Thursday, March 23, 2017¶

I continued to explore #1609. The problem is easily reproduceable on cpaseupen by setting Martha Mustermann to ClientStates.newcomer and then running RefuseClient with a lino_welfare.modlib.pcsw.choicelists.RefusalReason that contains non-ascii characters. I adapted lino_welfare.projects.eupen.tests.test_clients.

In order to see whether is fixes (or at least works around) our problem, I tried to change their MyISAM or InnoDB? from MyISAM to InnoDB by removing the ‘OPTIONS’: { “init_command”: “SET storage_engine=MyISAM” } from their DATABASES setting and restoring their last snapshot.

Changing the engine is not very difficult, but restoring a snapshot runs about 25 minutes on their site. Here are some numbers:

Loading 13724 objects to table contacts_partner...
Loading 13074 objects to table addresses_address...
Loading 21777 objects to table b2c_transaction...
Loading 6950 objects to table pcsw_client...
Loading 19700 objects to table cal_event...
Loading 93915 objects to table changes_change...
Loading 35140 objects to table debts_entry...
Loading 18658 objects to table notes_note...

The first restore failed because I had still a local change for #1354 in their code. Meanwhile I had realized that switching the engine might cause minor operation problems. So I decided to ask their permission before doing this move. It seems clear to me that we should do it one day (after reading e.g. this article), but they should decide which day suits them best.

Actually I don’t seriously believe I that this move will fix #1609, but it is not impossible, and I currently don’t see any other explanation. Here is the traceback:

  File "env/repositories/lino/lino/core/kernel.py", line 865, in run_action
    a.run_from_ui(ar)
  File "env/repositories/welfare/lino_welfare/modlib/pcsw/actions.py", line 89, in run_from_ui
    ar, subject=subject, body=body)
  File "env/repositories/xl/lino_xl/lib/notes/mixins.py", line 38, in emit_system_note
    note.save()
  File "env/repositories/lino/lino/modlib/gfks/mixins.py", line 128, in save
    super(Controllable, self).save(*args, **kw)
  File "env/lib/python2.7/site-packages/django/db/models/base.py", line 708, in save
    force_update=force_update, update_fields=update_fields)
  File "env/lib/python2.7/site-packages/django/db/models/base.py", line 736, in save_base
    updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
  File "env/lib/python2.7/site-packages/django/db/models/base.py", line 820, in _save_table
    result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
  File "env/lib/python2.7/site-packages/django/db/models/base.py", line 859, in _do_insert
    using=using, raw=raw)
  File "env/lib/python2.7/site-packages/django/db/models/manager.py", line 122, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "env/lib/python2.7/site-packages/django/db/models/query.py", line 1039, in _insert
    return query.get_compiler(using=using).execute_sql(return_id)
  File "env/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 1060, in execute_sql
    cursor.execute(sql, params)
  File "env/lib/python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
    return self.cursor.execute(sql, params)
  File "env/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 112, in execute
    return self.cursor.execute(query, args)
  File "env/lib/python2.7/site-packages/MySQLdb/cursors.py", line 187, in execute
    query = query % tuple([db.literal(item) for item in args])
  File "env/lib/python2.7/site-packages/MySQLdb/connections.py", line 278, in literal
    return self.escape(o, self.encoders)
  File "env/lib/python2.7/site-packages/MySQLdb/connections.py", line 203, in string_literal
    return db.string_literal(obj)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xd6' in position 0: ordinal not in range(128)

I tried to log the value of body and see what it produces on their server. But it seems to be a valid unicode string.

>>> from lino import startup
>>> startup('lino_welfare.projects.eupen.settings.demo')
>>> from lino_welfare.modlib.pcsw.choicelists import RefusalReasons
>>> ch = RefusalReasons.get_by_value('20')
>>> from builtins import str
>>> str(ch)
'\xd6SHZ ist nicht zust\xe4ndig'
>>> unicode(ch)
'\xd6SHZ ist nicht zust\xe4ndig'

About broken GFKs¶

I removed gfks.BrokenGFKs from the menu and created #1620.

Lino Avanti meeting¶

Today was next meeting with the Avanti team.

TODO:

Kann Hubert nicht als Klientenkontakt angeben
nicht sichtbar für Janina: NISS, Enrolment.pupil (Klient), Enrolments *
Kinderbetreuung : wieviele Kinder? Arten. Pro Haushaltsmitglied könnte man noch die Betreuungsart eingeben. Hier eventuell noch unterscheiden zwischen (1) Ist-Situation und (2) Bedarf. Betreuungsarten wären z.B. Keine, Betreutes Kind, Pfegebedürftiger Angehöriger, Behindert, Kleinkind, Kindergarten, Schulkind, … Aber wahrscheinlich ist das alles zu detailiert und nicht pflegbar.

Eher vielleicht eine allgemeinere Tabelle mit “Hindernisgründen” (mangelnde Kinderbetreuung, Krankheit, pflegebedürftige Angehörige, …). Zu beobachten.
Die drei Felder (Wartezeit, Datum Arbeit) aus Lebenslauf können weg, dafür neues Auswahlfeld “Berufliche Situation”:

Student Arbeitslos Eingeschrieben (Arbeitsamt) Angestellt Selbstständig
Feldbezeichnung “Sprache” -> “Kontaktsprache”
Zwei neue Felder “Niveau DE” und “Niveau FR”.
LanguageKnowledgesByPerson : (1) von Lebenslauf nach Person (oder eher sogar als eigener Reiter statt summary) (2) Zusammenfassung : lediglich das CEF-Level der Kontaktsprachen + alle Muttersprachen
Janina will eigentlich nur drei virtuelle Felder - ein Feld “Muttersprachen”, in dem alle als MS markierten Sprachen (space-separated iso3) stehen - CEF-Level DE und FR und EN
Janina will die Entwicklungsetappen nicht sehen.
Partnernummer aus den Excel-Daten raus
Drei Klientenstatuus: Eingeschrieben, Beendet und Abgebrochen. Kreuzt sich bewusst ein bisschen mit dem Beendigungsgrund.
Neues Feld “Referenz” (Dossiernummer, Aktenreferenz). Client should inherit from Referrable.
Eigentlich wollen wir immer nur einen Haushalt pro Klient. Und diesen Haushalt wollen wir gar nicht erstellen müssen. Zumindest stay_in_grid for MembersByPerson.
Lebenslauf in Ordnung
Wissen über Bewerbungsverfahren in Belgien
Führerschein
eigenes Fahrzeug
Entwicklungsbereich Detail –> Alle Etappen dieses Bereichs erfassen
Neues Feld “Kursbedarf” (Auswahlfeld -> Kursreihe”)
Neues Feld “Verfügbarkeit” (freies Textfeld)
Möglichkeit, gewisse Notizen im Reiter “Person” anzuzeigen, damit jeder Mitarbeiter sofort sieht, (nicht im Startbildschirm, sondern nur beim Klienten). Beispiele: “Diplomanerkennung anfragen!” “Wie ist es mit der Führerscheinprüfung?” New checkbox cal.Task.important which akts like Note.important.
Aufenthaltstitel: das ist eines der Felder im BeIdCardHolder. All diese Felder sollten irgendwo manuell eingebbar sein.

Miscellaneous¶

I had a problem #1624 (AttrDict instance has no key ‘pcsw’ during inv bd in book) because lino_welfare.modlib.welfare.user_types since recently was installing the merge actions to some models. This code fails when the lino_welfare.modlib.pcsw plugin is not installed. When I moved this code there, I wasn’t aware that the book also imports this module for displaying the inheritence diagram (in order to explain Introduction to permissions).

So I added lino_welfare to the dependencies of book. And lino_voga as well while we are there. I think this is the right direction. Yes, the Lino book requires all those “pilot applications”.

I removed the unused method Site.setup_choicelists().

Reorganizing tested docs and demo projects¶

The Lino project is organized into code repositories, test suites, demo projects and documentation trees. These don’t overlap exactly.

The book sometimes needs some applications in order to explain some framework feature. For example it is difficult to explain the tickets plugin without having some application that uses it. And it would be a waste of energy to create a demo application since we have a full-fledged application which we can use as demo.
Some parts of the documentation are being tested using doctest and therefore rely a demo project to be preparaed. These pages must be in the same repository as the demo projects they use.
The application docs should document only the application-specific things and should be able to refer to parts of the book for every shared feature. For example end-user documentation about user management can be shared because many concepts are common to all Lino applications.
We first must build the book docs and then the docs of the individual applications.
Installing the book and building its docs requires the applications to be installed.
Installing an application does not need to install any demo project. These are part of the book.
We need the demo projects of an application for writing tested docs about that application.
The application repositories will have only a very basic test suite.
Coverage can be measured only for all projects together as a whole.
The ordering for code installation is not the same as the ordering for building the docs:
- pip installation : atelier lino xl noi voga … book
- Sphinx build : book noi voga …

I think that this means for now that

We should do the following first with Noi, then evaluate it before doing it with Cosi, Voga and finally Welfare (the biggest one): move all the demo projects and all tested docs (e.g. everything below Lino Noi) from the application repo to the book.

Pooh, that’s a big series of changes! I created #1626 for it. Any comment about my plan are welcome. Tonis might work on this until Sunday.