20130323 (Saturday, 23 March 2013)

Extract messages using Babel instead of Django

Cool! Today I replaced the sophisticated and intransparent system for handling internationalization (which used only Django’s builtin possibilities and needed /Makefile) by a transparent and easy-to-use system using Babel (and Jinja’s Babel Integration).

The configuration is in /lino/setup_info.py:

SETUP_INFO.update(message_extractors = {
    'lino': [
        ('**/sandbox/**',        'ignore', None),
        ('**/cache/**',          'ignore', None),
        ('**.py',                'python', None),
        ('**.js',                'javascript', None),
        ('**/templates_jinja/**.html', 'jinja2', None),
    ],
})

The raw commands to issue (supposing locale “de” ) are:

setup.py extract_messages -o lino/locale/django.pot
setup.py init_catalog    --domain django -d lino/locale -l de -i lino/locale/django.pot
setup.py update_catalog  --domain django -d lino/locale -l de -i lino/locale/django.pot
setup.py compile_catalog --domain django -d lino/locale -l de

A first nice surprise is the following message:

SyntaxError: python refuses to compile code with both a UTF8 byte-order-mark and a magic encoding comment

There were indeed a few files with this problem (which had never occured on my machine, though, and I’m not going to research why).

One fundamental difference is: until now I had a lot of locale directories (one for each app), but I’m going to have now a single one in /lino/locale. Because that simplifies everything a lot! Except that I’ll need to find out how to recover the existing translations. Here is the Windows shell command used to get a list of the German ones:

T:\hgwork\lino>dir /s django.po /b | grep de\\

Another little difference: Django’s makemessages command automatically removed the .pot files, I am going to leave the /lino/locale/django.pot file hanging around.

The above commands are now fully implemented in djangosite.utils.fablib as:

fab em
fab im
fab um
fab cm

While converting lino_welfare to the new system I noticed that I can now finally also put the central things like the settings.py file, demo fixtures and django-admin commands to where they belong (to the top-level directory) I used the occasion to split the central parts of lino_welfare.modlib.pcsw.models into a new module lino_welfare.models.

north.north_site.Site now does the same for LOCALE_PATHS as for FIXTURE_DIRS: it extends Django’s automatic discovery system by searching also for “locale” subdirectories relative to any Python module that defines a subclass of north.north_site.Site (usually a settings.py file) and which are not part of an app (in which case a locale directory should not be mentioned there because Django already searches them, as explained in How Django discovers translations).

One “detail” causes still headache: Babel doesn’t seem to support message contexts, at least it doesn’t extract them using pgettext instead of gettext. A ticket 278 mentioned this, but it has been closed as duplicate of 277 which afaics doesn’t speak at all about pgettext.

An example of where this causes problem is the German male salutation “Herr” or “Herrn”: both salutations are “Mr” in English, but in German there are two cases. Lino provides some utility functions for this, see The Human mixin. This is done using pgettext (see source code at /lino/mixins/human.py), but Babel seems to simply ignore these. It does read the file:

...
extracting messages from lino\mixins\human.py
...

UPDATE: What a funny coincidence! The last post in Babel’s mailinglist was exactly about this problem, and it told me that Babel can do pgettext, but they just didn’t yet release it officially! Tried with a checkout of the latest trunk version (which is not completely trivial but there are clear instructions): yes, that works.

Only remaining problem is that I’d prefer to use a released version on my customer’s site.

To trust or not to trust?

Before releasing yesterdays new “textfield templates are now parsed using Jinja” feature to The Lino framework, I’d rather add the new attribute lino.ui.Site.trusted_templates which is False by default. So sorry, you cannot see this feature in our live demos.