20160716 (Saturday, 16 July 2016)

ValueError: invalid literal for float(): 0:10

There was a bug in lino.modlib.export_excel causing it to fail when the table had a DurationField.

When searching for a test case to reproduce this, I first wrote Exporting to Excel from Lino Noi, but then was surprised to see that calendar events didn’t have a Duration field. I changed the implementation: the StartedEnded mixin no longer exists. It caused the following behaviour: when saving a Session whose end_date is empty, it filled the end_date from start_date. This behaviour was disturbing because the end_date didn’t get changed when you afterwards changed the start_date. So I wrote a new implementation for get_duration(), end_date can now be empty (which means that we should use the same date as in start_date), and as a result, calendar events now have a Duration field.

To fix the bug itself was yet another hour of work. I completely reimplemented lino.modlib.export_excel: Joe had written it using a TableRenderer class, but this was unefficient (it called ar.get_field_info for every row), and it converted the values using an unreliable try-except mechanism.

ESF Statistics

I fixed two bugs in lino.modlib.summaries:

  • the last year was not being computed

  • counters were never being reset

I added first test cases to esf : European Social Fund.

The checksummaries command is now automatically being run daily.

New function decorators schedule_often schedule_daily

Testing testing

I repaired the Lino Noi test suite and stumbled over yet another unicode problem with inv test. It happens only under Python 2, and only if some test fails, and only if the output of that failure contains non-ascii characters.

To reproduce it, I added (to the Atelier repository) a test case which always fails using a message in German which I refuse to translate:

class BasicTests(TestCase):

    def test_fail_with_unicode_error(self):
        self.fail(u"Scheiße wa!")

Now I can run:

$ python setup.py -q test
.F.......
======================================================================
FAIL: test_02 (tests.BasicTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/media/dell1tb/luc/work/atelier/tests/__init__.py", line 20, in test_02
    self.fail(u"Scheiße wa!")
AssertionError: Schei\xdfe wa!

----------------------------------------------------------------------
Ran 9 tests in 1.524s

FAILED (failures=1)

Works perfectly under both Python 2 and 3.

But when I run inv test (which basically does the same), then I get:

$ inv test
.F.......
======================================================================
FAIL: test_fail_with_unicode_error (tests.BasicTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/media/dell1tb/luc/work/atelier/tests/__init__.py", line 14, in test_fail_with_unicode_error
    self.fail(u"Scheiße wa!")
AssertionError: Schei\xdfe wa!

----------------------------------------------------------------------
Ran 9 tests in 1.424s

FAILED (failures=1)
Traceback (most recent call last):
  File ".../virtualenvs/py27/bin/inv", line 11, in <module>
    sys.exit(program.run())
  File ".../virtualenvs/py27/local/lib/python2.7/site-packages/invoke/program.py", line 275, in run
    debug("Received a possibly-skippable exception: {0!r}".format(e))
  File ".../virtualenvs/py27/local/lib/python2.7/site-packages/invoke/exceptions.py", line 49, in __repr__
    return str(self)
  File ".../virtualenvs/py27/local/lib/python2.7/site-packages/invoke/exceptions.py", line 46, in __str__
    """.format(self.result.exited, err_label, err_text)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xdf' in position 369: ordinal not in range(128)

After two hours of poking around, I gave up for this time and created ticket #1052. I guess that the solution is somewhere in atelier.invlib, or in invlib.program. It is not very urgent, since it happens only under Python 2, and only when some test fails. And when it happens, then the simple workaround is to type python setup.py test yourself.

ExtJS6 is coming

Hamza and I had a long working session yesterday. I now merged the result of these hours into the master).

I added lino_xl.lib.extensible to the disables_plugins of ExtJS 6 front end. Because that’s a perfect second use case for disables_plugins.

Grigorij is back

Grigorij is back after a week of holidays. His task is to walk through the “Getting started” section of the Developer’s Guide, ask questions if necessary and report his successes and any problems he encounters. This task has a double benefit:

  1. he learns about installing Python packages, virtual environements, updating git clones etc. This is a knowledge which every Python professional needs.

  2. Lino gets better because he helps us to “test the documentation”. When he stumbles over a problem and does the work of reporting it, then we can fix it. Our goal is that every (motivated) beginner experiences joy when walking through this document.

So thanks in advance, Grigorij, for your contribution to the Lino project!

Here is a problem he helped me to discover: the “Collecting static files” section was absolutely useless in the installation instructions for developers. I moved it to the Set up a Lino production server, replacing a section “Serving JavaScript frameworks” which was hanging around there from old times. On a development server you don’t need to run collectstatic because the server does that dynamically.

Another problem which Grigorij would soon report is the following:

$ cd lino_book/projects/polly
$ python manage.py initdb_demo
We are going to flush your database (.../default.db).
Are you sure (y/n) ? [Y,n]?
Operations to perform:
...
Installed 18 object(s) from 5 fixture(s)
$ python manage.py runserver
CommandError: You must set settings.ALLOWED_HOSTS if DEBUG is False.

This was because lino_book.projects.polly.settings.demo did not set DEBUG to True.

So, Grigorij, before going on make sure to update your git clones (see How to update your copy of the repositories).

A third problem I discovered whil writing this: the online demo of Lino Polly was broken.

Grigorij also reported a problem TypeError: setup_from_tasks() got an unexpected keyword argument 'demo_projects' which was caused by my yesterday’s changes. So I released Atelier version 1.0.2.

More statistics for ESF

I continued on #584. See code changes in lino_welfare.modlib.esf.

I wrote a new utility function lino.utils.dates.weekdays() (thanks to earl for the basic idea).

Lino now (theoretically) knows all the requested types of statistical fields as described in esf : European Social Fund. Of course we are not yet at the end. But now it is time for a release in order to watch real data.