Sunday, March 13, 2016

In order to reproduce the problem, I wrote the following script:

from lino import startup
startup('lino_welfare.projects.std.settings.demo')
from django.conf import settings
settings.SITE.appy_params.update(raiseOnError=True)
from lino.api import rt
Excerpt = rt.models.excerpts.Excerpt
ses = rt.login('robin')
# qs = Excerpt.objects.filter(build_time__isnull=True)
qs = Excerpt.objects.all()
# for obj in qs:
if True:
    obj = Excerpt.objects.get(pk=4)
    print "Gonna print #", obj.pk
    ses.run(obj.do_clear_cache)
    # dd.logger.info("20150526 rendering %s", obj)
    rv = ses.run(obj.do_print)
    print(rv)

This is to be run after an initdb_demo with SEVERE set to False in lino_xl.lib.excerpts.fixtures.demo2. On my machine the script reproduces our error reliably.

It was not trival to get the following traceback. I had to change appy’s behaviour when the raiseOnError option is True: in that case it now really raises an exception (until now it still tried to “manage” that error by adding some information and then raise a new exception).

But finally here it is:

Error while evaluating the expression "html(body)" defined in the "from" part of a statement. TypeError: 'NoneType' object is not iterable
File "<string>", line 1, in <module>
File "/work/xl/lino_xl/lib/appypod/appy_renderer.py", line 170, in html_func
return self.renderXhtml(html, **kw)
File "/mypy/appy/pod/renderer.py", line 259, in renderXhtml
stylesMapping, self).run()
File "/mypy/appy/pod/xhtml2odt.py", line 643, in run
self.xhtmlParser.parse(self.xhtmlString)
File "/mypy/appy/shared/xml_parser.py", line 277, in parse
self.parser.parse(inputSource)
File "/usr/lib/python2.7/xml/sax/expatreader.py", line 107, in parse
xmlreader.IncrementalParser.parse(self, source)
File "/usr/lib/python2.7/xml/sax/xmlreader.py", line 123, in parse
self.feed(buffer)
File "/usr/lib/python2.7/xml/sax/expatreader.py", line 210, in feed
self._parser.Parse(data, isFinal)
File "/usr/lib/python2.7/xml/sax/expatreader.py", line 304, in start_element
self._cont_handler.startElement(name, AttributesImpl(attrs))
File "/mypy/appy/pod/xhtml2odt.py", line 553, in startElement
currentElem = e.onElementStart(elem, attrs)
File "/mypy/appy/pod/xhtml2odt.py", line 473, in onElementStart
styles = CssStyles(elem, attrs)
File "/mypy/appy/shared/css.py", line 57, in __init__
setattr(self, name, CssValue(attrs[name]))
File "/mypy/appy/shared/css.py", line 27, in __init__
value, unit = CssValue.valueRex.match(value)
<type 'exceptions.TypeError'>: 'NoneType' object is not iterable

Now that we have a reproduceable traceback, let’s look at the actual problem itself. It happens in the following code (in appy/shared/css.py):

class CssValue:
    valueRex = re.compile('(\d+)(%|px)?')

    def __init__(self, value):
        value, unit = CssValue.valueRex.match(value)
        if not unit: unit = 'px'
        self.value = int(value)
        self.unit = unit

Let’s try to reproduce it in a simplified snippet:

>>> mo = None
>>> value, unit = mo
Traceback (most recent call last):
  ...
TypeError: 'NoneType' object is not iterable

Okay that’s our error message, but it is useless because it is too much of a simplification. Here is a more accurate reproduction using the re module:

>>> import re
>>> valueRex = re.compile('(\\d+)(%|px)?')
>>> value = '10'
>>> mo = valueRex.match(value)
>>> type(mo)
<type '_sre.SRE_Match'>

Unfortunately the above snippet does not fail. Also the following lines confirm that everything is correct here:

>>> mo.groups()
('10', None)
>>> valueRex.pattern
'(\\d+)(%|px)?'
>>> valueRex.flags
0

And even the following snippet does not fail:

>>> from appy.shared.css import CssValue
>>> CssValue("b'10'")
10px

Why then does it obviously fail with the script at the top?

>>> from lino.utils.xmlgen.html import E
>>> name = "Hello World".encode('utf-8')
>>> html = E.table( E.tr(name=name) )
>>> print(tostring(html))
<table><tr name="Hello World" /></table>

>>> from lino import startup
>>> startup('lino_welfare.projects.std.settings.demo')
>>> from lino.api.doctest import *
>>> ar = users.Users.request()
>>> html = ar.table2xhtml()
>>> type(html)
<class 'xml.etree.ElementTree.Element'>

>>> type(tostring(html))
<type 'unicode'>

>>> print(tostring(html))
<table bgcolor="#ffffff" cellspacing="3px" name="b'users.Users.grid'" width="100%"><thead><tr><th align="left" bgcolor="#eeeeee" valign="top" width="b'21'">Username</th><th align="left" bgcolor="#eeeeee" valign="top" width="b'38'">User Profile</th><th align="left" bgcolor="#eeeeee" valign="top" width="b'21'">First name</th><th align="left" bgcolor="#eeeeee" valign="top" width="b'21'">Last name</th></tr></thead><tbody><tr><td align="left" bgcolor="#eeeeee" valign="top">alicia</td><td align="left" bgcolor="#eeeeee" valign="top">Integration agent</td><td align="left" bgcolor="#eeeeee" valign="top">Alicia</td><td align="left" bgcolor="#eeeeee" valign="top">Allmanns</td></tr><tr><td align="left" bgcolor="#eeeeee" valign="top">caroline</td><td align="left" bgcolor="#eeeeee" valign="top">Newcomers consultant</td><td align="left" bgcolor="#eeeeee" valign="top">Caroline</td><td align="left" bgcolor="#eeeeee" valign="top">Carnol</td></tr><tr><td align="left" bgcolor="#eeeeee" valign="top">hubert</td><td align="left" bgcolor="#eeeeee" valign="top">Integration agent</td><td align="left" bgcolor="#eeeeee" valign="top">Hubert</td><td align="left" bgcolor="#eeeeee" valign="top">Huppertz</td></tr><tr><td align="left" bgcolor="#eeeeee" valign="top">judith</td><td align="left" bgcolor="#eeeeee" valign="top">Social agent</td><td align="left" bgcolor="#eeeeee" valign="top">Judith</td><td align="left" bgcolor="#eeeeee" valign="top">Jousten</td></tr><tr><td align="left" bgcolor="#eeeeee" valign="top">kerstin</td><td align="left" bgcolor="#eeeeee" valign="top">Debts consultant</td><td align="left" bgcolor="#eeeeee" valign="top">Kerstin</td><td align="left" bgcolor="#eeeeee" valign="top">Kerres</td></tr><tr><td align="left" bgcolor="#eeeeee" valign="top">melanie</td><td align="left" bgcolor="#eeeeee" valign="top">Integration agent (Manager)</td><td align="left" bgcolor="#eeeeee" valign="top">M&#233;lanie</td><td align="left" bgcolor="#eeeeee" valign="top">M&#233;lard</td></tr><tr><td align="left" bgcolor="#eeeeee" valign="top">nicolas</td><td align="left" bgcolor="#eeeeee" valign="top" /><td align="left" bgcolor="#eeeeee" valign="top" /><td align="left" bgcolor="#eeeeee" valign="top" /></tr><tr><td align="left" bgcolor="#eeeeee" valign="top">robin</td><td align="left" bgcolor="#eeeeee" valign="top">Administrator</td><td align="left" bgcolor="#eeeeee" valign="top">Robin</td><td align="left" bgcolor="#eeeeee" valign="top">Rood</td></tr><tr><td align="left" bgcolor="#eeeeee" valign="top">rolf</td><td align="left" bgcolor="#eeeeee" valign="top">Administrator</td><td align="left" bgcolor="#eeeeee" valign="top">Rolf</td><td align="left" bgcolor="#eeeeee" valign="top">Rompen</td></tr><tr><td align="left" bgcolor="#eeeeee" valign="top">romain</td><td align="left" bgcolor="#eeeeee" valign="top">Administrator</td><td align="left" bgcolor="#eeeeee" valign="top">Romain</td><td align="left" bgcolor="#eeeeee" valign="top">Raffault</td></tr><tr><td align="left" bgcolor="#eeeeee" valign="top">theresia</td><td align="left" bgcolor="#eeeeee" valign="top">Reception clerk</td><td align="left" bgcolor="#eeeeee" valign="top">Theresia</td><td align="left" bgcolor="#eeeeee" valign="top">Thelen</td></tr><tr><td align="left" bgcolor="#eeeeee" valign="top">wilfried</td><td align="left" bgcolor="#eeeeee" valign="top">Accountant</td><td align="left" bgcolor="#eeeeee" valign="top">Wilfried</td><td align="left" bgcolor="#eeeeee" valign="top">Willems</td></tr></tbody></table>

>>> type(ar.bound_action.full_name())
'users.Users.grid'

>>> from builtins import str
>>> str(b'x')
>>> str
<class 'future.types.newstr.newstr'>