Monday, November 14, 2016¶

I had asked Hamza to do the folllowing:

rename notify.Notification to notify.Message
remove the Message.subject field. A message has just a body. This fixes our design problem of having redundant text in subject and body.
we no longer send emails every 10 seconds but only once per day. And that email has then as subject “X unseen notification messages on (SITE.title)”, and the mail body contains the full notification messages

Now I merged his work into master and started to adapt tests in other projects after these changes.

I changed the label of notify_silent from “Don’t send email message” to “Don’t notify others” (“Keine Mitteilung an andere”, “Ne pas avertir les autres”).

In the evening we deployed to The Lino framework, did the migration together (which required a manual change in restore.py because we didn’t write a migrator yet) and then explored the problem of our websocket connection:

reconnecting-websocket.min.js:1 WebSocket connection to 'wss://xxxxx.lino-framework.org/websocket/' failed: Error during WebSocket handshake: Unexpected response code: 404

pkg_resources.VersionConflict¶

I had the following problem (obviously as a result of my experiments last Friday):

$ cd ~/repositories/lino
$ inv test
Traceback (most recent call last):
  File "setup.py", line 9, in <module>
    setup(**SETUP_INFO)
  File "/usr/lib/python2.7/distutils/core.py", line 151, in setup
    dist.run_commands()
  File "/usr/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "env/local/lib/python2.7/site-packages/setuptools/command/test.py", line 198, in run
    installed_dists = self.install_dists(self.distribution)
  File "env/local/lib/python2.7/site-packages/setuptools/command/test.py", line 193, in install_dists
    ir_d = dist.fetch_build_eggs(dist.install_requires or [])
  File "env/local/lib/python2.7/site-packages/setuptools/dist.py", line 361, in fetch_build_eggs
    replace_conflicting=True,
  File "env/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 850, in resolve
    dist = best[req.key] = env.best_match(req, ws, installer)
  File "env/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 1115, in best_match
    dist = working_set.find(req)
  File "env/local/lib/python2.7/site-packages/pkg_resources/__init__.py", line 719, in find
    raise VersionConflict(dist, req)
pkg_resources.VersionConflict: (html5lib 0.9999999 (env/lib/python2.7/site-packages), Requirement.parse('html5lib>=0.999999999'))

Which I tried to fix by saying:

$ pip install -U lino
...
Successfully installed PyYAML-3.12 beautifulsoup4-4.5.1 cffi-1.9.1
cssselect-1.0.0 django-1.9.11 django-wkhtmltopdf-3.1.0
future-0.16.0 gitdb2-2.0.0 gitpython-2.1.0 html5lib-0.999999999
pycparser-2.17 pytest-3.0.4 pytest-cov-2.4.0 python-dateutil-2.6.0
pytidylib-0.3.1 pytz-2016.7 smmap2-2.0.1

Observations:

bleach has a quite complex requirement: html5lib (>=0.999,!=0.9999,!=0.99999,<0.99999999)
html5lib has a strange version numbering: (Please for the love of god use a normal version number.)
So pip install -U bleach will break something.

With bleach==1.5.0 and html5lib==0.999999999:

$ python -m bleach
Traceback (most recent call last):
  File "/usr/lib/python2.7/runpy.py", line 163, in _run_module_as_main
    mod_name, _Error)
  File "/usr/lib/python2.7/runpy.py", line 111, in _get_module_details
    __import__(mod_name)  # Do not catch exceptions initializing package
  File "/media/dell1tb/virtualenvs/py27/local/lib/python2.7/site-packages/bleach/__init__.py", line 14, in <module>
    from html5lib.sanitizer import HTMLSanitizer
ImportError: No module named sanitizer

The html5lib CHANGES.rst mentions the following changes which seem to indicate that actually it is bleach which needs an update:

Fix #72 by rewriting the sanitizer to apply only to treewalkers (instead of the tokenizer); as such, this will require amending all callers of it to use it via the treewalker API.
Get rid of the sanitizer package. Merge sanitizer.sanitize into the sanitizer.htmlsanitizer module and move that to saniziter. This means anyone who used sanitizer.sanitize or sanitizer.HTMLSanitizer needs no code changes.

I tried to understand what happened and to write a PR for bleach. But gave up, realizing that all this is a job for the developers of these packages.

Lino and Django 1.9.11¶

Django 1.9.11 validates the Host header against ALLOWED_HOSTS also when DEBUG is True. It gives me a warning:

Invalid HTTP_HOST header: 'testserver'. You may need to add u'testserver' to ALLOWED_HOSTS.

To get our test suites pass again, I now defined a default ALLOWED_HOSTS in lino.projects.std.settings:

ALLOWED_HOSTS = ['testserver', 'localhost', '127.0.0.1', '::1']