Friday, November 22, 2019

Which or That?

Congratulations to this nice article about the difference between “which” and “that”. It confirms my feeling that many people (including myself) tend to use “which” in places where we actually should say “that”.

Playing with Tox

Okay tox insists on creating its own separate environment for running tests. That’s why the tox.ini somehow resembles the .travis.yml.

When I run the book test suite on my machine, I don’t want it to clone the lino repository. Which means that I cannot do the following in the tox.ini:

pip install -r requirements.dev.python3.txt
inv install

skipsdist means: “Flag indicating to perform the packaging operation or not. Set it to true when using tox for an application, instead of a library.”

So skipsdist must be true, which means that I must install

pip install -U pip pip install -e {toxinidir}

Note that for me it wasn’t enought to say:

$ sudo apt-get install python3-dev

I had to say:

$ sudo apt-get install python3.7-dev

Seems that the default python is 3.6 on my machine but we support only 3.7

Together with Hamza I reviewed the requirements files for book : we no longer need Python 2 support, renamed LINO_VERSION to REQ_VERSION, and introduced a new REQ_VERSION “local”.

When I got this working, I realized that pytest does not discover the doctests (#3353). It is because the doctests use the unittest load_tests feature. I read their tickets 3548 and 992. So indeed they discussed about this topic some years ago and could not imagine that I would write atelier.test.make_docs_suite() and that we rely heavily on it.

So I tried unittest2 instead, and this discovered the doctests out of the box.

I checked in, but there are still many failures cause by some missing dependency. I have no local work now and will go offline for some time now. But I will continue on it later.

We don’t want Tox

Now that I got tox running, I realized that we don’t want tox! At least not on a contributor machine. A contributor does not want to test in a separate environment. A contributor has his default environment that he keeps up-to-date, as it has always been. We just need a new parameter test_command in atelier (like prep_command or coverage_command) which defaults to something like “unit2 discover tests”.

About coverage

I also understood a few things about coverage. For example the official coverage for atelier was 23% until now. That’s very low because actually we don’t yet use coverage correctly.

I read https://coverage.readthedocs.io/en/coverage-4.3.4/subprocess.html

The second option, “Create a .pth file in your Python installation” seemed best to me… until I read that pth files are going to be deprecated in 3.8.

So we must edit the sitecustomize.py file. This file is usually defined in the system Python:

>>> import sitecustomize
>>> sitecustomize.__file__
'/usr/lib/python3.6/sitecustomize.py'
>>>

I added this test and a warning in the inv cov command. When you run inv cov while not under coverage, you now get a warning.

But there is more. In order to get a “full coverage report”, we must run inv prep test clean -b bd in all projects. And all this must be covered by a single coverage report. For example

• many parts of atelier aren’t tested by the atelier test suite, but they are tested when we use them for testing the other projects.

• Running the tests for welfare tests parts of lino or lino_xl code that are not covered by the book test suite.

I created a file run_coverage.sh in the book repository which does exactly this.

I removed the combine and report parts from inv cov because we don’t want them for each project.

It seems to work; the average coverage for all our projects was now 47%, but this number is still not reliable because the book and getlino test suites are currently failing. Also the coverage report is now very long and has no sub-total per project. And there are still warnings that I don’t understand…