Wednesday, February 17, 2021

I had an avalanche of ideas about what’s to do next.

Note: This is a situation where it’s more efficient to write a developer blog entry. Otherwise I started to use our commenting system (Lino Noi) more intensively. Because a reporting-what-you-do approach based on comments in a database (rather than daily blog entries) is more suitable whenever you are not alone.

Here is an overview of the ideas (all except the first are just ideas ATM):

  • lino.modlib.uploads : Add a field file_size. Add a view where files are sorted by size. Move the Uploadable mixin to the uploads plugin because it is used only there. Add a UploadChecker, which verifies that the file still exists and the size matches. Report files under uploads root that do not have an Upload entry. I started working on this as #3982.

  • A future lino_xl.lib.gallery plugin would depend on uploads and add views to manage and uploads that are considered as pictures. But what is a picture? What about the existing Upload.mimetype field? Can we use this? Or do we need a an additional choicelist uploads.FileTypes and set file_type.is_image to True for pictures?

  • lino.modlib.comments : Review the demo comments to make them more convincing. Use a CommentingAreas choicelist instead of a generic foreign key. Add commenting to Lino Così.

  • lino.modlib.changes: Add a config parameter to say that we want to forget changes that are older than a number of days. Add a method Change.get_as_comment() that returns self formatted as a comment.

Intermediate research:

Can we use Django’s built-in size attribute of a FieldFile object for sorting all Uploads by their size? No, because a FieldFile is just a volatile object created on the fly, and its size property works by calling os.path.getsize() (at least when the uses the default file storage and unless it has been asked before doring the same request). os.path.getsize() returns the file size in bytes.

Oops, I discovered that the insert window of an upload doesn’t close after having uploaded. The JS console says “Uncaught TypeError: action.result is undefined”. The end user sees no error message and the window simply remains open. So the user is tempted click a second time on the submit button. And every click will upload the file again. So we definitively need the UploadChecker to find duplicate and unused files.

Ha! There was a typo bug in renderer_mixin.JsCacheRenderer which caused Lino to create an empty media directory “upload” (instead of “uploads”) at startup. This bug might be the cause for missing group write permission of files that were uploaded to a production site. If you encounter this problem, check whether your uploads directory has the sticky group bit set.

Here the actual code changes in lino.modlib.uploads:

  • Moved the Upgradable mixin from lino.mixins to UploadBase. And the upload_to value of Upload.file is no longer hard-coded but configurable via the new Plugin.upload_to_spec setting.

  • Added a new plugin parameter Plugin.max_file_size

  • The Model.save_new_instance() method now calls full_clean() on the new instance before anything else. This is now important because an upload file might be invalid due to its size.

  • In lino_book.projects.min9 I converted the settings package into a single settings.py module.

Some side effects on documentation and test suites:

A first side effect was that I removed the docs and max demo projects. These two project had become useless ballast. Most of the tested documents that relied on them can use lino_book.projects.min2 or lino_book.projects.min9 as well. But I had to add some more plugins to min9. I created ticket #3984 for this.

As a side effect of this side effect, the API generated by autodoc no longer includes the models.py files of the some plugins. This is a first step towards converting their docs to prose style (#1869). Next step will be to create a specs page for each of them and to move the docstrings from the models.py there. The plugins are lino_xl.lib.votes lino_xl.lib.skills lino_xl.lib.postings lino_xl.lib.outbox lino_xl.lib.online.users lino_voga.lib.invoicing lino_noi.lib.users lino_noi.lib.topics lino_noi.lib.tickets lino_noi.lib.contacts and lino_noi.lib.cal.

And a third side effect: I moved some attribute descriptions of lino.core.model.Model (e.g. allow_cascaded_copy) from the source code to a prosa file. This was my first usage of the :noindex: option for the class directive.