20130419 (Friday, 19 April 2013)¶
Running multiple Lino versions¶
I have a production site running Lino on a stable Debian (i.e. squeeze), without pip and virtualenv. I want to “upgrade” this server to use pip and virtualenv.
First I need to install pip and virtualenv. The next steps are to create two virtual environments “stable” and “testing”, play with them, and finally to tell Apache to use them.
How to install pip and virtualenv on a production site¶
Matthew Rankin explains on Stackoverflow how to do it: What’s the proper way to install pip, virtualenv, and distribute for Python?. Thank you, Matthew! Very good, so I can install them without danger on my production site.
The following is how I applied Matthew’s instructions.
It starts with a problem:
lsaffre@lino:~$ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.9.1.tar.gz
-bash: curl: command not found
Okay, we need to install curl first:
lsaffre@lino:~$ sudo aptitude install curl
The following NEW packages will be installed:
curl libcurl3{a} libssh2-1{a}
0 packages upgraded, 3 newly installed, 0 to remove and 40 not upgraded.
Need to get 594 kB of archives. After unpacking 1,147 kB will be used.
Do you want to continue? [Y/n/?]
Yes. But oops:
Get:1 http://ftp.be.debian.org/debian/ squeeze/main libssh2-1 amd64 1.2.6-1 [80.9 kB]
Err http://ftp.be.debian.org/debian/ squeeze/main libcurl3 amd64 7.21.0-1
404 Not Found [IP: 77.243.184.65 80]
Err http://ftp.be.debian.org/debian/ squeeze/main curl amd64 7.21.0-1
404 Not Found [IP: 77.243.184.65 80]
Fetched 80.9 kB in 0s (267 kB/s)
Selecting previously deselected package libssh2-1.
(Reading database ... 46423 files and directories currently installed.)
Unpacking libssh2-1 (from .../libssh2-1_1.2.6-1_amd64.deb) ...
Setting up libssh2-1 (1.2.6-1) ...
E: Failed to fetch http://ftp.be.debian.org/debian/pool/main/c/curl/libcurl3_7.21.0-1_amd64.deb: 404 Not Found [IP: 77.243.184.65 80]
Hm, is the Debian server down or not reachable? No, a ping ftp.be.debian.org confirms that this is not the reason. The solution was this:
lsaffre@lino:~$ sudo aptitude update
(TODO: explain why…)
Now it works:
lsaffre@lino:~$ python virtualenv-1.9.1/virtualenv.py --distribute envs/boot
lsaffre@lino:~$ sudo aptitude install curl
lsaffre@lino:~$ curl -O https://pypi.python.org/packages/source/v/virtualenv/virtualenv-1.9.1.tar.gz
lsaffre@lino:~$ tar xvzf virtualenv-1.9.1.tar.gz
I then created a directoy “pythonenvs” in my home directory, then a first environment “boot”:
lsaffre@lino:~$ mkdir pythonenvs
lsaffre@lino:~$ python virtualenv-1.9.1/virtualenv.py --distribute pythonenvs/boot
New python executable in pythonenvs/boot/bin/python
Installing distribute.............done.
Installing pip................done.
To activate the boot environment:
lsaffre@lino:~$ . pythonenvs/boot/bin/activate
(boot)lsaffre@lino:~$
Reuse the .tar.gz file to install virtualenv into the boot environment:
(boot)lsaffre@lino:~$ pip install virtualenv-1.9.1.tar.gz
Unpacking ./virtualenv-1.9.1.tar.gz
...
Successfully installed virtualenv
Cleaning up...
No longer needed:
(boot)lsaffre@lino:~$ rm -r virtualenv-1.9.1
(boot)lsaffre@lino:~$ rm virtualenv-1.9.1.tar.gz
Create a stable environment¶
How to create a virtualenv which is a copy of the current site-packages?
lsaffre@lino:~$ ll /usr/local/django/cpas_eupen/using
total 0
lrwxrwxrwx 1 lsaffre www-data 25 Mar 29 02:58 lino -> /var/snapshots/lino-1.6.4
lrwxrwxrwx 1 lsaffre www-data 26 Mar 29 02:58 north -> /var/snapshots/North-0.1.1
lrwxrwxrwx 1 lsaffre www-data 31 Mar 29 02:58 site -> /var/snapshots/djangosite-0.1.1
lrwxrwxrwx 1 lsaffre www-data 33 Mar 29 02:59 welfare -> /var/snapshots/lino-welfare-1.1.1
lsaffre@lino:~$ . pythonenvs/boot/bin/activate
(boot)lsaffre@lino:~$
(boot)lsaffre@lino:~$ virtualenv --distribute pythonenvs/stable
Using real prefix '/usr'
New python executable in pythonenvs/stable/bin/python
Installing distribute...........................................................................................................................................................................................................................done.
Installing pip................done.
(boot)lsaffre@lino:~$ . pythonenvs/stable/bin/activate
(stable)lsaffre@lino:~$ pip install lino-welfare
...
Voilà. Hm, there is at least one difference: this environment installed Django 1.5.1 because lino-welfare doesn’t specify any restriction on the Django version. But the current “environment” on this production server is:
$ python manage.py validate
...
INFO This is Lino Welfare 1.1.1 using Python 2.6.6, Django 1.4.2, djangosite 0.1.1, Babel 0.9.6, North 0.1.1, Lino 1.6.4, Jinja 2.6, Sphinx 1.1.3, python-dateutil 1.4.1, Cheetah 2.4.2.1, OdfPy ODFPY/0.9.4, docutils 0.10, suds 0.3.9, PyYaml 3.09, Appy 0.7.0 (2011/09/15 15:55).
INFO Languages: de, fr, en. 25 apps, 92 models, 294 actors.
0 errors found
INFO Done manage.py validate (PID 9408)
In order to really reproduce this environment I must say:
lsaffre@lino:~$ pythonenvs/stable/bin/pip freeze > pythonenvs/dsbe20130419.txt
lsaffre@lino:~$ nano pythonenvs/dsbe20130419.txt
I then edited that requirements file manually, and if I’d feel very careful-traditional then I’d use it like this:
lsaffre@lino:~$ pythonenvs/stable/bin/pip install -r pythonenvs/dsbe20130419.txt
But probably I’m rather going to feel audacious and have Django 1.5.1 tried on a production site.
New version numbers¶
Updated the version numbers in the repositories: from (lino-welfare-1.1.1 lino-1.6.4 north-0.1.1 djangosite-0.1.1) to (lino-welfare-1.1.2 lino-1.6.5 north-0.1.2 djangosite-0.1.2 atelier-0.0.1)
Followed by:
$ pp ci
$ pp test
$ pp sdist alldocs pub
(pp is my alias for atelier’s per_project script)
Oops, the sdist command should not yet register the new version number to PyPI since we are still testing after all. That’s not a big problem since I didn’t actually upload the files.
The testing environment¶
There are two possibilities to implement a testing environment:
using a pip install –editable on a checkout of my projects
using a temporary distribution site (and telling pip to prefer this over the PyPI index, i.e. pip install –find-links=http://www.lino-framework.org/dl
Why a Belgian lives in Estonia¶
An interview with me was published on the first page of a much-read Estonian news portal: Belglane jättis kodumaal tasuva töö ja kolis Eestisse. Miks?.
Estonians of course enjoy stories of (European) foreigners who feel well in their little country.
So I received public attention, unfortunately only because I happened to move from Belgium to Estonia 12 years ago and still feel happy about it. Which is not at all my merit. What a shame: it’s still not yet for my work on Lino that I’m getting famous ;-)
Apache mod_wsgi and virtualenv¶
The mod_wsgi documentation about Virtual Environments is perfect.
My summary:
use only mod_wsgi and not try and use mod_python on the same server at the same time.
in the global part of the main Apache configuration files:
WSGIPythonHome /home/lsaffre/pythonenvs/boot
This environment will be used as the baseline environment for all WSGI applications, whether they be run in embedded mode or daemon mode.
In every application’s wsgi.py file:
import site site.addsitedir('/usr/local/pythonenv/stable/lib/python2.6/site-packages')
The env symbolik link¶
Convention: per project I create a symbolic link called “env” which points to the virtualenv for this project.
How to install MySQLdb using pip¶
Theroretically it’s easy:
$ sudo aptitude install python-dev
$ pip install mysql-python
But reality is sometimes more complex:
Downloading/unpacking mysql-python
...
EnvironmentError: mysql_config not found
This error message meant that libmysqlclient-dev was missing:
$ sudo aptitude install libmysqlclient-dev
After having installed libmysqlclient-dev, it failed again with:
unable to execute gcc: No such file or directory
So I did:
$ sudo aptitude install gcc
After this it worked.