Last time on the blog, I started the process of packaging mediagoblin into a Nix package, and this time we are continuing on our journey.
Switching to v0.10.0 for mediagoblin
One thing that I discovered while I was working on this project was that there are release tags for mediagoblin. So, I decided to switch to using this tag instead of the stable branch, which will mean our build is more reproducible.
src = fetchgit {
url = "https://git.savannah.gnu.org/git/mediagoblin.git";
rev = "v0.10.0";
sha256 = "13s66snz9710xr7s2icnmmqb4756fq369gn0qa6cwja4xr25sjd4";
fetchSubmodules = true;
};
Fixing The First Build Problem
Last time, our build script looked like this:
buildPhase = ''
cd $src
./bootstrap.sh
'';
And I was getting an error about how aclocal
wasn’t found.
So, I decided that it was time to clone the repository and have a look at the bootstrap script myself. After I ran the git clone
command and changed into that directory, I read the bootstrap.sh
file and found this:
#!/bin/sh
set -e
aclocal -I m4 --install
autoreconf -fvi
git submodule init
git submodule update
So, the git submodule
commands are taken care of with our packaging, and the other commands are from autoconf
and automake
, which are taken care of in Nix using something called the autoreconfHook
. So, as it turns out, we don’t need the bootstrap script at all!
The ./configure and make command
So, the next command that we were told to run is a ./configure
command, that takes a VIRTUALENV
environment variable to configure the Python environment. Nix’s mkDerivation
assumes that the configure phase of building something is just the ./configure
command. Additionally, the make
command is assumed for the build phase.
Therefore, we don’t need to invoke either commands in our build, since they are the default. I might need to bring back the VIRTUALENV
version of the config command later, but I’ll tackle it then.
So, instead of redefining configure or build phases, all you have to do is add the autoreconfHook
to the buildInputs
argument of mkDerivation
.
{ autoreconfHook, fetchgit, stdenv }:
stdenv.mkDerivation rec {
name = "mediagoblin-${version}";
version = "stable";
src = ...;
buildInputs = [
autoreconfHook
];
meta = ...;
}
So, when I build this, I get the following error:
configure: error: --with-python3 given but no acceptable python3 could be found
So, at this point, the autoconf
error is for a missing python dependency. I worked on that next.
Adding Python Dependencies
While I was looking at the bootstrap.sh
script, I realized that there was Python code in the repository, when I remembered that mediagoblin stated that it’s implemented in Python for their backend. Once I knew that I was going to be working on Python depedencies, I started hunting for a requirements.txt
file. However, I didn’t find it; I found a setup.py
file instead.
After looking through it, I found a two places that it declares deps: for Python 2 and for Python 3. I took the list and pasted it into my Nix file, changed the syntax to match, and looked up the names of the Python packages as they appear in Nix instead of PyPI, and I ended up with this:
let
my-python-packages = python-packages: with python-packages; [
alembic
Babel
bcrypt
celery
certifi
configobj
dateutil
email_validator
exifread
gst-python
itsdangerous
jinja2
jsonschema
lxml
markdown
oauthlib
PasteDeploy
pasteScript
pillow
PyLD
pytest
pytest_xdist
pytz
requests
six
sphinx
sqlalchemy
unidecode
virtualenv
waitress
webtest
werkzeug
wtforms
];
python-with-my-packages = python3.withPackages my-python-packages;
in stdenv.mkDerivation rec {
The two variables that are bound in this let
statement show a common pattern for adding Python to a Nix package. The python3.withPackages
function call takes a function that takes a list of Python packages and selects a subset. That’s what the my-python-packages
does: it defines that filter function.
Python version mismatch
When I ran the build this time, I got this error:
error: cheetah-2.4.4 not supported for interpreter python3.8
So, it seems that a python package is missing for the Python 3.8, which is the assumed Python version when you use python3
in Nix currently. Let’s see what happens when we change python3
to python37
.
error: cheetah-2.4.4 not supported for interpreter python3.7
Oh boy. And I thought this was going to be easy.
The thing I did next was to see where cheetah-2.4.4
was in the official nixpkgs collection, if anywhere. I ran nix search python cheeta
and I got these results:
* nixpkgs.python27Packages.TurboCheetah (python2.7-TurboCheetah)
TurboGears plugin to support use of Cheetah templates
* nixpkgs.python27Packages.cheetah (python2.7-cheetah)
A template engine and code generation tool
* nixpkgs.python37Packages.TurboCheetah (python3.7-TurboCheetah)
TurboGears plugin to support use of Cheetah templates
* nixpkgs.python38Packages.TurboCheetah (python3.8-TurboCheetah)
TurboGears plugin to support use of Cheetah templates
OK, so cheetah
has not made it to the Python 3 side of nixpkgs.
So, this blog post has become long enough, so I’m going to do some investigation to help solve this problem, and report back next week.
Thanks for reading!