Overview

This repository overlays the upstream nixpkgs-channels repository with fixes and additional packages to create a repository suitable for use on Compute Canada HPC clusters. It is available to use by default on Compute Canada clusters. It's layout is modelled after the upstream one.

Using

The nix module must be loaded

module load nix

To update to the latest version of these packages (i.e., the last one for which all overlays built successfully) do

nix-channel --update

This is safe to do as the prior version can always be returned to by doing a rollback

nix-channel --rollback

The nix-env command can be used to install, update, or remove packages from the environment.

Installing

To install (or update) a package, specify the packages attribute to nix-env using the --attr option

nix-env --install --attr nixpkgs.attribute

where attribute is replaced with the attribute that designates the package (e.g., emacs).

Removing

Removing a package requires knowing its name. Get this by querying the current environment

nix-env --query

Then it can be uninstalled with the --uninstall options

nix-env --uninstall name

where name is a placeholder for the name returned by --query (e.g., emacs).

Reverting

These commands are always safe to do as they can always be undone by doing a rollback

nix-env --rollback

Using

A development environment consisting of a given set of packages can be entered using the nix-shell command with the --packages option (unlike nix-env, this automatically uses the nixpkgs channel)

nix-shell --packages attribute ...

Dependencies of non-installed packages are not tracked, so it is not guaranteed that the specified packages will remain on the system for more than the current day (nor does the fact that they sometimes remain for more than a day imply they will always remain). Should they be removed, rerunning the command will temporarily reinstall them again.

Developing

To add to this repository, git clone it into a local directory.

git clone https://git.computecanada.ca/nix/ccpkgs.git
cd ccpkgs

As it is an overlay, it also requires a matching release of the upstream nixpkgs-channels repository. This should be cloned into the upsteam subdirectory (if this directory does not it exist it will use the current channel if it is the correct version)

git clone https://github.com/NixOS/nixpkgs-channels.git -b nixos-$(< .version) upstream

Layout

Longer term items, like package additions, are put in the pkgs subdirectory. Shorter term items, like temporary upstream workarounds, are placed in the temporary subdirectory. The pkgs/all-packages.nix and temporary/all-temporary.nix files are applied as overlays on top of the the upstream nixpkgs-channel. This means attributes defined in them (first all-temporary.nix and then all-packages.nix) will replace or augment attributes defined in nixpkgs-channel.

Inside both of these files, the default scope is the final attribute set (i.e., after all overlays have been applied). The final, prior (before the file's overlays are applied), and next (after the file's overlays are applied) attribute sets are also available under the attributes pkgs, super, and self. Consider the emacs attribute for example. emacs and pkgs.emacs refer to the final emacs definition. The prior (before this overlay) definition is available as super.emacs, and the next (after this overlay) definition is available as self.emacs.

Adding

Packages are added by creating an appropriate pkgs/.../package.nix or, if additional patch files and such are required, pkgs/.../package/default.nix expression, where package is a placeholder for an appropriate name (e.g., emacs). This expression is then assigned to an attribute by adding an attribute = callPackage ... line to pkgs/all-packages.nix, where attribute is replaced with the attribute that will designate the package (e.g., emacs).

The easiest way to create the nix expression, is to look through the existing overlay expressions, find one that is suitably close to what is desired, and then duplicate and modify it. Upstream nixpkgs expressions can also be used as a starting point as they also follow the exact same format.

More information on creating packages can be found in the Nixpkgs Contributor Guide. More information about the nix expression language itself can be found in the Nix Package Manager Guide.

Testing

Nix expressions can be build from the top-level directory by using the nix-build command. For example

nix-build --attr attribute

It is highly recommended to do this before pushing any changes to the master repository as any breakage will prevent further releases of the channel to the clusters until fixed.

Debugging

The nix-shell command can be used to enter a non-sandboxed version of the build environment from the top-level directory. Passing the --pure option causes it to purge the environment (RHEL and Fedora do not guard against multiple evaluations of the /etc/profile.d scripts via __ETC_PROFILE_SOURCED so this may not be 100% effective)

nix-shell --pure --attr attribute

NIX_ENFORCE_PURITY should be set in the entered environment to enable additional checks against system libraries making it into the build. The build and install locations should also be reset to somewhere writable (e.g., /tmp)

export NIX_ENFORCE_PURITY=1
NIX_BUILD_TOP=/tmp/nix-$USER
out=$NIX_BUILD_TOP/out
prefix=$out

mkdir -p "$NIX_BUILD_TOP" "$out"
cd "$NIX_BUILD_TOP"

After this, the standard build phases can be manually invoked and debugged

eval "${unpackPhase:-unpackPhase}"
cd "$sourceRoot"

eval "${patchPhase:-patchPhase}"
eval "${configurePhase:-configurePhase}"
eval "${buildPhase:-buildPhase}"
eval "${installPhase:-installPhase}"

where the above evaluates the *Phase environment variable if it exists (the phase is overridden) or otherwise calls the standard *Phase function (the phase is not overridden). In some extreme cases it can be useful to look at the actual implementation of these functions and the genericBuild function which ties them all together.

Deployment

Both this repository and nixpkgs-channel are regularly checked for updates by the hydra server. If a newer version of either of these are found, the hydra server automatically clones them and attempts to build all the overlays defined in pkgs/all-packages.nix and temporary/all-temporary.nix and a channel tarball too. The Compute Canada clusters monitor this process, and, if it succeeds, download the channel tarball and make it available as a new release.

This means deployment is as simple as doing (please prefix commits message with package: ... as is done upstream)

git push

and waiting for a while (see the hydra server to check on progress). Once it propagates to the clusters, anyone can update to it by running

nix-channel --update

Normal Changes

As mentioned in the developing section, new or changed attribute should first be test built before pushing changes upstream

nix-build --attr attribute

where attribute is replaced with the appropriate attribute (e.g., emacs).

Sweeping Changes

The top-level release.nix expression contains the builds performed by the hydra server. If significant changes have been made, these can be manually tested before pushing

nix-build release.nix --arg stableBranch true --attr tested

A build of the channel tarball can also be done

nix-build release.nix --arg stableBranch true --attr channel

The resulting channel tarball can even be imported as new channel and used the same as the default channel

nix-channel --add "file://$(echo "$PWD"/result/tarballs/*)" channel
nix-channel --update channel
nix-env --install --attr channel.attribute

where channel is a place holder for an arbitrary new channel name (e.g., test) and attribute for some attribute (e.g., emacs).