ccpkgs/README.md

# Overview

This repository overlays the [upstream nixpkgs-channels
repository](https://github.com/NixOS/nixpkgs-channels) with fixes and
additional packages to create a repository suitable for use on Compute
Canada HPC clusters.  It is available to use by default on Compute
Canada clusters.  It's layout is modelled after the upstream one.


# Using

The `nix` module must be loaded

```bash
module load nix
```

To update to the latest version of these packages (i.e., the last one
for which all overlays built successfully) do

```bash
nix-channel --update
```

This is safe to do as the prior version can always be returned to by
doing a rollback

```bash
nix-channel --rollback
```

The `nix-env` command can be used to install, update, or remove
packages from the environment.

## Installing

To install (or update) a package, specify the packages attribute to
`nix-env` using the `--attr` option

```bash
nix-env --install --attr nixpkgs.attribute
```

where `attribute` is replaced with the attribute that designates the
package (e.g., `emacs`).

## Removing

Removing a package requires knowing its name.  Get this by querying
the current environment

```bash
nix-env --query
```

Then it can be uninstalled with the `--uninstall` options

```bash
nix-env --uninstall name
```

where `name` is a placeholder for the name returned by `--query`
(e.g., emacs).

## Reverting

These commands are always safe to do as they can always be undone by
doing a rollback

```bash
nix-env --rollback
```

## Using

A development environment consisting of a given set of packages can be
entered using the `nix-shell` command with the `--packages` option
(unlike `nix-env`, this automatically uses the `nixpkgs` channel)

```bash
nix-shell --packages attribute ...
```

Dependencies of non-installed packages are not tracked, so it is not
guaranteed that the specified packages will remain on the system for
more than the current day (nor does the fact that they sometimes
remain for more than a day imply they will always remain).  Should
they be removed, rerunning the command will temporarily reinstall them
again.


# Developing

To add to this repository, `git clone` it into a local directory.

```bash
git clone https://git.computecanada.ca/nix/ccpkgs.git
cd ccpkgs
```

As it is an overlay, it also requires a matching release of the
upstream `nixpkgs-channels` repository.  This should be cloned into
the *upsteam* subdirectory (if this directory does not it exist it
will use the current channel if it is the correct version)

```bash
git clone https://github.com/NixOS/nixpkgs-channels.git -b nixos-$(< .version) upstream
```

## Layout

Longer term items, like package additions, are put in the *pkgs*
subdirectory.  Shorter term items, like temporary upstream
workarounds, are placed in the *temporary* subdirectory.  The
*pkgs/all-packages.nix* and *temporary/all-temporary.nix* files are
applied as [overlays](https://nixos.org/nixpkgs/manual/#chap-overlays)
on top of the the upstream nixpkgs-channel.  This means attributes
defined in them (first *all-temporary.nix* and then
*all-packages.nix*) will replace or augment attributes defined in
nixpkgs-channel.

Inside both of these files, the default scope is the final attribute
set (i.e., after all overlays have been applied).  The final, prior
(before the file's overlays are applied), and next (after the file's
overlays are applied) attribute sets are also available under the
attributes `pkgs`, `super`, and `self`.  Consider the `emacs`
attribute for example.  `emacs` and `pkgs.emacs` refer to the final
`emacs` definition.  The prior (before this overlay) definition is
available as `super.emacs`, and the next (after this overlay)
definition is available as `self.emacs`.

## Adding

Packages are added by creating an appropriate *pkgs/.../package.nix*
or, if additional patch files and such are required,
*pkgs/.../package/default.nix* expression, where `package` is a
placeholder for an appropriate name (e.g., `emacs`).  This expression
is then assigned to an attribute by adding an
`attribute = callPackage ...` line to *pkgs/all-packages.nix*, where
`attribute` is replaced with the attribute that will designate the
package (e.g., `emacs`).

The easiest way to create the nix expression, is to look through the
existing overlay expressions, find one that is suitably close to what
is desired, and then duplicate and modify it.  [Upstream
nixpkgs](https://github.com/NixOS/nixpkgs/tree/master/pkgs)
expressions can also be used as a starting point as they also follow
the exact same format.

More information on creating packages can be found in the [Nixpkgs
Contributor Guide](https://nixos.org/nixpkgs/manual/).  More
information about the nix expression language itself can be found in
the [Nix Package Manager
Guide](https://nixos.org/nix/manual/#ch-expression-language).

## Testing

Nix expressions can be build from the top-level directory by using the
`nix-build` command.  For example

```bash
nix-build --attr attribute
```

It is highly recommended to do this before pushing any changes to the
master repository as any breakage will prevent further releases of the
channel to the clusters until fixed.

## Debugging

The `nix-shell` command can be used to enter a non-sandboxed version
of the build environment from the top-level directory.  Passing the
`--pure` option causes it to purge the environment (RHEL and Fedora do
not guard against multiple evaluations of the */etc/profile.d* scripts
via `__ETC_PROFILE_SOURCED` so this may not be 100% effective)

```bash
nix-shell --pure --attr attribute
```

`NIX_ENFORCE_PURITY` should be set in the entered environment to
enable additional checks against system libraries making it into the
build.  The build and install locations should also be reset to
somewhere writable (e.g., */tmp*)

```bash
export NIX_ENFORCE_PURITY=1
NIX_BUILD_TOP=/tmp/nix-$USER
out=$NIX_BUILD_TOP/out
prefix=$out
```
```bash
mkdir -p "$NIX_BUILD_TOP" "$out"
cd "$NIX_BUILD_TOP"
```

After this, the standard [build
phases](https://nixos.org/nixpkgs/manual/#sec-stdenv-phases) can be
manually invoked and debugged

```bash
eval "${unpackPhase:-unpackPhase}"
cd "$sourceRoot"
```
```bash
eval "${patchPhase:-patchPhase}"
eval "${configurePhase:-configurePhase}"
eval "${buildPhase:-buildPhase}"
eval "${installPhase:-installPhase}"
```

where the above evaluates the `*Phase` environment variable if it
exists (the phase is overridden) or otherwise calls the standard
`*Phase` function (the phase is not overridden).  In some extreme
cases it can be useful to look at the [actual
implementation](https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/generic/setup.sh)
of these functions and the `genericBuild` function which ties
them all together.


# Deployment

Both this repository and nixpkgs-channel are regularly checked for
updates by the [hydra server](https://hydra.sharcnet.ca).  If a newer
version of either of these are found, the hydra server automatically
clones them and attempts to build all the overlays defined in
*pkgs/all-packages.nix* and *temporary/all-temporary.nix* and a channel
tarball too.  The Compute Canada clusters monitor this process, and,
if it succeeds, download the channel tarball and make it available as
a new release.

This means deployment is as simple as doing (please prefix commits
message with `package: ...` as is done upstream)

```bash
git push
```

and waiting for a while (see the [hydra
server](https://hydra.sharcnet.ca/) to check on progress).  Once it
propagates to the clusters, anyone can update to it by running

```bash
nix-channel --update
```

## Normal Changes

As mentioned in the [developing section](#testing), new or changed
attribute should first be test built before pushing changes upstream

```bash
nix-build --attr attribute
```

where `attribute` is replaced with the appropriate attribute (e.g.,
`emacs`).

## Sweeping Changes

The top-level *release.nix* expression contains the builds performed
by the hydra server.  If significant changes have been made, these can
be manually tested before pushing

```bash
nix-build release.nix --arg stableBranch true --attr tested
```

A build of the channel tarball can also be done

```bash
nix-build release.nix --arg stableBranch true --attr channel
```

The resulting channel tarball can even be imported as new channel
and used the same as the default channel

```bash
nix-channel --add "file://$(echo "$PWD"/result/tarballs/*)" channel
nix-channel --update channel
nix-env --install --attr channel.attribute
```

where `channel` is a place holder for an arbitrary new channel name
(e.g., `test`) and `attribute` for some attribute (e.g., `emacs`).