diff --git a/README.md b/README.md index b925f13..373c6cd 100644 --- a/README.md +++ b/README.md @@ -4,17 +4,13 @@ This repository overlays the [upstream nixpkgs-channels repository](https://github.com/NixOS/nixpkgs-channels) with fixes and additional packages to create a repository suitable for use on Compute Canada HPC clusters. It is available to use by default on Compute -Canada clusters. It's layout is modelled after the [upstream -nixpkgs](https://github.com/NixOS/nixpkgs). +Canada clusters. It's layout is modelled after the upstream one. # Using -This repository overlayed onto the latest [upstream -nixpkgs-channel](https://github.com/NixOS/nixpkgs-channels) is the -default channel on Compute Canada clusters. To update to the latest -version (i.e., the last one for which all overlays built successfully) -do +To update to the latest version (i.e., the last one for which all +overlays built successfully) do ```bash nix-channel --update @@ -39,8 +35,8 @@ To install (or update) a package, specify the packages attribute to nix-env --install --attr attribute ``` -where `attribute` is a placeholder for the attribute that designates -the desired package (e.g., `emacs`). +where `attribute` is replaced with the attribute that designates the +package (e.g., `emacs`). ## Removing @@ -88,7 +84,7 @@ again. # Developing -To add to this this repository, `git clone` it into a local directory. +To add to this repository, `git clone` it into a local directory. ```bash git clone https://git.computecanada.ca/nix/ccpkgs.git @@ -106,41 +102,42 @@ git clone https://github.com/NixOS/nixpkgs-channels.git -b nixos-$(< .version) u ## Layout -Longer term overlays, like package additions, are put in the *pkgs* -subdirectory. Shorter term overlays, like temporary upstream +Longer term items, like package additions, are put in the *pkgs* +subdirectory. Shorter term items, like temporary upstream workarounds, are placed in the *temporary* subdirectory. The *pkgs/all-packages.nix* and *temporary/all-temporary.nix* files are applied as [overlays](https://nixos.org/nixpkgs/manual/#chap-overlays) on top of the the upstream nixpkgs-channel. This means attributes defined in them (first *all-temporary.nix* and then -*all-packages.nix*) will replace attributes defined in +*all-packages.nix*) will replace or augment attributes defined in nixpkgs-channel. Inside both of these files, the default scope is the final attribute set (i.e., after all overlays have been applied). The final, prior (before the file's overlays are applied), and next (after the file's overlays are applied) attribute sets are also available under the -top-level attributes `pkgs`, `super`, and `self`. Consider the -`emacs` attribute for example. `emacs` and `pkgs.emacs` refer to the -final `emacs` definition. The prior (before this overlay) and next -(after this overlay) definition are also available though -as`super.emacs` and `self.emacs`, respectively. +attributes `pkgs`, `super`, and `self`. Consider the `emacs` +attribute for example. `emacs` and `pkgs.emacs` refer to the final +`emacs` definition. The prior (before this overlay) definition is +available as `super.emacs`, and the next (after this overlay) +definition is available as `self.emacs`. ## Adding Packages are added by creating an appropriate *pkgs/.../package.nix* -or, if more multiple files are required, +or, if additional patch files and such are required, *pkgs/.../package/default.nix* expression, where `package` is a -placeholder for a suitably descriptive (e.g., `emacs`), add then -adding a `package = callPackage ...` line into -*pkgs/all-packages.nix*. This makes available under the attribute -`package`. +placeholder for an appropriate name (e.g., `emacs`). This expression +is then assigned to an attribute by adding an `attribute = callPackage +...` line to *pkgs/all-packages.nix*, where `attribute` is replaced +with the attribute that will designate the package (e.g., `emacs`). The easiest way to create the nix expression, is to look through the existing overlay expressions, find one that is suitably close to what is desired, and then duplicate and modify it. [Upstream -nixpkgs](https://github.com/NixOS/nixpkgs) expressions can also be -used as a starting point as they also follow the exact same format. +nixpkgs](https://github.com/NixOS/nixpkgs/tree/master/pkgs) +expressions can also be used as a starting point as they also follow +the exact same format. More information on creating packages can be found in the [Nixpkgs Contributor Guide](https://nixos.org/nixpkgs/manual/). More @@ -157,10 +154,9 @@ Nix expressions can be build from the top-level directory by using the nix-build --attr attribute ``` -where `attribute` is replaced with the appropriate attribute (e.g., -`emacs`). It is highly recommended to do this before pushing any -changes to the master repository as any breakage will prevent further -releases of the channel to the clusters until fixed. +It is highly recommended to do this before pushing any changes to the +master repository as any breakage will prevent further releases of the +channel to the clusters until fixed. ## Debugging @@ -168,9 +164,9 @@ The `nix-shell` command can be used to enter a non-sandboxed version of the build environment from the top-level directory. Passing the `--pure` option causes it to purge the environment. RHEL and Fedora do not guard against multiple evaluations of the */etc/profile.d* -scripts though, so the environment will re-initialize itself from -*~/.bashrc* script unless it is modified to not source */etc/bashrc* -when `__ETC_PROFILE_SOURCED` is set +scripts though, so the default *~/.bashrc* will restore it unless it +is modified to not source */etc/bashrc* when `__ETC_PROFILE_SOURCED` +is set ```bash sed -i -e 's|^\(\s*\)\(\. /etc/bashrc\)\s*$|\1test -n "$__ETC_PROFILE_SOURCED" \|\| \2|' ~/.bashrc @@ -192,13 +188,13 @@ somewhere writable (e.g., */tmp*) ```bash export NIX_ENFORCE_PURITY=1 NIX_BUILD_TOP=/tmp/nix-$USER - -mkdir "$NIX_BUILD_TOP" -cd "$NIX_BUILD_TOP" -mkdir out -out=$PWD/out +out=$NIX_BUILD_TOP/out prefix=$out ``` +```bash +mkdir -p "$NIX_BUILD_TOP" "$out" +cd "$NIX_BUILD_TOP" +``` After this, the standard [build phases](https://nixos.org/nixpkgs/manual/#sec-stdenv-phases) can be @@ -207,7 +203,8 @@ manually invoked and debugged ```bash eval "${unpackPhase:-unpackPhase}" cd "$sourceRoot" - +``` +```bash eval "${patchPhase:-patchPhase}" eval "${configurePhase:-configurePhase}" eval "${buildPhase:-buildPhase}" @@ -229,7 +226,7 @@ Both this repository and nixpkgs-channel are regularly checked for updates by the [hydra server](https://hydra.sharcnet.ca). If a newer version of either of these are found, the hydra server automatically clones them and attempts to build all the overlays defined in -*pkgs/all-packages.nix* and temporary/all-temporary.nix* and a channel +*pkgs/all-packages.nix* and *temporary/all-temporary.nix* and a channel tarball too. The Compute Canada clusters monitor this process, and, if it succeeds, download the channel tarball and make it available as a new release. @@ -240,9 +237,9 @@ This means deployment is as simple as doing git push ``` -waiting for a while (see the [hydra -server](https://hydra.sharcnet.ca/) to check on progress), and then -doing +and waiting for a while (see the [hydra +server](https://hydra.sharcnet.ca/) to check on progress). Once it +propagates to the clusters, anyone can update to it by running ```bash nix-channel --update