Nix users often create flakes to package software out of tree, like this Zen
+Browser flake I’ve been
+maintaining. Keeping them up to date is a hassle though, since you have to
+update the Subresource Integrity (SRI) hashes that Nix uses to ensure
+reproducibility.
+Here’s a neat method I’ve been using to cleanly handle automatic hash updates.
+I use Nushell to easily work with data, prefetch
+some hashes, and put it all in a JSON file that can be read by Nix at build
+time.
+First, let’s create a file called update.nu. At the top, place this shebang:
+#!/usr/bin/env -S nix shell nixpkgs#nushell --command nu
+This will execute the script in a Nushell environment, which is fetched by Nix.
+Get the up to date URLs
+We need to obtain the latest version of whatever software we want to update.
+In this case, I’ll use GitHub releases as my source of truth.
+You can use the GitHub API to fetch metadata about all the releases of a repository.
+https://api.github.com/repos/($repo)/releases
+Roughly speaking, the raw JSON returned by the GitHub releases API looks something like:
+[
+ {tag_name: "foo", prerelease: false, ...},
+ {tag_name: "bar", prerelease: true, ...},
+ {tag_name: "foobar", prerelease: false, ...},
+]
+
+Note that the ordering of the objects in the array is chronological.
+
+Even if you aren’t using GitHub releases, as long as there is a reliable way to
+programmatically fetch the latest download URLs of whatever software you’re
+packaging, you can adapt this approach for your specific case.
+
+We use Nushell’s http get to make a network request. Nushell will
+automatically detect and parse the JSON reponse into a Nushell table.
+In my case, Zen Browser frequently publishes prerelease “twilight” builds which
+we don’t want to update to. So, we ignore any releases tagged “twilight” or
+marked “prerelease” by filtering them out with the where selector.
+Finally, we retrieve the tag name of the item at the first index, which would
+be the latest release (since the JSON array was chronologically sorted).
+#!/usr/bin/env -S nix shell nixpkgs#nushell --command nu
+
+# get the latest tag of the latest release that isn't a prerelease
+def get_latest_release [repo: string] {
+ try {
+ http get $"https://api.github.com/repos/($repo)/releases"
+ | where prerelease == false
+ | where tag_name != "twilight"
+ | get tag_name
+ | get 0
+ } catch { |err| $"Failed to fetch latest release, aborting: ($err.msg)" }
+}
+Prefetching SRI hashes
+Now that we have the latest tags, we can easily obtain the latest download URLs, which are of the form:
+https://github.com/zen-browser/desktop/releases/download/$tag/zen.linux-x86_64.tar.bz2
+https://github.com/zen-browser/desktop/releases/download/$tag/zen.aarch64-x86_64.tar.bz2
+However, we still need the corresponding SRI hashes to pass to Nix.
+src = fetchurl {
+ url = "https://github.com/zen-browser/desktop/releases/download/1.0.2-b.5/zen.linux-x86_64.tar.bz2";
+ hash = "sha256-00000000000000000000000000000000000000000000";
+};
The easiest way to obtain these new hashes is to update the URL and then set
+the hash property to an empty string (""). Nix will spit out an hash mismatch
+error with the correct hash. However, this is inconvenient for automated
+command line scripting.
+The Nix documentation mentions
+nix-prefetch-url
+as a way to obtain these hashes, but as usual, it doesn’t work quite right and
+has also been replaced by a more powerful but underdocumented experimental
+feature instead.
+The nix store
+prefetch-file
+command does what nix-prefetch-url is supposed to do, but handles the caveats
+that lead to the wrong hash being produced automatically.
+Let’s write a Nushell function that outputs the SRI hash of the given URL. We
+tell prefetch-file to output structured JSON that we can parse.
+Since Nushell is a shell, we can directly invoke shell commands like usual,
+and then process their output with pipes.
+def get_nix_hash [url: string] {
+ nix store prefetch-file --hash-type sha256 --json $url | from json | get hash
+}
+Cool! Now get_nix_hash can give us SRI hashes that look like this:
+sha256-K3zTCLdvg/VYQNsfeohw65Ghk8FAjhOl8hXU6REO4/s=
+Putting it all together
+Now that we’re able to fetch the latest release, obtain the download URLs, and
+compute their SRI hashes, we have all the information we need to make an
+automated update. However, these URLs are typically hardcoded in our Nix
+expressions. The question remains as to how to update these values.
+A common way I’ve seen updates performed is using something like sed to
+modify the Nix expressions in place. However, there’s actually a more
+maintainable and easy to understand approach.
+Let’s have our Nushell script generate the URLs and hashes and place them in a
+JSON file! Then, we’ll be able to read the JSON file from Nix and obtain the
+URL and hash.
+def generate_sources [] {
+ let tag = get_latest_release "zen-browser/desktop"
+ let prev_sources = open ./sources.json
+
+ if $tag == $prev_sources.version {
+ # everything up to date
+ return $tag
+ }
+
+ # generate the download URLs with the new tag
+ let x86_64_url = $"https://github.com/zen-browser/desktop/releases/download/($tag)/zen.linux-x86_64.tar.bz2"
+ let aarch64_url = $"https://github.com/zen-browser/desktop/releases/download/($tag)/zen.linux-aarch64.tar.bz2"
+
+ # create a Nushell record that maps cleanly to JSON
+ let sources = {
+ # add a version field as well for convenience
+ version: $tag
+
+ x86_64-linux: {
+ url: $x86_64_url
+ hash: (get_nix_hash $x86_64_url)
+ }
+ aarch64-linux: {
+ url: $aarch64_url
+ hash: (get_nix_hash $aarch64_url)
+ }
+ }
+
+ echo $sources | save --force "sources.json"
+
+ return $tag
+}
+Running this script with
+chmod +x ./update.nu
+./update.nu
gives us the file sources.json:
+{
+ "version": "1.0.2-b.5",
+ "x86_64-linux": {
+ "url": "https://github.com/zen-browser/desktop/releases/download/1.0.2-b.5/zen.linux-x86_64.tar.bz2",
+ "hash": "sha256-K3zTCLdvg/VYQNsfeohw65Ghk8FAjhOl8hXU6REO4/s="
+ },
+ "aarch64-linux": {
+ "url": "https://github.com/zen-browser/desktop/releases/download/1.0.2-b.5/zen.linux-aarch64.tar.bz2",
+ "hash": "sha256-NwIYylGal2QoWhWKtMhMkAAJQ6iNHfQOBZaxTXgvxAk="
+ }
+}
Now, let’s read this from Nix. My file organization looks like the following:
+./
+| flake.nix
+| zen-browser-unwrapped.nix
+| ...other files...
+zen-browser-unwrapped.nix contains the derivation for Zen Browser. Let’s add
+version, url, and hash to its inputs:
+{
+ stdenv,
+ fetchurl,
+ # add these below
+ version,
+ url,
+ hash,
+ ...
+}:
+stdenv.mkDerivation {
+ # inherit version from inputs
+ inherit version;
+ pname = "zen-browser-unwrapped";
+
+ src = fetchurl {
+ # inherit the URL and hash we obtain from the inputs
+ inherit url hash;
+ };
+}
Then in flake.nix, let’s provide the derivation with the data from sources.json:
+let
+ supportedSystems = [
+ "x86_64-linux"
+ "aarch64-linux"
+ ];
+ forAllSystems = nixpkgs.lib.genAttrs supportedSystems;
+in
+{
+ # rest of file omitted for simplicity
+ packages = forAllSystems (
+ system:
+ let
+ pkgs = import nixpkgs { inherit system; };
+ # parse sources.json into a Nix attrset
+ sources = builtins.fromJSON (builtins.readFile ./sources.json);
+ in
+ rec {
+ zen-browser-unwrapped = pkgs.callPackage ./zen-browser-unwrapped.nix {
+ inherit (sources.${system}) hash url;
+ inherit (sources) version;
+
+ # if the above is difficult to understand, it is equivalent to the following:
+ hash = sources.${system}.hash;
+ url = sources.${system}.url;
+ version = sources.version;
+ };
+}
Now, running nix build .#zen-browser-unwrapped will be able to use the hashes
+and URLs from sources.json to build the package!
+Automating it in CI
+We now have a script that can automatically fetch releases and generate hashes
+and URLs, as well as a way for Nix to use the outputted JSON to build
+derivations. All that’s left is to fully automate it using CI!
+We are going to use GitHub actions for this, as it’s free and easy and you’re
+probably already hosting on GitHub.
+Ensure you’ve set up actions for your repo and given it sufficient permissions.
+We’re gonna run it on a cron timer that checks for updates at 8 PM PST every day.
+We use DeterminateSystems’ actions to help set up Nix. Then, we simply run our
+update script. Since we made the script return the tag it fetched, we can store
+it in a variable and then use it in our commit message.
+name: Update to latest version, and update flake inputs
+
+on:
+ schedule:
+ - cron: "0 4 * * *"
+ workflow_dispatch:
+
+jobs:
+ update:
+ name: Update flake inputs and browser
+ runs-on: ubuntu-latest
+ steps:
+ - name: Checkout Repository
+ uses: actions/checkout@v4
+
+ - name: Check flake inputs
+ uses: DeterminateSystems/flake-checker-action@v4
+
+ - name: Install Nix
+ uses: DeterminateSystems/nix-installer-action@main
+
+ - name: Set up magic Nix cache
+ uses: DeterminateSystems/magic-nix-cache-action@main
+
+ - name: Check for update and perform update
+ run: |
+ git config --global user.name "github-actions[bot]"
+ git config --global user.email "github-actions[bot]@users.noreply.github.com"
+
+ chmod +x ./update.nu
+ export ZEN_LATEST_VER="$(./update.nu)"
+
+ git add -A
+ git commit -m "github-actions: update to $ZEN_LATEST_VER" || echo "Latest version is $ZEN_LATEST_VER, no updates found"
+
+ nix flake update --commit-lock-file
+
+ git push
+Now, our repository will automatically check for and perform updates every day!
+
+]]>