Building MozJS with Nix

Nix is a package manager and build system for Linux and macOS. It is designed to be able to self-host an operating system (called NixOS) and to be able to run alongside an existing UNIX system (such as Ubuntu or macOS).

This tutorial shows how to use Nix to build a somewhat complicated Rust project.

= Getting started =

Each step in this tutorial can be found in the mozjs-example GitHub repository. You can follow along with the commits in the repository, since each one corresponds to a step here.

Start out by installing the nix package manager.

= Set up a crate =

View on GitHub

We'll start by putting some files and Cargo dependencies. Why are we even going to bother with Nix? Because this crate depends on SpiderMonkey (that's MozJS, or the Firefox JavaScript engine), and that has very specific dependencies on its C compiler and available libraries.

You can "cargo build" if you want, but unless you have libclang lying around, it won't build.

As is normal for Rust projects, this will generate a Cargo.toml and a source file]. The example code has added some code, plus a couple of dependencies.

# Cargo.toml [package] name = "mozjs-example" version = "0.0.1" authors = ["Michael Howell "] edition = "2018"

[dependencies] smol = "0.1.5" mozjs = { git = "https://github.com/servo/rust-mozjs" }

= Set up niv =

View on GitHub

This step is going to depend on niv, the nix version manager and the tool we use to handle auto-updating dependencies.

$ nix-env -i niv

Once that's installed, we'll use niv to generate a dependency file for the project repository itself. By making the nix repository a fixed constant, we can promise that as long as you have a spec-compliant nix interpreter, you'll be able to build the app itself.

$ niv init $ niv update nixpkgs -o NixOS -b master -r nixpkgs

The result will be two files: nix/sources.json and nix/sources.nix. The nix file is a script for importing the json file. The nix file is identical across almost all nix projects, since its purpose is just to bootstrap it, while the json file (since it's json) is easy to automatically update.

The step where we update to a newer nixpkgs is needed because we depend on some features that are currently only in nightly.

= Set up crate2nix and nix-shell = View on GitHub $ niv add kolloch/crate2nix

The result of running the niv command will be a new couple of lines in the dependencies file (and also switch to a newer version of nixpkgs itself, because the Rust compiler in the old version of nixpkgs is really, really old).

To really use it, though, we'll want to create a nix-shell file. Let's do a couple of things to make everything easier for us.

# nix/dependencies.nix
 * 1) This is a file we'll write ourselves to make it easier to ensure that configurations stay the same between each environment, both build and dev.

let sources = import ./sources.nix; pkgs = import sources.nixpkgs { }; in { # This will import our pinned instance of the nixpkgs upstream repository. inherit pkgs; # This will contain all of the CLI tools that our shell uses. devDeps = [ (import sources.crate2nix {inherit pkgs;}) pkgs.llvmPackages.lld pkgs.yasm pkgs.binutils pkgs.cargo pkgs.llvm pkgs.autoconf213 pkgs.python3 pkgs.python2 pkgs.which pkgs.perl ]; # Pinned versions of other tools. libclang = pkgs.llvmPackages.libclang; }

Then let's write the shell.nix file, which actually defines the shell proper.

# shell.nix

let # Import dependencies (this preamble will be very common) dependencies = import ./nix/dependencies.nix; pkgs = dependencies.pkgs; in # A "shell", in nix, defines the environment variables and other dependencies # for doing development. The "clangStdenv" line makes it use LLVM's C compiler # instead of the default GNU C Compiler, because that's what SpiderMonkey # needs. pkgs.mkShell.override { stdenv = pkgs.clangStdenv; } { # This name is pretty much arbitrary. name = "mozjs-example"; # This makes it pull in crate2nix and other tools. buildInputs = dependencies.devDeps; # This is needed to build SpiderMonkey. LIBCLANG_PATH = "${dependencies.libclang.lib}/lib"; }

Now, if you run nix-shell, it should pull in crate2nix and let you use it.

= Generate Cargo.nix and write default.nix = View on GitHub

First, run nix-shell, then use cargo to generate a lockfile, then run crate2nix:

$ nix-shell nix$ cargo update nix$ crate2nix generate

And then, finally, let's write our main build script.

# default.nix {release ? true}: let dependencies = import ./nix/dependencies.nix; pkgs = dependencies.pkgs; # Import Cargo.nix with supplied arguments. cargoNix = pkgs.callPackage ./Cargo.nix { # Pass the "release" flag on. inherit release; # Use our version if nixpkgs. inherit pkgs; nixpkgs = pkgs; # Tell it to build with clang (because mozjs needs it). stdenv = pkgs.clangStdenv; }; in cargoNix.rootCrate.build
 * 1) Allows you to specify if this is a release build on the CLI
 * 2) We're going to modify this file later

= Fix the mozjs-sys build = View on GitHub

Try running. It won't work, instead failing with a message like this:

Building build.rs (mozjs_sys) Running rustc --crate-name build_script_build build.rs --crate-type bin -C opt-level=3 -C codegen-units=8 --edition 2015 --cfg feature="default" --out-dir target/build/mozjs_sys --emit=dep-info,link -L dependency=target/buildDeps --extern bindgen=/nix/store/vf1w3q8svms49bcjhr4ssj57bb5np4aw-rust_bindgen-0.53.3-lib/lib/libbindgen-9df4e066ea.rlib --extern cc=/nix/store/jraman5ssgjs33a92gy6ky6hxr1pidl2-rust_cc-1.0.54-lib/lib/libcc-9e4ac1bad6.rlib --extern walkdir=/nix/store/7qmf2mlfgnjwvj96kvzlaxnr7qjd1jsv-rust_walkdir-2.3.1-lib/lib/libwalkdir-6344f26ad1.rlib --cap-lints allow -L native=/nix/store/0wq80m7c42wbynpxxdzhg0pxh2jqv6ph-rust_libloading-0.5.2-lib/lib/libloading.out -l dl -l static=global_static --color always make: /usr/bin/env: Command not found make: /usr/bin/env: Command not found make: *** [/build/mozjs-9a6d8fc/makefile.cargo:190: maybe-configure] Error 127 thread 'main' panicked at 'assertion failed: result.success', build.rs:177:5 stack backtrace: 0: ::fmt 1: core::fmt::write 2: std::io::Write::write_fmt 3: std::panicking::default_hook:: 4: std::panicking::default_hook 5: std::panicking::rust_panic_with_hook 6: std::panicking::begin_panic 7: build_script_build::main 8: std::rt::lang_start:: 9: std::panicking::try::do_call 10: __rust_maybe_catch_panic 11: std::rt::lang_start_internal 12: main 13: __libc_start_main 14: _start note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace. builder for '/nix/store/5w87199rvl60ksj5b7a05fw4g03hdjcs-rust_mozjs_sys-0.68.1.drv' failed with exit code 101 cannot build derivation '/nix/store/mzny9101sixpg4lb11a0r7wyv33wm5id-rust_mozjs-example-0.0.1.drv': 1 dependencies couldn't be built error: build of '/nix/store/mzny9101sixpg4lb11a0r7wyv33wm5id-rust_mozjs-example-0.0.1.drv' failed

To fix this, we need to give mozjs-sys access to its non-Rust dependencies, and patch its build script. Here's the changed version of dependencies.nix:

# default.nix {release ? true}: let dependencies = import ./nix/dependencies.nix; pkgs = dependencies.pkgs; # Import Cargo.nix with supplied arguments. cargoNix = pkgs.callPackage ./Cargo.nix { # Pass the "release" flag on. inherit release; # Use our version of nixpkgs. inherit pkgs; nixpkgs = pkgs; # Tell it to build with clang (because mozjs needs it). stdenv = pkgs.clangStdenv; }; in cargoNix.rootCrate.build.override { crateOverrides = pkgs.defaultCrateOverrides // { encoding_c_mem = {src, ...}: { # Copy .h files from target directory to output directory. # encoding_c_mem assumes that its include file in target/ will be     # available to crates that depend on it. This is not true in Nix. postInstall = '' echo "export DEP_ENCODING_C_MEM_INCLUDE_DIR=$lib/include" > $lib/env cp include -rP $lib/include '';   };    mozjs-example = {src, ...}: { # Expose libclang. extraRustcOpts=["-C" "linker=${pkgs.llvmPackages.clang}/bin/clang" "-C" "link-arg=-fuse-ld=${pkgs.llvmPackages.lld}/bin/ld.lld"]; LIBCLANG_PATH = "${dependencies.libclang.lib}/lib"; # Need to make the location of the system libraries available to `bindgen`, which reads the CLANGFLAGS env variable to get at this data. CLANGFLAGS="-isystem ${pkgs.clangStdenv.lib.getDev pkgs.clangStdenv.cc.cc}/lib/clang/${pkgs.clangStdenv.cc.cc.version}/include -isystem ${pkgs.clangStdenv.cc.cc.gcc}/include/c++/${pkgs.clangStdenv.cc.cc.gcc.version}/${pkgs.hostPlatform.config} -isystem ${pkgs.clangStdenv.cc.cc.gcc}/include/c++/${pkgs.clangStdenv.cc.cc.gcc.version} -isystem ${pkgs.clangStdenv.lib.getDev pkgs.clangStdenv.cc.libc}/include"; # Need to make sure everybody uses Clang, because mozjs doesn't really support building with GCC. CC="${pkgs.llvmPackages.clang}/bin/clang"; CXX="${pkgs.llvmPackages.clang}/bin/clang"; LD="${pkgs.llvmPackages.lld}/bin/ld.lld"; # Tell it to pull in all our specified deps (like autoconf). buildInputs = dependencies.devDeps; };   mozjs = {src, ...}: { # Expose libclang. extraRustcOpts=["-C" "linker=${pkgs.llvmPackages.clang}/bin/clang" "-C" "link-arg=-fuse-ld=${pkgs.llvmPackages.lld}/bin/ld.lld"]; LIBCLANG_PATH = "${dependencies.libclang.lib}/lib"; # Need to make the location of the system libraries available to `bindgen`, which reads the CLANGFLAGS env variable to get at this data. CLANGFLAGS="-isystem ${pkgs.clangStdenv.lib.getDev pkgs.clangStdenv.cc.cc}/lib/clang/${pkgs.clangStdenv.cc.cc.version}/include -isystem ${pkgs.clangStdenv.cc.cc.gcc}/include/c++/${pkgs.clangStdenv.cc.cc.gcc.version}/${pkgs.hostPlatform.config} -isystem ${pkgs.clangStdenv.cc.cc.gcc}/include/c++/${pkgs.clangStdenv.cc.cc.gcc.version} -isystem ${pkgs.clangStdenv.lib.getDev pkgs.clangStdenv.cc.libc}/include"; # Need to make sure everybody uses Clang, because mozjs doesn't really support building with GCC. CC="${pkgs.llvmPackages.clang}/bin/clang"; CXX="${pkgs.llvmPackages.clang}/bin/clang"; LD="${pkgs.llvmPackages.lld}/bin/ld.lld"; # Tell it to pull in all our specified deps (like autoconf). buildInputs = dependencies.devDeps; };   mozjs_sys = {src, ...}: { # Expose libclang. extraRustcOpts=["-C" "linker=${pkgs.llvmPackages.clang}/bin/clang" "-C" "link-arg=-fuse-ld=${pkgs.llvmPackages.lld}/bin/ld.lld"]; LIBCLANG_PATH = "${dependencies.libclang.lib}/lib"; # Need to make the location of the system libraries available to `bindgen`, which reads the CLANGFLAGS env variable to get at this data. CLANGFLAGS="-isystem ${pkgs.clangStdenv.lib.getDev pkgs.clangStdenv.cc.cc}/lib/clang/${pkgs.clangStdenv.cc.cc.version}/include -isystem ${pkgs.clangStdenv.cc.cc.gcc}/include/c++/${pkgs.clangStdenv.cc.cc.gcc.version}/${pkgs.hostPlatform.config} -isystem ${pkgs.clangStdenv.cc.cc.gcc}/include/c++/${pkgs.clangStdenv.cc.cc.gcc.version} -isystem ${pkgs.clangStdenv.lib.getDev pkgs.clangStdenv.cc.libc}/include"; # Need to make sure everybody uses Clang, because mozjs doesn't really support building with GCC. CC="${pkgs.llvmPackages.clang}/bin/clang"; CXX="${pkgs.llvmPackages.clang}/bin/clang"; LD="${pkgs.llvmPackages.lld}/bin/ld.lld"; # Tell it to pull in all our specified deps (like autoconf). buildInputs = dependencies.devDeps; prePatch = '' # Remove reference to /usr/bin/env from build scripts, # because that app doesn't actually exist in the Nix build sandbox. substituteInPlace makefile.cargo --replace '/usr/bin/env bash' '${pkgs.bash}/bin/bash' '';     # Copy .h files from target directory to output directory. # mozjs_sys assumes that its include file in target/ will be     # available to crates that depend on it. This is not true in Nix. postInstall = '' echo "export DEP_MOZJS_OUTDIR=$lib/lib/mozjs_sys.out/build/" > $lib/env cd $lib/lib/mozjs_sys.out/build/ xd=$(pwd) find. | grep -v virtualenv | while read path; do         target=`readlink $path || true` if [ -n "$target" ]; then dirname=`dirname $path` filename=`basename -s "" $path` cd "$dirname" echo "$target => $path" rm -f "$filename" cp -f "$target" "$filename" cd "$xd" fi       done '';   };  }; }
 * 1) Allows you to specify if this is a release build on the CLI

= Troubleshooting =

It says I'm out of disk space?
On my Ubuntu-based computer, the build ran in, in a tmpfs folder that didn't have room enough to build mozjs. I expanded it by running this command as root, since I actually do have enough memory to store all the temporary files:

# mount -o remount,size=8613616k /run/user/1000/

You can also change where build files are put using the  environment variable.