More Improvements in SingularityCE 3.9.0

We previously took a look at improved Docker compatibility and NVIDIA GPU Support coming to SingularityCE 3.9.0. The improvements we talked about in those posts are the headline features for 3.9.0, but we’ve been busy on a number of smaller changes and fixes too.

This week, let’s take a look at some of the other improvements you’ll benefit from when 3.9.0 is released on November 16th.

Writable tmpfs for build test sections – A SingularityCE definition file can include a %test section, which will be run during the build process. Previously, it was a bit difficult to use as the test script always ran in a read-only filesystem. You can now use the --writable-tmpfs flag with singularity build so that the %test script is able to create files when needed, inside a temporary filesystem that is cleaned up when the build completes.

Multi-stream library:// pulls – A singularity pull against a library:// URI will now attempt to download the image using multiple streams, each fetching parts of the image concurrently. This improves download speeds from the Sylabs Cloud and on-premise Singularity Enterprise installations.

Support for http-only remotes – Community projects such as hinkskalle and sregistry support the library:// protocol and can be configured with singularity remote add. For local testing we now support unenecrypted http:// connections for remote service discovery, via a new --insecure flag for singularity remote add.

Updated ORAS support – We updated the libraries we use for ORAS push/pull, and fixed some issues in the CLI code. This means SingularityCE can now push/pull successfully to a wider range of OCI registries, and won’t get stuck trying.

Consistent external binary handling – In SingularityCE 3.9.0 the way in which we find external binaries has been reviewed, and changed for consistency and clarity. For each external program we call (such as mksquashfs) we now have a single method of finding it. Previously, the code sometimes switched between PATH searches and configuration file settings for the same binary, which could result in confusing behavior. The improved handling of external binaries is documented in our admin guide. We also list the complete sets of external dependencies for various distribution bootstraps etc.

Cgroups v2 Support – With more distributions adopting the v2 unified cgroups hierarchy, we have added support for this to the SingularityCE oci command group, and --cgroups flag. SingularityCE continues to its existing TOML cgroups rules format, and will apply it to the new unified hierarchy.

Hopefully you’ll agree that 3.9.0 is a pretty comprehensive release! Check out the changelog for more, and grab the latest release candidate to test out the new stuff. We’d be really grateful for any testing, comments, and bug reports as we get everything tidied up for the stable release in just over a week.

In addition, as always, please let us know your ideas for future releases by commenting on our open SingularityCE roadmap, or reaching us on the community channels.