At Sylabs, our mission is to drive development of the open source Singularity project and expand the surrounding ecosystem positioning Singularity as the de-facto choice for HPC and Enterprise Performance Computing (EPC) workflows. .
Today, we’re happy to share with you a preview of one of the products we have been working on: the Sylabs Remote Build Service, a cloud-based service (It’s not the most creative name, I know, but as a team of software developers, you’ll have to forgive us for our lack of marketing prowess!)
Singularity enables many different workflows by giving you the opportunity to use compute resources where privileges are limited. But if we step back and think about the process of building an image, elevated privileges are sometimes necessary. Specifically, if you want to build an image from a recipe or definition file, you need root access.
Right now, Singularity users need a Linux workstation or Virtual Machine where they have elevated privileges to build and test their container images. In many HPC and EPC environments, however, there are a couple of potential issues with this approach:
Sylabs Remote Build Service addresses these challenges by removing the need for users to have elevated privileges, and potentially allowing users to build Singularity containers on non-Linux workstations.
Singularity 3.0 development is continuing at a fast pace, and new and exciting use cases are already popping up. For instance, using the the Remote Build System, it should be possible to build Singularity images using a Mac or Windows workstation; please stay tuned!
One of the goals of the Remote Build Service is for it to integrate seamlessly with Singularity so that current Singularity users don’t have to modify their workflow. In fact, with the addition of a single flag on the command line, the build can be done remotely:
$ singularity build --remote test.sif test.defThis causes Singularity to make a request to the Remote Build Service, which completes the build remotely. During the build process, output of build is streamed back to the console, so that the user can monitor its progress. Assuming the build completes successfully, the built SIF image file is transferred back to the user’s workstation, from which point it can be executed with Singularity.
Singularity and Sylabs have been focussed on security from day one. The astute Singularity user will note that building images within a managed service still requires elevated privileges. It simply shifts the location where elevated privileges are utilized.
This shift allows the Remote Build Service to implement appropriate levels of isolation between the components performing the builds with elevated privileges, and the rest of the infrastructure. As a system administrator, you get a turn-key solution that empowers users to build Singularity images, providing you with centralized auditing and monitoring of Singularity builds occuring on-site.
Do you have a use case for the Sylabs Remote Build Service? Interested in learning more? We’d love to hear from you for topics to cover in future Lab Notes, or joining community discussions.
Slack Channel or get an invitation at: Sylabs website.
Ensuring Long-term Trust and Reliability of SIF Images: What You Need to Know As container runtimes technologies become more prevalent, it presents a dilemma for government, public, and private sector organizations. They must consider the long-term implications of...
The Apptainer project has recently published CVE-2023-30549. The associated
Enabling Portable and Secure Computing Environments for High-Performance Workloads.As part of their ongoing efforts to streamline workflows, enhance productivity, and save time, engineers, and developers in enterprises and high performance computing (HPC) focused...