Sylabs Remote Build Service

Sylabs Remote Build Service

At Sylabs, our mission is to drive development of the open source Singularity project and expand the surrounding ecosystem positioning Singularity as the de-facto choice for HPC and Enterprise Performance Computing (EPC) workflows. .

Today, we’re happy to share with you a preview of one of the products we have been working on: the Sylabs Remote Build Service, a cloud-based service (It’s not the most creative name, I know, but as a team of software developers, you’ll have to forgive us for our lack of marketing prowess!)

Why Build Remotely?

Singularity enables many different workflows by giving you the opportunity to use compute resources where privileges are limited. But if we step back and think about the process of building an image, elevated privileges are sometimes necessary. Specifically, if you want to build an image from a recipe or definition file, you need root access.

Right now, Singularity users need a Linux workstation or Virtual Machine where they have elevated privileges to build and test their container images. In many HPC and EPC environments, however, there are a couple of potential issues with this approach:

• Users may not have access to Linux workstations. It’s possible to work around this limitation using virtual machines or cloud computing, but some sites prohibit this for technical and/or policy reasons.
• Some facilities have policies that do not allow elevated privileges to be granted to non-admin users, which limits the types of Singularity images users can build.

Sylabs Remote Build Service addresses these challenges by removing the need for users to have elevated privileges, and potentially allowing users to build Singularity containers on non-Linux workstations.

Singularity 3.0 development is continuing at a fast pace, and new and exciting use cases are already popping up. For instance, using the the Remote Build System, it should be possible to build Singularity images using a Mac or Windows workstation; please stay tuned!

Usage

One of the goals of the Remote Build Service is for it to integrate seamlessly with Singularity so that current Singularity users don’t have to modify their workflow. In fact, with the addition of a single flag on the command line, the build can be done remotely:

This causes Singularity to make a request to the Remote Build Service, which completes the build remotely. During the build process, output of build is streamed back to the console, so that the user can monitor its progress. Assuming the build completes successfully, the built SIF image file is transferred back to the user’s workstation, from which point it can be executed with Singularity.

Security as a Discipline

Singularity and Sylabs have been focussed on security from day one. The astute Singularity user will note that building images within a managed service still requires elevated privileges. It simply shifts the location where elevated privileges are utilized.

This shift allows the Remote Build Service to implement appropriate levels of isolation between the components performing the builds with elevated privileges, and the rest of the infrastructure. As a system administrator, you get a turn-key solution that empowers users to build Singularity images, providing you with centralized auditing and monitoring of Singularity builds occuring on-site.

Early Access

Do you have a use case for the Sylabs Remote Build Service? Interested in learning more? We’d love to hear from you for topics to cover in future Lab Notes, or joining community discussions.

Google Groups

Slack Channel or get an invitation at: Sylabs website.

GitHub

YouTube