Maintaining Open Source Singularity Since 2017

SCS Login

Hybrid Docker/Singularity Workflow

By Adam Hughes

Jun 21, 2023 | Blog, How To Guides

Overview

The hybrid workflow of Singularity with Docker containers combines the strengths of both technologies to provide a flexible and efficient containerization solution. With Singularity’s ability to natively run Docker containers, users can leverage the vast Docker ecosystem and easily incorporate Docker images into their Singularity workflows. This hybrid approach allows users to benefit from Singularity’s security, reproducibility, and compatibility with HPC systems while taking advantage of Docker’s extensive collection of pre-existing containers and tools. It provides a seamless bridge between the two container technologies, enabling users to run Docker containers within Singularity with minimal effort and enabling efficient collaboration across diverse computing environments. As work progresses towards SingularityCE 4.0 and beyond, let’s take a look at what is possible today with Singularity Enterprise and SingularityCE/PRO.
Docker and a brief OCI history: While Docker initially had its own container format and runtime, it actively collaborated with the Open Container Initiative (OCI) to align its technology with the industry standards. As a result, Docker adopted the OCI specifications as the foundation for its container format and runtime. Docker now utilizes the OCI image and runtime specifications, ensuring compatibility and interoperability with other OCI-compliant container runtimes and tools.
Singularity and the SIF format: With Singularity, the container image is a single file with a .sif extension, which contains a root filesystem in SquashFS format. This file can be easily distributed and executed on different systems running Singularity, making it portable across various environments. One advantage of using the SquashFS format is that Singularity containers can be mounted as read-only, providing enhanced security and reproducibility. Additionally, Singularity containers can directly access the host system’s file system, allowing users to work with data and files seamlessly without the need for complex data volume setups.
Hybrid Container Workflows: OCI powers cloud native technologies such as Docker and Kubernetes, which have changed the way the world develops and deploys software. Containers, such as those provided by Singularity, allow users to package their software into a single, reproducible unit, making it easy to deploy and run the application on HPC environments. Singularity is capable of using OCI containers as a source to either run the container or convert it to a SIF image. And while there has always been a high level of compatibility, it has never been a story of perfect compatibility. We’ve been talking to users in the open source community, as well as our customers, and partners to understand current and future needs. These conversations spark our imaginations to think a lot about what OCI workflows look like in HPC environments today, how they will evolve tomorrow. More importantly, we think about security (we’ll get to more of that in other blog posts) and what it will take to simplify interactions with the merging orchestrated and batch computing workflows.
In this article, we’ll look at a sample “hybrid” OCI/Singularity workflow that leverages Docker to build, test and push an image to Singularity Enterprise using the OCI standards, and then pull and run this image with both Docker and SingularityCE/PRO. Please note:
  • The following code examples require SingularityCE version 3.11.4 or later. If you are a customer using SingularityPRO, please update to either SingularityPRO 3.9-11, or SingularityPRO 3.11-3 or later.
  • The examples also works with Singularity Enterprise version 2.3 or later, as well as the Sylabs-hosted version called Singularity Container Services at https://cloud.sylabs.io.
  • Singularity Enterprise has three functions: Remote Builder, Library and KeyStore.
  • The terms Singularity Container Service and SCS are synonyms and used indistinctly.

Closing the gap

Singularity Enterprise and Singularity Container Services now include functionality to store OCI images and artifacts, in addition to SIF images. This means that we can build an image using a Dockerfile, upload it to the Library, and run it using SingularityCE/PRO.
Let’s take the following Dockerfile as an example:
FROM golang:1.20-alpine
RUN apk add git
RUN git clone https://github.com/golang/example
RUN cd example/hello && go build && mv hello /
CMD ["/hello"]
Then, let’s build it:
$ docker build -t example .             
[+] Building 3.7s (8/8) FINISHED
 => [internal] load .dockerignore                                      0.1s
 => => transferring context: 2B                                        0.0s
 => [internal] load build definition from Dockerfile                   0.0s
 => => transferring dockerfile: 252B                                   0.0s
 => [internal] load metadata for docker.io/library/golang:1.20.2       0.0s
 => [1/4] FROM docker.io/library/golang:1.20.2-alpine                  0.0s
 => CACHED [2/4] RUN apk add git                                       0.0s
 => CACHED [3/4] RUN git clone https://github.com/golang/example       0.0s
 => [4/4] RUN cd example/hello && go build && mv hello /               3.5s
 => exporting to image                                                 0.2s
 => => exporting layers                                                0.1s
 => => writing image sha256:f0c5b17de57822...5dbff                     0.0s
 => => naming to docker.io/library/example
When running this image, the output is:
$ docker run example                    
Hello, Go examples!
SingularityCE and PRO normally push images to the Library at either your private Singularity Enterprise endpoint or the SCS endpoint at cloud.sylabs.io. OCI native tools like Docker cannot communicate using these endpoints so we have created a new OCI compatible Library at registry.sylabs.io. Before pushing this image to the Library, we must configure Docker to push the image with tags. If your Singularity installation (SingularityCE or SingularityPRO) doesn’t have the OCI registry remote environment configuration, use the following example to add the capability.
$ singularity remote add local cloud.sylabs.io
INFO:    Remote "local" added.
Generate an access token at https://registry.sylabs.io/auth/tokens, and paste it here.
Token entered will be hidden for security.
Access Token:
$ singularity remote use local
Then, obtain a password to authenticate to the built-in registry and pipe the output to docker login command:
$ singularity remote get-login-password | \
              docker login -u USERNAME --password-stdin registry.sylabs.io
WARNING! Your password will be stored unencrypted in /home/almalinux/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
Once you are logged in, we must re-tag the image so Docker knows where to store the image, then, push it.
$ docker tag example registry.sylabs.io/USERNAME/example:v1.0
$ docker push registry.sylabs.io/USERNAME/example:v1.0
The push refers to repository [registry.sylabs.io/USERNAME/example]
dfc1c98e1bbf: Pushed 
775e259e565c: Pushing [==============================>                    ]  148.1MB/246.5MB
4984fbd72df1: Pushed 
bb01bd7e32b5: Pushed
This image will now be visible and classified as an OCI / Docker image, this new asset type is assigned an icon and a type field specified as DOCKER.
Continuing with the workflow, let’s run this image from Singularity as an OCI image, we will follow the same steps, obtain a password to authenticate to the registry and then run the image with the docker:// schema.
$ singularity remote get-login-password | \
  singularity remote login --username USERNAME \
              --password-stdin docker://registry.sylabs.io
INFO:    Token stored in /home/almalinux/.singularity/remote.yaml
$ singularity run \
  docker://registry.sylabs.io/USERNAME/example/example:v1.0
INFO:    Converting OCI blobs to SIF format
INFO:    Starting build...
Getting image source signatures
Copying blob 100b0fdef129 done  
Copying blob d694b5ae8c79 skipped: already exists  
Copying blob f56be85fc22e skipped: already exists  
Copying blob 9f32a84ed3da skipped: already exists  
Copying blob a9354d4031ea done  
Copying blob 85791d961cd3 skipped: already exists  
Copying blob c74873be681f done  
Copying config 61ce683a0a done  
Writing manifest to image destination
Storing signatures
2023/05/13 19:41:59  info unpack layer: sha256:f56be822e...d29d7f64b87a09
2023/05/13 19:41:59  info unpack layer: sha256:85791d961...cd35e9127a7907
2023/05/13 19:41:59  info unpack layer: sha256:d694b5ae8...c79da87345cbc6
2023/05/13 19:42:01  info unpack layer: sha256:9f32a84ed...3adb91a7ddf893
2023/05/13 19:42:01  info unpack layer: sha256:a938340aa...fa14813eeed798
2023/05/13 19:42:01  info unpack layer: sha256:100b0fdef...129a7d1fa7a44e
2023/05/13 19:42:01  info unpack layer: sha256:c74873be6...81fecc83acddcc7
INFO:    Creating SIF file...
Hello, Go examples!
Reuse of OCI image as a base for subsequent use is also possible, in your SingularityCE definition files, let’s add the following code to a file named example.def:
Bootstrap: docker
From: registry.sylabs.io/USERNAME/example/example:v1.0

%post
  # Any other instructions goes here
And finally issue the SingularityCE build command:
$ singularity build --fakeroot example.sif example.def
INFO:    Starting build...
INFO:    Setting maximum build duration to 20m0s
INFO:    Remote "registry.sylabs.io" added.
INFO:    Access Token Verified!
INFO:    Token stored in /root/.singularity/remote.yaml
INFO:    Remote "registry.sylabs.io" now in use.
INFO:    Starting build...
Getting image source signatures
Copying blob sha256:361237ddf3...18f0b3435
Copying config sha256:d710383bdc8e0a...394af54773632
...

Conclusion

Singularity Enterprise 2.3+ and Singularity Container Service allows more flexibility than ever to work with the increasing number of tools in the OCI ecosystem. Whether it’s building and testing container images with Docker, Kubernetes, SingularityPRO, or any other tools leveraging the OCI standards, Singularity Enterprise and Singularity Container Service provides a single place to store and manage your container images and OCI artifacts.
In this blog post, we’ve demonstrated a workflow that starts off by building an image in Docker, and can be run by SingularityCE/PRO in an efficient and seamless way in an HPC environment in its original OCI format. Stay tuned for more blog posts to come that look at other ways to use the new OCI compatibility built into Singularity Enterprise 2.3 and Singularity Container Service. If you have any questions or comments about hybrid OCI/Singularity workflows, feel free to contact us. We’d love to hear from you!

Join Our Mailing List

Recent Posts

Related Posts