Now that we have readied the WSL2 environment with Singularity and the relevant CUDA libraries, it’s time to run the sample Keras workflow.
The single-file container image of Singularity is a unique feature that allows movement from one system to another as simple as transferring a file. With a Singularity container image, there is no need to worry about managing layers, directories, or having to export. This extreme mobility of compute gives unparalleled freedom to Singularity users, allowing access to containerized software without additional infrastructure requirements. As Singularity containers continue to become a vital part of HPC and Enterprise Performance Computing (EPC) workflows, the Sylabs team will continue to build upon the features and services to ease management and security. As the saying goes, ‘the price of freedom is eternal vigilance’. Well, Sylabs’ cloud offerings, and on-premise services will help you keep things organized and secure.
Container Library for End Users
Later this year Sylabs will launch our Container Library, a comfortable home for your containers. Available as a cloud service, or for on-prem deployment, the Library will be available to manage, store and share containers. The cloud service portion will offer common Linux distributions, programming languages and AI frameworks, which will be updated regularly. A clear web interface and simple command-line syntax will let you search across containers and `singularity pull` them down to your system. If you are working with air-gapped or embedded systems, no problem! Thanks to the single file image format, you can simply download your container from a supported service, or another machine; an on-premise library does not require internet connected.
With Singularity 3.0 the new Singularity Image Format (SIF) will bring container signing and validation to Singularity and the library. Quickly identify containers signed by trusted sources (like companies or collaborators), and give a thumbs up to the images that work best for you. We know there are a lot of containers out there, and we want to make it easy to find an image that works best for your needs.
Container Security for HPC and EPC Professionals
The rise of a new technology will always disrupt existing processes and workflows, and the growing use of containers is challenging for administrators and InfoSec teams. How can you keep your systems and data secure without restricting users’ productivity? New vulnerabilities are discovered every day, and the software installed into containers is often out of reach of traditional monitoring tools.
Singularity already mitigates many security problems that could be present in software installed into a container. We encourage that containers are never run as root, and block privilege escalation in the container. However, out of date software in a container may still leak data via vulnerabilities.
A proof of concept open-source tool, `clair-singularity`, is already available in the community to scan Singularity containers using CoreOS Clair. This scanner can quickly identify OS packages installed inside the container that have outstanding publicly disclosed vulnerabilities. Running a quick test with `clair-singularity` against a test image containing a popular AI environment finds several vulnerabilities. Many of these are false positives for a container run with Singularity, due to the mitigations mentioned above. However, an issue with a component of the web application stack, which could be exploited, is also found:
$ clair-singularity xxx.simg --bind-ip=192.168.1.11 --bind-port=8888 … CVE-2017-1000158 (Medium) https://people.ubuntu.com/~ubuntu-security/cve/CVE-2017-1000158 CPython (aka Python) up to 2.7.13 is vulnerable to an integer overflow in the PyString_DecodeEscape function in stringobject.c, resulting in heap-based buffer overflow (and possible arbitrary code execution)
With a team of developers coming from backgrounds spanning academic HPC, to national intelligence agencies, we are acutely aware of the challenges in balancing security and usability. In the Container Library, we are is incorporating automated security scanning that can be integrated into existing security workflows. More uniquely, we’ll leverage the functionality of our new SIF image format validate signatures, either manually by a security team or through a trusted CI pipeline.
Additionally, the Singularity client will be able to subscribe to a library, and optionally or automatically check for updated versions of a container. Users can be empowered to update containers themselves, in response to notifications of upstream fixes. Administrators will be able to see if outdated containers remain in use, so they can follow up with users. Alternatively, on a sensitive production system Singularity could be configured to always obtain the latest versions of containers, to ensure the latest security fixes. With mandatory checks enabled, it will also be possible to deny execution of a container that the library has identified as vulnerable, or exceeds a specified rating threshold.
Do you have a use case or want to learn more about the Sylabs Container Library?
We’d love to hear from you for topics to cover in future Lab Notes, or join community discussions here:
Join Our Mailing List
Signing the Container The Singularity 3.0 family introduced the ability to create (and manage) PGP keys to sign and verify containers. This provides a trusted method for Singularity users to share containers and ensures a bit-for-bit reproduction of the original...
Create an Account & Authentication Token Now that we have SingularityCE installed in WSL2, and NVIDIA GPU support is enabled, we will create a Singularity Container Services account and configure the local Singularity client, followed by building a remote...