Containerization continues to revolutionize how high performance computing resources are used to help solve a variety of real-world problems. By packaging a workload’s application and its dependencies in a single file image (container), developers, scientists, and researchers are able more easily leverage shared computing resources to process large volumes of data, at high speed. This is made possible with fast network interconnects between compute and storage nodes, allowing for data reproducibility, portability, and scalability in a vast array of applications and scenarios. 

Singularity containerization provides a powerful means of addressing several critical HPC concerns, including the need for secure, fast and simplified mobility across hybrid computing resources. Broadening and simplifying the access to these systems for those who make societal advancements is why Sylabs is committed to the growing community that has already adopted Singularity.

Singularity is not just a runtime, it also includes a container image format. Sylabs is gratified that the Singularity Image File (SIF) is increasingly recognized and accepted as the defacto standard for HPC containers. The SIF image is also recognized as an important container format by other open source projects as well as companies that provide computing resources and technologies to the growing number of compute-driven ecosystems. This can be seen by recent announcements by Podman, the long-running support by NVIDIA NGC, the hub for GPU-optimized software, and SIF support in an ORAS compatible registry such as Azure Batch Shipyard. These are but a few examples.

The SIF file format, developed at Sylabs and invented by Yannic Côté during his tenure with Sylabs (who is now working at Red Hat), is continuing to be developed and maintained by Sylabs in concert with the SingularityCE community. Together, we aim to ensure it not only solves the problems originally addressed with its creation, but also works to solve new problems in the burgeoning compute-driven landscape as they appear. As displayed in the open SingularityCE roadmap, Sylabs and the community have a well-planned path for expanding and enhancing SIF’s use and utility across the myriad forms of architectures and computing resources, including stronger OCI compatibility. This is important in the merging of workflows of traditional cloud-native and batch-oriented systems. Without a doubt, SIF is the beating heart of Singularity. Sylabs and the open source community that developed, maintain and rely upon SIF are very proud of this technology. Sylabs will continue to provide this critical utility to the community at large through funding, open-source efforts and the BSD license.

What is Singularity?

We say it unequivocally: the open source SingularityCE project is Singularity, which has been developed under the guidance of Sylabs. 

Of course, there is bound to be confusion when an open source project reaches a point where there is disagreement between the founder of the project and the developers committing work to that project. Both sides had different ideas about the path forward and there came a point when those ideas were too divergent to be reconciled. Dave Trudgian is the lead developer on SingularityCE, Adam Hughes on SIF, and any examination of Sylabs repositories will see the bulk of work, especially around SIF, is being done (and pulled from) there. We will continue our efforts around Singularity via SingularityCE with the aim of providing function and stability, doing the unglamorous development work that other projects that absorb this effort are not doing. 

The community of individuals, companies, partners, and agencies rely on this effort for consistency and stability – Sylabs is not going anywhere. We will innovate. We will solve problems. We will continue to support our community and our customers. We will continue to do the development that is necessary in order to drive Singularity innovation. Singularity is the most widely adopted container runtime and image format in use in high performance computing environments. Singularity is at our core. The work of our community, our partners, and yes, the people in our company rely on our efforts. We have no intention of stopping or slowing down.

The SingularityCE community has exciting plans for the future of Singularity, and the entire Singularity ecosystem. These plans are laid bare in the SingularityCE community chat forums  (Google Groups, Slack, and GitHub). The SingularityCE roadmap is openly discussed on the SingularityCE community calls, where no invitation is required to join. The roadmap is well developed and is published on GitHub for anyone to comment on, contribute to, leverage for their own project, or simply view for interest. Sylabs continues to host monthly community meetings to share progress and solicit feedback from the community who uses the Singularity runtime and containers for their work. To that matter, Sylabs as a whole speaks openly about roadmaps for the entire portfolio of products. 

Sylabs is catering to the high-end of computing where the need for stability is critical. We are actively engaged in the development that is required for a stable project with predictable releases that supports various architectures and even OS distributions. And yes, part of this is because of our own commercial efforts: the benefits of the QA work that Sylabs puts into SingularityPRO, Singularity Enterprise and Singularity Container Services are a critical part of the maintenance and feeding of SingularityCE – and feeds and supports the Singularity ecosystem.

This is the entire point that we are hoping to impress in this posting: it is an ecosystem with multiple players. Of those players, the SingularityCE project will continue to provide the core development for Singularity and SIF.  SingularityCE is Singularity.

Advancing Singularity

It’s important to have open, and ongoing communication with the community so that it can understand and participate in the development happening and let us know how that development impacts them. People need to see where the project is going and feel like they have a voice in it. Because of the thought leadership and technological advancements happening within the community, the ecosystem is growing. The latest SingularityCE 3.10 release is ample evidence of this. Ultimately, the community roadmap is about making SingularityCE the most powerful and flexible container utility available. 

Sylabs will continue to support and build a compelling container runtime and image format that directly addresses HPC community needs.