IEEE Utility and Cloud Computing 2017 Conference: Is Singularity-based Container Technology Ready for Running MPI Applications on HPC Clouds?

By Staff

Feb 7, 2018 | Blog, News

From the Proceedings of the 10th International Conference on Utility and Cloud Computing:


The Message Passing Interface (MPI) standard has become the de facto programming model for parallel computing with the last 25-year continuous community effort. With the development of building efficient HPC clouds, more and more MPI-based HPC applications start running on cloud-based environments.

Singularity is one of the most attractive container technologies to build HPC clouds due to the claimed reproducible environments across the HPC centers. However, our investigations in the literature show that there is a lack of a systematical study on evaluating the performance of Singularity with various benchmarks and applications on different types of HPC platforms. Without these studies, it remains difficult to tell the community whether Singularity-based container technology is ready or not for running MPI applications on HPC clouds to gain desired performance.

To fill this gap in the literature, as a third-party, we first propose a four-dimension evaluation methodology to cover various aspects and based on that, we conduct extensive studies on evaluating the performance of Singularity on modern processors, and high-performance interconnects. Performance results prove that Singularity-based container technology can achieve near-native performance for both Intel Xeon and Intel Xeon Knights Landing (KNL) platforms with different memory access modes (i.e., cache, flat).

Singularity also shows very little overhead for running MPI-based HPC applications on both Omni-Path and InfiniBand networks. With the verification of our results, we believe that Singularity can be used for building next-generation HPC clouds with near-native performance as well as desired cloud features such as easy management and deployment.

Related Posts

Sylabs Appoints Adam Hughes as Chief Technology Officer

RENO, N.V., March 23, 2022 — Sylabs, the global leader in providing tools and services for performance-intensive container runtime technology, today announced that it has appointed Adam Hughes to the position of Chief Technology Officer. Hughes will lead Sylab’s...

read more

SingularityCE 4.0 and beyond

Bridging the gap between HPC and OCI After SingularityCE was forked, Sylabs committed to placing a heavy emphasis on open and transparent discussion of the future direction of the project. Our SingularityCE roadmap was created as an open living document that we...

read more

Pin It on Pinterest