Maintaining Open Source Singularity Since 2017

SCS Login

Towards Native Integration Between Singularity Containers And Kubernetes: Announcing An Open Source Project

By Staff

Nov 26, 2018 | Blog

Our work in progress towards native integration of Singularity with Kubernetes is now available on GitHub.

We are releasing this code while it remains under active development for your review, feedback, and hacking pleasure. You should not expect a working implementation at this time, but a clear indication of the direction this project is heading. Obviously, this POC is not intended for general consumption, but targets developers having interest in contributing to Singularity, Kubernetes, or similar technologies.

We anticipated this POC implementation just two weeks ago at SC18 in Dallas. To recap briefly statements made then regarding the roadmap:

… tightly coupled integration and interoperability with Kubernetes has emerged as a pressing requirement for Singularity. […] Thus immediately upon our agenda is the need to implement the Kubernetes [Container Runtime Interface] CRI ‘shim’ (see schematic) to allow Singularity containers to be orchestrated by Kubernetes.

[Image credit: Yu-Ju Hong, Software Engineer, Google
Introducing Container Runtime Interface (CRI) in Kubernetes
https://kubernetes.io/blog/2016/12/container-runtime-interface-cri-in-kubernetes/]

We’ve reproduced the above schematic for ease of reference, as it makes clear that the Singularity CRI implementation is a gRPC server which serves Kubelet requests by interoperating with the Singularity runtime. This server will interact with the Kubernetes CRI to natively spawn Singularity containers, thus adding Singularity Image Format (SIF) support to Kubernetes.

This POC implementation is the above pictured “CRI shim.” Specifically, the Singularity CRI incorporates two services:

  • ImageService – an interface responsible for pulling, removing, and maintaining the list of local Singularity images
  • RuntimeService – an interface responsible for creating, terminating and otherwise managing Singularity containers within Kubernetes pods

Our announcement at SC18 of the intention to integrate Singularity with Kubernetes served two purposes:

  • To engage in compelling discussions at the event – discussions especially that framed existing and emerging use cases in Enterprise Performance Computing (EPC)
  • To alert stakeholders in the broader container ecosystem that they could soon expect substantial progress towards a native integration between Singularity containers and Kubernetes

There has been pressing demand for Singularity containers to integrate with Kubernetes. We are delighted that stakeholders across the broader container ecosystem are expectant, have use cases in hand, and are poised to embrace the most appropriate fit for orchestrating existing and emerging use cases in EPC – i.e., via the tightly integrated combination of Singularity containers and Kubernetes. With this first POC implementation, initial steps in the journey towards integration have been taken. We anticipate many invaluable contributions from the Singularity Community, together with those from the open source communities that surround Kubernetes. Our collective efforts will accelerate progress and ensure that this integration exceeds the requirements and expectations of all involved. We urge you to get involved by joining our Community!

Join Our Mailing List

Categories

Related Posts