Maintaining Open Source Singularity Since 2017

SCS Login

Securing Container Workflows with the Power of OCI Registries

By Adam Hughes

Jul 6, 2023 | Blog

Overview

In the rapidly evolving world of containerization, where efficient software deployment and scalability are paramount, having a robust infrastructure to manage and distribute container images is crucial. OCI (Open Container Initiative) registries, the backbone of modern container ecosystems. These powerful repositories bring a plethora of advantages that revolutionize the way we handle container images.
In this blog, we’ll dive deep into the advantages of having a compatible OCI registry at Singularity Container Service as well as Singularity Enterprise and explore how it seamlessly integrates with existing tools and simplifies the management, security, distribution, and collaboration of container images. From compatibility and security to caching and performance optimization, OCI registries offer a comprehensive solution that streamlines the entire lifecycle of containerized applications.
Please read the blog Hybrid Docker/Singularity workflow, to get an insight on how to push OCI images to SCS and have the following examples working for you.

Unlocking the Advantages for Container Image Management

Having an OCI registry offers several advantages for managing and distributing container images. Here are some key benefits:

Image Versioning and Lifecycle Management

OCI registries offer features for managing the entire lifecycle of container images. This includes image promotion between different environments (such as development, staging, and production), garbage collection to remove unused images, and support for vulnerability scanning to identify and mitigate security risks.
Versioning of container images allows users to maintain different versions of the same image. This is particularly useful for managing updates, rollbacks, and compatibility across different stages of application development and deployment. Tagging is a method used to organize versioning, the following example demonstrates the use of the ls command, which lists all tags corresponding to a certain image:
$ crane ls registry.sylabs.io/USERNAME/alpine/alpine
3.17
3.17.0
3.17.0_rc1
3.17.1
3.17.2
3.17.3
3.18
3.18.0

Compatibility

OCI registry follows the open standards defined by the OCI, ensuring compatibility with various container runtimes, orchestrators, and tools. This interoperability allows users to seamlessly work with container images across different platforms and environments.
When working with private registries, a password is necessary to perform operations, to do this, the first thing to do is to login to the registry and then obtain a temporary password. The majority of the tools out there accept very similar arguments:
singularity remote get-login-password | \
            crane auth login -u USERNAME \
              --password-stdin registry.sylabs.io

singularity remote get-login-password | \
            skopeo login -u USERNAME --password-stdin registry.sylabs.io

singularity remote get-login-password | \
            cosign login -u USERNAME --password-stdin registry.sylabs.io

singularity remote get-login-password | \
            syft login -u USERNAME --password-stdin registry.sylabs.io

Digital Signatures

OCI registries provide mechanisms to enforce secure image distribution. They support content signing and verification, enabling users to ensure the integrity and authenticity of container images. For example, have the cosign CLI to sign directly on remote registries. Signing container images is an important security practice that provides integrity, authenticity, and non-repudiation of the image content. Here are a few reasons why signing container images is beneficial:
  • Integrity Verification: Image signing ensures the integrity of container images. By signing an image, a cryptographic hash is generated and embedded in the image metadata. This hash acts as a unique identifier for the image content. When the image is pulled from a registry, the signature can be verified against the image content, ensuring that the image has not been tampered with or modified during transit.
  • Authentication and Trust: Image signing provides a mechanism for authentication and trust. Signatures are created using private keys held by trusted entities, such as developers or organizations.
  • Supply Chain Security: Image signing enhances the security of the container image supply chain in the form of a Software Bill of Material which is described later in this blog post. Each step of the image’s lifecycle, from creation to deployment, can be signed by the responsible parties. This enables end-to-end verification, ensuring that images are created and modified by authorized entities.
  • Compliance and Auditing: Signed images facilitate compliance and auditing requirements.
The following is a very simplified demonstration, first generate a key-pair, then obtain the digest of the image which is about to be signed, then issue the sign command it like so:
$ cosign generate-key-pair
Enter password for private key: 
Enter password for private key again: 
Private key written to cosign.key
Public key written to cosign.pub
$ DIGEST=$(crane digest registry.sylabs.io/USERNAME/alpine/alpine:latest)
$ cosign sign -y --key cosign.key \
         registry.sylabs.io/USERNAME/alpine/alpine@$DIGEST
$ cosign verify --key cosign.pub \
         registry.sylabs.io/USERNAME/alpine/alpine@$DIGEST

Software Bill of Materials

SBOM stands for Software Bill of Materials. It is a structured inventory that provides a comprehensive list of all the components and dependencies used in building a software product. The primary purpose is to enhance transparency and traceability in software development and deployment processes. By having a detailed list of all the components and dependencies, organizations can better understand the composition of their software, identify potential security vulnerabilities or licensing issues, and effectively manage their software supply chain.
The following demonstrates how to generate an SBOM very easily directly on the registry, first lets copy an image from another source into our repository, and then, with the help of Syft, the SBOM is generated.
$ crane cp docker.io/ubuntu:23.04 \
           registry.sylabs.io/USERNAME/ubuntu/ubuntu:23.04
$ syft packages registry:registry.sylabs.io/USERNAME/ubuntu/ubuntu:23.04 \
       -o cyclonedx-json=sbom.ubuntu_23_04.cyclone.json
 ✔ Parsed image            
 ✔ Cataloged packages      [90 packages]

*Syft supports many SBOM formats; the chosen one was taken arbitrarily and with no preference.

After creating the SBOM it is normally kept for records for different purposes like:
  • Risk management.
  • Software maintenance and support.
  • Intellectual property management.
  • Supply chain transparency.
  • Compliance and auditing.
In terms of compliance and auditing, there are vulnerability scanning tools designed specifically for SBOM analysis and is known for its speed and efficiency in scanning SBOMs, Grype is suitable for integration into continuous integration/continuous deployment (CI/CD) pipelines. It supports a wide range of SBOM formats, such as SPDX, CycloneDX, and JFrog Xray, making it versatile and adaptable to different software development environments.
It also has the ability to scan a remote image stored in a OCI Compliant registry in search of known vulnerabilities:
$ grype registry:registry.sylabs.io/USERNAME/ubuntu/ubuntu:23.04                                            
 ✔ Vulnerability DB        [updated]
 ✔ Parsed image            
 ✔ Cataloged packages      [90 packages]
 ✔ Scanning image...       [0 vulnerabilities]
   ├── 0 critical, 0 high, 0 medium, 0 low, 0 negligible
   └── 0 fixed
No vulnerabilities found
For demonstration, let’s scan a well known insecure and old image:
$ crane cp docker.io/debian:10 registry.sylabs.io/USERNAME/debian/debian:10
$ grype registry:registry.sylabs.io/USERNAME/debian/debian:10
✔ Vulnerability DB        [no update available]
 ✔ Parsed image            
 ✔ Cataloged packages      [91 packages]
 ✔ Scanning image...       [119 vulnerabilities]
   ├── 0 critical, 24 high, 9 medium, 20 low, 62 negligible (4 unknown)
   └── 0 fixed

NAME           INSTALLED    FIXED-IN     TYPE  VULNERABILITY     SEVERITY   
apt            1.8.2.3                   deb   CVE-2011-3374     Negligible  
bash           5.0-4                     deb   CVE-2019-18276    Negligible  
bsdutils       1:2.33.1-0.1              deb   CVE-2022-0563     Negligible  
bsdutils       1:2.33.1-0.1 (won’t fix)  deb   CVE-2021-37600    Low         
coreutils      8.30-3                    deb   CVE-2017-18018    Negligible
...
Anyone can alter, modify or forge an SBOM and claim it belongs to a certain Image. Attestation is the process of verifying and providing evidence or proof of the integrity, authenticity, or configuration of the container image. An attestation typically involves a trusted entity, often referred to as an attestation authority or a trusted third party, that verifies and vouches for the properties of the system or component being attested. The attestation authority examines the container image, performs various checks, and generates a signed statement or report, commonly known as an attestation report or attestation token, that provides evidence of the image properties.
The following example demonstrates how to sign an attestation of our previous Ubuntu 23.04 SBOM:
$ cosign attest --key cosign.key \
         --predicate sbom.ubuntu_23_04.cyclone.json \
         registry.sylabs.io/USERNAME/ubuntu/ubuntu:23.04
And finally you can confirm the attestation is attached to the repository with the help of the crane command:
command:
$ crane ls registry.sylabs.io/USERNAME/ubuntu/ubuntu
23.04
sha256-9279f41cc6e4df8f87b13ac17c2c6f2a280fd3ca2638d18f8dc94b774486909f.att
Take a look at the last tag, it is an .att tag indicating an attestation, in a real scenario workflow we should run a verification by issuing the following cosign command:
cosign verify-attestation --key cosign.pub \
       registry.sylabs.io/USERNAME/ubuntu/ubuntu:23.04 | \
       jq '.payload | @base64d | fromjson | .predicate.Data | fromjson'
You should see this output along with the attached SBOM
Verification for registry.sylabs.io/USERNAME/ubuntu/ubuntu:23.04 --
The following checks were performed on each of these signatures:
  - The cosign claims were validated
  - Existence of the claims in the transparency log was verified offline
  - The signatures were verified against the specified public key
{
  "$schema": "http://cyclonedx.org/schema/bom-1.4.schema.json",
  "bomFormat": "CycloneDX",
  "specVersion": "1.4",
  "serialNumber": "urn:uuid:cd363e67-3790-4ebb-aa9d-8f5175132376",
  "version
...

Summary

Singularity Container Services and Singularity Enterprise offers an OCI compatible registry that brings a variety of toolchains which are common to the cloud native industry, for example, one can build an automated workflow in which SBOMs are kept as records and submitted to a management system supporting compliance, risk management, software maintenance, IP management, and supply chain transparency. They serve as a valuable source of information for various stakeholders, including developers, security teams, auditors, and regulatory bodies.

Join Our Mailing List

Recent Posts

Related Posts