Maintaining Open Source Singularity Since 2017

SCS Login

Improve Security in your CI/CD Workflows

By Staff

Oct 13, 2022 | Blog, How To Guides

In the development world, continuous integration is where members of a team integrate all their work frequently, for example, think of a team all working on the same code base, they are fixing bugs, implementing new features, so to prevent conflicts, all the code is merged together with automation. For this use case, a free Singularity Container Services account would be helpful, one click and you are done, and you get 11 GB for your images and 500 build time minutes. For more information, please see part 1 of the Singularity Container Workflow. Upcoming releases will include a SBOM (Software Bill of Materials), this is an important contribution to security. If you are not familiar with SBOM, let me put it in simple words, it is a list of dependencies: modules, libraries and components required to either run or build a piece of software. Like the ingredients you read on a can of food, if you are allergic to some specific ingredient and you may want to stay away, the same happens with an SBOM, you will probably want to stay away from a flagged or vulnerable component. Some questions may arise, How is this collection built? Does it have a specific format? What kind and type of information contains? There are many SBOM specifications, the most widely used today are SPDX and CycloneDX, CycloneDX being the format selected by default. And now, what can we do with an SBOM?, we can audit the components. The tool of choice is Grype, it is a vulnerability scanner for container images and filesystems, is very easy to use, has an extensive vulnerability database for many major linux operating systems including CentOS, Debian, Oracle Linux, Red Hat (RHEL), Ubuntu, and others, and also supports SIF images out of the box.

Use case

Build a SIF image and ensure security by documenting and analyzing packages and dependencies, for this, the following requirements are:
  • GitHub public repository.
  • A free Singularity Container Services account.
We will create a simple Hello World program in Go. Remember, this is a demonstration on how to build a Singularity image, generate a Software Bill of Materials (SBOM), and check to see if any of the software in our image has known vulnerabilities. The structure of our source code is as follows:
$ tree
├── cmd
│   └── main.go      <-- Command line program, imports hello package.
├── go.mod           <-- Standard module definition **no deps here**
├── hello.go         <-- Source code of the program
├── hello_test.go    <-- Testing source code of the program
├── helloworld.def   <-- SIF image definition file
├── main             <-- Compiled program
└── README.md        <-- This file
Our program is very basic, contents of the cmd/main.go file:
package main
 
import (
   "fmt"
   "hello"
)
 
func main() {
   message := hello.SayHello("World")
   fmt.Println(message)
}
Contents of the hello.go file:
package hello
 
func SayHello(a string) string {
   return "Hello " + a + "!"
}
The contents of the source code for testing are basic as well, so the contents of the hello_test.go file are:
package hello
 
import (
   "strings"
   "testing"
)
 
func TestLength(t *testing.T) {
   msg := SayHello("World")
   length := len(msg)
   if length != 12 {
       t.Errorf("SayHello(\"World\") length is %d; want 12", length)
   }
}
 
func TestContainsUTF(t *testing.T) {
   msg := SayHello("嗨")
   if !strings.Contains(msg, "嗨") {
       t.Error("SayHello(\"嗨\") doesn't support UTF8")
   }
}
Then, we will create a definition file for this image, this is the content of the helloworld.def file:
Bootstrap: library
From: josue-sylabs/devtools/golang:1.19.2
Stage: build
 
%files
    . /src
%post
    cd /src
    go test ./...
    go build -v -o /usr/bin/main ./cmd/main.go
 
Bootstrap: library
From: alpine
Stage: final
 
%files from build
    /usr/bin/main
%runscript
    /usr/bin/main

If you are new, let’s explain:

  • Bootstrap is telling the image builder where to get the base image, options are, library, dockerhub and others. At this time we are going to download the alpine image from the Singularity library.
  • From is the name of our image. In this example, the latest alpine image is used. Whether this is not good for production, the scope of this post is a demonstration.
  • %files. This section is going to copy our helloworld program which is named: main.
  • %post. This is the section where you perform the majority of the operations like download files, install software, compile source code, etc.
  • %runscript. This tells the singularity engine to run our program when called via the run subcommand.

At a very low level the SIF is made up of a global header, metadata, and the actual data. Our interest is in the metadata part, which in the SIF standard are called descriptors.

The descriptors are an important part of the image because it tells how the image was created using a definition file, it stores data partitions, labels, signing data, cryptographic data, and our data type of interest: SBOM data.

Fortunately, Grype supports SIF images out of the box, so you don’t have to worry about anything else, scan it by prepend “singularity:” to your image name, like so:

$ grype singularity:ubuntu-sbom.sif
 ✔ Vulnerability DB        [no update available]
 ✔ Parsed image            
 ✔ Cataloged packages      [102 packages]
 ✔ Scanned image           [27 vulnerabilities]
NAME      INSTALLED         FIXED-IN      TYPE  VULNERABILITY   SEVERITY   
coreutils 8.32-4.1ubuntu1                 deb   CVE-2016-2781   Low         
e2fsprogs 1.46.5-2ubuntu1   1.46.5-2ubunt deb   CVE-2022-1304   Medium 
...
Here comes the CI/CD part. Let’s create a GitHub action, I will explain it section by section. The first section, whose name is “Build Image”, is triggered on a push to the repository.
name: Build Image

on:
  push:
    branches: [ "master" ]

Next section is variable definition, in this part, the definition file is set, as well as the name of the SIF file to be created. As an option, we can also keep a copy of the image file on the build server.

env:
  DEF_FILE: "helloworld.def"
  OUTPUT_SIF: "helloworld.sif"
  LIBRARY_PATH: "library://josue-sylabs/temp/helloworld:latest"
  SCS_BUILDER_VERSION: “0.7.5"
  SYLABS_AUTH_TOKEN: ${{ secrets.SYLABS_AUTH_TOKEN }}
  GOLANG_IMAGE: "library://josue-sylabs/devtools/golang:1.19.2"

These are the steps the runner executes when a push occurs.

  1. Setup the standard “checkout” step
  2. Install the SCS builder from GitHub, which at the creation of this document is 0.7.5.
  3. Compile the program, test and build the actual SIF file remotely and create an image, this is important to understand, this is where you need a Singularity Container Services account already mentioned before. So, create an access token and call it SYLABS_AUTH_TOKEN, the value is given by the Singularity Container Services when you create the account there.
  4. Now it is time to scan the source code for known package vulnerabilities, Grype has a wide support of programming languages including Go, Rust, Javascript, Python, and others.
  5. Perform the actual scan of the SIF image.
  6. Save the generated SIF file in GitHub artifact list.
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
#1
    - uses: actions/checkout@v3
#2
    - name: Install scs-build
      run: |
        mkdir {tmp,tmp-sif-path}
        pushd tmp
        wget https://github.com/sylabs/scs-build-client/releases/download/v${SCS_BUILDER_VERSION}/scs-build-client_${SCS_BUILDER_VERSION}_linux_amd64.tar.gz
        tar -zxf scs-build-client_${SCS_BUILDER_VERSION}_linux_amd64.tar.gz
        sudo cp scs-build /usr/bin/
        popd   


#3
    - name: Compile program and build image
      run: scs-build build $DEF_FILE ${{ github.workspace }}/tmp-sif-path/$OUTPUT_SIF

#4
    - name: Scan source code
      uses: anchore/scan-action@v3
      with:
        path: "${{ github.workspace }}"

#5
    - name: Scan final image
      uses: anchore/scan-action@v3
      with:
        image: "singularity:${{ github.workspace }}/tmp-sif-path/{{ env.OUTPUT_SIF }}"
#6
    - name: Save artifacts
      uses: actions/upload-artifact@v2
      with:
        name: ${{ env.OUTPUT_SIF }}
        path: ${{ github.workspace }}/tmp-sif-path
By default, if any vulnerability is medium or higher the build is going to fail. To modify the behavior, set the severity-cutoff to low, high or critical:
   - name: Scan SBOM
      uses: anchore/scan-action@v3
      with:
        image: "singularity:${{ github.workspace }}/tmp-sif-path/{{ env.OUTPUT_SIF }}"
        severity-cutoff: critical

You could inspect the output of the report, for example in SARIF, like so:

   - name: Inspect the report
      run: cat ${{ steps.scan.outputs.sarif }}

Summary

Scanning for vulnerabilities with the help of Singularity’s SBOM and Grype in a CI/CD workflow is a good start. Using many open source tools provides value, saves time and effort, but every day a new bug comes and it is not always easy to see how critical they are. Source code: https://github.com/josueneo/helloworld.

Join Our Mailing List

Categories

Related Posts

An Introduction to Singularity Containers

Mar 30, 2023 |

Enabling Portable and Secure Computing Environments for High-Performance Workloads.As part of their ongoing efforts to streamline workflows, enhance productivity, and save time, engineers, and developers in enterprises and high performance computing (HPC) focused...

read more