This long awaited feature allows copying local files into remote builds via the %files section of the Singularity def file the same way as you would if you were building locally with singularity build.
No special command-line arguments or other user actions are required: simply add a %files section to your DEF file to be used with remote build and any referenced files will be automatically copied to the image.
Remote build files support is included in SingularityCE v3.10 (and later), scs-build 0.7.4 (and later), and will be back-ported to SingularityPRO v3.9 for an upcoming release.
scs-build is the go-to utility for integrating Singularity builds into a CI/CD pipeline, such as GitHub Actions, GitLab, Jenkins, CircleCI, and others. It is the same code used in SingularityCE and SingularityPRO to call out to a remote build service. There are currently two means to support the remote building of a SIF image, either in Sylabs Singularity Enterprise or in Sylabs Singularity Container Services (cloud.sylabs.io).
It is imperative that the Docker command-line arguments are properly formed to mimic the Singularity build from being invoked on the local system. This ensures the current local directory is mirrored in the scs-build container environment.
docker run \
--workdir `pwd` \
-e SYLABS_AUTH_TOKEN=${SYLABS_AUTH_TOKEN} \
-v `pwd`:`pwd` \
-u `id -u` \
sylabsio/scs-build build \
--skip-verify \
`pwd`/<def file in current local directory> \
<library ref>This command-line invocation ensures file privileges are maintained within the build container environment through the use of the -u argument.
Depending on the workflow, the DEF file might be in another directory. If that’s the case, it is necessary to (also) map that directory into the environment (using the docker run --volume argument) and reference it in place.
No special command-line arguments or actions are required to perform a remote build using a DEF file containing a %files section. Invoke singularity build --remote as normal:
singularity build --remote <library ref> <def file spec>If the DEF file specified contains a %files section, the build context is automatically created and managed by the build operation.
Support for remote build files is enabled now on Sylabs Cloud (https://cloud.sylabs.io) and will be available in the next release of Singularity Enterprise, using a compatible Singularity runtime (see Overview)
If you happen to experience any issues with the remote build files support, in either scs-build or singularity CLI please contact us via the community Slack channel (https://singularityce.slack.com) in the #sylabs-cloud channel.
Addressing The Grand Challenges of Our Time Through Singularity Container TechnologyAt Sylabs, our mission and vision aren't just statements on a wall, they're an ethos we embody daily. We're committed to facilitating cutting-edge research that seeks to address...
IntroductionOpen Container Initiative (OCI) registries offer extensibility and customization options. Users can define custom metadata, annotations, and labels for images, which aids in better categorization and organization. Furthermore, registries can be integrated...
Discover the role of Singularity and OCI in making computational workflows FAIR. Understand how container technology streamlines data management and improves research efficiency across platforms.