Maintaining Open Source Singularity Since 2017

SCS Login

New Containers – Apache, MariaDB, Blender

By Staff

Sep 25, 2018 | Blog, How To Guides

For 3.0, we’re developing containers for common use cases and expanding the size of our cloud library.  These containers and their recipe files will be uploaded GitHub repo with full instructions, and will be accessible via our cloud library at the time of our 3.0 GA release.  Over just the past week we’ve created containers for Apache, MariaDB, and Blender. We’ll be continuing to showcase what you can do with Singularity of the coming weeks with more recipes, guides, and containers.

Apache

This Apache container will run a basic http webserver.  Every file can be edited on the host system in “web/”.

Recipe:

Bootstrap: docker
From: httpd:2.4.34

%post
# Changes the port we are listening on to 8080 instead of 80
sed -ie 's/^\(Listen\).*/\1 8080/' /usr/local/apache2/conf/httpd.conf

%startscript
httpd

Setup:

Let’s start by creating a directory to house our container, its build file, and our web content and logs:

$ mkdir httpd && cd httpd

Create a recipe file (httpd.def) from the recipe above and then build the container:

$ sudo singularity build httpd.sif httpd.def

NOTE ON PORTS:

The configuration runs Apache on port 8080 by default. This can be changed before the container is built in the %post section, by modifying the sed line within the Singularity Definition file.

Our web content and logs are stored from a location on the host.  So we’ll create this directory tree on our host system:

$ mkdir -p web/{htdocs,logs}

Your tree should look like this:

web/
|-- htdocs/
|   `-- index.html
`-- logs/

Add an index file at web/htdocs/index.html:

$ echo '<h1>Apache server is running on Singularity!</h1>' >> web/htdocs/index.html

To serve files from our host system, we start an instance and bind our host path files to the container using “-B”:

$ singularity instance start \
    -B web/htdocs:/usr/local/apache2/htdocs \
    -B web/logs:/usr/local/apache2/logs \
    httpd.sif httpd

Test that your server is up and running by navigating to https://ec2-44-196-59-11.compute-1.amazonaws.com/index.html on the host system.

We can further secure the container by adding “:ro” to our “htdocs” bind line:

-B web/htdocs:/usr/local/apache2/htdocs:ro

This will bind mount the Web root directory in read-only mode. This will prevent an unauthorized user from modifying the web-root itself from within the container.

MariaDB

Running a Database server requires a bit more setup prior to the server being ready. The concept is similar to the HTTP server though. Please see the GitHub location for the source files.

Recipe:

Replace [User] in the recipe file with the host username that will be executing the container.

Bootstrap: docker
From: mariadb:10.3.9

%post
    # [User] is the user who will be executing the container.
    sed -ie 's/^#user.*/user = [User]/' /etc/mysql/my.cnf


%runscript
    exec "mysqld" "$@"

%startscript
    exec "mysqld_safe"

Setup:

Like with Apache, start by creating a directory to house our container, its build file, and our database directory.

$ mkdir mariadb && cd mariadb

Build the image:

$ sudo singularity build mariadb.sif mariadb.def

To setup our host database structure that we will later bind to the container:

$ mkdir -p mariadb/{db,run,log}

Directory structure should look like this:

mariadb/
|-- db/
|-- run/
`-- log/

Now, we will shell into the container.

$ singularity shell \
  -B mariadb/db:/var/lib/mysql \
  -B mariadb/log:/var/log/mysql \
  -B mariadb/run:/var/run/mysqld \
  mariadb.sif

Once in the container shell, we setup mariadb:

> mysql_install_db
> mysqld_safe --datadir=/var/lib/mysql &

SECURING:

At this point, we have the bare installation done, but it’s not yet secure. So, the next step is to run:

> mysql_secure_installation

During the execution, you should:

  • Set a root password (it’s blank by default)
  • Remove anonymous users
  • Disallow root login remotely
  • Remove the test database and access
  • Reload/flush the privilege table

After, connect as the root user to the database:

> mysql -u root -p
Enter password:
 [ … ]
MariaDB [(none)]>

NEW USER:

Now that we’re connected as root, we can create a new database and user:

MariaDB [(none)]> use mysql;
MariaDB [mysql]> CREATE DATABASE workdb;
Query OK, 1 row affected (0.000 sec)
MariaDB [mysql]> CREATE USER newuser@[Host IP] IDENTIFIED BY 'password';
MariaDB [mysql]> GRANT ALL PRIVILEGES ON workdb.* TO newuser@[Host IP] WITH GRANT OPTION;
MariaDB [mysql]> FLUSH PRIVILEGES;
MariaDB [mysql]> exit

Now we’ll shut down the mariadb service inside the container:

> mysqladmin -u root -p shutdown

And exit the container:

> exit

TESTING:

We now have a working database, and are ready to start the instance. The Database files are stored on the host under <em>mariadb/db/</em>:

$ singularity instance start \
    -B mariadb/db:/var/lib/mysql \
    -B mariadb/log:/var/log/mysql \
    -B mariadb/run:/var/run/mysqld \
    mariadb.sif mariadb

The instance is started so we’ll connect to it as the “newuser” account we created.

$ mysql -u newuser -p -h [Host IP address] workdb

For example:

$ mysql -u newuser -p -h 192.168.1.3 workdb

Insert a table and data for testing:

MariaDB [workdb]> CREATE TABLE test ( id INT UNSIGNED NOT NULL AUTO_INCREMENT UNIQUE, name varchar(64), PRIMARY KEY (id));
Query OK, 0 rows affected (0.23 sec)
MariaDB [workdb]> INSERT INTO test (name) VALUES ('name1'),('name2');

We have now setup a small database.  Exit from the Mysql client and run a test query against the database:

MariaDB [workdb]> exit
$ mysql -u newuser -p -h 192.168.1.3 workdb -e "SELECT * FROM test WHERE name = 'name2';"

You should get returned:

+----+-------+
| id | name  |
+----+-------+
|  2 | name2 |
+----+-------+

And on our host are the database files, owned by our user.

$ ls -l mariadb/db/workdb/
total 120
-rw-rw---- 1 test test     65 Sep 11 15:30 db.opt
-rw-rw---- 1 test test   1498 Sep 11 15:59 test.frm
-rw-rw---- 1 test test 114688 Sep 11 16:04 test.ibd

Blender

Our Blender container may not make açai bowls (yet), but you can use it to render Blender scenes.

Recipe:

Bootstrap: docker
From: ubuntu:16.04

%post
apt-get update
apt-get -y install blender blender-data
apt-get -y install less

%help
    singularity run blender.sif [scene file] [output directory] <frame | start frame:end frame>

    Example:
    Using `my_scene.blend`, render all frames, and output into `run/output`

        singularity run blender.sif my_scene.blend run/output

    Using `my_scene.blend`, render frames 100->200, and output into `run/output`

        singularity run blender.sif my_scene.blend run/output 100:200

    Using `my_scene.blend`, render frame 5, and output into `run/output`

        singularity run blender.sif my_scene.blend run/output 5

%runscript
    FRAME="-a"
    if [ -n "$3" ]; then

        if echo $3 | grep -q ":"; then
            STARTF=$(echo $3 | cut -f 1 -d ':')
            ENDF=$(echo $3 | cut -f 2 -d ':')

            FRAME="-s ${STARTF} -e ${ENDF} -a"
        else
            FRAME="-f $3"
        fi
    fi

    echo "Command to run is: /usr/bin/blender -b -noaudio $1 -o $2 ${FRAME}"
    /usr/bin/blender -b -noaudio $1 -o $2 ${FRAME}

Setup:

To use our Blender container, just build the file from the recipe file (blender.def) above:

$ singularity build blender.sif blender.def

Then you can run the following to render .blend (blender scene) files:

$ singularity run blender.sif [scene file] [output directory] <frame | start frame:end frame>

The documentation for the container (%help) can be accessed by running:

$ singularity run-help blender.sif

Running the following will render all frames:

$ singularity run --nv blender.sif my_scene.blend run/output

Render frames 100-200:

$ singularity run --nv blender.sif my_scene.blend run/output 100:200

Render frame 5:

v$ singularity run --nv blender.sif my_scene.blend run/output 5

The above examples are all using the –nv option, for bringing into the container the nVidia libraries from the host.

Join Our Mailing List

Categories

Related Posts

An Introduction to Singularity Containers

Mar 30, 2023 |

Enabling Portable and Secure Computing Environments for High-Performance Workloads.As part of their ongoing efforts to streamline workflows, enhance productivity, and save time, engineers, and developers in enterprises and high performance computing (HPC) focused...

read more