Introduction to Apptainer

Note

Prerequisites

This section uses the Myriad compute cluster to run Apptainer. An active account on Myriad is required. Before using Apptainer on another HPC system, consult the user guide for that system; but most of this content will apply to any system that supports Apptainer.

At face value, Apptainer is an alternative container implementation to Docker that has an overlapping set of features but some key differences as well. Apptainer is commonly available on shared clusters, such as UCL Research Computing’s HPC systems, because the Docker runtime is not secure on systems where users are not allowed to have “escalated privileges”. Importantly, the Apptainer runtime is compatible with Docker containers! So in general, we follow the practice of using Docker to develop containers and using Apptainer simply as a runtime to execute containers on HPC systems.

If you are familiar with Docker, Apptainer will feel familiar.

Login to Myriad

For today’s training, we will use the Myriad supercomputer. To login, you need to establish a SSH connection from your laptop to the Myriad system. Please follow these instructions to log in.

When you have successfully logged in, you should be greeted with some welcome text and a command prompt.

When you log in to the Myriad cluster, you enter onto a login node. The login node provides computing resources that are shared among everyone who is logged in to Myriad; therefore we cannot run any containers which are computationally demanding.

For now, we can familliarize ourselves with Apptainer on the login node. Later, we will use the compute nodes so that we can request sufficient resources for our container. In general, if you use Apptainer for research tasks you must run your containers on a compute node.

Load the Apptainer Module

By default, the apptainer command is not visible, but it can be added to the environment by loading the module.

$ module list

$ module show apptainer

$ module load apptainer

Now the apptainer command is available.

$ type apptainer

$ apptainer help

Core Apptainer Commands

Pull a Docker container

Containers in the Docker registry may be downloaded and used, assuming the underlying architecture (e.g. x86) is the same between the container and the host.

Create a new directory, then pull a container.

$ cd
$ mkdir apptainer
$ cd apptainer
$ apptainer pull docker://godlovedc/lolcow
INFO:    Converting OCI blobs to SIF format
WARNING: 'nodev' mount option set on /run/user/441890, it could be a source of failure during build process
INFO:    Starting build...
Getting image source signatures
Copying blob 8e860504ff1e done
Copying blob d010c8cf75d7 done
... # output ommitted
INFO:    Creating SIF file...
$ ls
lolcow_latest.sif

There may be some warning messages, but this command should download the latest version of the “lolcow” container and save it in your current working directory as lolcow_latest.sif.

Interactive shell

The shell command allows you to spawn a new shell within your container and interact with it as though it were a small virtual machine.

$ apptainer shell lolcow_latest.sif

Apptainer>

The change in prompt indicates that you have entered the container (though you should not rely on that to determine whether you are in container or not).

Once inside of an Apptainer container, you are the same user as you are on the host system. Also, a number of host directories are mounted by default.

Apptainer> whoami

Apptainer> id

Apptainer> pwd

Apptainer> exit

Note

Docker and Apptainer have very different conventions around how host directories are mounted within the container. In many ways, Apptainer has a simpler process for working with data on the host, but it is also more prone to inadvertantly having host configurations “leak” into the container.

Run a container’s default command

Just like with Docker, Apptainer can run the default “entrypoint” or default command of a container with the run subcommand. These defaults are defined in the Dockerfile (or Apptainer Definition file) that define the actions a container should perform when someone runs it.

    $ apptainer run lolcow_latest.sif

 ________________________________________
< The time is right to make new friends. >
 ----------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Note

You may receive a warning about “Setting locale failed”. This is because, by default, Apptainer sets all shell environment variables inside the container to match whatever is on the host. To override this behavior, add the --cleanenv argument to your command.

Executing arbitrary commands

The exec command allows you to execute a custom command within a container. For instance, to execute the cowsay program within the lolcow_latest.sif container:

    $ apptainer exec --cleanenv lolcow_latest.sif cowsay Apptainer runs Docker containers on HPC systems
 _______________________________________
/ Apptainer runs Docker containers on \
\ HPC systems                           /
 ---------------------------------------
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||

Note

exec also works with the library://, docker://, and shub:// URIs. This creates an ephemeral container that executes a command and disappears.

Once you are finished with your interactive session, you can end it and return to the login node with the exit command:

$ exit