Using singularity recipes

Creating a Singularity Definition File

Via Docker bootstrap

A Singularity Definition File is a text file that contains a series of statements that are used to create a container image. In line with the configuration as code approach mentioned above, the definition file can be stored in your code repository alongside your application code and used to create a reproducible image. This means that for a given commit in your repository, the version of the definition file present at that commit can be used to reproduce a container with a known state. It was pointed out earlier in the course, when covering Docker, that this property also applies for Dockerfiles.

We’ll now look at a very simple example of a definition file:

Bootstrap: docker
From: ubuntu:20.04

    apt-get -y update && apt-get install -y python

    python -c 'print("Hello World! Hello from our custom Singularity image!")'

A definition file has a number of optional sections, specified using the % prefix, that are used to define or undertake different configuration during different stages of the image build process. You can find full details in Singularity’s Definition Files documentation. In our very simple example here, we only use the %post and %runscript sections.

Let’s step through this definition file and look at the lines in more detail:

Bootstrap: docker
From: ubuntu:20.04

These first two lines define where to bootstrap our image from. Why can’t we just put some application binaries into a blank image? Any applications or tools that we want to run will need to interact with standard system libraries and potentially a wide range of other libraries and tools. These need to be available within the image and we therefore need some sort of operating system as the basis for our image. The most straightforward way to achieve this is to start from an existing base image containing an operating system. In this case, we’re going to start from a minimal Ubuntu 20.04 Linux Docker image. Note that we’re using a Docker image as the basis for creating a Singularity image. This demonstrates the flexibility in being able to start from different types of images when creating a new Singularity image.

The Bootstrap: docker line is similar to prefixing an image path with docker:// when using, for example, the singularity pull command. A range of different bootstrap options are supported. From: ubuntu:20.04 says that we want to use the ubuntu image with the tag 20.04 from Docker Hub.

Next we have the %post section of the definition file:

    apt-get -y update && apt-get install -y python3

In this section of the file we can do tasks such as package installation, pulling data files from remote locations and undertaking local configuration within the image. The commands that appear in this section are standard shell commands and they are run within the context of our new container image. So, in the case of this example, these commands are being run within the context of a minimal Ubuntu 20.04 image that initially has only a very small set of core packages installed.

Here we use Ubuntu’s package manager to update our package indexes and then install the python3 package along with any required dependencies. The -y switches are used to accept, by default, interactive prompts that might appear asking you to confirm package updates or installation. This is required because our definition file should be able to run in an unattended, non-interactive environment.

Finally we have the %runscript section:

    python3 -c 'print("Hello World! Hello from our custom Singularity image!")'

This section is used to define a script that should be run when a container is started based on this image using the singularity run command. In this simple example we use python3 to print out some text to the console.

We can now save the contents of the simple defintion file shown above to a file and build an image based on it. In the case of this example, the definition file has been named my_test_image.def. (Note that the instructions here assume you’ve bound the image output directory you created to the /home/singularity directory in your Docker Singularity container, as explained in the “Getting started with the Docker Singularity image” section above.):

$ singularity build /home/singularity/my_test_image.sif /home/singularity/my_test_image.def

Examples from Bioinformatics

BootStrap: docker
From: biocontainers/fastqc:v0.11.9_cv6

    echo "Welcome to FastQC Image"
    fastqc --version

    echo "Image built"
sudo singularity build fastqc.sif docker.singularity

Via Debian bootstrap

BootStrap: debootstrap
OSVersion: bionic
Include: build-essential curl python python-dev openjdk-11-jdk bzip2 zip unzip

    echo "Welcome to my Singularity Image"
    fastqc --version
    multiqc --version
    bowtie --version



    cd /usr/local; curl -k -L${FASTQC_VERSION}.zip >
    cd /usr/local; unzip; rm; chmod 775 FastQC/fastqc; ln -s /usr/local/FastQC/fastqc /usr/local/bin/fastqc

    cd /usr/local; curl --fail --silent --show-error --location --remote-name$BOWTIE_VERSION/bowtie-${BOWTIE_VERSION}
    cd /usr/local; unzip -d /usr/local bowtie-${BOWTIE_VERSION}
    cd /usr/local; rm bowtie-${BOWTIE_VERSION}
    cd /usr/local/bin; ln -s ../bowtie-${BOWTIE_VERSION}-linux-x86_64/bowtie* .

    curl --fail --silent --show-error --location --remote-name

    pip install numpy matplotlib
    pip install -I multiqc==${MULTIQC_VERSION}

    echo "Biocore image built"

    Maintainer Biocorecrg
Version 0.1.0

In order to run the command shown below, you have to install debootstrap before e.g. sudo apt-get install debootstrap.

sudo singularity build fastqc-multi-bowtie.sif debootstrap.singularity