Build, Run, and Manage Containers with Podman

Peng Lian

2023-10-11

Virtualization Technologies

Containers and Virtual Machines

Containers and virtual machines are very similar resource virtualization technologies. Virtualization is the process in which a system singular resource like RAM, CPU, Disk, or Networking can be ‘virtualized’ and represented as multiple resources. The key differentiator between containers and virtual machines is that virtual machines virtualize an entire machine down to the hardware layers and containers only virtualize software layers above the operating system level.

Containers vs Virtual Machines

Containers Virtual Machines
Pros lightweight, fast, robust ecosystem full isolation, interactive development
Cons shared host exploits slow, storage size cost
Popular providers Docker/Podman, Singularity, CRI-O Virtualbox, VMWare, QEMU


Containers are lightweight software packages that contain all the dependencies required to execute the contained software application. These dependencies include things like system libraries, external third-party code packages, and other operating system level applications. The dependencies included in a container exist in stack levels that are higher than the operating system.

Virtual machines are heavy software packages that provide complete emulation of low level hardware devices like CPU, Disk and Networking devices.

Podman vs Docker

Docker is the standard container management technology. It has so much weight in the industry that when most people think of containers, they think of Docker.

Podman is an open-source, Linux-native tool designed to develop, manage, and run containers and pods under the Open Container Initiative (OCI) standards. It is developed by Red Hat and is the default container engine in RedHat 8.

  • Podman does not require a separate daemon to run containers, making it more lightweight and secure.
  • Podman has better support for running containers as non-root users, which can improve security.
  • Podman can run Kubernetes pods natively without needing a separate tool like Docker Compose.

Access to Podman on BioHPC

Podman is now available on the BioHPC cluster after the latest update to RHEL 7.9. This training will demonstrate how to use rootless Podman under your user account.


An important thing to note is that because it is rootless, everything Podman creates - including images and containers - is stored in a temporary directory and removed when you log out. So remember to push your images to a container registry like the BioHPC GitLab (git.biohpc.swmed.edu:5050) or Docker Hub after building to avoid losing them.

Option 1. Through BioHPC WebGUI

  • Go to BioHPC Portal –> Cloud Services –>Web Visualization at here (https://portal.biohpc.swmed.edu/intranet/terminal/webgui/)

Option 1. Through BioHPC WebGUI (continued)

  • Select the Job Type from the dropdown list and launch the job.

  • Once the resource is ready, connect to the session with web or VNC Client.

Option 1. Through BioHPC WebGUI (continued)

  • Open Terminal and start to use the Podman

Option 2. Via SLURM interactive job

  • Log in to Nucleus with any SSH software you prefer
# E.g. run the ssh command from Linux to log in Nucleus
ssh YOUR_USER_NAME@nucleus.biohpc.swmed.edu
  • Start an interactive SLURM job on super partition. You can replace super with any partition you’d like to use, but super is quick.
srun -p super -N 1 --pty bash
  • Once the resource is ready for you, then you can start to run the podman commands.
podman --version

Option 3. Via SLURM sbatch job

Similar to Option 2, the user could create a sbatch script and submit the job to SLURM.

  • Log in to Nucleus with any SSH software you prefer
# E.g. run the ssh command from Linux to log in Nucleus
ssh YOUR_USER_NAME@nucleus.biohpc.swmed.edu
  • Create a sbatch script like the example below
#!/bin/bash
#
#SBATCH --job-name Podman
#SBATCH -p super
#SBATCH -N 1
#SBATCH -t 0-2:0:0
#SBATCH -o job_%j.out
#SBATCH -e job_%j.err

podman images

podman pull ubuntu

podman run --rm -it docker.io/library/ubuntu:latest cat /etc/os-release
  • Submit the script to SLURM
sbatch YOUR_SBATCH_SCRIPT

Basic Podman Commands

Basic Podman Commands - I

  • If docker is preferred. an alias for the podman command as docker could be made.
alias docker='podman'
  • To check the current images
podman images
  • To pull an image. By default, podman will try to pull images from registry.redhat.io. Please add docker.io if you want to pull from Docker Hub.
# Pull from Docker Hub
podman pull docker.io/ubuntu:22.04

# Pull from your personal repo of BioHPC GitLab
podman pull git.biohpc.swmed.edu:5050/astrocyte/container/r:4.2.2
  • To run an image.
# Run the image directly (Better know the default command of the image before running it.)
podman run docker.io/library/ubuntu:22.04

# Run the image directly and remove the containers after finished.
podman run --rm  docker.io/library/ubuntu:22.04

# Run the image interactively (Go inside the image) and remove the containers after finished.
podman run --rm -it docker.io/library/ubuntu:22.04 /bin/bash
  • To check the existing containers
# Check running containers
podman ps

# Check all containers (including stopped ones)
podman ps -a

Basic Podman Commands - II

  • To remove containers/images
# Remove the container with an ID as b61f5b266e6f
podman rm b61f5b266e6f

# Force to remove the container with an ID as b61f5b266e6f
podman rm -f b61f5b266e6f

# Remove a image
podman rmi docker.io/library/ubuntu:22.04

# Force to remove a image
podman rmi -f docker.io/library/ubuntu:22.04
  • To save/load an image
# Save an image to tar
podman save docker.io/library/ubuntu:latest > ubuntu_latest.tar

# Remove the image for testing
podman rmi docker.io/library/ubuntu:latest

# Load the archived image
podman load < ubuntu_latest.tar

# Check the images again
podman images
  • Convert an image to Singularity image
module load singularity/3.9.9

# Convert docker archive to Singularity image
singularity build ubuntu_latest.sif docker-archive://ubuntu_latest.tar

# Run with Singularity
singularity exec ubuntu_latest.sif /bin/bash

Basic Podman Commands - Personal Access Token

To push an image to BioHPC GitLab registry, you need to create an Access Tokens from your BioHPC GitLab settings and log in with podman first. There are two kinds of access tokens, Personal Access Token and Project Access Token. Either one with enough permission should be able to work with Podman. Here are the steps to create them:

  • To create a Personal Access Token for your BioHPC GitLab account. (This kind of token could be used to access all your repos on BioHPC GitLab).
  1. Log in to your BioHPC GitLab
  2. Select Preferences from the dropdown of your personal icon at the top right corner
  3. From the left menu select Access Tokens
  4. Fill out the form by adding Token name, Expiration date, Select a role, and Select scopes. To be able to push images, you need at least have the write_registry permission selected.
  5. Click the Create personal access token button.
  6. Write down the token string. (This will be your password and it will only show once!).

Basic Podman Commands - Project Access Token

  • To create a Project Access Token for your repo. (This kind of token only controls the access of the repo).
  1. Go to your project on BioHPC GitLab.
  2. Got to Settings –> Access Tokens (in the left-hand menu).
  3. Fill out the form by adding Token name, Expiration date, Select a role, and Select scopes. To be able to push images, you need at least have the write_registry permission selected.
  4. Click the Create project access token button.
  5. Write down the token string. (This will be your password and it will only show once!).
  • To log in to BioHPC GitLab registry
podman login git.biohpc.swmed.edu:5050
# Provide the "Token name" as your username
# Provide the "Token" as your password
  • To tag an image. (Need to tag an image before pushing to a private registry)
# Tag the image with private BioHPC GitLab repo
podman tag    docker.io/library/ubuntu:22.04    git.biohpc.swmed.edu:5050/astrocyte/container/ubuntu:22.04
  • To push the image to BioHPC GitLab registry
podman push git.biohpc.swmed.edu:5050/astrocyte/container/ubuntu:22.04

Acknowledgement

  • Thank all BioHPC team members for their support.

  • Please acknowledge our contribution by adding the following sentence to your paper:

This research was supported in part by the computational resources provided by the BioHPC supercomputing facility located in the Lyda Hill Department of Bioinformatics, UT Southwestern Medical Center.

Questions?

Thanks for your attention!