Package: singularity-permanent-cache
Singularity-permanent-cache creates a permanent cache for singularity images on
disk. It takes a URI as argument and returns the location of the image. It
utilizes a filelock to prevent cache corruption.It will use theSINGULARITY_PERMANENTCACHEDIRorSINGULARITY_CACHEDIRenvironment
variables to determine the location of the cache. Alternatively the cache dir
can be set with the-dor--cache-dirflags on the command line.Thesingularity-permanent-cachecommand can be used in scripts. It was
designed with multiprocess usage in mind: a filelock will prevent corruption
of the cache when multiple instances of singularity-permanent-cache are
running. It can be used in a script like this:#!/usr/bin/env bashset-eu-opipefailexportSINGULARITY_PERMANENTCACHEDIR=$HOME/.singularity/permanent_cacheMY_IMAGE_URI="docker://debian:buster-slim"IMAGE_LOCATION=$(singularity-permanent-cache$MY_IMAGE_URI)cluster_submit"singularity exec$IMAGE_LOCATIONecho 'Hello world!'"Singularity-permanent-cache will download the debian buster slim image
if it is not yet in the cache. It will not dowload anything if it is already
in the cache.WarningDo not usesingularity-permanent-cacheon images with unstable tags
such asdocker://ubuntu:latest. Once thedocker_ubuntu_latest.sifimage is in the cache,singularity-permanent-cachewill never check
for a newer version!Use containers with stable tags, such asbiocontainersor use hashes. (For example:ubuntu@sha256:a69390df0911533dd2fc552a8765481bf6a93b5d5952a9ddbe9cb64ca3479e17.)Notesingularity-permanent-cache utilizes a filelock which only works if
multiple singularity-permanent-cache processes are launched on the same
machine. If multiple processes are launched on multiple machines connected
to the same networked filesystem then cache corruption may occur.UsageBesidesingularity-permanent-cache, alsospcis added to PATH as a
short-hand when installing the package.singularity-permanent-cacheis
also available as a stand-alone scriptsingularity_permanent_cache.py.singularity-permant-cachehas no dependencies and only requires a modern
python version (3.5 or higher).usage: singularity-permanent-cache [-h] [-d CACHE_DIR] [-s SINGULARITY_EXE]
                               [--which-cache] [-v] [-q]
                               <IMAGE>

Creates a permanent cache on disk for singularity images. Returns the location
of the image in the cache. WARNING: This program will never check if a newer
image is available. Make sure unique tags or hashes are used!

positional arguments:
  <IMAGE>               The singularity URI to the image. For example:
                        'docker://debian:buster-slim'

optional arguments:
  -h, --help            show this help message and exit
  -d CACHE_DIR, --cache-dir CACHE_DIR
                        Path to the cache location. Uses the
                        SINGULARITY_PERMANENTCACHEDIR, or SINGULARITY_CACHEDIR
                        environment variable by default.
  -s SINGULARITY_EXE, --singularity-exe SINGULARITY_EXE
                        Path to singularity executable.
  --which-cache         Show which cache the program will use and exit.
  -v, --verbose         Increase log verbosity. Can be used multiple times.
  -q, --quiet           Decrease log verbosity. Can be used multiple times.AcknowledgementsLots of thanks to @TMiguelT, @illusional and @vsoch for their constructive
feedback onthis PR for Cromwellwhich led to the
development of this program.The filelock implementation is based onpy-filelock.
Huge thanks to @benediktschmitt & contributors for this filelock example
which they made Public Domain.