Package: damona
Python version:Python 3.8, 3.9, 3.10Source:Seehttp://github.com/cokelaer/damona.Issues:Please fill a report ongithubPlatform:This is currently only available for Linux distribution with zsh/fish/bash shells (contributions are welcome to port the tool on other platforms/shells)Quick StartAssuming Singularity (Apptainer) is installed on your system, install Damona usingpipfor Python:pip install damonaYou need to configure Damona before using it. In a bash shell, type:damonaAdd these lines in your .bashrc:if [ ! -f  "~/.config/damona/damona.sh" ] ; then
    source ~/.config/damona/damona.sh
fiopen a new shelland then use damona:damona create TEST
damona activate TEST
damona install fastqc
fastqcThis should installfastqcin the newly created Damona environment (TEST). Type:fastqcit should open a window with thefastqcinterface.OverviewDamona is a singularity environment manager.Damona started as a small collections of singularity recipes to help installing third-party tools forSequana NGS pipelines.Damona is now used in production to create reproducible environments where singularity images and their associated binaries are installed altogether.In a nutshell, Damona combines the logic of Conda environments with the
reproducibility of singularity containers. We believe that it could be useful for
other projects and therefore decided to release it as an independent tool.As of 11st Jna 2024,Damonacontains 82 software, 128 releases 456 binaries.InstallationIf you are in a hurry, just type:pip install damona --upgradeYou must installSingularityto make use ofDamona.If you are familiar with conda, I believe you can do:conda install singularityTypedamonain a shell. This will initiate the tool with a config file in your HOME/.config/damona directory for bash shell andfish shellusers.Bash users should add this code in their ~/.bashrc file:source ~/.config/damona/damona.shFish shell users should add the following code in their ~/.config/fish/config.fish file:source ~/.config/damona/damona.fishZsh users should add the following code in their ~/.config/fish/config.fish file:source ~/.config/damona/damona.zshOpen a new shell and you are ready to go. Please see theInstallation in detailssection for more information.Quick StartDamonaneeds environments to work with.
First, let uscreateone, which is called TEST:damona create TESTSecond, we need toactivateit. Subsequent insallation will happen in this environment:damona activate TESTFrom there, we can install some binaries/images:damona install fastqc:0.11.9That’s it. Time to test. Typefastqc.To rename this TEST example:damona rename TEST --new-name prodor delete it:damona delete prodSee more examples hereafter or in the user guide onhttps://damona.readthedocs.ioMotivationAs stated on their website,Condais
an open sourcepackagemanagement system
andenvironmentmanagement system.
Conda provides pre-compiled releases of software; they can be installed in
different local environments that do not interfer with your system. This has
great advantages for developers. For example, you can install a pre-compiled
libraries in a minute instead of trying to compile it yourself including all
dependencies. Different communities have emerged using this
framework. One of them isBioconda, which is dedicated to bioinformatics.Another great tool that emerged in the last years isSingularity. Singularity containers can be used
to package entire scientific workflows,
software and libraries, and even data. It is a simple file that can be shared
between environments and guarantee exectution and reproducibility.Originally, Conda provided pre-compiled version of a software. Nowadays, it also provides
a docker and a singularity image of the tool. On the other side, Singularity can include an
entire conda environment. As you can see everything is there to build reproducible tools and
environment.Now, what about a software in development that depends on third-party packages ?
You would create a conda environment and starts installing the required packages.
Quickly, you will install another package that will break your environment due
to unresolved conlicts; this is not common but it happens. In the worst case
scenario, the environment is broken. In facilities where users depends on you,
it can be quite stresful and time-consuming to maintain several such
environments. This is why we have moved little by little to a very light conda
environment where known-to-cause-problem packages have been shipped into
singularity containers. This means we have to create aliases to those
singularities. The singularities can be simple executable containers or full
environment containers with many executables inside. In both cases, one need to
manage those containers for different users, pipelines, versions etc. This
started to be cumbersome to have containers in different places and update
script that generate the aliases to those executables.That’s wheredamonastarted: we wanted to combine the conda-like environment
framework to manage our singularity containers more easily.AlthoughDamonawas started with theSequana projet,Damonamay be useful for others developers who wish to have a quick and easy
solution for their users when they need to install third-party libraries.Before showing real-case examples, let us install the software itself and
understand the details.Installation in detailsThe is the egg and chicken paradox. To get reproducible container with
singularity, at some point you need to install singularity itself. That the first
of the two software that you will need to install. Instructions
are onsingularity web site. This
is not obvious to be honest. You need the GO language to be installed as well. I
personally installed from source and it worked like a charm.Second, you needDamona. This is a pure Python sotfware with only a few
dependencies. Install it with thepipsoftware provided with your Python
installation (Python 3.X):pip install damona --upgradeTypedamonato create the Damona tree structure. Images and binaries
will be saved in your home directory within the
~/.config/damona directory. There, special files should be available:damona.sh,damona.fishanddamona.cfg. Check that those files are present.Finally, you need to tell your system where to find damona. For bashrc users,
please add this line to you bashrc file:source ~/.config/damona/damona.shopen a new shell and typedamonaand you should be ready to go.For fishshell users, please add this line in~/.config/fish/config.fish*:source ~/.config/damona/damona.fishTutorialTheDamonastandalone is calleddamona. It has a documentation that should suffice for most users.The main documentation is obtained using:damona --helpwhere you will see the list ofDamonacommands (may be different with time) (may be:activate    Activate a damona environment.
clean       Remove orphan images and binaries from all environments.
create      Create a new environment
deactivate  Deactivate the current Damona environment.
delete      Remove an environment
env         List all environemnts with some stats.
export      Create a bundle of a given environment.
info        Print information about a given environment.
install     Download and install an image and its binaries.
list        List all packages that can be installed
remove      Remove binaries or image from an environment.
rename      Rename an existing environment
search      Search for a container or binary.
stats       Get information about Damona images and binariesTo get help for theinstallcommand, type:damona install --help1.listavailable environmentsBy default you have an environment calledbase. Unlike thebaseenvironment found inconda, it is not
essential and may be altered. However, it cannot be removed or created. You can check the list of environments using:damona env2.createenvironmentsAll environments are stored in~/.config/damona/envs/. You can create a new one as follows:damona create TESTThere, you have abindirectory where binaries are going to be installed.You can check that it has been created:damona envNote the last line telling you that:Your current env is 'TEST'.3. activate and deactivate environmentsIn order to install new binaries or software package, you must activate an environment. You may activate several but the last one is theactiveone. Let us activate theTESTenvironment:damona activate TESTCheck that it is active using:damona envand look at the last line. It should look like:Your current env is 'TEST'.What is going on when you activate an environment called TEST ? Simple: we append the directory ~/.config/damona/envs/TEST/bin to your PATH where binaries are searched for. This directory is removed when you use thedeactivatecommand.damona deactivate TEST
damona envshould remove the TEST environment from your PATH. You may activate several and deactivate them. In such case, the
environments behave as a Last In First Out principle:damona activate base
damona activate TEST
damona deactivateRemoves the last activated environments. While this set of commands is more specific:damona activate base
damona activate TEST
damona deactivate baseand keep the TEST environment only in your PATH.4.installa softwareLet us now consider that the TEST environment is active.Damona provides software that may have several releases. Each software/release comes with binaries that will be
installed together with the underlying singularity image.:damona install fastqc:0.11.9Here, the singularity image corresponding to the release 0.11.9 of thefastqcsoftware is downloaded. Then, binaries registered in this release are installed (here thefastqcbinary only).All images are stored in~/.config/damona/imagesand are shared between environments.5. Getinfoabout installed images and binariesYou can get the binaries installed in an environment (and the images used by
them) using theinfocommand:damona info TEST6. Search the registryYou can search for a binary using:damona search PATTERNExternal registry can be set-up. For instance, a damona registry is accessible
as follows (for demonstration):damona search fastqc --url damonaWheredamonais an alias defined in the .config/damona/damona.cfg that
is set tohttps://biomics.pasteur.fr/drylab/damona/registry.txtYou may retrieve images from a website where a registry exists (see the developer
guide to create a registry yourself).7. combine two different environmentsIn damona, you can have sereral environments in parallel and later activate the
one you wish to use. Let us create a new one:damone create test1and check that you now have one more environment:damona envWe want to create an alias to the previously downloaded image of fastqc tool but
in thetest1environment. First we activate the newly create environment:damona activate test1then, we install the container:damona install fastqc:0.11.9This will not download the image again. It will just create a binary in the
~/.config/damona/envs/test1/bin directory.you can combine this new environment with the base one:damona activate baseIf you are interested to know more, please see the User Guide and Developer
guide here below.ChangelogFrom version 0.10 onwards, we will not mention the new software and their versions
but only changes made to the code itself.VersionDescription0.11.1add pbsim. update hifiasm0.11.0add precommit, update to use pyproject0.10.1Fix the get_stats_software wrt new  zenodo API0.10.0ADD zsh supportUPDATE flye 2.9.1ADD nanopolishUPDATE remove nanopolish from sequana_tools binaries0.9.1ADD hmmer 3.2.2ADD trinotate 4.0.1ADD transdecode 5.7.0ADD trinity 2.15.1UPDATE bioconvert 1.1.0update bowtie2 2.5.10.9.0refactorise the command ‘env’ by splitting into dedicated subcommands
create, delete, rename. add progress bar when downloading containerNEW micromamba image to work as a localimageNEW sequana_minimal package to hold common tools (bwa, samtools,
kraken, etc)NEW ivarm pangolin, nextclade, subread, mafft packagesUPDATE fastp to 0.23.3, gffread to 0.12.7 (3 times lighter).UPDATE sequana_tools to use micromamba (30% lighter)0.8.4fix damona stats command to return unique binariesmore recipes and version (e.g. fastqc 0.12.1, graphviz update, etc)0.8.3create registry specifically for the sandbox (for testing)add damona community in the uploadsadd pbbam, bioconvert, busco, canu, ccsadd polypolish, samtools 1.16.1, sequana 0.14.6, flye 2.9, canu 2.1.1
circlator 1.5.5, hifiasm0.8.2add idr, samtools, homer, bamtools, bedtools, sequana_denovoadd seqkit recipe and containeradd shustring0.8.1Include ability to interact with biocontainers by allowing retrieval
of all biocontainers docker images using this syntax:
‘damona install biocontainers/xx:1.2.3’ Note that although 9000
containers are available, in practice, only about 1000 dockers are
on dockerhub, which is already nice :-)0.8.0Fix regression to install a software with its version0.7.1Implement the fish shelladd command “damona list”rename recipes/ directory into software/ and created a new library/
directory for images used as library, that are not installed.0.7.0Check that singularity is installedimplement the remove commandhttps://github.com/cokelaer/damona/issues/15more recipes cleanup (https://github.com/cokelaer/damona/issues/12)removed damona recipes (pure python package)cleanup all recipesadd zenodo stats (for admin)0.6.0add ability to upload images on zenodo. No need for external
repositories.ability to add/delete a software from different imagesimplement –help for the activate/deactivate (non trivial)add –rename option in ‘damona env’‘base’ environment is now at the same level as other environmentsbetter bash script; no need for DAMONA_EXE variable anymore.0.5.3Fixing config/shell0.5.2add missing shell package0.5.1add DAMONA_SINGULARITY_OPTIONS env variable in the binaryFix the way binaries are found in the releases.new recipes: rtoolsnew releases: sequana_tools_0.10.0Fix shell script to handle DAMONA_EXE variable0.5.0Major refactoring.Simplification of the registries (dropping notion of exe/set
classMain script should now be fully functional with functional
activation/deactivation.New command to build images from local recipes or dockerhub
entries.Install command can now install local container.DAMONA_PATH can be set to install damona images/binaries
anywhere, not just in local home.check md5 of images to not download/copy again0.4.3Implement damona activate/deactivate0.4.2Fix typo in the creation of aliases for ‘set’ containers0.4.1implemented aliases for the –from-url option stored in a
damona.cfg file0.4.0implemented the ‘env’ and ‘activate’ commandability to setup an external registry on any https and retrieve
registry from there to download external images0.3.Xadd gffread, rnadiff recipes0.3.0A stable version with documentation and >95% coverage read-yto-use0.2.3add new recipes (rnadiff)0.2.2Download latest if no version providedincludebuildcommand to build image locally0.2.1fixed manifest0.2.0first working version of damona to pull image locally with binaries0.1.1small update to fix RTD, travis, coveralls0.1first release to test feasibility of the project