neulab/tldr huggingface dataset:
--sig-proxy=true|false Proxy received signals to the process (non-TTY mode only). SIGCHLD, SIGSTOP, and SIGKILL are not proxied. The default is true.
--memory-swappiness="" Tune a container's memory swappiness behavior. Accepts an integer between 0 and 100.
-t, --tty=true|false Allocate a pseudo-TTY. The default is false.
When set to true Docker can allocate a pseudo-tty and attach to the standard input of any container. This can be used, for example, to run a throwaway interactive shell. The default is false.
The -t option is incompatible with a redirection of the docker client standard input.
--tmpfs=[] Create a tmpfs mount
Mount a temporary filesystem (tmpfs) mount into a container, for example:
$ docker run -d --tmpfs /tmp:rw,size=787448k,mode=1777 my_image
This command mounts a tmpfs at /tmp within the container.  The supported mount options are the same as the Linux default mount flags. If you do not specify any options, the systems uses the following options: rw,noexec,nosuid,nodev,size=65536k.
See also --mount, which is the successor of --tmpfs and --volume. Even though there is no plan to deprecate --tmpfs, usage of --mount is recommended.
-u, --user="" Sets the username or UID used and optionally the groupname or GID for the specified command.
The followings examples are all valid: --user [user | user:group | uid | uid:gid | user:gid | uid:group ]
Without this argument the command will be run as root in the container.
--ulimit=[] Ulimit options
-v|--volume[=[[HOST-DIR:]CONTAINER-DIR[:OPTIONS]]] Create a bind mount. If you specify, -v /HOST-DIR:/CONTAINER-DIR, Docker bind mounts /HOST-DIR in the host to /CONTAINER-DIR in the Docker container. If 'HOST-DIR' is omitted,  Docker automatically creates the new volume on the host.  The OPTIONS are a comma delimited list and can be:
• [rw|ro]
• [z|Z]
• [[r]shared|[r]slave|[r]private]
• [delegated|cached|consistent]
• [nocopy]
The CONTAINER-DIR must be an absolute path such as /src/docs. The HOST- DIR can be an absolute path or a name value. A name value must start with an alphanumeric character, followed by a-z0-9, _ (underscore), . (period) or - (hyphen). An absolute path starts with a / (forward slash).
If you supply a HOST-DIR that is an absolute path,  Docker bind-mounts to the path you specify. If you supply a name, Docker creates a named volume by that name. For example, you can specify either /foo or foo for a HOST- DIR value. If you supply the /foo value, Docker creates a bind mount. If you supply the foo specification, Docker creates a named volume.
You can specify multiple  -v options to mount one or more mounts to a container. To use these same mounts in other containers, specify the --volumes-from option also.
You can supply additional options for each bind mount following an additional colon.  A :ro or :rw suffix mounts a volume in read-only or read-write mode, respectively. By default, volumes are mounted in read- write mode.  You can also specify the consistency requirement for the mount, either :consistent (the default), :cached, or :delegated. Multiple options are separated by commas, e.g. :ro,cached.
Labeling systems like SELinux require that proper labels are placed on volume content mounted into a container. Without a label, the security system might prevent the processes running inside the container from using the content. By default, Docker does not change the labels set by the OS.
To change a label in the container context, you can add either of two suffixes :z or :Z to the volume mount. These suffixes tell Docker to relabel file objects on the shared volumes. The z option tells Docker that two containers share the volume content. As a result, Docker labels the content with a shared content label. Shared volume labels allow all containers to read/write content.  The Z option tells Docker to label the content with a private unshared label.  Only the current container can use a private volume.
By default bind mounted volumes are private. That means any mounts done inside container will not be visible on host and vice-a-versa. One can change this behavior by specifying a volume mount propagation property. Making a volume shared mounts done under that volume inside container will be visible on host and vice-a-versa. Making a volume slave enables only one way mount propagation and that is mounts done on host under that volume will be visible inside container but not the other way around.
To control mount propagation property of volume one can use :[r]shared, :[r]slave or :[r]private propagation flag. Propagation property can be specified only for bind mounted volumes and not for internal volumes or named volumes. For mount propagation to work source mount point (mount point where source dir is mounted on) has to have right propagation properties. For shared volumes, source mount point has to be shared. And for slave volumes, source mount has to be either shared or slave.
Use df <source-dir> to figure out the source mount and then use findmnt -o TARGET,PROPAGATION <source-mount-dir> to figure out propagation properties of source mount. If findmnt utility is not available, then one can look at mount entry for source mount point in /proc/self/mountinfo. Look at optional fields and see if any propagation properties are specified.  shared:X means mount is shared, master:X means mount is slave and if nothing is there that means mount is private.
To change propagation properties of a mount point use mount command. For example, if one wants to bind mount source directory /foo one can do mount --bind /foo /foo and mount --make-private --make-shared /foo. This will convert /foo into a shared mount point. Alternatively one can directly change propagation properties of source mount. Say / is source mount for /foo, then use mount --make-shared / to convert / into a shared mount.
Note: When using systemd to manage the Docker daemon's start and stop, in the systemd unit file there is an option to control mount propagation for the Docker daemon itself, called MountFlags. The value of this setting may cause Docker to not see mount propagation changes made on the mount point. For example, if this value is slave, you may not be able to use the shared or rshared propagation on a volume.
To disable automatic copying of data from the container path to the volume, use the nocopy flag. The nocopy flag can be set on bind mounts and named volumes.
See also --mount, which is the successor of --tmpfs and --volume.  Even though there is no plan to deprecate --volume, usage of --mount is recommended.
--volume-driver="" Container's volume driver. This driver creates volumes specified either from a Dockerfile's VOLUME instruction or from the docker run -v flag. See docker-volume-create(1) for full details.
--volumes-from=[] Mount volumes from the specified container(s)
Mounts already mounted volumes from a source container onto another container. You must supply the source's container-id. To share a volume, use the --volumes-from option when running the target container. You can share volumes even if the source container is not running.
By default, Docker mounts the volumes in the same mode (read-write or read-only) as it is mounted in the source container. Optionally, you can change this by suffixing the container-id with either the :ro or :rw keyword.
If the location of the volume from the source container overlaps with data residing on a target container, then the volume hides that data on the target.
-w, --workdir="" Working directory inside the container
The default working directory for running binaries within a container is the root directory (/). The developer can set a different default with the Dockerfile WORKDIR instruction. The operator can override the working directory by using the -w option.
Exit Status The exit code from docker run gives information about why the container failed to run or why it exited.  When docker run exits with a non-zero code, the exit codes follow the chroot standard, see below:
125 if the error is with Docker daemon itself
$ docker run --foo busybox; echo $? # flag provided but not defined: --foo See 'docker run --help'. 125
126 if the contained command cannot be invoked
$ docker run busybox /etc; echo $? # exec: "/etc": permission denied docker: Error response from daemon: Contained command could not be invoked 126
127 if the contained command cannot be found
$ docker run busybox foo; echo $? # exec: "foo": executable file not found in $PATH docker: Error response from daemon: Contained command not found or does not exist 127
Exit code of contained command otherwise
$ docker run busybox /bin/sh -c 'exit 3' # 3
Running container in read-only mode During container image development, containers often need to write to the image content.  Installing packages into /usr, for example.  In production, applications seldom need to write to the image.  Container applications write to volumes if they need to write to file systems at all.  Applications can be made more secure by running them in read-only mode using the --read-only switch.  This protects the containers image from modification. Read only containers may still need to write temporary data.  The best way to handle this is to mount tmpfs directories on /run and /tmp.
# docker run --read-only --tmpfs /run --tmpfs /tmp -i -t fedora /bin/bash
Exposing log messages from the container to the host's log If you want messages that are logged in your container to show up in the host's syslog/journal then you should bind mount the /dev/log directory as follows.
# docker run -v /dev/log:/dev/log -i -t fedora /bin/bash
From inside the container you can test this by sending a message to the log.
(bash)# logger "Hello from my container"
Then exit and check the journal.
# exit
# journalctl -b | grep Hello
This should list the message sent to logger.
Attaching to one or more from STDIN, STDOUT, STDERR If you do not specify -a then Docker will attach everything (stdin,stdout,stderr) you'd like to connect instead, as in:
# docker run -a stdin -a stdout -i -t fedora /bin/bash
Sharing IPC between containers Using shm_server.c available here: https://www.cs.cf.ac.uk/Dave/C/node27.html
Testing --ipc=host mode:
Host shows a shared memory segment with 7 pids attached, happens to be from httpd:
$ sudo ipcs -m
------ Shared Memory Segments -------- key        shmid      owner      perms      bytes      nattch     status 0x01128e25 0          root       600        1000       7
Now run a regular container, and it correctly does NOT see the shared memory segment from the host:
$ docker run -it shm ipcs -m
------ Shared Memory Segments -------- key        shmid      owner      perms      bytes      nattch     status
Run a container with the new --ipc=host option, and it now sees the shared memory segment from the host httpd:
$ docker run -it --ipc=host shm ipcs -m
Testing --ipc=container:CONTAINERID mode:
Start a container with a program to create a shared memory segment:
$ docker run -it shm bash $ sudo shm/shm_server & $ sudo ipcs -m
------ Shared Memory Segments -------- key        shmid      owner      perms      bytes      nattch     status 0x0000162e 0          root       666        27         1
Create a 2nd container correctly shows no shared memory segment from 1st container:
$ docker run shm ipcs -m
Create a 3rd container using the new --ipc=container:CONTAINERID option, now it shows the shared memory segment from the first:
$ docker run -it --ipc=container:ed735b2264ac shm ipcs -m $ sudo ipcs -m
Linking Containers Note: This section describes linking between containers on the default (bridge) network, also known as "legacy links". Using --link on user-defined networks uses the DNS-based discovery, which does not add entries to /etc/hosts, and does not set environment variables for discovery.
The link feature allows multiple containers to communicate with each other. For example, a container whose Dockerfile has exposed port 80 can be run and named as follows:
# docker run --name=link-test -d -i -t fedora/httpd
A second container, in this case called linker, can communicate with the httpd container, named link-test, by running with the --link=:
# docker run -t -i --link=link-test:lt --name=linker fedora /bin/bash
Now the container linker is linked to container link-test with the alias lt.  Running the env command in the linker container shows environment variables with the LT (alias) context (LT_)
# env HOSTNAME=668231cb0978 TERM=xterm LT_PORT_80_TCP=tcp://172.17.0.3:80 LT_PORT_80_TCP_PORT=80 LT_PORT_80_TCP_PROTO=tcp LT_PORT=tcp://172.17.0.3:80 PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin PWD=/ LT_NAME=/linker/lt SHLVL=1 HOME=/ LT_PORT_80_TCP_ADDR=172.17.0.3 _=/usr/bin/env
When linking two containers Docker will use the exposed ports of the container to create a secure tunnel for the parent to access.
If a container is connected to the default bridge network and linked with other containers, then the container's /etc/hosts file is updated with the linked container's name.
Note Since Docker may live update the container's /etc/hosts file, there may be situations when processes inside the container can end up reading an empty or incomplete /etc/hosts file. In most cases, retrying the read again should fix the problem.
Mapping Ports for External Usage The exposed port of an application can be mapped to a host port using the -p flag. For example, an httpd port 80 can be mapped to the host port 8080 using the following:
# docker run -p 8080:80 -d -i -t fedora/httpd
Creating and Mounting a Data Volume Container Many applications require the sharing of persistent data across several containers. Docker allows you to create a Data Volume Container that other containers can mount from. For example, create a named container that contains directories /var/volume1 and /tmp/volume2. The image will need to contain these directories so a couple of RUN mkdir instructions might be required for you fedora-data image:
# docker run --name=data -v /var/volume1 -v /tmp/volume2 -i -t fedora-data true # docker run --volumes-from=data --name=fedora-container1 -i -t fedora bash
Multiple --volumes-from parameters will bring together multiple data volumes from multiple containers. And it's possible to mount the volumes that came from the DATA container in yet another container via the fedora-container1 intermediary container, allowing to abstract the actual data source from users of that data:
# docker run --volumes-from=fedora-container1 --name=fedora-container2 -i -t fedora bash
Mounting External Volumes To mount a host directory as a container volume, specify the absolute path to the directory and the absolute path for the container directory separated by a colon:
# docker run -v /var/db:/data1 -i -t fedora bash
When using SELinux, be aware that the host has no knowledge of container SELinux policy. Therefore, in the above example, if SELinux policy is enforced, the /var/db directory is not writable to the container. A "Permission Denied" message will occur and an avc: message in the host's syslog.
To work around this, at time of writing this man page, the following command needs to be run in order for the proper SELinux policy type label to be attached to the host directory:
# chcon -Rt svirt_sandbox_file_t /var/db
Now, writing to the /data1 volume in the container will be allowed and the changes will also be reflected on the host in /var/db.
Using alternative security labeling You can override the default labeling scheme for each container by specifying the --security-opt flag. For example, you can specify the MCS/MLS level, a requirement for MLS systems. Specifying the level in the following command allows you to share the same content between containers.
# docker run --security-opt label=level:s0:c100,c200 -i -t fedora bash
An MLS example might be:
# docker run --security-opt label=level:TopSecret -i -t rhel7 bash
To disable the security labeling for this container versus running with the --permissive flag, use the following command:
# docker run --security-opt label=disable -i -t fedora bash
If you want a tighter security policy on the processes within a container, you can specify an alternate type for the container. You could run a container that is only allowed to listen on Apache ports by executing the following command:
# docker run --security-opt label=type:svirt_apache_t -i -t centos bash
You would have to write policy defining a svirt_apache_t type.
Setting device weight If you want to set /dev/sda device weight to 200, you can specify the device weight by --blkio-weight-device flag. Use the following command:
# docker run -it --blkio-weight-device "/dev/sda:200" ubuntu
Specify isolation technology for container (--isolation) This option is useful in situations where you are running Docker containers on Microsoft Windows. The --isolation <value> option sets a container's isolation technology. On Linux, the only supported is the default option which uses Linux namespaces. These two commands are equivalent on Linux:
$ docker run -d busybox top $ docker run -d --isolation default busybox top
On Microsoft Windows, can take any of these values:
• default: Use the value specified by the Docker daemon's --exec- opt . If the daemon does not specify an isolation technology, Microsoft Windows uses process as its default value.
• process: Namespace isolation only.
• hyperv: Hyper-V hypervisor partition-based isolation.
In practice, when running on Microsoft Windows without a daemon option set,  these two commands are equivalent:
$ docker run -d --isolation default busybox top $ docker run -d --isolation process busybox top
If you have set the --exec-opt isolation=hyperv option on the Docker daemon, any of these commands also result in hyperv isolation:
$ docker run -d --isolation default busybox top $ docker run -d --isolation hyperv busybox top
Setting Namespaced Kernel Parameters (Sysctls) The --sysctl sets namespaced kernel parameters (sysctls) in the container. For example, to turn on IP forwarding in the containers network namespace, run this command:
$ docker run --sysctl net.ipv4.ip_forward=1 someimage
Not all sysctls are namespaced. Docker does not support changing sysctls inside of a container that also modify the host system. As the kernel evolves we expect to see more sysctls become namespaced.
See the definition of the --sysctl option above for the current list of supported sysctls.
April 2014, Originally compiled by William Henry (whenry at redhat dot com) based on docker.com source material and internal work.  June 2014, updated by Sven Dowideit SvenDowideit@home.org.au ⟨mailto:SvenDowideit@home.org.au⟩ July 2014, updated by Sven Dowideit SvenDowideit@home.org.au ⟨mailto:SvenDowideit@home.org.au⟩ November 2015, updated by Sally O'Malley somalley@redhat.com ⟨mailto:somalley@redhat.com⟩
samtools(1)                   Bioinformatics tools                   samtools(1)
samtools - Utilities for the Sequence Alignment/Map (SAM) format
samtools view -bt ref_list.txt -o aln.bam aln.sam.gz
samtools tview aln.sorted.bam ref.fasta
samtools quickcheck in1.bam in2.cram
samtools index aln.sorted.bam
samtools sort -T /tmp/aln.sorted -o aln.sorted.bam aln.bam
samtools collate -o aln.name_collated.bam aln.sorted.bam
samtools idxstats aln.sorted.bam
samtools flagstat aln.sorted.bam
samtools flags PAIRED,UNMAP,MUNMAP
samtools stats aln.sorted.bam
samtools bedcov aln.sorted.bam
samtools depth aln.sorted.bam
samtools ampliconstats primers.bed in.bam
samtools mpileup -C50 -f ref.fasta -r chr3:1,000-2,000 in1.bam in2.bam
samtools coverage aln.sorted.bam
samtools merge out.bam in1.bam in2.bam in3.bam
samtools split merged.bam
samtools cat out.bam in1.bam in2.bam in3.bam
samtools import input.fastq > output.bam
samtools fastq input.bam > output.fastq
samtools fasta input.bam > output.fasta
samtools faidx ref.fasta
samtools fqidx ref.fastq
samtools dict -a GRCh38 -s "Homo sapiens" ref.fasta
samtools calmd in.sorted.bam ref.fasta
samtools fixmate in.namesorted.sam out.bam
samtools markdup in.algnsorted.bam out.bam
samtools addreplacerg -r 'ID:fish' -r 'LB:1334' -r 'SM:alpha' -o output.bam input.bam
samtools reheader in.header.sam in.bam > out.bam
samtools targetcut input.bam
samtools phase input.bam
samtools depad input.bam
samtools ampliconclip -b bed.file input.bam
Samtools is a set of utilities that manipulate alignments in the SAM (Sequence Alignment/Map), BAM, and CRAM formats.  It converts between the formats, does sorting, merging and indexing, and can retrieve reads in any regions swiftly.
Samtools is designed to work on a stream. It regards an input file `-' as the standard input (stdin) and an output file `-' as the standard output (stdout). Several commands can thus be combined with Unix pipes. Samtools always output warning and error messages to the standard error output (stderr).
Samtools is also able to open files on remote FTP or HTTP(S) servers if the file name starts with `ftp://', `http://', etc.  Samtools checks the current working directory for the index file and will download the index upon absence. Samtools does not retrieve the entire alignment file unless it is asked to do so.
If an index is needed, samtools looks for the index suffix appended to the filename, and if that isn't found it tries again without the filename suffix (for example in.bam.bai followed by in.bai).  However if an index is in a completely different location or has a different name, both the main data filename and index filename can be pasted together with ##idx##.  For example /data/in.bam##idx##/indices/in.bam.bai may be used to explicitly indicate where the data and index files reside.
Each command has its own man page which can be viewed using e.g. man samtools-view or with a recent GNU man using man samtools view.  Below we have a brief summary of syntax and sub-command description.
Options common to all sub-commands are documented below in the GLOBAL COMMAND OPTIONS section.
view      samtools view [options] in.sam|in.bam|in.cram [region...]
With no options or regions specified, prints all alignments in the specified input alignment file (in SAM, BAM, or CRAM format) to standard output in SAM format (with no header by default).
You may specify one or more space-separated region specifications after the input filename to restrict output to only those alignments which overlap the specified region(s). Use of region specifications requires a coordinate-sorted and indexed input file.
Options exist to change the output format from SAM to BAM or CRAM, so this command also acts as a file format conversion utility.
tview     samtools tview [-p chr:pos] [-s STR] [-d display] <in.sorted.bam> [ref.fasta]
Text alignment viewer (based on the ncurses library). In the viewer, press `?' for help and press `g' to check the alignment start from a region in the format like `chr10:10,000,000' or `=10,000,000' when viewing the same reference sequence.
quickcheck samtools quickcheck [options] in.sam|in.bam|in.cram [ ... ]
Quickly check that input files appear to be intact. Checks that beginning of the file contains a valid header (all formats) containing at least one target sequence and then seeks to the end of the file and checks that an end-of-file (EOF) is present and intact (BAM only).
Data in the middle of the file is not read since that would be much more time consuming, so please note that this command will not detect internal corruption, but is useful for testing that files are not truncated before performing more intensive tasks on them.
This command will exit with a non-zero exit code if any input files don't have a valid header or are missing an EOF block. Otherwise it will exit successfully (with a zero exit code).
index     samtools index [-bc] [-m INT] aln.sam.gz|aln.bam|aln.cram [out.index]
Index a coordinate-sorted SAM, BAM or CRAM file for fast random access.  Note for SAM this only works if the file has been BGZF compressed first.
This index is needed when region arguments are used to limit samtools view and similar commands to particular regions of interest.
If an output filename is given, the index file will be written to out.index.  Otherwise, for a CRAM file aln.cram, index file aln.cram.crai will be created; for a BAM or SAM file aln.bam, either aln.bam.bai or aln.bam.csi will be created, depending on the index format selected.
sort      samtools sort [-l level] [-m maxMem] [-o out.bam] [-O format] [-n] [-t tag] [-T tmpprefix] [-@ threads] [in.sam|in.bam|in.cram]
Sort alignments by leftmost coordinates, or by read name when -n is used.  An appropriate @HD-SO sort order header tag will be added or an existing one updated if necessary.
The sorted output is written to standard output by default, or to the specified file (out.bam) when -o is used.  This command will also create temporary files tmpprefix.%d.bam as needed when the entire alignment data cannot fit into memory (as controlled via the -m option).
Consider using samtools collate instead if you need name collated data without a full lexicographical sort.
collate   samtools collate [options] in.sam|in.bam|in.cram [<prefix>]
Shuffles and groups reads together by their names.  A faster alternative to a full query name sort, collate ensures that reads of the same name are grouped together in contiguous groups, but doesn't make any guarantees about the order of read names between groups.
The output from this command should be suitable for any operation that requires all reads from the same template to be grouped together.
idxstats  samtools idxstats in.sam|in.bam|in.cram
Retrieve and print stats in the index file corresponding to the input file.  Before calling idxstats, the input BAM file should be indexed by samtools index.
If run on a SAM or CRAM file or an unindexed BAM file, this command will still produce the same summary statistics, but does so by reading through the entire file.  This is far slower than using the BAM indices.
The output is TAB-delimited with each line consisting of reference sequence name, sequence length, # mapped reads and # unmapped reads. It is written to stdout.
flagstat  samtools flagstat in.sam|in.bam|in.cram
Does a full pass through the input file to calculate and print statistics to stdout.
Provides counts for each of 13 categories based primarily on bit flags in the FLAG field. Each category in the output is broken down into QC pass and QC fail, which is presented as "#PASS + #FAIL" followed by a description of the category.
flags     samtools flags INT|STR[,...]
Convert between textual and numeric flag representation.
FLAGS:
0x1   PAIRED          paired-end (or multiple-segment) sequencing technology 0x2   PROPER_PAIR     each segment properly aligned according to the aligner 0x4   UNMAP           segment unmapped 0x8   MUNMAP          next segment in the template unmapped 0x10   REVERSE         SEQ is reverse complemented 0x20   MREVERSE        SEQ of the next segment in the template is reverse complemented 0x40   READ1           the first segment in the template 0x80   READ2           the last segment in the template 0x100   SECONDARY       secondary alignment 0x200   QCFAIL          not passing quality controls 0x400   DUP             PCR or optical duplicate 0x800   SUPPLEMENTARY   supplementary alignment
stats     samtools stats [options] in.sam|in.bam|in.cram [region...]
samtools stats collects statistics from BAM files and outputs in a text format.  The output can be visualized graphically using plot-bamstats.
bedcov    samtools bedcov [options] region.bed in1.sam|in1.bam|in1.cram[...]
Reports the total read base count (i.e. the sum of per base read depths) for each genomic region specified in the supplied BED file. The regions are output as they appear in the BED file and are 0-based.  Counts for each alignment file supplied are reported in separate columns.
depth     samtools depth [options] [in1.sam|in1.bam|in1.cram [in2.sam|in2.bam|in2.cram] [...]]
Computes the read depth at each position or region.
ampliconstats samtools ampliconstats [options] primers.bed in.sam|in.bam|in.cram[...]
samtools ampliconstats collects statistics from one or more input alignment files and produces tables in text format.  The output can be visualized graphically using plot-ampliconstats.
The alignment files should have previously been clipped of primer sequence, for example by samtools ampliconclip and the sites of these primers should be specified as a bed file in the arguments.
mpileup   samtools mpileup [-EB] [-C capQcoef] [-r reg] [-f in.fa] [-l list] [-Q minBaseQ] [-q minMapQ] in.bam [in2.bam [...]]
Generate textual pileup for one or multiple BAM files.  For VCF and BCF output, please use the bcftools mpileup command instead.  Alignment records are grouped by sample (SM) identifiers in @RG header lines.  If sample identifiers are absent, each input file is regarded as one sample.
See the samtools-mpileup man page for a description of the pileup format and options.
coverage  samtools coverage [options] [in1.sam|in1.bam|in1.cram [in2.sam|in2.bam|in2.cram] [...]]
Produces a histogram or table of coverage per chromosome.
merge     samtools merge [-nur1f] [-h inh.sam] [-t tag] [-R reg] [-b list] out.bam in1.bam [in2.bam in3.bam ... inN.bam]
Merge multiple sorted alignment files, producing a single sorted output file that contains all the input records and maintains the existing sort order.
If -h is specified the @SQ headers of input files will be merged into the specified header, otherwise they will be merged into a composite header created from the input headers.  If the @SQ headers differ in order this may require the output file to be re-sorted after merge.
The ordering of the records in the input files must match the usage of the -n and -t command-line options.  If they do not, the output order will be undefined.  See sort for information about record ordering.
split     samtools split [options] merged.sam|merged.bam|merged.cram
Splits a file by read group, producing one or more output files matching a common prefix (by default based on the input filename) each containing one read-group.
cat       samtools cat [-b list] [-h header.sam] [-o out.bam] in1.bam in2.bam [ ... ]
Concatenate BAMs or CRAMs. Although this works on either BAM or CRAM, all input files must be the same format as each other. The sequence dictionary of each input file must be identical, although this command does not check this. This command uses a similar trick to reheader which enables fast BAM concatenation.
import    samtools import [options] in.fastq [ ... ]
Converts one or more FASTQ files to unaligned SAM, BAM or CRAM. These formats offer a richer capability of tracking sample meta-data via the SAM header and per-read meta-data via the auxiliary tags.  The fastq command may be used to reverse this conversion.
fastq/a   samtools fastq [options] in.bam samtools fasta [options] in.bam
Converts a BAM or CRAM into either FASTQ or FASTA format depending on the command invoked. The files will be automatically compressed if the file names have a .gz or .bgzf extension.
The input to this program must be collated by name.  Use samtools collate or samtools sort -n to ensure this.
faidx     samtools faidx <ref.fasta> [region1 [...]]
Index reference sequence in the FASTA format or extract subsequence from indexed reference sequence. If no region is specified, faidx will index the file and create <ref.fasta>.fai on the disk. If regions are specified, the subsequences will be retrieved and printed to stdout in the FASTA format.
The input file can be compressed in the BGZF format.
FASTQ files can be read and indexed by this command.  Without using --fastq any extracted subsequence will be in FASTA format.
fqidx     samtools fqidx <ref.fastq> [region1 [...]]
Index reference sequence in the FASTQ format or extract subsequence from indexed reference sequence. If no region is specified, fqidx will index the file and create <ref.fastq>.fai on the disk. If regions are specified, the subsequences will be retrieved and printed to stdout in the FASTQ format.
samtools fqidx should only be used on fastq files with a small number of entries.  Trying to use it on a file containing millions of short sequencing reads will produce an index that is almost as big as the original file, and searches using the index will be very slow and use a lot of memory.
dict      samtools dict ref.fasta|ref.fasta.gz
Create a sequence dictionary file from a fasta file.
calmd     samtools calmd [-Eeubr] [-C capQcoef] aln.bam ref.fasta
Generate the MD tag. If the MD tag is already present, this command will give a warning if the MD tag generated is different from the existing tag. Output SAM by default.
Calmd can also read and write CRAM files although in most cases it is pointless as CRAM recalculates MD and NM tags on the fly. The one exception to this case is where both input and output CRAM files have been / are being created with the no_ref option.
fixmate   samtools fixmate [-rpcm] [-O format] in.nameSrt.bam out.bam
Fill in mate coordinates, ISIZE and mate related flags from a name-sorted alignment.
markdup   samtools markdup [-l length] [-r] [-s] [-T] [-S] in.algsort.bam out.bam
Mark duplicate alignments from a coordinate sorted file that has been run through samtools fixmate with the -m option.  This program relies on the MC and ms tags that fixmate provides.
rmdup     samtools rmdup [-sS] <input.srt.bam> <out.bam>
This command is obsolete. Use markdup instead.
addreplacerg samtools addreplacerg [-r rg-line | -R rg-ID] [-m mode] [-l level] [-o out.bam] in.bam
Adds or replaces read group tags in a file.
reheader  samtools reheader [-iP] in.header.sam in.bam
Replace the header in in.bam with the header in in.header.sam. This command is much faster than replacing the header with a BAM→SAM→BAM conversion.
By default this command outputs the BAM or CRAM file to standard output (stdout), but for CRAM format files it has the option to perform an in-place edit, both reading and writing to the same file.  No validity checking is performed on the header, nor that it is suitable to use with the sequence data itself.
targetcut samtools targetcut [-Q minBaseQ] [-i inPenalty] [-0 em0] [-1 em1] [-2 em2] [-f ref] in.bam
This command identifies target regions by examining the continuity of read depth, computes haploid consensus sequences of targets and outputs a SAM with each sequence corresponding to a target. When option -f is in use, BAQ will be applied. This command is only designed for cutting fosmid clones from fosmid pool sequencing [Ref. Kitzman et al. (2010)].
phase     samtools phase [-AF] [-k len] [-b prefix] [-q minLOD] [-Q minBaseQ] in.bam
Call and phase heterozygous SNPs.
depad     samtools depad [-SsCu1] [-T ref.fa] [-o output] in.bam
Converts a BAM aligned against a padded reference to a BAM aligned against the depadded reference.  The padded reference may contain verbatim "*" bases in it, but "*" bases are also counted in the reference numbering.  This means that a sequence base-call aligned against a reference "*" is considered to be a cigar match ("M" or "X") operator (if the base-call is "A", "C", "G" or "T").  After depadding the reference "*" bases are deleted and such aligned sequence base-calls become insertions. Similarly transformations apply for deletions and padding cigar operations.
ampliconclip samtools ampliconclip [-o out.file] [-f stat.file] [--soft- clip] [--hard-clip] [--both-ends] [--strand] [--clipped] [--fail] [--no-PG] -b bed.file in.file
Clip reads in a SAM compatible file based on data from a BED file.
These are options that are passed after the samtools command, before any sub-command is specified.
help, --help Display a brief usage message listing the samtools commands available.  If the name of a command is also given, e.g., samtools help view, the detailed usage message for that particular command is displayed.
--version Display the version numbers and copyright information for samtools and the important libraries used by samtools.
--version-only Display the full samtools version number in a machine-readable format.
Several long-options are shared between multiple samtools sub-commands: --input-fmt, --input-fmt-option, --output-fmt, --output-fmt-option, --reference, --write-index, and --verbosity.  The input format is typically auto-detected so specifying the format is usually unnecessary and the option is included for completeness.  Note that not all subcommands have all options.  Consult the subcommand help for more details.
Format strings recognised are "sam", "sam.gz", "bam" and "cram".  They may be followed by a comma separated list of options as key or key=value. See below for examples.
The fmt-option arguments accept either a single option or option=value. Note that some options only work on some file formats and only on read or write streams.  If value is unspecified for a boolean option, the value is assumed to be 1.  The valid options are as follows.
level=INT Output only. Specifies the compression level from 1 to 9, or 0 for uncompressed.  If the output format is SAM, this also enables BGZF compression, otherwise SAM defaults to uncompressed.
nthreads=INT Specifies the number of threads to use during encoding and/or decoding.  For BAM this will be encoding only.  In CRAM the threads are dynamically shared between encoder and decoder.
filter=STRING Apply filter STRING to all incoming records, rejecting any that do not satisfy the expression.  See the FILTER EXPRESSIONS section below for specifics.
reference=fasta_file Specifies a FASTA reference file for use in CRAM encoding or decoding.  It usually is not required for decoding except in the situation of the MD5 not being obtainable via the REF_PATH or REF_CACHE environment variables.
decode_md=0|1 CRAM input only; defaults to 1 (on).  CRAM does not typically store MD and NM tags, preferring to generate them on the fly.  When this option is 0 missing MD, NM tags will not be generated.  It can be particularly useful when combined with a file encoded using store_md=1 and store_nm=1.
store_md=0|1 CRAM output only; defaults to 0 (off).  CRAM normally only stores MD tags when the reference is unknown and lets the decoder generate these values on-the-fly (see decode_md).
store_nm=0|1 CRAM output only; defaults to 0 (off).  CRAM normally only stores NM tags when the reference is unknown and lets the decoder generate these values on-the-fly (see decode_md).
ignore_md5=0|1 CRAM input only; defaults to 0 (off).  When enabled, md5 checksum errors on the reference sequence and block checksum errors within CRAM are ignored.  Use of this option is strongly discouraged.
required_fields=bit-field CRAM input only; specifies which SAM columns need to be populated. By default all fields are used.  Limiting the decode to specific columns can have significant performance gains.  The bit-field is a numerical value constructed from the following table.
0x1   SAM_QNAME 0x2   SAM_FLAG 0x4   SAM_RNAME 0x8   SAM_POS 0x10   SAM_MAPQ 0x20   SAM_CIGAR 0x40   SAM_RNEXT 0x80   SAM_PNEXT 0x100   SAM_TLEN 0x200   SAM_SEQ 0x400   SAM_QUAL 0x800   SAM_AUX 0x1000   SAM_RGAUX
name_prefix=string CRAM input only; defaults to output filename.  Any sequences with auto-generated read names will use string as the name prefix.
multi_seq_per_slice=0|1 CRAM output only; defaults to 0 (off).  By default CRAM generates one container per reference sequence, except in the case of many small references (such as a fragmented assembly).
version=major.minor CRAM output only.  Specifies the CRAM version number.  Acceptable values are "2.1" and "3.0".
seqs_per_slice=INT CRAM output only; defaults to 10000.
slices_per_container=INT CRAM output only; defaults to 1.  The effect of having multiple slices per container is to share the compression header block between multiple slices.  This is unlikely to have any significant impact unless the number of sequences per slice is reduced.  (Together these two options control the granularity of random access.)
embed_ref=0|1 CRAM output only; defaults to 0 (off).  If 1, this will store portions of the reference sequence in each slice, permitting decode without having requiring an external copy of the reference sequence.
no_ref=0|1 CRAM output only; defaults to 0 (off).  If 1, sequences will be stored verbatim with no reference encoding.  This can be useful if no reference is available for the file.
use_bzip2=0|1 CRAM output only; defaults to 0 (off).  Permits use of bzip2 in CRAM block compression.
use_lzma=0|1 CRAM output only; defaults to 0 (off).  Permits use of lzma in CRAM block compression.
lossy_names=0|1 CRAM output only; defaults to 0 (off).  If 1, templates with all members within the same CRAM slice will have their read names removed.  New names will be automatically generated during decoding. Also see the name_prefix option.
samtools view --input-fmt-option decode_md=0 --output-fmt cram,version=3.0 --output-fmt-option embed_ref --output-fmt-option seqs_per_slice=2000 -o foo.cram foo.bam
The --write-index option enables automatic index creation while writing out BAM, CRAM or bgzf SAM files.  Note to get compressed SAM as the output format you need to manually request a compression level, otherwise all SAM files are uncompressed.  By default SAM and BAM will use CSI indices while CRAM will use CRAI indices.  If you need to create BAI indices note that it is possible to specify the name of the index being written to, and hence the format, by using the filename##idx##indexname notation.
For example: to convert a BAM to a compressed SAM with CSI indexing:
samtools view -h -O sam,level=6 --write-index in.bam -o out.sam.gz
To convert a SAM to a compressed BAM using BAI indexing:
samtools view --write-index in.sam -o out.bam##idx##out.bam.bai
The --verbosity INT option sets the verbosity level for samtools and HTSlib.  The default is 3 (HTS_LOG_WARNING); 2 reduces warning messages and 0 or 1 also reduces some error messages, while values greater than 3 produce increasing numbers of additional warnings and logging messages.
The CRAM format requires use of a reference sequence for both reading and writing.
When reading a CRAM the @SQ headers are interrogated to identify the reference sequence MD5sum (M5: tag) and the local reference sequence filename (UR: tag).  Note that http:// and ftp:// based URLs in the UR: field are not used, but local fasta filenames (with or without file://) can be used.
To create a CRAM the @SQ headers will also be read to identify the reference sequences, but M5: and UR: tags may not be present. In this case the -T and -t options of samtools view may be used to specify the fasta or fasta.fai filenames respectively (provided the .fasta.fai file is also backed up by a .fasta file).
The search order to obtain a reference is:
1. Use any local file specified by the command line options (eg -T).
2. Look for MD5 via REF_CACHE environment variable.
3. Look for MD5 in each element of the REF_PATH environment variable.
4. Look for a local file listed in the UR: header tag.
Filter expressions are used as an on-the-fly checking of incoming SAM, BAM or CRAM records, discarding records that do not match the specified expression.
The language used is primarily C style, but with a few differences in the precedence rules for bit operators and the inclusion of regular expression matching.
The operator precedence, from strongest binding to weakest, is:
Grouping        (, )             E.g. "(1+2)*3" Values:         literals, vars   Numbers, strings and variables Unary ops:      +, -, !, ~       E.g. -10 +10, !10 (not), ~5 (bit not) Math ops:       *, /, %          Multiply, division and (integer) modulo Math ops:       +, -             Addition / subtraction Bit-wise:       &                Integer AND Bit-wise        ^                Integer XOR Bit-wise        |                Integer OR Conditionals:   >, >=, <, <= Equality:       ==, !=, =~, !~   =~ and !~ match regular expressions Boolean:        &&, ||           Logical AND / OR
Expressions are computed using floating point mathematics, so "10 / 4" evaluates to 2.5 rather than 2.  They may be written as integers in decimal or "0x" plus hexadecimal, and floating point with or without exponents.However operations that require integers first do an implicit type conversion, so "7.9 % 5" is 2 and "7.9 & 4.1" is equivalent to "7 & 4", which is 4.  Strings are always specified using double quotes.  To get a double quote in a string, use backslash.  Similarly a double backslash is used to get a literal backslash.  For example ab\"c\\d is the string ab"c\d.
Comparison operators are evaluated as a match being 1 and a mismatch being 0, thus "(2 > 1) + (3 < 5)" evaluates as 2.
The variables are where the file format specifics are accessed from the expression.  The variables correspond to SAM fields, for example to find paired alignments with high mapping quality and a very large insert size, we may use the expression "mapq >= 30 && (tlen >= 100000 || tlen <= -100000)".  Valid variable names and their data types are:
flag                 int            Combined FLAG field flag.paired          int            Single bit, 0 or 1 flag.proper_pair     int            Single bit, 0 or 2 flag.unmap           int            Single bit, 0 or 4 flag.munmap          int            Single bit, 0 or 8 flag.reverse         int            Single bit, 0 or 16 flag.mreverse        int            Single bit, 0 or 32 flag.read1           int            Single bit, 0 or 64 flag.read2           int            Single bit, 0 or 128 flag.secondary       int            Single bit, 0 or 256 flag.qcfail          int            Single bit, 0 or 512 flag.dup             int            Single bit, 0 or 1024 flag.supplementary   int            Single bit, 0 or 2048 library              string         Library (LB header via RG) mapq                 int            Mapping quality mpos                 int            Synonym for pnext mrefid               int            Mate reference number (0 based) mrname               string         Synonym for rnext ncigar               int            Number of cigar operations pnext                int            Mate's alignment position (1-based) pos                  int            Alignment position (1-based) qlen                 int            Alignment length: no. query bases qname                string         Query name qual                 string         Quality values (raw, 0 based) refid                int            Integer reference number (0 based) rlen                 int            Alignment length: no. reference bases rname                string         Reference name rnext                string         Mate's reference name seq                  string         Sequence tlen                 int            Template length (insert size) [XX]                 int / string   XX tag value
Flags are returned either as the whole flag value or by checking for a single bit.  Hence the filter expression flag.dup is equivalent to flag & 1024.
"qlen" and "rlen" are measured using the CIGAR string to count the number of query (sequence) and reference bases consumed.  Note "qlen" may not exactly match the length of the "seq" field if the sequence is "*".
Reference names may be matched either by their string forms ("rname" and "mrname") or as the Nth @SQ line (counting from zero) as stored in BAM using "tid" and "mtid" respectively.
Auxiliary tags are described in square brackets and these expand to either integer or string as defined by the tag itself (XX:Z:string or XX:i:int).  For example [NM]>=10 can be used to look for alignments with many mismatches and [RG]=~"grp[ABC]-" will match the read-group string.
If no comparison is used with an auxiliary tag it is taken simply to be a test for the existence of that tag.  So "[NM]" will return any record containing an NM tag, even if that tag is zero (NM:i:0).
If you need to check specifically for a non-zero value then use [NM] && [NM]!=0.
Some simple functions are available to operate on strings.  These treat the strings as arrays of bytes, permitting their length, minimum, maximum and average values to be computed.
length   Length of the string (excluding nul char) min      Minimum byte value in the string max      Maximum byte value in the string avg      Average byte value in the string
Note that "avg" is a floating point value and it may be NAN for empty strings.  This means that "avg(qual)" does not produce an error for records that have both seq and qual of "*".  This value will fail any conditional checks, so e.g. "avg(qual) > 20" works and will not report these records.
HTS_PATH A colon-separated list of directories in which to search for HTSlib plugins.  If $HTS_PATH starts or ends with a colon or contains a double colon (::), the built-in list of directories is searched at that point in the search.