Archiving and compression

The traditional Unix archiving and compression tools are separated according to the
Unix philosophy
:
A
file archiver
combines several files into one archive file, e.g.
tar
.
A
compression
tool compresses and decompresses data, e.g.
gzip
.
These tools are often used in sequence by firstly creating an archive file and then compressing it.
Of course there are also
tools that do both
, which tend to additionally offer encryption, error detection and recovery.
Archiving only
Name
Package
Manuals
Description
GNU
tar
tar
tar(1)
,
info
Core utility
for manipulating the ubiquitous tar archives (tarballs).
libarchive
libarchive
bsdtar(1)
bsdcpio(1)
Implementation of
tar
and
cpio
that also offers a library. Used by
pacman
and
mkinitcpio
.
ar
binutils
ar(1)
Legacy Unix archiver before
tar
. Today only used for creating
static library
files.
GNU
cpio
cpio
cpio(1)
.
info
File archiver via stdin/stdout, supports cpio and tar formats.
Tip
Both GNU and BSD tar automatically do decompression delegation for bzip2, compress, gzip, lzip, lzma, lzop, zstd, and xz compressed archives. Only BSD tar supports lz4 natively (but GNU tar can do an equivalent with
--use-compress-program=lz4
/
-Ilz4
). When creating archives both support the
-a
switch to automatically filter the created archive through the right compression program based on the file extension. While BSD tar recognizes compression formats based on the format, GNU tar only guesses based on the file extension.
See also
#Archiving only usage
.
Compression tools
Compression only
These compression programs implement their own file format.
Name
Package
Manual
Ext
Tar ext
Description
Parallel implementations
bzip2
bzip2
bzip2(1)
.bz2, .bz
.tbz2, .tbz
Uses the
Burrows–Wheeler algorithm
.
lbzip2
,
pbzip2
bzip3
bzip3
bzip3(1)
.bz3
.tbz3
Uses the
Burrows–Wheeler algorithm
.
gzip
gzip
gzip(1)
.gz, .z
.tgz, .taz
GNU zip, based on
DEFLATE
algorithm.
pigz
,
bgzip(1)
(part of
htslib
AUR
),
crabz
AUR
,
python-rapidgzip
AUR
lrzip
lrzip
lrzip(1)
.lrz
Improved version of
rzip
, uses multiple algorithms.
Is multithreaded by default
LZ4
lz4
lz4(1)
.lz4
Written in C, focused on compression and decompression speed.
Is multithreaded by default. See
https://lz4.org/
for alternatives.
lzip
lzip
lzip(1)
.lz
Uses
LZMA
.
plzip
AUR
lzop
lzop
lzop(1)
.lzo
.tzo
Uses the
LZO
library (
lzo
).
xz
xz
xz(1)
.xz, .lzma
.txz, .tlz
Uses
LZMA2
. Default for GNU
coreutils
and kernel archive files.
Is multithreaded by default. An alternative is
pixz
.
zstd
zstd
zstd(1)
.zst
Uses
Zstandard
algorithm.
is multithreaded
Parallel implementations offer improved speeds by using multiple CPU cores.
Tar extensions refers to compressed archives where
tar
and the compression tool is used, e.g.
.tzo
is
.tar.lzo
.
See also
#Compression only usage
.
Archiving and compression
Name
Packages
Manuals
Ext
Description
7-Zip
7zip
Official manual
.7z
A file archiver with a high compression ratio.
DAR
dar
AUR
dar(1)
.dar
Archiver to backup large live filesystems, takes care of hard links,
extended attributes
, sparse files and inode types.
GNU tar
tar
tar(1)
,
info
.tar.
compression-type
Supports passing the archive for compression to
bzip2(1)
,
xz(1)
,
lzip(1)
,
lzma(1)
,
lzop(1)
,
gzip(1)
,
compress(1)
,
zstd(1)
, as well as any other custom compression program. See
tar(1) § Compression options
.
libarchive
libarchive
bsdtar(1)
.tar.
compression-type
Has built-in compression options for bzip2, gzip, lrzip, lzma, lzop, lz4, zstd and xz.
RAR
rar
AUR
,
unrar
rar(1)
.rar
Both the format and the
rar
utility are proprietary.
t2sz
t2sz
AUR
.tar.zst .tzst
Tar archiving utility in C with member-aligned zstd-compression
tarlz
tarlz
AUR
tarlz(1)
.tar.lz .tlz
Tar archiving utility in C++ with member-aligned lzip compression
ZIP
zip
,
unzip
zip(1)
,
unzip(1)
.zip
Widely used outside of the Linux world.
Unarchiver
unarchiver
unar(1)
,
lsar(1)
many
Command-line tool of a Mac application, supports over 40 archive formats.
ZPAQ
zpaq
AUR
zpaq(1)
.zpaq
A high compression ratio archiver written in C++, uses several algorithms.
LHa
lhasa
,
lha
AUR
lha(1)
.lzh (on Amiga: .lha)
LZH/LHA archiver, supports the lh7-method.
WinAce
unace
unace(1)
.ace
Both the
ACE
file format and the archiving tool are proprietary.
See also
#Archiving and compression usage
.
Feature charts
Some of the tools above are capable of handling multiple formats, allowing for fewer installed packages.
Decompress
Name
File
Archive
gzip
bzip2
LZMA
xz
zstd
ZIP
RAR
7z
CAB
gzip
Yes
No
No
No
No
Partial
1
No
No
No
7zip
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
unarchiver
Yes
Yes
Yes
Yes
No
Yes
Yes
Yes
Yes
zstd
Yes
No
Yes
Yes
Yes
No
No
No
No
gunzip(1)
can only decompress single member ZIP files.
Usage comparison
Archiving only usage
Name
Create archive
Extract archive
List content
tar(1)
tar cfv archive.tar file1 file2
tar xfv archive.tar
tar -tvf archive.tar
cpio(1)
ls file1 file2 | cpio -o > archive.cpio
cpio -i -vd < archive.cpio
cpio -t < archive.cpio
Compression only usage
Name
Compress
Decompress
Decompress to stdout
bzip2(1)
bzip2 file
bzip2 -d file.bz2
bzcat file.bz2
gzip(1)
gzip file
gzip -d file.gz
zcat file.gz
lrzip(1)
lrzip file
lrztar folder
lrzip -d file.lrz
lrztar -d folder.tar.lrz
lrzcat file.lrz
lz4(1)
lz4 file
lz4 file.lz4
lz4cat file.lz4
xz(1)
xz file
xz -d file.xz
xzcat file.xz
zstd(1)
zstd file
zstd -d file.zst
zstdcat file.zst
Archiving and compression usage
Name
Compress
Decompress
Decompress to stdout
List content
7z
7z a archive.7z file1 file2
7z x archive.7z
7z e -so archive.7z file1
7z l archive.7z
dar(1)
dar -c archive -g file1 -g file2
dar -x archive
dar -l archive
tar(1)
tar acvf archive.
format
file1 file2
tar xfv archive.
format
tar xfvO archive.
format
tar tvf archive.
format
rar(1)
rar a archive.rar file1 file2
rar x archive.rar
rar p -inul archive.rar file1
rar l archive.rar
zip(1)
,
unzip(1)
zip archive.zip file1 file2
unzip archive.zip
unzip -p archive.zip file1
unzip -l archive.zip
lha(1)
lha ao7 archive.lzh file1 file2
lha x archive.lzh
minimal:
lha l archive.lzh
verbose:
lha v archive.lzh
Convenience tools
See also
Bash/Functions#Extract
.
atool
— Script for managing file archives of various types.
https://www.nongnu.org/atool/
||
atool
dtrx
— An intelligent archive extraction tool.
https://github.com/brettcs/dtrx
||
dtrx
AUR
J7Z
— GUI for Linux in java which attempts to simplify data compression and backup. It can create 7z, BZip2, Zip, GZip, Tar archives.
https://sites.google.com/site/xavionsj7z
||
j7z
AUR
ouch
— A command line utility for easily compressing and decompressing files and directories.
https://github.com/ouch-org/ouch
||
ouch
patool
— Allows various archive types to be created, extracted, tested, listed, compared, searched and repacked.
https://wummel.github.io/patool/
||
patool
AUR
python-unp
— Command line tool that can unpack archives easily.
https://github.com/mitsuhiko/unp
||
python-unp
AUR
unp
— A script for unpacking a wide variety of archive formats.
https://tracker.debian.org/pkg/unp
||
unp
unpack
— Wrapper script for handling multiple archive formats.
https://github.com/githaff/unpack
||
unpack-git
AUR
Determining archive format
To extract an archive, its file format needs to be determined. If the file is properly named you can deduce its format from the file extension.
Otherwise you can use the
file
tool, see
file(1)
.
Esoteric, rare or deprecated tools
Name
Packages
Ext
Description
ARC
arc
AUR
.arc, .ark
Was very popular during the early days of the dial-up BBS. Superseded by ZIP.
ARJ
arj
.arj
An archiver used on DOS/Windows in mid-1990s. This is an open source clone.
Cabinet
cabextract
,
unshield
.cab, .exe
A variety of installation technologies in Windows use the CAB format.
compress
ncompress
.Z
The de facto standard UNIX compression utility to success the Huffman-based
pack(1)
before gzip become a thing.
Inno Setup
innoextract
.exe
Installers created by Inno Setup.
PAR2
par2cmdline
.par2
Parity archiver for increased data integrity. See also
Parchive
.
shar
sharutils
.shar
Creates self-extracting archives that are valid shell scripts.
Zoo
zoo
AUR
.zoo
Was mostly popular on the
OpenVMS
operating system before PKZIP became popular.
File system compression
Some
file systems
support on-the-fly compression of file data:
Btrfs
can be configured to compress individual files, directories, or entire volumes by default.
On
ZFS
, compression can be enabled on pools or file systems.
Device mapper compression
The open-sourced VDO project was integrated into the Linux kernel project, which provides a deduplication and compression device mapper layer in the interest of increasing storage efficiency. A userspace tools for managing VDO volumes is available in the AUR:
vdo
AUR
Compression libraries
Brotli
— Compression algorithm for data streams using the LZ77 algorithm, Huffman coding and 2nd order context modeling.
https://github.com/google/brotli
||
brotli
libzip
— Provides creation and extraction of ZIP files. Used by KDE and Deepin in place of the zip/unzip tools.
https://libzip.org
||
libzip
zlib
— Compression library implementing the deflate compression method found in gzip and PKZIP.
https://www.zlib.net/
||
zlib
Zopfli
— High compress ratio file compressor from Google, using a deflate-compatible algorithm called zopfli. Also available as compression level 11 in
pigz
.
https://github.com/google/zopfli
||
zopfli-git
AUR
Troubleshooting
Garbled file names when extracted
See
Character encoding#Troubleshooting
.
See also
List of applications/Utilities#Archiving and compression tools
List of applications/Multimedia#Image compression
Wikipedia:Comparison of file archivers
Wikipedia:List of archive formats
Wikipedia:Comparison of archive formats