neulab/tldr huggingface dataset:
The X perm symbol was adopted from BSD-based systems because it provides commonly desired functionality when doing recursive (-R option) modifications. Similar functionality is not provided by the find utility. Historical BSD versions of chmod, however, only supported X with op+; it has been extended in this volume of POSIX.1‐2017 because it is also useful with op=.  (It has also been added for op- even though it duplicates x, in this case, because it is intuitive and easier to explain.)
The grammar was extended with the permcopy non-terminal to allow historical-practice forms of symbolic modes like o=u -g (that is, set the ``other'' permissions to the permissions of ``owner'' minus the permissions of ``group'').
ls, umask
The Base Definitions volume of POSIX.1‐2017, Section 4.5, File Access Permissions, Chapter 8, Environment Variables, Section 12.2, Utility Syntax Guidelines
The System Interfaces volume of POSIX.1‐2017, chmod()
BLKDISCARD(8)                 System Administration                BLKDISCARD(8)
blkdiscard - discard sectors on a device
blkdiscard [options] [-o offset] [-l length] device
blkdiscard is used to discard device sectors. This is useful for solid-state drivers (SSDs) and thinly-provisioned storage. Unlike fstrim(8), this command is used directly on the block device.
By default, blkdiscard will discard all blocks on the device. Options may be used to modify this behavior based on range or size, as explained below.
The device argument is the pathname of the block device.
WARNING: All data in the discarded region on the device will be lost!
The offset and length arguments may be followed by the multiplicative suffixes KiB (=1024), MiB (=1024*1024), and so on for GiB, TiB, PiB, EiB, ZiB and YiB (the "iB" is optional, e.g., "K" has the same meaning as "KiB") or the suffixes KB (=1000), MB (=1000*1000), and so on for GB, TB, PB, EB, ZB and YB.
-f, --force Disable all checking. Since v2.36 the block device is open in exclusive mode (O_EXCL) by default to avoid collision with mounted filesystem or another kernel subsystem. The --force option disables the exclusive access mode.
-o, --offset offset Byte offset into the device from which to start discarding. The provided value must be aligned to the device sector size. The default value is zero.
-l, --length length The number of bytes to discard (counting from the starting point). The provided value must be aligned to the device sector size. If the specified value extends past the end of the device, blkdiscard will stop at the device size boundary. The default value extends to the end of the device.
-p, --step length The number of bytes to discard within one iteration. The default is to discard all by one ioctl call.
-s, --secure Perform a secure discard. A secure discard is the same as a regular discard except that all copies of the discarded blocks that were possibly created by garbage collection must also be erased. This requires support from the device.
-z, --zeroout Zero-fill rather than discard.
-v, --verbose Display the aligned values of offset and length. If the --step option is specified, it prints the discard progress every second.
Lukas Czerner <lczerner@redhat.com>
fstrim(8)
The blkdiscard command is part of the util-linux package which can be downloaded from Linux Kernel Archive <https://www.kernel.org/pub/linux/utils/util-linux/>.
SUPERVISORD(1)                     Supervisor                     SUPERVISORD(1)
supervisord - supervisord Documentation
supervisord The server piece of supervisor is named supervisord.  It is responsible for starting child programs at its own invocation, responding to commands from clients, restarting crashed or exited subprocesseses, logging its subprocess stdout and stderr output, and generating and handling "events" corresponding to points in subprocess lifetimes.
The server process uses a configuration file.  This is typically located in /etc/supervisord.conf.  This configuration file is a "Windows-INI" style config file.  It is important to keep this file secure via proper filesystem permissions because it may contain unencrypted usernames and passwords.
Adding a Program Before supervisord will do anything useful for you, you'll need to add at least one program section to its configuration.  The program section will define a program that is run and managed when you invoke the supervisord command.  To add a program, you'll need to edit the supervisord.conf file.
One of the simplest possible programs to run is the UNIX cat program.  A program section that will run cat when the supervisord process starts up is shown below.
[program:foo] command=/bin/cat
This stanza may be cut and pasted into the supervisord.conf file.  This is the simplest possible program configuration, because it only names a command.  Program configuration sections have many other configuration options which aren't shown here.  See programx_section for more information.
Running supervisord To start supervisord, run $BINDIR/supervisord.  The resulting process will daemonize itself and detach from the terminal.  It keeps an operations log at $CWD/supervisor.log by default.
You may start the supervisord executable in the foreground by passing the -n flag on its command line.  This is useful to debug startup problems.
WARNING: When supervisord starts up, it will search for its configuration file in default locations including the current working directory.  If you are security-conscious you will probably want to specify a "-c" argument after the supervisord command specifying an absolute path to a configuration file to ensure that someone doesn't trick you into running supervisor from within a directory that contains a rogue supervisord.conf file.  A warning is emitted when supervisor is started as root without this -c argument.
To change the set of programs controlled by supervisord, edit the supervisord.conf file and kill -HUP or otherwise restart the supervisord process.  This file has several example program definitions.
The supervisord command accepts a number of command-line options.  Each of these command line options overrides any equivalent value in the configuration file.
supervisord Command-Line Options -c FILE, --configuration=FILE The path to a supervisord configuration file.
-n, --nodaemon Run supervisord in the foreground.
-h, --help Show supervisord command help.
-u USER, --user=USER UNIX username or numeric user id.  If supervisord is started as the root user, setuid to this user as soon as possible during startup.
-m OCTAL, --umask=OCTAL Octal number (e.g. 022) representing the umask that should be used by supervisord after it starts.
-d PATH, --directory=PATH When supervisord is run as a daemon, cd to this directory before daemonizing.
-l FILE, --logfile=FILE Filename path to use as the supervisord activity log.
-y BYTES, --logfile_maxbytes=BYTES Max size of the supervisord activity log file before a rotation occurs.  The value is suffix-multiplied, e.g "1" is one byte, "1MB" is 1 megabyte, "1GB" is 1 gigabyte.
-y NUM, --logfile_backups=NUM Number of backup copies of the supervisord activity log to keep around.  Each logfile will be of size logfile_maxbytes.
-e LEVEL, --loglevel=LEVEL The logging level at which supervisor should write to the activity log.  Valid levels are trace, debug, info, warn, error, and critical.
-j FILE, --pidfile=FILE The filename to which supervisord should write its pid file.
-i STRING, --identifier=STRING Arbitrary string identifier exposed by various client UIs for this instance of supervisor.
-q PATH, --childlogdir=PATH A path to a directory (it must already exist) where supervisor will write its AUTO -mode child process logs.
-k, --nocleanup Prevent supervisord from performing cleanup (removal of old AUTO process log files) at startup.
-a NUM, --minfds=NUM The minimum number of file descriptors that must be available to the supervisord process before it will start successfully.
-t, --strip_ansi Strip ANSI escape sequences from all child log process.
-v, --version Print the supervisord version number out to stdout and exit.
--profile_options=LIST Comma-separated options list for profiling.  Causes supervisord to run under a profiler, and output results based on the options, which is a comma-separated list of the following: cumulative, calls, callers.  E.g. cumulative,callers.
--minprocs=NUM The minimum number of OS process slots that must be available to the supervisord process before it will start successfully.
Signals The supervisord program may be sent signals which cause it to perform certain actions while it's running.
You can send any of these signals to the single supervisord process id. This process id can be found in the file represented by the pidfile parameter in the [supervisord] section of the configuration file (by default it's $CWD/supervisord.pid).
Signal Handlers SIGTERM supervisord and all its subprocesses will shut down.  This may take several seconds.
SIGINT supervisord and all its subprocesses will shut down.  This may take several seconds.
SIGQUIT supervisord and all its subprocesses will shut down.  This may take several seconds.
SIGHUP supervisord will stop all processes, reload the configuration from the first config file it finds, and restart all processes.
SIGUSR2 supervisord will close and reopen the main activity log and all child log files.
Runtime Security The developers have done their best to assure that use of a supervisord process running as root cannot lead to unintended privilege escalation. But caveat emptor.  Supervisor is not as paranoid as something like DJ Bernstein's daemontools, inasmuch as supervisord allows for arbitrary path specifications in its configuration file to which data may be written.  Allowing arbitrary path selections can create vulnerabilities from symlink attacks.  Be careful when specifying paths in your configuration.  Ensure that the supervisord configuration file cannot be read from or written to by unprivileged users and that all files installed by the supervisor package have "sane" file permission protection settings.  Additionally, ensure that your PYTHONPATH is sane and that all Python standard library files have adequate file permission protections.
Running supervisord automatically on startup If you are using a distribution-packaged version of Supervisor, it should already be integrated into the service management infrastructure of your distribution.
There are user-contributed scripts for various operating systems at: https://github.com/Supervisor/initscripts
There are some answers at Serverfault in case you get stuck: How to automatically start supervisord on Linux (Ubuntu)
Subprocesses supervisord's primary purpose is to create and manage processes based on data in its configuration file.  It does this by creating subprocesses. Each subprocess spawned by supervisor is managed for the entirety of its lifetime by supervisord (supervisord is the parent process of each process it creates).  When a child dies, supervisor is notified of its death via the SIGCHLD signal, and it performs the appropriate operation.
Nondaemonizing of Subprocesses Programs meant to be run under supervisor should not daemonize themselves.  Instead, they should run in the foreground.  They should not detach from the terminal from which they are started.
The easiest way to tell if a program will run in the foreground is to run the command that invokes the program from a shell prompt.  If it gives you control of the terminal back, but continues running, it's daemonizing itself and that will almost certainly be the wrong way to run it under supervisor.  You want to run a command that essentially requires you to press Ctrl-C to get control of the terminal back.  If it gives you a shell prompt back after running it without needing to press Ctrl-C, it's not useful under supervisor.  All programs have options to be run in the foreground but there's no "standard way" to do it; you'll need to read the documentation for each program.
Below are configuration file examples that are known to start common programs in "foreground" mode under Supervisor.
Examples of Program Configurations Here are some "real world" program configuration examples:
Apache 2.2.6 [program:apache2] command=/path/to/httpd -c "ErrorLog /dev/stdout" -DFOREGROUND redirect_stderr=true
Two Zope 2.X instances and one ZEO server [program:zeo] command=/path/to/runzeo priority=1
[program:zope1] command=/path/to/instance/home/bin/runzope priority=2 redirect_stderr=true
[program:zope2] command=/path/to/another/instance/home/bin/runzope priority=2 redirect_stderr=true
Postgres 8.X [program:postgres] command=/path/to/postmaster ; we use the "fast" shutdown signal SIGINT stopsignal=INT redirect_stderr=true
OpenLDAP slapd [program:slapd] command=/path/to/slapd -f /path/to/slapd.conf -h ldap://0.0.0.0:8888 redirect_stderr=true
Other Examples Other examples of shell scripts that could be used to start services under supervisord can be found at http://thedjbway.b0llix.net/services.html.  These examples are actually for daemontools but the premise is the same for supervisor.
Another collection of recipes for starting various programs in the foreground is available from http://smarden.org/runit/runscripts.html.
pidproxy Program Some processes (like mysqld) ignore signals sent to the actual process which is spawned by supervisord.  Instead, a "special" thread/process is created by these kinds of programs which is responsible for handling signals.  This is problematic because supervisord can only kill a process which it creates itself.  If a process created by supervisord creates its own child processes, supervisord cannot kill them.
Fortunately, these types of programs typically write a "pidfile" which contains the "special" process' PID, and is meant to be read and used in order to kill the process.  As a workaround for this case, a special pidproxy program can handle startup of these kinds of processes.  The pidproxy program is a small shim that starts a process, and upon the receipt of a signal, sends the signal to the pid provided in a pidfile. A sample configuration program entry for a pidproxy-enabled program is provided below.
[program:mysql] command=/path/to/pidproxy /path/to/pidfile /path/to/mysqld_safe
The pidproxy program is put into your configuration's $BINDIR when supervisor is installed (it is a "console script").
Subprocess Environment Subprocesses will inherit the environment of the shell used to start the supervisord program.  Several environment variables will be set by supervisord itself in the child's environment also, including SUPERVISOR_ENABLED (a flag indicating the process is under supervisor control), SUPERVISOR_PROCESS_NAME (the config-file-specified process name for this process) and SUPERVISOR_GROUP_NAME (the config-file-specified process group name for the child process).
These environment variables may be overridden within the [supervisord] section config option named environment (applies to all subprocesses) or within the per- [program:x] section environment config option (applies only to the subprocess specified within the [program:x] section).  These "environment" settings are additive.  In other words, each subprocess' environment will consist of: The environment variables set within the shell used to start supervisord...
... added-to/overridden-by ...
... the environment variables set within the environment global config option ...
... supervisor-specific environment variables (SUPERVISOR_ENABLED, SUPERVISOR_PROCESS_NAME, SUPERVISOR_GROUP_NAME) ..
... the environment variables set within the per-process "environment" config option.
No shell is executed by supervisord when it runs a subprocess, so environment variables such as USER, PATH, HOME, SHELL, LOGNAME, etc. are not changed from their defaults or otherwise reassigned.  This is particularly important to note when you are running a program from a supervisord run as root with a user= stanza in the configuration.  Unlike cron, supervisord does not attempt to divine and override "fundamental" environment variables like USER, PATH, HOME, and LOGNAME when it performs a setuid to the user defined within the user= program config option.  If you need to set environment variables for a particular program that might otherwise be set by a shell invocation for a particular user, you must do it explicitly within the environment= program config option.  An example of setting these environment variables is as below.
[program:apache2] command=/home/chrism/bin/httpd -c "ErrorLog /dev/stdout" -DFOREGROUND user=chrism environment=HOME="/home/chrism",USER="chrism"
Process States A process controlled by supervisord will be in one of the below states at any given time.  You may see these state names in various user interface elements in clients.
STOPPED (0) The process has been stopped due to a stop request or has never been started.
STARTING (10) The process is starting due to a start request.
RUNNING (20) The process is running.
BACKOFF (30) The process entered the STARTING state but subsequently exited too quickly to move to the RUNNING state.
STOPPING (40) The process is stopping due to a stop request.
EXITED (100) The process exited from the RUNNING state (expectedly or unexpectedly).
FATAL (200) The process could not be started successfully.
UNKNOWN (1000) The process is in an unknown state (supervisord programming error).
Each process run under supervisor progresses through these states as per the following directed graph. [image: Subprocess State Transition Graph] [image] Subprocess State Transition Graph.UNINDENT
A process is in the STOPPED state if it has been stopped adminstratively or if it has never been started.
When an autorestarting process is in the BACKOFF state, it will be automatically restarted by supervisord.  It will switch between STARTING and BACKOFF states until it becomes evident that it cannot be started because the number of startretries has exceeded the maximum, at which point it will transition to the FATAL state.  Each start retry will take progressively more time.
When a process is in the EXITED state, it will automatically restart:
• never if its autorestart parameter is set to false.
• unconditionally if its autorestart parameter is set to true.
• conditionally if its autorestart parameter is set to unexpected.  If it exited with an exit code that doesn't match one of the exit codes defined in the exitcodes configuration parameter for the process, it will be restarted.
A process automatically transitions from EXITED to RUNNING as a result of being configured to autorestart conditionally or unconditionally.  The number of transitions between RUNNING and EXITED is not limited in any way: it is possible to create a configuration that endlessly restarts an exited process.  This is a feature, not a bug.
An autorestarted process will never be automatically restarted if it ends up in the FATAL state (it must be manually restarted from this state).
A process transitions into the STOPPING state via an administrative stop request, and will then end up in the STOPPED state.
A process that cannot be stopped successfully will stay in the STOPPING state forever.  This situation should never be reached during normal operations as it implies that the process did not respond to a final SIGKILL signal sent to it by supervisor, which is "impossible" under UNIX.
State transitions which always require user action to invoke are these:
FATAL   -> STARTING
RUNNING -> STOPPING
State transitions which typically, but not always, require user action to invoke are these, with exceptions noted:
STOPPED -> STARTING (except at supervisord startup if process is configured to autostart)
EXITED -> STARTING (except if process is configured to autorestart)
All other state transitions are managed by supervisord automatically.
LSLOGINS(1)                       User Commands                      LSLOGINS(1)
lslogins - display information about known users in the system
lslogins [options] [-s|-u[=UID]] [-g groups] [-l logins] [username]
Examine the wtmp and btmp logs, /etc/shadow (if necessary) and /passwd and output the desired data.
The optional argument username forces lslogins to print all available details about the specified user only. In this case the output format is different than in case of -l or -g and unknown is username reported as an error.
The default action is to list info about all the users in the system.
-a, --acc-expiration Display data about the date of last password change and the account expiration date (see shadow(5) for more info). (Requires root privileges.)
--btmp-file path Alternate path for btmp.
-c, --colon-separate Separate info about each user with a colon instead of a newline.
-e, --export Output data in the format of NAME=VALUE.
-f, --failed Display data about the users' last failed login attempts.
-G, --supp-groups Show information about supplementary groups.
-g, --groups=groups Only show data of users belonging to groups. More than one group may be specified; the list has to be comma-separated. Unknown group names are ignored.
Note that relation between user and group may be invisible for primary group if the user is not explicitly specify as group member (e.g., in /etc/group). If the command lslogins scans for groups than it uses groups database only, and user database with primary GID is not used at all.
-L, --last Display data containing information about the users' last login sessions.
-l, --logins=logins Only show data of users with a login specified in logins (user names or user IDS). More than one login may be specified; the list has to be comma-separated. Unknown login names are ignored.
-n, --newline Display each piece of information on a separate line.
--noheadings Do not print a header line.
--notruncate Donâ€™t truncate output.
-o, --output list Specify which output columns to print. The default list of columns may be extended if list is specified in the format +list.
--output-all Output all available columns. --help to get a list of all supported columns.
-p, --pwd Display information related to login by password (see also -afL).
-r, --raw Raw output (no columnation).
-s, --system-accs Show system accounts. These are by default all accounts with a UID between 101 and 999 (inclusive), with the exception of either nobody or nfsnobody (UID 65534). This hardcoded default may be overwritten by parameters SYS_UID_MIN and SYS_UID_MAX in the file /etc/login.defs.
--time-format type Display dates in short, full or iso format. The default is short, this time format is designed to be space efficient and human readable.
-u, --user-accs Show user accounts. These are by default all accounts with UID above 1000 (inclusive), with the exception of either nobody or nfsnobody (UID 65534). This hardcoded default maybe overwritten by parameters UID_MIN and UID_MAX in the file /etc/login.defs.
--wtmp-file path Alternate path for wtmp.
--lastlog path Alternate path for lastlog(8).
-Z, --context Display the users' security context.
-z, --print0 Delimit user entries with a nul character, instead of a newline.
0 if OK,
1 if incorrect arguments specified,
2 if a serious error occurs (e.g., a corrupt log).
The default UID thresholds are read from /etc/login.defs.
The lslogins utility is inspired by the logins utility, which first appeared in FreeBSD 4.10.
Ondrej Oprala <ooprala@redhat.com>, Karel Zak <kzak@redhat.com>
group(5), passwd(5), shadow(5), utmp(5)
The lslogins command is part of the util-linux package which can be downloaded from Linux Kernel Archive <https://www.kernel.org/pub/linux/utils/util-linux/>.
SPLIT(1)                          User Commands                         SPLIT(1)
split - split a file into pieces
split [OPTION]... [FILE [PREFIX]]
Output pieces of FILE to PREFIXaa, PREFIXab, ...; default size is 1000 lines, and default PREFIX is 'x'.
-a, --suffix-length=N generate suffixes of length N (default 2)
--additional-suffix=SUFFIX append an additional SUFFIX to file names
-b, --bytes=SIZE put SIZE bytes per output file
-C, --line-bytes=SIZE put at most SIZE bytes of records per output file
-d     use numeric suffixes starting at 0, not alphabetic
--numeric-suffixes[=FROM] same as -d, but allow setting the start value
-x     use hex suffixes starting at 0, not alphabetic
--hex-suffixes[=FROM] same as -x, but allow setting the start value
-e, --elide-empty-files do not generate empty output files with '-n'
--filter=COMMAND write to shell COMMAND; file name is $FILE
-l, --lines=NUMBER put NUMBER lines/records per output file
-n, --number=CHUNKS generate CHUNKS output files; see explanation below
-t, --separator=SEP use SEP instead of newline as the record separator; '\0' (zero) specifies the NUL character
-u, --unbuffered immediately copy input to output with '-n r/...'
--verbose print a diagnostic just before each output file is opened
CHUNKS may be: N      split into N files based on size of input
K/N    output Kth of N to stdout
l/N    split into N files without splitting lines/records
l/K/N  output Kth of N to stdout without splitting lines/records
r/N    like 'l' but use round robin distribution
r/K/N  likewise but only output Kth of N to stdout
Written by Torbjorn Granlund and Richard M. Stallman.
Full documentation <https://www.gnu.org/software/coreutils/split> or available locally via: info '(coreutils) split invocation'
YOUTUBE-DL(1)                                                      YOUTUBE-DL(1)
youtube-dl - download videos from youtube.com or other video platforms
youtube-dl [OPTIONS] URL [URL...]
youtube-dl is a command-line program to download videos from YouTube.com and a few more sites.  It requires the Python interpreter, version 2.6, 2.7, or 3.2+, and it is not platform specific.  It should work on your Unix box, on Windows or on macOS.  It is released to the public domain, which means you can modify it, redistribute it or use it however you like.
-h, --help Print this help text and exit
-U, --update Update this program to latest version.  Make sure that you have sufficient permissions (run with sudo if needed)
-i, --ignore-errors Continue on download errors, for example to skip unavailable videos in a playlist
--abort-on-error Abort downloading of further videos (in the playlist or the command line) if an error occurs
--dump-user-agent Display the current browser identification
--list-extractors List all supported extractors
--extractor-descriptions Output descriptions of all supported extractors
--force-generic-extractor Force extraction to use the generic extractor
--default-search PREFIX Use this prefix for unqualified URLs.  For example "gvsearch2:" downloads two videos from google videos for youtube- dl "large apple".  Use the value "auto" to let youtube-dl guess ("auto_warning" to emit a warning when guessing).  "error" just throws an error.  The default value "fixup_error" repairs broken URLs, but emits an error if this is not possible instead of searching.
--ignore-config Do not read configuration files.  When given in the global configuration file /etc/youtube-dl.conf: Do not read the user configuration in ~/.config/youtube-dl/config (%APPDATA%/youtube-dl/config.txt on Windows)
--config-location PATH Location of the configuration file; either the path to the config or its containing directory.
--flat-playlist Do not extract the videos of a playlist, only list them.
--mark-watched Mark videos watched (YouTube only)
--no-mark-watched Do not mark videos watched (YouTube only)
--no-color Do not emit color codes in output
Network Options: --proxy URL Use the specified HTTP/HTTPS/SOCKS proxy.  To enable SOCKS proxy, specify a proper scheme.  For example socks5://127.0.0.1:1080/. Pass in an empty string (--proxy "") for direct connection
--socket-timeout SECONDS Time to wait before giving up, in seconds
--source-address IP Client-side IP address to bind to
-4, --force-ipv4 Make all connections via IPv4
-6, --force-ipv6 Make all connections via IPv6
Geo Restriction: --geo-verification-proxy URL Use this proxy to verify the IP address for some geo-restricted sites.  The default proxy specified by --proxy (or none, if the option is not present) is used for the actual downloading.
--geo-bypass Bypass geographic restriction via faking X-Forwarded-For HTTP header
--no-geo-bypass Do not bypass geographic restriction via faking X-Forwarded-For HTTP header
--geo-bypass-country CODE Force bypass geographic restriction with explicitly provided two-letter ISO 3166-2 country code
--geo-bypass-ip-block IP_BLOCK Force bypass geographic restriction with explicitly provided IP block in CIDR notation
Video Selection: --playlist-start NUMBER Playlist video to start at (default is 1)
--playlist-end NUMBER Playlist video to end at (default is last)
--playlist-items ITEM_SPEC Playlist video items to download.  Specify indices of the videos in the playlist separated by commas like: "-- playlist-items 1,2,5,8" if you want to download videos indexed 1, 2, 5, 8 in the playlist.  You can specify range: " --playlist-items 1-3,7,10-13", it will download the videos at index 1, 2, 3, 7, 10, 11, 12 and 13.
--match-title REGEX Download only matching titles (regex or caseless sub-string)
--reject-title REGEX Skip download for matching titles (regex or caseless sub-string)
--max-downloads NUMBER Abort after downloading NUMBER files
--min-filesize SIZE Do not download any videos smaller than SIZE (e.g.  50k or 44.6m)
--max-filesize SIZE Do not download any videos larger than SIZE (e.g.  50k or 44.6m)
--date DATE Download only videos uploaded in this date
--datebefore DATE Download only videos uploaded on or before this date (i.e. inclusive)
--dateafter DATE Download only videos uploaded on or after this date (i.e. inclusive)
--min-views COUNT Do not download any videos with less than COUNT views
--max-views COUNT Do not download any videos with more than COUNT views
--match-filter FILTER Generic video filter.  Specify any key (see the "OUTPUT TEMPLATE" for a list of available keys) to match if the key is present, !key to check if the key is not present, key > NUMBER (like "comment_count > 12", also works with >=, <, <=, !=, =) to compare against a number, key = 'LITERAL' (like "uploader = 'Mike Smith'", also works with !=) to match against a string literal and & to require multiple matches.  Values which are not known are excluded unless you put a question mark (?) after the operator.  For example, to only match videos that have been liked more than 100 times and disliked less than 50 times (or the dislike functionality is not available at the given service), but who also have a description, use --match-filter "like_count > 100 & dislike_count <?  50 & description" .
--no-playlist Download only the video, if the URL refers to a video and a playlist.
--yes-playlist Download the playlist, if the URL refers to a video and a playlist.
--age-limit YEARS Download only videos suitable for the given age
--download-archive FILE Download only videos not listed in the archive file.  Record the IDs of all downloaded videos in it.
--include-ads Download advertisements as well (experimental)
Download Options: -r, --limit-rate RATE Maximum download rate in bytes per second (e.g.  50K or 4.2M)
-R, --retries RETRIES Number of retries (default is 10), or "infinite".
--fragment-retries RETRIES Number of retries for a fragment (default is 10), or "infinite" (DASH, hlsnative and ISM)
--skip-unavailable-fragments Skip unavailable fragments (DASH, hlsnative and ISM)
--abort-on-unavailable-fragment Abort downloading when some fragment is not available
--keep-fragments Keep downloaded fragments on disk after downloading is finished; fragments are erased by default
--buffer-size SIZE Size of download buffer (e.g.  1024 or 16K) (default is 1024)
--no-resize-buffer Do not automatically adjust the buffer size.  By default, the buffer size is automatically resized from an initial value of SIZE.
--http-chunk-size SIZE Size of a chunk for chunk-based HTTP downloading (e.g.  10485760 or 10M) (default is disabled).  May be useful for bypassing bandwidth throttling imposed by a webserver (experimental)
--playlist-reverse Download playlist videos in reverse order
--playlist-random Download playlist videos in random order
--xattr-set-filesize Set file xattribute ytdl.filesize with expected file size
--hls-prefer-native Use the native HLS downloader instead of ffmpeg
--hls-prefer-ffmpeg Use ffmpeg instead of the native HLS downloader
--hls-use-mpegts Use the mpegts container for HLS videos, allowing to play the video while downloading (some players may not be able to play it)
--external-downloader COMMAND Use the specified external downloader.  Currently supports aria2c,avconv,axel,c url,ffmpeg,httpie,wget
--external-downloader-args ARGS Give these arguments to the external downloader
Filesystem Options: -a, --batch-file FILE File containing URLs to download ('-' for stdin), one URL per line.  Lines starting with '#', ';' or ']' are considered as comments and ignored.
--id   Use only video ID in file name
-o, --output TEMPLATE Output filename template, see the "OUTPUT TEMPLATE" for all the info
--output-na-placeholder PLACEHOLDER Placeholder value for unavailable meta fields in output filename template (default is "NA")
--autonumber-start NUMBER Specify the start value for %(autonumber)s (default is 1)
--restrict-filenames Restrict filenames to only ASCII characters, and avoid "&" and spaces in filenames
-w, --no-overwrites Do not overwrite files
-c, --continue Force resume of partially downloaded files.  By default, youtube-dl will resume downloads if possible.
--no-continue Do not resume partially downloaded files (restart from beginning)
--no-part Do not use .part files - write directly into output file
--no-mtime Do not use the Last-modified header to set the file modification time
--write-description Write video description to a .description file
--write-info-json Write video metadata to a .info.json file
--write-annotations Write video annotations to a .annotations.xml file
--load-info-json FILE JSON file containing the video information (created with the "--write- info-json" option)
--cookies FILE File to read cookies from and dump cookie jar in
--cache-dir DIR Location in the filesystem where youtube-dl can store some downloaded information permanently.  By default $XDG_CACHE_HOME/youtube-dl or ~/.cache/youtube-dl .  At the moment, only YouTube player files (for videos with obfuscated signatures) are cached, but that may change.
--no-cache-dir Disable filesystem caching
--rm-cache-dir Delete all filesystem cache files
Thumbnail Options: --write-thumbnail Write thumbnail image to disk
--write-all-thumbnails Write all thumbnail image formats to disk
--list-thumbnails Simulate and list all available thumbnail formats
Verbosity / Simulation Options: -q, --quiet Activate quiet mode
--no-warnings Ignore warnings
-s, --simulate Do not download the video and do not write anything to disk
--skip-download Do not download the video
-g, --get-url Simulate, quiet but print URL
-e, --get-title Simulate, quiet but print title
--get-id Simulate, quiet but print id
--get-thumbnail Simulate, quiet but print thumbnail URL
--get-description Simulate, quiet but print video description
--get-duration Simulate, quiet but print video length
--get-filename Simulate, quiet but print output filename
--get-format Simulate, quiet but print output format
-j, --dump-json Simulate, quiet but print JSON information.  See the "OUTPUT TEMPLATE" for a description of available keys.
-J, --dump-single-json Simulate, quiet but print JSON information for each command-line argument.  If the URL refers to a playlist, dump the whole playlist information in a single line.
--print-json Be quiet and print the video information as JSON (video is still being downloaded).
--newline Output progress bar as new lines
--no-progress Do not print progress bar
--console-title Display progress in console titlebar
-v, --verbose Print various debugging information
--dump-pages Print downloaded pages encoded using base64 to debug problems (very verbose)
--write-pages Write downloaded intermediary pages to files in the current directory to debug problems
--print-traffic Display sent and read HTTP traffic
-C, --call-home Contact the youtube-dl server for debugging
--no-call-home Do NOT contact the youtube-dl server for debugging
Workarounds: --encoding ENCODING Force the specified encoding (experimental)
--no-check-certificate Suppress HTTPS certificate validation
--prefer-insecure Use an unencrypted connection to retrieve information about the video.  (Currently supported only for YouTube)
--user-agent UA Specify a custom user agent
--referer URL Specify a custom referer, use if the video access is restricted to one domain
--add-header FIELD:VALUE Specify a custom HTTP header and its value, separated by a colon ':'.  You can use this option multiple times
--bidi-workaround Work around terminals that lack bidirectional text support. Requires bidiv or fribidi executable in PATH
--sleep-interval SECONDS Number of seconds to sleep before each download when used alone or a lower bound of a range for randomized sleep before each download (minimum possible number of seconds to sleep) when used along with --max-sleep-interval.
--max-sleep-interval SECONDS Upper bound of a range for randomized sleep before each download (maximum possible number of seconds to sleep).  Must only be used along with --min- sleep-interval.
Video Format Options: -f, --format FORMAT Video format code, see the "FORMAT SELECTION" for all the info
--all-formats Download all available video formats
--prefer-free-formats Prefer free video formats unless a specific one is requested
-F, --list-formats List all available formats of requested videos
--youtube-skip-dash-manifest Do not download the DASH manifests and related data on YouTube videos
--merge-output-format FORMAT If a merge is required (e.g.  bestvideo+bestaudio), output to given container format.  One of mkv, mp4, ogg, webm, flv.  Ignored if no merge is required
Subtitle Options: --write-sub Write subtitle file
--write-auto-sub Write automatically generated subtitle file (YouTube only)
--all-subs Download all the available subtitles of the video
--list-subs List all available subtitles for the video
--sub-format FORMAT Subtitle format, accepts formats preference, for example: "srt" or "ass/srt/best"
--sub-lang LANGS Languages of the subtitles to download (optional) separated by commas, use --list-subs for available language tags
Authentication Options: -u, --username USERNAME Login with this account ID
-p, --password PASSWORD Account password.  If this option is left out, youtube-dl will ask interactively.
-2, --twofactor TWOFACTOR Two-factor authentication code
-n, --netrc Use .netrc authentication data
--video-password PASSWORD Video password (vimeo, youku)
Adobe Pass Options: --ap-mso MSO Adobe Pass multiple-system operator (TV provider) identifier, use --ap-list-mso for a list of available MSOs
--ap-username USERNAME Multiple-system operator account login
--ap-password PASSWORD Multiple-system operator account password.  If this option is left out, youtube-dl will ask interactively.
--ap-list-mso List all supported multiple-system operators
Post-processing Options: -x, --extract-audio Convert video files to audio-only files (requires ffmpeg/avconv and ffprobe/avprobe)
--audio-format FORMAT Specify audio format: "best", "aac", "flac", "mp3", "m4a", "opus", "vorbis", or "wav"; "best" by default; No effect without -x
--audio-quality QUALITY Specify ffmpeg/avconv audio quality, insert a value between 0 (better) and 9 (worse) for VBR or a specific bitrate like 128K (default 5)
--recode-video FORMAT Encode the video to another format if necessary (currently supported: mp4|flv|ogg|webm|mkv|avi)
--postprocessor-args ARGS Give these arguments to the postprocessor
-k, --keep-video Keep the video file on disk after the post-processing; the video is erased by default
--no-post-overwrites Do not overwrite post-processed files; the post-processed files are overwritten by default
--embed-subs Embed subtitles in the video (only for mp4, webm and mkv videos)
--embed-thumbnail Embed thumbnail in the audio as cover art
--add-metadata Write metadata to the video file
--metadata-from-title FORMAT Parse additional metadata like song title / artist from the video title.  The format syntax is the same as --output.  Regular expression with named capture groups may also be used.  The parsed parameters replace existing values.  Example: --metadata-from-title "%(artist)s - %(title)s" matches a title like "Coldplay - Paradise".  Example (regex): --metadata-from-title "(?P.+?) - (?P .+)"
--xattrs Write metadata to the video file's xattrs (using dublin core and xdg standards)
--fixup POLICY Automatically correct known faults of the file.  One of never (do nothing), warn (only emit a warning), detect_or_warn (the default; fix file if we can, warn otherwise)
--prefer-avconv Prefer avconv over ffmpeg for running the postprocessors
--prefer-ffmpeg Prefer ffmpeg over avconv for running the postprocessors (default)
--ffmpeg-location PATH Location of the ffmpeg/avconv binary; either the path to the binary or its containing directory.
--exec CMD Execute a command on the file after downloading and post-processing, similar to find's -exec syntax.  Example: --exec 'adb push {} /sdcard/Music/ && rm {}'
--convert-subs FORMAT Convert the subtitles to other format (currently supported: srt|ass|vtt|lrc)
You can configure youtube-dl by placing any supported command line option to a configuration file.  On Linux and macOS, the system wide configuration file is located at /etc/youtube-dl.conf and the user wide configuration file at ~/.config/youtube-dl/config.  On Windows, the user wide configuration file locations are %APPDATA%\youtube-dl\config.txt or C:\Users\<user name>\youtube-dl.conf.  Note that by default configuration file may not exist so you may need to create it yourself.
For example, with the following configuration file youtube-dl will always extract the audio, not copy the mtime, use a proxy and save all videos under Movies directory in your home directory:
# Lines starting with # are comments
# Always extract audio -x
# Do not copy the mtime --no-mtime
# Use this proxy --proxy 127.0.0.1:3128
# Save all videos under Movies directory in your home directory -o ~/Movies/%(title)s.%(ext)s
Note that options in configuration file are just the same options aka switches used in regular command line calls thus there must be no whitespace after - or --, e.g.  -o or --proxy but not - o or -- proxy.
You can use --ignore-config if you want to disable the configuration file for a particular youtube-dl run.
You can also use --config-location if you want to use custom configuration file for a particular youtube-dl run.
Authentication with .netrc file You may also want to configure automatic credentials storage for extractors that support authentication (by providing login and password with --username and --password) in order not to pass credentials as command line arguments on every youtube-dl execution and prevent tracking plain text passwords in the shell command history.  You can achieve this using a .netrc file (https://stackoverflow.com/tags/.netrc/info) on a per extractor basis.  For that you will need to create a .netrc file in your $HOME and restrict permissions to read/write by only you:
touch $HOME/.netrc chmod a-rwx,u+rw $HOME/.netrc
After that you can add credentials for an extractor in the following format, where extractor is the name of the extractor in lowercase:
machine <extractor> login <login> password <password>
machine youtube login myaccount@gmail.com password my_youtube_password machine twitch login my_twitch_account_name password my_twitch_password
To activate authentication with the .netrc file you should pass --netrc to youtube-dl or place it in the configuration file.
On Windows you may also need to setup the %HOME% environment variable manually.  For example:
set HOME=%USERPROFILE%
The -o option allows users to indicate a template for the output file names.
tl;dr: navigate me to examples.
The basic usage is not to set any template arguments when downloading a single file, like in youtube-dl -o funny_video.flv "https://some/video". However, it may contain special sequences that will be replaced when downloading each video.  The special sequences may be formatted according to python string formatting operations (https://docs.python.org/2/library/stdtypes.html#string-formatting).  For example, %(NAME)s or %(NAME)05d.  To clarify, that is a percent symbol followed by a name in parentheses, followed by formatting operations. Allowed names along with sequence type are:
• id (string): Video identifier
• title (string): Video title
• url (string): Video URL
• ext (string): Video filename extension
• alt_title (string): A secondary title of the video
• display_id (string): An alternative identifier for the video
• uploader (string): Full name of the video uploader
• license (string): License name the video is licensed under
• creator (string): The creator of the video
• release_date (string): The date (YYYYMMDD) when the video was released
• timestamp (numeric): UNIX timestamp of the moment the video became available
• upload_date (string): Video upload date (YYYYMMDD)
• uploader_id (string): Nickname or id of the video uploader
• channel (string): Full name of the channel the video is uploaded on
• channel_id (string): Id of the channel
• location (string): Physical location where the video was filmed
• duration (numeric): Length of the video in seconds
• view_count (numeric): How many users have watched the video on the platform
• like_count (numeric): Number of positive ratings of the video
• dislike_count (numeric): Number of negative ratings of the video
• repost_count (numeric): Number of reposts of the video
• average_rating (numeric): Average rating give by users, the scale used depends on the webpage
• comment_count (numeric): Number of comments on the video
• age_limit (numeric): Age restriction for the video (years)
• is_live (boolean): Whether this video is a live stream or a fixed-length video
• start_time (numeric): Time in seconds where the reproduction should start, as specified in the URL
• end_time (numeric): Time in seconds where the reproduction should end, as specified in the URL
• format (string): A human-readable description of the format
• format_id (string): Format code specified by --format
• format_note (string): Additional info about the format
• width (numeric): Width of the video
• height (numeric): Height of the video
• resolution (string): Textual description of width and height
• tbr (numeric): Average bitrate of audio and video in KBit/s
• abr (numeric): Average audio bitrate in KBit/s
• acodec (string): Name of the audio codec in use
• asr (numeric): Audio sampling rate in Hertz
• vbr (numeric): Average video bitrate in KBit/s
• fps (numeric): Frame rate
• vcodec (string): Name of the video codec in use
• container (string): Name of the container format
• filesize (numeric): The number of bytes, if known in advance
• filesize_approx (numeric): An estimate for the number of bytes
• protocol (string): The protocol that will be used for the actual download
• extractor (string): Name of the extractor
• extractor_key (string): Key name of the extractor
• epoch (numeric): Unix epoch when creating the file
• autonumber (numeric): Number that will be increased with each download, starting at --autonumber-start
• playlist (string): Name or id of the playlist that contains the video
• playlist_index (numeric): Index of the video in the playlist padded with leading zeros according to the total length of the playlist
• playlist_id (string): Playlist identifier
• playlist_title (string): Playlist title
• playlist_uploader (string): Full name of the playlist uploader
• playlist_uploader_id (string): Nickname or id of the playlist uploader
Available for the video that belongs to some logical chapter or section:
• chapter (string): Name or title of the chapter the video belongs to
• chapter_number (numeric): Number of the chapter the video belongs to
• chapter_id (string): Id of the chapter the video belongs to
Available for the video that is an episode of some series or programme:
• series (string): Title of the series or programme the video episode belongs to
• season (string): Title of the season the video episode belongs to
• season_number (numeric): Number of the season the video episode belongs to
• season_id (string): Id of the season the video episode belongs to
• episode (string): Title of the video episode
• episode_number (numeric): Number of the video episode within a season
• episode_id (string): Id of the video episode
Available for the media that is a track or a part of a music album:
• track (string): Title of the track