neulab/tldr huggingface dataset:
scontrol show job(s):        Admin, Operator, Coordinator scontrol update job:         Admin, Operator, Coordinator scontrol requeue:            Admin, Operator, Coordinator scontrol show step(s):       Admin, Operator, Coordinator scontrol update step:        Admin, Operator, Coordinator
scontrol show node:          Admin, Operator scontrol update node:        Admin
scontrol create partition:   Admin scontrol show partition:     Admin, Operator scontrol update partition:   Admin scontrol delete partition:   Admin
scontrol create reservation: Admin, Operator scontrol show reservation:   Admin, Operator scontrol update reservation: Admin, Operator scontrol delete reservation: Admin, Operator
scontrol reconfig:           Admin scontrol shutdown:           Admin scontrol takeover:           Admin
# scontrol scontrol: show part debug PartitionName=debug AllocNodes=ALL AllowGroups=ALL Default=YES DefaultTime=NONE DisableRootJobs=NO Hidden=NO MaxNodes=UNLIMITED MaxTime=UNLIMITED MinNodes=1 Nodes=snowflake[0-48] Priority=1 RootOnly=NO OverSubscribe=YES:4 State=UP TotalCPUs=694 TotalNodes=49 scontrol: update PartitionName=debug MaxTime=60:00 MaxNodes=4 scontrol: show job 71701 JobId=71701 Name=hostname UserId=da(1000) GroupId=da(1000) Priority=66264 Account=none QOS=normal WCKey=*123 JobState=COMPLETED Reason=None Dependency=(null) TimeLimit=UNLIMITED Requeue=1 Restarts=0 BatchFlag=0 ExitCode=0:0 SubmitTime=2010-01-05T10:58:40 EligibleTime=2010-01-05T10:58:40 StartTime=2010-01-05T10:58:40 EndTime=2010-01-05T10:58:40 SuspendTime=None SecsPreSuspend=0 Partition=debug AllocNode:Sid=snowflake:4702 ReqNodeList=(null) ExcNodeList=(null) NodeList=snowflake0 NumNodes=1 NumCPUs=10 CPUs/Task=2 ReqS:C:T=1:1:1 MinCPUsNode=2 MinMemoryNode=0 MinTmpDiskNode=0 Features=(null) Reservation=(null) OverSubscribe=OK Contiguous=0 Licenses=(null) Network=(null) scontrol: update JobId=71701 TimeLimit=30:00 Priority=500 scontrol: show hostnames tux[1-3] tux1 tux2 tux3 scontrol: create res StartTime=2009-04-01T08:00:00 Duration=5:00:00 Users=dbremer NodeCnt=10 Reservation created: dbremer_1 scontrol: update Reservation=dbremer_1 Flags=Maint NodeCnt=20 scontrol: delete Reservation=dbremer_1 scontrol: quit
Copyright (C) 2002-2007 The Regents of the University of California. Produced at Lawrence Livermore National Laboratory (cf, DISCLAIMER). Copyright (C) 2008-2010 Lawrence Livermore National Security. Copyright (C) 2010-2018 SchedMD LLC.
/etc/slurm.conf
scancel(1), sinfo(1), squeue(1), slurm_create_partition (3), slurm_delete_partition (3), slurm_load_ctl_conf (3), slurm_load_jobs (3), slurm_load_node (3), slurm_load_partitions (3), slurm_reconfigure (3), slurm_requeue (3), slurm_resume (3), slurm_shutdown (3), slurm_suspend (3), slurm_takeover (3), slurm_update_job (3), slurm_update_node (3), slurm_update_partition (3), slurm.conf(5), slurmctld(8)
SUBLIMINAL,(1)                    User Commands                   SUBLIMINAL,(1)
subliminal, - tool to search and download subtitles
subliminal [OPTIONS] COMMAND [ARGS]...
Subtitles, faster than your thoughts.
--addic7ed USERNAME PASSWORD Addic7ed configuration.
--opensubtitles USERNAME PASSWORD OpenSubtitles configuration.
--subscenter USERNAME PASSWORD SubsCenter configuration.
--cache-dir DIRECTORY Path to the cache directory.  [default: /sbuild-nonexistent/.config/subliminal]
--debug Print useful information for debugging subliminal and for reporting bugs.
Commands: cache  Cache management.
download Download best subtitles.
Suggestions and bug reports are greatly appreciated: https://github.com/Diaoul/subliminal/
kubectl describe - Show details of a specific resource or group of resources
kubectl describe [OPTIONS]
Show details of a specific resource or group of resources.
Print a detailed description of the selected resources, including related resources such as events or controllers. You may select a single object by name, all objects of that type, provide a name prefix, or label selector. For example:
$ kubectl describe TYPE NAME_PREFIX
will first check for an exact match on TYPE and NAME_PREFIX. If no such resource exists, it will output details for every resource that has a name prefixed with NAME_PREFIX.
Use "kubectl api-resources" for a complete list of supported resources.
-A, --all-namespaces=false      If present, list the requested object(s) across all namespaces. Namespace in current context is ignored even if specified with --namespace.
--chunk-size=500      Return large lists in chunks rather than all at once. Pass 0 to disable. This flag is beta and may change in the future.
-f, --filename=[]      Filename, directory, or URL to files containing the resource to describe
-k, --kustomize=""      Process the kustomization directory. This flag can't be used together with -f or -R.
-l, --selector=""      Selector (label query) to filter on, supports '=', '==', and '!='.(e.g. -l key1=value1,key2=value2)
--show-events=true      If true, display events related to the described object.
# Describe a node kubectl describe nodes kubernetes-node-emt8.c.myproject.internal
# Describe a pod kubectl describe pods/nginx
# Describe a pod identified by type and name in "pod.json" kubectl describe -f pod.json
# Describe all pods kubectl describe pods
# Describe pods by label name=myLabel kubectl describe po -l name=myLabel
# Describe all pods managed by the 'frontend' replication controller (rc-created pods # get the name of the rc as a prefix in the pod the name) kubectl describe pods frontend
TRANSMISSION-REMOTE(1)     BSD General Commands Manual    TRANSMISSION-REMOTE(1)
transmission-remote â€” a remote control utility for transmission-daemon(1) and transmission(1)
transmission-remote [host:port | host | port] [-a filenames-or-URLs] [-as] [-AS] [-asd number] [-asu number] [-asc] [-ASC] [-b] [-c path | -C] [-d number | -D] [-e size] [-er | -ep | -et] [--exit] [-f] [-g files] [-G files] [-gsr ratio] [-GSR] [-h] [-i] [-ic] [-if] [-ip] [-it] [-l] [-m | -M] [-n user:pass] [-ne] [-N netrc] [-o | -O] [-p port] [-Bh] [-Bn] [-Bl] [-ph files] [-pl files] [-pn files] [-pr peers] [-r] [-rad] [-s | -S] [-sr ratio] [-SR] [-srd] [-si] [-st] [-t all | active | id | hash] [-hl] [-HL] [-u number | -U] [-utp | -no-utp] [-v] [-V] [-w download-dir] [-x | -X] [-y | -Y] [-pi]
transmission-remote is a remote control utility for transmission(1) and transmission-daemon(1).
By default, transmission-remote connects to the transmission session at localhost:9091.  Other sessions can be controlled by specifying a different host and/or port.
-a --add filenames-or-URLs Add torrents to transmission.
-b --debug Enable debugging mode.
-as --alt-speed Use the alternate Limits.
-AS --no-alt-speed Don't use the alternate Limits.
-asd --alt-speed-downlimit limit Limit the alternate download speed to limit kilobytes per second.
-asu --alt-speed-uplimit limit Limit the alternate upload speed to limit kilobytes per second.
-asc --alt-speed-scheduler Use the scheduled on/off times.
-ASC --no-alt-speed-scheduler Don't use the scheduled on/off days and times.
--alt-speed-time-begin time Time to start using the alt speed limits (in hhmm).
--alt-speed-time-end time Time to stop using the alt speed limits (in hhmm).
--alt-speed-days days Set the number of days on which to enable the speed scheduler, using a list such as "2,4-6".
--torrent-done-script filename Specify a file to run each time a torrent finishes
--no-torrent-done-script Don't run any script when a torrent finishes
-c --incomplete-dir dir When adding new torrents, store their contents in directory until the torrent is done.
-C --no-incomplete-dir Don't store incomplete torrents in a different directory.
-d --downlimit limit Limit the maximum download speed to limit kB/s.  If current torrent(s) are selected this operates on them. Otherwise, it changes the global setting.
-D --no-downlimit Disable download speed limits.  If current torrent(s) are selected this operates on them. Otherwise, it changes the global setting.
-e --cache size Set the session's maximum memory cache size in MiB. This cache is used to reduce disk IO.
-er --encryption-required Encrypt all peer connections.
-ep --encryption-preferred Prefer encrypted peer connections.
-et --encryption-tolerated Prefer unencrypted peer connections.
--exit  Tell the Transmission to initiate a shutdown.
-f --files Get a file list for the current torrent(s)
-g --get all | file-index | files Mark file(s) for download.  all marks all all of the torrent's files for downloading, file-index adds a single file to the download list, and files adds multiple files to the download list, such as "-g1,3-5" to add files #1, #3, #4, and #5 to the download list.
-G --no-get all | file-index | files Mark file(s) for not downloading.
-gsr --global-seedratio ratio All torrents, unless overridden by a per-torrent setting, should seed until a specific ratio
-GSR --no-global-seedratio All torrents, unless overridden by a per-torrent setting, should seed regardless of ratio
-h --help Print command-line option descriptions.
-i --info Show details of the current torrent(s)
-if --info-files List the specified torrent's files
-ip --info-peers List the specified torrent's peers
-ic --info-pieces List the specified torrent's pieces
-it --info-trackers List the specified torrent's trackers
-si --session-info List session information from the server
-st --session-stats List statistical information from the server
-l --list List all torrents
-L --labels Set the specified torrent's labels
-m --portmap Enable portmapping via NAT-PMP or UPnP
-M --no-portmap Disable portmapping
-n --auth username:password Set the username and password for authentication
-ne --authenv Set the authentication information from the TR_AUTH environment variable which must be formatted as username:password.
-N --netrc filename Set the authentication information from a netrc file. See netrc(5) for more information.
-o --dht Enable distributed hash table (DHT).
-O --no-dht Disable distribued hash table (DHT).
-p --port port Set the port for use when listening for incoming peer connections
-Bh --bandwidth-high Give this torrent first chance at available bandwidth
-Bn --bandwidth-normal Give this torrent the bandwidth left over by high priority torrents
-Bl --bandwidth-low Give this torrent the bandwidth left over by high and normal priority torrents
-ph --priority-high all | file-index | files Try to download the specified file(s) first.  all marks all of the torrent's files as normal priority, file-index sets a single file's priority as normal, and files sets multiple files' priorities as normal, such as "-pn1,3-5" to normalize files #1, #3, #4, and #5.
-pn --priority-normal all | file-index | files Try to download the specified files normally.
-pl --priority-low all | file-index | files Try to download the specified files last
-pr --peers number Set the maximum number of peers.  If current torrent(s) are selected this operates on them. Otherwise, it changes the global setting.
-r --remove Remove the current torrent(s). This does not delete the downloaded data.
-rad --remove-and-delete Remove the current torrent(s) and delete their downloaded data.
--reannounce Reannounce the current torrent(s). This is the same as the GUI's "ask tracker for more peers" button.
--move  Move the current torrents' data from their current locations to the specified directory.
--find  Tell Transmission where to look for the current torrents' data.
-sr --seedratio ratio Let the current torrent(s) seed until a specific ratio
-SR --no-seedratio Let the current torrent(s) seed regardless of ratio
-srd --seedratio-default Let the current torrent(s) use the global seedratio settings
-td --tracker-add tracker Add a tracker to a torrent
-tr --tracker-remove trackerId Remove a tracker from a torrent
-s --start Start the current torrent(s)
-S --stop Stop the current torrent(s) from downloading or seeding
--start-paused Start added torrents paused
--no-start-paused Start added torrents unpaused
-t --torrent all | active | id | torrent-hash Set the current torrent(s) for use by subsequent options. The literal all will apply following requests to all torrents; the literal active will apply following requests to recently-active torrents; and specific torrents can be chosen by id or hash.  To set more than one current torrent, join their ids together in a list, such as "-t2,4,6-8" to operate on the torrents whose IDs are 2, 4, 6, 7, and 8.
--trash-torrent Delete torrents after adding
--no-trash-torrent Do not delete torrents after adding
-hl --honor-session Make the current torrent(s) honor the session limits.
-HL --no-honor-session Make the current torrent(s) not honor the session limits.
-u --uplimit limit Limit the maximum upload speed to limit kB/s.  If current torrent(s) are selected this operates on them. Otherwise, it changes the global setting.
-U --no-uplimit Disable upload speed limits.
--utp   Enable uTP for peer connections.
--no-utp Disable uTP for peer connections.  If current torrent(s) are selected this operates on them. Otherwise, it changes the global setting.
-v --verify Verify the current torrent(s)
-V --version Show version number and exit
-w --download-dir directory When used in conjunction with --add, set the new torrent's download folder. Otherwise, set the default download folder.
-x --pex Enable peer exchange (PEX).
-X --no-pex Disable peer exchange (PEX).
-y --lds Enable local peer discovery (LPD).
-Y --no-lds Disable local peer discovery (LPD).
-pi --peer-info List the current torrent's connected peers.  In the `status' section of the list, the following shorthand is used: D: Downloading from this peer d: We would download from this peer if they would let us E: Encrypted connection H: Peer was found through DHT I: Peer is an incoming connection K: Peer has unchoked us, but we're not interested O: Optimistic unchoked T: Peer is connected over uTP U: Uploading to peer u: We would upload to this peer if they asked X: Peer was discovered through Peer Exchange (PEX) ?: We unchoked this peer, but they're not interested
--blocklist-update Update blocklist from URL specified in remote client's settings with blocklist-url key.
List all torrents:
$ transmission-remote -l List all active torrents:
$ transmission-remote -tactive -l Set download and upload limits to 400 kB/sec and 60 kB/sec:
$ transmission-remote -d400 -u60 $ transmission-remote --downlimit=400 --uplimit=60 Set alternate download and upload limits to 100 kB/sec and 20 kB/sec:
$ transmission-remote -asd100 -asu20 $ transmission-remote --alt-speed-downlimit=100 --alt-speed-uplimit=20 Set the scheduler to use the alternate speed limits on weekdays between 10AM and 11PM
$ transmission-remote --alt-speed-time-begin=1000 $ transmission-remote --alt-speed-time-end=2300 $ transmission-remote --alt-speed-days=1-5 $ transmission-remote --alt-speed-scheduler List all torrents' IDs and states:
$ transmission-remote -l List all torrents from a remote session that requires authentication:
$ transmission-remote host:9091 --auth=username:password -l Start all torrents:
$ transmission-remote -tall --start Add two torrents:
$ transmission-remote -a one.torrent two.torrent Add all torrents in ~/Desktop:
$ transmission-remote -a ~/Desktop/*torrent Get detailed information on the torrent whose ID is '1':
$ transmission-remote -t1 -i Get a list of a torrent's files:
$ transmission-remote -t1 -f Download only its second and fourth files:
$ transmission-remote -t1 -Gall -g2,4 Set all torrents' first two files' priorities to high:
$ transmission-remote -tall -ph1,2 Set all torrents' files' priorities to normal:
$ transmission-remote -tall -pnall
http_proxy  Sets the proxy to use for http tracker announces.
Charles Kerr, Josh Elsasser, Eric Petit, Mitchell Livingston, and Mike Gelfand.
TEX(1)                       General Commands Manual                      TEX(1)
tex, initex - text formatting and typesetting
tex [options] [&format] [file|\commands]
Run the TeX typesetter on file, usually creating file.dvi.  If the file argument has no extension, ".tex" will be appended to it.  Instead of a filename, a set of TeX commands can be given, the first of which must start with a backslash.  With a &format argument TeX uses a different set of precompiled commands, contained in format.fmt; it is usually better to use the -fmt format option instead.
TeX formats the interspersed text and commands contained in the named files and outputs a typesetter independent file (called DVI, which is short for DeVice Independent).  TeX's capabilities and language are described in The TeXbook.  TeX is normally used with a large body of precompiled macros, and there are several specific formatting systems, such as LaTeX, which require the support of several macro files.
This version of TeX looks at its command line to see what name it was called under.  If they exist, then both initex and virtex are symbolic links to the tex executable.  When called as initex (or when the -ini option is given) it can be used to precompile macros into a .fmt file. When called as virtex it will use the plain format.  When called under any other name, TeX will use that name as the name of the format to use. For example, when called as tex the tex format is used, which is identical to the plain format.  The commands defined by the plain format are documented in The TeXbook.  Other formats that are often available include latex and amstex.
The non-option command line arguments to the TeX program are passed to it as the first input line.  (But it is often easier to type extended arguments as the first input line, since UNIX shells tend to gobble up or misinterpret TeX's favorite symbols, like backslashes, unless you quote them.)  As described in The TeXbook, that first line should begin with a filename, a \controlsequence, or a &formatname.
The normal usage is to say tex paper to start processing paper.tex.  The name paper will be the ``jobname'', and is used in forming output filenames.  If TeX doesn't get a filename in the first line, the jobname is texput.  When looking for a file, TeX looks for the name with and without the default extension (.tex) appended, unless the name already contains that extension.  If paper is the ``jobname'', a log of error messages, with rather more detail than normally appears on the screen, will appear in paper.log, and the output file will be in paper.dvi.
This version of TeX can look in the first line of the file paper.tex to see if it begins with the magic sequence %&.  If the first line begins with %&format -translate-file tcxname then TeX will use the named format and translation table tcxname to process the source file.  Either the format name or the -translate-file specification may be omitted, but not both.  This overrides the format selection based on the name by which the program is invoked.  The -parse-first-line option or the parse_first_line configuration variable controls whether this behaviour is enabled.
The e response to TeX's error prompt causes the system default editor to start up at the current line of the current file.  The environment variable TEXEDIT can be used to change the editor used.  It may contain a string with "%s" indicating where the filename goes and "%d" indicating where the decimal line number (if any) goes.  For example, a TEXEDIT string for emacs can be set with the sh command TEXEDIT="emacs +%d %s"; export TEXEDIT
A convenient file in the library is null.tex, containing nothing.  When TeX can't find a file it thinks you want to input, it keeps asking you for another filename; responding `null' gets you out of the loop if you don't want to input anything.  You can also type your EOF character (usually control-D).
This version of TeX understands the following command line options.
-fmt format Use format as the name of the format to be used, instead of the name by which TeX was called or a %& line.
-ipc   Send DVI output to a socket as well as the usual output file. Whether this option is available is the choice of the installer.
-output-comment string Use string for the DVI file comment instead of the date.
-output-directory directory Write output files in directory instead of the current directory. Look up input files in directory first, then along the normal search path.  See also description of the TEXMFOUTPUT environment variable.
-src-specials Insert source specials into the DVI file.
-src-specials where Insert source specials in certain places of the DVI file.  where is a comma-separated value list: cr, display, hbox, math, par, parent, or vbox.
See the Kpathsearch library documentation (the `Path specifications' node) for precise details of how the environment variables are used.  The kpsewhich utility can be used to query the values of the variables.
One caveat: In most TeX formats, you cannot use ~ in a filename you give directly to TeX, because ~ is an active character, and hence is expanded, not taken as part of the filename.  Other programs, such as Metafont, do not have this problem.
TEXMFOUTPUT Normally, TeX puts its output files in the current directory.  If any output file cannot be opened there, it tries to open it in the directory specified in the environment variable TEXMFOUTPUT. There is no default value for that variable.  For example, if you say tex paper and the current directory is not writable, if TEXMFOUTPUT has the value /tmp, TeX attempts to create /tmp/paper.log (and /tmp/paper.dvi, if any output is produced.) TEXMFOUTPUT is also checked for input files, as TeX often generates files that need to be subsequently read; for input, no suffixes (such as ``.tex'') are added by default, the input name is simply checked as given.
TEXINPUTS Search path for \input and \openin files.  This should probably start with ``.'', so that user files are found before system files.  An empty path component will be replaced with the paths defined in the texmf.cnf file.  For example, set TEXINPUTS to ".:/home/user/tex:" to prepend the current directory and ``/home/user/tex'' to the standard search path.
TEXPOOL search path for tex internal strings.
TEXEDIT Command template for switching to editor.  The default, usually vi, is set when TeX is compiled.
texmf.cnf Configuration file.  This contains definitions of search paths as well as other configuration parameters like parse_first_line.
tex.pool Text file containing TeX's internal strings.
texfonts.map Filename mapping definitions.
*.tfm  Metric files for TeX's fonts.
*.fmt  Predigested TeX format (.fmt) files.
$TEXMFMAIN/tex/plain/base/plain.tex The basic macro package described in the TeXbook.
This manual page is not meant to be exhaustive.  The complete documentation for this version of TeX can be found in the info manual Web2C: A TeX implementation.
This version of TeX implements a number of optional extensions.  In fact, many of these extensions conflict to a greater or lesser extent with the definition of TeX.  When such extensions are enabled, the banner printed when TeX starts is changed to print TeXk instead of TeX.
This version of TeX fails to trap arithmetic overflow when dimensions are added or subtracted.  Cases where this occurs are rare, but when it does the generated DVI file will be invalid.
mf(1), Donald E. Knuth, The TeXbook, Addison-Wesley, 1986, ISBN 0-201-13447-0. Leslie Lamport, LaTeX - A Document Preparation System, Addison-Wesley, 1985, ISBN 0-201-15790-X. K. Berry, Eplain: Expanded plain TeX, https://tug.org/eplain Michael Spivak, The Joy of TeX, 2nd edition, Addison-Wesley, 1990, ISBN 0-8218-2997-1. TUGboat (the journal of the TeX Users Group).  https://tug.org/TUGboat
TeX, pronounced properly, rhymes with ``blecchhh.''  The proper spelling in typewriter-like fonts is ``TeX'' and not ``TEX'' or ``tex.''
TeX was created by Donald E. Knuth, who implemented it using his Web system for Pascal programs.  It was ported to Unix at Stanford by Howard Trickey, and at Cornell by Pavel Curtis.  The version now offered with the Unix TeX distribution is that generated by the Web to C system (web2c), originally written by Tomas Rokicki and Tim Morgan.
The encTeX extensions were written by Petr Olsak.
SCRIPTREPLAY(1)                   User Commands                  SCRIPTREPLAY(1)
scriptreplay - play back typescripts, using timing information
scriptreplay [options] [-t] timingfile [typescript [divisor]]
This program replays a typescript, using timing information to ensure that output happens in the same rhythm as it originally appeared when the script was recorded.
The replay simply displays the information again; the programs that were run when the typescript was being recorded are not run again. Since the same information is simply being displayed, scriptreplay is only guaranteed to work properly if run on the same type of terminal the typescript was recorded on. Otherwise, any escape characters in the typescript may be interpreted differently by the terminal to which scriptreplay is sending its output.
The timing information is what script(1) outputs to file specified by --log-timing.
By default, the typescript to display is assumed to be named typescript, but other filenames may be specified, as the second parameter or with option --log-out.
If the third parameter or --divisor is specified, it is used as a speed-up multiplier. For example, a speed-up of 2 makes scriptreplay go twice as fast, and a speed-up of 0.1 makes it go ten times slower than the original session.
-I, --log-in file File containing script's terminal input.
-O, --log-out file File containing script's terminal output.
-B, --log-io file File containing script's terminal output and input.
-t, --timing file File containing script's timing output. This option overrides old-style arguments.
-T, --log-timing file This is an alias for -t, maintained for compatibility with script(1) command-line options.
-s, --typescript file File containing script's terminal output. Deprecated alias to --log-out. This option overrides old-style arguments.
-c, --cr-mode mode Specifies how to use the CR (0x0D, carriage return) character from log files. The default mode is auto, in this case CR is replaced with line break for stdin log, because otherwise scriptreplay would overwrite the same line. The other modes are never and always.
-d, --divisor number Speed up the replay displaying this number of times. The argument is a floating-point number. It’s called divisor because it divides the timings by this factor. This option overrides old-style arguments.
-m, --maxdelay number Set the maximum delay between updates to number of seconds. The argument is a floating-point number. This can be used to avoid long pauses in the typescript replay.
--summary Display details about the session recorded in the specified timing file and exit. The session has to be recorded using advanced format (see script(1)) option --logging-format for more details).
-x, --stream type Forces scriptreplay to print only the specified stream. The supported stream types are in, out, signal, or info. This option is recommended for multi-stream logs (e.g., --log-io) in order to print only specified data.
% script --log-timing file.tm --log-out script.out Script started, file is script.out % ls <etc, etc> % exit Script done, file is script.out % scriptreplay --log-timing file.tm --log-out script.out
The original scriptreplay program was written by Joey Hess <joey@kitenet.net>. The program was re-written in C by James Youngman <jay@gnu.org> and Karel Zak <kzak@redhat.com>
Copyright © 2008 James Youngman
Copyright © 2008-2019 Karel Zak
This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Released under the GNU General Public License version 2 or later.
script(1), scriptlive(1)
The scriptreplay command is part of the util-linux package which can be downloaded from Linux Kernel Archive <https://www.kernel.org/pub/linux/utils/util-linux/>.
http(n)                       Tcl Bundled Packages                       http(n)
http - Client-side implementation of the HTTP/1.1 protocol
package require http ?2.9?
::http::config ?-option value ...?
::http::geturl url ?-option value ...?
::http::formatQuery key value ?key value ...?
::http::quoteString value
::http::reset token ?why?
::http::wait token
::http::status token
::http::size token
::http::code token
::http::ncode token
::http::meta token
::http::data token
::http::error token
::http::cleanup token
::http::register proto port command
::http::registerError port ?message?
::http::unregister proto
________________________________________________________________________________
The http package provides the client side of the HTTP/1.1 protocol, as defined in RFC 7230 to RFC 7235, which supersede RFC 2616.  The package implements the GET, POST, and HEAD operations of HTTP/1.1.  It allows configuration of a proxy host to get through firewalls.  The package is compatible with the Safesock security policy, so it can be used by untrusted applets to do URL fetching from a restricted set of hosts. This package can be extended to support additional HTTP transport protocols, such as HTTPS, by providing a custom socket command, via ::http::register.
The ::http::geturl procedure does a HTTP transaction.  Its options determine whether a GET, POST, or HEAD transaction is performed.  The return value of ::http::geturl is a token for the transaction.  The value is also the name of an array in the ::http namespace that contains state information about the transaction.  The elements of this array are described in the STATE ARRAY section.
If the -command option is specified, then the HTTP operation is done in the background.  ::http::geturl returns immediately after generating the HTTP request and the callback is invoked when the transaction completes. For this to work, the Tcl event loop must be active.  In Tk applications this is always true.  For pure-Tcl applications, the caller can use ::http::wait after calling ::http::geturl to start the event loop.
Note: The event queue is even used without the -command option.  As a side effect, arbitrary commands may be processed while http::geturl is running.
::http::config ?options? The ::http::config command is used to set and query the name of the proxy server and port, and the User-Agent name used in the HTTP requests.  If no options are specified, then the current configuration is returned.  If a single argument is specified, then it should be one of the flags described below.  In this case the current value of that setting is returned.  Otherwise, the options should be a set of flags and values that define the configuration:
-accept mimetypes The Accept header of the request.  The default is */*, which means that all types of documents are accepted. Otherwise you can supply a comma-separated list of mime type patterns that you are willing to receive.  For example, “image/gif, image/jpeg, text/*”.
-pipeline boolean Specifies whether HTTP/1.1 transactions on a persistent socket will be pipelined.  See the PERSISTENT SOCKETS section for details. The default is 1.
-postfresh boolean Specifies whether requests that use the POST method will always use a fresh socket, overriding the -keepalive option of command http::geturl.  See the PERSISTENT SOCKETS section for details.  The default is 0.
-proxyhost hostname The name of the proxy host, if any.  If this value is the empty string, the URL host is contacted directly.
-proxyport number The proxy port number.
-proxyfilter command The command is a callback that is made during ::http::geturl to determine if a proxy is required for a given host.  One argument, a host name, is added to command when it is invoked.  If a proxy is required, the callback should return a two-element list containing the proxy server and proxy port.  Otherwise the filter should return an empty list.  The default filter returns the values of the -proxyhost and -proxyport settings if they are non- empty.
-repost boolean Specifies what to do if a POST request over a persistent connection fails because the server has half-closed the connection.  If boolean true, the request will be automatically retried; if boolean false it will not, and the application that uses http::geturl is expected to seek user confirmation before retrying the POST.  The value true should be used only under certain conditions. See the PERSISTENT SOCKETS section for details. The default is 0.
-urlencoding encoding The encoding used for creating the x-url-encoded URLs with ::http::formatQuery and ::http::quoteString.  The default is utf-8, as specified by RFC 2718.  Prior to http 2.5 this was unspecified, and that behavior can be returned by specifying the empty string ({}), although iso8859-1 is recommended to restore similar behavior but without the ::http::formatQuery or ::http::quoteString throwing an error processing non-latin-1 characters.
-useragent string The value of the User-Agent header in the HTTP request.  In an unsafe interpreter, the default value depends upon the operating system, and the version numbers of http and Tcl, and is (for example) “Mozilla/5.0 (Windows; U; Windows NT 10.0) http/2.9.0 Tcl/8.6.9”.  A safe interpreter cannot determine its operating system, and so the default in a safe interpreter is to use a Windows 10 value with the current version numbers of http and Tcl.
-zip boolean If the value is boolean true, then by default requests will send a header “Accept-Encoding: gzip,deflate,compress”.  If the value is boolean false, then by default this header will not be sent.  In either case the default can be overridden for an individual request by supplying a custom Accept-Encoding header in the -headers option of http::geturl. The default is 1.
::http::geturl url ?options? The ::http::geturl command is the main procedure in the package. The -query option causes a POST operation and the -validate option causes a HEAD operation; otherwise, a GET operation is performed. The ::http::geturl command returns a token value that can be used to get information about the transaction.  See the STATE ARRAY and ERRORS section for details.  The ::http::geturl command blocks until the operation completes, unless the -command option specifies a callback that is invoked when the HTTP transaction completes.  ::http::geturl takes several options:
-binary boolean Specifies whether to force interpreting the URL data as binary.  Normally this is auto-detected (anything not beginning with a text content type or whose content encoding is gzip or compress is considered binary data).
-blocksize size The block size used when reading the URL.  At most size bytes are read at once.  After each block, a call to the -progress callback is made (if that option is specified).
-channel name Copy the URL contents to channel name instead of saving it in state(body).
-command callback Invoke callback after the HTTP transaction completes.  This option causes ::http::geturl to return immediately.  The callback gets an additional argument that is the token returned from ::http::geturl. This token is the name of an array that is described in the STATE ARRAY section.  Here is a template for the callback:
proc httpCallback {token} { upvar #0 $token state # Access state as a Tcl array }
-handler callback Invoke callback whenever HTTP data is available; if present, nothing else will be done with the HTTP data. This procedure gets two additional arguments: the socket for the HTTP data and the token returned from ::http::geturl.  The token is the name of a global array that is described in the STATE ARRAY section.  The procedure is expected to return the number of bytes read from the socket.  Here is a template for the callback:
proc httpHandlerCallback {socket token} { upvar #0 $token state # Access socket, and state as a Tcl array # For example... ... set data [read $socket 1000] set nbytes [string length $data] ... return $nbytes }
The http::geturl code for the -handler option is not compatible with either compression or chunked transfer- encoding.  If -handler is specified, then to work around these issues http::geturl will reduce the HTTP protocol to 1.0, and override the -zip option (i.e. it will not send the header "Accept-Encoding: gzip,deflate,compress").
If options -handler and -channel are used together, the handler is responsible for copying the data from the HTTP socket to the specified channel.  The name of the channel is available to the handler as element -channel of the token array.
-headers keyvaluelist This option is used to add headers not already specified by ::http::config to the HTTP request.  The keyvaluelist argument must be a list with an even number of elements that alternate between keys and values.  The keys become header field names.  Newlines are stripped from the values so the header cannot be corrupted.  For example, if keyvaluelist is Pragma no-cache then the following header is included in the HTTP request:
-keepalive boolean If boolean true, attempt to keep the connection open for servicing multiple requests.  Default is 0.
-method type Force the HTTP request method to type. ::http::geturl will auto-select GET, POST or HEAD based on other options, but this option enables choices like PUT and DELETE for webdav support.
-myaddr address Pass an specific local address to the underlying socket call in case multiple interfaces are available.
-progress callback The callback is made after each transfer of data from the URL.  The callback gets three additional arguments: the token from ::http::geturl, the expected total size of the contents from the Content-Length meta-data, and the current number of bytes transferred so far.  The expected total size may be unknown, in which case zero is passed to the callback.  Here is a template for the progress callback:
proc httpProgress {token total current} { upvar #0 $token state }
-protocol version Select the HTTP protocol version to use. This should be 1.0 or 1.1 (the default). Should only be necessary for servers that do not understand or otherwise complain about HTTP/1.1.
-query query This flag causes ::http::geturl to do a POST request that passes the query as payload verbatim to the server.  The content format (and encoding) of query is announced by the header field content-type set by the option -type.  query is an x-url-encoding formatted query, if used for html forms.  The ::http::formatQuery procedure can be used to do the formatting.
-queryblocksize size The block size used when posting query data to the URL.  At most size bytes are written at once.  After each block, a call to the -queryprogress callback is made (if that option is specified).
-querychannel channelID This flag causes ::http::geturl to do a POST request that passes the data contained in channelID to the server. The data contained in channelID must be an x-url-encoding formatted query unless the -type option below is used.  If a Content-Length header is not specified via the -headers options, ::http::geturl attempts to determine the size of the post data in order to create that header.  If it is unable to determine the size, it returns an error.
-queryprogress callback The callback is made after each transfer of data to the URL (i.e. POST) and acts exactly like the -progress option (the callback format is the same).
-strict boolean Whether to enforce RFC 3986 URL validation on the request. Default is 1.
-timeout milliseconds If milliseconds is non-zero, then ::http::geturl sets up a timeout to occur after the specified number of milliseconds.  A timeout results in a call to ::http::reset and to the -command callback, if specified.  The return value of ::http::status is timeout after a timeout has occurred.
-type mime-type Use mime-type as the Content-Type value, instead of the default value (application/x-www-form-urlencoded) during a POST operation.
-validate boolean If boolean is non-zero, then ::http::geturl does an HTTP HEAD request.  This request returns meta information about the URL, but the contents are not returned.  The meta information is available in the state(meta)  variable after the transaction.  See the STATE ARRAY section for details.
::http::formatQuery key value ?key value ...? This procedure does x-url-encoding of query data.  It takes an even number of arguments that are the keys and values of the query.  It encodes the keys and values, and generates one string that has the proper & and = separators.  The result is suitable for the -query value passed to ::http::geturl.
::http::quoteString value This procedure does x-url-encoding of string.  It takes a single argument and encodes it.
::http::reset token ?why? This command resets the HTTP transaction identified by token, if any.  This sets the state(status) value to why, which defaults to reset, and then calls the registered -command callback.
::http::wait token This is a convenience procedure that blocks and waits for the transaction to complete.  This only works in trusted code because it uses vwait.  Also, it is not useful for the case where ::http::geturl is called without the -command option because in this case the ::http::geturl call does not return until the HTTP transaction is complete, and thus there is nothing to wait for.
::http::data token This is a convenience procedure that returns the body element (i.e., the URL data) of the state array.
::http::error token This is a convenience procedure that returns the error element of the state array.
::http::status token This is a convenience procedure that returns the status element of the state array.
::http::code token This is a convenience procedure that returns the http element of the state array.
::http::ncode token This is a convenience procedure that returns just the numeric return code (200, 404, etc.) from the http element of the state array.
::http::size token This is a convenience procedure that returns the currentsize element of the state array, which represents the number of bytes received from the URL in the ::http::geturl call.
::http::meta token This is a convenience procedure that returns the meta element of the state array which contains the HTTP response headers. See below for an explanation of this element.
::http::cleanup token This procedure cleans up the state associated with the connection identified by token.  After this call, the procedures like ::http::data cannot be used to get information about the operation.  It is strongly recommended that you call this function after you are done with a given HTTP request.  Not doing so will result in memory not being freed, and if your app calls ::http::geturl enough times, the memory leak could cause a performance hit...or worse.
::http::register proto port command This procedure allows one to provide custom HTTP transport types such as HTTPS, by registering a prefix, the default port, and the command to execute to create the Tcl channel. E.g.:
package require http package require tls
::http::register https 443 ::tls::socket
set token [::http::geturl https://my.secure.site/]
::http::registerError port ?message? This procedure allows a registered protocol handler to deliver an error message for use by http.  Calling this command does not raise an error. The command is useful when a registered protocol detects an problem (for example, an invalid TLS certificate) that will cause an error to propagate to http.  The command allows http to provide a precise error message rather than a general one.  The command returns the value provided by the last call with argument message, or the empty string if no such call has been made.
::http::unregister proto This procedure unregisters a protocol handler that was previously registered via ::http::register, returning a two-item list of the default port and handler command that was previously installed (via ::http::register) if there was such a handler, and an error if there was no such handler.
The ::http::geturl procedure will raise errors in the following cases: invalid command line options, an invalid URL, a URL on a non-existent host, or a URL at a bad port on an existing host.  These errors mean that it cannot even start the network transaction.  It will also raise an error if it gets an I/O error while writing out the HTTP request header. For synchronous ::http::geturl calls (where -command is not specified), it will raise an error if it gets an I/O error while reading the HTTP reply headers or data.  Because ::http::geturl does not return a token in these cases, it does all the required cleanup and there is no issue of your app having to call ::http::cleanup.
For asynchronous ::http::geturl calls, all of the above error situations apply, except that if there is any error while reading the HTTP reply headers or data, no exception is thrown.  This is because after writing the HTTP headers, ::http::geturl returns, and the rest of the HTTP transaction occurs in the background.  The command callback can check if any error occurred during the read by calling ::http::status to check the status and if its error, calling ::http::error to get the error message.
Alternatively, if the main program flow reaches a point where it needs to know the result of the asynchronous HTTP request, it can call ::http::wait and then check status and error, just as the callback does.
In any case, you must still call ::http::cleanup to delete the state array when you are done.
There are other possible results of the HTTP transaction determined by examining the status from ::http::status.  These are described below.
ok     If the HTTP transaction completes entirely, then status will be ok.  However, you should still check the ::http::code value to get the HTTP status.  The ::http::ncode procedure provides just the numeric error (e.g., 200, 404 or 500) while the ::http::code procedure returns a value like “HTTP 404 File not found”.
eof    If the server closes the socket without replying, then no error is raised, but the status of the transaction will be eof.
error  The error message will also be stored in the error status array element, accessible via ::http::error.
timeout A timeout occurred before the transaction could complete
reset  user-reset
Another error possibility is that ::http::geturl is unable to write all the post query data to the server before the server responds and closes the socket.  The error message is saved in the posterror status array element and then  ::http::geturl attempts to complete the transaction. If it can read the server's response it will end up with an ok status, otherwise it will have an eof status.
The ::http::geturl procedure returns a token that can be used to get to the state of the HTTP transaction in the form of a Tcl array.  Use this construct to create an easy-to-use array variable:
upvar #0 $token state
Once the data associated with the URL is no longer needed, the state array should be unset to free up storage.  The ::http::cleanup procedure is provided for that purpose.  The following elements of the array are supported:
binary This is boolean true if (after decoding any compression specified by the “Content-Encoding” response header) the HTTP response is binary.  It is boolean false if the HTTP response is text.
body   The contents of the URL.  This will be empty if the -channel option has been specified.  This value is returned by the ::http::data command.
charset The value of the charset attribute from the Content-Type meta-data value.  If none was specified, this defaults to the RFC standard iso8859-1, or the value of $::http::defaultCharset.  Incoming text data will be automatically converted from this charset to utf-8.
coding A copy of the Content-Encoding meta-data value.
currentsize The current number of bytes fetched from the URL.  This value is returned by the ::http::size command.
error  If defined, this is the error string seen when the HTTP transaction was aborted.
http   The HTTP status reply from the server.  This value is returned by the ::http::code command.  The format of this value is:
HTTP/1.1 code string
The code is a three-digit number defined in the HTTP standard.  A code of 200 is OK.  Codes beginning with 4 or 5 indicate errors.  Codes beginning with 3 are redirection errors.  In this case the Location meta-data specifies a new URL that contains the requested information.
meta   The HTTP protocol returns meta-data that describes the URL contents.  The meta element of the state array is a list of the keys and values of the meta-data.  This is in a format useful for initializing an array that just contains the meta-data:
array set meta $state(meta)
Some of the meta-data keys are listed below, but the HTTP standard defines more, and servers are free to add their own.
Content-Type The type of the URL contents.  Examples include text/html, image/gif, application/postscript and application/x-tcl.
Content-Length The advertised size of the contents.  The actual size obtained by ::http::geturl is available as state(currentsize).