neulab/tldr huggingface dataset:
It is difficult for an application to use every part of the file modes field of ls -l in a portable manner. Certain file types and executable bits are not guaranteed to be exactly as shown, as implementations may have extensions. Applications can use this field to pass directly to a user printout or prompt, but actions based on its contents should generally be deferred, instead, to the test utility.
The output of ls (with the -l and related options) contains information that logically could be used by utilities such as chmod and touch to restore files to a known state. However, this information is presented in a format that cannot be used directly by those utilities or be easily translated into a format that can be used. A character has been added to the end of the permissions string so that applications at least have an indication that they may be working in an area they do not understand instead of assuming that they can translate the permissions string into something that can be used. Future versions or related documents may define one or more specific characters to be used based on different standard additional or alternative access control mechanisms.
As with many of the utilities that deal with filenames, the output of ls for multiple files or in one of the long listing formats must be used carefully on systems where filenames can contain embedded white space. Systems and system administrators should institute policies and user training to limit the use of such filenames.
The number of disk blocks occupied by the file that it reports varies depending on underlying file system type, block size units reported, and the method of calculating the number of blocks. On some file system types, the number is the actual number of blocks occupied by the file (counting indirect blocks and ignoring holes in the file); on others it is calculated based on the file size (usually making an allowance for indirect blocks, but ignoring holes).
An example of a small directory tree being fully listed with ls -laRF a in the POSIX locale:
total 11 drwxr-xr-x   3 fox      prog          64 Jul  4 12:07 ./ drwxrwxrwx   4 fox      prog        3264 Jul  4 12:09 ../ drwxr-xr-x   2 fox      prog          48 Jul  4 12:07 b/ -rwxr--r--   1 fox      prog         572 Jul  4 12:07 foo*
a/b: total 4 drwxr-xr-x   2 fox      prog          48 Jul  4 12:07 ./ drwxr-xr-x   3 fox      prog          64 Jul  4 12:07 ../ -rw-r--r--   1 fox      prog         700 Jul  4 12:07 bar
Some historical implementations of the ls utility show all entries in a directory except dot and dot-dot when a superuser invokes ls without specifying the -a option. When ``normal'' users invoke ls without specifying -a, they should not see information about any files with names beginning with a <period> unless they were named as file operands.
Implementations are expected to traverse arbitrary depths when processing the -R option. The only limitation on depth should be based on running out of physical storage for keeping track of untraversed directories.
The -1 (one) option was historically found in BSD and BSD-derived implementations only. It is required in this volume of POSIX.1‐2017 so that conforming applications might ensure that output is one entry per line, even if the output is to a terminal.
The -S option was added in Issue 7, but had been provided by several implementations for many years. The description given in the standard documents historic practice, but does not match much of the documentation that described its behavior. Historical documentation typically described it as something like:
-S        Sort by size (largest size first) instead of by name. Special character devices (listed last) are sorted by name.
even though the file type was never considered when sorting the output. Character special files do typically sort close to the end of the list because their file size on most implementations is zero. But they are sorted alphabetically with any other files that happen to have the same file size (zero), not sorted separately and added to the end.
This volume of POSIX.1‐2017 is frequently silent about what happens when mutually-exclusive options are specified. Except for -R, -d, and -f, the ls utility is required to accept multiple options from each mutually- exclusive option set without treating them as errors and to use the behavior specified by the last option given in each mutually-exclusive set. Since ls is one of the most aliased commands, it is important that the implementation perform intuitively. For example, if the alias were:
alias ls="ls -C"
and the user typed ls -1 (one), single-text-column output should result, not an error.
The -g, -l (ell), -n, and -o options are not mutually-exclusive options. They all enable long format output. They work together to determine whether the file's owner is written (no if -g is present), file's group is written (no if -o is present), and if the file's group or owner is written whether it is written as the name (default) or a string representation of the UID or GID number (if -n is present). The -C, -m, -x, and -1 (one) are mutually-exclusive options and the first three of these disable long format output. The -1 (one) option does not directly change whether or not long format output is enabled, but by overriding -C, -m, and -x, it can re-enable long format output that had been disabled by one of these options.
Earlier versions of this standard did not describe the BSD -A option (like -a, but dot and dot-dot are not written out). It has been added due to widespread implementation.
Implementations may make -q the default for terminals to prevent trojan horse attacks on terminals with special escape sequences.  This is not required because:
*  Some control characters may be useful on some terminals; for example, a system might write them as "\001" or "^A".
*  Special behavior for terminals is not relevant to applications portability.
An early proposal specified that the <optional alternate access method flag> had to be '+' if there was an alternate access method used on the file or <space> if there was not. This was changed to be <space> if there is not and a single printable character if there is. This was done for three reasons:
1. There are historical implementations using characters other than '+'.
2. There are implementations that vary this character used in that position to distinguish between various alternate access methods in use.
3. The standard developers did not want to preclude future specifications that might need a way to specify more than one alternate access method.
Nonetheless, implementations providing a single alternate access method are encouraged to use '+'.
Earlier versions of this standard did not have the -k option, which meant that the -s option could not be used portably as its block size was implementation-defined, and the units used to specify the number of blocks occupied by files in a directory in an ls -l listing were fixed as 512-byte units. The -k option has been added to provide a way for the -s option to be used portably, and for consistency it also changes the aforementioned units from 512-byte to 1024-byte.
The <date and time> field in the -l format is specified only for the POSIX locale. As noted, the format can be different in other locales. No mechanism for defining this is present in this volume of POSIX.1‐2017, as the appropriate vehicle is a messaging system; that is, the format should be specified as a ``message''.
Allowing -f to ignore the -A, -g, -l, -n, -o, and -s options may be removed in a future version.
A future version of this standard may require that if the collating sequence for the current locale does not have a total ordering of all characters, any filenames or pathnames that collate equally are further compared byte-by-byte using the collating sequence for the POSIX locale.
chmod, find
The Base Definitions volume of POSIX.1‐2017, Section 7.3.2, LC_COLLATE, Section 4.5, File Access Permissions, Chapter 8, Environment Variables, Section 12.2, Utility Syntax Guidelines, <sys_stat.h>
The System Interfaces volume of POSIX.1‐2017, fstatat()
FC-PATTERN(1)                                                      FC-PATTERN(1)
fc-pattern - parse and show pattern
fc-pattern [ -cdVh ] [ --config ] [ --default ] [ -f format | --format format ] [ --version ] [ --help ] [ pattern  [ element ... ]  ]
fc-pattern parses pattern (empty pattern by default) and shows the parsed result.  If --config is given, config substitution is performed on the pattern before being displayed.  If --default is given, default substitution is performed on the pattern before being displayed.
If any elements are specified, only those are printed.
-c --config Perform config substitution on pattern.
-d --default Perform default substitution on pattern.
pattern Parses and displays pattern (uses empty pattern by default).
element If set, the element property is displayed for parsed pattern.
FcNameParse(3) FcConfigSubstitute(3) FcDefaultSubstitute(3) FcPatternPrint(3) FcPatternFormat(3) fc-cat(1) fc-cache(1) fc-list(1) fc- match(1) fc-query(1) fc-scan(1)
This manual page was updated by Behdad Esfahbod <behdad@behdad.org>.
ARITHMETIC(6)                   BSD Games Manual                   ARITHMETIC(6)
arithmetic — quiz on simple arithmetic
arithmetic [-o +-x/] [-r range]
arithmetic asks you to solve problems in simple arithmetic.  Each question must be answered correctly before going on to the next.  After every 20 problems, it prints the score so far and the time taken.  You can quit at any time by typing the interrupt or end-of-file character.
-o      By default, arithmetic asks questions on addition of numbers from 0 to 10, and corresponding subtraction.  By supplying one or more of the characters +-x/, you can ask for problems in addition, subtraction, multiplication, and division, respectively.  If you give one of these characters more than once, that kind of problem will be asked correspondingly more often.
-r      If a range is supplied, arithmetic selects the numbers in its problems in the following way.  For addition and multiplication, the numbers to be added or multiplied are between 0 and range, inclusive.  For subtraction and division, both the required result and the number to divide by or subtract will be between 0 and range.  (Of course, arithmetic will not ask you to divide by 0.) The default
range   is 10.
When you get a problem wrong, arithmetic will remember the numbers involved, and will tend to select those numbers more often than others, in problems of the same sort.  Eventually it will forgive and forget.
arithmetic cannot be persuaded to tell you the right answer.  You must work it out for yourself.
“What?” if you get a question wrong.  “Right!” if you get it right. “Please type a number.” if arithmetic doesn't understand what you typed.
bc(1), dc(1)
BG(1P)                      POSIX Programmer's Manual                     BG(1P)
bg — run jobs in the background
bg [job_id...]
If job control is enabled (see the description of set -m), the bg utility shall resume suspended jobs from the current environment (see Section 2.12, Shell Execution Environment) by running them as background jobs. If the job specified by job_id is already a running background job, the bg utility shall have no effect and shall exit successfully.
Using bg to place a job into the background shall cause its process ID to become ``known in the current shell execution environment'', as if it had been started as an asynchronous list; see Section 2.9.3.1, Examples.
job_id    Specify the job to be resumed as a background job. If no job_id operand is given, the most recently suspended job shall be used. The format of job_id is described in the Base Definitions volume of POSIX.1‐2017, Section 3.204, Job Control Job ID.
The following environment variables shall affect the execution of bg:
The output of bg shall consist of a line in the format:
"[%d] %s\n", <job-number>, <command>
where the fields are as follows:
<job-number> A number that can be used to identify the job to the wait, fg, and kill utilities. Using these utilities, the job can be identified by prefixing the job number with '%'.
If job control is disabled, the bg utility shall exit with an error and no job shall be placed in the background.
A job is generally suspended by typing the SUSP character (<control>‐Z on most systems); see the Base Definitions volume of POSIX.1‐2017, Chapter 11, General Terminal Interface.  At that point, bg can put the job into the background. This is most effective when the job is expecting no terminal input and its output has been redirected to non-terminal files. A background job can be forced to stop when it has terminal output by issuing the command:
stty tostop
A background job can be stopped with the command:
kill -s stop job ID
The bg utility does not work as expected when it is operating in its own utility execution environment because that environment has no suspended jobs. In the following examples:
... | xargs bg (bg)
each bg operates in a different environment and does not share its parent shell's understanding of jobs. For this reason, bg is generally implemented as a shell regular built-in.
The extensions to the shell specified in this volume of POSIX.1‐2017 have mostly been based on features provided by the KornShell. The job control features provided by bg, fg, and jobs are also based on the KornShell. The standard developers examined the characteristics of the C shell versions of these utilities and found that differences exist. Despite widespread use of the C shell, the KornShell versions were selected for this volume of POSIX.1‐2017 to maintain a degree of uniformity with the rest of the KornShell features selected (such as the very popular command line editing features).
The bg utility is expected to wrap its output if the output exceeds the number of display columns.
Section 2.9.3.1, Examples, fg, kill, jobs, wait
The Base Definitions volume of POSIX.1‐2017, Section 3.204, Job Control Job ID, Chapter 8, Environment Variables, Chapter 11, General Terminal Interface
nbtscan(1)       scan networks searching for NetBIOS information      nbtscan(1)
nbtscan - scan networks for NetBIOS name information
nbtscan [-v] [-d] [-e] [-l] [-t timeout] [-b bandwidth] [-r] [-q] [-s separator] [-h] [-m retransmits] [-f filename | target]
NBTscan is a program for scanning IP networks for NetBIOS name information. It sends NetBIOS status query to each address in supplied range and lists received information in human readable form. For each responded host it lists IP address, NetBIOS computer name, logged-in user name and MAC address (such as Ethernet).
NBTscan produces a report like that:
IP address     NetBIOS Name  Server    User           MAC address ----------------------------------------------------------------------- 192.168.1.2    MYCOMPUTER              JDOE           00-a0-c9-12-34-56 192.168.1.5    WIN98COMP     <server>  RROE           00-a0-c9-78-90-00 192.168.1.123  DPTSERVER     <server>  ADMINISTRATOR  08-00-09-12-34-56
First column lists IP address of responded host. Second column is computer name. Third column indicates if this computer shares or is able to share files or printers. For NT machine it means that Server Service is running on this computer. For Windows 95 it means that "I want to be able to give others access to my files" or "I want to be able to allow others to print on my printer(s)" checkbox is ticked (in Control Panel/Network/File and Print Sharing). Most often it means that this computer shares files. Third column shows user name. If no one is logged on from this computer it is same as computer name. Last column shows adapter MAC address.
If run with -v switch NBTscan lists whole NetBIOS name table for each responded address.  The output looks like that:
NetBIOS Name Table for Host 192.168.1.123:
Name             Service          Type ---------------------------------------- DPTSERVER        <00>             UNIQUE DPTSERVER        <20>             UNIQUE DEPARTMENT       <00>             GROUP DEPARTMENT       <1c>             GROUP DEPARTMENT       <1b>             UNIQUE DEPARTMENT       <1e>             GROUP DPTSERVER        <03>             UNIQUE DEPARTMENT       <1d>             UNIQUE ??__MSBROWSE__?  <01>             GROUP INet~Services    <1c>             GROUP IS~DPTSERVER     <00>             UNIQUE DPTSERVER        <01>             UNIQUE
Adapter address: 00-a0-c9-12-34-56 ----------------------------------------
-v     Verbose output. Print all names received from each host.
-d     Dump packets. Print whole packet contents. Cannot be used with -v, -s or -h options.
-e     Format output in /etc/hosts format.
-l     Format output in lmhosts format.
-t <timeout> Wait timeout seconds for response. Default 1.
-b <bandwidth> Output  throttling. Slow down output so that it uses no more that bandwidth bps. Useful on slow links, so that outgoing queries don't get dropped.
-r     Use local port 137 for scans. Win95 boxes respond to this only. You need to be root to use this option.
-q     Suppress banners and error messages.
-s <separator> Script-friendly output. Don't print column and record headers, separate fields with separator.
-h     Print human-readable names for services. Can only be used with -v option.
-m <retransmits> Number of retransmits. Default 0.
-f <filename> Take IP addresses to scan from file "filename"
target NBTscan is a command-line tool. You have to supply at least one argument, the address range, in one of three forms:
xxx.xxx.xxx.xxx Single IP in dotted-decimal notation. Example: 192.168.1.1
xxx.xxx.xxx.xxx/xx Net address and subnet mask. Example: 192.168.1.0/24
xxx.xxx.xxx.xxx-xxx Address range. Example: 192.168.1.1-127. This will scan all addresses from 192.168.1.1 to 192.168.1.127
Scans the whole C-class network:
nbtscan 192.168.1.0/24
Scans the whole C-class network, using port 137:
nbtscan -r 192.168.1.0/24
Scans a range from 192.168.1.25 to 192.168.1.137:
nbtscan 192.168.1.25-137
Scans C-class network. Prints results in script-friendly format using colon as field separator:
nbtscan -v -s : 192.168.1.0/24
The last command produces output like that:
192.168.0.1:NT_SERVER:00U 192.168.0.1:MY_DOMAIN:00G 192.168.0.1:ADMINISTRATOR:03U 192.168.0.2:OTHER_BOX:00U ...
Scans IP addresses specified in file iplist:
nbtscan -f iplist
NetBIOS Suffix, aka NetBIOS End Character (endchar), indicates service type for the registered name. The most known codes are listed below. (U = Unique Name, G = Group Name)
Name                Number(h)  Type  Usage --------------------------------------------------------------------------
<computername>         00       U    Workstation Service <computername>         01       U    Messenger Service <\--__MSBROWSE__>      01       G    Master Browser <computername>         03       U    Messenger Service <computername>         06       U    RAS Server Service <computername>         1F       U    NetDDE Service <computername>         20       U    File Server Service <computername>         21       U    RAS Client Service <computername>         22       U    Exchange Interchange(MSMail Connector) <computername>         23       U    Exchange Store <computername>         24       U    Exchange Directory <computername>         30       U    Modem Sharing Server Service <computername>         31       U    Modem Sharing Client Service <computername>         43       U    SMS Clients Remote Control <computername>         44       U    SMS Administrators Remote Control Tool <computername>         45       U    SMS Clients Remote Chat <computername>         46       U    SMS Clients Remote Transfer <computername>         87       U    Microsoft Exchange MTA <computername>         6A       U    Microsoft Exchange IMC <computername>         BE       U    Network Monitor Agent <computername>         BF       U    Network Monitor Application <username>             03       U    Messenger Service <domain>               00       G    Domain Name <domain>               1B       U    Domain Master Browser <domain>               1C       G    Domain Controllers <domain>               1D       U    Master Browser <domain>               1E       G    Browser Service Elections <INet~Services>        1C       G    IIS <IS~computer name>     00       U    IIS
1.  NBTscan lists my Windows boxes just fine but does not list my Unixes or routers. Why?
R: That is the way it is supposed to work. NBTscan uses NetBIOS for scanning and NetBIOS is only implemented by Windows (and some software on Unix such as Samba).
2.  Why do I get "Connection reset by peer" errors on Windows 2000?
R: NBTscan uses port 137 UDP for sending queries. If the port is closed on destination host destination will reply with ICMP "Port unreachable" message. Most operating system will ignore this message. Windows 2000 reports it to the application as "Connection reset by peer" error. Just ignore it.
3.  Why NBTscan doesn't scan for shares? Are you going to add share scanning to NBTscan?
R: No. NBTscan uses UDP for what it does. That makes it very fast. Share scanning requires TCP. For one thing, it will make nbtscan more slow. Also adding share scanning means adding a lot of new code to nbtscan. There is a lot of good share scanners around, so there is no reason to duplicate that work.
4.  Why do I get 00-00-00-00-00-00 instead of MAC address when I scan a Samba box?
R: Because that's what Samba send in response to the query. Nbtscan just prints out what it gets.
NBTscan was created by Alla Bezroutchko <alla@inetcat.org>. Currently is maintained by some volunteers at https://github.com/resurrecting-open- source-projects/nbtscan
This manual page was written for the first time by Ryszard Lach <rla@debian.org> and rewritten, from scratch, by Joao Eriberto Mota Filho <eriberto@debian.org> for the Debian GNU/Linux system (but may be used by others).
XH(1)                             User Commands                            XH(1)
xh - Friendly and fast tool for sending HTTP requests
xh [OPTIONS] <[METHOD] URL> [--] [REQUEST_ITEM]...
xh is a friendly and fast tool for sending HTTP requests.
It reimplements as much as possible of HTTPie's excellent design, with a focus on improved performance.
Positional Arguments METHOD
The HTTP method to use. One of get, post, head, put, patch, delete, or options.
Optional. If omitted, a GET or a POST will be done depending on whether the request sends data.
The URL to request. The scheme defaults to 'http://' unless --https is used.
A leading colon works as shorthand for localhost. :8000 is equivalent to localhost:8000, and :/path is equivalent to localhost/path.
REQUEST_ITEM...
Optional key-value pairs to be included in the request.
key==value to add a parameter to the URL
key=value to add a JSON field (--json) or form field (--form)
key=@value to add a JSON field (--json) or form field (--form) from a file.
key:=value to add a complex JSON value (e.g. `numbers:=[1,2,3]`)
key:=@value to add a complex JSON value (e.g. `numbers:=@data.json`) from a file.
key@filename to upload a file from filename (with --form)
@filename to use a file as the request body
header:value to add a header
header: to unset a header
header; to add a header with an empty value
A backslash can be used to escape special characters (e.g. weird\:key=value).
File uploads can take ';type=' and ';filename=' parameters to set the mimetype and filename, respectively. For example:
key@filename;type=text/html;filename=document.html
Flags -j, --json
(default) Serialize data items from the command line as a JSON object
-f, --form
Serialize data items from the command line as form fields
-m, --multipart
Like --form, but force a multipart/form-data request even without files
-h, --headers
Print only the response headers, shortcut for --print=h
-b, --body
Print only the response body, Shortcut for --print=b
Print the whole request as well as the response
Show any intermediary requests/responses while following redirects with --follow
Do not print to stdout or stderr
-S, --stream
Always stream the response body
-d, --download
Download the body to a file instead of printing it
-c, --continue
Resume an interrupted download. Requires --download and --output
--ignore-netrc
Do not use credentials from .netrc
--offline
Construct HTTP requests without sending them anywhere
(default) Exit with an error status code if the server replies with an error.
The exit code will be 4 on 4xx (Client Error), 5 on 5xx (Server Error), or 3 on 3xx (Redirect) if --follow isn't set.
If stdout is redirected then a warning is written to stderr.
-F, --follow
Do follow redirects
--native-tls
Use the system TLS library instead of rustls (if enabled at compile time)
--https
Make HTTPS requests if not specified in the URL
-I, --ignore-stdin
Do not attempt to read stdin
--curl
Print a translation to a `curl` command.
For translating the other way, try https://curl2httpie.online/.
--curl-long
Use the long versions of curl's flags
Prints help information
Prints version information
--pretty <STYLE>
Controls output processing [possible values: all, colors, format, none]
-s, --style <THEME>
Output coloring style [possible values: auto, solarized, monokai]
--response-charset <ENCODING>
Override the response encoding for terminal display purposes.
Example: `--response-charset=latin1`
--response-mime <MIME_TYPE>
Override the response mime type for coloring and formatting for the terminal
Example: `--response-mime=application/json`
-p, --print <FORMAT>
String specifying what the output should contain.
Use `H` and `B` for request header and body respectively, and `h` and `b` for response hader and body.
Example: `--print=Hb`
-P, --history-print <FORMAT>
The same as --print but applies only to intermediary requests/responses
-o, --output <FILE>
Save output to FILE instead of stdout
--session <FILE>
Create, or reuse and update a session.
Within a session, custom headers, auth credentials, as well as any cookies sent by the server persist between requests.
--session-read-only <FILE>
Create or read a session without updating it form the request/response exchange
-A, --auth-type <auth-type>
Specify the auth mechanism [possible values: basic, bearer, digest]
-a, --auth <USER[:PASS] | TOKEN>
Authenticate as USER with PASS or with TOKEN.
PASS will be prompted if missing. Use a trailing colon (i.e. `USER:`) to authenticate with just a username.
TOKEN is expected if `--auth-type=bearer`.
--max-redirects <NUM>
Number of redirects to follow, only respected if `follow` is set
--timeout <SEC>
Connection timeout of the request.
The default value is `0`, i.e., there is no timeout limit.
--proxy <PROTOCOL:URL>...
Use a proxy for a protocol. For example: `--proxy https:http://proxy.host:8080`.
PROTOCOL can be `http`, `https` or `all`.
If your proxy requires credentials, put them in the URL, like so: `--proxy http:socks5://user:password@proxy.host:8000`.
You can specify proxies for multiple protocols by repeating this option.
The environment variables `http_proxy` and `https_proxy` can also be used, but are completely ignored if --proxy is passed.
--verify <VERIFY>
If "no", skip SSL verification. If a file path, use it as a CA bundle.
Specifying a CA bundle will disable the system's built-in root certificates.
"false" instead of "no" also works. The default is "yes" ("true").
--cert <FILE>
Use a client side certificate for SSL
--cert-key <FILE>
A private key file to use with --cert.
Only necessary if the private key is not contained in the cert file.
--ssl <VERSION>
Force a particular TLS version.
"auto" or "ssl2.3" gives the default behavior of negotiating a version with the server. [possible values: auto, ssl2.3, tls1, tls1.1, tls1.2, tls1.3]
--http-version <VERSION>
HTTP version to use [possible values: 1, 1.0, 1.1, 2]
Each option can be reset with a --no-OPTION argument.
+native-tls
CSVSQL(1)                            csvkit                            CSVSQL(1)
csvsql - csvsql Documentation
Generate SQL statements for a CSV file or execute those statements directly on a database. In the latter case supports both creating tables and inserting data:
usage: csvsql [-h] [-d DELIMITER] [-t] [-q QUOTECHAR] [-u {0,1,2,3}] [-b] [-p ESCAPECHAR] [-z FIELD_SIZE_LIMIT] [-e ENCODING] [-L LOCALE] [-S] [--blanks] [--date-format DATE_FORMAT] [--datetime-format DATETIME_FORMAT] [-H] [-K SKIP_LINES] [-v] [-l] [--zero] [-V] [-i {firebird,mssql,mysql,oracle,postgresql,sqlite,sybase,crate}] [--db CONNECTION_STRING] [--query QUERY] [--insert] [--prefix PREFIX] [--tables TABLE_NAMES] [--no-constraints] [--unique-constraint UNIQUE_CONSTRAINT] [--no-create] [--create-if-not-exists] [--overwrite] [--db-schema DB_SCHEMA] [-y SNIFF_LIMIT] [-I] [--chunk-size NUM] [FILE [FILE ...]]
Generate SQL statements for one or more CSV files, or execute those statements directly on a database, and execute one or more SQL queries.
positional arguments: FILE                  The CSV file(s) to operate on. If omitted, will accept input on STDIN.
optional arguments: -h, --help            show this help message and exit -i {firebird,mssql,mysql,oracle,postgresql,sqlite,sybase,crate}, --dialect {firebird,mssql,mysql,oracle,postgresql,sqlite,sybase,crate} Dialect of SQL to generate. Only valid when --db is not specified. --db CONNECTION_STRING If present, a sqlalchemy connection string to use to directly execute generated SQL on a database. --query QUERY         Execute one or more SQL queries delimited by ";" and output the result of the last query as CSV. QUERY may be a filename. --insert              In addition to creating the table, also insert the data into the table. Only valid when --db is specified. --prefix PREFIX       Add an expression following the INSERT keyword, like OR IGNORE or OR REPLACE. --tables TABLE_NAMES  A comma-separated list of names of tables to be created. By default, the tables will be named after the filenames without extensions or "stdin". --no-constraints      Generate a schema without length limits or null checks. Useful when sampling big tables. --unique-constraint UNIQUE_CONSTRAINT A column-separated list of names of columns to include in a UNIQUE constraint. --no-create           Skip creating a table. Only valid when --insert is specified. --create-if-not-exists Create table if it does not exist, otherwise keep going. Only valid when --insert is specified. --overwrite           Drop the table before creating. Only valid when --insert is specified and --no-create is not specified. --db-schema DB_SCHEMA Optional name of database schema to create table(s) in. -y SNIFF_LIMIT, --snifflimit SNIFF_LIMIT Limit CSV dialect sniffing to the specified number of bytes. Specify "0" to disable sniffing entirely. -I, --no-inference    Disable type inference when parsing the input. --chunk-size NUM Chunk size for batch insert into the table. Only valid when --insert is specified.
For information on connection strings and supported dialects refer to the SQLAlchemy documentation.
If you prefer not to enter your password in the connection string, store the password securely in a PostgreSQL Password File, a MySQL Options File or similar files for other systems.
NOTE: Using the --query option may cause rounding (in Python 2) or introduce [Python floating point issues](‐ https://docs.python.org/3.4/tutorial/floatingpoint.html) (in Python 3).
Generate a statement in the PostgreSQL dialect:
csvsql -i postgresql examples/realdata/FY09_EDU_Recipients_by_State.csv
Create a table and import data from the CSV directly into PostgreSQL:
createdb test csvsql --db postgresql:///test --tables fy09 --insert examples/realdata/FY09_EDU_Recipients_by_State.csv
For large tables it may not be practical to process the entire table. One solution to this is to analyze a sample of the table. In this case it can be useful to turn off length limits and null checks with the --no-constraints option:
head -n 20 examples/realdata/FY09_EDU_Recipients_by_State.csv | csvsql --no-constraints --tables fy09
Create tables for an entire folder of CSVs and import data from those files directly into PostgreSQL:
createdb test csvsql --db postgresql:///test --insert examples/*_converted.csv
If those CSVs have identical headers, you can import them into the same table by using csvstack first:
createdb test csvstack examples/dummy?.csv | csvsql --db postgresql:///test --insert
Group rows by one column:
csvsql --query "select * from 'dummy3' group by a" examples/dummy3.csv
You can also use CSVSQL to “directly” query one or more CSV files. Please note that this will create an in-memory SQL database, so it won’t be very fast:
csvsql --query  "select m.usda_id, avg(i.sepal_length) as mean_sepal_length from iris as i join irismeta as m on (i.species = m.species) group by m.species" examples/iris.csv examples/irismeta.csv
Concatenate two columns:
csvsql --query "select a || b from 'dummy3'" examples/dummy3.csv
If a column contains null values, you must COALESCE the column:
csvsql --query "select a || COALESCE(b, '') from 'sort_ints_nulls'" --no-inference examples/sort_ints_nulls.csv
pngcrush(1)                  General Commands Manual                 pngcrush(1)
pngcrush — optimizes (or modifies) PNG (Portable Network Graphics) files.
pngcrush [options]  [infile.png]  [outfile.png]
pngcrush -e ext [options]  [file1.png]  [file2.png] ...
pngcrush -d dir [options]  [file1.png]  [file2.png] ...
pngcrush is an optimizer for PNG (Portable Network Graphics) files.  Its main purpose is to reduce the size of the file's IDAT chunk (or stream). pngcrush can also be used to modify a PNG's ancillary chunks (example: transparency information or textual comments). Some familiarity with the PNG (pronounced 'ping') format may be helpful to users of pngcrush. pngcrush has reasonable defaults so running with no options may produce smaller files.  This document briefly describes the PNG format where necessary to understand pngcrush.  For complete documentation, see :
http://www.libpng.org/pub/png/.
The actual pixel data of a PNG is contained in one or many IDAT chunks. To make an IDAT chunk PNG encoders take the raw pixel data, filter it with one of 5 different filters (See Section "Filter Types" ) then compress it.  pngcrush reduces the size of PNG files by choosing a different filter or compression methods from those used in the input file.  pngcrush tries multiple filter/compression methods and compares the size of the resulting IDAT chunk.  The filter/compression method with the smallest IDAT chunk is chosen for the output file.  The filter and compression levels pngcrush tries are controlled by command line options. All of the filter methods and compression levels are lossless; they will not reduce image quality.
In addition to reducing the size of a PNG file, pngcrush can modify ancillary chunks.   Ancillary chunks are optional information including transparency (tRNS), gamma (gAMMA), standard RGB color space conformance (sRGB) or textual information (iTXt, tEXt or zTXt).  See the associated command line options for details on manipulating these chunks.
-already size If file has an IDAT chunk greater than the integer size, it will be considered to be already crushed and will not be processed further.
-bit_depth n Force output bit depth to n. See Section "Color Types" for bit depth restrictions.
-bkgd r g b Specify the default background color for the image.  Some PNG viewers will use this background when displaying the image. Background is specified by r, g and b integers between 0 and (2^bitdepth)-1. For output color type 0 and 4 (gray, see section "Color Types" ) the green index is used as the gray level of the background.
-brute    Use brute force, try 114 different filter/compression methods [11-124]. This option is very time-consuming and generally not worthwhile. You can restrict this option to certain filter types, compression levels, or strategies by following it with -f filter,  -l level, or -z strategy. For example:
pngcrush -brute -f 0 infile.png
Will try only methods that use filter type 0.
-c type   Set the output image color type to type, one of [0, 2, 4, or 6] (see section "Color Types" ).  Future versions of pngcrush will also allow color type 3, if there are 256 or fewer colors present in the input file.  Color types 4 and 6 are padded with an opaque alpha channel if the input file does not have alpha information. You can use 0 or 4 to convert color to grayscale. Use 0 or 2 to delete an unwanted alpha channel. Default is to use same color type as the input file.
-d dir    Specify a directory for all output files. All output files will have the same name as their respective input files.
-dou      Double the images gamma.  This is used for fixing gamma in PhotoShop 5.0c5.02 files. It has been claimed that the PS5 bug is actually more complex than that, in some unspecified way.
-e ext    Specify a new extension ext for all output files.
-exit     Forces pngcrush to call exit() rather than return() when it is finished. It is unclear to this writer why this option exists.
-f filter Specify filter (see section "Filter Types" ) to use with the method specified in the preceding -m or -brute option.  Valid filter types are [0-4] : use specified filter, [5]: use adaptive filtering.
-fix      Fix otherwise fatal conditions such as bad CRCs. The CRC (Cyclic Redundancy Check) is present in every chunk and can alert the decoder of corrupt data.
-force    Write a new output file even if larger than input. Otherwise the input file will be copied to output if it is smaller than any generated file and no chunk additions, removals, or changes were requested.
-g gamma  Value to insert in gAMA chunk, only if the input file has no gAMA chunk.  To replace an existing gAMA chunk, use the -replace_gamma option.  Some decoders will use the gAMA chunk to more accurately display the image. gamma relates the output intensity to the input samples (input_sample = light_out GAMMA).  gamma can be any integer between 0 and 2 32 and is interpreted as GAMMA*100000. For example, if you wanted a GAMMA of 1/2.2, you would enter 45455 for gamma.
-h        Display help and legal notices.
-itxt b|a keyword text Insert an (uncompressed) iTXt chunk. b|a indicates whether chunk should be inserted before or after the IDAT chunk, (see Section "Text Chunks" ).
-keep chunk_name keep named chunk chunk_name even if the PNG datastream becomes invalid. Currently only dSIG is recognized as an acceptable value.
-l level  zlib compression level to use on the filtered IDAT chunk with the method specified by the preceding -m or -brute option. zlib compression levels are integers between 0 and 9.  0 = no compression, 1 = fastest compression, and 9 = best compression.
-loco     Make the file more compressible by performing a lossless, reversible, color transformation. The resulting file is a MNG format file, not a PNG, and should be given the .mng file extension. The loco option has no effect on grayscale or indexed-color PNG files.
-m method pngcrush method [0-200] to try (0 means try all of 1-10). Can be repeated as in -m 1 -m 4 -m 7. This can be useful if you run out of memory when pngcrush tries methods 2, 3, 5, 6, 8, 9, or 10 which use filtering and are memory intensive.  Method 1, 4, and 7 use no filtering; methods 11 and up use specified filter, compression level, and strategy.
-max maximum_IDAT_size Set the maximum idat size to be used when creating the compression buffer.[1 through 524288]
-mng      write a new MNG, do not crush embedded PNGs.
-n        Do not do compression or write output file. This is useful in conjunction with the -v option to get info, or to test decode speed.
-p        This option tells pngcrush to pause and wait for [enter] key whenever the screen fills.
-plte_len n Truncates the PLTE. The PLTE chunk contains from 1 to 256 palette entries. Be sure not to truncate it to less than the greatest index present in IDAT.
-q        quiet, the opposite of verbose.
-rem chunkname Name of an ancillary chunk or optional PLTE to be removed. Be careful with this.  Please don't use this feature to remove transparency, gamma, copyright, or other valuable information. To remove several different chunks, repeat: -rem tEXt -rem pHYs.  Known chunks (those in the PNG spec or extensions document) can be named with all lower-case letters, so -rem bkgd is equivalent to -rem bKGD.  But note: -rem text removes all forms of text chunks; Exact case is required to remove unknown chunks.  To do surgery with a chain-saw, -rem alla removes all known ancillary chunks except for tRNS, and -rem allb removes all but tRNS and gAMA.
-replace_gamma gamma Force a specified gamma in the output file even if gAMA is present in the input. See the -g for more information.
-res dpi  Write a pHYs chunk with a resolution of dpi.  The pHYs chunk indicates the desired pixel size.
-save     Force writing of unknown chunks.  If the input image has chunks that are not part of the PNG specification, they should not be discarded.
-srgb n   Set value of rendering intent for sRGB chunk to n where n is between 0 and 3. The appropriate rendering intent depends on how the image will be used:
0 - Perceptual: when good adaptation to the output device gamut at the expense of colorimetric accuracy is desired, example: photographs.
1 - Relative colorimetric: images requiring color appearance matching (relative to the output device white point), example: logos.
2: Saturation: preservation of saturation at the expense of hue and lightness is preferred, example: charts and graphs.
3: Absolute colorimetric: images requiring preservation of absolute colorimetry, example: proofs (previews of images destined for a different output device).
-text [b|a] keyword text Insert a tEXt chunk. [b|a] indicates whether chunk should be inserted before or after the IDAT chunk, (see Section "Text Chunks" ).
-newtimestamp Reset file modification time to newer time stamp. This is the default.
-oldtimestamp Retain input file modification time stamp. The default is -newtimestamp which is the opposite of this option.
-trns index red green blue gray Insert a tRNS (transparency) chunk, if no tRNS chunk found in file.  You must give all five parameters regardless of the color type, scaled to the output bit depth. See the PNG documentation for details.
-trns_array n trns[0] trns[1] ... trns[n-1] Insert a tRNS (transparency) chunk, if no tRNS chunk found in file. See the PNG documentation for details.
-v        Display more detailed information. Repeat the option (use "-v -v") for even more.
-w size   Specify size in kbytes (or bytes in the case of 512) of the sliding compression window where size is one of [32, 16, 8, 4, 2, 1, or 512]. It's best to use the default (32) unless you run out of memory.  The program will use a smaller window anyway when the uncompressed file is smaller than 16k.
-z strategy Specify the zlib compression strategy [0, 1, or 2] to be used on the filtered IDAT chunk for the method of the preceding -m. The zlib strategy parameter tunes the compression algorithm and is one of:
•  0 : default, most compression is aimed towards string matching
•  1 : some string matching, some Huffman coding
•  2 : use only Huffman coding
-zitxt [b|a] keyword text Insert a zTXt chunk. [b|a] indicates whether chunk should be inserted before or after the IDAT chunk, (see Section "Text Chunks" ).
-ztxt [b|a] keyword text zTXt chunk to insert (see -text).
Color Types The PNG specification provides for five color types.  The color type determines how the IDAT chunk will be interpreted by the decoder. Choosing a color type appropriate for the color information in an image can in principle reduce the size. However, certain combinations of -c and -bit_depth require color counting which is currently disabled. In such cases no reduction will take place. Following are the PNG color types followed by their supported bit depths (Note pngcrush does not support changing a file to color type 3 from another color type.):
•  0 : grayscale without alpha channel (1,2,4,8,16)
•  2 : true color without alpha channel (8,16)
•  3 : indexed color (1,2,4,8)
•  4 : grayscale with alpha channel (8,16)
•  6 : true color with alpha channel (8,16)
An alpha channel represents transparency on a per pixel basis. An alpha value of zero is completely transparent. An alpha channel of 2 bitdepth-1 is completely opaque.
Filter Types The IDAT chunk can optionally be filtered before compression.  These filters can make the IDAT chunk more compressible without losing any data and result in a smaller PNG file.  These filters are applied to the bytes of the IDAT chunk, not the pixels. Following is a brief description of the filters, see the PNG specification for details:
•  0 : no filter
•  1 : 'sub' transmits the difference between each byte and the value of the corresponding byte of the prior pixel.
•  2 : 'up' transmits the difference between each byte and the value of the corresponding byte of the pixel above this pixel
•  3 : 'average' transmits the difference between each byte and the average of the bytes described in filters 1 and 2
•  4 : 'paeth' computes a simple linear function of the corresponding byte in three neighboring pixels (paeth_predictor = left + above - upper left), then transmits the difference between the byte in question and the neighboring byte closest to the value of paeth_predictor.
Text Chunks Textual information pertaining to an image can be conveyed with the tEXt, iTXt and zTXt chunks. All text chunks consist of a keyword followed by a string.  The following keywords are defined in the PNG specification: (you may invent keywords for other purposes):
Title: Short title or caption for image
Author: Name of image's creator
Description: Longer description of image
Copyright: Copyright notice
Creation Time: Time of original image creation
Software: Software used to create the image
Disclaimer: Legal disclaimer
Warning: Warning of nature of content
Source: Device used to create the image
Comment: Miscellaneous comment.
A tEXt chunk stores text in the ISO/IEC 8859-1 (Latin-1) character set. zTXt chunks also use the Latin-1 character set, but the text is compressed.  This can be useful for large text chunks.  iTXt chunks consist of text in the UTF-8 of the Unicode character set.
keyword must be at least 1 character and less than 80 characters.  text must be less than 2048  characters when using pngcrush  For  now,  you can only add ten tEXt, iTXt, or zTXt chunks per pngcrush run.
png(5), libpng(3), zlib(3).
This manual page was written by David Whedon dwhedon@gordian.com for the Debian GNU/Linux system (but may be used by others).  Much of the information was gleaned from "PNG (Portable Network Graphics) Specification, Version 1.2"
GIT-INSTAWEB(1)                    Git Manual                    GIT-INSTAWEB(1)
git-instaweb - Instantly browse your working repository in gitweb
git instaweb [--local] [--httpd=<httpd>] [--port=<port>] [--browser=<browser>] git instaweb [--start] [--stop] [--restart]
A simple script to set up gitweb and a web server for browsing the local repository.
-l, --local Only bind the web server to the local IP (127.0.0.1).
-d, --httpd The HTTP daemon command-line that will be executed. Command-line options may be specified here, and the configuration file will be added at the end of the command-line. Currently apache2, lighttpd, mongoose, plackup, python and webrick are supported. (Default: lighttpd)
-m, --module-path The module path (only needed if httpd is Apache). (Default: /usr/lib/apache2/modules)
-p, --port The port number to bind the httpd to. (Default: 1234)
-b, --browser The web browser that should be used to view the gitweb page. This will be passed to the git web--browse helper script along with the URL of the gitweb instance. See git-web--browse(1) for more information about this. If the script fails, the URL will be printed to stdout.
start, --start Start the httpd instance and exit. Regenerate configuration files as necessary for spawning a new instance.
stop, --stop Stop the httpd instance and exit. This does not generate any of the configuration files for spawning a new instance, nor does it close the browser.
restart, --restart Restart the httpd instance and exit. Regenerate configuration files as necessary for spawning a new instance.
You may specify configuration in your .git/config
[instaweb] local = true httpd = apache2 -f port = 4321 browser = konqueror modulePath = /usr/lib/apache2/modules
If the configuration variable instaweb.browser is not set, web.browser will be used instead if it is defined. See git-web--browse(1) for more information about this.
gitweb(1)
PYENV(1)                     General Commands Manual                    PYENV(1)
pyenv - Simple Python version management
pyenv <command> [<args>]