neulab/tldr huggingface dataset:
mysql> help logs Many help items for your request exist. To make a more specific request, please type 'help <item>', where <item> is one of the following topics: SHOW SHOW BINARY LOGS SHOW ENGINE SHOW LOGS
Use a topic as the search string to see the help entry for that topic:
mysql> help show binary logs Name: 'SHOW BINARY LOGS' Description: Syntax: SHOW BINARY LOGS SHOW MASTER LOGS Lists the binary log files on the server. This statement is used as part of the procedure described in [purge-binary-logs], that shows how to determine which logs can be purged. mysql> SHOW BINARY LOGS; +---------------+-----------+-----------+ | Log_name      | File_size | Encrypted | +---------------+-----------+-----------+ | binlog.000015 |    724935 | Yes       | | binlog.000016 |    733481 | Yes       | +---------------+-----------+-----------+
The search string can contain the wildcard characters % and _. These have the same meaning as for pattern-matching operations performed with the LIKE operator. For example, HELP rep% returns a list of topics that begin with rep:
mysql> HELP rep% Many help items for your request exist. To make a more specific request, please type 'help <item>', where <item> is one of the following topics: REPAIR TABLE REPEAT FUNCTION REPEAT LOOP REPLACE REPLACE FUNCTION
The mysql client typically is used interactively, like this:
However, it is also possible to put your SQL statements in a file and then tell mysql to read its input from that file. To do so, create a text file text_file that contains the statements you wish to execute. Then invoke mysql as shown here:
mysql db_name < text_file
If you place a USE db_name statement as the first statement in the file, it is unnecessary to specify the database name on the command line:
mysql < text_file
If you are already running mysql, you can execute an SQL script file using the source command or \.  command:
mysql> source file_name mysql> \. file_name
Sometimes you may want your script to display progress information to the user. For this you can insert statements like this:
SELECT '<info_to_display>' AS ' ';
The statement shown outputs <info_to_display>.
You can also invoke mysql with the --verbose option, which causes each statement to be displayed before the result that it produces.
mysql ignores Unicode byte order mark (BOM) characters at the beginning of input files. Previously, it read them and sent them to the server, resulting in a syntax error. Presence of a BOM does not cause mysql to change its default character set. To do that, invoke mysql with an option such as --default-character-set=utf8.
For more information about batch mode, see Section 3.5, “Using mysql in Batch Mode”.
This section provides information about techniques for more effective use of mysql and about mysql operational behavior.
•   Input-Line Editing
•   Disabling Interactive History
•   Unicode Support on Windows
•   Displaying Query Results Vertically
•   Using Safe-Updates Mode (--safe-updates)
•   Disabling mysql Auto-Reconnect
•   mysql Client Parser Versus Server Parser Input-Line Editing
mysql supports input-line editing, which enables you to modify the current input line in place or recall previous input lines. For example, the left-arrow and right-arrow keys move horizontally within the current input line, and the up-arrow and down-arrow keys move up and down through the set of previously entered lines.  Backspace deletes the character before the cursor and typing new characters enters them at the cursor position. To enter the line, press Enter.
On Windows, the editing key sequences are the same as supported for command editing in console windows. On Unix, the key sequences depend on the input library used to build mysql (for example, the libedit or readline library).
Documentation for the libedit and readline libraries is available online. To change the set of key sequences permitted by a given input library, define key bindings in the library startup file. This is a file in your home directory: .editrc for libedit and .inputrc for readline.
For example, in libedit, Control+W deletes everything before the current cursor position and Control+U deletes the entire line. In readline, Control+W deletes the word before the cursor and Control+U deletes everything before the current cursor position. If mysql was built using libedit, a user who prefers the readline behavior for these two keys can put the following lines in the .editrc file (creating the file if necessary):
bind "^W" ed-delete-prev-word bind "^U" vi-kill-line-prev
To see the current set of key bindings, temporarily put a line that says only bind at the end of .editrc.  mysql shows the bindings when it starts.  Disabling Interactive History
The up-arrow key enables you to recall input lines from current and previous sessions. In cases where a console is shared, this behavior may be unsuitable.  mysql supports disabling the interactive history partially or fully, depending on the host platform.
On Windows, the history is stored in memory.  Alt+F7 deletes all input lines stored in memory for the current history buffer. It also deletes the list of sequential numbers in front of the input lines displayed with F7 and recalled (by number) with F9. New input lines entered after you press Alt+F7 repopulate the current history buffer. Clearing the buffer does not prevent logging to the Windows Event Viewer, if the --syslog option was used to start mysql. Closing the console window also clears the current history buffer.
To disable interactive history on Unix, first delete the .mysql_history file, if it exists (previous entries are recalled otherwise). Then start mysql with the --histignore="*" option to ignore all new input lines. To re-enable the recall (and logging) behavior, restart mysql without the option.
If you prevent the .mysql_history file from being created (see Controlling the History File) and use --histignore="*" to start the mysql client, the interactive history recall facility is disabled fully. Alternatively, if you omit the --histignore option, you can recall the input lines entered during the current session.  Unicode Support on Windows
Windows provides APIs based on UTF-16LE for reading from and writing to the console; the mysql client for Windows is able to use these APIs. The Windows installer creates an item in the MySQL menu named MySQL command line client - Unicode. This item invokes the mysql client with properties set to communicate through the console to the MySQL server using Unicode.
To take advantage of this support manually, run mysql within a console that uses a compatible Unicode font and set the default character set to a Unicode character set that is supported for communication with the server:
1. Open a console window.
2. Go to the console window properties, select the font tab, and choose Lucida Console or some other compatible Unicode font. This is necessary because console windows start by default using a DOS raster font that is inadequate for Unicode.
3. Execute mysql.exe with the --default-character-set=utf8 (or utf8mb4) option. This option is necessary because utf16le is one of the character sets that cannot be used as the client character set. See the section called “Impermissible Client Character Sets”.
With those changes, mysql uses the Windows APIs to communicate with the console using UTF-16LE, and communicate with the server using UTF-8. (The menu item mentioned previously sets the font and character set as just described.)
To avoid those steps each time you run mysql, you can create a shortcut that invokes mysql.exe. The shortcut should set the console font to Lucida Console or some other compatible Unicode font, and pass the --default-character-set=utf8 (or utf8mb4) option to mysql.exe.
Alternatively, create a shortcut that only sets the console font, and set the character set in the [mysql] group of your my.ini file:
[mysql] default-character-set=utf8
Displaying Query Results Vertically
Some query results are much more readable when displayed vertically, instead of in the usual horizontal table format. Queries can be displayed vertically by terminating the query with \G instead of a semicolon. For example, longer text values that include newlines often are much easier to read with vertical output:
mysql> SELECT * FROM mails WHERE LENGTH(txt) < 300 LIMIT 300,1\G *************************** 1. row *************************** msg_nro: 3068 date: 2000-03-01 23:29:50 time_zone: +0200 mail_from: Jones reply: jones@example.com mail_to: "John Smith" <smith@example.com> sbj: UTF-8 txt: >>>>> "John" == John Smith writes: John> Hi.  I think this is a good idea.  Is anyone familiar John> with UTF-8 or Unicode? Otherwise, I'll put this on my John> TODO list and see what happens. Yes, please do that. Regards, Jones file: inbox-jani-1 hash: 190402944 1 row in set (0.09 sec)
Using Safe-Updates Mode (--safe-updates)
For beginners, a useful startup option is --safe-updates (or --i-am-a-dummy, which has the same effect). Safe-updates mode is helpful for cases when you might have issued an UPDATE or DELETE statement but forgotten the WHERE clause indicating which rows to modify. Normally, such statements update or delete all rows in the table. With --safe-updates, you can modify rows only by specifying the key values that identify them, or a LIMIT clause, or both. This helps prevent accidents. Safe-updates mode also restricts SELECT statements that produce (or are estimated to produce) very large result sets.
The --safe-updates option causes mysql to execute the following statement when it connects to the MySQL server, to set the session values of the sql_safe_updates, sql_select_limit, and max_join_size system variables:
SET sql_safe_updates=1, sql_select_limit=1000, max_join_size=1000000;
The SET statement affects statement processing as follows:
•   Enabling sql_safe_updates causes UPDATE and DELETE statements to produce an error if they do not specify a key constraint in the WHERE clause, or provide a LIMIT clause, or both. For example:
UPDATE tbl_name SET not_key_column=val WHERE key_column=val; UPDATE tbl_name SET not_key_column=val LIMIT 1;
•   Setting sql_select_limit to 1,000 causes the server to limit all SELECT result sets to 1,000 rows unless the statement includes a LIMIT clause.
•   Setting max_join_size to 1,000,000 causes multiple-table SELECT statements to produce an error if the server estimates it must examine more than 1,000,000 row combinations.
To specify result set limits different from 1,000 and 1,000,000, you can override the defaults by using the --select-limit and --max-join-size options when you invoke mysql:
mysql --safe-updates --select-limit=500 --max-join-size=10000
It is possible for UPDATE and DELETE statements to produce an error in safe-updates mode even with a key specified in the WHERE clause, if the optimizer decides not to use the index on the key column:
•   Range access on the index cannot be used if memory usage exceeds that permitted by the range_optimizer_max_mem_size system variable. The optimizer then falls back to a table scan. See the section called “Limiting Memory Use for Range Optimization”.
•   If key comparisons require type conversion, the index may not be used (see Section 8.3.1, “How MySQL Uses Indexes”). Suppose that an indexed string column c1 is compared to a numeric value using WHERE c1 = 2222. For such comparisons, the string value is converted to a number and the operands are compared numerically (see Section 12.3, “Type Conversion in Expression Evaluation”), preventing use of the index. If safe-updates mode is enabled, an error occurs.
As of MySQL 8.0.13, safe-updates mode also includes these behaviors:
•   EXPLAIN with UPDATE and DELETE statements does not produce safe-updates errors. This enables use of EXPLAIN plus SHOW WARNINGS to see why an index is not used, which can be helpful in cases such as when a range_optimizer_max_mem_size violation or type conversion occurs and the optimizer does not use an index even though a key column was specified in the WHERE clause.
•   When a safe-updates error occurs, the error message includes the first diagnostic that was produced, to provide information about the reason for failure. For example, the message may indicate that the range_optimizer_max_mem_size value was exceeded or type conversion occurred, either of which can preclude use of an index.
•   For multiple-table deletes and updates, an error is produced with safe updates enabled only if any target table uses a table scan. Disabling mysql Auto-Reconnect
If the mysql client loses its connection to the server while sending a statement, it immediately and automatically tries to reconnect once to the server and send the statement again. However, even if mysql succeeds in reconnecting, your first connection has ended and all your previous session objects and settings are lost: temporary tables, the autocommit mode, and user-defined and session variables. Also, any current transaction rolls back. This behavior may be dangerous for you, as in the following example where the server was shut down and restarted between the first and second statements without you knowing it:
mysql> SET @a=1; Query OK, 0 rows affected (0.05 sec) mysql> INSERT INTO t VALUES(@a); ERROR 2006: MySQL server has gone away No connection. Trying to reconnect... Connection id:    1 Current database: test Query OK, 1 row affected (1.30 sec) mysql> SELECT * FROM t; +------+ | a    | +------+ | NULL | +------+ 1 row in set (0.05 sec)
The @a user variable has been lost with the connection, and after the reconnection it is undefined. If it is important to have mysql terminate with an error if the connection has been lost, you can start the mysql client with the --skip-reconnect option.
For more information about auto-reconnect and its effect on state information when a reconnection occurs, see Automatic Reconnection Control[5].  mysql Client Parser Versus Server Parser
The mysql client uses a parser on the client side that is not a duplicate of the complete parser used by the mysqld server on the server side. This can lead to differences in treatment of certain constructs. Examples:
•   The server parser treats strings delimited by " characters as identifiers rather than as plain strings if the ANSI_QUOTES SQL mode is enabled.
The mysql client parser does not take the ANSI_QUOTES SQL mode into account. It treats strings delimited by ", ', and ` characters the same, regardless of whether ANSI_QUOTES is enabled.
•   Within /*! ... */ and /*+ ... */ comments, the mysql client parser interprets short-form mysql commands. The server parser does not interpret them because these commands have no meaning on the server side.
If it is desirable for mysql not to interpret short-form commands within comments, a partial workaround is to use the --binary-mode option, which causes all mysql commands to be disabled except \C and \d in noninteractive mode (for input piped to mysql or loaded using the source command).
1. MySQL Shell 8.0 https://dev.mysql.com/doc/mysql-shell/8.0/en/
2. C API Basic Data Structures https://dev.mysql.com/doc/c-api/8.0/en/c-api-data-structures.html
3. Multiple Statement Execution Support https://dev.mysql.com/doc/c-api/8.0/en/c-api-multiple-queries.html
4. mysql_change_user() https://dev.mysql.com/doc/c-api/8.0/en/mysql-change-user.html
5. Automatic Reconnection Control https://dev.mysql.com/doc/c-api/8.0/en/c-api-auto-reconnect.html
source(n)                     Tcl Built-In Commands                    source(n)
source - Evaluate a file or resource as a Tcl script
source fileName
source -encoding encodingName fileName
________________________________________________________________________________
This command takes the contents of the specified file or resource and passes it to the Tcl interpreter as a text script.  The return value from source is the return value of the last command executed in the script. If an error occurs in evaluating the contents of the script then the source command will return that error.  If a return command is invoked from within the script then the remainder of the file will be skipped and the source command will return normally with the result from the return command.
The end-of-file character for files is “\32” (^Z) for all platforms.  The source command will read files up to this character.  This restriction does not exist for the read or gets commands, allowing for files containing code and data segments (scripted documents).  If you require a “^Z” in code for string comparison, you can use “\032” or “\u001a”, which will be safely substituted by the Tcl interpreter into “^Z”.
A leading BOM (Byte order mark) contained in the file is ignored for unicode encodings (utf-8, unicode).
The -encoding option is used to specify the encoding of the data stored in fileName.  When the -encoding option is omitted, the system encoding is assumed.
Run the script in the file foo.tcl and then the script in the file bar.tcl:
source foo.tcl source bar.tcl
foreach scriptFile {foo.tcl bar.tcl} { source $scriptFile }
file(n), cd(n), encoding(n), info(n)
file, script
GITLINT,(1)                       User Commands                      GITLINT,(1)
gitlint, - Git commit message linter
gitlint [OPTIONS] COMMAND [ARGS]...
Git lint tool, checks your git commit messages for styling issues
Documentation: http://jorisroovers.github.io/gitlint
--target DIRECTORY Path of the target git repository. [default: current working directory]
-C, --config FILE Config file location [default: .gitlint]
-c TEXT Config flags in format <rule>.<option>=<value> (e.g.: -c T1.line-length=80). Flag can be used multiple times to set multiple config values.
--commits TEXT The range of commits to lint. [default: HEAD]
-e, --extra-path PATH Path to a directory or python module with extra user-defined rules
--ignore TEXT Ignore rules (comma-separated by id or name).
--contrib TEXT Contrib rules to enable (comma-separated by id or name).
--msg-filename FILENAME Path to a file containing a commit-msg.
--ignore-stdin Ignore any stdin data. Useful for running in CI server.
--staged Read staged commit meta-info from the local repository.
-v, --verbose Verbosity, more v's for more verbose output (e.g.: -v, -vv, -vvv). [default: -vvv]
-s, --silent Silent mode (no output). Takes precedence over -v, -vv, -vvv.
-d, --debug Enable debugging output.
--version Show the version and exit.
--help Show this message and exit.
Commands: generate-config Generates a sample gitlint config file.
install-hook Install gitlint as a git commit-msg hook.
lint   Lints a git repository [default command]
run-hook Runs the gitlint commit-msg hook.
uninstall-hook Uninstall gitlint commit-msg hook.
When no COMMAND is specified, gitlint defaults to 'gitlint lint'.
The full documentation for gitlint, is maintained as a Texinfo manual. If the info and gitlint, programs are properly installed at your site, the command
info gitlint,
SM(6)                             Games Manual                             SM(6)
sm - Displays a short text fullscreen
sm [ OPTIONS ] [ text | - ]
Screen Message will display a given multi-line message as large as possible, fullscreen and black on white. You can specify the text either when launching sm, or edit it while the program is running.
After a short timeout, the text entry and the quit button will disappear, leaving nothing on the screen but the entered text. To continue entering text, just start typing or (left-)click anywhere on the screen.
To clear the displayed text, press Escape.
To invert the colors of the text and the background, press Ctrl-I.
To quit the program, press Ctrl-Q, or Escape twice, or click the "Quit"-button.
[ text | - ] Text to display at start up. Defaults to ":-)". If "-" is passed to sm, it will read the text to display from the standard input, see REMOTE CONTROLLING SM.
-f, --foreground=colordesc Define a different color to use for the foreground of the text to display than black. The text string can be in any of the forms accepted by XParseColor; these include name for a color from rgb.txt, such as DarkSlateGray, or a hex specification such as #3050b2 or #35b.
-b, --background=colordesc Define a different color to use for the background of the text to display than white. For possible values, see above.
-i, --invert Switch the roles for foreground and background colors. Useful if you prefer white-on-black.
-n, --font=fontspec Define a different font to use than the default sans-serif font of your system. The fontspec be the complete name for a truetype font (like "DejaVu Sans" or "Bitstream Vera Serif") or just a short font family specification ("serif", "sans-serif").
-r, --rotate=rotation Rotates the display by rotation*90 degrees counter-clock-wise. So -r 1 rotates the display to the left, and -r 2 puts it upside down.
-a, --align=alignment Aligns the text centered (-a 0), left (-a 1) or right (-a 2).
--     (Double dash) End option parsing. This is used to be able to actually hand over text that starts of with an dash.
-h, --help This option will give you a short usage message summarizing the recognized options and quits.
-V, --version This prints the project name together with its version number quits.
If sm is called with - as a command line argument, it will read the text to be shown from the standard input. It will read the input until it reaches the end of the file, or the form feed character \f, and show the input read so far at once. Newline characters at the beginning or the end are ignored. The input is assumed to be UTF-8 encoded (see utf8(7)).
This can be used to create automatic displays of changing data or similar tricks. For example, the following command will create a simple digital watch:
(while sleep 1; do date +%T; printf '\f'; done) | sm -
This manual page was written by Joachim Breitner <mail@joachim- breitner.de> and updated by Gerfried Fuchs <rhonda@deb.at> to reflect additions for commandline option handling.
PIPENV(1)                            pipenv                            PIPENV(1)
pipenv - pipenv Documentation
Pipenv — the officially recommended Python packaging tool from Python.org, free (as in freedom).
Pipenv is a tool that aims to bring the best of all packaging worlds (bundler, composer, npm, cargo, yarn, etc.) to the Python world. Windows is a first–class citizen, in our world.
It automatically creates and manages a virtualenv for your projects, as well as adds/removes packages from your Pipfile as you install/uninstall packages. It also generates the ever–important Pipfile.lock, which is used to produce deterministic builds.
The problems that Pipenv seeks to solve are multi-faceted:
• You no longer need to use pip and virtualenv separately. They work together.
• Managing a requirements.txt file can be problematic, so Pipenv uses the upcoming Pipfile and Pipfile.lock instead, which is superior for basic use cases.
• Hashes are used everywhere, always. Security. Automatically expose security vulnerabilities.
• Give you insight into your dependency graph (e.g. $ pipenv graph).
• Streamline development workflow by loading .env files.
INSTALL PIPENV TODAY! $ pip install pipenv ✨🍰✨
If you have excellent taste, there's also a  fancy installation method.
Pipenv & Virtual Environments [image]
This tutorial walks you through installing and using Python packages.
It will show you how to install and use the necessary tools and make strong recommendations on best practices. Keep in mind that Python is used for a great many different purposes, and precisely how you want to manage your dependencies may change based on how you decide to publish your software. The guidance presented here is most directly applicable to the development and deployment of network services (including web applications), but is also very well suited to managing development and testing environments for any kind of project.
NOTE: This guide is written for Python 3, however, these instructions should work fine on Python 2.7—if you are still using it, for some reason.
☤ Make sure you've got Python & pip Before you go any further, make sure you have Python and that it's available from your command line. You can check this by simply running:
$ python --version
You should get some output like 3.6.2. If you do not have Python, please install the latest 3.x version from python.org or refer to the Installing Python section of The Hitchhiker's Guide to Python.
NOTE: If you're newcomer and you get an error like this:
>>> python Traceback (most recent call last): File "<stdin>", line 1, in <module> NameError: name 'python' is not defined
It's because this command is intended to be run in a shell (also called a terminal or console). See the Python for Beginners getting started tutorial for an introduction to using your operating system's shell and interacting with Python.
Additionally, you'll need to make sure you have pip available. You can check this by running:
$ pip --version pip 9.0.1
If you installed Python from source, with an installer from python.org, or via Homebrew you should already have pip9. If you're on Linux and installed using your OS package manager, you may have to install pip separately.
☤ Installing Pipenv Pipenv is a dependency manager for Python projects. If you're familiar with Node.js' npm or Ruby's bundler, it is similar in spirit to those tools. While pip can install Python packages, Pipenv is recommended as it's a higher-level tool that simplifies dependency management for common use cases.
Use pip to install Pipenv:
$ pip install --user pipenv
NOTE: This does a user installation to prevent breaking any system-wide packages. If pipenv isn't available in your shell after installation, you'll need to add the user base's binary directory to your PATH.
On Linux and macOS you can find the user base binary directory by running python -m site --user-base and adding bin to the end. For example, this will typically print ~/.local (with ~ expanded to the absolute path to your home directory) so you'll need to add ~/.local/bin to your PATH. You can set your PATH permanently by modifying ~/.profile.
On Windows you can find the user base binary directory by running py -m site --user-site and replacing site-packages with Scripts. For example, this could return C:\Users\Username\AppData\Roaming\Python36\site-packages so you would need to set your PATH to include C:\Users\Username\AppData\Roaming\Python36\Scripts. You can set your user PATH permanently in the Control Panel. You may need to log out for the PATH changes to take effect.
☤ Installing packages for your project Pipenv manages dependencies on a per-project basis. To install packages, change into your project's directory (or just an empty directory for this tutorial) and run:
$ cd myproject $ pipenv install requests
Pipenv will install the excellent Requests library and create a Pipfile for you in your project's directory. The Pipfile is used to track which dependencies your project needs in case you need to re-install them, such as when you share your project with others. You should get output similar to this (although the exact paths shown will vary):
Creating a Pipfile for this project... Creating a virtualenv for this project... Using base prefix '/usr/local/Cellar/python3/3.6.2/Frameworks/Python.framework/Versions/3.6' New python executable in ~/.local/share/virtualenvs/tmp-agwWamBd/bin/python3.6 Also creating executable in ~/.local/share/virtualenvs/tmp-agwWamBd/bin/python Installing setuptools, pip, wheel...done.
Virtualenv location: ~/.local/share/virtualenvs/tmp-agwWamBd Installing requests... Collecting requests Using cached requests-2.18.4-py2.py3-none-any.whl Collecting idna<2.7,>=2.5 (from requests) Using cached idna-2.6-py2.py3-none-any.whl Collecting urllib3<1.23,>=1.21.1 (from requests) Using cached urllib3-1.22-py2.py3-none-any.whl Collecting chardet<3.1.0,>=3.0.2 (from requests) Using cached chardet-3.0.4-py2.py3-none-any.whl Collecting certifi>=2017.4.17 (from requests) Using cached certifi-2017.7.27.1-py2.py3-none-any.whl Installing collected packages: idna, urllib3, chardet, certifi, requests Successfully installed certifi-2017.7.27.1 chardet-3.0.4 idna-2.6 requests-2.18.4 urllib3-1.22
Adding requests to Pipfile's [packages]... P.S. You have excellent taste! ✨ 🍰 ✨
☤ Using installed packages Now that Requests is installed you can create a simple main.py file to use it:
response = requests.get('https://httpbin.org/ip')
print('Your IP is {0}'.format(response.json()['origin']))
Then you can run this script using pipenv run:
$ pipenv run python main.py
You should get output similar to this:
Your IP is 8.8.8.8
Using $ pipenv run ensures that your installed packages are available to your script. It's also possible to spawn a new shell that ensures all commands have access to your installed packages with $ pipenv shell.
☤ Next steps Congratulations, you now know how to install and use Python packages! ✨ 🍰 ✨
☤ Fancy Installation of Pipenv To install pipenv in a fancy way, we recommend using pipsi.
Pipsi is a powerful tool which allows you to install Python scripts into isolated virtual environments.
To install pipsi, first run this:
$ curl https://raw.githubusercontent.com/mitsuhiko/pipsi/master/get-pipsi.py | python
Follow the instructions, you'll have to update your PATH.
Then, simply run:
$ pipsi install pew $ pipsi install pipenv
To upgrade pipenv at any time:
$ pipsi upgrade pipenv
This will install both pipenv and pew (one of our dependencies) in an isolated virtualenv, so it doesn't interfere with the rest of your Python installation!
☤ Pragmatic Installation of Pipenv If you have a working installation of pip, and maintain certain "toolchain" type Python modules as global utilities in your user environment, pip user installs allow for installation into your home directory. Note that due to interaction between dependencies, you should limit tools installed in this way to basic building blocks for a Python workflow like virtualenv, pipenv, tox, and similar software.
To install:
For more information see the user installs documentation, but to add the installed cli tools from a pip user install to your path, add the output of:
$ python -c "import site; import os; print(os.path.join(site.USER_BASE, 'bin'))"
$ pip install --user --upgrade pipenv
☤ Crude Installation of Pipenv If you don't even have pip installed, you can use this crude installation method, which will bootstrap your whole system:
$ curl https://raw.githubusercontent.com/kennethreitz/pipenv/master/get-pipenv.py | python
Congratulations, you now have pip and Pipenv installed!
Jannis Leidel, former pip maintainer— Pipenv is the porcelain I always wanted to build for pip9. It fits my brain and mostly replaces virtualenvwrapper and manual pip calls for me. Use it.
Justin Myles Holmes— Pipenv is finally an abstraction meant to engage the mind instead of merely the filesystem.
Isaac Sanders— Pipenv is literally the best thing about my day today. Thanks, Kenneth!
☤ PIPENV FEATURES • Enables truly deterministic builds, while easily specifying only what you want.
• Generates and checks file hashes for locked dependencies.
• Automatically install required Pythons, if pyenv is available.
• Automatically finds your project home, recursively, by looking for a Pipfile.
• Automatically generates a Pipfile, if one doesn't exist.
• Automatically creates a virtualenv in a standard location.
• Automatically adds/removes packages to a Pipfile when they are un/installed.
• Automatically loads .env files, if they exist.
The main commands are install, uninstall, and lock, which generates a Pipfile.lock. These are intended to replace $ pip install usage, as well as manual virtualenv management (to activate a virtualenv, run $ pipenv shell).
Basic Concepts • A virtualenv will automatically be created, when one doesn't exist.
• When no parameters are passed to install, all packages [packages] specified will be installed.
• To initialize a Python 3 virtual environment, run $ pipenv --three.
• To initialize a Python 2 virtual environment, run $ pipenv --two.
• Otherwise, whatever virtualenv defaults to will be the default.
Other Commands • graph will show you a dependency graph, of your installed dependencies.
• shell will spawn a shell with the virtualenv activated.
• run will run a given command from the virtualenv, with any arguments forwarded (e.g. $ pipenv run python).
• check checks for security vulnerabilities and asserts that PEP 508 requirements are being met by the current environment.
Basic Usage of Pipenv [image]
This document covers some of Pipenv's more basic features.
☤ Example Pipfile & Pipfile.lock Here is a simple example of a Pipfile and the resulting Pipfile.lock.
Example Pipfile [[source]] url = "https://pypi.python.org/simple" verify_ssl = true name = "pypi"
[packages] requests = "*"
[dev-packages] pytest = "*"
Example Pipfile.lock { "_meta": { "hash": { "sha256": "8d14434df45e0ef884d6c3f6e8048ba72335637a8631cc44792f52fd20b6f97a" }, "host-environment-markers": { "implementation_name": "cpython", "implementation_version": "3.6.1", "os_name": "posix", "platform_machine": "x86_64", "platform_python_implementation": "CPython", "platform_release": "16.7.0", "platform_system": "Darwin", "platform_version": "Darwin Kernel Version 16.7.0: Thu Jun 15 17:36:27 PDT 2017; root:xnu-3789.70.16~2/RELEASE_X86_64", "python_full_version": "3.6.1", "python_version": "3.6", "sys_platform": "darwin" }, "pipfile-spec": 5, "requires": {}, "sources": [ { "name": "pypi", "url": "https://pypi.python.org/simple", "verify_ssl": true } ] }, "default": { "certifi": { "hashes": [ "sha256:54a07c09c586b0e4c619f02a5e94e36619da8e2b053e20f594348c0611803704", "sha256:40523d2efb60523e113b44602298f0960e900388cf3bb6043f645cf57ea9e3f5" ], "version": "==2017.7.27.1" }, "chardet": { "hashes": [ "sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691", "sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae" ], "version": "==3.0.4" }, "idna": { "hashes": [ "sha256:8c7309c718f94b3a625cb648ace320157ad16ff131ae0af362c9f21b80ef6ec4", "sha256:2c6a5de3089009e3da7c5dde64a141dbc8551d5b7f6cf4ed7c2568d0cc520a8f" ], "version": "==2.6" }, "requests": { "hashes": [ "sha256:6a1b267aa90cac58ac3a765d067950e7dbbf75b1da07e895d1f594193a40a38b", "sha256:9c443e7324ba5b85070c4a818ade28bfabedf16ea10206da1132edaa6dda237e" ], "version": "==2.18.4" }, "urllib3": { "hashes": [ "sha256:06330f386d6e4b195fbfc736b297f58c5a892e4440e54d294d7004e3a9bbea1b", "sha256:cc44da8e1145637334317feebd728bd869a35285b93cbb4cca2577da7e62db4f" ], "version": "==1.22" } }, "develop": { "py": { "hashes": [ "sha256:2ccb79b01769d99115aa600d7eed99f524bf752bba8f041dc1c184853514655a", "sha256:0f2d585d22050e90c7d293b6451c83db097df77871974d90efd5a30dc12fcde3" ], "version": "==1.4.34" }, "pytest": { "hashes": [ "sha256:b84f554f8ddc23add65c411bf112b2d88e2489fd45f753b1cae5936358bdf314", "sha256:f46e49e0340a532764991c498244a60e3a37d7424a532b3ff1a6a7653f1a403a" ], "version": "==3.2.2" } } }
☤ Importing from requirements.txt If you only have a requirements.txt file available when running pipenv install, pipenv will automatically import the contents of this file and create a Pipfile for you.
You can also specify $ pipenv install -r path/to/requirements.txt to import a requirements file.
Note, that when importing a requirements file, they often have version numbers pinned, which you likely won't want in your Pipfile, so you'll have to manually update your Pipfile afterwards to reflect this.
☤ Specifying Versions of a Package To tell pipenv to install a specific version of a library, the usage is simple:
$ pipenv install requests==2.13.0
This will update your Pipfile to reflect this requirement, automatically.
☤ Specifying Versions of Python To create a new virtualenv, using a specific version of Python you have installed (and on your PATH), use the --python VERSION flag, like so:
Use Python 3:
$ pipenv --python 3
Use Python3.6:
$ pipenv --python 3.6
Use Python 2.7.14:
$ pipenv --python 2.7.14
When given a Python version, like this, Pipenv will automatically scan your system for a Python that matches that given version.
If a Pipfile hasn't been created yet, one will be created for you, that looks like this:
[[source]] url = "https://pypi.python.org/simple" verify_ssl = true
[dev-packages]
[requires] python_version = "3.6"
Note the inclusion of [requires] python_version = "3.6". This specifies that your application requires this version of Python, and will be used automatically when running pipenv install against this Pipfile in the future (e.g. on other machines). If this is not true, feel free to simply remove this section.
If you don't specify a Python version on the command–line, either the [requires] python_full_version or python_version will be selected automatically, falling back to whatever your system's default python installation is, at time of execution.
☤ Editable Dependencies (e.g. -e . ) You can tell Pipenv to install a path as editable — often this is useful for the current working directory when working on packages:
$ pipenv install '-e .' --dev
$ cat Pipfile [dev-packages] "e1839a8" = {path = ".", editable = true}
Note that all sub-dependencies will get added to the Pipfile.lock as well.
☤ Environment Management with Pipenv The three primary commands you'll use in managing your pipenv environment are $ pipenv install, $ pipenv uninstall, and $ pipenv lock.
$ pipenv install $ pipenv install is used for installing packages into the pipenv virtual environment and updating your Pipfile.
Along with the basic install command, which takes the form:
$ pipenv install [package names]
The user can provide these additional parameters:
• --two — Performs the installation in a virtualenv using the system python2 link.
• --three — Performs the installation in a virtualenv using the system python3 link.
• --python — Performs the installation in a virtualenv using the provided Python interpreter.
WARNING: None of the above commands should be used together. They are also destructive and will delete your current virtualenv before replacing it with an appropriately versioned one.
NOTE: The virtualenv created by Pipenv may be different from what you were expecting.  Dangerous characters (i.e. $`!*@" as well as space, line feed, carriage return, and tab) are converted to underscores. Additionally, the full path to the current folder is encoded into a "slug value" and appended to ensure the virtualenv name is unique.
• --dev — Install both develop and default packages from Pipfile.lock.
• --system — Use the system pip command rather than the one from your virtualenv.
• --ignore-pipfile — Ignore the Pipfile and install from the Pipfile.lock.
• --skip-lock — Ignore the Pipfile.lock and install from the Pipfile. In addition, do not write out a Pipfile.lock reflecting changes to the Pipfile.
$ pipenv uninstall $ pipenv uninstall supports all of the parameters in pipenv install, as well as one additional, --all.
• --all — This parameter will purge all files from the virtual environment, but leave the Pipfile untouched.
$ pipenv lock $ pipenv lock is used to create a Pipfile.lock, which declares all dependencies (and sub-dependencies) of your project, their latest available versions, and the current hashes for the downloaded files. This ensures repeatable, and most importantly deterministic, builds.
☤ About Shell Configuration Shells are typically misconfigured for subshell use, so $ pipenv shell --fancy may produce unexpected results. If this is the case, try $ pipenv shell, which uses "compatibility mode", and will attempt to spawn a subshell despite misconfiguration.
A proper shell configuration only sets environment variables like PATH during a login session, not during every subshell spawn (as they are typically configured to do). In fish, this looks like this:
if status --is-login set -gx PATH /usr/local/bin $PATH end
You should do this for your shell too, in your ~/.profile or ~/.bashrc or wherever appropriate.
NOTE: The shell launched in interactive mode. This means that if your shell reads its configuration from a specific file for interactive mode (e.g. bash by default looks for a ~/.bashrc configuration file for interactive mode), then you'll need to modify (or create) this file.
☤ A Note about VCS Dependencies Pipenv will resolve the sub–dependencies of VCS dependencies, but only if they are editable, like so:
[packages] requests = {git = "https://github.com/requests/requests.git", editable=true}
If editable is not true, sub–dependencies will not get resolved.
☤ Pipfile.lock Security Features Pipfile.lock takes advantage of some great new security improvements in pip.  By default, the Pipfile.lock will be generated with the sha256 hashes of each downloaded package. This will allow pip to guarantee you're installing what you intend to when on a compromised network, or downloading dependencies from an untrusted PyPI endpoint.
We highly recommend approaching deployments with promoting projects from a development environment into production. You can use pipenv lock to compile your dependencies on your development environment and deploy the compiled Pipfile.lock to all of your production environments for reproducible builds.
Advanced Usage of Pipenv [image]
This document covers some of Pipenv's more glorious and advanced features.
☤ Specifying Package Indexes If you'd like a specific package to be installed with a specific package index, you can do the following:
[[source]] url = "https://pypi.python.org/simple" verify_ssl = true name = "pypi"
[[source]] url = "http://pypi.home.kennethreitz.org/simple" verify_ssl = false name = "home"
[packages] requests = {version="*", index="home"} maya = {version="*", index="pypi"} records = "*"
Very fancy.
☤ Specifying Basically Anything If you'd like to specify that a specific package only be installed on certain systems, you can use PEP 508 specifiers to accomplish this.
Here's an example Pipfile, which will only install pywinusb on Windows systems:
[packages] requests = "*" pywinusb = {version = "*", os_name = "== 'windows'"}
Here's a more complex example:
[packages] unittest2 = {version = ">=1.0,<3.0", markers="python_version < '2.7.9' or (python_version >= '3.0' and python_version < '3.4')"}
Magic. Pure, unadulterated magic.
☤ Deploying System Dependencies You can tell Pipenv to install things into its parent system with the --system flag:
$ pipenv install --system
This is useful for Docker containers, and deployment infrastructure (e.g. Heroku does this).
Also useful for deployment is the --deploy flag:
$ pipenv install --system --deploy
This will fail a build if the Pipfile.lock is out–of–date, instead of generating a new one.
☤ Generating a requirements.txt You can convert a Pipfile and Pipenv.lock into a requirements.txt file very easily, and get all the benefits of hashes, extras, and other goodies we have included.
Let's take this Pipfile:
[packages] requests = {version="*"}
And generate a requirements.txt out of it:
$ pipenv lock -r chardet==3.0.4 --hash=sha256:fc323ffcaeaed0e0a02bf4d117757b98aed530d9ed4531e3e15460124c106691  --hash=sha256:84ab92ed1c4d4f16916e05906b6b75a6c0fb5db821cc65e70cbd64a3e2a5eaae requests==2.18.4 --hash=sha256:6a1b267aa90cac58ac3a765d067950e7dbbf75b1da07e895d1f594193a40a38b  --hash=sha256:9c443e7324ba5b85070c4a818ade28bfabedf16ea10206da1132edaa6dda237e certifi==2017.7.27.1 --hash=sha256:54a07c09c586b0e4c619f02a5e94e36619da8e2b053e20f594348c0611803704  --hash=sha256:40523d2efb60523e113b44602298f0960e900388cf3bb6043f645cf57ea9e3f5 idna==2.6 --hash=sha256:8c7309c718f94b3a625cb648ace320157ad16ff131ae0af362c9f21b80ef6ec4  --hash=sha256:2c6a5de3089009e3da7c5dde64a141dbc8551d5b7f6cf4ed7c2568d0cc520a8f urllib3==1.22 --hash=sha256:06330f386d6e4b195fbfc736b297f58c5a892e4440e54d294d7004e3a9bbea1b  --hash=sha256:cc44da8e1145637334317feebd728bd869a35285b93cbb4cca2577da7e62db4f
☤ Detection of Security Vulnerabilities Pipenv includes the safety package, and will use it to scan your dependency graph for known security vulnerabilities!
$ cat Pipfile [packages] django = "==1.10.1"
$ pipenv check Checking PEP 508 requirements… Passed! Checking installed package safety…
33075: django >=1.10,<1.10.3 resolved (1.10.1 installed)! Django before 1.8.x before 1.8.16, 1.9.x before 1.9.11, and 1.10.x before 1.10.3, when settings.DEBUG is True, allow remote attackers to conduct DNS rebinding attacks by leveraging failure to validate the HTTP Host header against settings.ALLOWED_HOSTS.
33076: django >=1.10,<1.10.3 resolved (1.10.1 installed)! Django 1.8.x before 1.8.16, 1.9.x before 1.9.11, and 1.10.x before 1.10.3 use a hardcoded password for a temporary database user created when running tests with an Oracle database, which makes it easier for remote attackers to obtain access to the database server by leveraging failure to manually specify a password in the database settings TEST dictionary.
33300: django >=1.10,<1.10.7 resolved (1.10.1 installed)! CVE-2017-7233: Open redirect and possible XSS attack via user-supplied numeric redirect URLs ============================================================================================
Django relies on user input in some cases  (e.g. :func:`django.contrib.auth.views.login` and :doc:`i18n </topics/i18n/index>`) to redirect the user to an "on success" URL. The security check for these redirects (namely ``django.utils.http.is_safe_url()``) considered some numeric URLs (e.g. ``http:999999999``) "safe" when they shouldn't be.
Also, if a developer relies on ``is_safe_url()`` to provide safe redirect targets and puts such a URL into a link, they could suffer from an XSS attack.
CVE-2017-7234: Open redirect vulnerability in ``django.views.static.serve()`` =============================================================================
A maliciously crafted URL to a Django site using the :func:`~django.views.static.serve` view could redirect to any other domain. The view no longer does any redirects as they don't provide any known, useful functionality.
Note, however, that this view has always carried a warning that it is not hardened for production use and should be used only as a development aid.
✨🍰✨
☤ Code Style Checking Pipenv has Flake 8 built into it. You can check the style of your code like so, without installing anything:
$ cat t.py import requests
$ pipenv check --style t.py t.py:1:1: F401 'requests' imported but unused t.py:1:16: W292 no newline at end of file
Super useful :)
☤ Open a Module in Your Editor Pipenv allows you to open any Python module that is installed (including ones in your codebase), with the $ pipenv open command:
$ pipenv install -e git+https://github.com/kennethreitz/background.git#egg=background Installing -e git+https://github.com/kennethreitz/background.git#egg=background… ... Updated Pipfile.lock!
$ pipenv open background Opening '/Users/kennethreitz/.local/share/virtualenvs/hmm-mGOawwm_/src/background/background.py' in your EDITOR.
This allows you to easily read the code you're consuming, instead of looking it up on GitHub.
NOTE: The standard EDITOR environment variable is used for this. If you're using Sublime Text, for example, you'll want to export EDITOR=subl (once you've installed the command-line utility).
☤ Automatic Python Installation If you have pyenv installed and configured, Pipenv will automatically ask you if you want to install a required version of Python if you don't already have it available.
This is a very fancy feature, and we're very proud of it:
$ cat Pipfile [[source]] url = "https://pypi.python.org/simple" verify_ssl = true
$ pipenv install Warning: Python 3.6 was not found on your system… Would you like us to install latest CPython 3.6 with pyenv? [Y/n]: y Installing CPython 3.6.2 with pyenv (this may take a few minutes)… ... Making Python installation global… Creating a virtualenv for this project… Using /Users/kennethreitz/.pyenv/shims/python3 to create virtualenv… ... No package provided, installing all dependencies. ... Installing dependencies from Pipfile.lock… 🐍   ❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒❒ 5/5 — 00:00:03 To activate this project's virtualenv, run the following: $ pipenv shell
Pipenv automatically honors both the python_full_version and python_version PEP 508 specifiers.
💫✨🍰✨💫
☤ Automatic Loading of .env If a .env file is present in your project, $ pipenv shell and $ pipenv run will automatically load it, for you:
$ cat .env HELLO=WORLD⏎
$ pipenv run python Loading .env environment variables… Python 2.7.13 (default, Jul 18 2017, 09:17:00) [GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.42)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import os >>> os.environ['HELLO'] 'WORLD'
This is very useful for keeping production credentials out of your codebase.  We do not recommend committing .env files into source control!