# Python Programming/Print version

The current, editable version of this book is available in Wikibooks, the open-content textbooks collection, at
https://en.wikibooks.org/wiki/Python_Programming

# Overview

Python is a high-level, structured, open-source programming language that can be used for a wide variety of programming tasks. Python was created by Guido Van Rossum in the early 1990s; its following has grown steadily and interest has increased markedly in the last few years or so. It is named after Monty Python's Flying Circus comedy program.
Python is used extensively for system administration (many vital components of Linux distributions are written in it); also, it is a great language to teach programming to novices. NASA has used Python for its software systems and has adopted it as the standard scripting language for its Integrated Planning System. Python is also extensively used by Google to implement many components of its Web Crawler and Search Engine & Yahoo! for managing its discussion groups.
Python within itself is an interpreted programming language that is automatically compiled into bytecode before execution (the bytecode is then normally saved to disk, just as automatically, so that compilation need not happen again until and unless the source gets changed). It is also a dynamically typed language that includes (but does not require one to use) object-oriented features and constructs.
The most unusual aspect of Python is that whitespace is significant; instead of block delimiters (braces → "" in the C family of languages), indentation is used to indicate where blocks begin and end.
For example, the following Python code can be interactively typed at an interpreter prompt, display the famous "Hello World!" on the user screen:
Another great feature of Python is its availability for all platforms. Python can run on Microsoft Windows, Macintosh and all Linux distributions with ease. This makes the programs very portable, as any program written for one platform can easily be used on another.
Python provides a powerful assortment of built-in types (e.g., lists, dictionaries and strings), a number of built-in functions, and a few constructs, mostly statements. For example, loop constructs that can iterate over items in a collection instead of being limited to a simple range of integer values. Python also comes with a powerful standard library, which includes hundreds of modules to provide routines for a wide variety of services including regular expressions and TCP/IP sessions.
Python is used and supported by a large Python Community that exists on the Internet. The mailing lists and news groups like the tutor list actively support and help new python programmers. While they discourage doing homework for you, they are quite helpful and are populated by the authors of many of the Python textbooks currently available on the market.
Python 2 vs. Python 3: Years ago, the Python developers made the decision to come up with a major new version of Python, which became the 3.x series of versions. The 3.x versions are backward-incompatible with Python 2.x: certain old features (like the handling of Unicode strings) were deemed to be too unwieldy or broken to be worth carrying forward. Instead, new, cleaner ways of achieving the same results were added. See also Python 2 vs. Python 3 chapter.

# Getting Python

To program in Python, you need a Python interpreter to run your code—we will discuss interpreters later. If it's not already installed, or if the version you are using is obsolete, you will need to obtain and install Python using the methods below. The current Python versions are 3.x; versions 2.x are discontinued and no longer maintained.

## Installing Python in Windows

Go to the Python Homepage and get the proper version for your platform. Download it, read the instructions and get it installed.
To run Python from the command line, you will need to have the python directory in your PATH. You can instruct the Python installer to add Python to the path, but if you do not do that, you can add it manually. The PATH variable can be modified from the Window's System control panel. To expand the PATH in Windows 7:
* Go to Start.
* Right click on computer.
* Click on properties.
* Click on 'Advanced System Settings'
* Click on 'Environmental Variables'.
* In the system variables select Path and edit it, by appending a ';' (without quote) and adding 'C:\python27'(without quote).
If you prefer having a temporary environment, you can create a new command prompt short-cut that automatically executes the following statement:
If you downloaded a different version (such as Python 3.1), change the "27" for the version of Python you have (27 is 2.7.x, the current version of Python 2.)

### Cygwin

By default, the Cygwin installer for Windows does not include Python in the downloads. However, it can be selected from the list of packages.

## Installing Python on Mac

Users on Mac OS X will find that it already ships with Python 2.3 (OS X 10.4 Tiger) or Python 2.6.1 (OS X Snow Leopard), but if you want the more recent version head to Python Download Page follow the instruction on the page and in the installers. As a bonus you will also install the Python IDE.

## Installing Python on Unix environments

Python is available as a package for most Linux distributions. In some cases, the distribution CD will contain the python package for installation, while other distributions require downloading the source code and using the compilation scripts.

### Gentoo Linux

Gentoo includes Python by default—the package management system Portage depends on Python.

### Ubuntu Linux

Users of Ubuntu will notice that Python comes installed by default, only it sometimes is not the latest version. To check which version of Python is installed, type
into the terminal.

### Arch Linux

Arch Linux does not come with Python pre-installed by default, but it is easily available for installation through the package manager to pacman. As root (or using sudo if you've installed and configured it), type:
This will be update package databases and install Python 3. Python 2 can be installed with:
Other versions can be built from source from the Arch User Repository.

### Source code installations

Some platforms do not have a version of Python installed, and do not have pre-compiled binaries. In these cases, you will need to download the source code from the official site. Once the download is complete, you will need to unpack the compressed archive into a folder.
To build Python, simply run the configure script (requires the Bash shell) and compile using make.

### Other Distributions

Python, also referred to as CPython to avoid confusion, is written in the C programming language, and is the official reference implementation. CPython can run on various platforms due to its portability.
Apart from CPython there are also other implementations that run on top of a virtual machine. For example, on Java's JRE (Java Runtime Environment) or Microsoft's .NET CLR (Common Language Runtime). Both can access and use the libraries available on their platform. Specifically, they make use of reflection that allows complete inspection and use of all classes and objects for their very technology.
Python Implementations (Platforms)
| Environment | Description | Get From |
| Jython | Java Version of Python | Jython |
| IronPython | C# Version of Python | IronPython |

### Integrated Development Environments (IDE)

It's common to use a simple text editor for writing Python code, but you may feel the need to upgrade to a more advanced IDE. CPython ships with IDLE; however, IDLE is not considered user-friendly. For Linux, KDevelop and Spyder are popular. For Windows, PyScripter is free, quick to install, and comes included with PortablePython.
Some Integrated Development Environments (IDEs) for Python
| Environment | Description | Get From |
| ActivePython | Highly flexible, Pythonwin IDE | ActivePython |
| Anjuta | IDE Linux/Unix | Anjuta |
| Eclipse (PyDev plugin) | Open-source IDE | Eclipse |
| Eric | Open-source Linux/Windows IDE. | Eric |
| KDevelop | Cross-language IDE for KDE | KDevelop |
| Ninja-IDE | Cross-platform open-source IDE. | Nina-IDE |
| PyScripter | Free Windows IDE (portable) | PyScripter |
| Pythonwin | Windows-oriented environment | Pythonwin |
| Spyder | Free cross-platform IDE (math-oriented) | Spyder |
| VisualWx | Free GUI Builder | VisualWx |
The Python official wiki has a complete list of IDEs.
There are several commercial IDEs such as Komodo, BlackAdder, Code Crusader, Code Forge, and PyCharm. However, for beginners learning to program, purchasing a commercial IDE is unnecessary.

## Trying Python online

You can try Python online, thereby avoiding the need to install. The online Python shell at Python's official site provides a web Python REPL (read–eval–print loop).

## Keeping Up to Date

Python has a very active community and the language itself is evolving continuously. Make sure to check python.org for recent releases and relevant tools. The website is an invaluable asset.
Public Python-related mailing lists are hosted at mail.python.org. Two examples of such mailing lists are the Python-announce-list to keep up with newly released third party-modules or software for Python and the general discussion list Python-list. These lists are mirrored to the Usenet newsgroups comp.lang.python.announce & comp.lang.python.

## Notes

* ↑ The Things I Hate About IDLE That I Wish Someone Would Fix .

# Development Environments

You can edit Python source code files in any plain text editor such as Notepad, Notepad++ or Vim. However, there are multiple integrated development environments (IDEs) available for Python. A full list can be found on the Python wiki.

## Visual Studio Code

There is a Python extension for Visual Studio Code, a popular cross-platform code editor.
Links:
* Python in Visual Studio Code, code.visualstudio.com

## Eclipse

You can use the Eclipse IDE as your Python IDE. The only requirement is Eclipse and the Eclipse PyDev Plug-in.
Go to http://www.eclipse.org/downloads/ and get the proper Eclipse IDE version for your OS platform. There are also updates on the site, but just look for the basic program, Download and install it. The install just requires you to unpack the downloaded Eclipse install file onto your system.
You can install PyDev Plug-in two ways:
* Suggested: Use Eclipse's update manager, found in the tool bar under "Help" -> "install new Software". add http://pydev.org/updates/ in "work with" click add, and select PyDev ,and click "Next" and let Eclipse do the rest. Eclipse will now check for any updates to PyDev when it searches for updates.
** If you get an error stating a requirement for the plugin "org.eclipse.mylyn", expand the PyDev tree, and deselect the optional mylyn components.
* Or install PyDev manually, by going to http://pydev.sourceforge.net and get the latest PyDev Plug-in version. Download it, and install it by unpacking it into the Eclipse base folder.

## Emacs

There is a Python mode for Emacs which provides features such as running pieces of code, and changing the tab level for blocks.
Links:
* python-mode, gitlab.com

## External links

* Integrated Development Environments, wiki.python.org

# Interactive mode

Python has two basic modes: script and interactive. The normal mode is the mode where the scripted and finished .py files are run in the Python interpreter. Interactive mode is a command line shell which gives immediate feedback for each statement, while running previously fed statements in active memory. As new lines are fed into the interpreter, the fed program is evaluated both in part and in whole.
Interactive mode is a good way to play around and try variations on syntax.
On macOS or linux, open a terminal and simply type "python". On Windows, bring up the command prompt and type "py", or start an interactive Python session by selecting "Python (command line)", "IDLE", or similar program from the task bar / app menu. IDLE is a GUI which includes both an interactive mode and options to edit and run files.
Python should print something like this:
(If Python doesn't run, make sure it is installed and your path is set correctly. See Getting Python.)
The >>> is Python's way of telling you that you are in interactive mode. In interactive mode what you type is immediately run. Try typing 1+1 in. Python will respond with 2. Interactive mode allows you to test out and see what Python will do. If you ever feel the need to play with new Python statements, go into interactive mode and try them out.
A sample interactive session:
However, you need to be careful in the interactive environment to avoid confusion. For example, the following is a valid Python script:
If you try to enter this as written in the interactive environment, you might be surprised by the result:
What the interpreter is saying is that the indentation of the second print was unexpected. You should have entered a blank line to end the first (i.e., "if") statement, before you started writing the next print statement. For example, you should have entered the statements as though they were written:
Which would have resulted in the following:

### Interactive mode

Instead of Python exiting when the program is finished, you can use the -i flag to start an interactive session. This can be very useful for debugging and prototyping.
For i in range(-1,-5,-1):

# Self Help

| Previous: Interactive mode | Index | Next: Creating Python programs |
This book is useful for learning Python, but there might be a topic that the book does not cover. You might want to search for modules in the standard library, or inspect an unknown object's functions, or perhaps you know there is a function that you have to call inside an object but you don't know its name. This is where the interactive help comes into play.

## Built-in help overview

Built-in interactive help at a glance:

## Navigating Help

To start Python's interactive help, type "help" at the prompt.
You will be presented with a greeting and a quick introduction to the help system. For Python 2.6, the prompt will look something like this:
Notice also that the prompt will change from ">>>" (three right angle brackets) to "help>" You can access the different portions of help simply by typing in modules, keywords, or topics.
Typing in the name of one of these will print the help page associated with the item in question. To get a list of available modules, keywords, or topics, type "modules","keywords", or "topics". Each module also comes with a one-line summary of what it does; to list the modules whose summaries contain a given word such as "spam", type "modules spam".
You can exit the help system by typing "quit" or by entering a blank line to return to the interpreter.

## Help Parameter

You can obtain information on a specific command without entering interactive help.
For example, you can obtain help on a given topic simply by adding a string in quotes, such as help("object"). You may also obtain help on a given object as well, by passing it as a parameter to the help function.

## External links

* help in 2. Built-in Functions, docs.python.org
| Previous: Interactive mode | Index | Next: Creating Python programs |

# Creating Python programs

| Previous: Self Help | Index | Next: Variables and Strings |
Welcome to Python! This tutorial will show you how to start writing programs.
Python programs are nothing more than text files, and they may be edited with a standard text editor program. What text editor you use will probably depend on your operating system: any text editor can create Python programs. However, it is easier to use a text editor that includes Python syntax highlighting.

## Hello, World

The very first program that beginning programmers usually write or learn is the "Hello, World!" program. This program simply outputs the phrase "Hello, World!" then terminates itself. Let's write "Hello, World!" in Python!
Open up your text editor and create a new file called hello.py containing just this line (you can copy-paste if you want):
The below line is used for Python 3.x.x
You can also put the below line to pause the program at the end until you press anything.
This program uses the print function, which simply outputs its parameters to the terminal. By default, print appends a newline character to its output, which simply moves the cursor to the next line.
Now that you've written your first program, let's run it in Python! This process differs slightly depending on your operating system.

### Windows

* Create a folder on your computer to use for your Python programs, such as C:\pythonpractice, and save your hello.py program in that folder.
* In the Start menu, select "Run...", and type in cmd. This will cause the Windows terminal to open.
* Type cd \pythonpractice to change directory to your pythonpractice folder, and hit Enter.
* Type hello.py to run your program!
If it didn't work, make sure your PATH contains the python directory. See Getting Python.

### Mac

* Create a folder on your computer to use for your Python programs. A good suggestion would be to name it pythonpractice and place it in your Home folder (the one that contains folders for Documents, Movies, Music, Pictures, etc). Save your hello.py program into it. Open the Applications folder, go into the Utilities folder, and open the Terminal program.
* Type cd pythonpractice to change directory to your pythonpractice folder, and hit Enter.
* Type python ./hello.py to run your program!

### Linux

* Create a folder on your computer to use for your Python programs, such as ~/pythonpractice, and save your hello.py program in that folder.
* Open up the terminal program. In KDE, open the main menu and select "Run Command..." to open Konsole. In GNOME, open the main menu, open the Applications folder, open the Accessories folder, and select Terminal.
* Type cd ~/pythonpractice to change directory to your pythonpractice folder, and hit Enter.
* Don't forget to make the script executable by chmod +x.
* Type python ./hello.py to run your program!

### Linux (advanced)

* Create a folder on your computer to use for your Python programs, such as ~/pythonpractice.
* Open up your favorite text editor and create a new file called hello.py containing just the following 2 lines (you can copy-paste if you want):
* save your hello.py program in the ~/pythonpractice folder.
* Open up the terminal program. In KDE, open the main menu and select "Run Command..." to open Konsole. In GNOME, open the main menu, open the Applications folder, open the Accessories folder, and select Terminal.
* Type cd ~/pythonpractice to change directory to your pythonpractice folder, and hit Enter.
* Type chmod a+x hello.py to tell Linux that it is an executable program.
* Type ./hello.py to run your program!
* In addition, you can also use ln -s hello.py /usr/bin/hello to make a symbolic link hello.py to /usr/bin under the name hello, then run it by simply executing hello.
Note that this mainly should be done for complete, compiled programs, if you have a script that you made and use frequently, then it might be a good idea to put it somewhere in your home directory and put a link to it in /usr/bin. If you want a playground, a good idea is to invoke mkdir ~/.local/bin and then put scripts in there. To make ~/.local/bin content executable the same way /usr/bin does type $PATH = $PATH:~/local/bin (you can add this line to your shell rc file, for example ~/.bashrc).

### Result

The program should print:
Congratulations! You're well on your way to becoming a Python programmer.

## Exercises

* Modify the hello.py program to say hello to someone from your family or your friends (or to Ada Lovelace).
* Change the program so that after the greeting, it asks, "How did you get here?".
* Re-write the original program to use two print statements: one for "Hello" and one for "world". The program should still only print out on one line.
Solutions

## Notes

* ↑ Sometimes, Python programs are distributed in compiled form. We won't have to worry about that for quite a while.
* ↑ A Quick Introduction to Unix/My First Shell Script explains what a hash bang line does.
| Previous: Self Help | Index | Next: Variables and Strings |

# Variables and Strings

| Previous: Creating Python programs | Index | Next: Basic Syntax |
In this section, you will be introduced to two different kinds of data in Python: variables and strings. Please follow along by running the included programs and examining their output.

## Variables

A variable is something that holds a value that may change. In simplest terms, a variable is just a box that you can put stuff in. You can use variables to store all kinds of stuff, but for now, we are just going to look at storing numbers in variables.
This code creates a variable called lucky, and assigns to it the integer number 7. When we ask Python to tell us what is stored in the variable lucky, it returns that number again.
We can also change what is inside a variable. For example:
We declare a variable called changing, put the integer 3 in it, and verify that the assignment was properly done. Then, we assign the integer 9 to changing, and ask again what is stored in changing. Python has thrown away the 3, and has replaced it with 9. Next, we create a second variable, which we call different, and put 12 in it. Now we have two independent variables, different and changing, that hold different information, i.e., assigning a new value to one of them is not affecting the other.
You can also assign the value of a variable to be the value of another variable. For example:
To understand this code, keep in mind that the name of the variable is always on the left side of the equals sign (the assignment operator), and the value of the variable is on the right side of the equals sign. First the name, then the value.
We start out declaring that red is 5, and blue is 10. As you can see, you can pass several arguments to print to tell it to print multiple items on one line, separating them by spaces. As expected, Python reports that red stores 5, and blue holds 10.
Now we create a third variable, called yellow. To set its value, we tell Python that we want yellow to be whatever red is. (Remember: name to the left, value to the right.) Python knows that red is 5, so it also sets yellow to be 5.
Now we're going to take the red variable, and set it to the value of the blue variable. Don't get confused — name on the left, value on the right. Python looks up the value of blue, and finds that it is 10. So, Python throws away red's old value (5), and replaces it with 10. After this assignment Python reports that yellow is 5, red is 10, and blue is 10.
But didn't we say that yellow should be whatever value red is? The reason that yellow is still 5 when red is 10, is because we only said that yellow should be whatever red is at the moment of the assignment. After Python has figured out what red is and assigned that value to yellow, yellow doesn't care about red any more. yellow has a value now, and that value is going to stay the same no matter what happens to red.
For the name of the variable, it can only consist of uppercase and lowercase letters (A-Z, a-z), digits (0-9), and the underscore character (_), and the first character of the name cannot be a digit. For example, 1abc and _#$ad are not valid variable names, while _123 and a__bc are valid variable names.

## String

A 'string' is simply a list of characters in order. A character is anything you can type on the keyboard in one keystroke, like a letter, a number, or a backslash. For example, "hello" is a string. It is five characters long — h, e, l, l, o. Strings can also have spaces: "hello world" contains 11 characters: 10 letters and the space between "hello" and "world". There are no limits to the number of characters you can have in a string — you can have anywhere from one to a million or more. You can even have a string that has 0 characters, which is usually called an "empty string."
There are three ways you can declare a string in Python: single quotes ('), double quotes ("), and triple quotes ("""). In all cases, you start and end the string with your chosen string declaration. For example:
You can use quotation marks within strings by placing a backslash directly before them, so that Python knows you want to include the quotation marks in the string, instead of ending the string there. Placing a backslash directly before another symbol like this is known as escaping the symbol.
If you want to include a backslash in a string, you have to escape said backslash. This tells Python that you want to include the backslash in the string, instead of using it as an escape character. For example:
As you can see from the above examples, only the specific character used to quote the string needs to be escaped. This makes for more readable code.
To see how to use strings, let's go back for a moment to an old, familiar program:
Look at that! You've been using strings since the very beginning!
You can add two strings together using the + operator: this is called concatenating them.
Notice that there is a space at the end of the first string. If you don't put that in, the two words will run together, and you'll end up with Hello,world!
You can also repeat strings by using the * operator, like so:
The string bouncy gets repeated 5 times in the 1st example and 10 times in the 2nd.
If you want to find out how long a string is, you use the len function, which simply takes a string and counts the number of characters in it. (len stands for "length.") Just put the string that you want to find the length of inside the parentheses of the function. For example:

## Strings and Variables

Now that you've learned about variables and strings separately, let's see how they work together.
Variables can store much more than just numbers. You can also use them to store strings! Here's how:
In this program, we are creating a variable called question, and storing the string "What did you have for lunch?" in it. Then, we just tell Python to print out whatever is inside the question variable. Notice that when we tell Python to print out question, there are no quotation marks around the word question: this tells Python that we are using a variable, not a string. If we put in quotation marks around question, Python would treat it as a string, as shown below:
Let's try something different. Sure, it's all fine and dandy to ask the user what they had for lunch, but it doesn't make much difference if they can't respond! Let's edit this program so that the user can type in what they ate.
To ask the user to write something, we used a function called raw_input, which waits until the user writes something and presses enter, and then returns what the user wrote. Don't forget the parentheses! Even though there's nothing inside of them, they're still important, and Python will give you an error if you don't put them in. You can also use a different function called input, which works in nearly the same way. We will learn the differences between these two functions later.
In this program, we created a variable called answer, and put whatever the user wrote into it. Then, we print out a new string, which contains whatever the user wrote. Notice the extra space at the end of the "You had " string, and the exclamation mark at the start of the "! That sounds delicious!" string. They help format the output and make it look nice, so that the strings don't all run together.

## Combining Numbers and Strings

Take a look at this program, and see if you can figure out what it's supposed to do.
This program should take a number from the user, add 10 to it, and print out the result. But if you try running it, it won't work! You'll get an error that looks like this:
What's going on here? Python is telling us that there is a TypeError, which means there is a problem with the types of information being used. Specifically, Python can't figure out how to reconcile the two types of data that are being used simultaneously: integers and strings. For example, Python thinks that the number variable is holding a string, instead of a number. If the user enters 15, then number will contain a string that is two characters long: a 1, followed by a 5. So how can we tell Python that 15 should be a number, instead of a string?
Also, when printing out the answer, we are telling Python to concatenate together a string ("If we add 10 to your number, we get ") and a number (plusTen). Python doesn't know how to do that -- it can only concatenate strings together. How do we tell Python to treat a number as a string, so that we can print it out with another string?
Luckily, there are two functions that are perfect solutions for these problems. The int function will take a string and turn it into an integer, while the str function will take an integer and turn it into a string. In both cases, we put what we want to change inside the parentheses. Therefore, our modified program will look like this:
That's all you need to know about strings and variables! We'll learn more about types later.

## List of Learned Functions

* print: Print the output information to the user
* input or raw_input: asks the user for a response, and returns that response. (Note that in version 3.x raw_input does not exist and has been replaced by input)
* len: returns the length of a string (number of characters)
* str: returns the string representation of an object
* int: given a string or number, returns an integer

## Exercises

* Write a program that asks the user to type in a string, and then tells the user how long that string was.
* Ask the user for a string, and then for a number. Print out that string, that many times. (For example, if the string is hello and the number is 3 you should print out hello hello hello .)
* What would happen if a mischievous user typed in a word when you ask for a number? Try it.
Solutions
Quiz
| Previous: Creating Python programs | Index | Next: Basic Syntax |

# Basic syntax

| Previous: Variables and Strings | Index | Next: Sequences |
There are five fundamental concepts in Python.

### Semicolons

Python does not normally use semicolons, but they are allowed to separate statements on the same line, if your code has semicolons; your code isn't "Pythonic"

### Case Sensitivity

All variables are case-sensitive. Python treats 'number' and 'Number' as separate, unrelated entities.

### Spaces and tabs don't mix

Instead of block delimiters (braces → "" in the C family of languages), indentation is used to indicate where blocks begin and end. Because whitespace is significant, remember that spaces and tabs don't mix, so use only one or the other when indenting your programs. A common error is to mix them. While they may look the same in editor, the interpreter will read them differently and it will result in either an error or unexpected behavior. Most decent text editors can be configured to let tab key emit spaces instead.
Python's Style Guideline described that the preferred way is using 4 spaces.
Tips: If you invoked python from the command-line, you can give -t or -tt argument to python to make python issue a warning or error on inconsistent tab usage.
This will issue an error if you have mixed spaces and tabs.

### Objects

In Python, like all object-oriented languages, there are aggregations of code and data called objects, which typically represent the pieces in a conceptual model of a system.
Objects in Python are created (i.e., instantiated) from templates called classes (which are covered later, as much of the language can be used without understanding classes). They have attributes, which represent the various pieces of code and data which make up the object. To access attributes, one writes the name of the object followed by a period (henceforth called a dot), followed by the name of the attribute.
An example is the 'upper' attribute of strings, which refers to the code that returns a copy of the string in which all the letters are uppercase. To get to this, it is necessary to have a way to refer to the object (in the following example, the way is the literal string that constructs the object).
Code attributes are called methods. So in this example, upper is a method of 'bob' (as it is of all strings). To execute the code in a method, use a matched pair of parentheses surrounding a comma separated list of whatever arguments the method accepts (upper doesn't accept any arguments). So to find an uppercase version of the string 'bob', one could use the following:

### Scope

In a large system, it is important that one piece of code does not affect another in difficult to predict ways. One of the simplest ways to further this goal is to prevent one programmer's choice of a name from blocking another's use of that name. The concept of scope was invented to do this. A scope is a "region" of code in which a name can be used and outside of which the name cannot be easily accessed. There are two ways of delimiting regions in Python: functions or modules. They each have different ways of providing their content outside of their scope. Functions return data as the result of execution. Modules leads us to another concept, namespace.

### Namespaces

It would be possible to teach Python without the concept of namespaces because they are so similar to attributes, which we have already mentioned, but the concept of namespaces is one that transcends any particular programming language, and so it is important to teach. To begin with, there is a built-in function dir that can be used to help one understand the concept of namespaces. When you first start the Python interpreter (i.e., in interactive mode), you can list the objects in the current (or default) namespace using this function.
This function can also be used to show the names available within a module's namespace. To demonstrate this, first we can use the type function to show what kind of object __builtins__ is:
Since it is a module, it has a namespace. We can list the names within the __builtins__ namespace, again using the dir function (note that the complete list of names has been abbreviated):
Namespaces are a simple concept. A namespace is a particular place in which names specific to a module reside. Each name within a namespace is distinct from names outside of that namespace. This layering of namespaces is called scope. A name is placed within a namespace when that name is given a value. For example:
Note that I was able to add the "name" variable to the namespace using a simple assignment statement. The import statement was used to add the "math" name to the current namespace. To see what math is, we can simply:
Since it is a module, it also has a namespace. To display the names within this namespace, we:
If you look closely, you will notice that both the default namespace and the math module namespace have a '__name__' object. The fact that each layer can contain an object with the same name is what scope is all about. To access objects inside a namespace, simply use the name of the module, followed by a dot, followed by the name of the object. This allows us to differentiate between the __name__ object within the current namespace, and that of the object with the same name within the math module. For example:
| Previous: Variables and Strings | Index | Next: Sequences |

# Sequences

| Previous: Basic syntax | Index | Next: Data types |
Sequences allow you to store multiple values in an organized and efficient fashion. There are seven sequence types: strings, bytes, lists, tuples, bytearrays, buffers, and range objects. Dictionaries and sets are containers for sequential data.

## Strings

We already covered strings, but that was before you knew what a sequence is. In other languages, the elements in arrays and sometimes the characters in strings may be accessed with the square brackets, or subscript operator. This works in Python too:
Indexes are numbered from 0 to n-1 where n is the number of items (or characters), and they are given to the characters from the start of the string:
Negative indexes (numbered from -1 to -n) are counted from the end of the string:
If the number given in the bracket is less than -n or greater than n-1, then we will get an IndexError.
But in Python, the colon : allows the square brackets to take two numbers. For any sequence which only uses numeric indexes, this will return the portion which is between the specified indexes. This is known as "slicing" and the result of slicing a string is often called a "substring."
As demonstrated above, if the first and second number is omitted, then they take their default values, which are 0 and n-1 respectively, corresponding to the beginning and end of the sequence respectively (in this case). Note also that the brackets are inclusive on the left but exclusive on the right: in the first example above with [3:9] the character at index 3, 'l', is included while the character at index 9, 'r', is excluded.
We can add a third number in the bracket, by adding one more colon in the bracket, which indicates the increment step of the slicing:
The increment step can be positive or negative. For positive increment step, the slicing is from left to right, and for negative increment step, the slicing is from right to left. Also, we should aware that
* the default value of the increment step is 1
* the default value for the first and second number becomes -1 (or n) and -n (or 0) respectively, when the increment step is negative (same default value as above when the increment step is positive)
* the increment step cannot be 0

## Lists

A list is just what it sounds like: a list of values, organized in order. A list is created using square brackets. For example, an empty list would be initialized like this:
The values of the list are separated by commas. For example:
Lists may contain objects of varying types. It may hold both the strings "eggs" and "bacon" as well as the number 42.
Like characters in a string, items in a list can be accessed by indexes starting at 0. To access a specific item in a list, you refer to it by the name of the list, followed by the item's number in the list inside brackets. For example:
You can also use negative numbers, which count backwards from the end of the list:
The len function also works on lists, returning the number of items in the array:
Note that the len function counts the number of item inside a list, so the last item in spam (42) has the index (len(spam) - 1).
The items in a list can also be changed, just like the contents of an ordinary variable:
(Strings, being immutable, are impossible to modify.) As with strings, lists may be sliced:
It is also possible to add items to a list. There are many ways to do it, the easiest way is to use the append method of list:
Note that you cannot manually insert an element by specifying the index outside of its range. The following code would fail:
Instead, you must use the insert function. If you want to insert an item inside a list at a certain index, you may use the insert method of list, for example:
You can also delete items from a list using the del statement:
As you can see, the list re-orders itself, so there are no gaps in the numbering.
Lists have an unusual characteristic. Given two lists a and b, if you set b to a, and change a, b will also be changed.
This can easily be worked around by using b=a[:] instead.
For further explanation on lists, or to find out how to make 2D arrays, see Data Structure/Lists

## Tuples

Tuples are similar to lists, except they are immutable. Once you have set a tuple, there is no way to change it whatsoever: you cannot add, change, or remove elements of a tuple. Otherwise, tuples work identically to lists.
To declare a tuple, you use commas:
It is often necessary to use parentheses to differentiate between different tuples, such as when doing multiple assignments on the same line:
Unnecessary parentheses can be used without harm, but nested parentheses denote nested tuples:
both produce:
but:
For further explanation on tuple, see Data Structure/Tuples

## Dictionaries

Dictionaries are also like lists, and they are mutable -- you can add, change, and remove elements from a dictionary. However, the elements in a dictionary are not bound to numbers, the way a list is. Every element in a dictionary has two parts: a key, and a value. Calling a key of a dictionary returns the value linked to that key. You could consider a list to be a special kind of dictionary, in which the key of every element is a number, in numerical order.
Dictionaries are declared using curly braces, and each element is declared first by its key, then a colon, and then its value. For example:
Also, adding an element to a dictionary is much simpler: simply declare it as you would a variable.
For further explanation on dictionary, see Data Structure/Dictionaries

## Sets

Sets are just like lists except that they are unordered and they do not allow duplicate values. Elements of a set are neither bound to a number (like list and tuple) nor to a key (like dictionary). The reason for using a set over other data types is that a set is much faster for a large number of items than a list or tuple and sets provide fast data insertion, deletion, and membership testing. Sets also support mathematical set operations such as testing for subsets and finding the union or intersection of two sets.
Note that sets are unordered, items you add into sets will end up in an indeterminable position, and it may also change from time to time.
Sets cannot contain a single value more than once. Unlike lists, which can contain anything, the types of data that can be included in sets are restricted. A set can only contain hashable, immutable data types. Integers, strings, and tuples are hashable; lists, dictionaries, and other sets (except frozensets, see below) are not.

### Frozenset

The relationship between frozenset and set is like the relationship between tuple and list. Frozenset is an immutable version of set. An example:

## Other data types

Python also has other types of sequences, though these are used less frequently and need to be imported from the standard library before being used. We will only brush over them here.

### array

    A typed-list, an array may only contain homogeneous values.

### collections.defaultdict

    A dictionary that, when an element is not found, returns a default value instead of error.

### collections.deque

    A double ended queue, allows fast manipulation on both sides of the queue.

### heapq

    A priority queue.

### Queue

    A thread-safe multi-producer, multi-consumer queue for use with multi-threaded programs. Note that a list can also be used as queue in a single-threaded code.
For further explanation on set, see Data Structure/Sets

### 3rd party data structure

Some useful data types in Python do not come in the standard library. Some of these are very specialized in their use. We will mention some of the more well known 3rd party types.

#### numpy.array

    useful for heavy number crunching => see numpy section

#### sorteddict

    like the name says, a sorted dictionary

## Exercises

* Write a program that puts 5, 10, and "twenty" into a list. Then remove 10 from the list.
* Write a program that puts 5, 10, and "twenty" into a tuple.
* Write a program that puts 5, 10, and "twenty" into a set. Put "twenty", 10, and 5 into another set purposefully in a different order. Print both of them out and notice the ordering.
* Write a program that constructs a tuple, one element of which is a frozenset.
* Write a program that creates a dictionary mapping 1 to "Monday," 2 to "Tuesday," etc.

## External links

* Sequence Types — list, tuple, range in The Python Library Reference, docs.python.org
| Previous: Basic syntax | Index | Next: Data types |

# Data types

| Previous: Sequences | Index | Next: Numbers |
Data types determine whether an object can do something, or whether it just would not make sense. Other programming languages often determine whether an operation makes sense for an object by making sure the object can never be stored somewhere where the operation will be performed on the object (this type system is called static typing). Python does not do that. Instead it stores the type of an object with the object, and checks when the operation is performed whether that operation makes sense for that object (this is called dynamic typing).

## Built-in Data types

Python's built-in (or standard) data types can be grouped into several classes. Sticking to the hierarchy scheme used in the official Python documentation these are numeric types, sequences, sets and mappings (and a few more not discussed further here). Some of the types are only available in certain versions of the language as noted below.
* boolean: the type of the built-in values True and False. Useful in conditional expressions, and anywhere else you want to represent the truth or falsity of some condition. Mostly interchangeable with the integers 1 and 0. In fact, conditional expressions will accept values of any type, treating special ones like boolean False, integer 0 and the empty string "" as equivalent to False, and all other values as equivalent to True.
Numeric types:
* int: Integers; equivalent to C longs in Python 2.x, non-limited length in Python 3.x
* long: Long integers of non-limited length; exists only in Python 2.x
* float: Floating-Point numbers, equivalent to C doubles
* complex: Complex Numbers
Sequences:
* str: String; represented as a sequence of 8-bit characters in Python 2.x, but as a sequence of Unicode characters (in the range of U+0000 - U+10FFFF) in Python 3.x
* bytes: a sequence of integers in the range of 0-255; only available in Python 3.x
* byte array: like bytes, but mutable (see below); only available in Python 3.x
* list
* tuple
Sets:
* set: an unordered collection of unique objects; available as a standard type since Python 2.6
* frozen set: like set, but immutable (see below); available as a standard type since Python 2.6
Mappings:
* dict: Python dictionaries, also called hashmaps or associative arrays, which means that an element of the list is associated with a definition, rather like a Map in Java
Some others, such as type and callables

## Mutable vs Immutable Objects

In general, data types in Python can be distinguished based on whether objects of the type are mutable or immutable. The content of objects of immutable types cannot be changed after they are created.
| Some immutable types | Some mutable types |
| int, float, complex str bytes tuple frozenset bool | array bytearray list set dict |
Only mutable objects support methods that change the object in place, such as reassignment of a sequence slice, which will work for lists, but raise an error for tuples and strings.
It is important to understand that variables in Python are really just references to objects in memory. If you assign an object to a variable as below,
all you really do is make this variable (a, s, or l) point to the object (1, 'abc', ['a string', 456, ('a', 'tuple', 'inside', 'a', 'list')]), which is kept somewhere in memory, as a convenient way of accessing it. If you reassign a variable as below
you make the variable point to a different object (newly created ones in our examples). As stated above, only mutable objects can be changed in place (l[0] = 1 is ok in our example, but s[0] = 'a' raises an error). This becomes tricky, when an operation is not explicitly asking for a change to happen in place, as is the case for the += (increment) operator, for example. When used on an immutable object (as in a += 1 or in s += 'qwertz'), Python will silently create a new object and make the variable point to it. However, when used on a mutable object (as in l += [1,2,3]), the object pointed to by the variable will be changed in place. While in most situations, you do not have to know about this different behavior, it is of relevance when several variables are pointing to the same object. In our example, assume you set p = s and m = l, then s += 'etc' and l += [9,8,7]. This will change s and leave p unaffected, but will change both m and l since both point to the same list object. Python's built-in id function, which returns a unique object identifier for a given variable name, can be used to trace what is happening under the hood.
Typically, this behavior of Python causes confusion in functions. As an illustration, consider this code:
This will give the above indicated, and usually unintended, output. myseq is a local variable of the append_to_sequence function, but when this function gets called, myseq will nevertheless point to the same object as the variable that we pass in (t or l in our example). If that object is immutable (like a tuple), there is no problem. The += operator will cause the creation of a new tuple, and myseq will be set to point to it. However, if we pass in a reference to a mutable object, that object will be manipulated in place (so myseq and l, in our case, end up pointing to the same list object).
Links:
* 3.1. Objects, values and types, The Python Language Reference, docs.python.org
* 5.6.4. Mutable Sequence Types, The Python Standard Library, docs.python.org

## Creating Objects of Defined Types

Literal integers can be entered in three ways:
* decimal numbers can be entered directly
* hexadecimal numbers can be entered by prepending a 0x or 0X (0xff is hex FF, or 255 in decimal)
* the format of octal literals depends on the version of Python:
   * Python 2.x: octals can be entered by prepending a zero ( 0 ) (0732 is octal 732, or 474 in decimal)
   * Python 3.x: octals can be entered by prepending a zero followed by the letter O (0o or 0O) (0o732 is octal 732, or 474 in decimal)
Floating point numbers can be entered directly.
Long integers are entered either directly (1234567891011121314151617181920 is a long integer) or by appending an L (0L is a long integer). Computations involving short integers that overflow are automatically turned into long integers.
Complex numbers are entered by adding a real number and an imaginary one, which is entered by appending a j (i.e. 10+5j is a complex number. So is 10j). Note that j by itself does not constitute a number. If this is desired, use 1j.
Strings can be either single or triple quoted strings. The difference is in the starting and ending delimiters, and in that single quoted strings cannot span more than one line. Single quoted strings are entered by entering either a single quote (') or a double quote (") followed by its match. So therefore
Triple quoted strings are like single quoted strings, but can span more than one line. Their starting and ending delimiters must also match. They are entered with three consecutive single or double quotes, so
Tuples are entered in parentheses, with commas between the entries:
Also, the parenthesis can be left out when it's not ambiguous to do so:
Note that one-element tuples can be entered by surrounding the entry with parentheses and adding a comma like so:
Lists are similar, but with brackets:
Dicts are created by surrounding with curly braces a list of key/value pairs separated from each other by a colon and from the other entries with commas:
Any of these composite types can contain any other, to any depth:

## Null object

The Python analogue of null pointer known from other programming languages is None. None is not a null pointer or a null reference but an actual object of which there is only one instance. One of the uses of None is in default argument values of functions, for which see Python Programming/Functions. Comparisons to None are usually made using is rather than ==.
Testing for None and assignment:
Using None in a default argument value:
PEP8 states that "Comparisons to singletons like None should always be done with is or is not, never the equality operators." Therefore, "if item == None:" is inadvisable. A class can redefine the equality operator (==) such that instances of it will equal None.
You can verify that None is an object by dir(None) or id(None).
See also Python Programming/Operators/ chapter.
Links:
* 4. Built-in Constants, docs.python.org
* 3.11.7 The Null Object, docs.python.org
* Python None comparison: should I use “is” or ==?, stackoverflow.com
* PEP 0008 -- Style Guide for Python Code, python.org

## Type conversion

Type conversion in Python by example:
Implicit type conversion:
Keywords: type casting.
Links:
* 2. Built-in Functions # bool, docs.python.org
* 2. Built-in Functions # list, docs.python.org
* 2. Built-in Functions # float, docs.python.org
* 2. Built-in Functions # int, docs.python.org
* 2. Built-in Functions # set, docs.python.org
* 2. Built-in Functions # str, docs.python.org
* 2. Built-in Functions # type, docs.python.org
* 2. Built-in Functions # tuple, docs.python.org

## Exercises

* Write a program that instantiates a single object, adds [1,2] to the object, and returns the result.
** Find an object that returns an output of the same length (if one exists?).
** Find an object that returns an output length 2 greater than it started.
** Find an object that causes an error.
* Find two data types X and Y such that X = X + Y will cause an error, but X += Y will not.
| Previous: Sequences | Index | Next: Numbers |

# Numbers

| Previous: Data types | Index | Next: Strings |
Python 2.x supports 4 built-in numeric types - int, long, float and complex. Of these, the long type has been dropped in Python 3.x - the int type is now of unlimited length by default. You don’t have to specify what type of variable you want; Python does that automatically.
* Int: The basic integer type in python, equivalent to the hardware 'c long' for the platform you are using in Python 2.x, unlimited in length in Python 3.x.
* Long: Integer type with unlimited length. In python 2.2 and later, Ints are automatically turned into long ints when they overflow. Dropped since Python 3.0, use int type instead.
* Float: This is a binary floating point number. Longs and Ints are automatically converted to floats when a float is used in an expression, and with the true-division / operator. In CPython, floats are usually implemented using the C languages double, which often yields 52 bits of significand, 11 bits of exponent, and 1 sign bit, but this is machine dependent.
* Complex: This is a complex number consisting of two floats. Complex literals are written as a + bj where a and b are floating-point numbers denoting the real and imaginary parts respectively.
In general, the number types are automatically 'up cast' in the following order:
Int → Long → Float → Complex. The farther to the right you go, the higher the precedence.
The result of divisions is somewhat confusing. In Python 2.x, using the / operator on two integers will return another integer, using floor division. For example, 5/2 will give you 2. At least one of the operands must be float to get true division, e.g. 5/2. or 5./2 (the dot makes a number a float) will yield 2.5. Starting with Python 2.2 this behavior can be changed to true division by the future division statement from __future__ import division. In Python 3.x, the result of using the / operator is always true division (you can ask for floor division explicitly by using the // operator since Python 2.2).
This illustrates the behavior of the / operator in Python 2.2+:
For operations on numbers, see chapters Basic Math and Math.

## Links

* 5.4. Numeric Types — int, float, long, complex, docs.python.org
| Previous: Data types | Index | Next: Strings |

# Strings

| Previous: Numbers | Index | Next: Lists |

## Overview

Strings in Python at a glance:
See also chapter Regular Expression for advanced pattern matching on strings in Python.

## String operations
### Equality

Two strings are equal if they have exactly the same contents, meaning that they are both the same length and each character has a one-to-one positional correspondence. Many other languages compare strings by identity instead; that is, two strings are considered equal only if they occupy the same space in memory. Python uses the is operator to test the identity of strings and any two objects in general.
Examples:

### Numerical

There are two quasi-numerical operations which can be done on strings -- addition and multiplication. String addition is just another name for concatenation, which is simply sticking the strings together. String multiplication is repetitive addition, or concatenation. So:

### Containment

There is a simple operator 'in' that returns True if the first operand is contained in the second. This also works on substrings:
Note that 'print(x in y)' would have also returned the same value.

### Indexing and Slicing

Much like arrays in other languages, the individual characters in a string can be accessed by an integer representing its position in the string. The first character in string s would be s[0] and the nth character would be at s[n-1].
Unlike arrays in other languages, Python also indexes the arrays backwards, using negative numbers. The last character has index -1, the second to last character has index -2, and so on.
We can also use "slices" to access a substring of s. s[a:b] will give us a string starting with s[a] and ending with s[b-1].
None of these are assignable.
Outputs (assuming the errors were suppressed):
Another feature of slices is that if the beginning or end is left empty, it will default to the first or last index, depending on context:
You can also use negative numbers in slices:
To understand slices, it's easiest not to count the elements themselves. It is a bit like counting not on your fingers, but in the spaces between them. The list is indexed like this:
So, when we ask for the [1:3] slice, that means we start at index 1, and end at index 2, and take everything in between them. If you are used to indexes in C or Java, this can be a bit disconcerting until you get used to it.

## String constants

String constants can be found in the standard string module. An example is string.digits, which equals to '0123456789'.
Links:
* Python documentation of "string" module -- python.org

## String methods

There are a number of methods or built-in string functions:
* capitalize
* center
* count
* decode
* encode
* endswith
* expandtabs
* find
* index
* isalnum
* isalpha
* isdigit
* islower
* isspace
* istitle
* isupper
* join
* ljust
* lower
* lstrip
* replace
* rfind
* rindex
* rjust
* rstrip
* split
* splitlines
* startswith
* strip
* swapcase
* title
* translate
* upper
* zfill
Only emphasized items will be covered.

### is*

isalnum, isalpha, isdigit, islower, isupper, isspace, and istitle fit into this category.
The length of the string object being compared must be at least 1, or the is* methods will return False. In other words, a string object of len(string) == 0, is considered "empty", or False.
* isalnum returns True if the string is entirely composed of alphabetic and/or numeric characters (i.e. no punctuation).
* isalpha and isdigit work similarly for alphabetic characters or numeric characters only.
* isspace returns True if the string is composed entirely of whitespace.
* islower, isupper, and istitle return True if the string is in lowercase, uppercase, or titlecase respectively. Uncased characters are "allowed", such as digits, but there must be at least one cased character in the string object in order to return True. Titlecase means the first cased character of each word is uppercase, and any immediately following cased characters are lowercase. Curiously, 'Y2K'.istitle returns True. That is because uppercase characters can only follow uncased characters. Likewise, lowercase characters can only follow uppercase or lowercase characters. Hint: whitespace is uncased.
Example:

### Title, Upper, Lower, Swapcase, Capitalize

Returns the string converted to title case, upper case, lower case, inverts case, or capitalizes, respectively.
The title method capitalizes the first letter of each word in the string (and makes the rest lower case). Words are identified as substrings of alphabetic characters that are separated by non-alphabetic characters, such as digits, or whitespace. This can lead to some unexpected behavior. For example, the string "x1x" will be converted to "X1X" instead of "X1x".
The swapcase method makes all uppercase letters lowercase and vice versa.
The capitalize method is like title except that it considers the entire string to be a word. (i.e. it makes the first character upper case and the rest lower case)
Example:
Keywords: to lower case, to upper case, lcase, ucase, downcase, upcase.

### count

Returns the number of the specified substrings in the string. i.e.
Hint: .count is case-sensitive, so this example will only count the number of lowercase letter 'o's. For example, if you ran:

### strip, rstrip, lstrip

Returns a copy of the string with the leading (lstrip) and trailing (rstrip) whitespace removed. strip removes both.
Note the leading and trailing tabs and newlines.
Strip methods can also be used to remove other types of characters.
Outputs:
Note that string.lowercase and string.printable require an import string statement

### ljust, rjust, center

left, right or center justifies a string into a given field size (the rest is padded with spaces).

### join

Joins together the given sequence with the string as separator:
map may be helpful here: (it converts numbers in seq into strings)
now arbitrary objects may be in seq instead of just strings.

### find, index, rfind, rindex

The find and index methods return the index of the first found occurrence of the given subsequence. If it is not found, find returns -1 but index raises a ValueError. rfind and rindex are the same as find and index except that they search through the string from right to left (i.e. they find the last occurrence)
Because Python strings accept negative subscripts, index is probably better used in situations like the one shown because using find instead would yield an unintended value.

### replace

Replace works just like it sounds. It returns a copy of the string with all occurrences of the first parameter replaced with the second parameter.
Or, using variable assignment:
Outputs:
Notice, the original variable (string) remains unchanged after the call to replace.

### expandtabs

Replaces tabs with the appropriate number of spaces (default number of spaces per tab = 8; this can be changed by passing the tab size as an argument).
Outputs:
Notice how (although these both look the same) the second string (t) has a different length because each tab is represented by spaces not tab characters.
To use a tab size of 4 instead of 8:
Outputs:
Please note each tab is not always counted as eight spaces. Rather a tab "pushes" the count to the next multiple of eight. For example:
Output:
Outputs:

### split, splitlines

The split method returns a list of the words in the string. It can take a separator argument to use instead of whitespace.
Note that in neither case is the separator included in the split strings, but empty strings are allowed.
The splitlines method breaks a multiline string into many single line strings. It is analogous to split('\n') (but accepts '\r' and '\r\n' as delimiters as well) except that if the string ends in a newline character, splitlines ignores that final character (see example).
The method split also accepts multi-character string literals:

## Unicode

In Python 3.x, all strings (the type str) contain Unicode per default.
In Python 2.x, there is a dedicated unicode type in addition to the str type: u = u"Hello"; type(u) is unicode.
The topic name in the internal help is UNICODE.
Examples for Python 3.x:
* v = "Hello Günther"
** Uses a Unicode code point directly in the source code; that has to be in UTF-8 encoding.
* v = "Hello G\xfcnther"
** Specifies 8-bit Unicode code point using \xfc.
* v = "Hello G\u00fcnther"
** Specifies 16-bit Unicode code point using \u00fc.
* v = "Hello G\U000000fcnther"
** Specifies 32-bit Unicode code point using \U000000fc, the U being capitalized.
* v = "Hello G\N{LATIN SMALL LETTER U WITH DIAERESIS}nther"
** Specifies a Unicode code point using \N followed by the unicode point name.
* v = "Hello G\N{latin small letter u with diaeresis}nther"
** The code point name can be in lowercase.
* n = unicodedata.name(chr(252))
** Obtains Unicode code point name given a Unicode character, here of ü.
* v = "Hello G" + chr(252) + "nther"
** chr accepts Unicode code points and returns a string having one Unicode character.
* c = ord("ü")
** Yields the code point number.
* b = "Hello Günther".encode("UTF-8")
** Creates a byte sequence (bytes) out of a Unicode string.
* b = "Hello Günther".encode("UTF-8"); u = b.decode("UTF-8")
** Decodes bytes into a Unicode string via decode method.
* v = b"Hello " + "G\u00fcnther"
** Throws TypeError: can't concat bytes to str.
* v = b"Hello".decode("ASCII") + "G\u00fcnther"
** Now it works.
* f = open("File.txt", encoding="UTF-8"); lines = f.readlines; f.close
** Opens a file for reading with a specific encoding and reads from it. If no encoding is specified, the one of locale.getpreferredencoding is used.
* f = open("File.txt", "w", encoding="UTF-8"); f.write("Hello G\u00fcnther"); f.close
** Writes to a file in a specified encoding.
* f = open("File.txt", encoding="UTF-8-sig"); lines = f.readlines; f.close
** The -sig encoding means that any leading byte order mark (BOM) is automatically stripped.
* f = tokenize.open("File.txt"); lines = f.readlines; f.close
** Automatically detects encoding based on an encoding marker present in the file, such as BOM, stripping the marker.
* f = open("File.txt", "w", encoding="UTF-8-sig"); f.write("Hello G\u00fcnther"); f.close
** Writes to a file in UTF-8, writing BOM at the beginning.
Examples for Python 2.x:
* v = u"Hello G\u00fcnther"
** Specifies 16-bit Unicode code point using \u00fc.
* v = u"Hello G\U000000fcnther"
** Specifies 32-bit Unicode code point using \U000000fc, the U being capitalized.
* v = u"Hello G\N{LATIN SMALL LETTER U WITH DIAERESIS}nther"
** Specifies a Unicode code point using \N followed by the unicode point name.
* v = u"Hello G\N{latin small letter u with diaeresis}nther"
** The code point name can be in lowercase.
* unicodedata.name(unichr(252))
** Obtains Unicode code point name given a Unicode character, here of ü.
* v = "Hello G" + unichr(252) + "nther"
** chr accepts Unicode code points and returns a string having one Unicode character.
* c = ord(u"ü")
** Yields the code point number.
* b = u"Hello Günther".encode("UTF-8")
** Creates a byte sequence (str) out of a Unicode string. type(b) is str.
* b = u"Hello Günther".encode("UTF-8"); u = b.decode("UTF-8")
** Decodes bytes (type str) into a Unicode string via decode method.
* v = "Hello" + u"Hello G\u00fcnther"
** Concatenates str (bytes) and Unicode string without an error.
* f = codecs.open("File.txt", encoding="UTF-8"); lines = f.readlines; f.close
** Opens a file for reading with a specific encoding and reads from it. If no encoding is specified, the one of locale.getpreferredencoding is used[VERIFY].
* f = codecs.open("File.txt", "w", encoding="UTF-8"); f.write(u"Hello G\u00fcnther"); f.close
** Writes to a file in a specified encoding.
** Unlike the Python 3 variant, if told to write newline via \n, does not write operating system specific newline but rather literal \n; this makes a difference e.g. on Windows.
** To ensure text mode like operation one can write os.linesep.
* f = codecs.open("File.txt", encoding="UTF-8-sig"); lines = f.readlines; f.close
** The -sig encoding means that any leading byte order mark (BOM) is automatically stripped.
Links:
* Unicode HOWTO for Python 3, docs.python.org
* Unicode HOWTO for Python 2, docs.python.org
* Processing Text Files in Python 3, curiousefficiency.org
* PEP 263 – Defining Python Source Code Encodings, python.org
* unicodedata — Unicode Database in Python Library Reference, docs.python.org
* Get a list of all the encodings Python can encode to, stackoverflow.com

## External links

* "String Methods" chapter -- python.org
* Python documentation of "string" module -- python.org
| Previous: Numbers | Index | Next: Lists |

# Lists

| Previous: Strings | Index | Next: Tuples |
A list in Python is an ordered group of items (or elements). It is a very general structure, and list elements don't have to be of the same type: you can put numbers, letters, strings and nested lists all on the same list.

## Overview

Lists in Python at a glance:

## List creation

There are two different ways to make a list in Python. The first is through assignment ("statically"), the second is using list comprehensions ("actively").

### Plain creation

To make a static list of items, write them between square brackets. For example:
Observations:
* The list contains items of different data types: integer, string, and Donkey class.
* Objects can be created 'on the fly' and added to lists. The last item is a new instance of Donkey class.
Creation of a new list whose members are constructed from non-literal expressions:

### List comprehensions

Using list comprehension, you describe the process using which the list should be created. To do that, the list is broken into two pieces. The first is a picture of what each element will look like, and the second is what you do to get it.
For instance, let's say we have a list of words:
To take the first letter of each word and make a list out of it using list comprehension, we can do this:
List comprehension supports more than one for statement. It will evaluate the items in all of the objects sequentially and will loop over the shorter objects if one object is longer than the rest.
List comprehension supports an if statement, to only include members into the list that fulfill a certain condition:
In version 2.x, Python's list comprehension does not define a scope. Any variables that are bound in an evaluation remain bound to whatever they were last bound to when the evaluation was completed. In version 3.x Python's list comprehension uses local variables:
This is exactly the same as if the comprehension had been expanded into an explicitly-nested group of one or more 'for' statements and 0 or more 'if' statements.

### List creation shortcuts

You can initialize a list to a size, with an initial value for each element:
This works for any data type:
But there is a caveat. When building a new list by multiplying, Python copies each item by reference. This poses a problem for mutable items, for instance in a multidimensional array where each element is itself a list. You'd guess that the easy way to generate a two dimensional array would be:
and this works, but probably doesn't do what you expect:
What's happening here is that Python is using the same reference to the inner list as the elements of the outer list. Another way of looking at this issue is to examine how Python sees the above definition:
Assuming the above effect is not what you intend, one way around this issue is to use list comprehensions:

## List size

To find the length of a list use the built in len method.

## Combining lists

Lists can be combined in several ways. The easiest is just to 'add' them. For instance:
Another way to combine lists is with extend. If you need to combine lists inside of a lambda, extend is the way to go.
The other way to append a value to a list is to use append. For example:
However, [3,4] is an element of the list, and not part of the list. append always adds one element only to the end of a list. So if the intention was to concatenate two lists, always use extend.

## Getting pieces of lists (slices)
### Continuous slices

Like strings, lists can be indexed and sliced:
Much like the slice of a string is a substring, the slice of a list is a list. However, lists differ from strings in that we can assign new values to the items in a list:
We can assign new values to slices of the lists, which don't even have to be the same length:
It's even possible to append items onto the start of lists by assigning to an empty slice:
Similarly, you can append to the end of the list by specifying an empty slice after the end:
You can also completely change the contents of a list:
The right-hand side of a list assignment statement can be any iterable type:
With slicing you can create copy of list since slice returns a new list:
Note, however, that this is a shallow copy and contains references to elements from the original list, so be careful with mutable types:

### Non-Continuous slices

It is also possible to get non-continuous parts of an array. If one wanted to get every n-th occurrence of a list, one would use the :: operator. The syntax is a:b:n where a and b are the start and end of the slice to be operated upon.

## Comparing lists

Lists can be compared for equality.
Lists can be compared using a less-than operator, which uses lexicographical order:

## Sorting lists

Sorting at a glance:
Sorting lists is easy with a sort method.
Note that the list is sorted in place, and the sort method returns None to emphasize this side effect.
If you use Python 2.4 or higher there are some more sort parameters:
* sort(cmp,key,reverse)
** cmp : function to determine the relative order between any two elements
** key : function to obtain the value against which to sort for any element.
** reverse : sort(reverse=True) or sort(reverse=False)
Python also includes a sorted function.
Note that unlike the sort method, sorted(list) does not sort the list in place, but instead returns the sorted list. The sorted function, like the sort method also accepts the reverse parameter.
Links:
* 2. Built-in Functions # sorted, docs.python.org
* Sorting HOW TO, docs.python.org

## Iteration

Iteration over lists:
Read-only iteration over a list, AKA for each element of the list:
Writable iteration over a list:
From a number to a number with a step:
For each element of a list satisfying a condition (filtering):
See also Python Programming/Loops.

## Removing

Removing aka deleting an item at an index (see also #pop(i)):
Removing an element by value:
Creating a new list by copying a filtered selection of items from the old list:
This uses a list comprehension.
Update a list by retaining a filtered selection of items in it by using "[:]":
For more complex condition a separate function can be used to define the criteria:
Removing items while iterating a list usually leads to unintended outcomes unless you do it carefully by using an index:
Links:
* Remove items from a list while iterating, stackoverflow.com

## Aggregates

There are some built-in functions for arithmetic aggregates over lists. These include minimum, maximum, and sum:
The max and min functions also apply to lists of strings, returning maximum and minimum with respect to alphabetical order:

## Copying

Copying AKA cloning of lists:
Making a shallow copy:
The above does not make a deep copy, which has the following consequence:
Making a deep copy:
See also #Continuous slices.
Links:
* 8.17. copy — Shallow and deep copy operations at docs.python.org

## Clearing

Clearing a list:
Clearing using a proper approach makes a difference when the list is passed as an argument:
Keywords: emptying a list, erasing a list, clear a list, empty a list, erase a list.

## Removing duplicate items

Removing duplicate items from a list (keeping only unique items) can be achieved as follows.
If each item in the list is hashable, using list comprehension, which is fast:
If each item in the list is hashable, using index iteration, much slower:
If some items are not hashable, the set of visited items can be kept in a list:
If each item in the list is hashable and preserving element order does not matter:
In the above examples where index iteration is used, scanning happens from the end to the beginning. When these are rewritten to scan from the beginning to the end, the result seems hugely slower.
Links:
* How do you remove duplicates from a list? at python.org Programming FAQ
* Remove duplicates from a sequence (Python recipe) at activestate.com
* Removing duplicates in lists at stackoverflow.com

## List methods
### append(x)

Add item x onto the end of the list.
See pop(i)

### pop(i)

Remove the item in the list at the index i and return it. If i is not given, remove the last item in the list and return it.

## Operators

To concatenate two lists.
To create a new list by concatenating the given list the given number of times. i.e. list1 * 0 ==; list1 * 3 == list1 + list1 + list1;

### in

The operator 'in' is used for two purposes; either to iterate over every item in a list in a for loop, or to check if a value is in a list returning true or false.

## Difference

To get the difference between two lists, just iterate:

## Intersection

To get the intersection between two lists (by preserving its elements order, and their doubles), apply the difference with the difference:

## Exercises

* Use a list comprehension to construct the list ['ab', 'ac', 'ad', 'bb', 'bc', 'bd'].
* Use a slice on the above list to construct the list ['ab', 'ad', 'bc'].
* Use a list comprehension to construct the list ['1a', '2a', '3a', '4a'].
* Simultaneously remove the element '2a' from the above list and print it.
* Copy the above list and add '2a' back into the list such that the original is still missing it.
* Use a list comprehension to construct the list ['abe', 'abf', 'ace', 'acf', 'ade', 'adf', 'bbe', 'bbf', 'bce', 'bcf', 'bde', 'bdf']

## Solutions

Question 1 :
Question 2 :
Question 3 :
Question 4 :
Question 5 :
Question 6 :

## External links

* Python documentation, chapter "Sequence Types" -- python.org
* Python Tutorial, chapter "Lists" -- python.org
| Previous: Strings | Index | Next: Tuples |

# Tuples

| Previous: Lists | Index | Next: Dictionaries |
A tuple in Python is much like a list except that it is immutable (unchangeable) once created. A tuple of hashable objects is hashable and thus suitable as a key in a dictionary and as a member of a set.

## Overview

Tuples in Python at a glance:

## Tuple notation

Tuples may be created directly or converted from lists. Generally, tuples are enclosed in parentheses.
A one item tuple is created by an item in parentheses followed by a comma:
Also, tuples will be created from items separated by commas.

## Packing and Unpacking

You can also perform multiple assignment using tuples.
Note that either, or both sides of an assignment operator can consist of tuples.
The example above: article, noun, verb, adjective, direct_object = t is called "tuple unpacking" because the tuple t was unpacked and its values assigned to each of the variables on the left. "Tuple packing" is the reverse: t=article, noun, verb, adjective, direct_object. When unpacking a tuple, or performing multiple assignment, you must have the same number of variables being assigned to as values being assigned.

## Operations on tuples

These are the same as for lists except that we may not assign to indices or slices, and there is no "append" operator.
For lists we would have had:

### Tuple Attributes

Length: Finding the length of a tuple is the same as with lists; use the built in len method.

### Conversions

Convert list to tuples using the built in tuple method.
Converting a tuple into a list using the built in list method to cast as a list:
Dictionaries can also be converted to tuples of tuples using the items method of dictionaries:

## Uses of Tuples

Tuples can be used in place of lists where the number of items is known and small, for example when returning multiple values from a function. Many other languages require creating an object or container to return, but with Python's tuple assignment, multiple-value returns are easy:
This resulting tuple can be easily unpacked with the tuple assignment technique explained above:

## Using List Comprehension to process Tuple elements

Occasionally, there is a need to manipulate the values contained within a tuple in order to create a new tuple. For example, if we wanted a way to double all of the values within a tuple, we can combine some of the above information in addition to list comprehension like this:

## Exercises

* Create the list ['a', 'b', 'c'], then create a tuple from that list.
* Create the tuple ('a', 'b', 'c'), then create a list from that tuple. (Hint: the material needed to do this has been covered, but it's not entirely obvious)
* Make the following instantiations simultaneously: a = 'a', b=2, c='gamma'. (That is, in one line of code).
* Create a tuple containing just a single element which in turn contains the three elements 'a', 'b', and 'c'. Verify that the length is actually 1 by using the len function.

## External links

* Python documentation, chapter "Sequence Types" -- python.org
* Python documentation, chapter "Tuples and Sequences" -- python.org
| Previous: Lists | Index | Next: Dictionaries |

# Dictionaries

| Previous: Tuples | Index | Next: Sets |
A dictionary in Python is a collection of unordered values accessed by key rather than by index. The keys have to be hashable: integers, floating point numbers, strings, tuples, and, frozensets are hashable. Lists, dictionaries, and sets other than frozensets are not hashable. Dictionaries were available as early as in Python 1.4.

## Overview

Dictionaries in Python at a glance:

## Dictionary notation

Dictionaries may be created directly or converted from sequences. Dictionaries are enclosed in curly braces,
Also, dictionaries can be easily created by zipping two sequences.

## Operations on Dictionaries

The operations on dictionaries are somewhat unique. Slicing is not supported, since the items have no intrinsic order.

## Combining two Dictionaries

You can combine two dictionaries by using the update method of the primary dictionary. Note that the update method will merge existing elements if they conflict.

## Exercises

Write a program that:
* Asks the user for a string, then creates the following dictionary. The values are the letters in the string, with the corresponding key being the place in the string. https://docs.python.org/2/tutorial/datastructures.html#looping-techniques
* Replaces the entry whose key is the integer 3, with the value "Pie".
* Asks the user for a string of digits, then prints out the values corresponding to those digits.

## External links

* 5.5. Dictionaries in Tutorial, docs.python.org
* 5.8. Mapping Types in Library Doc, docs.python.org
| Previous: Tuples | Index | Next: Sets |

# Sets

| Previous: Dictionaries | Index | Next: Operators |
Starting with version 2.3, Python comes with an implementation of the mathematical set. Initially this implementation had to be imported from the standard module set, but with Python 2.6 the types set and frozenset became built-in types. A set is an unordered collection of objects, unlike sequence objects such as lists and tuples, in which each element is indexed. Sets cannot have duplicate members - a given object appears in a set 0 or 1 times. All members of a set have to be hashable, just like dictionary keys. Integers, floating point numbers, tuples, and strings are hashable; dictionaries, lists, and other sets (except frozensets) are not.

### Overview

Sets in Python at a glance:

### Constructing Sets

One way to construct sets is by passing any sequential object to the "set" constructor.
We can also add elements to sets one by one, using the "add" function.
Note that since a set does not contain duplicate elements, if we add one of the members of s to s again, the add function will have no effect. This same behavior occurs in the "update" function, which adds a group of elements to a set.
Note that you can give any type of sequential structure, or even another set, to the update function, regardless of what structure was used to initialize the set.
The set function also provides a copy constructor. However, remember that the copy constructor will copy the set, but not the individual elements.

### Membership Testing

We can check if an object is in the set using the same "in" operator as with sequential data types.
We can also test the membership of entire sets. Given two sets $S_{1}$ and $S_{2}$, we check if $S_{1}$ is a subset or a superset of $S_{2}$.
Note that "issubset" and "issuperset" can also accept sequential data types as arguments
Note that the <= and >= operators also express the issubset and issuperset functions respectively.
Like lists, tuples, and string, we can use the "len" function to find the number of items in a set.

### Removing Items

There are three functions which remove individual items from a set, called pop, remove, and discard. The first, pop, simply removes an item from the set. Note that there is no defined behavior as to which element it chooses to remove.
We also have the "remove" function to remove a specified element.
However, removing a item which isn't in the set causes an error.
If you wish to avoid this error, use "discard." It has the same functionality as remove, but will simply do nothing if the element isn't in the set
We also have another operation for removing elements from a set, clear, which simply removes all elements from the set.

### Iteration Over Sets

We can also have a loop move over each of the items in a set. However, since sets are unordered, it is undefined which order the iteration will follow.

### Set Operations

Python allows us to perform all the standard mathematical set operations, using members of set. Note that each of these set operations has several forms. One of these forms, s1.function(s2) will return another set which is created by "function" applied to $S_{1}$ and $S_{2}$. The other form, s1.function_update(s2), will change $S_{1}$ to be the set created by "function" of $S_{1}$ and $S_{2}$. Finally, some functions have equivalent special operators. For example, s1 & s2 is equivalent to s1.intersection(s2)

#### Intersection

Any element which is in both $S_{1}$ and $S_{2}$ will appear in their intersection.

#### Union

The union is the merger of two sets. Any element in $S_{1}$ or $S_{2}$ will appear in their union.
Note that union's update function is simply "update" above.

#### Symmetric Difference

The symmetric difference of two sets is the set of elements which are in one of either set, but not in both (also called exclusive-or in logic).

#### Set Difference

Python can also find the set difference of $S_{1}$ and $S_{2}$, which is the elements that are in $S_{1}$ but not in $S_{2}$.

### Multiple sets

Starting with Python 2.6, "union", "intersection", and "difference" can work with multiple input. For example, using "set.intersection":

### frozenset

A frozenset is basically the same as a set, except that it is immutable - once it is created, its members cannot be changed. Since they are immutable, they are also hashable, which means that frozensets can be used as members in other sets and as dictionary keys. frozensets have the same functions as normal sets, except none of the functions that change the contents (update, remove, pop, etc.) are available.

### Exercises

* Create the set {'cat', 1, 2, 3}, call it s.
* Create the set {'c', 'a', 't', '1', '2', '3'}.
* Create the frozen set {'cat', 1, 2, 3}, call it fs.
* Create a set containing the frozenset fs, it should look like {frozenset({'cat', 2, 3, 1})}.

### Reference

* Python Tutorial, section "Data Structures", subsection "Sets" -- python.org
* Python Library Reference on Set Types -- python.org
* PEP 218 -- Adding a Built-In Set Object Type, python.org, a nice concise overview of the set type
| Previous: Dictionaries | Index | Next: Operators |

# Basic Math

| Previous: Operators | Index | Next: Control Flow |
Now that we know how to work with numbers and strings, let's write a program that might actually be useful! Let's say you want to find out how much you weigh in stone. A concise program can make short work of this task. Since a stone is 14 pounds, and there are about 2.2 pounds in a kilogram, the following formula should do the trick:
$m_{stone}={\frac {m_{kg}\times 2.2}{14}}$
So, let's turn this formula into a program!
Run this program and get your weight in stone! Notice that applying the formula was as simple as putting in a few mathematical statements:

## Mathematical Operators

Here are some commonly used mathematical operators
| Syntax | Math | Operation Name |
| a+b | $a+b\,$ | addition |
| a-b | $a-b\,$ | subtraction |
| a*b | $a\times b\,$ | multiplication |
| a/b | $a\div b\,$ | division (see note below) |
| a//b | $\lfloor a\div b\,\rfloor$ | floor division (e.g. 5//2=2) - Available in Python 2.2 and later |
| a%b | $a~{\bmod {~}}b\,$ | modulo |
| -a | $-a\,$ | negative value |
| abs(a) | $|a|\,$ | absolute value |
| a**b | $a^{b}\,$ | exponent |
| math.sqrt(a) | ${\sqrt {a}}\,$ | square root |
Beware that due to the limitations of floating point arithmetic, rounding errors can cause unexpected results. For example:
For the Python 2.x series, / does "floor division" for integers and longs (e.g. 5/2=2) but "true division" for floats and complex (e.g. 5.0/2.0=2.5). For Python 3.x, / does "true division" for all types.
This can be fixed by putting a round([math]-0.5) around a normal division sign, because of a Python error causing round(0.5) to round down.
The operator // always performs Euclidean (or integer-type) division, which includes a quotient term (obtained from the // operator) and a remainder term (obtained from the % operator). In the previous example we have seen that the quotient term 0.6 // 0.2 is 2.0, which can be verified by extending the above example:
The difference between the operations / and // when applied to decimal numbers is due to the way decimal numbers are stored in Python and rounding.

## Order of Operations

Python uses the standard order of operations as taught in Algebra and Geometry classes at high school or secondary school. That is, mathematical expressions are evaluated in the following order (memorized by many as PEMDAS), which is also applied to parentheticals.
(Note that operations which share a table row are performed from left to right. That is, a division to the left of a multiplication, with no parentheses between them, is performed before the multiplication simply because it is to the left.)
| Name | Syntax | Description | PEMDAS Mnemonic |
| Parentheses | ( ... ) | Before operating on anything else, Python must evaluate all parentheticals starting at the innermost level. (This includes functions.) | Please |
| Exponents | ** | As an exponent is simply short multiplication or division, it should be evaluated before them. | Excuse |
| Multiplication and Division | * / // % | Again, multiplication is rapid addition and must, therefore, happen first. | My Dear |
| Addition and Subtraction | + - | | Aunt Sally |

## Formatting output

Wouldn't it be nice if we always worked with nice round numbers while doing math? Unfortunately, the real world is not quite so neat and tidy as we would like it to be. Sometimes, we end up with long, ugly numbers like the following:
By default, Python's print statement prints numbers to 10 decimal places. But what if you only want one or two? We can use the round function, which rounds a number to the number of decimal points you choose. round takes two arguments: the number you want to round, and the number of decimal places to round it to. For example:
Now, let's change our program to only print the result to two decimal places.
This also demonstrates the concept of nesting functions. As you can see, you can place one function inside another function, and everything will still work exactly the way you would expect. If you don't like this, you can always use multiple variables, instead:
To learn about more formatting options, see section Formatting in chapter Input and Output.

## Notes

* ↑ What's New in Python 2.2
* ↑ PEP 238 -- Changing the Division Operator
| Previous: Operators | Index | Next: Control Flow |

# Operators

| Previous: Sets | Index | Next: Basic Math |

## Basics

Python math works as expected:
Note that Python adheres to the PEMDAS order of operations.

## Powers

There is a built in exponentiation operator **, which can take either integers, floating point or complex numbers. This occupies its proper place in the order of operations.

## Floor Division and True Division

In Python 3.x, slash operator ("/") does true division for all types including integers, and therefore, e.g. 3/2==1.5. The result is of a floating-point type even if both inputs are integers: 4 / 2 yields 2.0.
In Python 3.x and latest 2.x, floor division for both integer arguments and floating-point arguments is achieved by using the double slash ("//") operator. For negative results, this is unlike the integer division in the C language since -3 // 2 == -2 in Python while -3 / 2 == -1 in C: C rounds the negative result toward zero while Python toward negative infinity.
Beware that due to the limitations of floating point arithmetic, rounding errors can cause unexpected results. For example:
For Python 2.x, dividing two integers or longs using the slash operator ("/") uses floor division (applying the floor function after division) and results in an integer or long. Thus, 5 / 2 == 2 and -3 / 2 == -2. Using "/" to do division this way is deprecated; if you want floor division, use "//" (available in Python 2.2 and later). Dividing by or into a floating point number will cause Python to use true division. Thus, to ensure true division in Python 2.x: x=3; y=2; float(x)/y == 1.5.
Links:
* 6.7. Binary arithmetic operations, in The Python Language Reference, docs.python.org
* Python-style integer division & modulus in C, stackoverflow.com
* Integer division rounding with negatives in C++, stackoverflow.com
* In Python, what is a good way to round towards zero in integer division?, stackoverflow.com
* Why Python's Integer Division Floors, python-history.blogspot.com

## Modulus

The modulus (remainder of the division of the two operands, rather than the quotient) can be found using the % operator, or by the divmod builtin function. The divmod function returns a tuple containing the quotient and remainder.
Note that -10 % 7 is equal to +4 while in the C language it is equal to -3. That is because Python floors towards negative infinity not zero. As a result, remainders add towards positive infinity. Consequently, since -10 / 7 = -1.4286 becomes floored to -2.0 the remainder becomes x such that -14 + x = -10.
Links:
* 6.7. Binary arithmetic operations, in The Python Language Reference, docs.python.org

## Negation

Unlike some other languages, variables can be negated directly:

## Comparison

| Operation | Means |
| < | Less than |
| > | Greater than |
| <= | Less than or equal to |
| >= | Greater than or equal to |
| == | Equal to |
| != | Not equal to |
Numbers, strings and other types can be compared for equality/inequality and ordering:

## Identity

The operators is and is not test for object identity and stand in contrast to == (equals): x is y is true if and only if x and y are references to the same object in memory. x is not y yields the inverse truth value. Note that an identity test is more stringent than an equality test since two distinct objects may have the same value.
For the built-in immutable data types (like int, str and tuple) Python uses caching mechanisms to improve performance, i.e., the interpreter may decide to reuse an existing immutable object instead of generating a new one with the same value. The details of object caching are subject to changes between different Python versions and are not guaranteed to be system-independent, so identity checks on immutable objects like 'hello' is 'hello', (1,2,3) is (1,2,3), 4 is 2**2 may give different results on different machines.
In some Python implementations, the following results are applicable:
Links:
* 3. Data model, python.org
* 2. Built-in Functions # id, python.org
* 5. Expressions # is, python.org

## Augmented Assignment

There is shorthand for assigning the output of an operation to one of the inputs:

## Logical Operators

Logical operators are operators that act on booleans.

### or

The or operator returns true if any one of the booleans involved are true. If none of them are true (in other words, they are all false), the or operator returns false.

### and

The and operator only returns true if all of the booleans are true. If any one of them is false, the and operator returns false.

### not

The not operator only acts on one boolean and simply returns its opposite. So, true turns into false and false into true.
The order of operations here is: not first, and second, or third. In particular, "True or True and False or False" becomes "True or False or False" which is True.
Warning, logical operators can act on things other than booleans. For instance "1 and 6" will return 6. Specifically, "and" returns either the first value considered to be false, or the last value if all are considered true. "or" returns the first true value, or the last value if all are considered false. In Python the number zero and empty strings, lists, sets, etc. are considered false. You may use bool to check whether a thing is considered to be true or false in Python. For instance, bool(0.0) and bool() both return False.

## Bitwise Operators

Python operators for bitwise arithmetic are like those in the C language. They include & (bitwise and), | (bitwise or), ^ (exclusive or AKA xor), << (shift left), >> (shift right), and ~ (complement). Augmented assignment operators (AKA compound assignment operators) for the bitwise operations include &=, |=, ^=, <<=, and >>=. Bitwise operators apply to integers, even negative ones and very large ones; for the shift operators, the second operand must be non-negative. In the Python internal help, this is covered under the topics of EXPRESSIONS and BITWISE.
Examples:
* 0b1101 & 0b111 == 0b101
** Note: 0b starts a binary literal, just like 0x starts a hexadecimal literal.
* 0b1 | 0b100 == 0b101
* 0b111 ^ 0b101 == 0b10
* 1 << 4 == 16
* 7 >> 1 == 3
* 1 << 100 == 0x10000000000000000000000000
** Large results are supported.
* 1 << -1
** An error: the 2nd operand must be non-negative.
* -2 & 15 == 14
** For bitwise operations, negative integers are treated as if represented using two's complement with an infinite sequence of leading ones. Thus -2 is as if 0x...FFFFFFFFFE, which ANDed with 15 (0xF) yields 0xE, which is 14.
* format(-2 % (1 << 32), "032b")
** Determines a string showing the last 32 bits of the implied two's complement representation of -2.
* ~-2 == 1
** The above note about treatment of negative integers applies. For the complement (~), this treatment yields ~x == -1 * (x + 1). This can be verified: min((~x == -1 * (x + 1) for x in range(-10 ** 6, 10 ** 6))) == True.
* ~1 == -2
** The formula ~x == -1 * (x + 1) mentioned above applies. In the bitwise complement interpretation, all the imaginary leading leading zeros of 1 are toggled to leading ones, which is then interpreted as two's complement representation of -2, which is as if 0x...FFFFFFFFFE.
* x = 0b11110101; x &= ~(0xF << 1); x == 0b11100001
** A common idiom clears the least significant bits 5 to 2 using complement, showing the usefulness of the infinite-leading-ones two's complement implied representation of negative numbers for bitwise operations. Works for arbitrarily large x.
* ~2 & 0xFFFFFFFF == 0xFFFFFFFD
** We can emulate bitwise complement on 32-bit integers by ANDing the complement with the maximum unsigned 32-bit integer, 0xFFFFFFFF. We can proceed similarly for 8-bit complement, 16-bit complement, etc., by ANDing the complement with 0xFF, 0xFFFF, etc.
* 2 ^ 0xFFFFFFFF == 0xFFFFFFFD
** Another way to emulate fixed-size bitwise complement is by XORing with all Fs for the size.
* v = 2.0 & 2
** Yields an error: no automatic conversion of floats to ints, and no operation on the underlying representation of the float.
* int.bit_length(0x8000) == 16
** Determines how many bits are needed to represent the integer. Therefore, min((int.bit_length(1 << x) == x + 1 for x in range(100000))) == True.
Examples of augmented assignment operators:
* a = 0b1101; a &= 0b111; a == 0b101
* a = 0b1; a |= 0b100; a == 0b101
* a = 0b111; a ^= 0b101; a == 0b10
* a = 1; a <<= 4; a == 16
* a = 7; a >>= 1; a == 3
Class definitions can overload the operators for the instances of the class; thus, for instance, sets overload the pipe (|) operator to mean set union: {1,2} | {3,4} == {1,2,3,4}. The names of the override methods are __and__ for &, __or__ for |, __xor__ for ^, __invert__ for ~, __lshift__ for <<, __rshift__ for >>, __iand__ for &=, __ior_ for |=, __ixor__ for ^=, __ilshift__ for <<=, and __irshift__ for >>=.
Examples of use of bitwise operations include calculation of CRC and MD5. Admittedly, these would usually be implemented in C rather than Python for maximum speed; indeed, Python has libraries for these written in C. Nonetheless, implementations in Python are possible and are shown in the links to Rosetta Code below.
Links:
* BitwiseOperators, wiki.python.org
* BitManipulation, wiki.python.org
* Bitwise Operations on Integer Types in Library Reference, docs.python.org
* 2.5. Operators in The Python Language Reference, docs.python.org
* 6. Expressions in The Python Language Reference, docs.python.org
* 3. Data model in in The Python Language Reference, docs.python.org
    "Integers (int) [...] For the purpose of shift and mask operations, a binary representation is assumed, and negative numbers are represented in a variant of 2’s complement which gives the illusion of an infinite string of sign bits extending to the left."
* Bitwise operation, wikipedia.org
* Bitwise operations in C, wikipedia.org
* PEP 203 – Augmented Assignments, python.org
* Bitwise operation and usage, stackoverflow.com
* CRC-32 # Python, rosettacode.org
* MD5/Implementation # Python, rosettacode.org
* longobject.c at python/cpython, github.com - functions long_bitwise, long_rshift, long_lshift, and long_invert

## References

* ↑ What's New in Python 2.2
* ↑ PEP 238 -- Changing the Division Operator

## External Links

* 3.1.1. Numbers in The Python Tutorial, docs.python.org
* 6. Expressions in The Python Language Reference, docs.python.org
| Previous: Sets | Index | Next: Basic Math |

# Control Flow

| Previous: Operators | Index | Next: Decision Control |
As with most imperative languages, there are three main categories of program control flow:
* loops
* branches
* function calls
Function calls are covered in the next section.
Generators and list comprehensions are advanced forms of program control flow, but they are not covered here.

### Overview

Control flow in Python at a glance:

### Loops

In Python, there are two kinds of loops, 'for' loops and 'while' loops.

#### For loops

A for loop iterates over elements of a sequence (tuple or list). A variable is created to represent the object in the sequence. For example,
This will output
The for loop loops over each of the elements of a list or iterator, assigning the current element to the variable name given. In the example above, each of the elements in x is assigned to i.
A built-in function called range exists to make creating sequential lists such as the one above easier. The loop above is equivalent to:
Similar to the slicing operation, in the range function, the first argument is the starting integer (we can just pass one argument to the range, which will be interpreted as the second argument, and then the default value: 0 is used for the first argument), and the second argument is the ending integer but excluded from the list.
The next example uses a negative step (the third argument for the built-in range function, which is similar to the slicing operation):
This will output
The negative step can be -2:
This will output
For loops can have names for each element of a tuple, if it loops over a sequence of tuples:
This will output
Links:
* 4.2. for Statements, The Python Tutorial, docs.python.org
* 4.3. The range Function, The Python Tutorial, docs.python.org

#### While loops

A while loop repeats a sequence of statements until the condition becomes false. For example:
Will output:
Python's while loops can also have an 'else' clause, which is a block of statements that is executed (once) when the while condition evaluates to false. The break statement (see the next section) inside the while loop will not direct the program flow to the else clause. For example:
This will output:
Unlike some languages, there is no post-condition loop.
When the while condition never evaluates to false, i.e., is always true, then we have an infinite loop. For example,
This results in an infinite loop, which prints 1,2,3,4,... . To stop an infinite loop, we need to use the break statement.
Links:
* 3.2. First Steps Towards Programming, The Python Tutorial, docs.python.org

#### Breaking and continuing

Python includes statements to exit a loop (either a for loop or a while loop) prematurely. To exit a loop, use the break statement:
This will output
The statement to begin the next iteration of the loop without waiting for the end of the current loop is 'continue'.
This will not produce any output.

#### Else clause of loops

The else clause of loops will be executed if no break statements are met in the loop.
Another example of a while loop using the break statement and the else statement:
This will output:

#### White Space

Python determines where a loop repeats itself by the indentation in the whitespace. Everything that is indented is part of the loop, the next entry that is not indented is not. For example, the code below prints "1 1 2 1 1 2"
On the other hand, the code below prints "1 2 1 2 1 2 1 2"

### Branches

There is basically only one kind of branch in Python, the 'if' statement. The simplest form of the if statement simple executes a block of code only if a given predicate is true, and skips over it if the predicate is false
For instance,
You can also add "elif" (short for "else if") branches onto the if statement. If the predicate on the first “if” is false, it will test the predicate on the first elif, and run that branch if it’s true. If the first elif is false, it tries the second one, and so on. Note, however, that it will stop checking branches as soon as it finds a true predicate, and skip the rest of the if statement. You can also end your if statements with an "else" branch. If none of the other branches are executed, then python will run this branch.
Links:
* 4.1. if Statements, The Python Tutorial, docs.python.org

### Conclusion

Any of these loops, branches, and function calls can be nested in any way desired. A loop can loop over a loop, a branch can branch again, and a function can call other functions, or even call itself.

## Exercises

* Print the numbers from 0 to 1000 (including both 0 and 1000).
* Print the numbers from 0 to 1000 that are multiples of 5.
* Print the numbers from 1 to 1000 that are multiples of 5.
* Use a nested for-loop to prints the 3x3 multiplication table below
* Print the 3x3 multiplication table below.

## External links

* 4. More Control Flow Tools, The Python Tutorial, docs.python.org
| Previous: Operators | Index | Next: Decision Control |

# Decision Control

| Previous: Control Flow | Index | Next: Conditional Statements |
Python, like many other computer programming languages, uses Boolean logic for its decision control. That is, the Python interpreter compares one or more values in order to decide whether to execute a piece of code or not, given the proper syntax and instructions.
Decision control is then divided into two major categories, conditional and repetition. Conditional logic simply uses the keyword if and a Boolean expression to decide whether or not to execute a code block. Repetition builds on the conditional constructs by giving us a simple method in which to repeat a block of code while a Boolean expression evaluates to true.

## Boolean Expressions

Here is a little example of boolean expressions (you don't have to type it in):
With the output being:
What is going on? The program consists of a bunch of funny looking print statements. Each print statement prints a number and an expression. The number is to help keep track of which statement I am dealing with. Notice how each expression ends up being either True or False; these are built-in Python values. The lines:
print out True and False respectively, just as expected, since the first is true and the second is false. The third print, print (3, a == 6 and b == 7), is a little different. The operator and means if both the statement before and the statement after are true then the whole expression is true otherwise the whole expression is false. The next line, print (4, a == 7 and b == 7), shows how if part of an and expression is false, the whole thing is false. The behavior of and can be summarized as follows:
| expression | result |
| true and true | true |
| true and false | false |
| false and true | false |
| false and false | false |
Note that if the first expression is false Python does not check the second expression since it knows the whole expression is false.
The next line, print (5, not a == 7 and b == 7), uses the not operator. not just gives the opposite of the expression (The expression could be rewritten as print (5, a != 7 and b == 7)). Here's the table:
| expression | result |
| not true | false |
| not false | true |
The two following lines, print (6, a == 7 or b == 7) and print (7, a == 7 or b == 6), use the or operator. The or operator returns true if the first expression is true, or if the second expression is true or both are true. If neither are true it returns false. Here's the table:
| expression | result |
| true or true | true |
| true or false | true |
| false or true | true |
| false or false | false |
Note that if the first expression is true Python doesn't check the second expression since it knows the whole expression is true. This works since or is true if at least one of the expressions are true. The first part is true so the second part could be either false or true, but the whole expression is still true.
The next two lines, print (8, not (a == 7 and b == 6)) and print (9, not a == 7 and b == 6), show that parentheses can be used to group expressions and force one part to be evaluated first. Notice that the parentheses changed the expression from false to true. This occurred since the parentheses forced the not to apply to the whole expression instead of just the a == 7 portion.
Here is an example of using a boolean expression:
See the Lists chapter for an explanation of the slice operator, [:], occurring in copy = list[:].
Here is the output:
This program works by continuing to check for a match while count < len(copy) and copy[count] != prev. When either count is greater than the last index of copy or a match has been found the and is no longer true so the loop exits. The if simply checks to make sure that the while exited because a match was found.
The other 'trick' of and is used in this example. If you look at the table for and notice that the third entry is "false and won't check". If count >= len(copy) (in other words count < len(copy) is false) then copy[count] is never looked at. This is because Python knows that if the first is false then they both can't be true. This is known as a short circuit and is useful if the second half of the and will cause an error if something is wrong. I used the first expression ( count < len(copy)) to check and see if count was a valid index for copy. (If you don't believe me remove the matches `Jill' and `Life', check that it still works and then reverse the order of count < len(copy) and copy[count] != prev to copy[count] != prev and count < len(copy).)
Boolean expressions can be used when you need to check two or more different things at once.

## Examples

password1.py
Sample runs

## Exercises

* Write a program that has a user guess your name, but they only get 3 chances to do so until the program quits.
Solutions
| Previous: Control Flow | Index | Next: Conditional Statements |

# Conditional Statements

| Previous: Decision Control | Index | Next: Loops |

## Decisions

A Decision is when a program has more than one choice of actions depending on a variable's value. Think of a traffic light. When it is green, we continue our drive. When we see the light turn yellow, we reduce our speed, and when it is red, we stop. These are logical decisions that depend on the value of the traffic light. Luckily, Python has a decision statement to help us when our application needs to make such decision for the user.

## If statements

Here is a warm-up exercise - a short program to compute the absolute value of a number:
absoval.py
Here is the output from the two times that I ran this program:
What does the computer do when it sees this piece of code? First it prompts the user for a number with the statement "n = input("Integer? ")". Next it reads the line "if n < 0:". If n is less than zero Python runs the line "print ("The absolute value of",n,"is",-n)". Otherwise python runs the line "print ("The absolute value of",n,"is",n)".
More formally, Python looks at whether the expression n < 0 is true or false. An if statement is followed by an indented block of statements that are run when the expression is true. After the if statement is an optional else statement and another indented block of statements. This 2nd block of statements is run if the expression is false.
Expressions can be tested several different ways. Here is a table of all of them:
| operator | function |
| < | less than |
| <= | less than or equal to |
| > | greater than |
| >= | greater than or equal to |
| == | equal |
| != | not equal |
Another feature of the if command is the elif statement. It stands for "else if," which means that if the original if statement is false and the elif statement is true, execute the block of code following the elif statement. Here's an example:
ifloop.py
and the output:
Notice how the elif a <= 7 is only tested when the if statement fails to be true. elif allows multiple tests to be done in a single if statement.

### If Example

High_low.py
Sample run:
As it states in its comments, this code is not prepared to handle invalid input (i.e., strings instead of numbers). If you are wondering how you would implement such functionality in Python, you are referred to the Errors Chapter of this book, where you will learn about error handling. For the above code you may try this slight modification of the while loop:
even.py
Sample runs.
average1.py
Sample runs
average2.py
Sample runs
average3.py
Sample runs

### If Exercises

* Write a password guessing program to keep track of how many times the user has entered the password wrong. If it is more than 3 times, print You have been denied access. and terminate the program. If the password is correct, print You have successfully logged in. and terminate the program.
* Write a program that asks for two numbers. If the sum of the numbers is greater than 100, print That is a big number and terminate the program.
* Write a program that asks the user their name. If they enter your name, say "That is a nice name." If they enter "John Cleese" or "Michael Palin", tell them how you feel about them ;), otherwise tell them "You have a nice name."
* Ask the user to enter the password. If the password is correct print "You have successfully logged in" and exit the program. If the password is wrong print "Sorry the password is wrong" and ask the user to enter the password 3 times. If the password is wrong print "You have been denied access" and exit the program.

### Conditional Statements

Many languages (like Java and PHP) have the concept of a one-line conditional (called The Ternary Operator), often used to simplify conditionally accessing a value. For instance (in Java):
For many years Python did not have the same construct natively, however you could replicate it by constructing a tuple of results and calling the test as the index of the tuple, like so:
It is important to note that, unlike a built in conditional statement, both the true and false branches are evaluated before returning, which can lead to unexpected results and slower executions if you're not careful. To resolve this issue, and as a better practice, wrap whatever you put in the tuple in anonymous function calls (lambda notation) to prevent them from being evaluated until the desired branch is called:
Since Python 2.5 however, there has been an equivalent operator to The Ternary Operator (though not called such, and with a totally different syntax):

## Switch

A switch is a control statement present in most computer programming languages to minimize a bunch of If - elif statements. The switch case functionality was added to Python 3.10 with a new syntax called match case. Prior to version 3.10 switch was not officially supported. The switch statement can be recreated by cleverly using an array or dictionary.
This works because Python stores a reference of the function in the array at its particular index, and by adding a pair of parentheses we are actually calling the function. Here the last line is equivalent to:

## Another way. Using function through user Input
### Another way

Another way is to use lambdas. Code pasted here with permissions1
For more information on lambda see anonymous functions in the function section.
| Previous: Decision Control | Index | Next: Loops |

# Loops

| Previous: Conditional Statements | Index | Next: Functions |

## While loops

This is our first control structure.The version used is 2.7. Ordinarily the computer starts with the first line and then goes down from there. Control structures change the order that statements are executed or decide if a certain statement will be run. As a side note, decision statements (e.g., if statements) also influence whether or not a certain statement will run. Here's the source for a program that uses the while control structure:
And here is the output:
So what does the program do? First it sees the line a = 0 and sets the variable a to zero. Then it sees while a < 5: and so the computer checks to see if a < 5. The first time the computer sees this statement, a is zero, and zero is less than 5. In other words, while a is less than five, the computer will run the indented statements.
Here is another example of the use of while:
Notice how print 'Total Sum =',s is only run at the end. The while statement only affects the lines that are tabbed in (a.k.a. indented). The != means does not equal so while a != 0 : means until a is zero run the tabbed in statements that are afterwards.
Now that we have while loops, it is possible to have programs that run forever. An easy way to do this is to write a program like this:
This program will output Help, I'm stuck in a loop. until the heat death of the universe or you stop it. The way to stop it is to hit the Control (or Ctrl) button and `c' (the letter) at the same time. This will kill the program. (Note: sometimes you will have to hit enter after the Control C.)

### Examples

Fibonacci.py
Output:
Password.py
Sample run:

## For Loops

The next type of loop in Python is the for loop. Unlike in most languages, for requires some __iterable__ object like a Set or List to work.
The output:
The output looks very familiar, but the program code looks different. The first line uses the range function. The range function uses two arguments like this range(start,finish). start is the first number that is produced. finish is one larger than the last number. Note that this program could have been done in a shorter way:
Here are some examples to show what happens with the range function:
Remark: for the current version of Python, we need to pass the range function to list to actually print a list. For instance, we need to use list(range(1,10)) to get [1, 2, 3, 4, 5, 6, 7, 8, 9].
Another way to use the range function in a for loop is to supply only one argument:
The above code acts exactly the same as:
with 0 implied as the starting point. The output is
The code would cycle through the for loop 10 times as expected, but starting with 0 instead of 1.
The next line for count in onetoten: uses the for control structure. A for control structure looks like for variable in list:. list is gone through starting with the first element of the list and going to the last. As for goes through each element in a list it puts each into variable. That allows variable to be used in each successive time the for loop is run through. Here is another example to demonstrate:
The output is:
Notice how the for loop goes through and sets item to each element in the list. (Notice how if you don't want print to go to the next line add a comma at the end of the statement (i.e. if you want to print something else on that line). ) So, what is for good for? The first use is to go through all the elements of a list and do something with each of them. Here a quick way to add up all the elements:
with the output simply being:
Or you could write a program to find out if there are any duplicates in a list like this program does:
and for good measure:
How does it work? Here is a special debugging version:
with the output being:
Note: The reason there are so many print statements is because print statements can show the value of each variable at different times, and help debug the program. First the program starts with a old list. Next the program sorts the list. This is so that any duplicates get put next to each other. The program then initializes a prev(ious) variable. Next the first element of the list is deleted so that the first item is not incorrectly thought to be a duplicate. Next a for loop is gone into. Each item of the list is checked to see if it is the same as the previous. If it is a duplicate was found. The value of prev is then changed so that the next time the for loop is run through prev is the previous item to the current. Sure enough, the 7 is found to be a duplicate.
The other way to use for loops is to do something a certain number of times. Here is some code to print out the first 9 numbers of the Fibonacci series:
with the surprising output:
Everything that can be done with for loops can also be done with while loops but for loops give an easy way to go through all the elements in a list or to do something a certain number of times.

## range Versus xrange

Python 3 Note: In Python 3.x, there is no function named xrange. Instead, the range function has the behaviour described below for xrange. The following applies to the legacy Python 2.
Above, you were introduced to the range function, which returns a list of all the integers in a specified range. Supposing you were to write an expression like range(0, 1000000): that would construct a list consisting of a million integers! That can take up a lot of memory.
Often, you do indeed need to process all the numbers over a very wide range. But you might only need to do so one at a time; as each number is processed, it can be discarded from memory before the next one is obtained.
To do this, you can use the xrange function instead of range. For example, the following simple loop
will print the million integers from 0 to 999999, but it will get them one at a time from the xrange call, instead of getting them all at once as a single list and going through that.
This is an example of an iterator, which yields values one at a time as they are needed, rather than all at once. As you learn more about Python, you will see a lot more examples of iterators in use, and you will learn how to write your own.

## The break Statement

A while-loop checks its termination condition before each entry into the loop body, and terminates if the condition has gone False. Thus, the loop body will normally iterate zero, one or more complete times.
A for-loop iterates its body once for each value returned from the iterator expression. Again, each iteration is normally of the complete loop body.
Sometimes you want to conditionally stop the loop in the middle of the loop body. An example situation might look like this:
* Obtain the next value to check
* Is there in fact another value to check? If not, exit the loop with failure.
* Is the value what I’m looking for? If so, exit the loop with success.
* Otherwise, go back to step 1.
As you can see, there are two possible exits from this loop. You can exit from the middle of a Python while- or for-loop with the break-statement. Here is one way to write such a loop:
The trouble with this is the asymmetry between the two ways out of the loop: one through normal for-loop termination, the other through the break. As a stylistic matter, it would be more consistent to follow this principle:
    If one exit from a loop is represented by a break, then all exits from the loop should be represented by breaks.
In particular, this means that the loop construct itself becomes a “loop-forever” loop; the only out of it is via a break statement.
We can do this by explicitly dealing with an iterator yielding the values to be checked. Perhaps the values_to_check call above already yields an iterator; if not, it can be converted to one by wrapping it in a call to the iter built-in function, and then using the next built-in function to obtain successive values from this iterator. Then the loop becomes:
This uses the special Python value None to indicate that the iterator has reached its end. This is a common Python convention. But if you need to use None as a valid item in your sequence of values, then it is easy enough to define some other unique value (e.g. a custom dummy class) to represent the end of the list.

## Exercises

* Create a program to count by prime numbers. Ask the user to input a number, then print each prime number up to that number.
* Instruct the user to pick an arbitrary number from 1 to 100 and proceed to guess it correctly within seven tries. After each guess, the user must tell whether their number is higher than, lower than, or equal to your guess.
Solutions
| Previous: Conditional Statements | Index | Next: Functions |

# Functions

| Previous: Loops | Index | Next: Scoping |

## Function Calls

A callable object is an object that can accept some arguments (also called parameters) and possibly return an object (often a tuple containing multiple objects).
A function is the simplest callable object in Python, but there are others, such as classes or certain class instances.

#### Defining Functions

A function is defined in Python by the following format:
If a function takes no arguments, it must still include the parentheses, but without anything in them:
The arguments in the function definition bind the arguments passed at function invocation (i.e. when the function is called), which are called actual parameters, to the names given when the function is defined, which are called formal parameters. The interior of the function has no knowledge of the names given to the actual parameters; the names of the actual parameters may not even be accessible (they could be inside another function).
A function can 'return' a value, for example:
A function can define variables within the function body, which are considered 'local' to the function. The locals together with the arguments comprise all the variables within the scope of the function. Any names within the function are unbound when the function returns or reaches the end of the function body.
You can return multiple values as follows:
Keywords: returning multiple values, multiple return values.

### Declaring Arguments

When calling a function that takes some values for further processing, we need to send some values as Function Arguments. For example:

#### Default Argument Values

If any of the formal parameters in the function definition are declared with the format "arg = value," then you will have the option of not specifying a value for those arguments when calling the function. If you do not specify a value, then that parameter will have the default value given when the function executes.
Links:
* 4.7.1. Default Argument Values, The Python Tutorial, docs.python.org

#### Variable-Length Argument Lists

Python allows you to declare two special arguments which allow you to create arbitrary-length argument lists. This means that each time you call the function, you can specify any number of arguments above a certain number.
When calling the above function, you must provide value for each of the first two arguments. However, since the third parameter is marked with an asterisk, any actual parameters after the first two will be packed into a tuple and bound to "remaining."
If we declare a formal parameter prefixed with two asterisks, then it will be bound to a dictionary containing any keyword arguments in the actual parameters which do not correspond to any formal parameters. For example, consider the function:
If we call this function with any keyword arguments other than max_length, they will be placed in the dictionary "entries." If we include the keyword argument of max_length, it will be bound to the formal parameter max_length, as usual.
Links:
* 4.7.3. Arbitrary Argument Lists, The Python Tutorial, docs.python.org

#### By Value and by Reference

Objects passed as arguments to functions are passed by reference; they are not being copied around. Thus, passing a large list as an argument does not involve copying all its members to a new location in memory. Note that even integers are objects. However, the distinction of by value and by reference present in some other programming languages often serves to distinguish whether the passed arguments can be actually changed by the called function and whether the calling function can see the changes.
Passed objects of mutable types such as lists and dictionaries can be changed by the called function and the changes are visible to the calling function. Passed objects of immutable types such as integers and strings cannot be changed by the called function; the calling function can be certain that the called function will not change them. For mutability, see also Data Types chapter.
An example:

### Preventing Argument Change

If an argument is of an immutable type any changes made to it will remain local to the called function. However, if the argument is of a mutable type, such as a list, changes made to it will update the corresponding value in the calling function. Thus, if the calling function wants to make sure its mutable value passed to some unknown function will not be changed by it must create and pass a copy of the value.
An example:

### Calling Functions

A function can be called by appending the arguments in parentheses to the function name or an empty pair of parentheses if the function takes no arguments.
A function's return value can be used by assigning it to a variable, like so:
As shown above, when calling a function you can specify the parameters by name and you can do so in any order
This above is valid and start will have the default value of 0. A restriction placed on this is after the first named argument then all arguments after it must also be named. The following is not valid
because the third argument ("my message") is an unnamed argument.

## Nested functions

Nested functions are functions defined within other functions. Arbitrary level of nesting is possible.
Nested functions can read variables declared in the immediately outside function. For such variables that are mutable, nested functions can even modify them. For such variables that are immutable such as integers, attempt at modification in the nested function throws UnboundLocalError. In Python 3, an immutable immediately outside variable can be declared in the nested function to be nonlocal, in an analogy to global. Once this is done, the nested function can assign a new value to that variable and that modification is going to be seen outside of the nested function.
Nested functions can be used in #Closures, as shown below. Furthermore, they can be used to reduce repetion of code that pertains only to a single function, often with reduced argument list owing to seeing the immediately outside variables.
An example of a nested function that modifies an immediately outside variable that is a list and therefore mutable:
An example in which the outside variable is first accessed below the nested function definition and it still works:
Keywords: inner functions, internal functions, local functions.
Links:
* Nested Function in Python stackoverflow.com
* 7. Simple statements # 7.13. The nonlocal statement, docs.python.org/3
* Python nonlocal statement, stackoverflow.com

## Lambda Expressions

A lambda is an anonymous (unnamed) function. It is used primarily to write very short functions that are a hassle to define in the normal way. A function like this:
may also be defined using lambda
Lambda is often used as an argument to other functions that expects a function object, such as sorted's 'key' argument.
The lambda form is often useful as a closure, such as illustrated in the following example:
Note that the lambda function can use the values of variables from the scope in which it was created similar to regular locally defined functions described above. In fact, exporting the precalculations embodiedby its constructor function is one of the essential utilities of closures.
Links:
* 4.7.5. Lambda Expressions, The Python Tutorial, docs.python.org

## Generator Functions

When discussing loops, you came across the concept of an iterator. This yields in turn each element of some sequence, rather than the entire sequence at once, allowing you to deal with sequences much larger than might be able to fit in memory at once.
You can create your own iterators, by defining what is known as a generator function. To illustrate the usefulness of this, let us start by considering a simple function to return the concatenation of two lists:
Imagine wanting to do something like concat(list(range(0, 1000000)), list(range(1000000, 2000000)))
That would work, but it would consume a lot of memory.
Consider an alternative definition, which takes two iterators as arguments:
Notice the use of the yield statement, instead of return. We can now use this something like
and print out an awful lot of numbers, without using a lot of memory at all.
Note: You can still pass a list or other sequence type wherever Python expects an iterator (like to an argument of your concat function); this will still work, and makes it easy not to have to worry about the difference where you don’t need to.
Links:
* Functional Programming HOWTO # Generators, docs.python.org

## External Links

* 4.6. Defining Functions, The Python Tutorial, docs.python.org
| Previous: Loops | Index | Next: Scoping |

# Scoping

| Previous: Functions | Index | Next: Classes |

### Variables

Variables in Python are automatically declared by assignment. Variables are always references to objects, and are never typed. Variables exist only in the current scope or global scope. When they go out of scope, the variables are destroyed, but the objects to which they refer are not (unless the number of references to the object drops to zero).
Scope is delineated by function and class blocks. Both functions and their scopes can be nested. So therefore
Now when this code is tested,
The name 'bar' is not found because a higher scope does not have access to the names lower in the hierarchy.
It is a common pitfall to fail to assign an object to a variable before use. In its most common form:
Here, to correct this problem, one must add y = before the for loop executes.
A loop does not create its own scope:

### Keyword global

Global variables of a Python module are read-accessible from functions in that module. In fact, if they are mutable, they can be also modified via method call. However, they cannot be modified by a plain assignment unless they are declared global in the function.
An example to clarify:
Links:
* 6.13. The global statement, docs.python.org
* What are the rules for local and global variables in Python? in Programming FAQ, docs.python.org

### Keyword nonlocal

Keyword nonlocal, available since Python 3.0, is an analogue of global for nested scopes. It enables a nested function to assign-modify even an immutable variable that is local to the outer function.
An example:
Simulation of nonlocal in Python 2 via a mutable object:
Links:
* 7.13. The nonlocal statement, docs.python.org

### globals and locals

To find out which variables exist in the global and local scopes, you can use locals and globals functions, which return dictionaries:
Write access to locals dictionary is discouraged by the Python documentation.
Links:
* 2. Built-in Functions # globals, docs.python.org
* 2. Built-in Functions # locals, docs.python.org

### External links

* 4. Execution model, docs.python.org
* 7.13. The nonlocal statement, docs.python.org
* PEP 3104 -- Access to Names in Outer Scopes, python.org
| Previous: Functions | Index | Next: Classes |

# Input and Output

| Previous: Scoping | Index | Next: Files |

## Input

Python 3.x has one function for input from user, input. By contrast, legacy Python 2.x has two functions for input from user: input and raw_input.
There are also very simple ways of reading a file and, for stricter control over input, reading from stdin if necessary.

### input in Python 3.x

In Python 3.x, input asks the user for a string of data (ended with a newline), and simply returns the string. It can also take an argument, which is displayed as a prompt before the user enters the data. E.g.
prints out
Example: to assign the user's name, i.e. string data, to a variable "x" you would type
In legacy Python 2.x, the above applies to what was raw_input function, and there was also input function that behaved differently, automatically evaluating what the user entered; in Python 3, the same would be achieved via eval(input).
Links:
* input in Built-in Functions in Library Reference for Python 3, docs.python.org
* raw_input in Built-in Functions in Library Reference for Python 2, docs.python.org

### input in Python 2.x

In legacy Python 2.x, input takes the input from the user as a string and evaluates it.
Therefore, if a script says:
it is possible for a user to input:
which yields the correct answer in list form. Note that no inputted statement can span more than one line.
input should not be used for anything but the most trivial program, for security reasons. Turning the strings returned from raw_input into Python types using an idiom such as:
is preferable, as input uses eval to turn a literal into a Python type, which allows a malicious person to run arbitrary code from inside your program trivially.
Links:
* input in Built-in Functions in Library Reference for Python 2, docs.python.org

### File Input
#### File Objects

To read from a file, you can iterate over the lines of the file using open:
This will print the first character of each line. A newline is attached to the end of each line read this way. The second argument to open can be 'r', 'w', or 'rw', among some others.
The newer and better way to read from a file:
The advantage is that the opened file will close itself after finishing the part within the with statement, and will do so even if an exception is thrown.
Because files are automatically closed when the file object goes out of scope, there is no real need to close them explicitly. So, the loop in the previous code can also be written as:
You can read a specific numbers of characters at a time:
This will read the characters from f one at a time, and then print them if they're not whitespace.
A file object implicitly contains a marker to represent the current position. If the file marker should be moved back to the beginning, one can either close the file object and reopen it or just move the marker back to the beginning with:

#### Standard File Objects

There are built-in file objects representing standard input, output, and error. These are in the sys module and are called stdin, stdout, and stderr. There are also immutable copies of these in __stdin__, __stdout__, and __stderr__. This is for IDLE and other tools in which the standard files have been changed.
You must import the sys module to use the special stdin, stdout, stderr I/O handles.
For finer control over input, use sys.stdin.read. To implement the UNIX 'cat' program in Python, you could do something like this:
Note that sys.stdin.read will read from standard input till EOF. (which is usually Ctrl+D.)

### Parsing command line

Command-line arguments passed to a Python program are stored in sys.argv list. The first item in the list is name of the Python program, which may or may not contain the full path depending on the manner of invocation. sys.argv list is modifiable.
Printing all passed arguments except for the program name itself:
Parsing passed arguments for passed minus options:
Above, the arguments at which options are found are removed so that sys.argv can be looped for all remaining arguments.
Parsing of command-line arguments is further supported by library modules optparse (deprecated), argparse (since Python 2.7) and getopt (to make life easy for C programmers).
A minimum parsing example for argparse:
Parse with argparse--specify the arg type as int:
Parse with argparse--add optional switch -m to yield multiplication instead of addition:
Parse with argparse--set an argument to consume one or more items:
Usage example: python ArgparseTest.py 1 3 5
Parse with argparse--as above but with a help epilog to be output after parameter descriptions upon -h:
Parse with argparse--make second integer argument optional via nargs:
Links:
* The Python Standard Library - 28.1. sys, docs.python.org
* The Python Standard Library - 15.4. argparse, docs.python.org
* The Python Standard Library - 15.5. optparse, docs.python.org
* The Python Standard Library - 15.6. getopt, docs.python.org
* Argparse Tutorial, docs.python.org

## Output

The basic way to do output is the print statement.
To print multiple things on the same line separated by spaces, use commas between them:
This will print out the following:
While neither string contained a space, a space was added by the print statement because of the comma between the two objects. Arbitrary data types can be printed:
This will output the following:
Objects can be printed on the same line without needing to be on the same line:
This will output the following:
To end the printed line with a newline, add a print statement without any objects.
This will output the following:
If the bare print statement were not present, the above output would look like:
You can print to a file instead of to standard output:
This will print to any object that implements write, which includes file objects.
Note on legacy Python 2: in Python 2, print is a statement rather than a function and there is no need to put brackets around its arguments. Instead of print(i, end=" "), one would write print i,.

### Omitting newlines

In Python 3.x, you can output without a newline by passing end="" to the print function or by using the method write:
In Python 2.x, to avoid adding spaces and newlines between objects' output with subsequent print statements, you can do one of the following:
Concatenation: Concatenate the string representations of each object, then later print the whole thing at once.
This will output the following:
Write function: You can make a shorthand for sys.stdout.write and use that for output.
This will output the following:
You may need sys.stdout.flush to get that text on the screen quickly.

### Examples

Examples of output with Python 3.x:
* from __future__ import print_function
** Ensures Python 2.6 and later Python 2.x can use Python 3.x print function.
* print("Hello", "world")
** Prints the two words separated with a space. Notice the surrounding brackets, ununsed in Python 2.x.
* print("Hello world", end="")
** Prints without the ending newline.
* print("Hello", "world", sep="-")
** Prints the two words separated with a dash.
* print("Hello", 34)
** Prints elements of various data types, separating them by a space.
* print("Hello " + 34)
** Throws an error as a result of trying to concatenate a string and an integer.
* print("Hello " + str(34))
** Uses "+" to concatenate strings, after converting a number to a string.
* sum=2+2; print "The sum: %i" % sum
** Prints a string that has been formatted with the use of an integer passed as an argument. See also #Formatting.
* print ("Error", file=sys.stderr)
** Outputs to a file handle, in this case standard error stream.
Examples of output with Python 2.x:
* print "Hello"
* print "Hello", "world"
** Separates the two words with a space.
* print "Hello", 34
** Prints elements of various data types, separating them by a space.
* print "Hello " + 34
** Throws an error as a result of trying to concatenate a string and an integer.
* print "Hello " + str(34)
** Uses "+" to concatenate strings, after converting a number to a string.
* print "Hello",
** Prints "Hello " without a newline, with a space at the end.
* sys.stdout.write("Hello")
** Prints "Hello" without a newline. Doing "import sys" is a prerequisite. Needs a subsequent "sys.stdout.flush" in order to display immediately on the user's screen.
* sys.stdout.write("Hello\n")
** Prints "Hello" with a newline.
* print >> sys.stderr, "An error occurred."
** Prints to standard error stream.
* sys.stderr.write("Hello\n")
** Prints to standard error stream.
* sum=2+2; print "The sum: %i" % sum
** Prints a string that has been formatted with the use of an integer passed as an argument.
* formatted_string = "The sum: %i" % (2+2); print formatted_string
** Like the previous, just that the formatting happens outside of the print statement.
* print "Float: %6.3f" % 1.23456
** Outputs "Float: 1.234". The number 3 after the period specifies the number of decimal digits after the period to be displayed, while 6 before the period specifies the total number of characters the displayed number should take, to be padded with spaces if needed.
* print "%s is %i years old" % ("John", 23)
** Passes two arguments to the formatter.

### File Output

Printing numbers from 1 to 10 to a file, one per line:
With "w", the file is opened for writing. With "file=file1", print sends its output to a file rather than standard output.
Printing numbers from 1 to 10 to a file, separated with a dash:
Opening a file for appending rather than overwriting:
In Python 2.x, a redirect to a file is done like print >>file1, i.
See also Files chapter.

### Formatting

Formatting numbers and other values as strings using the string percent operator:
Formatting numbers and other values as strings using the format string method, since Python 2.6:
Formatting numbers and other values as strings using literal string interpolation, since Python 3.6:
Links:
* 5.6.2. String Formatting Operations, docs.python.org
* 2. Built-in Functions # format, docs.python.org
* 7.1.2. Custom String Formatting, docs.python.org
* 7.1.3.1. Format Specification Mini-Language, docs.python.org
* 7.1.4. Template strings, docs.python.org
* PEP 3101 -- Advanced String Formatting, python.org
* PEP 498 -- Literal String Interpolation, python.org

## External Links

* 7. Input and Output in The Python Tutorial, python.org
* 6.6. The print statement in The Python Language Reference, python.org
* 2. Built-in Functions #open in The Python Standard Library at Python Documentation, python.org
* 5. Built-in Types #file.write in The Python Standard Library at Python Documentation, python.org
* 27.1. sys — System-specific parameters and functions in Python Documentation, python org -- mentions sys.stdout, and sys.stderr
* 2.3.8 File Objects in Python Library Reference, python.org, for "flush"
* 5.6.2. String Formatting Operations in The Python Standard Library at Python Documentation, python.org -- for "%i", "%s" and similar string formatting
* 7.2.2. The string format operator, in Python 2.5 quick reference, nmt.edu, for "%i", "%s" and similar string formatting
| Previous: Scoping | Index | Next: Files |

# Files

| Previous: Input and Output | Index | Next: Modules |

## File I/O

Read entire file:
In this case the "r" parameter means the file will be opened in read-only mode.
Read certain amount of bytes from a file:
When opening a file, one starts reading at the beginning of the file, if one would want more random access to the file, it is possible to use seek to change the current position in a file and tell to get to know the current position in the file. This is illustrated in the following example:
Here a file is opened, twice ten bytes are read, tell shows that the current offset is at position 20, now seek is used to go back to position 10 (the same position where the second read was started) and ten bytes are read and printed again. And when no more operations on a file are needed the close function is used to close the file we opened.
Read one line at a time:
In this case readlines will return an array containing the individual lines of the file as array entries. Reading a single line can be done using the readline function which returns the current line as a string. This example will output an additional newline between the individual lines of the file, this is because one is read from the file and print introduces another newline.
Write to a file requires the second parameter of open to be "w", this will overwrite the existing contents of the file if it already exists when opening the file:
Append to a file requires the second parameter of open to be "a" (from append):
Note that this does not add a line break between the existing file content and the string to be added.
Since Python 2.5, you can use with keyword to ensure the file handle is released as soon as possible and to make it exception-safe:
Or one line at a time:
Related to the with keywords is Context Managers chapter.
Links:
* 7.5. The with statement, python.org
* PEP 343 -- The "with" Statement, python.org

## Testing Files

Determine whether path exists:
When working on systems such as Microsoft Windows™, the directory separators will conflict with the path string. To get around this, do the following:
A better way however is to use "raw", or r:
But there are some other convenient functions in os.path, where os.path.exists only confirms whether or not path exists, there are functions which let you know if the path is a file, a directory, a mount point or a symlink. There is even a function os.path.realpath which reveals the true destination of a symlink:

## Common File Operations

To copy or move a file, use the shutil library.
To perform a recursive copy it is possible to use copytree, to perform a recursive remove it is possible to use rmtree
To remove an individual file there exists the remove function in the os module:

## Finding Files

Files can be found using glob:
The content of a directory can be listed using listdir:
Getting a list of all items in a directory, including the nested ones:
Above, root takes value of each directory in /user/Joe Hoe including /user/Joe Hoe itself, and directories and files are only those directly present in each root.
Getting a list of all files in a directory, including the nested ones, ending in .txt, using list comprehension:
Links:
* glob, python.org
* glob, Py 3, python.org
* os.listdir, python.org
* os.walk, python.org
* os.path.join, python.org

## Current Directory

Getting current working directory:
Changing current working directory:

## External Links

* os — Miscellaneous operating system interfaces in Python documentation
* glob — Unix style pathname pattern expansion in Python documentation
* shutil — High-level file operations in Python documentation
* Brief Tour of the Standard Library in The Python Tutorial
| Previous: Input and Output | Index | Next: Modules |

# Modules

| Previous: Files | Index | Next: Classes |
Modules are a way to structure a program and create reusable libraries. A module is usually stored in and corresponds to a separate .py file. Many modules are available from the standard library. You can create your own modules. Python searches for modules in the current directory and other locations; the list of module search locations can be expanded by expanding PYTHONPATH environment variable and by other means.

## Importing a Module

To use the functions and classes offered by a module, you have to import the module:
The above imports the math standard module, making all of the functions in that module namespaced by the module name. It imports all functions and all classes, if any.
You can import the module under a different name:
You can import a single function, making it available without the module name namespace:
You can import a single function and make it available under a different name:
You can import multiple modules in a row:
You can make an import as late as in a function definition:
Such an import only takes place when the function is called.
You can import all functions from the module without the module namespace, using an asterisk notation:
However, if you do this inside a function, you get a warning in Python 2 and error in Python 3:
You can guard for a module not found:
Modules can be different kinds of things:
* Python files
* Shared Objects (under Unix and Linux) with the .so suffix
* DLL's (under Windows) with the .pyd suffix
* Directories
Modules are loaded in the order they're found, which is controlled by sys.path. The current directory is always on the path.
Directories should include a file in them called __init__.py, which should probably include the other files in the directory.
Creating a DLL that interfaces with Python is covered in another section.

## Imported Check

You can check whether a module has been imported as follows:
Links:
* 28.1. sys # sys.modules, docs.python.org

## Creating a Module
### From a File

The easiest way to create a module is by having a file called mymod.py either in a directory recognized by the PYTHONPATH variable or (even easier) in the same directory where you are working. If you have the following file mymod.py
you can already import this "module" and create instances of the object Object1.

### From a Directory

It is not feasible for larger projects to keep all classes in a single file. It is often easier to store all files in directories and load all files with one command. Each directory needs to have a __init__.py file which contains python commands that are executed upon loading the directory.
Suppose we have two more objects called Object2 and Object3 and we want to load all three objects with one command. We then create a directory called mymod and we store three files called Object1.py, Object2.py and Object3.py in it. These files would then contain one object per file but this not required (although it adds clarity). We would then write the following __init__.py file:
The first three commands tell python what to do when somebody loads the module. The last statement defining __all__ tells python what to do when somebody executes from mymod import *. Usually we want to use parts of a module in other parts of a module, e.g. we want to use Object1 in Object2. We can do this easily with an from . import * command as the following file Object2.py shows:
We can now start python and import mymod as we have in the previous section.

## Making a program usable as a module

In order to make a program usable both as a standalone program to be called from a command line and as a module, it is advisable that you place all code in functions and methods, designate one function as the main one, and call then main function when __name__ built-in equals '__main__'. The purpose of doing so is to make sure that the code you have placed in the main function is not called when your program is imported as a module; the code would be called upon import if it were placed outside of functions and methods.
Your program, stored in mymodule.py, can look as follows:
The uses of the above program can look as follows:
Links:
* The Python Tutorial - 6.1.1. Executing modules as scripts, python.org

## Extending Module Path

When import is requested, modules are searched in the directories (and zip files?) in the module path, accessible via sys.path, a Python list. The module path can be extended as follows:
Above, if ModuleFileName.py is located at /My/Path/To/Module/Directory and contains a definition of my_function, the 2nd line ensures the 3rd line actually works.
Links:
* The Module Search Path at Python doc

## Module Names

Module names seem to be limited to alphanumeric characters and underscore; dash cannot be used. While my-module.py can be created and run, importing my-module fails. The name of a module is the name of the module file minus the .py suffix.
Module names are case sensitive. If the module file is called MyModule.py, doing "import mymodule" fails while "import MyModule" is fine.
PEP 0008 recommends module names to be in all lowercase, with possible use of underscores.
Examples of module names from the standard library include math, sys, io, re, urllib, difflib, and unicodedata.
Links:
* Package and Module Names at PEP 0008 -- Style Guide for Python Code
* The Python Standard Library at docs.python.org

## Built-in Modules

For a module to be built-in is not the same as to be part of the standard library. For instance, re is not a built-in module but rather a module written in Python. By contrast, _sre is a built-in module.
Obtaining a list of built-in module names:
Links:
* 28.1. sys # sys.builtin_module_names, docs.python.org
* The Python Standard Library, docs.python.org

## External links

* 6. Modules, The Python Tutorial, python.org
* Python Module Index, python.org
* 31. Importing Modules, python.org
* Installing Python Modules, python.org
* Idioms and Anti-Idioms in Python, python.org
* Python: Why should 'from <module> import *' be prohibited?, stackoverflow.com
* Error handling when importing modules, stackoverflow.com
| Previous: Files | Index | Next: Classes |

# Classes

| Previous: Modules | Index | Next: Exceptions |
Classes are a way of aggregating similar data and functions. A class is basically a scope inside which various code (especially function definitions) is executed, and the locals to this scope become attributes of the class, and of any objects constructed by this class. An object constructed by a class is called an instance of that class.

### Overview

Classes in Python at a glance:

### Defining a Class

To define a class, use the following format:
The capitalization in this class definition is the convention, but is not required by the language. It's usually good to add at least a short explanation of what your class is supposed to do. The pass statement in the code above is just to say to the python interpreter just go on and do nothing. You can remove it as soon as you are adding your first statement.

### Instance Construction

The class is a callable object that constructs an instance of the class when called. Let's say we create a class Foo.
To construct an instance of the class, Foo, "call" the class object:
This constructs an instance of class Foo and creates a reference to it in f.

### Class Members

In order to access the member of an instance of a class, use the syntax <class instance>.<member>. It is also possible to access the members of the class definition with <class name>.<member>.

#### Methods

A method is a function within a class. The first argument (methods must always take at least one argument) is always the instance of the class on which the function is invoked. For example
If this code were executed, nothing would happen, at least until an instance of Foo were constructed, and then bar were called on that instance.

##### Why a mandatory argument?

In a normal function, if you were to set a variable, such as test = 23, you could not access the test variable. Typing test would say it is not defined. This is true in class functions unless they use the self variable.
Basically, in the previous example, if we were to remove self.x, function bar could not do anything because it could not access x. The x in setx would disappear. The self argument saves the variable into the class's "shared variables" database.

#### Why self?

You do not need to use self. However, it is a norm to use self.

#### Invoking Methods

Calling a method is much like calling a function, but instead of passing the instance as the first parameter like the list of formal parameters suggests, use the function as an attribute of the instance.
This will output
It is possible to call the method on an arbitrary object, by using it as an attribute of the defining class instead of an instance of that class, like so:
This will have the same output.

#### Dynamic Class Structure

As shown by the method setx above, the members of a Python class can change during runtime, not just their values, unlike classes in languages like C++ or Java. We can even delete f.x after running the code above.
Another effect of this is that we can change the definition of the Foo class during program execution. In the code below, we create a member of the Foo class definition named y. If we then create a new instance of Foo, it will now have this new member.

#### Viewing Class Dictionaries

At the heart of all this is a dictionary that can be accessed by "vars(ClassName)"
At first, this output makes no sense. We just saw that g had the member y, so why isn't it in the member dictionary? If you remember, though, we put y in the class definition, Foo, not g.
And there we have all the members of the Foo class definition. When Python checks for g.member, it first checks g's vars dictionary for "member," then Foo. If we create a new member of g, it will be added to g's dictionary, but not Foo's.
Note that if we now assign a value to g.y, we are not assigning that value to Foo.y. Foo.y will still be 10, but g.y will now override Foo.y
Sure enough, if we check the values:
Note that f.y will also be 10, as Python won't find 'y' in vars(f), so it will get the value of 'y' from vars(Foo).
Some may have also noticed that the methods in Foo appear in the class dictionary along with the x and y. If you remember from the section on lambda functions, we can treat functions just like variables. This means that we can assign methods to a class during runtime in the same way we assigned variables. If you do this, though, remember that if we call a method of a class instance, the first parameter passed to the method will always be the class instance itself.

#### Changing Class Dictionaries

We can also access the members dictionary of a class using the __dict__ member of the class.
If we add, remove, or change key-value pairs from g.__dict__, this has the same effect as if we had made those changes to the members of g.

### Why use classes?

Classes are special due to the fact once an instance is made, the instance is independent of all other instances. I could have two instances, each with a different x value, and they will not affect the other's x.
f.boo and g.boo will print different values.

### New Style Classes

New style classes were introduced in python 2.2. A new-style class is a class that has a built-in as its base, most commonly object. At a low level, a major difference between old and new classes is their type. Old class instances were all of type instance. New style class instances will return the same thing as x.__class__ for their type. This puts user defined classes on a level playing field with built-ins. Old/Classic classes are slated to disappear in Python 3. With this in mind all development should use new style classes. New Style classes also add constructs like properties and static methods familiar to Java programmers.
Old/Classic Class
New Style Class

#### Properties

Properties are attributes with getter and setter methods.
and since Python 2.6, with @property decorator

#### Static Methods

Static methods in Python are just like their counterparts in C++ or Java. Static methods have no "self" argument and don't require you to instantiate the class before using them. They can be defined using staticmethod
They can also be defined using the function decorator @staticmethod.

### Inheritance

Like all object oriented languages, Python provides support for inheritance. Inheritance is a simple concept by which a class can extend the facilities of another class, or in Python's case, multiple other classes. Use the following format for this:
ClassName is what is known as the derived class, that is, derived from the base classes. The derived class will then have all the members of its base classes. If a method is defined in the derived class and in the base class, the member in the derived class will override the one in the base class. In order to use the method defined in the base class, it is necessary to call the method as an attribute on the defining class, as in Foo.setx(f,5) above:
Once again, we can see what's going on under the hood by looking at the class dictionaries.
When we call g.x, it first looks in the vars(g) dictionary, as usual. Also as above, it checks vars(Bar) next, since g is an instance of Bar. However, thanks to inheritance, Python will check vars(Foo) if it doesn't find x in vars(Bar).

### Multiple inheritance

As shown in section #Inheritance, a class can be derived from multiple classes:
A tricky part about multiple inheritance is method resolution: upon a method call, if the method name is available from multiple base classes or their base classes, which base class method should be called.
The method resolution order depends on whether the class is an old-style class or a new-style class. For old-style classes, derived classes are considered from left to right, and base classes of base classes are considered before moving to the right. Thus, above, BaseClass1 is considered first, and if method is not found there, the base classes of BaseClass1 are considered. If that fails, BaseClass2 is considered, then its base classes, and so on. For new-style classes, see the Python documentation online.
Links:
* 9.5.1. Multiple Inheritance, docs.python.org
* The Python 2.3 Method Resolution Order, python.org

### Special Methods

There are a number of methods which have reserved names which are used for special purposes like mimicking numerical or container operations, among other things. All of these names begin and end with two underscores. It is convention that methods beginning with a single underscore are 'private' to the scope they are introduced within.

#### Initialization and Deletion
##### __init__

One of these purposes is constructing an instance, and the special name for this is '__init__'. __init__ is called before an instance is returned (it is not necessary to return the instance manually). As an example,
outputs
__init__ can take arguments, in which case it is necessary to pass arguments to the class in order to create an instance. For example,
outputs
Here is an example showing the difference between using __init__ and not using __init__:
outputs

##### __del__

Similarly, '__del__' is called when an instance is destroyed; e.g. when it is no longer referenced.

##### __enter__ and __exit__

These methods are also a constructor and a destructor but they're only executed when the class is instantiated with with. Example:

##### __new__

Metaclass constructor.

#### Representation

String Representation Override Functions :
| __str__ Converting an object to a string, as with the print statement or with the str conversion function, can be overridden by overriding __str__. Usually, __str__ returns a formatted version of the objects content. This will NOT usually be something that can be executed. For example: class Bar: def __init__ (self, iamthis): self.iamthis = iamthis def __str__ (self): return self.iamthis bar = Bar('apple') print(bar) outputs apple __repr__ This function is much like __str__. If __str__ is not present but this one is, this function's output is used instead for printing. __repr__ is used to return a representation of the object in string form. In general, it can be executed to get back the original object. For example: class Bar: def __init__ (self, iamthis): self.iamthis = iamthis def __repr__(self): return "Bar('%s')" % self.iamthis bar = Bar('apple') bar outputs (note the difference: it may not be necessary to put it inside a print, however in Python 2.7 it does) Bar('apple') | String Representation Override Functions Function Operator __str__str(A) __repr__repr(A) __unicode__unicode(x) (2.x only) |

#### Attributes

Attribute Override Functions :
| __setattr__ This is the function which is in charge of setting attributes of a class. It is provided with the name and value of the variables being assigned. Each class, of course, comes with a default __setattr__ which simply sets the value of the variable, but we can override it. >>> class Unchangable: ... def __setattr__(self, name, value): ... print("Nice try") ... >>> u = Unchangable >>> u.x = 9 Nice try >>> u.x Traceback (most recent call last): File "<stdin>", line 1, in ? AttributeError: Unchangable instance has no attribute 'x' __getattr___ Similar to __setattr__, except this function is called when we try to access a class member, and the default simply returns the value. >>> class HiddenMembers: ... def __getattr__(self, name): ... return "You don't get to see " + name ... >>> h = HiddenMembers >>> h.anything "You don't get to see anything" __delattr__ This function is called to delete an attribute. >>> class Permanent: ... def __delattr__(self, name): ... print(name, "cannot be deleted") ... >>> p = Permanent >>> p.x = 9 >>> del p.x x cannot be deleted >>> p.x 9 | Attribute Override Functions Function Indirect form Direct Form __getattr__getattr(A, B)A.B __setattr__setattr(A, B, C)A.B = C __delattr__delattr(A, B)del A.B |

#### Operator Overloading

Operator overloading allows us to use the built-in Python syntax and operators to call functions which we define.

##### Binary Operators

Binary Operator Override Functions :
| If a class has the __add__ function, we can use the '+' operator to add instances of the class. This will call __add__ with the two instances of the class passed as parameters, and the return value will be the result of the addition. >>> class FakeNumber: ... n = 5 ... def __add__(A,B): ... return A.n + B.n ... >>> c = FakeNumber >>> d = FakeNumber >>> d.n = 7 >>> c + d 12 To override the augmented assignment operators, merely add 'i' in front of the normal binary operator, i.e. for '+=' use '__iadd__' instead of '__add__'. The function will be given one argument, which will be the object on the right side of the augmented assignment operator. The returned value of the function will then be assigned to the object on the left of the operator. >>> c.__imul__ = lambda B: B.n - 6 >>> c *= d >>> c 1 It is important to note that the augmented assignment operators will also use the normal operator functions if the augmented operator function hasn't been set directly. This will work as expected, with "__add__" being called for "+=" and so on. >>> c = FakeNumber >>> c += d >>> c 12 | Binary Operator Override Functions Function Operator __add__A + B __sub__A - B __mul__A * B __truediv__A / B __floordiv__A // B __mod__A % B __pow__A ** B __and__A & B __or__A | B __xor__A ^ B __eq__A == B __ne__A != B __gt__A > B __lt__A < B __ge__A >= B __le__A <= B __lshift__A << B __rshift__A >> B __contains__A in B A not in B |

##### Unary Operators

Unary Operator Override Functions :
| Unary operators will be passed simply the instance of the class that they are called on. >>> FakeNumber.__neg__ = lambda A : A.n + 6 >>> -d 13 | Unary Operator Override Functions Function Operator __pos__+A __neg__-A __inv__~A __abs__abs(A) __len__len(A) |

##### Item Operators

Item Operator Override Functions :
| It is also possible in Python to override the indexing and slicing operators. This allows us to use the class[i] and class[a:b] syntax on our own objects. The simplest form of item operator is __getitem__. This takes as a parameter the instance of the class, then the value of the index. >>> class FakeList: ... def __getitem__(self,index): ... return index * 2 ... >>> f = FakeList >>> f['a'] 'aa' We can also define a function for the syntax associated with assigning a value to an item. The parameters for this function include the value being assigned, in addition to the parameters from __getitem__ >>> class FakeList: ... def __setitem__(self,index,value): ... self.string = index + " is now " + value ... >>> f = FakeList >>> f['a'] = 'gone' >>> f.string 'a is now gone' We can do the same thing with slices. Once again, each syntax has a different parameter list associated with it. >>> class FakeList: ... def __getslice___(self,start,end): ... return str(start) + " to " + str(end) ... >>> f = FakeList >>> f[1:4] '1 to 4' Keep in mind that one or both of the start and end parameters can be blank in slice syntax. Here, Python has default value for both the start and the end, as show below. >> f[:] '0 to 2147483647' Note that the default value for the end of the slice shown here is simply the largest possible signed integer on a 32-bit system, and may vary depending on your system and C compiler. __setslice__ has the parameters (self,start,end,value) We also have operators for deleting items and slices. __delitem__ has the parameters (self,index) __delslice__ has the parameters (self,start,end) Note that these are the same as __getitem__ and __getslice__. | Item Operator Override Functions Function Operator __getitem__C[i] __setitem__C[i] = v __delitem__del C[i] __getslice__C[s:e] __setslice__C[s:e] = v __delslice__del C[s:e] |

#### Other Overrides

Other Override Functions :
| Function | Operator |
| __cmp__ | cmp(x, y) |
| __hash__ | hash(x) |
| __nonzero__ | bool(x) |
| __call__ | f(x) |
| __iter__ | iter(x) |
| __reversed__ | reversed(x) (2.6+) |
| __divmod__ | divmod(x, y) |
| __int__ | int(x) |
| __long__ | long(x) |
| __float__ | float(x) |
| __complex__ | complex(x) |
| __hex__ | hex(x) |
| __oct__ | oct(x) |
| __index__ | |
| __copy__ | copy.copy(x) |
| __deepcopy__ | copy.deepcopy(x) |
| __sizeof__ | sys.getsizeof(x) (2.6+) |
| __trunc__ | math.trunc(x) (2.6+) |
| __format__ | format(x, ...) (2.6+) |

### Programming Practices

The flexibility of python classes means that classes can adopt a varied set of behaviors. For the sake of understandability, however, it's best to use many of Python's tools sparingly. Try to declare all methods in the class definition, and always use the <class>.<member> syntax instead of __dict__ whenever possible. Look at classes in C++ and Java to see what most programmers will expect from a class.

#### Encapsulation

Since all python members of a python class are accessible by functions/methods outside the class, there is no way to enforce encapsulation short of overriding __getattr__, __setattr__ and __delattr__. General practice, however, is for the creator of a class or module to simply trust that users will use only the intended interface and avoid limiting access to the workings of the module for the sake of users who do need to access it. When using parts of a class or module other than the intended interface, keep in mind that the those parts may change in later versions of the module, and you may even cause errors or undefined behaviors in the module, since encapsulation is private.

#### Doc Strings

When defining a class, it is convention to document the class using a string literal at the start of the class definition. This string will then be placed in the __doc__ attribute of the class definition.
Docstrings are a very useful way to document your code. Even if you never write a single piece of separate documentation (and let's admit it, doing so is the lowest priority for many coders), including informative docstrings in your classes will go a long way toward making them usable.
Several tools exist for turning the docstrings in Python code into readable API documentation, e.g., EpyDoc.
Don't just stop at documenting the class definition, either. Each method in the class should have its own docstring as well. Note that the docstring for the method explode in the example class Documented above has a fairly lengthy docstring that spans several lines. Its formatting is in accordance with the style suggestions of Python's creator, Guido van Rossum in PEP 8.

#### Adding methods at runtime
##### To a class

It is fairly easy to add methods to a class at runtime. Lets assume that we have a class called Spam and a function cook. We want to be able to use the function cook on all instances of the class Spam:
This will output

##### To an instance of a class

It is a bit more tricky to add methods to an instance of a class that has already been created. Lets assume again that we have a class called Spam and we have already created eggs. But then we notice that we wanted to cook those eggs, but we do not want to create a new instance but rather use the already created one:
Now we can cook our eggs and the last statement will output:

##### Using a function

We can also write a function that will make the process of adding methods to an instance of a class easier.
All we now need to do is call the attach_method with the arguments of the function we want to attach, the instance we want to attach it to and the class the instance is derived from. Thus our function call might look like this:
Note that in the function add_method we cannot write instance.fxn = f since this would add a function called fxn to the instance.

### External links

* 9. Classes, docs.python.org
* 2. Built-in Functions # vars, docs.python.org
| Previous: Modules | Index | Next: Errors |

# Exceptions

| Previous: Errors | Index | Next: Source Documentation and Comments |
Python 2 handles all errors with exceptions.
An exception is a signal that an error or other unusual condition has occurred. There are a number of built-in exceptions, which indicate conditions like reading past the end of a file, or dividing by zero. You can also define your own exceptions.

### Overview

Exceptions in Python at a glance:

### Raising exceptions

Whenever your program attempts to do something erroneous or meaningless, Python raises exception to such conduct:
This traceback indicates that the ZeroDivisionError exception is being raised. This is a built-in exception -- see below for a list of all the other ones.

### Catching exceptions

In order to handle errors, you can set up exception handling blocks in your code. The keywords try and except are used to catch exceptions. When an error occurs within the try block, Python looks for a matching except block to handle it. If there is one, execution jumps there.
If you execute this code:
Then Python will print this:
You can't divide by zero!
If you don't specify an exception type on the except line, it will cheerfully catch all exceptions. This is generally a bad idea in production code, since it means your program will blissfully ignore unexpected errors as well as ones which the except block is actually prepared to handle.
Exceptions can propagate up the call stack:
In this code, the print statement calls the function f. That function calls the function g, which will raise an exception of type ValueError. Neither f nor g has a try/except block to handle ValueError. So the exception raised propagates out to the main code, where there is an exception-handling block waiting for it. This code prints:
That value was invalid.
Sometimes it is useful to find out exactly what went wrong, or to print the python error text yourself. For example:
Which will print:
Sorry, 'the_parrot' has apparently joined the choir invisible.

#### Custom Exceptions

Code similar to that seen above can be used to create custom exceptions and pass information along with them. This can be very useful when trying to debug complicated projects. Here is how that code would look; first creating the custom exception class:
And then using that exception:

#### Trying over and over again
### Recovering and continuing with finally

Exceptions could lead to a situation where, after raising an exception, the code block where the exception occurred might not be revisited. In some cases this might leave external resources used by the program in an unknown state.
finally clause allows programmers to close such resources in case of an exception. Between 2.4 and 2.5 version of python there is change of syntax for finally clause.
* Python 2.4
* Python 2.5

### Built-in exception classes

All built-in Python exceptions

### Exotic uses of exceptions

Exceptions are good for more than just error handling. If you have a complicated piece of code to choose which of several courses of action to take, it can be useful to use exceptions to jump out of the code as soon as the decision can be made. The Python-based mailing list software Mailman does this in deciding how a message should be handled. Using exceptions like this may seem like it's a sort of GOTO -- and indeed it is, but a limited one called an escape continuation. Continuations are a powerful functional-programming tool and it can be useful to learn them.
Just as a simple example of how exceptions make programming easier, say you want to add items to a list but you don't want to use "if" statements to initialize the list we could replace this:
Using exceptions, we can emphasize the normal program flow—that usually we just extend the list—rather than emphasizing the unusual case:

### External links

* 8. Errors and Exceptions in The Python Tutorial, docs.python.org
* 8. Errors and Exceptions in The Python Tutorial for Python 2.4, docs.python.org
* 6. Built-in Exceptions, docs.python.org
| Previous: Errors | Index | Next: Source Documentation and Comments |

# Errors

| Previous: Modules | Index | Next: Exceptions |
In python there are three types of errors; syntax errors, logic errors and exceptions.

## Syntax errors

Syntax errors are the most basic type of error. They arise when the Python parser is unable to understand a line of code. Syntax errors are almost always fatal, i.e. there is almost never a way to successfully execute a piece of code containing syntax errors. Some syntax errors can be caught and handled, like eval(""), but these are rare.
In IDLE, it will highlight where the syntax error is. Most syntax errors are typos, incorrect indentation, or incorrect arguments. If you get this error, try looking at your code for any of these.

## Logic errors

These are the most difficult type of error to find, because they will give unpredictable results and may crash your program. A lot of different things can happen if you have a logic error. However these are very easy to fix as you can use a debugger, which will run through the program and fix any problems.
A simple example of a logic error can be showcased below, the while loop will compile and run however, the loop will never finish and may crash Python:
Logic errors are only erroneous in the perspective of the programming goal one might have; in many cases Python is working as it was intended, just not as the user intended. The above while loop is functioning correctly as Python is intended to, but the exit condition the user needs is missing.

## Exceptions

Exceptions arise when the python parser knows what to do with a piece of code but is unable to perform the action. An example would be trying to access the internet with python without an internet connection; the python interpreter knows what to do with that command but is unable to perform it.

### Dealing with exceptions

Unlike syntax errors, exceptions are not always fatal. Exceptions can be handled with the use of a try statement.
Consider the following code to display the HTML of the website 'example.com'. When the execution of the program reaches the try statement it will attempt to perform the indented code following, if for some reason there is an error (the computer is not connected to the internet or something) the python interpreter will jump to the indented code below the 'except:' command.
Another way to handle an error is to except a specific error.
If the user enters a numeric value as his/her age, the output should look like this:
However, if the user enters a non-numeric value as his/her age, a ValueError is thrown when trying to execute the int method on a non-numeric string, and the code under the except clause is executed:
You can also use a try block with a while loop to validate input:
The program will prompt you for your age until you enter a valid age:
In certain other cases, it might be necessary to get more information about the exception and deal with it appropriately. In such situations the except as construct can be used.
| Previous: Modules | Index | Next: Exceptions |

# Source Documentation and Comments

| Previous: Errors | Index | Next: Idioms |
Documentation is the process of leaving information about your code. The two mechanisms for doing this in Python are comments and documentation strings.

## Comments

There will always be a time in which you have to return to your code. Perhaps it is to fix a bug, or to add a new feature. Regardless, looking at your own code after six months is almost as bad as looking at someone else's code. What one needs is a means to leave reminders to yourself as to what you were doing.
For this purpose, you leave comments. Comments are little snippets of text embedded inside your code that are ignored by the Python interpreter. A comment is denoted by the hash character (#) and extends to the end of the line. For example:
As you can see, you can also use comments to temporarily remove segments of your code, like the second print statement.

### Comment Guidelines

The following guidelines are from PEP 8, written by Guido van Rossum.
* General
** Comments that contradict the code are worse than no comments. Always make a priority of keeping the comments up-to-date when the code changes!
** Comments should be complete sentences. If a comment is a phrase or sentence, its first word should be capitalized, unless it is an identifier that begins with a lower case letter (never alter the case of identifiers!).
** If a comment is short, the period at the end can be omitted. Block comments generally consist of one or more paragraphs built out of complete sentences, and each sentence should end in a period.
** You should use two spaces after a sentence-ending period.
** When writing English, Strunk and White applies.
** Python coders from non-English speaking countries: please write your comments in English, unless you are 120% sure that the code will never be read by people who don't speak your language.
* Inline Comments
** An inline comment is a comment on the same line as a statement. Inline comments should be separated by at least two spaces from the statement. They should start with a # and a single space.
** Inline comments are unnecessary and in fact distracting if they state the obvious. Don't do this: x = x + 1 # Increment x But sometimes, this is useful: x = x + 1 # Compensate for border

## Documentation Strings

But what if you just want to know how to use a function, class, or method? You could add comments before the function, but comments are inside the code, so you would have to pull up a text editor and view them that way. But you can't pull up comments from a C extension, so that is less than ideal. You could always write a separate text file with how to call the functions, but that would mean that you would have to remember to update that file. If only there was a mechanism for being able to embed the documentation and get at it easily...
Fortunately, Python has such a capability. Documentation strings (or docstrings) are used to create easily-accessible documentation. You can add a docstring to a function, class, or module by adding a string as the first indented statement. For example:
The convention is to use triple-quoted strings, because it makes it easier to add more documentation spanning multiple lines.
To access the documentation, you can use the help function inside a Python shell with the object you want help on, or you can use the pydoc command from your system's shell. If we were in the directory where docstringexample.py lives, one could enter pydoc docstringexample to get documentation on that module.
| Previous: Errors | Index | Next: Idioms |

# Idioms

| Previous: Source Documentation and Comments | Index | Next: Decorators |
Python is a strongly idiomatic language: there is generally a single optimal way of doing something (a programming idiom), rather than many ways: “There’s more than one way to do it” is not a Python motto.
This section starts with some general principles, then goes through the language, highlighting how to idiomatically use operations, data types, and modules in the standard library.

## Principles

Use exceptions for error-checking, following EAFP (It's Easier to Ask Forgiveness than Permission) instead of LBYL (Look Before You Leap): put an action that may fail inside a try...except block.
Use context managers for managing resources, like files. Use finally for ad hoc cleanup, but prefer to write a context manager to encapsulate this.
Use properties, not getter/setter methods.
Use dictionaries for dynamic records, classes for static records (for simple classes, use collections.namedtuple): if a record always has the same fields, make this explicit in a class; if the fields may vary (be present or not), use a dictionary.
Use _ for throwaway variables, like discarding a return value when a tuple is returned, or to indicate that a parameter is being ignored (when required for an interface, say). You can use *_, **__ to discard positional or keyword arguments passed to a function: these correspond to the usual *args, **kwargs parameters, but explicitly discarded. You can also use these in addition to positional or named parameters (following the ones you use), allowing you to use some and discard any excess ones.
Use implicit True/False (truthy/falsy values), except when needing to distinguish between falsy values, like None, 0, and, in which case use an explicit check like is None or == 0.
Use the optional else clause after try, for, while not just if.

## Imports

For readable and robust code, only import modules, not names (like functions or classes), as this creates a new (name) binding, which is not necessarily in sync with the existing binding. For example, given a module m which defines a function f, importing the function with from m import f means that m.f and f can differ if either is assigned to (creating a new binding).
In practice, this is frequently ignored, particularly for small-scale code, as changing a module post-import is rare, so this is rarely a problem, and both classes and functions are imported from modules so they can be referred to without a prefix. However, for robust, large-scale code, this is an important rule, as it risks creating very subtle bugs.
For robust code with low typing, one can use a renaming import to abbreviate a long module name:
Note that importing submodules (or subpackages) from a package using from is completely fine:

## Operations
### Attribute access on nullable value

To access an attribute (esp. to call a method) on a value that might be an object, or might be None, use the boolean shortcircuiting of and:
Particularly useful for regex matches:

### in

Use in for substring checking.

## Data types
### All sequence types
#### Indexing during iteration

Use enumerate if you need to keep track of iteration cycles over an iterable:
Anti-idiom:

#### Finding first matching element

Python sequences do have an index method, but this returns the index of the first occurrence of a specific value in the sequence. To find the first occurrence of a value that satisfies a condition, instead, use next and a generator expression:
If you need the value, not the index of its occurrence, you can get it directly through:
The reason for this construct is twofold:
* Exceptions let you signal “no match found” (they solve the semipredicate problem): since you're returning a single value (not an index), this can't be returned in the value.
* Generator expressions let you use an expression without needing a lambda or introducing new grammar.

#### Truncating

For mutable sequences, use del, instead of reassigning to a slice:
Anti-idiom:
The simplest reason is that del makes your intention clear: you're truncating.
More subtly, slicing creates another reference to the same list (because lists are mutable), and then unreachable data can be garbage-collected, but generally this is done later. Deleting instead immediately modifies the list in-place (which is faster than creating a slice and then assigning it to the existing variable), and allows Python to immediately deallocate the deleted elements, instead of waiting for garbage collection.
In some cases you do want 2 slices of the same list – though this is rare in basic programming, other than iterating once over a slice in a for loop – but it's rare that you'll want to make a slice of a whole list, then replace the original list variable with a slice (but not change the other slice!), as in the following funny-looking code:

#### Sorted list from an iterable

You can create a sorted list directly from any iterable, without needing to first make a list and then sort it. These include sets and dictionaries (iterate on the keys):

### Tuples

Use tuples for constant sequences. This is rarely necessary (primarily when using as keys in a dictionary), but makes intention clear.

### Strings
#### Substring

Use in for substring checking.
However, do not use in to check if a string is a single-character match, since it matches substrings and will return spurious matches – instead use a tuple of valid values. For example, the following is wrong:
Instead, use a tuple:

#### Building a string

To make a long string incrementally, build a list and then join it with '' – or with newlines, if building a text file (don't forget the final newline in this case!). This is faster and clearer than appending to a string, which is often slow. (In principle can be $O(nk)$ in overall length of string and number of additions, which is $O(n^{2})$ if pieces are of similar sizes.)
However, there are some optimizations in some versions CPython that make simple string appending fast – string appending in CPython 2.5+, and bytestring appending in CPython 3.0+ are fast, but for building Unicode strings (unicode in Python 2, string in Python 3), joining is faster. If doing extensive string manipulation, be aware of this and profile your code. See Performance Tips: String Concatenation and Concatenation Test Code for details.
Don't do this:
Instead:
You can even use generator expressions, which are extremely efficient:
If you do want a mutable string-like object, you can use StringIO.
* Efficient String Concatenation in Python – old article (so benchmarks out of date), but gives overview of some techniques.

### Dictionaries

To iterate through a dictionary, either keys, values, or both:
Anti-patterns:
FIXME:
* setdefault
* usually better to use collections.defaultdict
dict.get is useful, but using dict.get and then checking if it is None as a way of testing if the key is in the dictionary is an anti-idiom, as None is a potential value, and whether the key is in the dictionary can be checked directly. It's ok to use get and compare with None if this is not a potential value, however.
Simple:
Anti-idiom (unless None is not a potential value):

#### Dict from parallel sequences of keys and values

Use zip as: dict(zip(keys, values))

## Modules
### re

Match if found, else None:
...returns None if no match, and the match contents if there is one.

## References

* ↑ “Idioms and Anti-Idioms in Python”: from module import name1, name2
* “Idioms and Anti-Idioms in Python”, Moshe Zadka

## Further reading

* “PEP 20 -- The Zen of Python”, Tim Peters
| Previous: Source Documentation and Comments | Index | Next: Decorators |

# Package management

pip is the standard Python package manager, making it easy to download and install packages from the PyPI repository. pip seems to be part of Python distribution since Python 2.7.9 and 3.5.4. If you do not have pip, you can install it by downloading get-pip.py from bootstrap.pypa.io and running python get-pip.py. Other package managers are not covered in this chapter.
Examples of pip use:
* pip install xlrd
** Installs xlrd package from the PyPI repository, or from another repository if customized to search in additional repositories.
* pip install --upgrade xlrd
** Upgrades a package to the latest version.
* pip install mypackage.whl
** Installs the package from wheel file mypackage.whl. This is useful when for whatever reason installation from PyPI fails and you need to download the wheel file (.whl) of the package manually.
* pip freeze
** Lists installed packages and their versions.
* pip show xlrd
** Outputs information about an installed package (here xlrd), including version, author and license.
* python -m pip install xlrd
** Calls pip via python and -m option. Useful e.g. for installing packages for PyPy (a just-in-time compiler for Python), in which case you use pypy -m pip install xlrd.
* pip --version
** Outputs the pip version.
* pip install --upgrade pip
** Upgrades pip itself.
PyPI is an online repository of Python packages, many of which are published under a rather permissive license such as MIT license or one of the BSD licenses. PyPI hosts both pure-Python packages and Python packages taking advantage of the C language. Installing pure-Python packages such as xlrd is usually seamless. As for C language packages, many of them have precompiled binaries for multiple operating systems, making the installation seamless as well. However, for a C language package that has only sources published, pip needs a working and properly set up compiler to successfully install the package.
A wheel file is a package distribution. It can contain pure-Python code but also precompiled executable binaries if required. A single package can offer multiple wheels per different Python versions and operating systems. An example wheel file containing precompiled binaries is numpy-1.16.2-cp27-cp27m-win32.whl, for numpy package, available from pypi Download files section for the package. If you are using pip with no problems, you do not need to worry about wheel files.

## requirements.txt

This file lists the application dependencies. It's equivalent to composer.json in PHP or package.json in JavaScript.
File generation:
Dependencies installation:

## External links

* pip - The Python Package Installer, pip.pypa.io
* Installation in pip documentation, pip.pypa.io
* How do I install pip on Windows?, stackoverflow.com
* Installing Packages, packaging.python.org
* Packaging Python Projects, packaging.python.org
* The Python Package Index (PyPI), pypi.org
* Over 10% of Python Packages on PyPI are Distributed Without Any License, 2018, snyk.io

# Python 2 vs. Python 3

Python 3 was created incompatible with Python 2. Support for Python 2.7 ended in 2020.
One noticeable difference is that in Python 3, print is not a statement but rather a function, and therefore, invoking it requires placing brackets around its arguments. Differences with deeper impact include making all strings Unicode and introducing a bytes type, making all integers big integers, letting slash (/) denote a true division rather than per default integer division, etc.; for a compact overview, see Python wiki.
Python 2 code can be made ready for a switch to Python 3 by importing features from __future__ module. For instance, from __future__ import print_function makes Python 2 behave as if it had Python 3 print function.
Python 3 was first released in 2008.
A list of Python packages ready for Python 3 is available from py3readiness.org.
A survey conducted in 2018 by JetBrains and Python Software Foundation suggests significant adoption of Python 3 among Python users.

## External links

* W:History of Python#Version 3, en.wikipedia.org
* What’s New In Python 3.0, docs.python.org
* 28.11. __future__ — Future statement definitions, docs.python.org
* Porting Python 2 Code to Python 3, docs.python.org
* Cheat Sheet: Writing Python 2-3 compatible code, python-future.org
* Should I use Python 2 or Python 3 for my development activity?, wiki.python.org
* Python3.0, wiki.python.org
* Python 3 Readiness, pyreadiness.org
* PEP 373 -- Python 2.7 Release Schedule, legacy.python.org
* Python Developers Survey 2018 Results, jetbrains.com (conducted with Python Software Foundation)
* How we rolled out one of the largest Python 3 migrations ever, 2018, blogs.dropbox.com

# Decorators

| Previous: Idioms | Index | Next: Context Managers |
Duplicated code is recognized as bad practice in software for lots of reasons, not least of which is that it requires more work to maintain. If you have the same algorithm operating twice on different pieces of data you can put the algorithm in a function and pass in the data to avoid having to duplicate the code. However, sometimes you find cases where the code itself changes, but two or more places still have significant chunks of duplicated boilerplate code. A typical example might be logging:
In a case like this, it's not obvious how to factor out the duplication. We can follow our earlier pattern of moving the common code to a function, but calling the function with different data is not enough to produce the different behavior we want (add or multiply). Instead, we have to pass a function to the common function. This involves a function that operates on a function, known as a higher-order function.
Decorator in Python is a syntax sugar for high-level function.
Minimal example of property decorator:
The above example is really just a syntax sugar for codes like this:
Minimal Example of generic decorator:
A good use for the decorators is to allow you to refactor your code so that common features can be moved into decorators. Consider for example, that you would like to trace all calls to some functions and print out the values of all the parameters of the functions for each invocation. Now you can implement this in a decorator as follows:
Then you can use the decorator on any function that you defined by:
On running this code you would see output like
Alternately, instead of creating the decorator as a class, you could have used a function as well.
Remember it is good practice to return the function or a sensible decorated replacement for the function so that decorators can be chained.
| Previous: Idioms | Index | Next: Context Managers |

# Context Managers

| Previous: Reflection | Index | Next: Tips and Tricks |
A basic issue in programming is resource management: a resource is anything in limited supply, notably file handles, network sockets, locks, etc., and a key problem is making sure these are released after they are acquired. If they are not released, you have a resource leak, and the system may slow down or crash. More generally, you may want cleanup actions to always be done, other than simply releasing resources.
Python provides special syntax for this in the with statement, which automatically manages resources encapsulated within context manager types, or more generally performs startup and cleanup actions around a block of code. You should always use a with statement for resource management. There are many built-in context manager types, including the basic example of File, and it is easy to write your own. The code is not hard, but the concepts are slightly subtle, and it is easy to make mistakes.

## Basic resource management

Basic resource management uses an explicit pair of open...close functions, as in basic file opening and closing. Don't do this, for the reasons we are about to explain:
The key problem with this simple code is that it fails if there is an early return, either due to a return statement or an exception, possibly raised by called code. To fix this, ensuring that the cleanup code is called when the block is exited, one uses a try...finally clause:
However, this still requires manually releasing the resource, which might be forgotten, and the release code is distant from the acquisition code. The release can be done automatically by instead using with, which works because File is a context manager type:
This assigns the value of open(filename) to f (this point is subtle and varies between context managers), and then automatically releases the resource, in this case calling f.close, when the block exits.

## Technical details

Newer objects are context managers (formally context manager types: subtypes, as they implement the context manager interface, which consists of __enter__, __exit__), and thus can be used in with statements easily (see With Statement Context Managers).
For older file-like objects that have a close method but not __exit__, you can use the @contextlib.closing decorator. If you need to roll your own, this is very easy, particularly using the @contextlib.contextmanager decorator.
Context managers work by calling __enter__ when the with context is entered, binding the return value to the target of as, and calling __exit__ when the context is exited. There's some subtlety about handling exceptions during exit, but you can ignore it for simple use.
More subtly, __init__ is called when an object is created, but __enter__ is called when a with context is entered.
The __init__/__enter__ distinction is important to distinguish between single use, reusable and reentrant context managers. It's not a meaningful distinction for the common use case of instantiating an object in the with clause, as follows:
...in which case any single use context manager is fine.
However, in general it is a difference, notably when distinguishing a reusable context manager from the resource it is managing, as in here:
Putting resource acquisition in __enter__ instead of __init__ gives a reusable context manager.
Notably, File objects do the initialization in __init__ and then just returns itself when entering a context, as in def __enter__: return self. This is fine if you want the target of the as to be bound to an object (and allows you to use factories like open as the source of the with clause), but if you want it to be bound to something else, notably a handle (file name or file handle/file descriptor), you want to wrap the actual object in a separate context manager. For example:
For simple uses you don't need to do any __init__ code, and only need to pair __enter__/__exit__. For more complicated uses you can have reentrant context managers, but that's not necessary for simple use.

## Caveats
### try...finally

Note that a try...finally clause is necessary with @contextlib.contextmanager, as this does not catch any exceptions raised after the yield, but is not necessary in __exit__, which is called even if an exception is raised.

### Context, not scope

The term context manager is carefully chosen, particularly in contrast to “scope”. Local variables in Python have function scope, and thus the target of a with statement, if any, is still visible after the block has exited, though __exit__ has already been called on the context manager (the argument of the with statement), and thus is often not useful or valid. This is a technical point, but it's worth distinguishing the with statement context from the overall function scope.

### Generators

Generators that hold or use resources are a bit tricky.
Beware that creating generators within a with statement and then using them outside the block does not work, because generators have deferred evaluation, and thus when they are evaluated, the resource has already been released. This is most easily seen using a file, as in this generator expression to convert a file to a list of lines, stripping the end-of-line character:
When lines is then used – evaluation can be forced with list(lines) – this fails with ValueError: I/O operation on closed file. This is because the file is closed at the end of the with statement, but the lines are not read until the generator is evaluated.
The simplest solution is to avoid generators, and instead use lists, such as list comprehensions. This is generally appropriate in this case (reading a file) since one wishes to minimize system calls and just read the file all at once (unless the file is very large):
In case that one does wish to use a resource in a generator, the resource must be held within the generator, as in this generator function:
As the nesting makes clear, the file is kept open while iterating through it.
To release the resource, the generator must be explicitly closed, using generator.close, just as with other objects that hold resources (this is the dispose pattern). This can in turn be automated by making the generator into a context manager, using @contextlib.closing, as:

### Not RAII

Resource Acquisition Is Initialization is an alternative form of resource management, particularly used in C++. In RAII, resources are acquired during object construction, and released during object destruction. In Python the analogous functions are __init__ and __del__ (finalizer), but RAII does not work in Python, and releasing resources in __del__ does not work. This is because there is no guarantee that __del__ will be called: it's just for memory manager use, not for resource handling.
In more detail, Python object construction is two-phase, consisting of (memory) allocation in __new__ and (attribute) initialization in __init__. Python is garbage-collected via reference counting, with objects being finalized (not destructed) by __del__. However, finalization is non-deterministic (objects have non-deterministic lifetimes), and the finalizer may be called much later or not at all, particularly if the program crashes. Thus using __del__ for resource management will generally leak resources.
It is possible to use finalizers for resource management, but the resulting code is implementation-dependent (generally working in CPython but not other implementations, such as PyPy) and fragile to version changes. Even if this is done, it requires great care to ensure references drop to zero in all circumstances, including: exceptions, which contain references in tracebacks if caught or if running interactively; and references in global variables, which last until program termination. Prior to Python 3.4, finalizers on objects in cycles were also a serious problem, but this is no longer a problem; however, finalization of objects in cycles is not done in a deterministic order.

## References

* ↑ Nils von Barth’s answer to “how to delete dir created by python tempfile.mkdtemp”, StackOverflow
* Context Manager Types
* With Statement Context Managers

## External links

* Get With the Program as Contextmanager
* PyMOTW (Module of the Week): contextlib
* Markus Gattol: Context Manager
| Previous: Reflection | Index | Next: Tips and Tricks |

# Reflection

| Previous: Context Managers | Index | Next: Metaclasses |
A Python script can find out about the type, class, attributes and methods of an object. This is referred to as reflection or introspection. See also Metaclasses.
Reflection-enabling functions include type, isinstance, callable, dir and getattr.

## Type

The type method enables to find out about the type of an object. The following tests return True:
* type(3) is int
* type(3.0) is float
* type(10**10) is long # Python 2
* type(1 + 1j) is complex
* type('Hello') is str
* type([1, 2]) is list
* type([1, [2, 'Hello']]) is list
* type({'city': 'Paris'}) is dict
* type((1,2)) is tuple
* type(set) is set
* type(frozenset) is frozenset
* type(3).__name__ == "int"
* type('Hello').__name__ == "str"
* import types, re, Tkinter # For the following examples
* type(re) is types.ModuleType
* type(re.sub) is types.FunctionType
* type(Tkinter.Frame) is types.ClassType
* type(Tkinter.Frame).__name__ == "classobj"
* type(Tkinter.Frame).__name__ == "instance"
* type(re.compile('myregex')).__name__ == "SRE_Pattern"
* type(type(3)) is types.TypeType
The type function disregards class inheritance: "type(3) is object" yields False while "isinstance(3, object)" yields True.
Links:
* 2. Built-in Functions # type, python.org
* 8.15. types — Names for built-in types, python.org

## Isinstance

Determines whether an object is an instance of a type or class.
The following tests return True:
* isinstance(3, int)
* isinstance([1, 2], list)
* isinstance(3, object)
* isinstance([1, 2], object)
* import Tkinter; isinstance(Tkinter.Frame, Tkinter.Frame)
* import Tkinter; Tkinter.Frame.__class__.__name__ == "Frame"
Note that isinstance provides a weaker condition than a comparison using #Type.
Function isinstance and a user-defined class:
Links:
* Built-in Functions # isinstance, python.org
* isinstance considered harmful, canonical.org

## Issubclass

Determines whether a class is a subclass of another class. Pertains to classes, not their instances.
Links:
* Built-in Functions # issubclass, python.org

## Duck typing

Duck typing provides an indirect means of reflection. It is a technique consisting in using an object as if it was of the requested type, while catching exceptions resulting from the object not supporting some of the features of the class or type.
Links:
* Glossary # duck-typing, python.org

## Callable

For an object, determines whether it can be called. A class can be made callable by providing a __call__ method.
Examples:
* callable(2)
** Returns False. Ditto for callable("Hello") and callable([1, 2]).
* callable([1,2].pop)
** Returns True, as pop without "" returns a function object.
* callable([1,2].pop)
** Returns False, as [1,2].pop returns 2 rather than a function object.
Links:
* Built-in Functions # callable, python.org

## Dir

Returns the list of names of attributes of an object, which includes methods. Is somewhat heuristic and possibly incomplete, as per python.org.
Examples:
* dir(3)
* dir("Hello")
* dir([1, 2])
* import re; dir(re)
** Lists names of functions and other objects available in the re module for regular expressions.
Links:
* Built-in Functions # dir, python.org

## Getattr

Returns the value of an attribute of an object, given the attribute name passed as a string.
An example:
* getattr(3, "imag")
The list of attributes of an object can be obtained using #Dir.
Links:
* Built-in Functions # getattr, python.org

## Keywords

A list of Python keywords can be obtained from Python:
Links:
* 32.6. keyword — Testing for Python keywords, python.org

## Built-ins

A list of Python built-in objects and functions can be obtained from Python:
Links:
* 28.3. __builtin__ — Built-in objects, python.org
* Built-in Functions # dir, python.org

## External links

* 2. Built-in Functions, docs.python.org
* How to determine the variable type in Python?, stackoverflow.com
* Differences between isinstance and type in python, stackoverflow.com
* W:Reflection (computer_programming)#Python, Wikipedia
* W:Type introspection#Python, Wikipedia
| Previous: Context Managers | Index | Next: Metaclasses |

# Metaclasses

| Previous: Reflection | Index | Next: Tips and Tricks |
In Python, classes are themselves objects. Just as other objects are instances of a particular class, classes themselves are instances of a metaclass.

### Python3

The Pep 3115 defines the changes to python 3 metaclasses. In python3 you have a method __prepare__ that is called in the metaclass to create a dictionary or other class to store the class members. Then there is the __new__ method that is called to create new instances of that class.

### The type Metaclass

The metaclass for all standard Python types is the "type" object.
Just like list, int and object, "type" is itself a normal Python object, and is itself an instance of a class. In this case, it is in fact an instance of itself.
It can be instantiated to create new class objects similarly to the class factory example above by passing the name of the new class, the base classes to inherit from, and a dictionary defining the namespace to use.
For instance, the code:
Could also be written as:

### Metaclasses

It is possible to create a class with a different metaclass than type by setting the metaclass keyword argument when defining the class. When this is done, the class, and its subclass will be created using your custom metaclass. For example
This will print
By creating a custom metaclass in this way, it is possible to change how the class is constructed. This allows you to add or remove attributes and methods, register creation of classes and subclasses creation and various other manipulations when the class is created.

### More resources

* Wikipedia article on Aspect Oriented Programming
* Unifying types and classes in Python 2.2
* O'Reilly Article on Python Metaclasses

### References

* ↑ http://www.python.org/dev/peps/pep-3115/
* ↑ http://eli.thegreenplace.net/2011/08/14/python-metaclasses-by-example/
| Previous: Reflection | Index | Next: Tips and Tricks |

# Performance

Since Python is an interpreted language in its most commonly used CPython implementation, it is many times slower in a variety of tasks than the most commonly used compiled non-managed languages such as C and C++; for some tasks, it is more than 100 times slower. CPython seems to be on par with Perl, another interpreted language, slower on some tasks, faster on other tasks.
Performance can be measured using benchmarks. Benchmarks are often far from representative of the real-world usage and have to be taken with a grain of salt. Some benchmarks are outright wrong in that non-idiomatic code is used for a language, yielding avoidably low performance for the language.
PyPy is a just-in-time (JIT) compiler that often runs faster than CPython. Another compiler that can lead to greater speeds is Numba, which works for a subset of Python. Yet another compiler is Cython, not to be confused with CPython.

## External links

* Python 3 versus C gcc fastest programs, benchmarksgame-team.pages.debian.net
* Python 3 versus Java fastest programs, benchmarksgame-team.pages.debian.net
* Python 3 versus Go fastest programs, benchmarksgame-team.pages.debian.net
* Perl vs Python 3 - Which programs are fastest?, benchmarksgame-team.pages.debian.net
* Python speed center, speed.python.org
* pypy speed center, speed.pypy.org
* Python Interpreters Benchmarks, pybenchmarks.org
* Performance of Python runtimes on a non-numeric scientific code by Riccardo Murri, arxiv.org
* How To Make Python Run As Fast As Julia by Jean Francois Puget, ibm.com

# PyPy

PyPy is a Python interpreter containing a just-in-time compiler. Python programs can usually be run by PyPy without modification but for availability of 3rd party modules since a module made for CPython, the normal Python interpreter, does not automatically work with PyPy. Furthermore, some Python programs can run into trouble because of PyPy's different strategy of when to free up allocated objects including file handles.
The speed-up brought by PyPy compared to CPython depends on the nature of the task. For some computationally heavy tasks, the speed-up factor can reach as high as 50. PyPy speed center reports the geometric average speed-up factor as 7.6, calculated from a set of benchmarks.
PyPy is available both for Python 2 and Python 3, but the version for Python 3 is slower; the above speed statements pertain to Python 2.
Interactive use of PyPy is possible: you can type "pypy" into a command line, and start interacting with it just like with CPython.
The outputs from PyPy are not guaranteed to be exactly the same as from CPython. For instance, PyPy can yield items from a set in an order different from that of CPython since the item order in a set is arbitrary and not guaranteed to the same between different Python implementations; for verification, you can compare the results of {1,2}.pop. Dictionaries have an arbitrary key order as well.
Floating point results may be slightly different between PyPy and CPython in some setups as long as PyPy is built with SSE2 instruction set enabled and CPython is not.
See also Performance.

## External links

* PyPy, pypy.org
* PyPy's Speed Center, speed.pypy.org
* PyPy, en.wikipedia.org

# Cython

Cython (not to be confused with CPython) is a compiler of Python-like source code to the C language, from which it is compiled by a C compiler to binary executable. The objective is a significant speedup compared to interpreting the Python code in CPython, the standard interpreter. Cython is usually used to create extension modules for Python. The source code language compilable by Cython is a near superset of Python.
You can install Cython using pip install Cython. However, in order for Cython to work, you will need a working C compiler. On Linux, you usually have one; on Windows, you can install and use Microsoft Visual C++ compiler or MinGW.
Beyond the normal Python, Cython-compilable Python source can contain C-like declarations of variable types, leading to speedups of the compiled code.
Cython-compilable Python source code files conventionally use extension pyx.
The compiled extension module still needs CPython (the normal Python interpreter) to run, and can call other Python modules, including pure Python modules. This is because, where required, Cython compiles to C code that uses the CPython API to achieve general Python-like behavior.

## External links

* Official website, cython.org
* Basic Tutorial, docs.cython.org
* Cython, pypi.org
* cython, github.com
* Cython, en.wikipedia.org
* Python Interpreters Benchmarks, pybenchmarks.org

# Command-line one-liners

Python can run one-liners from an operating system command line using option -c:
* python -c "print(3.0/2)"
** Calculates and outputs the result.
* python -c "import math;print(math.sin(1))"
** Imports a module required and outputs sine value.
* python -c "for i in range(1,11):print(i)"
** Uses a loop to output numbers from 1 to 10.
* python -c "for i in range(1,11):for j in range(1,11): print(i,j)"
** Does not work; two loops in a line are an invalid syntax.
* python -c "for i, j in ((i,j) for i in range (1,11) for j in range(1,11)): print(i, j)"
** Outputs pairs using two analogues of loop within a comprehension.
* echo hey | python -c "import sys,re;[sys.stdout.write(line) for line in sys.stdin if re.search('he.', line)]"
** Acts as grep: outputs each line of the input containing a substring matching a regular expression. Not a Python one-liner strength.
* echo hallo | python -c "import sys,re;[sys.stdout.write(re.sub('h[au]llo', 'hello', line)) for line in sys.stdin]"
** Acts as sed: for each line of the input, performs a regex replacement and outputs the results. Again, not a Python one-liner strength.
* python -m calendar
** Outputs a year's calendar using calendar module.
* python -c "import playsound as p;p.playsound(r'C:\WINDOWS\Media\notify.wav')"
** On Windows, plays notification sound. Requires installation of playsound module. The module works across platforms; what is Windows specific above is the file path.

## External links

* Powerful Python One-Liners, wiki.python.org

# Tips and Tricks

| Previous: Metaclasses | Index | Next: Namespace |
There are many tips and tricks you can learn in Python:

## Strings

* Triple quotes are an easy way to define a string with both single and double quotes.
* String concatenation is expensive. Use percent formatting and str.join for concatenation:
(but don't worry about this unless your resulting string is more than 500-1000 characters long)

## Optimized C modules

Several modules have optimized versions written in C, which provide an almost-identical interface and are frequently much faster or more memory-efficient than the pure Python implementations. Module behavior generally does differ in some respects, often minor, and thus C versions are frequently used.
This is primarily a Python 2.x feature, which has been largely removed in Python 3, with modules automatically using optimized implementations if available. However, the cProfile/profile pair still exists (as of Python 3.4).

### importing

The C version of a module named module or Module is called cModule, and frequently imported using import...as to strip off the prefix, as:
For compatibility, one can try to import the C version and fall back to the Python version if the C version is not available; in this case using import...as is required, so the code does not depend on which module was imported:

### Examples

Notable examples include:
* (Python 2.x) cPickle for pickle, up to 1000× faster.
* (Python 2.x) cStringIO for StringIO, replaced by io.StringIO in Python 3
* cProfile for profile – the Python profile adds significant overhead, and thus cProfile is recommended for most use.
* (not needed in Python 3.3+) cElementTree for ElementTree, 15–20 times faster and uses 2–5 times less memory; not needed in Python 3.3+, which automatically uses a fast implementation if possible.

## List comprehension and generators

* List comprehension and generator expressions are very useful for working with small, compact loops. Additionally, it is faster than a normal for-loop.
* List comprehension and generator expression can be used to work with two (or more) lists with zip or itertools.izip

## Data type choice

Choosing the correct data type can be critical to the performance of an application. For example, say you have 2 lists:
and you want to find the entries common to both lists. You could iterate over one list, checking for common items in the other:
For such small lists, this will work fine, but for larger lists, for example if each contains thousands of entries, the following will be more efficient, and produces the same result:
Sets are optimized for speed in such functions. Dictionaries themselves cannot be used as members of a set as they are mutable, but tuples can. If one needs to do set operations on a list of dictionaries, one can convert the items to tuples and the list to a set, perform the operation, then convert back. This is often much faster than trying to replicate set operations using string functions.

## Other

* Decorators can be used for handling common concerns like logging, db access, etc.
* While Python has no built-in function to flatten a list you can use a recursive function to do the job quickly.
* To stop a Python script from closing right after you launch one independently, add this code:
* Python already has a GUI built in: Tkinter, based on Tcl's Tk. More are available, such as PyQt4, pygtk3, and wxPython.
* Ternary Operators:
* Booleans as indexes:

## References

* ↑ "'concat vs join - followup' on 'Python Rocks! and other rants 27.8.2004 Weblog of Kent S Johnson'". August 27, 2004. Retrieved 2008-08-29.
* ↑ What’s New In Python 3.0, Guido van Rossum
* ↑ "The cElementTree Module", January 30, 2005, Fredrik Lundh
| Previous: Metaclasses | Index | Next: Namespace |

# Standard Library

| Previous: Namespace | Index | Next: Regular Expression |
The Python Standard Library is a collection of script modules accessible to a Python program to simplify the programming process and removing the need to rewrite commonly used commands. They can be used by 'calling/importing' them at the beginning of a script.
A list of the Standard Library modules can be found at http://www.python.org/doc/.
The following are among the most important:
* time
* sys
* os
* math
* random
* pickle
* urllib
* re
* cgi
* socket
| Previous: Namespace | Index | Next: Regular Expression |

# Regular Expression

| Previous: Standard Library | Index | Next: XML Tools |
Python includes a module for working with regular expressions on strings. For more information about writing regular expressions and syntax not specific to Python, see the regular expressions wikibook. Python's regular expression syntax is similar to Perl's
To start using regular expressions in your Python scripts, import the "re" module:

## Overview

Regular expression functions in Python at a glance:

## Matching and searching

One of the most common uses for regular expressions is extracting a part of a string or testing for the existence of a pattern in a string. Python offers several functions to do this.
The match and search functions do mostly the same thing, except that the match function will only return a result if the pattern matches at the beginning of the string being searched, while search will find a match anywhere in the string.
In this example, match2 will be None, because the string st2 does not start with the given pattern. The other 3 results will be Match objects (see below).
You can also match and search without compiling a regexp:
Here we use the search function of the re module, rather than of the pattern object. For most cases, its best to compile the expression first. Not all of the re module functions support the flags argument and if the expression is used more than once, compiling first is more efficient and leads to cleaner looking code.
The compiled pattern object functions also have parameters for starting and ending the search, to search in a substring of the given string. In the first example in this section, match2 returns no result because the pattern does not start at the beginning of the string, but if we do:
it works, because we tell it to start searching at character number 3 in the string.
What if we want to search for multiple instances of the pattern? Then we have two options. We can use the start and end position parameters of the search and match function in a loop, getting the position to start at from the previous match object (see below) or we can use the findall and finditer functions. The findall function returns a list of matching strings, useful for simple searching. For anything slightly complex, the finditer function should be used. This returns an iterator object, that when used in a loop, yields Match objects. For example:
If you're going to be iterating over the results of the search, using the finditer function is almost always a better choice.

### Match objects

Match objects are returned by the search and match functions, and include information about the pattern match.
The group function returns a string corresponding to a capture group (part of a regexp wrapped in) of the expression, or if no group number is given, the entire match. Using the search1 variable we defined above:
Capture groups can also be given string names using a special syntax and referred to by matchobj.group('name'). For simple expressions this is unnecessary, but for more complex expressions it can be very useful.
You can also get the position of a match or a group in a string, using the start and end functions:
This returns the start and end locations of the entire match, and the start and end of the first (and in this case only) capture group, respectively.

## Replacing

Another use for regular expressions is replacing text in a string. To do this in Python, use the sub function.
sub takes up to 3 arguments: The text to replace with, the text to replace in, and, optionally, the maximum number of substitutions to make. Unlike the matching and searching functions, sub returns a string, consisting of the given text with the substitution(s) made.
This takes any single alphanumeric character (\w in regular expression syntax) preceded by "a" or "an" and wraps in in single quotes. The \1 and \2 in the replacement string are backreferences to the 2 capture groups in the expression; these would be group(1) and group(2) on a Match object from a search.
The subn function is similar to sub, except it returns a tuple, consisting of the result string and the number of replacements made. Using the string and expression from before:
Replacing without constructing and compiling a pattern object:

## Splitting

The split function splits a string based on a given regular expression:

## Escaping

The escape function escapes all non-alphanumeric characters in a string. This is useful if you need to take an unknown string that may contain regexp metacharacters like ( and . and create a regular expression from it.

## Flags

The different flags use with regular expressions:
| Abbreviation | Full name | Description |
| re.I | re.IGNORECASE | Makes the regexp case-insensitive |
| re.L | re.LOCALE | Makes the behavior of some special sequences (\w, \W, \b, \B, \s, \S) dependent on the current locale |
| re.M | re.MULTILINE | Makes the ^ and $ characters match at the beginning and end of each line, rather than just the beginning and end of the string |
| re.S | re.DOTALL | Makes the . character match every character including newlines. |
| re.U | re.UNICODE | Makes \w, \W, \b, \B, \d, \D, \s, \S dependent on Unicode character properties |
| re.X | re.VERBOSE | Ignores whitespace except when in a character class or preceded by an non-escaped backslash, and ignores # (except when in a character class or preceded by an non-escaped backslash) and everything after it to the end of a line, so it can be used as a comment. This allows for cleaner-looking regexps. |

## Pattern objects

If you're going to be using the same regexp more than once in a program, or if you just want to keep the regexps separated somehow, you should create a pattern object, and refer to it later when searching/replacing.
To create a pattern object, use the compile function.
The first argument is the pattern, which matches the string "foo", followed by up to 5 of any character, then the string "bar", storing the middle characters to a group, which will be discussed later. The second, optional, argument is the flag or flags to modify the regexp's behavior. The flags themselves are simply variables referring to an integer used by the regular expression engine. In other languages, these would be constants, but Python does not have constants. Some of the regular expression functions do not support adding flags as a parameter when defining the pattern directly in the function, if you need any of the flags, it is best to use the compile function to create a pattern object.
The r preceding the expression string indicates that it should be treated as a raw string. This should normally be used when writing regexps, so that backslashes are interpreted literally rather than having to be escaped.

## External links

* re — Regular expression operations, docs.python.org
* Python Programming/RegEx, wikiversity.org
| Previous: Standard Library | Index | Next: XML Tools |

# External commands

The traditional way of executing external commands is using os.system:
The modern way, since Python 2.4, is using subprocess module:
The traditional way of executing external commands and reading their output is via popen2 module:
The modern way, since Python 2.4, is using subprocess module:
Keywords: system commands, shell commands, processes, backtick, pipe.

## External links

* 17.1. subprocess — Subprocess management, python.org, since Python 2.4
* 15.1. os — Miscellaneous operating system interfaces, python.org
* 17.5. popen2 — Subprocesses with accessible I/O streams, python.org, deprecated since Python 2.6

# XML Tools

| Previous: Regular Expression | Index | Next: Email |

## Introduction

Python includes several modules for manipulating xml.

## xml.sax.handler

Python Doc

## xml.dom.minidom

An example of doing RSS feed parsing with DOM
XML document provided by pyxml documentation.
| Previous: Regular Expression | Index | Next: Email |

# Email

| Previous: XML Tools | Index | Next: Threading |
Python includes several modules in the standard library for working with emails and email servers.

## Sending mail

Sending mail is done with Python's smtplib using an SMTP (Simple Mail Transfer Protocol) server. Actual usage varies depending on complexity of the email and settings of the email server, the instructions here are based on sending email through Google's Gmail.
The first step is to create an SMTP object, each object is used for connection with one server.
The first argument is the server's hostname, the second is the port. The port used varies depending on the server.
Next, we need to do a few steps to set up the proper connection for sending mail.
These steps may not be necessary depending on the server you connect to. ehlo is used for ESMTP servers, for non-ESMTP servers, use helo instead. See Wikipedia's article about the SMTP protocol for more information about this. The starttls function starts Transport Layer Security mode, which is required by Gmail. Other mail systems may not use this, or it may not be available.
Next, log in to the server:
Then, send the mail:
Note that this is a rather crude example, it doesn't include a subject, or any other headers. For that, one should use the email package.

## The email package

Python's email package contains many classes and functions for composing and parsing email messages, this section only covers a small subset useful for sending emails.
We start by importing only the classes we need, this also saves us from having to use the full module name later.
Then we compose some of the basic message headers:
Next, we attach the body of the email to the MIME message:
For sending the mail, we have to convert the object to a string, and then use the same procedure as above to send using the SMTP server..
If we look at the text, we can see it has added all the necessary headers and structure necessary for a MIME formatted email. See MIME for more details on the standard:
* The full text of our example message
* >>> print text
* Content-Type: multipart/mixed; boundary="===============1893313573=="
* MIME-Version: 1.0
* From: you@gmail.com
* To: target@example.com
* Subject: Python email
* --===============1893313573==
* Content-Type: text/plain; charset="us-ascii"
* MIME-Version: 1.0
* Content-Transfer-Encoding: 7bit
* Python test mail
* --===============1893313573==--
| Previous: XML Tools | Index | Next: Threading |

# Threading

| Previous: Email | Index | Next: Sockets |
Threading in python is used to run multiple threads (tasks, function calls) at the same time. Note that this does not mean that they are executed on different CPUs. Python threads will NOT make your program faster if it already uses 100 % CPU time. In that case, you probably want to look into parallel programming. If you are interested in parallel programming with python, please see here.
Python threads are used in cases where the execution of a task involves some waiting. One example would be interaction with a service hosted on another computer, such as a webserver. Threading allows python to execute other code while waiting; this is easily simulated with the sleep function.

## Examples
### A Minimal Example with Function Call

Make a thread that prints numbers from 1-10 and waits a second between each print:

### A Minimal Example with Object

The output looks like this:
| Previous: Email | Index | Next: Sockets |

# Sockets

| Previous: Threading | Index | Next: GUI Programming |

## HTTP Client

Make a very simple HTTP client

## NTP/Sockets

Connecting to and reading an NTP time server, returning the time as follows
| Previous: Threading | Index | Next: GUI Programming |

# GUI Programming

| Previous: Sockets | Index | Next: CGI interface |
There are various GUI toolkits usable from Python.
Very productive are true GUI-builders, where the programmer can arrange the GUI window and other components such as database by using the mouse only in an intuitive fashion like in Windows Delphi 2.0. Very little typing is required. For python, only Boa Constructor follows this paradigm. WXglade and Qt-designer, monkey studio etc. come somewhat near but remain incomplete.
Disadvantages with the following kits described below are:
* Difficult deployment - the apps won't run on a particular GNU-Linux installation without major additional work
* breakage - apps won't work due to bit-rot.

## Tkinter

Tkinter is a Python wrapper for Tcl/Tk providing a cross-platform GUI toolkit. On Windows, it comes bundled with Python; on other operating systems, it can be installed. The set of available widgets is smaller than in some other toolkits, but since Tkinter widgets are extensible, many of the missing compound widgets can be created using the extensibility, such as combo box and scrolling pane.
A minimal example:
Main chapter: Tkinter.
Links:
* 24.1. Tkinter — Python interface to Tcl/Tk, python.org
* Tkinter 8.5 reference: a GUI for Python, infohost.nmt.edu; the same as pdf
* An Introduction to Tkinter, effbot.org

## PyGTK

See also book PyGTK For GUI Programming
PyGTK provides a convenient wrapper for the GTK+ library for use in Python programs, taking care of many of the boring details such as managing memory and type casting. The bare GTK+ toolkit runs on Linux, Windows, and Mac OS X (port in progress), but the more extensive features — when combined with PyORBit and gnome-python — require a GNOME install, and can be used to write full featured GNOME applications.
Home Page

## PyQt

PyQt is a wrapper around the cross-platform Qt C++ toolkit. It has many widgets and support classes supporting SQL, OpenGL, SVG, XML, and advanced graphics capabilities. A PyQt hello world example:
PyQt is a set of bindings for the cross-platform Qt application framework. PyQt v4 supports Qt4 and PyQt v3 supports Qt3 and earlier.

## wxPython

Bindings for the cross platform toolkit wxWidgets. WxWidgets is available on Windows, Macintosh, and Unix/Linux.
* wxPython

## Dabo

Dabo is a full 3-tier application framework. Its UI layer wraps wxPython, and greatly simplifies the syntax.
* Dabo

## pyFltk

pyFltk is a Python wrapper for the FLTK, a lightweight cross-platform GUI toolkit. It is very simple to learn and allows for compact user interfaces.
The "Hello World" example in pyFltk looks like:

## Other Toolkits

* PyKDE - Part of the kdebindings package, it provides a python wrapper for the KDE libraries.
* PyXPCOM provides a wrapper around the Mozilla XPCOM component architecture, thereby enabling the use of standalone XUL applications in Python. The XUL toolkit has traditionally been wrapped up in various other parts of XPCOM, but with the advent of libxul and XULRunner this should become more feasible. These days, nobody uses PyXPCOM for very good reasons: PyXPCOM gives one dead links and outdated incompatible firefox extensions.

## External links

* Graphic User Interface FAQ, python.org
* An excerpt from Chapter 20: GUI Development, from Python Programming on Win32, onlamp.com, by Mark Hammond, Andy Robinson; covers Tkinter, PythonWin and wxPython
* Google Ngram Viewer: Tkinter, wxPython, wxWidgets, PyGTK, PyQt
| Previous: Sockets | Index | Next: CGI interface |

# Tkinter

Tkinter is a Python wrapper for Tcl/Tk providing a cross-platform GUI toolkit. On Windows, it comes bundled with Python; on other operating systems, it can be installed. The set of available widgets is smaller than in some other toolkits, but since Tkinter widgets are extensible, many of the missing compound widgets can be created using the extensibility, such as combo box and scrolling pane.
IDLE, Python's Integrated Development and Learning Environment, is written using Tkinter and is often distributed with Python. You can learn about features of Tkinter by playing around with menus and dialogs of IDLE. For instance, Options > Configure IDLE... dialog shows a broad variety of GUI elements including tabbed interface. You can learn about programming using Tkinter by studying IDLE source code, which, on Windows, is available e.g. in C:\Program Files\Python27\Lib\idlelib.
Python 3: The examples on this page are for Python 2. In Python 3, what was previously module Tkinter is tkinter, what was tkMessageBox is messagebox, etc.

## Minimal example

A minimal example:
A minimal example made more compact - later references to GUI items not required:
A minimal example creating an application class derived from Frame:

## Message boxes

Simple message boxes can be created using tkMessageBox as follows:
Links:
* The tkMessageBox dialogs module, infohost.nmt.edu
* Standard Dialogs, effbot.org

## File dialog

File dialogs can be created as follows:
Links:
* The tkFileDialog module, infohost.nmt.edu
* File Dialogs, effbot.org
* tkFileDialog, tkinter.unpythonic.net

## Radio button

A radio button can be used to create a simple choice dialog with multiple options:
An alternative to radio button that immediately reacts to button press:
Links:
* The Radiobutton widget, infohost.nmt.edu
* The Tkinter Radiobutton Widget, effbot.org

## List box

A list box can be used to create a simple multiple-choice dialog:
Links:
* The Listbox widget, infohost.nmt.edu
* The Tkinter Listbox Widget, effbot.org
* TKinter Listbox example (Python recipe), code.activestate.com

## Checkbox

Checkbox or check button can be created as follows:
Links:
* The Checkbutton widget, infohost.nmt.edu
* The Tkinter Checkbutton Widget, effbot.org

## Entry

Entry widget, a single-line text input field, can be used as follows:
Links:
* The Entry widget, infohost.nmt.edu
* The Tkinter Entry Widget, effbot.org

## Menu

Menus can be created as follows:
Links:
* The Menu widget, infohost.nmt.edu
* The Tkinter Menu Widget, effbot.org

## LabelFrame

A frame around other elements can be created using LabelFrame widget as follows:
Links:
* The LabelFrame widget, infohost.nmt.edu
* Tkinter LabelFrame Widget, effbot.org

## Message

Message is like Label but ready to wrap across multiple lines. An example:
Links:
* The Message widget, infohost.nmt.edu
* The Tkinter Message Widget, effbot.org

## Option menu

Drop-down list, in Tkinter option menu, can be created as follows:
Links:
* The OptionMenu widget, infohost.nmt.edu
* The Tkinter OptionMenu Widget, effbot.org

## Text

Text widget is a more complex one, allowing editing of both plain and formatted text, including multiple fonts.
Example to be added.
Links:
* The Text widget, infohost.nmt.edu
* The Tkinter Text Widget, effbot.org

## Tcl/Tk version

The Windows installer for Python 2.3 ships with Tcl/Tk 8.4.3. You can find out about the version:
Links:
* What's new in Python 2.3, python.org

## Related books

* Tcl Programming/Tk

## External links

* 24.1. Tkinter — Python interface to Tcl/Tk, python.org
* Tkinter 8.5 reference: a GUI for Python, infohost.nmt.edu; the same as pdf
* An Introduction to Tkinter, effbot.org
* Tkinter Summary, staff.washington.edu
* TkInter, wiki.python.org
* GUI Tk « Python, java2s.com, many examples

# CGI interface

| Previous: GUI Programming | Index | Next: WSGI web programming |
The Common Gateway Interface (CGI) allows to execute some Python programs on an HTTP server.

## Installation

By default, open a .py file in HTTP returns its content. In order to make the server compile and execute the source code, it must be placed in a directory including an .htaccess file, with the lines:
Attention: on the Unix-like servers the files aren't executable by default, so this must be set with the command: chmod +x *.py.

## Examples

The module cgitb is used for debugging:
The usage of a form needs an import cgi.
For a MySQL database, its import MySQLdb.
The following file is called CGI_MySQL.py, and uses the both modules:

## References

* ↑ "HOWTO Use Python in the web".
* ↑ http://fr.openclassrooms.com/informatique/cours/apercu-de-la-cgi-avec-python
* ↑ https://pypi.python.org/pypi/MySQL-python/1.2.5
| Previous: GUI Programming | Index | Next: WSGI web programming |

# WSGI web programming

| Previous: CGI interface | Index | Next: Web Page Harvesting |

# WSGI Web Programming
## External Resources

http://docs.python.org/library/wsgiref.html
| Previous: CGI interface | Index | Next: Web Page Harvesting |

# Internet

| Previous: Web Page Harvesting | Index | Next: Networks |
The urllib module which is bundled with python can be used for web interaction. This module provides a file-like interface for web urls.

## Getting page text as a string

An example of reading the contents of a webpage
Processing page text line by line:
Get and post methods can be used, too.

## Downloading files

To save the content of a page on the internet directly to a file, you can read it and save it as a string to a file object
This will download the file from here and save it to a file "pythonbook.pdf" on your hard drive.

## Other functions

The urllib module includes other functions that may be helpful when writing programs that use the internet:
The urlencode function, described above converts a dictionary of key-value pairs into a query string to pass to a URL, the quote and quote_plus functions encode normal strings. The quote_plus function uses plus signs for spaces, for use in submitting data for form fields. The unquote and unquote_plus functions do the reverse, converting urlencoded text to plain text.

## Email

With Python, MIME compatible emails can be sent. This requires an installed SMTP server.
This sends the sample message from 'sender@example.com' to 'receiver@example.com'.

## External links

* urllib.request, docs.python.org
* HOWTO Fetch Internet Resources Using The urllib Package, docs.python.org
* urllib2 for Python 2, docs.python.org
* HOWTO Fetch Internet Resources Using urllib2 — Python 2.7, docs.python.org
| Previous: Web Page Harvesting | Index | Next: Networks |

# Networks

| Previous: Internet | Index | Next: Databases |

## Sockets

Python can also communicate via sockets.

### Connecting to a server

This simple Python program will fetch a 4096 byte HTTP response from Google:

## High-level interfaces

Most Python developers will prefer to use a high-level interface over using sockets, such as Twisted and urllib2.
| Previous: Internet | Index | Next: Databases |

# Math

For basic math including addition, subtraction, multiplication and the like, see Basic Math and Operators chapters. For quick reference, the built-in Python math operators include addition (+), subtraction (-), multiplication (*), division (/), floor division (//), modulo (%), and exponentiation (**). The built-in Python math functions include rounding (round), absolute value (abs), minimum (min), maximum (max), division with a remainder (divmod), and exponentiation (pow). Sign function can be created as "sign = lambda n: 1 if n > 0 else -1 if n < 0 else 0".

## Math

A range of mathematical functions is available from math module of the standard library:

## Example code using in-built operators

This code was made to replicate the log function in a calculator

## Cmath

The cmath module provides similar functions like the math module but for complex numbers, and then some.

## Random

Pseudo-random generators are available from the random module:

## Decimal

The decimal module enables decimal floating point arithmethic, avoiding certain artifacts of the usual underlying binary representation of floating point numbers that are unintuitive to humans.

## Fractions

The fractions module provides fraction arithmetic via Fraction class. Compared to floating point numbers representing fractions, Fraction fractions do not lose precision.

## Statistics

The statistics module, available since Python 3.4, provides some basic statistical functions. It only provides basics; it does not replace full-fledged 3rd party libraries such as numpy. For Python 2.7, the statistics module can be installed from pypi.

## External links

* 2. Built-in Functions, python.org
* 9. Numeric and Mathematical Modules, python.org
* 9.2. math — Mathematical functions, python.org
* 9.3. cmath — Mathematical functions for complex numbers, python.org
* 9.4. decimal — Decimal fixed point and floating point arithmetic, python.org
* 9.5. fractions — Rational numbers, python.org
* 9.6. random — Generate pseudo-random numbers, python.org
* 9.7. itertools — Functions creating iterators for efficient looping, python.org
* 9.7. statistics — Mathematical statistics functions, python.org

# Databases

| Previous: Networks | Index | Next: Database Programming |
Python has support for working with databases via a simple API. Modules included with Python include modules for SQLite and Berkeley DB. Modules for MySQL , PostgreSQL , FirebirdSQL and others are available as third-party modules. The latter have to be downloaded and installed before use. The package MySQLdb can be installed, for example, using the Debian package "python-mysqldb".

## DBMS Specifics
### MySQL

An Example with MySQL would look like this:
On the first line, the Module MySQLdb is imported. Then a connection to the database is set up and on line 4, we save the actual SQL statement to be executed in the variable query. On line 5 we execute the query and on line 6 we fetch all the data. After the execution of this piece of code, lines contains the number of lines fetched (e.g. the number of rows in the table sampletable). The variable data contains all the actual data, e.g. the content of sampletable. In the end, the connection to the database would be closed again. If the number of lines are large, it is better to use row = cursor.fetchone and process the rows individually:
Obviously, some kind of data processing has to be used on a row, otherwise the data will not be stored. The result of the fetchone command is a Tuple.
In order to make the initialization of the connection easier, a configuration file can be used:
Here, the file .my.cnf in the home directory contains the necessary configuration information for MySQL.

### Sqlite

An example with SQLite is very similar to the one above and the cursor provides many of the same functionalities.
When writing to the db, one has to remember to call db.commit, otherwise the changes are not saved:

### Firebird
## General Principles
### Parameter Quoting

You will frequently need to substitute dynamic data into a query string. It is important to ensure this is done correctly.
This example is wrong, because it doesn’t correctly deal with special characters, like apostrophes, in the string being substituted. If your code has to deal with potentially hostile users (like on a public-facing Web server), this could leave you open to an SQL injection attack.
For simple cases, use the automatic parameter substitution provided by the execute method, e.g.
The DBMS interface itself will automatically convert the values you pass into the correct SQL syntax.
For more complex cases, the DBMS module should provide a quoting function that you can explicitly call. For example, MySQLdb provides the escape_string method, while APSW (for SQLite3) provides format_sql_value. This is necessary where the query structure takes a more dynamic form:
This will dynamically construct queries like “select name from employees where company = 'some company'” or “select name from employees where company = 'some company' and department = 'some department'”, depending on which fields have been filled in by the user.

### Use Iterators

Python iterators are a natural fit for the problem of iterating over lots of database records. Here is an example of a function that performs a database query and returns an iterator for the results, instead of returning them all at once. It relies on the fact that, in APSW (the Python 3 interface library for SQLite), the cursor.execute method itself returns an iterator for the result records. The result is that you can write very concise code for doing complex database queries in Python.
Example uses of this function:
and
In the first example, since db_iter returns a tuple for each record, this can be directly assigned to individual variables for the record fields. In the second example, the tuple has only one element, so a custom mapfn is used to extract this element and return it instead of the tuple.

### Never Use “SELECT *” in a Script

Database table definitions are frequently subject to change. As application requirements evolve, fields and even entire tables are often added, or sometimes removed. Consider a statement like
You may happen to know that the employees table currently contains, say, 4 fields. But tomorrow someone may add a fifth field. Did you remember to update your code to deal with this? If not, it’s liable to crash. Or even worse, produce an incorrect result!
Better to always list the specific fields you’re interested in, no matter how many there are:
That way, any extra fields added will simply be ignored. And if any of the named fields are removed, the code will at least fail with a runtime error, which is a good reminder that you forgot to update it!

### Looping on Field Breaks

Consider the following scenario: your sales company database has a table of employees, and also a table of sales made by each employee. You want to loop over these sale entries, and produce some per-employee statistics. A naïve approach might be:
* Query the database to get a list of employees
* For each employee, do a database query to get the list of sales for each employee.
If you have a lot of employees, then the first query may produce a large list, and the second step will involve a correspondingly large number of database queries.
In fact, the entire processing loop can run off a single database query, using the standard SQL construct called a join.
Here is what an example of such a loop could look like:
Here the statistics are quite simple: earliest and latest sale, and number and total amount of sales, and could be computed directly within the SQL query. But the same loop could compute more complex statistics (like standard deviation) that cannot be represented directly within a simple SQL query.
Note how the statistics for each employee are written out under either of two conditions:
* The employee name of the next record is different from the previous one
* The end of the query results has been reached.
Both conditions are tested with row == None or employee_name != prev_employee_name; after writing out the employee statistics, a separate check for the second condition row == None is used to terminate the loop. If the loop doesn’t terminate, then processing is initialized for the new employee.
Note also the use of a left join in this case: if an employee has had no sales, then the join will return a single row for that employee, with SQL null values (represented by None in Python) for the fields from the sales table. This is why we need checks for such None values before processing those fields.
Alternatively, we could have used an inner join, which would have returned no results for an employee with no sales. Whether you want to omit such an employee from your report, or include them with totals of zero, is really up to your application.

## See Also

* Python Programming/Database Programming

## External links

* APSW module, code.google.com — SQLite3 for Python 2.x and 3.x
* SQLite documentation
* Psycopg2 (PostgreSQL module - newer), initd.org
* PyGreSQL (PostgreSQL module - older), pygresql.org
* MySQL module, sourceforge.net
* FirebirdSQL module, github.com
| Previous: Networks | Index | Next: Database Programming |

# Database Programming

| Previous: Networks | Index | Next: Database Programming |
Python has support for working with databases via a simple API. Modules included with Python include modules for SQLite and Berkeley DB. Modules for MySQL , PostgreSQL , FirebirdSQL and others are available as third-party modules. The latter have to be downloaded and installed before use. The package MySQLdb can be installed, for example, using the Debian package "python-mysqldb".

## DBMS Specifics
### MySQL

An Example with MySQL would look like this:
On the first line, the Module MySQLdb is imported. Then a connection to the database is set up and on line 4, we save the actual SQL statement to be executed in the variable query. On line 5 we execute the query and on line 6 we fetch all the data. After the execution of this piece of code, lines contains the number of lines fetched (e.g. the number of rows in the table sampletable). The variable data contains all the actual data, e.g. the content of sampletable. In the end, the connection to the database would be closed again. If the number of lines are large, it is better to use row = cursor.fetchone and process the rows individually:
Obviously, some kind of data processing has to be used on a row, otherwise the data will not be stored. The result of the fetchone command is a Tuple.
In order to make the initialization of the connection easier, a configuration file can be used:
Here, the file .my.cnf in the home directory contains the necessary configuration information for MySQL.

### Sqlite

An example with SQLite is very similar to the one above and the cursor provides many of the same functionalities.
When writing to the db, one has to remember to call db.commit, otherwise the changes are not saved:

### Firebird
## General Principles
### Parameter Quoting

You will frequently need to substitute dynamic data into a query string. It is important to ensure this is done correctly.
This example is wrong, because it doesn’t correctly deal with special characters, like apostrophes, in the string being substituted. If your code has to deal with potentially hostile users (like on a public-facing Web server), this could leave you open to an SQL injection attack.
For simple cases, use the automatic parameter substitution provided by the execute method, e.g.
The DBMS interface itself will automatically convert the values you pass into the correct SQL syntax.
For more complex cases, the DBMS module should provide a quoting function that you can explicitly call. For example, MySQLdb provides the escape_string method, while APSW (for SQLite3) provides format_sql_value. This is necessary where the query structure takes a more dynamic form:
This will dynamically construct queries like “select name from employees where company = 'some company'” or “select name from employees where company = 'some company' and department = 'some department'”, depending on which fields have been filled in by the user.

### Use Iterators

Python iterators are a natural fit for the problem of iterating over lots of database records. Here is an example of a function that performs a database query and returns an iterator for the results, instead of returning them all at once. It relies on the fact that, in APSW (the Python 3 interface library for SQLite), the cursor.execute method itself returns an iterator for the result records. The result is that you can write very concise code for doing complex database queries in Python.
Example uses of this function:
and
In the first example, since db_iter returns a tuple for each record, this can be directly assigned to individual variables for the record fields. In the second example, the tuple has only one element, so a custom mapfn is used to extract this element and return it instead of the tuple.

### Never Use “SELECT *” in a Script

Database table definitions are frequently subject to change. As application requirements evolve, fields and even entire tables are often added, or sometimes removed. Consider a statement like
You may happen to know that the employees table currently contains, say, 4 fields. But tomorrow someone may add a fifth field. Did you remember to update your code to deal with this? If not, it’s liable to crash. Or even worse, produce an incorrect result!
Better to always list the specific fields you’re interested in, no matter how many there are:
That way, any extra fields added will simply be ignored. And if any of the named fields are removed, the code will at least fail with a runtime error, which is a good reminder that you forgot to update it!

### Looping on Field Breaks

Consider the following scenario: your sales company database has a table of employees, and also a table of sales made by each employee. You want to loop over these sale entries, and produce some per-employee statistics. A naïve approach might be:
* Query the database to get a list of employees
* For each employee, do a database query to get the list of sales for each employee.
If you have a lot of employees, then the first query may produce a large list, and the second step will involve a correspondingly large number of database queries.
In fact, the entire processing loop can run off a single database query, using the standard SQL construct called a join.
Here is what an example of such a loop could look like:
Here the statistics are quite simple: earliest and latest sale, and number and total amount of sales, and could be computed directly within the SQL query. But the same loop could compute more complex statistics (like standard deviation) that cannot be represented directly within a simple SQL query.
Note how the statistics for each employee are written out under either of two conditions:
* The employee name of the next record is different from the previous one
* The end of the query results has been reached.
Both conditions are tested with row == None or employee_name != prev_employee_name; after writing out the employee statistics, a separate check for the second condition row == None is used to terminate the loop. If the loop doesn’t terminate, then processing is initialized for the new employee.
Note also the use of a left join in this case: if an employee has had no sales, then the join will return a single row for that employee, with SQL null values (represented by None in Python) for the fields from the sales table. This is why we need checks for such None values before processing those fields.
Alternatively, we could have used an inner join, which would have returned no results for an employee with no sales. Whether you want to omit such an employee from your report, or include them with totals of zero, is really up to your application.

## See Also

* Python Programming/Database Programming

## External links

* APSW module, code.google.com — SQLite3 for Python 2.x and 3.x
* SQLite documentation
* Psycopg2 (PostgreSQL module - newer), initd.org
* PyGreSQL (PostgreSQL module - older), pygresql.org
* MySQL module, sourceforge.net
* FirebirdSQL module, github.com
| Previous: Networks | Index | Next: Database Programming |

# numpy

| Previous: Database Programming | Index | Next: Game Programming in Python |
Numpy is a numeric library for python.

## Installation

It's provided with the main Linux distribution, however it can be installed through the Debian package python-numpy. On Windows, it can be downloaded on http://sourceforge.net/projects/numpy/files/.
Then, once the .zip unpacked, the installation is done by entering into the console:
In case of error:
* ImportError “No Module named Setuptools”, save the file https://bootstrap.pypa.io/ez_setup.py into the Python folder, and install it with: python ez_setup.py install.
** Microsoft Visual C++ 9.0 is required, download it on https://www.microsoft.com/en-us/download/details.aspx?id=44266

## See also

* http://numpy.scipy.org/
* http://docs.scipy.org/doc/numpy/reference/
* Research using FOSS
| Previous: Database Programming | Index | Next: Game Programming in Python |

# Game Programming in Python

| Previous: numpy | Index | Next: PyQt4 |

### 3D Game Programming
#### 3D Game Engine with a Python binding

* Irrlicht Engine (Python binding website: )
Both are very good free open source C++ 3D game Engine with a Python binding.
* CrystalSpace is a free cross-platform software development kit for real-time 3D graphics, with particular focus on games. Crystal Space is accessible from Python in two ways: (1) as a Crystal Space plugin module in which C++ code can call upon Python code, and in which Python code can call upon Crystal Space; (2) as a pure Python module named ‘cspace’ which one can ‘import’ from within Python programs. To use the first option, load the ‘cspython’ plugin as you would load any other Crystal Space plugin, and interact with it via the SCF ‘iScript’ interface .The second approach allows you to write Crystal Space applications entirely in Python, without any C++ coding. CS Wiki

#### 3D Game Engines written for Python

Engines designed for Python from scratch.
* Blender is an impressive 3D tool with a fully integrated 3D graphics creation suite allowing modeling, animation, rendering, post-production, real-time interactive 3D and game creation and playback with cross-platform compatibility. The 3D game engine uses an embedded python interpreter to make 3D games.
* Panda3D is a 3D game engine. It's a library written in C++ with Python bindings. Panda3D is designed in order to support a short learning curve and rapid development. This software is available for free download with source code under the BSD License. The development was started by [Disney]. Now there are many projects made with Panda3D, such as Disney's Pirate's of the Caribbean Online, ToonTown, Building Virtual World, Shell Games and many others. Panda3D supports several features: Procedural Geometry, Animated Texture, Render to texture, Track motion, fog, particle system, and many others.
* Crystal-space Is a 3D game engine, with a Python bindings, named *Crystal, view Wikipedia page of *CrystalSpace.

### 2D Game Programming

* Pygame is a cross platform Python library which wraps SDL. It provides many features like Sprite groups and sound/image loading and easy changing of an objects position. It also provides the programmer access to key and mouse events. A full tutorial can be found in the free book "Making Games with Python & Pygame".
* Phil's Pygame Utilities (PGU) is a collection of tools and libraries that enhance Pygame. Tools include a tile editor and a level editor (tile, isometric, hexagonal). GUI enhancements include full featured GUI, HTML rendering, document layout, and text rendering. The libraries include a sprite and tile engine (tile, isometric, hexagonal), a state engine, a timer, and a high score system. (Beta with last update March, 2007. APIs to be deprecated and isometric and hexagonal support is currently Alpha and subject to change.) [Update 27/02/08 Author indicates he is not currently actively developing this library and anyone that is willing to develop their own scrolling isometric library offering can use the existing code in PGU to get them started.]
* Pyglet is a cross-platform windowing and multimedia library for Python with no external dependencies or installation requirements. Pyglet provides an object-oriented programming interface for developing games and other visually-rich applications for Windows, Mac OS X and Linux. Pyglet allows programs to open multiple windows on multiple screens, draw in those windows with OpenGL, and play back audio and video in most formats. Unlike similar libraries available, pyglet has no external dependencies (such as SDL) and is written entirely in Python. Pyglet is available under a BSD-Style license.
* Kivy Kivy is a library for developing multi-touch applications. It is completely cross-platform (Linux/OSX/Win & Android with OpenGL ES2). It comes with native support for many multi-touch input devices, a growing library of multi-touch aware widgets and hardware accelerated OpenGL drawing. Kivy is designed to let you focus on building custom and highly interactive applications as quickly and easily as possible.
* Rabbyt A fast Sprite library for Python with game development in mind. With Rabbyt Anims, even old graphics cards can produce very fast animations of 2,400 or more sprites handling position, rotation, scaling, and color simultaneously.

### See Also

* 10 Lessons Learned - How To Build a Game In A Week From Scratch With No Budget
| Previous: numpy | Index | Next: PyQt4 |

# PyQt4

| Previous: Game Programming in Python | Index | Next: Dbus |
WARNING: The examples on this page are a mixture of PyQt3 and PyQt4 - use with caution!
This tutorial aims to provide a hands-on guide to learn the basics of building a small Qt4 application in Python.
To follow this tutorial, you should have basic Python knowledge. However, knowledge of Qt4 is not necessary. I'm using Linux in these examples and am assuming you already have a working installation of Python and PyQt4. To test this, open a Python shell (by typing 'Python' in a console to start the interactive interpreter) and type:
If no error message appears, you should be ready to go.
The examples in this tutorial as easy as possible, showing useful ways to write and structure your program. It is important that you read the source code of the example files, most of the explanations are in the code. The best way to get comfortable with PyQt is play around with the examples and try to change things.

## Hello, world!

Let's start easy: popping up a window and displaying something. The following small program will pop up a window showing "Hello, world!".
About 7 lines of code, and that's about as easy as it can get.

## A Button

Let's add some interaction! We'll replace the label saying "Hello, World!" with a button and assign an action to it. This assignment is done by connecting a signal, an event which is sent out when the button is pushed, to a slot, which is an action, normally a function that is run in the case of that event.
You can imagine that coding this way is not scalable nor the way you'll want to continue working. So let's make that stuff pythonic, adding structure and actually using object-orientation in it. We create our own application class, derived from a QApplication and put the customization of the application into its methods: One method to build up the widgets and a slot which contains the code that's executed when a signal is received.

## GUI Coding

... so we want to use Qt3 Designer for creating our GUI. In the picture, you can see a simple GUI, with in green letters the names of the widgets. What we are going to do is We compile the .ui file from Qt designer into a python class We subclass that class and use it as our mainWidget This way, we're able to change the user interface afterwards from Qt designer, without having it messing around in the code we added.
makes a Python file from it which we can work with.
The way our program works can be described like this: We fill in the lineedit Clicking the add button will be connected to a method that reads the text from the lineedit, makes a listviewitem out of it and adds that to our listview. Clicking the deletebutton will delete the currently selected item from the listview. Here's the heavily commented code (only works in PyQt3):
also this code is useful and it works on PyQt4 and it has many useful options

## Useful to Know

Creating the GUI in Qt Designer not only makes creating the GUI easier, but it's also a great learning tool. You can test what a widget looks like, see what's available in Qt, and have a look at properties you might want to use.
The C++ API documentation is also a very useful (read: necessary) tool when working with PyQt. The API translation is straightforward, and after a little experience, you'll find the developers API docs one of the tools you really need. When working from KDE, konqueror's default shortcut is qt:[widgetname], so [alt]+[F2], "qt:qbutton directly takes you to the right API documentation page. Trolltech's doc section has much more documentation which you might want to have a look at.
The first 3 examples in this tutorial have been created using PyQt4, the last one uses syntax that only works with PyQt3.
Note: The previous version of this page (applicable to pyqt3) is/was available at http://vizzzion.org/?id=pyqt.
This document is published under the GNU Free Documentation License.
| Previous: Game Programming in Python | Index | Next: Dbus |

# Dbus

| Previous: PyQt4 | Index | Next: pyFormex |
In Linux, Dbus is a way for processes to communicate with each other. For example, programs like Pidgin instant messenger allow other programs to find out or change the user's status (Available, Away, etc). Another example is the network-manager service that publishes which internet connection is active. Programs that sometimes connect to the internet can then pick the best time to download updates to the system.

## Buses

Messages are sent along buses. Services attach themselves to these buses, and allow clients to pass messages to and from them.
There are two main buses, the system bus and session bus. Services on the system bus affect the whole system, such as providing information about the network or disk drives. Services on the session bus provide access to programs running on the desktop, like Pidgin.

## Objects and interfaces

Services attached to a bus can be contacted using their well-known name. While this could be any string, the format is normally that of a reverse domain name: an example for a spreadsheet program called "CalcProgram" from "My Corp Inc." could be "com.mycorp.CalcProgram".
Services publish objects using slash-separated paths (this is similar to webpages). Someone on dbus can request an object if they know this path.
The object passed back is not a full object: it just refers to the service's copy of the object. It is called a proxy object.
Before the proxy object can be used, we need to specify what type of object it is. We do this by creating an interface object.
Whatever methods are set up for this type of object can be called:
| Name | Example | Description |
| service well known name | com.mycorp.CalcProgram | Identifies the application |
| path of an object | /spreadsheet1/cells/a2 | Identifies an object published by a service |
| interface | com.mycorp.CalcProgram.SpreadsheetCell | Identifies what type of object we expect |

## dbus-python examples

These examples have been tested with dbus-python 0.83.0. Older library versions may not have the same interface.
Calling an interface's methods / Listing HAL Devices:
Introspecting an object:
Avahi:

## pydbus examples

These examples have been tested with pydbus 0.2 and 0.3.
Calling an interface's methods / Listing systemd units:
Introspecting an object:
Avahi:

## References

* dbus-python tutorial, dbus.freedesktop.org, Simon McVittie, 2006-06-14
* pydbus README.rst, github.com
* DbusExamples, wiki.python.org
* D-Bus Howto, developer.pidgin.im
* Rough notes: Python and D-Bus, archived version, A.M. Kuchling, 2007, originally at www.amk.ca
| Previous: PyQt4 | Index | Next: pyFormex |

# pyFormex

| Previous: Dbus | Index | Next: matplotlib |
pyFormex is a module for Python, which allows the generation, manipulation, and operation of 3D geometric models using mathematical operations. Its uses include automated 3D design and finite-element preprocessing.
| Previous: Dbus | Index | Next: matplotlib |

# matplotlib

| Previous: pyFormex | Index | Next: Natural Language Toolkit |
matplotlib is a Python library that allows Python to be used like Matlab, visualizing data on the fly. It is able to create plots, histograms, power spectra, bar charts, errorcharts, scatterplots, etc. It can be used from normal Python and also from iPython.
Examples:
Plot a data series that represents the square function:
Plot a data series that represents the square function but in reverse order, by providing not only the series of the y-axis values but also the series of x-axis values:
Plot the square function, setting the limits for the y-axis:

## Links

* matplotlib, matplotlib.org
* Pyplot tutorial, matplotlib.org
* Plotting, wiki.python.org
| Previous: pyFormex | Index | Next: Natural Language Toolkit |

# Sorted Container Types

Python does not provide modules for sorted set and dictionary data types as part of its standard library. This is a concious decision on the part of Guido van Rossum, et al. to preserve "one obvious way to do it." Instead, Python delegates this task to third-party libraries that are available on the Python Package Index. These libraries use various techniques to maintain list, dict, and set types in sorted order. Maintaining order using a specialized data structure can avoid very slow behavior (quadratic run-time) in the naive approach of editing and constantly re-sorting.
Third-party modules supporting sorted containers:
* SortedContainers - Pure-Python implementation that is fast-as-C implementations. Implements sorted list, dict, and set. Testing includes 100% code coverage and hours of stress. Documentation includes full API reference, performance comparison, and contributing/development guidelines. License: Apache2.
* rbtree - Provides a fast, C-implementation for sorted dict and set data types. Based on a red-black tree implementation.
* treap - Provides a sorted dict data type. Uses a treap for implementation and improves performance using Cython.
* bintrees - Provides several tree-based implementations for dict and set data types. Fastest implementations are based on AVL and Red-Black trees. Implemented in C. Extends the conventional API to provide set operations for dict data types.
* banyan - Provides a fast, C-implementation for dict and set data types.
* skiplistcollections - Pure-Python implementation based on skip-lists providing a limited API for dict and set data types.
* blist - Provides sorted list, dict and set data types based on the "blist" data type, a B-tree implementation. Implemented in Python and C.

## External links

* SortedContainers, pypi.python.org
* rbtree, pypi.python.org
* treap, pypi.python.org
* bintrees, pypi.python.org
* Banyan, pypi.python.org
* skiplistcollections, pypi.python.org
* blist, pypi.python.org

# Excel

Python has multiple 3rd party libraries for reading and writing Microsoft Excel spreadsheet files, including .xls and .xlsx.
For working with .xls files, there is xlrd for reading and xlwt for writing.
For working with .xlsx files, there is xlrd for reading, openpyxl for reading and writing, and XlsxWriter and PyExcelerate for writing.
To interact with the Excel application and create Python-based add-ins: xlwings, xlOil, PyXLL (commercial).

## xlrd

Supports reading .xls Excel files. Support for .xlsx files was removed in xlrd version 2.0.0 from Dec 2020 due to security concerns, but is still available in xlrd version 1.2.0 from Dec 2018. License: BSD.
Example:
Links:
* xlrd, pypi.python.org
* xlrd 1.2.0, pypi.python.org
* xlrd documentation, readthedocs.io
* xlrd API documentation, readthedocs.io
* Python: xlrd discerning dates from floats, stackoverflow.com
* xlrd.biffh.XLRDError: Excel xlsx file; not supported, 11 Dec 2020, stackoverflow.com
* xlrd 2 released, 11 Dec 2020, groups.google.com

## xlwt

Supports writing .xls files. License: BSD.
Links:
* xlwt, pypi.python.org
* xlwt documentation, readthedocs.io

## openpyxl

Supports reading and writing .xlsx Excel files. Does not support .xls files. License: MIT.
Reading a workbook:
Creating a new workbook:
Changing an existing workbook:
Links:
* openpyxl, pypi.org
* openpyxl - A Python library to read/write Excel 2010 xlsx/xlsm files, readthedocs.io

## XlsxWriter

Supports writing of .xlsx files. License: BSD.
Links:
* XlsxWriter, pypi.org
* Creating Excel files with Python and XlsxWriter, readthedocs.io

## PyExcelerate

Supports writing .xlsx files. License: BSD.
Links:
* PyExcelerate, pypi.org

## xlutils

Supports various operations and queries on .xls files; depends on xlrd and xlwt. License: MIT.
Links:
* xlutils, pypi.org

## xlOil

Supports creation of Python-based Excel add-ins. Requires Python 3.6 or later; requires Excel 2010 or later installed. Supports: global and local scope worksheet functions, ribbon customisation, custom task panes, RTD/async functions, numpy, matplotlib, pandas, jupyter. Low overhead function calls due to use of the Excel's C-API and embedded in-process Python
Examples:
Create a function to add one day to a date:
Create a function which give a live ticking clock in an cell (uses RTD):
Links:
* xlOil, pypi.org
* xloil, readthedocs.io

## pywin32

Supports access to Windows applications via Windows Component Object Model (COM). Thus, on Windows, if Excel is installed, PyWin32 lets you call it from Python and let it do various things. You can install PyWin32 by downloading a .exe installer from SourceForge, where it is currently hosted.
Links:
* 3.4.1. PyWin32, docs.python.org
* pywin32, pypi.org
* Python for Windows Extensions files, sourceforge.net
* Python Excel Mini Cookbook, pythonexcels.com

## External links

* Working with Excel Files in Python, python-excel.org -- has links to source code repositories (GitHub, etc.) for the packages

# MS Word

Microsoft Word documents of the .docx format can be created or changed using python-docx 3rd party module.
The .doc format can be worked with on Windows using PyWin32 via COM interface, provided Word is installed. An example:

## External links

* python-docx, pypi.python.org
* python-docx, readthedocs.io

# Extending with C

| Previous: MS Word | Index | Next: Extending with C++ |
Python modules can be written in pure Python but they can also be written in the C language. The following shows how to extend Python with C.

## Using the Python/C API
### A minimal example

To illustrate the mechanics, we will create a minimal extension module containing a single function that outputs "Hello" followed by the name passed in as the first parameter.
We will first create the C source code, placing it to hellomodule.c:
Then we will need a setup file, setup.py:
Then we can build the module using a procedure whose details depends on the operating system and the compiler suite.

#### Building with GCC for Linux

Before our module can be compiled, you must install the Python development headers if you have not already. On Debian and Debian-based systems such as Ubuntu, these can be installed with the following command:
On openSUSE, the required package is called python-devel and can be installed with zypper:
Now that Python.h is available, we can compile the module source code we created in the previous section as follows:
The will compile the module to a file called hello.so in build/lib.linux-i686-x.y.

#### Building with GCC for Microsoft Windows

Microsoft Windows users can use MinGW to compile the extension module from the command line. Assuming gcc is in the path, you can build the extension as follows:
The above will produce file hello.pyd, a Python Dynamic Module, similar to a DLL. The file will land in build\lib.win32-x.y.
An alternate way of building the module in Windows is to build a DLL. (This method does not need an extension module file). From cmd.exe, type:
where XY represents the version of Python, such as "24" for version 2.4.

#### Building using Microsoft Visual C++

With VC8, distutils is broken. Therefore, we will use cl.exe from a command prompt instead:

#### Using the extension module

Change to the subdirectory where the file hello.so resides. In an interactive Python session you can use the module as follows.

### A module for calculating Fibonacci numbers

In this section, we present a module for Fibonacci numbers, thereby expanding on the minimal example above. Compared to the minimal example, what is worth noting is the use of "i" in PyArg_ParseTuple and Py_BuildValue.
The C source code in (fibmodule.c):
The build script (setup.py):
Usage:

## Using SWIG

SWIG is a tool that helps a variety of scripting and programming languages call C and C++ code. SWIG makes creation of C language modules much more straightforward.
To use SWIG, you need to get it up and running first.
You can install it on an Ubuntu system as follows:
To get SWIG for Windows, you can use binaries available from the SWIG download page.
Once you have SWIG, you need to create the module source file and the module interface file:
hellomodule.c:
hello.i:
Then we let SWIG do its work:
The above produces files hello.py and hello_wrap.c.
The next step is compiling; substitute /usr/include/python2.4/ with the correct path to Python.h for your setup:
As the last step, we do the linking:
The module is used as follows:

## External links

* Extending and Embedding the Python Interpreter, python.org
* Python/C API Reference Manual, python.org
* SWIG, swig.org
* Download SWIG, swig.org
| Previous: pyFormex | Index | Next: Extending with C++ |

# Extending with C++

| Previous: Extending with C | Index | Next: Extending with Pyrex |
There are different ways to extend Python with C and C++ code:
* In plain C, using Python.h
* Using Swig
* Using Boost.Python, optionally with Py++ preprocessing
* Using pybind11
* Using Cython.
This page describes Boost.Python. Before the emergence of Cython, it was the most comfortable way of writing C++ extension modules.
Boost.Python comes bundled with the Boost C++ Libraries. To install it on an Ubuntu system, you might need to run the following commands

## A Hello World Example
### setup.py

Now we can build our module with
The module `hello.so` will end up in e.g `build/lib.linux-i686-2.4`.

### Using the extension module

Change to the subdirectory where the file `hello.so` resides. In an interactive python session you can use the module as follows.

## An example with CGAL

Some, but not all, functions of the CGAL library already have Python bindings. Here an example is provided for a case without such a binding and how it might be implemented. The example is taken from the CGAL Documentation.
We then compile and run the module as follows:

## Handling Python objects and errors

One can also handle more complex data, e.g. Python objects like lists. The attributes are accessed with the extract function executed on the objects "attr" function output. We can also throw errors by telling the library that an error has occurred and returning. In the following case, we have written a C++ function called "afunction" which we want to call. The function takes an integer N and a vector of length N as input, we have to convert the python list to a vector of strings before calling the function.

## External links

* Boost.Python, boost.org
* pybind11, pypi.org
| Previous: Extending with C | Index | Next: Extending with Pyrex |

# Extending with Pyrex

| Previous: Extending with C++ | Index | Next: Extending with ctypes |
Pyrex is a compiler of Python-like source code to the C language, intended to make it relatively easy for Python programmers to write fast Python extension modules without having to learn the C language.
Cython is an actively developed derivative of Pyrex. Pyrex is no longer actively developed, the last stable release being from 2010.
Links:
* Pyrex - a Language for Writing Python Extension Modules, cosc.canterbury.ac.nz
* Pyrex (programming language), en.wikipedia.org
| Previous: Extending with C++ | Index | Next: Extending with ctypes |

# Extending with ctypes

| Previous: Extending with Pyrex | Index | Next: Extending with Perl |
ctypes is a foreign function interface module for Python (included with Python 2.5 and above), which allows you to load in dynamic libraries and call C functions. This is not technically extending Python, but it serves one of the primary reasons for extending Python: to interface with external C code.

## Basics

A library is loaded using the ctypes.CDLL function. After you load the library, the functions inside the library are already usable as regular Python calls. For example, if we wanted to forego the standard Python print statement and use the standard C library function, printf, you would use this:
Of course, you must use the libName line that matches your operating system, and delete the other. If all goes well, you should see the infamous Hello World string at your console.

## Getting Return Values

ctypes assumes, by default, that any given function's return type is a signed integer of native size. Sometimes you don't want the function to return anything, and other times, you want the function to return other types. Every ctypes function has an attribute called restype. When you assign a ctypes class to restype, it automatically casts the function's return value to that type.

### Common Types

| ctypes name | C type | Python type | Notes |
| None | void | None | the None object |
| c_bool | C99 _Bool | bool | |
| c_byte | signed char | int | |
| c_char | signed char | str | length of one |
| c_char_p | char * | str | |
| c_double | double | float | |
| c_float | float | float | |
| c_int | signed int | int | |
| c_long | signed long | long | |
| c_longlong | signed long long | long | |
| c_short | signed short | long | |
| c_ubyte | unsigned char | int | |
| c_uint | unsigned int | int | |
| c_ulong | unsigned long | long | |
| c_ulonglong | unsigned long long | long | |
| c_ushort | unsigned short | int | |
| c_void_p | void * | int | |
| c_wchar | wchar_t | unicode | length of one |
| c_wchar_p | wchar_t * | unicode | |
| Previous: Extending with Pyrex | Index | Next: Extending with Perl |

# Extending with Perl

| Previous: Extending with ctypes | Index | Next: Extending with R |
It is possible to call Perl functions and modules in Python. One way to do that is the PyPerl Module. It is not developed actively any more and for it to work in newer versions, one has to use this version and apply the patches.
It is thus possible to handle Perl objects, change their attributes and call their methods.

## See also

* http://wiki.python.org/moin/IntegratingPythonWithOtherLanguages#Perl
| Previous: Extending with ctypes | Index | Next: Extending with R |

# Popularity

The popularity of a programming language is one factor that people consider when choosing a language for a task or a project. Beware that even a good programming language can be unpopular.
The popularity of Python can be determined with the help of various indices, which are not identical to the real popularity in the real world. While the statistics are linked from the external links section below, it is advisable that you use them with a grain of salt.
Python is the main scripting language used by Google, per Google Python Style Guide.
Python is embedded in various software packages to support their extensibility and automation, including Gimp and Inkscape; see also Wikipedia's W:Python (programming_language)#Uses and W:List of Python software#Embedded as a scripting language.

## External links

* Tiobe index, tiobe.com
* PYPL PopularitY of Programming Languages, pypl.github.io
* RedMonk rankings, redmonk.com, combines Stack Overflow data with GitHub data
* The 2015 Top Ten Programming Languages, spectrum.ieee.org, 2015
* W:Measuring programming language popularity
* W:Programming languages used in most popular websites
* W:Python (programming_language)
* W:List of Python software
* Google Python Style Guide, google.github.io
* Python Success Stories, python.org
* Organizations using Python, wiki.python.org

# Links

Web resources:
Python.org:
* The Python.org documents section
** Current Python 2.x documentation
** Current Python 3.x documentation
* Python Wiki at python.org
** Python IDEs
Online books:
* Dive into Python by Mark Pilgrim, a free book published under GNU Free Documentation License, with around 100 000 words
* Text Processing in Python, a book by David Mertz, published by Addison Wesley
* Introduction to Python Programming with NCLab, a book available as a single PDF, with around 44 000 words
* Invent Your Own Computer Games with Python - A complete eBook for total beginners available for free under a Creative Commons license.
* Making Games with Python & Pygame - A Creative Commons-licensed book that covers the Pygame library.
* Hacking Secret Ciphers with Python - A Creative Commons-licensed book for complete beginners with a focus on cryptography.
Various:
* Python Programming Tutorials -- a list of Python tutorials
* ActiveState Python Cookbook -- a collection of Python recipes
* Dev Shed's Python Tutorials -- a collection of Python articles
* Python Tutorials at awaretek.com
* Python 101 -- Introduction to Python by Dave Kuhlman, around 9000 words
* Category:Python, rosettacode.org

# Authors

| Previous: Links | Index | Next: Orphaned Pages |

## Authors of Python textbook

* Quartz25
* Jesdisciple
* Hannes Röst
* David Ross
* Lawrence D’Oliveiro
* User:BLibrestez55
| Previous: Links | Index | Next: OrphanedPages |

# Library Modules

This is a list of python modules in the standard library as of Python 3.6.
* __future__: Future statement definitions
* __main__: The environment where the top-level script is run.
* _dummy_thread: Drop-in replacement for the _thread module.
* _thread: Low-level threading API.
* abc: Abstract base classes according to PEP 3119.
* aifc: Read and write audio files in AIFF or AIFC format.
* argparse: Command-line option and argument parsing library.
* array: Space efficient arrays of uniformly typed numeric values.
* ast: Abstract Syntax Tree classes and manipulation.
* asynchat: Support for asynchronous command/response protocols.
* asyncio: Asynchronous I/O, event loop, coroutines and tasks.
* asyncore: A base class for developing asynchronous socket handling services.
* atexit: Register and execute cleanup functions.
* audioop: Manipulate raw audio data.
* base64: RFC 3548: Base16, Base32, Base64 Data Encodings; Base85 and Ascii85
* bdb: Debugger framework.
* binascii: Tools for converting between binary and various ASCII-encoded binary representations.
* binhex: Encode and decode files in binhex4 format.
* bisect: Array bisection algorithms for binary searching.
* builtins: The module that provides the built-in namespace.
* bz2: Interfaces for bzip2 compression and decompression.
* calendar: Functions for working with calendars, including some emulation of the Unix cal program.
* cgi: Helpers for running Python scripts via the Common Gateway Interface.
* cgitb: Configurable traceback handler for CGI scripts.
* chunk: Module to read IFF chunks.
* cmath: Mathematical functions for complex numbers.
* cmd: Build line-oriented command interpreters.
* code: Facilities to implement read-eval-print loops.
* codecs: Encode and decode data and streams.
* codeop: Compile (possibly incomplete) Python code.
* collections: Container datatypes
* colorsys: Conversion functions between RGB and other color systems.
* compileall: Tools for byte-compiling all Python source files in a directory tree.
* concurrent:
* configparser: Configuration file parser.
* contextlib: Utilities for with-statement contexts.
* copy: Shallow and deep copy operations.
* copyreg: Register pickle support functions.
* cProfile
* crypt (Unix): The crypt function used to check Unix passwords.
* csv: Write and read tabular data to and from delimited files.
* ctypes: A foreign function library for Python.
* curses (Unix): An interface to the curses library, providing portable terminal handling.
* datetime: Basic date and time types.
* dbm: Interfaces to various Unix "database" formats.
* decimal: Implementation of the General Decimal Arithmetic Specification.
* difflib: Helpers for computing differences between objects.
* dis: Disassembler for Python bytecode.
* distutils: Support for building and installing Python modules into an existing Python installation.
* doctest: Test pieces of code within docstrings.
* dummy_threading: Drop-in replacement for the threading module.
* email: Package supporting the parsing, manipulating, and generating email messages.
* encodings:
* ensurepip: Bootstrapping the "pip" installer into an existing Python installation or virtual environment.
* enum: Implementation of an enumeration class.
* errno: Standard errno system symbols.
* faulthandler: Dump the Python traceback.
* fcntl (Unix): The fcntl and ioctl system calls.
* filecmp: Compare files efficiently.
* fileinput: Loop over standard input or a list of files.
* fnmatch: Unix shell style filename pattern matching.
* formatter: Deprecated: Generic output formatter and device interface.
* fpectl (Unix): Provide control for floating point exception handling.
* fractions: Rational numbers.
* ftplib: FTP protocol client (requires sockets).
* functools: Higher-order functions and operations on callable objects.
* gc: Interface to the cycle-detecting garbage collector.
* getopt: Portable parser for command line options; support both short and long option names.
* getpass: Portable reading of passwords and retrieval of the userid.
* gettext: Multilingual internationalization services.
* glob: Unix shell style pathname pattern expansion.
* grp (Unix): The group database (getgrnam and friends).
* gzip: Interfaces for gzip compression and decompression using file objects.
* hashlib: Secure hash and message digest algorithms.
* heapq: Heap queue algorithm (a.k.a. priority queue).
* hmac: Keyed-Hashing for Message Authentication (HMAC) implementation
* html: Helpers for manipulating HTML.
* http: HTTP status codes and messages
* imaplib: IMAP4 protocol client (requires sockets).
* imghdr: Determine the type of image contained in a file or byte stream.
* imp: Deprecated: Access the implementation of the import statement.
* importlib: The implementation of the import machinery.
* inspect: Extract information and source code from live objects.
* io: Core tools for working with streams.
* ipaddress: IPv4/IPv6 manipulation library.
* itertools: Functions creating iterators for efficient looping.
* json: Encode and decode the JSON format.
* keyword: Test whether a string is a keyword in Python.
* lib2to3: the 2to3 library
* linecache: This module provides random access to individual lines from text files.
* locale: Internationalization services.
* logging: Flexible event logging system for applications.
* lzma: A Python wrapper for the liblzma compression library.
* macpath: Mac OS 9 path manipulation functions.
* mailbox: Manipulate mailboxes in various formats
* mailcap: Mailcap file handling.
* marshal: Convert Python objects to streams of bytes and back (with different constraints).
* math: Mathematical functions (sin etc.).
* mimetypes: Mapping of filename extensions to MIME types.
* mmap: Interface to memory-mapped files for Unix and Windows.
* modulefinder: Find modules used by a script.
* msilib (Windows): Creation of Microsoft Installer files, and CAB files.
* msvcrt (Windows): Miscellaneous useful routines from the MS VC++ runtime.
* multiprocessing: Process-based parallelism.
* netrc: Loading of .netrc files.
* nis (Unix): Interface to Sun's NIS (Yellow Pages) library.
* nntplib: NNTP protocol client (requires sockets).
* numbers: Numeric abstract base classes (Complex, Real, Integral, etc.).
* operator: Functions corresponding to the standard operators.
* optparse: Deprecated: Command-line option parsing library.
* os: Miscellaneous operating system interfaces.
* ossaudiodev (Linux, FreeBSD): Access to OSS-compatible audio devices.
* parser: Access parse trees for Python source code.
* pathlib: Object-oriented filesystem paths
* pdb: The Python debugger for interactive interpreters.
* pickle: Convert Python objects to streams of bytes and back.
* pickletools: Contains extensive comments about the pickle protocols and pickle-machine opcodes, as well as some useful functions.
* pipes (Unix): A Python interface to Unix shell pipelines.
* pkgutil: Utilities for the import system.
* platform: Retrieves as much platform identifying data as possible.
* plistlib: Generate and parse Mac OS X plist files.
* poplib: POP3 protocol client (requires sockets).
* posix (Unix): The most common POSIX system calls (normally used via module os).
* pprint: Data pretty printer.
* profile: Python source profiler.
* pstats: Statistics object for use with the profiler.
* pty (Linux): Pseudo-Terminal Handling for Linux.
* pwd (Unix): The password database (getpwnam and friends).
* py_compile: Generate byte-code files from Python source files.
* pyclbr: Supports information extraction for a Python class browser.
* pydoc: Documentation generator and online help system.
* queue: A synchronized queue class.
* quopri: Encode and decode files using the MIME quoted-printable encoding.
* random: Generate pseudo-random numbers with various common distributions.
* re: Regular expression operations.
* readline (Unix): GNU readline support for Python.
* reprlib: Alternate repr implementation with size limits.
* resource (Unix): An interface to provide resource usage information on the current process.
* rlcompleter: Python identifier completion, suitable for the GNU readline library.
* runpy: Locate and run Python modules without importing them first.
* sched: General purpose event scheduler.
* secrets: Generate secure random numbers for managing secrets.
* select: Wait for I/O completion on multiple streams.
* selectors: High-level I/O multiplexing.
* shelve: Python object persistence.
* shlex: Simple lexical analysis for Unix shell-like languages.
* shutil: High-level file operations, including copying.
* signal: Set handlers for asynchronous events.
* site: Module responsible for site-specific configuration.
* smtpd: A SMTP server implementation in Python.
* smtplib: SMTP protocol client (requires sockets).
* sndhdr: Determine type of a sound file.
* socket: Low-level networking interface.
* socketserver: A framework for network servers.
* spwd (Unix): The shadow password database (getspnam and friends).
* sqlite3: A DB-API 2.0 implementation using SQLite 3.x.
* ssl: TLS/SSL wrapper for socket objects
* stat: Utilities for interpreting the results of os.stat, os.lstat and os.fstat.
* statistics: mathematical statistics functions
* string: Common string operations.
* stringprep: String preparation, as per RFC 3453
* struct: Interpret bytes as packed binary data.
* subprocess: Subprocess management.
* sunau: Provide an interface to the Sun AU sound format.
* symbol: Constants representing internal nodes of the parse tree.
* symtable: Interface to the compiler's internal symbol tables.
* sys: Access system-specific parameters and functions.
* sysconfig: Python's configuration information
* syslog (Unix): An interface to the Unix syslog library routines.
* tabnanny: Tool for detecting white space related problems in Python source files in a directory tree.
* tarfile: Read and write tar-format archive files.
* telnetlib: Telnet client class.
* tempfile: Generate temporary files and directories.
* termios (Unix): POSIX style tty control.
* test: Regression tests package containing the testing suite for Python.
* textwrap: Text wrapping and filling
* threading: Thread-based parallelism.
* time: Time access and conversions.
* timeit: Measure the execution time of small code snippets.
* tkinter: Interface to Tcl/Tk for graphical user interfaces
* token: Constants representing terminal nodes of the parse tree.
* tokenize: Lexical scanner for Python source code.
* trace: Trace or track Python statement execution.
* traceback: Print or retrieve a stack traceback.
* tracemalloc: Trace memory allocations.
* tty (Unix): Utility functions that perform common terminal control operations.
* turtle: An educational framework for simple graphics applications
* turtledemo: A viewer for example turtle scripts
* types: Names for built-in types.
* typing: Support for type hints (see PEP 484).
* unicodedata: Access the Unicode Database.
* unittest: Unit testing framework for Python.
* urllib:
* uu: Encode and decode files in uuencode format.
* uuid: UUID objects (universally unique identifiers) according to RFC 4122
* venv: Creation of virtual environments.
* warnings: Issue warning messages and control their disposition.
* wave: Provide an interface to the WAV sound format.
* weakref: Support for weak references and weak dictionaries.
* webbrowser: Easy-to-use controller for Web browsers.
* winreg (Windows): Routines and objects for manipulating the Windows registry.
* winsound (Windows): Access to the sound-playing machinery for Windows.
* wsgiref: WSGI Utilities and Reference Implementation.
* xdrlib: Encoders and decoders for the External Data Representation (XDR).
* xml: Package containing XML processing modules
* zipapp: Manage executable python zip archives
* zipfile: Read and write ZIP-format archive files.
* zipimport: support for importing Python modules from ZIP archives.
* zlib: Low-level interface to compression and decompression routines compatible with gzip.

# Naming conventions

There are various naming conventions used in Python programs.
PEP 0008 specifies naming conventions for class names (e.g. GenericTree), package and module names (e.g. generictree and generic_tree), function and variable names (storestate or store_state; mixedCase is dispreferred), and more. Google Python Style Guide follows similar naming conventions.
The above stands in contrast to Java naming convention (e.g. storeState for method names) and C# naming convention (e.g. StoreState for method names).
In Python 2, the standard library contains multiple deviations from PEP 0008. For instance, Tkinter module is spelled as Tkinter with capital T; this was renamed to tkinter in Python 3.

## External links

* Naming Conventions in PEP 0008: Style Guide for Python Code, python.org
* Naming in Google Python Style Guide, google.github.io
* pycodestyle - Python style guide checker, pypi.python.org
* Modules to Rename in PEP 3108, python.org
* W:Naming convention (programming)