Source code: Lib/functools.py
The functools module is for higher-order functions: functions that act on or return other functions. In general, any callable object can be treated as a function for the purposes of this module.
The functools module defines the following functions:
Simple lightweight unbounded function cache. Sometimes called “memoize”.
Returns the same as lru_cache(maxsize=None), creating a thin wrapper around a dictionary lookup for the function arguments. Because it never needs to evict old values, this is smaller and faster than lru_cache() with a size limit.
For example:
New in version 3.9.
Transform a method of a class into a property whose value is computed once and then cached as a normal attribute for the life of the instance. Similar to property(), with the addition of caching. Useful for expensive computed properties of instances that are otherwise effectively immutable.
Example:
The mechanics of cached_property() are somewhat different from property(). A regular property blocks attribute writes unless a setter is defined. In contrast, a cached_property allows writes.
The cached_property decorator only runs on lookups and only when an attribute of the same name doesn’t exist. When it does run, the cached_property writes to the attribute with the same name. Subsequent attribute reads and writes take precedence over the cached_property method and it works like a normal attribute.
The cached value can be cleared by deleting the attribute. This allows the cached_property method to run again.
Note, this decorator interferes with the operation of PEP 412 key-sharing dictionaries. This means that instance dictionaries can take more space than usual.
Also, this decorator requires that the __dict__ attribute on each instance be a mutable mapping. This means it will not work with some types, such as metaclasses (since the __dict__ attributes on type instances are read-only proxies for the class namespace), and those that specify __slots__ without including __dict__ as one of the defined slots (as such classes don’t provide a __dict__ attribute at all).
If a mutable mapping is not available or if space-efficient key sharing is desired, an effect similar to cached_property() can be achieved by a stacking property() on top of cache():
New in version 3.8.
Transform an old-style comparison function to a key function. Used with tools that accept key functions (such as sorted(), min(), max(), heapq.nlargest(), heapq.nsmallest(), itertools.groupby()). This function is primarily used as a transition tool for programs being converted from Python 2 which supported the use of comparison functions.
A comparison function is any callable that accept two arguments, compares them, and returns a negative number for less-than, zero for equality, or a positive number for greater-than. A key function is a callable that accepts one argument and returns another value to be used as the sort key.
Example:
For sorting examples and a brief sorting tutorial, see Sorting HOW TO.
New in version 3.2.
Decorator to wrap a function with a memoizing callable that saves up to the maxsize most recent calls. It can save time when an expensive or I/O bound function is periodically called with the same arguments.
Since a dictionary is used to cache results, the positional and keyword arguments to the function must be hashable.
Distinct argument patterns may be considered to be distinct calls with separate cache entries. For example, f(a=1, b=2) and f(b=2, a=1) differ in their keyword argument order and may have two separate cache entries.
If user_function is specified, it must be a callable. This allows the lru_cache decorator to be applied directly to a user function, leaving the maxsize at its default value of 128:
If maxsize is set to None, the LRU feature is disabled and the cache can grow without bound.
If typed is set to true, function arguments of different types will be cached separately. For example, f(3) and f(3.0) will be treated as distinct calls with distinct results.
The wrapped function is instrumented with a cache_parameters() function that returns a new dict showing the values for maxsize and typed. This is for information purposes only. Mutating the values has no effect.
To help measure the effectiveness of the cache and tune the maxsize parameter, the wrapped function is instrumented with a cache_info() function that returns a named tuple showing hits, misses, maxsize and currsize. In a multi-threaded environment, the hits and misses are approximate.
The decorator also provides a cache_clear() function for clearing or invalidating the cache.
The original underlying function is accessible through the __wrapped__ attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache.
An LRU (least recently used) cache works best when the most recent calls are the best predictors of upcoming calls (for example, the most popular articles on a news server tend to change each day). The cache’s size limit assures that the cache does not grow without bound on long-running processes such as web servers.
In general, the LRU cache should only be used when you want to reuse previously computed values. Accordingly, it doesn’t make sense to cache functions with side-effects, functions that need to create distinct mutable objects on each call, or impure functions such as time() or random().
Example of an LRU cache for static web content:
Example of efficiently computing Fibonacci numbers using a cache to implement a dynamic programming technique:
New in version 3.2.
Changed in version 3.3: Added the typed option.
Changed in version 3.8: Added the user_function option.
New in version 3.9: Added the function cache_parameters()
Given a class defining one or more rich comparison ordering methods, this class decorator supplies the rest. This simplifies the effort involved in specifying all of the possible rich comparison operations:
The class must define one of __lt__(), __le__(), __gt__(), or __ge__(). In addition, the class should supply an __eq__() method.
For example:
Note
While this decorator makes it easy to create well behaved totally ordered types, it does come at the cost of slower execution and more complex stack traces for the derived comparison methods. If performance benchmarking indicates this is a bottleneck for a given application, implementing all six rich comparison methods instead is likely to provide an easy speed boost.
New in version 3.2.
Changed in version 3.4: Returning NotImplemented from the underlying comparison function for unrecognised types is now supported.
Return a new partial object which when called will behave like func called with the positional arguments args and keyword arguments keywords. If more arguments are supplied to the call, they are appended to args. If additional keyword arguments are supplied, they extend and override keywords. Roughly equivalent to:
The partial() is used for partial function application which “freezes” some portion of a function’s arguments and/or keywords resulting in a new object with a simplified signature. For example, partial() can be used to create a callable that behaves like the int() function where the base argument defaults to two:
Return a new partialmethod descriptor which behaves like partial except that it is designed to be used as a method definition rather than being directly callable.
func must be a descriptor or a callable (objects which are both, like normal functions, are handled as descriptors).
When func is a descriptor (such as a normal Python function, classmethod(), staticmethod(), abstractmethod() or another instance of partialmethod), calls to __get__ are delegated to the underlying descriptor, and an appropriate partial object returned as the result.
When func is a non-descriptor callable, an appropriate bound method is created dynamically. This behaves like a normal Python function when used as a method: the self argument will be inserted as the first positional argument, even before the args and keywords supplied to the partialmethod constructor.
Example:
New in version 3.4.
Apply function of two arguments cumulatively to the items of iterable, from left to right, so as to reduce the iterable to a single value. For example, reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates ((((1+2)+3)+4)+5). The left argument, x, is the accumulated value and the right argument, y, is the update value from the iterable. If the optional initializer is present, it is placed before the items of the iterable in the calculation, and serves as a default when the iterable is empty. If initializer is not given and iterable contains only one item, the first item is returned.
Roughly equivalent to:
See itertools.accumulate() for an iterator that yields all intermediate values.
Transform a function into a single-dispatch generic function.
To define a generic function, decorate it with the @singledispatch decorator. Note that the dispatch happens on the type of the first argument, create your function accordingly:
To add overloaded implementations to the function, use the register() attribute of the generic function. It is a decorator. For functions annotated with types, the decorator will infer the type of the first argument automatically:
For code which doesn’t use type annotations, the appropriate type argument can be passed explicitly to the decorator itself:
To enable registering lambdas and pre-existing functions, the register() attribute can be used in a functional form:
The register() attribute returns the undecorated function which enables decorator stacking, pickling, as well as creating unit tests for each variant independently:
When called, the generic function dispatches on the type of the first argument:
Where there is no registered implementation for a specific type, its method resolution order is used to find a more generic implementation. The original function decorated with @singledispatch is registered for the base object type, which means it is used if no better implementation is found.
If an implementation registered to abstract base class, virtual subclasses will be dispatched to that implementation:
To check which implementation will the generic function choose for a given type, use the dispatch() attribute:
To access all registered implementations, use the read-only registry attribute:
New in version 3.4.
Changed in version 3.7: The register() attribute supports using type annotations.
Transform a method into a single-dispatch generic function.
To define a generic method, decorate it with the @singledispatchmethod decorator. Note that the dispatch happens on the type of the first non-self or non-cls argument, create your function accordingly:
@singledispatchmethod supports nesting with other decorators such as @classmethod. Note that to allow for dispatcher.register, singledispatchmethod must be the outer most decorator. Here is the Negator class with the neg methods being class bound:
The same pattern can be used for other similar decorators: staticmethod, abstractmethod, and others.
New in version 3.8.
Update a wrapper function to look like the wrapped function. The optional arguments are tuples to specify which attributes of the original function are assigned directly to the matching attributes on the wrapper function and which attributes of the wrapper function are updated with the corresponding attributes from the original function. The default values for these arguments are the module level constants WRAPPER_ASSIGNMENTS (which assigns to the wrapper function’s __module__, __name__, __qualname__, __annotations__ and __doc__, the documentation string) and WRAPPER_UPDATES (which updates the wrapper function’s __dict__, i.e. the instance dictionary).
To allow access to the original function for introspection and other purposes (e.g. bypassing a caching decorator such as lru_cache()), this function automatically adds a __wrapped__ attribute to the wrapper that refers to the function being wrapped.
The main intended use for this function is in decorator functions which wrap the decorated function and return the wrapper. If the wrapper function is not updated, the metadata of the returned function will reflect the wrapper definition rather than the original function definition, which is typically less than helpful.
update_wrapper() may be used with callables other than functions. Any attributes named in assigned or updated that are missing from the object being wrapped are ignored (i.e. this function will not attempt to set them on the wrapper function). AttributeError is still raised if the wrapper function itself is missing any attributes named in updated.
New in version 3.2: Automatic addition of the __wrapped__ attribute.
New in version 3.2: Copying of the __annotations__ attribute by default.
Changed in version 3.2: Missing attributes no longer trigger an AttributeError.
Changed in version 3.4: The __wrapped__ attribute now always refers to the wrapped function, even if that function defined a __wrapped__ attribute. (see bpo-17482)
This is a convenience function for invoking update_wrapper() as a function decorator when defining a wrapper function. It is equivalent to partial(update_wrapper, wrapped=wrapped, assigned=assigned, updated=updated). For example:
Without the use of this decorator factory, the name of the example function would have been 'wrapper', and the docstring of the original example() would have been lost.
partial objects are callable objects created by partial(). They have three read-only attributes:
A callable object or function. Calls to the partial object will be forwarded to func with new arguments and keywords.
The leftmost positional arguments that will be prepended to the positional arguments provided to a partial object call.
The keyword arguments that will be supplied when the partial object is called.
partial objects are like function objects in that they are callable, weak referencable, and can have attributes. There are some important differences. For instance, the __name__ and __doc__ attributes are not created automatically. Also, partial objects defined in classes behave like static methods and do not transform into bound methods during instance attribute look-up.


Simple lightweight unbounded function cache. Sometimes called “memoize”.
Returns the same as lru_cache(maxsize=None), creating a thin wrapper around a dictionary lookup for the function arguments. Because it never needs to evict old values, this is smaller and faster than lru_cache() with a size limit.
For example:
New in version 3.9.


Transform a method of a class into a property whose value is computed once and then cached as a normal attribute for the life of the instance. Similar to property(), with the addition of caching. Useful for expensive computed properties of instances that are otherwise effectively immutable.
Example:
The mechanics of cached_property() are somewhat different from property(). A regular property blocks attribute writes unless a setter is defined. In contrast, a cached_property allows writes.
The cached_property decorator only runs on lookups and only when an attribute of the same name doesn’t exist. When it does run, the cached_property writes to the attribute with the same name. Subsequent attribute reads and writes take precedence over the cached_property method and it works like a normal attribute.
The cached value can be cleared by deleting the attribute. This allows the cached_property method to run again.
Note, this decorator interferes with the operation of PEP 412 key-sharing dictionaries. This means that instance dictionaries can take more space than usual.
Also, this decorator requires that the __dict__ attribute on each instance be a mutable mapping. This means it will not work with some types, such as metaclasses (since the __dict__ attributes on type instances are read-only proxies for the class namespace), and those that specify __slots__ without including __dict__ as one of the defined slots (as such classes don’t provide a __dict__ attribute at all).
If a mutable mapping is not available or if space-efficient key sharing is desired, an effect similar to cached_property() can be achieved by a stacking property() on top of cache():
New in version 3.8.


Transform an old-style comparison function to a key function. Used with tools that accept key functions (such as sorted(), min(), max(), heapq.nlargest(), heapq.nsmallest(), itertools.groupby()). This function is primarily used as a transition tool for programs being converted from Python 2 which supported the use of comparison functions.
A comparison function is any callable that accept two arguments, compares them, and returns a negative number for less-than, zero for equality, or a positive number for greater-than. A key function is a callable that accepts one argument and returns another value to be used as the sort key.
Example:
For sorting examples and a brief sorting tutorial, see Sorting HOW TO.
New in version 3.2.


Decorator to wrap a function with a memoizing callable that saves up to the maxsize most recent calls. It can save time when an expensive or I/O bound function is periodically called with the same arguments.
Since a dictionary is used to cache results, the positional and keyword arguments to the function must be hashable.
Distinct argument patterns may be considered to be distinct calls with separate cache entries. For example, f(a=1, b=2) and f(b=2, a=1) differ in their keyword argument order and may have two separate cache entries.
If user_function is specified, it must be a callable. This allows the lru_cache decorator to be applied directly to a user function, leaving the maxsize at its default value of 128:
If maxsize is set to None, the LRU feature is disabled and the cache can grow without bound.
If typed is set to true, function arguments of different types will be cached separately. For example, f(3) and f(3.0) will be treated as distinct calls with distinct results.
The wrapped function is instrumented with a cache_parameters() function that returns a new dict showing the values for maxsize and typed. This is for information purposes only. Mutating the values has no effect.
To help measure the effectiveness of the cache and tune the maxsize parameter, the wrapped function is instrumented with a cache_info() function that returns a named tuple showing hits, misses, maxsize and currsize. In a multi-threaded environment, the hits and misses are approximate.
The decorator also provides a cache_clear() function for clearing or invalidating the cache.
The original underlying function is accessible through the __wrapped__ attribute. This is useful for introspection, for bypassing the cache, or for rewrapping the function with a different cache.
An LRU (least recently used) cache works best when the most recent calls are the best predictors of upcoming calls (for example, the most popular articles on a news server tend to change each day). The cache’s size limit assures that the cache does not grow without bound on long-running processes such as web servers.
In general, the LRU cache should only be used when you want to reuse previously computed values. Accordingly, it doesn’t make sense to cache functions with side-effects, functions that need to create distinct mutable objects on each call, or impure functions such as time() or random().
Example of an LRU cache for static web content:
Example of efficiently computing Fibonacci numbers using a cache to implement a dynamic programming technique:
New in version 3.2.
Changed in version 3.3: Added the typed option.
Changed in version 3.8: Added the user_function option.
New in version 3.9: Added the function cache_parameters()


Return a new partial object which when called will behave like func called with the positional arguments args and keyword arguments keywords. If more arguments are supplied to the call, they are appended to args. If additional keyword arguments are supplied, they extend and override keywords. Roughly equivalent to:
The partial() is used for partial function application which “freezes” some portion of a function’s arguments and/or keywords resulting in a new object with a simplified signature. For example, partial() can be used to create a callable that behaves like the int() function where the base argument defaults to two:


The leftmost positional arguments that will be prepended to the positional arguments provided to a partial object call.


A callable object or function. Calls to the partial object will be forwarded to func with new arguments and keywords.


The keyword arguments that will be supplied when the partial object is called.


Return a new partialmethod descriptor which behaves like partial except that it is designed to be used as a method definition rather than being directly callable.
func must be a descriptor or a callable (objects which are both, like normal functions, are handled as descriptors).
When func is a descriptor (such as a normal Python function, classmethod(), staticmethod(), abstractmethod() or another instance of partialmethod), calls to __get__ are delegated to the underlying descriptor, and an appropriate partial object returned as the result.
When func is a non-descriptor callable, an appropriate bound method is created dynamically. This behaves like a normal Python function when used as a method: the self argument will be inserted as the first positional argument, even before the args and keywords supplied to the partialmethod constructor.
Example:
New in version 3.4.


Apply function of two arguments cumulatively to the items of iterable, from left to right, so as to reduce the iterable to a single value. For example, reduce(lambda x, y: x+y, [1, 2, 3, 4, 5]) calculates ((((1+2)+3)+4)+5). The left argument, x, is the accumulated value and the right argument, y, is the update value from the iterable. If the optional initializer is present, it is placed before the items of the iterable in the calculation, and serves as a default when the iterable is empty. If initializer is not given and iterable contains only one item, the first item is returned.
Roughly equivalent to:
See itertools.accumulate() for an iterator that yields all intermediate values.


Transform a function into a single-dispatch generic function.
To define a generic function, decorate it with the @singledispatch decorator. Note that the dispatch happens on the type of the first argument, create your function accordingly:
To add overloaded implementations to the function, use the register() attribute of the generic function. It is a decorator. For functions annotated with types, the decorator will infer the type of the first argument automatically:
For code which doesn’t use type annotations, the appropriate type argument can be passed explicitly to the decorator itself:
To enable registering lambdas and pre-existing functions, the register() attribute can be used in a functional form:
The register() attribute returns the undecorated function which enables decorator stacking, pickling, as well as creating unit tests for each variant independently:
When called, the generic function dispatches on the type of the first argument:
Where there is no registered implementation for a specific type, its method resolution order is used to find a more generic implementation. The original function decorated with @singledispatch is registered for the base object type, which means it is used if no better implementation is found.
If an implementation registered to abstract base class, virtual subclasses will be dispatched to that implementation:
To check which implementation will the generic function choose for a given type, use the dispatch() attribute:
To access all registered implementations, use the read-only registry attribute:
New in version 3.4.
Changed in version 3.7: The register() attribute supports using type annotations.


Transform a method into a single-dispatch generic function.
To define a generic method, decorate it with the @singledispatchmethod decorator. Note that the dispatch happens on the type of the first non-self or non-cls argument, create your function accordingly:
@singledispatchmethod supports nesting with other decorators such as @classmethod. Note that to allow for dispatcher.register, singledispatchmethod must be the outer most decorator. Here is the Negator class with the neg methods being class bound:
The same pattern can be used for other similar decorators: staticmethod, abstractmethod, and others.
New in version 3.8.


Given a class defining one or more rich comparison ordering methods, this class decorator supplies the rest. This simplifies the effort involved in specifying all of the possible rich comparison operations:
The class must define one of __lt__(), __le__(), __gt__(), or __ge__(). In addition, the class should supply an __eq__() method.
For example:
Note
While this decorator makes it easy to create well behaved totally ordered types, it does come at the cost of slower execution and more complex stack traces for the derived comparison methods. If performance benchmarking indicates this is a bottleneck for a given application, implementing all six rich comparison methods instead is likely to provide an easy speed boost.
New in version 3.2.
Changed in version 3.4: Returning NotImplemented from the underlying comparison function for unrecognised types is now supported.


Update a wrapper function to look like the wrapped function. The optional arguments are tuples to specify which attributes of the original function are assigned directly to the matching attributes on the wrapper function and which attributes of the wrapper function are updated with the corresponding attributes from the original function. The default values for these arguments are the module level constants WRAPPER_ASSIGNMENTS (which assigns to the wrapper function’s __module__, __name__, __qualname__, __annotations__ and __doc__, the documentation string) and WRAPPER_UPDATES (which updates the wrapper function’s __dict__, i.e. the instance dictionary).
To allow access to the original function for introspection and other purposes (e.g. bypassing a caching decorator such as lru_cache()), this function automatically adds a __wrapped__ attribute to the wrapper that refers to the function being wrapped.
The main intended use for this function is in decorator functions which wrap the decorated function and return the wrapper. If the wrapper function is not updated, the metadata of the returned function will reflect the wrapper definition rather than the original function definition, which is typically less than helpful.
update_wrapper() may be used with callables other than functions. Any attributes named in assigned or updated that are missing from the object being wrapped are ignored (i.e. this function will not attempt to set them on the wrapper function). AttributeError is still raised if the wrapper function itself is missing any attributes named in updated.
New in version 3.2: Automatic addition of the __wrapped__ attribute.
New in version 3.2: Copying of the __annotations__ attribute by default.
Changed in version 3.2: Missing attributes no longer trigger an AttributeError.
Changed in version 3.4: The __wrapped__ attribute now always refers to the wrapped function, even if that function defined a __wrapped__ attribute. (see bpo-17482)


This is a convenience function for invoking update_wrapper() as a function decorator when defining a wrapper function. It is equivalent to partial(update_wrapper, wrapped=wrapped, assigned=assigned, updated=updated). For example:
Without the use of this decorator factory, the name of the example function would have been 'wrapper', and the docstring of the original example() would have been lost.


Source code: Lib/asyncio/futures.py, Lib/asyncio/base_futures.py
Future objects are used to bridge low-level callback-based code with high-level async/await code.
Return True if obj is either of:
New in version 3.5.
Return:
If obj is neither of the above a TypeError is raised.
Important
See also the create_task() function which is the preferred way for creating new Tasks.
Changed in version 3.5.1: The function accepts any awaitable object.
Wrap a concurrent.futures.Future object in a asyncio.Future object.
A Future represents an eventual result of an asynchronous operation. Not thread-safe.
Future is an awaitable object. Coroutines can await on Future objects until they either have a result or an exception set, or until they are cancelled.
Typically Futures are used to enable low-level callback-based code (e.g. in protocols implemented using asyncio transports) to interoperate with high-level async/await code.
The rule of thumb is to never expose Future objects in user-facing APIs, and the recommended way to create a Future object is to call loop.create_future(). This way alternative event loop implementations can inject their own optimized implementations of a Future object.
Changed in version 3.7: Added support for the contextvars module.
Return the result of the Future.
If the Future is done and has a result set by the set_result() method, the result value is returned.
If the Future is done and has an exception set by the set_exception() method, this method raises the exception.
If the Future has been cancelled, this method raises a CancelledError exception.
If the Future’s result isn’t yet available, this method raises a InvalidStateError exception.
Mark the Future as done and set its result.
Raises a InvalidStateError error if the Future is already done.
Mark the Future as done and set an exception.
Raises a InvalidStateError error if the Future is already done.
Return True if the Future is done.
A Future is done if it was cancelled or if it has a result or an exception set with set_result() or set_exception() calls.
Return True if the Future was cancelled.
The method is usually used to check if a Future is not cancelled before setting a result or an exception for it:
Add a callback to be run when the Future is done.
The callback is called with the Future object as its only argument.
If the Future is already done when this method is called, the callback is scheduled with loop.call_soon().
An optional keyword-only context argument allows specifying a custom contextvars.Context for the callback to run in. The current context is used when no context is provided.
functools.partial() can be used to pass parameters to the callback, e.g.:
Changed in version 3.7: The context keyword-only parameter was added. See PEP 567 for more details.
Remove callback from the callbacks list.
Returns the number of callbacks removed, which is typically 1, unless a callback was added more than once.
Cancel the Future and schedule callbacks.
If the Future is already done or cancelled, return False. Otherwise, change the Future’s state to cancelled, schedule the callbacks, and return True.
Changed in version 3.9: Added the msg parameter.
Return the exception that was set on this Future.
The exception (or None if no exception was set) is returned only if the Future is done.
If the Future has been cancelled, this method raises a CancelledError exception.
If the Future isn’t done yet, this method raises an InvalidStateError exception.
Return the event loop the Future object is bound to.
New in version 3.7.
This example creates a Future object, creates and schedules an asynchronous Task to set result for the Future, and waits until the Future has a result:
Important
The Future object was designed to mimic concurrent.futures.Future. Key differences include:


Base class for warnings about deprecated features when those warnings are intended for end users of applications that are written in Python.


This module provides an interface to the optional garbage collector. It provides the ability to disable the collector, tune the collection frequency, and set debugging options. It also provides access to unreachable objects that the collector found but cannot free. Since the collector supplements the reference counting already used in Python, you can disable the collector if you are sure your program does not create reference cycles. Automatic collection can be disabled by calling gc.disable(). To debug a leaking program call gc.set_debug(gc.DEBUG_LEAK). Notice that this includes gc.DEBUG_SAVEALL, causing garbage-collected objects to be saved in gc.garbage for inspection.
The gc module provides the following functions:
Enable automatic garbage collection.
Disable automatic garbage collection.
Return True if automatic collection is enabled.
With no arguments, run a full collection. The optional argument generation may be an integer specifying which generation to collect (from 0 to 2). A ValueError is raised if the generation number is invalid. The number of unreachable objects found is returned.
The free lists maintained for a number of built-in types are cleared whenever a full collection or collection of the highest generation (2) is run. Not all items in some free lists may be freed due to the particular implementation, in particular float.
Set the garbage collection debugging flags. Debugging information will be written to sys.stderr. See below for a list of debugging flags which can be combined using bit operations to control debugging.
Return the debugging flags currently set.
Returns a list of all objects tracked by the collector, excluding the list returned. If generation is not None, return only the objects tracked by the collector that are in that generation.
Changed in version 3.8: New generation parameter.
Raises an auditing event gc.get_objects with argument generation.
Return a list of three per-generation dictionaries containing collection statistics since interpreter start. The number of keys may change in the future, but currently each dictionary will contain the following items:
New in version 3.4.
Set the garbage collection thresholds (the collection frequency). Setting threshold0 to zero disables collection.
The GC classifies objects into three generations depending on how many collection sweeps they have survived. New objects are placed in the youngest generation (generation 0). If an object survives a collection it is moved into the next older generation. Since generation 2 is the oldest generation, objects in that generation remain there after a collection. In order to decide when to run, the collector keeps track of the number object allocations and deallocations since the last collection. When the number of allocations minus the number of deallocations exceeds threshold0, collection starts. Initially only generation 0 is examined. If generation 0 has been examined more than threshold1 times since generation 1 has been examined, then generation 1 is examined as well. With the third generation, things are a bit more complicated, see Collecting the oldest generation for more information.
Return the current collection counts as a tuple of (count0, count1,
count2).
Return the current collection thresholds as a tuple of (threshold0,
threshold1, threshold2).
Return the list of objects that directly refer to any of objs. This function will only locate those containers which support garbage collection; extension types which do refer to other objects but do not support garbage collection will not be found.
Note that objects which have already been dereferenced, but which live in cycles and have not yet been collected by the garbage collector can be listed among the resulting referrers. To get only currently live objects, call collect() before calling get_referrers().
Warning
Care must be taken when using objects returned by get_referrers() because some of them could still be under construction and hence in a temporarily invalid state. Avoid using get_referrers() for any purpose other than debugging.
Raises an auditing event gc.get_referrers with argument objs.
Return a list of objects directly referred to by any of the arguments. The referents returned are those objects visited by the arguments’ C-level tp_traverse methods (if any), and may not be all objects actually directly reachable. tp_traverse methods are supported only by objects that support garbage collection, and are only required to visit objects that may be involved in a cycle. So, for example, if an integer is directly reachable from an argument, that integer object may or may not appear in the result list.
Raises an auditing event gc.get_referents with argument objs.
Returns True if the object is currently tracked by the garbage collector, False otherwise. As a general rule, instances of atomic types aren’t tracked and instances of non-atomic types (containers, user-defined objects…) are. However, some type-specific optimizations can be present in order to suppress the garbage collector footprint of simple instances (e.g. dicts containing only atomic keys and values):
New in version 3.1.
Returns True if the given object has been finalized by the garbage collector, False otherwise.
New in version 3.9.
Freeze all the objects tracked by gc - move them to a permanent generation and ignore all the future collections. This can be used before a POSIX fork() call to make the gc copy-on-write friendly or to speed up collection. Also collection before a POSIX fork() call may free pages for future allocation which can cause copy-on-write too so it’s advised to disable gc in parent process and freeze before fork and enable gc in child process.
New in version 3.7.
Unfreeze the objects in the permanent generation, put them back into the oldest generation.
New in version 3.7.
Return the number of objects in the permanent generation.
New in version 3.7.
The following variables are provided for read-only access (you can mutate the values but should not rebind them):
A list of objects which the collector found to be unreachable but could not be freed (uncollectable objects). Starting with Python 3.4, this list should be empty most of the time, except when using instances of C extension types with a non-NULL tp_del slot.
If DEBUG_SAVEALL is set, then all unreachable objects will be added to this list rather than freed.
Changed in version 3.2: If this list is non-empty at interpreter shutdown, a ResourceWarning is emitted, which is silent by default. If DEBUG_UNCOLLECTABLE is set, in addition all uncollectable objects are printed.
Changed in version 3.4: Following PEP 442, objects with a __del__() method don’t end up in gc.garbage anymore.
A list of callbacks that will be invoked by the garbage collector before and after collection. The callbacks will be called with two arguments, phase and info.
phase can be one of two values:
“start”: The garbage collection is about to start.
“stop”: The garbage collection has finished.
info is a dict providing more information for the callback. The following keys are currently defined:
“generation”: The oldest generation being collected.
“collected”: When phase is “stop”, the number of objects successfully collected.
“uncollectable”: When phase is “stop”, the number of objects that could not be collected and were put in garbage.
Applications can add their own callbacks to this list. The primary use cases are:
Gathering statistics about garbage collection, such as how often various generations are collected, and how long the collection takes.
Allowing applications to identify and clear their own uncollectable types when they appear in garbage.
New in version 3.3.
The following constants are provided for use with set_debug():
Print statistics during collection. This information can be useful when tuning the collection frequency.
Print information on collectable objects found.
Print information of uncollectable objects found (objects which are not reachable but cannot be freed by the collector). These objects will be added to the garbage list.
Changed in version 3.2: Also print the contents of the garbage list at interpreter shutdown, if it isn’t empty.
When set, all unreachable objects found will be appended to garbage rather than being freed. This can be useful for debugging a leaking program.
The debugging flags necessary for the collector to print information about a leaking program (equal to DEBUG_COLLECTABLE | DEBUG_UNCOLLECTABLE |
DEBUG_SAVEALL).


A list of callbacks that will be invoked by the garbage collector before and after collection. The callbacks will be called with two arguments, phase and info.
phase can be one of two values:
“start”: The garbage collection is about to start.
“stop”: The garbage collection has finished.
info is a dict providing more information for the callback. The following keys are currently defined:
“generation”: The oldest generation being collected.
“collected”: When phase is “stop”, the number of objects successfully collected.
“uncollectable”: When phase is “stop”, the number of objects that could not be collected and were put in garbage.
Applications can add their own callbacks to this list. The primary use cases are:
Gathering statistics about garbage collection, such as how often various generations are collected, and how long the collection takes.
Allowing applications to identify and clear their own uncollectable types when they appear in garbage.
New in version 3.3.


With no arguments, run a full collection. The optional argument generation may be an integer specifying which generation to collect (from 0 to 2). A ValueError is raised if the generation number is invalid. The number of unreachable objects found is returned.
The free lists maintained for a number of built-in types are cleared whenever a full collection or collection of the highest generation (2) is run. Not all items in some free lists may be freed due to the particular implementation, in particular float.


Print information on collectable objects found.


The debugging flags necessary for the collector to print information about a leaking program (equal to DEBUG_COLLECTABLE | DEBUG_UNCOLLECTABLE |
DEBUG_SAVEALL).


When set, all unreachable objects found will be appended to garbage rather than being freed. This can be useful for debugging a leaking program.


Print statistics during collection. This information can be useful when tuning the collection frequency.


Print information of uncollectable objects found (objects which are not reachable but cannot be freed by the collector). These objects will be added to the garbage list.
Changed in version 3.2: Also print the contents of the garbage list at interpreter shutdown, if it isn’t empty.


Freeze all the objects tracked by gc - move them to a permanent generation and ignore all the future collections. This can be used before a POSIX fork() call to make the gc copy-on-write friendly or to speed up collection. Also collection before a POSIX fork() call may free pages for future allocation which can cause copy-on-write too so it’s advised to disable gc in parent process and freeze before fork and enable gc in child process.
New in version 3.7.


A list of objects which the collector found to be unreachable but could not be freed (uncollectable objects). Starting with Python 3.4, this list should be empty most of the time, except when using instances of C extension types with a non-NULL tp_del slot.
If DEBUG_SAVEALL is set, then all unreachable objects will be added to this list rather than freed.
Changed in version 3.2: If this list is non-empty at interpreter shutdown, a ResourceWarning is emitted, which is silent by default. If DEBUG_UNCOLLECTABLE is set, in addition all uncollectable objects are printed.
Changed in version 3.4: Following PEP 442, objects with a __del__() method don’t end up in gc.garbage anymore.


Return the current collection counts as a tuple of (count0, count1,
count2).


Return the debugging flags currently set.


Return the number of objects in the permanent generation.
New in version 3.7.


Returns a list of all objects tracked by the collector, excluding the list returned. If generation is not None, return only the objects tracked by the collector that are in that generation.
Changed in version 3.8: New generation parameter.
Raises an auditing event gc.get_objects with argument generation.


Return a list of objects directly referred to by any of the arguments. The referents returned are those objects visited by the arguments’ C-level tp_traverse methods (if any), and may not be all objects actually directly reachable. tp_traverse methods are supported only by objects that support garbage collection, and are only required to visit objects that may be involved in a cycle. So, for example, if an integer is directly reachable from an argument, that integer object may or may not appear in the result list.
Raises an auditing event gc.get_referents with argument objs.


Return the list of objects that directly refer to any of objs. This function will only locate those containers which support garbage collection; extension types which do refer to other objects but do not support garbage collection will not be found.
Note that objects which have already been dereferenced, but which live in cycles and have not yet been collected by the garbage collector can be listed among the resulting referrers. To get only currently live objects, call collect() before calling get_referrers().
Warning
Care must be taken when using objects returned by get_referrers() because some of them could still be under construction and hence in a temporarily invalid state. Avoid using get_referrers() for any purpose other than debugging.
Raises an auditing event gc.get_referrers with argument objs.


Return a list of three per-generation dictionaries containing collection statistics since interpreter start. The number of keys may change in the future, but currently each dictionary will contain the following items:
New in version 3.4.


Return the current collection thresholds as a tuple of (threshold0,
threshold1, threshold2).


Return True if automatic collection is enabled.


Returns True if the given object has been finalized by the garbage collector, False otherwise.
New in version 3.9.


Returns True if the object is currently tracked by the garbage collector, False otherwise. As a general rule, instances of atomic types aren’t tracked and instances of non-atomic types (containers, user-defined objects…) are. However, some type-specific optimizations can be present in order to suppress the garbage collector footprint of simple instances (e.g. dicts containing only atomic keys and values):
New in version 3.1.


Set the garbage collection debugging flags. Debugging information will be written to sys.stderr. See below for a list of debugging flags which can be combined using bit operations to control debugging.


Set the garbage collection thresholds (the collection frequency). Setting threshold0 to zero disables collection.
The GC classifies objects into three generations depending on how many collection sweeps they have survived. New objects are placed in the youngest generation (generation 0). If an object survives a collection it is moved into the next older generation. Since generation 2 is the oldest generation, objects in that generation remain there after a collection. In order to decide when to run, the collector keeps track of the number object allocations and deallocations since the last collection. When the number of allocations minus the number of deallocations exceeds threshold0, collection starts. Initially only generation 0 is examined. If generation 0 has been examined more than threshold1 times since generation 1 has been examined, then generation 1 is examined as well. With the third generation, things are a bit more complicated, see Collecting the oldest generation for more information.


Unfreeze the objects in the permanent generation, put them back into the oldest generation.
New in version 3.7.


Raised when a generator or coroutine is closed; see generator.close() and coroutine.close(). It directly inherits from BaseException instead of Exception since it is technically not an error.


This attribute is a tuple (possibly of length 1) of generic types passed to the original __class_getitem__() of the generic container:


This attribute points at the non-parameterized generic class:


This attribute is a lazily computed tuple (possibly empty) of unique type variables found in __args__:


Return the value of the named attribute of object. name must be a string. If the string is the name of one of the object’s attributes, the result is the value of that attribute. For example, getattr(x, 'foobar') is equivalent to x.foobar. If the named attribute does not exist, default is returned if provided, otherwise AttributeError is raised.


Source code: Lib/getopt.py
Note
The getopt module is a parser for command line options whose API is designed to be familiar to users of the C getopt() function. Users who are unfamiliar with the C getopt() function or who would like to write less code and get better help and error messages should consider using the argparse module instead.
This module helps scripts to parse the command line arguments in sys.argv. It supports the same conventions as the Unix getopt() function (including the special meanings of arguments of the form ‘-‘ and ‘--‘). Long options similar to those supported by GNU software may be used as well via an optional third argument.
This module provides two functions and an exception:
Parses command line options and parameter list. args is the argument list to be parsed, without the leading reference to the running program. Typically, this means sys.argv[1:]. shortopts is the string of option letters that the script wants to recognize, with options that require an argument followed by a colon (':'; i.e., the same format that Unix getopt() uses).
Note
Unlike GNU getopt(), after a non-option argument, all further arguments are considered also non-options. This is similar to the way non-GNU Unix systems work.
longopts, if specified, must be a list of strings with the names of the long options which should be supported. The leading '--' characters should not be included in the option name. Long options which require an argument should be followed by an equal sign ('='). Optional arguments are not supported. To accept only long options, shortopts should be an empty string. Long options on the command line can be recognized so long as they provide a prefix of the option name that matches exactly one of the accepted options. For example, if longopts is ['foo', 'frob'], the option --fo will match as --foo, but --f will not match uniquely, so GetoptError will be raised.
The return value consists of two elements: the first is a list of (option,
value) pairs; the second is the list of program arguments left after the option list was stripped (this is a trailing slice of args). Each option-and-value pair returned has the option as its first element, prefixed with a hyphen for short options (e.g., '-x') or two hyphens for long options (e.g., '--long-option'), and the option argument as its second element, or an empty string if the option has no argument. The options occur in the list in the same order in which they were found, thus allowing multiple occurrences. Long and short options may be mixed.
This function works like getopt(), except that GNU style scanning mode is used by default. This means that option and non-option arguments may be intermixed. The getopt() function stops processing options as soon as a non-option argument is encountered.
If the first character of the option string is '+', or if the environment variable POSIXLY_CORRECT is set, then option processing stops as soon as a non-option argument is encountered.
This is raised when an unrecognized option is found in the argument list or when an option requiring an argument is given none. The argument to the exception is a string indicating the cause of the error. For long options, an argument given to an option which does not require one will also cause this exception to be raised. The attributes msg and opt give the error message and related option; if there is no specific option to which the exception relates, opt is an empty string.
Alias for GetoptError; for backward compatibility.
An example using only Unix style options:
Using long option names is equally easy:
In a script, typical usage is something like this:
Note that an equivalent command line interface could be produced with less code and more informative help and error messages by using the argparse module:
See also
Alternative command line option and argument parsing library.


Alias for GetoptError; for backward compatibility.


Parses command line options and parameter list. args is the argument list to be parsed, without the leading reference to the running program. Typically, this means sys.argv[1:]. shortopts is the string of option letters that the script wants to recognize, with options that require an argument followed by a colon (':'; i.e., the same format that Unix getopt() uses).
Note
Unlike GNU getopt(), after a non-option argument, all further arguments are considered also non-options. This is similar to the way non-GNU Unix systems work.
longopts, if specified, must be a list of strings with the names of the long options which should be supported. The leading '--' characters should not be included in the option name. Long options which require an argument should be followed by an equal sign ('='). Optional arguments are not supported. To accept only long options, shortopts should be an empty string. Long options on the command line can be recognized so long as they provide a prefix of the option name that matches exactly one of the accepted options. For example, if longopts is ['foo', 'frob'], the option --fo will match as --foo, but --f will not match uniquely, so GetoptError will be raised.
The return value consists of two elements: the first is a list of (option,
value) pairs; the second is the list of program arguments left after the option list was stripped (this is a trailing slice of args). Each option-and-value pair returned has the option as its first element, prefixed with a hyphen for short options (e.g., '-x') or two hyphens for long options (e.g., '--long-option'), and the option argument as its second element, or an empty string if the option has no argument. The options occur in the list in the same order in which they were found, thus allowing multiple occurrences. Long and short options may be mixed.


This is raised when an unrecognized option is found in the argument list or when an option requiring an argument is given none. The argument to the exception is a string indicating the cause of the error. For long options, an argument given to an option which does not require one will also cause this exception to be raised. The attributes msg and opt give the error message and related option; if there is no specific option to which the exception relates, opt is an empty string.


This function works like getopt(), except that GNU style scanning mode is used by default. This means that option and non-option arguments may be intermixed. The getopt() function stops processing options as soon as a non-option argument is encountered.
If the first character of the option string is '+', or if the environment variable POSIXLY_CORRECT is set, then option processing stops as soon as a non-option argument is encountered.


Source code: Lib/getpass.py
The getpass module provides two functions:
Prompt the user for a password without echoing. The user is prompted using the string prompt, which defaults to 'Password: '. On Unix, the prompt is written to the file-like object stream using the replace error handler if needed. stream defaults to the controlling terminal (/dev/tty) or if that is unavailable to sys.stderr (this argument is ignored on Windows).
If echo free input is unavailable getpass() falls back to printing a warning message to stream and reading from sys.stdin and issuing a GetPassWarning.
Note
If you call getpass from within IDLE, the input may be done in the terminal you launched IDLE from rather than the idle window itself.
A UserWarning subclass issued when password input may be echoed.
Return the “login name” of the user.
This function checks the environment variables LOGNAME, USER, LNAME and USERNAME, in order, and returns the value of the first one which is set to a non-empty string. If none are set, the login name from the password database is returned on systems which support the pwd module, otherwise, an exception is raised.
In general, this function should be preferred over os.getlogin().


Prompt the user for a password without echoing. The user is prompted using the string prompt, which defaults to 'Password: '. On Unix, the prompt is written to the file-like object stream using the replace error handler if needed. stream defaults to the controlling terminal (/dev/tty) or if that is unavailable to sys.stderr (this argument is ignored on Windows).
If echo free input is unavailable getpass() falls back to printing a warning message to stream and reading from sys.stdin and issuing a GetPassWarning.
Note
If you call getpass from within IDLE, the input may be done in the terminal you launched IDLE from rather than the idle window itself.


A UserWarning subclass issued when password input may be echoed.


Return the “login name” of the user.
This function checks the environment variables LOGNAME, USER, LNAME and USERNAME, in order, and returns the value of the first one which is set to a non-empty string. If none are set, the login name from the password database is returned on systems which support the pwd module, otherwise, an exception is raised.
In general, this function should be preferred over os.getlogin().


Source code: Lib/gettext.py
The gettext module provides internationalization (I18N) and localization (L10N) services for your Python modules and applications. It supports both the GNU gettext message catalog API and a higher level, class-based API that may be more appropriate for Python files. The interface described below allows you to write your module and application messages in one natural language, and provide a catalog of translated messages for running under different natural languages.
Some hints on localizing your Python modules and applications are also given.
The gettext module defines the following API, which is very similar to the GNU gettext API. If you use this API you will affect the translation of your entire application globally. Often this is what you want if your application is monolingual, with the choice of language dependent on the locale of your user. If you are localizing a Python module, or if your application needs to switch languages on the fly, you probably want to use the class-based API instead.
Bind the domain to the locale directory localedir. More concretely, gettext will look for binary .mo files for the given domain using the path (on Unix): localedir/language/LC_MESSAGES/domain.mo, where language is searched for in the environment variables LANGUAGE, LC_ALL, LC_MESSAGES, and LANG respectively.
If localedir is omitted or None, then the current binding for domain is returned. 1
Bind the domain to codeset, changing the encoding of byte strings returned by the lgettext(), ldgettext(), lngettext() and ldngettext() functions. If codeset is omitted, then the current binding is returned.
Deprecated since version 3.8, will be removed in version 3.10.
Change or query the current global domain. If domain is None, then the current global domain is returned, otherwise the global domain is set to domain, which is returned.
Return the localized translation of message, based on the current global domain, language, and locale directory. This function is usually aliased as _() in the local namespace (see examples below).
Like gettext(), but look the message up in the specified domain.
Like gettext(), but consider plural forms. If a translation is found, apply the plural formula to n, and return the resulting message (some languages have more than two plural forms). If no translation is found, return singular if n is 1; return plural otherwise.
The Plural formula is taken from the catalog header. It is a C or Python expression that has a free variable n; the expression evaluates to the index of the plural in the catalog. See the GNU gettext documentation for the precise syntax to be used in .po files and the formulas for a variety of languages.
Like ngettext(), but look the message up in the specified domain.
Similar to the corresponding functions without the p in the prefix (that is, gettext(), dgettext(), ngettext(), dngettext()), but the translation is restricted to the given message context.
New in version 3.8.
Equivalent to the corresponding functions without the l prefix (gettext(), dgettext(), ngettext() and dngettext()), but the translation is returned as a byte string encoded in the preferred system encoding if no other encoding was explicitly set with bind_textdomain_codeset().
Warning
These functions should be avoided in Python 3, because they return encoded bytes. It’s much better to use alternatives which return Unicode strings instead, since most Python applications will want to manipulate human readable text as strings instead of bytes. Further, it’s possible that you may get unexpected Unicode-related exceptions if there are encoding problems with the translated strings.
Deprecated since version 3.8, will be removed in version 3.10.
Note that GNU gettext also defines a dcgettext() method, but this was deemed not useful and so it is currently unimplemented.
Here’s an example of typical usage for this API:
The class-based API of the gettext module gives you more flexibility and greater convenience than the GNU gettext API. It is the recommended way of localizing your Python applications and modules. gettext defines a GNUTranslations class which implements the parsing of GNU .mo format files, and has methods for returning strings. Instances of this class can also install themselves in the built-in namespace as the function _().
This function implements the standard .mo file search algorithm. It takes a domain, identical to what textdomain() takes. Optional localedir is as in bindtextdomain(). Optional languages is a list of strings, where each string is a language code.
If localedir is not given, then the default system locale directory is used. 2 If languages is not given, then the following environment variables are searched: LANGUAGE, LC_ALL, LC_MESSAGES, and LANG. The first one returning a non-empty value is used for the languages variable. The environment variables should contain a colon separated list of languages, which will be split on the colon to produce the expected list of language code strings.
find() then expands and normalizes the languages, and then iterates through them, searching for an existing file built of these components:
localedir/language/LC_MESSAGES/domain.mo
The first such file name that exists is returned by find(). If no such file is found, then None is returned. If all is given, it returns a list of all file names, in the order in which they appear in the languages list or the environment variables.
Return a *Translations instance based on the domain, localedir, and languages, which are first passed to find() to get a list of the associated .mo file paths. Instances with identical .mo file names are cached. The actual class instantiated is class_ if provided, otherwise GNUTranslations. The class’s constructor must take a single file object argument. If provided, codeset will change the charset used to encode translated strings in the lgettext() and lngettext() methods.
If multiple files are found, later files are used as fallbacks for earlier ones. To allow setting the fallback, copy.copy() is used to clone each translation object from the cache; the actual instance data is still shared with the cache.
If no .mo file is found, this function raises OSError if fallback is false (which is the default), and returns a NullTranslations instance if fallback is true.
Changed in version 3.3: IOError used to be raised instead of OSError.
Deprecated since version 3.8, will be removed in version 3.10: The codeset parameter.
This installs the function _() in Python’s builtins namespace, based on domain, localedir, and codeset which are passed to the function translation().
For the names parameter, please see the description of the translation object’s install() method.
As seen below, you usually mark the strings in your application that are candidates for translation, by wrapping them in a call to the _() function, like this:
For convenience, you want the _() function to be installed in Python’s builtins namespace, so it is easily accessible in all modules of your application.
Deprecated since version 3.8, will be removed in version 3.10: The codeset parameter.
Translation classes are what actually implement the translation of original source file message strings to translated message strings. The base class used by all translation classes is NullTranslations; this provides the basic interface you can use to write your own specialized translation classes. Here are the methods of NullTranslations:
Takes an optional file object fp, which is ignored by the base class. Initializes “protected” instance variables _info and _charset which are set by derived classes, as well as _fallback, which is set through add_fallback(). It then calls self._parse(fp) if fp is not None.
No-op in the base class, this method takes file object fp, and reads the data from the file, initializing its message catalog. If you have an unsupported message catalog file format, you should override this method to parse your format.
Add fallback as the fallback object for the current translation object. A translation object should consult the fallback if it cannot provide a translation for a given message.
If a fallback has been set, forward gettext() to the fallback. Otherwise, return message. Overridden in derived classes.
If a fallback has been set, forward ngettext() to the fallback. Otherwise, return singular if n is 1; return plural otherwise. Overridden in derived classes.
If a fallback has been set, forward pgettext() to the fallback. Otherwise, return the translated message. Overridden in derived classes.
New in version 3.8.
If a fallback has been set, forward npgettext() to the fallback. Otherwise, return the translated message. Overridden in derived classes.
New in version 3.8.
Equivalent to gettext() and ngettext(), but the translation is returned as a byte string encoded in the preferred system encoding if no encoding was explicitly set with set_output_charset(). Overridden in derived classes.
Warning
These methods should be avoided in Python 3. See the warning for the lgettext() function.
Deprecated since version 3.8, will be removed in version 3.10.
Return the “protected” _info variable, a dictionary containing the metadata found in the message catalog file.
Return the encoding of the message catalog file.
Return the encoding used to return translated messages in lgettext() and lngettext().
Deprecated since version 3.8, will be removed in version 3.10.
Change the encoding used to return translated messages.
Deprecated since version 3.8, will be removed in version 3.10.
This method installs gettext() into the built-in namespace, binding it to _.
If the names parameter is given, it must be a sequence containing the names of functions you want to install in the builtins namespace in addition to _(). Supported names are 'gettext', 'ngettext', 'pgettext', 'npgettext', 'lgettext', and 'lngettext'.
Note that this is only one way, albeit the most convenient way, to make the _() function available to your application. Because it affects the entire application globally, and specifically the built-in namespace, localized modules should never install _(). Instead, they should use this code to make _() available to their module:
This puts _() only in the module’s global namespace and so only affects calls within this module.
Changed in version 3.8: Added 'pgettext' and 'npgettext'.
The gettext module provides one additional class derived from NullTranslations: GNUTranslations. This class overrides _parse() to enable reading GNU gettext format .mo files in both big-endian and little-endian format.
GNUTranslations parses optional metadata out of the translation catalog. It is convention with GNU gettext to include metadata as the translation for the empty string. This metadata is in RFC 822-style key: value pairs, and should contain the Project-Id-Version key. If the key Content-Type is found, then the charset property is used to initialize the “protected” _charset instance variable, defaulting to None if not found. If the charset encoding is specified, then all message ids and message strings read from the catalog are converted to Unicode using this encoding, else ASCII is assumed.
Since message ids are read as Unicode strings too, all *gettext() methods will assume message ids as Unicode strings, not byte strings.
The entire set of key/value pairs are placed into a dictionary and set as the “protected” _info instance variable.
If the .mo file’s magic number is invalid, the major version number is unexpected, or if other problems occur while reading the file, instantiating a GNUTranslations class can raise OSError.
The following methods are overridden from the base class implementation:
Look up the message id in the catalog and return the corresponding message string, as a Unicode string. If there is no entry in the catalog for the message id, and a fallback has been set, the look up is forwarded to the fallback’s gettext() method. Otherwise, the message id is returned.
Do a plural-forms lookup of a message id. singular is used as the message id for purposes of lookup in the catalog, while n is used to determine which plural form to use. The returned message string is a Unicode string.
If the message id is not found in the catalog, and a fallback is specified, the request is forwarded to the fallback’s ngettext() method. Otherwise, when n is 1 singular is returned, and plural is returned in all other cases.
Here is an example:
Look up the context and message id in the catalog and return the corresponding message string, as a Unicode string. If there is no entry in the catalog for the message id and context, and a fallback has been set, the look up is forwarded to the fallback’s pgettext() method. Otherwise, the message id is returned.
New in version 3.8.
Do a plural-forms lookup of a message id. singular is used as the message id for purposes of lookup in the catalog, while n is used to determine which plural form to use.
If the message id for context is not found in the catalog, and a fallback is specified, the request is forwarded to the fallback’s npgettext() method. Otherwise, when n is 1 singular is returned, and plural is returned in all other cases.
New in version 3.8.
Equivalent to gettext() and ngettext(), but the translation is returned as a byte string encoded in the preferred system encoding if no encoding was explicitly set with set_output_charset().
Warning
These methods should be avoided in Python 3. See the warning for the lgettext() function.
Deprecated since version 3.8, will be removed in version 3.10.
The Solaris operating system defines its own binary .mo file format, but since no documentation can be found on this format, it is not supported at this time.
GNOME uses a version of the gettext module by James Henstridge, but this version has a slightly different API. Its documented usage was:
For compatibility with this older module, the function Catalog() is an alias for the translation() function described above.
One difference between this module and Henstridge’s: his catalog objects supported access through a mapping API, but this appears to be unused and so is not currently supported.
Internationalization (I18N) refers to the operation by which a program is made aware of multiple languages. Localization (L10N) refers to the adaptation of your program, once internationalized, to the local language and cultural habits. In order to provide multilingual messages for your Python programs, you need to take the following steps:
In order to prepare your code for I18N, you need to look at all the strings in your files. Any string that needs to be translated should be marked by wrapping it in _('...') — that is, a call to the function _(). For example:
In this example, the string 'writing a log message' is marked as a candidate for translation, while the strings 'mylog.txt' and 'w' are not.
There are a few tools to extract the strings meant for translation. The original GNU gettext only supported C or C++ source code but its extended version xgettext scans code written in a number of languages, including Python, to find strings marked as translatable. Babel is a Python internationalization library that includes a pybabel script to extract and compile message catalogs. François Pinard’s program called xpot does a similar job and is available as part of his po-utils package.
(Python also includes pure-Python versions of these programs, called pygettext.py and msgfmt.py; some Python distributions will install them for you. pygettext.py is similar to xgettext, but only understands Python source code and cannot handle other programming languages such as C or C++. pygettext.py supports a command-line interface similar to xgettext; for details on its use, run pygettext.py
--help. msgfmt.py is binary compatible with GNU msgfmt. With these two programs, you may not need the GNU gettext package to internationalize your Python applications.)
xgettext, pygettext, and similar tools generate .po files that are message catalogs. They are structured human-readable files that contain every marked string in the source code, along with a placeholder for the translated versions of these strings.
Copies of these .po files are then handed over to the individual human translators who write translations for every supported natural language. They send back the completed language-specific versions as a <language-name>.po file that’s compiled into a machine-readable .mo binary catalog file using the msgfmt program. The .mo files are used by the gettext module for the actual translation processing at run-time.
How you use the gettext module in your code depends on whether you are internationalizing a single module or your entire application. The next two sections will discuss each case.
If you are localizing your module, you must take care not to make global changes, e.g. to the built-in namespace. You should not use the GNU gettext API but instead the class-based API.
Let’s say your module is called “spam” and the module’s various natural language translation .mo files reside in /usr/share/locale in GNU gettext format. Here’s what you would put at the top of your module:
If you are localizing your application, you can install the _() function globally into the built-in namespace, usually in the main driver file of your application. This will let all your application-specific files just use _('...') without having to explicitly install it in each file.
In the simple case then, you need only add the following bit of code to the main driver file of your application:
If you need to set the locale directory, you can pass it into the install() function:
If your program needs to support many languages at the same time, you may want to create multiple translation instances and then switch between them explicitly, like so:
In most coding situations, strings are translated where they are coded. Occasionally however, you need to mark strings for translation, but defer actual translation until later. A classic example is:
Here, you want to mark the strings in the animals list as being translatable, but you don’t actually want to translate them until they are printed.
Here is one way you can handle this situation:
This works because the dummy definition of _() simply returns the string unchanged. And this dummy definition will temporarily override any definition of _() in the built-in namespace (until the del command). Take care, though if you have a previous definition of _() in the local namespace.
Note that the second use of _() will not identify “a” as being translatable to the gettext program, because the parameter is not a string literal.
Another way to handle this is with the following example:
In this case, you are marking translatable strings with the function N_(), which won’t conflict with any definition of _(). However, you will need to teach your message extraction program to look for translatable strings marked with N_(). xgettext, pygettext, pybabel extract, and xpot all support this through the use of the -k command-line switch. The choice of N_() here is totally arbitrary; it could have just as easily been MarkThisStringForTranslation().
The following people contributed code, feedback, design suggestions, previous implementations, and valuable experience to the creation of this module:
The default locale directory is system dependent; for example, on RedHat Linux it is /usr/share/locale, but on Solaris it is /usr/lib/locale. The gettext module does not try to support these system dependent defaults; instead its default is sys.base_prefix/share/locale (see sys.base_prefix). For this reason, it is always best to call bindtextdomain() with an explicit absolute path at the start of your application.
See the footnote for bindtextdomain() above.


Bind the domain to the locale directory localedir. More concretely, gettext will look for binary .mo files for the given domain using the path (on Unix): localedir/language/LC_MESSAGES/domain.mo, where language is searched for in the environment variables LANGUAGE, LC_ALL, LC_MESSAGES, and LANG respectively.
If localedir is omitted or None, then the current binding for domain is returned. 1


Bind the domain to codeset, changing the encoding of byte strings returned by the lgettext(), ldgettext(), lngettext() and ldngettext() functions. If codeset is omitted, then the current binding is returned.
Deprecated since version 3.8, will be removed in version 3.10.


Like gettext(), but look the message up in the specified domain.


Like ngettext(), but look the message up in the specified domain.


Similar to the corresponding functions without the p in the prefix (that is, gettext(), dgettext(), ngettext(), dngettext()), but the translation is restricted to the given message context.
New in version 3.8.


This function implements the standard .mo file search algorithm. It takes a domain, identical to what textdomain() takes. Optional localedir is as in bindtextdomain(). Optional languages is a list of strings, where each string is a language code.
If localedir is not given, then the default system locale directory is used. 2 If languages is not given, then the following environment variables are searched: LANGUAGE, LC_ALL, LC_MESSAGES, and LANG. The first one returning a non-empty value is used for the languages variable. The environment variables should contain a colon separated list of languages, which will be split on the colon to produce the expected list of language code strings.
find() then expands and normalizes the languages, and then iterates through them, searching for an existing file built of these components:
localedir/language/LC_MESSAGES/domain.mo
The first such file name that exists is returned by find(). If no such file is found, then None is returned. If all is given, it returns a list of all file names, in the order in which they appear in the languages list or the environment variables.


Return the localized translation of message, based on the current global domain, language, and locale directory. This function is usually aliased as _() in the local namespace (see examples below).


The following methods are overridden from the base class implementation:
Look up the message id in the catalog and return the corresponding message string, as a Unicode string. If there is no entry in the catalog for the message id, and a fallback has been set, the look up is forwarded to the fallback’s gettext() method. Otherwise, the message id is returned.
Do a plural-forms lookup of a message id. singular is used as the message id for purposes of lookup in the catalog, while n is used to determine which plural form to use. The returned message string is a Unicode string.
If the message id is not found in the catalog, and a fallback is specified, the request is forwarded to the fallback’s ngettext() method. Otherwise, when n is 1 singular is returned, and plural is returned in all other cases.
Here is an example:
Look up the context and message id in the catalog and return the corresponding message string, as a Unicode string. If there is no entry in the catalog for the message id and context, and a fallback has been set, the look up is forwarded to the fallback’s pgettext() method. Otherwise, the message id is returned.
New in version 3.8.
Do a plural-forms lookup of a message id. singular is used as the message id for purposes of lookup in the catalog, while n is used to determine which plural form to use.
If the message id for context is not found in the catalog, and a fallback is specified, the request is forwarded to the fallback’s npgettext() method. Otherwise, when n is 1 singular is returned, and plural is returned in all other cases.
New in version 3.8.
Equivalent to gettext() and ngettext(), but the translation is returned as a byte string encoded in the preferred system encoding if no encoding was explicitly set with set_output_charset().
Warning
These methods should be avoided in Python 3. See the warning for the lgettext() function.
Deprecated since version 3.8, will be removed in version 3.10.


Look up the message id in the catalog and return the corresponding message string, as a Unicode string. If there is no entry in the catalog for the message id, and a fallback has been set, the look up is forwarded to the fallback’s gettext() method. Otherwise, the message id is returned.


Equivalent to gettext() and ngettext(), but the translation is returned as a byte string encoded in the preferred system encoding if no encoding was explicitly set with set_output_charset().
Warning
These methods should be avoided in Python 3. See the warning for the lgettext() function.
Deprecated since version 3.8, will be removed in version 3.10.


Do a plural-forms lookup of a message id. singular is used as the message id for purposes of lookup in the catalog, while n is used to determine which plural form to use. The returned message string is a Unicode string.
If the message id is not found in the catalog, and a fallback is specified, the request is forwarded to the fallback’s ngettext() method. Otherwise, when n is 1 singular is returned, and plural is returned in all other cases.
Here is an example:


Do a plural-forms lookup of a message id. singular is used as the message id for purposes of lookup in the catalog, while n is used to determine which plural form to use.
If the message id for context is not found in the catalog, and a fallback is specified, the request is forwarded to the fallback’s npgettext() method. Otherwise, when n is 1 singular is returned, and plural is returned in all other cases.
New in version 3.8.


Look up the context and message id in the catalog and return the corresponding message string, as a Unicode string. If there is no entry in the catalog for the message id and context, and a fallback has been set, the look up is forwarded to the fallback’s pgettext() method. Otherwise, the message id is returned.
New in version 3.8.


This installs the function _() in Python’s builtins namespace, based on domain, localedir, and codeset which are passed to the function translation().
For the names parameter, please see the description of the translation object’s install() method.
As seen below, you usually mark the strings in your application that are candidates for translation, by wrapping them in a call to the _() function, like this:
For convenience, you want the _() function to be installed in Python’s builtins namespace, so it is easily accessible in all modules of your application.
Deprecated since version 3.8, will be removed in version 3.10: The codeset parameter.


Equivalent to the corresponding functions without the l prefix (gettext(), dgettext(), ngettext() and dngettext()), but the translation is returned as a byte string encoded in the preferred system encoding if no other encoding was explicitly set with bind_textdomain_codeset().
Warning
These functions should be avoided in Python 3, because they return encoded bytes. It’s much better to use alternatives which return Unicode strings instead, since most Python applications will want to manipulate human readable text as strings instead of bytes. Further, it’s possible that you may get unexpected Unicode-related exceptions if there are encoding problems with the translated strings.
Deprecated since version 3.8, will be removed in version 3.10.


Like gettext(), but consider plural forms. If a translation is found, apply the plural formula to n, and return the resulting message (some languages have more than two plural forms). If no translation is found, return singular if n is 1; return plural otherwise.
The Plural formula is taken from the catalog header. It is a C or Python expression that has a free variable n; the expression evaluates to the index of the plural in the catalog. See the GNU gettext documentation for the precise syntax to be used in .po files and the formulas for a variety of languages.


Takes an optional file object fp, which is ignored by the base class. Initializes “protected” instance variables _info and _charset which are set by derived classes, as well as _fallback, which is set through add_fallback(). It then calls self._parse(fp) if fp is not None.
No-op in the base class, this method takes file object fp, and reads the data from the file, initializing its message catalog. If you have an unsupported message catalog file format, you should override this method to parse your format.
Add fallback as the fallback object for the current translation object. A translation object should consult the fallback if it cannot provide a translation for a given message.
If a fallback has been set, forward gettext() to the fallback. Otherwise, return message. Overridden in derived classes.
If a fallback has been set, forward ngettext() to the fallback. Otherwise, return singular if n is 1; return plural otherwise. Overridden in derived classes.
If a fallback has been set, forward pgettext() to the fallback. Otherwise, return the translated message. Overridden in derived classes.
New in version 3.8.
If a fallback has been set, forward npgettext() to the fallback. Otherwise, return the translated message. Overridden in derived classes.
New in version 3.8.
Equivalent to gettext() and ngettext(), but the translation is returned as a byte string encoded in the preferred system encoding if no encoding was explicitly set with set_output_charset(). Overridden in derived classes.
Warning
These methods should be avoided in Python 3. See the warning for the lgettext() function.
Deprecated since version 3.8, will be removed in version 3.10.
Return the “protected” _info variable, a dictionary containing the metadata found in the message catalog file.
Return the encoding of the message catalog file.
Return the encoding used to return translated messages in lgettext() and lngettext().
Deprecated since version 3.8, will be removed in version 3.10.
Change the encoding used to return translated messages.
Deprecated since version 3.8, will be removed in version 3.10.
This method installs gettext() into the built-in namespace, binding it to _.
If the names parameter is given, it must be a sequence containing the names of functions you want to install in the builtins namespace in addition to _(). Supported names are 'gettext', 'ngettext', 'pgettext', 'npgettext', 'lgettext', and 'lngettext'.
Note that this is only one way, albeit the most convenient way, to make the _() function available to your application. Because it affects the entire application globally, and specifically the built-in namespace, localized modules should never install _(). Instead, they should use this code to make _() available to their module:
This puts _() only in the module’s global namespace and so only affects calls within this module.
Changed in version 3.8: Added 'pgettext' and 'npgettext'.


Add fallback as the fallback object for the current translation object. A translation object should consult the fallback if it cannot provide a translation for a given message.


Return the encoding of the message catalog file.


If a fallback has been set, forward gettext() to the fallback. Otherwise, return message. Overridden in derived classes.


Return the “protected” _info variable, a dictionary containing the metadata found in the message catalog file.


This method installs gettext() into the built-in namespace, binding it to _.
If the names parameter is given, it must be a sequence containing the names of functions you want to install in the builtins namespace in addition to _(). Supported names are 'gettext', 'ngettext', 'pgettext', 'npgettext', 'lgettext', and 'lngettext'.
Note that this is only one way, albeit the most convenient way, to make the _() function available to your application. Because it affects the entire application globally, and specifically the built-in namespace, localized modules should never install _(). Instead, they should use this code to make _() available to their module:
This puts _() only in the module’s global namespace and so only affects calls within this module.
Changed in version 3.8: Added 'pgettext' and 'npgettext'.


Equivalent to gettext() and ngettext(), but the translation is returned as a byte string encoded in the preferred system encoding if no encoding was explicitly set with set_output_charset(). Overridden in derived classes.
Warning
These methods should be avoided in Python 3. See the warning for the lgettext() function.
Deprecated since version 3.8, will be removed in version 3.10.


If a fallback has been set, forward ngettext() to the fallback. Otherwise, return singular if n is 1; return plural otherwise. Overridden in derived classes.


If a fallback has been set, forward npgettext() to the fallback. Otherwise, return the translated message. Overridden in derived classes.
New in version 3.8.


Return the encoding used to return translated messages in lgettext() and lngettext().
Deprecated since version 3.8, will be removed in version 3.10.


If a fallback has been set, forward pgettext() to the fallback. Otherwise, return the translated message. Overridden in derived classes.
New in version 3.8.


Change the encoding used to return translated messages.
Deprecated since version 3.8, will be removed in version 3.10.


No-op in the base class, this method takes file object fp, and reads the data from the file, initializing its message catalog. If you have an unsupported message catalog file format, you should override this method to parse your format.


Change or query the current global domain. If domain is None, then the current global domain is returned, otherwise the global domain is set to domain, which is returned.


Return a *Translations instance based on the domain, localedir, and languages, which are first passed to find() to get a list of the associated .mo file paths. Instances with identical .mo file names are cached. The actual class instantiated is class_ if provided, otherwise GNUTranslations. The class’s constructor must take a single file object argument. If provided, codeset will change the charset used to encode translated strings in the lgettext() and lngettext() methods.
If multiple files are found, later files are used as fallbacks for earlier ones. To allow setting the fallback, copy.copy() is used to clone each translation object from the cache; the actual instance data is still shared with the cache.
If no .mo file is found, this function raises OSError if fallback is false (which is the default), and returns a NullTranslations instance if fallback is true.
Changed in version 3.3: IOError used to be raised instead of OSError.
Deprecated since version 3.8, will be removed in version 3.10: The codeset parameter.


Source code: Lib/glob.py
The glob module finds all the pathnames matching a specified pattern according to the rules used by the Unix shell, although results are returned in arbitrary order. No tilde expansion is done, but *, ?, and character ranges expressed with [] will be correctly matched. This is done by using the os.scandir() and fnmatch.fnmatch() functions in concert, and not by actually invoking a subshell. Note that unlike fnmatch.fnmatch(), glob treats filenames beginning with a dot (.) as special cases. (For tilde and shell variable expansion, use os.path.expanduser() and os.path.expandvars().)
For a literal match, wrap the meta-characters in brackets. For example, '[?]' matches the character '?'.
See also
The pathlib module offers high-level path objects.
Return a possibly-empty list of path names that match pathname, which must be a string containing a path specification. pathname can be either absolute (like /usr/src/Python-1.5/Makefile) or relative (like ../../Tools/*/*.gif), and can contain shell-style wildcards. Broken symlinks are included in the results (as in the shell). Whether or not the results are sorted depends on the file system. If a file that satisfies conditions is removed or added during the call of this function, whether a path name for that file be included is unspecified.
If recursive is true, the pattern “**” will match any files and zero or more directories, subdirectories and symbolic links to directories. If the pattern is followed by an os.sep or os.altsep then files will not match.
Raises an auditing event glob.glob with arguments pathname, recursive.
Note
Using the “**” pattern in large directory trees may consume an inordinate amount of time.
Changed in version 3.5: Support for recursive globs using “**”.
Return an iterator which yields the same values as glob() without actually storing them all simultaneously.
Raises an auditing event glob.glob with arguments pathname, recursive.
Escape all special characters ('?', '*' and '['). This is useful if you want to match an arbitrary literal string that may have special characters in it. Special characters in drive/UNC sharepoints are not escaped, e.g. on Windows escape('//?/c:/Quo vadis?.txt') returns '//?/c:/Quo vadis[?].txt'.
New in version 3.4.
For example, consider a directory containing the following files: 1.gif, 2.txt, card.gif and a subdirectory sub which contains only the file 3.txt. glob() will produce the following results. Notice how any leading components of the path are preserved.
If the directory contains files starting with . they won’t be matched by default. For example, consider a directory containing card.gif and .card.gif:
See also
Shell-style filename (not path) expansion


Escape all special characters ('?', '*' and '['). This is useful if you want to match an arbitrary literal string that may have special characters in it. Special characters in drive/UNC sharepoints are not escaped, e.g. on Windows escape('//?/c:/Quo vadis?.txt') returns '//?/c:/Quo vadis[?].txt'.
New in version 3.4.


Return a possibly-empty list of path names that match pathname, which must be a string containing a path specification. pathname can be either absolute (like /usr/src/Python-1.5/Makefile) or relative (like ../../Tools/*/*.gif), and can contain shell-style wildcards. Broken symlinks are included in the results (as in the shell). Whether or not the results are sorted depends on the file system. If a file that satisfies conditions is removed or added during the call of this function, whether a path name for that file be included is unspecified.
If recursive is true, the pattern “**” will match any files and zero or more directories, subdirectories and symbolic links to directories. If the pattern is followed by an os.sep or os.altsep then files will not match.
Raises an auditing event glob.glob with arguments pathname, recursive.
Note
Using the “**” pattern in large directory trees may consume an inordinate amount of time.
Changed in version 3.5: Support for recursive globs using “**”.


Return an iterator which yields the same values as glob() without actually storing them all simultaneously.
Raises an auditing event glob.glob with arguments pathname, recursive.


Return a dictionary representing the current global symbol table. This is always the dictionary of the current module (inside a function or method, this is the module where it is defined, not the module from which it is called).