# Introspection in Python

## Summary


**Introspection** is about getting information about an object at runtime. This information could be about the object's type, its data attributes and methods, its class inheritance hierarchy, documentation, method signature, and more. Introspection can also be about knowing the current runtime context such as stack information. 

Introspection is an additional language feature in the programmer's toolbox. It can help the programmer code for specific scenarios that are perhaps harder to solve otherwise. Tracing and debugging tools benefit from introspection. In Python, data attributes and methods can be added or removed from instances even after instantiation. Given this dynamic nature of objects, introspection becomes a useful feature.

## Discussion

### What built-in functions facilitate object introspection in Python?

The following built-in functions are useful:  

    + `callable()`: Returns True if object is callable. Eg. `callable(mycar)`. Objects can be made callable by defining `&lowbar;&lowbar;call&lowbar;&lowbar;` method in their class.
    + `dir()`: Obtain all attributes (data and methods) of an object. Eg. `dir(mycar)`.
    + `hasattr()`: Check if object has an attribute. Eg. `hasattr(mycar, 'num_gears')`. This is useful since attributes can be dynamically added/removed at runtime. Note that `if 'num_gears' in dir(mycar)` is equivalent to `hasattr(mycar, 'num_gears')`.
    + `help()`: Obtain help on an object's interface based on docstrings.
    + `id()`: Obtain the unique ID of the object. Eg. `id(mycar)`.
    + `issubclass()`: Check if a class is a subclass of a specified class. Eg. `issubclass(Honda, Car)`.
    + `isinstance()`: Check if an object is of a specified class. Eg. `isinstance(mycar, Honda)` will be True when `mycar = Honda()`. If Honda is a subclass of Car, `isinstance(mycar, Car)` is also True. Note that `type(mycar) is Car` is False.
    + `sys.getsizeof()`: Part of the `sys` module, this obtains the size of the object in bytes.
    + `type()`: Obtain the object's type. Eg. `type(2)` will give `<class 'int'>`.
    + `vars()`: Obtain all instance variables (name and value) as a dictionary. Eg. `vars(mycar)`. This is equivalent to `mycar.&lowbar;&lowbar;dict&lowbar;&lowbar;`.

### What object attributes help with introspection in Python?

There are some attributes that give useful information about objects. For example, given an instance `mycar`, `mycar.&lowbar;&lowbar;class&lowbar;&lowbar;` retrieves its class. Attributes of the class give more information: `&lowbar;&lowbar;name&lowbar;&lowbar;`, `&lowbar;&lowbar;qualname&lowbar;&lowbar;`, `&lowbar;&lowbar;str&lowbar;&lowbar;`, `&lowbar;&lowbar;repr&lowbar;&lowbar;`, `&lowbar;&lowbar;doc&lowbar;&lowbar;`, and `&lowbar;&lowbar;self&lowbar;&lowbar;`. We briefly describe these: 

    + `&lowbar;&lowbar;name&lowbar;&lowbar;`: Original name of the class, function or method.
    + `&lowbar;&lowbar;qualname&lowbar;&lowbar;`: Qualified name of the class function or method. This is often useful where class/function/method definitions are nested.
    + `&lowbar;&lowbar;doc&lowbar;&lowbar;`: Documentation string, which can also be retrieved by calling built-in function `help()`.
    + `&lowbar;&lowbar;self&lowbar;&lowbar;`: Instance to which a method is bound.

### Could you describe Python's inspect module?

Python module `inspect` provides many functions to enable introspection. The module documentation notes that it provides four main kinds of services: "type checking, getting source code, inspecting classes and functions, and examining the interpreter stack." 

Inspect documentation has full details. We note some functions: 

    + **Checking types**: `ismodule()`, `isclass()`, `ismethod()`, `isfunction()`, `isgenerator()`, `isabstract()`, `isbuiltin()`, `isawaitable()`, and many more.
    + **Inspecting objects**: `getmembers()` is quite useful. For example, `inspect.getmembers(mycar, inspect.ismethod)` will return only methods of the instance `mycar`. Function `getfullargspec()` returns names and default values of a function. These can be nicely formatted using `formatargspec()`. Callables can be inspected using `signature()`. Class definitions relevant here are `Signature`, `Parameter` and `BoundArguments`.
    + **Retrieving source code**: `getdoc()`, `getcomments()`, `getfile()`, `getmodule()`, `getsourcefile()`, `getsourcelines()`, `getsource()`, and `cleandoc()`.
    + **Examining the stack**: `getframeinfo()`, `getouterframes()`, `getinnerframes()`, `currentframe()`, `stack()` and `trace()`.

### How can I obtain the current runtime context in Python?

The Python interpreter maintains a **stack of frames**. The bottom-most frame is called the global frame or module frame. When a function is called, a new frame is created and pushed to the stack. When that function returns, the frame is popped off the stack. 

Python's `inspect` module provides many functions to inspect the interpreter stack. Function `currentframe()` returns a frame object representing the current frame. More information about the frame object can be obtained by calling `getframeinfo()`, which returns a named tuple `Traceback(filename, lineno, function, code_context, index)`. 

To obtain the entire stack, call `stack()` instead. To inspect the call graph, we can use `getouterframes()`. When an exception is raised and later handled in an outer frame, we can use `trace()` or `getinnerframes()`. These can be useful for logging and debugging. 

Apart from inspecting the stack, two built-in functions help us inspect variable scope. These are `globals()` and `locals()`. At the module level, both functions return the same dictionary. 


### How can I inspect class hierarchy in Python?

Class hierarchy can easily be inspected by accessing the class attribute `&lowbar;&lowbar;bases&lowbar;&lowbar;` and recursively printing the class names up the hierarchy. If an instance is being inspected, we can first get to its class via the `&lowbar;&lowbar;class&lowbar;&lowbar;` attribute and then process the `&lowbar;&lowbar;bases&lowbar;&lowbar;` attribute. In the figure, we show an example of how this can be done. 

The method `inspect.getclasstree()` from `inspect` module is an alternative. Given a list of classes, it returns the class hierarchy. For every class, its base classes are returned as a tuple. 

Another useful method is `inspect.getmro()` to which a class name must be passed as an argument. This is equivalent to calling the built-in class method `mro()`. Either way, these don't return the inheritance hierarchy but they do tell us the order in which methods are looked up the hierarchy. 


### What are some specialized modules for Python introspection?

Python processes source code in three steps: (i) parsing, where it tokenizes the source code and produces an Abstract Syntax Tree (AST); (ii) compiling, where it transforms the AST into Python bytecode; (iii) interpreting, where the Python Virtual Machine (VM) interprets and executes the bytecode. In this context, there are two Python modules that help with introspection: `ast` to inspect the AST, and `dis` to inspect the bytecode. 

While `inspect` module has functions to inspect the call stack, `sys` module also has functions that can help with debugging recursive calls: `_getframe()`, `getrecursionlimit()` and `setrecursionlimit()`. 

Support we decorate the function `foo()`. Accessing `foo.&lowbar;&lowbar;name&lowbar;&lowbar;` will return the name of the function returned by the decorator. This is probably not what we want. We can solve this with the `wraps()` function from the `functools` module. The figure shows an example how to use this. An alternative is to use `inspect.signature()` function. 


### Could you share some tips on using Python's introspection features?

While `type()` and `isinstance()` could be useful, code that rely on them are hard to maintain. We might be checking for specific types but when another type must be handled, we need to update the code. Instead, Python programmers prefer the approach of *duck typing*. 

With duck typing, we don't care about an object's type. Instead, we focus on the interfaces the object exposes. In this approach, `hasattr()` can be useful though this method can cause problems in Python 2 when called on a property attribute. 

An alternative to `hasattr()` is to simply call a method or access a data attribute without doing any checks. If it doesn't exist, we should handle the exception. Another alternative is to use an Abstract Base Class (ABC) from the `collections.abc` module. 

## Milestones

Jan  
1994

Python 1.0 is released. 

Apr  
2001

Python 2.1 is released. This release introduces the `inspect` module. 

Sep  
2012

Python 3.3 is released. Via PEP 3155, this release adds **qualified name** attribute `&lowbar;&lowbar;qualname&lowbar;&lowbar;` to classes and functions. This enhances introspection for nested classes and functions. Via PEP 362, function `inspect.signature()` is introduced for **inspecting callables** including decorated functions, classes and partial function objects. In January 2013, PEP 362 is backported by community developers to older versions of Python, in a package named `funcsigs`. 

Sep  
2015

Python 3.5 is released. Via PEP 492, Python gets better support for **asynchronous programming**. As part of this enhancement, `inspect` module includes many functions to inspect coroutine functions and coroutine objects. The module's functions `stack()`, `trace()`, `getouterframes()`, and `getinnerframes()` now return named tuples. `Signature` and `Parameter` objects can be pickled and hashed.