Dynamic importing stuff in Python
Tags: python
Published: 2021-06-17 08:28:10

---
title: Dynamic importing stuff in Python
published: true
date: 2021-06-09 15:09:23 UTC
tags: Python
canonical_url: https://hackberry.xyz/dynamic-importing-stuff-in-python
---

In my artificial intellingence project's prototype [Okai](https://github.com/0xcrypto/ok) (which is completely written in python), I did some dynamic importing which surprisingly was unknown to many. I think you might be interested in knowing about it as well.

Before I explain dynamic importing, lemme explain traditional import too for the sake of readers of all levels. To follow along, you must be atleast familiar with basics of Python. If you don't understand anything, read anyway... you will learn something new. Let's start with the most basic example and increment on it. If you feel bored, skip to next section.

## import module

```python
import os

```

Very familiar import. Here, we are simply importing `os` module. `os` is now available in the scope as a "module" object (Python treats literally everything as an object) identified as `os`.

## import module as something

```python
import random as r

```

Again, we are simply importing a module. But now, the module will be available as given identifier (here it is `r`) in the scope.

## import module's module

```python
import urllib.request

```

_Note that you can't import methods and properties of a class without importing class._

`.` is the scope resolution operator in python. It simply allows you to go a level deeper in the namespace and find the next identifier ie. `request` in above example. but `urllib` whole is imported and request is not defined! Why is that?

Something seems to be wrong here. But it is how it should be. To keep things interesting, let's continue our journey and pin it for later.

## import module's module as something

```python
import urllib.request as request

```

We have fixed the buggy import we did before. Now the `urllib` is not available in the scope, request is the scope and refers to `urllib.request`.

## from module import module

```python
from urllib import request

```

It works exactly like the above fix but more self explanatory and simpler. Remember, we still have a question pinned. We need to find the answer.

## from module import all

```python
from urllib import *

```

This one will import modules that have their identifiers listed in ` **all** ` list or simply, exist within the namespace of urllib's ` **init** `. Since we used `from` keyword, `urllib` does not exist in the scope. But all the modules from within `urllib` 's namespace now exist with their default identifiers! So many identifiers within same scope can be dangerous. Consider the following example:

```python
walk = 'Some string'

from os import *

```

I just committed a crime! `walk` has been overridden by `os.walk` ! To avoid such incidents, avoid importing `*`. As a thumb rule, do imports at the beginning of your code.

## from module's module import module

```python
from urllib.request import Request

```

It works as expected. Nothing new. You can keep going deeper in the modules as long as you want. Anything that comes inbetween `from` and `import` does not exist in global scope.

## import self's module

Let's say that you wanted a module named as `sys` in your package, and you made one. But now, how do you import it when there is already a standard module with similar name? You use relative importing.

```python
from . import sys

```

This will import `sys` from within current package.

```python
from .sys import mod

```

This will import `mod` from within `sys` module that resides in current package.

```python
from .. import sys

```

This imports `sys` from the parent package. But we still have that question pinned. Let's find the answer to that question. We need to find how import works behind the scene first.

## How import works

We can always look in the documentation and find how import works. If not willing to do so, you can always dump global identifiers available with `globals()`. Other than `globals()`, ` __builtins__ ` list usually contains most of the built in classes and functions (as strings). If you `dir( __builtins__ )`, you will find ` __import__ `. This is invoked by import keyword. if you look in the help,

```python
help( __import__ )

```

it explains what ` __import__ ` accepts as argument:

```
Help on built-in function __import__ in module builtins:

__import__ (...)
    __import__ (name, globals=None, locals=None, fromlist=(), level=0) -> module

    Import a module. Because this function is meant for use by the Python
    interpreter and not for general use it is better to use
    importlib.import_module() to programmatically import a module.

    The globals argument is only used to determine the context;
    they are not modified. The locals argument is unused. The fromlist
    should be a list of names to emulate ``from name import ...'', or an
    empty list to emulate ``import name''.
    When importing a module from a package, note that __import__ ('A.B', ...)
    returns package A when fromlist is empty, but its submodule B when
    fromlist is not empty. Level is used to determine whether to perform 
    absolute or relative imports. 0 is absolute while a positive number
    is the number of parent directories to search relative to the current module.

```

It also answered our question! When importing modules with scope resolution and `fromList` is empty, it returns the topmost object.

Note that is also indicates that you can use `importlib.import_module()` to programmatically import modules. Or should I say dynamically. As ` **import** ` provides functionality to import the modules, you can utilize it for dynamic importing. But that would be a bad practice as the documentation depreciates its direct use. So let's see what `importlib` can do for us.

## Hey import library, import module named something...

Let's checkout how `importlib.import_module()` works:

```python
import importlib
help(importlib.import_module)

```

As you can see, in the following, `import_module` accepts name of the module and package.

```python
Help on function import_module in module importlib:

import_module(name, package=None)
    Import a module.

    The 'package' argument is required when performing a relative import. It
    specifies the package to use as the anchor point from which to resolve the
    relative import to an absolute import.

```

If you `dir()` on `importlib`, you will see that it also contains default ` **import** ` as well. Obviously, it is just a wrapper over ` **import** ` .

But it allows us to import modules by their name. Following is the code snippet of okai. Highlighted line is responsible for dynamic importing.

```python
import importlib
from mind.thought import thought

class action(thought):
  def do(action, act_on):
    if(type(act_on) != thought):
      act_on = thought(act_on)

    todo = action.split('.')[-1]

    if( not callable(action) ):
      try:
        action = importlib.import_module('actions.'+ action)

      except ImportError:
        print("\n[ok]: I don't know how to - " + action + '.\n')
        # return thought('raise', 'EXIT_FAILURE')
        return thought('')

      try:
        toAct = getattr(action, todo)()
        return toAct.do(act_on)

      except Exception as e:
        print("[ok]: I can - " + action. __name__ + " but cannot - " + todo)
        print("[...] Because " + e.args[0])
        print()
        print(e)

    else:
      return action(act_on)

    return thought('')
# return thought('raise', 'EXIT_FAILURE')

```

That's that. You now know how import works and how to do dynamic importing. I hope you learned something new.

This article first appeared on my old blog, then dev.to. Since both are now dead, reposting here.