3 Techniques to Effortlessly Import and Execute Python Modules | by Khuyen Tran | Towards Data Science

As a data scientist, you most likely want to share the helpful modules you created with your teammates or other users. Although your module might be useful, others will not use it if it takes them a lot of effort to access the useful functions in your module.
Thus, you want to make it easy for users to use your module. The code to import and run your module should be short. In this article, I will show you 3 ways to make it easy to import and execute your Python modules.
Imagine we have a file called utils.py that contains all important functions and classes
In another script, we want to import everything from utils.py except the variablea using from utils import *. How can we do that?
This could easily be done by adding__all__ = [“add_two”, “multiply_by_two”] . Functions, classes, and packages specified in __all__ will be imported when using import * .
Now try to use import * in another script
56Traceback (most recent call last):  File "main.py", line 6, in <module>    print(a)NameError: name 'a' is not defined
Cool! The error shows us that only add_two and multiply_by_two from utils.py are imported while ais not imported.
Imagine what the structure of the files in our directory looks like below:
.├── data_modules│   ├── load_data.py│   └── process_data.py└── main.py
load_data.py looks like this:
and process_data.py looks like this:
To use classes from 2 different files, we need to import each class from each file.
This method is fine, but it is redundant to use 2 import statements. Is there a way that we can turn two import statements into one import statement like below?
This can be easily solved with __init__.py file. Insert __init__.py file under thedata_modules directory:
touch data_modules/__init__.py
Then insert the import statements mentioned earlier to the__init__.py file:
We use one . here because load_data.py is in the same directory as __init__.py .
Now let’s try to import DataLoader and DataProcessor from data_modules
Loading data from data/Processing data1
Nice! It works!
Our directory looks like this:
.└── data_modules    ├── __init__.py    ├── load_data.py    ├── main.py    └── process_data.py
Instead of running
$ python data_modules/main.py
we might want to make it simpler for our users or teammates to run the code main.py file by simply running the parent directory:
$ python data_modules
This is better than runningpython data_modules/main.py because it is shorter, and users don’t need to know what files are in data_modules .
How can we do that?
This is when __main__.py comes in handy. Simply change the script you want to run when running the top-level directory to __main__.py. In our example, main.py will become __main__.py .
# Rename main.py to __main__.py$ mv data_modules/main.py data_modules/__main__.py
Our new directory will look like this
.└── data_modules    ├── __init__.py    ├── load_data.py    ├── __main__.py    └── process_data.py
Now run
$ python data_modulesLoading data from data/Processing data1
And it works like a charm!
Congratulations! You have just learned how to make it easier for you and those who use your package by:
Controlling what to import when using import *
Shortening your import statements
Running the top-level directory as the main script
I hope these tricks will make your Python package more user-friendly. Your teammates and users will be happy to use your package if it is easy for them to access your useful modules.
The source code of this article could be found here:
github.com
I like to write about basic data science concepts and play with different algorithms and data science tools. You could connect with me on LinkedIn and Twitter.
Star this repo if you want to check out the codes for all of the articles I have written. Follow me on Medium to stay informed with my latest data science articles like these:
towardsdatascience.com
towardsdatascience.com
towardsdatascience.com
towardsdatascience.com
Beazley, D. M., & Jones, B. K. (2014). Python cookbook. Beijing ; Cambridge ; Farnham ; Köln ; Sebastopol ; Tokyo: O’Reilly.