Package: smart-imports
ChangelogAutomatically discovers & imports entities, used in the current module.No magic or monkey patching. Only standard Python functionality.BeforeAfterimportmathfrommy_projectimportcalc# 100500 other importsdefmy_code(argument,function=calc):returnmath.log(function(argument))importsmart_importssmart_imports.all()# no any other importsdefmy_code(argument,function=calc):returnmath.log(function(argument))MyPysupported.SummaryGet source code of the module, from whichsmart_imports.all()has called.Parse it, find all not initialized variables.Search imports, suitable for found variables.Import them.Library process only modules, from whichsmart_importscalled explicitly.Main ideaWith time every complex project develops own naming convention. If we translate that convention into more formal rules, we will be able to make automatic imports of every entity, knowing only its name.For example, we will not need to writeimport mathto callmath.pi, since our system will understand thatmathis the module of the standard library.How it worksCode from the header works in such way:smart_imports.all()buildsASTof the module from which it has called.Library analyses AST and searches for not initialized variables.Name of every found variable processed thought chain of rules to determine the correct module (or its attribute) to import. If the rule finds the target module, chain breaks and the next rules will not be processed.Library load found modules and add imported entities into the global namespace.Smart Importssearches not initialized variables in every part of code (including new Python syntax).Automatic importing turns on only for modules, that do explicit call ofsmart_imports.all().Moreover, you can use normal imports withSmart Importsat the same time. That helps to integrateSmart Importsstep by step.You can notice, that AST of module builts two times:when CPython imports module;whenSmart Importsprocess call ofsmart_imports.all().We can build AST once (for that we can add hook into the process of importing modules with help ofPEP-0302), but it will make import event slower. I think that it is because at import time CPython builds AST in terms of its internal structures (probably implemented in C). Conversion from them to Python AST cost more than building new AST from scratch.Smart Importsbuild AST only once for every module.Default import rulesSmart Importscan be used without configuration. By default it uses such rules:By exact match looks for the module with the required name in the folder of the current module.Checks if the standard library has a module with the required name.By exact match with top-level packages (for example,math).For sub-packages and modules checks complex names with dots replaced by underscores (for example,os.pathwill be imported for nameos_path).By exact match looks for installed packages with the required name (for example,requests).PerformanceSmart Importsdoes not slow down runtime but increases startup time.Because of building AST, startup time increased in 1.5-2 times. For small projects it is inconsequential. At the same time, the startup time of large projects depends mostly on architecture and dependencies between modules, than from the time of modules import.In the future, part ofSmart Importscan be rewritten in C — it should eliminate startup delays.To speed up startup time, results of AST processing can be cached on the file system. That behavior can be turned on in the config.SmartImportsinvalidates cache when module source code changes.Also,Smart Imports’ work time highly depends on rules and their sequence. You can reduce these costs by modifying configs. For example, you can specify an explicit import path for a name withRule 4: custom names.ConfigurationThe logic of default configuration was already described. It should be enough to work with the standard library.Default config:{"cache_dir":null,"rules":[{"type":"rule_local_modules"},{"type":"rule_stdlib"},{"type":"rule_predefined_names"},{"type":"rule_global_modules"}]}If necessary, a more complex config can be put on a file system.Example of complex config(from my pet project).At the time of callsmart_import.all()library detects a location of config file by searching filesmart_imports.jsonfrom the current folder up to root. If a file will be found, it will become config for the current module.You can use multiple config files (place them in different folders).There are few config parameters now:{// folder to store cached AST// if not specified or null, cache will not be used"cache_dir":null|"string",// list of import rules (see further)"rules":[]}Import rulesA sequence of rules in configs determines the order of their application. The first success rule stops processing and makes import.Rule 1: predefined nameswill be often used in the examples below. It required for the correct processing of default names likeprint.Rule 1: predefined namesRule silences import search for predefined names like__file__and builtins likeprint.# config:# {#    "rules": [{"type": "rule_predefined_names"}]# }importsmart_importssmart_imports.all()# Smart Imports will not search for module with name __file__# event if variable is not initialized explicity in codeprint(__file__)Rule 2: local modulesRule checks if a module with the required name exists in the folder of the current module. If the module found, it will be imported.# config:# {#    "rules": [{"type": "rule_predefined_names"},#              {"type": "rule_local_modules"}]# }## project on file sytem:## my_package# |-- __init__.py# |-- a.py# |-- b.py# b.pyimportsmart_importssmart_imports.all()# module "a" will be found and importedprint(a)Rule 3: global modulesRule tries to import the module by name.# config:# {#    "rules": [{"type": "rule_predefined_names"},#              {"type": "rule_global_modules"}]# }## install external package## pip install requestsimportsmart_importssmart_imports.all()# module "requests" will be found and importedprint(requests.get('http://example.com'))Rule 4: custom namesRule links a name to the specified module and its attribute (optionally).# config:# {#    "rules": [{"type": "rule_predefined_names"},#              {"type": "rule_custom",#               "variables": {"my_import_module": {"module": "os.path"},#                             "my_import_attribute": {"module": "random", "attribute": "seed"}}}]# }importsmart_importssmart_imports.all()# we use modules of the standard library in that example# but any module can be usedprint(my_import_module)print(my_import_attribute)Rule 5: standard libraryRule checks if the standard library has a module with the required name. For examplemathoros.path(which will be imported for the nameos_path).That rule works faster thanRule 3: global modules, since it searches module by predefined list. Lists of modules for every Python version was collected with help ofstdlib-list.# config:# {#    "rules": [{"type": "rule_predefined_names"},#              {"type": "rule_stdlib"}]# }importsmart_importssmart_imports.all()print(math.pi)Rule 6: import by prefixRule imports module by name from the package, which associated with name prefix. It can be helpful when you have a package used in the whole project. For example, you can access modules from packageutilswith prefixutils_.# config:# {#    "rules": [{"type": "rule_predefined_names"},#              {"type": "rule_prefix",#               "prefixes": [{"prefix": "utils_", "module": "my_package.utils"}]}]# }## project on filesystem## my_package# |-- __init__.py# |-- utils# |-- |-- __init__.py# |-- |-- a.py# |-- |-- b.py# |-- subpackage# |-- |-- __init__.py# |-- |-- c.py# c.pyimportsmart_importssmart_imports.all()print(utils_a)print(utils_b)Rule 7: modules from parent packageIf you have sub-packages with the same name in different parts of your project (for example,testsormigrations), you can allow for them to search modules by name in parent packages.# config:# {#    "rules": [{"type": "rule_predefined_names"},#              {"type": "rule_local_modules_from_parent",#               "suffixes": [".tests"]}]# }## project on file system:## my_package# |-- __init__.py# |-- a.py# |-- tests# |-- |-- __init__.py# |-- |-- b.py# b.pyimportsmart_importssmart_imports.all()print(a)Rule 8: modules from namespaceThe rule allows for modules from a specified package to import by name modules from another package.# config:# {#    "rules": [{"type": "rule_predefined_names"},#              {"type": "rule_local_modules_from_namespace",#               "map": {"my_package.subpackage_1": ["my_package.subpackage_2"]}}]# }## project on filesystem:## my_package# |-- __init__.py# |-- subpackage_1# |-- |-- __init__.py# |-- |-- a.py# |-- subpackage_2# |-- |-- __init__.py# |-- |-- b.py# a.pyimportsmart_importssmart_imports.all()print(b)How to add custom rule?Subclasssmart_imports.rules.BaseRule.Implement required logic.Register rule with methodsmart_imports.rules.register.Add rule to config.???Profit.Look into the implementation of current rules, if you need an example.Similar projectsThere are a couple of projects with a similar approach:autoimport— automatically fixes wrong import statements.If you think that some projects should be listed here, feel free to make a pull request.MyPYPlugin for integration with MyPy implemented.MyPy config (mypy.ini) example:[mypy]plugins=smart_imports.plugins.mypyPlansI love the idea of determining code properties by used names. So, I will try to develop it in the borders ofSmart Importsand other projects.What I planning forSmart Imports:Continue support and patch it for new versions of Python.Research usage of type annotations to import automatization.Try to implement lazy imports.Implement utilities for automatic config generation and code refactoring.Rewrite part of code in C, to speedup AST construction.Implement integrations with popular IDEs.I open to your suggestions. Feel free to contact me in any way.