Package: fastapi-module-loader
fastapi-module-loaderThis library allows you to load modules in a structured and well defined way. The overall idea is explained pretty
quick, we have a list of modules and then ensure all those are basically imported. Also we allow the module classes to
execute additional code on loading. This is useful to for example resolve forward references with pydantic, to register
some hooks or just load all SQLAlchemy ORM models so relationships work.The module loader idea is based on what Django does usingAppConfig's - so if you know those you might find this
pretty straight forward.Example FastAPI setupLets say we are using the following folder structure in your project,modulesmay contain a list of modules
used by your application:yourapp
|-- modules
|   `-- something
|       `-- __init__.py     <- SomethingModule is defined here, see technical details below
|-- config.py               <- Your config file
|-- loader.py               <- This is where you put the global loader instance
`-- main.py                 <- Your normal FastAPI application lives hereNow lets put the MODULES list intoconfig.py:MODULES=["yourapp.modules.something.SomethingModules",]Then add the loader to yourloader.py:fromfastapi_module_loaderimportModuleLoaderfromyourapp.configimportMODULESloader=ModuleLoader(MODULES)Note:We areNOTcallingloader.load()orloader.setup()here. This is because we want to do this in ourmain.pyfile so we have control when this actually happens.Then change yourmain.pyto look like this:fromfastapiimportFastAPIfromyourapp.loaderimportloader@asynccontextmanagerasyncdeflifespan(app:FastAPI):loader.setup()# Setup everything on FastAPI startupyieldloader.load()# Ensure everything is loadedapp=FastAPI(lifespan=lifespan)# ...put the actual FastAPI code hereUse casesThose are some use cases we encountered while working on projects. Those use cases led us to create this library:main.py: Load the modules to ensure everything is setup for the FastAPI app.In alembic migrationsenv.py: Ensure allorm.pyare loaded and the alembic migrations can "see" all the ORM
models. Otherwise, detecting any changes would not be possible at all.(seehttps://alembic.sqlalchemy.org/en/latest/tutorial.html#the-migration-environment)To register any hooks bound toasync-signalsSignal instances.(seehttps://github.com/team23/async-signals)There will be many additional use cases for using the module loader. If you have any interesting use case please
open an issue and tell us about it. 😉Technical detailsBasic structureThe classModuleLoaderis the main entry point for the module loader system. It needs to be instantiated with a
list of modules as strings. You have to initialize the module loader in some global place for your application. This
can be done directly inmain.py, although it is recommended to use a separate file for this likeloader.py.Getting you own loader instance can look like this:fromfastapi_module_loaderimportModuleLoader# We recommend putting the module list into your normal config file - if you have oneMODULES=["yourapp.modules.something.SomethingModules","yourapp.modules.something_else.SomethingElseModules","yourapp.modules.etc.EtcModules",]loader=ModuleLoader(MODULES)The actual loading process then works like this:Usingloader.load()will import all the modules passed to the constructor. It will useimportlibto load the
module and then instantiate the module class. After this is done it will call theload()method on the modules -
you can use this hook to load additional modules or similar.Theloader.setup()will execute thesetup()method of all the modules. This is where the modules should do
additional setup work like resolvingForwardRef's or registering hooks.Note thatloader.setup()will actually callpre_setup(), thensetup()and finallypost_setup()on the
modules. This allows you to have different stages of your setup process.Module structureA module is a class that inherits fromBaseModule. It may implement thesetup()and other methods as hooks. All
methods will be called on module loading/setup.Example for a module:fromfastapi_module_loaderimportBaseModuleclassSomethingModule(BaseModule):defsetup(self):# providing a setup method is optionalprint("Hello world!")Loading processloader.load()will import all the modules and then call theload()method on them. This is the place where you
should load further modules if you need to.fromyourapp.loaderimportloader# Import has to match your own setuploader.load()# will import all modules and call load() on each of themWhen all modules are loaded you may useloader.setup()to execute thesetup()method on all modules. This is the
place where you should do your actual setup work.fromyourapp.loaderimportloader# Import has to match your own setuploader.setup()# will call pre_setup(), setup() and then post_setup() on all modulesLoading further modules inload()It is a very common use case to load further modules in thesetup()method. To support this use case theBaseModuleclass provides a methodload_in_module()then will load given module names inside the current module scope.Example:fromfastapi_module_loaderimportBaseModuleclassSomethingModule(BaseModule):defsetup(self):self.load_in_module("orm")This will loadyourapp.modules.something.ormif the module file itself is placed inyourapp/modules/something/__init__.py.See this example structure:yourapp
`-- modules
    `-- something
        |-- __init__.py     <- SomethingModule is defined here
        |-- etc...py
        |-- orm.py          <- This is loaded
        `-- etc...pyNote:Forload_in_moduleto work you are required to put the modules class (SomethingModulein this example)
into the__init__.pyfile of the module. Otherwise the module loader will not be able to find the module you want
it to load.Integrating the module loaderTo ensure everything is setup properly you should just call the methods mentioned above. This may for example look
like this:fromfastapi_module_loaderimportloaderloader.load()loader.setup()# Put the actual code hereExceptionsThe module loader will raiseImproperlyConfiguredModulesif anything went wrong during handling the modules and
loading those, this includes:A module could not be importedA module does not inherit fromBaseModuleYou are callingloader.setup()whenloader.load()has not been called beforeNote however that all exceptions raised by thesetup()method of the modules will just be passed through. The module
loader will not catch now handle them. This is also the case if a modules to be loaded bySelf.load_in_module()does not exist. Generally you have to handle those errors insetup()yourself.ContributingIf you want to contribute to this project, feel free to just fork the project,
create a dev branch in your fork and then create a pull request (PR). If you
are unsure about whether your changes really suit the project please create an
issue first, to talk about this.