FastAPI Hive Framework, modulization of code layout
Tags: fastapi, python, modulization, decoupling
Published: 2023-10-22 17:27:21

## Introduction

FastAPI Hive Framework is a developer friendly and easy to be integrated framework for managing your code by endpoints and cornerstones folder structure.

The key features are:

- **Conerstone Container**: a top-level folder to layout codes by function folder, like db and authentication.

- **Endpoint Container**: a top-level folder to layout service codes by endpoint folder,
- **Endpoint folder**: a sub-folder in Endpoint Container, layout one service code by function folder(router, db, service, etc).
- **Router Mounting Automatically**: the router defined by module will be mounted into app automatically.
- **Model Preloading Easily**: the service(such as ML model) defined by module will be mounted into app easily, in order to reduce loading time during endpoint request.
- **Developer-Friendly**: all one-endpoint/cornerstone codes are put in one same folders, easy to review and update.
- **Easy-to-be-Integrated**: Just several line codes to integrate it in your app.


## Why

### Regular project layout & its cons.

&emsp;&emsp;Let's look at the regular project folder layout.There are serval function folders in the app, and the folders are arranged by functions, such as routes/models/services.No problem if folders are set like this for small project.

&emsp;&emsp;But when it comes to large scale project which contains too many services and functions, it will bring tough task to maintain these codes, because those services' code files are existent in different folders, hard to review, you have to jumping among different folders.

&emsp;&emsp;Beyond the problem, developer also need to register some functions in main.py, such as router registering.

&emsp;&emsp;So it is ideal for developer to maintain each related functional code files in one container_name folder, and register all service and function codes into app automatically.

For the classical code layout:

Reference: https://github.com/eightBEC/fastapi-ml-skeleton/tree/master/fastapi_skeleton

>     app
>         router
>             heartbeat.py
>             prediction.py
>         models
>             heartbeat.py
>             prediction.py
>         services
>             heartbeat.py
>             prediction.py
>         main.py

### Ideal project layout & its pros.

&emsp;&emsp;Here is the ideal folder structure layout from developers' perspective. For each of service container_name(heartbeat and prediction), there is one module folder for containing all functional code files.

&emsp;&emsp;Furthermore, developers do not have to register function into app, such as router.

>     app
>         packages
>             heartbeat
>                 router.py
>                 models.py
>                 service.py
>             prediction
>                 router.py
>                 models.py
>                 service.py
>         main.py


## How

&emsp;&emsp;FastAPI Hive Framework is the solution to the problems in "why" chapter. In this chapter, let see how to apply it in project.

### Install it.

```
pip3 install fastapi_hive
```

### Integrate it into your app

Note: You can reference example code to complete this part.
Make packages of cornerstones and endpoints

First, create or refactor you code into cornerstones and endpoints folders:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xxko9ltuvzm78fwrt8zl.png)


Code Folder Structure

```
app
    cornerstones
        db
            __init__.py
            implement.py
        auth
            __init__.py
            implement.py
    endpoint_packages
        heartbeat
            api.py
            models.py
            service.py
            __init__.py
        house_price
            api.py
            models.py
            service.py
            __init__.py
```

From code view, the setup or teardown hooks should be set in __init__.py if needed.

For cornerstone

```python

from fastapi import FastAPI
from fastapi_hive.ioc_framework.cornerstone_hooks import CornerstoneHooks, CornerstoneAsyncHooks
from example.cornerstone.auth.implement import validate_request


class CornerstoneHooksImpl(CornerstoneHooks):

    def __init__(self):
        super(CornerstoneHooksImpl, self).__init__()

    def pre_endpoint_setup(self):
        print("call pre setup from CornerstoneHooksImpl!!!")
        print("---- get fastapi app ------")
        print(self.app)

    def post_endpoint_setup(self):
        print("call post setup from CornerstoneHooksImpl!!!")

    def pre_endpoint_teardown(self):
        print("call pre teardown from CornerstoneHooksImpl!!!")

    def post_endpoint_teardown(self):
        print("call pre teardown from CornerstoneHooksImpl!!!")

    def pre_endpoint_call(self):
        pass

    def post_endpoint_call(self):
        pass


class CornerstoneAsyncHooksImpl(CornerstoneAsyncHooks):

    def __init__(self):
        super(CornerstoneAsyncHooksImpl, self).__init__()

    async def pre_endpoint_setup(self):
        print("call pre setup from CornerstoneAsyncHooksImpl!!!")

    async def post_endpoint_setup(self):
        print("call post setup from CornerstoneAsyncHooksImpl!!!")

    async def pre_endpoint_teardown(self):
        print("call pre teardown from CornerstoneAsyncHooksImpl!!!")

    async def post_endpoint_teardown(self):
        print("call pre teardown from CornerstoneAsyncHooksImpl!!!")

    async def pre_endpoint_call(self):
        pass

    async def post_endpoint_call(self):
        pass

```

For endpoint

```python
from fastapi import FastAPI
from fastapi_hive.ioc_framework.endpoint_hooks import EndpointHooks, EndpointAsyncHooks


class EndpointHooksImpl(EndpointHooks):

    def __init__(self):
        super(EndpointHooksImpl, self).__init__()

    def setup(self):
        print("call pre setup from EndpointHooksImpl!!!")
        print("---- get fastapi app ------")
        print(self.app)

    def teardown(self):
        print("call pre teardown from EndpointHooksImpl!!!")


class EndpointAsyncHooksImpl(EndpointAsyncHooks):

    def __init__(self):
        super(EndpointAsyncHooksImpl, self).__init__()

    async def setup(self):
        print("call pre setup from EndpointAsyncHooksImpl!!!")

    async def teardown(self):
        print("call pre teardown from EndpointAsyncHooksImpl!!!")
```

For the hooks running flow, please reference the belowing diagram: Note: it only depict the startup flow, it is same as shutdown flow.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w6nivemhq3474wrju3db.png)

### Setup hive framework init codes

Second, setup the initial code snippet of ioc_framework in main.py

```python
from fastapi import FastAPI
from loguru import logger
from example.cornerstone.config import (APP_NAME, APP_VERSION, API_PREFIX,
                                        IS_DEBUG)

from fastapi_hive.ioc_framework import IoCFramework


def get_app() -> FastAPI:
    logger.info("app is starting.")

    fast_app = FastAPI(title=APP_NAME, version=APP_VERSION, debug=IS_DEBUG)

    def hive_pre_setup():
        logger.info("------ call pre setup -------")

    def hive_post_setup():
        logger.info("------ call post setup -------")

    async def hive_async_pre_setup():
        logger.info("------ call async pre setup -------")

    async def hive_async_post_setup():
        logger.info("------ call async post setup -------")

    ioc_framework = IoCFramework(fast_app)
    ioc_framework.config.CORNERSTONE_PACKAGE_PATH = "./example/cornerstone/"

    ioc_framework.config.API_PREFIX = API_PREFIX
    ioc_framework.config.ENDPOINT_PACKAGE_PATHS = ["./example/endpoints_package1", "./example/endpoints_package2"]
    ioc_framework.config.ROUTER_MOUNT_AUTOMATED = True
    ioc_framework.config.HIDE_ENDPOINT_CONTAINER_IN_API = True
    ioc_framework.config.HIDE_ENDPOINT_IN_API = False
    ioc_framework.config.HIDE_ENDPOINT_IN_TAG = True
    ioc_framework.config.PRE_ENDPOINT_SETUP = hive_pre_setup
    ioc_framework.config.POST_ENDPOINT_SETUP = hive_post_setup
    ioc_framework.config.ASYNC_PRE_ENDPOINT_SETUP = hive_async_pre_setup
    ioc_framework.config.ASYNC_POST_ENDPOINT_SETUP = hive_async_post_setup

    ioc_framework.init_modules()

    @fast_app.get("/")
    def get_root():
        return "Go to docs URL to look up API: http://localhost:8000/docs"

    return fast_app


app = get_app()
```

If you want to try, please download the source repo.

Source Code: https://github.com/fanqingsong/fastapi-hive

Then follow the document's index page to run the example app

Documentation: https://fanqingsong.github.io/fastapi-hive


## Use Cases

Under the modulization layout principle, there are some use cases which is useful, including:

* preload machine learning model before request
* setup db dependency

https://fanqingsong.github.io/fastapi-hive/usecases/


## Reference

For any part of code usage, please refer to doc:

https://fanqingsong.github.io/fastapi-hive/reference/

## Design

The fastapi-hive framework confront to DIP Principle.
If you are interesting, please look into it.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xqd9u1mh0vv6249uxkwi.png)


https://fanqingsong.github.io/fastapi-hive/design/