Package: pyctuator
PyctuatorMonitor Python web apps usingSpring Boot Admin.Pyctuator supportsFlask,FastAPI,aiohttpandTornado.Djangosupport is planned as well.The following video shows a FastAPI web app being monitored and controled using Spring Boot Admin.The complete example can be found inAdvanced example.RequirementsPython 3.9+Pyctuator has zero hard dependencies.InstallingInstall Pyctuator using pip:pip3 install pyctuatorWhy?Many Java shops use Spring Boot as their main web framework for developing
microservices.
These organizations often use Spring Actuator together with Spring Boot Admin
to monitor their microservices' status, gain access to applications'
state and configuration, manipulate log levels, etc.While Spring Boot is suitable for many use-cases, it is very common for organizations
to also have a couple of Python microservices, as Python is often more suitable for
some types of applications. The most common examples are Data Science and Machine Learning
applications.Setting up a proper monitoring tool for these microservices is a complex task, and might
not be justified for just a few Python microservices in a sea of Java microservices.This is where Pyctuator comes in. It allows you to easily integrate your Python
microservices into your existing Spring Boot Admin deployment.Main FeaturesPyctuator is a partial Python implementation of theSpring Actuator API.It currently supports the following Actuator features:Application detailsMetricsMemory usageDisk usageCustom metricsHealth monitorsBuilt in MySQL health monitorBuilt in Redis health monitorCustom health monitorsEnvironmentLoggers- Easily change log levels during runtimeLog file- Tail the application's log fileThread dump- See which threads are runningHTTP traces- Tail recent HTTP requests, including status codes and latencyQuickstartThe examples below show a minimal integration ofFastAPI,Flaskandaiohttpapplications withPyctuator.After installing Flask/FastAPI/aiohttp and Pyctuator, start by launching a local Spring Boot Admin instance:dockerrun--rm-p8080:8080--add-host=host.docker.internal:host-gatewaymichayaak/spring-boot-admin:2.2.3-1Then go tohttp://localhost:8080to get to the web UI.FlaskThe following example is complete and should run as is.fromflaskimportFlaskfrompyctuator.pyctuatorimportPyctuatorapp_name="Flask App with Pyctuator"app=Flask(app_name)@app.route("/")defhello():return"Hello World!"Pyctuator(app,app_name,app_url="http://host.docker.internal:5000",pyctuator_endpoint_url="http://host.docker.internal:5000/pyctuator",registration_url="http://localhost:8080/instances")app.run(debug=False,port=5000)The application will automatically register with Spring Boot Admin upon start up.Log in to the Spring Boot Admin UI athttp://localhost:8080to interact with the application.FastAPIThe following example is complete and should run as is.fromfastapiimportFastAPIfromuvicornimportServerfromuvicorn.configimportConfigfrompyctuator.pyctuatorimportPyctuatorapp_name="FastAPI App with Pyctuator"app=FastAPI(title=app_name)@app.get("/")defhello():return"Hello World!"Pyctuator(app,"FastAPI Pyctuator",app_url="http://host.docker.internal:8000",pyctuator_endpoint_url="http://host.docker.internal:8000/pyctuator",registration_url="http://localhost:8080/instances")Server(config=(Config(app=app,loop="asyncio"))).run()The application will automatically register with Spring Boot Admin upon start up.Log in to the Spring Boot Admin UI athttp://localhost:8080to interact with the application.aiohttpThe following example is complete and should run as is.fromaiohttpimportwebfrompyctuator.pyctuatorimportPyctuatorapp=web.Application()routes=web.RouteTableDef()@routes.get("/")defhello():returnweb.Response(text="Hello World!")Pyctuator(app,"aiohttp Pyctuator",app_url="http://host.docker.internal:8888",pyctuator_endpoint_url="http://host.docker.internal:8888/pyctuator",registration_url="http://localhost:8080/instances")app.add_routes(routes)web.run_app(app,port=8888)The application will automatically register with Spring Boot Admin upon start up.Log in to the Spring Boot Admin UI athttp://localhost:8080to interact with the application.Registration NotesWhen registering a service in Spring Boot Admin, note that:Docker- If the Spring Boot Admin is running in a container while the managed service is running in the docker-host directly, theapp_urlandpyctuator_endpoint_urlshould usehost.docker.internalas the url's host so Spring Boot Admin will be able to connect to the monitored service.Http Traces- In order for the "Http Traces" tab to be able to hide requests sent by Spring Boot Admin to the Pyctuator endpoint,pyctuator_endpoint_urlmust be using the same host and port asapp_url.HTTPS- If Pyctuator is to be registered with Spring Boot Admin using HTTPS and the default SSL context is inappropriate, you can provide your ownssl.SSLContextusing thessl_contextoptional parameter of thePyctuatorconstructor.Insecure HTTPS- If Spring Boot Admin is using HTTPS with self-signed certificate, set thePYCTUATOR_REGISTRATION_NO_CERTenvironment variable so Pyctuator will disable certificate validation when registering (and deregistering).Advanced ConfigurationThe following sections are intended for advanced users who want to configure advanced Pyctuator features.Application InfoWhile Pyctuator only needs to know the application's name, we recommend that applications monitored by Spring
Boot Admin will show additional build and git details.
This becomes handy when scaling out a service to multiple instances by showing the version of each instance.
To do so, you can provide additional build and git info using methods of the Pyctuator object:pyctuator=Pyctuator(...)# arguments removed for brevitypyctuator.set_build_info(name="app",version="1.3.1",time=datetime.fromisoformat("2019-12-21T10:09:54.876091"),)pyctuator.set_git_info(commit="7d4fef3",time=datetime.fromisoformat("2019-12-24T14:18:32.123432"),branch="origin/master",)Once you configure build and git info, you should see them in the Details tab of Spring Boot Admin:Additional Application InfoIn addition to adding build and git info, Pyctuator allows adding arbitrary application details to the "Info" section in SBA.This is done by initializing theadditional_app_infoparameter with an arbitrary dictionary.
For example, you can provide links to your application's metrics:Pyctuator(app,"Flask Pyctuator",app_url=f"http://172.18.0.1:5000",pyctuator_endpoint_url=f"http://172.18.0.1:5000/pyctuator",registration_url=f"http://localhost:8080/instances",app_description="Demonstrate Spring Boot Admin Integration with Flask",additional_app_info=dict(serviceLinks=dict(metrics="http://xyz/service/metrics"),podLinks=dict(metrics=["http://xyz/pod/metrics/memory","http://xyz/pod/metrics/cpu"])))This will result with the following Info page in SBA:DB HealthFor services that use SQL database via SQLAlchemy, Pyctuator can easily monitor and expose the connection's health
using the DbHealthProvider class as demonstrated below:engine=create_engine("mysql+pymysql://root:root@localhost:3306")pyctuator=Pyctuator(...)# arguments removed for brevitypyctuator.register_health_provider(DbHealthProvider(engine))Once you configure the health provider, you should see DB health info in the Details tab of Spring Boot Admin:Redis healthIf your service is using Redis, Pyctuator can monitor the connection to Redis by simply initializing aRedisHealthProvider:r=redis.Redis()pyctuator=Pyctuator(...)# arguments removed for brevitypyctuator.register_health_provider(RedisHealthProvider(r))Custom EnvironmentOut of the box, Pyctuator exposes Python's environment variables to Spring Boot Admin.In addition, an application may register an environment provider to provide additional configuration that should be exposed via Spring Boot Admin.When the environment provider is called it should return a dictionary describing the environment. The returned dictionary is exposed to Spring Boot Admin.Since Spring Boot Admin doesn't support hierarchical environment (only a flat key/value mapping), the provided environment is flattened as dot-delimited keys.Pyctuator tries to hide secrets from being exposed to Spring Boot Admin by replacing the values of "suspicious" keys with ***.Suspicious keys are keys that contain the words "secret", "password" and some forms of "key".For example, if an application's configuration looks like this:config={"a":"s1","b":{"secret":"ha ha","c":625,},"d":{"e":True,"f":"hello","g":{"h":123,"i":"abcde"}}}An environment provider can be registered like so:pyctuator.register_environment_provider("config",lambda:config)Filesystem and Memory MetricsPyctuator can provide filesystem and memory metrics.To enable these metrics, installpsutilNote that thepsutildependency isoptionaland is only required if you want to enable filesystem and memory monitoring.LoggersPyctuator leverages Python's builtinloggingframework and allows controlling log levels at runtime.Note that in order to control uvicorn's log level, you need to provide a logger object when instantiating it. For example:myFastAPIServer=Server(config=Config(logger=logging.getLogger("uvi"),app=app,loop="asyncio"))Spring Boot Admin Using Basic AuthenticationPyctuator supports registration with Spring Boot Admin that requires basic authentications. The credentials are provided when initializing the Pyctuator instance as follows:# NOTE: Never include secrets in your code !!!auth=BasicAuth(os.getenv("sba-username"),os.getenv("sba-password"))Pyctuator(app,"Flask Pyctuator",app_url="http://localhost:5000",pyctuator_endpoint_url=f"http://localhost:5000/pyctuator",registration_url=f"http://spring-boot-admin:8080/instances",registration_auth=auth,)Protecting Pyctuator with authenticationSince there are numerous standard approaches to protect an API, Pyctuator doesn't explicitly support any of them. Instead, Pyctuator allows to customize its integration with the web-framework.
See the example infastapi_with_authentication_example_app.py.Full blown examplesTheexamplesfolder contains full blown Python projects that are built usingPoetry.To run these examples, you'll need to have Spring Boot Admin running in a local docker container. A Spring Boot Admin Docker image is availablehere.Unless the example includes a docker-compose file, you'll need to start Spring Boot Admin using docker directly:dockerrun--rm-p8080:8080--add-host=host.docker.internal:host-gatewaymichayaak/spring-boot-admin:2.2.3-1(the docker image's tag represents the version of Spring Boot Admin, so if you need to use version2.0.0, usemichayaak/spring-boot-admin:2.0.0instead, note it accepts connections on port 8082).The examples includeFastAPI Example- demonstrates integrating Pyctuator with the FastAPI web framework.Flask Example- demonstrates integrating Pyctuator with the Flask web framework.Advanced Example- demonstrates configuring and using all the advanced features of Pyctuator.ContributingTo set up a development environment, make sure you have Python 3.9 or newer installed, and runmake bootstrap.Usemake checkto run static analysis tools.Usemake testto run tests.