Silvan Melchior – Pydantic Configuration Management With ConfZ – SPS22
Channel: Swiss Python Summit
Date: 2022-09-27

"Config management in larger applications can become quite complex: Information needs to be loaded from different sources like environment variables, config files and command line arguments, the loaded data needs to be validated to ensure all expected information is present and in the correct format and then distributed to different locations in the codebase. This holds true especially in data science projects, having rich model and training configurations.

To simplify this process, we developed and open-sourced ConfZ, a config management library for Python based on pydantic. It easily allows to load config values from heterogeneous sources, validates them and makes them accessible as Python dataclass-like objects with full IDE support. It furthermore supports in common use cases like having multiple environments, lazy loading and unit testing. Within two months, we already reached more than 100 stars on GitHub.

In this talk, we show how ConfZ is used and how it compares to other config management solutions. We then dig into the pythonic details and see how meta classes drive the internals of the library."

About the speaker: Silvan is a data scientist who really likes to get his hands on the full stack from cloud data pipeline setup over ML processing to frontend web development and data display. Since 2019, he works for Zühlke, where he focuses on computer vision and medical data science applications.

Talk recorded at the Swiss Python Summit on September 22nd, 2022.

Licensed as Creative Commons Attribution 4.0 International.