semvec/pypi-packages huggingface dataset:
emp-config 0.1.1
Custom configuration loader with integrated Fallback to local retrievement of environment variables in case Spring Config Server isn't available
License: MIT

# EMP - Config

This repository provides a configuration loader utility tailored for the **EMP project**. It simplifies the management and retrieval of configuration values across services by integrating with a secured Spring Cloud Config Server and supports fallback to OS environment variables when needed. The package is available via PyPi and can be included in any Python-based EMP service.

## Configuration Loader Module

The `load_cnf` function enables EMP services to load configurations dynamically, based on the specified environment, for seamless transitions between development, staging, and production setups. This approach ensures consistent, secure configuration access across all EMP services.

### Key Features

- **Primary Source: Spring Cloud Config Server**: The loader first attempts to connect to a Spring Cloud Config Server at a configurable address and port. By default, it connects to `localhost` on port `8888`, but you can specify a different host via the `BASE_URL` environment variable.
- **Secured Access**: Since the Config Server is protected by Spring Security, it requires a username and password, provided through the `EMP_CONFIG_USERNAME` and `EMP_ENCRYPT_KEY` environment variables.
- **Fallback Mechanism**: If the Config Server is unavailable, the loader retrieves values from OS environment variables, ensuring uninterrupted access to necessary configurations.

### Installation

Install the package via pip:

```bash
pip3 install emp-config
```

### Usage

1. Import the `load_cnf` function

    ```python
    from emp_config import load_cnf
    ```

2. Retrieve configurations for your service
    
    ```python
    env_vars: Dict[str, str] = load_cnf(
        service_name = 'location-service',
        required_keys = ['AWS_KEY_IAM', 'AWS_IAM_SECRET'])
    ```

3.	The env_vars dictionary now contains the required configuration keys.

__Additional Notes__

- Environment Configuration: Set the environment variable BASE_URL to customize the host for the Config Server. The default port is 8888, but this can be changed in the server setup if needed.
- Fallback to OS Variables: If the Config Server cannot be reached, the module attempts to load configurations from the OS environment, ensuring critical variables remain accessible.
- Security Requirements: Ensure EMP_CONFIG_USERNAME and EMP_ENCRYPT_KEY are set in your environment for authenticated access to the Config Server.

*This configuration loader module is custom-built for the EMP project, facilitating secure, centralized configuration management.*
emp-logger 0.1.1
Custom logger with dynamic customizability for all emp-microservices that require a python logger module
License: MIT

# EMP - Logger

This repository is focusing on shared functions, code(-snippets) and generic functionalities. The goal is to reduce redundancy and to increase productivity in the usage of shared modules / functions. To maximize the advantage of this repository, the selected type of sharing is python packaging for python modules. Although github submodules and docker do offer a professional solution - It is specifically for this usecase not applicable.

## Logging Module

The setup_logger function in the Logging Module simplifies consistent logging across services, creating structured, environment-aware log files. The JSON-formatted output is ideal for integration with monitoring tools like AWS CloudWatch, Splunk, and New Relic.

### Key Features

- Environment-based Logging: Generates separate log files based on the specified environment (dev, int, stg, prd), automatically creating a logs directory.
- Formatted JSON Logs: Logs are formatted in JSON, making them easily readable and compatible with modern monitoring tools.
- Dynamic Log Levels: Automatically switches to overwrite mode in non-production environments and append mode in production, facilitating easy log management.

### Usage

1. Installing the package

    `pip3 install emp-logger`

2. Setting up the logger

    ```python
    from emp_logger import setup_logger()

    # Example usage
    logger = setup_logger(ENVIRONMENT = 'dev', SERVICE = 'Customer', module = 'Serverless')
    logger.info('Logger setup successfully for service customer')
    ```

### Additional Notes

- __Environment Detection__: If ENVIRONMENT is not specified, the logger defaults to dev.
- __Custom File Naming__: Log files are generated with a name format of EMP-<SERVICE>-<module>_<ENVIRONMENT>.log, aiding easy identification.
- __Directory Creation__: The module will automatically create a logs folder if it does not already exist.

*This logger module is built to seamlessly integrate into various environments, supporting straightforward and organized logging across applications.*
observability-toolkit 0.0.1
A python library for adding observability to an application.
License: MIT
Requires Python: >=3.8

# observability-toolkit

[![PyPI - Version](https://img.shields.io/pypi/v/observability-toolkit.svg)](https://pypi.org/project/observability-toolkit)
[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/observability-toolkit.svg)](https://pypi.org/project/observability-toolkit)
[![Hatch project](https://img.shields.io/badge/%F0%9F%A5%9A-Hatch-4051b5.svg)](https://github.com/pypa/hatch)
-----

Observability patterns for python applications.


*No dependencies outside of standard python libraries*


## Table of Contents

- [Installation](#installation)
- [Examples](#examples)
- [License](#license)

## Installation

```console
pip install observability-toolkit
```


## License

`observability-toolkit` is distributed under the terms of the
[MIT](https://spdx.org/licenses/MIT.html) license.


## Future Scope

The following is an excerpt from a conversation with an LLM around what the
features of this library should be:

#### Core Observability Components
1. Metrics collection and reporting
    - Ability to collect and expose application metrics
    - Integration with metrics systems like Prometheus
2. Logging capabilities
    - Standardized logging implementation
    - Support for different log levels (e.g. debug, info, warning, error)
    - Ability to configure log output destinations
3. Tracing and distributed tracing
    - Support for collecting and analyzing trace data
    - Integration with OpenTelemetry or other tracing standards
#### Data Collection and Instrumentation
1. Automatic instrumentation
    - Ability to automatically instrument common Python frameworks and libraries
2.  Manual instrumentation APIs
    - Easy-to-use APIs for manually adding observability to specific parts of the application
3. Context propagation
    - Ability to propagate context across different components and services
#### Configuration and Customization
1. Flexible configuration options
    - Ability to customize various aspects of observability collection and reporting
2. Pluggable architecture
    - Support for custom plugins/extensions to integrate with different systems
3. Sampling and filtering
    - Options for sampling and filtering telemetry data to manage volume and performance impact
#### Developer Experience
1. Easy integration
    - Simple setup process for integrating the toolkit into existing applications
2. Documentation and examples
    - Clear guides on usage and best practices
    - Sample implementations for reference