semvec/pypi-packages huggingface dataset:
aioworkers-kafka 0.2.0
aioworkers plugin for kafka
Requires Python: >=3.9
Keywords: aioworkers

aioworkers-kafka
================

.. image:: https://img.shields.io/pypi/v/aioworkers-kafka.svg
  :target: https://pypi.org/project/aioworkers-kafka

.. image:: https://github.com/aioworkers/aioworkers-kafka/workflows/Tests/badge.svg
  :target: https://github.com/aioworkers/aioworkers-kafka/actions?query=workflow%3ATests

.. image:: https://codecov.io/gh/aioworkers/aioworkers-kafka/branch/master/graph/badge.svg
  :target: https://codecov.io/gh/aioworkers/aioworkers-kafka
  :alt: Coverage

.. image:: https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/charliermarsh/ruff/main/assets/badge/v0.json
  :target: https://github.com/charliermarsh/ruff
  :alt: Code style: ruff

.. image:: https://img.shields.io/badge/types-Mypy-blue.svg
  :target: https://github.com/python/mypy
  :alt: Code style: Mypy

.. image:: https://readthedocs.org/projects/aioworkers-kafka/badge/?version=latest
  :target: https://github.com/aioworkers/aioworkers-kafka#readme
  :alt: Documentation Status

.. image:: https://img.shields.io/pypi/pyversions/aioworkers-kafka.svg
  :target: https://pypi.org/project/aioworkers-kafka
  :alt: Python versions

.. image:: https://img.shields.io/pypi/dm/aioworkers-kafka.svg
  :target: https://pypistats.org/packages/aioworkers-kafka

.. image:: https://img.shields.io/badge/%F0%9F%A5%9A-Hatch-4051b5.svg
  :alt: Hatch project
  :target: https://github.com/pypa/hatch


Usage
-----

config.yaml:

.. code-block:: yaml

    producer:
      cls: aioworkers_kafka.producer.KafkaProducer
      format: json
      kafka:
        bootstrap.servers: kafka:9092
      topic: test

    consumer:
      cls: aioworkers_kafka.consumer.KafkaConsumer
      format: json  # default format is json
      kafka:
        bootstrap.servers: kafka:9092
        group.id: test
      topics:
        - test

    worker:
      cls: mymodule.MyWorker
      input: .consumer
      output: .producer
      autorun: true


mymodule.py:

.. code-block:: python

    from aioworkers.worker.base import Worker

    class MyWorker(Worker):
        async def run(self, value):  # consume value from input
            assert isinstance(value, dict)

            value["test"] += 1

            return value  # produce value to output


.. code-block:: shell

    $ aioworkers -c config.yaml


Development
-----------

Check code:

.. code-block:: shell

    hatch run lint:all


Format code:

.. code-block:: shell

    hatch run lint:fmt


Run tests:

.. code-block:: shell

    hatch run pytest


Run tests with coverage:

.. code-block:: shell

    hatch run cov
aidbox-logging 0.1.0b1
Aidbox $loggy python logging
Requires Python: >=3.11
Keywords: aidbox, logging

[![build status](https://github.com/beda-software/aidbox-logging-py/actions/workflows/main.yml/badge.svg)](https://github.com/beda-software/aidbox-logging-py/actions/workflows/main.yml)
[![codecov](https://codecov.io/gh/beda-software/aidbox-logging-py/branch/master/graph/badge.svg)](https://codecov.io/gh/beda-software/aidbox-logging-py)
[![pypi](https://img.shields.io/pypi/v/aidbox-logging.svg)](https://pypi.org/project/aidbox-logging)
[![Checked with mypy](https://www.mypy-lang.org/static/mypy_badge.svg)](https://mypy-lang.org/)
[![Supported Python version](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/release/python-311/)

# aidbox-logging-py
Aidbox $loggy python logging handler with queue

## Installation

Install from pypi as `aidbox-logging`

## Usage

```python
from aidbox_logging import init_queued_aidbox_loggy_handler, enable_excepthook_logging


logging.basicConfig(
    format="[%(asctime)s] [%(process)d] [%(levelname)s] %(name)s %(message)s",
    datefmt="%Y-%m-%d %H:%M:%S %z",
)
root_logger = logging.getLogger()
root_logger.setLevel(logging.DEBUG)
if env_config.loggy_url:
    loggy_handler = init_queued_aidbox_loggy_handler(
        env_config.loggy_url,
        app="lab-backend",
        version=f"{version.__version__}-{version.__build_commit_hash__}",
        meta={
            "env": env_config.lab_room_id,
        },
    )

    # By default all log messages will be handled by loggy
    root_logger.addHandler(loggy_handler)

    # By default gunicorn errors are not propagated, so we handle it explicitly
    logging.getLogger("gunicorn.error").addHandler(loggy_handler)

    # Optional: enable excepthook logging that intercepts all exceptions inside threads
    enable_excepthook_logging()
```