eventy3 3.0.0
Qotto/eventy
Requires Python: >=3.7

# Release 3.0.0

This is the `release-3.0.0` branch, will be merged into main tagged `3.0.0`. Original README at the bottom

## Quickstart

The first feature is a concept of `trace_id` and `request_id` context that works with threaded and async execution contexts (using contextvars). This allows to use logging handler defined in eventy to add this information in your logs automatically.

### Logging setup API

Out of the box:

```python
import logging

from eventy.logging import GkeHandler, SimpleHandler

# Apply to all loggers
# Log levels can then be reduced in handlers
root_logger = logging.getLogger()
root_logger.setLevel('DEBUG')

# All INFO and above messages with GKE format
gke_handler = GkeHandler(level='INFO')
root_logger.addHandler(gke_handler)

# Additionally, all messages with default colored text format
colortext_handler = SimpleHandler(colored=True, level='DEBUG')
root_logger.addHandler(colortext_handler)

# Additionally, all WARNING with a specific format
custom_handler = SimpleHandler(
    fmt='%(asctime)s SPECIAL WARNING! TRACE_id=%(trace_id)s %(message)s',
    datefmt='[%Y-%m-%d %H:%M:%S %z]',
    level='WARNING'
)
root_logger.addHandler(custom_handler)
```

Customize everything:

```python
import logging
from eventy.logging import add_trace_id_filter, add_request_id_filter

handler = logging.StreamHandler()
formatter = logging.Formatter(
    fmt='%(asctime)s [CID:%(correlation_id)s] [RID:%(request_id)s] '
        '%(levelname)s (%(module)s L%(lineno)d) '
        '%(message)s',
    datefmt='[%Y-%m-%d %H:%M:%S %z]',
)
handler.setFormatter(formatter)

# You need to add this if you don't use GkeHandler or SimpleHandler
handler.addFilter(add_trace_id_filter)
handler.addFilter(add_request_id_filter)

handler.setLevel('INFO')
root_logger.addHandler(handler)
```

## Context API

Consider this example:

```python
import asyncio
import logging

from eventy.context import request_id
from eventy.logging import SimpleHandler

logger = logging.getLogger(__name__)

root_logger = logging.getLogger()
root_logger.setLevel('DEBUG')
root_logger.addHandler(SimpleHandler())

async def handle_request(delay):
    logger.info(f'Begin')
    await asyncio.sleep(delay)
    logger.info(f'End')

async def main():
    tasks = []

    request_id.set("R-1")
    tasks.append(asyncio.create_task(handle_request(1.0)))
    request_id.set("R-2")
    tasks.append(asyncio.create_task(handle_request(2.0)))
    request_id.set("R-3")
    tasks.append(asyncio.create_task(handle_request(1.5)))

    for task in tasks:
        await task

asyncio.run(main())
```

Note that the only parameter fo `handle_request` is the delay to process. The logs are:

```
[2021-06-22 12:15:08 +0200] [TID:] [RID:R-1] INFO __main__ (ex2 L18) Begin
[2021-06-22 12:15:08 +0200] [TID:] [RID:R-2] INFO __main__ (ex2 L18) Begin
[2021-06-22 12:15:08 +0200] [TID:] [RID:R-3] INFO __main__ (ex2 L18) Begin
[2021-06-22 12:15:09 +0200] [TID:] [RID:R-1] INFO __main__ (ex2 L20) End
[2021-06-22 12:15:09 +0200] [TID:] [RID:R-3] INFO __main__ (ex2 L20) End
[2021-06-22 12:15:10 +0200] [TID:] [RID:R-2] INFO __main__ (ex2 L20) End
```

Threading support will come later.

# Eventy

## What is Eventy?

Eventy is both a protocol and a library for making the design of fault-tolerant, event-driven, concurrent and distributed applications in a microservices-oriented architecture easier.

As a protocol, Eventy is language-agnostic.

However, the reference implementation is written in Python. Whatever is your programming language, you can use Eventy. If you are using Python, you will be able to make the most of it quicker.

## Motivation

The reality is, a distributed, microservices-oriented architecture is a bit hard to make right.

Martin Fowler even [ended up stating](https://www.drdobbs.com/errant-architectures/184414966):

> First Law of Distributed Object Design: “don't distribute your objects”.

He later [detailed his view](https://martinfowler.com/articles/distributed-objects-microservices.html) of the First Law regarding microservices.

As Martin Folwer points out, inherently microservices come with their undeniable advantages and a few companies use them with great success. The only reason why a lot of people end up struggling with microservices, is because it greatly increases the complexity of the whole system, which makes it harder to get it right.

Eventy is adressing exactly that issue — providing you with a reliable framework to make microservices work for you.

A big part of the design process to create Eventy was to understand different use cases and see how the design can be simplified. For instance, not only does Eventy offer state partitionning for free, it actually comes up with a few stratagies that eliminate the need to persist state altogether.

## Inspiration

[Kafka Streams](https://kafka.apache.org/documentation/streams/) was a great influence in making Eventy. [Celery](http://www.celeryproject.org/) and [Faust](https://github.com/robinhood/faust) are also worth to be looked at if you are looking for an opiniated framework easy to get started with.

However, these frameworks only partially solve all the issues you will have with microservices. And, in our opinion, these frameworks are not suitable for designing large critical systems.

They're both opinanated, and therefore cannot be easily integrated in your existing software. You will have to build your software around the framework, instead of the other way around. They also don't give you the full control on the way you can use them: you can only use them as a whole, or not at all.

## What Eventy can do for you

Eventy implements multiple features, but all of them simply solve two main problems:

* How to make services communicate with each other
* How to access and persist state

With Eventy, you can serialize data the way you want. You can use [Avro](https://avro.apache.org/), [JSON](https://www.json.org/), [gRPC](https://grpc.io/), or whatever customer serializer you like.

With Eventy, you can use any system you like as a persistency layer, as long as it supports transactions, if you need strong processing guarantees. The most obvious choice is to use [Apache Kafka](https://kafka.apache.org/), but persisting messages over [PostgreSQL](https://www.postgresql.org/) is completely feasable, too.

Eventy was destined with the mindset of a library of related but independently usable components - and not a framework: the behaviour is explicit and you're the one in charge: you can design your software your own way.

This explicit behaviour, albeit requiring more boilerplate, gives you better clarity on what is happening. Recipes and examples are provided so that you can understand how to use Eventy for most use cases.

You're free to use any part of Eventy as well. Even if you end up not using the Eventy protocol at all, simply reading the documentation and understanding the issues that are adressed and how they are adressed can help you to get on the right path.

## Main components of Eventy

* a **well-defined communication protocol** for sending various types of persisted messages, called _Records_: _Events_, _Requests_ and _Responses_
* **persistency of _Records_** that can be stored forever, which lets you keep track of all the changes in your system (especially useful for audits and business analytics)
* **queues** so that _Records_ can be processed asynchroneously, and aren't lost even if your system is down or overloaded
* **strong processing guarantees**: a Record can be designed to be processed _at least once_, _at most once_ and _exactly once_ even if your system encounters a process or network failure at any point
* **self-propagating _Contexts_** that in many cases entirely eliminate the need of persisting state
* **partitionned state persistency** so that you no longer have a single point of failure in your system (aka _the database_) and can scale up as your business grows