Package: eventy3
Release 3.0.0This is therelease-3.0.0branch, will be merged into main tagged3.0.0. Original README at the bottomQuickstartThe first feature is a concept oftrace_idandrequest_idcontext that works with threaded and async execution contexts (using contextvars). This allows to use logging handler defined in eventy to add this information in your logs automatically.Logging setup APIOut of the box:importloggingfromeventy.loggingimportGkeHandler,SimpleHandler# Apply to all loggers# Log levels can then be reduced in handlersroot_logger=logging.getLogger()root_logger.setLevel('DEBUG')# All INFO and above messages with GKE formatgke_handler=GkeHandler(level='INFO')root_logger.addHandler(gke_handler)# Additionally, all messages with default colored text formatcolortext_handler=SimpleHandler(colored=True,level='DEBUG')root_logger.addHandler(colortext_handler)# Additionally, all WARNING with a specific formatcustom_handler=SimpleHandler(fmt='%(asctime)sSPECIAL WARNING! TRACE_id=%(trace_id)s%(message)s',datefmt='[%Y-%m-%d%H:%M:%S %z]',level='WARNING')root_logger.addHandler(custom_handler)Customize everything:importloggingfromeventy.loggingimportadd_trace_id_filter,add_request_id_filterhandler=logging.StreamHandler()formatter=logging.Formatter(fmt='%(asctime)s[CID:%(correlation_id)s] [RID:%(request_id)s] ''%(levelname)s(%(module)sL%(lineno)d) ''%(message)s',datefmt='[%Y-%m-%d%H:%M:%S %z]',)handler.setFormatter(formatter)# You need to add this if you don't use GkeHandler or SimpleHandlerhandler.addFilter(add_trace_id_filter)handler.addFilter(add_request_id_filter)handler.setLevel('INFO')root_logger.addHandler(handler)Context APIConsider this example:importasyncioimportloggingfromeventy.contextimportrequest_idfromeventy.loggingimportSimpleHandlerlogger=logging.getLogger(__name__)root_logger=logging.getLogger()root_logger.setLevel('DEBUG')root_logger.addHandler(SimpleHandler())asyncdefhandle_request(delay):logger.info(f'Begin')awaitasyncio.sleep(delay)logger.info(f'End')asyncdefmain():tasks=[]request_id.set("R-1")tasks.append(asyncio.create_task(handle_request(1.0)))request_id.set("R-2")tasks.append(asyncio.create_task(handle_request(2.0)))request_id.set("R-3")tasks.append(asyncio.create_task(handle_request(1.5)))fortaskintasks:awaittaskasyncio.run(main())Note that the only parameter fohandle_requestis the delay to process. The logs are:[2021-06-22 12:15:08 +0200] [TID:] [RID:R-1] INFO __main__ (ex2 L18) Begin
[2021-06-22 12:15:08 +0200] [TID:] [RID:R-2] INFO __main__ (ex2 L18) Begin
[2021-06-22 12:15:08 +0200] [TID:] [RID:R-3] INFO __main__ (ex2 L18) Begin
[2021-06-22 12:15:09 +0200] [TID:] [RID:R-1] INFO __main__ (ex2 L20) End
[2021-06-22 12:15:09 +0200] [TID:] [RID:R-3] INFO __main__ (ex2 L20) End
[2021-06-22 12:15:10 +0200] [TID:] [RID:R-2] INFO __main__ (ex2 L20) EndThreading support will come later.EventyWhat is Eventy?Eventy is both a protocol and a library for making the design of fault-tolerant, event-driven, concurrent and distributed applications in a microservices-oriented architecture easier.As a protocol, Eventy is language-agnostic.However, the reference implementation is written in Python. Whatever is your programming language, you can use Eventy. If you are using Python, you will be able to make the most of it quicker.MotivationThe reality is, a distributed, microservices-oriented architecture is a bit hard to make right.Martin Fowler evenended up stating:First Law of Distributed Object Design: “don't distribute your objects”.He laterdetailed his viewof the First Law regarding microservices.As Martin Folwer points out, inherently microservices come with their undeniable advantages and a few companies use them with great success. The only reason why a lot of people end up struggling with microservices, is because it greatly increases the complexity of the whole system, which makes it harder to get it right.Eventy is adressing exactly that issue — providing you with a reliable framework to make microservices work for you.A big part of the design process to create Eventy was to understand different use cases and see how the design can be simplified. For instance, not only does Eventy offer state partitionning for free, it actually comes up with a few stratagies that eliminate the need to persist state altogether.InspirationKafka Streamswas a great influence in making Eventy.CeleryandFaustare also worth to be looked at if you are looking for an opiniated framework easy to get started with.However, these frameworks only partially solve all the issues you will have with microservices. And, in our opinion, these frameworks are not suitable for designing large critical systems.They're both opinanated, and therefore cannot be easily integrated in your existing software. You will have to build your software around the framework, instead of the other way around. They also don't give you the full control on the way you can use them: you can only use them as a whole, or not at all.What Eventy can do for youEventy implements multiple features, but all of them simply solve two main problems:How to make services communicate with each otherHow to access and persist stateWith Eventy, you can serialize data the way you want. You can useAvro,JSON,gRPC, or whatever customer serializer you like.With Eventy, you can use any system you like as a persistency layer, as long as it supports transactions, if you need strong processing guarantees. The most obvious choice is to useApache Kafka, but persisting messages overPostgreSQLis completely feasable, too.Eventy was destined with the mindset of a library of related but independently usable components - and not a framework: the behaviour is explicit and you're the one in charge: you can design your software your own way.This explicit behaviour, albeit requiring more boilerplate, gives you better clarity on what is happening. Recipes and examples are provided so that you can understand how to use Eventy for most use cases.You're free to use any part of Eventy as well. Even if you end up not using the Eventy protocol at all, simply reading the documentation and understanding the issues that are adressed and how they are adressed can help you to get on the right path.Main components of Eventyawell-defined communication protocolfor sending various types of persisted messages, calledRecords:Events,RequestsandResponsespersistency ofRecordsthat can be stored forever, which lets you keep track of all the changes in your system (especially useful for audits and business analytics)queuesso thatRecordscan be processed asynchroneously, and aren't lost even if your system is down or overloadedstrong processing guarantees: a Record can be designed to be processedat least once,at most onceandexactly onceeven if your system encounters a process or network failure at any pointself-propagatingContextsthat in many cases entirely eliminate the need of persisting statepartitionned state persistencyso that you no longer have a single point of failure in your system (akathe database) and can scale up as your business grows