Package: eventsourcing-grpc
Event Sourcing with gRPCThis is an extension forthe Python eventsourcing library.This package provides a gRPC server class that runs event-sourced applications
in a gRPC server. A gRPC client class can be used to interact with the application
server. Application command and query methods can be called, and
notifications selected, remotely using an application proxy object
provided by the client.This package also provides a system runner class, with the same interface as the Runner
classes in the core eventsourcing library. It can run a system of event-sourced
applications as a set of gRPC servers. Leader and follower connections, for prompting
and pulling event notifications, are implemented with gRPC clients.This package provides console commands that can be used to start servers and run systems
from a command line interface.This package also provides an example of running an event-sourced application in
gRPC server in a Docker container.This package also provides an example of running a system of applications
in gRPC servers in Docker containers with docker-compose.This package also provides an example of running a system of applications
in gRPC servers in Docker containers with Kubernetes, with clients and
servers authenticated with SSL/TLS PKI certificates.InstallationUse pip to install thestable distributionfrom the Python Package Index.$ pip install eventsourcing_grpcIt is recommended to install Python packages into a Python virtual environment.gRPC clients and serversTheGrpcApplicationServerclass can be used to run an event-sourced
application as a gRPC server, and theGrpcApplicationClientclass can
be used to connect to a server.Define an event-sourced application.fromeventsourcing_grpc.exampleimportOrdersConfigure with environment variables.importosos.environ["ORDERS_GRPC_SERVER_ADDRESS"]="localhost:50051"Application persistence can be configured in the usual way (see core library docs).Use thestart_server()function to create and start a gRPC application server.fromeventsourcing_grpc.application_serverimportstart_serverserver=start_server(Orders)Use theconnect()function to create a gRPC client and connect to the server.fromeventsourcing_grpc.application_clientimportconnectclient=connect(Orders,max_attempts=10)Use the application proxy to call command and query methods on the application server.# Create order.order_id=client.app.create_new_order()# Get order.order=client.app.get_order(order_id)assertorder["id"]==order_idassertorder["is_reserved"]==Falseassertorder["is_paid"]==FalseUse the application proxy to select event notifications.notifications=client.app.notification_log.select(start=1,limit=10)assertlen(notifications)==1assertnotifications[0].id==1assertnotifications[0].originator_id==order_idassertnotifications[0].originator_version==1assertnotifications[0].topic=="eventsourcing_grpc.example:Order.Created"Close client and stop server.client.close()server.stop(wait=True)gRPC runnerTheGrpcRunnerclass can be used to run a system of event-sourced applications
as an inter-connected set of gRPC servers.fromeventsourcing_grpc.runnerimportGrpcRunnerDefine a system of event-sourced applications.fromeventsourcing_grpc.exampleimportsystemConfigure with environment variables.importosos.environ["ORDERS_GRPC_SERVER_ADDRESS"]="localhost:50051"os.environ["RESERVATIONS_GRPC_SERVER_ADDRESS"]="localhost:50052"os.environ["PAYMENTS_GRPC_SERVER_ADDRESS"]="localhost:50053"Create and start a gRPC system runner (starts servers and connects clients).runner=GrpcRunner(system)runner.start(with_subprocesses=True)Get an application proxy from the runner.app=runner.get(Orders)Call command and query methods on application proxy.order1_id=app.create_new_order()# Wait for the processing to happen.fromtimeimportsleepfor_inrange(100):sleep(0.1)ifapp.is_order_paid(order1_id):breakelifrunner.has_errored.is_set():raiseAssertionError("Runner error")else:raiseAssertionError("Timeout waiting for order to be paid")Get application's event notifications.notifications=app.notification_log.select(start=1,limit=10)assertlen(notifications)==3assertnotifications[0].id==1assertnotifications[0].originator_id==order1_idassertnotifications[0].originator_version==1assertnotifications[0].topic=="eventsourcing_grpc.example:Order.Created"assertnotifications[1].id==2assertnotifications[1].originator_id==order1_idassertnotifications[1].originator_version==2assertnotifications[1].topic=="eventsourcing_grpc.example:Order.Reserved"assertnotifications[2].id==3assertnotifications[2].originator_id==order1_idassertnotifications[2].originator_version==3assertnotifications[2].topic=="eventsourcing_grpc.example:Order.Paid"Stop runner (closes clients and stops servers).runner.stop()Start server from command lineThe console commandeventsourcing_grpc_servercan be used to start
a gRPC application server. The application topic can be given as a
command line argument.$ export ORDERS_GRPC_SERVER_ADDRESS=localhost:50051
$ eventsourcing_grpc_server eventsourcing_grpc.example:OrdersA system of interconnected application servers can be started in this way,
after settingSYSTEM_TOPICin the environment.$ export SYSTEM_TOPIC=eventsourcing_grpc.example:system
$ export ORDERS_GRPC_SERVER_ADDRESS=localhost:50051
$ export RESERVATIONS_GRPC_SERVER_ADDRESS=localhost:50052
$ export PAYMENTS_GRPC_SERVER_ADDRESS=localhost:50053

$ eventsourcing_grpc_server eventsourcing_grpc.example:Orders &
$ eventsourcing_grpc_server eventsourcing_grpc.example:Reservations &
$ eventsourcing_grpc_server eventsourcing_grpc.example:Payments &In the example above, the servers are run as background processes.Start runner from command lineThe console commandeventsourcing_grpc_runnercan be used to run a system
of event-sourced applications. Only the system topic must be configured.The system topic can be given as a command line argument.$ eventsourcing_grpc_runner eventsourcing_grpc.example:systemAlternatively, the system topic can be set using theSYSTEM_TOPICenvironment variable.$ export SYSTEM_TOPIC=eventsourcing_grpc.example:system
$ eventsourcing_grpc_runnerThe server addresses will be automatically configured, unless they are set as
environment variables.Docker containerA Docker file can be defined to run theeventsourcing_grpc_servercommand
mentioned above, after installing an event-sourced system and theeventsourcing_grpcpackage.FROMpython:3.10WORKDIR/app# Install package(s).RUNpipinstalleventsourcing_grpc# Run application server.ENVPYTHONUNBUFFERED=1CMD["eventsourcing_grpc_server"]A container image can then be built with Docker.$ docker build -t eventsourcing-grpc -f Dockerfile .Docker ComposeA system of gRPC application servers can be run with Docker Compose.
The following Docker Compose file assumes SSL/TLS PKI certificates
have been generated in a local./sslfolder.version: '2'

services:
  orders:
    image: "eventsourcing-grpc:v1"
    environment:
      APPLICATION_TOPIC: "eventsourcing_grpc.example:Orders"
      SYSTEM_TOPIC: "eventsourcing_grpc.example:system"
      ORDERS_GRPC_SERVER_ADDRESS: "orders:50051"
      RESERVATIONS_GRPC_SERVER_ADDRESS: "reservations:50052"
      PAYMENTS_GRPC_SERVER_ADDRESS: "payments:50053"
      MAX_PULL_INTERVAL: "10"
      GRPC_SSL_PRIVATE_KEY_PATH: /app/ssl/orders.key
      GRPC_SSL_CERTIFICATE_PATH: /app/ssl/orders.crt
      GRPC_SSL_ROOT_CERTIFICATE_PATH: /app/ssl/root.crt
    volumes:
      - ./ssl/orders:/app/ssl
    ports:
      - "50051:50051"

  reservations:
    image: "eventsourcing-grpc:v1"
    environment:
      APPLICATION_TOPIC: "eventsourcing_grpc.example:Reservations"
      SYSTEM_TOPIC: "eventsourcing_grpc.example:system"
      ORDERS_GRPC_SERVER_ADDRESS: "orders:50051"
      RESERVATIONS_GRPC_SERVER_ADDRESS: "reservations:50052"
      PAYMENTS_GRPC_SERVER_ADDRESS: "payments:50053"
      MAX_PULL_INTERVAL: "10"
      GRPC_SSL_PRIVATE_KEY_PATH: /app/ssl/reservations.key
      GRPC_SSL_CERTIFICATE_PATH: /app/ssl/reservations.crt
      GRPC_SSL_ROOT_CERTIFICATE_PATH: /app/ssl/root.crt
    volumes:
      - ./ssl/reservations:/app/ssl
    ports:
      - "50052:50052"

  payments:
    image: "eventsourcing-grpc:v1"
    environment:
      APPLICATION_TOPIC: "eventsourcing_grpc.example:Payments"
      SYSTEM_TOPIC: "eventsourcing_grpc.example:system"
      ORDERS_GRPC_SERVER_ADDRESS: "orders:50051"
      RESERVATIONS_GRPC_SERVER_ADDRESS: "reservations:50052"
      PAYMENTS_GRPC_SERVER_ADDRESS: "payments:50053"
      MAX_PULL_INTERVAL: "10"
      GRPC_SSL_PRIVATE_KEY_PATH: /app/ssl/payments.key
      GRPC_SSL_CERTIFICATE_PATH: /app/ssl/payments.crt
      GRPC_SSL_ROOT_CERTIFICATE_PATH: /app/ssl/root.crt
    volumes:
      - ./ssl/payments:/app/ssl
    ports:
      - "50053:50053"The containers can then be started.$ docker-compose up -dKubernetesA system of gRPC application servers can be run with Kubernetes.Create Kubernetes Secrets from the SSL/TLS PKI certificates.$ kubectl create secret generic root-ssl-secret \
--from-file=root.crt=ssl/root/root.crt

$ kubectl create secret tls orders-ssl-secret \
--key ./ssl/orders/orders.key \
--cert ./ssl/orders/orders.crt

$ kubectl create secret tls reservations-ssl-secret \
--key ./ssl/reservations/reservations.key \
--cert ./ssl/reservations/reservations.crt

$ kubectl create secret tls payments-ssl-secret \
--key ./ssl/payments/payments.key \
--cert ./ssl/payments/payments.crtCreate a Kubernetes Deployment for each application.apiVersion:apps/v1kind:Deploymentmetadata:name:orderslabels:system:ordersspec:selector:matchLabels:app:ordersreplicas:1template:metadata:labels:app:ordersspec:volumes:-name:ssl-secret-volumesecret:secretName:orders-ssl-secret-name:root-ssl-secret-volumesecret:secretName:root-ssl-secretcontainers:-name:ordersimage:"eventsourcing-grpc:latest"imagePullPolicy:Neverports:-containerPort:50051volumeMounts:-mountPath:/app/sslname:ssl-secret-volume-mountPath:/app/root-sslname:root-ssl-secret-volumeenv:-name:APPLICATION_TOPICvalue:"eventsourcing_grpc.example:Orders"-name:SYSTEM_TOPICvalue:"eventsourcing_grpc.example:system"-name:ORDERS_GRPC_SERVER_ADDRESSvalue:"0.0.0.0:50051"-name:RESERVATIONS_GRPC_SERVER_ADDRESSvalue:"reservations:50052"-name:PAYMENTS_GRPC_SERVER_ADDRESSvalue:"payments:50053"-name:GRPC_SSL_PRIVATE_KEY_PATHvalue:/app/ssl/tls.key-name:GRPC_SSL_CERTIFICATE_PATHvalue:/app/ssl/tls.crt-name:GRPC_SSL_ROOT_CERTIFICATE_PATHvalue:/app/root-ssl/root.crtCreate a Kubernetes Service for each Deployment.apiVersion:v1kind:Servicemetadata:name:orderslabels:system:ordersspec:selector:app:orderstype:LoadBalancerports:-port:50051targetPort:50051protocol:TCPThe above configuration will run each application on a separate Pod. Alternatively,
you can run all the applications in the same Pod, by defining a multi-container
Deployment.Please refer to the project repo (and its GitHub actions workflow and Makefile)
to see a fully working example of running a system of applications with minikube.DevelopersAfter cloning the eventsourcing-grpc repository, set up a virtual
environment and install dependencies by running the following command in the
root folder.$ make installThemake installcommand uses the build tool Poetry to create a dedicated
Python virtual environment for this project, and installs popular development
dependencies such as Black, isort and pytest.Add tests in./tests. Add code in./eventsourcing_grpc.Generate SSL/TLS certificates and private keys for testing.$ make sslRun tests.$ make testCheck the formatting of the code.$ make lintReformat the code.$ make fmtAdd dependencies inpyproject.tomland then update installed packages.$ make update-packagesIf you edit the .proto files, regenerate the project's protos package.$ make grpc-stubs