Package: esdbclient
Python gRPC Client for EventStoreDBThisPython packageprovides a Python
gRPC client for theEventStoreDBdatabase.This client has been developed in collaboration with the EventStoreDB
team. Although not all the features of EventStoreDB are supported
by this client, many of the most useful features are presented
in an easy-to-use interface.This client has been tested to work with EventStoreDB LTS versions 21.10,
22.10, and 23.10, without and without SSL/TLS, with single-server and cluster
modes, with Python versions 3.7, 3.8, 3.9, 3.10, 3.11 and 3.12.The test suite has 100% line and branch coverage. The code has typing annotations
checked strictly with mypy. The code is formatted with black and isort, and checked
with flake8. Poetry is used for package management during development, and for
building and publishing distributions toPyPI.SynopsisYou can connect and interact with an EventStoreDB server using theEventStoreDBClientclass.TheEventStoreDBClientclass can be imported from theesdbclientpackage.Probably the three most useful methods ofEventStoreDBClientare:append_to_stream()This method can be used to record new events in a particular
"stream". This is useful, for example, when executing a command in an application
that mutates an aggregate. This method is "atomic" in that either all or none of
the events will be recorded.read_stream()This method can be used to retrieve all the recorded
events in a "stream". This is useful, for example, when reconstructing
an aggregate from recorded events before executing a command in an
application that creates new events.subscribe_to_all()This method can be used to receive all recorded events in
the database. This is useful, for example, in event-processing components because
it supports processing events with "exactly-once" semantics.The example below uses an "insecure" EventStoreDB server running locally on port 2113.importuuidfromesdbclientimportEventStoreDBClient,NewEvent,StreamState# Construct EventStoreDBClient with an EventStoreDB URI. The# connection string URI specifies that the client should# connect to an "insecure" server running on port 2113.client=EventStoreDBClient(uri="esdb://localhost:2113?Tls=false")# Generate new events. Typically, domain events of different# types are generated in a domain model, and then serialized# into NewEvent objects. An aggregate ID may be used as the# name of a stream in EventStoreDB.stream_name1=str(uuid.uuid4())event1=NewEvent(type='OrderCreated',data=b'{"order_number": "123456"}')event2=NewEvent(type='OrderSubmitted',data=b'{}')event3=NewEvent(type='OrderCancelled',data=b'{}')# Append new events to a new stream. The value returned# from the append_to_stream() method is the overall# "commit position" in the database of the last new event# recorded by this operation. The returned "commit position"# may be used in a user interface to poll an eventually# consistent event-processing component until it can# present an up-to-date materialized view. New events are# each allocated a "stream position", which is the next# available position in the stream, starting from 0.commit_position1=client.append_to_stream(stream_name=stream_name1,current_version=StreamState.NO_STREAM,events=[event1,event2],)# Append events to an existing stream. The "current version"# is the "stream position" of the last recorded event in a# stream. We have recorded two new events, so the "current# version" is 1. The exception 'WrongCurrentVersion' will be# raised if an incorrect value is given.commit_position2=client.append_to_stream(stream_name=stream_name1,current_version=1,events=[event3],)# - allocated commit positions increase monotonicallyassertcommit_position2>commit_position1# Read events from a stream. This method returns a# "read response" iterator, which returns recorded# events. The iterator will stop when there are no# more events to be returned.read_response=client.read_stream(stream_name=stream_name1)# Iterate over "read response" to get recorded events.# The recorded events may be deserialized to domain event# objects of different types and used to reconstruct an# aggregate in a domain model.recorded_events=tuple(read_response)# - stream 'stream_name1' now has three eventsassertlen(recorded_events)==3# - allocated stream positions are zero-based and gaplessassertrecorded_events[0].stream_position==0assertrecorded_events[1].stream_position==1assertrecorded_events[2].stream_position==2# - event attribute values are recorded faithfullyassertrecorded_events[0].type=="OrderCreated"assertrecorded_events[0].data==b'{"order_number": "123456"}'assertrecorded_events[0].id==event1.idassertrecorded_events[1].type=="OrderSubmitted"assertrecorded_events[1].data==b'{}'assertrecorded_events[1].id==event2.idassertrecorded_events[2].type=="OrderCancelled"assertrecorded_events[2].data==b'{}'assertrecorded_events[2].id==event3.id# Start a catch-up subscription from last recorded position.# This method returns a "catch-up subscription" iterator,# which returns recorded events. The iterator will not stop# when there are no more recorded events to be returned, but# will block, and continue when further events are recorded.catchup_subscription=client.subscribe_to_all()# Iterate over the catch-up subscription. Process each recorded# event in turn. Within an atomic database transaction, record# the event's "commit position" along with any new state generated# by processing the event. Use the component's last recorded commit# position when restarting the catch-up subscription.received_events=[]foreventincatchup_subscription:received_events.append(event)ifevent.commit_position==commit_position2:# Break so we can continue with the example.break# - events are received in the order they were recordedassertreceived_events[-3].type=="OrderCreated"assertreceived_events[-3].data==b'{"order_number": "123456"}'assertreceived_events[-3].id==event1.idassertreceived_events[-2].type=="OrderSubmitted"assertreceived_events[-2].data==b'{}'assertreceived_events[-2].id==event2.idassertreceived_events[-1].type=="OrderCancelled"assertreceived_events[-1].data==b'{}'assertreceived_events[-1].id==event3.id# Stop the catch-up subscription iterator.catchup_subscription.stop()# Close the client's gRPC connection.client.close()See below for more details.For an example of usage, see theeventsourcing-eventstoredbpackage.Table of contentsInstall packageFrom PyPIWith PoetryEventStoreDB serverRun containerStop containerEventStoreDB clientImport classConstruct clientConnection stringsTwo schemesUser info stringQuery stringExamplesEvent objectsNew eventsRecorded eventsStreamsAppend eventsIdempotent append operationsRead stream eventsGet current versionHow to implement snapshotting with EventStoreDBRead all eventsGet commit positionGet stream metadataSet stream metadataDelete streamTombstone streamCatch-up subscriptionsSubscribe to all eventsSubscribe to stream eventsHow to implement exactly-once event processingPersistent subscriptionsCreate subscription to allRead subscription to allHow to write a persistent subscription consumerUpdate subscription to allCreate subscription to streamRead subscription to streamUpdate subscription to streamReplay parked eventsGet subscription infoList subscriptionsList subscriptions to streamDelete subscriptionCall credentialsConstruct call credentialsConnectionReconnectCloseAsyncio clientSynopsisNotesRegular expression filtersReconnect and retry method decoratorsContributorsInstall PoetrySetup for PyCharm usersSetup from command lineProject Makefile commandsInstall packageIt is recommended to install Python packages into a Python virtual environment.From PyPIYou can use pip to install this package directly fromthe Python Package Index.$ pip install esdbclientWith PoetryYou can use Poetry to add this package to your pyproject.toml and install it.$ poetry add esdbclientEventStoreDB serverThe EventStoreDB server can be run locally using the official Docker container image.Run containerFor development, you can run a "secure" EventStoreDB server using the following command.$ docker run -d --name eventstoredb-secure -it -p 2113:2113 --env "HOME=/tmp" eventstore/eventstore:21.10.9-buster-slim --devAs we will see, your client will need an EventStoreDB connection string URI as the value
of itsuriconstructor argument. The connection string for this "secure" EventStoreDB
server would be:esdb://admin:changeit@localhost:2113To connect to a "secure" server, you will usually need to include a "username"
and a "password" in the connection string, so that the server can authenticate the
client. With EventStoreDB, the default username is "admin" and the default password
is "changeit".When connecting to a "secure" server, you may also need to provide an SSL/TLS certificate
as the value of theroot_certificatesconstructor argument. If the server certificate
is publicly signed, the root certificates may be picked up by the grpc package from a
default location. The client uses the root SSL/TLS certificate to authenticate the server.
For development, you can either use the SSL/TLS certificate of a self-signing certificate
authority used to create the server's certificate. Or, when using a single-node cluster,
you can just use the server certificate itself, getting the server certificate with
the following Python code.importsslserver_certificate=ssl.get_server_certificate(addr=('localhost',2113))Alternatively, you can start an "insecure" server using the following command.$ docker run -d --name eventstoredb-insecure -it -p 2113:2113 eventstore/eventstore:21.10.9-buster-slim --insecureThe connection string URI for this "insecure" server would be:esdb://localhost:2113?Tls=falseAs we will see, when connecting to an "insecure" server, there is no need to include
a "username" and a "password" in the connection string. If you do, these values will
be ignored by the client, so that they are not sent over an insecure channel.Please note, the "insecure" connection string uses a query string with the field-valueTls=false. The value of this field is by defaulttrue.Stop containerTo stop and remove the "secure" container, use the following Docker commands.$ docker stop eventstoredb-secure
$ docker rm eventstoredb-secureTo stop and remove the "insecure" container, use the following Docker commands.$ docker stop eventstoredb-insecure
$ docker rm eventstoredb-insecureEventStoreDB clientThis EventStoreDB client is implemented in theesdbclientpackage with
theEventStoreDBClientclass.Import classTheEventStoreDBClientclass can be imported from theesdbclientpackage.fromesdbclientimportEventStoreDBClientConstruct clientTheEventStoreDBClientclass has one required constructor argument,uri, and one
optional constructor argument,root_certificates.Theuriargument is expected to be an EventStoreDB connection string URI that
conforms with the standard EventStoreDB "esdb" or "esdb+discover" URI schemes.For example, the following connection string specifies that the client should
attempt to create a "secure" connection to port 2113 on "localhost", and use the
client credentials "username" and "password" when making calls to the server.esdb://username:password@localhost:2113?Tls=trueThe client must be configured to create a "secure" connection to a "secure" server,
or alternatively an "insecure" connection to an "insecure" server. By default, the
client will attempt to create a "secure" connection. And so, when connecting to an
"insecure" server, the connection string must specify that the client should attempt
to make an "insecure" connection.The following connection string specifies that the client should
attempt to create an "insecure" connection to port 2113 on "localhost".
When connecting to an "insecure" server, the client will ignore any
username and password information included in the connection string,
so that usernames and passwords are not sent over an "insecure" connection.esdb://localhost:2113?Tls=falsePlease note, the "insecure" connection string uses a query string with the field-valueTls=false. The value of this field is by defaulttrue. Unless the connection string
URI includes the field-valueTls=falsein the query string, theroot_certificatesconstructor argument can be used to supply SSL/TLS root certificate(s).When connecting to a "secure" server, theroot_certificatesargument can be
a Pythonstrcontaining PEM encoded SSL/TLS root certificates. This value is
passed directly togrpc.ssl_channel_credentials(). It is used for authenticating the
server to the client. It is commonly the certificate of the certificate authority that
was responsible for generating the SSL/TLS certificate used by the EventStoreDB server.
Often it is unnecessary to provide these certificates explicitly, if they are installed
locally in a such a way that the Python grpc library can pick them up from a default
location. Alternatively, for development, you can use the server's certificate itself.In the example below, the constructor argument values are taken from the operating
system environment. This is a typical arrangement in a production environment. It is
done this way here so that the code in this documentation can be tested with both
a "secure" and an "insecure" server.importosclient=EventStoreDBClient(uri=os.getenv("ESDB_URI"),root_certificates=os.getenv("ESDB_ROOT_CERTIFICATES"),)Connection stringsAn EventStoreDB connection string is a URI that conforms with one of two possible
schemes: either the "esdb" scheme, or the "esdb+discover" scheme.The syntax and semantics of the EventStoreDB URI schemes are described below. The
syntax is defined usingEBNF.Two schemesThe "esdb" URI scheme can be defined in the following way.esdb-uri = "esdb://" , [ user-info , "@" ] , grpc-target, { "," , grpc-target } , [ "?" , query-string ] ;In the "esdb" URI scheme, after the optional user info string, there must be at least
one gRPC target. If there are several gRPC targets, they must be separated from each
other with the "," character. Each gRPC target should indicate an EventStoreDB gRPC
server socket, by specifying a host and a port number separated with the ":" character.
The host may be a hostname that can be resolved to an IP address, or an IP address.grpc-target = ( hostname | ip-address ) , ":" , port-number ;The "esdb+discover" URI scheme can be defined in the following way.esdb-discover-uri = "esdb+discover://" , [ user-info, "@" ] , cluster-domainname, [ ":" , port-number ] , [ "?" , query-string ] ;In the "esdb+discover" URI scheme, after the optional user info string, there must be a
domain name which should identify a cluster of EventStoreDB servers. The client will use
a DNS server to resolve the domain name to a list of addresses of EventStoreDB servers,
by querying for 'A' records. The specified port number, or the default port number
"2113", will be used to construct gRPC targets from the addresses obtained from 'A'
records provided by the DNS server. Therefore, if you want to use the "esdb+discover"
URI scheme, you will need to configure DNS when setting up your EventStoreDB cluster.With both the "esdb" and "esdb+discover" URI schemes, the client firstly obtains
a list of gRPC targets: either directly from "esdb" connection strings; or indirectly
from "esdb+discover" connection strings via DNS. This list of targets is known as the
"gossip seed". The client will then attempt to connect to each gRPC target in turn,
attempting to call the EventStoreDB Gossip API to obtain information about the
EventStoreDB cluster. A member of the cluster is selected by the client, according
to the "node preference" option. The client may then need to close its
connection and reconnect to the selected server.User info stringIn both the "esdb" and "esdb+discover" schemes, the URI may include a user info string.
If it exists in the URI, the user info string must be separated from the rest of the URI
with the "@" character. The user info string must include a username and a password,
separated with the ":" character.user-info = username , ":" , password ;The user info is sent by the client as "call credentials" in each call to a "secure"
server, in a "basic auth" authorization header. This authorization header is used by
the server to authenticate the client. The authorization header is not sent to
"insecure" servers.Query stringIn both the "esdb" and "esdb+discover" schemes, the optional query string must be one
or many field-value arguments, separated from each other with the "&" character.query-string = field-value, { "&", field-value } ;Each field-value argument must be one of the supported fields, and an
appropriate value, separated with the "=" character.field-value = ( "Tls", "=" , "true" | "false" )
            | ( "TlsVerifyCert", "=" , "true" | "false" )
            | ( "ConnectionName", "=" , string )
            | ( "NodePreference", "=" , "leader" | "follower" | "readonlyreplica" | "random" )
            | ( "DefaultDeadline", "=" , integer )
            | ( "GossipTimeout", "=" , integer )
            | ( "MaxDiscoverAttempts", "=" , integer )
            | ( "DiscoveryInterval", "=" , integer )
            | ( "KeepAliveInterval", "=" , integer )
            | ( "KeepAliveTimeout", "=" , integer ) ;The table below describes the query field-values supported by this client.FieldValueDescriptionTls"true", "false" (default: "true")If "true" the client will create a "secure" gRPC channel. If "false" the client will create an "insecure" gRPC channel. This must match the server configuration.TlsVerifyCert"true", "false" (default: "true")This value is currently ignored.ConnectionNamestring (default: auto-generated version-4 UUID)Sent in call metadata for every call, to identify the client to the cluster.NodePreference"leader", "follower", "readonlyreplica", "random" (default: "leader")The node state preferred by the client. The client will select a node from the cluster info received from the Gossip API according to this preference.DefaultDeadlineinteger (default:None)The default value (in seconds) of thetimeoutargument of client "write" methods such asappend_to_stream().GossipTimeoutinteger (default: 5)The default value (in seconds) of thetimeoutargument of gossip read methods, such asread_gossip().MaxDiscoverAttemptsinteger (default: 10)The number of attempts to read gossip when connecting or reconnecting to a cluster member.DiscoveryIntervalinteger (default: 100)How long to wait (in milliseconds) between gossip retries.KeepAliveIntervalinteger (default:None)The value of the "grpc.keepalive_ms" gRPC channel option.KeepAliveTimeoutinteger (default:None)The value of the "grpc.keepalive_timeout_ms" gRPC channel option.ExamplesHere are some examples of EventStoreDB connection string URIs.The following URI will cause the client to connect to, and get
cluster info, from "secure" server socketlocalhost:2113. And
then to connect to a "leader" node. And also to use "admin" and
"changeit" as the username and password when making calls to
EventStoreDB API methods.esdb://admin:changeit@localhost:2113The following URI will cause the client to get cluster info from
"insecure" server socket 127.0.0.1:2113.  And then to connect to
a "leader" node.esdb://127.0.0.1:2113?Tls=falseThe following URI will cause the client to get cluster info from
addresses in DNS 'A' records for cluster1.example.com. And then
to connect to a "leader" node. And use a default deadline of 5
seconds when making calls to EventStore API "write" methods.esdb+discover://admin:changeit@cluster1.example.com?DefaultDeadline=5The following URI will cause the client to get cluster info from either
localhost:2111, or localhost:2112, or localhost:2113. And then to connect
to a "follower" node.esdb://admin:changeit@localhost:2111,localhost:2112,localhost:2113?NodePreference=followerThe following URI will cause the client to get cluster info from addresses in
DNS 'A' records for cluster1.example.com. And to configure "keep alive" timeout
and interval in the gRPC channel.esdb+discover://admin:changeit@cluster1.example.com?KeepAliveInterval=10000&KeepAliveTimeout=10000Please note, the client is insensitive to the case of fields and values. If fields are
repeated in the query string, the query string will be parsed without error. However,
the connection options used by the client will use the value of the first field. All
the other field-values in the query string with the same field name will be ignored.
Fields without values will also be ignored.If the client's node preference is "leader" and the node becomes a
"follower", the client will attempt to reconnect to the current leader when a method
is called that expects to call a leader. Methods which mutate the state of the database
expect to call a leader. For such methods, the HTTP header "requires-leader" is set to
"true", and this header is observed by the server, and so a node which is not a leader
that receives such a request will return an error. This error is detected by the client,
which will then close the current gRPC connection and create a new connection to the
leader. The request will then be retried with the leader.If the client's node preference is "follower" and there are no follower
nodes in the cluster, then the client will raise an exception. Similarly, if the
client's node preference is "readonlyreplica" and there are no read-only replica
nodes in the cluster, then the client will also raise an exception.The gRPC channel option "grpc.max_receive_message_length" is automatically
configured to the value17 * 1024 * 1024. This value cannot be changed.Event objectsThis package defines aNewEventclass and aRecordedEventclass. TheNewEventclass should be used when writing events to the database. TheRecordedEventclass is used when reading events from the database.New eventsTheNewEventclass should be used when writing events to an EventStoreDB database.
You will need to construct new event objects before callingappend_to_stream().TheNewEventclass is a frozen Python dataclass. It has two required constructor
arguments (typeanddata) and three optional constructor arguments (metadata,content_typeandid).The requiredtypeargument is a Pythonstr, used to describe the type of
domain event that is being recorded.The requireddataargument is a Pythonbytesobject, used to state the
serialized data of the domain event that is being recorded.The optionalmetadataargument is a Pythonbytesobject, used to indicate any
metadata of the event that will be recorded. The default value is an emptybytesobject.The optionalcontent_typeargument is a Pythonstr, used to indicate the
kind of data that is being recorded. The default value is'application/json',
which indicates that thedatawas serialised using JSON. An alternative value
for this argument is the more general indication'application/octet-stream'.The optionalidargument is a PythonUUIDobject, used to specify the unique ID
of the event that will be recorded. If no value is provided, a new version-4 UUID
will be generated.new_event1=NewEvent(type='OrderCreated',data=b'{"name": "Greg"}',)assertnew_event1.type=='OrderCreated'assertnew_event1.data==b'{"name": "Greg"}'assertnew_event1.metadata==b''assertnew_event1.content_type=='application/json'assertisinstance(new_event1.id,uuid.UUID)event_id=uuid.uuid4()new_event2=NewEvent(type='ImageCreated',data=b'01010101010101',metadata=b'{"a": 1}',content_type='application/octet-stream',id=event_id,)assertnew_event2.type=='ImageCreated'assertnew_event2.data==b'01010101010101'assertnew_event2.metadata==b'{"a": 1}'assertnew_event2.content_type=='application/octet-stream'assertnew_event2.id==event_idRecorded eventsTheRecordedEventclass is used when reading events from an EventStoreDB
database. The client will return event objects of this type from all methods
that return recorded events, such asget_stream(),subscribe_to_all(),
andread_subscription_to_all(). You do not need to construct recorded event objects.LikeNewEvent, theRecordedEventclass is also a frozen Python dataclass. It has
all the attributes thatNewEventhas (type,data,metadata,content_type,id)
and some additional attributes that follow from the fact that an event was recorded
(stream_name,stream_position,commit_position). It has aretry_countwhich is set only when reading persistence subscriptions. It also has alinkattribute, which is set only when resolving "link events".Thetypeattribute is a Pythonstr, used to indicate the type of an event
that was recorded.Thedataattribute is a Pythonbytesobject, used to indicate the data of an
event that was recorded.Themetadataattribute is a Pythonbytesobject, used to indicate the metadata of
an event that was recorded.Thecontent_typeattribute is a Pythonstr, used to indicate the type of
data that was recorded for an event. It is usually'application/json', indicating
that the data can be parsed as JSON. Alternatively, it is'application/octet-stream'.Theidattribute is a PythonUUIDobject, used to indicate the unique ID of an
event that was recorded.Thestream_nameattribute is a Pythonstr, used to indicate the name of a
stream in which an event was recorded.Thestream_positionattribute is a Pythonint, used to indicate the position in a
stream at which an event was recorded.In EventStoreDB, a "stream position" is an integer representing the position of a
recorded event in a stream. Each recorded event is recorded at a position in a stream.
Each stream position is occupied by only one recorded event. New events are recorded at the
next unoccupied position. All sequences of stream positions are zero-based and gapless.Thecommit_positionattribute is a Pythonint, used to indicate the position in the
database at which an event was recorded.In EventStoreDB, a "commit position" is an integer representing the position of a
recorded event in the database. Each recorded event is recorded at a position in the
database. Each commit position is occupied by only one recorded event. Commit positions
are zero-based and increase monotonically as new events are recorded. But, unlike stream
positions, the sequence of successive commit positions is not gapless. Indeed, there are
usually large differences between the commit positions of successively recorded events.Please note, in EventStoreDB 21.10, thecommit_positionof allRecordedEventobjects
obtained fromread_stream()isNone, whereas those obtained fromread_all()have
the actual commit position of the recorded event. This was changed in version 22.10, so
that event objects obtained from bothget_stream()andread_all()have the actual
commit position. Thecommit_positionattribute of theRecordedEventclass is
annotated with the typeOptional[int]for this reason only.Theretry_countis a Pythonint, used to indicate the number of times a persistent
subscription has retried sending the event to a consumer.Thelinkattribute is an optionalRecordedEventthat carries information about
a "link event" that has been resolved. This allows link events to be acknowledged or
negatively acknowledged when using persistence subscriptions with theresolve_linksargument set toTrue. See theack_idproperty.fromdataclassesimportdataclass@dataclass(frozen=True)classRecordedEvent:"""Encapsulates event data that has been recorded in EventStoreDB."""type:strdata:bytesmetadata:bytescontent_type:strid:UUIDstream_name:strstream_position:intcommit_position:Optional[int]retry_count:Optional[int]=Nonelink:Optional["RecordedEvent"]=None@propertydefack_id(self)->UUID:ifself.linkisnotNone:returnself.link.idelse:returnself.id@propertydefis_system_event(self)->bool:returnself.type.startswith("$")@propertydefis_link_event(self)->bool:returnself.type=="$>"@propertydefis_resolved_event(self)->bool:returnself.linkisnotNone@propertydefis_checkpoint(self)->bool:returnFalseThe propertyack_idcan be used to obtain the correct event ID toack()ornack()events received when reading persistent subscriptions. The returned value is either the
value of theidattribute of thelinkattribute, iflinkis notNone, otherwise
it is the value of theidattribute.The propertyis_system_eventindicates whether the event is a "system event". System
events have atypevalue that starts with'$'.The propertyis_link_eventindicates whether the event is a "link event". Link
events have atypevalue of'$>'.The propertyis_resolve_eventindicates whether the event has been resolved from a
"link event". The returned value isTrueiflinkis notNone.The propertyis_checkpointisFalse. This can be used to identifyCheckpointinstances returned when receiving events frominclude_checkpoints=True.StreamsIn EventStoreDB, a "stream" is a sequence of recorded events that all have
the same "stream name". There will normally be many streams in a database,
each with many recorded events. Each recorded event has a position in its stream
(the "stream position"), and a position in the database (the "commit position").
Stream positions are zero-based and gapless. Commit positions are also zero-based,
but are not gapless.The methodsappend_to_stream(),get_stream()andread_all()can
be used to read and record in the database.Append eventsrequires leaderTheappend_to_stream()method can be used atomically to record a sequence of new events.
If the operation is successful, it returns the commit position of the last event in the
sequence that has been recorded.This method has three required arguments,stream_name,current_versionandevents.The requiredstream_nameargument is a Pythonstrthat uniquely identifies a
stream to which a sequence of events will be appended.The requiredcurrent_versionargument is expected to be either a Pythonintthat indicates the stream position of the last recorded event in the stream, orStreamState.NO_STREAMif the stream does not yet exist or has been deleted. The
stream positions are zero-based and gapless, so that if a stream has two events, thecurrent_versionshould be 1. If an incorrect value is given, this method will raise aWrongCurrentVersionexception. This behavior is designed to provide concurrency
control when recording new events. The correct value ofcurrent_versionfor any stream
can be obtained by callingget_current_version(). However, the typical approach is to
reconstruct an aggregate from the recorded events, so that the version of the aggregate
is the stream position of the last recorded event, then have the aggregate generate new
events, and then use the current version of the aggregate as the value of thecurrent_versionargument when appending the new aggregate events. This ensures
the consistency of the recorded aggregate events, because operations that generate
new aggregate events can be retried with a freshly reconstructed aggregate if
aWrongCurrentVersionexception is encountered when recording new events. This
controlling behavior can be entirely disabled by setting the value of thecurrent_versionargument to the constantStreamState.ANY. More selectively, this behaviour can be
disabled for existing streams by setting the value of thecurrent_versionargument to the constantStreamState.EXISTS.The requiredeventsargument is expected to be a sequence of new event objects. TheNewEventclass should be used to construct new event objects. Theappend_to_stream()operation is atomic, so that either all or none of the new events will be recorded. It
is not possible with EventStoreDB atomically to record new events in more than one stream.This method has an optionaltimeoutargument, which is a Pythonfloatthat sets a maximum duration, in seconds, for the completion of the gRPC operation.This method has an optionalcredentialsargument, which can be used to
override call credentials derived from the connection string URI.In the example below, a new event,event1, is appended to a new stream. The stream
does not yet exist, socurrent_versionisStreamState.NO_STREAM.# Construct a new event object.event1=NewEvent(type='OrderCreated',data=b'data1')# Define a new stream name.stream_name1=str(uuid.uuid4())# Append the new events to the new stream.commit_position1=client.append_to_stream(stream_name=stream_name1,current_version=StreamState.NO_STREAM,events=[event1],)In the example below, two subsequent events are appended to an existing
stream. The stream has one recorded event, socurrent_versionis0.event2=NewEvent(type='OrderUpdated',data=b'data2')event3=NewEvent(type='OrderDeleted',data=b'data3')commit_position2=client.append_to_stream(stream_name=stream_name1,current_version=0,events=[event2,event3],)The returned values,commit_position1andcommit_position2, are the
commit positions in the database of the last events in the recorded sequences.
That is,commit_position1is the commit position ofevent1andcommit_position2is the commit position ofevent3.Commit positions that are returned in this way can be used by a user interface to poll
a downstream component until it has processed all the newly recorded events. For example,
consider a user interface command that results in the recording of new events, and an
eventually consistent materialized view in a downstream component that is updated from
these events. If the new events have not yet been processed, the view might be stale,
or out-of-date. Instead of displaying a stale view, the user interface can poll the
downstream component until it has processed the newly recorded events, and then display
an up-to-date view to the user.Idempotent append operationsTheappend_to_stream()method is "idempotent" with respect to theidvalue of aNewEventobject. That is to say, ifappend_to_stream()is called with events
whoseidvalues are equal to those already recorded in the stream, then the
method call will successfully return, with the commit position of the last new event,
without making any changes to the database.This is because sometimes it may happen, when callingappend_to_stream(), that the new
events are successfully recorded, but somehow something bad happens before the method call
can return successfully to the caller. In this case, we cannot be sure that the events have
in fact been recorded, and so we may wish to retry.If the events were in fact successfully recorded, it is convenient for the retried method call
to return successfully, and without either raising an exception (whencurrent_versionis eitherStreamState.NO_STREAMan integer value) or creating further event records
(whencurrent_versionisStreamState.ANYorStreamState.EXISTS), as it would
if theappend_to_stream()method were not idempotent.If the method call initially failed and the new events were not in fact recorded, it
makes good sense, when the method call is retried, that the new events are recorded
and that the method call returns successfully. If the concurrency controls have not been disabled,
that is if thecurrent versionis eitherStreamState.NO_STREAMor an integer value, and
if aWrongCurrentVersionexception is raised when retrying the method call, then we can assume
both that the initial method call did not in fact successfully record the events, and also
that subsequent events have in the meantime been recorded by somebody else. In this case,
an application command which generated the new events may need to be executed again. And
the user of the application may need to be given an opportunity to decide if they still wish to
proceed with their original intention, by displaying a suitable error with an up-to-date view of
the recorded state. In the case where concurrency controls have been disabled, by usingStreamState.ANYorStreamState.EXISTSas the value ofcurrent_position, retrying a method call that failed to
return successfully will, more simply, just attempt to ensure the new events are recorded, regardless
of their resulting stream positions. In either case, when the method call does return successfully, we
can be sure the events have been recorded.The example below shows theappend_to_stream()method being called again with eventsevent2andevent3, and withcurrent_version=0. We can see that repeating the call
toappend_to_stream()returns successfully without raising aWrongCurrentVersionexception, as it would if the `append_to_stream() operation were not idempotent.# Retry appending event3.commit_position_retry=client.append_to_stream(stream_name=stream_name1,current_version=0,events=[event2,event3],)We can see that the same commit position is returned as above.assertcommit_position_retry==commit_position2The example below shows theappend_to_stream()method being called again with eventsevent2andevent3, with andcurrent_version=StreamState.ANY.# Retry appending event3.commit_position_retry=client.append_to_stream(stream_name=stream_name1,current_version=0,events=[event2,event3],)We can see that the same commit position is returned as above.assertcommit_position_retry==commit_position2By callingget_stream(), we can also see the stream has been unchanged.
That is, there are still only three events in the stream.events=client.get_stream(stream_name=stream_name1)assertlen(events)==3This idempotent behaviour depends on theidattribute of theNewEventclass.
This attribute is, by default, assigned a new and unique version-4 UUID when an
instance ofNewEventis constructed. To set theidvalue of aNewEvent,
the optionalidconstructor argument can be used when constructingNewEventobjects.Read stream eventsTheread_stream()method can be used to get events that have been appended
to a stream. This method returns a "read response" object.A "read response" object is a Python iterator. Recorded events can be
obtained by iterating over the "read response" object. Recorded events are
streamed from the server to the client as the iteration proceeds. The iteration
will automatically stop when there are no more recorded events to be returned.
The streaming of events, and hence the iterator, can also be stopped by calling
thestop()method on the "read response" object.Theget_stream()method can be used to get events that have been appended
to a stream. This method returns a Pythontupleof recorded event objects.
The recorded event objects are instances of theRecordedEventclass. It
callsread_stream()and passes the "read response" iterator into a Pythontuple, so that the streaming will complete before the method returns.Theread_stream()andget_stream()methods have one required argument,stream_name.The requiredstream_nameargument is a Pythonstrthat uniquely identifies a
stream from which recorded events will be returned.Theread_stream()andget_stream()methods also have six optional arguments,stream_position,backwards,resolve_links,limit,timeout, andcredentials.The optionalstream_positionargument is a Pythonintthat can be used to
indicate the position in the stream from which to start reading. The default value
ofstream_positionisNone. When reading a stream from a specific position in the
stream, the recorded event at that position will be included, both when reading
forwards from that position, and when reading backwards.The optionalbackwardsargument is a Pythonbool. The default value ofbackwardsisFalse, which means the stream will be read forwards, so that events are returned
in the order they were recorded. IfbackwardsisTrue, the events are returned in
reverse order.IfbackwardsisFalseandstream_positionisNone, the stream's events will be
returned in the order they were recorded, starting from the first recorded event. IfbackwardsisTrueandstream_positionisNone, the stream's events will be
returned in reverse order, starting from the last recorded event.The optionalresolve_linksargument is a Pythonbool. The default value ofresolve_linksisFalse, which means any event links will not be resolved, so that the events that are
returned may represent event links. Ifresolve_linksisTrue, any event links will
be resolved, so that the linked events will be returned instead of the event links.The optionallimitargument is a Pythonintwhich restricts the number of events
that will be returned. The default value oflimitissys.maxint.The optionaltimeoutargument is a Pythonfloatwhich sets a
maximum duration, in seconds, for the completion of the gRPC operation.This method has an optionalcredentialsargument, which can be used to
override call credentials derived from the connection string URI.The example below shows the default behavior, which is to return all the recorded
events of a stream forwards from the first recorded events to the last.events=client.get_stream(stream_name=stream_name1)assertlen(events)==3assertevents[0]==event1assertevents[1]==event2assertevents[2]==event3The example below shows how to use thestream_positionargument to read a stream
from a specific stream position to the end of the stream. Stream positions are
zero-based, and sostream_position=1corresponds to the second event that was
recorded in the stream, in this caseevent2.events=client.get_stream(stream_name=stream_name1,stream_position=1,)assertlen(events)==2assertevents[0]==event2assertevents[1]==event3The example below shows how to use thebackwardsargument to read a stream backwards.events=client.get_stream(stream_name=stream_name1,backwards=True,)assertlen(events)==3assertevents[0]==event3assertevents[1]==event2assertevents[2]==event1The example below shows how to use thelimitargument to read a limited number of
events.events=client.get_stream(stream_name=stream_name1,limit=2,)assertlen(events)==2assertevents[0]==event1assertevents[1]==event2Theread_stream()andget_stream()methods will raise aNotFoundexception if the
named stream has never existed or has been deleted.fromesdbclient.exceptionsimportNotFoundtry:client.get_stream('does-not-exist')exceptNotFound:pass# The stream does not exist.else:raiseException("Shouldn't get here")Please note, theget_stream()method is decorated with the@autoreconnectand@retrygrpcdecorators, whilst theread_stream()method is not. This means that
all errors due to connection issues will be caught by the retry and reconnect decorators
when calling theget_stream()method, but not when callingread_stream(). Theread_stream()method has no such decorators because the streaming only starts
when iterating over the "read response" starts, which means that the method returns
before the streaming starts, and so there is no chance for any decorators to catch
any connection issues.For the same reason,read_stream()will not raise aNotFoundexception when
the stream does not exist, until iterating over the "read response" object begins.If you are reading a very large stream, then you might prefer to callread_stream(),
and begin iterating through the recorded events whilst they are being streamed from
the server, rather than both waiting and having them all accumulate in memory.Get current versionTheget_current_version()method is a convenience method that essentially callsget_stream()withbackwards=Trueandlimit=1. This method returns
the value of thestream_positionattribute of the last recorded event in a
stream. If a stream does not exist, the returned value isStreamState.NO_STREAM.
The returned value is the correct value ofcurrent_versionwhen appending events
to a stream, and when deleting or tombstoning a stream.This method has one required argument,stream_name.The requiredstream_nameargument is a Pythonstrthat uniquely identifies a
stream from which a stream position will be returned.This method has an optionaltimeoutargument, which is a Pythonfloatthat sets a maximum duration, in seconds, for the completion of the gRPC operation.This method has an optionalcredentialsargument, which can be used to
override call credentials derived from the connection string URI.In the example below, the last stream position ofstream_name1is obtained.
Since three events have been appended tostream_name1, and because positions
in a stream are zero-based and gapless, so the current version is2.current_version=client.get_current_version(stream_name=stream_name1)assertcurrent_version==2If a stream has never existed or has been deleted, the returned value isStreamState.NO_STREAM, which is the correct value of thecurrent_versionargument both when appending the first event of a new stream, and also when
appending events to a stream that has been deleted.current_version=client.get_current_version(stream_name='does-not-exist')assertcurrent_versionisStreamState.NO_STREAMHow to implement snapshotting with EventStoreDBSnapshots can improve the performance of aggregates that would otherwise be
reconstructed from very long streams. However, it is generally recommended to design
aggregates to have a finite lifecycle, and so to have relatively short streams,
thereby avoiding the need for snapshotting. This "how to" section is intended merely
to show how snapshotting of aggregates can be implemented with EventStoreDB using
this Python client.Event-sourced aggregates are typically reconstructed from recorded events by calling
a mutator function for each recorded event, evolving from an initial stateNoneto the current state of the aggregate. The functionget_aggregate()shows
how this can be done. The aggregate ID is used as a stream name. The exceptionAggregateNotFoundis raised if the aggregate stream is not found.classAggregateNotFound(Exception):"""Raised when an aggregate is not found."""defget_aggregate(aggregate_id,mutator_func):stream_name=aggregate_id# Get recorded events.try:events=client.get_stream(stream_name=stream_name,stream_position=None)exceptNotFoundase:raiseAggregateNotFound(aggregate_id)fromeelse:# Reconstruct aggregate from recorded events.aggregate=Noneforeventinevents:aggregate=mutator_func(aggregate,event)returnaggregateSnapshotting of aggregates can be implemented by recording the current state of
an aggregate as a new event.If an aggregate object has a version number that corresponds to the stream position of
the last event that was used to reconstruct the aggregate, and this version number
is recorded in the snapshot metadata, then any events that are recorded after the
snapshot can be selected using this version number. The aggregate can then be
reconstructed from the last snapshot and any subsequent events, without having
to replay the entire history.We will use a separate stream for an aggregate's snapshots that is named after the
stream used for recording its events. The name of the snapshot stream will be
constructed by prefixing the aggregate's stream name with'snapshot-$'.SNAPSHOT_STREAM_NAME_PREFIX='snapshot-$'defmake_snapshot_stream_name(stream_name):returnf'{SNAPSHOT_STREAM_NAME_PREFIX}{stream_name}'defremove_snapshot_stream_prefix(snapshot_stream_name):assertsnapshot_stream_name.startswith(SNAPSHOT_STREAM_NAME_PREFIX)returnsnapshot_stream_name[len(SNAPSHOT_STREAM_NAME_PREFIX):]Now, let's redefine theget_aggregate()function, so that it looks for a snapshot event,
then selects subsequent aggregate events, and then calls a mutator function for each
recorded event.Notice that the aggregate events are read from a stream for serialized aggregate
events, whilst the snapshot is read from a separate stream for serialized aggregate
snapshots. We will use JSON to serialize and deserialize event data.importjsondefget_aggregate(aggregate_id,mutator_func):stream_name=aggregate_idrecorded_events=[]# Look for a snapshot.try:snapshots=client.get_stream(stream_name=make_snapshot_stream_name(stream_name),backwards=True,limit=1)exceptNotFound:stream_position=Noneelse:assertlen(snapshots)==1snapshot=snapshots[0]stream_position=deserialize(snapshot.metadata)['version']+1recorded_events.append(snapshot)# Get subsequent events.try:events=client.get_stream(stream_name=stream_name,stream_position=stream_position)exceptNotFoundase:raiseAggregateNotFound(aggregate_id)fromeelse:recorded_events+=events# Reconstruct aggregate from recorded events.aggregate=Noneforeventinrecorded_events:aggregate=mutator_func(aggregate,event)returnaggregatedefserialize(d):returnjson.dumps(d).encode('utf8')defdeserialize(s):returnjson.loads(s.decode('utf8'))To show howget_aggregate()can be used, let's define aDogaggregate class, with
attributesnameandtricks. The attributesidandversionwill indicate an
aggregate object's ID and version number. The attributeis_from_snapshotis added
here merely to demonstrate below when an aggregate object has been reconstructed using
a snapshot.fromdataclassesimportdataclass@dataclass(frozen=True)classAggregate:id:strversion:intis_from_snapshot:bool@dataclass(frozen=True)classDog(Aggregate):name:strtricks:listLet's also define a mutator functionmutate_dog()that evolves the state of aDogaggregate given various different types of events,'DogRegistered','DogLearnedTrick', and'Snapshot'.defmutate_dog(dog,event):data=deserialize(event.data)ifevent.type=='DogRegistered':returnDog(id=event.stream_name,version=event.stream_position,is_from_snapshot=False,name=data['name'],tricks=[],)elifevent.type=='DogLearnedTrick':assertevent.stream_position==dog.version+1assertevent.stream_name==dog.id,(event.stream_name,dog.id)returnDog(id=dog.id,version=event.stream_position,is_from_snapshot=dog.is_from_snapshot,name=dog.name,tricks=dog.tricks+[data['trick']],)elifevent.type=='Snapshot':returnDog(id=remove_snapshot_stream_prefix(event.stream_name),version=deserialize(event.metadata)['version'],is_from_snapshot=True,name=data['name'],tricks=data['tricks'],)else:raiseException(f"Unknown event type:{event.type}")For convenience, let's also define aget_dog()function that callsget_aggregate()with themutate_dog()function as the value of itsmutator_funcargument.defget_dog(dog_id):returnget_aggregate(aggregate_id=dog_id,mutator_func=mutate_dog,)We can also define some "command" functions that append new events to the
database. Theregister_dog()function appends aDogRegisteredevent. Therecord_trick_learned()appends aDogLearnedTrickevent. The functionsnapshot_dog()appends aSnapshotevent. Notice that therecord_trick_learned()andsnapshot_dog()functions useget_dog().Notice also that theDogRegisteredandDogLearnedTrickevents are appended to a
stream for aggregate events, whilst theSnapshotevent is appended to a separate
stream for aggregate snapshots.defregister_dog(name):dog_id=str(uuid.uuid4())event=NewEvent(type='DogRegistered',data=serialize({'name':name}),)client.append_to_stream(stream_name=dog_id,current_version=StreamState.NO_STREAM,events=event,)returndog_iddefrecord_trick_learned(dog_id,trick):dog=get_dog(dog_id)event=NewEvent(type='DogLearnedTrick',data=serialize({'trick':trick}),)client.append_to_stream(stream_name=dog_id,current_version=dog.version,events=event,)defsnapshot_dog(dog_id):dog=get_dog(dog_id)event=NewEvent(type='Snapshot',data=serialize({'name':dog.name,'tricks':dog.tricks}),metadata=serialize({'version':dog.version}),)client.append_to_stream(stream_name=make_snapshot_stream_name(dog_id),current_version=StreamState.ANY,events=event,)We can callregister_dog()to register a new dog.# Register a new dog.dog_id=register_dog('Fido')dog=get_dog(dog_id)assertdog.name=='Fido'assertdog.tricks==[]assertdog.version==0assertdog.is_from_snapshotisFalseWe can callrecord_trick_learned()to record that some tricks have been learned.# Record that 'Fido' learned a new trick.record_trick_learned(dog_id,trick='roll over')dog=get_dog(dog_id)assertdog.name=='Fido'assertdog.tricks==['roll over']assertdog.version==1assertdog.is_from_snapshotisFalse# Record that 'Fido' learned another new trick.record_trick_learned(dog_id,trick='fetch ball')dog=get_dog(dog_id)assertdog.name=='Fido'assertdog.tricks==['roll over','fetch ball']assertdog.version==2assertdog.is_from_snapshotisFalseWe can callsnapshot_dog()to record a snapshot of the current state of theDogaggregate. After we callsnapshot_dog(), theget_dog()function will return aDogobject that has been constructed using theSnapshotevent.# Snapshot 'Fido'.snapshot_dog(dog_id)dog=get_dog(dog_id)assertdog.name=='Fido'assertdog.tricks==['roll over','fetch ball']assertdog.version==2assertdog.is_from_snapshotisTrueWe can continue to evolve the state of theDogaggregate, using
the snapshot both during the call torecord_trick_learned()and
when callingget_dog()directly.record_trick_learned(dog_id,trick='sit')dog=get_dog(dog_id)assertdog.name=='Fido'assertdog.tricks==['roll over','fetch ball','sit']assertdog.version==3assertdog.is_from_snapshotisTrueWe can see from theis_from_snapshotattribute that theDogobject was indeed
reconstructed from the snapshot.Snapshots can be created at fixed version number intervals, fixed time
periods, after a particular type of event, immediately after events are
appended, or as a background process.Read all eventsTheread_all()method can be used to get all recorded events
in the database in the order they were recorded. This method returns
a "read response" object, just likeread_stream().A "read response" is an iterator, and not a sequence. Recorded events can be
obtained by iterating over the "read response" object. Recorded events are
streamed from the server to the client as the iteration proceeds. The iteration
will automatically stop when there are no more recorded events to be returned.
The streaming of events, and hence the iterator, can also be stopped by calling
thestop()method on the "read response" object. The recorded event objects
are instances of theRecordedEventclass.This method has nine optional arguments,commit_position,backwards,resolve_links,filter_exclude,filter_include,filter_by_stream_name,limit,timeout,
andcredentials.The optionalcommit_positionargument is a Pythonintthat can be used to
specify a commit position from which to start reading. The default value ofcommit_positionisNone. Please note, if a commit position is specified,
it must be an actually existing commit position in the database. When reading
forwards, the event at the commit position may be included, depending upon the
filter. When reading backwards, the event at the commit position will not be
included.The optionalbackwardsargument is a Pythonbool. The default ofbackwardsisFalse, which means events are returned in the order they were recorded, IfbackwardsisTrue, then events are returned in reverse order.IfbackwardsisFalseandcommit_positionisNone, the database's events will
be returned in the order they were recorded, starting from the first recorded event.
This is the default behavior ofread_all(). IfbackwardsisTrueandcommit_positionisNone, the database's events will be returned in reverse order,
starting from the last recorded event.The optionalresolve_linksargument is a Pythonbool. The default value ofresolve_linksisFalse, which means any event links will not be resolved, so that the events that are
returned may represent event links. Ifresolve_linksisTrue, any event links will
be resolved, so that the linked events will be returned instead of the event links.The optionalfilter_excludeargument is a sequence of regular expressions that
specifies which recorded events should be returned. This argument is ignored
iffilter_includeis set to a non-empty sequence. The default value of this
argument matches the event types of EventStoreDB "system events", so that system
events will not normally be included. See the Notes section below for more
information about filter expressions.The optionalfilter_includeargument is a sequence of regular expressions
that specifies which recorded events should be returned. By default, this
argument is an empty tuple. If this argument is set to a non-empty sequence,
thefilter_excludeargument is ignored.The optionalfilter_by_stream_nameargument is a Pythonboolthat indicates
whether the filtering will apply to event types or stream names. By default, this
value isFalseand so the filtering will apply to the event type strings of
recorded events.The optionallimitargument is an integer which restricts the number of events that
will be returned. The default value issys.maxint.The optionaltimeoutargument is a Pythonfloatwhich sets a
maximum duration, in seconds, for the completion of the gRPC operation.The optionalcredentialsargument can be used to
override call credentials derived from the connection string URI.The filtering of events is done on the EventStoreDB server. Thelimitargument is applied on the server after filtering.The example below shows how to get all the events we have recorded in the database
so far, in the order they were recorded. We can see the three events ofstream_name1(event1,event2andevent3) are included, along with others.# Read all events (creates a streaming gRPC call).read_response=client.read_all()# Convert the iterator into a sequence of recorded events.events=tuple(read_response)assertlen(events)>3# more than three# Convert the sequence of recorded events into a set of event IDs.event_ids=set(e.idforeinevents)assertevent1.idinevent_idsassertevent2.idinevent_idsassertevent3.idinevent_idsThe example below shows how to read all recorded events in the database from
a particular commit position, in this casecommit_position1. When reading
forwards from a specific commit position, the event at the specified position
will be included. The value ofcommit_position1is the position we obtained
when appendingevent1. And soevent1is the first recorded event we shall
receive,event2is the second, andevent3is the third.# Read all events forwards from a commit position.read_response=client.read_all(commit_position=commit_position1)# Step through the "read response" iterator.assertnext(read_response)==event1assertnext(read_response)==event2assertnext(read_response)==event3# Stop the iterator.read_response.stop()The example below shows how to read all events recorded in the database in reverse
order. We can see that the first events we receive are the last events that were
recorded: the events of theDogaggregate from the section about snapshotting
and the snapshot.# Read all events backwards from the end.read_response=client.read_all(backwards=True)# Step through the "read response" iterator.assertnext(read_response).type=="DogLearnedTrick"assertnext(read_response).type=="Snapshot"assertnext(read_response).type=="DogLearnedTrick"assertnext(read_response).type=="DogLearnedTrick"assertnext(read_response).type=="DogRegistered"# Stop the iterator.read_response.stop()The example below shows how to read a limited number of events
forwards from a specific commit position.events=tuple(client.read_all(commit_position=commit_position1,limit=1,))assertlen(events)==1assertevents[0]==event1The example below shows how to read a limited number of the recorded events
in the database backwards from the end. In this case, the limit is 1, and
so we receive the last recorded event.events=tuple(client.read_all(backwards=True,limit=1,))assertlen(events)==1assertevents[0].type=='DogLearnedTrick'assertdeserialize(events[0].data)['trick']=='sit'Please note, like theread_stream()method, theread_all()method
is not decorated with retry and reconnect decorators, because the streaming of recorded
events from the server only starts when iterating over the "read response" starts, which
means that the method returns before the streaming starts, and so there is no chance for
any decorators to catch any connection issues.Get commit positionTheget_commit_position()method can be used to get the commit position of the
last recorded event in the database. It simply callsread_all()withbackwards=Trueandlimit=1, and returns the value of thecommit_positionattribute of the last recorded event.commit_position=client.get_commit_position()This method has five optional arguments,filter_exclude,filter_include,filter_by_stream_name,timeoutandcredentials. These values are passed toread_all().The optionalfilter_exclude,filter_includeandfilter_by_stream_namearguments
work in the same way as they do in theread_all()method.The optionaltimeoutargument is a Pythonfloatwhich sets a
maximum duration, in seconds, for the completion of the gRPC operation.The optionalcredentialsargument can be used to override call credentials
derived from the connection string URI.This method might be used to measure progress of a downstream component
that is processing all recorded events, by comparing the current commit
position with the recorded commit position of the last successfully processed
event in a downstream component. In this case, the value of thefilter_exclude,filter_includeandfilter_by_stream_namearguments should equal those used
by the downstream component to obtain recorded events.Get stream metadataTheget_stream_metadata()method returns the metadata for a stream, along
with the version of the stream metadata.This method has one required argument,stream_name, which is a Pythonstrthat
uniquely identifies a stream for which a stream metadata will be obtained.This method has an optionaltimeoutargument, which is a Pythonfloatthat sets a maximum duration, in seconds, for the completion of the gRPC operation.This method has an optionalcredentialsargument, which can be used to
override call credentials derived from the connection string URI.In the example below, metadata forstream_name1is obtained.metadata,metadata_version=client.get_stream_metadata(stream_name=stream_name1)The returnedmetadatavalue is a Pythondict. The returnedmetadata_versionvalue is either anintif the stream exists, orStreamState.NO_STREAMif the stream
does not exist and no metadata has been set. These values can be used as the arguments
ofset_stream_metadata().Set stream metadatarequires leaderThe methodset_stream_metadata()sets metadata for a stream. Stream metadata
can be set before appending events to a stream.This method has one required argument,stream_name, which is a Pythonstrthat
uniquely identifies a stream for which a stream metadata will be set.This method has an optionaltimeoutargument, which is a Pythonfloatthat sets a maximum duration, in seconds, for the completion of the gRPC operation.This method has an optionalcredentialsargument, which can be used to
override call credentials derived from the connection string URI.In the example below, metadata forstream_name1is set.metadata["foo"]="bar"client.set_stream_metadata(stream_name=stream_name1,metadata=metadata,current_version=metadata_version,)Thecurrent_versionargument should be the current version of the stream metadata
obtained fromget_stream_metadata().Please refer to the EventStoreDB documentation for more information about stream
metadata.Delete streamrequires leaderThe methoddelete_stream()can be used to "delete" a stream.This method has two required arguments,stream_nameandcurrent_version.The requiredstream_nameargument is a Pythonstrthat uniquely identifies a
stream to which a sequence of events will be appended.The requiredcurrent_versionargument is expected to be either a Pythonintthat indicates the stream position of the last recorded event in the stream.This method has an optionaltimeoutargument, which is a Pythonfloatthat sets a maximum duration, in seconds, for the completion of the gRPC operation.This method has an optionalcredentialsargument, which can be used to
override call credentials derived from the connection string URI.In the example below,stream_name1is deleted.commit_position=client.delete_stream(stream_name=stream_name1,current_version=2)After deleting a stream, it's still possible to append new events. Reading from a
deleted stream will return only events that have been appended after it was
deleted.Tombstone streamrequires leaderThe methodtombstone_stream()can be used to "tombstone" a stream.This method has two required arguments,stream_nameandcurrent_version.The requiredstream_nameargument is a Pythonstrthat uniquely identifies a
stream to which a sequence of events will be appended.The requiredcurrent_versionargument is expected to be either a Pythonintthat indicates the stream position of the last recorded event in the stream.This method has an optionaltimeoutargument, which is a Pythonfloatthat sets a maximum duration, in seconds, for the completion of the gRPC operation.This method has an optionalcredentialsargument, which can be used to
override call credentials derived from the connection string URI.In the example below,stream_name1is tombstoned.commit_position=client.tombstone_stream(stream_name=stream_name1,current_version=2)After tombstoning a stream, it's not possible to append new events.Catch-up subscriptionsA "catch-up" subscription can be used to receive events that have already been
recorded and events that are recorded subsequently. A catch-up subscription can
be used by an event-processing component that processes recorded events with
"exactly-once" semantics.Thesubscribe_to_all()method starts a catch-up subscription that can receive
all events in the database. Thesubscribe_to_stream()method starts a catch-up
subscription that can receive events from a specific stream. Both methods return a
"catch-up subscription" object, which is a Python iterator. Recorded events can be
obtained by iteration. Recorded event objects obtained in this way are instances
of theRecordedEventclass.Before the "catch-up subscription" object is returned to the caller, the client will
firstly obtain a "confirmation" response from the server, which allows the client to
detect that both the gRPC connection and the streaming gRPC call is operational. For
this reason, thesubscribe_to_all()andsubscribe_to_stream()methods are both
usefully decorated with the reconnect and retry decorators. However, once the method
has returned, the decorators will have exited, and any exceptions that are raised
due to connection issues whilst iterating over the subscription object will have to
be handled by your code.A "catch-up subscription" iterator will not automatically stop when there are no more
events to be returned, but instead the iteration will block until new events are
subsequently recorded in the database. Any subsequently recorded events will then be
immediately streamed to the client, and the iteration will then continue. The streaming
of events, and hence the iteration, can be stopped by calling thestop()method on the
"catch-up subscription" object.Subscribe to all eventsThesubscribe_to_all()method can be used to start a catch-up subscription
from which all events recorded in the database can be obtained in the order
they were recorded. This method returns a "catch-up subscription" iterator.This method also has nine optional arguments,commit_position,from_end,resolve_links,filter_exclude,filter_include,filter_by_stream_name,include_checkpoints,timeoutandcredentials.The optionalcommit_positionargument specifies a commit position. The default
value ofcommit_positionisNone, which means the catch-up subscription will
start from the first recorded event in the database. If a commit position is given,
it must match an actually existing commit position in the database. Only events
recorded after that position will be obtained.The optionalfrom_endargument specifies whether or not the catch-up subscription
will start from the last recorded event in the database. By default, this argument
isFalse. Iffrom_endisTrue, only events recorded after the subscription
is started will be obtained. This argument will be disregarded ifcommit_positionis notNone.The optionalresolve_linksargument is a Pythonbool. The default value ofresolve_linksisFalse, which means any event links will not be resolved, so that the events that are
returned may represent event links. Ifresolve_linksisTrue, any event links will
be resolved, so that the linked events will be returned instead of the event links.The optionalfilter_excludeargument is a sequence of regular expressions that
specifies which recorded events should be returned. This argument is ignored
iffilter_includeis set to a non-empty sequence. The default value of this
argument matches the event types of EventStoreDB "system events", so that system
events will not normally be included. See the Notes section below for more
information about filter expressions.The optionalfilter_includeargument is a sequence of regular expressions
that specifies which recorded events should be returned. By default, this
argument is an empty tuple. If this argument is set to a non-empty sequence,
thefilter_excludeargument is ignored.The optionalfilter_by_stream_nameargument is a Pythonboolthat indicates
whether the filtering will apply to event types or stream names. By default, this
value isFalseand so the filtering will apply to the event type strings of
recorded events.The optionalinclude_checkpointsargument is a Pythonboolwhich indicates
whether checkpoints should be included when recorded events are received. Checkpoints
have acommit_positionvalue that can be used by an event processing component to
update its recorded commit position value, so that, when lots of events are being
filter out, the subscriber does not have to start from the same old position when
the event processing component is restarted.The optionaltimeoutargument is a Pythonfloatwhich sets a
maximum duration, in seconds, for the completion of the gRPC operation.The optionalcredentialsargument can be used to
override call credentials derived from the connection string URI.The example below shows how to start a catch-up subscription that starts
from the first recorded event in the database.# Subscribe from the first recorded event in the database.catchup_subscription=client.subscribe_to_all()The example below shows that catch-up subscriptions do not stop
automatically, but block when the last recorded event is received,
and then continue when subsequent events are recorded.fromtimeimportsleepfromthreadingimportThread# Append a new event to a new stream.stream_name2=str(uuid.uuid4())event4=NewEvent(type='OrderCreated',data=b'data4')client.append_to_stream(stream_name=stream_name2,current_version=StreamState.NO_STREAM,events=[event4],)# Receive events from the catch-up subscription in a different thread.received_events=[]defreceive_events():foreventincatchup_subscription:received_events.append(event)defwait_for_event(event):for_inrange(100):forreceivedinreversed(received_events):ifevent==received:returnelse:sleep(0.1)else:raiseAssertionError("Event wasn't received")thread=Thread(target=receive_events,daemon=True)thread.start()# Wait to receive event4.wait_for_event(event4)# Append another event whilst the subscription is running.event5=NewEvent(type='OrderUpdated',data=b'data5')client.append_to_stream(stream_name=stream_name2,current_version=0,events=[event5],)# Wait for the subscription to block.wait_for_event(event5)# Stop the subscription.catchup_subscription.stop()thread.join()The example below shows how to subscribe to events recorded after a
particular commit position, in this case from the commit position of
the last recorded event that was received above. Then, another event is
recorded before the subscription is restarted. And three more events are
recorded whilst the subscription is running. These four events are
received in the order they were recorded.# Append another event.event6=NewEvent(type='OrderDeleted',data=b'data6')client.append_to_stream(stream_name=stream_name2,current_version=1,events=[event6],)# Restart subscribing to all events after the# commit position of the last received event.catchup_subscription=client.subscribe_to_all(commit_position=received_events[-1].commit_position)thread=Thread(target=receive_events,daemon=True)thread.start()# Wait for event6.wait_for_event(event6)# Append three more events to a new stream.stream_name3=str(uuid.uuid4())event7=NewEvent(type='OrderCreated',data=b'data7')event8=NewEvent(type='OrderUpdated',data=b'data8')event9=NewEvent(type='OrderDeleted',data=b'data9')client.append_to_stream(stream_name=stream_name3,current_version=StreamState.NO_STREAM,events=[event7,event8,event9],)# Wait for events 7, 8 and 9.wait_for_event(event7)wait_for_event(event8)wait_for_event(event9)# Stop the subscription.catchup_subscription.stop()thread.join()The catch-up subscription call is ended as soon as the subscription object'sstop()method is called. This happens automatically when it goes out of scope,
or when it is explicitly deleted from memory using the Pythondelkeyword.Subscribe to stream eventsThesubscribe_to_stream()method can be used to start a catch-up subscription
from which events recorded in a single stream can be obtained. This method
returns a "catch-up subscription" iterator.This method has a requiredstream_nameargument, which specifies the name of the
stream from which recorded events will be received.This method also has five optional arguments,stream_position,from_end,resolve_links,timeoutandcredentials.The optionalstream_positionargument specifies a position in the stream from
which to start subscribing. The default value ofstream_positionisNone,
which means that all events recorded in the stream will be obtained in the
order they were recorded, unlessfrom_endis set toTrue. If a stream
position is given, then only events recorded after that position will be obtained.The optionalfrom_endargument specifies that the subscription will start
from the last position in the stream. The default value offrom_endisFalse.
Iffrom_endisTrue, then only events recorded after the subscription was
created will be obtained. This argument if ignored isstream_positionis set.The optionalresolve_linksargument is a Pythonbool. The default value ofresolve_linksisFalse, which means any event links will not be resolved, so that the events that are
returned may represent event links. Ifresolve_linksisTrue, any event links will
be resolved, so that the linked events will be returned instead of the event links.The optionaltimeoutargument is a Pythonfloatwhich sets a
maximum duration, in seconds, for the completion of the gRPC operation.The optionalcredentialsargument can be used to
override call credentials derived from the connection string URI.The example below shows how to start a catch-up subscription from
the first recorded event in a stream.# Subscribe from the start of 'stream2'.subscription=client.subscribe_to_stream(stream_name=stream_name2)The example below shows how to start a catch-up subscription from
a particular stream position.# Subscribe to stream2, from the second recorded event.subscription=client.subscribe_to_stream(stream_name=stream_name2,stream_position=1,)How to implement exactly-once event processingThe commit positions of recorded events that are received and processed by a
downstream component are usefully recorded by the downstream component, so that
the commit position of last processed event can be determined when processing is
resumed.The last recorded commit position can be used to specify the commit position from which
to subscribe when processing is resumed. Since this commit position will represent the
position of the last successfully processed event in a downstream component, so it
will be usual to want the next event after this position, because that is the next
event that has not yet been processed. For this reason, when subscribing for events
from a specific commit position using a catch-up subscription in EventStoreDB, the
recorded event at the specified commit position will NOT be included in the sequence
of recorded events that are received.To accomplish "exactly-once" processing of recorded events in a downstream
component when using a catch-up subscription, the commit position of a recorded
event should be recorded atomically and uniquely along with the result of processing
recorded events, for example in the same database as materialised views when
implementing eventually-consistent CQRS, or in the same database as a downstream
analytics or reporting or archiving application. By recording the commit position
of recorded events atomically with the new state that results from processing
recorded events, "dual writing" in the consumption of recorded events can be
avoided. By also recording the commit position uniquely, the new state cannot be
recorded twice, and hence the recorded state of the downstream component will be
updated only once for any recorded event. By using the greatest recorded commit
position to resume a catch-up subscription, all recorded events will eventually
be processed. The combination of the "at-most-once" condition and the "at-least-once"
condition gives the "exactly-once" condition.The danger with "dual writing" in the consumption of recorded events is that if a
recorded event is successfully processed and new state recorded atomically in one
transaction with the commit position recorded in a separate transaction, one may
happen and not the other. If the new state is recorded but the position is lost,
and then the processing is stopped and resumed, the recorded event may be processed
twice. On the other hand, if the commit position is recorded but the new state is
lost, the recorded event may effectively not be processed at all. By either
processing an event more than once, or by failing to process an event, the recorded
state of the downstream component might be inaccurate, or possibly inconsistent, and
perhaps catastrophically so. Such consequences may or may not matter in your situation.
But sometimes inconsistencies may halt processing until the issue is resolved. You can
avoid "dual writing" in the consumption of events by atomically recording the commit
position of a recorded event along with the new state that results from processing that
event in the same atomic transaction. By making the recording of the commit positions
unique, so that transactions will be rolled back when there is a conflict, you will
prevent the results of any duplicate processing of a recorded event being committed.Recorded events received from a catch-up subscription cannot be acknowledged back
to the EventStoreDB server. Acknowledging events, however, is an aspect of "persistent
subscriptions". Hoping to rely on acknowledging events to an upstream
component is an example of dual writing.Persistent subscriptionsIn EventStoreDB, "persistent" subscriptions are similar to catch-up subscriptions,
in that reading a persistent subscription will block when there are no more recorded
events to be received, and then continue when new events are subsequently recorded.Persistent subscriptions can be consumed by a group of consumers operating with one
of the supported "consumer strategies".The significant different with persistent subscriptions is the server will keep track
of the progress of the consumers. The consumer of a persistent subscription will
therefore need to "acknowledge" when a recorded event has been processed successfully,
and otherwise "negatively acknowledge" a recorded event that has been received but was
not successfully processed.All of this means that for persistent subscriptions there are "create", "read", "update"
"delete", "ack", and "nack" operations to consider.Whilst there are some advantages of persistent subscriptions, in particular the
concurrent processing of recorded events by a group of consumers, by tracking in
the server the position in the commit sequence of events that have been processed,
the issue of "dual writing" in the consumption of events arises. Reliability in the
processing of recorded events by a group of persistent subscription consumers will
rely on their idempotent handling of duplicate messages, and their resilience to
out-of-order delivery.Create subscription to allrequires leaderThecreate_subscription_to_all()method can be used to create a "persistent subscription"
to all the recorded events in the database across all streams.This method has a requiredgroup_nameargument, which is the
name of a "group" of consumers of the subscription.This method has twelve optional arguments,from_end,commit_position,resolve_links,filter_exclude,filter_include,filter_by_stream_name,consumer_strategy,message_timeout,max_retry_count,max_subscriber_count,timeoutandcredentials.The optionalfrom_endargument can be used to specify that the group of consumers
of the subscription should only receive events that were recorded after the subscription
was created.Alternatively, the optionalcommit_positionargument can be used to specify a commit
position from which the group of consumers of the subscription should
receive events. Please note, the recorded event at the specified commit position might
be included in the recorded events received by the group of consumers.If neitherfrom_endnorcommit_positionare specified, the group of consumers
of the subscription will potentially receive all recorded events in the database.The optionalresolve_linksargument is a Pythonbool. The default value ofresolve_linksisFalse, which means any event links will not be resolved, so that the events that are
returned may represent event links. Ifresolve_linksisTrue, any event links will
be resolved, so that the linked events will be returned instead of the event links.The optionalfilter_excludeargument is a sequence of regular expressions that
specifies which recorded events should be returned. This argument is ignored
iffilter_includeis set to a non-empty sequence. The default value of this
argument matches the event types of EventStoreDB "system events", so that system
events will not normally be included. See the Notes section below for more
information about filter expressions.The optionalfilter_includeargument is a sequence of regular expressions
that specifies which recorded events should be returned. By default, this
argument is an empty tuple. If this argument is set to a non-empty sequence,
thefilter_excludeargument is ignored.The optionalfilter_by_stream_nameargument is a Pythonboolthat indicates
whether the filtering will apply to event types or stream names. By default, this
value isFalseand so the filtering will apply to the event type strings of
recorded events.The optionalconsumer_strategyargument is a Pythonstrthat defines
the consumer strategy for this persistent subscription. The value of this argument
can be'DispatchToSingle','RoundRobin','Pinned', or'PinnedByCorrelation'. The
default value is'DispatchToSingle'.The optionalmessage_timeoutargument is a Pythonfloatwhich sets a maximum duration,
in seconds, from the server sending a recorded event to a consumer of the persistent
subscription until either an "acknowledgement" (ack) or a "negative acknowledgement"
(nack) is received by the server, after which the server will retry to send the event.
The default value ofmessage_timeoutis30.0.The optionalmax_retry_countargument is a Pythonintwhich sets the number of times
the server will retry to send an event. The default value ofmax_retry_countis10.The optionalmax_subscriber_countargument is a Pythonintwhich sets the maximum
number of concurrent readers of the persistent subscription, beyond which attempts to
read the persistent subscription will raise aMaximumSubscriptionsReachederror.The optionaltimeoutargument is a Pythonfloatwhich sets a
maximum duration, in seconds, for the completion of the gRPC operation.The optionalcredentialsargument can be used to
override call credentials derived from the connection string URI.The methodcreate_subscription_to_all()does not return a value. Recorded events are
obtained by calling theread_subscription_to_all()method.In the example below, a persistent subscription is created to operate from the
first recorded non-system event in the database.# Create a persistent subscription.group_name1=f"group-{uuid.uuid4()}"client.create_subscription_to_all(group_name=group_name1)Read subscription to allrequires leaderTheread_subscription_to_all()method can be used by a group of consumers to receive
recorded events from a persistent subscription that has been created using
thecreate_subscription_to_all()method.This method has a requiredgroup_nameargument, which is
the name of a "group" of consumers of the subscription specified
whencreate_subscription_to_all()was called.This method has an optionaltimeoutargument, which is a Pythonfloatthat sets a maximum duration, in seconds, for the completion of the gRPC operation.This method has an optionalcredentialsargument, which can be used to
override call credentials derived from the connection string URI.This method returns aPersistentSubscriptionobject, which is an iterator
givingRecordedEventobjects. It also hasack(),nack()andstop()methods.subscription=client.read_subscription_to_all(group_name=group_name1)Theack()method should be used by a consumer to "acknowledge" to the server that
it has received and successfully processed a recorded event. This will prevent that
recorded event being received by another consumer in the same group. Theack()has anitemargument which can be either aRecordedEventor aUUID. If you pass
in aRecordedEvent, the value of itsack_idattribute will be used to acknowledge
the event to the server. If you pass in a UUID, then used the value of theack_idof theRecordedEventthat is being acknowledged, in case the event has been resolved
from a link event.The example below iterates over the subscription object, and callsack()with the
receivedRecordedEventobjects. The subscription'sstop()method is called when
we have receivedevent9, so that we can continue with the examples below.received_events=[]foreventinsubscription:received_events.append(event)# Acknowledge the received event.subscription.ack(event)# Stop when 'event9' has been received.ifevent==event9:subscription.stop()Thenack()should be used by a consumer to "negatively acknowledge" to the server that
it has received but not successfully processed a recorded event. Thenack()method has
anitemargument that works in the same way asack(). Use the recorded event or itsack_idattribute. Thenack()method also has anactionargument, which should be
a Pythonstr: either'unknown','park','retry','skip'or'stop'.Thestop()method can be used to stop the gRPC streaming operation.How to write a persistent subscription consumerThe reading of a persistent subscription can be encapsulated in a "consumer" that calls
a "policy" function when a recorded event is received and then automatically callsack()if the policy function returns normally, andnack()if it raises an exception,
perhaps retrying the event for a certain number of times before parking the event.The simple example below shows how this might be done. We can see that 'event9' is
acknowledged before 'event5' is finally parked.The  number of time aRecordedEventhas been retried is presented by the itsretry_countattribute.acked_events={}nacked_events={}classExampleConsumer:def__init__(self,subscription,max_retry_count,final_action):self.subscription=subscriptionself.max_retry_count=max_retry_countself.final_action=final_actionself.error=Nonedefrun(self):try:foreventinself.subscription:try:self.policy(event)exceptException:ifevent.retry_count<self.max_retry_count:action="retry"else:action=self.final_actionself.subscription.nack(event,action)self.after_nack(event,action)else:self.subscription.ack(event)self.after_ack(event)exceptException:self.stop()raisedefstop(self):self.subscription.stop()defpolicy(self,event):# Raise an exception when we see "event5".ifevent==event5:raiseException()defafter_ack(self,event):# Track retry count of acked events.acked_events[event.id]=event.retry_countdefafter_nack(self,event,action):# Track retry count of nacked events.nacked_events[event.id]=event.retry_countifaction==self.final_action:# Stop the consumer, so we can continue with the examples.self.stop()# Create subscription.group_name=f"group-{uuid.uuid4()}"client.create_subscription_to_all(group_name,commit_position=commit_position1)# Read subscription.subscription=client.read_subscription_to_all(group_name)# Construct consumer.consumer=ExampleConsumer(subscription=subscription,max_retry_count=5,final_action="park",)# Run consumer.consumer.run()# Check 'event5' was nacked and never acked.assertevent5.idinnacked_eventsassertevent5.idnotinacked_eventsassertnacked_events[event5.id]==5# Check 'event9' was acked and never nacked.assertevent9.idinacked_eventsassertevent9.idnotinnacked_eventsUpdate subscription to allrequires leaderTheupdate_subscription_to_all()method can be used to update a
"persistent subscription". Please note, the filter options and consumer
strategy cannot be adjusted.This method has a requiredgroup_nameargument, which is the
name of a "group" of consumers of the subscription.This method also has eight optional arguments,from_end,commit_position,resolve_links,message_timeout,max_retry_count,max_subscriber_count,timeoutandcredentials.The optionalfrom_endargument can be used to specify that the group of consumers
of the subscription should only receive events that were recorded after the subscription
was updated.Alternatively, the optionalcommit_positionargument can be used to specify a commit
position from which commit position the group of consumers of the subscription should
receive events. Please note, the recorded event at the specified commit position might
be included in the recorded events received by the group of consumers.If neitherfrom_endnorcommit_positionare specified, the group of consumers
of the subscription will potentially receive all recorded events in the database.The optionalresolve_linksargument is a Pythonbool. The default value ofresolve_linksisFalse, which means any event links will not be resolved, so that the events that are
returned may represent event links. Ifresolve_linksisTrue, any event links will
be resolved, so that the linked events will be returned instead of the event links.The optionalmessage_timeoutargument is a Pythonfloatwhich sets a maximum duration,
in seconds, from the server sending a recorded event to a consumer of the persistent
subscription until either an "acknowledgement" (ack) or a "negative acknowledgement"
(nack) is received by the server, after which the server will retry to send the event.
The default value ofmessage_timeoutis30.0.The optionalmax_retry_countargument is a Pythonintwhich sets the number of times
the server will retry to send an event. The default value ofmax_retry_countis10.The optionalmax_subscriber_countargument is a Pythonintwhich sets the maximum
number of concurrent readers of the persistent subscription, beyond which attempts to
read the persistent subscription will raise aMaximumSubscriptionsReachederror.The optionaltimeoutargument is a Pythonfloatwhich sets a
maximum duration, in seconds, for the completion of the gRPC operation.The optionalcredentialsargument can be used to
override call credentials derived from the connection string URI.The methodupdate_subscription_to_all()does not return a value.In the example below, a persistent subscription is updated to run from the end of the
database.# Create a persistent subscription.client.update_subscription_to_all(group_name=group_name1,from_end=True)Create subscription to streamrequires leaderThecreate_subscription_to_stream()method can be used to create a persistent
subscription to a stream.This method has two required arguments,group_nameandstream_name. Thegroup_nameargument names the group of consumers that will receive events
from this subscription. Thestream_nameargument specifies which stream
the subscription will follow. The values of both these arguments are expected
to be Pythonstrobjects.This method also has nine optional arguments,stream_position,from_end,resolve_links,consumer_strategy,message_timeout,max_retry_count,max_subscriber_count,timeoutandcredentials.The optionalstream_positionargument specifies a stream position from
which to subscribe. The recorded event at this stream
position will be received when reading the subscription.The optionalfrom_endargument is a Pythonbool.
By default, the value of this argument isFalse. If this argument is set
toTrue, reading from the subscription will receive only events
recorded after the subscription was created. That is, it is not inclusive
of the current stream position.The optionalresolve_linksargument is a Pythonbool. The default value ofresolve_linksisFalse, which means any event links will not be resolved, so that the events that are
returned may represent event links. Ifresolve_linksisTrue, any event links will
be resolved, so that the linked events will be returned instead of the event links.The optionalconsumer_strategyargument is a Pythonstrthat defines
the consumer strategy for this persistent subscription. The value of this argument
can be'DispatchToSingle','RoundRobin','Pinned', or'PinnedByCorrelation'. The
default value is'DispatchToSingle'.The optionalmessage_timeoutargument is a Pythonfloatwhich sets a maximum duration,
in seconds, from the server sending a recorded event to a consumer of the persistent
subscription until either an "acknowledgement" (ack) or a "negative acknowledgement"
(nack) is received by the server, after which the server will retry to send the event.
The default value ofmessage_timeoutis30.0.The optionalmax_retry_countargument is a Pythonintwhich sets the number of times
the server will retry to send an event. The default value ofmax_retry_countis10.The optionalmax_subscriber_countargument is a Pythonintwhich sets the maximum
number of concurrent readers of the persistent subscription, beyond which attempts to
read the persistent subscription will raise aMaximumSubscriptionsReachederror.The optionaltimeoutargument is a Pythonfloatwhich sets a
maximum duration, in seconds, for the completion of the gRPC operation.The optionalcredentialsargument can be used to
override call credentials derived from the connection string URI.This method does not return a value. Events can be received by callingread_subscription_to_stream().The example below creates a persistent stream subscription from the start of the stream.# Create a persistent stream subscription from start of the stream.group_name2=f"group-{uuid.uuid4()}"client.create_subscription_to_stream(group_name=group_name2,stream_name=stream_name2,)Read subscription to streamrequires leaderTheread_subscription_to_stream()method can be used to read a persistent
subscription to a stream.This method has two required arguments,group_nameandstream_name, which
should match the values of arguments used when callingcreate_subscription_to_stream().This method has an optionaltimeoutargument, which is a Pythonfloatthat sets a maximum duration, in seconds, for the completion of the gRPC operation.This method has an optionalcredentialsargument, which can be used to
override call credentials derived from the connection string URI.This method returns aPersistentSubscriptionobject, which is an iterator
givingRecordedEventobjects, that also hasack(),nack()andstop()methods.subscription=client.read_subscription_to_stream(group_name=group_name2,stream_name=stream_name2,)The example below iterates over the subscription object, and callsack().
The for loop breaks when we have received the last event in the stream, so
that we can finish the examples in this documentation.events=[]foreventinsubscription:events.append(event)# Acknowledge the received event.subscription.ack(event)# Stop when 'event6' has been received.ifevent==event6:subscription.stop()We can check we received all the events that were appended tostream_name2in the examples above.assertlen(events)==3assertevents[0]==event4assertevents[1]==event5assertevents[2]==event6Update subscription to streamrequires leaderTheupdate_subscription_to_stream()method can be used to update a persistent
subscription to a stream. Please note, the consumer strategy cannot be adjusted.This method has a requiredgroup_nameargument, which is the
name of a "group" of consumers of the subscription, and a requiredstream_nameargument, which is the name of a stream.This method also has eight optional arguments,from_end,stream_position,resolve_links,message_timeout,max_retry_count,max_subscriber_count,timeoutandcredentials.The optionalfrom_endargument can be used to specify that the group of consumers
of the subscription should only receive events that were recorded after the subscription
was updated.Alternatively, the optionalstream_positionargument can be used to specify a stream
position from which commit position the group of consumers of the subscription should
receive events. Please note, the recorded event at the specified stream position might
be included in the recorded events received by the group of consumers.If neitherfrom_endnorcommit_positionare specified, the group of consumers
of the subscription will potentially receive all recorded events in the stream.The optionalresolve_linksargument is a Pythonbool. The default value ofresolve_linksisFalse, which means any event links will not be resolved, so that the events that are
returned may represent event links. Ifresolve_linksisTrue, any event links will
be resolved, so that the linked events will be returned instead of the event links.The optionalmessage_timeoutargument is a Pythonfloatwhich sets a maximum duration,
in seconds, from the server sending a recorded event to a consumer of the persistent
subscription until either an "acknowledgement" (ack) or a "negative acknowledgement"
(nack) is received by the server, after which the server will retry to send the event.
The default value ofmessage_timeoutis30.0.The optionalmax_retry_countargument is a Pythonintwhich sets the number of times
the server will retry to send an event. The default value ofmax_retry_countis10.The optionalmax_subscriber_countargument is a Pythonintwhich sets the maximum
number of concurrent readers of the persistent subscription, beyond which attempts to
read the persistent subscription will raise aMaximumSubscriptionsReachederror.The optionaltimeoutargument is a Pythonfloatwhich sets a
maximum duration, in seconds, for the completion of the gRPC operation.The optionalcredentialsargument can be used to
override call credentials derived from the connection string URI.Theupdate_subscription_to_stream()method does not return a value.In the example below, a persistent subscription to a stream is updated to run from the
end of the stream.# Create a persistent subscription.client.update_subscription_to_stream(group_name=group_name2,stream_name=stream_name2,from_end=True,)Replay parked eventsrequires leaderThereplay_parked_events()method can be used to "replay" events that have
been "parked" (negatively acknowledged with the action'park') when reading
a persistent subscription. Parked events will then be received again by consumers
reading from the persistent subscription.This method has a requiredgroup_nameargument and an optionalstream_nameargument. The values of these arguments should match those used when callingcreate_subscription_to_all()orcreate_subscription_to_stream().This method has an optionaltimeoutargument, which is a Pythonfloatthat sets a maximum duration, in seconds, for the completion of the gRPC operation.This method has an optionalcredentialsargument, which can be used to
override call credentials derived from the connection string URI.The example below replays parked events for groupgroup_name1.client.replay_parked_events(group_name=group_name1,)The example below replays parked events for groupgroup_name2.client.replay_parked_events(group_name=group_name2,stream_name=stream_name2,)Get subscription inforequires leaderTheget_subscription_info()method can be used to get information for a
persistent subscription.This method has a requiredgroup_nameargument and an optionalstream_nameargument, which should match the values of arguments used when calling eithercreate_subscription_to_all()orcreate_subscription_to_stream().This method has an optionaltimeoutargument, which is a Pythonfloatthat sets a maximum duration, in seconds, for the completion of the gRPC operation.This method has an optionalcredentialsargument, which can be used to
override call credentials derived from the connection string URI.The example below gets information for the persistent subscriptiongroup_name1which
was created by callingcreate_subscription_to_all().subscription_info=client.get_subscription_info(group_name=group_name1,)The example below gets information for the persistent subscriptiongroup_name2onstream_name2which was created by callingcreate_subscription_to_stream().subscription_info=client.get_subscription_info(group_name=group_name2,stream_name=stream_name2,)The returned value is aSubscriptionInfoobject.List subscriptionsrequires leaderThelist_subscriptions()method can be used to get information for all
existing persistent subscriptions, both "subscriptions to all" and
"subscriptions to stream".This method has an optionaltimeoutargument, which is a Pythonfloatthat sets a maximum duration, in seconds, for the completion of the gRPC operation.This method has an optionalcredentialsargument, which can be used to
override call credentials derived from the connection string URI.The example below lists all the existing persistent subscriptions.subscriptions=client.list_subscriptions()The returned value is a list ofSubscriptionInfoobjects.List subscriptions to streamrequires leaderThelist_subscriptions_to_stream()method can be used to get information for all
the persistent subscriptions to a stream.This method has one required argument,stream_name.This method has an optionaltimeoutargument, which is a Pythonfloatthat sets a maximum duration, in seconds, for the completion of the gRPC operation.This method has an optionalcredentialsargument, which can be used to
override call credentials derived from the connection string URI.subscriptions=client.list_subscriptions_to_stream(stream_name=stream_name2,)The returned value is a list ofSubscriptionInfoobjects.Delete subscriptionrequires leaderThedelete_subscription()method can be used to delete a persistent
subscription.This method has a requiredgroup_nameargument and an optionalstream_nameargument, which should match the values of arguments used when calling eithercreate_subscription_to_all()orcreate_subscription_to_stream().This method has an optionaltimeoutargument, which is a Pythonfloatthat sets a maximum duration, in seconds, for the completion of the gRPC operation.This method has an optionalcredentialsargument, which can be used to
override call credentials derived from the connection string URI.The example below deletes the persistent subscriptiongroup_name1which
was created by callingcreate_subscription_to_all().client.delete_subscription(group_name=group_name1,)The example below deleted the persistent subscriptiongroup_name2onstream_name2which was created by callingcreate_subscription_to_stream().client.delete_subscription(group_name=group_name2,stream_name=stream_name2,)Call credentialsDefault call credentials are derived by the client from the user info part of the
connection string URI.Many of the client methods described above have an optionalcredentialsargument,
which can be used to set call credentials for an individual method call that override
those derived from the connection string URI.Construct call credentialsThe client methodconstruct_call_credentials()can be used to construct a call
credentials object from a username and password.call_credentials=client.construct_call_credentials(username='admin',password='changeit')The call credentials object can be used as the value of thecredentialsargument in other client methods.ConnectionReconnectThereconnect()method can be used to manually reconnect the client to a
suitable EventStoreDB node. This method uses the same routine for reading the
cluster node states and then connecting to a suitable node according to the
client's node preference that is specified in the connection string URI when
the client is constructed. This method is thread-safe, in that when it is called
by several threads at the same time, only one reconnection will occur. Concurrent
attempts to reconnect will block until the client has reconnected successfully,
and then they will all return normally.client.reconnect()An example of when it might be desirable to reconnect manually is when (for performance
reasons) the client's node preference is to be connected to a follower node in the
cluster, and, after a cluster leader election, the follower becomes the leader.
Reconnecting to a follower node in this case is currently beyond the capabilities of
this client, but this behavior might be implemented in a future release.Reconnection will happen automatically in many cases, due to the@autoreconnectdecorator.CloseTheclose()method can be used to cleanly close the client's gRPC connection.client.close()Asyncio clientTheesdbclientpackage also includes an early version of an asynchronous I/O
gRPC Python client. It follows exactly the same behaviors as the multithreadedEventStoreDBClient, but uses thegrpc.aiopackage and theasynciomodule, instead ofgrpcandthreading.The async functionAsyncioEventStoreDBClientconstructs the client, and connects to
a server. It can be imported fromesdbclient, and can be called with the same
arguments asEventStoreDBClient. It supports both the "esdb" and the "esdb+discover"
connection string URI schemes, and can connect to both "secure" and "insecure"
EventStoreDB servers. It reconnects or retries when connection issues or server
errors are encountered.fromesdbclientimportAsyncioEventStoreDBClientThe asynchronous I/O client has the following methods:append_to_stream(),get_stream(),read_all(),subscribe_to_all(),delete_stream(),tombstone_stream(), andreconnect().These methods are equivalent to the methods onEventStoreDBClient. They have the same
method signatures, and can be called with the same arguments, to the same effect.
The methods which appear onEventStoreDBClientbut not onAsyncioEventStoreDBClientwill be
added soon.SynopsisThe example below demonstrates theappend_to_stream(),get_stream()andsubscribe_to_all()methods. These are the most useful methods for writing
an event-sourced application, allowing new aggregate events to be recorded, the
recorded events of an aggregate to be obtained so aggregates can be reconstructed,
and the state of an application to propagated and processed with "exactly-once"
semantics.importasyncioasyncdefdemonstrate_asyncio_client():# Construct client.client=awaitAsyncioEventStoreDBClient(uri=os.getenv("ESDB_URI"),root_certificates=os.getenv("ESDB_ROOT_CERTIFICATES"),)# Append events.stream_name=str(uuid.uuid4())event1=NewEvent("OrderCreated",data=b'{}')event2=NewEvent("OrderUpdated",data=b'{}')event3=NewEvent("OrderDeleted",data=b'{}')commit_position=awaitclient.append_to_stream(stream_name=stream_name,current_version=StreamState.NO_STREAM,events=[event1,event2,event3])# Read stream events.recorded=awaitclient.get_stream(stream_name)assertlen(recorded)==3assertrecorded[0]==event1assertrecorded[1]==event2assertrecorded[2]==event3# Subscribe all events.received=[]asyncforeventinawaitclient.subscribe_to_all():received.append(event)ifevent.commit_position==commit_position:breakassertreceived[-3]==event1assertreceived[-2]==event2assertreceived[-1]==event3# Close the client.awaitclient.close()# Run the demo.asyncio.get_event_loop().run_until_complete(demonstrate_asyncio_client())NotesRegular expression filtersTheread_all(),subscribe_to_all(),create_subscription_to_all()andget_commit_position()methods havefilter_excludeandfilter_includearguments. This section provides some more details about the values of these
arguments.The first thing to note is that the values of these arguments should be sequences
of regular expressions.Please note, they are concatenated together by the client as bracketed alternatives in a larger
regular expression that is anchored to the start and end of the strings being
matched. So there is no need to include the'^'and'$'anchor assertions.You should use wildcards if you want to match substrings, for example'.*Snapshot'to match all strings that end with'Snapshot', or'Order.*'to match all strings
that start with'Order'.System events generated by EventStoreDB havetypestrings that start with
the$sign. Persistence subscription events generated when manipulating
persistence subscriptions havetypestrings that start withPersistentConfig.For example, to match the type of EventStoreDB system events, use the regular
expression stringr'\$.+'. Please note, the constantESDB_SYSTEM_EVENTS_REGEXis
set to this value. You can import this constant fromesdbclientand use it when
building longer sequences of regular expressions.Similarly, to match the type of EventStoreDB persistence subscription events, use the
regular expressionr'PersistentConfig\d+'. The constantESDB_PERSISTENT_CONFIG_EVENTS_REGEXis set to this value. You can import this constant fromesdbclientand use it when
building longer sequences of regular expressions.The constantDEFAULT_EXCLUDE_FILTERis a sequence of regular expressions that includes
bothESDB_SYSTEM_EVENTS_REGEXandESDB_PERSISTENT_CONFIG_EVENTS_REGEX. It is used
as the default value offilter_excludeso that the events generated internally by
EventStoreDB are excluded by default.In all methods that have afilter_excludeargument, the default value of the argument
is the constantDEFAULT_EXCLUDE_FILTER, which is designed to match (and therefore
to exclude) both "system" and "persistence subscription config" event types, which
would otherwise be included.This value can be extended. For example, if you want to exclude system events and
persistent subscription events and also events that have a type that ends with'Snapshot', then you can useDEFAULT_EXCLUDE_FILTER + ['.*Snapshot']as thefilter_excludeargument.Thefilter_includeandfilter_excludearguments are designed to have exactly
the opposite effect from each other, so that a sequence of strings given tofilter_includewill return exactly those events which would be excluded if
the same argument value were used withfilter_exclude. And vice versa, so that
a sequence of strings given tofilter_excludewill return exactly those events
that would not be included if the same argument value were used withfilter_include.Reconnect and retry method decoratorsPlease note, nearly all the client methods are decorated with the@autoreconnectand
the@retrygrpcdecorators.The@autoreconnectdecorator will reconnect to a suitable node in the cluster when
the server to which the client has been connected has become unavailable, or when the
client's gRPC channel happens to have been closed. The client will also reconnect when
a method is called that requires a leader, and the client's node preference is to be
connected to a leader, but the node that the client has been connected to stops being
the leader. In this case, the client will reconnect to the current leader. After
reconnecting, the failed operation will be retried.The@retrygrpcdecorator selectively retries gRPC operations that have failed due to
a timeout, network error, or server error. It doesn't retry operations that fail due to
bad requests that will certainly fail again.Please also note, the aspects not covered by the reconnect and retry decorator
behaviours have to do with methods that return iterators. For example, consider
the "read response" iterator returned from theread_all()method. Theread_all()method will have returned, and the method decorators will therefore
have exited, before iterating over the "read response" begins. Therefore, if a
connection issues occurs whilst iterating over the "read response", it isn't possible
for any decorator on theread_all()method to trigger a reconnection.With the "catch-up subscription" objects, there is an initial "confirmation" response
from the server which is received and checked by the client. And so, when a call is
made tosubscribe_to_all()orsubscribe_to_stream(), if the server is unavailable,
or if the channel has somehow been closed, or if the request fails for some other reason,
then the client will reconnect and retry. However, if an exception is raised when iterating over a
successfully returned "catch-up subscription" object, the catch-up subscription will
need to be restarted. Similarly, when reading persistent subscriptions, if there are
connection issues whilst iterating over a successfully received response, the consumer
will need to be restarted.ContributorsInstall PoetryThe first thing is to check you have Poetry installed.$ poetry --versionIf you don't, then pleaseinstall Poetry.$ curl -sSL https://install.python-poetry.org | python3 -It will help to make sure Poetry's bin directory is in yourPATHenvironment variable.But in any case, make sure you know the path to thepoetryexecutable. The Poetry
installer tells you where it has been installed, and how to configure your shell.Please refer to thePoetry docsfor guidance on
using Poetry.Setup for PyCharm usersYou can easily obtain the project files using PyCharm (menu "Git > Clone...").
PyCharm will then usually prompt you to open the project.Open the project in a new window. PyCharm will then usually prompt you to create
a new virtual environment.Create a new Poetry virtual environment for the project. If PyCharm doesn't already
know where yourpoetryexecutable is, then set the path to yourpoetryexecutable
in the "New Poetry Environment" form input field labelled "Poetry executable". In the
"New Poetry Environment" form, you will also have the opportunity to select which
Python executable will be used by the virtual environment.PyCharm will then create a new Poetry virtual environment for your project, using
a particular version of Python, and also install into this virtual environment the
project's package dependencies according to the project'spoetry.lockfile.You can add different Poetry environments for different Python versions, and switch
between them using the "Python Interpreter" settings of PyCharm. If you want to use
a version of Python that isn't installed, either use your favourite package manager,
or install Python by downloading an installer for recent versions of Python directly
from thePython website.Once project dependencies have been installed, you should be able to run tests
from within PyCharm (right-click on thetestsfolder and select the 'Run' option).Because of a conflict between pytest and PyCharm's debugger and the coverage tool,
you may need to add--no-covas an option to the test runner template. Alternatively,
just use the Python Standard Library'sunittestmodule.You should also be able to open a terminal window in PyCharm, and run the project's
Makefile commands from the command line (see below).Setup from command lineObtain the project files, using Git or suitable alternative.In a terminal application, change your current working directory
to the root folder of the project files. There should be a Makefile
in this folder.Use the Makefile to create a new Poetry virtual environment for the
project and install the project's package dependencies into it,
using the following command.$ make install-packagesIt's also possible to also install the project in 'editable mode'.$ make installPlease note, if you create the virtual environment in this way, and then try to
open the project in PyCharm and configure the project to use this virtual
environment as an "Existing Poetry Environment", PyCharm sometimes has some
issues (don't know why) which might be problematic. If you encounter such
issues, you can resolve these issues by deleting the virtual environment
and creating the Poetry virtual environment using PyCharm (see above).Project Makefile commandsYou can start EventStoreDB using the following command.$ make start-eventstoredbYou can run tests using the following command (needs EventStoreDB to be running).$ make testYou can stop EventStoreDB using the following command.$ make stop-eventstoredbYou can check the formatting of the code using the following command.$ make lintYou can reformat the code using the following command.$ make fmtTests belong in./tests. Code-under-test belongs in./esdbclient.Edit package dependencies inpyproject.toml. Update installed packages (and thepoetry.lockfile) using the following command.$ make update-packages