Package: python-liftbridge
python-liftbridgeThis project is under development.Python client forLiftbridge, a system that provides lightweight, fault-tolerant message streams forNATS.Liftbridge provides the following high-level features:Log-based API for NATSReplicated for fault-toleranceHorizontally scalableWildcard subscription supportAt-least-once delivery support and message replayMessage key-value supportLog compaction by keyInstallation$ pip install python-liftbridgeBasic Usagefrompython_liftbridgeimportLift,Message,Stream,ErrStreamExists# Create a Liftbridge client.client=Lift(ip_address='localhost:9292',timeout=5)# Create a Liftbridge stream with name "foo-stream"try:client.create_stream(Stream(subject='foo',name='foo-stream'))exceptErrStreamExists:print('This stream already exists!')# Publish a message to the stream with the name "foo-stream".client.publish(Message(value='hello',stream='foo-stream'))# Subscribe to the stream starting from the beginning.formessageinclient.subscribe(Stream(subject='foo',name='foo-stream',).start_at_earliest_received(),):print("Received: '{}'".format(message.value))Create StreamStreamsare a durable message log attached to a NATS subject. They record messages published to the subject for consumption.Streams have a few key properties: a subject, which is the corresponding NATS subject, a name, which is a human-readable identifier for the stream, and a replication factor, which is the number of nodes the stream should be replicated to for redundancy.  Optionally, there is a group which is the name of a load-balance group for the stream to join. When there are multiple streams in the same group, messages will be balanced among them."""Create a stream attached to the NATS subject "foo.*" that is replicated toall the brokers in the cluster. ErrStreamExists is returned if a stream withthe given name already exists for the subject."""client.create_stream(Stream(subject='foo.*',name='my-stream',max_replication=True))Subscription Start/Replay OptionsSubscriptionsare how Liftbridge streams are consumed. Clients can choose where to start consuming messages from in a stream. This is controlled using options passed to Subscribe.# Subscribe starting with new messages only.client.subscribe(Stream(subject='foo',name='foo-stream'))# Subscribe starting with the most recently published value.client.subscribe(Stream(subject='foo',name='foo-stream').start_at_earliest_received())# Subscribe starting with the oldest published value.client.subscribe(Stream(subject='foo',name='foo-stream').start_at_latest_received())# Subscribe starting at a specific offset.client.subscribe(Stream(subject='foo',name='foo-stream').start_at_offset(4))# Subscribe starting at a specific time.client.subscribe(Stream(subject='foo',name='foo-stream').start_at_time(datetime.now()))# Subscribe starting at a specific amount of time in the past.client.subscribe(Stream(subject='foo',name='foo-stream').start_at_time_delta(timedelta(days=1)))PublishingA publish API is provided to make it easy to write messages to streams. This includes a number of options for decorating messages with metadata like a message key.Keys are used by Liftbridge's log compaction. When enabled, Liftbridge streams will retain only the last message for a given key.# Publish a message with a keyclient.publish(Message(stream='foo-stream',value='Hello',key='key'))Also, it is possible to publish a message to the NATS subject (and, in turn, any streams that match the subject).# Publish a message to the NATS subjectclient.publish_to_subject(Message(subject='foo',value='Hello foo'))Publishing Directly with NATSSince Liftbridge is an extension ofNATS, aNATS clientcan also be used to publish messages. This means existing NATS publishers do not need any changes for messages to be consumed in Liftbridge.How to contributeCheck for open issues or open a fresh issue to start a discussion around a feature idea or a bug.Forkthe repositoryon GitHub to start making your changes to the master branch (or branch off of it).Write a test which shows that the bug was fixed or that the feature works as expected.Send apull requestand bugmeuntil it gets merged and published.Some things on the backlog:Add documentation (Sphynx)Add CI (CircleCI or TravisCI)Add testsAdd code coverageAdd TLS support for gRPCAdd message headers supportAdd message ACK support (scaffolding is already done)Add method to close connectionAdd async clientAdd gRPC connection poolAdd logging (and remove all the random prints)Add proper docstringsAdd version fileAdd Contributing.md and explanation of the workflow (pyenv,tox,make,pre-commit...)Improve fetch metadataImprove error handlingAdd to the makefile run-liftbridge using DockercontainerBetter instrumentation/observability (OpenCensus support?)