callosum 1.0.3
Callosum RPC Library
License: MIT
Requires Python: >=3.11
Keywords: asyncio,rpc,pubsub,messaging

Callosum
========

An RPC Transport Library

It provides an asynchronous multi-channel order-preserving message and data
streaming transport for *upper* RPC layers (e.g., Apache Thrift) by wrapping
*lower* transport implementations (e.g., ZeroMQ).

It aims to follow the latest coding style and conventions in Python asyncio.

*Corpus callosum* is a bundle of neuron fibers that connects two cerebral
hemispheres of a human brain.

Prerequisite
------------

Python 3.11 or higher.

Features
--------

* RPC
  - Native timeout and cancellation support
  - Explicit server-to-client error propagation including stringified tracebacks
  - Order preserving based on user-defined keys while keeping executions asynchronous
* Streaming
  - Broadcast & shared pipelines
* Optional client authentication and encrypted communication
  - Currently supported for only ZeroMQ with its CURVE library
* Optional message compression using [snappy](https://pypi.org/project/python-snappy/)
* Replacible and combinable lower/upper layers (ZeroMQ/Redis + JSON/msgpack/Thrift)

Planned features
----------------

* Managed streaming (with acks)
* Tunneling to bundle other channels and generic network traffic in a single connection
* Bidirectional RPC
* Chunked transfer of large messages

Installation
------------

To install the core:

```console
$ pip install -U pip setuptools
$ pip install callosum
```

You may add extra dependencies like:

```console
$ pip install 'callosum[zeromq,redis,thrift,snappy]'
```

Examples
--------

Please check out [the examples directory.](https://github.com/lablup/callosum/tree/master/examples)

Development
-----------

Create a virtual environment or an isolated Python environment using your favorite tool.

Inside it, run editable installation as follows:

```console
$ pip install -U pip setuptools
$ pip install -U -r requirements/dev.txt
```

# Changes

<!--
    You should *NOT* be adding new change log entries to this file, this
    file is managed by towncrier. You *may* edit previous change logs to
    fix problems like typo corrections or such.

    To add a new change log entry, please refer
    https://pip.pypa.io/en/latest/development/contributing/#news-entries

    We named the news folder "changes".

    WARNING: Don't drop the last line!
-->

<!-- towncrier release notes start -->

## 1.0.3 (2024-03-04)

### Fixes
* Relax yarl version requirements to avoid potential dependency conflicts in the downstream Backend.AI platform ([#28](https://github.com/lablup/callosum/issues/28))


## 1.0.2 (2024-02-29)

### Fixes
* Stabilize ZMQ RPC auth and malformed packet handling ([#27](https://github.com/lablup/callosum/issues/27))

### Miscellaneous
* Update dependencies including pyzmq ('&gt;=23' &rarr; '&gt;=25.1.1' to support Python 3.12) and development tools including ruff to replace the black formatter ([#26](https://github.com/lablup/callosum/issues/26))


## v1.0.1 (2023-09-18)

### Fixes
* Prevent leaking secret keys in the logs and allow infinite timeouts on connection pings ([#24](https://github.com/lablup/callosum/issues/24))


## v1.0.0 (2023-09-14)

### Breaking change
* Now it requires Python 3.11 to work!

### Features
* Implement the full version of secure, encrypted RPC communication based on ZeroMQ's ZAP protocol using CURVE keypairs ([#21](https://github.com/lablup/callosum/issues/21))


## v0.9.10 (2022-02-17)

### Fixes
* Fix pyzmq attribute error on Ubuntu 20.04 at aarch64, which is built using older libzeromq without some socket monitoring event constants, by loading the constant declarations dynamically ([#20](https://github.com/lablup/callosum/issues/20))


## v0.9.9 (2021-10-05)

### Fixes
* Add explicit `task_done()` call to the output queue of `rpc.Peer` instances to avoid potential missing wakeup on joined coroutines ([#16](https://github.com/lablup/callosum/issues/16))


## v0.9.8 (2021-10-05)

### Features
* lower.zeromq: Add a transport option to attach monitors for logging underlying socket events ([#17](https://github.com/lablup/callosum/issues/17))

### Fixes
* Improve `zsock_opts` handling when explicitly specified by the library users, as previously it caused argument errors in binders and connectors ([#18](https://github.com/lablup/callosum/issues/18))


## v0.9.7 (2020-12-22)

### Features
* Provide `repr()` of exceptions in RPC user/internal errors for better application-level error logging ([#15](https://github.com/lablup/callosum/issues/15))


## v0.9.6 (2020-06-05)

### Features
* `upper.rpc`: Support server-side cancellation and propagation to clients by adding the `CANCELLED` RPC message type ([#14](https://github.com/lablup/callosum/issues/14))

### Deprecations
* Drop use of aiojobs in favor of native semaphores for limiting the task concurrency and less clutters on job scheduling semantics ([#14](https://github.com/lablup/callosum/issues/14))

### Fixes
* Stability updates for the RPC layer: ([#14](https://github.com/lablup/callosum/issues/14))
  - Fix wrong message sequence calcuation with `SEQ_BITS` and clarify the roles of `cleanup()` and `cancel()` methods in the schedulers.
  - Now we use the exit-ordered scheduler by default.


## v0.9.5 (2020-05-12)

### Fixes
* lower.zeromq: Use destroy() for zmq context termination to improve stability and shutdown open socekts cleanly ([#13](https://github.com/lablup/callosum/issues/13))


## v0.9.4 (2020-04-10)

### Fixes
* Fix a race condition due to overlapping RPC message sequence IDs by separating server/client message sequence IDs [(#12)](https://github.com/lablup/callosum/issues/12)

### Miscellaneous
* Adopt towncrier for changelog management [(#11)](https://github.com/lablup/callosum/issues/11)


## v0.9.3 (2020-03-20)

* FIX: Mitigate bogus KeyError when cleaning up task futures in RPC servers that has caused event
  loop spinning.


## v0.9.2 (2020-02-28)

* MAINTENANCE: Update dependencies and only specify the minimum versions since
  Callosum is a library.


## v0.9.1 (2020-01-05)

* FIX: wrong typing of `RPCMessage.body` field

* IMPROVE: Add `debug_rpc` option to `rpc.Peer` for logging exceptions in RPC
  scheduler and user-defined handlers explicitly.

* Update dependencies and remove unused ones.


## v0.9.0 (2019-12-06)

* First public release with a working RPC based on ZeroMQ DEALER/ROUTER sockets.


## 2018-05-02

* Started the project.