pyzmqrpc3 3.2.2
A simple ZMQ RPC extension with command/service architecture
License: MIT
Keywords: zmq,zeromq,pyzmq,rpc

pyzmqrpc3
=========

!`Publish <https://github.com/brgirgis/pyzmqrpc3/workflows/Publish/badge.svg>`_

!`Test <https://github.com/brgirgis/pyzmqrpc3/workflows/Test/badge.svg>`_

# Introduction
==============

This Python package adds basic Remote Procedure Call (RPC) functionalities to

ZeroMQ.

The supported command/service architecture allows for complex serialization of

user defined data and modern-looking implementation.

# Install
=========

	pip install pyzmqrpc3

# Usage
=======

Implement a concrete class of the interface class `ICommand` that can

de/serialize itself and has a default constructor

(i.e. can be constructed without any arguments):

	from typing import Optional

	from zmqrpc import ICommand

	class SimpleCommand(ICommand):

		def \_\_init\_\_(

				self,

				param1: Optional[str] = None,

				param2: Optional[str] = None):

			super().\_\_init\_\_()

			self.\_\_param1 = param1 or ''

			self.\_\_param2 = param2 or ''

		@property

		def param1(self) \-> str:

			return self.\_\_param1

		@property

		def param2(self) \-> str:

			return self.\_\_param2

		def set\_command\_state(self, state: dict) \-> None:

			self.\_\_param1 = state['param1']

			self.\_\_param2 = state['param2']

		def get\_command\_state(self) \-> dict:

			return dict(

				param1=self.param1,

				param2=self.param2,

			)

The two methods, `set*command*state()` and `get*command*state()`, are

essential for marshaling the command data between the client and the server.

It is the user's responsibility to make sure that the implementation of these

methods is correct to avoid any data loss.

Both the client and the server side need to be aware of all the system

commands' implementations.

Implement a concrete service functor which inherits from `IService` and

handles one kind of a command by the server:

	from typing import Optional

	from zmqrpc import IService

	class SimpleService(IService):

		def \_\_call\_\_(self, command: SimpleCommand) \-> Optional[object]:

			print(

				'SimpleCommand executed with params "{0}" and "{1}"'.format(

					command.param1,

					command.param2,

				)

			)

			return 'SimpleService response text for SimpleCommand is "%s"' % str(

				dict(

					param1=command.param1,

					param2=command.param2,

				)

			)

Although it is technically possible to make one service to handle more

than one command,

it is highly recommended from architecture point of view to dedicate

one service for one command type.

Services' implementations need not to be visible on the client side

from code organization point of view.

On the server side, create a ZeroMQ RPC server:

	from zmqrpc import ZmqRpcServer

	server = ZmqRpcServer(

		zmq\_rep\_bind\_address='tcp://\*:30000',

	)

Register all the services:

	server.register\_service(

		command\_class=SimpleCommand,

		service=SimpleService(),

	)

Note that this call takes the ***actual*** command class and an ***instance***

of the service functor.

After registering all the services, start the RPC server:

	server.start()

On the client side, create a client that connects to that server endpoint:

	client = ZmqRpcClient(

		zmq\_req\_endpoints=['tcp://localhost:30000'],

	)

Have the client execute commands on the server:

	client.execute\_remote(

		command=SimpleCommand(param1='value1', param2='value2'),

	)

For more examples, take a look at the `examples <./examples>`_ directory.

Even more examples can be found in the `tests <./tests>`_ directory.

# Rationale
===========

Working with ZeroMQ is great!

It is fun, fast and simply works.

It can be used with many applications out of the box with minimal effort.

However, there is no clear structure for the RPC workflow.

This package is a lightweight layer to bridge this gap with minimal restrictions

on what we can already do with the barebone ZMQ.

# Requirements
==============

1. It should be possible to create a network by simply starting apps and

configure them with the location of the end-points.

The apps will typically be started on a process level, however,

threading should also be supported.

2. Must have support for PUB/SUB (non-reliable, whoever is listening) and

REQ/REP sockets (reliable).

The latter should have support for timeouts and automatic recreation of a

REQ socket if no message is received in the timeout period.

3. If somewhere in the network there is connection failing, messages

should be automatically queued up to a certain queue size.

Right now, this has been implemented on the PUB/SUB interface.

4. Password protection is important when traversing non-secure networks.

However, no CURVE based protection is needed for now, just simple

username/password.

The fact that a network can be sniffed is not relevant for general use cases.

5. Since it is common to use a lot of devices together, like Raspberry devices,

it shall be able to work around via proxy connections.

# Components
============

## ZmqReceiver/Thread
=====================

Starts a loop listening via a SUB or REP socket for new messages.

Multiple SUB end-points may be provided.

If a message is received, it calls the `handle*incoming*message()` method

which can be overridden by any subclassed implementation.

The thread version, `ZmqReceiverThread`, can be used for testing or with

applications that might be running multiple server threads.

## ZmqSender
============

Upon creation it starts a PUB socket and/or creates a REQ socket.

The REQ socket may point to multiple end-points, which then use round-robin

message delivery.

The ZmqSender implements the `send()` method that sends a message.

## ZmqProxy
===========

Forwards messages from a SUB --> REQ socket or from a PUB --> REP socket using

a receiver/sender pair.

## ZmqRpcServer/Thread
======================

Implements service(s) that can be remotely executed by receiving a distinct

command type.

It inherits the `ZmqReceiver` functionality to listen for messages on a

REP or SUB socket.

However, it overrides the `handle*incoming*message()` method to deserialize

command messages, identify their type and execute the corresponding service

implementation.

The thread version, `ZmqRpcServerThread`, can be used for testing or with

applications that might be running multiple server threads.

## ZmqRpcClient
===============

Executes a remotely implemented service over a PUB or REQ socket using a

command argument.

For PUB sockets, no response messages should be expected.

It inherits the `ZmqSender` functionality to send messages over the wire.

## ICommand
===========

The base interface class for all concrete command types.

It enforces the implementation of two methods; `set*command*state()` and

`get*command*state()`.

These two methods are essential in marshaling any complex user data from the

client side to the server side.

## IService
===========

The base interface class for all concrete service functors.

It enforces the implementation of the `**call**()` method which is the entry

point of handling a certain command type on the server side.

# Available Standard Proxies
============================

A number of proxies are available out of the box:

* REQ to REQ by means of ZmqProxySub2Req/Thread

* SUB to PUB by means of ZmqProxySub2Pub/Thread

* REP to PUB by means of ZmqProxyRep2Pub/Thread

* REP to REQ by means of ZmqProxyRep2Req/Thread

* Buffered REP to REQ via ZmqBufferedProxyRep2ReqThread

Each of these proxies/proxy-threads will take a message from the input

format/socket and marshal it to the output socket.

One model could be to collect all samples from all sub-processes on a site

and multiplex them via the proxy in a reliable manner over a REP/REQ socket.

Note that when a PUB/SUB connection is used, there is no return message or

in case of method invocation, any function response is discarded.

The buffered REP/REQ proxy quietly uses a PUB/SUB socket to introduce a

means to buffer messages and method invocations.

# Known Issues and Limitations (KIL)
====================================

* Only localhost type of testing done with passwords.

Not sure if auth works over remote connections.

* The `inproc://` transport of ZMQ is not supported by current implementation.

# Notes
=======

Please note that this implementation is very pre-mature, although it works

fine for the projects it is currently being used in and has operated stable

for months.


For more info visit the project repository at
https://github.com/brgirgis/pyzmqrpc3