pycapnp-async 0.7.0
A cython wrapping of the C++ Cap'n Proto library with support for asyncio
License: BSD
Keywords: capnp,capnproto,Cap'n Proto,pycapnp,pycapnp-async

pycapnp-async
=============

|Actions Status|

Requirements
------------

-  C++14 supported compiler

   -  gcc 6.1+ (5+ may work)
   -  clang 6 (3.4+ may work)
   -  Visual Studio 2017+

-  cmake (needed for bundled capnproto)

   -  ninja (macOS + Linux)
   -  Visual Studio 2017+

-  capnproto-0.7.0

   -  Not necessary if using bundled capnproto

32-bit Linux requires that capnproto be compiled with ``-fPIC``. This is
usually set correctly unless you are compiling canproto yourself. This
is also called ``-DCMAKE_POSITION_INDEPENDENT_CODE=1`` for cmake.

pycapnp has additional development dependencies, including cython and
pytest. See requirements.txt for them all.

Building and installation
-------------------------

Install with ``pip install pycapnp``. You can set the CC environment
variable to control which compiler is used, ie
``CC=gcc-8.2 pip install pycapnp``.

Or you can clone the repo like so:

.. code:: bash

   git clone https://github.com/haata/pycapnp-async.git
   cd pycapnp-async
   pip install .

If you wish to install using the latest upstream C++ Cap’n Proto:

.. code:: bash

   pip install \
       --install-option "--libcapnp-url" \
       --install-option "https://github.com/sandstorm-io/capnproto/archive/master.tar.gz" \
       --install-option "--force-bundled-libcapnp" .

To force bundled python:

.. code:: bash

   pip install --install-option "--force-bundled-libcapnp" .

Python Versions
---------------

Python 3.7+ is supported. Earlier versions of Python have asyncio bugs
that might be possible to work around, but may require significant work
(3.5 and 3.6).

Development
-----------

Git flow has been abandoned, use master.

To test, use a pipenv (or install requirements.txt and run pytest
manually).

.. code:: bash

   pip install pipenv
   pipenv install
   pipenv run pytest

Binary Packages
~~~~~~~~~~~~~~~

Building a dumb binary distribution:

.. code:: bash

   python setup.py bdist_dumb

Building a Python wheel distributiion:

.. code:: bash

   python setup.py bdist_wheel

Pypi Upload Instructions
~~~~~~~~~~~~~~~~~~~~~~~~

Only necessary if uploading release to pypi.org.

TODO

Documentation/Example
---------------------

There is some basic documentation
`here <http://jparyani.github.io/pycapnp/>`__.

The examples directory has one example that shows off pycapnp quite
nicely. Here it is, reproduced:

.. code:: python

   from __future__ import print_function
   import os
   import capnp

   import addressbook_capnp

   def writeAddressBook(file):
       addresses = addressbook_capnp.AddressBook.new_message()
       people = addresses.init('people', 2)

       alice = people[0]
       alice.id = 123
       alice.name = 'Alice'
       alice.email = 'alice@example.com'
       alicePhones = alice.init('phones', 1)
       alicePhones[0].number = "555-1212"
       alicePhones[0].type = 'mobile'
       alice.employment.school = "MIT"

       bob = people[1]
       bob.id = 456
       bob.name = 'Bob'
       bob.email = 'bob@example.com'
       bobPhones = bob.init('phones', 2)
       bobPhones[0].number = "555-4567"
       bobPhones[0].type = 'home'
       bobPhones[1].number = "555-7654"
       bobPhones[1].type = 'work'
       bob.employment.unemployed = None

       addresses.write(file)


   def printAddressBook(file):
       addresses = addressbook_capnp.AddressBook.read(file)

       for person in addresses.people:
           print(person.name, ':', person.email)
           for phone in person.phones:
               print(phone.type, ':', phone.number)

           which = person.employment.which()
           print(which)

           if which == 'unemployed':
               print('unemployed')
           elif which == 'employer':
               print('employer:', person.employment.employer)
           elif which == 'school':
               print('student at:', person.employment.school)
           elif which == 'selfEmployed':
               print('self employed')
           print()


   if __name__ == '__main__':
       f = open('example', 'w')
       writeAddressBook(f)

       f = open('example', 'r')
       printAddressBook(f)

Also, pycapnp has gained RPC features that include pipelining and a
promise style API. Refer to the calculator example in the examples
directory for a much better demonstration:

.. code:: python

   import capnp
   import socket

   import test_capability_capnp


   class Server(test_capability_capnp.TestInterface.Server):

       def __init__(self, val=1):
           self.val = val

       def foo(self, i, j, **kwargs):
           return str(i * 5 + self.val)


   def server(write_end):
       server = capnp.TwoPartyServer(write_end, bootstrap=Server(100))


   def client(read_end):
       client = capnp.TwoPartyClient(read_end)

       cap = client.bootstrap()
       cap = cap.cast_as(test_capability_capnp.TestInterface)

       remote = cap.foo(i=5)
       response = remote.wait()

       assert response.x == '125'


   if __name__ == '__main__':
       read_end, write_end = socket.socketpair(socket.AF_UNIX)
       # This is a toy example using socketpair.
       # In real situations, you can use any socket.

       server(write_end)
       client(read_end)

.. |Actions Status| image:: https://github.com/haata/pycapnp-async/workflows/Python%20Test%20Packaging/badge.svg
   :target: https://github.com/haata/pycapnp-async/actions

Changelog
=============
v0.6.4 (2019-01-31)
-------------------

-  Fix bugs in ``read_multiple_bytes`` (thanks to @tsh56)
-  Remove end-of-life Python versions 2.6, 3.2, and 3.3. Add CI tests
   for 3.6
-  Expose SchemaParser in Cython header

v0.6.3 (2018-01-14)
-------------------

-  Bump bundled capnp version to v0.6.1 (thanks to @E8Yuval)
-  Fix a memleak in RemotePromise (thanks to @E8Yuval)

v0.6.2 (2017-11-30)
-------------------

-  Add support for buffers/memoryviews in ``from_bytes`` (thanks to
   @aldanor)

v0.6.1 (2017-07-27)
-------------------

-  Fixed upload to PyPi (forgot to cythonize)

v0.6.0 (2017-07-27)
-------------------

-  Update bundled capnp version to v0.6.0 and fix related problems
   (thanks to @benmoran)
-  Fix memleak with KjException (thanks to @tsh56)

v0.5.12 (2017-04-18)
--------------------

-  Bump bundled capnp version to v0.5.3.1

v0.5.11 (2017-04-10)
--------------------

-  Make enums hashable (thanks to @madeleine-empirical)
-  Rework logic on when to build bundled libcapnp. Fixes
   cross-compilation (thanks to @benizl)
-  Add traversal_limit_in_words and nesting_limit to RPC classes (thanks
   to @asilversempirical)
-  Include class attributes in **dir**. This allows for code completion
   of class methods (thanks to @chaoflow )
-  Allow setting lists with python tuples (thanks to @chaoflow)
-  Fix traversal_limit_in_words and nesting_limit being ignored by
   ``from_bytes`` (thanks to @plesner)

v0.5.10 (2016-11-28)
--------------------

-  Fix bug that prevented event loop from actually being lazy
   initialized
-  Fix possible recursive loop in KjException
-  Add ``clear_write_flag`` method to builder classes

v0.5.9 (2016-07-07)
-------------------

-  Make the event loop be lazy initialized
-  Add support for segment (de)serialization (thanks to @gcv). See
   to_segments/from_segments methods.
-  Fix response objects not referencing parents correctly
-  Add test for large reads

v0.5.8 (2016-05-27)
-------------------

-  Fix build problem with Cython v0.24
-  Include the changelog in the manifest (should fix install problems if
   pandoc is present)
-  Include the traceback in exceptions
-  Make sure to encode to utf-8, not the default encoding (thanks to
   @novas0x2a)
-  Add –libcapnp-url option in installer to allow installing arbitrary
   libcapnp versions
-  Support mmap objects for reading with from_bytes (thanks to
   @bpiwowar)
-  Change read_multiple and read_multiple_packed to copy by default
-  Fix mistakenly discarding the file parameter on reads
-  Add reraise_kj_exception to the prettyPrint functions. (thanks to
   @kdienes)
-  Fix KjException init (missing wrapper). (thanks to @E8-Storage)
-  Add ``result_type`` to InterfaceMethodSchema

v0.5.7 (2015-06-16)
-------------------

-  Update bundled libcapnp to v0.5.2
-  Add warnings for using old restorer methods. You should use
   ``bootstrap`` instead
-  Fix warning from PyEventPort
-  Handle AnyPointers better as arguments to RPC functions
-  Add support for using keyword arguments with a named struct in an RPC
-  Add bootstrap method to TwoPartyServer
-  Add ``init`` method to lists
-  Add support for unix sockets in RPC

v0.5.6 (2015-04-13)
-------------------

-  Fix a serious bug in TwoPartyServer that was preventing it from
   working when passed a string address.
-  Fix bugs that were exposed by defining KJDEBUG (thanks @davidcarne
   for finding this)

v0.5.5 (2015-03-06)
-------------------

-  Update bundled C++ libcapnp to v0.5.1.2 security release

v0.5.4 (2015-03-02)
-------------------

-  Update bundled C++ libcapnp to v0.5.1.1 security release
-  Add bootstrap RPC methods
-  Fix possible segfault when importing multiple schemas

v0.5.3 (2015-02-23)
-------------------

-  Fix possible crash due to bad destructor ordering in MessageReader
   (by @JohnEmhoff)
-  Default to no longer using cython

v0.5.2 (2015-02-20)
-------------------

-  Add read_multiple_bytes/read_multiple_bytes_packed methods
-  Added Python 3.4 to the travis build matrix
-  Bump version for bundled C++ libcapnp to v0.5.1

v0.5.1 (2014-12-27)
-------------------

-  Remove installation dependency on cython. We now have no dependencies
   since libcapnp will automatically build as well.

v0.5.0 (2014-12-15)
-------------------

-  Timer class ``capnp.getTimer()``
-  pycapnp is now thread-safe and allows an event loop to be run in each
   thread

   -  You must destroy and re-create the event loop to get this
      functionality (see ``test_threads.py``)

-  Inheritance now works correctly for interfaces (previously inherited
   methods were inaccessible from pycapnp)
-  Add ability to import modules with dashes or spaces. Use underscores
   in place of them
-  ``from_bytes`` with builder=True is no longer zero copy. It never
   worked correctly, and is much safer now
-  Add ``num_first_segment_words`` argument wherever message creation
   can occur
-  Allow restoring a null objectId by passing None to restore
-  Support ordered dictionary in ``to_dict``
-  Add ListSchema class and schemas for native types under
   ``capnp.types`` which completes all the Schemas needed to be wrapped.
   See ``test_schema.py`` for examples using it
-  Add automatic build of C++ libcapnp if it’s not detected on the
   system. Also add flags –force-bundled-libcapnp and
   –force-system-libcapnp respectively

v0.4.6 (2014-9-10)
------------------

-  Fix build for new 0.21 release of Cython. 0.21 is now the minimum
   supported version of Cython.

v0.4.5 (2014-6-26)
------------------

-  Fix ``to_dict`` not converting enums to strings

v0.4.4 (2014-04-25)
-------------------

-  Fix compilation problem with gcc 4.8

v0.4.3 (2014-02-18)
-------------------

-  Fix problem with uninitialized unions in \_from_dict
-  Add accesible version numbers for C++ libcapnp

v0.4.2 (2014-02-13)
-------------------

-  Remove onDrained since it was removed upstream
-  Replace usage of strings as enum type with custom ``_DynamicEnum``
   class.
-  Also change ``Struct.which()`` method to be a property
   ``Struct.which`` and return an enum type (``_DynamicEnumField``,
   which behaves much like ``_DynamicEnum``).
-  TwoPartyServer.run_forever() now will handle more than 1
   simulataneous connection.
-  Change exception wrapper to detect and raise AttributeError for field
   lookup exceptions (Fixes problem in Python3.x ``__dir__``)
-  Allow setting of fields with python dicts.

0.4.1 (2013-12-18)
------------------

-  Remove python 3.2 from travis tests. Python 3.2 still should work
   fine, but it’s more trouble than it’s worth to write unicode tests
   that work in both it and Python2.
-  Fix problems with null characters in Text/Data fields. Fixes #19

.. _section-1:

0.4.0 (2013-12-12)
------------------

-  Initial working version of RPC
-  Add get_root_as_any to \_MessageReader
-  Add capnp.pxd for public declarations of cython classes
-  Fix problems compiling with gcc4.7

v0.3.18 (2013-11-05)
--------------------

-  Change naming of ReaderOption parameters to be pep8 compliant

v0.3.17 (2013-11-05)
--------------------

-  Add ReaderOptions to read/read_packed/from_bytes

v0.3.16 (2013-10-28)
--------------------

-  Add defaults flag to capnp-json. Also remove ‘which’ field
-  Add capnp-json serializer script. Also fix bugs in from_dict
-  Fix build for clang/python3. Also remove -fpermissive
-  Add ``as_builder`` method to Struct Reader
-  Add warning when writing the same message more than once
-  First working version of capability interfaces
-  Wrap InterfaceSchema
-  Fix setting string fields to support all types of strings
-  Fix changed API for DynamicObject/ObjectPointer

v0.3.15 (2013-09-19)
--------------------

-  Add not having installed the C++ libcapnp library to ‘Common
   Problems’
-  Add \_short_str function for use in capnp_test_pycapnp.py
-  Add test script for testing with https://github.com/kaos/capnp_test
-  Add handling of DynamicObject
-  Fix lists of lists or dicts for from_dict

v0.3.14 (2013-09-04)
--------------------

-  Fix problem with to_dict

v0.3.13 (2013-09-04)
--------------------

-  Add \_DynamicStructBuilder.to_bytes() and .from_bytes()
-  Change == on StructSchema to return cbool
-  Add Builder and Reader ABCs for each struct type

v0.3.12 (2013-09-03)
--------------------

-  Fix handling of empty path ’’ in load_module
-  Add from_dict
-  Fix bug in exception handling for which(). Also standardize
   exceptions.
-  Change import hook to require modules to end in ’_capnp’
-  Add import monkey patch function.
-  Change naming for functions to conform to PEP 8. Also deprecate old
   read/write API
-  Update preferred method for reading/writing messages from files

v0.3.11 (2013-09-01)
--------------------

-  Forgot to change project name in setup.py

v0.3.10 (2013-09-01)
--------------------

-  Change all references to old project name (change from
   capnpc-python-cpp to pycapnp)
-  Change DynamicValue.Reader lists to be returned as
   \_DynamicListReader
-  Unify setters for DynamicList and DynamicStruct
-  Add shortcuts for reading from / writing to files. In Python, it
   doesn’t make much sense to force people to muck around with
   MessageReaders and MessageBuilders since everything is landing on the
   heap anyway. Instead, let’s make it easy:
   MyType.read[Packed]From(file) reads a file and returns a MyType
   reader. MyType.newMessage() returns a MyType builder representing the
   root of a new message. You can call this builder’s
   write[Packed]To(file) method to write it to a file.
-  Store Builders by value rather than allocate them separately on the
   heap (matches treatment of Readers). v0.3 fixes the bug that made
   this not work.
-  Wrap MessageBuilder::setRoot().
-  Add tests based on TestAllTypes from the C++ test.capnp. Fix problems
   uncovered in capnp.pyx.
-  Implement **str** and **repr** for struct and list builders. **str**
   uses prettyPrint while **repr** shows the type name and the
   low-whitespace stringification. Also implement **repr** for
   StructSchema, just because why not?

v0.3.9 (2013-08-30)
-------------------

-  Change load to use a global SchemaParser. Make structs settable as
   field
-  Add docstrings for new functions and \_DynamicResizableListBuilder

v0.3.8 (2013-08-29)
-------------------

-  Add initial tests
-  Add \_capnp for original Cython module. Meant for testing.
-  Lowercase schema so it conforms to member naming conventions
-  Expose \_StructSchema’s raw node
-  Add some useful \_StructSchema, reader, and builder methods
-  Add full orphan functionality. Also, allow special orphan lists
-  Finish up adding docstrings to all public classes/methods

v0.3.7 (2013-08-26)
-------------------

-  Add a ton of docstrings and add to official docs
-  Add DynamicOrphan

v0.3.6 (2013-08-26)
-------------------

-  Add intersphinx for linking to python docs
-  Add C++ library version check

v0.3.5 (2013-08-25)
-------------------

-  Add handling of constants in schemas
-  Fix new error with DynamicValue.Builder no longer being copyable

v0.3.4 (2013-08-22)
-------------------

-  Fix Void namespace change
-  Updated capnp schema to conform with new union rules

v0.3.3 (2013-08-22)
-------------------

-  Fix for the removal of DynamicUnion from the C++ API

v0.3.2 (2013-08-21)
-------------------

-  Add MANIFEST.in to include README

v0.3.1 (2013-08-21)
-------------------

-  Update docs with lines about upgrading setuptools

.. _section-2:

0.3.0 (2013-08-21)
------------------

-  Initial commit of docs
-  Add querying unnamed enums to structs

.. _section-3:

0.2.1 (2013-08-13)
------------------

-  Fix enum interface change for benchmark
-  Random formatting cleanup
-  Allow import paths in the schema loader
-  Add travis CI

.. _section-4:

0.2.0 (2013-08-12)
------------------

-  Initial working version