Package: katcp
katcp-pythonPython client and server library for the Karoo Array Telescope Control
Protocol (KATCP).For more info, see the doc directory in this
repository, or for online documentation of the latest release,read the katcp-python documentation.Changelog21 October 2022 (0.9.3)Address deprecation issues in Python 3.9 (PR#263)22 September 2022 (0.9.2)consistent error message in py2 and py3 for error in Timestamp decode (PR#258)13 May 2021 (0.9.1)Fix issues in KATCPReply__repr__in py3 (PR#256)29 January 2021 (0.9.0)Add asyncio compatible ioloop to ioloop manager (PR#254)29 October 2020 (0.8.0)Add bulk sensor sampling feature (PR#197)3 September 2020 (0.7.2)Support for handling generator expressions in Discrete type (PR#237)Fix handling of strings and bytes inget_sensorin testutils (#240)Allow strings or bytes for assert_request_fails andtest_assert_request_succeedsfunction arguments. (PR(#242)Handle str type correctly('easier') intestutils.get_sensorfor python 2 and python 3 (#243)Allow bytes and strings intest_sensor_listcomparison of sensors (#244)Correct handling of floatstest_sensor_list(#245)black formatting on certain test files. (PR#246)12 May 2020 (0.7.1)All params infuture_get_sensorare now cast to byte strings (PR#235).Added tests totest_fake_clients.pyandtest_inspecting_client.py(PR#235).Ensure testutils method casts expected requests to byte strings (PR#233).28 October 2019 (0.7.0)Added Python 3 compatibility (PR#215)23 August 2019 (0.6.4)Fix some client memory leaks, and adduntil_stoppedmethods (PR #206).Increase server's MAX_QUEUE_SIZE from 30 to 300 to handle more simultaneous
client connections (PR #208).Use correct ioloop for client AsyncEvent objects (PR #210).14 May 2019 (0.6.3)Put docs on readthedocs (PR #192).Added error handling when parsing Message arguments that include non-ASCII
characters (not following KATCP specification).  Invalid characters are replaced
with question marks, possibly breaking functionality (PR #193).Increase container sync time from 8 sec to 20 sec, and initial resync
timeout from 1 sec to 5 sec, to better support large containers (PR #194).Limit tornado version to <5 because the openssl version on Ubuntu 14.04
is not compatible (PR #195).Allow sampling strategy to be removed from cache, to better support clients
working with devices that dynamically remove sensors (PR #196).Provide a meaningful error message for the DeviceMetaClass assert statements
(PR #198).Increase server's MAX_QUEUE_SIZE from 30 to 300 to handle more simultaneous
client connections (PR #199).  (Not in release 0.6.3 - it was only on the
release/karoocamv20 branch)Updates to Jenkinsfile for pipeline improvements (PR #200, #201, #202).Add information on how to contribute to the project (PR #203).19 July 2017 (0.6.2)Bumped the tornado dependency to at least 4.3Started work to support for python3 (not production ready)Added the ability to let ClientGroup wait for a quorum of clients
Give ClientGroup.wait() the ability to wait until a subset of clients
(a 'quorum') satisfy a sensor condition.Added an until_some helper functionAdded timeout hints usage to the resource clientAdded default request-timeout-hint implementation to server.py
It is only activated if the server protocol is set to the (proposed) KATCP v5.1
spec and the timeout hint protocol flag is enabled.Moved IOLoopThreadWrapper to ioloop_manager.py, a more sensible locationImproved unit test coverage and implementation in some casesStarted work on implementing an async device handlerNumerous docstring and code style improvements throughoutFixed a bug where the timeout parameter was not passed down in until_state()Fixed a bug where model updates were never sent by the inspecting client
callback when sensor sync failedChanged thestrmethod for InspectingClientStateType to be more compactAdded a random-exponential retry backoff process to the inspecting client,
to ensure that the thundering herd does not continue thunderingProperly handle errors during inspection in inspecting clientAdded support for continuous integration using a Jenkinsfile30 August 2016 (0.6.1)Increased the maximum message size that can be received up from 128k to 2MMinor docstring fixesFixed testutilsFixed inspecting client from losing units valuesAvoid blocking python interpreter shutdown and avoid leaking file descriptorsImproved setup.py dependencies and removed pip-build-requirements.txtAllow high level clients to preset KATCP flagsSimplify sensor wait() interfaceAdded a check that the sensor exists before removing itBetter handling of malformed log linesUse katversion and git tags to determine package version on install15 June 2016 (0.6.0)Added high level, simple to use, client API (KATCPResource) that connects to,
inspects and presents a high level, hierarchical object oriented interface to
multiple KATCP devices.Added mid level client API (InspectingClient) that abstracts the inspection
of a KATCP device.Backwards-incompatible change in Sensor observer APISensor strategies now call back with raw Python datatype values rather than
KATCP formatted values, using the same 'reading' API as Sensor
observers' update methods.Tornado ioloop / iostream based sensor samplingTornado ioloop / iostream based serverSever message handlers can now return futuresServer supports fully async mode with Tornado futures.Threaded message handle compatibility mode + new Async mode (performance
penalty for threaded mode, but is default for now)Tornado ioloop / iostream based client AsyncClient, single implementation for
both BlockingClient and CallbackClient based on thin wrapper around
AsyncClient.Clients now stop much faster due to removal of 0.5s select timeout in
previous implementation.Backwards incompatible change in BlockingClient.blocking_request(): now
returns a failed reply rather than raising an exception for timeouts (like
CallbackClient.blocking_request() always did).Threadsafe mode (default for BlockingClient and CallbackClient) available, but
comes with performance overhead even async for usage.Hundreds of test-suite reliablility improvements.14 July 2014 (0.5.5)Add utility code to katcp.version.py that other modules can use to parse svn
or git revisions for use in build-state informs or sensorsAdd katcp.client.request_check: a convenience function for making KATCP
requests and raising an exception if there is an error.Add katcp.testutils.handle_mock_req() to allow mocked testing of server request
handlers and dispatch, removing the need to start() a DeviceServer for
handler tests.Make katcp.testutils.WaitingMock robust against race conditionsExplicitly list set of base KATCP requests in katcp.server.BASE_REQUESTSMake method-name to KATCP-name function available outside KATCP server
metaclassImprovements to logging (i.a. prevents issues when using SocketHandler)Docstring cleanups10 June 2013 (0.5.4)Change event-rate strategy to always send an update if the sensor has
changed and shortest-period has passedAdd differential-rate strategy31 August 2013 (0.5.3)Add convert_seconds() method to katcp client classes that converts
seconds into the device timestamp format.24 April 2013 (0.5.2)Fix memory leak in sample reactor when replacing sensor strategiesMinor error reporting improvements in sampling strategiesFix 2012 -> 2013 in changelog :)8 March 2013 (0.5.1)Minor code comment cleanupsMinor documentation and docstring updatesFix twisted katcp server's on-connect inform messages to be consistent
with the selected katcp versionImprove error messages when an invalid sensor sampling strategy is setFix bug in event-rate sampling strategy where events were sent whenever a sensor
object is updated, even if the value is unchanged. Also fixes same
bug for the event sampling strategy, since it uses the event-rate
code for its implementationWork around potential CPU livelock when sending messages and the network
buffers are full by using timeouts. Problem will be properly solved when the
current networking code is replaced by twisted.Fixed some buggy logger callsImprove the handling of sensor removal from the sensor treemade server main event loop robust against disconnect race-conditions when
looking up the client connection object29 January 2013 (0.5.0)Minor documentation updatesVersion 0.5.0 released13 December 2012 (0.5.0a0)Add support for protocol flags to katcp.core -- a standard
data structure for storing KATCP versions and capabilitiesClean-up various bits of code, docstring fixes, etc.katcp.Message constructors are now strict about unknown keyword argumentsAdd clear_strategies() method to server class to clear all sampling
strategies for a connection.Support for serving either KATCP v4 or v5sensor timestamp conversions as neededlog timestamp conversion as neededkattypes decorators can be configured for the appropriate protocol versionsgenerate appropriate on-connect messagesSupport for being a client of ether KATCP v4 or v5Auto-detect server version using on-connect informsAdd ability for clients to wait until protocol information is received from
serverRefactor use of raw 'sock' object as a connection identifierAbstract sock to a transport-neutral connection objectPass a request-bound connection object to request handlers, rather than a
raw socket. Automatically use the right name and mid for informs and repliesDeprecate use of self.reply(sock, ....) and self.reply_inform(sock,
....). Rather use req.reply(...), req.reply_with_message(...),
req.inform(...)Renamed CallbackClient.request to callback_request to be more consistent with
superclass APIAdd sensor construction class methods, e.g. Sensor.integer(...), rather than
Sensor(Sensor.INTEGER, ...)Removed non-standard rate-limit extension from 'event' sample strategy
implementation. KATCP v5 event-rate strategy covers that use case.Fix race condition between timeout and reply handlersVarious improvements to tests and testutils to make tests more robust and
(sometimes) fasterAdd implementation of subset of new KATCP v5 featuresAdd Address sensor typeImplement ?version-list and #version-connectAdd 'unreachable' and 'inactive' sensor statesMake float and integer sensor ranges optional and informational.Add Message Identifier (mid) support. Enabled by default in client
for servers that support mids.Use seconds instead of milliseconds for all timestamps and periods (except
when explicitly configured to use an older KATCP, see multi-version support
above)Version 0.5.0a0 released18 September 2012PEP-8 cleanup of code.Store sensor state as a tuple to allow 'atomic' read.Use nose as the default test runner.Fix thread leak in callback client when async requests are outstanding when
the client is being shut down.Fix server deadlock on sensor strategy lock at shutdown.Add variable number of arguments support to kattypes decorators.Version 0.3.5 released -- Should be last public release adhering to the KATCP
v4 specifications. Next release should be based on the KATCP v5
specifications.01 June 2011:Merge r9501,9706,9764,9769,9771,9776,9780:Numerous improvements to katcp.tx.proxy to add hooks that proxy
subclasses can connect functionality to (for example, arrival of
devices) and improve the behaviour of proxied sensors.Merge r9783,9784,9785:Fix bug in sensortree that was triggered by sensor updates
occurring while sensors were being added to the tree.Merge r9791:Fix katcp.tx build state and API version sending.Version 0.3.4 released.16 May 2011:Merge r9447,9448,9491:Add a better reconnecting interface to katcp.tx clients.Version 0.3.3 released.05 May 2011:Merge r9321,9357,9370,9371,9373,9397 from trunk:Fix kattypes decorator bug that caused it to break on request
methods with no arguments.Simplify disconnecting clients in katcp.tx.Allow katcp.tx clients to wait to connect.Report connections failures better inside katcp.tx.Minor docstring fix.Version 0.3.2 released.09 Feb 2011:Merge r7520,7522,7546,7578-7580,7588-7591,7635,7637-7638,7674-7678,
7701-7702,7707,7726-7729 from trunkFix unbounded memory usage issue in katcp.tx.Fix katcp.tx so it doesn't send sensor-list twice.Fix katcp.tx to clear running strategies when a connection is lost.Minor improvements to katcp.tx.Benchmark suite improvements.Merge r7806-7807,7812,7818,8081-8083,8085,8404,8420,8432-8434,8660
from trunkRefactor katcp.tx interface (breaks code that uses katcp.tx but Twisted
support was experimental in 0.3.0).Fix lack of build and api version reporting by katcp.tx server.Fix katcp.tx bug in reporting methods with names containing
underscores (affects ?help).Fix katcp.tx bug which causes messages to be lost if socket buffer
filled up.Change katcp.tx demo client to only use official katcp.tx API.Merge r8703 from trunkFix check in test (sensor_list should have been sensor-list).Version 0.3.1 released.14 Dec 2010:Merge r8034,8234-8235,8238,8260,8265 from trunkCleanup of testutils.25 Oct 2010:Created 0.3.x branch.22 Oct 2010:Add a NullHandler for katcp library logging as recommended by Python logging
documentation.12 Oct 2010:Add regular expression pattern support to ?sensor-list and ?sensor-value
requests.8 Oct 2010:Add .wait_connected() method to KATCP client.4 Oct 2010:Start of KATCP benchmarking effort.28 Sep 2010:Refactor Twisted KATCP into katcp.tx sub-package.Update and flesh out example KATCP server.27 Sep 2010:Implementation of Twisted KATCP proxy. A proxy mutliplexes queries and
sensor sampling from a single client to multiple KATCP servers.17 Sep 2010:Completion of Twisted KATCP DeviceServer.16 Aug 2010:Implementation ofeqandnefor katcp.Message.30 Jul 2010katcp.katcp renamed to katcp.core to avoid module name clashing with
package name.Allow Python types to be used in addition to katcp.Sensor object types
when constructing sensors.27 Jul 2010Customreprfor katcp.Message added.22 Jul 2010Initial sensor tree implementation. A sensor tree is a directed acyclic
graph of sensor dependencies. The tree triggers updates of dependent
sensors when dependee sensors change.15 Jul 2010Start of work on Twisted katcp client and server implementations.13 Jul 2010Major rework of katcp.testutils.11 Mar 2010Add check to client to bail out of re-trying sends if the client
has disconnected.02 Feb 2010Allow adding and removing sensors on the fly.20 Jan 2010Allow logging using .warn(fmt, arg1, arg2, ...) format as used
in Python logging (allows msg formatting to be delayed until it
is known whether the message will actually be logged).26 Oct 2009Add include_msg option to request decorator to give decorated
requests a means to access the original message and the message id.Add support for msg ids to callback client.23 Oct 2009Add support for message ids to server and client request handling.Server gained new .reply() and .reply_inform() methods that set
message ids.Server had .send_message() moved to ._send_message() since it is
now rather important to use .reply() and .reply_inform() when
replying. Old servers will need to upgrade. Calls to .send_message()
should be replaced with calls to either .reply(), .reply_inform() or
.inform(). Calls to .inform() should be changed to calls to
.reply_inform() if they are part of the response to a request.Client request handler handles message ids correctly.22 Oct 2009Add support for message ids to KATCP message and message parser classes.21 Oct 2009Add blocking_request implementation to callback client.Reduce client sleep to 0.5s (from 1.0s) to make shutdown more responsive.Add Message.reply_ok() method for checking that a message represents
a successful reply.07 Oct 2009Fix bug in new callback client time outs.05 Oct 2009Support for commands timing out in callback client.21 Sep 2009Fix bug in client run loop where self._sock could become
None while it was being used by select(..) and .recv(...).02 Sep 2009Implement TimestampOrNow kattype.01 Sep 2009Fix bug where AUTO strategy did not set updates on connect.26 Aug 2009Downgrade logging of successful request handling to DEBUG log level.29 Jul 2009Enable TCP_NODELAY on server and client sockets since katcp protocol messages
are often small.22 Jul 2009Update version to 0.2.x.Fix download url and license in setup.py.Have Sphinx read the version number from katcp.VERSION and katcp.VERSION_STR.29 Jun 2009Start of CHANGELOG in preparation for 0.1.0 release.