Repository: ManiacalLabs/BiblioPixel
Pull request #769 (2018-04-18 11:59:47)

New `Runnable` interface + grab bag of tiny commits
My big artnet branch got out of control and I had to extract out a lot of simple commits.

The "Runnable" one is the one that needs explanation:

As I get more UDP tests, I'm having significant issues with sockets not shutting down correctly and this problem is only going to get worse as we have more facilities.  The main artnet branch is stymied now because of this.

In the real world, these will correspond to `bp` refusing to shut down and having to be killed.

So I extracted out a class called `Runnable` that basically abstracts a `Thread` and gives it a bit more structure, as well as forcing all joins to have a timeout.  

It's useful both for threads, and for other services that have their own threads hidden within them (like `pynput` and `remote`).



adammhaile: +1!

On Wed, Apr 18, 2018 at 7:59 AM, Tom Swirly <notifications@github.com>
wrote:

> My big artnet branch got out of control and I had to extract out a lot of
> simple commits.
>
> The "Runnable" one is the one that needs explanation:
>
> As I get more UDP tests, I'm having significant issues with sockets not
> shutting down correctly and this problem is only going to get worse as we
> have more facilities. The main artnet branch is stymied now because of this.
>
> In the real world, these will correspond to bp refusing to shut down and
> having to be killed.
>
> So I extracted out a class called Runnable that basically abstracts a
> Thread and gives it a bit more structure, as well as forcing all joins to
> have a timeout.
>
> It's useful both for threads, and for other services that have their own
> threads hidden within them (like pynput and remote).
> ------------------------------
> You can view, comment on, or merge this pull request online at:
>
>   https://github.com/ManiacalLabs/BiblioPixel/pull/769
> Commit Summary
>
>    - Replace "dimensions" with "shape" in tests and projects
>    - Make Layout more forgiving of partial color_lists
>    - Correctly handle Exceptions with no args in main
>    - New Runnable hierarchy replaces threads.py
>    - Pull code up from server_driver.py into server_cache.py
>    - Split Control.set_target() off Control.start()
>    - Rename `Address.target` to `Address.root` (fix #767)
>    - Add a timeout to UDP sockets.
>
> File Changes
>
>    - *M* bibliopixel/control/action.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-0>
>    (8)
>    - *M* bibliopixel/control/address.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-1>
>    (50)
>    - *M* bibliopixel/control/control.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-2>
>    (13)
>    - *M* bibliopixel/control/midi.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-3>
>    (2)
>    - *M* bibliopixel/control/queued_address.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-4>
>    (16)
>    - *M* bibliopixel/control/receiver.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-5>
>    (4)
>    - *M* bibliopixel/control/routing.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-6>
>    (10)
>    - *M* bibliopixel/drivers/server_driver.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-7>
>    (16)
>    - *M* bibliopixel/layout/layout.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-8>
>    (4)
>    - *M* bibliopixel/main/main.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-9>
>    (2)
>    - *M* bibliopixel/main/monitor.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-10>
>    (3)
>    - *M* bibliopixel/util/server_cache.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-11>
>    (12)
>    - *A* bibliopixel/util/threads/runnable.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-12>
>    (147)
>    - *M* bibliopixel/util/threads/task_thread.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-13>
>    (4)
>    - *D* bibliopixel/util/threads/threads.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-14>
>    (48)
>    - *M* bibliopixel/util/threads/update_threading.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-15>
>    (6)
>    - *M* bibliopixel/util/udp.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-16>
>    (40)
>    - *M* projects/12-gifwriter.yml
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-17>
>    (2)
>    - *M* projects/21-key-control.json
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-18>
>    (2)
>    - *M* projects/23-midi-control.json
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-19>
>    (2)
>    - *M* test/bibliopixel/control/midi_test.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-20>
>    (12)
>    - *M* test/bibliopixel/control/queued_address_test.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-21>
>    (4)
>    - *M* test/bibliopixel/drivers/artnet_test.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-22>
>    (2)
>    - *M* test/bibliopixel/layout/layout_test.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-23>
>    (6)
>    - *M* test/bibliopixel/project/project_test.py
>    <https://github.com/ManiacalLabs/BiblioPixel/pull/769/files#diff-24>
>    (2)
>
> Patch Links:
>
>    - https://github.com/ManiacalLabs/BiblioPixel/pull/769.patch
>    - https://github.com/ManiacalLabs/BiblioPixel/pull/769.diff
>
> —
> You are receiving this because you are subscribed to this thread.
> Reply to this email directly, view it on GitHub
> <https://github.com/ManiacalLabs/BiblioPixel/pull/769>, or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AA6a6v4N4qKbNoiR7EdS5ZNatV2xAPeBks5tpyq0gaJpZM4TZ7sI>
> .
>


Patch:
diff --git a/bibliopixel/control/action.py b/bibliopixel/control/action.py
index 4eade53..fc0ef17 100644
--- a/bibliopixel/control/action.py
+++ b/bibliopixel/control/action.py
@@ -13,8 +13,8 @@ class Action(Receiver):
         self.address = QueuedAddress(address)
         self.ops = Ops(*ops)
 
-    def set_target(self, project):
-        self.address.set_target(project)
+    def set_root(self, root):
+        self.address.set_root(root)
 
     def receive(self, values):
         if self.ops:
@@ -51,9 +51,9 @@ class ActionList(Receiver):
             actions = [actions]
         self.actions = tuple(Action.make(a) for a in actions or ())
 
-    def set_target(self, project):
+    def set_root(self, root):
         for a in self.actions:
-            a.set_target(project)
+            a.set_root(root)
 
     def receive(self, msg):
         values = tuple(msg.values())
diff --git a/bibliopixel/control/address.py b/bibliopixel/control/address.py
index 92d75e9..afa21bf 100644
--- a/bibliopixel/control/address.py
+++ b/bibliopixel/control/address.py
@@ -1,6 +1,6 @@
 """
-An address identifies how to get or set a piece of data within a Python object
-using attributes and indexing.
+An address identifies how to get or set a piece of data within a Python object,
+called the "root", using attributes and indexing.
 
 An address description is a string looking like:
 
@@ -10,7 +10,7 @@ An address description is a string looking like:
 
 which would mean
 
-"given an object target, the value ``target.foo.bar[32][5]['baz'].bang()``".
+"given an object "root", the value ``root.foo.bar[32][5]['baz'].bang()``".
 
 Addresses are divided into "segments".
 
@@ -21,7 +21,7 @@ In the example above, the segments are ``foo``, ``bar``, ``[32]``, ``[5]``,
 ``[baz]`` and ``bang``; ``foo`` and ``bar`` are attributes.
 ``baz`` is a string index, and ``32`` and ``5`` are numeric indexes.
 
-You can use an Address to either get or set values in a target object.
+You can use an Address to either get or set values in the root object.
 
 Any key that's entirely numeric is taken to be an integer index.  This is
 convenient but prevents the creation of dictionaries like ``{1: 'x', '1': 'y'}``
@@ -36,25 +36,25 @@ class Address:
         def __init__(self, name):
             self.name = name
 
-        def set(self, target, *value):
-            self._set(target, (value[0] if len(value) == 1 else value))
+        def set(self, root, *value):
+            self._set(root, (value[0] if len(value) == 1 else value))
 
     class Attribute(Segment):
-        def get(self, target):
-            return getattr(target, self.name)
+        def get(self, root):
+            return getattr(root, self.name)
 
-        def _set(self, target, value):
-            setattr(target, self.name, value)
+        def _set(self, root, value):
+            setattr(root, self.name, value)
 
         def __str__(self):
             return '.%s' % self.name
 
     class Index(Segment):
-        def get(self, target):
-            return target[self.name]
+        def get(self, root):
+            return root[self.name]
 
-        def _set(self, target, value):
-            target[self.name] = value
+        def _set(self, root, value):
+            root[self.name] = value
 
         def __str__(self):
             return '[%s]' % self.name
@@ -63,11 +63,11 @@ class Address:
         def __init__(self):
             pass
 
-        def get(self, target):
-            return target()
+        def get(self, root):
+            return root()
 
-        def set(self, target, *value):
-            target(*value)
+        def set(self, root, *value):
+            root(*value)
 
         def __str__(self):
             return '()'
@@ -106,17 +106,17 @@ class Address:
         return self.name
 
     @staticmethod
-    def _get(target, address):
+    def _get(root, address):
         for a in address:
-            target = a.get(target)
-        return target
+            root = a.get(root)
+        return root
 
-    def get(self, target):
-        return self._get(target, self.segments)
+    def get(self, root):
+        return self._get(root, self.segments)
 
-    def set(self, target, *values):
+    def set(self, root, *values):
         *first, last = self.segments
-        parent = self._get(target, first)
+        parent = self._get(root, first)
         last.set(parent, *(self.assignment + values))
 
 
diff --git a/bibliopixel/control/control.py b/bibliopixel/control/control.py
index 62e36f3..4f98fe3 100644
--- a/bibliopixel/control/control.py
+++ b/bibliopixel/control/control.py
@@ -29,13 +29,14 @@ class Control:
         self.routing = Routing(routing or {}, default or {}, python_path)
         self.running = False
 
-    def start(self, project):
+    def set_root(self, root):
+        self.pre_routing.set_root(root)
+        self.routing.set_root(root)
+
+    def start(self):
         if self.verbose:
             log.info('Starting %s', self)
 
-        self.project = project  # Can I now delete this?
-        self.pre_routing.set_target(project)
-        self.routing.set_target(project)
         self.running = True
         self.thread = self._make_thread()
         self.thread.start()
@@ -47,7 +48,7 @@ class Control:
         self.running = False
 
     def loop(self):
-        for msg in self:
+        for msg in self.messages():
             self.receive(msg)
             if not self.running:
                 return
@@ -77,7 +78,7 @@ class Control:
         """
         raise NotImplementedError
 
-    def __iter__(self):
+    def messages(self):
         """Should yield a sequence of messages from the input source."""
         raise NotImplementedError
 
diff --git a/bibliopixel/control/midi.py b/bibliopixel/control/midi.py
index 638d6c8..a8c7047 100644
--- a/bibliopixel/control/midi.py
+++ b/bibliopixel/control/midi.py
@@ -54,7 +54,7 @@ class Midi(control.ExtractedControl):
         super().__init__(**kwds)
         self.use_note_off = use_note_off
 
-    def __iter__(self):
+    def messages(self):
         if not mido:
             raise ValueError(MIDO_ERROR)
         try:
diff --git a/bibliopixel/control/queued_address.py b/bibliopixel/control/queued_address.py
index 4220d7e..44c595a 100644
--- a/bibliopixel/control/queued_address.py
+++ b/bibliopixel/control/queued_address.py
@@ -7,7 +7,7 @@ class QueuedAddress(Receiver):
     A `QueuedAddress` is a `Receiver` which sets `Address`es, perhaps using an
     an optional `EventQueue`.
 
-    When the `set_target` method is called, `QueuedAddress` searches
+    When the `set_root` method is called, `QueuedAddress` searches
     down through the address and stores the most recent `event_queue`
     method it finds.
 
@@ -16,16 +16,16 @@ class QueuedAddress(Receiver):
     def __init__(self, address=None):
         self.address = Address(address)
 
-    def set_target(self, target):
+    def set_root(self, root):
         self.last_segment = self.address.segments[-1]
         self.queue = None
-        self.subtarget = target
+        self.subroot = root
 
         for segment in self.address.segments:
-            self.queue = getattr(self.subtarget, 'event_queue', self.queue)
+            self.queue = getattr(self.subroot, 'event_queue', self.queue)
             if segment is not self.last_segment:
-                self.subtarget = segment.get(self.subtarget)
-                if self.subtarget is None:
+                self.subroot = segment.get(self.subroot)
+                if self.subroot is None:
                     raise ValueError(
                         'Resolving the address "%s" failed at "%s"' %
                         (self.address, segment))
@@ -48,8 +48,8 @@ class QueuedAddress(Receiver):
         return str(self.address)
 
     def get(self):
-        return self.last_segment.get(self.subtarget)
+        return self.last_segment.get(self.subroot)
 
     def _set(self, values):
         args = self.address.assignment + values
-        self.last_segment.set(self.subtarget, *args)
+        self.last_segment.set(self.subroot, *args)
diff --git a/bibliopixel/control/receiver.py b/bibliopixel/control/receiver.py
index e460a04..33b4927 100644
--- a/bibliopixel/control/receiver.py
+++ b/bibliopixel/control/receiver.py
@@ -6,12 +6,12 @@ class Receiver(abc.ABC):
     Several classes receive data and route it to a target, and this is their
     base class, mainly for documentation.
 
-    Receivers must have two methods, `set_target` and `receive`.  `set_target`
+    Receivers must have two methods, `set_root` and `receive`.  `set_root`
     must be called exactly once for each Receiver, and this must be before
     `receive` is ever.
     """
     @abc.abstractmethod
-    def set_target(self, target):
+    def set_root(self, root):
         pass
 
     @abc.abstractmethod
diff --git a/bibliopixel/control/routing.py b/bibliopixel/control/routing.py
index 23fc0d3..2cbadd1 100644
--- a/bibliopixel/control/routing.py
+++ b/bibliopixel/control/routing.py
@@ -38,13 +38,13 @@ class Routing(Receiver):
         routing = flatten.unflatten(routing)
         self.routing = make(routing)
 
-    def set_target(self, project):
+    def set_root(self, root):
         """Set the base project for routing."""
         def visit(x):
-            # Try to set_target, then recurse through any values()
-            set_target = getattr(x, 'set_target', None)
-            if set_target:
-                set_target(project)
+            # Try to set_root, then recurse through any values()
+            set_root = getattr(x, 'set_root', None)
+            if set_root:
+                set_root(root)
             values = getattr(x, 'values', lambda: ())
             for v in values():
                 visit(v)
diff --git a/bibliopixel/drivers/server_driver.py b/bibliopixel/drivers/server_driver.py
index fe081ca..1939b0f 100644
--- a/bibliopixel/drivers/server_driver.py
+++ b/bibliopixel/drivers/server_driver.py
@@ -1,12 +1,9 @@
 from . driver_base import DriverBase
-from .. util.server_cache import ServerCache
+from .. util.server_cache import StaticCache
 from .. project import construct
 
 
-class ServerDriver(DriverBase):
-    SERVER_CLASS = None
-    SERVER_KWDS = {}
-
+class ServerDriver(DriverBase, StaticCache):
     def __init__(self, *args, address, pixel_positions=None, **kwds):
         """
         Args:
@@ -32,15 +29,6 @@ class ServerDriver(DriverBase):
             self.pixel_positions = pixel_positions
             self._on_positions()
 
-    @classmethod
-    def cache(cls):
-        try:
-            return cls.CACHE
-        except AttributeError:
-            pass
-        cls.CACHE = ServerCache(cls.SERVER_CLASS, **cls.SERVER_KWDS)
-        return cls.CACHE
-
     def _send_packet(self):
         pass
 
diff --git a/bibliopixel/layout/layout.py b/bibliopixel/layout/layout.py
index ceadf1d..96fb4b3 100644
--- a/bibliopixel/layout/layout.py
+++ b/bibliopixel/layout/layout.py
@@ -133,7 +133,9 @@ class Layout(object):
             color_list[0][0]
         except TypeError:
             # It's a 1-dimensional list
-            assert not (len(color_list) % 3)
+            extra = len(color_list) % 3
+            if extra:
+                color_list = color_list[:-extra]
             color_list = list(zip(*[iter(color_list)] * 3))
 
         size = len(self._colors) - offset
diff --git a/bibliopixel/main/main.py b/bibliopixel/main/main.py
index 7c83cdd..0e9b0f1 100644
--- a/bibliopixel/main/main.py
+++ b/bibliopixel/main/main.py
@@ -79,7 +79,7 @@ def main():
     except Exception as e:
         if args.verbose:
             raise
-        log.printer('ERROR:', e.args[0], file=sys.stderr)
+        log.printer('ERROR:', e.args and e.args[0], file=sys.stderr)
         log.printer(*e.args[1:], sep='\n', file=sys.stderr)
         result = getattr(e, 'errorcode', -1)
 
diff --git a/bibliopixel/main/monitor.py b/bibliopixel/main/monitor.py
index dd00603..4525208 100644
--- a/bibliopixel/main/monitor.py
+++ b/bibliopixel/main/monitor.py
@@ -16,7 +16,8 @@ def run(args):
         raise
 
     control_object = tc(pre_routing='()')
-    control_object.start(print)  # noqa: T002
+    control_object.set_root(log.printer)
+    control_object.start()
     control_object.thread.join()
 
 
diff --git a/bibliopixel/util/server_cache.py b/bibliopixel/util/server_cache.py
index 6dd4ae5..94f6019 100644
--- a/bibliopixel/util/server_cache.py
+++ b/bibliopixel/util/server_cache.py
@@ -97,3 +97,15 @@ class ServerCache:
         if server:
             server.server.close()
             return True
+
+
+class StaticCache:
+    SERVER_CLASS = None
+    SERVER_KWDS = {}
+    CACHE = None
+
+    @classmethod
+    def cache(cls):
+        if not cls.CACHE:
+            cls.CACHE = ServerCache(cls.SERVER_CLASS, **cls.SERVER_KWDS)
+        return cls.CACHE
diff --git a/bibliopixel/util/threads/runnable.py b/bibliopixel/util/threads/runnable.py
new file mode 100644
index 0000000..464c947
--- /dev/null
+++ b/bibliopixel/util/threads/runnable.py
@@ -0,0 +1,147 @@
+from .. import log
+import contextlib, functools, threading, queue
+
+
+class Runnable:
+    """
+    Base class for all objects that contain threads - including threads
+    created and started by bp code, and threads started by external libraries.
+
+    You can use a Runnable in two ways:  you can call the methods start, stop,
+    and join just like for a thread, or you can use it as a context manager:
+
+    with some_runnable() as runnable:
+         do_some_things(runnable)
+
+         # At the end of the block, runnable.stop() is called and then
+         # the thread is joined
+
+    """
+
+    timeout = 0.1
+    running = True
+
+    def start(self):
+        self.running = True
+
+    def stop(self):
+        self.running = False
+
+    def join(self):
+        """Join this thread, or timeout after self.timeout seconds"""
+        pass
+
+    def cleanup(self):
+        pass
+
+    def is_alive(self):
+        return self.running
+
+    def __enter__(self):
+        self.start()
+        return self
+
+    def __exit__(self, exc_type, exc_val, exc_tb):
+        self.stop()
+        try:
+            self.join()
+        except Exception as e:
+            log.error('Unable to join thread %s: %s', self, e)
+
+
+class RunnableTarget(Runnable):
+    def run(self):
+        try:
+            self.target()
+        finally:
+            self.cleanup()
+
+    def target(self):
+        """The target code that is actually executed in the run method"""
+        pass
+
+
+class RunnableThread(threading.Thread, RunnableTarget):
+    def __init__(self, daemon=True, **kwds):
+        super().__init__(daemon=daemon, **kwds)
+
+    def start(self):
+        RunnableTarget.start(self)
+        threading.Thread.start(self)
+
+    def run(self):
+        RunnableTarget.run(self)
+
+    def join(self):
+        threading.Thread.join(self, self.timeout)
+
+
+class Loop(RunnableThread):
+    def __init__(self, loop=None, **kwds):
+        super().__init__(**kwds)
+        self.loop = loop or self.loop
+
+    def target(self):
+        while self.running:
+            self.loop()
+
+    def loop(self):
+        raise NotImplementedError
+
+
+class Iterable(RunnableThread):
+    def __init__(self, iterator=None, **kwds):
+        super().__init__(**kwds)
+        self.iterator = iterator or self.iterator
+
+    def target(self):
+        for i in self.iterator():
+            self.receiver(i)
+            if not self.running:
+                break
+
+    def iterator(self):
+        pass
+
+
+class RunnableCollection(Runnable):
+    def __init__(self, runnables):
+        self.runnables = tuple(runnables)
+
+    def start(self):
+        for r in self.runnables:
+            r.start()
+
+    def stop(self):
+        for r in self.runnables:
+            r.stop()
+
+    def join(self):
+        for r in self.runnables:
+            r.join()
+
+    def cleanup(self):
+        for r in self.runnables:
+            r.cleanup()
+
+    def is_alive(self):
+        return any(r.is_alive() for r in self.runnables)
+
+
+class QueueHandler(Loop):
+    def __init__(self, timeout=0.1, send=None, **kwds):
+        super().__init__(**kwds)
+        self.timeout = timeout
+        self.queue = queue.Queue()
+        self.send = send or self.send
+
+    def loop(self):
+        try:
+            msg = self.queue.get(timeout=self.timeout)
+        except queue.Empty:
+            pass
+        else:
+            self.send(msg)
+
+    def send(self, msg):
+        pass
diff --git a/bibliopixel/util/threads/task_thread.py b/bibliopixel/util/threads/task_thread.py
index 3693497..42960e6 100644
--- a/bibliopixel/util/threads/task_thread.py
+++ b/bibliopixel/util/threads/task_thread.py
@@ -1,5 +1,5 @@
 import collections, threading
-from . import threads
+from . import runnable
 
 
 class Task(object):
@@ -16,7 +16,7 @@ class Task(object):
         next_task.event.set()
 
 
-class TaskThread(threads.Loop):
+class TaskThread(runnable.Loop):
     def __init__(self, producer_task, consumer_task, daemon=True, **kwds):
         super().__init__(daemon=daemon, **kwds)
         self.producer_task = producer_task
diff --git a/bibliopixel/util/threads/threads.py b/bibliopixel/util/threads/threads.py
deleted file mode 100644
index 6a48604..0000000
--- a/bibliopixel/util/threads/threads.py
+++ /dev/null
@@ -1,48 +0,0 @@
-import functools, threading, queue
-
-
-class Loop(threading.Thread):
-    def __init__(self, loop=None, daemon=True, **kwds):
-        super().__init__(daemon=daemon, **kwds)
-        self.running = True
-        self.loop = loop or self.loop
-
-    def run(self):
-        while self.running:
-            self.loop()
-
-    def stop(self):
-        self.running = False
-
-    def loop(self):
-        pass
-
-
-class Loops(object):
-    def __init__(self, loops):
-        self.loops = list(loops)
-
-    def start(self):
-        map(lambda x: x.start(), self.loops)
-
-    def stop(self):
-        map(lambda x: x.stop(), self.loops)
-
-
-class QueueHandler(Loop):
-    def __init__(self, timeout=0.1, send=None, **kwds):
-        super().__init__(**kwds)
-        self.timeout = timeout
-        self.queue = queue.Queue()
-        self.send = send or self.send
-
-    def loop(self):
-        try:
-            msg = self.queue.get(timeout=self.timeout)
-        except queue.Empty:
-            pass
-        else:
-            self.send(msg)
-
-    def send(self, msg):
-        pass
diff --git a/bibliopixel/util/threads/update_threading.py b/bibliopixel/util/threads/update_threading.py
index b750ed1..c5a4053 100644
--- a/bibliopixel/util/threads/update_threading.py
+++ b/bibliopixel/util/threads/update_threading.py
@@ -1,9 +1,9 @@
 import threading, time
 from .. import log
-from . import compose_events, threads
+from . import compose_events, runnable
 
 
-class UpdateDriverThread(threads.Loop):
+class UpdateDriverThread(runnable.Loop):
 
     def __init__(self, driver):
         super().__init__()
@@ -35,7 +35,7 @@ class UpdateDriverThread(threads.Loop):
         self._updating.set()
 
 
-class UpdateThread(threads.Loop):
+class UpdateThread(runnable.Loop):
 
     def __init__(self, drivers):
         super().__init__()
diff --git a/bibliopixel/util/udp.py b/bibliopixel/util/udp.py
index b49b29e..ed77f49 100644
--- a/bibliopixel/util/udp.py
+++ b/bibliopixel/util/udp.py
@@ -4,40 +4,26 @@ separate thread.
 """
 
 import queue, socket
-from .. util.threads import threads
+from .. util.threads import runnable
 
 
-class Sender:
+class Sender(runnable.Runnable):
     def __init__(self, address):
         """
-        :param str address: a pair (port, ip_address) to pass to socket.connect
+        :param str address: a pair (ip_address, port) to pass to socket.connect
         """
         super().__init__()
         self.address = address
-        self.running = True
-        self._socket = None
-        self.reuse_socket = True
-
-    @property
-    def socket(self):
-        if not (self.reuse_socket and self._socket):
-            self._socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
-        return self._socket
-
-    def start(self):
-        pass
-
-    def stop(self):
-        pass
 
     def send(self, msg):
         with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:
+            sock.settimeout(self.timeout)
             sock.connect(self.address)
             sock.send(msg)
             # sock.sendto(msg, self.address)
 
 
-class QueuedSender(threads.QueueHandler):
+class QueuedSender(runnable.QueueHandler):
     """
     The UPD protocol is stateless but not necessarily thread-safe.
 
@@ -48,9 +34,7 @@ class QueuedSender(threads.QueueHandler):
 
     def __init__(self, address, **kwds):
         """
-        :param str address: a pair (port, ip_address) to pass to socket.connect
-        :param bool use_queue: if True, run the connection in a different thread
-            with a queue
+        :param str address: a pair (ip_address, port) to pass to socket.connect
         """
         super().__init__(**kwds)
         self.sender = Sender(address)
@@ -61,12 +45,14 @@ class QueuedSender(threads.QueueHandler):
 
 def sender(address, use_queue=True, **kwds):
     """
-    :param tuple address:
+    :param str address: a pair (ip_address, port) to pass to socket.connect
+    :param bool use_queue: if True, run the connection in a different thread
+        with a queue
     """
     return QueuedSender(address, **kwds) if use_queue else Sender(address)
 
 
-class Receiver(threads.Loop):
+class Receiver(runnable.Loop):
     """
     Receive UDP messages in a thread
     """
@@ -80,11 +66,15 @@ class Receiver(threads.Loop):
     def run(self):
         self.socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
         self.socket.bind(self.address)
+        self.socket.settimeout(self.timeout)
 
         super().run()
 
     def loop(self):
-        data, addr = self.socket.recvfrom(self.bufsize)
+        try:
+            data, addr = self.socket.recvfrom(self.bufsize)
+        except socket.timeout:
+            return
         if data:
             self.receive(data)
 
diff --git a/projects/12-gifwriter.yml b/projects/12-gifwriter.yml
index 410369a..f69454b 100644
--- a/projects/12-gifwriter.yml
+++ b/projects/12-gifwriter.yml
@@ -1,4 +1,4 @@
-dimensions: [16, 16]
+shape: [16, 16]
 
 aliases:
   bpa: BiblioPixelAnimations.matrix
diff --git a/projects/21-key-control.json b/projects/21-key-control.json
index b0cb5e6..a89e3fc 100644
--- a/projects/21-key-control.json
+++ b/projects/21-key-control.json
@@ -17,7 +17,7 @@
         }
     },
 
-    "dimensions": [50],
+    "shape": [50],
 
     "run": {
         "fps": 30
diff --git a/projects/23-midi-control.json b/projects/23-midi-control.json
index fbd6c01..b62ba8e 100644
--- a/projects/23-midi-control.json
+++ b/projects/23-midi-control.json
@@ -17,7 +17,7 @@
         }
     },
 
-    "dimensions": [50],
+    "shape": [50],
 
     "run": {
         "fps": 30


Test patch:
diff --git a/test/bibliopixel/control/midi_test.py b/test/bibliopixel/control/midi_test.py
index d654d6a..2b80750 100644
--- a/test/bibliopixel/control/midi_test.py
+++ b/test/bibliopixel/control/midi_test.py
@@ -42,15 +42,15 @@ class MidiTest(unittest.TestCase):
 
     def run_test(self, msgs, expected, routing=None, **kwds):
         with patch.patch(midi, 'mido', FakeMido(msgs)):
-            class Settable:
-                def deferred_set(self, address, *value):
-                    address.set(self, *value)
+            class Root:
+                pass
 
-            settable = Settable()
+            root = Root()
             m = midi.Midi(routing=routing or self.routing, **kwds)
-            m.start(settable)
+            m.set_root(root)
+            m.start()
             m.thread.join()
-            self.assertEqual(vars(settable), expected)
+            self.assertEqual(vars(root), expected)
 
     def test_one(self):
         expected = {'note': (32, fractions.Fraction(96, 127))}
diff --git a/test/bibliopixel/control/queued_address_test.py b/test/bibliopixel/control/queued_address_test.py
index a93526e..319faf5 100644
--- a/test/bibliopixel/control/queued_address_test.py
+++ b/test/bibliopixel/control/queued_address_test.py
@@ -31,7 +31,7 @@ class QueuedAddressTest(unittest.TestCase):
     def test_no_queue(self):
         test = Base()
         qa = QueuedAddress('baz.bar.foo.foo')
-        qa.set_target(test)
+        qa.set_root(test)
         self.assertEqual(qa.get(), 12)
         qa.receive(('hello', ))
         self.assertEqual(qa.get(), 'hello')
@@ -39,7 +39,7 @@ class QueuedAddressTest(unittest.TestCase):
     def test_queued_address(self):
         test = Base()
         qa = QueuedAddress('baz.barq.foo.foo')
-        qa.set_target(test)
+        qa.set_root(test)
         self.assertEqual(qa.get(), 12)
         qa.receive(('hello',))
         self.assertEqual(qa.get(), 12)
diff --git a/test/bibliopixel/drivers/artnet_test.py b/test/bibliopixel/drivers/artnet_test.py
index 5addce2..6fbf0bc 100644
--- a/test/bibliopixel/drivers/artnet_test.py
+++ b/test/bibliopixel/drivers/artnet_test.py
@@ -16,7 +16,7 @@ PROJECT = {
         "typename": "artnet",
         "ip_address": LOCALHOST
     },
-    "dimensions": 12,
+    "shape": 12,
     "animation": ".tests.StripChannelTest",
     "run": {
         "max_steps": MAX_STEPS
diff --git a/test/bibliopixel/layout/layout_test.py b/test/bibliopixel/layout/layout_test.py
index 6a5029d..9cc2937 100644
--- a/test/bibliopixel/layout/layout_test.py
+++ b/test/bibliopixel/layout/layout_test.py
@@ -44,6 +44,12 @@ class LayoutTest(unittest.TestCase):
         expected = [RED, GREEN, HOT_PINK, GREEN]
         self.assertEquals(expected, strip.color_list)
 
+    def test_set_partial_color(self):
+        strip = Strip(MockDriver(4))
+        strip.color_list = (RED + GREEN + HOT_PINK + GREEN)[:-1]
+        expected = [RED, GREEN, HOT_PINK, BLACK]
+        self.assertEquals(expected, strip.color_list)
+
     def test_set_offset(self):
         strip = Strip(MockDriver(4))
         strip.set_color_list(RED + GREEN, 2)
diff --git a/test/bibliopixel/project/project_test.py b/test/bibliopixel/project/project_test.py
index 6836c1e..1c2bab2 100644
--- a/test/bibliopixel/project/project_test.py
+++ b/test/bibliopixel/project/project_test.py
@@ -89,7 +89,7 @@ class ProjectTest(unittest.TestCase):
 PROJECT = """
 {
     "driver": "dummy",
-    "dimensions": 12,
+    "shape": 12,
     "layout": "strip",
     "animation": "strip_test",
     "run": {