oaeproject/Hilary — pull request #712 by simong (2013-11-21 10:58:44+00:00)
Title: Push notifications

The main body of work in this commit is the introduction of push notifications.
A websocket endpoint is available at `/api/push` on which clients can subscribe for activity streams.
So far, no work has been made to make this exensible and support non-activity related features.

Overview
--------

Each app server will have their own transient queue on which it can receive activities.
If the app server dies, this queue and all the activities in it will be removed.

When activities are routed, each one of them will be published to a special exchange `oae-activity-push` with a routing key of `resourceId#activityStreamId`.
This allows us to route activities from the exchange to the app server queues who are interested in those feeds.

When a client connects to a socket he'll end up at an app server. The client will need to subscribe
to certain activityfeeds (his/her activity stream, his/her notification stream, a piece of content's activity stream, ..)
in order to start receiving push notifications for them. When a user subscribes for a feed, the app server will instruct
the message exchange to send all activities for that particular feed to its own queue. When an activity occurs in that feed,
it will end up at the queue, the app server will receive it and then immediately send it out to the client.

In theory, a user will get a push notification as soon as an activity has been routed. Since routing happens as soon as a new
activity has been received from RabbitMQ, this should be very fast.

Refactor work
-------------

Some things had to be refactored in order to support all of the above.

1. Split off task queue functionality from util/MQ into util/TaskQueue
To support the creation of custom queues/routing of messages, the MQ-utility now exposes more direct access to RabbitMQ.
The old functionality is still available in a new utility called `TaskQueue` which is a small wrapper over the `MQ` utility.

2. New activity streams
Activity streams are no longer tied to just `user`s and `group`s, all top-level resources (`content`, `discussion`) can
have their own activity streams.
`activity` and `notification` are also no longer the only supported streams. Users can declare their own streams to which
activities can be routed.
So far the following streams are available:
* activity
* notification
* message (for content comments and discussion messages)

3. Transient routes
Some activity streams don't really need to be persisted to disk (ex: the `message` stream).
When a stream has been marked as transient, its activities will only be emitted after the routing phase and will then be dropped.
These will **NOT** be aggregated.

TODO
----

1. Migration
We will require some migration/transitioning period as activity streams now have slightly different keys.

2. More tests (coming soon)