Cogs: start Broadcaster after its dependencies.
pipeline: Uses a set for requisite_urls. Uses remove() to reduce memory leak.
Bumps pipeline version 20140411.01
Add /jetpack-comment/ ignore pattern
Ignore &replytocom= as well
Dashboard: ignore Sass cache.
IA pack finder: switch to archivebot collection.
Publish a message when job settings change.

At present, a pipeline checks its settings after every request; however,
settings change relatively infrequently.  This polling behavior is
therefore unnecessarily heavy, and (I suspect) is slowing down pipeline
execution.

The Right Thing to do is to have each pipeline listen for change
notifications and update their job data when such a notification is
received.  (Pipelines should still poll -- it ensures that a pipeline's
job settings are in sync, and provides a way to recover from control
node communication failure -- however, we can poll at a much reduced
rate whilst minimizing blockage on the main thread.)
Switch to bleeding-edge redis-py.

Andy McCurdy has merged a large number of changes to redis-py's pubsub
implementation.  Of particular interest is pubsub.run_in_thread(), which
subscribes to a pubsub channel on a separate thread.  In released
versions of redis-py, doing this is very painful.

These changes are scheduled for redis-py 2.10.0, which is not yet
released.  Hence this commit.

We also drag in hiredis because redis-py edge + native Python parser
gets into infinite loops on sudden connection failure in the pubsub
client.  With hiredis, a redis.exceptions.ConnectionError is raised,
which is much easier to work with.
Use per-job pubsub channel.
A webapp for ArchiveBot integration testing.
When job settings change, publish a notification.
A more Pythonic way of raising exceptions-from-exceptions.
Wrap connect/disconnect in methods.
Introduce and wire in the settings listener.

This commit also moves abort_requested to the settings module, because
ce91de3039af33008a312c7b14823cd6fd95d1ac turns it into something that
acts more like a setting.
Make !yahoo increment settings age by one.

The old mechanism used separate calls to modify delay, pagereq delay,
and concurrency.  This incremented the settings age by three.  Under the
listen-for-settings-changes model, this means we also publish three
update messages.  However, a !yahoo is conceptually one settings change.
Tweak ignore pattern compilation error message.
Bump pipeline version.
Turning it up to 11 is a bit much.  Let's do 4.

(At least, it's too much for a single EC2 m1.medium to handle.)
Pipeline: report absolute available memory and disk.

In #archivebot, we have had a few occasions where it was useful to ask
"does any pipeline have enough space for a job of X GB".