Merging latest master into branch.

commit c039597e2ada561823318d2492004577945a9962
Author: Michael Spiegel <spiegel@addthis.com>
Date:   Mon Apr 7 15:28:28 2014 -0400

    Addition of enforcer.fail property.

commit 40f73e4c48db52f0da6b37217c5a7c4bbf555020
Author: Michael Spiegel <spiegel@addthis.com>
Date:   Mon Apr 7 14:36:21 2014 -0400

    Increment bark dependency version.

commit cd612ee9a7c5b7193cf8b6e7f4752e74aba71d6f
Author: Michael Spiegel <spiegel@addthis.com>
Date:   Mon Apr 7 14:21:13 2014 -0400

    Increment meshy version dependency.

commit f818c6276a0947046f665792359de609c9604909
Author: Michael Spiegel <spiegel@addthis.com>
Date:   Mon Apr 7 13:48:28 2014 -0400

    Increment bundle and muxy dependency versions.

commit ceb94335618341f5b795c79d9b1fc8a84cf36217
Author: Michael Spiegel <spiegel@addthis.com>
Date:   Mon Apr 7 13:02:31 2014 -0400

    Increment basis and codec dependency versions.

commit 667c3adfea60d314feac2017dfbe6e9bbc87e9e3
Merge: 53939f8 17fecc2
Author: mspiegel <michael.m.spiegel@gmail.com>
Date:   Fri Apr 4 14:10:00 2014 -0400

    Merge pull request #62 from stewartoallen/at-pull-proper-json-responses

    ensure properly formatted json responses

commit 17fecc20047c58e856597abd6f901cf3cf1112e5
Author: Stewart Allen <stewart@neuron.com>
Date:   Fri Apr 4 14:09:01 2014 -0400

    remove commented out lines

commit 53939f8e908b7a615238a775a9b333861d2d6b17
Merge: f361c66 9c74d90
Author: Matt Abrams <abramsm@gmail.com>
Date:   Fri Apr 4 10:25:04 2014 -0400

    Merge pull request #64 from erichees/spawn2-metrics

    add metrics endpoint to spawn2 servlet

commit f361c661515fbe6406fc70a221d951303c5b248b
Merge: 0f2d952 148eab0
Author: mspiegel <michael.m.spiegel@gmail.com>
Date:   Thu Apr 3 16:23:20 2014 -0400

    Merge pull request #66 from erichees/dependencies

    Dependencies

commit 0f2d952f75daab7a2437f2183e80ed3cca3123ca
Merge: ebe73ff 31b89c0
Author: mspiegel <michael.m.spiegel@gmail.com>
Date:   Thu Apr 3 16:16:56 2014 -0400

    Merge pull request #65 from erichees/ui-improvements

    Ui improvements

commit 9c74d90bafb5d6791e53e57a862623503c2a6698
Author: hees <hees@addthis.com>
Date:   Wed Apr 2 14:35:40 2014 -0400

    add metrics endpoint to spawn2 servlet

    - fixed bug in MinionsDown Gauge

commit 148eab00ab04c89a13fa515d76a05321173e7da4
Author: hees <hees@addthis.com>
Date:   Wed Apr 2 14:54:32 2014 -0400

    add links to dependency view

commit 012c06525d3ccc0028411f07349a935fcce1cf89
Author: hees <hees@addthis.com>
Date:   Mon Mar 31 15:49:32 2014 -0400

    addons to dependencies tab

    - use all connections instead of sinks
    - highlight current job in green

commit 0fc59736c0f5d8f5c228595cf248be2d7e4ed42e
Author: hees <hees@addthis.com>
Date:   Mon Mar 31 13:08:48 2014 -0400

    make all dependency text render rightside-up

commit 31b89c0805a6f53f78a4c0e0a41d6a676f37a2fc
Author: hees <hees@addthis.com>
Date:   Thu Apr 3 14:52:29 2014 -0400

    adjust table header width on jobs-list redraw

commit 0364bfe3816260eb8c08d0c6d6c9ea8356044f73
Author: hees <hees@addthis.com>
Date:   Mon Mar 31 17:38:53 2014 -0400

    added enabled only filter

commit 14bfc5435f1402b716b27f480f5a4fe1d15acb7e
Author: hees <hees@addthis.com>
Date:   Mon Mar 31 12:04:40 2014 -0400

    add user name to macro save

commit ebe73fff3e7d42dae42d352f12dab0ba794dff89
Author: Michael Spiegel <spiegel@addthis.com>
Date:   Thu Apr 3 13:56:23 2014 -0400

    Upgrade to bark 2.1.2.

    This version of bark has a bugfix. The change is to strip
    the trailing "/" character from the zookeeper path. The
    underlying Zookeeper client will throw an exception if the
    path ends in a slash.

commit b600625849531e65357345a55db6165bad60d098
Author: Stewart Allen <stewart@neuron.com>
Date:   Thu Apr 3 13:29:52 2014 -0400

    ensure properly formatted json responses

commit 984639518e5921511371d4274ecc735bd129d4fa
Author: ian <ian@addthis.com>
Date:   Thu Apr 3 09:54:49 2014 -0400

    quick fix for legacy source type inflating

commit bfb01110e7da6348098c1be149006b938dc728b5
Author: al <al@addthis.com>
Date:   Thu Apr 3 08:56:08 2014 -0400

    Tweak to minion/spawn minionType logic to better support minions with multiple types

commit c354ffd36630f37892633f9d0690f32bbf712e3b
Merge: 761720a 9406db1
Author: mspiegel <michael.m.spiegel@gmail.com>
Date:   Thu Apr 3 09:16:35 2014 -0400

    Merge pull request #61 from stewartoallen/at-pull-fix-replicate-script

    fix missing space in conditional in generated replication script

commit 9406db1d972343ed289b2c2b586e0f7b482cdeb4
Author: Stewart Allen <stewart@neuron.com>
Date:   Wed Apr 2 15:17:19 2014 -0400

    fix missing space in conditional in generated replication script

commit 761720a4db33e368dbbb6aee0e6b549c37177d83
Author: al <al@addthis.com>
Date:   Wed Apr 2 13:37:04 2014 -0400

    Minor -- add license info to new test.

commit fe9b556f02e174a16ffd2fdc805f3ef6724db821
Merge: c1b87ea 9f4a806
Author: alstaplesmoore <alstaplesmoore@gmail.com>
Date:   Wed Apr 2 13:35:31 2014 -0400

    Merge pull request #60 from addthis/spawn_queue

    Spawn queue

commit 9f4a806ae90657b3cf90307d0b859beac3793b9e
Author: al <al@addthis.com>
Date:   Wed Apr 2 13:28:05 2014 -0400

    Minor NPE fix

commit 44018e0962628d6dfa7d0d83db42588c56f492e7
Author: al <al@addthis.com>
Date:   Tue Apr 1 15:43:05 2014 -0400

    Additional sanity check to ensure task moves obey minion types

commit 8d05bfb42df353f5735eacdc4129063a2f2714ab
Author: al <al@addthis.com>
Date:   Tue Apr 1 10:45:13 2014 -0400

    Attempt to simplify spawn queue slot tracking.

commit 27f23aa66421617ec869a964095ede077b9aab82
Author: al <al@addthis.com>
Date:   Tue Apr 1 10:04:12 2014 -0400

    Disable last-host-slot feature on local stack, where it will be mostly a hindrance

commit 1ba8085e69218bebb787c9dc2811f513c9248309
Author: al <al@addthis.com>
Date:   Tue Apr 1 09:34:15 2014 -0400

    Minor changes to signatures/naming

commit 7b0801f8126ff8cec78fd0575a82ad5ffcc355ec
Author: al <al@addthis.com>
Date:   Tue Apr 1 08:53:15 2014 -0400

    Rebalance operations attempt to use the lightest valid host.

commit ffdf2469589377808e2111a939d18b95f8e3421f
Author: al <al@addthis.com>
Date:   Mon Mar 31 15:52:46 2014 -0400

    New jobs won't use up the last slot on a minion until after a configurable delay period

commit 1de70b567b8e38d34ed7ebf63de472605c2854a7
Author: al <al@addthis.com>
Date:   Mon Mar 31 12:24:25 2014 -0400

    Minions report meanActiveTasks as a percentage of their respective maxima

commit 073658ae27d1df359c476c035b440af483e19114
Author: al <al@addthis.com>
Date:   Mon Mar 31 12:01:16 2014 -0400

    Revise method signatures.

commit e542323859b07ec6f051625f895ab52efeb44ead
Author: al <al@addthis.com>
Date:   Mon Mar 31 11:12:49 2014 -0400

    Use meanActiveTasks as a tiebreak when availSlots are equal. Add tests for this behavior.

commit 2d9d8dcc1263bf21ec5c053724eadbd4011e8318
Author: al <al@addthis.com>
Date:   Fri Mar 28 11:06:55 2014 -0400

    Initial work at better task allocation

commit c1b87ea9342527a7c199e5731652fe7cea295c6d
Author: ian <ian@addthis.com>
Date:   Wed Apr 2 11:00:40 2014 -0400

    correctly handle ends of dates in SSMeshy

commit 699b4e79dddb9e15ee524994dbd338af17a59cd3
Merge: 4a9f8ef b73362b
Author: Ian Barfield <ian.barfield@gmail.com>
Date:   Tue Apr 1 17:37:52 2014 -0400

    Merge pull request #50 from addthis/prefetch-meshy2

    Prefetch meshy2

commit 4a9f8ef2a25bf8d9fcf09b2d3ca76c896c605140
Merge: b1e32cc d02070d
Author: Ian Barfield <ian.barfield@gmail.com>
Date:   Tue Apr 1 13:20:37 2014 -0400

    Merge pull request #58 from stewartoallen/at-pull-mesh-mq-v2

    add mesh queueing option

commit d02070d63493b3494e3f729ce961039980bf204f
Merge: 8f194af d9a7081
Author: Stewart Allen <stewart@neuron.com>
Date:   Tue Apr 1 12:49:00 2014 -0400

    Merge branch 'at-pull-mesh-mq-v2' of github.com:stewartoallen/hydra into at-pull-mesh-mq-v2

commit 8f194afaf727c6e534e0e586572e72affda37768
Author: Stewart Allen <stewart@neuron.com>
Date:   Tue Apr 1 12:48:36 2014 -0400

    patch to support hydra tests

commit d9a708178b2913d2febb1dbd53e2ca5c02b5ef28
Author: ian <ian@addthis.com>
Date:   Tue Apr 1 11:54:16 2014 -0400

    use stable meshy version

commit 0145bb7262eb4d59d25903ce1763afef8d31a8b6
Author: Stewart Allen <stewart@neuron.com>
Date:   Mon Mar 31 17:23:55 2014 -0400

    fix zero queue length polling lockup

commit ecacbf1c00c8ed2132d4460d12e568e79cc5d924
Author: Stewart Allen <stewart@neuron.com>
Date:   Mon Mar 31 16:34:56 2014 -0400

    add more extensive state debugging

commit 88edd09d90c25c9fcfd5826a702a94a2e20c65f8
Author: Stewart Allen <stewart@neuron.com>
Date:   Wed Mar 26 12:49:05 2014 -0400

    fix user-friendly output to send just a string, not a length-prefixed string

commit f187b8f071db796fc0cc992f43f30148a967c449
Author: Stewart Allen <stewart@neuron.com>
Date:   Wed Mar 26 09:47:26 2014 -0400

    replace stack traces with log.warn() calls

commit 3b9973af7a5099df89c553fcfe81d4943e7ef501
Author: Stewart Allen <stewart@neuron.com>
Date:   Tue Mar 25 15:50:39 2014 -0400

    on-demand creation of producer queues based on routing keys
    fix first-time creation bug for queues

commit 5de3f3a5a4cff794839f163a043f26fa50f4f39f
Author: Stewart Allen <stewart@neuron.com>
Date:   Tue Mar 25 01:15:12 2014 -0400

    optional mesh queueing. a replacement for rabbit
    minion/spawn comms need significant cleanup, doc and externalization
    the abstraction should be completed in subsequent patches

commit b1e32cc936a7d45480b83018dbe6af101f5011fa
Merge: fd555e6 503dd6a
Author: mspiegel <michael.m.spiegel@gmail.com>
Date:   Fri Mar 28 17:15:41 2014 -0400

    Merge pull request #56 from erichees/master

    update spawn2 templates to use auto-escaping syntax

commit 503dd6a9c064380df4eac90bef6cc01e61901e86
Author: hees <hees@addthis.com>
Date:   Fri Mar 28 16:24:07 2014 -0400

    update spawn2 templates to use auto-escaping syntax

commit b73362bfd2dc6f771c7a790853ccc5514bd0a321
Author: ian <ian@addthis.com>
Date:   Fri Mar 21 14:41:39 2014 -0400

    clean up for mesh2 source improvements et al

    - cacheMap is accessed within the sync lock. It looks like that
    would only ever manifest a problem in a NPE. That didn't occur
    in testing, but better safe etc

    - firstData / processing range re-implemented. It had been commented
    out for convenience (it is pretty much unused). There is some more
    explicit logic that makes it clearer what and when it is used for. I
    kept it outside of the sync block, but changed the moreData variable
    to volatile. This is kind of a hedge - it may not need to be volatile
    because after all, setting firstDate to be not null isn't sync'd there
    to begin with - I don't feel great about it being actually safe but that
    is a problem for another time.

    - some optimizations related to object creation. Fewer collections are
    made, and more use initial capacity settings

    - inverse lottery selection now takes a varargs param of primitive ints.
    this helps a lot with the previous goal (collection creation), but also
    makes it a bit simpler

commit 8b290d49c83bd27b2ea209026f1f0ec71361ca9c
Author: ian <ian@addthis.com>
Date:   Wed Mar 19 19:37:52 2014 -0400

    fix type typo

commit b6d6dcd9035a6784b0130fa5526d39adbe09683d
Author: ian <ian@addthis.com>
Date:   Wed Mar 19 19:23:11 2014 -0400

    different stealing method and linked queues

    linked queues can get size() without locking. the stealing
    variant is a little faster and adjustable

commit a5292f5f075165d3c38904c9ca560303c938443b
Author: ian <ian@addthis.com>
Date:   Wed Mar 19 18:55:50 2014 -0400

    removing cross-buffer max from task feeder

    much less concerned about a couple thousand bundles on heap than
    source feeder shutting down for 1000ms

commit 80fd7e3b668a15093d8df607c72c223fce89a97c
Author: ian <ian@addthis.com>
Date:   Wed Mar 19 18:23:23 2014 -0400

    just choose the meshy host outside of the lock

commit 2fc6d23888908e03bee052e53af994f9ca9303d5
Author: ian <ian@addthis.com>
Date:   Wed Mar 19 18:13:27 2014 -0400

    use linked queue instead of array backed

    producers were hoarding the global lock the array version uses in
    some cases

commit 3a4df93f2214f91f9ed3b7e854bece8d35da7ca2
Author: ian <ian@addthis.com>
Date:   Wed Mar 19 16:57:31 2014 -0400

    more logging for score cache

commit 2d14b5c73604b5514dee762c2c1e46e2eb07d425
Author: ian <ian@addthis.com>
Date:   Wed Mar 19 16:55:16 2014 -0400

    poll time cutoff and metric for the score cache

commit 9ee9d04e23f6b4ceb235e5de7cbced48f0fa6635
Author: ian <ian@addthis.com>
Date:   Wed Mar 19 16:26:06 2014 -0400

    delay bundle skipping and compression handling

    currently we usually block waiting for a remote response as soon
    as we try to wrap the input stream in a decompression wrapper. this
    is because the constructor sometimes tries to immediately read bytes
    (like for a header or something). Bundle skipping is similar in that
    it _would_ block given the chance. This change tries to move any
    blocking network wait until after the source is pulled in turn from
    the preOpen queue.

    in order to take advantage of this change, jobs will probably need
    to make sure meshy.stream.prefetch is set to true, preOpen is large,
    and multiBundleReads is set relatively high (to try to pull as much
    available data from wraps that have already paid the initial wait -
    which tends to be the longest - before hitting the new wrap)