swipely/streamingly — pull request #6 by peakxu (2014-10-10 19:45:50+00:00)
Title: Make reducer catch errors so that processing will contiue in spite of lo...

...cal failures

@mattgillooly, @mguymon please review.
/cc @dstuebe

Context:
For a big data system, malformed input or other unexpected conditions may pop up a lot. Rather than having the reducer fail and stop on the first such input, it is often better to keep going and instead use some other mechanism to capture and document the failure (for instance, HoneyBadger exception reporting, etc.)

Solution:
This pull makes the reducer resilient to such errors.

Design tradeoffs:
1. Should this be make an option instead to ensure no changes to current behavior? I can see an argument for failing extremely loudly.
2. Should we add a limit on the number and/or percentage of failures allowed? This is to prevent systems like HoneyBadgers from being inundated w/ erroneous output.