Yelp/mrjob — Issue #361: SkipBadRecords --- missing in mrjob?
Author: tokarina
Created: 2012-02-10 09:28:54+00:00
State: open

Is there a way to utilize Hadoop's SkipBadRecords functionality using mrjob?

RIght now, for a particular job, I'm running my map and reduce steps separately because there are a couple bad records that get caught in the map step and cause reduce to get canceled entirely.

Did I just not read the right tutorial? Or is this feature missing? Or is it not possible in streaming?

I can work on a patch if someone points me in the right direction.

Thanks.