Yelp/mrjob — Issue #388: log-parsing regexes should handle long lines
Author: davidmarin
Created: 2012-03-07 21:21:16+00:00
State: open
Labels: Bug

It's not the greatest idea in the world to dump a 3MB data structure into a Hadoop counter, but people do apparently do it from time to time.

Unfortunately, our log-parsing regexes allow for a lot of backtracking, so we can end up taking a very large amount of time to parse a single long line.

Basically, we need less `".*?"` and more `"([^"\\]|\.)*"`.