commoncrawl/commoncrawl-crawler — pull request #1 by matpalm (2013-01-13 07:53:25+00:00)
Title: input format for sequence files and streaming

input format for sequence files that assumes Text value and preps data for streaming (ie remove CRs, escapes LF and escapes tabs)

i have some toy python examples showing explicit I'll add to the wiki