something is rotten in the state of the common crawl script (not tokenizing properly I think)
fixed tokenization bug
fixed performance issues by reimplementing in pure pig + 1 java udf