Michael's changes: logging, file input.
Added try-catch block around the call to boilerpipe.
Added incrementing of current document on None returns so we have accurate currentDocument count.
Added another WARC record for boilerpipe exception.
Apparently it's not an exception. Now we just catch everything.
Now skips files that have a corresponding output already; comments
Fudged around with the indentation, moved creation of processors to outside of for loop.
Just the Apache 2.0 license for now.
Initial commit of notice
Added apache license header; some comments; checks that the payload isn't empty now.
Removed <1MB requirement; added some try/catch for data input stream reads; removed creating two of the same new byte arrays
Removed profile for sentencer
Added a lot of try-catch blocks -- bad style (in my mind), but it'll work.
Merge branch 'master' of https://github.com/xdavidjung/cluewebextractor

Conflicts:
	src/main/scala/edu/washington/cs/knowitall/cluewebextractor/WarcRecordIterator.scala