PrintAllUris add to appassembler which will output a urls.html file with all the domains in the hbase table in the form of a search query. Could be used for load testing.
Merge remote-tracking branch 'forkOrigin/master'
started a new branch to extract text from warcbase data and organize the data to run topic modeling
Started a new branch to extract text from warcbase data and organize the data to run topic modeling.
removed dead code
Extract links and Lucene FST for URLs.
Simple MapReduce program to count number of unique URLs.
modify UriMappingBuilder to read all files under given directory
check text/html type and modify Jsoup.parse charset as ISO-8859-1
build hadoop job using Maven Assembly Plugin, modified pom.xml, added hadoop-job.xml
modify UriMappingBuilder in pom.xml
new hbase structure for ingest files
servlet updated with the new hbase structure.
Update README.md
Update README.md
Merge branch 'extract-links' of https://github.com/Jeffyrao/warcbase into working
Whitespace.
Tweaked POM
More POM tweaks to get MR jobs running correctly.
Update README.