add exceptions to crawler and indexer

Try/catch statement added in to deal with term too long issue, and
encoding issues.
checking in data dump

dump for docs and indexterms under db storage