Revert "Temporarily turn off index processing exception handling"

This reverts commit f4dea8f3993765e118d42f613eb8c41fa56323bc.
Added URL downloading/processing thread num command line option