Exception handling

add exception handling code to robust crawler