wrapped all token processing in try except to look for index errors
added more failure tests