small updates to crawler