added rating detection to crawler