crawler updated for regex