updated crawler code
updated crawler test