Added explanation on how the crawler will work