Changed the way the site test crawler work.

Signed-off-by: David Saradini <david.saradini@me.com>