crawler tests and fix