beta version of integrated test for crawler