cleaned up crawler. started auth