Disassemble class into functions.

In addition, add crawler test and modify auto_discovery function