update adns, fixed a bug in larbin and add igoring robots.txt