add allow, disallow and controller for robots