Remove security groups from the ODL agent.