Remove acl when publisher role is removed.

This fixes the acl sync'ing code to push all new acls and then
difference to remove any missing roles.