Rely on Whitehall::Authority for reject perms

It's getting easier...