Refine the way that we do permissions work