Restrict assignment access to owner and admins