Some minor optimizations when checking permissions

Change-Id: I46b7310302c3d6c1be7e3e0fa0879a1ad6b57ed3