Some speed improvements to access rights handling