Sadly yes. I've filed a bug to improve logging of the results since it impacts more than just the permissions tests.