Disable the perm panel in a few more situations. Closes #104