Some fixes for missing perms, and actually do the owner check