Remove attribute modifiers based on skills, since skill-based attribute modifiers have were removed some time ago.