Always run xmodmap in xinitrc.