Use PLUGIN for key-mappings and actually have both number and relativenumber set in relativenumber mode.