Remove the default key binding for toggling the numbers

Please consider removing the default key binding. I don't use this feature and only messes up my existing key binding for C-n. Since it's still configurable, everybody who wants to have this key binding can still enable it.