Ignore VCS directories and local tmp directories
Make relative number toggle only happen on Ctrl+n, not on buffer changes