Only show numbers for listed buffers

If we don't check if a buffer is listed, vim-numbertoggle will enable
numbers in buffers where it's not desired to have numbers at all (e.g.
in NERDTree, tagbar, help, etc.)