Revert "Revert "Only show numbers for listed buffers""

This reverts commit c5a9e6d0290db4697084e35e1a4eacd1a09a26d1.