Add exception to not look for file-specific vimrc if it's an empty buffer