Clean up .gitignore

I don't think I'm ever going to meet non-.swp Vim swap files, and even
if I am, I'm unlikely to commit them into the repository.