Stop including unnecessary files in .gem

As suggested by Rob Gleeson in discussion on #1147. Also removed an obsolete
dotfile, because why not.