Leave the gemspec out of it for now too