The gemspec problem

Today I started up a project with the intent to use goat. The project was not large yet, but I knew it was going to be. And that brought me crashing into an architectural issue with goat, which I call "the gemspec problem".

Imagine a world where Ruby is the same as it is now, except there are no gemspecs. Gems can still specify a version via a VCS tag, but since there's no gemspec, they can't specify what versions of other gems they depend on. Pop quiz: how do you implement bundler in this world?

The answer is that you can't. Your Gemfile can specify the versions of your top-level dependencies, but once Bundler fetches those, it has to ask you, "okay... what now?" The top level dependencies don't have gemspecs and bundler doesn't know what version to retrieve. So you have to specify all the versions for the dependencies of your dependencies. This continues recursively. In effect, you would have to specify the version of every single gem in your dependency tree to use bundler.

Now imagine using rails in this world. Every single Gemfile will look something like this: http://patshaughnessy.net/assets/2011/9/17/gem_graph.png This guy's dependency graph has over 40 gems in it, and that's just his asset pipeline. For fun, go check your Gemfile.lock and see how many unique names are in there. The Gemfile.lock for Redmine 2.5.2 has 52 of them. In this world, you now have to specify a version constraint for every single one of those gems.

The "gemspec problem" is what goat faces. goat is to bundler as .go.yaml is to gemspec, except gemspecs are a standard in Ruby and .go.yaml is not a standard in Go, and you can't implement bundler without ubiquitous gemspecs. You can greatly weaken the problem by fetching head/master/tip/trunk for all transitive dependencies, but right now that's just not enough for a large tree.

To be fair to goat, I think this is an issue that all recursive get implementations will inevitably need to face. How do you resolve the dependencies of your dependencies? One way or another, your dependencies need to tell you. And that eventually forces the community to settle on standard metadata or else find another, non-bundler-esque solution to the problem. Cough godep cough.

I need to think some more.