Clean up the gems, at least for now

I think that it is better to start with as little noise as possible.