Handle freeing objects and finalization

This seems to allow ruby to build
Make git happy with ruby building

1. don't commit the ruby binary
2. remove files the ruby build removed