Ok c++ references suck, changed all meshes to pointers. Apparently before every object was making a copy of the mesh object ...