Clean up some crashes in client side handling of voxels