Fix server used for deleting model instances when using sharding

This was previously only handled correctly in the unoptimized case,
not in the optimized case.