As you get always `Collection` object, you can just call:
```php
$quantityDifference = $quantity - $order->getInventoryUnitsByVariant($variant)->count();