"mainly prediction latency and throughput" (as one could also consider the training throughput but this often less impacting in a production setup).