Early-Exit and Instant Confidence Translation Quality Estimation
Authors: name: Vilém Zouhar, name: Maike Züfle, name: Beni Egressy, name: Julius Cheng, name: Jan Niehues

Quality estimation is omnipresent in machine translation, for both evaluation
and generation. Unfortunately, quality estimation models are often opaque and
computationally expensive, making them impractical to be part of large-scale
pipelines. In this work, we tackle two connected challenges: (1) reducing the
cost of quality estimation at scale, and (2) developing an inexpensive
uncertainty estimation method for quality estimation. To address the latter, we
introduce Instant Confidence COMET, an uncertainty-aware quality estimation
model that matches the performance of previous approaches at a fraction of
their costs. We extend this to Early-Exit COMET, a quality estimation model
that can compute quality scores and associated confidences already at early
model layers, allowing us to early-exit computations and reduce evaluation
costs. We also apply our model to machine translation reranking. We combine
Early-Exit COMET with an upper confidence bound bandit algorithm to find the
best candidate from a large pool without having to run the full evaluation
model on all candidates. In both cases (evaluation and reranking) our methods
reduce the required compute by 50% with very little degradation in performance.