  In this paper, we propose and study Open-World Tracking (OWT). Open-world
tracking goes beyond current multi-object tracking benchmarks and methods which
focus on tracking object classes that belong to a predefined closed-set of
frequently observed object classes. In OWT, we relax this assumption: we may
encounter objects at inference time that were not labeled for training. The
main contribution of this paper is the formalization of the OWT task, along
with an evaluation protocol and metric (Open-World Tracking Accuracy, OWTA),
which decomposes into two intuitive terms, one for measuring recall, and
another for measuring track association accuracy. This allows us to perform a
rigorous evaluation of several different baselines that follow design patterns
proposed in the multi-object tracking community. Further we show that our
Open-World Tracking Baseline, while performing well in the OWT setting, also
achieves near state-of-the-art results on traditional closed-world benchmarks,
without any adjustments or tuning. We believe that this paper is an initial
step towards studying multi-object tracking in the open world, a task of
crucial importance for future intelligent agents that will need to understand,
react to, and learn from, an infinite variety of objects that can appear in an
open world.
