  In multi-object tracking, the tracker maintains in its memory the appearance
and motion information for each object in the scene. This memory is utilized
for finding matches between tracks and detections and is updated based on the
matching result. Many approaches model each target in isolation and lack the
ability to use all the targets in the scene to jointly update the memory. This
can be problematic when there are similar looking objects in the scene. In this
paper, we solve the problem of simultaneously considering all tracks during
memory updating, with only a small spatial overhead, via a novel multi-track
pooling module. We additionally propose a training strategy adapted to
multi-track pooling which generates hard tracking episodes online. We show that
the combination of these innovations results in a strong discriminative
appearance model, enabling the use of greedy data association to achieve online
tracking performance. Our experiments demonstrate real-time, state-of-the-art
performance on public multi-object tracking (MOT) datasets.
