SlowGuess/NeurIPS_2023_OCR huggingface dataset:
More Exploration of PMDP Tea-skip4 is a special initialization method, which chooses the skipped four layers (layer-3/6/9/12) of the teacher (MixViT-B) for initialization. In other words, Tea-skip4 is an extreme case of ours PMDP when the eliminating epoch  $m$  equal to 0. So it is reasonable that Tea-skip4 performs better than the baseline Tea-fir4, which employs the first four layers of the teacher (MixViT-B) to initialize the student backbone. In Table 8b, we further evaluate the performance on more benchmarks. It can be seen that ours PMDP surpasses Tea-skip4 by  $1.0\%$  on LaSOT_ext, which demonstrate its effectiveness.
Computation Loads of Different Localization Head We showcase the FLOPs of different heads as follows. Formally, we denote  $C_{in}$  as input feature dimension,  $C_{out}$  as output feature dimension,  $H_{in}, W_{in}$  as input feature map shape of convolution layer,  $H_{out}, W_{out}$  as output feature map shape, and  $K$  as the convolution kernel size. The computational complexity of one linear layer is  $O(C_{in}C_{out})$ , and that of one convolutional layer is  $O(C_{in}C_{out}H_{out}W_{out}K^{2})$ .
In our situation, for T4, the Localization Head contains four MLP to predict four coordinates. Each MLP contains two linear layers, whose input and output dimensions are all 768. The loads can be calculated as:
$$
L o a d _ {T 4} = 4 \times (7 6 8 \times 7 6 8 + 7 6 8 \times 7 2) = 2 5 8 0 4 8 0 \sim 2. 5 M
$$
For Py-Corner, totally 24 convolution layers are used. The loads can be calculated as:
$$
L o a d _ {P y - C o r n e r} = 2 * (7 6 8 * 3 8 4 * 1 8 * 1 8 * 3 * 3 +
$$
$$
\begin{array}{l} 3 8 4 * 1 9 2 * 1 8 * 1 8 * 3 * 3 + \\ 3 8 4 * 1 9 2 * 1 8 * 1 8 * 3 * 3 + \\ 1 9 2 * 9 6 * 3 6 * 3 6 * 3 * 3 + \\ 3 8 4 * 9 6 * 1 8 * 1 8 * 3 * 3 + \\ 9 6 * 4 8 * 7 2 * 7 2 * 3 * 3 + \\ 4 8 * 1 * 7 2 * 7 2 * 3 * 3 + \\ 1 9 2 * 9 6 * 1 8 * 1 8 * 3 * 3 + \\ 9 6 * 4 8 * 1 8 * 1 8 * 3 * 3 + \\ 4 8 * 1 * 1 8 * 1 8 * 3 * 3 + \\ 9 6 * 4 8 * 3 6 * 3 6 * 3 * 3 + \\ 4 8 * 1 * 3 6 * 3 6 * 3 * 3) \\ = 3 9 0 2 5 8 7 7 7 6 \sim 3. 9 B \\ \end{array}
$$
For simplicity, we do not include some operations such as bias terms and Layer/Batch-Normalization, which does not affect the overall calculation load level. Besides, the Pyramid Corner Head utilize additional ten interpolation operations. Obviously the calculation load of Py-Corner is still hundreds of times of T4.
S.4 Visualization Results
Visualization of Attention Map To explore how the introduced learnable prediction tokens work within the P-MAM, we visualize the attention maps of prediction-token-to-search and prediction-token-to-template in Fig. 5 and Fig. 6, where the prediction tokens are served as query and the others as key/value of the attention operation. From the visualization results, we can arrive that the four prediction tokens are sensitive to corresponding part of the targets and thus yielding a compact object bounding box. We suspect that the performance gap between the dense corner head based MixViT-B and our fully transformer MixFormerV2-B without distillation lies in the lack of holistic target modeling capability. Besides, the prediction tokens tend to extract partial target information in both the template and the search so as to relate the two ones.
Visualization of Predicted Probability Distribution We show two good cases and bad cases in Figure 7. In Figure 7a MixFormerV2 deals with occlusion well and locate the bottom edge correctly. As show in Figure 7b, the probability distribution of box representation can effectively alleviate issue of ambiguous boundaries. There still exist problems like strong occlusion and similar objects which will lead distribution shift, as demonstrated in Figure 7c and 7d.
Figure 5: Visualization of prediction-token-to- Figure 6: Visualization of prediction-token-tosearch attention maps, where the prediction to- template attention maps, where the prediction tokens are served as query of attention operation. kens are served as query of attention operation.
Figure 7: In each figure, the left one is plot of the probability distribution of predicted box (red), which demonstrates how our algorithm works. The right one is heatmap of attention weights in the backbone. The examples are from LaSOT test subset and the green boxes are ground truths.