SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Impact of different sequence lengths. Regarding the impact of sequence lengths, if the sequence is longer and has larger variations, generally the equivariance error would be worse. To better illustrate this point, we perform an ablation study on MNIST and present the results in Table 10. Specifically, we change the sequence length from 9 to 12, which increases the extent of scaling from maximally 1.8 times to maximally 2.1 times, and increases the rotation angle from maximally 80 degrees to maximally 110 degrees. As can be observed, the equivariance error gets larger when the sequence becomes longer and the variations are larger. Notice that even for the longer sequence, our method still outperforms other baselines with shorter sequences.
A.5 HJ equations as dynamic optimal transport
We now turn to introduce why HJ equations could minimize the Wasserstein distance. As stated in [4], the  $L_{2}$  Wasserstein distance can be re-formulated in the fluid mechanical interpretation as
$$
W ^ {2} = \inf  \int_ {D} \int_ {0} ^ {1} \frac {1}{2} \rho (x, t) v (x, t) ^ {2} d x d t \tag {16}
$$
where the density satisfies the continuity equation  $(\partial_t\rho = -\nabla \cdot (\rho (x,t)v(x,t))$ . If we introduce the momentum  $m(x,t) = \rho (x,t)v(x,t)$  and two Lagrange multipliers  $u$  and  $\lambda$ , the Lagrangian function of the Wasserstein distance would be:
$$
L (\rho , m, \phi) = \int_ {D} \int_ {0} ^ {1} \frac {\left\| m \right\| ^ {2}}{2 \rho} + u \left(\partial_ {t} \rho + \nabla \cdot m\right) - \lambda \left(\rho - s ^ {2}\right) \tag {17}
$$
Figure 10: More visualizations of the learned latent flows on MNIST [54].
where the second term is the equality constraint, and the third term is an equality constraint with a slack variable  $s$ . Using integration by parts formula, the above equation can be re-written as
$$
L (\rho , m, \phi) = \int_ {D} \int_ {0} ^ {1} \frac {\left| | m | \right| ^ {2}}{2 \rho} + \int_ {D} u \rho | _ {0} ^ {1} - \int_ {D} \int_ {0} ^ {1} \left(\partial_ {t} u \rho + \nabla u \cdot m\right) - \lambda \left(\rho - s ^ {2}\right) \tag {18}
$$
Based on the set of Karush-Kuhn-Tucker (KKT) conditions  $(\partial_{m}L = 0, \partial_{u}L = 0, \partial_{\rho}L = 0,$  and  $\lambda \geq 0)$ , we would have:
$$
\left\{ \begin{array}{l} \partial_ {m} L = \frac {m}{\rho} - \nabla u = v - \nabla u = 0 \\ \partial_ {u} L = \partial_ {t} \rho + \nabla \cdot m = 0 \\ \partial_ {\rho} L = - \frac {| | m | | ^ {2}}{2 \rho^ {2}} - \partial_ {t} u - \lambda = - \frac {1}{2} | | v | | ^ {2} - \partial_ {t} u - \lambda = 0 \end{array} \right. \tag {19}
$$
where the first condition indicates that the gradient  $\nabla u$  acts as the velocity field, and the third condition implies the optimal solution is given by the generalized HJ equation:
$$
\partial_ {t} u + \frac {1}{2} | | \nabla u | | ^ {2} = - \lambda \leq 0 \tag {20}
$$
We thus apply the generalized HJ equation (i.e.,  $\partial_t u + \frac{1}{2} ||\nabla u||^2\leq 0$ ) as the constraints. We further use an extra negative force because this would give more dynamics for modeling the posterior flow.
A.6 More visualizations
Fig. 10, 11, and 12 display more visualization results of the latent evolution on MNIST, Shapes3D, Falcol3D and Isaac3D, respectively. Across all the datasets, our method presents precise control of the given transformations. Fig. 13 and 14 show more latent evolution results of switching transformations (top) and combining transformations (bottom) on MNIST and Shapes3D, respectively. Fig. 15 also visualizes a few examples of superposing and switching transformation on Falcol3D and Isaac3D. Our latent flows learn to compose or switch different transformations precisely and flexibly.
Figure 11: More visualizations of the learned latent flows on Shapes3D [10].
Figure 12: Exemplary visualizations of learned latent flows on Falcol3D (top) and Isaac3D (bottom).
Figure 13: More visualizations of switching and superposing transformations on MNIST [54].
Figure 14: More visualizations of switching and superposing transformations on Shapes3D [10].
Figure 15: Exemplary visualization results of superposing transformations on Isaac3D (left) and switching transformations on Falcol3D (right).