SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} p (\mathbf {x}) p (\mathbf {y}) k _ {p} ^ {j} (\mathbf {x}, \mathbf {y}) = \nabla_ {x _ {j}} ^ {2} p (\mathbf {x}) \nabla_ {y _ {j}} ^ {2} p (\mathbf {y}) k (\mathbf {x}, \mathbf {y}) \\ + 2 \nabla_ {x _ {j}} ^ {2} p (\mathbf {x}) \nabla_ {y _ {j}} p (\mathbf {y}) \nabla_ {y _ {j}} k (\mathbf {x}, \mathbf {y}) + p (\mathbf {y}) \nabla_ {x _ {j}} ^ {2} p (\mathbf {x}) \nabla_ {y _ {j}} ^ {2} k (\mathbf {x}, \mathbf {y}) \\ + 2 \nabla_ {x _ {j}} p (\mathbf {x}) \nabla_ {y _ {j}} ^ {2} p (\mathbf {y}) \nabla_ {x _ {j}} k (\mathbf {x}, \mathbf {y}) + 4 \nabla_ {x _ {j}} p (\mathbf {x}) \nabla_ {y _ {j}} p (\mathbf {y}) \nabla_ {x _ {i}} \nabla_ {y _ {j}} k (\mathbf {x}, \mathbf {y}) \tag {12} \\ + 2 p (\mathbf {y}) \nabla_ {x _ {j}} p (\mathbf {x}) \nabla_ {x _ {j}} \nabla_ {y _ {j}} ^ {2} k (\mathbf {x}, \mathbf {y}) + p (\mathbf {x}) \nabla_ {y _ {j}} ^ {2} p (\mathbf {y}) \nabla_ {x _ {j}} ^ {2} k (\mathbf {x}, \mathbf {y}) \\ + 2 p (\mathbf {x}) \nabla_ {y _ {j}} p (\mathbf {y}) \nabla_ {x _ {j}} ^ {2} \nabla_ {y _ {j}} k (\mathbf {x}, \mathbf {y}) + p (\mathbf {x}) p (\mathbf {y}) \nabla_ {x _ {j}} ^ {2} \nabla_ {y _ {j}} ^ {2} k (\mathbf {x}, \mathbf {y}) \\ \end{array}
$$
A closed-form expression can then be obtained in the case of, e.g., an inverse multiquadratic kernel of the form  $k(\mathbf{x},\mathbf{y}) = (1 + \| \mathbf{x} - \mathbf{y}\|_{2}^{2} / \ell^{2})^{-1 / 2}$  where  $\ell$  denotes the bandwidth. In this case, one has
$$
\nabla_ {y _ {j}} k (\mathbf {x}, \mathbf {y}) = \frac {1}{\ell^ {2}} k (\mathbf {x}, \mathbf {y}) ^ {3} (x _ {j} - y _ {j}), \quad \nabla_ {x _ {j}} k (\mathbf {x}, \mathbf {y}) = - \nabla_ {y _ {j}} k (\mathbf {x}, \mathbf {y})
$$
$$
\nabla_ {y _ {j}} ^ {2} k (\mathbf {x}, \mathbf {y}) = \nabla_ {x _ {j}} ^ {2} k (\mathbf {x}, \mathbf {y}) = - \frac {1}{\ell^ {2}} k (\mathbf {x}, \mathbf {y}) ^ {3} + \frac {3}{\ell^ {4}} k (\mathbf {x}, \mathbf {y}) ^ {5} (x _ {j} - y _ {j}) ^ {2}
$$
$$
\nabla_ {x _ {j}} \nabla_ {y _ {j}} k (\mathbf {x}, \mathbf {y}) = \frac {1}{\ell^ {2}} k (\mathbf {x}, \mathbf {y}) ^ {3} - \frac {3}{\ell^ {4}} k (\mathbf {x}, \mathbf {y}) ^ {5} (x _ {j} - y _ {j}) ^ {2} \tag {13}
$$
$$
\nabla_ {x _ {j}} ^ {2} \nabla_ {y _ {j}} k (\mathbf {x}, \mathbf {y}) = \frac {- 9 k (\mathbf {x} , \mathbf {y}) ^ {5}}{\ell^ {4}} (x _ {j} - y _ {j}) + \frac {1 5 k (\mathbf {x} , \mathbf {y}) ^ {7}}{\ell^ {6}} (x _ {j} - y _ {j}) ^ {3}
$$
$$
\nabla_ {x _ {j}} \nabla_ {y _ {j}} ^ {2} k (\mathbf {x}, \mathbf {y}) = - \nabla_ {x _ {j}} ^ {2} \nabla_ {y _ {j}} k (\mathbf {x}, \mathbf {y})
$$
$$
\nabla_ {x _ {j}} ^ {2} \nabla_ {y _ {j}} ^ {2} k (\mathbf {x}, \mathbf {y}) = \frac {9 k (\mathbf {x} , \mathbf {y}) ^ {5}}{\ell^ {4}} - \frac {9 0 k (\mathbf {x} , \mathbf {y}) ^ {7}}{\ell^ {6}} (x _ {j} - y _ {j}) ^ {2} + \frac {1 0 5 k (\mathbf {x} , \mathbf {y}) ^ {9}}{\ell^ {8}} (x _ {j} - y _ {j}) ^ {4}
$$
A closed-form expression of  $k_{p}$  can then be obtained by combining Equations (10)-(13). In contrast to the Langevin Stein kernel (see Equation (2) of the main article), the above Stein kernel involves second-order derivatives of the density  $p$ .
We run experiments based on our Example 1 of Gaussian mixtures for this new Stein operator  $\mathcal{T}_p g = \Delta(pg) / p$ . We sequentially set  $\mu = 2$  and  $\mu = 5$ , with  $\sigma = 1$  and  $w = 0.5$ . Results are displayed in Figure 17. In the left panel with  $\mu = 2$ , we see that Pathology II does not occur, as opposed to Figure 2 of the main article with the Langevin Stein operator. However, when  $\mu$  is set to 5 in the right panel, all particles are concentrated in spurious minimum again. Therefore, introducing higher-order derivatives of the target density through the Stein operator does not seem to be a promising route.
Figure 17: Stein thinning with the Stein operator  $\mathcal{T}_p g = \Delta(pg) / p$ , for Gaussian mixtures of Example 1, with  $\mu = 2$  (left panel), and  $\mu = 5$  (right panel),  $\sigma = 1$ , and  $w = 0.5$ .