SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Weijian Luo *
Boya Zhang†
Efficiently sampling from un-normalized target distributions is a fundamental problem in scientific computing and machine learning. Traditional approaches such as Markov Chain Monte Carlo (MCMC) guarantee asymptotically unbiased samples from such distributions but suffer from computational inefficiency, particularly when dealing with high-dimensional targets, as they require numerous iterations to generate a batch of samples. In this paper, we introduce an efficient and scalable neural implicit sampler that overcomes these limitations. The implicit sampler can generate large batches of samples with low computational costs by leveraging a neural transformation that directly maps easily sampled latent vectors to target samples without the need for iterative procedures. To train the neural implicit samplers, we introduce two novel methods: the KL training method and the Fisher training method. The former method minimizes the Kullback-Leibler divergence, while the latter minimizes the Fisher divergence between the sampler and the target distributions. By employing the two training methods, we effectively optimize the neural implicit samplers to learn and generate from the desired target distribution. To demonstrate the effectiveness, efficiency, and scalability of our proposed samplers, we evaluate them on three sampling benchmarks with different scales. These benchmarks include sampling from 2D targets, Bayesian inference, and sampling from high-dimensional energy-based models (EBMs). Notably, in the experiment involving high-dimensional EBMs, our sampler produces samples that are comparable to those generated by MCMC-based methods while being more than 100 times more efficient, showcasing the efficiency of our neural sampler. Besides the theoretical contributions and strong empirical performances, the proposed neural samplers and corresponding training methods will shed light on further research on developing efficient samplers for various applications beyond the ones explored in this study.
Efficiently sampling from un-normalized distributions is a fundamental challenge that arises in various research domains, including Bayesian statistics [1], biology and physics simulations [2, 3], as well as generative modeling and machine learning [4, 5]. The task at hand involves generating batches of samples from a target distribution defined by a differentiable un-normalized potential function, denoted as  $\log q(x)$ . This problem seeks effective techniques for obtaining samples that accurately represent the underlying target distribution while minimizing computational costs.
Two main classes of methods exist for addressing this challenge. The first class comprises Markov Chain Monte Carlo (MCMC) algorithms [6-9]. MCMC methods involve the design of Markov Chains with stationary distributions that match the target distribution. By simulating these Markov
*School of Mathematical Sciences; Peking University; luoweijian@stu.pku.edu.cn;
$^{\dagger}$  Academy for Advanced Interdisciplinary Studies; Peking University; zhangboya@pku.edu.cn;
Chains, it becomes possible to transition a batch of initial samples towards samples approximately distributed following the target distribution. While MCMC methods offer asymptotically unbiased samples, they often suffer from computational inefficiency, particularly when dealing with high-dimensional target distributions. This inefficiency stems from the need to perform a significant number of iterations to generate a single batch of samples.
The second class of methods, known as learning to sample (L2S) models [10-18], focuses on training neural components, such as generative models, to generate samples following the target distribution. L2S models aim to improve the quality and efficiency of sample generation compared to traditional training-free algorithms like MCMCs. Among the L2S models, the neural implicit sampler stands out due to its remarkable efficiency. The neural implicit sampler is an L2S model that incorporates a neural transformation  $\pmb{x} = g_{\theta}(\pmb{z})$ , where  $\pmb{z}$  is an easy-to-sample latent vector from a known distribution  $p_z(\pmb{z})$ . This transformation allows the sampler to generate samples directly from the target distribution without the need for iterations or gradient operations, which are commonly required in MCMCs. Such a sampler offers significant advantages in terms of efficiency and scalability, particularly when dealing with high-dimensional distributions such as those found in image spaces. However, training a neural implicit sampler poses technical challenges due to the intractability of sampler distributions that are caused by complex neural architectures of  $g_{\theta}$ . Achieving effective training requires careful consideration of various factors and the development of appropriate training strategies.
In this paper, we introduce two innovative training approaches, namely the KL training methods and the Fisher training methods, for training implicit samplers. We theoretically demonstrate that our proposed KL and Fisher training methods are equivalent to minimizing the Kullback-Leibler divergence and the Fisher divergence [19], respectively, between the sampler and the target distribution. To evaluate the effectiveness, efficiency, and scalability of our proposed training approach and sampler, we conduct experiments on three different sampling benchmarks that span various scales. These benchmarks include sampling from 2D targets with low dimensions, Bayesian inference with moderate dimensions, and sampling from high-dimensional energy-based models (EBMs) trained on the MNIST dataset. The results of experiments consistently demonstrate the superior performance of our proposed method and the quality of the generated samples. Notably, in the EBM experiment, our sampler trained with the KL training method achieves sample quality comparable to that of the EBM model while being more than 100 times more efficient than traditional MCMC methods. These findings highlight the effectiveness, efficiency, and scalability of the proposed approach across different sampling scenarios.
2 Backgrounds
2.1 Generative Models for Sampling
Generative models have emerged as powerful tools in various domains, demonstrating their ability to produce diverse, high-quality samples. They have been successfully applied in tasks such as text-to-image generation [20-28], audio generation[29], video and 3D creation[30-33], and even molecule design [34, 35]. Recently, there has been a growing interest in leveraging generative models for sampling from target distributions.
Specifically, consider the problem that we have access to an un-normalized target distribution  $q(\pmb{x})$ , or its logarithm  $\log q(\pmb{x})$ . Our objective is to train generative models that can effectively generate samples following the target distribution. By learning the underlying structure of the target distribution, these generative models can generate samples that capture the characteristics of the target distribution, facilitating tasks such as posterior inference in Bayesian statistics.
Three primary classes of generative models have been extensively studied for sampling tasks. The first class comprises normalizing flows (NFs) [36], while the second class consists of diffusion models (DMs) [37]. NFs employ invertible neural transformations to map Gaussian latent vectors  $\mathbf{z}$  to obtain samples  $\mathbf{x}$ . The strict invertibility of NF transformations enables the availability of likelihood values for generated samples, which are differentiable with respect to the model's parameters. Training NFs often involves minimizing the KL divergence between the NF and the target distribution [12]. On the other hand, DMs employ neural score networks to model the marginal score functions of a data-initialized diffusion process. The score functions are learned using techniques related to score-matching [38]. DMs have been successfully employed to enhance the sampler per
formance of the annealed importance sampling algorithm [39], a widely recognized MCMC method for various sampling benchmarks. Despite the successes of NFs and DMs, both models have their limitations. The invertibility of NFs restricts their expressiveness, which can hinder their ability to effectively model high-dimensional targets. Moreover, DMs still require a considerable number of iterations for sample generation, resulting in computational inefficiency.
The third class is the implicit generative model. An implicit generative model (IGM) uses a flexible neural transform  $g_{\theta}(.)$  to push forward easy-to-sample latent vectors  $z \sim p_z$  to obtain samples  $\boldsymbol{x} = g_{\theta}(\boldsymbol{z})$ . The main difference between IGMs and NFs is that IGMs' neural transformation is not required to be strictly invertible, which unlocks both the flexibility and the modeling power of deep neural networks. For instance, Hu et al. [11] proposed to train implicit samplers by minimizing the Stein discrepancy between the sampler and the target distribution. The Stein Discrepancy (SD) [40] between  $p$  and  $q$  is defined as
$$
\mathcal {D} _ {S D} := \sup  _ {\boldsymbol {f} \in \mathcal {F}} \left\{\mathbb {E} _ {p} \langle \nabla_ {\boldsymbol {x}} \log q (\boldsymbol {x}), \boldsymbol {f} (\boldsymbol {x}) \rangle + \sum_ {d = 1} ^ {D} \frac {\partial}{\partial \boldsymbol {x} _ {d}} f _ {d} (\boldsymbol {x}) \right\}.
$$
The notation  $D$  represents the dimension of data space and  $f_{d}(.)$  is the  $d$ -th component of the vector-valued function  $\pmb{f}$ . The calculation of Stein's discrepancy relies on solving a maximization problem w.r.t. the test function  $\pmb{f}$ . When the function class  $\mathcal{F}$  is carefully chosen, the optimal  $\pmb{f}$  may have an explicit solution or easier formulation. For instance, Hu et al. [11] found that if  $\mathcal{F}$  is taken to be  $\mathcal{F} = \{\pmb{f} \colon \mathbb{E}_{\pmb{x} \sim p} \| \pmb{f}(\pmb{x}) \|_2^2 \leq \delta\}$ , the SD is equivalent to a regularized representation
$$
\mathcal {D} _ {S D} (p, q) = \max _ {f} \Bigg \{\mathbb {E} _ {\boldsymbol {x} \sim p} \langle \nabla_ {\boldsymbol {x}} \log q (\boldsymbol {x}), \boldsymbol {f} (\boldsymbol {x}) \rangle + \sum_ {d = 1} ^ {D} \frac {\partial}{\partial \boldsymbol {x} _ {d}} f _ {d} (\boldsymbol {x}) - \lambda \| \boldsymbol {f} (\boldsymbol {x}) \| _ {2} ^ {2} \Bigg \}.
$$
They used two neural networks:  $g_{\theta}$  to parametrize an implicit sampler and  $f_{\eta}$  to parametrize the test function. Let  $p_{\theta}(\pmb{x})$  denote the implicit sampler distribution induced by  $\pmb{x} = g_{\theta}(\pmb{z})$  with  $z \sim p_z(z)$ . They solved a bi-level mini-max problem on parameter pair  $(\theta, \eta)$  to obtain a sampler that tries to minimize the SD with
$$
\min  _ {\theta} \max  _ {\eta} L (\theta , \eta)
$$
$$
L (\theta , \eta) = \mathbb {E} _ {p _ {\theta}} \left\langle \nabla_ {\boldsymbol {x}} \log q (\boldsymbol {x}), \boldsymbol {f} _ {\eta} (\boldsymbol {x}) \right\rangle + \sum_ {d = 1} ^ {D} \frac {\partial}{\partial \boldsymbol {x} _ {d}} f _ {d} (\boldsymbol {x}) - \lambda \| \boldsymbol {f} _ {\eta} (\boldsymbol {x}) \| _ {2} ^ {2}. \tag {1}
$$
Hu et al. [11] opened the door to training implicit samplers by minimizing divergences which are implemented with a two-networks bi-level optimization problem. Since the general Stein's discrepancy does not have explicit formulas, in our paper we study training approaches of an implicit sampler by minimizing the KL divergence and the Fisher divergences. For the implementation of minimization of Fisher divergence, some backgrounds of score function estimation are needed as we put in Section 2.2.
2.2 Score Function Estimation
Since the implicit sampler does not have an explicit log-likelihood function or score function, training it with score-based divergence requires inevitably estimating the score function (or equivalent component). Score matching [41] and its variants provided practical approaches to estimating score functions. Assume one only has available samples  $\boldsymbol{x} \sim p$  and wants to use a parametric approximated distribution  $q_{\phi}(\boldsymbol{x})$  to approximate  $p$ . Such an approximation can be made by minimizing the Fisher Divergence between  $p$  and  $q_{\phi}$  with the definition
$$
\mathcal {D} _ {F D} (p, q _ {\phi}) := \mathbb {E} _ {\boldsymbol {x} \sim p} \Bigg \{\| \nabla_ {\boldsymbol {x}} \log p (\boldsymbol {x}) \| _ {2} ^ {2} + \| \nabla_ {\boldsymbol {x}} \log q _ {\phi} (\boldsymbol {x}) \| _ {2} ^ {2} - 2 \langle \nabla_ {\boldsymbol {x}} \log p (\boldsymbol {x}), \nabla_ {\boldsymbol {x}} \log q _ {\phi} (\boldsymbol {x}) \rangle \Bigg \}.
$$
Under certain conditions, the equality
$$
\mathbb {E} _ {\boldsymbol {x} \sim p} \langle \nabla_ {\boldsymbol {x}} \log p (\boldsymbol {x}), \nabla_ {\boldsymbol {x}} \log q _ {\phi} (\boldsymbol {x}) \rangle = - \mathbb {E} _ {\boldsymbol {x} \sim p} \Delta_ {\boldsymbol {x}} \log q _ {\phi} (\boldsymbol {x})
$$
holds (usually referred to as Stein's Identity[42]). Here  $\Delta_{\pmb{x}}\log q_{\phi}(\pmb {x}) = \sum_{i}\frac{\partial^{2}}{\partial\pmb{x}_{i}^{2}}\log q_{\phi}(\pmb {x})$  denotes the Laplacian operator applied on  $\log q_{\phi}(\pmb {x})$ . Combining this equality and noting that the first term
of FD  $\mathbb{E}_{\boldsymbol{x}\sim p}\| \nabla_{\boldsymbol{x}}\log p(\boldsymbol {x})\| _2^2$  does not rely on the parameter  $\phi$ , minimizing  $\mathcal{D}_{FD}(p,q_{\phi})$  is equivalent to minimizing the following objective
$$
\mathcal {L} _ {S M} (\phi) = \mathbb {E} _ {\boldsymbol {x} \sim p} \left\{\| \nabla_ {\boldsymbol {x}} \log q _ {\phi} (\boldsymbol {x}) \| _ {2} ^ {2} + 2 \Delta_ {\boldsymbol {x}} \log q _ {\phi} (\boldsymbol {x}) \right\}.
$$
This objective can be estimated only through samples from  $p$ , thus is tractable when  $q_{\phi}$  is well-defined. Moreover, one only needs to define a score network  $s_{\phi}(\pmb{x}) \colon \mathbb{R}^{D} \to \mathbb{R}^{D}$  instead of a density model to represent the parametric score function. This technique was named after Score Matching. Other variants of score matching were also studied [43-48]. Score Matching related techniques have been widely used in the domain of generative modeling, so we use them to estimate the score function of the sampler's distribution.
3 Training Approaches of Neural Implicit Samplers
The neural implicit sampler has the advantage of sampling efficiency and scalability. But the training approach for such samplers still requires to explore. In this section, we start by introducing the proposed KL and Fisher training methods for implicit samplers.
3.1 Training Neural Implicit Sampler by Minimizing the KL Divergence
Let  $g_{\theta}(\cdot) \colon \mathbb{R}^{Dz} \to \mathbb{R}^{Dx}$  be an implicit sampler (i.e., a neural transform),  $p_z$  the latent distribution,  $p_{\theta}$  the sampler induced distribution  $x = g_{\theta}(z)$ , and  $q(x)$  the un-normalized target. Our goal in this section is to train the implicit sampler by minimizing the KL divergence between the sampler distribution and the un-normalized target. For a neural implicit sampler, we can efficiently sample from it but the density  $p_{\theta}$  has no explicit expression. Our goal is to minimize the KL divergence between  $p_{\theta}$  and target  $q$  in order to train the sampler  $g_{\theta}$ . The KL divergence between  $p_{\theta}$  and  $q$  is defined as
$$
\mathcal {D} ^ {(K L)} \left(p _ {\theta}, q\right) := \mathbb {E} _ {\boldsymbol {x} \sim p _ {\theta}} \left[ \log p _ {\theta} (\boldsymbol {x}) - \log q (\boldsymbol {x}) \right] = \mathbb {E} _ {\boldsymbol {z} \sim p _ {z}} \left[ \log p _ {\theta} \left(g _ {\theta} (\boldsymbol {z})\right) - \log q \left(g _ {\theta} (\boldsymbol {z})\right) \right]. \tag {2}
$$
In order to update the sampler  $g_{\theta}$ , we need to calculate the gradient of  $\theta$  for the divergence equation 2. So we take the gradient of  $\theta$  with respect to the KL divergence equation 2 and obtain
$$
\begin{array}{l} \frac {\partial}{\partial \theta} \mathcal {D} ^ {(K L)} (p _ {\theta}, q) \\ = \mathbb {E} _ {\boldsymbol {z} \sim p _ {z}} \left[ \nabla_ {\boldsymbol {x}} \log p _ {\theta} (g _ {\theta} (\boldsymbol {z})) - \nabla_ {\boldsymbol {x}} \log q (g _ {\theta} (\boldsymbol {z})) \right] \frac {\partial g _ {\theta} (\boldsymbol {z})}{\partial \theta} + \mathbb {E} _ {\boldsymbol {z} \sim p _ {z}} \left[ \frac {\partial \log p _ {\theta} (\boldsymbol {x})}{\partial \theta} | _ {\boldsymbol {x} = g _ {\theta} (\boldsymbol {z})} \right]. \\ \end{array}
$$
The second term vanishes under loose conditions (we put detailed discussions in the Appendix A.1)
$$
\begin{array}{l} \mathbb {E} _ {\boldsymbol {z} \sim p _ {z}} \left[ \frac {\partial \log p _ {\theta} (\boldsymbol {x})}{\partial \theta} | _ {\boldsymbol {x} = g _ {\theta} (\boldsymbol {z})} \right] = \mathbb {E} _ {\boldsymbol {x} \sim p _ {\theta}} \left[ \frac {\partial \log p _ {\theta} (\boldsymbol {x})}{\partial \theta} \right] = \int \frac {\partial p _ {\theta} (\boldsymbol {x})}{\partial \theta} d \boldsymbol {x} \\ = \frac {\partial}{\partial \theta} \int p _ {\theta} (\boldsymbol {x}) d \boldsymbol {x} = \frac {\partial}{\partial \theta} 1 = \mathbf {0}. \\ \end{array}
$$
So the gradient of the KL term only remains the first term and the gradient can be calculated with
$$
\operatorname {G r a d} ^ {(K L)} (\theta) := \mathbb {E} _ {\boldsymbol {z} \sim p _ {z}} \left[ \left[ \boldsymbol {s} _ {p} \left(g _ {\theta} (\boldsymbol {z})\right) - \boldsymbol {s} _ {q} \left(g _ {\theta} (\boldsymbol {z})\right) \right] \frac {\partial g _ {\theta} (\boldsymbol {z})}{\partial \theta} \right], \tag {3}
$$
where  $s_q \coloneqq \nabla_x \log q(\pmb{x})$  is the score function of target distribution so we have the explicit expression. The  $s_p \coloneqq \nabla_x \log p_\theta(\pmb{x})$  is the score function of the implicit sampler but we do not have the explicit expression. However, thanks to advanced techniques of neural score function estimation as we introduced in 2.2, we can use another score neural network to estimate the score function  $s_p(\pmb{x}) \approx s_\phi(\pmb{x})$  with samples consistently drawn from  $p_\theta$ . With the estimated  $s_p(\pmb{x})$ , the formula equation 3 provides a practical method for updating the implicit sampler that is shown to minimize the KL divergence.
More precisely, we can alternate between a score-estimation phase and a KL minimization phase in order to minimize the KL divergence between the sampler and the target distribution. The former phase uses score estimation techniques to train a score network  $s_{\phi}$  to approximate the sampler's
score function with samples consistently generated from the implicit sampler. In the latter phase, we can minimize the KL divergence with gradient-based optimization algorithms according to gradient formula equation 3. By alternating the optimization of a score network  $s_{\phi}$  and implicit sampler  $g_{\theta}$ , one can eventually train an implicit sampler that can approximately generate samples that are distributed according to the target distribution. We name such method the KL training method.
3.2 Training Neural Implicit Sampler by Minimizing Fisher Divergence
In the previous section, we proposed a KL training method for implicit samplers. In this section, we proposed an alternative training method motivated by minimizing the Fisher divergence instead of the KL. We use the same notation as the section 3.1. Recall the definition of the Fisher divergence,
$$
\mathcal {D} ^ {(F)} \left(p _ {\theta}, q\right) := \mathbb {E} _ {p _ {\theta}} \frac {1}{2} \| \nabla_ {\boldsymbol {x}} \log p _ {\theta} (\boldsymbol {x}) - \nabla_ {\boldsymbol {x}} \log q (\boldsymbol {x}) \| _ {2} ^ {2} = \mathbb {E} _ {\boldsymbol {z} \sim p _ {z}} \frac {1}{2} \| \boldsymbol {s} _ {\theta} \left(g _ {\theta} (\boldsymbol {z})\right) - \boldsymbol {s} _ {q} \left(g _ {\theta} (\boldsymbol {z})\right) \| _ {2} ^ {2}. \tag {4}
$$
The score functions  $s_{\theta}$  and  $s_q$  remain the same meaning as in the equation 3. For an implicit sampler, we do not know the explicit expression of  $s_{\theta}$ . In order to minimize the Fisher divergence, we take the  $\theta$  gradient of equation 4 and obtain
$$
\begin{array}{l} \frac{\partial}{\partial\theta}\mathcal{D}^{(F)}(p_{\theta},q) = \mathbb{E}_{\substack{\boldsymbol {x} = g_{\theta}(\boldsymbol {z}),\\ \boldsymbol {z}\sim p_{z}}}\left[\boldsymbol{s}_{\theta}(\boldsymbol {x}) - \boldsymbol{s}_{q}(\boldsymbol {x})\right]\left[\frac{\partial\boldsymbol{s}_{\theta}(\boldsymbol{x})}{\partial\boldsymbol{x}} -\frac{\partial\boldsymbol{s}_{q}(\boldsymbol{x})}{\partial\boldsymbol{x}}\right]\frac{\partial\boldsymbol{x}}{\partial\theta} \\ + \mathbb {E} _ {\boldsymbol {z} \sim p _ {z}} \left[ \boldsymbol {s} _ {\theta} \left(g _ {\theta} (\boldsymbol {z})\right) - \boldsymbol {s} _ {q} \left(g _ {\theta} (\boldsymbol {z})\right) \right] \frac {\partial \boldsymbol {s} _ {\theta} (\boldsymbol {x})}{\partial \theta} | _ {\boldsymbol {x} = g _ {\theta} (\boldsymbol {z})} \tag {5} \\ = \operatorname {G r a d} ^ {(F, 1)} (\theta) + \operatorname {G r a d} ^ {(F, 2)} (\theta). \\ \end{array}
$$
We use  $s_{\theta^{\dagger}}$  to represent a function that does not differentiate with respect to parameter  $\theta$ , so the first term gradient equation 5 is equivalent to taking the gradient of an equivalent objective
$$
\mathcal {L} ^ {(F, 1)} (\theta) = \mathbb {E} _ {\boldsymbol {z} \sim p _ {z}} \frac {1}{2} \| \boldsymbol {s} _ {\theta^ {\dagger}} (g _ {\theta} (\boldsymbol {z})) - \boldsymbol {s} _ {q} (g _ {\theta} (\boldsymbol {z})) \| _ {2} ^ {2}. \tag {6}
$$
As for the second term of equation 5, we notice that under regularity conditions
$$
\frac {\partial \boldsymbol {s} _ {\theta} (\boldsymbol {x})}{\partial \theta} = \frac {\partial}{\partial \theta} \frac {\partial}{\partial \boldsymbol {x}} \log p _ {\theta} (\boldsymbol {x}) = \frac {\partial}{\partial \boldsymbol {x}} \frac {\partial}{\partial \theta} \log p _ {\theta} (\boldsymbol {x}).
$$
So the second term of equation 5 equals to
$$
\begin{array}{l} \operatorname {G r a d} ^ {(F, 2)} (\theta) = \mathbb {E} _ {\boldsymbol {x} \sim p _ {\theta}} \left[ \boldsymbol {s} _ {\theta} (\boldsymbol {x}) - \boldsymbol {s} _ {q} (\boldsymbol {x}) \right] \frac {\partial}{\partial \theta} \frac {\partial}{\partial \boldsymbol {x}} \log p _ {\theta} (\boldsymbol {x}) = \int \left[ \boldsymbol {s} _ {\theta} (\boldsymbol {x}) - \boldsymbol {s} _ {q} (\boldsymbol {x}) \right] p _ {\theta} (\boldsymbol {x}) \frac {\partial}{\partial \boldsymbol {x}} \frac {\partial}{\partial \theta} \log p _ {\theta} (\boldsymbol {x}) d \boldsymbol {x} \\ = - \frac {\partial}{\partial \theta} \mathbb {E} _ {p _ {\theta}} \left[ \boldsymbol {s} _ {\theta^ {\dagger}} ^ {T} (\boldsymbol {x}) \left[ \boldsymbol {s} _ {\theta^ {\dagger}} (\boldsymbol {x}) - \boldsymbol {s} _ {q} (\boldsymbol {x}) \right] + \nabla_ {\boldsymbol {x}} \left[ \boldsymbol {s} _ {\theta^ {\dagger}} (\boldsymbol {x}) - \boldsymbol {s} _ {q} (\boldsymbol {x}) \right] \right]. \tag {7} \\ \end{array}
$$
We put detailed derivation and required conditions in Appendix A.2. With the expression of the  $\mathrm{Grad}^{(F,2)}(\theta)$ , minimizing the second part of  $\mathcal{D}^{(F)}(\theta)$  is equivalent to minimizing an equivalent objective
$$
\mathcal {L} ^ {(F, 2)} (\theta) = - \mathbb {E} _ {\boldsymbol {z} \sim p _ {z}} \left[ \boldsymbol {s} _ {\theta^ {\dagger}} ^ {T} \left(g _ {\theta} (\boldsymbol {z})\right) \left[ \boldsymbol {s} _ {\theta^ {\dagger}} \left(g _ {\theta} (\boldsymbol {z})\right) - \boldsymbol {s} _ {q} \left(g _ {\theta} (\boldsymbol {z})\right) \right] + \nabla_ {\boldsymbol {x}} \left[ \boldsymbol {s} _ {\theta^ {\dagger}} \left(g _ {\theta} (\boldsymbol {z})\right) - \boldsymbol {s} _ {q} \left(g _ {\theta} (\boldsymbol {z})\right) \right] \right]. \tag {8}
$$
Combining equivalent loss function equation 6 and equation 8, we obtain a final objective function for training implicit sampler that minimizes the Fisher divergence