user: text: Write a concise one-sentence TLDR summary of the scientific paper content below.

Title: Generalized Transformation-based Gradient

Paper content:
The reparameterization trick has become one of the most useful tools in the field of variational inference.
However, the reparameterization trick is based on the standardization transformation which restricts the scope of application of this method to distributions that have tractable inverse cumulative distribution functions or are expressible as deterministic transformations of such distributions.
In this paper, we generalized the reparameterization trick by allowing a general transformation.
We discover that the proposed model is a special case of control variate indicating that the proposed model can combine the advantages of CV and generalized reparameterization.
Based on the proposed gradient model, we propose a new polynomial-based gradient estimator which has better theoretical performance than the reparameterization trick under certain condition and can be applied to a larger class of variational distributions.
In studies of synthetic and real data, we show that our proposed gradient estimator has a significantly lower gradient variance than other state-of-the-art methods thus enabling a faster inference procedure.
Most machine learning objective function can be rewritten in the form of an expectation:
where θ is a parameter vector.
However, due to the intractability of the expectation, it's often impossible or too expensive to calculate the exact gradient w.r.t θ, therefore it's inevitable to estimate the gradient ∇ θ L in practical applications.
Stochastic optmization methods such as reparameterization trick and score function methods have been widely applied to address the stochastic gradient estimation problem.
Many recent advances in large-scale machine learning tasks have been brought by these stochastic optimization tricks.
Like in other stochastic optimzation related works, our paper mainly focus on variational inference tasks.
The primary goal of variational inference (VI) task is to approximate the posterior distribution in probabilistic models (Jordan et al., 1999; Wainwright & Jordan, 2008) .
To approximate the intractable posterior p(z|x) with the joint probability distribution p(x, z) over observed data x and latent random variables z given, VI introduces a parameteric family of distribution q θ (z) and find the best parameter θ by optimizing the Kullback-Leibler (KL) divergence D KL (q(z; θ) p(z|x)).
The performance of VI methods depends on the capacity of the parameteric family of distributions (often measured by Rademacher complexity) and the ability of the optimizer.
In this paper, our method tries to introduce a better optimizer for a larger class of parameteric family of distributions.
The main idea of our work is to replace the parameter-independent transformation in reparameterization trick with generalized transformation and construct the generalized transformation-based (G-TRANS) gradient with the velocity field which is related to the characteristic curve of the sublinear partial differential equation associated with the generalized transformation.
Our gradient model further generalizes the G-REP (Ruiz et al., 2016) and provides a more elegant and flexible way to construct gradient estimators.
We mainly make the following contributions:
1. We develop a generalized transformation-based gradient model based on the velocity field related to the generalized transformation and explicitly propose the unbiasedness constraint on the G-TRANS gradient.
The proposed gradient model provides a more poweful and flexible way to construct gradient estimators.
2. We show that our model is a generalization of the score function method and the reparameterization trick.
Our gradient model can reduce to the reparameterization trick by enforcing a transport equation constraint on the velocity field.
We also show our model's connection to control variate method.
3. We propose a polynomial-based gradient estimator that cannot be induced by any other existing generalized reparameterization gradient framework, and show its superiority over similar works on several experiments.
The rest of this paper is organized as follows.
In Sec.2 we review the stochastic gradient variational inference (SGVI) and stochastic gradient estimators.
In Sec.3 we propose the generalized transformation-based gradient.
In Sec.4 we propose the polynomial-based G-TRANS gradient estimator.
In Sec.5 we study the performance of our gradient estimator on synthetic and real data.
In Sec.6 we review the related works.
In Sec.7 we conclude this paper and discuss future work.
We proposed a generalized transformation-based (G-TRANS) gradient model which extends the reparameterization trick to a larger class of variational distributions.
Our gradient model hides the details of transformation by introducing the velocity field and provides a flexible way to construct gradient estimators.
Based on the proposed gradient model, we introduced a polynomial-based G-TRANS gradient estimator that cannot be induced by any other existing generalized reparameterization gradient framework.
In practice, our gradient estimator provides a lower gradient variance than other state-of-the-art methods, leading to a fast converging process.
For future work, We can consider how to construct G-TRANS gradient estimators for distributions that don't have analytical high-order moments.
We can also utilize the results from the approximation theory to find certain kinds of high-order polynomial functions that can approximate the test function effectively with cheap computations for the coefficients.
Constructing velocity fields with the optimal transport theory is also a promising direction.

assistant: text: a generalized transformation-based gradient model for variational inference