FLUID FLOW MASS TRANSPORT FOR GENERATIVE NETWORKS
Authors: Jingrong Lin
Keegan Lensink
Eldad Haber
Venue:  (2020)
Keywords: generative network
optimal mass transport
gaussian mixture
model matching

Abstract: Generative Adversarial Networks have been shown to be powerful tools for generating content resulting in them being intensively studied in recent years. Training these networks requires maximizing a generator loss and minimizing a discriminator loss, leading to a difficult saddle point problem that is slow and difficult to converge. Motivated by techniques in the registration of point clouds and the fluid flow formulation of mass transport, we investigate a new formulation that is based on strict minimization, without the need for the maximization. This formulation views the problem as a matching problem rather than an adversarial one, and thus allows us to quickly converge and obtain meaningful metrics in the optimization path.

Under review as a conference paper at ICLR 2020
FLUID FLOW MASS TRANSPORT FOR GENERATIVE NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
Generative Adversarial Networks have been shown to be powerful tools for gener- ating content resulting in them being intensively studied in recent years. Training these networks requires maximizing a generator loss and minimizing a discrimi- nator loss, leading to diffieult saddle point problem that is slow and difficult converge. Motivated by techniques in the registration of point clouds and the fluid flow formulation of mass transport, we investigate new formulation that based on strict minimization, without the need for the maximization. This formulation views the problem matching problem rather than adversarial one, and thus allows us to quickly converge and obtain meaningful metrics in the optimization path.
INTRODUCTION
Generative Networks have been intensively studied in recent years yielding some impressive results in different fields (see for example Salimans et al. (2016); Karras et al. (2017): Brock et al. (2018); Karras et al. (2018), 8); Zhu et al. (017)) and reference within). The most common technique used train generative network is by formulating the problem as a Generative Adversarial Networks (GANs). Nonetheless, GANs are notoriously difficult to train and in many cases do not converge, converge to undesirable points, or suffer from problems such as mode collapse.
Our goal here is to better understand the generative network problem and to investigate new formulation and numerical optimization techniques that do not suffer from similar shortcomings. To be more specific, we consider a data set made up of two sets of vectors, template vectors (organized as matrix) = (T...... Tn] and reference vectors R [R...... R., E R. The goal of our network is to find transformation f(T, 0) that generates reference vectors, that vectors from the space R. from template vectors in the space T, where are parameters that control the function f. For simplicity, write our generator
T( (
(1.1)
Equation equation I.I defines generator that depends the template data and some unknown parameters to be learned in the training process. We will use a deep residual network to approximate the function with 0 being the weights of the network.
In order to find the parameters for the generator we need to minimize some loss that captures how well the generator works. Assume first that have correspondence between the data points in T and R, that is, we know that the vector T, (0) R, Vj. In other words, our training set consists of paired input and output vectors. In this case we can find 0 by simply minimizing the sum of squares difference (and may add some regularization term such as weight decay).
1 Σ 2n
(())
T ( 12,
However, such correspondence is not always available and therefore, different approach is needed if we are to estimate the generator parameters.
The most commonly used approach to solve the lack correspondence is Generative Adversarial Networks (GANs) and more recently, the Wasserstein GANs (WGANs) Arjovsky et al. (2017). In

---

Under review as a conference paper at ICLR 2020
these approaches one generates a discriminator network, critic in the of WGANs, c(r, m) that gives a score for a vector, r, to be in the space R. In the original formulation c(:,)) yields the probability and in more recent work WGANs yields real number. The discriminator/crittc depends on the parameters to be estimated from the data in the training process. One then defines the function
Jnn 0) - g(c(R, n)) h(c(T(())
(1.2)
In the case simple GAN we have that
g(x) log(sm(x))) and h(x) = log(1 (x))
where the (.) function, is a soft max function that converts the score to probability, In the case of WGANs a simpler expression is derived where we use the score directly setting the identity and h
9. -9. and require some extra regularity on the score function.
Minimizing with respect to r7 detects the "fake" vectors generated by the generator while maximiz- ing J with respect to 0 "fools" the discriminator, thus generating vectors T(Ø) that are similar to vectors that are drawn from Training GANs a minimax problem where [(n, 0) minimized with respect to 17 and maximize with respect to O. Minimax problems are very difficult to solve and are typically unstable. Furthermore, the solution based on gradient d/ascent which is known to be slow, especially when considering saddle point problem Nocedal & Wright (199),, and this can be demonstrated by solving the following simple quadratic problem.
Example 1.1 Let
1
1 20n² 100
(n, -
20
The d/ascent algorithm used for the solution of the problem reads
1
0x+1
02 p(0x 10T))
ak+1
"10"
100x+1)
The convergence path of the method is plotted in Figure 7 It is evident that the algorithm takes
110
The convergence path for the solution of the quadratic problem in The blue points represent the ascent step and the red the descent step. Note the circular path the algorithm takes for convergence. This path is typical when solving saddle point problem using a/descent method.
Figure 1:
inefficient path to reach its destination. This inefficient path a known property of the method and avoiding it requires much more complex algorithms that, for example, eliminate one of the unknowns at least locally (see discussion in Nocedal & Wright (1999)). While such approaches have been derived for problems in fluid dynamics and constrained optimization, they much more difficult to derive for deep learning due to the non-linearity of the learning problem.
Besides the slowly converging algorithm, the simple GAN approach has a number of known funda- mental problems. It has been shown in Zhang al. 2017) that deep network classify vectors
2

---

Under review as a conference paper at ICLR 2020
with random labels. This implies that given sufficient capacity in the classifier, it is always possible to obtain 0 loss even if R + T E R. implying that there is no real metrics to stop the process (see further discussion in Arjovsky & Bottou (017)). This problem also leads to mode collapse, it is possible to obtain the saddle point of the function when T(Ø) = R. however this yields local solution that is redundant. Therefore, it is fairly well known that the discriminator needs be heavily regularized in order to effectively train the generator Gulrajani et al. (2017). such as by using very small learning rate or by weight clipping the case of WGANs, number of improvements have been proposed for GANs and WGANs, however these techniques still involve minimax problem that can be difficult solve.
Another common property of all GANs is that they minimize some distance between the two vector spaces that are spanned by some probabilities. While simple GAN is minimizing the JS-Divergence, the Wasserstein GAN minimizes the Wasserstein Distance. Minimizing the distance for probabilities makes sense when the probabilities are known exactly. However, as we discuss the next section, such process is not reasonable when the probabilities are estimated, that is, sampled and are therefore noisy. Since both spaces, and T are only sampled, we only have noisy estimations the probabilities and this has be taken into consideration when solving the problem.
In this paper we therefore propose a different point of view that allows us to solve the problem without the need of minimax. Our point of view stems from the following observation. Assume for moment that the vectors R and T()) are R² or R. Then, the problem described above is nothing but registration of point clouds under non-common transformation rule (i.e. not affine transformation). Such problem has been addressed in computer graphics and image registration for decades (see for example Eckart al. (2018); Myronenko & Song (2010) and reference within) yielding successful software packages, numerical analysis and computational treatment. This observation is demonstrated in the following example, that we use throughout the paper.
Assume that the space T is defined by vectors T; that are in R2 and that each vector drawn from a simple Gaussian distribution with mean and standard deviation of 1. To generate the space R
Example 1.2
a known generator that is simple resnet with fully connected layers
Xj+1 X, + (K,,)),)) == 1.......7 =
Here (-) is the tanh activation function and we choose Kj and save them for later use. The original points as well as points from the same distribution that are transformed using this simple resenet are plotted in Figure 2 Finding the transformation parameters (the matrices Kj) is kin registering the red and blue points, when no correspondence map is given.
Figure 2: Template points drawn from Gaussian random distribution (blue) are transformed using a simple ResNet to generate new reference points (red). There is no correspondence between the red/blue points. The goal of the training is to recover transformation that move the blue points into the red ones without known correspondence.
We therefore propose to extend the ideas behind cloud point registration to higher dimensions, adapting them to Generative Models. Furthermore, recent techniques for such registration
3

---

Under review as a conference paper at ICLR 2020
strongly connected to the fluid flow formulation of optimal mass transport. As explore next, there is a strong connection between our formulation and the Wasserstein GAN. Similar connections have been proposed in Lei al. (2019). Our approach can be viewed discretization of the fluid flow formulation of the optimal mass transport problem proposed in Benamou & Brenier (2003) and further discussed in Habeer & Horesh (2014); Ryu al. (2017). It has some commonalities with normalized flow generators Kingma & Dhariwal (2018) with two main differences. First, our flow become regular due the addition of regularization that explicitly keep the flow normalized and second, and more importantly, it solves the correspondence problem between the two data sets. The fluid formulation is known be easier solve and less nonlinear than the standard approach and (at least in 3D) leads to better and faster algorithms. We name our algorithm Mass Transport Generative Networks (MTGN) since our algorithm is based on mass transport that tries to match distributions but not adversarial networks. Similar to the fluid flow formulation for the OMT problem, our algorithm leads to a simple minimization problem and does not require the solution of minimax.
The rest of this paper is organized as follows. In Section we lay out the foundation of the idea used to solve the problem, including discretization and numerical optimization. In Section 3 we demonstrate the idea very simple example that help us gain insight into the method. In Section 4 we perform numerical experiments in higher dimensions and discuss how to effectively use the method and finally in Section we summarize the paper and suggest future work.
GENERATIVE MODELS, MASS TRANSPORT AND CLOUD POINT REGISTRATION
In this section we discuss our approach for the solution of the problem and the connection to the registration of cloud of points and the fluid flow formulation of optimal
mass transport.
2.1 GENERATIVE MODELS AND FLUID FLOW MASS TRANSPORT
We start by associating the spaces T and R with two probability density functions pr (x) and px(x). The goal is to find a transformation such that the probability, pr is transformed to the probability PR, minimizing some distance (see the review paper Evans (189) for details). An L2 distance leads to the Monge Kantorovich problem but it is possible to use different distances to obtain different transformations Burger et al. (2013). The computation of such transformation has been addressed by vast amount of literature. Solution techniques range from linear programming. to the solution of the notoriously nonlinear Monge Ampre equation Evans (1989). However, in seminal paper Benamou & Brenier (2003), was shown that the problem can be formed as minimizing the energy of simple flow
{(v(x,t))= = ./¹ [ ((xxx))v(xx, 1) dx dt P + V. (vp)=0)
min
(2.3a)
s.t
(2.3b)
x) = (x) p(1, x) = pr(x)
(2.3c)
Here is the total energy that depends on the velocity and density of the flow. The idea was extended in Chen et al. (2017) to solve the problem with different distances vector spaces. The problem is also commonly solved in fields such as computational flow of fluids in porous media, where different energy and more complex transport equations are considered Sarma et all (2007). A simple modification which we use here, is to relax the constraint p(1, x) pn(x) and to demand it holds only approximately. This formulation is better where have noisy realizations of the distributions and a perfect fit may lead to overfitting. The formulation leads to the optimization problem
p2(x)) E(v(x, t)) (o(l,x) p()x))) dx + a L.') / plx, t)lv(xx))) t)1 dx dt (2.4a)
1
min s.t Here, the first term
S V+V. (vp) 0 p(0,x) = (x)
(2.4b)
M = / ()(1,x)- -pn(x))? dx
2
(2.5)
4

---

Under review as a conference paper at ICLR 2020
can be viewed as data misfit, while the second term
s= / 1, p(x, t)lv t)l² dxot
(2.6)
can be thought of as regularization. Typical to all learning problems, the regularization parameter needs to be chosen, usually by using some solved in applied inverse transport problems Dean & Chen (201T).
validation set. Such formulation is commonly
Here
that our formulation equation 2.4a differs from both WGAN and normalized flow. It allows us to have a tradeoff between the regularity of the transformation as expressed in equation 2.6
to fitting the data expressed in equation 2.5 This is different from both WGAN and normalized flow when such a choice is not given.
2.2 DISCRETIZATION OF THE REGULARIZZATI TERM
The optimization problem equation 2.4 is defined in continuous space x, and in order solve we need to discretize it. The work in Benamou & Brenier (2003): : Haber & Horesh (2014) used an Fulerian framework, and discretize both x and on grid 2D and 3D. Such formulation is not suitable for our problems as the dimensionality of the problem can be much higher. In this case, Lagrangian approach that is based on sampling is suitable although some care must taken if the discrete problem to be solved is faithful to the continuous one. To this end, the flow equation 2.3b is approximated by placing particles, of equal mass for now, at locations X; = t;, = I...., 7 and letting them flow by the equation
dx; dt
v(x;, 4) == LE [0.1]
(2.7)
(0) t;
Here v(x; f; 0) is the velocity field that depends the parameters O. If use the forward Buler discretization in time, equation equation 2.7 nothing but a resnet that transforms particles located in x; t, from the original distribution pr (x) to the final distribution Pr(o) (x), that is sampled at points (t), I..... n. It important to stress that other discretizations in time may be more suitable for the problem. Using the point mass approximation to estimate the density, the regularization part of the energy can be approximate in a straight forward way as
ôt n
(0)
(yxx:,t;0))1
(2.8)
where 5l is the time interval used to discretize the ODE's equation 2.7 We see that the T.2 OMT energy is simply a sum of squared activations for all particles and layers. Other energies can be used well and can sometimes lead to more regular transportation maps Burger al. (2013).
2.3 DISCRETIZING THE MISFIT AND POINT OF CLOUD REGISTRATION
Estimating the first term in the objective function M, the misfit, requires further discussion. Assume that we have used some parameters 0 and push the particles forward. The main problem how to compare the distributions PT( that is sampled at points T,( (t), i 1,..., n and pn(x) sampled at R,, I,.... n.
In standard Lagrangian framework one usually assumes correspondence between the particles in the different distributions, however this is not the case here. Since we have unpaired data there no way to know which particle in R corresponds to particle in T(O). This is exactly the problem solved when registering two point clouds to each other. We thus discuss the connection between our approach to point cloud registration.
One approach for measuring the difference between two point clouds is using the closest point match. This is the basis for the Iterative Closest Point (ICP) Besl & McKay (1992) algorithm that is commonly used to solve the problem. Nonetheless, the ICP algorithm tends to converge only locally and thus we turn to other algorithms that usually exhibit better properties.
Following the work Myronenko Song (2010) we use the idea of coherent point drift for the solution of the problem. To this end, use Gaussian Mixture Model to evaluate the distribution of each of
5

---

Under review as a conference paper at ICLR 2020
the data. We define the approximations Pr (x, R) and pn(x, T(Ø)) to p(x, R) and p(x, T(Ø))
[_1x-RR.1e))
||x R.l:
Px R
exp
(2.9)
(2TT?)
Ø²
and
x (0)
(x T()))
exp
(2.10)
(2xo²)
σ²
The integral equation 2.5 can be now written as
1
(,) 2 (p.((x, T(0)) - Px, R.)² dx
(2.11)
Finally, we approximate Mn by replacing x by the sampled points T(Ø) and R. in symmetric distance obtaining
(0) (T(O), T())) Pr. (T(O), R) pl (R, T(O)) Pm. (R, R)
(2.12)
To summarize, we minimize the fluid flow formulation equation 2.4 2 by discretizing the misfit term equation 2.5aand and the regularization term equation 2.6
2.4 NUMERICAL OPTIMIZATION
The optimization problem equation 2.4 can be solved using any standard optimization technique, however, there are number of points that require special attention. First, the batch size in both R and T(0) cannot be too small. This is because we are trying to match probabilities that approximated by particles. For example, using batch of single vector is very likely to not represent the probability density. Better approximations the distribution, for example, by using a small number of Gaussians but this is not explored here. A second point is the choice of is the estimation of the probability. When the distributions are very far it is best to pick a rather large a. Such a choice yields a very "low resolution" approximation to the density, that is, only the low frequencies of the densities approximated. As the fit becomes better, we decrease a and obtain more details in the density's 's surfaces. This principle is very well known in image registration Modersitzki (200)).
be obtained by using different approximation to
NUMERICAL EXPERIMENTS ON SYNTHETIC DATA
In this section we perform numerical experiments using synthetic data. The goals of these experiments twofold. First, experimenting in 2D allows us plot the distributions and obtain some highly needed intuition. Second, synthetic experiments allow us to quantitatively test the results as we can always compute the correspondence for new data point.
Returning to Example 1.2 we use the data generated with some chosen parameters Otrue We train the generator estimate O and obtain convergence
epochs. The optimization path is plotted in Figure 3 We have also used standard GAN Zhu et al. (2017) in order to achieve the same goal. The GAN converged much slower and visually less pleasing solution that can be qualitatively assessed to be of lower accuracy.
One of the advantages of synthetic experiments that have the "true" transformation and therefore can qualitatively validate our results. To this end, we choose a new set of random points, Trett and used them with the optimal parameters to generate (0) and its associate approximate distribution. We also generate the "true" distribution from the chosen parameters @true by pushing Trist with the true parameters, generating Trest (Qtrue) We then compute the mean square error
Tlest
test ( (trre
(true)

---

Under review as a conference paper at ICLR 2020
Epoch 1
Epoch 2
Epoch
Epoch 4
Epoch 5
Epoch 6
Epoch
Epoch 8
Epoch 2000
Epoch 4000
Epoch 6000
Epoch 8000
Figure 3: Path of the optimization. Top two panels: The template points are transformed into the reference space using approach in 8 epochs. The density estimated from the red points is displayed in colour. The bottom panel the path optimization taken by the GAN training. takes more than 8000 epochs and the results clearly not worse.
For the experiment at hand we obtained error of E x 10 with our method, while the GAN gave us an error of 3.6 x 10-1, which is substantially worse than our estimated network. This implics that the using training the network managed to learn the transformation rather well. Unfortunately, this quantitative measure can only be obtained when the transformation known and this is why we believe that such simple tests important.
NUMERICAL EXPERIMENTS IN HIGHER DIMENSIONS
In order to match higher dimensional vectors and distributions slightly modify the architecture of the problem. Rather than working on the spaces T and R directly, we use a feature extractor to obtain latent space Te and R. respectively. Such spaces auto-encoder and then use the encoded space as the latent space. We then register the points in T the points in R₁, In the experiments have done here used the MNIST data set and used simple encoder similar to Kingma & Welling (019)) to learn the latent space of R. We then use our framework to obtain the transformation that maps template vector sampled from a Gaussian random distribution, T, with mean and standard deviation of to the latent space R2 In our experiments the size of the latent space was only 32 which seems be sufficient to represent the MNIST images. We use simple ResNet50 network with a single layer at every step that utilizes 3 x 3 convolutions. run our network for 200 epochs in total with constant learning rate, Better results are obtained we change the kernel width, throughout the optimization. We start with very large, value of 50, and slowly decrease it, dividing by every 30 steps. The final value of a is 0.78 which yields a rather local support. Convergence curve for method is plotted in Figure 4 Convergence is generally monotonic and the misfit grows only when we choose a,, changing the problem to a more challenging one.
be formed for example by training
7

---

Under review as a conference paper at ICLR 2020
Figure Convergence path of method. Note the jumps in misfit when change a.
Results of our results are presented in Figure
Although not all images look real, we have very
4
7
9
5
7
X
3
6
7
3
/
7
9
9
1
4
4
9
/
4
q
4
7
2
4
6
8
6
9
9
9
3
6
3
/
/
Figure 5: A random set of transformed template vectors that has been trained using encoded MNIST images reference vectors.
large number (over 80% by visual inspection) that look like they can taken from the reference set. Unlike the previous experiment where we have a quantitative measure of how successful our approach is, here we have to rely on visual inspection.
5 CONCLUSIONS AND FUTURE WORK
In this work we have introduced new approach for Generative Networks, Rather than viewing the problem as "fooling" an adversary which leads to minimax we view the problem matching problem, where correspondence between points is unknown. This enables us formulate the problem as strictly minimization problem, using the theory of optimal mass transport that is designed to match probabilities, coupled with numerical implementation that is based on particles and cloud point registration.
When comparing our approach to typical GANs in low dimensions, where it is possible to construct examples with known solution it is evident that our algorithm is superior in terms of iterations to convergence and also in terms visual inspection. Although we have shown only preliminary results in higher dimensions be pursuing variations of this problem in the future. Indeed, is it not better to find a match, that is commonalities, rather than to be adversary?
believe that our approach is more appropriate for the problem and will
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Martin Arjovsky and Léon Bottou. Towards Principled Methods for Training Generative Adversarial Networks. arXiv e-prints, art. arXiv: 170..040222 Jan 2017.
Martin Arjovsky, Soumith Chintala, and Léon Bottou. Wasserstein generative adversarial networks. In Doina Precup and Yee Whye Tch (eds.), Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pp. 214-223, International Convention Centre, Sydney, Australia, 06-11 Aug 2017. PMLR. URL http //proceedings.co mir...mrr..... wwtppppeeeeer//s////////
html
D. Benamou and Y. Brenier. A computational fluid mechanies solution the monge kantorovich mass transfer problem. SIAM J. Math. Analysis, 35:61-977 2003.
Paul J. Besl and Neil D. McKay. A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell., 14(2):39-2566, 1992.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high fidelity natural image synthesis. CoRR, 0bs/1809.11096 2018. URL http:///axxvv.org/
1809. 11096
Martin Burger, Modersitzki, and Lars Ruthotto. A hyperelastic regularization energy for image registration. SIAM Journal Scientific Computing, 35:B132-BAL -148,, 01 2013. doi: 10.1137/ 110835955.
Yongxin Chen, Eldad Haber, Kaoru Yamamoto, Tryphon T. Georgiou, and Allen R. Tannenbaum. An efficient algorithm for matrix-valued and vector-valued optimal mass transport. CoRR, abs/1706.08841, 2017.
O.S. Dean and Y. Chen. Recent progress on reservoir history matching: review. Comput Geoscience, 15:185-221, 2011.
Ben Eckart, Kihwan Kim, and Jan Kautz. Fast and Accurate Point Cloud Registration using Trees of Gaussian Mixtures. arXiv e-prints, art. arXiv: 180::100.000 002587 Jul 2018.
L.C. Evans. Partial Differential Equations and Monge Kantorovich transfer. Lecture Notes, 1989.
Ishaan Guirajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron Courville. Improved Training of Wasserstein GANs. arXiv e-prints, art. arXiv: 770.00022, Mar 2017.
E. Haber and R. Horesh. Efficient numerical methods for the solution of the monge kantorovich optimal transport map. SIAM J. on Scientific Computing, pp. 36-49,10 2014.
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. CoRR, abs/1710.10196, 2017. URL
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. CORR, abs/1812.00498,, 2018. URL [RRl::ttpp//aaxi..orr//a////////
04948
Diederik P. Kingma and Prafulla Dhariwal. Glow: Generative Flow with Invertible 1x1 Convolutions, arXiv e-prints, art. I 1807 1807.03039, 2018.
Diederik Kingma and Max Welling. An Introduction Variational Autoencoders. arXiv e-prints, pp. Nxiv:1996.0069. 2019.
Na Lei, Kehua Su, Li Cui, Shing-Tung Yau, and Xianfeng David Gu. A geometric view of optimal transportation and generative model. Computer Aided Geometric Design, 68: 21, 2019. URL http://wwwy http://wwww..attt.cooooeeeeeeaaa
S016 $0167839618301243
J. Modersitzki. Numerical methods for image registration. Oxford, 2004. 2004.

---

Under review as a conference paper at ICLR 2020
Andriy Myronenko and Xubo Song. Point set registration: Coherent point drift. IEEE Trans. Pattern 2275, December 2010. ISSN 0162-8828. doi: 10.1109/TPAMI Anal. Mach. Intell., (2(12):2262 22222222222222,,,,.-2-22222
htt.://///1..11//PAPA... 2010.46. 1ttp///////a............0.011111
2010 46.
Nocedal and Wright. Numerical Optimization. Springer, New York, 1999.
Ernest K. Ryu, Yongxin Chen, Wuchen Li, and Stanley Osher. Vector and Matrix Optimal Mass Transport: Theory, Algorithm, and Applications. arXiv e-prints, art. arXiv:1712.1 10279, Dec 2017.
Tim Salimans, Ian J. Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. CoRR, abs/1606.03398,, 2016. URL http:://axii:
arg/abs/Dn 606 03498
Sarma, L.J. Durlofsky, K. Aziz, and W. Chen. A approach automatic history matching using kemee pea. SPE Reservoir Simulation Symposium, Houston, Texas, 2007.
Chiyuan Zhang, Samy Bengio, Moritz Hardt, Benjamin Recht, and Oriol Vinyals. Understanding deep learning requires rethinking generalization. 2017. URL http::/s///xxx..rrr//////
1611 1 03530
Jun Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Unpaired image-to-image translation using cycle-consisten adversarial networks. CoRR, abs/1703.10593, 2017. URL http://
arxiv.rgg/bbs wtt.:/rrrrbbb//13...oo 10593
Jun-Yan Yan Zhu, Taesung Park, Phillip Isola, and Alexei A. Efros. Unpaired Stage-to-Image Translation using Cycle-Consistent Adversarial Networks, arXiv e-prints, art. arXiv: 100..110011000. 10593, Mar 2017.
10


Reviews:
[{"review_id": "rklblJ7LKr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposed a generative network based on fluid flow solutions of mass transport problems. The paper is difficult to follow due to a poor structure and obvious technical mistakes. Detailed comments are as follows:\n\n1. The dual formulation used in the objective of WGANS involves expectations with respect to data distributions. When authors introduced WGANs, it is extremely loose and essential wrong to state that \"In the case of WGANs a simpler expression is derived where we use the score directly setting g to the identity and h = −g, and require some extra regularity on the score function\".\n\n2. The organisation of the paper makes it difficult to read:\na) While the relevant work is discussed only briefly in Section 1 and contain incorrect statements (see an example above), detailed discussions of two toy examples in the introduction section distract the reading.\n\nb) Section 2 is a combination of related work and proposed work. For example, it starts more like a section introducing the dynamic transport formulation of mass transport problems (Equations 2.3a-2.3c), but in fact it contains the authors' proposed approach (2.4a-2.4b), which makes it difficult to tell the authors' contributions. Moreover, the connection between 2.3a-2.3c to 2.4a-2.4b is not clear to me.\n\n3. Multiple notations are not properly defined or conflicting: what are $\\rho(x,t)$ and $\\rho(1,x)$?\n\n4. Very limited experimental validation with no comparison to other algorithms.\n\n5. Multiple typos in the paper, e.g. \"Equation equation 1.1\".", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "SJxhj397cB", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper addresses the task of constructing a generative model for data using a novel optimal transport-based method. The paper proposes an alternative view of obtaining generative models by viewing the generation process as a transport problem (specifically, fluid flow mass transport) between two point clouds living in high-dimensional space. To solve the transport problem, a discretization scheme is proposed, which gives rise to a variant of point cloud registration problem, which is solved using numerical optimization. The results are provided on synthetic data and a real MNIST data. \n\nWith generative models, and particularly generative adversarial networks (GANs) being notoriously hard to train, alternative ways of constructing generative models are needed. The paper does a good job displaying the potential optimization issues arising when training GANs, and the basic theoretical foundations used in the paper have been validated in prior work. However, with the introduction of a transport-based formulation, its respective issues may arise, that are not described in the paper. Will the optimization always converge, and if yes, to which kind of optimum? What are the requirements for the point clouds R and T? \n\nWhile the conceptual contribution is that the , the technical novelty is limited and mostly amounts to the appropriate choice of distributions in R and T and applying the discretization schemes to be able to compute the experiments. Unfortunately, the model is not studied theoretically, i.e. no description is given regarding the class of generative tasks that could be solved using the method, or the class of functions that could be learn in such a way.\n\nThe experimental evaluation demonstrates on only two simple examples the results of the work. More experiments are needed to fully understand the possibilities of the framework. \n\nThe convergence speed is not indicated, and the efficiency of the optimization is not described. \n\nTo summarize, I believe the paper should not be accepted in its present (early) form, as (1) more detailed theoretical insight are needed, and (2) much more computational experiments are needed to fully validate the method.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "B1gXbw_nFB", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper targets training generative adversarial networks with a formulation that is motivated by mass transport of fluid flows. While generalized transport formulations are popular for GANs by now, especially the Earth Mover's distance and the Wasserstein GAN version that this paper is based on, this submission instead frames the problem with a divergence-free flow model inspired by Navier-Stokes. The problem of matching output distributions then becomes one of inferring a suitable flow field that aligns the distributions. \n\nThe proposed model is discretized in a Lagrangian manner, and divergence freeness is ensured trivially by a constant particle count. Matching the point clouds in terms of distribution is done via a gaussian mixture model to obtain smooth distributions. (This probably doesn't scale too well to large paricle numbers due to the global support, but the paper also focuses on smaller 2D cases.) As mentioned in the text, the method in the end shares similarities with point cloud registration (ICP).\n\nA synthetic test with a square target shape and an initial centered distribution is shown to highlight that the method can re-distribute parts of the initial cloud towards all sides of the target shape. \n\nAs a tougher case, a digit generator with MNIST data is shown, which however, does not reach a level of quality we could expect from other existing methods for generative models.\n\nWhile I found the initial motivation of the paper quite interesting (and novel as far as I can tell), the discretized version is somewhat disappointing. The smooth matching of point clouds does not retain too much of the initial fluid flow model. Unfortunately, the MNIST test also indicates that the method has problems scaling to higher dimensions. This is a central challenge for GANs, and based on the submission I don't have the impression that the proposed formulation is competitive with existing GAN methods. ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The submission is concerned with providing a transport based formulation for generative modeling in order to avoid the standard max/min optimization challenge of GANs. The authors propose representing the divergence with a fluid flow model, the solution of which can be found by discretizing the space, resulting in an alignment of high dimensional point clouds. 

The authors disagreed about the novelty and clarity of the work, but they did agree that the empirical and theoretical support was lacking, and that the paper could be substantially improved through better validation and better results - in particular, the approach struggles with MNIST digit generation compared to other methods.

The recommendation is to not accept the submission at this time.
Decision: Reject