End-to-end learning of energy-based representations for irregularly-sampled signals and images
Authors: Ronan Fablet
Lucas Drumetz
François Rousseau
Venue:  (2020)
Keywords: end-to-end-learning
irregularly-sampled data
energy representations
optimal interpolation

Abstract: For numerous domains, including for instance earth observation, medical imaging, astrophysics,..., available image and signal datasets often irregular space-time sampling patterns and large missing data rates. These sampling properties is a critical issue to apply state-of-the-art learning-based (e.g., auto-encoders, CNNs,...) to fully benefit from the available large-scale observations and reach breakthroughs in the reconstruction and identification of processes of interest. In this paper, we address the end-to-end learning of representations of signals, images and image sequences from irregularly-sampled data, {\em i.e.} when the training data involved missing data. From an analogy to Bayesian formulation, we consider energy-based representations. Two energy forms are investigated: one derived from auto-encoders and one relating to Gibbs energies. The learning stage of these energy-based representations (or priors) involve a joint interpolation issue, which resorts to solving an energy minimization problem under observation constraints. Using a neural-network-based implementation of the considered energy forms, we can state an end-to-end learning scheme from irregularly-sampled data. We demonstrate the relevance of the proposed representations for different case-studies: namely, multivariate time series, 2{\sc } images and image sequences.

Under review as a conference paper at ICLR 2020
END-TO-END LEARNING OF ENERGY-BASED REPRE- SENTATIONS FOR IRREGULARLY-SAMPLED SIGNALS AND IMAGES
Anonymous authors Paper under double-blind review
ABSTRACT
For numerous domains, including for instance earth observation, medical imag- ing, astrophysics..., available image and signal datasets often irregular space- time sampling patterns and large missing data rates. These sampling properties is critical issue to apply state-of-the CNNs.... fully benefit from the available large-scale observations and reach breakthroughs the reconstruction and identification of processes of interest. this paper, we address the end-to-end learning of representations of signals, im- ages and image sequences from aregularly-samplee data, i.e. when the training data involved missing data. From an analogy to Bayesian formulation, sider energy-based representations. Two energy forms are investigated: one de- rived from auto-encoders and relating to Gibbs energies. The learning stage of these energy-basel representations (or priors) involve joint interpolation is- sue, which resorts solving an energy minimization problem under observation constraints. Using neural-network based implementation of the considered en- ergy forms, we can state an end-to-end learning scheme from irregularly-sann stmplee data. We demonstrate the relevance of the proposed representations for different case-studies: namely, multivariate time series, 2 images and image sequences.
learning-based (e.g., auto-encoders,
con-
INTRODUCTION
In numerous application domains, the available observation datasets do not involve gap-free and regularly-gridde signals or images. The irreulaar-saplling may result both from the characteristics of the sensors and sampling strategy, e.g. considered orbits and swaths in spacebone earth observa- tion and astrophysics, sampling schemes in medical imaging, as well as environmental conditions which may affect the sensor, e.g. atmospheric conditions and clouds for earth observation.
A rich literature exists on interpolation for irregularlysamplle signals and images (also referred to as inpainting in image processing (4). A classic framework states the interpolation issue as the miminisation of energy, which may be interpreted in Bayesian framework. A variety of en- ergy forms, including Markovian priors (12), patch-bacc based priors (20), gradient norms variational and/or PDE-based formulations (4), Gaussian priors () as well as dynamical priors in fluid dynamics 3.. The later relates to optimal interpolation and kriging (8), which is among the state-of-the-an and operational schemes in geoscience (TO). Optimal schemes classically involve the inference of the considered covariance-based priors from trregularly-sample data. This may however be at the expense of Gaussianity and linearity assumptions. which do not often apply for real signals and im- ages. For the other types of energy forms, their parameterization are generally set a priori and not learnt from the data. Regarding more particularly data-driven and learning- based approaches, most previous works 2 1 20) have addressed the learning of interpolation schemes under the assump- tion that a representative gap-free dataset is available. This gap-free dataset may be the image itself 9 20 18). For numerous application domains, as mentionned above, this assumption cannot be fulfilled, Regarding recent advances learning-based schemes, a variety of deep learning models, e.g. (7) 24 23), have been proposed. Most of these works focus on learning interpolator. One may however expect to learn not only an interpolator but also some representation of considered data, which may be of interest for other applications. In this respect, RBM models (Restricted Boltzmann

---

Under review as a conference paper at ICLR 2020
Machines) (22 66 MCMC schemes.
are particularly appealing at the expense however of computationallyyexeesssie
In this paper, aim to learn representations of signals or images from treguuaaly-samplle obser- vation datasets. Our contribution three-fold:
end-to-end learning of energy-based representations from trrgularll-ssmmples training data. Based on neural-network architecture, it jointly embeds the considered energy form and an associated interpolation scheme.
besides classic auto-encoder representations, we introduce NN-based Gibbs-Energy repre- sentations, which relate Markovian priors embedded in CNNs.
the demonstration of the relevance of the proposed end-to-end learning framework for dif- ferent data types, namely time series, images and image sequences, with possibly very high missing data rates.
The remainder is organized follows. Section 2 formally states the considered issue. We introduce the proposed end-to-end learning scheme in Section We report numerical experiments in Section and discuss our contribution with respect related work in Section ??.
PROBLEM STATEMENT
In this section, we formally introduce the considered issue, namely the end-to-end learning of rep- resentations and interpolators from pregularly-samplee data. Within a classic Bayesian or energy- based framework, interpolation issues may be stated as minimization issue
8 = arg min Ua (X) subject to Xy = Ya
X
where X is the considered signal, image or image series (referred to hereafter as the hidden state), Y the observation data, only available considered energy prior parameterized by 0. As briefly introduced above, variety of energy priors have been proposed in the literature, e.g. 20 5).
subdomain Ω of the entire domain D, and Up() the
We assume we are provided with a series of ttregularly-samplle observations, that is to say a set (Y) (2()) Jie such that 20) C D and (+) only defined on subdomain Q Assum- ing that all (() share some underlying energy representation Us(,, we may define the following operator I
I UUo y()) ,(0),2())) arg min Up (X) subject to Xy =Y(0)
such that Z(Y(),, 20) (1), Here, we aim learn the parameters 8() of the energy Us() from the available observation dataset (y()), ) Assuming operator I known, this learning issue can be stated as the minimization of reconstruction error for the observed data
i arg min Σ Σ|ro 1 ( .20) |2
(3)
where I1-112, refers the L2 norm evaluated on subdomain. Learning energy Uo() from observation dataset (2()) 3. clearly involves a joint interpolation issue solved by operator I.
Given this general formulation, the end-to-end learning issue comes to solve minimization 3 ac- cording some given parameterization of energy Ug(). In 3 interpolation operator clearly critical. In Section 3 we investigate a neural-networt implementation of this general framework, which embeds neural-network formulations both for energy ()() and interpolation operator I.
PROPOSED END-TO-END LEARNING FRAMEWORK
In this section, we detail the proposed neural-network-baseee implementation of the end-to-end for- mulation introduced in the previous section. We first present the considered paramaterizations for energy Ua() in B (Section 3.1b. We derive associated NN-based interpolation operators I (Section (.2) and describe our overall NN architectures for the end-to-end learning of representations and interpolators from irregularly-sampled datasets (Section 3.3
2

---

Under review as a conference paper at ICLR 2020
3.1 NN-BASED ENERGY FORMULATION
We first investigate NN-based energy representations based on auto-encoders (1)). Let us denote by 0g and Φn the encoding and decoding operators of auto-encoder (AE), which may comprise both dense auto-encoders (AEs), convolutional AEs as well as recurrent AEs when dealing with time-related processes. The key feature of AEs is that the encoding operator dg maps the state X into a low-dimensional space. Auto-encoders are naturally associated with the following energy
(4) Minimizing according to this energy amounts to retrieving the hidden state whose low- dimensional representation in the encoding space matches the observed data in the original decoded space. Here, parameters refer to the parameters of the encoder dg and decoder Φn. respectively 0g and 0D.
Ux(X) ||X ΦD (ØE (x))]I
The mapping to lower-dimensional space may be regarded as a potential loss in the representation potential of the representation. Gibbs models provide an appealing framework for an alternative energy-based representation, with no such dimensionality reduction constraint. Gibbs models intro- duced in statistical physics have also been widely explored in computer vision and pattern recog- nition (13) from the 80s. Gibbs models relate to the decomposition of Up as a sum of potentials (X) Eaec Ve where C is a set of cliques, i.e. a set of interacting sites (typically, local neighbors), and V. the potential on clique c. In statistical physics, this formulation states the global energy of the system as the sum of local energies (the potential over locally-interactio sites). Here, we focus the following parameterization of the potential function
(x(X) = Σ IX. ve 112
(5)
.DD
with N₂ the set of neighbors of site s for the entire domain D and & a potential function. Low- energy state for this energy refers to state X which operator ab provides a good prediction at any site knowing the state in the neighborhood N₂ of 8. This type of Gibbs energy relates to Gaussian Markov random fields, where the conditional likelihood at one site given its neighborhood follows a Gaussian distribution. We implement this type of Gibbs energy using the following NN-based parameterization of operator ei:
(XX) = t/2 (t/1 (X))
(6)
It involves the composition of space and/or time convolutional operator th and a coordinate-wiss operator 1/g. The convolutional kernel for operator 1/ is such that the coefficients for the center of convolutional window are set to zero. This property fulfills the constraint that X (s) is not involved in the computation of (Xx.) at site s. As an example, for univariate image, ee can be set as a convolutional layer with Np filters with kernels of size 3x3x1, such that for each kemnel K3 (1,
such case, operator f/s would be convolution layer with one filter with kernel of size IxlxNN.. Both th and th2 can also involve non-linear activations. Without loss of generality, given this parameterization for operator eb we may rewrite energy Ug as (x( = ||X & (X) 1 where vi (X)) at site is given by &a (XN,).
0 (the same applies to biases).
Overall, we may use the following common formulation for the two types of energy-based represen- tation
(x = |X e 12
They differ in the parameterization chosen for operator vb.
3.2 NN-BASED INTERPOLATOR
Besides the NN-based energy formulation, the general formulation stated in (3) involves the def- inition of interpolation operator Z, which refers to minimization ([. We here derive NN-based interpolation architectures from the considered NN-based energy parameterization.
Given parameterization 77. simple fixed-point algorithm may be considered to solve for 3 This algorithm at the basis of DINBOF algorithm and XXX for matrix completion under subspace con- straints (2) 14) involves the following iterative update
x (k+1)
3 (Xx(A)
( (2)
Y (D)
(8)
(R++1) 1
= X (k+11)
(2)
3

---

Under review as a conference paper at ICLR 2020
Interestingly, the algorithm is parameter-free and can be readily implemented in NN architecture given the number of iterations to be considered.
Given some initialisation, applics at each iteration
may typically consider an iterative mudientt-bseed descent which
x(==1)
X
AJu. (X(())
X (1+1) (S)) Y (
x(++1) (2)
((++1)
!2)
with Ju, the gradient of energy Up w.r.. state X, A the gradient step and ! the missing data area. Automatic differentiation tool embedded in neural network frameworks may provide the numerical computation for gradient Ju, given the NN-based parameterization for energy Ug. This proved numerically too expensive and was not further investigated in our experiments. Given the considered form for energy Ug. its gradient w.r.t X decomposes as product
Ju, (X) = J& (x) (X - (X))
(10)
and X th (X) may be regarded suboptimal gradient descent. Hence, rather than considering the true Jacobian Jp for operator 1h, we may consider an approximation through trainable CNN G() such that the gradient descent becomes
x = X(k) - G (X((),, (x(1))
(1++) (S2) = Y
(1I)
X (1++1) (12) -Xp x((+1) (22
where G(X(*), (x(())) = ((x()) - (x((x)) and G is a CNN to be learnt jointly to ip during the learning stage. Interestingly, this gradient descent embeds the fixed-port algorithm when & is the identity.
Let us denote respectively by I and Ig the fixed-point and gradient- based NN-based interpola- tors, which implement N, iterations of the proposed interpolation updates. Below, INN will denote both Ipr and Ig. Whereas Tpp is parameter-free, Ig involves the parameterization of operator G. We typically consider CNN with ReLu activations with increasing numbers of filter through layers up to the final layer which applies linear convolutional with a number of filters given by the dimension of the state.
G
(k)
(k+1)
In
Ω (0)
B
B
(...)
B
(N))))
Figure 1: Sketch of the considered end-to-end architecture: we depict the considered Ny-block architecture which implements Ny-step interpolation algorithm described in Section 3.2. Operator defined through energy representation (7) and refers to the NN-hased approximation of the gradient-based update for minimization
(T). This architecture uses input mask Ω corresponding to the missing data- free domain and an initial gap-filloe vap-filling (0) for state X. We typically initially fill missing data with zeros centered and normalized states.
3.3 END-TO-END ARCHITECTURE AND IMPLEMENTATION DETAILS
Given the parameterizations for energy Ug and the associated NN-based interpolators presented pre- viously, design an end-to-end learning for energy representation Up and associated interpolator
4

---

Under review as a conference paper at ICLR 2020
INN, which uses inputs observed sample y()) and the associated missing-date data-free domain 2(0. Using normalization preprocessing step, we initially fill missing data with zeros to provide an initial interpolated state to the architecture. We provide a sketch of the architecture in Fig
Regarding implementation details, beyond the design of the architectures, which may be application- dependent for operators de and G (see Section 4). we consider an implementation under keras using tensorflow as backend. Regarding the training strategy, we use adam optimizer. We iteratively in- crease the number of blocks N₁ (number of gradient steps) to avoid the training to diverge. Similarly, we decrease the learning rate accross iterations, typically from 1e-3 to 1e-6. In our experiments, we typically consider from 5 to 15 blocks. All the experiments were run under workstations with single GPU (Nvidia GTX 1080 and GTX 1080 Ti).
EXPERIMENTS
In this section, we report numerical experiments on different datasets to evaluate and demonstrate the proposed scheme. We consider three different case-studies: an image dataset, namely MNIST; multivariate time-series through an application to Lorenz-63 dynamics (17) and image sequence dataset through application to ocean remote sensing data with real missing data patterns. In all experiments, we refer to the AE-based framework, respectively as FP(d)-Conv AE and G(d)-ConvAID using the fixed- interpolation steps. Similarly, we refer to the Gibbs-base frameworks respectively as FP(d)-GENN and ((d))
gradient-baser interpolator where the value of d refers to the number of
4.1 MNIST DATASETS
We evaluate the proposed framework on MNIST datasets for which simulate missing data pat- terns. The dataset comprises 60000 28x28 grayscale images. For this dataset, we only evaluate the AE-based setting. We consider the following convolutional AE architecture with a 20-dimensional encoding space:
Encoder operator dE: Conv2D(20)+ ReLU AvPooling Conv2D(40) + ReLUU Aver-
agePooling + Dense(80) + ReLU + Dense(20);
Decoder operator Φg: ComYDD((0++RRLLU-CCW/D/D Conv2DTranspos(440) + ResNet(2),
ResNet:
We generate random missing data patterns composed of N' squares of size WsrWWs the center of the square is randomly sampled uniformly over the image grid. As illustrated in Fig 3 we consider four missing data patterns: Nş- = 20 and Ws = 5, Nşs 30 and Wss = 5, N₂- = and Ws = 9, Ng 6 and Ws 9. As performance measure, we evaluate an interpolation score (I-score), global reconstruction score (R-score) for the interpolated images and an auto-encoding (AE-scorc) score of the trained auto-encoder applied to gap-free data, in terms of explained variance. We also evaluate classification score (C-score), in terms mean accurcay, using the 20-dimensional encod- ing space feature space for classification with 3-yer MLP. We report all performance measures for both the test dataset in Tab[] for MNIST dataset. For benchmarking purposes, we also report the performance of DINEOF framework, which dataset, the auto-encoder architecture trained on gap-free dataset as well as the considered convo- lutional nuto-encoder trained using an initial zero-filling for missing data areas and training loss computed only of observed data areas. The later can be regarded as a ing a single block in F Overall, these results illustrate that representations trained from gap-free data may not apply when considering significant missing data rates as illustrated by relatively poor performance of PCA-based and AE schemes, when trained from gap-free data. Similarly, training an AE representations using input zero-filling strategy lowers the auto-encoding power when applied to gap-free data. Overall, the proposed scheme guarantees a good representation in terms of AE score with additional gain in terms of interpolation performance, typically between z 15% and 30% depending of the missing data patterns, the gain being greater when considering larger missing data areas.
20-dimensional PCA trained the gap-free
architecture us-
5

---

Under review as a conference paper at ICLR 2020
New MNIST
Model
1-score
R-score
AE-score
C-score
-9.41%
21.54%
64.36%
Ng
DINEOF
96.23%
(-10.95%)
(20,48%)
(65.11%)
55.98%
80.98%
93.42%
W 5
ConvAE
98.12% %
(55.39%)
(80.58%)
(92.35%)
61.79%
82.22%
87.64%
Zeno-ConvAF.
97.55%
(61.63%)
(81.64%)
(87.56%)
74.99%
78%
91.62%
FP(15)-ConvAE
97.96%
(72.80%)
(87.31%)
(91.13%)
76.50 %
89.81
91.77%
G(14)-ConvAE
97.91%
(75.56 )
(88.81 %)
(91.21%)
-8.86%
13.89%
64.36%
N3 = 30
DINEOF
96.23%
(-10.19%)
(12.71%)
(65.11%)
38.32%
67.42%
93.42%
W 5
ConvAE
98.12%
(38.16%)
(67.32%)
(92.35 %)
53.69%
74.97%
85.67%
Zero-ConvAE
97.03%
(53.44%)
(74.44%
(85.83%)
69.27%
83.81%
90.22%
FP(15)-CovAAE
97.59%
(67.68%)
(82.54%)
(90.04%)
69.82% %
84.96%
90.98%
G(14)-ConvAE
97.45%
(68.52%)
(83.76%)
(90.66%)
-1..5%%
33.08%
64.36%
Ns =
DINEOF
96.23%
(44.77%)
(32.26%)
(65.11%)
-1.21%
72.57%
93.42 %
W 9
ConvAE
98.12%
(-3.08%)
(71.96%)
(92.35 %
3,55%
74.04%
89.21 %
Zero-ConvAE
97.76%
(1.85%)
(72.93%)
(89.05%)
46,91 %
85.13%
91.87%
(P(15)-ConVAE
97.90%
(44.12%)
(83.79%)
(91.38%)
G(14)-ConvAE
46.74%
85.72%
92.09%
97.76%
8#
(43.76%)
(83.98%)
(91.39%)
-37.47%
16.83%
64.36%
Ns
DINEOF
96.23%
(-40.00%)
(15.50%)
(65.11%) (1))
-27.02%
46.95%
93.42%
W 9
ConvAE
98.12% %
(-28.28%)
(46.44%)
(92.35%)
-9.94%
55.41%
86.52%
Zeno-ConvAE.
97.33%
(-12.03%)
(54.09%)
(86.73%)
26.90 %
71.18%
91.03%
FP(15)-ConVAE
97.71%
(22.56 %
(68.45%)
(90.41%)
26. 18%
70.70%
90.30%
G(10)-ConvAE
97.86%
(24.73%)
(69.58%)
(90.23%)
Table 1: Performance of AE schemes in presence of missing data for Fashion MNIST dataset: for a given convolutional AE architecture (see main text for details), a PCA and ConvAE mod- els trained gap-free data with 15-iteration projection-based interpolation (resp., DINEOF and ConvAF), a zero-filling stratefy with the same ConvAR architecture (Zero-ConvABi and the fixed- point and gradient- based versions of the proposed scheme. For each experiment, we evaluate four measures: the reconstruction performance for the known image areas (R-score), the interpolation performance for the missing data areas (I-score), the reconstruction performance of the trained AE when applied to gap-free images (AE-score), the classification score of MLP classifier trained in trained latent space for training images involving missing data.

---

Under review as a conference paper at ICLR 2020
4.2 MULTIVARIATE TIME SERIES
We present an application to the Lorenz-63 dynamics (17), which involve a 3-dimensional state governed by the following ordinary differential equation:
dX.1 (X12 X1,2)
d. L2 = px - X1,2 - X1,
(12)
dAra
di = X1,1 - 3,x,3
Under parameterization a = 10, = 28 and B = 8/3 considered here, Lorenz-63 dynamics are chaotic dynamics, which make then challening in context. They can be regarded reduced- order model turbulence dynamics. We simulate Lorenz-63 time series of 200 time steps using a Runge-Kutta44 ODE solver with an integration step of 0.01 from initial condition in the attractor. For given experiment, we first subsample the simulated series to a given time step di and then generate using a uniform random samplong missing data mask accounting for 75% of the data. Overall, training and test time series are formed by subsequences of 200 time steps. We report experiments with the GE-NN setting. The AF-based framework showed lower performance and not included here. The considered GE-NN architecture is as follows: a ID convolution layer with 120 filters with kernel width of 3, zero- zero-weigee www.-w-eeestttaaaeeee and Relu activation, ID convolution layer with 6 filters kernel width of I and Relu activation, a residual network with residual units using filters with a kernel width and a linear activation. The last layer is convolutional layer with 3 filters, kernel width of I and linear activation.
for the center the convolution kernel
0.
45
60
-05
-05
-10
-15
100
100 100
Figure 2: Example of missing data interpolation for Lorenz-63 dynamics: from left to right, the time series of each of the three components of Lorenz-63 states for dt 0.02 and 75% miss- observed data (black dots), the true state (green,-),
ing data rate. We depict the the interpolated states using DINEOF (blue, -) and the interpolated states using the proposed ap- proach (G-NN-PP. (red, -). Visually, the interpolated sequence using our approach can hardly be distinguished from the true states.
For benchmarking purposes, report the interpolation performance issued from ensemble Kalman smoother (EnKS) (10) knowing the true model, regarded as a lower-bound of the inter- polation performance. The parameter setting of the EnKS dynamical model and spherical observation covariannce to 0.1 I. We also compare the proposed approach DINEOF 21). Here, the learning of the PCA decomposition used in the DINEOF scheme relies on gap-free data. Fig2iillss illustratss this comparison for one sequence of 200 time steps with dt = 0.02. In this example, one can hardly distinguish the interpolated sequence using the pro- posed approach (FP(15)-GE-NN) By contrast, DINEOF scheme cannot retrieve some of the largest deviations. We report Appendix Tab the performance the different interpolation schemes. The proposed approach clearly outperforms DINEOF by about one order of magnitude for the ex- periments with a time-step dt 0.02. The interpolation error for observed states (first line in Tab also stresses the improved prior issued from the proposed Gibbs-like energy setting. For chaotic dy- namics, global PCA representation seems poorly adapted where local representations as embedded by the considered Gibbs energy setting appear more appealing.
follows: 200 members. noise-free
7

---

Under review as a conference paper at ICLR 2020
4.3 IMAGE SEQUENCE DATASET WITH VERY LARGE MISSING DATA RATES
The third case-study addresses satellite-derived Sea Surface Temperature (SST) image time series. Due to their sensitivity the cloud cover, such SST datasets issued from infrared sensors may involve large missing data rates (typically, between 70% and 90%, Fig.?? for an illustration). For evaluation purposes,
build groundtruthed dataset from high-resolution numerical simulations, namely NATL60 data (I), using real cloud masks from METOP AVHRR sensor (I9). We sample 128x512 images over five consecutive days from June August ocean region in the North-East Atlantic from (40.58 N,46.3 to (53.04%N, 16. W)). Overall, randomly sample 400 128x512x1 image series training data and 150 test dataset.
resolution in open
For this case-study, we consider the following four architectures for the AEs and the GE-NNs:
ConvAE1: the first convolutional auto-encoder involves the following encoder architecture: five consecutive blocks with Conv2D layer, ReLu layer and 2x2 average pooling layer, the first one with 20 filters the following four ones with 40 filters, and a final linear convolutional layer with 20 filters. The output of the encoder is 4x 16x20. The decoder involves CranpDTrasspose layer with ReLu activation for an initial 16x16 upsampling stage CDaansosassp layer with ReLu activation for additional 2x2 upsampling stage, a Conv2D layer with 16 filters and last Conv2D layer with 5 filters. All Conv2D layers use 3x3 kernels. Overall, this model involves 400,000 parameters.
ConvAE2: we consider a more complex auto-encoder with an architecture similar to ConvAE; where the number of filters is doubled (e.g. The output of the encoder is 4x16x40 tensor). Overall, this model involves = 900,000 parameters.
GE-NN,.2 22 we consider two GE-NN architectures. They share the same global architecture with an initial 4x4 average pooling, a Conv2D layer with ReLu activation with a zero- weight constraint on the center of the convolution window, a 1x1 Conv2D layer with N filters, ResNet with bilinear residual unit, composed of initial mapping to initial 32x N2)x( space with a Conv2D+ReLu layer, a linear 1xl Conv2D+ReLu layer with N filters and final 4x4 Conv2DTranspos layer with a linear activation for an upsampling to the input shape. GE-NN, and GE-NN, differ in the convolutional parameters of the first Conv2D layers and in the number of residual units. GE-NN, involves 5x5 kernels, N 20 and residual units for a total of v 30,000 parameters. For GE-NN2, we consider 11x11 kernels, N 100 and 10 residual units for total of Rs 570,000 parameters.
These different parameterizations were selected so that ConvAE, and GE-NN, involve modeling complexity in the same range. We may point out that the considered GE-NN architecture are not applied to the finest resolution but to downscaled grids by factor of 4. The application of GE- NNs to the finest resolution showed poor performance. This is regarded as an illustration of the requirement for considering scale-selection problem when applying given prior. The upscaling involves the combination of a Cons2DTranspose layer with 11 filters, a Conv2D layer with a ReLu activation with 22 filters and linear Conv2D layer with 11 filters.
Similarly to MNIST dataset, we report the performance of the different models in terms of interpola- tion score (I-score), reconstruction score (R- and auto-encoding score (AE-score) both for the training and test dataset. We compare the performance of the four models using the fixed-point and gradient-based interpolation. Overall, we can draw conlusions similar MNIST case-study. Rep- resentations trained from gap-free data lead to poor performance and the proposed scheme reaches the best performance (gain over 50% in terms of explained variance for the interpolation and re- construction score). Here, models trained with a aero-filling strategy show good interpolation and reconstruction performance, but very poor AE score, stressing that cannot apply beyond the con- sidered interpolation task. When comparing GE-NN and AE settings, GE-NNs show slightly bet- ter performance with a much lower complexity (e.g., 30,000 parameters for GE-NN, vs. 400,000 parameters for Ao,) Regarding the comparison between the fixed-point and gradient-based interpolation strategies, the later reaches slightly better interpolation and reconstruction score. We may point out the significant gain w.r.t. OI, which is the current operational tool for ocean remote sensing data. We illustrate these results in Appendix (Fig which further stresses the gain w.r.t OI for the reconstruction of finer-scale structures.
8

---

Under review as a conference paper at ICLR 2020
SST
Model
I-Score
R-score
A E-score
67.59%
70.97%
OI
(57.29%)
(61.00%)
32.52%
34.94%
74.17%
FP(5)PCA((20)
(39.22%)
(30.39%)
(56.00%)
28.01% %
30.91%
89.95%
AE models
FP(5)-CA((8)
(34.83%)
(25.28%)
(64.53%)
89.12%
89.65%
67 67.42%
Zero-ConvAEi
(86.98%)
(87.33%)
(60.41%)
87.63%
89. 89.82%
83.81%
FP(1O)-CovVAB
(85.24%)
(87.28 (87.28%)
(77.20%)
89.08%
89.51 %
84.22%
G(8)-ConvABi
(87.89%)
(88.25%)
(76.32%)
86.70%
87.14%
67.20%
Zero-ConvAEz
(86.37%)
(86.87%)
(54.77%)
88.71 %
89.14%
86.24%
FR(1O)-ConAAE,
(85.02%)
(85.49%)
(80.76)
90.47%
90.98%
86.33%
G(8)-ConvAE2
(88.00%)
(88.39%)
(78.33%)
85.46%
86.71%
-94.84%
Zero-GE-NN;
(79.39%)
(80.30%)
(-172.68%)
89.22%
90.07%
92.61 %
FP(1S)-GE-NN
(87.45%)
(88. 58%5
(90.18%)
GE-NN models
89.83%
90.56%
92.23%
G(12)-GE-NN,
(89.16%)
(90.00 %)
(90.98%)
86.60%
87.48%
-141.64%
Zero-GE-NN;
(77.38%)
(78.01 %
(-235.50%%
90.56%
91.33%
93.04%
FR(15)-GE-N:
(85.93%)
(87.26%)
(91.17%)
91.10%
91.83 %
92.36%
G(12)GEENNN
(87.98%)
(88.81%)
(90.37%)
We evaluate for each model interpolation, reconstruction and auto-encoding scores, resp. I-score, R-score and AB-score, in terms of percentage of explained variance resp. for the interpolation with missing data and the reconstruction of gap-free images. For each model, evaluate these score for the training data (first row) and the test dataset (second row in brackets). We consider four different auto-encoder models, namely 20 and 80-dimensional PCAs and ConvAE, models, and two GE-NN models, GE-NN 1.2, combined with three interpolation strategies: the classic zero-filling strategy (Zero) and proposed iterative fixed-point (FP) and gradient-base (G) schemes, the figure refers to GE-NN, with a 10-step fixed-point interpolation scheme. The PCAs are trained from gap-free data. We also consider an Optimal Interpolation (OI) with space-time Gaussian covariance with empirically- tuned parameters. We refer the reader to the main text for the detailed paramcterization of the considered models.
Table 2: Performance on SST dataset:
missing data areas, the reconstruction of the whole image
in brackets denoting the number of iterations. For instance, FP(10)-GE-M
5 CONCLUSION
In this paper, we have addressed the learning of energy-based representations of signals and images from observation datasets involving missing data (with possibly very large missing data rates). Us- ing the proposed architectures, we can jointly learn relevant representations of signals and images while jointly providing the associated interpolation schemes. Our experiments stress that learning representations from gap-free data may lead to representations poorly adapted to the analysis of data with large missing data areas. We have also introduced a Gibbs priors embedded in neural net- work architecture. Relying on local characteristics rather than global ones as in AE schemes, these priors involve much lower complexity. Our experiments support their relevance for addressing inverse problems signal and image analysis. Future work may further explore multi-scale exten- sions of the proposed schemes along with couplings between global and local energy representations

---

Under review as a conference paper at ICLR 2020
and hybrid minimization schemes combining both gradient-based based and fixed-point strategies in the considered end-to-end formulation.
REFERENCES
[]] NATL60 data: http://meeennggggggggiiibboaoooo
[2] A. Avera-Azcarate, A. Barth, G. Parard, and J.M. Beckers. Analysis of SMOS sea surface salinity data using DINEOF. RSE, 180:1377145, 2016.
[3] M. Bertalmio, L.. Bertozzi, and G. Sapiro. Navier-stokes, fluid dynamies, image and video painting. JEEE CVPR, 2001.
[4] Marcelo Bertalmio, Guillermo Sapiro, Vincent Caselles, and Coloma Ballester. Image inpainting. ACM SIGGRAPH, 117-424, 2000.
[5] E. Candes and J. Romberg. Sparsity and incoherence in compressive sampling. Inverse Problems, 23:969-968 905,995,22 2007.
[6] C. L. P. Chen, C. Zhang L. Chen, and M. Gan. Fuzzy Restricted Boltzmann Machine for the Enhance- ment of Deep Learning. IEEE Trans. Fuzzy Systems, (166):216-2213, 2015.
[7] Yunjin Chen, Wei Yu, and Thomas Pock. On Learning Optimized Reaction Diffusion Processes for Effective Image Restoration. IEEE CVPR, 5261-5269, 2015.
[8] N. Cressie and K. Wikle. Statistics for Spatio- Temporal Data. Wiley Sons, 2015.
19] A. Criminisi, P. Perez, and K. Toyama. Region filling and object removal by exemplar-based image inpainting. IEEE TIP, 13(9): 1200-1212022221 2004.
[10] G. Evensen. Data Assimilation. Springer Berlin Heidelberg, Berlin, Heidelberg, 2009.
[11] R. Fablet, P. Viet, R. Lguensat. Data-Driven Models the Spatio-Tempont Interpolation of Satellite- Derived SST Fields. IEEE Trans. on Comp. Im., 3(4):647--66 657, December 2017.
[12] W. T. Freeman and Liu. Markov Random Fields for Super-Resolution. In Advances in Markov Random Fields for Vision Image Processing. MIT Press, a. blake, p. kohli, and rother, eds. edition, 2011.
[13] D. Geman. Random fields and inverse problems in imaging. LNM, 1427:113-199 1990.
[14] P. Jain, P. Netrapalli, and S. Sanghavi. Low-rank Matrix Completion Using Alternating Minimization. In ACM STOC, 665-674, 2013.
[15] LeCun, Y. Bengio, and G. Hinton. Deep learning. Nature, 521((553))436 444, 2015.
[16] G. Liu, F. A. Reda, K. J. Shih, T.-C. Wang. A. Tao, and B. Catanzaro. Image Inpainting for Irregular Holes Using Partial Convolutions. ECCV, 2018.
[17] E. N. Lorenz. Deterministic Nonperiodic Flow. J. Atm. Sc., 20(2):130-14)) 1963.
[18] Lorenzi, F. Melgani, and G. Mercier. Inpainting Strategies for Reconstruction Missing Data VHR Images. IEEE GRSL, 8(5)914-911, 2011.
[19] S. Ouala, R. Fablet, C. Herzet, B. Chapron, A. Pascual, F. Collard, and L. Gaultier. Neural Network Based Kalman Filters for the Seatio-Temporal Interpolation of Satellite-eerived Sea Surface Tempera- ture. Remote Sensing. 10(12)1884, 2018.
[20] G. Peyre, S. Bougleux, and L.D. Cohen. Non-local Regularization of Inverse Problems. Inverse Problems and Imaging, 5(2):11 1--3300 2011.
[21] B. Ping. F. Su, and Y. Meng. An Improved DINEOF Algorithm for Filling Missing Values in Spatio- Temporal Sea Surface Temperature Data. PLOS ONE, 11(5):e0155928. 2016.
[22] R Salakhutdinov and G. Hinton. Deep boltzmann machines. Art. Int. and Stat., pages 448-455, 2009.
[23] Ulusoy and C. M. Bishop. Generative versus discriminative methods object recognition. IEEE CVPR, 2-258-265. 2005.
[24] Xie, L. Xu, and Chen. Image Denoising and Inpainting with Deep Neural Networks. Pereira,
C. J. C, Burges, L. Bottou, and K. Q. Weinberger, editors, NIPS, 341-3491 2012.
10

---

Under review as a conference paper at ICLR 2020
A
APPENDIX
A.1
SUPPLEMENTARY FOR MNIST DATASET
We illustrate below both the considered masking patterns well reconstruction examples for the proposed framework applied to MNIST dataset.
7 719
9 9 8
4
46
6 8
$
N 20, W 5
N = 30, W =5
3 0 1 0
3 0 3 9
0
N = 3, W =
N = 6, W = 9
Figure 3: Illustration of the considered MNIST dataset with the selected missing data patterns: randomly remove data from N squared areas of size W.
2
2
8
8
3
6
22
88
3 3
Figure 4: Illustration of reconstruction results for FP-ConvAE model for MNIST examples: for each panel, the first column refers to Aero-ConvAE, results and the second one to Fp(15)-ConvAE,, The first row depicts the reference image, the second row the missing data mask and the third one the interpolated image. The first two panels illustrate interpolation results for training data and last two for test data. We depict grayscale mnist images using false colors to highmight differences.
A.2 SUPPLEMENTARY FOR LORENZ-63 DYNAMICS
We report below Table which details the interpolation performance the proposed GE-NN rep- resentation applied to Lorenz-63 time series in comparison with PCA-based scheme and lower- bound provided by the interpolation assuming the ODE model known.
A.3 SUPPLEMENTARY FOR SST DATASET
We report below reconstruction examples for the application of the proposed GE-NN approach to SST time series with real missing data masks, which involve very large missing data rates (typi- cally above 80%). The consistency between the interpolation results and the reconstruction of the gap-free image from the learnt mnergy-based representation further stresses the ability of the pro- posed approach extract generic representation from irregularl-ssampled data. These reulsts

---

Under review as a conference paper at ICLR 2020
Missing data
EnKS
DINEOF
G-NN-FP-OI
1/4 (dt 0.01)
9.91e-3
5.27E-01
5.97e-04
2.83e-3
1.10e0
6,55e-03
1/4 (dt 0.02)
6.15e-02
4.53e-01
2.17e-03
1.92E-02
4.31e0
5.60e-02
1/4 (dt 0.04)
5.41e-01
5.78e-01
7.90c-03
4.17c-01
1.33e01
7.39c-01
Table Interpolation performance for Lorenz-63 dynamies with different missing data rates: we compare the proposed neural-network aproach (FP(L5)-GE-NN) to an ensemble Kalman Smoother (EnKS) assuming the dynamical model (12) is known, and a DINEOF scheme (21). We report interpolation results for a 75% missing data rate with uniform random sampling for three different sampling time steps, = 0.01, dt 0.02 and dt 0.04. We report the mean square error of the interpolation for the observed data (first row) and masked ones (second row).
also emphasize a much greater ability of the proposed learning- baaning-bbsed based scheme to reconstruct fine- scale structures, which can hardly be reconstructed by Ol scheme with Gaussian space-time covariance model. We may recall that later is the stae-of-the-art approach for the processing of satellite-derived earth observation data (8).
Figure 5: Interpolation examples for SST data used during training: first row, reference SST im- ages corresponding to the center of the considered 1 dddyyy time window; second row, associated SST observations with missing data, third row, interpolation issued from FP(15)-GE-NN) model; third row, reconstruction of the gap-free image series issued from (P(15)-GE-NN, model; interpolation issued from optimal interpolation scheme using Gaussian covariance model with empirically tuned parameters.
12

---

Under review as a conference paper at ICLR 2020
Figure 6: Interpolation examples for SST data never seen during training: first row, reference SST images corresponding to the center of the considered 11-day time window; second row, as- sociated SST observations with missing data, third row, interpolation issued from FE-N5)-GE-NN, model; third row, reconstruction of the gap-free image series issued from GEE-N)-GE-N,, model; interpolation issued from optimal interpolation scheme using Gaussian covariance model with empirically tuned parameters.
13


Reviews:
[{"review_id": "rJx-I2HJ9S", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes an end-to-end learning framework for interpolation problems, motivated by problems such as irregularly-sampled images or time-series.\n\nIt was not clear after reading the paper where the key novelty of the proposal lies. The energy formulations for Uθ, namely an autoencoder and Gibbs model, are not to my knowledge new in this context. I gather the novelty must then be in Section 3.2, which uses a neural-network based interpolation scheme. This builds on an existing update scheme (Equation 8), but replaces the full operator ψ with an iterative update. The key idea is then to replace the full gradient for the energy function with a learned function, based on a CNN. While this is not my area of expertise, I am not sure as to technical significance or novelty of such a proposal.\n\nThe paper emphasises their view of an interpolation operator I, which is a little confusing in an autoencoder context. Typically, we pick \\hat{X} to be the solution attaining minimal squared error compared to the reference Y. It was not clear why one needs a further invocation of a minimisation between Y^i and I(Uθ, Y^i, Ω^i).\n\nThe writing could be improved, as there are a number of grammatical issues, as well as missing section or equation #'s. In terms of presentation, the Introduction delves far too soon into a detailed discussion of related work in the area. I would suggest instead to provide a crisp high-level overview of the limitations of existing work, and how they are overcome in the present paper.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "Byetnaz8cr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a framework to learn representations of signals when they the signals are irregularly sampled.  They propose to do this by using some modified iteration steps from DINEOF algorithm. In addition to this, they propose a new energy function which is inspired by the idea of Gibbs distribution. The parametrize the energy function by some convolutional filters with a constraint. As claimed by the paper in the introduction, they only use the under sampled signals to learn as opposed to using the fully sampled ground truth signals in the previous deep learning basic approaches.\n\nI choose to reject this paper. 1) The paper doesn't justify what these representations are or where these representations will be used 2) There is no theoretical or extensive empirical motivation that the representation that is coming out of this network is actually useful for downstream tasks of interest. 3) Ill-defined evaluation metrics 4) Missing comparisons with basic models - why not compare with in-painting models for images and time series in experiments? 5) The paper is hard to read, not well organized and missing a lot of details. \n\nMain Argument:\nIt is not clear to me the motivation for learning these representations or how the usefulness of this representation will be evaluated. The only place where I see a use for representation other than reconstruction of the signal is in Table 1. They use a 3 layered MLP on top of the representation from auto encoder. Why is this interesting? If you're trying to claim that the representation is powerful enough, shouldn't you be able to do the classification with just a linear classifier? Can you justify the usefulness of representation for some downstream tasks in your other experiments? \n\nIn any case, I don't understand why the authors don't compare with a simple auto-encoder. The authors assumes that they know where the signal is sampled and where is is not. Why not train an auto-encoder with MSE loss, but calculating MSE over only those pixels which the authors know are sampled. Comparison with such a model will be helpful in understanding if the framework is adding anything of value.\n\nAuthors useful I-Score, R-Score, AE-Score and C-Score to compare across models. These metrics are not defined in the main text (or any reference) but there is a loose (english) definition in the caption of Table 1. For example, R-Score, is the \"reconstruction performance of known image areas\". Why are these numbers in percentages? Shouldn't you be using MSE, PSNR or SSIM? Why are some of scores negative?\n\nIf you are comparing how well you're interpolating for natural images or time series, you should compare with standard interpolation techniques in image processing and time series analysis. \n\nThe paper have a lot of missing details, including simple things like not linking to a Figure or Section the authors are referring to. Some examples are:\n1. Authors motivate using Gibbs energy by saying that in auto encoders you have to project to a lower dimensional representation and gibbs energy based solution has no such dimensionality reduction constraint. There are no such constraint in AE as well - you can vey well have your hidden representation to be over complete - so this is not a good motivation for Gibbs energy\n2. Main section of 4.1 says that they use MNIST but the caption of Table 1 says they use Fashion MNIST\n3. The naming convention of the model is hard to follow and the authors notation itself is inconsistent. \n\nSome other problems:\n1. In Abstract, they say 2. images. Why 2?\n2. Introduction, the citation for Gaussian priors is empty\n3. Last word before beginning of section to, broken link\n4. In the second para of 3.2 did you mean to say solve for (2) instead of (3)?\n5.  What is \"XXX\" matrix in second para of 3.2?\n\n\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "N/A"}, "extra_text": {}}, {"review_id": "B1lNNWYdcr", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper discusses various approaches to predict missing values in the input (filling/inpainting task).\nThey define an energy function equal to the squared L2 distance between the input and its reconstruction by various kinds of neural nets. They show slightly better performance compared to a PCA-based baseline.\n\nPositive things about this work\n- the topic is interesting\n- the last application is interesting (water temperature prediction) \n\nNegative things about this work\n- this work is very poorly  written and lacks sufficient clarity. This work needs a major rewrite. I do not even know where to start, but to give an example:\n1st sentence  of the abstract reads:  “For numerous domains, including for instance earth observation, medical imaging, astrophysics,..., available image and signal datasets often irregular spacetime sampling patterns and large missing data rates.”, a sentence that misses the verb.\n2nd sentence of the abstract reads: \"These sampling properties is” which is not grammatically correct\nAnd so on so forth. Speaking of which, the Authors make excessive use of \"....\".\n- because of the lack of clarity throughout the paper, I have had hard time figuring out what exactly the authors do. From the limited understanding I got after reading this draft twice, I think they consider a few different variants of auto-encoder and a \"log-prior\"/energy function equal to the squared L2 distance between input and its reconstruction. However, this formulation has barely any novelty. For instance, see old work like:\nS. Roth and M. J. Black, “Fields of experts: A framework for learning image priors,” in Proc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), vol. 2, San Diego, California, June 2005, pp. 860–867\nwhere they used a different log-prior, but essentially the same optimization process. If the novelty is the use of auto-encoders, then the comparison should be against methods that do not auto-encode (like the above or more modern versions of it).\n- several choices made by the Authors seem not well motivated. For instance, it's written that computing gradients is too expensive and therefore these are replaced by the G network. However, doesn't the G network also need gradients to be updated?\n- the terminology is not standard and confusing. Hidden state usually refers to the encoder output, not to the decoder output.\n- the Authors never introduce the metrics they use, reconstruction and interpolation scores.\n- in general, the motivation is unclear. Why is it a problem if the data does not come from a grid? In vision, inpainting on unconstrained masks has been standard for decades. \nMore recently, transformer architectures and GNNs seem quite good at representing sets and graph structured data.\nSo considering this context, the current motivation provided by the authors needs some refinement. In fact, the authors could consider building on top of these other more modern approaches.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This work looks at ways to fill in incomplete data, through two different energy terms.
Reviewers find the work interesting, however it is very poorly written and nowhere near ready for publication. This comes on top of poorly stated motivation and insufficient comparison to prior work.
Authors have chosen not to answer the reviewers' comments.
We recommend rejection.
Decision: Reject