Year: 2021

Abstract:
Simulations that produce three-dimensional data are ubiquitous in science, ranging from fluid flows to plasma physics. We propose a similarity model based on entropy, which allows for the creation of physically meaningful ground truth distances for the similarity assessment of scalar and vectorial data, produced from transport and motion-based simulations. Utilizing two data acquisition methods derived from this model, we create collections of fields from numerical PDE solvers and existing simulation data repositories, and highlight the importance of an appropriate data distribution for an effective training process. Furthermore, a multiscale CNN architecture that computes a volumetric similarity metric (VolSiM) is proposed. To the best of our knowledge this is the first learning method inherently designed to address the challenges arising for the similarity assessment of high-dimensional simulation data. Additionally, the tradeoff between a large batch size and an accurate correlation computation for correlation-based loss functions is investigated, and the metric’s invariance with respect to rotation and scale operations is analyzed. Finally, the robustness and generalization of VolSiM is evaluated on a large range of test data, as well as a particularly challenging turbulence case study, that is close to potential real-world applications.

heading: 1 INTRODUCTION
text: Making comparisons is a fundamental operation that is essential for any kind of computation. This is especially true for the simulation of physical phenomena, as we are often interested in comparing simulations against other types of models or measurements from the physical system. Mathematically, such comparisons require metric functions that determine scalar distance values as a similarity assessment. A fundamental problem is that traditional comparisons are typically based on simple, element-wise metrics like the L1 or L2 distances, due to their computational simplicity and a lack of alternatives. Such metrics can work reliably for systems with few elements of interest, e.g. if we want to analyze the position of a moving object at different points in time, matching our intuitive understanding of distances. However, more complex physical problems often exhibit large numbers of degrees of freedom, and strong dependencies between elements in their solutions. Those coherences should be considered when comparing physical data, but element-wise operations by definition ignore such interactions between elements. With the curse of dimensionality this situation becomes significantly worse for systems that are modeled with dense grid data, as the number of interactions grows exponentially with a linearly increasing number elements. Such data representations are widely used, e.g. for medical blood flow simulations (Olufsen et al., 2000), over climate and weather predictions (Stocker et al., 2014), to the famous unsolved problem of turbulence (Holmes et al., 2012). Another downside of element-wise metrics is that each element is weighted equally, which is typically suboptimal: E.g. in the context of fluids, smoke plumes behave differently along the vertical dimension due to gravity or buoyancy, and small key features like vortices are more indicative of the fluid’s general behavior than large areas of near constant flow (Pope, 2000).
In the domain of natural images, neural networks have been successfully employed for similarity models that can consider larger structures, typically via learning from the semantic meaning of the images through class labels or data that encodes our human perception. Similarly, physical systems exhibit spatial and temporal coherence due to the underlying laws of physics, from which we can derive a similarity model. To robustly learn similarity assessments of scalar and vectorial volumetric data from such a model, our work makes the following contributions:
• We propose a novel similarity model based on the entropy of physical systems. It is employed to synthesize sequences of volumetric physical fields suitable for metric learning.
• We show that our Siamese, multiscale feature network results in a stable metric that successfully generalizes to new physical phenomena. To the best of our knowledge this is the first learned metric inherently designed for the similarity assessment of volumetric fields.
• The metric is employed to analyze turbulence in a case study, and its invariance to rotation and scale are evaluated. In addition, we analyze correlation-based loss functions with respect to their tradeoff between batch size and accuracy of correlation computation.
The central application of the proposed VolSiM metric is the similarity assessment of new physical simulation methods, numerical or learning-based, against a known ground truth. This ground truth can take the form of measurements, higher resolution simulations, or existing models. Furthermore, the trained metric can be used as a differentiable similarity loss for physical learning problems, similar to perceptual losses for computer vision tasks.

heading: 2 RELATED WORK
text: Apart from simple Ln distances, the two metrics PSNR and SSIM (Wang et al., 2004) are commonly used across disciplines for the similarity assessment of data. Similar to the underlying L2 distance, PSNR shares the issues of element-wise metrics (Huynh-Thu & Ghanbari, 2008; 2012). SSIM computes a more intricate function, but it was shown to be closely related to PSNR (Horé & Ziou, 2010) and thus has similar problems (Nilsson & Akenine-Möller, 2020). Furthermore, statistical tools like the Pearson correlation coefficient PCC (Pearson, 1920) and Spearman’s rank correlation coefficient SRCC (Spearman, 1904) can be employed as a simple similarity measurement. There are several learning-based metrics specialized for different domains such as rendered (Andersson et al., 2020) and natural images (Bosse et al., 2016), interior object design (Bell & Bala, 2015), audio (Avgoustinakis et al., 2020), and haptic signals (Kumari et al., 2019). Especially for images, similarity measurements have been studied in multiple settings, mainly by combining deep embeddings as perceptually more accurate metrics (Prashnani et al., 2018; Talebi & Milanfar, 2018). Similarly, such metrics can be employed for various image related tasks like super-resolution (Johnson et al., 2016) or image generation (Dosovitskiy & Brox, 2016). We additionally study the behavior of invariance and equivariance to transformations, which was targeted previously for rotational symmetries (Weiler et al., 2018; Chidester et al., 2019) and improved generalization (Wang et al., 2021).
Traditional metric learning for natural image domains typically works in one of two ways: Either, the training is directly supervised by learning from manually created labels, e.g. via two-alternative forced choice where humans pick the most similar option to a reference (Zhang et al., 2018). Or, the training is indirectly semi-supervised through images with class labels and a contrastive loss (Chopra et al., 2005; Hadsell et al., 2006). In that case, triplets of reference, same class image, and other class images are sampled, and the corresponding latent space representations are pulled together or pushed apart. We refer to Roth et al. (2020) for an overview of different image metric training strategies. Similarity metrics for simulation data have not been studied extensively yet. For smoke flow synthesis, Siamese networks for finding similar fluid descriptors have been applied (Chu & Thuerey, 2017). Um et al. (2017; 2021) used crowd-sourced user studies for the similarity assessment of liquid simulations. Simulation data was compared via a learned metric (Kohl et al., 2020) using a feature extractor with a structure similar to a classification CNN, however only considering scalar 2D data and a simple, linear distance model.
Deep learning techniques also receive attention for improving simulations, e.g. for predicting states over time (Morton et al., 2018; Kim et al., 2019), for learning via physical laws (Watters et al., 2017; Greydanus et al., 2019), and for the dynamics of complex materials (Ummenhofer et al., 2020; Sanchez-Gonzalez et al., 2020). While these approaches have potential to be combined with our metric, we focus on traditional numerical methods to acquire data in the following.

heading: 3 MODELING SIMILARITY OF SIMULATIONS
text: To formulate our methodology for learning similarity metrics that target dissipative physical systems, we turn to the fundamental quantity of entropy. The second law of thermodynamics states
that the entropy S of a physical system never decreases, thus ∆S ≥ 0. In the following, we make the reasonable assumption that the behavior of the system is continuous and non-oscillating,
S = kB log(W ) (1)
S̃(s) = k log(cws) (2)
D(s) = k log(cws + 1) (3)
D(s) = log(cws + 1)
log(c+ 1) (4) and that ∆S > 0. On the right, Eq. 1 is the Boltzmann equation from statistical mechanics (Boltzmann, 1866), that describes S in terms of the Boltzmann constant kb and the number of microstates W of a system (here an ideal gas).1 Since entropy only depends on a single system state, it can be reformulated to take the relative change between two states into account: given a sequence of states s0, s1, . . . , sn, the relative entropy S̃(s) is defined in Eq. 2, where ws is the monotonically increasing, relative number of microstates defined as 0 for s0 and as 1 for sn. c > 0 is a system-dependent factor that determines how quickly the number of microstates increases. Next, we shift Eq. 2 such that the lower bound is 0 to derive a first similarity model D. This offset is necessary, since the properties of similarity metrics dictate that distances are always non-negative and only zero for identical states. Finally, we choose the system-dependent constant k appropriately, as relative similarities do not change through linear transformations, and thus k does not influence the general behavior of D. Choosing k = 1/(log c+ 1) leads to the full similarity model D(s) in Eq. 4, that is bound to the range [0, 1], and directly predicts the level of similarity between the reference s0 and any state s from a sequence.
Fig. 1 illustrates this connection between the logarithmic increase of entropy within a system and the proposed similarity model. In addition, trajectories for different ∆ through this space are shown. This ∆ between states corresponds to c in the similarity model in Eq. 4, and is directly proportional to it: For large changes ∆inf to the system, the entropy quickly increases and the states s1, . . . , sn contain little information, hence are quickly very dissimilar to s0 (red dotted curve). For the smallest possible changes ∆ , entropy slowly increases while similarity linearly decreases, due to the continuous nature of the system (green dashed line). Ideally, a sequence of states employed for learning tasks should evenly exhibit
both regimes as well as intermediate ones, as indicated by the black curve. For practicality, this should not rely on an extremely large number of states, as would be necessary for ∆ . In the following, we will refer to this property of a sequence being informative with respect to a similarity analysis as difficulty of a sequence with a fixed length n. The red area in Fig. 1 generally corresponds to sequences that are too difficult, i.e. that do not bare any similarity to s0 very quickly, while the green area generally corresponds to sequences that are too easy, i.e. that barely change.
Algorithm 1 Computation of distance prediction d of metric m and ground truth distances g on an ordered sequence s0, s1, . . . , sn.
for i = 1 to n do q.append (1− PCC(s0, si)) q ← normalize q to [0, 1] c← fit log(10
cq+1) log(10c+1) for c
for i = 0 to n− 1 do for j = i+ 1 to n do
d.append(m(si, sj)) ws.append((j − i)/n)
g ← log(10 cws+1)
log(10c+1) The resulting formulation for D represents an idealized similarity model and does not directly work as a metric due to two reasons: 1) a metric only takes two states as an input, so estimating c would be difficult, and 2) both inputs do not necessarily have to come from the same system, for example when comparing a velocity observation to a simulated ground truth. To overcome this, we formulate a semi-supervised learning problem by creating sequences s0, s1, . . . , sn of different physical systems, and training a neural network m to predict the distance d given the reference distances g as computed via D. This technique incorporates the underlying physical behavior, compared to adding variations in a post-process (as it is often done in the domain of images, e.g. by Ponomarenko et al. (2015)). Alg. 1 shows how d and g are computed: First, we estimate c by evaluating Pearson’s distance that indicates how quickly the states become uncorrelated, normalizing it to [0, 1], and optimizing Eq. 4 for c via a standard least-squares
1We do not have any a priori information about the distribution of the likelihood of each microstate in a general physical system. Thus, more generic entropy models such as Gibb’s entropy or Shannon’s entropy, both of which assume a non-uniform distribution of microstates, are not immediately applicable.
optimization. Then, g directly follows from Eq. 4 through the fit of c and the linearly increasing number of microstates ws, while d is computed by a pairwise evaluation of m.

heading: 4 SEQUENCE CREATION
text: To create sequences s0, s1, . . . , sn within a controlled environment, we make use of the knowledge about the underlying physical processes: We either employ direct changes, based on spatial or temporal coherences to s0, or use changes to the initial conditions of the process that lead to s0. The central challenge now becomes how to find suitable sequences via the magnitude of ∆, without knowing a priori which degree of non-linearity in the similarity model provides the most information for each setup. However, given a value of ∆ and a corresponding sequence, we can compute how strongly it differs from a sequence with ∆ , as depicted in Fig. 1. This is achieved by a proxy similarity function, via computing the PCC between this proxy and a simple linearly decreasing similarity as predicted by ∆ . If the resulting correlation is high, the sequence is close to a similarity behavior as predicted by ∆ , while lower correlations indicate stronger deviations. Finally, we sample sequences based on simple correlation thresholds to acquire a suitable difficulty level. By using the mean squared error (MSE) as a proxy, we empirically determined that correlations between 0.65 and 0.85 work well for all cases we considered. Hence, we propose two semi-automatic iterative methods (see Fig. 2) to create data, depending on the method to introduce variations to a given state. Both methods iteratively sample a small set of sequences to calibrate ∆ to a suitable magnitude and use that for the full data set. Compared to a stricter sampling of every single sequence, this method avoids potential biases in the data and is computationally more efficient as less sampling is needed.
[A] Variations based on Initial Conditions of Simulations Given a numerical PDE solver and a set of initial conditions or parameters p, the solver computes a solution to the PDE over the time steps t0, t1, . . . , tt. To create a larger number of different sequences, we make the systems nondeterministic by adding noise to a simulation field and randomly generating the initial conditions from a given range. Adjusting one of the parameters pi in steps with a small perturbation ∆i, allows for the creation of a sequence s0, s1, . . . , sn with decreasing similarity to the unperturbed simulation s0. This is repeated for every suitable parameter in p, and the corresponding ∆ is updated individually until the targeted MSE correlation range is reached. The global noise strength factor also influences the difficulty to some degree and can be updated.
[B] Variations based on Spatio-temporal Coherences For a source D of volumetric spatialtemporal data without access to a solver, we rely on a larger spatial and/or temporal dimension than the one required for a sequence. We start at a random spatio-temporal position p to extract a cubical spatial area s0 around it. p can be repeatedly translated in space and/or time by ∆t,x,y,z to create a sequence s0, s1, . . . , sn of decreasing similarity. Using a different position leads to new sequences, as long as the repository features enough diverse data. It is possible to add a global amount of random perturbations q with a chosen strength to the positions, to further increase the difficulty.
Data Sets To create training data with method [A], we utilize solvers for a basic AdvectionDiffusion model (Adv), Burger’s equation (Bur) with an additional viscosity term, and the full Navier-Stokes equations via a Eulerian smoke simulation (Smo) and a hybrid Eulerian-Lagrangian liquid simulation (Liq). The corresponding validation sets are generated with a separate set of
random seeds. Furthermore, we use adjusted versions of the noise integration for two test sets, by adding noise to the density instead of the velocity field in the Advection-Diffusion model (AdvD), and overlayed noise to the background area of the liquid simulation (LiqN).
We create seven test sets via method [B]. Four come from the Johns Hopkins Turbulence Database JHTDB (Perlman et al., 2007) that contains a large amount of direct numerical simulation (DNS) data, where each is based on a subset of the JHTDB and features different characteristics: isotropic turbulence (Iso), a channel flow (Cha), magneto-hydrodynamic turbulence (Mhd), and a transitional boundary layer (Tra). Since turbulence contains structures of interest across all length scales, we additionally randomly stride or interpolate the query points for scale factors in [0.25, 4] to create sequences of different physical size. One additional test set (SF) via temporal translations is based on ScalarFlow (Eckert et al., 2019), consisting of 3D reconstructions of real smoke plumes. Furthermore, method [B] is slightly modified for two synthetic test sets: Instead of using a data
repository, we procedurally synthesize spatial fields. For that, we employ linearly moving randomized shapes (Sha), and randomized damped waves (Wav) of the general form f(x) = cos(x) ∗ e−x. All data was gathered in sequences with n = 10 at resolution 1283, and downsampled to 643 for computational efficiency during training and evaluations. Fig. 3 shows parts of multiple example sequences via a mean projection to 2D, and App. B and C contain further details on each data set.
Data Distribution There are many independent factors that influence the difficulty of the created data sequences, and measuring the similarity in the generation process with a proxy function introduces additional uncertainties. In addition, our iteration schemes only calibrate ∆ to a suitable magnitude, instead of enforcing stricter bounds on every single sequence. As a result, the computed PCC values from the MSE (see Fig. 2) on the full data sets exhibit a more natural, smooth distribution with controllable difficulty, instead of introducing an artificial distribution with hard cutoffs. Due to the central limit theorem, we expect a normal distribution of the PCC values that indicate the sequence difficulty. However, correlations have a fixed upper bound of 1, meaning a truncated normal distribution is a more accurate prediction. Intuitively, this corresponds to trajectories with different curvature in the similarity model in Fig. 1. In fact, we empirically determined that only training data distributions which reasonably closely follow this prediction, result in a successful training of our model. We assume, that significantly different distributions indicate unwanted biases in the data, which prevented effective learning in our experiments. Fig. 4 shows the correlation histograms of our training and validation data sets, all of which roughly follow this desired truncated normal distribution.

heading: 5 LEARNING A DISTANCE FUNCTION
text: For our method, we generally follow the established Siamese structure, that was originally proposed for 2D domains (Zhang et al., 2018; Kohl et al., 2020): First, two inputs are embedded in a latent
3Here the predominant x-component in Cha and is separately normalized for a more clear visualization.
space using a CNN as feature extractor. The Siamese network structure means that the weights are shared, which ensures the mathematical requirements for a pseudo-metric. Next, the features are normalized and compared with an element-wise comparison like an absolute or squared difference. Finally, the feature difference is aggregated with sum, mean, and learned weighted average functions. To compute the proposed VolSiM metric, that operates on inherently more complex 3D data, the following changes to this framework are proposed.
Multiscale Network Scale is important for a reliable similarity assessment, since physical systems often exhibit self-similar behavior that does not significantly change across scales, as indicated by the large number of dimensionless quantities in physics. Generally, scaling a data pair should not alter its similarity, and networks can learn such an invariance to scale most effectively by processing data at different scales. One example where this is crucial is the energy cascade in turbulence (Pope, 2000), which is also analyzed in our case study below. For learned image metrics, this invariance is also useful (but less crucial), and often introduced with large strides and kernels in the convolutions, e.g. via a feature extractor based on AlexNet (Zhang et al., 2018). In fact, our experiments with similar architectures showed, that models with large strides and kernels generally perform better than models that modify the scale over the course of the network to a lesser extent. However, we propose to directly encode this scale structure in a multiscale architecture for a more accurate similarity assessment, and a network with a smaller resource footprint. Fig. 5 shows the proposed fully convolutional network: Four scale blocks individually process the input on increasingly smaller scales, where each block follows the same layer structure, but deeper blocks effectively cover a significantly larger volume due to the reduced input resolutions. Furthermore, deeper architectures can model complex functions more easily, so we additionally include resolution connections from each scale block to the next resolution level via concatenation. Effectively, the network learns a mixture of connected deep features and similar representations across scales as a result.
Training and Evaluation To increase the model’s robustness during training, we used the following data augmentations for each sequence: the data is normalized to [−1, 1], and together randomly flipped and rotated in increments of 90° around a random axis. The velocity channels are randomly swapped to prevent directional biases from some simulations, while scalar data is extended to the three input channels via repetition. For inference, only the normalization operation and the repetition of scalar data is performed. The final metric model was trained with the Adam optimizer with a learning rate of 10−4 for 30 epochs via early stopping, further details can be found in App. A. To train the metric end-to-end, Alg. 1 is employed to compute the predicted distances d and the ground truth g according to the similarity model. Both are passed to the correlation loss function below, that compares them and provides gradients. To determine the accuracy of a metric during inference, we compute the SRCC between d and ws, where a value closer to 1 indicates a better reconstruction.4
Loss Function Given predicted distances d and a ground truth g of size n, we train our metric networks with the loss in Eq. 5. It consists of a weighted combination of an MSE and an inverted correlation term r, where d̄ and ḡ denote the mean. While the formulation follows existing work (Kohl et al., 2020), it is important to note that g is computed by our similarity model from Sec. 3, and below we introduce a slicing technique to apply this loss formulation to high-dimensional data sets. For the training process via stochastic gradient descent, Eq. 5 requires a trade-off: A large batch size b is useful to improve training stability via less random gradients for optimization. Similarly, a
4This is equivalent to SRCC(d, g), since the SRCC measures monotonic relationships and is not affect by monotonic transformations. SRCC(d,ws) is just computationally more efficient and has numerical benefits.
sufficiently large value of n is required to keep the correlation values accurate and stable.
L(d, g) = λ1(d− g)2 + λ2 1− ∑ni=1(di − d̄) (gi − ḡ)√∑n i=1(di − d̄)2 √∑n i=1(gi − ḡ)2  (5) However, with finite amounts of memory, choosing large values for n and b is not possible in practice. Especially so for 3D cases, where a single sample can already be memory intensive. In general, n is implicitly determined by the length of the created sequences, via the number of possible pairs within a sequence. Thus, we provide an analysis how the correlation can be approximated in multiple steps for a fixed n, to allow for increasing b in return. In the following, the batch dimension is not explicitly shown, but all expressions can be extended with a vectorized first dimension.
The full distance vectors d and g are split in slices with v elements, where v should be a proper divisor of n. For any slice k, we can compute a partial correlation rk with
rk = ∑k+v i=k (di − d̄) (gi − ḡ)√∑k+v
i=k (di − d̄)2 √∑k+v i=k (gi − ḡ)2 . (6)
Note that this is only an approximation, and choosing larger values of v for a given b is always beneficial, if sufficient memory is available. For all slices, the gradients are accumulated during backpropagation since other aggregations would required a computational graph of the original, impractical size. Eq. 6 still requires the computation of the means d̄ and ḡ as a pre-process over all samples. Both can be approximated with the running
means d̃ and g̃ for efficiency (RM ). For small values of v, the slicing results in very coarse, unstable correlation results. To alleviate that, it is possible to use a running mean over all previous values r̃k = (1/k)(rk + ∑k−1 l=1 rl). This aggregation (AG) can stabilize the gradients of individual rk as they converge to the true correlation value during training, as App. D shows in more detail.
Fig. 6 displays the resulting performance on our data, when training with different combinations of b, v, RM , and AG. All models exhibit similar levels of memory consumption and were trained with the same random training seed. When comparing models with and without RM (blue vs red and purple vs orange) both are on par in most cases, even though computation times for a running mean are about 20% lower. The results of models with and without AG (blue vs purple and red vs orange) confirm the expected stabilization effect, where reduced fluctuations in the training translate to better generalizing metrics. Overall, this experiment demonstrates that choosing larger v consistently leads to better results (marker shape), so more accurate correlations are beneficial over a large batch size b in memory limited scenarios. Thus, we use b = 1 and v = 55 for the final model.

heading: 6 RESULTS
text: We compare the proposed VolSiM metric to a variety of existing methods in the upper section of Tab. 1. All metrics were evaluated on the volumetric data from Sec. 4, which contain a wide range of test sets that differ strongly from the training data. VolSiM consistently reconstructs the ground truth distances from the entropy-based similarity model more reliably than other approaches on most data sets. As expected, this effect is most apparent on the validation sets since their distribution is closest to the training data. But even on the majority of test sets with a very different distribution, VolSiM is the best performing or close to the best performing metric. Metrics without deep learning often fall short, indicating that they were initially designed for different use cases, like SSIM (Wang et al., 2004) for images, or variation of information VI (Meilă, 2007) for clustering. The strictly element-wise metrics MSE and PSNR exhibit almost identical performance, and both work poorly on a variety of data sets. The learning-based methods LPIPS (Zhang et al., 2018) for images, and LSiM (Kohl et al., 2020) are only available for two-dimensional data. Hence, they can not correlate structures along all dimensions. To evaluate them in our setting, the 3D data was sliced along all three dimensions to avoid biases against one direction, and the resulting distances are averaged over all slices. As shown in Tab. 1, this results in a reasonable performance overall.
Table 1: Top: performance comparison of different metrics for 3D data via the SRCC, where values closer to 1 indicate a better reconstruction of the ground truth distances. Bold+underlined values
show the best method for each data set, bold values are within a 0.01 margin of the best performing. Bottom: ablation study of the proposed method. Gray models make use of a different training setup.
Validation data sets Test data sets
Simulated Simulated Generated JHTDB a SF b c
Metric Adv Bur Liq Smo AdvD LiqN Sha Wav Iso Cha Mhd Tra SF All
MSE 0.61 0.70 0.51 0.68 0.77 0.76 0.75 0.65 0.76 0.86 0.80 0.79 0.79 0.70 PSNR 0.61 0.68 0.52 0.68 0.78 0.76 0.75 0.65 0.78 0.86 0.81 0.83 0.79 0.73 SSIM 0.75 0.68 0.49 0.64 0.81 0.80 0.76 0.88 0.49 0.55 0.62 0.60 0.44 0.61 VI 0.57 0.69 0.43 0.60 0.69 0.82 0.67 0.87 0.59 0.76 0.68 0.67 0.41 0.62 LPIPS (2D) 0.63 0.62 0.35 0.56 0.76 0.62 0.87 0.92 0.71 0.83 0.79 0.76 0.87 0.76 LSiM (2D) 0.57 0.55 0.48 0.71 0.79 0.75 0.93 0.97 0.69 0.86 0.79 0.81 0.98 0.81 VolSiM (ours) 0.75 0.73 0.66 0.77 0.84 0.88 0.95 0.96 0.77 0.86 0.81 0.88 0.95 0.85
CNNtrained 0.60 0.71 0.63 0.76 0.81 0.77 0.92 0.93 0.75 0.86 0.78 0.85 0.95 0.82 MSrand 0.57 0.66 0.45 0.69 0.76 0.75 0.80 0.78 0.74 0.86 0.80 0.82 0.84 0.74 CNNrand 0.52 0.66 0.49 0.69 0.77 0.70 0.93 0.96 0.74 0.85 0.79 0.83 0.95 0.81 MSno skip 0.80 0.70 0.78 0.75 0.86 0.88 0.80 0.95 0.76 0.86 0.78 0.86 0.91 0.82 MS3 scales 0.70 0.69 0.70 0.73 0.83 0.82 0.95 0.94 0.76 0.87 0.80 0.88 0.93 0.83 MS5 scales 0.78 0.72 0.78 0.78 0.81 0.90 0.94 0.93 0.75 0.85 0.77 0.88 0.93 0.82 MSadded Iso 0.73 0.72 0.77 0.79 0.84 0.84 0.92 0.97 0.79 0.87 0.80 0.86 0.97 0.84 MSonly Iso 0.58 0.62 0.32 0.63 0.78 0.65 0.72 0.92 0.82 0.77 0.86 0.79 0.65 0.75 a Johns Hopkins Turbulence Database (Perlman et al., 2007) b ScalarFlow data set (Eckert et al., 2019) c Combined test data sets
The bottom half of Tab. 1 contains an ablation study of the proposed architecture MS, and a simple CNN model. This model is similar to an extension of the convolution layers of AlexNet (Krizhevsky et al., 2017) to three dimension, and does not utilize a multiscale structure. Even though VolSiM has more than 80% fewer weights compared to CNNtrained, it can fit the training data more easily and works better or equally well on every single dataset in Tab. 1, indicating the strengths of the proposed multiscale architecture. The performance of untrained models CNNrand and MSrand confirm the findings from Zhang et al. (2018), who also report a surprisingly strong performance of random networks. Using no resolution skip connections for MSno skip (see Fig. 5) slightly lowers the metrics generalization across most data sets. Removing the last resolution scale block for MS3 scales overly reduces the capacity of the model, while adding another block for MS5 scales only leads to minor differences. In addition, we also investigate two slightly different training setups: for MSadded Iso we integrate extra sequences created like the Iso data in the training, while MSonly Iso is exclusively trained on such sequences. MSadded Iso only slightly improves upon the baseline, and even the turbulence-specific MSonly Iso model does not consistently improve the results on the JHTDB data sets. Both cases indicate a high level of generalization for VolSiM, as it was not trained on any turbulence data. App. E contains more details for each model and App. F features further ablations.
Transformation Invariance Physical systems are often characterized by Galilean invariance (McComb, 1999), i.e. identical laws of motion across inertial frames. Likewise, a metric should be invariant to transformations of the input, meaning a constant distance output when translating, scaling, or rotating both inputs. Element-wise metrics fulfill these properties by construction, but our Siamese network structure requires an equivariant feature representation that changes along with input transformations to achieve them. As CNN features are translation equivariant by design, we only empirically examine rotation and scale invariance for our multiscale metric and a standard Conv+ReLU model on a fixed set of 8 random data pairs from each data set. For the rotation experiment, we rotate the pairs in steps of 5° around a random coordinate axis. The empty volume inside the original frame is filled with a value of 0, and data outside the frame is cut to prevent any scaling.
For scaling, the data is bilinearly up- or downsampled according to the scale factor, and processed fully convolutionally by each network. In Fig. 7, the resulting distance deviation from the mean of the predicted distances is plotted for rotation and scaling operations. The optimal result would be a perfectly equal distance with zero deviation across all transformations. Compared to the model CNNtrained, it can be observed that VolSiM produces less deviations overall, and leads to significantly smoother distance curves. Note that we observe scale equivariance rather than invariance for VolSiM, i.e. a mostly linear scaling of the distances according to the input size. This is not as problematic as the unstable behavior of CNNtrained, as the general order is preserved and data is typically not compared across resolutions without resampling. The main reason is that the fully convolutional features are larger and thus lead to scale equivariant distances when compared. With a normalization for the physical size of the system, this effect could be further reduced in future work.
Case Study: Turbulence Analysis As a particularly challenging test for generalization, we further perform a detailed case study on forced isotropic turbulence. This study resembles a potential realworld scenario for our metric. For this purpose, fully resolved raw data over a long temporal interval from the isotropic turbulence data set from JHTDB is utilized (see bottom of Fig. 8). The 10243 domain is filtered and reduced to a size of 1283 with strides of 8, meaning VolSiM is applied in a fully convolutional manner, and has to generalize beyond the training resolution of 643. Three different time spans of the simulation are investigated, where the long span also uses a temporal stride of 20. Traditionally, turbulence research makes use of two-point correlations to study such cases (Pope, 2000). Since we are interested in a comprehensive spatial analysis instead of two single points, we can make use of Pearson’s distance on the full fields to obtain reference values for an evaluation. App. G contains further details on the implementation of this case study.
At the top of Fig. 8, we find that our similarity model from Sec. 3 (gray dotted) closely follows this correlation-based trajectory (red dashed) over time, as also indicated by the value SRCCa between both cases. Even though there are smaller fluctuations, the proposed VolSiM metric (blue) results in a very similar distance trajectory to the similarity model (see SRCCb) across all time spans. Our similarity metric faithfully recovers the correlation-based reference, despite not having seen any turbulence data at training time. Hence, this experiment indicates the accuracy of the similarity model, and the capabilities for generalization of the multiscale metric network to new cases.

heading: 7 CONCLUSION
text: We presented the multiscale CNN architecture VolSiM, and demonstrated its capabilities as a similarity metric for volumetric simulation data. A similarity model based on the behavior of entropy in physical systems was proposed and utilized to learn a robust, physical similarity assessment. Different methods to compute correlations inside a loss function were analyzed, and the invariance to scale and rotation transformations investigated. The proposed metric potentially has an impact on a broad range of disciplines where volumetric simulation data arises. An interesting area for future work is designing a metric specifically for turbulence simulations, first steps towards which were taken with our case study. Additionally, investigating learning-based methods with features that are by construction equivariant to rotation and scaling may lead to further improvements in the future.

heading: ETHICS STATEMENT
text: Since we target the fundamental problem of the similarity assessment of numerical simulations, we do not see any direct negative ethical implications of our work. However, there could be indirect negative effects since this work can act as a tool for more accurate and/or robust numerical simulation methods in the future, for which a military relevance exists. A further indirect issue could be explainability, e.g. when simulations in an engineering process yield unexpected inaccuracies.

heading: REPRODUCIBILITY STATEMENT
text: To ensure the reproducibility of our work, we took the following steps: An extensive appendix with details and hyperparameters regarding the implementation, data sets, and sequence creation is provided below. Furthermore, the trained VolSiM model and source code to evaluate it are provided alongside this submission.
The complete data and implementation of this work will be published upon acceptance. This includes the source code to create the data sets as well as the full data sets themselves, the source code to recreate all figures, and the source code train the metric end-to-end and evaluate it.

heading: APPENDIX
text: In the following, additional details for the proposed VolSiM metric are provided: App. A contains implementation details regarding the training and the metric model setup, and App. B features generation details for all our data sets. Next, additional examples of the created data sequences are displayed in App. C, and the training stability for the loss experiment is investigated in App. D. Details for the experimental setup of the ablation study models are given in App. E, and App. F contains additional ablation studies. Finally, details regarding the turbulence case study can be found in App. G.
A IMPLEMENTATION DETAILS
The training and evaluation process of the metric was implemented in PyTorch, while the data was simulated and collected with specialized solvers and data interfaces as described in App B. The data acquisition, training, and metric evaluation was performed on a server with an Intel i7-6850 (3.60Ghz) CPU and an NVIDIA GeForce GTX 1080 Ti GPU. It took about 38 hours of training to fully optimize the final VolSiM model for the data sequences with a spatial resolution of 643.
In addition to the multiscale feature extractor network, the following operations were used for the Siamese architecture of the metric: Each feature map is normalized via a mean and standard deviation normalization to a standard normal distribution. The mean and standard deviation of each feature map is computed in a pre-processing step for the initialization of the network over all data samples. Both values are fixed for training the metric afterwards. To compare both sets of feature maps in the latent space, a simple element-wise, squared difference is employed. To keep the mathematical metric properties, this also requires a square root operation before the final distance output. The spatial squared feature map differences are then aggregated along all dimensions into a scalar distance output. Here, we used a single learned weight with dropout for every feature map, to combine them to a weighted average per network layer. The activations of the average feature maps are spatially combined with a simple mean, and summed over all network layers afterwards. This process of normalizing, comparing, and aggregating the feature maps computed by the feature extractor follows previous work (Kohl et al., 2020; Zhang et al., 2018).
The weights to adjust the influence of each feature map are initialized to 0.1, all other weights of the multiscale feature extractor are initialized with the default PyTorch initialization. For the final loss, the MSE term was weighted with λ1 = 1.0, while the correlation term was weighted with λ2 = 0.7.

heading: B DATA SET DETAILS
text: In the following sections, the details underlying each data set are described. Tab. 3 contains a summary of simulator, simulation setup, varied parameters, noise integration, and used fields for the simulated and generated data sets. Tab. 4 features a summary of the collected data sets, with repository details, jitter and cutout settings, and spatial and temporal ∆ values. Both tables also contain the number of sequences created for training, validation, and testing for every data source. These values only apply for the general metric setup, and changes for the ablation study models can be found in App. E below.

heading: B.1 ADVECTION-DIFFUSION AND BURGER’S EQUATION
text: In its simplest form, the transport of matter in a flow can be described by the two phenomena of advection and diffusion. Advection describes the movement of a passive quantity inside a velocity field over time, and diffusion describes the process of dissipation of this quantity due to the second law of thermodynamics.
∂d ∂t = ν∇2d− u · ∇d (7)
Eq. 7 is the simplified Advection-Diffusion equation with constant diffusivity and no sources or sinks, where u denotes the velocity, d is a scalar passive quantity that is transported, and ν is the diffusion coefficient or kinematic viscosity.
Burger’s Equation in Eq. 8 is similar to the Advection-Diffusion equation, but it describes how the velocity field itself changes over time with advection and diffusion. The diffusion term can also be interpreted as a viscosity, that models the resistance of the material to deformations. Furthermore, this variation can develop discontinuities (also called shock waves). Here, u also denotes the velocity and ν the kinematic viscosity or diffusion coefficient.
∂u ∂t = ν∇2u− u · ∇u (8)
To solve both PDEs, the differentiable fluid framework PhiFlow (Holl et al., 2020) was used. The solver utilizes a Semi-Lagrangian advection scheme, and we chose periodic domain boundary conditions to allow for the usage of a Fourier space diffusion solver. We introduced additional continuous forcing to the simulations by adding a force term f to the velocity after every simulation step. Thus, f depends on the time steps t, that is normalized by division of the simulation domain size beforehand. For Adv, Bur, and AdvD, we initialized the fields for velocity, density, and force with multiple layered parameterized sine functions. This leads to a large range of patterns across multiple scales and frequencies when varying the sine parameters.
ux(p) = sum ( fx1 + 4∑ i=1 fxi+1 ∗ sin(2iπp + ci ox(i+1mod 2)+1) )
where c = (1, 1, 0.4, 0.3)
(9)
fx(p, t) = sum ( fx6 ∗ (1 + fx6 ∗ 20) ∗ 4∑ i=1 fxi+1 ∗ sin(2iπp̃ + ci ox(imod 2)+1) )
where p̃ = p + fx7 ∗ 0.5 + fx7 ∗ sin(3t) and c = (0, 1, 1, 0.7)
(10)
d(p) = sum ( {x,y,z}∑
i
sin(f id ∗ 24πpi + oid) )
(11)
Eq. 9, 10, and 11 show the layered sine functions in ux(p), fx(p), and d(p) for a spatial grid position p ∈ R3. The sum operation denotes a sum of all vector elements here, and all binary operations on vectors and scalars use broadcasting of the scalar value to match the dimensions. Eq. 12 shows the definition of the function parameters used above, all of which are randomly sampled based on the simulation seed for more diverse simulations. Note that ν was multiplied by 0.1 for Bur. The remaining velocity and force components uy(p), uz(p), fy(p), and fz(p) and corresponding parameters are omitted for brevity here, since they follow the same initialization pattern as ux(p) and fx(p). Tab. 3 shows the function parameters that were varied, by using the random initializations and adjusting one of them in linear steps to create a sequence.
fx1 ∼ U(−0.2, 0.2)3
fx3 ∼ U(−0.15, 0.15)3
fx5 ∼ U(−0.1, 0.1)3
fx7 ∼ U(−0.1, 0.1)3
ox1 ∼ U(0, 100)3
fx,y,zd ∼ {1, 1
2 ,
1 3 , 1 4 , 1 5 , 1 6 }3
fx2 ∼ U(−0.2, 0.2)3
fx4 ∼ U(−0.15, 0.15)3
fx6 ∼ U(0.0, 0.1)3 ν ∼ U(0.0002, 0.1002) ox2 ∼ U(0, 100)3
ox,y,zd ∼ U(0, 100) 3
(12)
The main difference between the Advection-Diffusion training data and the test set is the method of noise integration: For Adv it is integrated into the simulation velocity, while for AdvD it is added to the density field instead. The amount of noise added to the velocity for Bur and Adv and to the density for AdvD was varied in isolation as well.

heading: B.2 NAVIER-STOKES EQUATIONS
text: The Navier-Stokes Equations fully describe the behavior of fluids like gases and liquids, via modelling advection, viscosity, and pressure effects, as well as mass conservation. Pressure can be interpreted as the force exerted by surrounding fluid mass at a given point, and the conservation of mass means that the fluid resists compression.
∂u ∂t + (u · ∇)u = −∇P ρ + ν∇2u+ g (13)
∇ · u = 0. (14) Eq. 13 describes the conservation of momentum, and Eq. 14 describes mass conservation. Again, u denotes the velocity, P is the pressure, ρ is the fluids density, ν is the kinematic viscosity, and g denotes external forces like gravity.
Smoke To create the smoke data set Smo, the fluid framework MantaFlow that provides a gridbased Eulerian smoke solver for the Navier-Stokes Equations was used. It is based on a SemiLagrangian advection scheme and on the conjugate gradient method as a pressure solver. The simulation setup consists of a cylindrical smoke source at the bottom of the domain with a fixed noise pattern initialization to create more diverse smoke plumes. Furthermore, a constant spherical force field ff is positioned over the source. This setup allows for a variation of multiple simulation parameters, like the smoke buoyancy, the source position and different force field settings. They include position, rotation, radius and strength. In addition, the amount of added noise to the velocity can also be varied in isolation.
Liquid Both liquid data sets, Liq and LiqN, were created with a liquid solver in MantaFlow. It utilizes the hybrid Eulerian-Lagrangian fluid implicit particle method (Zhu & Bridson, 2005), that combines the advantages of particle and grid-based liquid simulations for reduced numerical dissipation. The simulation setup consists of two liquid cuboids of different shapes, similar to the common breaking dam setup. After 25 simulation time steps a liquid drop is added near the top of the simulation domain, and it falls down on the water surface that is still moving. Here, the external gravity force as well as the drops position and radius are varied to create similarity sequences. As for the smoke data, a modification of the amount of noise added to the velocity was also employed as a varied parameter. The main difference between the liquid training data and the test set is the method of noise integration: For Liq it is integrated into the simulation velocity, while for LiqN it is overlayed on the simulation background.

heading: B.3 GENERATED DATA
text: To create the shape data set Sha and the wave data set Wav, a random number of straight paths are created by randomly generating a start and end point inside the domain. It is ensured that both are not too close to the boundaries and that the path has a sufficient length. The intermediary positions for the sequence are a result of linearly interpolating on these paths. The positions on the path determine the center for the generated objects that are added to a occupancy marker grid. For both data sets, overlapping shapes and waves are combined additively, and variations with and without overlayed noise to the marker grid were created.
Shapes For Sha, random shapes (box or sphere) are added to the positions, where the shape’s size is a random fraction of the path length, with a minimum and maximum constraint. The created shapes are then applied to the marker grid either with or without smoothed borders.
Waves For Wav, randomized volumetric damped cosine waves are added around the positions instead. The marker grid value m at point p for a single wave around a center c is defined as
m(p) = cos(w ∗ p̃) ∗ e−(3.7p̃/r) where p̃ = ‖p− c‖2 . (15)
Here, r is the radius given by the randomized size that is computed as for Sha, and w ∼ U(0.1, 0.3) is a randomized waviness value, that determines the frequency of the damped wave.

heading: B.4 COLLECTED DATA
text: The collected data sets Iso, Cha, Mhd, and Tra are based on different subsets from the Johns Hopkins Turbulence Database JHTDB (Perlman et al., 2007), that contain different types of data
from direct numerical simulations (DNS). In these simulations, all spatial scales of turbulence up to the dissipative regime are resolved. The data set SF is based on the ScalarFlow data (Eckert et al., 2019), that contains dense 3D velocity fields of real buoyant smoke plumes, created via multi-view reconstruction technique.
ScalarFlow Since 100 reconstructions of different smoke plumes are provided in ScalarFlow, there is no need to add additional randomization to create multiple test sequences. Instead, we directly use each reconstruction sequence to create one similarity sequence in equal temporal steps. The only necessary pre-processing step is cutting off the bottom part of domain that contains the smoke inflow, since it is frequently not fully reconstructed. Afterwards, the data is interpolated to the full spatial size of 1283 to match the other data sets.
JHTDB The JHTDB subsets typically contain a single simulation with a very high spatial and temporal resolution and a variety of fields. We focus on the velocity fields, since turbulent flow data is especially complex and potentially benefits most from a better similarity assessment. We can mainly rely on using temporal sequences, and only need to add spatial jitters in some cases to increase the difficulty. As turbulence generally features structures of interest across all length scales, we create sequences of different spatial scales for each subset. To achieve this, we randomly pick a cutout scale factor s. If s = 1, we directly use the native spatial discretization provided by the database. For s > 1 we stride the spatial query points of the normal cubical cutout by s after filtering the data. For s < 1 the size of the cubical cutout is reduced by a factor of s in each dimension, and the cutout is interpolated to the full spatial size of 1283 afterwards. Among other details, Tab. 4 shows the cutout scale factors, as well as the corresponding random weights.

heading: C ADDITIONAL EXAMPLE SEQUENCES
text: Fig. 10, 11, and 12 show multiple full example sequences from all our data sets. In every sequence, the leftmost image is the baseline field. Moving further to the right, the change of one initial parameter increases for simulated data sets, and the spatio-temporal position offset increases for generated and collected data. To plot the sequences, the 3D data is projected along the z-axis to 2D via a simple mean operation. This means, noise that was added to the data or the simulation is typically significantly less obvious due to statistical averaging in the projection. Velocity data is directly mapped to RGB color channels, and scalar data is shown via different shades of gray. Unless note otherwise, the data is jointly normalized to [0, 1] for all channels at the same time, via the overall minimum and maximum of the data field.

heading: D TRAINING STABILITY FOR LOSS EXPERIMENT
text: To further analyze the resulting stability of different variants of the loss investigated in Sec. 5, Fig. 9 shows training trajectories of multiple models. Displayed are the direct training loss over the training iterations, i.e. one loss value per batch, and a smoothed version via an exponential weighted moving average computed with α = 0.05, corresponding to a window size of 40. The models are colorcoded according to Fig. 6, and were trained with different batch sizes b, slicing values v, usage of the running sample mean RM , and usage of the correlation aggregation AG. Note that the models were generally trained with the same training seed, meaning the order of samples is should be very similar across all runs (only with potential deviations due to race conditions and GPU processing). It
can be observed that models trained with lower batch sizes generally achieve lower training losses, leading to the better generalization on test and validation sets in Fig. 6. Using no AG makes the training procedure less stable and occasionally large loss spikes occur, leading worse generalization as a result.

heading: E ABLATION STUDY DETAILS
text: In the following, details for the ablation study models in Sec. 6 are provided. The proposed VolSiM metric uses the multiscale architecture MS described in Sec. 5 as a feature extractor. For all models, the general training setup mentioned in App. A stays identical, apart from: 1) changes to the feature extractor, 2) a different feature extractor, 3) the training amount, or 4) the training data.
Changes to MS Architecture For MSno skip the resolution skip connections (red dashed arrows in Fig. 5) are entirely removed. This means each resolution is isolated, and sharing information across features of different levels more difficult. For MS3 scales the last scale block is removed, meaning the architecture from Fig. 5 ends with the 1/4 resolution level, while the 1/8 resolution level is omitted. For the MS5 scales model, one additional scale block is added. In Fig. 5, this corresponds to a 1/16 resolution level with a 16× 16× 16 AvgPool and a scale block with 64 channels.
Simple CNN Feature Extractor The CNNtrained model employs an entirely different feature extractor network, similar to the classical AlexNet architecture (Krizhevsky et al., 2017). It consists of 5 layers, each with a 3D convolution followed by a ReLU activation. The kernel sizes (12 - 5 - 3 - 3 - 3) decrease along with the strides (4 - 1 - 1 - 1 - 1), while the number of features first increase, then decreases again (32 - 96 - 192 - 128 - 128). To create some spatial reductions, two 4 × 4 × 4 MaxPools with stride 2 are included before the second and third convolution. The normalization and aggregation of the resulting feature maps is performed as described in App. A, in the same way as for the proposed feature extractor network. As a result, CNNtrained can also be applied in a fully convolutional manner for the scaling invariance experiment in Fig. 7.
Random Models For the random models CNNrand and MSrand, the corresponding feature extractor along with the aggregation weights are only initialized and not trained. However, to normalize every feature to a standard normal distribution, the training data is processed once to determine the feature mean and standard deviation before evaluating the models, as detailed in App. A.
Different Training Data For the MSadded Iso model, we created 400 additional sequences from the JHTDB according to the Iso column in Tab. 4, that are added to the other training data. Note that they utilize different random seeds than the 60 test sequences, so the Iso data set becomes an additional validation set in Tab. 1. Similarly for the MSonly Iso model, 1000 sequences according to the Iso column in Tab. 4 were collected. They replace all other training data from the original model, meaning the Iso data set becomes a validation set, and Adv, Bur, Liq, and Smo become further test sets in Tab. 1. For consistency, the All column in Tab. 1 still reports the combined values of the original test sets for both cases.

heading: F ADDITIONAL ABLATIONS
text: To further investigate our method, we perform three additional ablations in the following. The first ablation focuses on the benefits of our entropy-based similarity model, by replacing it with a simplified identity transformation. The other two replace the multiscale aspect in different ways, to demonstrate the robustness and memory efficiency of the proposed architecture. Tab. 2 shows the resulting SRCC values for these models and the corresponding baselines on our data sets, which are computed as for Tab. 1.
Simplified Similarity Model We compare the VolSiM model as proposed in the main paper with MSidentity that does not make use of the similarity model based on entropy during training. For that, the logarithmic transformation of ws is replaced with an identity transformation, corresponding to linear ground truth distances, i.e. g ← ws in Alg. 1. The results in the first block in Tab. 2 indicate the benefits of the entropy-based ground truth, with improvements across most data sets. Note that the simplified network version is slightly more efficient as the computation of the parameter c in Alg. 1 can be omitted. However, pre-computing these values for all sequences is computationally very light compared to training the full metric network.
Singlescale Model To investigate the multiscale aspect of our architecture, we removed all scale blocks at lower resolutions, such that only the component at the native input resolution remains (see Fig. 5). To compensate for the lower number of network parameters, we scale the number channels in each layer by a factor of 11, leading to the rather shallow but wide network MS1 scale with around 360k weights. The original network structure MS4 scales with the four scale blocks reconstructs the ground truth more accurate for almost all data sets as displayed in the middle block of Tab. 2. Furthermore, the proposed structure requires about five times less memory during training.
No Pooling Layers A different approach to analyze the multiscale aspect, is to eliminate all AvgPool layers and set all convolution strides to 1 (also see Fig.5). Here, no further adjustments to the network are required as neither pooling layers nor strides alter the number of network weights. The resulting MSno pool model is only slightly worse compared to the baseline, as indicated by bottom block in Tab. 2. However, the MSwith pool model needs about ten times less memory compared to the adjust variant during training due to the significantly smaller feature sizes.
Note that for both ablations on the multiscale structure, the training and test data resolution was reduced to 323. Furthermore, the number of channels in each layer for the MSwith pool and MSno pool models was reduced by a factor of 0.5. Both choices are motivated by memory limitations and are the reason for the slightly different results across the three baseline models in Tab. 2.

heading: G TURBULENCE CASE STUDY DETAILS
text: For the case study in Sec. 6, the velocity field from the isotropic1024coarse data set from the JHTDB (Perlman et al., 2007) is utilized. Instead of creating adjusted sequences according to the similarity model, the data is directly converted to three sequences of different time spans without any randomization in this case. A spatial stride of 8 is employed is employed for all cases, and to reduce memory consumption an additional temporal stride of 20 is used for the long time span only. The resulting sequences exhibit a spatial resolution of 1283, with 40 frames for the short span, 400 frames for medium span, and 200 frames for the long span. Before further processing, each frame is individually normalized to [−1, 1]. To create the results in Fig. 8, the first simulation frame t1 is compared to all following simulation frames ti via Pearson’s distance 1−PCC(t1, ti) (red dashed trajectory) and VolSiM(t1, ti) (blue solid trajectory). Note that VolSiM is applied fully convolutionally here, as it was trained on 643 data. For both cases, the resulting distances are normalized to [0, 1] to visually compare them more easily. The gray dotted curve, is the direct application of the proposed similarity model via Alg. 1 by fitting c for each time span, as described in Sec. 3. The examples from the sequences are visualized as described in App. C, while a projecting along the x-axis here.

Review summaries:
This paper describes a learned similarity metric for spatial scalar and vector fields obtained from physics simulations. The metric is like a perceptual loss but for physical fields. The training is self-supervised by comparing one-parameter families of fields obtained by modifying initial conditions or restricting or resampling.

The paper presents a learning-based method for evaluating the similarity between 3D volumetric data. A new similarity model based on the entropy of a physical system is proposed, which in my opinion is the paper's major contribution. The method combines the new entropy-based model with a Siamese network that is commonly used for similarity measurement on 2D imagery data. The proposed method further modifies the Siamese network in order to process 3D data. Experimental results are demonstrated on turbulence flow data, and are compared with other 2D similarity measurements. 

The paper proposes a similarity model for volumetric simulations derived from Boltzmann’s equation. A formulation of a similarity measure between a reference state and any other microstate in a dissipative physical system is presented. This similarity cannot be used as a metric as is, but can be used to train a Siamese CNN (VolSiM) that is introduced in this paper. The authors formulate a learning task by generating a sequence of physical systems. They use Pearson’s distance to estimate how quickly the microstates in a system are changing, and compute a ground truth for the CNN training using the proposed similarity measure. The network is trained to predict the distances. The CNN is designed to preserve a certain degree of translational, rotational and scale equivariances. Translational equivariance, apart from pooling operations, is given by the nature of CNNs; scale equivariance is directly encoded in the architecture of VolSiM by using four blocks processing the input data in multiple scales; and rotational equivariance is encouraged by data augmentation. The network is given ground truth distances based on the proposed similarity model, predicts distances and uses a loss which is a weighted sum of an MSE term and a correlation term. The authors propose two approaches to generate data, one which is based on varying the initial conditions of PDEs and using PDE solvers for a number of physical simulations, which is used as training data, and one which is based on spatial and temporal variations, which is used to generate test data. Furthermore, a publicly available database is used for testing. The datasets in the experiments cover a wide range of different physical models.