Unsupervised Learning of Automotive 3D Crash Simulations using LSTMs
Authors: Amin Abbasloo
Jochen Garcke
Rodrigo Iza-Teran
Venue:  (2020)
Keywords: LSTM
surface data
geometric deep learning
numerical simulation

Abstract: Long short-term memory (LSTM) networks allow to exhibit temporal dynamic behavior with feedback connections and seem a natural choice for learning sequences of 3D meshes. We introduce an approach for dynamic mesh representations as used for numerical simulations of car crashes. To bypass the complication of using 3D meshes, we transform the surface mesh sequences into spectral descriptors that efficiently encode the shape. A two branch LSTM based network architecture is chosen to learn the representations and dynamics of the crash during the simulation. The architecture is based on unsupervised video prediction by an LSTM without any convolutional layer. It uses an encoder LSTM to map an input sequence into a fixed length vector representation. On this representation one decoder LSTM performs the reconstruction of the input sequence, while the other decoder LSTM predicts the future behavior by receiving initial steps of the sequence as seed. The spatio-temporal error behavior of the model is analysed to study how well the model can extrapolate the learned spectral descriptors into the future, that is, how well it has learned to represent the underlying dynamical structural mechanics. Considering that only a few training examples are available, which is the typical case for numerical simulations, the network performs very well.

Under review as a conference paper at ICLR 2020
UNSUPERVISED LEARNING OF AUTOMOTIVE 3D CRASH SIMULATIONS USING LSTMs
Anonymous authors Paper under double-blind review
ABSTRACT
Long short-term memory (LSTM) networks allow to exhibit temporal dynamic behavior with feedback connections and seem a natural choice for learning se- quences of 3D meshes. We introduce an approach for dynamic mesh representa- tions as used for numerical simulations of car crashes. To bypass the complication of using 3D meshes, we transform the surface mesh sequences into spectral de- scriptors that efficiently encode the shape. A two branch LSTM based network architecture is chosen to learn the representations and dynamics of the crash dur- ing the simulation. The architecture is based on unsupervised video prediction by an LSTM without any convolutional layer. It uses encoder LSTM to map input sequence into fixed length vector representation. On this representation decoder LSTM performs the reconstruction of the input sequence, while the other decoder LSTM predicts the future behavior by receiving initial steps of the sequence as seed. The spatio-temporal error behavior of the model is analysed to study how well the model can extrapolate the learned spectral descriptors into the future, that is, how well it has learned to represent the underlying dynamical structural mechanics. Considering that only few training examples are available, which is the typical case for numerical simulations, the network performs very well.
INTRODUCTION
Data driven virtual product design is nowadays an essential tool in the automotive industry saving time and resources during the development process. For new car model, numerical crash simu- lations are performed where design parameters are changed to study their effects on physical and functional properties the car such as firewall intrusion, weight, or cost (Fang et al. (017).. Since one simulation run takes couple of hours ulation not feasible. Therefore, a system that is able to use a limited dataset and predict new simulations would make the development process faster and more efficient.
compute cluster, running a large number of sim-
The rise of deep neural networks (DNNs) in recent years encourages further research and industrial usages. Besides manifold research for autonomous driving, it natural for the automotive industry to seek and evaluate the possible applications of DNNs also in the product design stages. As an example, we investigate car crash tests, in which for example the plate thickness of certain parts strongly influences the bending behavior of structural beams and as result also the intrusion of the firewall into the passenger compartment. Here, numerical crash simulations for different variations of such thicknesses are used as a dataset for learning. The aim is to design system based on a DNN architecture that learns the crash behavior and would be able imitate the crash dynamics.
Car crash simulations are based on mathematical model of the plastic deformations and other physical and mechanical effects. They are defined on a computing mesh of currently up to three million points and up to hundred time steps are stored. Each data instance is simulation run of pre-selected parts and/or time steps-that is very high dimensional. Working with this data directly exasperates any machine learning (ML) method, but transformation of this data presented in Iza- Teran & Garcke (2019) allows to obtain a new representation that uses only small number of coefficients represent the high resolution numerical solutions. The transformed representation employed here to compress the mesh geometries to feature sets suitable for neural networks, while avoiding to directly handle geometries in the machine learning method. This way, network

---

Under review as a conference paper at ICLR 2020
designed for video prediction and embedding based on long short-term memory (LSTM) based architecture (Srivastava et al. 2015) can be adapted for mesh data. Since LSTM is a recurrent neural network that allows to exhibit temporal dynamic behavior with feedback connections, it is natural choice for learning the 3D sequences. The aim is that the network learns the observed crash behavior including translation, rotation, deformation of the parts in the model.
Since the contribution of this paper is using DNNs for analyzing car crash data, the related works are categorized into group of publications in which DNNs are extended for 3D graphics and one that concerns the use of ML techniques for analyzing car crash simulations. For the latter, one typically uses different embedding techniques to obtain low dimensional representation for the intrinsic underlying data space and to cluster simulations with similar characteristics together (Bohn et al., 2013
Diez 2018
Garcke & Iza- Ter-Teran 2015 Iza- TaaTTern& & Garcke 2019 Le Guennec et al. 2018).
The majority of publications about 3D DNN tried to extend CNN for 3D space and focus on de- scription learning and shape correspondence, also known as geometric deep learning, (Bronstein) et alll|22000 2017 Masci et al. [2015] Monti et al., 2017 Boscaini et al.) 2013 2016; Litany et al.] 2017 Halimi et al. 2018 Maturana & Scherer 2015] Su et al. 2015: Wang et al. 2017) and some devel- oped CNN filters for unorganized point clouds (Qi et al. 201 b). The very active research is so far very compute resource consuming and there is no extension of ConvLSTM for 3D space to our
knowledge, but for prediction one would need an LSTM (or GAN) approach.
However, a couple of very recent works introduce new feature sets and architectures for mesh em- bedding using autoencoders and LSTM (Tan et al.) 2018b) Qiao et al., 2018, Tan et al., 8a). The feature representation is using local shape deformations obtained by solving an optimization problem at each node and a global optimization for compensating for rotations. They have shown that after training the network, a sequences of 3D shapes as an animation can be generated by do- ing operations in the latent space. The bidirectional LSTM architecture shown outperform autoeconders (Tan et al. 8a). An LSTM based learning network has also been proposed in Qiao al. (2018), where the obtained feature representation is then taken as the temporal data to be feed into a CNN that takes the features and represents them in a lower dimensional latent space. This information subsequently feed into the LSTM module.
2 MESH DATA AND ITS REPRESENTATION
Data collection is bottleneck for deep learning applications. If the training data is not diverse enough, the network would neither be able to properly learn the intrinsic data space nor be able to return reasonable output for before unseen data. We focus on surface mesh data from numerical sim- ulations of car crashes, where in the industrial setting the car model is divided into several physcical car parts.
As simple car model we use Chevrolet C2500 pick-up truck, model with around 60,000 nodes from the National Crash Analysis Center] The data stems from numerical crash simulation of a frontal crash for random variations of the plate thickness of nine structural components, a setup similar to Bohn et al. 2013).. The thickness variations result in different deformation behavior. Figure shows snapshot of the crash simulation for the truck model.
The geometries the car model and parts are available in a regular mesh format and the correspon- dence of vertices between different simulations and over the time of the simulation are known by their node id. Therefore, instead of working with the meshes one can simply work with vertices and treat them like organized point clouds, while being able to recover the mesh and connectivity at any time. Now, the features for training the network are obtained from these point clouds and the network outputs feature vector that is later post-processed to a point cloud mesh.
Instead of working directly with 3D surface meshes, a set of features is extracted for training the network. Such a feature set should be able to represent the dynamics of the crash efficiently. In Iza- Teran & Garcke (2019) a compact representation for deforming shapes has been presented for the car crash case. The approach based on the property that the Laplace Beltrami Operator (LBO) on a surface is invariant to isometric deformations. That is, the LBO is the same if surface mesh is de-
1 from httprh/hhhrr//////b..areerrrreer "computed with hSSpDM//ttpp//wow/..accoooooooop www.e///uwwuuuuuu/uu//u/ /ml /mode 1 htm]
2

---

Under review as a conference paper at ICLR 2020
snapshot of the crash simulation of the 3D model used for data collection with zoom unto the studied longitudinal beams, from Bohn et al. (2013) (left). The four selected parts (beams) after the crash for one selected simulation, later used for illustration (right). The colors of the parts used to display error behavior per part.
Figure 1:
formed in such a way that it is neither stretched teared apart, which is the case for the considered simulations, which additionally all start from the same geometry. Consequently the eigenvectors, which form an orthogonal basis, do change under an isometric transformation and can be used to represent mesh functions such as the deformations (as three functions, one each for y, 2). The representation obtained by projecting mesh functions onto the common orthogonal basis com- pute so-called spectral coefficients. It turns out that most of the variations of the deformed shapes are concentrated small number of coefficients. Therefore an efficient representation obtained using few spectral coefficients. As shown by Brezis & Gómez-Castro (017) using suitable assump- tions, for the Lz-approximation of functions controlled in the H SSbolevnoom the orthonormal basis stemming from the Laplace operator provides an optimal approximation in certain sense; this result can be extended to the LBO and functions in the Sobolev space 2 Furthermore, the spectral representation can be understood of signals. This representation is introduced here for training LSTMs and allows to bypass the complexity of dealing with large meshes directly.
mesh surface analogy to the Fourier decomposition
To formalise, in Baxm we collect the first m unit eigenvectors of the LBO, where n is the number of points in the considered 3D shape. The spectral coefficients C3xm time step obtained by
Caxm Raxm
(1)
where Raxn contains the æ, y, z coordinates of the points from the considered 3D shape. Four B matrices are required for four distinct parts in the dataset, which are four distinct geometries.
Recovering the 3D shapes from their spectral coefficients is possible by
Rixnn CaxmB
since Baxm is orthonormal. R' is an approximation of R and by choosing larger m, the approxima- tion error gets smaller. In other words, the more eigenvectors are used Baxms the more details are saved in the spectral representation. Figure V visualizes the localization and histogram of average error for reconstructing four selected parts from their spectral coefficients. The averaging is done over 205 x 10 samples for each part independently. Note that the bounding box of the entire data [2750, 4500) x [-600, 600] x [250, 650), determined over simulation time. Comparing the his- togram with the tight bounding box 's dimensions shows that the reconstruction error is not very high for m = 40, while the part with blue color coding seems to have more overall error. Note that the error is localized mostly toward the front of the parts, this area goes through very large deformations during the crash. Note that with m 100 the observed maximal observed error would go down from 40 to 20.
MODEL DESCRIPTION
For frame prediction in video and obtain latent representations number of quite interesting applications robotics and computer vision have been developed. Different DNN architectures have been investigated very extensively. Srriasttvaa et al. 2015) was of the first proposals of an
3

---

Under review as a conference paper at ICLR 2020
160
120
100
80
10
30
Figure 2: Localization and histogram of errors between original parts and their reconstruction from their spectral representation with m = 40 spectral coefficients. Histogram color coding follows the color convention in Figure [ right.
unsupervised LSTM based representation learning method for video. The logic behind the choice is that the same operation must be applied at each step to propagate dynamics the next step. This implies that the underlying dynamics of the dataset remains the same, i.e. the same dynamics acting on any state, anytime, should produce the next state (Srivastava al. 2015). Others extended this work by adding convolutional operations in LSTM, introducing new recurrent networks, or using
model based architectures (Xingjian et al. 2015 Finn al., 2016 Kalchbrenner al. 2017 Lotter
et al. 2016 Sharma et al. 2015).
Car crash simulation data can basically be considered sequence of 3D geometries. analogy the video processing case, one can see each geometry in the sequence as video frame. Inspired from the work on video prediction, we use similar architecture for the case of car crash representation learning. We choose a two branch LSTM architecture with reconstruction and prediction decoders from SSivastava et al. (2015). The encoder LSTM maps an input sequence of & time steps into fixed length vector representation. This representation is decoded using a decoder LSTM to perform the reconstruction the input sequence of size k. Receiving a few steps 1, 1 k, from the beginning of the sequence, the other decoder LSTM predicts future sequences of length A see Figure where we here use = k/2 for simplicity. The architecture is Srivastava et al. (2015) shown to have better performance for predicting future frames compared to other approaches.
The design of the prediction encoder-decoder LSTM is same as that the autoencoder LSTM, except that it is like supervised method in which the decoder LSTM predicts future behavior that comes after the input sequence while the encoders hidden state will capture information about the representation of the input sequence. In comparing k time steps of the simulations and reconstruct them again, the prediction encoder-decoder LSTM receives just the first / steps and predicts the remaining time steps. Therefore, the input sequence acts as seed points for generating the rest of the sequence.
autoencoder L.STM that receives the entire
This compositional architecture is supposed address the shortcomings that one has to confront encoder-predictor by the use of an autoencoder or encoder- predictor alone. Namely, on the one hand an autoencoder LSTM suffers from a bias by memorizing the inputs, which not sufficient for predicting future frames. On the other hand, encoder-decoder LSTM suffers from the bias to save information mostly about the last few frames since these carry more information for predicting the future frames, but then the representation at the end of the encoder will ignore large parts of the input. In the compositional architecture, the model is trained to also predict all of the input sequence, therefore cannot just store information about the last few frames (Srivastava al. 2015).
is worth to mention that the dataset contains the translation, rotation, and deformation of car parts during the crash. Therefore the network would learn the entire degrees of freedom of the crash dynamics and would be able to imitate the crash by the prediction decoder after receiving the initial time steps seeds.

---

Under review as a conference paper at ICLR 2020
Input Reconstruction
LSTM
f
LSTM
ISTM
Sequence of Inputs
Future Prediction
fivi
Figure 3: The two branches LSTM autoencoder network architecture. The data sequence of fs are feed into the network in two lengths and a joint encoder is learned. The top decoder is learning the reconstruction, while the bottom decoder is performing future prediction.
EXPERIMENTS AND EVALUATION
Although the 3D truck model consists of several parts, only the left and right structural beam, which made of two parts each, considered for evaluating approach. These beams are structurally very important parts of the car and typically investigated by engineers, therefore it is justified to concentrate these parts.
Overall 205 different simulations were performed. For each simulation, ten snapshots equally dis- tributed in time selected and the deformation of the two beams extracted from it. Therefore a data sample consists of the four selected parts during ten time steps. Together, the dataset after the extraction from the crash simulations contains 205 samples and each full sample is long con- catenated vector of 4 x 3 x 40 x 10 elements. The entire dataset is divided into training and test set of 105 and 100 samples, respectively. Each data point, i.e. each time step of a simulation, is normalized by the l2 norm of the features at t = 0 beforehand.
Note that one could consider different scenario in which each part, out of the four, over all ten time steps is a data sample, that is treat the for beams with individual machine learning models. Since the dynamics of the two beams are coupled during the crash, considering them together can reduce the error for learning the crash dynamics by the network and we thereforce consider this setup.
We employ the two branch LSTM from Figure where the implementation done with Keras. In our experiments, the encoder component of the network has 1000, the decoder has 1500, and the prediction part has 2000 LSTM units without any convolutional layer, as defining a convolutional layer for 3D graphics is not trivial and existing approaches resource intensive. Further, ReLU as the activation function. Using ADAM with default parameters the entire network trained together, which includes the encoder, reconstruction, and prediction parts 100 iterations, until the mean squared eтrоr (over all parts) reaches the order of 10-7 and becomes stable. The training phase takes about 30 minutes on Intel i7-7700 CPU@ 3.60GHZ 8. The achieved minimum during training is trade-off' between reconstruction and prediction loss functions. The prediction part of the network receives the first five time steps as inputs and generates the next five steps of the crash (which are both vector of 4 x 3 x 40 x 5 elements).
For each grid point j we compute in the following the least squares error at time and averaged over the simulations, as well as accumulated over time:
E (t) (t) - (t))l AE =
Σ E Et
where s is the number of simulations and (t), St(() are the diree-dimessional (one for each direc- tion) original mesh function and its reconstruction (or prediction) for simulation i time t.
5

---

Under review as a conference paper at ICLR 2020
30000
20000
4000
2000
5000
Figure 4: Histogram of the reconstruction (blue) and prediction (red) errors for the spectral coeffi- cients by the network, left for the training data, on the right for the testing data.
4.1 NETWORK PERFORMANCE AND EXISTING BIFURCATION
Figure 4 shows the histograms of the reconstruction and prediction error for the training and test datasets of all four parts, only here for the spectral coefficients. The reconstruction error is larger than the prediction error since the reconstruction part recovers the entire k = 10 time steps while the prediction part just generates the last
time steps.
Bohn et al. 2013) and Iza- Zeran & Garcke (2019) have reported a bifurcation for this car crash dataset. That is, two different bending behavior for the beams arise in the simulation data, which due to changes in the plate thicknesses. One can cluster the 205 simulations into two groups of 71 and 134 for the two bending behaviors. It gives an unique opportunity to investigate the performance of the proposed system by analyzing and visualizing the enror for each branch of bifurcation indi- vidually. Moreover, it is possible to see and how well the network can recognize this bifurcation and how the spectral coefficients preserve this.
The 105 simulations used for training include samples from both branches of the bifurcation, as do the remaining 100 for testing. The distances between ground truth and the outputs of two decoders, after decompression and re-normalization from the spectral coefficient back to 3D) shapes, are vi- sualized template present the spatial localization of the average error of each bifurcation branch and its histogram is also shown for better comparison (all for the testing dataset), Figure 5 gives the accumulated error over time. Overall, the reconstruction and prediction errors small in relation to the bounding box of the data, the network based on the spectral coefficients is able to learn the complex structural mechanics.
It can be seen in first two histograms of Figure 5 that the errors accumulation for the reconstruction in the two branches is similar, which can also seen the first and second row in Figure 5 The part color coded red shows somewhat higher error values, while the blue part has more mid-range errors. The third and fourth row in Figure for the prediction have very different error localization.
Considering the localization and histograms of the average error over time for the reconstruction branch of the network, one can observe that the error behavior stays roughly the same over time, i.e. the localization of error and histograms show that the error stay the same.
Regarding the localization and histogram of the average error over time for the prediction branch of the network, now the error increases in time and the behavior changes at the eighth time step. Here, one can observe that the localization of the error changes and the histograms show that the error increases more strongly for the first branch, see Figure 7 in the Appendix.
4.2 EMBEDDING
The encoder LSTM weights can be used for visualizing the intrinsic underlying space of the dataset. Therefore, order to see how well the encoder learned the representation and dynamics of the crash simulations, the weights are visualized using different markers for the two branches of bifurcation. Due to the encoder having 1000 layers, to which each sample is mapped, one needs to some visualization techniques for embedding from higher dimensions
lower dimensional
space. We use t-SNE (Van Der Maaten & Hinton, 2008) and show the result in Figure 6 As can be seen, the bifurcation is shown two well-separated clusters. There few points from each

---

Under review as a conference paper at ICLR 2020
140
reconstruction branch reconstruction branch prediction branch
prediction branch 2
Figure 5: Localization and histogram of the еπоо accumulation AE. First and third rows show the reconstruction and prediction error, respectively, for the first bifurcation branch. Second and forth rows show the reconstruction and prediction error, respectively, for the second bifurcation branch. The error shown on a representative final time step of each branch to illustrate the different deformation behaviors. Histogram color coding follows the color convention in Figure right.
Figure 6: 2D embedding LSTM's reconstruction weights. Two branches of the bifurcation well separated in the 2D visualization made by t-SNE. First bifurcation branch is marrked by red dots and the second one by blue triangles.
bifurcation branch that are misaligned with the rest of their branch members. This might be because of using the spectral coefficients as approximation which leads to losing some details about the bifurcation, therefore increasing m might rectify the issue. Or 2D embedding is not sufficient for properly visualization the encoder LSTM weights. Nevertheless, this 2D visualization proves that the network was able to learn the complex dynamics of the crash since the bifurcation is well represented by the encoder weights.

---

Under review as a conference paper at ICLR 2020
5 CONCLUSIONS AND FUTURE WORK
Video frames prediction has been in the center of attention of researchers for while, but there has been only very few extensions of these works to the 3D case so far. The problem is addressed here by introducing spectral coefficients to encode functions on the geometry together with two branch LSTM based architecture without any convolutional layer, which has already proven to be feasible for video embedding and future frames prediction. The employed LBO basis and the resulting spectral coefficients provide trade-off between accuracy and required computational resources. We encode the 3D shapes by a set of features using the eigenvectors of the LBO.
For empirical evaluation, dataset is employed from set of numerical simulations of car during crash under different design conditions, i.e. plate thickness variations. The appearance of bifur- cation during the crash in the dataset, motivates an error analysis done for both groups see how good the network performs in the presence of a bifurcation. In both branches, the network is able to perform very good predictions, while we observe different error localisations for reconstruction versus prediction. Moreover, the 2D visualization of the reconstruction branch shows the bifurca- tion as two clusters. In any case, from a relatively small number of data, the proposed network using spectral coefficients is able to learn complex dynamical structural mechanical behaviors.
Future work could go toward scaling the pipeline for learning the crash dynamics of the entire car and larger mesh sizes, which increases the needed computational effort. On the other hand, one might be able to use smaller number of eigenvectors by not simply selecting the first few ones, but those with a large variance in the spectral coefficients of the data set. Furthermore, in practical settings, re-meshing of the parts can take place, here using spectral coefficients can ease this step since
encode shapes with different vertices number fixed size feature vectors, long as the geometry is (approximately) isometric. Still, there is the overall question, if and how a trained network can be evaluated for changed geometries (relevant question for any 3D DNN approach introduced so far) or different crash setups. Moreover, adding design parameters could also improve the accuracy but requires modifications of the networks architecture.
For practical applications, as each crash simulation requires hours of heavy computation running computational solvers on a large cluster, a system that is able to learn the representation of exper- iments with very few training data and generate the predicted simulation results for new design parameters would save much resources. Moreover, the ultimate goal of research along this direction would be a data driven system that receives very little information about the simulation (like design parameters) and output the crash sequences with minimum error.
Another application of the current system could be feasibility detectors while running the simulation on the compute cluster. Using the network, one could check if the simulation goes well if for some reasons it should be terminated. From the current stage of the system, the parts of the future simulation simply by extrapolating the learned spectral coefficients from a few initial time steps, which are already computed on the cluster, inputs. If the distance between network predicts and simulation gets very large over the iterations, the simulation can be terminated since it failed the feasibility check.
would be able generate
Further, related works such as Qiao et al. (2018) introduce a specific feature set and LSTM autoen- coders, where also graph convolution operation is required. This approach could be applied for crash data under the assumption that the local optimization can still be applied for large deforma- tions as the ones occurring in our applications. Further, the resulting features are long vectors, which results in 8 hours for learning on a CPU/GPU system for a data set similar in size to ours, where we need 30 minutes. Nevertheless, a comparison of these two approach will be worthwhile future work.
REFERENCES
Yoshua Bengio and Yann LeCun. Scaling learning algorithms towards AI. In Large Scale Kernel Machines. MIT Press, 2007.
Bastian Bohn, Jochen Garcke, Rodrigo Iza- Teran, Alexander Paprotny, Benjamin Peherstorfer, Ulf Schepsmeier, and Clemens-Auuust Thole. Analysis of car crash simulation data with nonlinear machine learning methods. Procedia Computer Science, 18:621-630, 2013.
8

---

Under review as a conference paper at ICLR 2020
Bastian Bohn, Jochen Garcke, and Michael Griebel. A sparse grid based method for generative dimensionality reduction of high- nigh-dimensiooal data. Journal of Computational Physics, 309:1-17, 2016.
D. Boscaini, J. Masci, S. Melzi, M. M. Bronstein, U. Castellani, and P. Vandergheynst. Learn- ing class-specific descriptors for deformable shapes using localized spectral convolutional net- works. Computer Graphics Forum, 34(5):13 23, 2015. doi: 10.111 12693, URL https: onlinelibrary www.y. doi abs /I0 1111 cgf.12693.
D. Boscaini, J. Masci, E. Rodolå, M. M. Bronstein, and D. Cremers. Anisotropic diffusion de- scriptors. Computer Graphics Forum, 35(2):331-441, 2016. doi: 10.1111/cgf.122 12844. URL
hittps://on. htt.///miilibbaarrr.... www.w..wwe....... com /ti/abs///////aa/cc///.aba/aaaeb 10 1111/cgf....... 12844
Haim Brezis and David Gómez-Castro. Rigidity of optimal bases for signal spaces. Comptes Rendus Mathematique, 355(7):780-785, jul 2017. doi: 10. 20016f.ccma.2017.0666.
Michael M Bronstein, Joan Bruna, Yann LeCun, Arthur Szlam, and Pierre Vandergheynst. Geomet- ric deep learning: Going beyond euclidean data. IEEE Signal Processing Magazine, 34(4):18-42, 2017.
Constantin Diez. Using artificial intelligence to analyze crash simulations. NAFEMS Benchmark, pp. 20-23. 2018.
Jianguang Fang, Guangyong Sun, Qiu, Nam H. Kim, and Qing Li. On design optimiza- tion for structural crashworthiness and its state of the art. Structural and Multidisciplinnry Optimization. 1591)119-- mar 2017. doi: 10. (0007//00118-001-55777 //link. www.sprinnggeeo spw.igerrgggr....m com/10 1007/ 581110000000-111-5555-8-
URL http:
Chelsea Finn, Ian Goodfellow, and Sergey Levine. Unsupervised learning for physical interaction through video prediction. In Advances in neural information processing systems, pp. 64-72, 2016.
Jochen Garcke and Rodrigo Iza-Teran. Machine learning approaches for repositories of numerical simulation results. In 10th European LS-DYNA Conference, 2015.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016.
Oshri Halimi, Or Litany, Emanuele Rodolå, Alex Bronstein, and Ron Kimmel. Self-supervised learning of dense shape correspondence. arXiv preprint arXiv:1812.0244 2018.
Geoffrey E. Hinton, Simon Osindero, and Yee Whye Teh. A fast learning algorithm for deep belief nets. Neural Computation, 18:1527-1554, 2006.
Rodrigo Iza-Teran and Jochen Garcke. A geometrical method for low-dimensional representations of simulations. SIAM/ASA Journal on Uncertainty Quantification, 7(2):472 496, 2019. doi: dot::0..133//7MM154400
Nal Kalchbrenner, Aäron van den Oord, Karen Simonyan, Ivo Danihelka, Oriol Vinyals, Alex Graves, and Koray Kavukcuoglu. Video pixel networks. Proceedings of the 34th Interna- tional Conference on Machine Learning-
Scamning-Volume Volume 70, pp. 1771-1779. JMLR. org, 2017.
Yves Le Guennec, J-P Brunet, F-Z Daim, Ming Chau, and Yves Tourbier. A parametric and non- intrusive reduced order model car crash simulation. Computer Methods in Applied Mechanics and Engineering, 338:186-207, 2018.
Or Litany, Tal Remez, Emanuele Rodolà, Alex Bronstein, and Michael Bronstein. Deep functional maps: Structured prediction for dense shape correspondence. In Proceedings of the IEEE Inter- national Conference on Computer Vision, pp. 5659 5667, 2017.
William Lotter, Gabriel Kreiman, and David Cox. Deep predictive coding networks for video pre- diction and unsupervised learning, arXiv preprint arXiv:1605.081 080400 2016.
Jonathan Masci, Davide Boscaini, Michael Bronstein, and Pierre Vandergheynst. Geodesic volutional neural networks on riemannian manifolds, In Proceedings of the IEEE international conference on computer vision workshops, pp. 37-45, 2015.

---

Under review as a conference paper at ICLR 2020
Daniel Maturana and Sebastian Scherer. Voxnet: A 3d convolutional neural network for real- time- object recognition. In 2015 JEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 922-928 -28... IEEE, 2015.
Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodola, Jan Svoboda, and Michael M Bronstein. Geometric deep learning on graphs and manifolds using mixture model CNNs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5115- 5124, 2017.
Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas. Pointnet: Deep learning on point sets for classification and segmentation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 652-660, 2017a.
Charles Ruizhongtai Qi, Li Yi, Hao Su, and Leonidas J Guibas. Pointnet++: Deep hierarchical feature learning on point sets in a metric space. In Advances in Neural Information Processing Systems, pp. 5099-5108, 2017b.
Yi-Ling Qiao, Lin Gao, Yu-Kun Lai, and Shihong Xia. Learning bidirectional LSTM networks for synthesizing 3d mesh animation sequences. arXiv preprint arXiv: 2018.
Shikhar Sharma, Ryan Kiros, and Ruslan Salakhutdinov. Action recognition using visual attention. arXiv preprint arXiv: 141.0115,,.0411,, 2015.
Nitish Srivastava, Elman Mansimov, and Ruslan Salakhudinov. Unsupervised learning of video representations using LSTMs. In International Conference on Machine Learning, pp. 843-852, 2015.
Hang Su. Subhransu Maji, Evangelos Kalogerakis, and Erik Leaned-Miller. Multi-view convo- lutional neural networks for 3d shape recognition. Proceedings of the IEFE international conference on computer vision, pp. 945-953, 2015.
Qingyang Tan, Lin Gao, Yu-Kun Lai, and Shihong Xia. Variational autoencoders for deforming 3d mesh models. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recogni- fion, pp. 5841-5850, 2018a.
Qingyang Tan, Lin Gao, Yu-Kun Lai, Jie Yang, and Shihong Xia. Mesh-based autoencoders for localized deformation component analysis. In Thirty-Second AAAI Conference on Artificial Intel- ligence, 2018b.
LJP Van Der Maaten and G E Hinton. Visualizing high-dimennsioall data using t-sne, Journal of Machine Learning Research, 9:2579-2605, 2008. doi: 10. 10.10077/10449--01-008111
Peng-Shuad Wang, Yang Liu, Yu-Xiao Guo, Chun-Yu Sun, and Xin Tong. O-enn: Oetree-based convolutional neural networks for 3d shape analysis. ACM Transactions on Graphics (TOG), 36 (4):72, 2017.
SHI Xingjian, Zhourong Chen, Hao Wang, Yan Yeung. Wii-Kin Wong, and Wang-chun Woo. Convolutional LSTM network: A machine learning approach for precipitation nowcasting. In Advances in neural information processing systems, pp. 802-810 810, 2015.
10

---

Under review as a conference paper at ICLR 2020
A
APPENDIX
time step
time step 7
time step
time step
time step 10
20
time step 6
time step 7
time step 8
time step
time step 10
Figure 7: Localization and histogram the error accumulation AE for the prediction per time step. Top rows show the prediction error for the first bifurcation branch. Bottom emror for the second bifurcation branch. The error shown on a representative final time step of each branch. Histogram color coding follows the color convention in Figure [ right.
show the prediction


Reviews:
[{"review_id": "rylPJuDatH", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary.\nLong short-term memory (LSTM) networks are trained to learn the underlying dynamics of 3D structures -- An encoder LSTM encodes a sequence of 3D mesh representations and two decoder LSTMs reconstruct the input itself and predict the future structural geometry. Such structural deformation sequences are collected from a simulation and used to train and evaluate the model. \n\nStrengths.\n1. Deep neural network architecture is applied to learn a 3D mesh structure deformation. Its approach looks reasonable to me -- (i) a high-level (user-specified) feature is extracted to represent 3D structural mesh, and (ii) LSTMs are trained in this feature space. \n\n2. The paper shows that LSTM can learn the underlying dynamics of 3D structural deformation (but since it does not provide any comparison with other work, I cannot determine the prediction error is within a reasonable bound for this task).\n\nWeaknesses.\n1. Weak technical and theoretical novelty.\nAn existing LSTM-based seq-to-seq style architecture is simply chosen and applied to the 3D mesh reconstruction and prediction task. LSTM networks have been successfully applied to various tasks, thus it would be less impactful even if a paper shows LSTM works well in the specific task. \n\n2. Missing comparison with existing work.\n(As far as I know) the task of predicting 3D structure deformation has long been studied and various simulation and analysis tools have widely used in academia and industries. The paper, however, does not thoroughly compare or cite any prior work in this area. As a reader from a different area, I would like to see more thorough analysis to support (i) what are the main bottlenecks of the conventional approaches, (ii) how the proposed data-driven approach can help to address these issues, and (iii) how the proposed approach can further extend.\n\nFor example, the paper mentions (as the main reason to use the deep neural network) that “Since one simulation run takes a couple of hours on a compute cluster, running a large number of simulation is not feasible” (in the 1st paragraph of Introduction). Can the authors provide an analysis of flops (floating-point operations per second)?\n\n3. Simulated data for training. \n205 samples are collected from a simulation to train and validate the proposed model and half an hour has taken to train this model. Judging from the common practices, does this imply the underlying dynamics function (of a simulation) is trivial to learn? \n\nAlso, only two parts (i.e. the left and right structural beam) of the whole 3D structure are analyzed. This implies that relations between any other parts are not considered and only a sole part is individually analyzed.\n\nMinor concerns.\n- Typo (In Figure 1 caption, ‘undo’).\n- “Due to the encoder having 1000 layers”?\n- Labels are needed in Figures.", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "SJxVoXsJqr", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper presents an LSTM model to predict the evolution for 3D models for crash tests. The core model is a video prediction model (Srivastava et al. 2015). Instead of using the original 3D geometry, the authors propose to predict a feature representation based on prior work (Brezis & Gomez-Castro 2017, Bohn et al. 2013, Teran & Garcke 2019).\n\nStrength:\n + Interesting problem\n + Good writing\nAreas of improvement:\n - Technical contribution\n - Experiments\n\nThe paper is quite easy to read, although it would help to clearly separate prior work from the actual technical contributions. The problem of simulating 3D meshes instead of raw pixels is interesting.\n\nMy main issue with the paper was that I had a hard time seeing a contribution. It is unclear what is technically new, or if everything was just copied from prior work. It was also unclear which parts of the technical approach mattered? Is the LSTM the right solution, or are there other simple baselines to compare to?\n\nFinally, the experiments are not very intuitive. It is unclear what impact the presented technique has on the down-stream tasks of crash testing. Does it improve the current simulation of crash testing? Is the prediction error acceptable to replace certain safety tests?", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HyeC3PS8cr", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes to use autoencoder LSTM to learn car 3D crash simulation in an unsupervised manner.\n\nBasically, it proposes to apply autoencoder LSTM, which is an existing and well-studied technique, to a very special task of car crash simulation. Neither a new method nor an in-depth investigation of the task itself is carried out. It is hard to identify contributions that are significant enough to support the publication in top conferences such as ICLR. Considering the fact that autoencoder LSTM is wildly employed in solving various unsupervised learning tasks, readers can hardly acquire knowledge in terms of novelty.", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: 
The paper proposes to train LSTMs to encode car crashes (a temporal sequence of 3D mesh representations).  Decoder LSTMs can then be used to 1) reconstruct the input or 2) predict the future sequence of structural geometry.  The authors propose to use a spectral feature representation based on prior work as input into the encoding LSTM.  The main contribution of the paper (based on the author response) is the introduction of this spectral feature representation to the ML community.  The authors used single 3D truck model to generate 205 simulations, of which 105 was used for training, and 100 for testing.  The authors presented reconstruction errors and TSNE visualization of the LSTM's reconstruction weights.

Discussion Summary:
The paper got three weak rejects.  The response provided by the authors failed to convince any of the reviewers to adjust their scores.  The authors did not provide a revision based on the reviewer comments.

Overall, the reviewers found the problem statement to be interesting.  However, they had concerns about the following:
1. It's unclear what is the main technical contribution of the work. 
Several of the reviewers pointed out the lack of technical novelty.  From the writing, it's unclear if the proposed spectral feature representation is taken directly from prior work or there was some additional innovation in this submission.  Based on the author response, it seems the proposed feature representation is taken directly from prior work as the authors themselves acknowledge that the submission is taking two known ideas and combining them.  This can be made more explicit in the paper itself.  

2. Lack of comparison with existing work and experimental analysis
There is no comparison against existing work on predicting 3D structure deformation over time. While the proposed representation is interesting, the is no comparison with other methods or other alternative representations.  Without any comparisons it is difficult to judge how the reconstruction error corresponding to actual reconstruction quality.  How much error is acceptable?  The submissions also fails to elucidate when the proposed representation should be used.  Is it better than alternative representations (use 3D mesh directly? use point clouds? use alternate basis functions?) 

3. What is being learned by the model?  
R3 pointed out that the authors mention that the model is trained in just half an hour and questioned whether the dynamics function is trivial to learn and that the only two parts of the 3D structure is analyzed.  The authors responded that the "coarse" dynamic is easier to learn than the "fine" scale dynamics.  Is what is learned by the model sufficient?  How well would a model that just modeled the car as a rigid object and predicted the position do?  The lack of comparison against baselines and alternative methods/representations makes it difficult to judge usefulness of the representation/approach that is presented.

4. The paper also has minor typos. 
Page 5: "treat the for beams" --> "treat the four beams"
Page 7: "marrked" --> "marked"

Overall the paper addresses a interesting problem domain, and introduces a interesting representation to the ML community, but fails to do a proper experimental analysis showing how the representation compares to alternatives.  Since the paper does not claim the novelty of the representation as its contribution, it is essential that it performs a thorough investigation of the task and perform empirical studies comparing the proposed representation/method against baselines and alternatives.
Decision: Reject