Reduced Order Modeling using TensorFlow | by Sunil | Towards Data Science

While in grad school, I worked on an unsupervised machine learning (ML) problem with computational fluid dynamics (CFD) data (link to the paper and the journal article). The singular value decomposition (SVD) based learning algorithm was written in C++ and ran on the CPU. I was recently recalling this work and decided to re-implement it using TensorFlow. The goals of this post are to write about my experiences with using TensorFlow for :
training an SVD based unsupervised learning ML model using TensorFlowdeploy the trained model with TensorFlow serving.
training an SVD based unsupervised learning ML model using TensorFlow
deploy the trained model with TensorFlow serving.
The concept behind CFD is the numerical solution of Navier-Stokes equations on a discretized grid. The N-S equations are coupled partial differential equations that require boundary and initial conditions to be solved numerically. There are several solution methods that have been developed over the last few decades with researchers and practitioners taking advantage of advances in hardware and software to compute accurate solutions faster.
The outcome of numerically solving these equations are datasets that depict the flowfield, characterized by the x, y and z components of velocity i.e. (u,v,w) and the pressure (things get more complex when there are other physical phenomena involved). The time-to-solution is usually of the order of several minutes to days, depending on the complexity of the problem. Hence the need for reduced order modeling, where one can reduce the time to solution using a model developed from data as opposed to one developed from first-principles using conservation laws. A data-driven model usually has lower accuracy than physics based models derived from first-principles.
Proper orthogonal decomposition (POD), also called principal components analysis (PCA) by many, is a matrix decomposition technique that projects high-dimensional data into a low-dimensional subspace and in the process separating out linearly correlated data. The low-dimensional subspace is an optimal subspace and is computed from the solution of the eigenvalue problem:
The data from the CFD computations have been arranged into a matrix as shown below. The matrix has been mean-centered by taking the row-wise mean of each component and subtracting it from each matrix element. There are m datasets each with n variables (velocity vectors at each discretized grid point, so actually 3n variables).
If the multi-dimensional data is represented by the matrix A, the SVD of the matrix A is given by
The U and V matrices are the left and right eigenvectors U and V, satisfy the relationships shown.
I have put some asserts in the python code to ensure that the left and right eigenvectors satisfy the orthogonality properties above.
Once the SVD is computed, the coefficients of the POD expansion are computed by projecting the left eigen-vectors onto the mean-centered data i.e. matrix C is:
The coefficient matrix of dimensions (n X m) can also be expressed as a (1 X m) matrix of n dimensional vectors as:
Computations in TensorFlow
All the math now needs to be converted into a computational graph using TensorFlow. The data from the CFD runs are arranged into a tensor (matrix really).
Mean Centering the data: The data is first mean centered by computing the mean of each row and subtracting this mean from the original dataset. These computations are performed in TF as shown below.
#compute the mean tensor...actually a vectormean_tensor = tf.reduce_mean(data_tensor, axis=1,keep_dims=True)#subtract the meanmean_centered_data = tf.subtract(data,mean_data_tensor)
The “mean flow” can be plotted like shown below.
SVD Computation: Computing the SVD is straightforward in TF.
svd_out = tf.svd(mean_centered_data, compute_uv=True, full_matrices=False, name=”svd”)s, u, v = sess.run(svd_out)
Computing the “energy” : The energy captured is a function of the singular values from the SVD. e is the total energy and the energy captured by each mode is computed as a fraction of the total energy. Using TF’s reduce_sum function these steps are simple.
e = sess.run(tf.reduce_sum(s))s_energy = sess.run(tf.div(s,e)*100)
Computing the coefficients : The coefficients are computed by projecting the left eigenvectors onto the original data.
coeffs = tf.matmul(tf.transpose(u), data)
Pickle it : We now have everything needed to make ROM evaluations. Before doing that all the data is used to create a RomObject which is pickled into a file that can be loaded for making evaluations.
rom_object = RomObject.romobject(u, s_energy, coeffs, mean_array)with open(‘rom-object.pkl’, ‘wb’) as output:       pickle.dump(rom_object, output, pickle.HIGHEST_PROTOCOL)
Evaluating the ROM : The first step in making ROM evaluations is to load the pickled objects. To evaluate the POD based ROM (i.e. make a prediction) for an input that is not in the training data, the coefficients are interpolated based on the input vector. In the case of multi-dimensional input parameter space, the appropriate coefficient vectors to interpolate can be found using a similarity criterion. A simple technique could be the cosine similarity index. The interpolated coefficient vector is multiplied by the dominant left-eigenvectors to yield the ROM (ROM Evaluator.ipynb).
with tf.Session() as sess:    sess.run(init_op)    modal_result = sess.run(tf.transpose(result_op))    modal_result.flatten()    mean_tensor.eval()    result_op = tf.add( modal_result, mean_tensor)    result = sess.run(result_op)
Some Results
Some sample results from the above work are shown in this blog. If you are interested in reading more about this particular problem, please refer to the paper.
Comparing the velocity magnitudes from the CFD and the ROM:
Comparing the velocity magnitudes from the CFD and the ROM:
2. Plotting some of the dominant modes:
I won’t go into the detailed analysis of this because it has already been addressed in the paper. Moreover, the goal of this blog post is to highlight the usage of TensorFlow in building reduced order models. The code is published (needs some cleanup). I won’t be making the data available. But it should be easy enough to use your own datasets with this baseline code.
A Jupyter notebook is published in this bitbucket repository. If you have any suggestions for improvements or other comments please do share them.
TensorFlow (TF) provides a low level interface to perform linear algebraic computations. It is built as a python layer over cuDNN and GPU-based BLAS libraries. It really speaks to the power of the library which can be used for solving just about any problem that involves linear algebra or tensor math. It is thus not restricted to ML or DL problems. The key here is being able to understand the relationships between the math and the TF API and construct a data-flow graph that can be executed on a GPU (or CPU).
Also, the fact that TensorFlow can efficiently utilize the GPU for general-purpose computations opens up several opportunities for applied scientific computing researchers and practitioners to leverage TF to speed up several types of computations. Proper orthogonal decomposition based reduced order modeling is only one such example.
With the recent announcements about TPUs from Google, there will be more opportunities in the future to adapt scientific codes to “exotic” hardware.
In the interest of keep this post concise, I will be publishing this in Part 2.