The deep learning leveraged FE2algorithm for two-scale modeling of elastic solids eliminates the need to solve the RVE problem on-the- y by replacing the e ective input-output causality by a neural network. This potentially reduces the computational cost of the FE2algorithm signi cantly. In this work, we put forth the use of snapshot proper orthogonal decomposition to improve the accuracy of the machine learning leveraged algorithm. Instead of training one neural net, multiple neural nets are trained with the coecients of the basis of the snapshot matrix as the target. Keywords Deep learningFE2homogenizationElastic solidsProper orthogonal decomposition 1 Introduction The RVE concept [1{10] is commonly used in the manufacturing sector to avoid using computationally expensive simulation platforms necessary to capture mi- crostructural features [11,12]. In essence, the features are captured in the RVE and averaged out over the RVE before any discretization technique is employed at the macroscale with the averaged properties as parameters. The popular FE2 algorithm [13{15] is commonly employed in which each gauss point for the nite element calculations at the macroscale is associated with a RVE and the infor- mation exchange between the two scales occurs at each of those gauss points. The macroscale problem is discretized into nite elements and the gauss point level computations work in conjunction with RVE scale nite element solve corre- sponding to each gauss point. The information fed by the macroscale to the RVE is the deformation gradient and the information fed by the RVE to the macroscale is the homogenized stress measure. A 2D depiction of the algorithmic framework for elastic solids is given in Figure 1. The nite element solves for the macroscale and the mesoscale are performed sequentialy and iteratively, with one macroscale nite S. Dana University of Southern California, CA 90007 E-mail: sdana@usc.edu 2 Saumik Dana macroscale nite element gauss point RVE 1 2 3 4 B R T L N+ N deformation gradient first P Kstress Fig. 1 A 2D depiction of the FE2framework for elastic solids. element solve followed by multiple simultaneous RVE nite element solves. This is then followed by a convergence check and subsequent macroscale nite element solve. The concepts of deformation gradient and the particular stress measure are explained in Appendix A. The deformation gradient manifests itself as periodic boundary conditions on the RVE as explained in Appendix B. The macroscale rst P-K stress is obtained as P=1 V0NX i=1fi Xi (1) where f= (f1;f2;:::;fN)Tis the vector of forces on the Nboundary nodes and V0 is the area (or volume) of the RVE. The derivation of Eq. (1) is given in Appendix C. 1.1 A neural network without proper orthogonal decomposition The information exchange would need to occure multiple times at every incre- ment of macro load to satisfy convergence. Depending on the complexity of the microstructure, the RVE solve itself would entail a lot of nite elements to resolve all the features in the RVE. The cumbersome computational cost would make the algorithm infeasible for complex microstructures. In lieu of that, [4] designed a ma- chine learning leveraged FE2algorithm which eliminates the need for on-the- y nite element solves at the RVE scale. This is achieved by following the outline laid out in [16] to build a neural net input-output causality relationship for the RVE thereby rendering the RVE as a blackbox which takes the macroscale deformation gradient as input and spits out the macroscale rst P-K stress as output as shown in Figure 2. A basic understanding of neural networks in given in Appendix D. Title Suppressed Due to Excessive Length 3 input layer output layer hidden layer F P b b target bias bias PNN Fig. 2 A naive 1-3-1 neural network with macroscale deformation gradient as input and homogenized rst P-K stress as the target output. We know from Appendix A that the macroscale deformation gradient has 9 components based o the spatial gradients of the the displacement eld compo- nents. Each of these components would honor the constraints that the problem statement poses for a well-posed solution. A prior knowledge of the range of pos- sible values each of these components is used to construct a bunch of candidate deformation gradients that act as input to the neural network. For each of these Mdeformation gradients, the RVE problem is solved using a conventional nite element procedure with periodic boundary conditions, and the resulting boundary forces are noted as the target. Any choice for the constitutive relationship cab be used to model the RVE problem based on the physical understanding of the phe- nomenon being studied and the material properties. The homogenized rst P-K stress is obtained for each of these boundary force vectors using Eq. (1). This data is then used to build the input-output causality as follows PNN=O(F) (2) whereOis the map between the deformation gradient and the output of the neural network. 2 A neural network with proper orthogonal decomposition Equation (1) clearly shows that the macroscale P-K stress is obtained from the RVE scale nite element solve through the forces obtained on the boundary nodes of the discretization. The boundary forces are summed and e ectively averaged over the boundary to obtain the homogenized rst P-K stress. As a result, a neural network with the homogenized rst P-K stress as the target would only capture the volume average of the RVE scale nite element solution features. With that in mind, a neural net is to be designed with the boundary forces as targets as shown in Figure 3. The steps in the POD and deep learning enhanced FE2algorithm are as follows: 4 Saumik Dana input layer output layer hidden layer F b target bias b bias f1 f3N Fig. 3 A neural network with macroscale deformation gradient as input and forces at N boundary nodes as the target output. {The non-linear neo-Hookean model is taken to be the stress-strain relation for the nite element simulations at the RVE level =1 2 J(J21)I+ J(bI) whereandare Lame parameters and bis left Cauchy Green strain tensor. {A collection of Mcandidate macroscale deformation gradients is selected {The RVE boundary value problem is solved for each one of the Mdeformation gradients using nite element method with periodic boundary conditions {The snapshots of ftaken over a range of deformation gradients is recorded as ff(k);k= 1;;Mg Title Suppressed Due to Excessive Length 5 with each snapshot generated from the nite element solution of the RVE BVP with periodic boundary conditions corresponding to each of the Mdeformation gradients. {The deviation matrix is built as follows F=2 64f(1) 1f1f(M) 1f1 :: f(1) NfNf(M) NfN3 75 3NM where fj=1 MMP i=1f(i) j;j= 1;;N. {The SVD of Fis computed as follows F=UVT {The targets (1); (2)and so on till (M)are obtained as follows (k)= TFk whereFkis thekthcolumn of Fand = U1;UM where Uiis theithcolumn of U {Instead of one neural net with fas target, multiple neural nets are trained with (1); (2)and so on as target. 8 >>>>>< >>>>>: (1)NN=O1(F) (2)NN=O2(F) : : (M)NN=OM(F)(3) whereO1;O2and so on are the maps between the deformation gradient and the outputs of the neural network. {The proper orthogonal decomposition [17] states that any of the snapshots can be reconstructed by a linear combination of the basis f(k)=1 M(MX i=1f(i) 1;;MX i=1f(i) N)T+MX i=1 (k) iUi 1 M(MX i=1f(i) 1;;MX i=1f(i) N)T+ (k)(4) {The rstm<M dominant modes can be taken. The reason for increased speed of training is that the neural net training optimizer works on a \local" error de ned over each of the deformation gradients with the POD incorporated as opposed to a \global" error de ned over all the sampled deformation gradients combined in the absence of POD. 6 Saumik Dana 3 Algorithmic framework in a nutshell In the initialization stage, the input-output causality of the RVE problem as in Eqs. (3), (4) and (1) is established. Once the initalization phase is complete, the following steps are followed {An increment of macro load is applied {Macroscale BVP is solved with macroscale sti ness given by Eq. (12) {The macroscale deformation gradient is updated {Periodic boundary conditions are imposed on RVE in accordance with Eq. (5) {Homogenized rst P-K stress is obtained in accordance with Eqs. (3), (4) and (1) {The gauss point level homogenized rst P-K stress is used to compute internal forces at macroscale nite element nodes If these internal forces are in balance with the prescribed macro load, incremental convergence has been achieved and steps 1 6 are repeated. If that is not the case, steps 26 are repeated. The algorithmic owchart is given in Algorithm 1. Algorithm 1 Machine learning based FE2homogenization for elastic solids Use machine learning to establish the relationship (3) FM I .Initialize deformation gradient forE2Thdo .loop over macroscale nite elements forg2Gdo .loop over gauss points RVE$g . Assign a RVE to each gauss point Compute macroscopic tangent sti ness at each gauss point in accordance with (12) Assemble macroscopic tangent sti ness over gauss points Assemble macroscopic tangent sti ness over nite elements whiletTdo Apply increment of macro load while (Internal force-Macro load >TOL) do.at macroscale nite element nodes Solve macroscale problem for FM FM FM+FM .Update deformation gradient forE2Thdo .loop over macroscale nite elements forg2Gdo .loop over gauss points Prescribe periodic BCs in accordance with (5) Calculate rst P-K stress in accordance with (3), (4) and (1) Compute internal forces at nite element nodes Con ict of interest The authors declare that they have no con ict of interest. A The deformation gradient and rst P-K stress As shown in Figure 4, let XfX;Y;Zgis position vector of point in reference con guration, ufu;v;wgbe the macroscale deformation eld and x=X+uis the position vector the same point in the deformed con guration. Meanwhile, an elemental area dA0with unit normal Title Suppressed Due to Excessive Length 7 x y z X x u undeformed con guration deformed con guration dA0 n0 n df dA Fig. 4 Kinematics of deformation n0deforms to dAwith unit normal nunder the transformation. The macroscale deformation gradient Fis the macroscale spatial derivative of xin the reference con guration as follows F=x rXI+u r whereris w.r.t X. The deformation gradient has the following components Fijij+@ui @Xj written in matrix form as F=2 41 +@u @X@u @Y@u @Z@v @X1 +@v @Y@v @Z@w @X@w @Y1 +@w @Z3 5 An incremental force dfis de ned with respect to the Cauchy stress and the rst Piola- Kircho stress Pin the deformed and reference con gurations respectively as follows df=ndA=Pn0dA0 B Periodic boundary conditions on RVE The typical RVE for imposition of periodic boundary conditions is shown in Figure 5. After each macroscale BVP solve, the deformation gradient is updated and the new position vectors of the vertices of the RVE are obtained using x=FX (5) This alongwith the shape periodicity of the RVE enables the implementation of periodic bound- ary conditions on RVE. It is easy to see that the prescribed periodic boundary conditions are Dirichlet boundary conditions. C Computation of homogenized rst P-K stress at the macroscale The linear momentum balance for the macroscale BVP in the reference con guration is given by rP+b=0 (6) 8 Saumik Dana 1 2 3 4 B R T L N+ N Fig. 5 Typical 2D RVE with pertinent microstructural features. LandRare mirror images so areTandB. This helps in easy implementation of periodic boundary conditions on the RVE in accordance with [18]. B R T L N+ N Fig. 6 Typical 2D RVE with nite element mesh. Boundary nodes are marked with crosses. where bis the body force vector. Equation (6) is expressed in the indicial notation as Pik;k+bi= 0i;k= 1;2;3 where the notation ( );k@() @Xkis used to denote the spatial derivative in the reference con guration. Before we proceed, we assume that the body force is zero, and obtain the following using chain rule for di erentiation (PikXj);k=Pik;kXj+Pikjk= 0 biXj+Pij (7) We express the macroscale rst P-K stress in indicial notation as follows PMij=1 V0Z V0PijdV0=1 V0Z V0(PikXj);kdV0=1 V0Z 0Pikn0kXjd0 (8) where the third equality follows from (7) and the fourth equality follows from divergence theorem. We then write (8) in tensorial notation as PM=1 V0Z 0t0 Xd0 (9) D Neural network A neural network is composed of several connected layers of arti cial neurons and biases where the data is fed into the input layer and ows through some hidden layers. The output is Title Suppressed Due to Excessive Length 9 predicted in the output layer. The neurons from di erent layers are connected through weights w. In the data collection phase, the data ows in one way from the input layer to the target. At each neuron, an activation function is attached. The output of each neuron is computed by multiplying the outputs from the previous layer with the corresponding weights. For the neuronjin layerk, the data from the previous layer k1 is summed up and then altered by an activation function. The output of neuron jin layerkis computed as ok j=F(NX i=1wijok1 i+bk1 i) whereNis the number of neurons in the previous layer k1,wijis the weight connecting neuronsiandj,ok1 iis the output of neuron iin layerk1 andbk1 iis its bias. A common choice for the activation function is the sigmoid F=2 1 +e2x1 In the training phase, the weights of neural network will be initialized rstly, (see [19]), which is followed by the weights updating using a training algorithm such that the global error is minimized. The global error, also named as loss function or network performance, is de ned according to the di erence between the network prediction and the target data. To minimize the global error, the Levenberg-Marquardt algorithm (see [20]) is applied to update the weights. E Computation of homogenized tangent sti ness at macroscale Letufrepresent the displacement DOFs corresponding to the interior nodes and urepresent the displacement DOFs corresponding to the boundary nodes. The force displacement relation for the RVE scale problem is   *KRVE KppKpf KfpKff up u = f 0 where the matrix KRVEis dictated by the microstructure and is known apriori. We knock o DOFs corresponding to internal nodes to obtain [  :K KppKpf(Kff)1Kfp]fug=ffg (10) The incremental macroscopic rst PK stress is obtained as PM=1 V0NX i=1fi Xi (from (1)) =1 V0NX i=1NX j=1Kijuj Xi (from (10)) =1 V0NX i=1NX j=1KijFMXj Xi (u=FMX) (11) Comparing (11) with PM=CMFM, we get CMabcd=1 V0NX i=1NX j=1KijacXibXida;b;c;d = 1;2;3 (12) 10 Saumik Dana