Pixel Chem: A Representation for Predicting Material Properties with Neural Network

1 INTRODUCTION . The most common way of representing material structures is using the coordinates of atoms together with the periodic unit cell vectors and categorized this information as unstructured data ( i.e . cif files ) ( Lam Pham et al. , 2017 ) . However , those unstructured data can not put into neural network directly and need to be reformatted as matrices for the advanced GPU based neural network computational architecture , which is extremely robust in pixel processing. ( ? ) As Scholkopf & Smola ( 2002 ) indicated , when using Kernel-based machine learning models , in order to reach a fast and accurate prediction of first principle properties , a single Hilbert space of the atomistic system is required , in which regression is carried out . In all above situation , an ideal representation needs to possess several essential characters like invariant , unique , continuous , general and efficient etc . ( Rupp , 2015 ; Bartok et al. , 2013 ; von Lilienfeld et al. , 2015 ) : In addition , apart from the R-space information , other K-space information might be more important for the overall property of material , especially when the research target relates to the band gap or excitation . Various kinds of representation have been developed and reported . The Coulomb matrix representation ( CM , Rupp ( 2015 ) ; Faber et al . ( 2015 ) ; Rupp et al . ( 2012 ) ) coded nuclear charge and atom displacement inside to predict atomization energies and formation energies . Based on it , bag of bond ( BOB ) ( Hansen et al. , 2015 ) representation constructed an effective pairwise interatomic potential to strip down pair-wise variant of CM , which improved the predictive model remarkably . Many-body tensor representation ( MBTR , Huo & Rupp ( 2017 ) ) is derived from 2 above and reached a significant drop in error rate . Symmetry functions ( Behler , 2011 ; Eshet et al. , 2010 ; 2012 ) is another method which focuses on the representation of local chemical environment of atoms and maps this representation to atomic energy . Recently , based on Simplified molecular-input line-entry system ( SMILES , Weininger ( 1988 ) ) representation , a new 3N-MCTS module based on Mote-Carlo tree search is proposed by Segler et al . ( 2018 ) . This module keeps only the structure-connection information and trained large dataset to provide supplementary instructions for retrosynthetic analysis . In addition , representations like bonding-angular machine learning ( Hansen et al. , 2015 ) and others ( Huang & von Lilienfeld , 2016 ; Schutt et al. , 2014 ) have also been reported and tested . However , most of these representations only satisfy those requirements partially , in which unique and continuous principles are often violated . The most severe shortcoming for both CM and BoB is that they focus on predicting the extentive quantity , in R space like structural energy but left other intensive quantities in K space like band gap . Though some reported researches also included other properties ( Huang & von Lilienfeld , 2016 ) , their error rates are way larger comparing with energy . Overall , the R-space and K-space are still separated for this kind of representation . The possible improvements could be made by endorsing more R/K space information to the targeted representations . Recall that CM only coded atomic number together with distance , BoB added roughly designed bonds energy in compliment , both failed to represent most of the properties in K-space . Therefore , defining a new representation with enough physical identity from R/K space is a vital task for material informatics to ensure the uniqueness and continuity . 2 DEFINITION . 2.1 PIXEL CHEM REPRESENTATION FOR FINITE STRUCTURES . Here in this work , we proposed representations with projected both R and K space chemical information into the pixel space ( Pixel Chem ) . We start from the property of different atoms , mainly binding strength and charge transfer ability . To describe these , we first introduce two new matrices , C and B . C , which mainly reflects the charge transfer ability , is based on electron affinity energy and ionization energy of atoms . Inspired by Mullikan Electronegativity ( Mulliken , 1934 ) , it is defined as Ci , j = ( IEi , : +EA : ,j ) /2 . As for binding strength , Bi , j ( UBond ) is imported , which represents the typical bond energy formed by two atoms . Note that though Bi , j = Bj , i , Ci , j 6= Cj , i , which means this matrix is not symmetric between diagonal . Finally , we proposed a displacement matrix Di , j = Dj , i = exp ( −|ri − rj |/a ) to represent the relative position of different atoms , where |ri−rj | represent the Euclidean distance between two atoms and a is the interaction strength coefficient trained by machine learning methods . To mix all the things up , we proposed a charge & energy matrix Ei , j which can be produced by multiplying Di , j with CTi , j and B T i , j as follows : E = D ×CT ×BT ( 1 ) This operation ensures that for every element E′ in Ei , j , there will be a product of the D′ in Di , j and B′ in BTi , j corresponding to the same atom . Therefore , for all the atoms , we have : E′i , j = D ′ i , j ·C ′i , j ·B′i , j ( 2 ) Atomic orbits information , including arrangement , position and number , is another important factor that highly influence the property of a structure . In order to represent those information , we first construct a 1 × 10 sized orbital charge vector oi = [ N∑ core , Ns , Npx , Npy . . . Ndyz , Ndxz ] ( core stands for all the core electrons ) is created to represent ( mainly ) the electron configuration of every atom in valance electron orbitals . The binding energy BE of each orbital also needs to be considered to distinguish those orbitals in different chemical environments . We therefore product the exp ( −BorbitE ) of every orbital in oi to get an energy encoded orbital matrix oBEi , where : oBEi = [ N∑ core · exp ( −B∑ coreE ) , Ns · exp ( −BsE ) , · · ·Ndyz · exp ( −Bdx2−y2E ) ] ( 3 ) To better represent the chemical environment of each atom , especially the different condition of each orbital , we hereby introduce two closely related methods , tight-binding method and linear combination of atomic orbitals ( LCAO , Slater & Koster ( 1954 ) ) method , to our representation model . With the existence of atomic Hamiltonian , this matrix can precisely describe the interaction between two atoms orbital by orbital . In detail , a 10 × 10 sized angular interaction matrix Ai , j , representing from all core electrons to dz orbital electrons , is implemented for all atoms in one specific structure , which formats as : Int ( i [ ∑ core ] , j [ ∑ core ] ) Int ( i [ ∑ core ] , b [ s ] ) · · · Int ( i [ ∑ core ] , j [ dx2−y2 ] ) Int ( i [ s ] , j [ ∑ core ] ) Int ( i [ s ] , j [ s ] ) · · · Int ( i [ s ] , j [ dx2−y2 ] ) ... ... . . . ... Int ( i [ dx2−y2 ] , j [ ∑ core ] ) Int ( i [ dx2−y2 ] , j [ s ] ) · · · Int ( i [ dx2−y2 ] , j [ dx2−y2 ] ) 10×10 ( 4 ) which i [ m ] represent the m orbital of i atom . Recalled 3 , every element inside the matrix of atoms , shown as Int ( A [ orbita ] , B [ orbitb ] ) , represent the interaction strength between two orbitals which belong to two atoms respectively , it is yielded out as follows : Int ( i [ m ] , j [ n ] ) = Ni [ m ] · exp ( −B i [ m ] E ) ·Nj [ n ] · exp ( −B j [ n ] E ) · Coeff ( 5 ) Which can be summarized as : Int ( i [ m ] , j [ n ] ) = oBEi [ m ] · oBEj [ n ] · Coeff ( 6 ) The coefficient inside this formula is defined by the position of two atoms , the direction of two target orbits and the energy of them . To be specific , the calculation was primarily based on the Slater-Koster Approximation ( Hehre et al. , 1972 ) , which was designed to solve the overlapping area between two selected orbits . Some orbits , such as px , py and pz orbits for the same atom , are perpendicular to each other , which will result in 0 at the corresponding element . The calculation detail could be found in supplementary notes . In all , this angular interaction matrix expanded the previous matrix for 10 times , which encodes more physical meanings , especially the Hamiltonian of each atom , which could benefit the learning efficiency and accuracy of our training model . To combine all the angular matrices Ai , j with the previous E. We product every element Ei , j with its corresponding Ai , j to get new element Mi , j . Note that new element Mi , j is also a 10 × 10 matrix . Therefore , M will be the final Pixel Chem representation , denoted as : Mi , j = Ei , j ·Ai , j ( 7 ) Which can also be written as : Mi , j = Di , j ·Ci , j ·Bi , j ·Ai , j ( 8 ) Moreover , when implementing Pixel Chem into organic structures , this representation could be simplified because most of the elements in organic structures are from the first 3 periods which means they do not have d orbitals . This simplification results in a smaller matrix size and could handle the representation of organic molecules in a faster way . When considering organic structures , hybridization , especially for carbon and nitrogen , the type of chemical bonds need to be considered . We hereby implement an adjoint classifier to estimate the chemical state for each carbon , which will return the bond type and hybridization state of carbon atoms . Take sp3 as an example , instead of which proposed above , the Ai , j for carbon atom will be transferred to : Int ( a [ ∑ core ] , b [ ∑ core ] ) Int ( i [ ∑ core ] , j [ s ] ) · · · Int ( i [ ∑ core ] , j [ pz ] ) Int ( i [ s ] , j [ ∑ core ] ) Int ( i [ s ] , j [ s ] ) · · · Int ( i [ s ] , j [ pz ] ) ... ... . . . ... Int ( i [ pz ] , j [ ∑ core ] ) Int ( i [ pz ] , j [ s ] ) · · · Int ( i [ pz ] , j [ pz ] ) 5×5 ( 9 ) Hereby the oBEi = [ N∑ core · exp ( −B∑ coreE ) , Ns · exp ( −BsE ) , · · · , Npz · exp ( −BpzE ) ] . as 4 sp3 orbitals present the same property . Bsp3E will also take the place of their initial values . Figure 1 shows different orbital matrix for different hybridization state . To illustrate this representation in a straightforward way , we visualized the pixel chem using matplotlib in Figure 2 . Each colored square represents a numerical value . Black color stands for absolute 0 , which also means there is no electron in the corresponding orbital . Some of the offdiagonal and diagonal parts are carried out with the explanation , which are used to represent atoms properties ( for diagonal parts ) or interaction strength between atoms ( for off-diagonal parts ) .

Summary: This paper proposes a neural network architecture PCnet for the prediction of intensive and extensive chemical properties of molecules and materials. The authors claim that the use of prior chemical knowledge such as Mulliken electronegativity, bond strength and orbital information improves prediction accuracy. While the idea of incorporating chemical domain knowledge in the interactions of an atomistic neural network is interesting in principle, this paper has severe issues ranging from presentation over the proposed approach to the results.