# Equivariant Graph Neural Network for Crystalline Materials

Astrid Klipfel, Zied Bouraoui, Yaël Frégier, Adlane Sayede

► To cite this version:

Astrid Klipfel, Zied Bouraoui, Yaël Frégier, Adlane Sayede. Equivariant Graph Neural Network for Crystalline Materials. 1st International Workshop on Spatio-Temporal Reasoning and Learning, Jul 2022, JeJu, South Korea. http://dx.doi.org/10.4097/ha-04037916

HAL Id: hal-04037916

https://univ-artois.hal.science/hal-04037916

Submitted on 1 Dec 2023

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_1.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Equivariant graph neural network for crystalline materials

Astrid Klipfel¹,²,³, Zied Bouraoui², Yaël Frégier¹ and Adlane Sayede³

¹Laboratoire de Mathématiques de Lens (LML)-UR 2462, Artois University, France

²Centre de Recherche en Informatique de Lens (CRIL)-CNRS UMR 8188, Artois University, France

³Unité de Catalyse et Chimie du Solide (UCCS)-CNRS UMR 8181, Artois University, France

## Abstract

Materials generation is an essential task in material science that aims to discover new materials. While most of the existing models have shown interesting results in simulation, they struggle to produce new original and stable materials. This paper discusses the salient properties required for material generation and studies the difficulties related to material pattern repetition, which impacts the stability of the generated structures.

## 1. Introduction

Crystalline materials are involved everywhere in our modern society. From metal alloy to semi-conductor, several technological objects contain crystalline materials. Discovering new materials remains a difficult task in material science. While existing algorithms can search for new structures in the materials space [1], searching for a new material with a given set of desirable properties is not a trivial task. The set of potential candidate materials is not countable by a computer, and the portion of stable materials (i.e. materials that can exist without self-destructing) is small. Moreover, estimating the properties of a single material with chemical simulation as Density Functional Theory (DFT) is computationally expensive. To this end, methods based on evolutionary algorithms (e.g. [2]) are introduced for generating new materials from existing datasets composed of stable materials. However, most of these approaches work by hybridization and mutation of existing materials. As a consequence, these methods are not able to find complex materials.

Discovering new materials is a challenging problem. But contrary to most generation problems, theoretical chemistry provides a powerful set of tools to analyse synthetic and real data. As a matter of fact, simulation techniques like hartree-fock or DFT are able to estimate the properties of a given structure by applying physics laws. Consequently, the stability of the materials can be estimated through simulation. These methods can also be used to perform the relaxation of crystalline materials. Relaxation is the process of minimizing the energy of a crystalline structure by deforming it. This process is very common in material science to study a given material.

<figure><img src="image_2.png" /><figcaption>**Figure 1:** The energy of a structure is given by its geometry and chemistry. Stable structures are the local minima of the energy (green dots). Relaxing a structure is the process of modifying the geometry of a structure to minimize its energy (an unstable structure represented as a red dot converges to the nearest stable minima).</figcaption></figure>

It permits the discovery of new materials since minimizing the total energy of the structure allows it to be more stable. Consequently, the local minima of the energy corresponding to stable materials and relaxation lead to producing stable crystalline structures from an initially unstable crystal around the stable structures in the materials space. This is illustrated in Figure 1.

Several methods were recently introduced for chemical simulation either by enhancing current algorithms with machine learning techniques (e.g. [3]) or by using end-to-end models (e.g. [4]). Most of these approaches focus on accelerating DFT by approaching relaxation as a supervised task. End-to-end models generally require expensive labels to be trained as interaction forces. However, these labels are generally not available and need to be produced through DFT calculation. Thus, the benefits of end-to-end models is limited because such models required data from atomistic simulation. Finally, even if the relaxation may lead to a crystalline structure, there is no guarantee on its stability. In fact, most of the random structures do not result in stable structures. Consequently, approaches based on simulation for new materials discovery are limited.

Another direction consists in directly generating stable crystalline structures using machine learning techniques.

STRL '22: First International Workshop on Spatio-Temporal Reasoning and Learning, July 24, 2022, Vienna, Austria

<figure><img src="image_3.png" /></figure>

klipfel@cril.fr (A. Klipfel); zied.bouraoui@cril.fr (Z. Bouraoui);

yael.fregier@univ-artois.fr (Y. Frégier);

adlane.sayede@univ-artois.fr (A. Sayede)

<figure><img src="image_4.png" /></figure>

© 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License
Attribution 4.0 International (CC BY 4.0)

<figure><img src="image_5.png" /></figure>

CEUR Workshop Proceedings (CEUR-WS.org)
In this case, the generative process can be performed with successive actions applied to the structure [5]. This process doesn't require strictly following physics laws as long as the stable points (i.e. local minima of the energy) learned by the machine learning models remain the same as the real stable points. This fact can be advantageous in some cases because the realistic relaxation of a random structure is not guaranteed to converge to a stable material. As a result, learning realistic stable points without realistic physics can help machine learning models produce stable structures.

Indeed, graph neural networks (GNN) have already been used for the representation and generation of organic molecules [6]. However, crystalline materials are known to be more difficult to generate since they generally have more complicated chemistry. Also, they contain repetitive patterns defined by the lattice of the crystals, which make them harder to process. In this paper, we present how graphs representation of materials are defined. We also discuss some important properties required for generative models.

## 2. Problem statement

**Crystalline systems** As molecules, crystalline systems can be defined as coloured point clouds. However, as crystals are periodic structures, additional information about how the point could be repeated in the space is required to represent it. The periodicity of the material can be then represented as a network where a group of points is repeated by discrete translation, which is equivalent to tiling space with a parallelepiped containing a cloud of atoms as illustrated in Figure 2.

As a result, a periodic system can be described as atomic positions $x_i \in [0, 1]^3$ with an associated feature space representing the chemical information of each atom $z_i \in \mathcal{F}$ and a lattice $\rho \in GL_3(\mathbb{R})$ representing the periodicity of the material. The infinite point cloud generated by this representation can be defined as

$$ \{(\rho \cdot (x_i + \tau), z_i) | \tau \in \mathbb{Z}^3, 1 \le i \le n\} \subseteq \mathbb{R}^3 \times \mathcal{F} \quad (1) $$

Where $\tau$ act like a $\mathbb{Z}^3$ vector that translate the point cloud.

**Material graph** There are multiple ways to define the graph of a material. Chemical bonds can be used to build the set of edges, but generally, edges are built from the atoms under a given threshold distance of from the k nearest neighbourhood [7]. The resulting graph is a multi-graph because the local environment of an atom can be on a translated point cloud near the border of the lattice as depicted in Figure 2. As a result, an edge can

have multiple edges between a pair of nodes and between a node with itself.

<figure><img src="image_6.png" /><figcaption>**Figure 2:** The periodic structure of a material is represented as a lattice (in dotted lines). The multi-graph associated with a material (blue arrow) can overlap on the adjacent repetition of the lattice and a pair of nodes can be connected multiple times.</figcaption></figure>

**Material deformation** To tackle the crystalline system generation problem, we should define action on the geometry of the material. This action can be seen as an action on the lattice of the material $\rho$ resulting in the updated lattice $\rho'$ and action on the atomic positions $x_i$ resulting in the updated atomic position $x'_i$.

$$ \left\{ \begin{array}{l} \rho' = g\rho \\ x'_i = [x_i + g_i] \end{array} \right. . \quad (2) $$

In this case, the goal is to predict the action $g \in GL_3(\mathbb{R})$ on the lattice and the actions $g_i \in \mathbb{R}^3$ on the atomic position. The atomic positions are brought back into the lattice of the crystal by truncation.

## 3. Method

**Equivariance** To obtain meaningful actions applied to a material, we should satisfy the equivariance property. Indeed, translations and rotations have no impact on the material's properties. Consequently, a machine learning model acting on the material should not be dependent on the orientation and position of the structure but only be dependent on its geometry as shown in Figure 3.

<figure><img src="image_7.png" /><figcaption>**Figure 3:** Equivariance between the action applied on the material $g$ and the actions of translation and rotation group $h \in \mathbb{R}^3 \cup SO(3)$. $M$ and $M'$ denoting the original material and the material after having applied the action $g$.</figcaption></figure>
**Actions on the lattice** When *g* acts on *rho* the lattice of the crystal, we would restrain *g* to the group of actions deforming the structure. However, any matrix in R³×³ can be decomposed as a sum of a symmetric and an anti-symmetric matrix. As we define the action of *g* in GL(3), the anti-symmetric part of the transformation can be seen as a rotation. But as we discussed earlier, applying rotations on a crystal doesn't act on a material as the rotated material is equivalent to the original material. This is illustrated in Figure 4. Consequently, it is better to restrain *g* to GL(3) \ SO(3). In other words, to restrain *g* to the subset of matrices of GL(3) that are symmetric.

<figure><img src="image_8.png" /><figcaption>**Figure 4:** The SO(2) group doesn't act on the materials but only rotates them without deformation.</figcaption></figure>

**Acting on the crystalline structure** To act on both the lattice and the atomic position, we can define actions on the edge of the graph. These actions can be then decomposed into a global action on the lattice of the crystal and local action on the atomic position. In order to define these actions, the contributions of all edges are aggregated to compute the action on the lattice. To act on the atomic positions, only the actions of the edges connected to the node are taken into account.

## 4. Conclusion

Crystalline materials are difficult to process because of the complexity and the variety of their chemistry, but also because of their repetitive structure. However, materials can be represented as graphs containing both chemical and geometrical information about the structures. Consequently, geometric machine learning techniques such as graph neural networks can be used in a wide variety of tasks including supervised and unsupervised learning. But in order to enhance the generalization capability of machine learning models, some properties such as the symmetry of the actions on the lattice or the equivariance can be beneficial.

## Acknowledgments

This work has been supported by HPC resources from GENCI-IDRIS (Grant 2021-[AD011013288]) and FEI INS2I 2022-EMILIE.

<figure><img src="image_9.png" /><figcaption>**Figure 5:** Actions defined on the edges of the graph can be decomposed into action on the atomic positions and action on the lattice of the crystalline material.</figcaption></figure>

## References

[1] C. J. Pickard, R. J. Needs, *Ab initio* random structure searching, Journal of Physics: Condensed Matter 23 (2011) 053201. URL: https://doi.org/10.1088/0953-8984/23/5/053201. doi:10.1088/0953-8984/23/5/053201.

[2] A. R. Oganov, A. O. Lyakhov, M. Valle, How evolutionary crystal structure prediction works—and why, Accounts of Chemical Research 44 (2011) 227–237. URL: https://doi.org/10.1021/ar1001318. doi:10.1021/ar1001318. arXiv:https://doi.org/10.1021/ar1001318, pMID: 21361336.

[3] Y. Yang, O. A. Jiménez-Negrón, J. R. Kitchin, Machine-learning accelerated geometry optimization in molecular simulation, The Journal of Chemical Physics 154 (2021). URL: https://par.nsf.gov/biblio/10250472. doi:10.1063/5.0049665.

[4] F. Ekström Kelvinius, R. Armiento, F. Lindsten, Graph-based machine learning beyond stable materials and relaxed crystal structures, Phys. Rev. Materials 6 (2022) 033801. URL: https://link.aps.org/doi/10.1103/PhysRevMaterials.6.033801. doi:10.1103/PhysRevMaterials.6.033801.

[5] T. Xie, X. Fu, O.-E. Ganea, R. Barzilay, T. S. Jaakkola, Crystal diffusion variational autoencoder for periodic material generation, in: International Conference on Learning Representations, 2022. URL: https://openreview.net/forum?id=03RLpj-tc.

[6] V. G. Satorras, E. Hoogeboom, M. Welling, E(n) equivalent graph neural networks, in: M. Meila, T. Zhang (Eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of *Proceedings of Machine Learning Research*, PMLR, 2021, pp. 9323–9332. URL: https://proceedings.mlr.press/v139/satorras21a.html.

[7] T. Xie, J. C. Grossman, Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties, Phys. Rev. Lett. 120 (2018) 145301. URL: https://link.aps.org/
doi/10.1103/PhysRevLett.120.145301. doi:10.1103/