MOFDiff: Coarse-grained Diffusion for Metal-Organic Framework Design
Authors: Xiang Fu
Tian Xie
Andrew Scott Rosen
Tommi S. Jaakkola
Jake Allen Smith
Venue: ICLR 2024 poster (2024)
Keywords: Materials design
diffusion model
metal-organic framework
carbon capture
generative model
AI for Science
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
TL;DR: We develop a generative model for metal organic frameworks using a coarse-grained diffusion approach to discover carbon capture materials and validate them through molecular simulations.

Abstract: Metal-organic frameworks (MOFs) are of immense interest in applications such as gas storage and carbon capture due to their exceptional porosity and tunable chemistry. Their modular nature has enabled the use of template-based methods to generate hypothetical MOFs by combining molecular building blocks in accordance with known network topologies. However, the ability of these methods to identify top-performing MOFs is often hindered by the limited diversity of the resulting chemical space. In this work, we propose MOFDiff: a coarse-grained (CG) diffusion model that generates CG MOF structures through a denoising diffusion process over the coordinates and identities of the building blocks. The all-atom MOF structure is then determined through a novel assembly algorithm. As the diffusion model generates 3D MOF structures by predicting scores in E(3), we employ equivariant graph neural networks that respect the permutational and roto-translational symmetries. We comprehensively evaluate our model's capability to generate valid and novel MOF structures and its effectiveness in designing outstanding MOF materials for carbon capture applications with molecular simulations.

Published as a conference paper at ICLR 2024
MOFDIFF: COARSE-GRAINED DIFFUSION FOR METAL- ORGANIC FRAMEWORK DESIGN
Xiang Ful f Tian Xie?
Andrew S. Rosen
Tommi Jaakkola'
Jake Smith
MIT CSAIL
Microsoft Research Al4Science
3 Department of Materials Science and Engineering, UC Berkeley
Materials Science Division, Lawrence Berkeley National Laboratory
ABSTRACT
Metal-organic frameworks (MOFs) are of immense interest in applications such as gas storage and carbon capture due to their exceptional porosity and tunable chemistry. Their modular nature has enabled the of template-based methods to generate hypothetical MOFs by combining molecular building blocks in accor- dance with known network topologies. However, the ability of these methods identify top-performing MOFs is often hindered by the limited diversity of the resulting chemical space. In this work, we propose MOFDiff: a coarse-grained (CG) diffusion model that generates CG MOF structures through a denoising dif- fusion process over the coordinates and identities of the building blocks. The all-atom MOF structure is then determined through a novel assembly algorithm. Equivariant graph neural networks are used for the diffusion model to respect the permutational and roto-translational symmetries. We comprehensively evaluate our model's capability to generate valid and novel MOF structures and its effective- ness in designing outstanding MOF materials for carbon capture applications with molecular simulations
INTRODUCTION
Metal-organic frameworks (MOFs), characterized by their permanent porosity and highly tun- able structures, are emerging as a versatile class of materials with applications spanning gas stor- age (Gomez-Gualdre) al., 2014: Li al., 2018), gas separations (Lin et al., 2020; Qian et al., 2020), catalysis (Yang & Gates, 2019; Bavykina et al., 2020; Rosen et al., 2022), and drug delivery (Cao al., 2020; Lawson al., 2021). These frameworks are constructed from metal ions or clusters ("nodes"") coordinated to organic ligands ("linkers"), forming a vast and diverse family of crystal structures (Moghadam et al., 2017). Unlike traditional solid-state materials, MOFs offer unparalleled tunability, as their structure and function can be engineered by varying the choice of metal nodes and organic linkers. The surge in interest surrounding MOF' is evident in the increasing number of research studies dedicated to their synthesis, characterization, and computational design (Kalmutzki et al., 2018; Boyd et al., 2017a: Yusuf et al., 2022).
The modular nature of MOFs naturally lends itself to template-based representations and algorithmic assembly. These algorithms create hypothetical MOFs by connecting metal nodes and organic linkers (collectively, building blocks) along connectivity templates known as topologies (Boyd et al., 201 7a; Yaghi, 2020; Lee et al., 2021). Given combination of topology, metal nodes, and organic linkers, the MOF structure is obtained through heuristic algorithms that arrange the building blocks, aligning them with the vertices and edges designated by the chosen topology, followed by structural relaxation process based on classical force fields.
The template-based approach to MOF design has led to the use of high tigh-throughhut computational screening approaches (Boyd et al., 2019), variational autoencoders (Yao et al., 2021), genetic algo-
*Corresoondence to Xiang Fu (xiangfulmit edu) and Jake Smith fiakesmitt.mmierccceee I Work partially done during an internship at Microsoft Research AI4Science.
Code available https: : //gi thub. com/ /i crosoft/MorDiif

---

Published as a conference paper at ICLR 2024
Coarse graining
Denoising diffusion over the unit cell
Initiaize L.K
Enoode
Condition
Inverse design
[=]]
1=
0.1
0
Decode building block identities
Assamble building blocks
Force field
Figure 1: (a) MOFDiff encodes a coarse-grained (CG) representation of MOF structures and decodes CG MOF structures with denoising diffusion process. To generate CG MOF structure, the lattice parameters L and the number of building blocks K are predicted from the latent vector z to initialize random structure. A denoising diffusion process conditional z generates the building block identities and coordinates. Inverse design is enabled through gradient-based optimization over z in the latent space. (b) The all-atom MOF structure is recovered from the bearse-rrained representation through three steps: (1) the building block identities are decoded from the learned representation; (2) building block orientations are randomly initialized, then the assembly algorithm (Figure 4) is run to re-orient the building blocks; (3) the assembled structure goes through an energetic minimization process using the UFF force field. The relaxed structure is then used to compute structural and gas adsorption properties. Atom color code: Zn (purple), O (red), C (gray), N (blue), H (white).
rithms (Day & Wilmer, 2020; Zhou & Wu, 2022; Park et al., 2022), Bayesian optimization (Comlek et al., 2023), and reinforcement learning (Zhang et al., 2019; Park et al., 2023b) to discover new identify top-performing materials. However,
combinations of building blocks and topologies template-based methods enforce a set of pre-curated topology templates and building block identities. This inherently narrows the range of designs these hypothetical MOF construction methods can produce (Moosavi et al., 2020), possibly excluding materials suited for some applications. Therefore, we aim to derive a generative model based on 3D representations of MOFs without the need for pre-defined templates that often rely chemical intuition.
Diffusion models (Song & Ermon, 2019; Ho et al., 2020; Song et al., 2021) have made significant progress in generating molecular and inorganic crystal structures (Shi et al., 2021; ; Luo al., 2021; Xie et al., 2022; Xu et al., 2022; 2023; Hoogeboom et al., 2022; Jing et al., 2022; Corso et al., 2022: Ingraham al., 2022: Luo et al., 2022: Yim et al., 2023; Watson et al., 2023; Lee et al., 2023; Gruver et al., 2023; Park et al., 2023c; Jiao et al., 2023). Recent work (Park et al., 2023a) also explored using diffusion model to design linker molecules in specific MOFs. In terms of data characteristics, both inorganic crystals and MOFs are represented as atoms in a unit cell. However, a typical MOF unit cell contains hundreds of atoms (Figure 1), while the most challenging dataset studied in previous works (Xie et al., 2022; Lyngby & Thygesen, 2022) only focused on inorganic crystals with less than 20 atoms in the unit cell. Training diffusion model for complex MOF systems with atomic-scale resolution is not only technically challenging and computationally expensive but also suffers from extremely poor data efficiency. To name one challenge, without accounting for the internal structures of the metal clusters and the molecular linkers, directly applying diffusion models over the atomic representation of MOFs can very easily lead unphysical structures for the inorganic nodes and/or organic linkers.
To address the challenges above, we propose MOFDiff, a coarse-grained diffusion model for generat- ing 3D MOF structures that leverages the modular and hierarchical structure of MOFs (Figure I (a)). We derive a coarse -raennrrriddddd 3D representation of MOFs, a diffusion process over this CG MOF repre- sentation, and assembly algorithm for recovering the all-atom MOF structure. In experiments, we adapt the MOF dataset from Boyd et al. 2019 (B (BW-DB) that contains hypothetical MOF structures and computed property labels related separation of carbon dioxide (CO2) from flue gas. We train
2

---

Published as a conference paper at ICLR 2024
Figure 2: MOF decomposition with connections visualized. Connection points are light blue. (a) A MOF unit cell. (b) For visibility, we visualize the metal node and one other organic linker, one at time. (c) four building blocks this example MOF.
MOFDiff on BW-DB and use MOFDiff to generate and optimize MOF structures for carbon capture. In summary, the contributions of this work are:
We derive a coarse- coarse-grained representation for MOFs where we specify the identities and coordinates of structural building blocks. We propose to learn a contrastive embedding to represent the vast building block design space.
We formulate diffusion process for generating coarse-grained MOF 3D structures. We then design an assembling algorithm that, given the identities and coordinates of building blocks, re-orients the building blocks recover the atomic MOF structures. The generated atomic structures are further refined with force field relaxation (Figure 1 (b)).
We demonstrate that MOFDiff can generate valid and novel MOF structures. MOFDiff surpasses the scope of previous template-based methods, producing MOFs that extend beyond simple combinations of pre-specified building blocks.
We use MOFDiff to optimize MOF structures for carbon capture and evaluate the perfor- mance of the generated MOFs using molecular simulations. We show that MOFDiff can discover MOFs with exceptional CO2 adsorption properties with excellent efficiency.
REPRESENTATION OF 3D MOF STRUCTURES
Like any solid-state material, a MOF structure can be represented as the periodic arrangement of atoms in 3D space, defined by the infinite extension of 3-dimensional unit cell. A unit cell that includes N atoms is described by three components: (1) atom types A (a), aN) E A where A denotes the of all chemical elements; (2) atom coordinates X (æ.,..., xx) ) IR Ngx,,, and (3) periodic lattice L = (41, 12, 13) E 3333 The periodic lattice defines the periodic translation symmetry of the material. Given M (A, X L). the infinite periodic structure is represented as,
x/)aa dg x. x; + k14 kal2 + Kala, k1, k2, k3 Ø},
(I)
where k2, ki are any integers that translate the unit cell using L to tile the entire 3D space. A MOF generative model aims to generate 3-tuples M that correspond to valid?, novel. and functional MOFs. As noted in the introduction, prior research (Xie et al., 2022) employed a diffusion model on atomic types and coordinates to produce valid and novel inorganic crystal structures, specifically with fewer than 20 atoms in the unit cell. However, MOFs present distinct challenge: their unit cells typically comprise tens to hundreds of atoms, composed of diverse range of metal nodes and organic linkers. Directly applying the atomic diffusion model to MOFs poses formidable learning and computational challenges due to their increased size and complexity. This necessitates new approach that can leverage the hierarchical nature of MOFs.
Hierarchical representation of MOFs, A coarse geeeeerraiddddd 3D structural representation of a MOF can be derived from the coordinates and identities of the building blocks constituting the MOF. Such a representation is attractive, as the number of building blocks (denoted K) in a MOF is generally orders of magnitude smaller than the number of atoms (denoted N. K & N). We denote coarse- grained MOF structure with K building blocks as MC = (A X L). The three components:
Assesssng the validity of MOF 3D structures is hard in practice. We defer our protocol for validity determination the experiment section.
3

---

Published as a conference paper at ICLR 2024
(9) Extract building blocks
3D GNN embedding
(b)
Contrastive Learning
19
Percant
15
100
150
50
10
Number connection points
Figure (a) Learning a compact representation of building blocks for CG diffusion. Building blocks are extracted from MOF structures and embedded through GemNet-OC encoder. The representation is trained through a contrastive learning loss such that similar building blocks have similar embeddings. (b) The distribution of the number of atoms and the distribution of the number of connection points for the building blocks extracted from BW-DB. Atom color code: Cu (brown), Zn (purple), O (red), N (bluc), C (gray), H (white).
(1) -af all building blocks; (2) x = (①f ..... xx) E R Kx3 are the coordinates of the building blocks; (3) L are the lattice parameters. To obtain this coarse-grained representation, we need a systematic procedure to determine which atoms constitute which building blocks. In other words. we need an algorithm to assign the N atoms to K connected components, which correspond to K building blocks.
E K are the identities of the building blocks, where BB denotes the set of
Luckily, multiple methods have been developed for decomposing MOFs into building blocks based on network topology and MOF chemistry (Bucior et al., 2019; Nandy et al., 2022; Bonneau et al., 2018; Barthel et al., 2018; Li et al., 2014; O' Keeffe Yaghi, 2012). We employ the mwwaaloxtee... algorithm from the popular MOF identification method MOF id (Bucior et al., 2019). Figure 1 (a) demonstrates the coarse- coarse-graining process: the atoms of each building block are identified with MOF id and assigned compute the building
the same color in the visualization. From these segmented atom groups, block coordinates X and identities A C for all K building blocks to construct the coarse-grained representation. Each building block is extracted by removing single bonds that connect it other building blocks. Every atom that forms such bonds to another building block is then assigned a special pscudo atom, called connection point, at the midpoint of the original bonds that were removed. Figure 2 illustrates this process. We can now compute building block coordinates X C by computing the centroid of the connection points for each building block".
The building block identities A are, on the other hand, tricky to represent because there is huge space of possible building blocks for any non non-trivial dataset. Furthermore, many building blocks share an identical chemical composition, varying only by small geometric variations and translation/rotation. Example building blocks are visualized in Figure 3 (a). To illustrate the vast space of building blocks, extracted million building blocks from the training split of the BW-DB dataset (289k MOFs). To quantify the extent of geometric variation among building blocks with the same molccule/metal cluster, computed the ECFP4 fingerprints (Rogers & Hahn, 2010) for each building block using their molecular graphs and found 242k unique building block identities. This building block space is too large to be represented as a categorical variable in a generative model. A brief introduction the tools such as MOFie id is included in Appendix B.3.
Contrastive representation of building blocks. In order construct a compact representation of building blocks for diffusion-based modeling, we use a contrastive learning approach (Hadsell et al.,
We compute the coarse granind coordinates based on the connection points hecause the assembly algorithm introduced later relies on matching the connection points align the building blocks.
4

---

Published as a conference paper at ICLR 2024
Figure 4: The MOF assembly process. Connection points (light blue) are highlighted for visibility.
2006; Chen et al., 2020) to embed building blocks into low dimensional latent space. A building block is encoded as a vector b, using a GemNet-OC encoder (Gasteiger et al., 2021; 2022), an SE(3)-invariant graph neural network model. We then train the GNN building block encoder using a contrastive loss to map small geometric variations of the same building block to similar latent vectors in the embedding space. In other words, two building blocks are positive pair for contrastive learning if they have the same ECFP4 fingerprint. Figure (a) illustrates the contrastive learning process, while Figure (b) shows the distribution of the number of atoms and the distribution of the number of connection points for building blocks extracted from BW-DB. The contrastive loss is defined
ΣJEB eXp(SL3/T) iEB EEEBYB; exp(s13/T)
1
Σ log
Lc
B
where B is training batch, B are the other data points in B that have the same ECFP4 fingerprint as i, St, is the similarity between building block i and building block j, and is the temperature factor. We define 8, = p//p//C PEPp//(lp:ll (IIe:IIll,,11) which is the cosine similarity between projected embeddings P, and pj. The projected embedding obtained by projecting the building block embedding b, using a multi-layer perceptron (MLP) projection head: p: = MLP((;, (,). The projection layer is a standard practice contrastive learning frameworks for improved performance.
With trained building block encoder, we encode all building blocks extracted from a MOF to construct the building block identities in the coarse-graine gaaseee representation: A (b...... by E RK where d is the embedding dimension of the contrastive building block encoder (d 32 for BW-DB). The contrastive embedding allows accurate retrieval through finding the nearest neighbor in the embedding space.
3 MOF DESIGN WITH COARSE-GRAINER DIFFUSION
MOFDiff. Equipped with the CG MOF representation, we encode MOFs as latent vectors and decode MOF structures with conditional diffusion. The MOFDiff model is composed of four components (Figure (a)): (1) A periodic GemNet-OC encoder* that outputs latent vector
PGNNN((M (MC)
(2) MLP predictor that predicts the lattice parameters and the number of building blocks from the latent code L, K MLP L.K (z); (3) a periodic GemNet-OC denoiser that denoises random structures to CG MOF structures conditional on the latent code: AA $ xxc = PGNND (MC. z), where $ AC, $ ex the predicted scores for building block identities A C and coordinates X and MC is noisy CG structure at time t in the diffusion process; (4) MLP predictor that predicts properties e (such as CO2 working capacity) from z: é MLPp(zz..
The first three components used to generate MOF structures, while the property predictor MLPp can be used for property-driven inverse design. To sample a CG MOF structure from MOFDiff, we follow three steps: (1) randomly sample latent code parameters L and the number of building blocks K from z, use L and z to initialize a random coarsc-grained MOF structure MC = (
A I); (2) decode the lattice
(3) generate the yearse-grained MOF structure M = (AC, C L) through the denoising diffusion process conditional on z. Given the final building block embedding A k we decode the building block identities by finding the nearest neighbors in the building block embedding space of the training set. More details on the training and diffusion processes and the training schematics (Figure 10) are included in Appendix B.
*We refer interested readers to Xie & Grossman 2018; Chen et al. 2019; Xie et al. 2022 for details about handling periodicity in graph neural networks.
5

---

Published as a conference paper at ICLR 2024
Recover all-atom MOF structures. The orientations of building blocks are not specified by the CG MOF representation, but they can be determined by forming connections between the building blocks. We design an assembly algorithm that optimizes the building block orientations to match the connection points of adjacent building blocks such that the MOF becomes connected (visualized in Figure 4). This optimization algorithm places Gaussian densities at the position of each connection point and maximizes the overlap of these densities between compatible connection points. Two connection points are compatible if they come from two different building blocks: one from metal atom, and the other is from non con-metal atom (Figure 2). The radius of the Gaussian densities is gradually reduced in the optimization process: at the beginning, the radius high, so the optimization problem is smoother, and it is simpler to find approximate solution. At the end of optimization, the radius of the densities is small, so the algorithm can find accurate orientations for matching the connection points closely. This overlap-based loss function is di differentiable with regard to the building block orientation, and we optimize for the building block orientations using the L-BFGS optimizer (Byrd et al., 1995). Details regarding the assembly algorithm are included in Appendix B.2.
The assembly algorithm outputs an all-atom MOF structure that is fed to a structural relaxation procedure using the UFF foree field (Rappé et al., 1992). We modify a relaxation workflow from previous work (Nandy et al., 2023) implemented with LAMMPS (Thompson et al., 2022) and LAMMPS s Interface (Boyd al., 201 to refine both atomic positions and the lattice parameters using the conjugate gradient algorithm.
Full generation process. Six steps needed to generate MOF structure: (1) sample latent vector 2; (2) decode the lattice parameters L and the number of building blocks K from 2, use L and K to initialize a random gaars-graiieed MOF structure; (3) generate the coarse-grained MOF structure through the denoising diffusion process conditional on 2; (4) decode the building block identities by finding their nearest neighbors from the building block vocabulary: (5) use the assembly algorithm to re-orient building blocks such that compatible connection points' overlap is maximized; (6) relax the all-atom structure using the UFF force field to refine the lattice parameter and atomic coordinates. All steps are demonstrated in Figure 1.
4 EXPERIMENTS
Our experiments aim to evaluate two capabilities of MOFDiff:
1. Can MOFDiff generate valid and novel MOF structures?
2. Can MOFDiff design functional MOF structures optimized for carbon capture?
We train and evaluate our method on the BW-DB dataset, which contains 304k MOFs with less than 20 building blocks (as defined by the met meloo-oxo decomposition algorithm) from the 324k MOFs in Boyd et al. 2019. We limit the size of MOFs within the dataset under the hypothesis that MOFs with extremely large primitive cells may be difficult synthesize. The median lattice constant in the primitive cell of an experimentally realized MOF in the Computation-Reade Experiment (CoRE) MOF 2019 dataset is, for example, only 13.8 A (Chung et al., 2019). We use 289k MOFs (95%) for training and the rest for validation. We do not keep test split, as we evaluate our generative model on random sampling and inverse design capabilities. On average, each MOF contains 185 atoms (6.9 building blocks) in the unit cell; each building block contains 26.8 atoms average.
4.1 GENERATE VALID AND NOVEL MOF STRUCTURES
Determine the validity and novelty of MOF structures. Assessing MOF validity is generally challenging. We employ series of validity checks:
1. The number of metal connection points and the number of non-metal connection points should be equal. We call this criterion Matched Connection.
2. The MOF atomic structure should successfully converge in the force field relaxation process.
3. For the relaxed structure, we adopt MOFChecker (Jablonka, 2023) to check validity. MOF Chockee includes variety of criteria: the presence of metal and organic elements,

---

Published as a conference paper at ICLR 2024
MOFDIF Reference
15
20
Percent
15
10
10
2000
0.00
0.25
0.50
PLD (A]
Gravimetric surface area [m//g]
Density [g/cm²]
Void fraction
Figure 5: MOFDiff samples match the reference distribution for various structural properties.
porosity, no overlapping atoms, no non-physical atomic valences or coordination envi- ronments, atoms molecules disconnected from the primary MOF structure, and no excessively large atomic charges. We refer interested readers to Jablonka 2023 for details.
We say a MOF structure is valid all three criteria above are satis- fied. For novelty, we adopt the MOF identifier extracted by MOFid and say a MOF is novel if its MOFic id differs from any other MOFs in the training dataset. We also count the number of unique gener- ations by filtering out replicate samples using their MOF id. We are ultimately interested in the valid, novel, and unique (VNU) MOFs discovered.
100
80
Percert
MOFDiff generates valid and novel MOFs. prerequisite for functional MOF design is the capability generate novel and valid MOF structures. We randomly sample 10,000 latent vectors from (O, I), decode through MOFDiff, assemble, and apply force field relaxation to obtain the atomic structures. Figure shows the num- ber of MOFs satisfying the validity and novelty criteria: out of the 10,000 generations, 5,865 samples satisfy the matching connection criterion; 3012 samples satisfy the validity criteria, and 2998 MOFs are valid, novel, and unique. To evaluate the structural diversity of the MOFDiff samples, we investigate the distribution of four im- portant structural properties calculated with Zeo++ (Willems et al., 2012): the diameter of the smallest passage in the pore structure, or pore limiting diameter (PLD); the surface area per unit mass, gravimetric surface area; the mass per unit volume, or density; and the ratio of total pore volume to total cell volume, or void fraction (Martin & Haranczyk, 2014). These structural properties, which characterize the geometry of the pore network within the MOF, have been shown to correlate directly with important properties of the bulk material (Krishnapriyan et al., 2020). The distributions of MOFDiff samples and the reference distribution of BW-DB are shown in Figure 5. We observe that the property distribution of generated samples matches well with the reference distribution of BW-DB, covering a wide range of property values.
Vold VNU
Figure 6: The validity of MOFDiff samples for increas- ingly strict criteria. "Match" stands for matched connec- tion. "VNU" stands for valid, novel, and unique. Almost all valid samples are also novel and unique. The last column shows the validity percentage of BW-DB under our criteria.
4.2 OPTIMIZE MOFs FOR CARBON CAPTURE
Climate change one of the most significant and urgent challenges that humanity needs address. Carbon capture is one of the few technologies that can mitigate current CO2 emissions, for which MOFs are promising candidate materials (Trickett et al., 2017; Ding et al., 2019). In this experiment, we evaluate MOFDiff's capability to optimize MOF structures for use as CO2-selective sorbents in point-capture applications.
Molecular simulations for gas adsorption property calculations. For faithful evaluation, we carry out grand canonical Monte Carlo (GCMC) simulations to calculate the gas adsorption properties of MOF structures. We implement the protocol for simulation of CO2 separation from simulated flue gas with vacuum swing regeneration proposed in Boyd et al. 2019 from scratch, using equlp to calculate per-atom charges on the MOF (Kadantsev et al., 2013; Rappe & Goddard III,, 1991) and RASPA2 to carry out GCMC simulations (Dubbeldam et al., 2016) since the original simulation code not publicly available. Parameters for CO2 and N₂ were taken from Garcia-Sanchez, et al. 2009 and
7

---

Published as a conference paper at ICLR 2024
80
MOFDiff
70
Reference
20
60
MOFDiff-1
InOF-1
MOFDiff-2
Percent
10a
10 10" 10a COy/N, selectivity
0
2
CO2 uptake [mol/kg] (0.15bar, 298K)
40
SIFSIX-3-CU NI4-PyC
MOFDifT-3
30
MIL)53(A))
MOFD
50
20
GMOF-4
AI-PMOF
50
10 15 CO2 heat of adsorption
2 CO2 uptake [mal/kg] (0.1bar, 363K)
Working capacity [mol/kg]
[kcal/mol] (0.15bar, 298K)
Figure 7: CO2 adsorption properties for MOFDiff optimized samples (top-5 annotated with green boxes) compared to the reference distribution and selected literature MOFs (grey boxes). The four small panels breakdown working capacity to more fundamental gas adsorption properties.
TraPPE (Potoff & Siepmann, 2001), respectively. Under this protocol, the adsorption stage considers the flue exhaust mixture of CO2 and N2 at ratio of 0.15:0885 at 298 K and total pressure of bar. The regeneration stage uses a temperature of 363 K and a vacuum pressure of 0.1 bar for desorption.
The key property for practical carbon capture purposes is high CO2 working capacity, the net quantity of CO2 capturable by a given quantity of MOF in adsorption/deeooption cycle. Several factors contribute to a high working capacity, such as the CO2 selectivity N2, CO,/N2 uptake for each condition, and CO2 heat of adsorp- tion, which reflects the average binding energy of the adsorbing gas molecules. In the appendix, Figure 11 shows the benchmark results of our implementation compared to the original labels of B BW-DB, which demonstrate a strong positive correlation with our implemen- tation underestimating the original labels by average of around 30%. MOFDiff is trained over the original BW-DB labels and uses latent-space optimization to maximize the BW-DB property values. In the final evaluation, we use our re-implemented simulation code.
40
20
Match
Velid VNU
Figure 8: The validity of MOFDiff samples optimized for CO2 working capacity. Al- most all valid samples are also novel and unique.
MOFDifT discovers promising candidates for carbon capture. randomly sample 10,000 MOFs from the training dataset and encode these MOFs to get 10,000 latent vectors. We use the Adam optimizer (Kingma & Ba, 2015) to maximize the model preec-predicted CO2 working capacity for 5,000 steps with a learning rate of 0.0003. The resulting optimized latent vectors then decoded, assembled, and relaxed. After conducting the validity checks described Section 4.1, we find 2054 MOFs that are valid, novel, and unique (Figure 7 (a)). These 2054 MOFs then simulated with our GCMC workflow compute gas adsorption properties. Given the systematic differences between the original labels of BW-DB and those calculated with our reimplemented GCMC workflow, randomly sampled 5,000 MOFs from the BW-DB dataset and recalculated the gas adsorption properties using our GCMC workflow to provide a fair baseline for comparison. Figure shows the CO2 working capacity distribution of the BW-DB MOFs and the MOFDiff optimized MOFs: the MOFs generated by MOFDiff have significantly higher CO2 working capacity. The four smaller panels break down the contributions to CO2 working capacity from COy/N, selectivity, CO2 heat of adsorption, as well CO2 uptake the adsorption 10.15 bar, 298 K) and the desorption stages (0. bar, 363 K). We observe that MOFDiff generates distribution of MOFs that more selective towards CO2, have higher CO₂ uptakes under adsorption conditions, and bind more strongly to CO2-
From efficiency perspective, GCMC simulations take orders of magnitude computational time (tens minutes to hours) than other components of the MOF design pipeline (seconds to tens of seconds). These simulations also be made more accurate at significantly higher computational
8

---

Published as a conference paper at ICLR 2024
MONDIE
MOFD
3OFDEF
MOEDIE
MONDIE' 5
Working cxpecity: 4.59 melkig C0,IN, selaetivity 197.7
Working espurit: 4.86 molig CO,/N, selesivvy: 65.2
Working cxpicity 4.01 malkk CO)/N, sclerivity 39.6
Working expseity 4.03 malkk CO.NN, sclectiityy 26.2
Working cnpucity: 3.57 mel CO2/N, selaciiv By: 1026
MOFDaEE
MOFDat-7
MOFDIDI
MOFDIE 9
MORD 10
Working expacty: 3,50 molikg C0,NN, selacivayy
Working espacity: 3.70 malky C0,NN, sclectivity: 19.8
Working espacity 3.65 molkk CO,NN, selectivity
Working expucity: 3.61 malkkg CO./N, selectivity; 19.1
Wirking expacity 3.61 molrkg CO,/N, selactvvtyy 45.6
Figure 9: The top ten samples from MOFDiff in terms of the highest CO2 working capacity. Atom color code: Cu (brown), Zn (purple), (yellow), O (red), N (blue), C (gray), H (white).
costs (days) by converging sampling to tighter confidence intervals using more advanced techniques, such as including blocking spheres, which prohibit Monte Carlo insertion of gas molecules into kinetically prohibited pores of the MOF, and calculating atomic charges with density functional theory (DFT). Therefore, the efficiency of a MOF design pipeline can be evaluated by the average number of GCMC simulations required find one qualifying MOF for carbon capture applications. Naively sampling from the BW-DB dataset requires, on average, 58.1 GCMC simulations to find one MOF with working capacity of more than mol/kg. For MOFDiff, only 14.6 GCMC simulations are needed to find one MOF with a working capacity of more than 2 mol/kg, a 75% decrease in compute cost per candidate structure.
Compare to carbon capture MOFs from literature. Beyond efficiency, MOFDiff's generation flexibility also allows discover top MOF candidates that are outstanding for carbon capture. We compute gas adsorption properties of 27 MOFs that have been investigated for CO2 adsorption from previous literature (Madden al., 2017: Coelho et al., 2016; González-Zamon et al., 2019; Yao et al., 2021) using our GCMC simulation workflow. We compare the gas adsorption properties of the top ten MOFs discovered from our 10,000 samples (visualized in Figure 9) to these 27 MOFs in Table of Appendix C and annotate selected MOFs in Figure 7. MOFDiff can discover highly promising candidates, making up 9 out of the top 10 MOFs. particular, AI-PMOF is the top MOF selected by authors of Boyd al. 2019 from BW-DB. GMOF-1 to GMOF-9 the top carbon capture candidates proposed in Yao et al. 2021. This comparison confirms MOFDiff's capability in advancing functional MOF design.
Ibarra, 2017; 77; Boyd
5 CONCLUSION
We proposed MOFDiff, coarse -rasined diffusion model for metal oorgani framework design. Our work presents complete pipeline of representation, generative model, structural relaxation, and molecular simulation to address specific carbon capture materials design problem. To design 3D MOF structures without using pre-defined templates, we derive coarse coarse-grained representation and the corresponding diffusion process. We then design an assembly algorithm to realize the all-atom MOF structures and characterize their properties with molecular simulations. MOFDiff can generate valid and novel MOF structures covering a wide range of structural properties as well as optimize MOFs for carbon capture applications that surpass sate-offthe-ar MOFs in molecular simulations.
ACKNOWLEDGMENTS
We thank Karin Strauss, Bichlien Nguyen, Yuan-Jyue Chen, Daniel Zügner, Gabriel Corso, Bowen Jing, Hannes Stärk, and the rest of Microsoft Research AI4Science members and TJ group members for their helpful comments and suggestions. The authors acknowledge Peter Boyd and Christopher

---

Published as a conference paper at ICLR 2024
Wilmer for their advice the replication of previously reported adsorption simulation methods. X. F. and T. J. acknowledge support from the MIT-GIST collaboration. A. R. acknowledges support via Miller Research Fellowship from the Miller Institute for Basic Research in Science, University of California, Berkeley.
REFERENCES
Senja Barthel, Eugeny V Alexandrov, Davide M Proserpio, and Berend Smit. Distinguishing metal- organic frameworks. Crystal growth design, 18()):138-1777, 2018.
Simon Batzner, Albert Musaelian, Lixin Sun, Mario Geiger, Jonathan P Mailoa, Mordechai Korbluthh Nicola Molinari, Tess E Smidt, and Boris Kozinsky. E (3)-eqivvariant graph neural networks for data-efficient and accurate interatomic potentials. Nature communications, 13(1):1-11, 2022.
Anastasiya Bavykina, Nikita Kolobov, II Son Khan, Jeremy A Bau, Adrian Ramirez, and Jorge Gascon. -rgal-organic frameworks in heterogeneous catalysis: recent progress, new trends, and future perspectives. Chemical reviews, 120(16):8468- 111184666--355,,,, 2020.
Charlotte Bonneau, Michael O O'eefffe, Davide M Proserpio, Vladislav Blatov, Stuart Batten, Susan A Bourne, Myoung Soo Lah, Jean- Main-Gull Eon, Stephen T Hyde, Seth B Wiggin, et al. Deconstruction of crystalline networks into underlying nets: Relevance for terminology guidelines and crystallographic databases. Crystal Growth & Design, 18(6):34113341 2018.
Peter G Boyd, Yongjin Lee, and Berend Smit. Computational development of the nanoporous materials genome. Nature Reviews Materials, 208):1-15, 2017a.
Peter G Boyd, Seyed Mohamad Moosavi, Matthew Witman, and Berend Smit. Force-field prediction of materials properties in metal-organic frameworks. The journal of physical chemistry letters, (2):357-363, 2017b.
Peter G Boyd, Arunraj Chidambaram, Enrique Gareía-Diez, Christopher Ireland, Thomas D Daff, Richard Bounds, Andrzej Gladysiak, Pascal Schouwink, Seyed Mohamad Moosavi, M Mercedes MtarotoVVales al. Data-driven design of metal-organie frameworks for wet flue gas co2 capture. Nature, 576(7786):253-22) 2019.
Benjamin J Bucior, Andrew S Rosen, Maciej Haranczyk, Zhenpeng Yao, Michael E Ziebel, Omar K Farha, Joseph T Hupp, J Ilja Siepmann, Alán Aspuru-Guzit and Randall Q Snurr. Identification schemes for mogal-organic frameworks to enable rapid search and cheminformatics analysis. Crystal Growth & Design, 19(11):6622-667,, 2019.
Richard H Byrd, Peihuang Lu, Jorge Nocedal, and Ciyou Zhu. A limited memory algorithm for bound constrained optimization. SIAM Journal on scientific computing, 1605):1100-1208, 1995.
Ruojin Cai, Guandao Yang. Hadar Averbuch-Eie -loor, Zekun Hao. Serge Belongie. Noah Snavely, and Bharath Hariharan. Learning gradient fields for shape generation. In Computer Vision- ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28, 2020, Proceedings, Part III 16, pp. 364 381. Springer, 2020.
Jian Cao, Xuejiao Li, and Hongqi Tian. Metal-organic framework (mof)-based drug delivery. Current medicinal chemistry, 27(35):594-5596 2020.
Lowik Chanussot, Abhishek Das, Siddharth Goyal, Thibaut Lavril, Muhammed Shuaibi, Morgane Riviere, Kevin Tran, Javier Heras-Domingo. Caleb Ho, Weihua Hu, et al. Open catalyst 2020 (oc20) dataset and community challenges. Acs Catalysis, 11(10)6009-6000 6072, 2021.
Chi Chen, Weike Ye, Yunxing Zuo, Chen Zheng, and Shyue Ping Ong. Graph networks as universal machine learning framework for molecules and crystals. Chemistry of Materials, 314))3364-3357. 2019.
Ting Chen, Simon Korbllihh, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In International conference on machine learning, pp. 1597-1607. PMLR, 2020.
10

---

Published as a conference paper at ICLR 2024
Yongchul G Chung, Emmanuel Haldoupis, Benjamin J Bucior, Maciej Haranczyk, Seulchan Lee, Hongda Zhang, Konstantinos D Vogiatzis, Marija Milisavljevic, Sanliang Ling. Jeffrey Camp, et al. Advances, updates, and analytics for the computation-reace experimental metal-organic framework database: Core mof 2019. Journal of Chemical & Engineering Data, 64( 64(12):598555990 2019.
Juliana A Coelho, Ana Mafalda Ribeiro, Alexandre FP Ferreira, Sebastiao MP Lucena, Alirio E Rodrigues, and Diana CS Azevedo. Stability of an al-fumarate mof and its potential for co2 capture from wet stream. Industrial & Engineering Chemistry Research, 55(7):2134-1143 2016.
Yigitcan Comlek, Thang Duc Pham, Randall Q. Snurr, and Wei Chen. Rapid design of top-performing metal-organic frameworks with qualitative representations of building blocks. npj Computational Materials, 91):1770, 2023.
Gabriele Corso, Hannes Stärk, Bowen Jing, Regina Barzilay, and Tommi Jaakkola. Diffdock: Diffusion steps, twists, and turns for molecular docking, arXiv preprint arXiv:2210.01776, 2022.
Brian A Day and Christopher Wilmer. Genetic algorithm design of mof-based gas sensor arrays for co2-in-air sensing. Sensors, 20(3):924, 2020.
Meili Ding, Robinson W Flaig, Hai-Long Jiang, and Omar M Yaghi. Carbon capture and conversion using gnetal-organic frameworks and mof-based materials. Chemical Society Reviews, 48(10): 2783-2828, 2019.
David Dubbeldam, Sofía Calero, Donald Ellis, and Randall Q Snurr. Raspa: molecular simulation software for adsorption and diffusson in flexible nanoporous materials. Molecular Simulation, 42 (2)111-101, 2016.
William Falcon and The PyTorch Lightnnng team. PyTorch Lightning, March 2019. URL https: AI /iighhhihhtnn....
Matthias Fey and Jan Lenssen. Fast graph representation learning with PyTorch Geometric, In ICLR Workshop Representation Learning Graphs and Manifolds, 2019.
Almudena Garcia-Sanchez, Conchi O Ania, José Parra, David Dubbeldam, Thijs JH Vlugt, Rajamani Krishna, and Sofia Calero. Transferable force field for carbon dioxide adsorption in zeolites. The Journal of Physical Chemistry C, 11((0))88114888)) 2009.
Johannes Gasteiger, Janek GroB, and Stephan Günnemann. Directional message passing for molecular graphs. arXiv preprini arXiv:2003.00 030033 2020.
Johannes Gasteiger, Florian Becker, and Stephan Günnemann. Gemmet: Universal directional graph neural networks for molecules. Advances in Neural Information Processing Systems, 34: 6790-6802, 2021.
Johannes Gasteiger, Muhammed Shuaibi, Anuroop Sriram, Stephan Günnemann, Zachary Ward Ulissi, C. Lawrence Zitnick, and Abhishek Das. Gemnet-OC: Developing graph neural networks for large and diverse molecular simulation datasets. Transactions on Machine Learning Research, 2022. URL
net. htr/ffrruse coormmmdd=uuse 55m44BB.
Diego Gomez-Gualdrrom Oleksii Gutov, Vaiva Krungleviciute, Bhaskarjyoti Borah, Joseph Mondloch, Joseph T Hupp. Taner Yildirim, Omar K Farha, and Randall Q Snum. Computational design of metal-organic frameworks based on stable zirconium building units for storage and delivery of methane. Chemistry of Materials, 26(19):563--5639, 2014.
Eduardo Monzález-Zamoos and llich A lbarra. Co 2 capture under humid conditions in frameworks. Muterials Chemistry Frontiers, 1(8): (09::177-1484, 2017.
Nate Gruver, Samuel Stanton, Nathan C Frey, Tim GJ Rudner, Isidro Hotzel, Julien Lafrance- Vanasse, Arvind Rajpal, Kyunghyun Cho, and Andrew Gordon Wilson. Protein design with guided discrete diffusion. arXiv preprint 2305 2023.

---

Published as a conference paper at ICLR 2024
Raia Hadsell, Sumit Chopra, and Yann LeCun. Dimensionality reduction by learning invariant mapping. In 2006 IEEE computer sociery conference on computer vision and pattern recognition (CVPR 06), volume 2. pp. 1735-1742. IEEE, 2006.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. International conference on learning representations, 2016.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-688 2020.
Emiel Hoogeboom, Victor Garcia Satorras, Clément Vignac, and Max Welling, Equivariant diffusion for molecule generation 3d. In International conference on machine learning, pp. 8867-8887.
PMLR, 2022.
John Ingraham, Max Baranov, Zak Costello, Vincent Frappier, Ahmed Ismail, Shan Tie, Wujie Wang, Vincent Xue, Fritz Obermeyer, Andrew Beam, et al. Illuminating protein space with a programmable generative model. BioRxiv, pp. 2022-12, 2022.
Kevin Maik Jablonka. mofchecker. June 2023. URL https: atttp::/ggtthan /gtth/gtttthhh//eg wwt.ccoo////kc http//jjpppllabamm mofchecker.
Rui Jiao, Wenbing Huang, Peijia Lin, Jiaqi Han, Pin Chen, Yutong Lu, and Yang Liu. Crystal structure prediction by joint equivariant diffusion. arXiv preprint arXiv:2309.04473 04475, 2023.
Bowen Jing, Gabriele Corso, Jeffrey Chang, Regina Barzilay, and Tommi Jaakkola. Torsional diffusion for molecular conformer generation. Advances in Neural Information Processing Systems, 35:24240-24253, 2022.
Ian T Jolliffe. Principal component analysis for special types of data. Springer, 2002.
Eugene S Kadantsev, Peter G Boyd, Thomas D Daff, and Tom K Woo. Fast and accurate elec- trostatics in metal organic frameworks with a robust charge equilibration parameterization for high-throughput virtual screening of gas adsorption. The Journal of Physical Chemistry Letters, 4 (18):3056-306) 10 2013.
Markus Kalmutzki, Nikita Hanikel, and Omar M Yaghi. Secondary building units as the turning point in the development of the reticular chemistry of mofs. Science advances, 4 A(10):caat9180 2018.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), San Diego, CA, USA, 2015.
Aditi S Krishnapriyan. Maciej Haranezyk, and Dmitriy Morozov. Topological descriptors help predict guest adsorption in nanoporous materials. The Journal of Physical Chemistry C, 124(17): 9360-9368, 2020.
Harrison D Lawson, Patrick Walton, and Christina Chan. Metal-organic frameworks for drug delivery: design perspective. ACS applied materials & interfaces, 13(6):7044-722) 2021.
Jin Sub Lee, Jisun Kim, and Philip M Kim. Score-based generative modeling for de novo protein design. Nature Computational Science, pp. 1-11, 2023.
Sangwon Lee, Backjun Kim, Hyun Cho, Hooseung Lee, Sarah Yunmi Lee, Eun Seon Cho, and Jihan Kim. Computational screening of trillions of orgal-organic frameworks for perforaeform methane storage. ACS Applied Materials & Interfaces, 13(20)23364723364, 2021.
Hao Li, Kecheng Wang, Yujia Sun, Christina T Lollar, Jialuo Li, and Hong-Cai Zhou. Recent advances gas storage and separation using metal-organic frameworks. Materials Today, 21(2): 108-121, 2018.
Mian Li, Dan Li, Michael O' O'Keeffe, and Omar M Yaghi. Topological analysis of metal-organe frameworks with polytopic linkers and/or multiple building units and the minimal transitivity principle. Chemical reviews, 114(2):1343-137) 2014.
12

---

Published as a conference paper at ICLR 2024
Rui-Biao Lin, Shengehang Xiang, Wei Zhou, and Banglin Chen. Microporous metal-organie framework materials for gas separation. Chem, 6(2):337-363, 2020.
Yi Liu, Limei Wang. Meng Liu, Xuan Zhang, Bora Oztekin, and Shuiwang Ji. Spherical message passing for 3d graph networks. arXiv preprint arXiv:2102.05013 2021.
Shitong Luo and Wei Hu. Diffusion probabilistic models for 3d point cloud generation. In Proceedings of the IEEF/CVF Conference Computer Vision and Pattern Recognition, pp. 2837-2845, 2021.
Shitong Luo, Chence Shi, Minkai Xu, and Jian Tang. Predicting molecular conformation via dynamic graph score matching. Advances in Neural Information Processing Systems, 14:19784-19795, 2021.
Shitong Luo, Yufeng Su, Xingang Peng. Sheng Wang. Jian Peng, and Jianzhu Ma. Antigen-specine antibody design and optimization with diffusion- based generative models for protein structures. Advances in Neural Information Processing Systems, 35:9754-9767, 2022.
Peder Lyngby and Kristian Sommer Thygesen. Data-driven discovery of 2d materials by deep generative models. npj Computational Materials, 8(1):232, 2022.
David G Madden, Hayley S Scott, Amrit Kumar, Kai-Jie Chen, Rana Sanii, Alankriti Bajpai, Matteo Lusi, Teresa Curtin, John Perry, and Michael Zaworotko. Flue-gas gas and direct-air capture of co2 by porous orgalio materials. Philosophical Trunsactions of the Royal Society A: Mathematical, Physical and Engineering Sciences, 375(2084):201 60025, 2017.
Richard Luis Martin and Maciej Haranczyk. Construction and characterization of structure models of crystalline porous polymers. Crystal growth & design, 14(5):2431-244 2014.
Peyman Z Moghadam, Aurelia Li, Seth B Wiggin, Andi Tao, Andrew GP Maloney, Peter A Wood, Suzanna C Ward, and David FairennJimenez. Development cambridge structural database subset: a collection of metal-organts frameworks for past, present, and future. Chemistry of Materials, 29(7):2618 2222222516666..... 2017.
Seyed Mohamad Moosavi, Aditya Nandy, Kevin Maik Jablonka, Daniele Ongari, Jon Paul Janet, Peter G Boyd, Yongjin Lee, Berend Smit, and Heather Kulik. Understanding the diversity of the metal-organic framework ecosystem. Nature communications, 1(():11-0, 2020.
Albert Musaelian, Simon Batzner, Anders Johansson, Lixin Sun, Cameron Owen, Mordechai Kornbluth, and Boris Kozinsky. Learning local equivariant representations for large-scale atomistic dynamics. arXiv preprint arXiv: arXiv:2204.055499 2022.
Aditya Nandy, Gianmarco Terrones, Naveen Arunachalam, Chenru Duan, David W Kastner, and Heather J Kulik. Mofsimplify, machine learning models with extracted stability data of three thousand metal-organic frameworks. Scientific Data, 91)::4, 2022.
Aditya Nandy, Shuwen Yue, Changhwan Oh, Chenru Duan, Gianmarco G Terrones, Yongchul G Chung, and Heather J Kulik. A database of ultrastable mofs reassembled from stable fragments with machine learning models. Matter, 6(5):1585-1603) 2023.
Michael O' O'Keeffe and Omar M Yaghi. Deconstructing the crystal structures of metal-organic frameworks and related materials into their underlying nets. Chemical reviews, 1 112(2):655-702 2012.
Hyun Park, Xiaoli Yan, Ruijie Zhu, EA Huerta, Santanu Chaudhuri, Donny Cooper, Ian Foster, and Emad Tajkhorshid. Ghp-mofasse mofasseeble::: Diffusion modeling, high throughput screening, and molecular dynamics for rational discovery of novel metal-organic frameworks for carbon capture at scale. arXiv preprint arXiv:2306.00 08695, 2023a.
Hyunsoo Park, Sauradeep Majumdar, Xiaoqi Zhang, Jihan Kim, and Berend Smit. Inverse design of -etal-organic frameworks for direct air capture co2 via deep reinforcement learning. ChemRxiv, 2023b.
13

---

Published as a conference paper at ICLR 2024
Junkil Park, Yunsung Lim, Sangwon Lee, and Jihan Kim. Computational design of metal-organic frameworks with unprecedented high hydrogen working capacity and high synthesizability. Chem- istry of Materials, 35(1):9-16 16, 2022.
Junkil Park, Aseem Partap Singh Gill, Seyed Mohamad Moosavi, and JIHAN KIM. Inverse design of porous materials: diffusion model approach. ChemRxiv, 2023c.
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. Pytorch: An imperative style, perhopeeformance deep learning library. Advances in neural information processing systems, 32, 2019.
Jeffrey Potoff and J Iija Siepmann. Vapor-liquid equilibria of mixtures containing alkanes, carbon dioxide, and nitrogen. AIChE journal, 47(7):1676-16533 2001.
Qihui Qian, Patrick A Asinger, Moon Joo Lee, Gang Han, Katherine Mizrahi Rodriguez, Sharon Lin, Francesco M Benedetti, Albert X Wu, Won Seok Chi, and Zachary P Smith. Mof-based membranes for gas separations. Chemical reviews, 120(16):8161-8266 2020.
Zhuoran Qiao, Matthew Welborn, Animashree Anandkumar, Frederick R Manby, and Thomas F Miller. Orbnet: Deep learning for quantum chemistry using symmetry-adapted atomic-orbital features. The Journal of chemical physics, 153(12), 2020.
Anthony K Rappe and William A Goddard III. Charge equilibration for molecular dynamics simulations. The Journal of Physical Chemistry, 95(8):3358-3360 1991.
Anthony K Rappé, Carla J Casewit, KS Colwell, William A Goddard III, and W Mason Skiff. Uff, full periodic table force field for molecular mechanics and molecular dynamics simulations. Journal of the American chemical society, (4(25):10024-1000 1992.
David Rogers and Mathew Hahn. Extended-ccomecctvvi fingerprints. Journal of chemical information and modeling, 50(5):742-754, 2010.
Andrew S Rosen, Justin M Notestein, and Randall Q Snurr. Realizing the data-driven, computational discovery of metal-organic framework catalysts. Current Opinion in Chemical Engineering, 35: 100760, 2022.
Kristof Schütt, Pieter-Jan Kindermans, Huziel Enoc Sauceda Felix, Stefan Chmiela, Alexandre Tkatchenko, and Klaus-Robert Müller. Schnet: A continuous-filee convolutional neural network for modeling quantum interactions. Advances in neural information processing systems, 30, 2017.
Chence Shi, Shitong Luo, Minkai Xu, and Jian Tang. Learning gradient fields for molecular conformation generation. In International conference on machine learning, pp. 9558-9568. PMLR, 2021.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
Yang Song, Jascha Sohl-Dickstein, Diederik Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021. URL https: / //tp://opppe/ee/ee/// net forum?
1d=PxTIG12RRHS.
Alexander Stukowski. Visualization and analysis of atomistic simulation data with OVITO-the Open Visualization Tool. MODELLING AND SIMULATION IN MATERIALS SCIENCE AND ENGINEERING, 18(1), JAN 2010. ISSN 0965-0392 doi: (10.1 1088/0966 0393318//
Nathaniel Thomas, Tess Smidt, Steven Kearnes, Lusann Yang, Li Li, Kai Kohlhoff, and Patrick Riley. Tensor field networks: Rotation-and translation-equivariant neural networks for 3d point clouds. arXiv preprint arXiv:1802.082199 2018
A. P. Thompson, H. M. Aktulga, R. Berger, D. S. Bolintineanu, W. M. Brown, P. S. Crozier, P. J. in 't Veld, A. Kohlmeyer, S. G. Moore, T. D. Nguyen, R. Shan, M. J. Stevens, J. Tranchida, C. Trott, and S. J. Plimpton. LAMMPS a flexible simulation tool for particle- based materials modeling at the atomic, meso, and continuum scales. Comp. Phys. Comm., 271 271:108171, 1717 2022.
14

---

Published as a conference paper at ICLR 2024
Christopher A Trickett, Aasif Helal, Bassem A Al-Maythalony, Zain H Yamani, Kyle E Cordova, and Omar M Yaghi. The chemistry of aactal-organic frameworks for co2 capture, regeneration and conversion. Nature Reviews Materials, 2(8):1-16 2017.
Limei Wang, Yi Liu, Yuchao Lin, Haoran Liu, and Shuiwang Ji. Comenet: Towards complete and efficient message passing for 3d molecular graphs. Advances in Neural Information Processing Systems, 35:650-664, 2022.
Joseph L Watson, David Juergens, Nathaniel R Bennett, Brian L Trippe, Jason Yim, Helen E. Eisenach, Woody Ahern, Andrew Borst, Robert J Ragotte, Lukas F Milles, et al. De novo design of protein 2023.
structure and function with rfdiffusion. Nature, (10179766:108--111
Thomas F Willems, Chris H Rycroft, Michaeel Kazi, Juan C Meza, and Maciej Haranczyk. Al- gorithms and tools for high-throughput geometry- based analysis of crystalline porous materials. Microporous and Mesoporous Materials, 144(11:1341141, 2012.
Tian Xie and Jeffrey C Grossman. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Physical review letters, 120(14):14300, 2018.
Tian Xie, Xiang Fu, Octavian-Fuut Fugen Ganea, Regina Barzilay, and Tommi Jaakkola. Crystal diffu-
sion variational autoencoder for periodic material generation. In International Conference on Learn- ing Representations, 2022. URL ht tttpps: / opw.review.eeeeeeeeeee net/ forum 2id 0.RL-
Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geometric diffusion model for molecular conformation generation. arXiv preprint arXiv:2203.029) 2022.
Minkai Xu, Alexander S Powers, Ron O Dror, Stefano Ermon, and Jure Leskovee. Geometric latent diffusion models for 3d molecule generation. In International Conference on Machine Learning. pp. 38592-38610. PMI.R, 2023.
Omar M Yaghi. The reticular chemist, 2020.
Dong Yang and Bruce Gates. Catalysis by metal organic frameworks: perspective and suggestions for future research. Acs Catalysis, 93):1779-1798, 2019.
Zhenpeng Yao, Benjamín Länchez-Lengeling, N Scott Bobbitt, Benjamin J Bucior, Sai Govind Hari Kumar, Sean P Collins, Thomas Burns, Tom K Woo, Omar K Farha, Randall Q SnurT, et al. Inverse design of nanoporous crystalline reticular materials with deep generative models. Nature Machine Intelligence, 3(1):76-86 86, 2021.
Jason Yim, Brian L Trippe, Valentin De Bortoli, Emile Mathicu, Arnaud Doucet, Regina Barzilay, and Tommi Jaakkola. Se (3) diffusion model with application to protein backbone generation. arXiv preprint a 2xXi:2332..02277 2023.
Vadia Foziya Yusuf, Naved I Malek, and Suresh Kumar Kailasa. Review metal-organic framework classification, synthetic approaches, and influencing factors: Applications in energy, drug delivery, and wastewater treatment. ACS omega, 7(49):4500744451, 2022.
Xiangyu Zhang, Kexin Zhang, and Yongjin Lee. Machine learning enabled tailor-made design of application-specific metal-organic frameworks. ACS applied muterials & interfaces, 12(1): 734-743, 2019.
Musen Zhou and Jianzhong Wu. Inverse design of metal organic frameworks for c2h4/c2h6 separa- tion. npj Computational Materials, 8(1):256, 2022.
15

---

Published as a conference paper at ICLR 2024
A RELATED WORKS
Machine learning generative models for MOF design. MOF generative models combined with template-bascd assembly (Martin & Haranczyk, 2014; Lee et al., 2021) have been studied by several previous works. These models use a qualitative representation of MOF that describes the identity of the metal node, linker molecule, and topology template. The generative model can sample combination of these components, while the 3D MOF structure is determined by the assembly software. In particular, Yao et al. 2021 proposes a variational autoencoder and property-driven inverse design is enabled through latent-space Bayesian optimization. Other works employ monte-carlo tree search (Zhang et al., 2019), Bayesian optimization (Comlek al., 2023), genetic algorithm (Day & Wilmer, 2020: Zhou & Wu, 2022; Park et al., 2022), and deep reinforcement learning (Park et al., 2023b), respectively, to iteratively optimize MOF structures for desired properties with feedback from molecular simulation or surrogate property predictor. Distinct from previous works, our method does not rely on predefined templates. Instead, we use diffusion model to directly generate coarse-grained 3D structures of MOFs and maintain the physical symmetries through the use of equivariant and invariant networks. Additionally, we adopt a contrastive learning approach to model vast space of building blocks directly extracted from MOF datasets, without the need to curate the building blocks for template compatibility.
Diffusion Models aims to generate samples from the training data distribution through denoising diffusion process (Song & Ermon, 2019; Ho et al., 2020; Song et al., 2021). The iterative refinement sampling procedure has achieved great progress in generating 3D configurations, which is reflected in various applications spanning 3D point clouds and shapes in computer vision (Cai et al., 2020; Luo & Hu, 2021), molecular and protein conformation generation (Shi et al., 2021; Xu al., 2022; Jing et al., 2022: Xu et al., 2023; Yim et al., 2023), protein-ligand interactions (Corso et al., 2022), molecular and protein design (Hoogeboom
al., 2022; Luo al., 2022; Ingraham et al., 2022; Watson et al.. 2023; Gruver et al.. 2023), as well as inorganic crystal design (Xie et al., 2022; Jiao et al.. 2023). Compared to previous works, the novelty of this paper from the diffusion model perspective lies the treatment of coarse-gras grained building block representation. While existing molecular diffusion models commonly use discrete diffusion to generate discrete atom/amino-ace types, our model denoises learned embedding, encompassing vast space of building blocks that is unfeasible to represent with a discrete variable.
Equivariant and invariant neural networks have shown great advantages in various machine learning tasks for molecules and materials (Schütt et al., 2017; Thomas et al., 2018; Xie & Grossman, 2018; Chen et al., 2019; Qiao al., 2020; Schütt et al., 2017; Gasteiger al., 2020; 2021: Liu et al., 2021; Wang et al., 2022; Batzner et al., 2022; Musaelian et al., 2022; Gasteiger et al., 2022). These network architectures respect the permutational, translational, and rotational symmetries in molecular systems while being expressive. The current work employs SE(3)-invariant representation for encoding building blocks and MOF and 3E(3)-equivariant network for score prediction.
B MODEL DETAILS
B. MOFDIFF
Building block representation. The building block encoder is a GemNet-OC model that inputs the 3D configuration of the building block, including the connection points, and outputs building block embedding A radius-cutoff graph is built as the building block for message passing. addition to the contrastive loss Lc, we also train the building block latent representation to encode the number of atoms N₁. the number of connection points C., and the largest distance between any pair of atoms I in the building block by predicting these quantities. Cross- Cross-entropy loss is used for N₂ and C₁ while mean squared error loss is used for I:
LB CrossEntropy (N,, No) + CrossEntropy (Co, O,) + +
(3)
where No. C., are model predictions. These quantities are important indicators of the size and connection pattern of the building block. The overall loss for the building block encoder is:
CBB Lo + LB + 6||2
where the last term is an L2 regularization over the building block embedding with a loss weighting of Bb 0.0001 to constrain the norm of building block embedding. The regularization makes
16

---

Published as a conference paper at ICLR 2024
the embedding numerically stable use in diffusion modeling later. We do not apply weighting over Le and Lg. Hyperparameters of the building block encoder are reported in Table GemNet- OC hyperparameters are the default values for the Base version from Gasteiger et al. 2022 unless otherwise noted. After being trained to convergence, the building block encoder is frozen and used for encoding all building blocks construct the CG representation of MOFs.
MOFDiff encoding. Before feeding the MOF structures to the periodic GNN encoder, we normalize all MOFs by dividing all lattice lengths by the mean lattice length and dividing all building block embedding by the mean of all building block embedding's is L2-norms. This normalization makes easier to select the noisy distributions for diffusion modeling. The coarse- coarse-grained diffusion model only operates on the coarse- coarse-grained representation. To encode a CG MOF structure, we build the coarse-grained graph with the CG connections inferred from the all-atom inter-building-hiee connections: two building blocks i and, j have an edge with the periodic image 7 if an atom in building block has a bond connection to an atom in building block j (considering periodic image I). We refer interested readers to Xie et al. 2022 2 for more details on the multi-graph representation of crystals. The periodic GNN encoder is an Snv3a-invariant GemNet-OC model. After invariant message passing, apply pooling to the node embedding to obtain the CG MOF latent code z.
Diffusion process. The forward diffusion process injects noise into the coarse coarse-grained MOF MC AC, " L) to obtain the noisy structure M (Af, X L) for t = 0 to T, where at T the data is diffused to the prior distribution. At time step i, the denoiser PGNND inputs the noisy structure M latent code z, and the time step then predicts scores Af x ,s for building block embedding and coordinates. The lattice parameter remains fixed throughout the diffusion process. With contrastive building block embedding in Ig K xd (d = 32 for BW-DB), we employ a DDPM (Ho al., 2020) variance-pressenvve forward process for type embedding:
AC
(5)
q( (A.11A) = =N(va,- (1- (1-ag) I)
(6)
where BI...., Br is the variance schedule, 04 = 1 3r and ā II.=11 0gg. The corresponding reverse diffuusin sampling process is:
q( 12) N (1 AC
1 1 az āi
L-
1- - oi
B.1
VI
We refer interested readers to Ho et al. 2020 for a more detailed derivation of the DDPM diffusion process. We use the same noise schedule as Hoogeboom al. 2022, a. for the building block type diffusion.
With building block coordinates in R Kx3 we employ a variance-exploding forward diffusion process for the coordinates:
x (x0)
(8)
where o1...., or are noise levels. The corresponding reverse diffusion sampling process is:
X M,, 2) = N x (xx:
Vo oi_ 8x
-
of
We refer interested readers to Song et al. 2021 for more detailed derivation of the variance-exploding
diffusion process. We use the same noise schedule Song al. 2021: a. m
Zmaz
We
Tonin
handle the denoising target under periodicity similarly as Xie et al. 2022 and direct readers interested in further details to this reference.
To train the denoising score network PGNND. we use the following loss functions:
LA E,Mc..s .ea [IEEA 11 and Lx = EI,MC [oillex =xx
(10)
17

---

Published as a conference paper at ICLR 2024
Add noise
Denoise
L. Fx
B KL
PGNND
Mc (AC,XC,))
Encode
L, K,
L I,Ke
PGNN
MLPL, L.K- MLPp
B
Z
Initialize without
Condition
M LL
PGNND
Figure 10: Training and sampling schematics of MOFDiff. Steps that are only used for training are represented with blue arrows. Steps that are only used for sampling are represented with green arrows. Steps that are used in both training and sampling are represented with black arrows. The training steps lead to the loss functions. The sampling steps lead to CG MOF samples.
where EA, (), I) sampled Gaussian noises, injected through the forward diffusion processes defined Equation (6) and Equation (8). The reverse diffusion process defined in Equation (7) and Equation (9) used for sampling MOF structures at inference time.
In addition to the diffusion losses LA and Lx, MOFDiff also trained to predict the lattice parameters L, the number of building blocks and property labels è from the latent code z. We use mean squared error loss for the lattice parameters and the property labels, and cross-entropy loss for the number of building blocks:
LL,Ke IL
i1 CrossEntropy(KK (K, K) 1ecllla)) all?
(11)
The entire MOFDiff then trained end-to-end with the loss function:
(MOFDIf LA Lx + LL.K. BKL
(12)
Where the ΣKL is the KL regularization for variational autoencoders. We did not use weighting over the different loss terms except for the KL regularization, which is weighted with fKL 0.01 (Higgins et al., 2016). The training and sampling process is illustrated in Figure 10. All hyperparameters are reported in Table 3.
B.2 RECOVER ATOMIC MOF STRUCTURES
The coarse-grained MOF structures generated by the diffusion model specify the lattice parameters, building block identities, and building block coordinates (the centroid of connection points). However, they do not specify the orientations of building blocks. The assembly algorithm finds the orientations of the building blocks to connect them to each other. Throughout the assembly process, we fix the centroids of the building blocks, the internal structures (atom relative coordinates) of the building blocks, and the lattice parameters. The building block orientations are the only variables that are allowed to change (Figure 4). As we change the orientation of building block, all atoms and connection points within rotate around its centroid.
For any ground truth structure, the connection points (as defined in Section 2) of adjacent building blocks will perfectly overlap since they are midpoints of the bonds connecting inter- mnte-building-blee atoms. Therefore, a viable objective for the assembly algorithm to maximize the overlap of compatible inter-building-ue connection point from a metal atom, and the other is from non-metn metal atom; (2) they are not from the same building block. We denote the set of all connection points C, the number of connection points as C. the coordinate connection point as 2;, the Euclidean distance between connection points i and di-- | æ.yl1 and the connection points compatible with C,. We define
connection points. Two connection points are compatible if (1) one
18

---

Published as a conference paper at ICLR 2024
Algorithm 1 Optimize building block orientations for MOF assembly
1: Input: MOF structure M = (AC, X L), the number of optimization rounds U, Gaussian
kernel width σ1 > ... > av, number of nearest neighbors for overlap evaluation k > ... > ky
2: Output: Building block orientations: Ω = w for all of E AC
3: Randomly initialize building block orientations $2
4: for round u = 1....., do
5: Let + ou k ky
6: minimize Lo (22) with respect to Ω using L-BFGS
7: end for
the objective function:
Lo.k,= - Σ exp( بي ) JEC;
mill:: C₁ dig S &dg)]ll k)
(13)
Where II is the indicator function. This loss can be thought of as measuring the inverse of the overlap under a Gaussian kernel width o, and the overlap is only evaluated for the nearest neighbors among the compatible connection points. Minimizing this loss maximizes the overlap. This loss is related to the building block orientations because the coordinate a connection point æ; is related to the orientation Wa (under the axis-angle representation) and CG coordinate xy of the corresponding building block a through:
x
x Uo,
(14)
where Va.i is the vector from the building block centroid to the connection point under a canonical orientation (which is invariant throughout the assembly process), and Rg is the rotation matrix corresponding to wa- The distance between pair of connection points dis can then be related the orientations of the two corresponding building blocks through Equation (14). Lo is wiice-iifferentiable with respect to building block rotations w for all building blocks as Lo,k.o is twice-differentiable with respect to dy for all connection points i, and dij is dwice-differentiabll with respect to w, and wp. This allows us to use L-BFGS, second-order optimization algorithm.
We can now define an annealed optimization process by gradually reducing a and at the beginning, the width and the number of other connection points we evaluate overlap with & are high, so is easier to find overlap between connection points, and the optimization problem becomes smoother. This makes it simpler to find an approximate solution. At the end of optimization, the kernel width small, and we are only computing the overlap for the closest compatible connection points. At this stage, the algorithm should have already found approximate solution, and stricter evaluation over overlapping can let the algorithm find more accurate orientations for matching the connection points closely.
The assembly algorithm starts by randomly initializing the orientations of the building blocks. Using the L-BFGS method, the algorithm iteratively minimizes Lo,k,a ar by adjusting the building block orientations Ω: wa (using the axis-angle representation) for all building blocks a. We use the axis-angle representations because rotation matrices need to follow specific constraints. As explained above, we start with relatively high a and and gradually reduce them in the optimization process gradually refine the optimized orientations. The full algorithm is shown in Algorithm 1. In our experiments, we use rounds: U 3, with = [3. 1.65, 0.3] and i- assembly process is visualized in Figure 4.
[30, 16. 1]. An example
Force field relaxation. The relaxation process is modified from workflow proposed in Nandy et al. 2022 and has four rounds of energy minimization using the UFF force field and the conjugate gradient algorithm in LAMMPS. At each round, we use LAMMP S's minimize function with etol=1 x 10-5 ftol=1 x 10-8, ma maxiter=1 106, and maxeval=1 x 106. the first and third rounds, we only relax the atom coordinates while keeping the lattice parameters frozen. In the second and fourth rounds, we relax both atom coordinates and the lattice parameters. The relaxation process can refine the all-atom structures based on the complete MOF configuration and correct minor errors in the previous steps (such as slightly smaller/bigger unit cells). Structural optimization using classical force field is commonly done in materials and MOF design (Lee et al., 2021; Nandy al., 2022).
19

---

Published as a conference paper at ICLR 2024
002 uptake [mol/kg] (0 15.ar, 238K) PCC-o .000
Working capacity [mo(/a] PCC-0.85
COy/Na selectivity PCC=0,78
102
10:
107
Our implemmertaan
10
20a
10
10
10
CD2 heat of addd [kcaa/mml] (0. (0.15barr, 298K)
002 uptake (moliaa (0. 1bar. 363K)
PCCw0.
2.5
2.0
1.5
10
1.0
0.0
0.5
Original
Original
Original
Figure 11: Benchmark GCMC results. PCC stands for "Pearson correlation coefficient",
B.3 MOF REPRESENTATION METHODS AND NETWORK ARCHITECTURE
Building block identification. We use the widely adopted Extended-Connectinn Fingerprints (ECFPs) (Rogers & Hahn, 2010) for building block identification. The fingerprint is built by iteratively updating identifier for each atom that reflects its neighboring atoms. After de-duplication, the fingerprint bit array is obtained by hashing the final atom identifiers. For ECFP4, the number of interactions is and the largest possible fragment will have width of 4 bonds.
MOF deconstruction is through the metal -oxo algorithm (Bucior et al., 2019). The inorganic building blocks are first identified metal-oxo clusters. These clusters include oxides, bound hydroxide, peroxide, and various node water species. The remaining fragments are identified as organic building blocks. Carboxylate functional groups, including the oxygen atoms, are considered part of the organic building blocks because they are covalently bonded to the rest of the molecule.
MOF identification. MoFid (Bucior al., 2019) systematic identifier developed represent MOFs and facilitate their analysis. It deconstructs the MOF to analyze its building blocks and topology to provide a string that contains the identities of the MOF building blocks and the metadata about the simplified overall topology. MOFid can be used to unambiguously tag individual MOF structures and is therefore used for determining the novelty and uniqueness of a MOF structure.
Network Architecture. All graph neural networks use the GemNet-OC architecture (Gasteiger et al., 2022). GemNet-CC an improved version of the GemNet architecture, a symmetry-respection possage-passing neural network that incorporates directional, angular, and dihedral information in the embedding process of the edge features. On top of GemNet, GemNet-OC implements a series of enhancements in radial/angular basis functions, interaction hierarchy, and hyperparameters to achieve impressive performance on the OC20 Chanussot et al. (2021) benchmark. We refer interested readers Gasteiger et al. 2021; 2022 for further details on GemNet-OC.
C EXPERIMENT DETAILS
Molecular simulation. As stated in Section 4.2, we implement GCMC simulation workflow from scratch, which may produce different results compared to the closed-source workflow used in BW-DB (Boyd et al., 2019). Per-atom charges on the MOF were calculated with egulp using the MEPO parameter set and the default configuration. GCMC simulations were performed with RASPA2 using the default configuration unless otherwise noted. Charge-charge interactions were modeled with Ewald sums at precision of 10 10-6 J. Other interactions were modeled with the Lennard-Jones 12-6 potential using UFF parameters for the MOF atoms with the epsilon parameters scaled by 0.635, parameters from Garcia-Sanchez et al. 2009 for CO2, and TraPPE parameters for N2.
20

---

Published as a conference paper at ICLR 2024
Gravimetric [m
jolom")
25
7000
2.0
08
20
6000
5000
-06
1.0
-10
0.5
CC2y uptake [mokkaa] 10. 15ear, 298K]
CO2 heat acsprtion [kcal/mol]
Warking capacity (molkg]
CO./N, selectivity
(0.15bar, 2983)
100
167
PC1
-109
PC1
104 PC1
10sl
PC1
Figure 12: Principle component analysis the MOFDiff latent space of the validation set, color- coded with various structural and gas adsorption properties.
12.0 A cutoff was applied to all interactions, with potentials shifted to zero the cutoff radius. The minimum sized supercell was constructed for each MOF such that all lattice vectors greater than 24.0 in length. The allowed Monte Carlo moves for gas atoms were identity change, swap, translation, rotation, and reinsertion a likelihood ratio of 2:2:1:1:1, and the MOF atoms were held constant throughout the simulation.
for 2000 equilibrium cycles followed by 2000 production cycles, with the uptake of each gas calculated as the average loading over the 2000 production cycles implemented in RASPA2. Similarly, each enthalpy of adsorption was calculated as the average internal energy of guest molecules within the MOF averaged over the 2000 production cycles as implemented in RASP A2 and converted to heat adsorption by changing the sign. Adsorption conditions were modeled using mixture of CO2 and N₂ at partial pressure ratio of 0.15:0.85, an external temperature of 298 K. and an external pressure of bar. Regeneration conditions were modeled using only CO2, an external temperature of 363 K, and external pressure of 0.1 bar. Working capacity was calculated as the difference in CO2 uptake under adsorption and regeneration conditions. CO,/N2 selectivity was calculated the ratio of each gas's respective uptake under adsorption conditions.
Simulations
Figure 11 shows benchmark that compares the gas adsorption labels obtained from BW-DB (original labels) and the labels obtained from workflow (our implementation) for 5,000 randomly sampled MOFs from BW-BB. The Pearson correlation coefficient (PCC) is also reported for each property. We observe strong positive correlation, while the working capacity is generally underestimated. Our model is trained with the original labels, and for property optimizing inverse design, we use property predictor trained over the original labels. Our model still demonstrates significant property improvement (Figure 7), which demonstrates the robustness of our method under shifted property evaluator.
MOF latent space. In Figure 12, we conduct principle component analysis (Jolliffe, 2002) produce two-dimensional visualization of the MOFDiff latent space. The latent space exhibits smooth tran- sitions for property values, indicating a smooth property landscape.
30
Percent
20
Compare to literature MOFs. In Table 1, we compare the top-ten MOFs generated by MOFDiff and 27 MOFs from previous litera- ture (Madden et al., 2017: Coelho et al., 2016; GonzálezzZamora & Ibarra, 2017; Boyd et al., 2019; Yao et al., 2021), Notably, AI-PMOF was proposed in Boyd et al. 2019, synthesized, and validated through real-world experiments. GMOF-1 to GMOF-9 are the top carbon capture candidates proposed in Yao et al. 2021.
10
1-5
6-10 11:15 16-20 Number building blocks
Figure 13: The percent of valid samples declines with more building blocks.
One limitation of MOFDiff is its generated samples have a lower validity rate when the size of the MOF becomes bigger. Figure 13
21

---

Published as a conference paper at ICLR 2024
shows declining validity percentage for samples of more building blocks. This result is unsurprising since bigger MOF with more building blocks inherently more complex. For a generated structure to be valid, the coordinates of every atom need to be correct, especially at every connection. The lattice parameters also need to be very accurate. Reformulating the diffusion process to enable the iterative refinement of the lattice parameters through the generation process and regularizing the diffusion process with known templates are two future directions to overcome this challenge.
Software versions. MOFid-v1.1.0, MoFChecke
grulp-vI.0.0. RA5PA2-V2.0.47,
LAMMP 5-2021-9-29, and 2eo++ are used in our experiments. Neural network modules are implemented with PyTorch-v1.1.0 (Paszke al., 2019), Pyg-v2.0.4 (Fey & Lenssen, 2019), and Lighhting.vii... (Falcon & The PyTorch Lightning team, 2019) with CUDA 11.3. OVITO- v3.8.3 (Stukowski, 2010) used for visualization.
Table Carbon capture properties of top ten MOFDiff optimized samples and MOFs from previous literature, sorted by CO2 working capacity.
CO2 working capacity [mal/kg]
CO.NN selectivity
CO uptake [mal/ee] (0.15 har, 298 K)
COs uptake [mol/kgl (0.1 bar, 363 K)
CO: heat of adsooption [kcal/mol]
CO: lest of adsorption [kenl/mool]
MOFDiff-
4.89
197.66
7.05
2.16
10.
10:05
MOFD
4.86
65.17
6.57
1,71
9.39
9.00
MOHDIISS
1,055
8.44
MOFD
4.03
26.21
1.85
0.82
9.05
8.41
MOFD
3.87
1026.38
13.27
9.40
12.61
11.27
AL-PMOF
4.95
8:26
MOFDII-6
3.80
73.34
1.73
0.93
9.13
9.02
MONDII-7
3.70
19.80
4.28
0.57
7.36
7.90
MOFD
3.65
50.62
1.68
1.02
8.94
8.98
4.18
0.57
MOHDIT--10
3.61
45.60
4.57
0.96
9.41
9:51
InOF-1
3.11
9.26
3.43
0.32
7.61
6.69
0.92
SABVOH manual
2.26
5.16
1.57
0.31
6.90
6.09
GMOF-4
2.20
11.46
2,53
0.33
7.96
7.57
GMOF-1
0.33
YEZKUI,ccaam
2.15
2.64
0.49
RUBTAKOI
2.11
19.15
2.70
0.59
7.82
8.72
AIFa
2.08
5.30
2.46
0.38
6.95
6.45
1.78
0.23
SIPSIX-3-Cu
1.22
2.69
1.47
11.80
11.79
NOTT-100
0.95
3.57
1.09
0.13
6.05
5.54
QOWRAV
1.02
0.14
DICRO-3-Nii
0.61
10.36
0.69
0.07
7.54
7.47
GMOF-2
0.59
2,44
0.69
0.10
5.58
5.27
CIGXIA manual
0.53
3.11
0.63
0.10
5.82
6.88
GMOF&
0.47
U92
0.61
0.14
4.44
3.51
OCUNAC manual
0.38
2.87
0.16
0.08
5.29
5.06
CuBTC
0.36
2.21
0.45
0.09
5.52
5.82
GMOF-9
0.12
DMOF-1
0.35
2.10
0.41
0.07
5.07
4.82
FAWCEN SL
0.33
2,42
0.38
0.05
5.37
5.16
NIMXAK
GMOP-5
0.20
0.45
0.29
0.08
2.68
2.11
GMOF-6
0.18
0.41
0.28
0.10
2.96
2.38
GMOF-7
0.10
0.30
0.16
0.06
2.06
1.59
MOF-5
0.09
1.02
0.12
0.03
3.34
3.11
22

---

Published as a conference paper at ICLR 2024
Table 2: Hyperparameters for building block representation learning.
Hyperparameter
Value
building block embedding dimension
GNN hidden layer dimension
256
projection dimension
128
# encoder GNN layers
radius cutoff
20 Á
maximum number of neighbors
50
temperature (r)
0.1
S
0.0001
batch size
512
optimizer Adam
initial learning rate
0.0003
learning rate scheduler learning rate patience learning rate factor
Reducel ReaeeeRROnPPPPP Plateau
10 epochs
0.6
Table 3: Hyperparameters for MOFDiff.
Hyperparameter
Value
latent dimension 256
GNN hidden layer dimension
256
# encoder GNN layers
# decoder GNN layers radius cutoff'
(after normalization)
maximum number of neighbors 24
total number of diffusion steps (T) 2000
Tmin for coordinate diffusion 0.001
mmax for coordinate diffusion
noise schedule for coordinate diffusion noise schedule for embedding diffusion time step embedding time step embedding dimension
Zuxx
Twin
Hoogeboomn et al. 2022
Fourier
64
3KL 0.01
batch size
128
optimizer Adam
initial learning rate 0.0003
learning rate scheduler
Plnneaa
learning rate patience learning rate factor
50 epochs
0.6
23


Reviews:
[{"review_id": "azbLZzQ3h1", "reviewer": "Reviewer_e39L", "summary": "The paper presents a method for generating Metal-Organic Framework (MOF) structures with target properties such as carbon capture.\n\nThe proposed method consists of a contrastive learning framework for embedding the MOF blocks based on their ECFP4 fingerprint. These representations are then used to work with a coarse-grained representation of the MOF. A latent variable model is then trained to generate a new MOF. Sampling from the latent variable allows for estimating the lattice structure and the number of building blocks. Then a diffusion-based decoder is used to produce the building block types and their location in the unit cell. The building blocks are then oriented and connections are established between them. Finally, a force field relaxation is computed to finalize the position and orientation of all atoms in the MOF. \n\nThe MOFs are validated based on well-established domain knowledge. The quality of the generated MOF structures is assessed for key application tasks such as carbon capture using well-established Monte Carlo simulation. \n\nThe proposal goes beyond the limitations of template-based methods and has computational advantages in proposing novel MOF structures with desired properties.", "questions": "One potentially problematic aspect is that the number of blocks K and the lattice structure L are computed based on the sample z which also conditions the diffusion of the attributes and the locations of the blocks. This can also lead to invalid MOF structures as the block's location and orientation may not match the lattice geometry. You identify this as well in your last sentence. \n1. Why not incorporate these in the diffusion process? \n2. Or alternatively, why not compute the lattice structure based on the atoms and the bond in the unit cell? Wouldn't the atoms (block) and the bonds uniquely identify the lattice structure of the crystal?\n3. Why generate from an uninformed prior N(0, I)? It seems unreasonable to expect that sampling from such a distribution would give good coverage of the vast space of possible MOF configurations. Would it not be more effective to condition on a number of building blocks or present partial coarse structures? Possibly many other well-understood properties of the MOFs?\n4. Your generation process is limited to using the building blocks present in the training data. How broad of a coverage does this give the generating process? Are there many other MOFs possible with building blocks not present in the training data?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 2, "strengths": "The paper clearly demonstrates the strong technical background in Material Science of the authors. The focus is on achieving concrete results in MOF generation that can have an impact on key applications of MOF materials.  Correspondingly, the validation of the results is substantive. \n\nThe selected Machine Learning components, such as the choice of the latent variable model, diffusion model, and contrastive learning are precisely selected to overcome specific challenges towards the goal of generating MOFs.", "weaknesses": "The above-given strengths are closely related to the weaknesses of the paper. Clearly, a very good application paper that makes important advances for computational material science may have limited contribution to the field of Machine Learning (ML). \n\nWhile a strong contribution to Material Science, from the ML perspective the work is a good combination of existing methods but does not go much further than that. The paper tells a story about many of the design decisions that were taken, but many other choices could have been made with even more recent approaches. \n\nIn that sense, the main weakness is the alignment to the ICLR conference.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VoCWjoGGBJ", "reviewer": "Reviewer_nQJ6", "summary": "The authors present MOFDiff, a diffusion model that generates a coarse-grained (CG) representation of metal-organic framework (MOF) structures using a diffusion model. MOFs are represented hierarchically with sets of atoms grouped to represent building blocks. Each MOF is represented with K building blocks such that K << N, where N is the number of atoms in the periodic cell. This is particularly important as typical MOF unit cells may contain on the order of 100s of atoms. The authors first embed the building blocks using a model trained on contrastive learning loss. The latent vector is then used to condition a denoising diffusion model to generate CG representations of MOFs. CG represents are converted back to MOFs using a novel assembly algorithm. The authors define a validity criterion for their generated MOFs and show they are capable of generating valid, novel, diverse MOFs. MOFDiff is also capable of guided inverse design and optimizing MOFs CO2 separation for carbon capture.", "questions": "* In Figure 5, are only novel MOFs (i.e. not in the reference dataset) used to generate the histogram?\n* In the representation learning of building blocks, the authors mention small geometric variations of the building. Are these variations in the coordinate space? In other words, what transformations are used to provide positive samples in the contrastive loss?\n* How are the coordinates of the building blocks assigned?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "* The paper is easy to follow and the authors motivate their reasoning for model and design choices\n* The contrastive representation learning of building blocks allows the authors to learn a meaningful latent space of building blocks including metal nodes and linkers. This extends previous work which only modified linkers and as a result, generated low-diversity samples\n* The proposed assembly algorithm to orient multiple building blocks using gradient-based optimization is novel and provides a strong justification for the authors' data representation choices", "weaknesses": "* In the generation process, it is unclear why the latent vector Z can be sampled from the standard normal distribution. The latent vector is generated using the OrbNet encoder trained with the contrastive learning loss. Without further regularization, like in the case of variational autoencoders using the KL divergence, the latent vector should not conform to the distribution.\n* It would be useful to compare with previous work [1] on the inverse design of MOFs. The authors mention and cite a source on the low diversity of the generated MOFs with template-based systems such as SmVAE, but a comparison isn’t present in the experiments section.\n* The validity and novelty of generated  structures are low compared to template-based models as in [1].\n* In general, while the authors include relevant work in the paper, the authors should have a related work section to place their work in the context of the field.\n    * The following papers could potentially be related to the present work [2], [3].\n\n[1] Yao, Z., Sánchez-Lengeling, B., Bobbitt, N. S., Bucior, B. J., Kumar, S. G. H., Collins, S. P., ... & Aspuru-Guzik, A. (2021). Inverse design of nanoporous crystalline reticular materials with deep generative models. Nature Machine Intelligence, 3(1), 76-86.\n\n[2] Zhou, M., & Wu, J. (2022). Inverse design of metal–organic frameworks for C2H4/C2H6 separation. npj Computational Materials, 8(1), 256.\n\n[3] Park, Junkil, et al. \"Computational design of metal–organic frameworks with unprecedented high hydrogen working capacity and high synthesizability.\" Chemistry of Materials 35.1 (2022): 9-16.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "o29SpBUTNI", "reviewer": "Reviewer_yqBG", "summary": "The paper introduces a novel method for generating metal-organic frameworks (MOFs) using a coarse-grained diffusion model called MOFDiff. MOFDiff employs a coarse-grained (CG) representation derived from MOF building blocks. A graph neural network (GNN) encoder trained via contrastive learning is used to map these building blocks into a latent space (z). MOFDiff consists of four main components: \n1. A periodic contrastive GNN encoder for latent embedding\n2. A MLP for lattice parameter (L) and building block (K) count prediction\n3. A periodic GNN denoiser for diffusion-based MOF generation\n4. Finally another MLP for MOF property prediction e.g., CO2 capacity. \n\nThe denoising diffusion process consists of two steps, first on the CG based building block types and then on their 3D coordinates. Then the all atom structure is recovered using an assembly algorithm, followed by simple force field (UFF) based relaxation. MOFDiff is trained on a dataset of ~304k MOFs and shows capability in generating valid and diverse MOFs. Overall, this paper leverages a diffusion model over a CG representation of MOFs to efficiently generate complex new structures, with proven efficacy in carbon capture applications.", "questions": "1. Why was the UFF chosen as force field? Would a MOF specific force field result in better structures? \n\n2. ECFP4 was chosen as similarity measure for contrastive learning. Would a better similarity measure e.g., learned embeddings (ChemBERTa) result in better GNN encoder? \n\n3. How difficult would it be extend the current framework to allow novel building blocks? \n\n\n### Corrections:\n\n* On page 19, there is a mismatch between Py-G and pytorch references.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The CG representation of building blocks for the diffusion model makes MOFDiff computationally efficient to train and sample new MOFs. The all atom  \n\n2. Underlying physical symmetries are appropriately handled with contrastively trained GNN encoder for the building blocks\n\n3. Optimizing MOFs in the latent space rather than decoding repeatedly is an efficient inverse design strategy demonstrated in the paper.", "weaknesses": "1. As acknowledged in the paper, the validity of the MOFs decreases as number of building blocks, this could limit applicability to larger and more complex MOFs.\n\n2. The authors use a MOF dataset with less than <20 building blocks. But the paper doesn't discuss about size extensivity of the GNN encoder and denoising module in detail.  \n\n3. There is limited discussion on the synthetic accessibility of the designed MOFs. Finally, the building blocks are constrained to a certain set  to allow coarse graining, this may led to less diverse MOFs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xxSdaIvfBR", "reviewer": "Reviewer_1BzN", "summary": "The paper presents a coarse grained diffusion model for generating metal organic frameworks. Coarse structures are generated by a diffusion model, after which 3d atomic structures are assembled, oriented using a heuristic optimization procedure, and optimized using a force field. The coarse grained building blocks are represented using a learned (contrastive) representation. The generative model is evaluated on an inverse design problem using GCMC simulation.", "questions": "In the coarse grained structure, are building blocks only represented using their identity and position? Did you consider including other features such as their orientation?\n\nHow sensitive is the representation learning to the choice of fingerprint? Is ECFP4 the only fingerprint you have considered, and why did you choose that?\n\nAbstract: What does \"predicting scores in E(3)\" mean? At this point it is not clear which scores this refers to.\n\n\"small geometric variations in 3D orientation\": I assume \"orientation\" here means something other than rotation/tranlation?\n\nWould there be room to include a short technical description of the metal-oxo / MOFid algorithms that are used? Similarly, a sentence that technically describes ECFP4?\n\nCould you include a very brief technical description of the gemnet-oc architecture?\n\nThe first paragraph in section 3 is difficult to read - would you consider including a \"concept figure\" that outlines the overall training + generative process?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "For the most part, the paper is easy to follow. There are several illustrative figures that provide a good overview\n\nThe paper addresses a very important and difficult problem, and the proposed method is sensible and (at least to some degree) effective.\n\nThere are several technical novelties, including the particular way the coarse graining and reassembly is conducted.\n\nThe method is validated with GCMC simulation.", "weaknesses": "There are many \"moving parts\" in the modeling pipeline. It would be enlightening with more ablation studies or comparisons of different modeling choices to yield some insight into the sensitivity to the different modeling choices.\n\nSome technical details are not described in detail. The paper builds on several existing techniques, which are referenced but not described technically. This is just a minor point, but in my view the paper could perhaps be more self contained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (poster)