SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Overall we find in over  $80\%$  of test cases learned densities achieve faster convergence versus the baseline. Importantly these test cases are out-of-distribution, involving structures with new combinations of elements not seen at train time. Our results show that training on simpler structures, i.e. unary and binary, can lead to generalization on more complex structures, i.e. ternary.
In this work we only test generalization for combinations of elements, and ignore stoichiometric ratios. In addition, we only investigate bulk catalysts, not the full complexity of catalyst systems as modeled in Chanussot et al. [8] which includes adsorbates, surfaces, and non-equilibrium points. Extending our analysis to these more complex systems is left to future work.
Our energy values are not directly comparable with those of Chanussot et al. [8], because we generated our dataset with a different DFT solver and pseudo-potentials. Systematic numerical differences may exist between different solvers, which makes comparison of energy values difficult.
Another limitation is that we do not report timings of our method versus traditional CPU-bound methods. Because of the high GPU-bound inference costs of our method, we do not expect timing improvement versus traditional methods. Sufficiently many GPUs will help mitigate this, as well as recent hardware development trends towards larger GPU memory.
We generated a new dataset of bulk catalyst relaxations with charge densities using open-source software. We trained a density-model pointwise on unary and binary catalysts. We showed this model can generalize to new binary and ternary catalysts, a form of combinatorial generalization. We found learned densities lead to faster convergence than standard baselines used to initialize the SCF cycle of DFT in over  $80\%$  of cases, amounting to an average of  $13\%$  saving in iterations.
7.1 Future Work
There are several directions for improvement to this work. The first is improving the proportion of test samples with positive savings, where clearly higher is better. In addition, expanding the class of structures tested, e.g. to quaternary structures and beyond, is of keen interest in applications.
A second direction is improving the rate of convergence of the learned densities. At present a number of SCF iterations are still necessary to reach high levels of convergence e.g.  $10^{-9}$ . The ideal case would be to converge rapidly to a single or few SCF iterations.
Rapid convergence across many kinds of structures opens up the interesting possibility of an end-to-end hybrid learning/numerical model which first precisely predicts the density then computes the energy in one or a few SCF iterations. Such models may have different Pareto curve properties than current energy-based approaches, and may be useful in applications with high precision requirements.
8 Broader Impact
One way this work may have positive broader impact is through reducing costs of DFT computations. For example, supposing that the USA National Labs spend  $100M on DFT per year, and a negligible cost of inference, then a \(13\%$  reduction in SCF iterations would yield a savings of \)13M $^{1}$ . Although there are a number of practical details to resolve, we believe this work is an important first step.
The potential impacts of new catalysts discoveries may have significant economic and environmental consequences. Positive impacts may include accelerating the transition towards renewable energy, an important goal for future societies. Negative impacts may include unintended consequences such as contributing to unsustainable population growth. In addition, as with any new technology with potentially large impacts in the economy, there is risk that control of the technology may concentrate in the hands of the few and limit the distribution of benefits.
In early stages of research is difficult to discern if the aims will come to fruition. The computational discovery of new catalysts does not necessarily imply that the results are translatable into real-world materials, e.g. not all materials are known how to be made easily or economically. If such technology can be discovered, we believe that the benefits can be more equitably distributed by making this research completely open-source and accessible to global researchers. Our use of open-source DFT solvers is one notable step towards this goal over previous approaches [8].
PP thanks Professors Maria Cameron, Howard Elman, Tom Goldstein, Ramani Duraiswami, Pratyush Tiwary, and Hong-Zhou Ye, as well as Dr. Ruiyu Wang for feedback on this project. PP also thanks the University of Maryland Institute for Advanced Computer Studies (UMIACS) for providing the computing infrastructure used in this project.
<sup>1</sup>We thank an anonymous reviewer for suggesting this way of interpreting our results.
[48] Richard Tran, Janice Lan, Muhammed Shuaibi, Brandon M. Wood, Siddharth Goyal, Abhishek Das, Javier Heras-Domingo, Adeesh Kolluru, Ammar Rizvi, Nima Shoghi, Anuroop Sriram, Félix Therrien, Jehad Abed, Oleksandr Voznyy, Edward H. Sargent, Zachary Ulissi, and C. Lawrence Zitnick. The open catalyst 2022 (OC22) dataset and challenges for oxide electrocatalysts. ACS Catalysis, 13(5):3066-3084, feb 2023. doi: 10.1021/acscatal.2c05426. URL https://doi.org/10.1021/2Facscatal.2c05426.
[49] Martin Uhrin, Sebastiaan P Huber, Jusong Yu, Nicola Marzari, and Giovanni Pizzi. Workflows in aiida: Engineering a high-throughput, event-based engine for robust and modular computational workflows. Computational Materials Science, 187:110086, 2021.
[50] Marin Vlastelica, Michal Rolinek, and Georg Martius. Neuro-algorithmic policies enable fast combinatorial generalization. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pages 10575-10585. PMLR, 18-24 Jul 2021. URL https://proceedings.mlr.press/v139/vlastelica21a.html.
[51] Tian Xie and Jeffrey C. Grossman. Crystal graph convolutional neural networks for an accurate and interpretable prediction of material properties. Phys. Rev. Lett., 120:145301, Apr 2018. doi: 10.1103/ PhysRevLett.120.145301. URL https://link.aps.org/doi/10.1103/PhysRevLett.120.145301.
[52] Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. When and why vision-language models behave like bags-of-words, and what to do about it?, 2023.
[53] Leonardo Zepeda-Núñez, Yixiao Chen, Jiefu Zhang, Weile Jia, Linfeng Zhang, and Lin Lin. Deep Density: Circumventing the Kohn-Sham equations via symmetry preserving neural networks. Journal of Computational Physics, 443:110523, October 2021. ISSN 0021-9991. doi: 10.1016/j.jcp.2021.110523. URL https://www.sciencedirect.com/science/article/pii/S0021999121004186.
[54] C. Lawrence Zitnick, Lowik Chanussot, Abhishek Das, Siddharth Goyal, Javier Heras-Domingo, Caleb Ho, Weihua Hu, Thibaut Lavril, Aini Palizhati, Morgane Riviere, Muhammed Shuaibi, Anuroop Sriram, Kevin Tran, Brandon Wood, Junwooong Yoon, Devi Parikh, and Zachary Ulissi. An Introduction to Electrocatalyst Design using Machine Learning for Renewable Energy Storage. arXiv:2010.09435 [cond-mat], October 2020. URL http://arxiv.org/abs/2010.09435.arXiv:2010.09435.
[55] C. Lawrence Zitnick, Abhishek Das, Adeesh Kolluru, Janice Lan, Muhammed Shuaibi, Anuroop Sriram, Zachary Ward Ulissi, and Brandon M Wood. Spherical channels for modeling atomic interactions. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022. URL https://openreview.net/forum?id=5Z3GURcqwT.