<figure><img src="image_18.png" /><figcaption>Figure 15: **Comparison of datasets.** The same models trained at temperature $T = 0.44$ on each dataset.</figcaption></figure>

By contrast, in our model we do not compute invariants after one layer, we keep equivariant features and let them interact over multiple layers in order to increase the expressivity.

Although this architecture qualitatively reproduces these expert features, for a quantitative match one would need to use a much larger cutoff radius $d_c = 5$ for building the graph, and a maximum rotational order of $l_{max} = 12$.

## D Shiba vs Bapst dataset comparison

In Bapst's dataset [36], a timescale actually corresponds to a fixed value of the self-intermediate scattering function $F_k(t)$, so that different samples are measured at slightly different times. Equilibration is performed under an NPT thermostat, i.e. at constant pressure and temperature, i.e. the volume is varying (and thus the density as well).

In Shiba's dataset [38], equilibration is performed at constant volume and temperature (NVT) so that the density is exactly $\rho = 1.2$ in all samples and at all temperatures. Furthermore, sampling of the trajectories is performed at fixed times, not fixed $F_k(t)$.

In Figure 15 we report the performance of our main model (thermal positions + $E_{pot}(IS)$ inputs) for these two Kob-Anderson 3D datasets. Performances are shifted in time but otherwise rather comparable (a bit better on Shiba's dataset).

## References

[1] H. Tong and H. Tanaka, *Structural order as a genuine control parameter of dynamics in simple glass formers*, Nat. Commun. **10**, 5596 (2019), doi:10.1038/s41467-019-13606-3.

[2] M. Lerbinger, A. Barbot, D. Vandembroucq and S. Patinet, *Relevance of shear transformations in the relaxation of supercooled liquids*, Phys. Rev. Lett. **129**, 195501 (2022), doi:10.1103/PhysRevLett.129.195501.

[3] J. Rottler, S. S. Schoenholz and A. J. Liu, *Predicting plasticity with soft vibrational modes: From dislocations to glasses*, Phys. Rev. E **89**, 042304 (2014), doi:10.1103/PhysRevE.89.042304.

[4] S. Golde, T. Palberg and H. J. Schöpe, *Correlation between dynamical and structural heterogeneities in colloidal hard-sphere suspensions*, Nat. Phys. **12**, 712 (2016), doi:10.1038/nphys3709.
[5] H. Tong and H. Tanaka, *Revealing hidden structural order controlling both fast and slow glassy dynamics in supercooled liquids*, Phys. Rev. X **8**, 011041 (2018), doi:10.1103/PhysRevX.8.011041.

[6] H. Tanaka, H. Tong, R. Shi and J. Russo, *Revealing key structural features hidden in liquids and glasses*, Nat. Rev. Phys. **1**, 333 (2019), doi:10.1038/s42254-019-0053-3.

[7] M. Lerbinger, *Local shear rearrangements in glassy systems: From micromechanics to structural relaxation in supercooled liquids*, PhD thesis, Université Paris Sciences et Lettres, Paris, France (2020).

[8] D. Coslovich, *Locally preferred structures and many-body static correlations in viscous liquids*, Phys. Rev. E **83**, 051505 (2011), doi:10.1103/PhysRevE.83.051505.

[9] A. S. Keys, C. R. Iacovella and S. C. Glotzer, *Characterizing structure through shape matching and applications to self-assembly*, Annu. Rev. Condens. Matter Phys. **2**, 263 (2011), doi:10.1146/annurev-conmatphys-062910-140526.

[10] C. P. Royall and W. Kob, *Locally favoured structures and dynamic length scales in a simple glass-former*, J. Stat. Mech.: Theory Exp. 024001 (2017), doi:10.1088/1742-5468/aa4e92.

[11] F. Turci, G. Tarjus and C. P. Royall, *From glass formation to icosahedral ordering by curving three-dimensional space*, Phys. Rev. Lett. **118**, 215501 (2017), doi:10.1103/PhysRevLett.118.215501.

[12] C. P. Royall, F. Turci and T. Speck, *Dynamical phase transitions and their relation to structural and thermodynamic aspects of glass physics*, J. Chem. Phys. **153**, 090901 (2020), doi:10.1063/5.0006998.

[13] E. Boattini, S. Marín-Aguilar, S. Mitra, G. Foffi, F. Smallenburg and L. Filion, *Autonomously revealing hidden local structures in supercooled liquids*, Nat. Commun. **11**, 5479 (2020), doi:10.1038/s41467-020-19286-8.

[14] J. Paret, R. L. Jack and D. Coslovich, *Assessing the structural heterogeneity of supercooled liquids through community inference*, J. Chem. Phys. **152**, 144502 (2020), doi:10.1063/5.0004732.

[15] D. Coslovich, R. L. Jack and J. Paret, *Dimensionality reduction of local structure in glassy binary mixtures*, J. Chem. Phys. **157**, 204503 (2022), doi:10.1063/5.0128265.

[16] P. Ronhovde, S. Chakrabarty, D. Hu, M. Sahu, K. K. Sahu, K. F. Kelton, N. A. Mauro and Z. Nussinov, *Detecting hidden spatial and spatio-temporal structures in glasses and complex physical systems by multiresolution network clustering*, Eur. Phys. J. E **34**, 105 (2011), doi:10.1140/epje/i2011-11105-9.

[17] P. Ronhovde, S. Chakrabarty, D. Hu, M. Sahu, K. K. Sahu, K. F. Kelton, N. A. Mauro and Z. Nussinov, *Detection of hidden structures for arbitrary scales in complex physical systems*, Sci. Rep. **2**, 329 (2012), doi:10.1038/srep00329.

[18] D. Richard et al., *Predicting plasticity in disordered solids from structural indicators*, Phys. Rev. Mater. **4**, 113609 (2020), doi:10.1103/PhysRevMaterials.4.113609.

[19] R. Candelier, A. Widmer-Cooper, J. K. Kummerfeld, O. Dauchot, G. Biroli, P. Harrowell and D. R. Reichman, *Spatiotemporal hierarchy of relaxation events, dynamical heterogeneities, and structural reorganization in a supercooled liquid*, Phys. Rev. Lett. **105**, 135702 (2010), doi:10.1103/PhysRevLett.105.135702.
[20] L. Berthier, G. Biroli, J.-P. Bouchaud, L. Cipelletti and W. van Saarloos, Dynamical heterogeneities in glasses, colloids, and granular media, Oxford University Press, Oxford, UK, ISBN 9780199691470 (2011), doi:10.1093/acprof:oso/9780199691470.001.0001.

[21] R. N. Chacko, F. P. Landes, G. Biroli, O. Dauchot, A. J. Liu and D. R. Reichman, Elastoplasticity mediates dynamical heterogeneity below the mode coupling temperature, Phys. Rev. Lett. 127, 048002 (2021), doi:10.1103/PhysRevLett.127.048002.

[22] L. Berthier and G. Biroli, Glasses and aging, a statistical mechanics perspective on, in Encyclopedia of complexity and systems science, Springer, New York, USA, ISBN 9780387758886 (2009), doi:10.1007/978-0-387-30440-3_248.

[23] S. S. Schoenholz, A. J. Liu, R. A. Riggleman and J. Rottler, Understanding plastic deformation in thermal glasses from single-soft-spot dynamics, Phys. Rev. X 4, 031014 (2014), doi:10.1103/PhysRevX.4.031014.

[24] E. D. Cubuk, S. S. Schoenholz, J. M. Rieser, B. D. Malone, J. Rottler, D. J. Durian, E. Kaxiras and A. J. Liu, Identifying structural flow defects in disordered solids using machine-learning methods, Phys. Rev. Lett. 114, 108001 (2015), doi:10.1103/PhysRevLett.114.108001.

[25] S. S. Schoenholz, E. D. Cubuk, E. Kaxiras and A. J. Liu, Relationship between local structure and relaxation in out-of-equilibrium glassy systems, Proc. Natl. Acad. Sci. 114, 263 (2016), doi:10.1073/pnas.1610204114.

[26] S. S. Schoenholz, E. D. Cubuk, D. M. Sussman, E. Kaxiras and A. J. Liu, A structural approach to relaxation in glassy liquids, Nat. Phys. 12, 469 (2016), doi:10.1038/nphys3644.

[27] T. A. Sharp, S. L. Thomas, E. D. Cubuk, S. S. Schoenholz, D. J. Srolovitz and A. J. Liu, Machine learning determination of atomic dynamics at grain boundaries, Proc. Natl. Acad. Sci. 115, 10943 (2018), doi:10.1073/pnas.1807176115.

[28] E. D. Cubuk et al., Structure-property relationships from universal signatures of plasticity in disordered solids, Science 358, 1033 (2017), doi:10.1126/science.aai8830.

[29] D. M. Sussman, S. S. Schoenholz, E. D. Cubuk and A. J. Liu, Disconnecting structure and dynamics in glassy thin films, Proc. Natl. Acad. Sci. 114, 10601 (2017), doi:10.1073/pnas.1703927114.

[30] F. P. Landes, G. Biroli, O. Dauchot, A. J. Liu and D. R. Reichman, Attractive versus truncated repulsive supercooled liquids: The dynamics is encoded in the pair correlation function, Phys. Rev. E 101, 010602 (2020), doi:10.1103/PhysRevE.101.010602.

[31] I. Tah, S. A. Ridout and A. J. Liu, Fragility in glassy liquids: A structural approach based on machine learning, J. Chem. Phys. 157, 124501 (2022), doi:10.1063/5.0099071.

[32] G. Zhang, H. Xiao, E. Yang, R. J. S. Ivancic, S. A. Ridout, R. A. Riggleman, D. J. Durian and A. J. Liu, Structuro-elasto-plasticity model for large deformation of disordered solids, Phys. Rev. Res. 4, 043026 (2022), doi:10.1103/PhysRevResearch.4.043026.

[33] S. Kearnes, K. McCloskey, M. Berndl, V. Pande and P. Riley, Molecular graph convolutions: Moving beyond fingerprints, J. Comput.-Aided Mol. Des. 30, 595 (2016), doi:10.1007/s10822-016-9938-8.
[34] C. Chen, W. Ye, Y. Zuo, C. Zheng and S. P. Ong, *Graph networks as a universal machine learning framework for molecules and crystals*, Chem. Mater. **31**, 3564 (2019), doi:10.1021/acs.chemmater.9b01294.

[35] W. Hu, M. Shuaibi, A. Das, S. Goyal, A. Sriram, J. Leskovec, D. Parikh and C. L. Zitnick, *ForceNet: A graph neural network for large-scale quantum calculations*, (arXiv preprint) doi:10.48550/arXiv.2103.01436.

[36] V. Bapst et al., *Unveiling the predictive power of static structure in glassy systems*, Nat. Phys. **16**, 448 (2020), doi:10.1038/s41567-020-0842-8.

[37] E. Boattini, F. Smallenburg and L. Filion, *Averaging local structure to predict the dynamic propensity in supercooled liquids*, Phys. Rev. Lett. **127**, 088007 (2021), doi:10.1103/PhysRevLett.127.088007.

[38] H. Shiba, M. Hanai, T. Suzumura and T. Shimokawabe, *BOTAN: BOnD TAargeting Network for prediction of slow glassy dynamics by machine learning relative motion*, J. Chem. Phys. **158**, 084503 (2023), doi:10.1063/5.0129791.

[39] G. Jung, G. Biroli and L. Berthier, *Predicting dynamic heterogeneity in glass-forming liquids by physics-inspired machine learning*, Phys. Rev. Lett. **130**, 238202 (2023), doi:10.1103/PhysRevLett.130.238202.

[40] R. M. Alkemade, F. Smallenburg and L. Filion, *Improving the prediction of glassy dynamics by pinpointing the local cage*, J. Chem. Phys. **158**, 134512 (2023), doi:10.1063/5.0144822.

[41] X. Jiang, Z. Tian, K. Li and W. Hu, *A geometry-enhanced graph neural network for learning the smoothness of glassy dynamics from static structure*, J. Chem. Phys. **159**, 144504 (2023), doi:10.1063/5.0162463.

[42] N. Thomas, T. Smidt, S. Kearnes, L. Yang, L. Li, K. Kohlhoff and P. Riley, *Tensor field networks: Rotation- and translation-equivariant neural networks for 3D point clouds*, (arXiv preprint) doi:10.48550/arXiv.1802.08219.

[43] S. Batzner, A. Musaelian, L. Sun, M. Geiger, J. P. Mailoa, M. Kornbluth, N. Molinari, T. E. Smidt and B. Kozinsky, *E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials*, Nat. Commun. **13**, 2453 (2022), doi:10.1038/s41467-022-29939-5.

[44] M. Weiler, M. Geiger, M. Welling, W. Boomsma and T. S. Cohen, *3d steerable cnns: Learning rotationally equivariant features in volumetric data*, in *Advances in neural information processing systems* 31, Curran Associates, New York, USA, ISBN 9781510884472 (2018).

[45] J. Brandstetter, R. Hesselink, E. van der Pol, E. J. Bekkers and M. Welling, *Geometric and physical quantities improve E(3) equivariant message passing*, (arXiv preprint) doi:10.48550/arXiv.2110.02905.

[46] I. Batatia, D. P. Kovacs, G. Simm, C. Ortner and G. Csányi, *Mace: Higher order equivariant message passing neural networks for fast and accurate force fields*, in *Advances in neural information processing systems* 35, Curran Associates, New York, USA, ISBN 9781713871088 (2022).

[47] Y.-L. Liao, B. Wood, A. Das and T. Smidt, *EquiformerV2: Improved equivariant transformer for scaling to higher-degree representations*, (arXiv preprint) doi:10.48550/arXiv.2306.12059.
[48] D. Ruhe, J. Brandstetter and P. Forré, Clifford group equivariant neural networks, (arXiv preprint) doi:10.48550/arXiv.2305.11141.

[49] D. Ruhe, J. K. Gupta, S. de Keninck, M. Welling and J. Brandstetter, Geometric Clifford algebra networks, (arXiv preprint) doi:10.48550/arXiv.2302.06594.

[50] L. Berthier and R. L. Jack, Structure and dynamics of glass formers: Predictability at large length scales, Phys. Rev. E 76, 041509 (2007), doi:10.1103/PhysRevE.76.041509.

[51] L. Berthier, G. Biroli, J.-P. Bouchaud, W. Kob, K. Miyazaki and D. R. Reichman, Spontaneous and induced dynamic fluctuations in glass formers. I. General results and dependence on ensemble and dynamics, J. Chem. Phys. 126, 184503 (2007), doi:10.1063/1.2721554.

[52] G. Jung et al., Roadmap on machine learning glassy liquids, (arXiv preprint) doi:10.48550/arXiv.2311.14752.

[53] R. Kondor, Z. Lin and S. Trivedi, Clebsch-Gordan nets: A fully Fourier space spherical convolutional neural network, in Advances in neural information processing systems 31, Curran Associates, New York, USA, ISBN 9781510884472 (2018).

[54] S. Batzner, A. Musaelian, L. Sun, M. Geiger, J. P. Mailoa, M. Kornbluth, N. Molinari, T. E. Smidt and B. Kozinsky, E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials, Nat. Commun. 13, 2453 (2022), doi:10.1038/s41467-022-29939-5.

[55] K. He, X. Zhang, S. Ren and J. Sun, Deep residual learning for image recognition, in IEEE conference on computer vision and pattern recognition, Las Vegas, USA (2016), doi:10.1109/CVPR.2016.90.

[56] S. Ioffe and C. Szegedy, Batch normalization: Accelerating deep network training by reducing internal covariate shift, in Proceedings of the 32nd international conference on machine learning, Lille, France (2015).

[57] M. Geiger et al., Euclidean neural networks: e3nn, Zenodo (2022), doi:10.5281/zenodo.6459381.

[58] Y.-L. Liao and T. Smidt, Equiformer: Equivariant graph attention transformer for 3D atomic graphs, (arXiv preprint) doi:10.48550/arXiv.2206.11990.

[59] M. J. Hutchinson, C. Le Lan, S. Zaidi, E. Dupont, Y. W. Teh and H. Kim, Lietransformer: Equivariant self-attention for Lie groups, in International conference on machine learning, Online (2021).

[60] F. Fuchs, D. Worrall, V. Fischer and M. Welling, SE(3)-transformers: 3d roto-translation equivariant attention networks, in Advances in neural information processing systems 33, Curran Associates, New York, USA, ISBN 9781713829546 (2020).

[61] C. Scalliet, B. Guiselin and L. Berthier, Thirty milliseconds in the life of a supercooled liquid, Phys. Rev. X 12, 041028 (2022), doi:10.1103/PhysRevX.12.041028.

[62] D. Coslovich and G. Pastore, Understanding fragility in supercooled Lennard-Jones mixtures. I. Locally preferred structures, J. Chem. Phys. 127, 124504 (2007), doi:10.1063/1.2773716.

[63] Y. Bengio, Deep learning of representations for unsupervised and transfer learning, in ICML workshop on unsupervised and transfer learning, Bellevue, USA (2012).
[64] Y. Wang, Q. Yao, J. T. Kwok and L. M. Ni, Generalizing from a few examples, ACM Comput. Surv. 53, 1 (2020), doi:10.1145/3386252.

[65] E. Perez, F. Strub, H. De Vries, V. Dumoulin and A. Courville, *FiLM: Visual reasoning with a general conditioning layer*, Proc. AAAI Conf. Artif. Intell. **32** (2018), doi:10.1609/aaai.v32i1.11671.

[66] J. K. Gupta and J. Brandstetter, *Towards multi-spatiotemporal-scale generalized PDE modeling*, (arXiv preprint) doi:10.48550/arXiv.2209.15616.

[67] N. Oyama, S. Koyama and T. Kawasaki, *What do deep neural networks find in disordered structures of glasses?*, Front. Phys. **10** (2023), doi:10.3389/fphy.2022.1007861.

[68] S. Liu, H. Wang, W. Liu, J. Lasenby, H. Guo and J. Tang, *Pre-training molecular graph representation with 3D geometry*, (arXiv preprint) doi:10.48550/arXiv.2110.07728.

[69] X. Fang, L. Liu, J. Lei, D. He, S. Zhang, J. Zhou, F. Wang, H. Wu and H. Wang, *Geometry-enhanced molecular representation learning for property prediction*, Nat. Mach. Intell. **4**, 127 (2022), doi:10.1038/s42256-021-00438-4.

[70] Z. Zhang, M. Xu, A. Jamasb, V. Chenthamarakshan, A. Lozano, P. Das and J. Tang, *Protein representation learning by geometric structure pretraining*, (arXiv preprint) doi:10.48550/arXiv.2203.06125.

[71] M. M. Bronstein, J. Bruna, T. Cohen and P. Veličković, *Geometric deep learning: Grids, groups, graphs, geodesics, and gauges*, (arXiv preprint) doi:10.48550/arXiv.2104.13478.

[72] M. Fey and J. E. Lenssen, *Fast graph representation learning with PyTorch geometric*, (arXiv preprint) doi:10.48550/arXiv.1903.02428.