SlowGuess/NeurIPS_2023_OCR huggingface dataset:
However, we argue that the invariant pose update limits the model's capacity since it is regardless of the geometric information of the system in the prediction phase. Considering we treat fragments as rigid 3D graphs whose local coordinates will not be changed, it is natural to take inspiration from rigid body mechanics to predict their poses.
The Newton-Euler equations describe a rigid body's combined translational and rotational dynamics. In the Center of Mass (CoM) frame, it can be expressed as the following matrix form:
$$
\left( \begin{array}{c} \boldsymbol {F} \\ \boldsymbol {\tau} \end{array} \right) = \left( \begin{array}{c c} m \mathbf {I} & 0 \\ 0 & \mathbf {I} _ {c} \end{array} \right) \left( \begin{array}{l} d \boldsymbol {v} / d t \\ d \boldsymbol {\omega} / d t \end{array} \right) + \left( \begin{array}{c} 0 \\ \boldsymbol {\omega} \times \mathbf {I} _ {c} \boldsymbol {\omega} \end{array} \right) \tag {13}
$$
where  $\pmb{F}$  and  $\pmb{\tau}$  are the total force and torque acting on CoM,  $\mathbf{v}$  and  $\omega$  are the velocity of CoM and the angular velocity around CoM,  $m$  and  $\mathbf{I}_c$  is the mass and inertia matrix of the rigid body, which are constant for a given rigid body.
In our fragment pose prediction module, the outputs of the neural network act as the forces  $\pmb{f}_i$  on each fragment atom  $i$ , with which we can compute the total force  $\pmb{F}$  and torque  $\pmb{\tau}$  for each fragment:
$$
\boldsymbol {f} _ {i} = \sum_ {j \notin \mathcal {V} _ {F _ {c}}} \left(\boldsymbol {x} _ {i} - \boldsymbol {x} _ {j}\right) \phi_ {f} \left(\boldsymbol {h} _ {i}, \boldsymbol {h} _ {j}, \boldsymbol {e} _ {i j} \| \boldsymbol {x} _ {i} - \boldsymbol {x} _ {j} \|\right) \tag {14}
$$
$$
\boldsymbol {F} _ {c} = \sum_ {i \in \mathcal {V} _ {F _ {c}}} \boldsymbol {f} _ {i}, \quad \boldsymbol {\tau} _ {c} = \sum_ {i \in \mathcal {V} _ {F _ {c}}} \left(\boldsymbol {x} _ {i} - \boldsymbol {p} _ {c}\right) \times \boldsymbol {f} _ {i} \tag {15}
$$
where  $c = 1$  or 2, corresponding to two fragments.
We assume the system is stationary at each discrete time step, i.e.  $\omega = 0$  and  $\pmb{v} = 0$ . Thus, the Newton-Euler equations 13 can be simplified as  $\pmb{\tau} = \mathbf{I}_c\frac{d\pmb{\omega}}{dt}$  and  $\pmb{F} = m\frac{dv}{dt}$ . For a short enough time period  $\Delta t$ , we have the velocity and angular velocity of the fragment as  $\omega = \mathbf{I}_c^{-1}\pmb{\tau}_c\Delta t$  and  $\pmb{v} = \frac{1}{m}\mathbf{F}\Delta t$ . Assuming each atom in the fragment has the unit mass and absorbing the time period  $\Delta t$  into  $\pmb{F}_c$  and  $\pmb{\tau}_c$ , the fragment pose will be updated as follows:
$$
\hat {\boldsymbol {p}} _ {c, 0} = \boldsymbol {p} _ {t} + \frac {1}{| \mathcal {V} _ {F _ {c}} |} \boldsymbol {F} _ {c} \quad \hat {\boldsymbol {R}} _ {c, 0} = R _ {\omega} \left(\mathbf {I} _ {c} ^ {- 1} \boldsymbol {\tau} _ {c}\right) \boldsymbol {R} _ {t} \tag {16}
$$
where  $|\mathcal{V}_{F_c}|$  denotes the number of atoms in  $F_{c}$  and  $R_{\omega}$  denotes the operation of converting a vector in  $\mathfrak{so}(3)$  to a rotation matrix in  $SO(3)$  (See Appendix for details).
It can be seen that the predicted force and torque take advantage of the geometric information and are equivariant to the global rigid transformation. Moreover, we also show that the predicted fragment poses are equivariant to the global rigid transformation and are independent of the choices of local coordinate systems. The final training loss is the weighted sum of MSE loss of linker atom positions and fragment center positions, a discrepancy loss of rotation matrix, and KL-divergence of linker atom types and bond types. Please see the proof, more training and modeling details, and the complete training/sampling algorithm in Appendix.
3.4 Constrained Generation with Guided Sampling
The LinkerNet we introduced so far is applicable in the case where the fragment poses are fully unknown. However, in real scenarios, we have some prior knowledge, such as the fragment distance should be in a reasonable range, the anchors (the atom on the fragment to connect with the linker) should be selected from a candidate set, etc. To incorporate these constraints, we leverage the idea of classifier guidance [10] to perform guided sampling. In our formulated problem, for a condition  $y$ , the diffusion score can be modified as:
$$
\nabla \log p \left(\mathcal {G} _ {M _ {t}} | y\right) = \nabla \log p \left(\mathcal {G} _ {M _ {t}}\right) + \nabla \log p (y | \mathcal {G} _ {M _ {t}}). \tag {17}
$$
For the fragment distance constraint, we assume the expected distance range is  $[d_{\min}, d_{\max}]$ , then we have the following guidance term:
$$
- \nabla_ {\mathbf {p}} \max  \left(\| \boldsymbol {p} _ {1} - \boldsymbol {p} _ {2} \| - d _ {\max }, 0\right) + \max  \left(d _ {\min } - \| \boldsymbol {p} _ {1} - \boldsymbol {p} _ {2} \|, 0\right). \tag {18}
$$
To generate a complete and valid molecule, at least one atom from the candidate anchor set should form a bond with one linker atom, and the atoms outside the anchor set should not form bonds with any linker atoms. These principles can be formulated as the following anchor proximity guidance:
$$
- \nabla_ {\mathbf {R}} \max  \left(d _ {a} - r _ {\max }, 0\right) + \max  \left(r _ {\min } - d _ {a}, 0\right) + \max  \left(r _ {\max } - d _ {n a}, 0\right). \tag {19}
$$
where  $\mathcal{A}$  is the candidate anchor set,  $d_{a} = \min_{i\in \mathcal{V}_{\mathcal{G}_{L}},j\in \mathcal{A}}\| \pmb{x}_{i} - \pmb{x}_{j}\|$  and  $d_{na} = \min_{i\in \mathcal{V}_{\mathcal{G}_L},j\in \mathcal{V}_{\mathcal{G}_F}\backslash \mathcal{A}}\| \pmb{x}_i - \pmb{x}_j\|$ .  $r_{\mathrm{min}}$  and  $r_{\mathrm{max}}$  denote the minimum and maximum of a bond length, which are set to  $1.2\AA$  and  $1.9\AA$  in practice. Besides the soft constraint on anchor proximity, we can also set a hard constraint by applying a bond mask during the sampling phase, i.e. we adopt  $\mathbf{b}_L\in \mathbb{R}^{N_L\times |\mathcal{A}|\times N_b}$  instead of  $\mathbb{R}^{N_L\times N\times N_b}$
We mainly conduct experiments in two settings: unconstrained generation and constrained generation. In the unconstrained generation setting, only 3D fragment graphs  $\mathcal{G}_{F_1},\mathcal{G}_{F_2}$  are known, and the goal is to validate whether the model can co-design linker and fragment poses to generate molecules  $\mathcal{G}_M$  with low-energy and other desired properties. In the constrained generation setting, the candidate anchor set  $\mathcal{A}$  on each fragment is also known besides  $\mathcal{G}_{F_1},\mathcal{G}_{F_2}$ , and we set a fragment center distance constraint. The goal is to validate whether the model can perform well in a more realistic scenario. We describe the general setting about data, baselines, and evaluation metrics as follows and the detailed task-specific settings in the corresponding subsections.
Data We use a subset of ZINC [43] for the unconstrained generation. Following [22, 20, 21], the reference conformation for each molecule is obtained by running 20 times MMFF [16] optimization using RDKit [1] and selecting the one with lowest energy. We use the same procedure as [21] to create fragments-linker pairs and randomly split the dataset, which results in a training/validation/test set with  $438,610 / 400 / 400$  examples. For the constrained generation, we use PROTAC-DB [45], a database collecting PROTACs from the literature or calculated by programs. The same procedure is applied to obtain reference conformations and create data pairs. We select 10 different warheads as the test set (43 examples) and the remaining as the training set (992 examples).
Table 1: Unconstrained generation results on ZINC. (*) denotes additional anchor information is utilized and thus scores are not directly comparable.  $(++)$  denotes a huge number  $(>100\mathrm{k}$  in energy). The mean and standard deviation values are reported by running the sampling procedure 3 times with different random seeds.
Method	Valid,%	Unique,%	Novel,%	Rec,%	QED (↑)	SA (↓)	Emin(↓)	RMSD (↓)	EL(↓)	ΔEL(↓)
DeLinker	96.8 ± 0.2	43.5 ± 0.4	43.3 ± 0.2	55.8 ± 1.2	0.61 ± 0.0	3.13 ± 0.0	-	-	-	-
3DLinker	40.3 ± 0.1	53.6 ± 0.6	47.4 ± 2.7	43.2 ± 0.9*	0.55 ± 0.00	3.08 ± 0.00	++	2.42 ± 0.01	5178.2 ± 108.5	++
DiffLinker	48.7 ± 0.1	90.1 ± 3.5	99.5 ± 0.0	0.0	0.59 ± 0.00	7.25 ± 0.00	179.2 ± 6.7	1.92 ± 0.00	216.8 ± 1.0	93.3 ± 0.4
Ours	83.1 ± 0.0	14.8 ± 0.1	11.4 ± 0.1	24.4 ± 0.1	0.70 ± 0.00	3.01 ± 0.00	32.7 ± 1.2	1.44 ± 0.00	54.8 ± 10.5	67.9 ± 1.3
Baselines For benchmarking, we compare our model with three baselines: DeLinker [22], 3DLinker [20] and DiffLinker [21]. DeLinker is a 2D graph generative model, while 3DLinker and DiffLinker are 3D generative models with VAE and diffusion models, respectively. Since there is no existing generative model to perform the fragment poses and linker co-design, we randomly sample fragment rotations and add noise to fragment center positions. Then, the noisy fragments are fed to these models to generate linkers.
Evaluation Metrics For ZINC and PROTAC-DB dataset, we generate 250 and 100 samples per fragment pair respectively for further evaluation. We evaluate the generated molecules on both 2D graphs and 3D conformations. For 2D metrics, we report the standard ones including validity, uniqueness and novelty [4], the recovery rate (the percentage of generated molecules that can successfully recover the reference molecules), and property-related metrics drug-likeness (QED) [3] and synthetic accessibility (SA) [13]. To evaluate 3D conformations, we first perform MMFF [16] optimization, and report the average minimum energy of generated molecules per fragment pair before optimization as  $E_{\mathrm{min}}$ , and the average Root Mean Square Deviation (RMSD) of the molecule coordinates before and after optimization as RMSD.  $E_{\mathrm{min}}$  indicates the best quality of the overall generated conformations and RMSD indicates the gap between generated conformations and the best possible ones. To further investigate the model's performance in generating linker and fragment poses respectively, we perform another constrained MMFF optimization by fixing the atoms in fragments. Then, we report the average median energy after optimization as  $E_L$  and the average median energy difference before and after optimization as  $\Delta E_L$ . Since fragment atoms are fixed, the optimization will fix the unrealistic conformation inside the linker and thus a lower  $E_L$  indicates the better fragment poses, and a lower  $\Delta E_L$  means less adjustment on linker atoms and thus indicates the better linker conformation.
4.2 Unconstrained Generation Results
For baseline models, we randomly sample fragment rotations and add a Gaussian noise on the fragment distance (stddev = 0.1 distance). We filter out clashing initial fragments and feed each baseline with 250 valid initial fragment poses per fragment pair. We fix the number of linker atoms to be the same as the reference molecule. For DeLinker and 3DLinker baselines, anchor atoms are also provided to utilize their published models while they are unavailable for DiffLinker and our model.
From Table 1, we can first see that our model can generate much more valid molecules compared to other 3D linker generative models (3DLinker and DiffLinker). Although our model's uniqueness and novelty scores are lower than other baselines, we believe that is because the chemical linker design space to form a low-energy molecule is limited. Our model achieves a recovery rate of  $24.5\%$ . In contrast, DiffLinker achieves zero recovery rate since it can not co-design fragment poses, while DeLinker / 3DLinker utilizes the anchor information and thus achieve a higher recovery rate. In terms of QED and SA, our model clearly outperforms other models, which indicates our generated linkers are more realistic. For 3D-related evaluation metrics, our model can achieve much lower energy and RMSD compared to other baselines, which justifies the effectiveness of our modeling approach.
Since we argue that our physics-inspired fragment pose prediction module is more effective in Sec. 3.3, we conduct an ablation study (See Table 2) on it to further investigate the role of each part. We take equation 12 as the counterpart of the Newton-Euler equations-based update formula (equation 16). Firstly, we find the fragment translation prediction based on Newton equation is more effective than predicting the change of translation in the local frame. The latter approach will result in severe drift (the fragments are far away from each other) during the sampling phase, as reflected in
Table 2: Ablation study on the fragment pose prediction module.
Pose Pred	Valid, %	Rec, %	Emin(↓)	RMSD(↓)	EL(↓)	ΔEL(↓)
Newton	Euler
(a)	81.1	47.3	++	++	++	++
(b)	✓	97.2	70.8	180.4	2.22	2052.2	++
(c)	✓	81.6	20.3	++	++	++	++
(d)	✓	✓	83.1	24.5	32.2	1.44	49.3	67.1
Figure 4: Rotation loss.
Figure 5: Visualization of reference molecules and molecules generated by LinkerNet and DiffLinker.
the huge energy and RMSD in (a) and (c), even though the model can learn to connect one linker atom with each fragment to achieve reasonable scores in 2D metrics such as validity and recovery rate. Secondly, we can see the fragment rotation prediction based on Euler equation (d) can achieve lower energy and RMSD compared to its counterpart (b), indicating our design can boost the model's capacity and make more accurate rotation predictions. We further plot the rotation training loss in Fig. 4, which confirms our argument since the rotation loss decreases as expected with Euler-equation rotation prediction while it almost remains unchanged with the counterpart design.
4.3 Constrained Generation Results
In the constrained generation setting, we mimic the real scenario in the PROTAC linker design. Since two fragments also need to bind the protein of interest and E3 ligase separately, the possible anchor can only be chosen from a subset of the fragment atoms. In addition, the linker length has a critical effect on the PROTAC's selectivity, and we usually need to restrict the range of fragment distance. To include these constraints, we take the atoms within two hops of the real anchor atom as the imaginary candidate anchor atom set for each fragment. We set the fragment center distance constraint as  $[0.8d, 1.2d]$ , where  $d$  is the fragment distance in the reference molecule.
We generate samples with guided sampling and hard bond mask as described in 3.4. For baselines, the fragment distance is uniformly sampled within the constraint, and random anchors from candidate anchor sets are provided. The number of linker atoms is sampled according to the approach proposed in their original paper. Since the public pretrained 3DLinker model can not support the linker generation to large molecular graphs such as PROTACs, we do not include it for comparison.
Table 3 shows the constrained generation results on PROTAC-DB. Our model achieves a higher validity and recovery rate, and outperforms other baselines in three of 3D metrics ( $E_{\mathrm{min}}$ , RMSD and  $E_L$ ) with a clear margin. DiffLinker has a lower energy difference before and after linker force field optimization than ours, which indicates molecules generated by DiffLinker has better geometry inside the linker. It makes sense since DiffLinker focuses on learning the linker distribution only, while our model is trained on a more complex fragment poses and linker co-design task.
Table 3: Constrained generation results on PROTAC-DB. The mean and standard deviation values are reported by running the sampling procedure 3 times with different random seeds.
Method	Valid,%	Unique,%	Novel,%	Rec,%	Emin(↓)	RMSD(↓)	EL(↓)	ΔEL(↓)
DeLinker	42.8 ± 0.4	89.9 ± 2.8	99.0 ± 0.1	0.0 ± 0.0	-	-	-	-
DiffLinker	24.0 ± 0.1	99.4 ± 0.0	98.9 ± 0.2	0.0 ± 0.0	416.2 ± 13.4	2.44 ± 0.05	501.0 ± 17.6	80.8 ± 4.6
Ours	55.5 ± 0.0	47.9 ± 11.6	41.4 ± 4.9	5.4 ± 1.4	113.7 ± 8.0	1.55 ± 0.02	29.9 ± 17.1	500.0 ± 99.1
Table 4: Ablation study on the guided sampling.
Guidance	Valid,%	DistSucc,%	Emin(↓)	RMSD(↓)	EL(↓)	ΔEL(↓)
Anchor	Distance
(a)	45.9	25.2	++	1.66	124.3	++
(b)	✓	53.2	25.4	1249.7	1.61	432.5	1495.0
(c)	✓	49.3	53.7	208.4	1.56	45.4	4971.0
(d)	✓	✓	55.5	52.6	115.6	1.54	18.2	610.7
To justify the effect of each guidance term, we conduct another ablation study as shown in Tab. 4. Without guided sampling, the model will generate unrealistic 3D linkers for some fragment pairs and result in a very large average energy  $E_{\mathrm{min}}$  and  $\Delta E_L$ . Anchor proximity guidance (b) and fragment distance (c) are both of benefit for meeting the constraints and achieving lower energy and RMSD. The combination of them (d) achieves the best result.
We introduce LinkerNet for 3D fragment poses and linker co-design, the first work which addresses this more general and challenging linker design task. The limitation of our model is that it does not directly incorporate fragment rotation constraints or explicitly consider the protein context in modeling. These aspects could be valuable directions for future research.
Reproducibility Statements The model implementation, experimental data and model checkpoints can be found here: https://github.com/guanjq/LinkerNet
Acknowledgement We thank all the reviewers for their feedbacks through out the review cycles of the manuscript. This work was supported by the National Key Research and Development Program of China grants 2022YFF1203100 and 2021YFF1201600
[38] Kathleen M Sakamoto, Kyung B Kim, Akiko Kumagai, Frank Mercurio, Craig M Crews, and Raymond J Deshaies. Protacs: Chimeric molecules that target proteins to the skp1-cullin-f box complex for ubiquitination and degradation. Proceedings of the National Academy of Sciences, 98(15):8554-8559, 2001.
[39] Arne Schneuing, Yuanqi Du, Charles Harris, Arian Jamasb, Ilia Igashov, Weitao Du, Tom Blundell, Pietro Lio, Carla Gomes, Max Welling, et al. Structure-based drug design with equivariant diffusion models. arXiv preprint arXiv:2210.13695, 2022.
[40] Stuart L Schreiber. The rise of molecular glues. Cell, 184(1):3-9, 2021.
[41] Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learning, pages 2256-2265. PMLR, 2015.
[42] Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems, 32, 2019.
[43] Teague Sterling and John J Irwin. Zinc 15-ligand discovery for everyone. Journal of chemical information and modeling, 55(11):2324-2337, 2015.
[44] Robert Ian Troup, Charlene Fallon, and Matthias GJ Baud. Current strategies for the design of protac linkers: a critical review. Exploration of Targeted Anti-tumor Therapy, 1:273-312, 2020.
[45] Gaoqi Weng, Xuanyan Cai, Dongsheng Cao, Hongyan Du, Chao Shen, Yafeng Deng, Qiaojun He, Bo Yang, Dan Li, and Tingjun Hou. Protac-db 2.0: an updated database of protacs. Nucleic Acids Research, 51(D1):D1367-D1372, 2023.
[46] Minkai Xu, Lantao Yu, Yang Song, Chence Shi, Stefano Ermon, and Jian Tang. Geodiff: A geometric diffusion model for molecular conformation generation. arXiv preprint arXiv:2203.02923, 2022.
[47] Yuyao Yang, Shuangjia Zheng, Shimin Su, Chao Zhao, Jun Xu, and Hongming Chen. Syntalinker: automatic fragment linking with deep conditional transformer neural networks. Chemical science, 11(31):8312-8322, 2020.
[48] Shuangjia Zheng, Youhai Tan, Zhenyu Wang, Chengtao Li, Zhiqing Zhang, Xu Sang, Hongming Chen, and Yuedong Yang. Accelerated rational protac design via deep learning and molecular simulations. Nature Machine Intelligence, 4(9):739-748, 2022.
A Exponential and Logarithmic Mapping between  $\mathfrak{so}(3)$  and  $SO(3)$
A rotation matrix has an associated axis-angle representation. The transformation between them relies upon the exponential and logarithmic mapping between the Lie algebra  $\mathfrak{so}(3)$  and  $SO(3)$ .
Following standard definitions [5], the logarithm of a rotation matrix  $\pmb{R}$  is:
$$
\log (\boldsymbol {R}) = \frac {\theta}{2 \sin \theta} \left(\boldsymbol {R} - \boldsymbol {R} ^ {T}\right), \tag {20}
$$
where  $\operatorname{Tr}(\pmb{R}) = 1 + 2\cos \theta$ . It can be shown that the logarithm of  $\pmb{R} \in SO(3)$  is a skew-symmetric matrix  $S \in \mathfrak{so}(3)$ :
$$
\boldsymbol {S} := \log (\boldsymbol {R}) = \left( \begin{array}{c c c} 0 & - v _ {z} & v _ {y} \\ v _ {z} & 0 & - v _ {x} \\ - v _ {y} & v _ {x} & 0 \end{array} \right), \tag {21}
$$
where  $\pmb{v} = [v_{x}, v_{y}, v_{z}]$  is the rotation axis and  $\theta = \| \pmb{v} \|_2$  is the rotation angle.
Correspondingly, the exponential of a skew-symmetric matrix is a rotation matrix:
$$
\boldsymbol {R} := \exp (\boldsymbol {S}) = \boldsymbol {I} + \frac {\sin \| \boldsymbol {v} \| _ {2}}{\| \boldsymbol {v} \| _ {2}} \boldsymbol {S} + \frac {1 - \cos \| \boldsymbol {v} \| _ {2}}{\| \boldsymbol {v} \| _ {2} ^ {2}} \boldsymbol {S} ^ {2}. \tag {22}
$$
Following [28], we diffuse the rotation matrix by scaling the angle of rotation along the geodesic from the identity, which can be done by logarithmic mapping the rotation matrix to values in  $\mathfrak{so}(3)$ , element-wise multiplying by a scalar, and exponentially mapping them back to  $SO(3)$ , i.e.  $\lambda (\gamma ,\mathbf{R}) = \exp (\gamma \log (\mathbf{R}))$ . We follow the same way as [31] to pre-compute and cache discretized angle distribution to draw samples efficiently.
B Proof of Equivariance
B.1 Equivariance w.r.t Global SE(3)-Transformation
We denote the global SE(3)-transformation as  $T_{g}$ , which can be written explicitly as  $\pmb{x}' = T_g(\pmb{x}) = R_g\pmb{x} + \pmb{b}$ , where  $R_{g}\in \mathbb{R}^{3\times 3}$  is the rotation matrix and  $\pmb {b}\in \mathbb{R}^3$  is the translation vector. Applying  $T_{g}$  will result in the same transformation on fragment poses, i.e.  $\pmb{p}_c' = \pmb {R}\pmb {p}_c$ ,  $\pmb{R}_c' = \pmb {R}_g\pmb {R}_c$ .
First, considering the zero-CoM (Center of Mass) operation  $\bar{x}_i = x_i - (p_1 + p_2) / 2$ , the translation vector in the global SE(3)-transformation will be cancelled out:
$$
\bar {x} _ {i} ^ {\prime} = T _ {g} (\bar {\boldsymbol {x}}) = \boldsymbol {R} \boldsymbol {x} _ {i} + \boldsymbol {b} - \left(\boldsymbol {R} \boldsymbol {p} _ {1} + \boldsymbol {b} + \boldsymbol {R} \boldsymbol {p} _ {2} + \boldsymbol {b}\right) / 2 = \boldsymbol {R} \left(\boldsymbol {x} _ {i} - \left(\boldsymbol {p} _ {1} + \boldsymbol {p} _ {2}\right) / 2\right) = \boldsymbol {R} \bar {\boldsymbol {x}} _ {i}. \tag {23}
$$
Thus, we only need to consider the rotation transformation. Next, we will prove the equivariance in the linker denoising process and fragment pose prediction.
Linker Denoising Process It is easy to see the atomic distance  $\| \pmb{x}_i - \pmb{x}_j\|$  is  $T_{g}$ . Thus,  $\tilde{e}_{ij}, h_i, e_{ij}$  are also invariant since the update of them (as shown in Eq. (8, 9, 10)) only involves invariant inputs. For  $\tilde{\pmb{x}}_i$ , the update formula is
$$
\phi \left(\boldsymbol {x} _ {i} ^ {l}\right) = \boldsymbol {x} _ {i} ^ {l} + \sum_ {j \in \mathcal {V} _ {M} \backslash \{i \}} \left(\boldsymbol {x} _ {i} ^ {l} - \boldsymbol {x} _ {j} ^ {l}\right) \phi_ {x} \left(\boldsymbol {h} _ {i} ^ {l + 1}, \boldsymbol {h} _ {j} ^ {l + 1}, \boldsymbol {e} _ {i j} ^ {l + 1}, t\right) \cdot \mathbb {1} _ {\text {l i n k e r}}. \tag {24}
$$
After applying  $T_{g}$ , we have