SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Jiaqi Guan
jiaqi@illinois.edu
Xingang Peng
xingang.peng@gmail.com
Peiqi Jiang
jpq20@mails.tsinghua.edu.cn
Yunan Luo
yunan@gatech.edu
Jian Peng
jianpeng@illinois.edu
Jianzhu Ma
majianzhu@tsinghua.edu.cn
Targeted protein degradation techniques, such as PROteolysis TTargeting Chimeras (PROTACs), have emerged as powerful tools for selectively removing disease-causing proteins. One challenging problem in this field is designing a linker to connect different molecular fragments to form a stable drug-candidate molecule. Existing models for linker design assume that the relative positions of the fragments are known, which may not be the case in real scenarios. In this work, we address a more general problem where the poses of the fragments are unknown in 3D space. We develop a 3D equivariant diffusion model that jointly learns the generative process of both fragment poses and the 3D structure of the linker. By viewing fragments as rigid bodies, we design a fragment pose prediction module inspired by the Newton-Euler equations in rigid body mechanics. Empirical studies on ZINC and PROTAC-DB datasets demonstrate that our model can generate chemically valid, synthetically-accessible, and low-energy molecules under both unconstrained and constrained generation settings.
Targeted Protein Degradation (TPD) techniques [27, 7], including PROteolysis TTargeting Chimeras (PROTACs) [35], molecular glues [40], etc., are emerging powerful tools for selectively removing disease-causing proteins. These techniques typically involve multiple molecular fragments connected by a linker, with each fragment binding to a specific protein. For instance, a PROTAC consists of three components: a ligand (warhead) that targets the protein of interest, another ligand that recruits an E3 ubiquitin ligase, and a linker that connects two ligands [38, 9]. PROTACs induce the ubiquitination of target proteins, which is a fundamental biological process where small proteins called ubiquitins are attached to target proteins, marking them for degradation by the proteasome. Unlike traditional small molecule inhibitors that temporarily inhibit protein function by binding to their active sites, PROTAC techniques result in complete protein elimination and offer several advantages, including increased selectivity, reduced off-target effects, and the potential to target previously undruggable proteins [27]. However, designing effective PROTACs remains a significant challenge, particularly in optimizing the linker, which is crucial in maintaining the conformational stability and other important biological properties of the entire molecule.
(a) PROTAC-DB ID: 146 (BRD4, CRBN)
(b) PROTAC-DB ID: 147 (BRD4, CRBN)
Figure 1: An example showing that the fragment poses are not fixed in PROTAC design. The above sub-figures show two PROTACs, both of which have one fragment binding with the BRD4 target and the other fragment binding with the CRBN E3 ligase. The linkers differ with a 'COC' motif.
(a) Fixed fragment design (2D)
(b) Fixed fragment design (3D)
(c) Free fragment design (3D)
Figure 2: An overview of different linker design settings. (a)/(b): Previous work focuses on 2D/3D linker design with fixed fragment poses. (c): We focus on co-designing fragment poses and linker.
In recent years, computational approaches, particularly deep learning methods, have been employed for linker design to accelerate the drug discovery process [47, 22, 23, 20, 21]. Many of these approaches utilize 3D structural information to enhance their performance. For example, Delinker [22] incorporates the distance and angle between anchor atoms as additional structural information to generate the 2D linker given the fragment molecular graphs. Going a step further, 3DLinker [20] and DiffLinker [21] operate directly in 3D space to process fragments and generate linkers with a conditional VAE and diffusion model, respectively. These models commonly assume that the relative position between fragments is fixed. This assumption is reasonable in traditional fragment-based drug design, where fragments are designed to bind to the same protein pocket and the binding poses are largely deterministic [12]. However, in scenarios involving two proteins (drug target and E3 ligase in the PROTAC case), the relative position between fragments may not be readily available due to the flexibility of protein-protein binding pose, as shown in Figure 1. As a result, it becomes necessary to adjust the fragment poses dynamically during the linker design process. One potential solution is to randomly sample multiple fragment poses and design a 3D linker for each pose. The drawback of this approach is that it limits the design space for the linker by immobilizing the relative positions of the two molecular fragments. It may not be possible to find a stable linker that can connect the two molecular fragments with fixed positions due to the narrow range in which a bond can form. To the best of our knowledge, there is currently no existing computational approach that addresses this challenging 3D linker design problem in the absence of fragment relative positions.
In this work, we first address the problem of co-designing fragment poses and the 3D linker. We represent each fragment as a rigid body and its pose as the position of its center and the rotation. The linker is represented as a 3D graph, consisting of atom positions, atom types, and bond types. To
tackle this co-designing problem, we propose LinkerNet, a 3D equivariant diffusion model, leveraging recent advancements in equivariant diffusion models [19] and diffusion models on Riemann manifolds [8]. Our proposed model can jointly learn the generative process of both fragment poses and the linker, which enables the model to find a stable linker and connect the fragments to form a low-energy conformation of the whole molecule. Moreover, we design a fragment pose prediction module inspired by the Newton-Euler equations in rigid body mechanics. This module employs neural networks to predict atomic forces, which are then used to update the fragment center positions and rotations through the aggregation of neural forces and torques. We introduce two guidance terms that restrict fragment distances and potential anchors to incorporate real-world constraints into our model. To evaluate our method, we perform comprehensive experiments on ZINC [43] and PROTAC-DB [45] datasets and demonstrate that our model can generate chemically valid, synthetically-accessible, and low-energy molecules under both unconstrained and constrained generation settings.
To summarize, our main contributions are:
- Our work presents the first computational model that abandons the unrealistic assumptions in PROTAC drug design.
- We propose a 3D equivariant diffusion model which enables the co-design of fragment poses and the 3D linker structure in a unified framework.
- We develop an effective fragment pose prediction module inspired by the Newton-Euler equations in rigid body mechanics, allowing for the accurate adjustment of fragment center positions and rotations.
- We conduct comprehensive experiments on ZINC and PROTAC-DB datasets, showcasing the superiority of our proposed model over other baseline methods in both unconstrained and constrained generation settings.
Molecular Linker Design Molecular linker design is a critical step in the rational compound design. SyntaLinker [47] operates on the SMILES representation of molecules and formulates the linker design as a sentence completion problem. The lack of 3D structural information and the drawbacks of the SMILES representation itself limits the performance of this method. DeLinker [22] and Develop [23] overcome this limitation by operating on graphs and utilizing the distance and angle between anchor atoms as the additional structural information. However, only limited structural information is used and the generation is still in 2D space. More recently, 3DLinker [20], DiffLinker[21] are proposed to directly generate linkers in 3D space with conditional VAE and diffusion models, respectively. All of these models assume the fragment poses are known. However, this is not always the case, especially in the emerging Targeted Protein Degradation (TPD) techniques. Instead, our model focuses on a more general linker design problem where the fragment poses are unknown.
PROTAC Linker Design PROteolysis TTargeting Chimeras (PROTAC) is a promising technique with many advantages over traditional small molecule inhibitors. The first proof of concept study of PROTAC was proposed in [38]. Most PROTAC linker design strategies rely on empirical optimization of linker composition, which consists of only a few main chemical motifs [44, 32]. Currently, there are no generally accepted rules for de novo PROTAC linker design. [48] uses deep reinforcement learning to facilitate rational PROTAC design, but it is still generating SMILES representation instead of molecules in 3D space.
Diffusion Generative Models Diffusion generative models [41, 42, 17] learn to denoise samples from a prior noise distribution and have achieved remarkable progress in generating images [36, 37], text [18, 2], etc. Recently, diffusion models are also applied in molecular data by considering the rotation-translation equivariance, such as molecular conformation generation [46], 3D molecule generation [19] and structure-based drug design [15, 39, 29]. In addition, diffusion models have also been extended to Riemann manifolds [8, 28], and many applications in molecular data have emerged, including conformation generation [25], molecular docking [6], antibody design [31] and protein-ligand binding affinity prediction [24]. Leveraging these advances in equivariance diffusion models and diffusion models in Riemann manifolds, we propose a diffusion model for the fragment
Figure 3: Overview of one denoising step of LinkerNet. An equivariant GNN is applied to update linker atom positions  $\mathbf{x}$ , atom types  $\mathbf{v}$  and bond types  $\mathbf{b}$ . The updated embedding and positions are then utilized by a pose prediction module to predict neural forces  $\mathbf{F}_c$  and torques  $\boldsymbol{\tau}_c$  to further update fragment poses  $\mathbf{R}$  and  $\mathbf{p}$ .
poses and linker co-design task. We inject task-specific network and sampling designs, including a physics-inspired fragment poses prediction module and the constrained linker sampling approach.
3 Fragment Poses and Linker Co-Design
In this section, we present LinkerNet, which co-designs fragment poses and the linker with 3D equivariant diffusion. We first define notations and this problem formally in Sec. 3.1. Then, we decompose the joint distribution as the product of positions, rotations and atom / bond types in Sec. 3.2 and show how the diffusion processes are constructed for them. In Sec. 3.3, we illustrate the equivariant network and physics-inspired prediction module for denoising fragment poses and the linker. Finally, in Sec. 3.4, we describe how our model can be applied with guided sampling in the constrained generation setting.
3.1 Notations and Problem Definition
In our formulation, each molecular fragment is represented as a 3D graph  $\mathcal{G}_F = \{\mathbf{v}_F,\mathbf{b}_F,\tilde{\mathbf{x}}_F\}$ . We denote the number of atom features, the number of bond types and the number of fragment atoms as  $N_{a},N_{b}$  and  $N_{F}$ , respectively, and then  $\mathbf{v}_F\in \mathbb{R}^{N_F\times N_a}$  represents one-hot atom types (including elements and atom charges),  $\mathbf{b}_F\in \mathbb{R}^{N_F\times N_F\times N_b}$  represents one-hot bond types (the absence of bond is treated as a special bond type), and  $\tilde{\mathbf{x}}_F\in \mathbb{R}^{3N_F}$  is the atom coordinates in the local coordinate system (i.e., conformation). Since we assume fragments are rigid,  $\tilde{\mathbf{x}}_F$  is unchanged in our setting. We use PCA to construct the local coordinate system [14] for robustness. However, we will see the equivariance of our model is independent of the choice of the local coordinate system (Sec. 3.3). The global pose of each fragment is determined by a rotation transformation  $\pmb {R}\in SO(3)$  and a translation transformation  $\pmb {p}\in \mathbb{T}(3)\cong \mathbb{R}^3$ , i.e.  $\mathbf{x}_F = R\tilde{\mathbf{x}}_F + \pmb{p}$ . The linker is represented as a set of atom types, bond types and atom positions  $\mathcal{G}_L = \{\mathbf{v}_L,\mathbf{b}_L,\mathbf{x}_L\}$ , where  $\mathbf{v}_L\in \mathbb{R}^{N_L\times N_a}$  denotes linker atom types,  $\mathbf{b}_L\in \mathbb{R}^{N_L\times N\times N_b}$  denotes linker bond types and  $\mathbf{x}_L\in \mathbb{R}^{3N_L}$  denotes linker atom coordinates in the global coordinate system. Here,  $N = N_{L} + N_{F_{1}} + N_{F_{2}}$  is the total number of atoms of two fragments and the linker.
Given two fragments  $(\mathcal{G}_{F_1},\mathcal{G}_{F_2})$  whose global poses are unknown, our goal is to design the linker  $\mathcal{G}_L$  and recover the global poses of two fragments  $(\pmb {R}_1,\pmb {p}_1)$  and  $(\pmb {R}_2,\pmb {p}_2)$  to connect fragments with the linker to form a conformationally stable molecule  $\mathcal{G}_M$ . Specifically, denoting the fragment rotations and translations as  $\mathbf{R}$  and  $\mathbf{p}$  separately, we aim to learn the distribution  $p_{\theta}(\mathbf{R},\mathbf{p},\mathcal{G}_L|\mathcal{G}_{F_1},\mathcal{G}_{F_2})$  with a neural network parameterized by  $\theta$ .
3.2 Diffusion Processes
A diffusion probabilistic model involves two Markov chains: a forward diffusion process and a reverse generative process. The diffusion process gradually injects noise into data, and the generative process
learns to recover the data distribution from the noise distribution. According to the different types of variables, the joint distribution  $p_{\theta}(\mathbf{R},\mathbf{p},\mathcal{G}_L|\mathcal{G}_{F_1},\mathcal{G}_{F_2})$  can be further decomposed as a product of fragment / linker positions  $(\mathbf{x}_L,\mathbf{p})$ , fragment rotations  $(\mathbf{R})$  and linker atom / bond types  $(\mathbf{v}_L,\mathbf{b}_L)$ . Next, we will describe how these diffusion processes are constructed in detail.
Diffusion on Positions Denote linker positions  $\mathbf{x}_L$  or fragment translations  $\mathbf{p}$  as a random variable  $\mathbf{x}$ . The diffusion on  $\mathbf{x}$  involves standard Gaussian diffusion kernels and has been well-studied in [17]. At each time step  $t$ , a small Gaussian noise is added according to a Markov chain with fixed variance schedules  $\beta_1, \ldots, \beta_T$ :
$$
q \left(\mathbf {x} _ {t} \mid \mathbf {x} _ {t - 1}\right) = \mathcal {N} \left(\mathbf {x} _ {t}; \sqrt {1 - \beta_ {t}} \mathbf {x} _ {t - 1}, \beta_ {t} \mathbf {I}\right) \tag {1}
$$
Under this formulation, we can efficiently draw samples from the noisy data distribution  $q(\mathbf{x}_t|\mathbf{x}_0)$  and compute the posterior distribution  $q(\mathbf{x}_{t - 1}|\mathbf{x}_t,\mathbf{x}_0)$  in closed-form:
$$
q \left(\mathbf {x} _ {t} \mid \mathbf {x} _ {0}\right) = \mathcal {N} \left(\mathbf {x} _ {t}; \sqrt {\bar {\alpha} _ {t}} \mathbf {x} _ {0}, (1 - \bar {\alpha} _ {t}) \mathbf {I}\right), \quad q \left(\mathbf {x} _ {t - 1} \mid \mathbf {x} _ {t}, \mathbf {x} _ {0}\right) = \mathcal {N} \left(\mathbf {x} _ {t - 1}; \tilde {\boldsymbol {\mu}} _ {t} \left(\mathbf {x} _ {t}, \mathbf {x} _ {0}\right), \tilde {\beta} _ {t} \mathbf {I}\right), \tag {2}
$$
where  $\alpha_{t} = 1 - \beta_{t}$ ,  $\bar{\alpha}_{t} = \Pi_{s=1}^{t} \alpha_{s}$  and  $\tilde{\pmb{\mu}}_{t}(\mathbf{x}_{t},\mathbf{x}_{0}) = \frac{\sqrt{\bar{\alpha}_{t-1}} \beta_{t}}{1-\bar{\alpha}_{t}} \mathbf{x}_{0} + \frac{\sqrt{\alpha_{t}} (1-\bar{\alpha}_{t-1})}{1-\bar{\alpha}_{t}} \mathbf{x}_{t}$ ,  $\tilde{\beta}_{t} = \frac{1-\bar{\alpha}_{t-1}}{1-\bar{\alpha}_{t}} \beta_{t}$ .
Diffusion on Fragment Rotations The diffusion kernel on fragment rotation  $\mathbf{R}$  is  $\mathcal{IG}_{SO(3)}(\pmb{\mu},\epsilon^2)$ , i.e. the isotropic Gaussian on  $SO(3)$  [34, 28] parameterized by a mean rotation  $\pmb{\mu}$  and scalar variance  $\epsilon^2$ . The standard  $\mathcal{IG}_{SO(3)}(\mathbf{I},\epsilon^2)$  can be sampled in an axis-angle form, with uniformly sampled axes  $\hat{\omega} \sim \mathfrak{so}(3)$  and rotation angle  $\omega \in [0,\pi]$  with density
$$
f (\omega) = \frac {1 - \cos \omega}{\pi} \sum_ {l = 0} ^ {\infty} (2 l + 1) e ^ {- l (l + 1) \epsilon^ {2}} \frac {\sin ((l + 1 / 2) \omega)}{\sin (\omega / 2)}. \tag {3}
$$
To sample from  $\mathcal{IG}_{SO(3)}(\pmb{\mu},\epsilon^2)$ , we can first sample a rotation  $e = \omega \hat{\pmb{\omega}}$  and apply it to  $\pmb{\mu}$  to obtain the desired sample. Similar to the Euclidean diffusion process, we can also draw samples and compute the posterior distribution at any time step in closed-form:
$$
q \left(\mathbf {R} _ {t} \mid \mathbf {R} _ {0}\right) = \mathcal {I G} _ {S O (3)} \left(\lambda \left(\sqrt {\bar {\alpha} _ {t}}, \mathbf {R} _ {0}\right), 1 - \bar {\alpha} _ {t}\right) q \left(\mathbf {R} _ {t - 1} \mid \mathbf {R} _ {t}, \mathbf {R} _ {0}\right) = \mathcal {I G} _ {S O (3)} \left(\tilde {\boldsymbol {\mu}} _ {t} \left(\mathbf {R} _ {t}, \mathbf {R} _ {0}\right), \tilde {\beta} _ {t}\right), \tag {4}
$$
where  $\tilde{\mu}_t(\mathbf{R}_t,\mathbf{R}_0) = \lambda \left(\frac{\sqrt{\bar{\alpha}_{t - 1}}\beta_t}{1 - \bar{\alpha}_t},\mathbf{R}_0\right) + \lambda \left(\frac{\sqrt{\bar{\alpha}_t(1 - \bar{\alpha}_{t - 1})}}{1 - \bar{\alpha}_t},\mathbf{R}_t\right)$  and  $\lambda (\gamma ,\mathbf{R}) = \exp (\gamma \log (\mathbf{R}))$  denotes the rotation scaling operation, which scales rotation matrices  $\mathbf{R}$  by converting them to values in  $\mathfrak{so}(3)$ , multiplying by a scalar  $\gamma$ , and converting them back to  $SO(3)$  [5].
Diffusion on Atom and Bond Types Following [18], we use categorical distributions to model discrete linker atom types  $\mathbf{v}_L$  and bond types  $\mathbf{b}_L$ . Take atom types as an example (same for bond types), a uniform noise across all  $K$  categories is added according to a Markov chain during the diffusion process:
$$
q \left(\mathbf {v} _ {t} \mid \mathbf {v} _ {t - 1}\right) = \mathcal {C} \left(\mathbf {v} _ {t} \mid \left(1 - \beta_ {t}\right) \mathbf {v} _ {t - 1} + \beta_ {t} / K\right). \tag {5}
$$
Similarly, we can compute  $q(\mathbf{v}_t|\mathbf{v}_0)$  and  $q(\mathbf{v}_{t - 1}|\mathbf{v}_t,\mathbf{v}_0)$  in closed-forms:
$$
q \left(\mathbf {v} _ {t} \mid \mathbf {v} _ {0}\right) = \mathcal {C} \left(\mathbf {v} _ {t} \mid \bar {\alpha} _ {t} \mathbf {v} _ {0} + (1 - \bar {\alpha} _ {t}) / K\right), \quad q \left(\mathbf {v} _ {t - 1} \mid \mathbf {v} _ {t}, \mathbf {v} _ {0}\right) = \mathcal {C} \left(\mathbf {v} _ {t - 1} \mid \tilde {\mathbf {c}} _ {t} \left(\mathbf {v} _ {t}, \mathbf {v} _ {0}\right)\right), \tag {6}
$$
where  $\tilde{c}_t(\mathbf{v}_t,\mathbf{v}_0) = c^{\star} / \sum_{k = 1}^{K}c_k^{\star}$  and  $c^{\star}(\mathbf{v}_t,\mathbf{v}_0) = [\alpha_t\mathbf{v}_t + (1 - \alpha_t) / K]\odot [\bar{\alpha}_{t - 1}\mathbf{v}_0 + (1 - \bar{\alpha}_{t - 1}) / K]$ .
3.3 Equivariant and Physics-Inspired Neural Network
The likelihood  $p_{\theta}(\mathbf{R}, \mathbf{p}, \mathcal{G}_L | \mathcal{G}_{F_1}, \mathcal{G}_{F_2})$  should be invariant to the global SE(3)-transformation, which can be achieved by composing an invariant initial distribution and an equivariant transition [46, 19]. Thus, we define the distribution on the subspace  $\sum_{i=1}^{2} p_i = \mathbf{0}$ , i.e., the center of fragment positions is zero. It is also consistent with the truth that the linker is around the center of fragments, and thus we can set the prior distribution of  $\mathbf{x}_L$  as a standard normal distribution. For the equivariant transition, we model the atomic interaction with a 3D Equivariant GNN  $\phi_{\theta}$ :
$$
\left[ \hat {\mathbf {x}} _ {L, 0}, \hat {\mathbf {v}} _ {L, 0}, \hat {\mathbf {b}} _ {L, 0}, \hat {\mathbf {R}} _ {0}, \hat {\mathbf {p}} _ {0} \right] = \phi_ {\theta} \left(\mathcal {G} _ {M _ {t}}, t\right) = \phi_ {\theta} \left(\left[ \mathbf {x} _ {L, t}, \mathbf {v} _ {L, t}, \mathbf {b} _ {L, t}, \mathbf {R} _ {t}, \mathbf {p} _ {t} \right], \mathcal {G} _ {F _ {1}}, \mathcal {G} _ {F _ {2}}, t\right). \tag {7}
$$
To obtain  $\mathcal{G}_{M_t}$  from  $[\mathbf{x}_t,\mathbf{v}_t,\mathbf{b}_t,\mathbf{R}_t,\mathbf{p}_t]$  and  $\{\mathcal{G}_{F_1},\mathcal{G}_{F_2}\}$ , we perform a local-to-global coordinate transformation first and compose the atoms together, i.e.,  $\mathbf{x}_{F,t} = \mathbf{R}_t\tilde{\mathbf{x}}_{F,t} + \mathbf{p}_t$  and  $\mathbf{x}_{M,t} = \{\mathbf{x}_{F,t},\mathbf{x}_{L,t}\}$ . Next, we will elaborate on how the linker and fragment poses are denoised.
**Equivariant Linker Denoising Process**  $\phi_{\theta}$  is a  $\mathcal{L}$ -layer 3D equivariant GNN. The initial atom and bond embedding  $h_i^0$  and  $e_{ij}^0$  are obtained by two embedding layers that encode the atom and bond information. At the  $l$ -th layer, the atom embedding  $h_i$ , bond embedding  $e_{ij}$  and linker atom positions  $\boldsymbol{x}_i$  are updated as follows:
$$
\tilde {\boldsymbol {e}} _ {i j} = \phi_ {d} \left(\boldsymbol {e} _ {i j} ^ {l}, \| \boldsymbol {x} _ {i} ^ {l} - \boldsymbol {x} _ {j} ^ {l} \|\right) \tag {8}
$$
$$
\boldsymbol {h} _ {i} ^ {l + 1} = \boldsymbol {h} _ {i} ^ {l} + \sum_ {j \in \mathcal {V} _ {M} \backslash \{i \}} \phi_ {h} \left(\boldsymbol {h} _ {i} ^ {l}, \boldsymbol {h} _ {j} ^ {l}, \tilde {\boldsymbol {e}} _ {i j}, t\right) \tag {9}
$$
$$
\boldsymbol {e} _ {i j} ^ {l + 1} = \boldsymbol {e} _ {i j} ^ {l} + \sum_ {k \in \mathcal {V} _ {M} \backslash \{i \}} \phi_ {h} \left(\boldsymbol {h} _ {k} ^ {l}, \boldsymbol {h} _ {i} ^ {l}, \tilde {\boldsymbol {e}} _ {k i}, t\right) + \sum_ {k \in \mathcal {V} _ {M} \backslash \{j \}} \phi_ {h} \left(\boldsymbol {h} _ {j} ^ {l}, \boldsymbol {h} _ {k} ^ {l}, \tilde {\boldsymbol {e}} _ {j k}, t\right) \tag {10}
$$
$$
\boldsymbol {x} _ {i} ^ {l + 1} = \boldsymbol {x} _ {i} ^ {l} + \sum_ {j \in \mathcal {V} _ {M} \backslash \{i \}} \left(\boldsymbol {x} _ {i} ^ {l} - \boldsymbol {x} _ {j} ^ {l}\right) \phi_ {x} \left(\boldsymbol {h} _ {i} ^ {l + 1}, \boldsymbol {h} _ {j} ^ {l + 1}, \boldsymbol {e} _ {i j} ^ {l + 1}, t\right) \cdot \mathbb {1} _ {\text {l i n k e r}} \tag {11}
$$
where  $\mathcal{V}_M$  is the set of all atoms in the molecule and  $\mathbb{1}_{\mathrm{linker}}$  is the linker atom mask. The final atom embedding  $h_i^{\mathcal{L}}$  and bond embedding  $e_{ij}^{\mathcal{L}}$  will be fed into two multi-layer perceptrs and softmax functions to obtain  $[\hat{\mathbf{v}}_{L,0},\hat{\mathbf{b}}_{L,0}]$ . In addition, they will also be used to predict the fragment poses  $[\hat{\mathbf{R}},\hat{\mathbf{p}}]$  with a physics-inspired prediction module.
Physics-Inspired Fragment Pose Prediction A straightforward way to predict the global pose is by predicting an invariant pose change  $(\mathbf{R}_{t\rightarrow 0},\mathbf{p}_{t\rightarrow 0})$  in the local coordinate system and applying it to the current pose  $(\mathbf{R}_t,\mathbf{p}_t)$  in the global coordinate system, i.e.  $(\mathbf{R}_t,\mathbf{p}_t)\circ (\mathbf{R}_{t\rightarrow 0},\mathbf{p}_{t\rightarrow 0}) = (\mathbf{R}_t\mathbf{R}_{t\rightarrow 0},\mathbf{p}_t + \mathbf{R}_t\mathbf{p}_{t\rightarrow 0})$ . This trick has been commonly applied in protein structure prediction [26, 31]. By applying this trick in our setting, it will be applying the invariant pose change prediction from  $h_i,e_{ij}$  to the current noisy rotation  $\mathbf{R}_t$  and translation  $\mathbf{p}_t$  to obtain the denoised ones  $\hat{\mathbf{R}}_0$  and  $\hat{\mathbf{p}}_0$ , i.e.:
$$
\hat {\mathbf {R}} _ {0} = \mathbf {R} _ {t} \phi_ {R} \left(\boldsymbol {h} _ {i}, \boldsymbol {h} _ {j}, \boldsymbol {e} _ {i j}\right) \quad \hat {\mathbf {p}} _ {0} = \mathbf {p} _ {t} + \mathbf {R} _ {t} \phi_ {p} \left(\boldsymbol {h} _ {i}, \boldsymbol {h} _ {j}, \boldsymbol {e} _ {i j}\right) \tag {12}
$$