SlowGuess/NeurIPS_2023_OCR huggingface dataset:
$$
\begin{array}{l} \phi (T (\boldsymbol {x} _ {i} ^ {l})) = T (\boldsymbol {x} _ {i} ^ {l}) + \sum_ {j \in \mathcal {V} _ {M} \backslash \{i \}} (T (\boldsymbol {x} _ {i} ^ {l}) - T (\boldsymbol {x} _ {j} ^ {l})) \phi_ {x} (\boldsymbol {h} _ {i} ^ {l + 1}, \boldsymbol {h} _ {j} ^ {l + 1}, \boldsymbol {e} _ {i j} ^ {l + 1}, t) \cdot \mathbb {1} _ {\text {l i n k e r}} \\ = \boldsymbol {R} \boldsymbol {x} _ {i} ^ {l} + \sum_ {j \in \mathcal {V} _ {M} \backslash \{i \}} \boldsymbol {R} \left(\boldsymbol {x} _ {i} ^ {l} - \boldsymbol {x} _ {j} ^ {l}\right) \phi_ {x} \left(\boldsymbol {h} _ {i} ^ {l + 1}, \boldsymbol {h} _ {j} ^ {l + 1}, \boldsymbol {e} _ {i j} ^ {l + 1}, t\right) \cdot \mathbb {1} _ {\text {l i n k e r}} \tag {25} \\ = \boldsymbol {R} \left(\boldsymbol {x} _ {i} ^ {l} + \sum_ {j \in \mathcal {V} _ {M} \backslash \{i \}} \left(\boldsymbol {x} _ {i} ^ {l} - \boldsymbol {x} _ {j} ^ {l}\right) \phi_ {x} \left(\boldsymbol {h} _ {i} ^ {l + 1}, \boldsymbol {h} _ {j} ^ {l + 1}, \boldsymbol {e} _ {i j} ^ {l + 1}, t\right) \cdot \mathbb {1} _ {\text {l i n k e r}}\right) \\ = T \left(\phi \left(\boldsymbol {x} _ {i} ^ {l}\right)\right), \\ \end{array}
$$
which implies that the linker atom position update is equivariant. By stacking multiple layers together, we can draw the conclusion that the denoised linker atom positions are SE(3)-equivariant.
Fragment Poses Prediction We recap the fragment poses prediction as follows:
$$
\boldsymbol {f} _ {i} = \sum_ {j \notin \mathcal {V} _ {F _ {c}}} (\boldsymbol {x} _ {i} - \boldsymbol {x} _ {j}) \phi_ {f} (\boldsymbol {h} _ {i}, \boldsymbol {h} _ {j}, \boldsymbol {e} _ {i j} \| \boldsymbol {x} _ {i} - \boldsymbol {x} _ {j} \|)
$$
$$
\boldsymbol {F} _ {c} = \sum_ {i \in \mathcal {V} _ {F _ {c}}} \boldsymbol {f} _ {i}, \quad \boldsymbol {\tau} _ {c} = \sum_ {i \in \mathcal {V} _ {F _ {c}}} \left(\boldsymbol {x} _ {i} - \boldsymbol {p} _ {c}\right) \times \boldsymbol {f} _ {i} \tag {26}
$$
$$
\hat {\pmb {p}} _ {c, 0} = \pmb {p} _ {t} + \frac {1}{| \mathcal {V} _ {F _ {c}} |} \pmb {F} _ {c}, \quad \hat {\pmb {R}} _ {c, 0} = R _ {\omega} (\mathbf {I} _ {c} ^ {- 1} \pmb {\tau} _ {c}) \pmb {R} _ {t}
$$
First, it is easy to see  $\pmb{f}_i$  is equivariant w.r.t  $T_g$  following the similar proof about the equivariance of linker atom positions. Thus, we can prove that the total force  $\pmb{F}_c$  and torque  $\pmb{\tau}_c$  are equivariant:
$$
\begin{array}{l} \boldsymbol {F} _ {c} ^ {\prime} = \sum_ {i \in \mathcal {V} _ {F _ {c}}} \boldsymbol {R} _ {g} \boldsymbol {f} _ {i} = \boldsymbol {R} _ {g} \sum_ {i \in \mathcal {V} _ {F _ {c}}} \boldsymbol {f} _ {i} = \boldsymbol {R} _ {g} \boldsymbol {F} _ {c}, \\ \boldsymbol {\tau} _ {c} ^ {\prime} = \sum_ {i \in \mathcal {V} _ {F _ {c}}} \boldsymbol {R} _ {g} \left(\boldsymbol {x} _ {i} - \boldsymbol {p} _ {c}\right) \times \boldsymbol {R} _ {g} \boldsymbol {f} _ {i} \tag {27} \\ = \boldsymbol {R} _ {g} \sum_ {i \in \mathcal {V} _ {F _ {c}}} \left(\boldsymbol {x} _ {i} - \boldsymbol {p} _ {c}\right) \times \boldsymbol {f} _ {i} = \boldsymbol {R} _ {g} \boldsymbol {\tau} _ {c}. \\ \end{array}
$$
Second, we follow the same inertia matrix definition as [24]:  $\mathbf{I}_c = \sum_{i\in \mathcal{V}_{F_c}}\| \pmb {x}_i - \pmb {p}_c\| ^2\mathbf{I} - (\pmb {x}_i - p_c)(\pmb {x}_i - \pmb {p}_c)^\top$ . After applying  $T_{g}$ , we have  $\mathbf{I}_c' = \pmb {R}_g\mathbf{I}_c\pmb{R}_g^{-1}$ , and thus the angular velocity is equivariant:
$$
\boldsymbol {\omega} ^ {\prime} = \mathbf {I} _ {c} ^ {\prime - 1} \boldsymbol {\tau} _ {c} ^ {\prime} = \boldsymbol {R} _ {g} \mathbf {I} _ {c} ^ {- 1} \boldsymbol {\widetilde {R}} _ {g} ^ {- 1} \boldsymbol {\widetilde {R}} _ {g} \boldsymbol {\tau} _ {c} = \boldsymbol {R} _ {g} \boldsymbol {\omega} \tag {28}
$$
Third, we notice one nice property of Lie group is that the adjoint transformation is linear:
$$
\mathbf {R} \exp (\boldsymbol {\omega}) = \exp (\operatorname {A d j} _ {\mathbf {R}} \boldsymbol {\omega}) \mathbf {R}, \quad \text {f o r} \boldsymbol {\omega} \in \mathfrak {s o} (3), \mathbf {R} \in S O (3) \tag {29}
$$
In the case of SO(3), the adjoint transformation for an element is exact the same rotation matrix used to represent the element [11], i.e.  $\mathrm{Adj}_{\mathbf{R}} = \mathbf{R}$ . Thus, we have  $\mathbf{R} \exp(\omega) = \exp(\mathbf{R} \omega) \mathbf{R}$ .
As a result, the predicted fragment center positions are equivariant:
$$
\hat {\boldsymbol {p}} _ {c, 0} ^ {\prime} = \boldsymbol {R} _ {g} \boldsymbol {p} _ {t} + \frac {1}{| \mathcal {V} _ {F _ {c}} |} \boldsymbol {R} _ {g} \boldsymbol {F} _ {c} = \boldsymbol {R} _ {g} \hat {\boldsymbol {p}} _ {c, 0}, \tag {30}
$$
and the predicted fragment rotations are also equivariant:
$$
\hat {\boldsymbol {R}} _ {c, 0} ^ {\prime} = \exp (\boldsymbol {R} _ {g} \boldsymbol {\omega}) \boldsymbol {R} _ {g} \boldsymbol {R} _ {t} = \boldsymbol {R} _ {g} \exp (\boldsymbol {\omega}) \boldsymbol {R} _ {g} ^ {- 1} \boldsymbol {R} _ {g} \boldsymbol {R} _ {t} = \boldsymbol {R} _ {g} \exp (\boldsymbol {\omega}) \boldsymbol {R} _ {t} = \boldsymbol {R} _ {g} \hat {\boldsymbol {R}} _ {c, 0}. \tag {31}
$$
B.2 Independence w.r.t Local Coordinate System
Since we always take the fragment center as the origin of the local coordinate system, we only need to consider the effect of the orientation change. Suppose the global coordinates remain unchanged, and the change of local coordinate system results in a rotation  $\mathbf{R}_c$  applied on local coordinates, i.e.  $\tilde{\mathbf{x}}' = \mathbf{R}_s\tilde{\mathbf{x}}$ . Thus, the transformation between local coordinates and global coordinates becomes  $\mathbf{x} = \mathbf{R}\mathbf{R}_s^{-1}\tilde{\mathbf{x}}' + \mathbf{p}$ , which implies the new rotation matrix representing the fragment pose is  $\mathbf{R}\mathbf{R}_s^{-1}$ .
Recall that the fragment rotation update is  $\hat{R}_0 = R_{\omega}(\mathbf{I}_c^{-1}\pmb{\tau}_c)\pmb{R}_t$ . With the change of local coordinate system, the predicted fragment rotation is  $\hat{R}_0' = R_{\omega}(\mathbf{I}_c^{-1}\pmb{\tau}_c)\pmb{R}_t\pmb{R}_s^{-1}$ . Since our rotation loss is defined as  $L_{\mathrm{rot}} = \| \pmb{R}_0\hat{\pmb{R}}_0^\top -\mathbf{I}\| _F^2$ , we have
$$
L _ {\text {r o t}} ^ {\prime} = \left\| \boldsymbol {R} _ {0} \boldsymbol {R} _ {s} ^ {- 1} \left(\hat {\boldsymbol {R}} _ {0} \boldsymbol {R} _ {s} ^ {- 1}\right) ^ {\top} - \mathbf {I} \right\| _ {F} ^ {2} = \left\| \boldsymbol {R} _ {0} \boldsymbol {R} _ {s} ^ {- 1} \widetilde {\boldsymbol {R}} _ {s} \hat {\boldsymbol {R}} _ {0} ^ {\top} - \mathbf {I} \right\| _ {F} ^ {2} = L _ {\text {r o t}}, \tag {32}
$$
which means the training of our model is independent w.r.t the choice of local coordinate system.
C.1 Featurization
The molecular graph is extended as a fully-connected graph. The atom features include a one-hot element and charge indicator (H, C, N,  $\mathrm{N}^{-}$ ,  $\mathrm{N}^{+}$ , O,  $\mathrm{O}^{-}$ , F, Cl, Br, I, S(2), S(4), S(6)) and a one-hot fragment/linker indicator. Note that the fragment/linker can be predetermined. Thus, it will only serve as the input feature without getting involved in the network's prediction. The edge features include a one-hot bond type indicator (None, Single, Double, Triple, Aromatic), and a 4-dim one-hot vector indicating the edge is between fragment atoms, linker atoms, fragment-linker atoms or linker-fragment atoms.
C.2 Model Details
Atom features and edge features are firstly fed to two embedding layers with node_emb_dim=256 and edge_emb_dim=64. The hidden embeddings get involved in three types of layers: atom update layer, bond update layer, and position update layer as described in Eq. (9, 10, 11). In each layer, we concatenate the input features and update the hidden embedding / positions with a 2-layer MLP with LayerNorm and ReLU activation. The stack of these three layers is viewed as a block, and our model consists of 6 blocks. For the force prediction layer, we apply graph attention to aggregate the message of each node/edge. The key/value/query embedding is also obtained with a 2-layer MLP.
We set the number of diffusion steps as 500. For this diffusion noise schedule, we choose to use a cosine  $\beta$  schedule suggested in [33] with  $s = 0.01$ .
C.3 Loss Functions
For the linker atom positions and fragment center positions losses, we use the standard mean Squared Error (MSE). For the rotation loss, we measure the discrepancy between the real and the predicted rotation matrices by computing the Frobenius-norm of  $\| \pmb{R}_0\hat{\pmb{R}}_0^\top -\mathbf{I}\|$ . For the atom and bond type losses, we compute the KL divergence between the real posterior and the predicted posterior. Specifically, the loss functions can be summarized as follows:
$$
L _ {\text {l i n k e r}} = \left\| \mathbf {x} _ {L} - \hat {\mathbf {x}} _ {L, 0} \right\| ^ {2} \tag {33}
$$
$$
L _ {\mathrm {t r}} = \Sigma_ {c = 1} ^ {2} \| \boldsymbol {p} _ {c} - \hat {\boldsymbol {p}} _ {c, 0} \| ^ {2} \tag {34}
$$
$$
L _ {\text {r o t}} = \Sigma_ {c = 1} ^ {2} \| \boldsymbol {R} _ {c, 0} \hat {\boldsymbol {R}} _ {c, 0} ^ {\top} - \mathbf {I} \| _ {F} ^ {2} \tag {35}
$$
$$
L _ {\text {a t o m}} = \sum_ {k = 1} ^ {N _ {\mathrm {a}}} \tilde {\mathbf {c}} \left(\mathbf {v} _ {L, t}, \mathbf {v} _ {L, 0}\right) _ {k} \log \frac {\tilde {\mathbf {c}} \left(\mathbf {v} _ {L , t} , \mathbf {v} _ {L , 0}\right) _ {k}}{\tilde {\mathbf {c}} \left(\mathbf {v} _ {L , t} , \hat {\mathbf {v}} _ {L , 0}\right) _ {k}} \tag {36}
$$
$$
L _ {\text {b o n d}} = \sum_ {k = 1} ^ {N _ {b}} \tilde {c} \left(\mathbf {b} _ {L, t}, \mathbf {b} _ {L, 0}\right) _ {k} \log \frac {\tilde {c} \left(\mathbf {b} _ {L , t} , \mathbf {b} _ {L , 0}\right) _ {k}}{\tilde {c} \left(\mathbf {b} _ {L , t} , \hat {\mathbf {b}} _ {L , 0}\right) _ {k}} \tag {37}
$$
The final loss is a weighted sum of them:
$$
L = \lambda_ {1} L _ {\text {l i n k e r}} + \lambda_ {2} L _ {\mathrm {t r}} + \lambda_ {3} L _ {\mathrm {r o t}} + \lambda_ {4} L _ {\mathrm {a t o m}} + \lambda_ {5} L _ {\mathrm {b o n d}} \tag {38}
$$
C.4 Training Details
The model is trained via AdamW [30] with init_learning_rate=5e-4, betas=(0.99, 0.999), batch_size=64 and clip_grad率=50.0. To balance the scales of different losses, we multiply a factor  $\lambda = 100$  on the atom type loss and bond type loss. During the training phase, we add a small Gaussian noise with a standard deviation of 0.05 to linker atom coordinates as data augmentation. We also schedule to decay the learning rate exponentially with a factor of 0.6 and a minimum learning rate of 1e-6. The learning rate is decayed if there is no improvement for the validation loss in 10 consecutive evaluations. The evaluation is performed for every 2000 training steps. We trained our model on one NVIDIA RTX A6000 GPU, and it could converge within 350k steps.
C.5 Overall Training and Sampling Procedures
In this section, we summarize the overall training and sampling procedures of LinkerNet as Algorithm 1 and Algorithm 2, respectively.
Algorithm 1 Training Procedure of LinkerNet
Input: Linker dataset  $\{\mathbf{R},\mathbf{p},\mathcal{G}_L,\mathcal{G}_{F_1},\mathcal{G}_{F_2}\}_{i = 1}^N$  , where  $\mathcal{G}_L = \{\mathbf{v}_L,\mathbf{b}_L,\mathbf{x}_L\}$ $\mathcal{G}_F = \{\mathbf{v}_F,\mathbf{b}_F,\tilde{\mathbf{x}}_F\}$  and  $p_1 + p_2 = 0$  ; Neural network  $\phi_{\theta}$    
1: while  $\phi_{\theta}$  not converge do   
2: Sample diffusion time  $t\in \mathcal{U}(0,\dots,T)$    
3: Add noise to  $(\mathbf{x}_L,\mathbf{p})$  , R and  $(\mathbf{v}_L,\mathbf{b}_L)$  according to Eq. (2, 4, 6), respectively   
4:Compose  $[\mathbf{x}_t,\mathbf{v}_t,\mathbf{b}_t,\mathbf{R}_t,\mathbf{p}_t]$  to form  $\mathcal{G}_{M_t}$  by performing  $\mathbf{x}_{F,t} = \mathbf{R}_{t}\hat{\mathbf{x}}_{F,t} + \mathbf{p}_{t}$    
5: Move the molecule to make  $p_{1,t} + p_{2,t} = 0$    
6: Predict  $[\hat{\mathbf{x}}_{L,0},\hat{\mathbf{v}}_{L,0},\hat{\mathbf{b}}_{L,0},\hat{\mathbf{R}}_0,\hat{\mathbf{p}}_0]$  from  $\mathcal{G}_{M_t}$  with  $\phi_{\theta}$  as described in Sec. 3.3   
7: Compute the training loss  $L$  according to Eq. (38)   
8: Update  $\theta$  by minimizing  $L$    
9: end while
Algorithm 2 Sampling Procedure of LinkerNet
Input: The molecular fragments  $\mathcal{G}_{F_1},\mathcal{G}_{F_2}$  , the learned model  $\phi_{\theta}$  . Optional: the fragment distance constraint  $[d_{\mathrm{min}},d_{\mathrm{max}}]$  , the fragment candidate anchor sets  $\mathcal{A}$    
Output: Generated fragment poses (R, p) and linker  $\mathcal{G}_L$    
1: If the fragment distance constraint is provided, sample the number of atoms in  $\mathcal{G}_L$  based on a prior distribution summarized from the training set.   
2: Sample initial fragment poses  $(\mathbf{R}_T,\mathbf{p}_T)$  and linker  $\mathcal{G}_{L,T}$    
3: for  $t$  in  $T,T - 1,\ldots ,1$  do   
4: Compose  $[\mathbf{x}_t,\mathbf{v}_t,\mathbf{b}_t,\mathbf{R}_t,\mathbf{p}_t]$  to form  $\mathcal{G}_{M_t}$  by performing  $\mathbf{x}_{F,t} = \mathbf{R}_{t}\tilde{\mathbf{x}}_{F,t} + \mathbf{p}_{t}$    
5: Move the molecule to make  $p_{1,t} + p_{2,t} = 0$    
6: Predict  $[\hat{\mathbf{x}}_{L,0},\hat{\mathbf{v}}_{L,0},\hat{\mathbf{b}}_{L,0},\hat{\mathbf{R}}_{0},\hat{\mathbf{p}}_{0}]$  from  $\mathcal{G}_{M_t}$  with  $\phi_{\theta}$  as described in Sec. 3.3   
7: Sample  $[\hat{\mathbf{x}}_{L,t - 1},\hat{\mathbf{v}}_{L,t - 1},\hat{\mathbf{b}}_{L,t - 1},\hat{\mathbf{R}}_{t - 1},\hat{\mathbf{p}}_{t - 1}]$  from the posterior according to Eq. (2, 4, 6)   
8: Optional: Compute the guidance according to Eq. (18, 19) if the corresponding constraint is provided. Update the prediction with guidance according to Eq. (17).   
9: end for
D Training and Sampling Efficiency
For the training efficiency, DiffLinker converges within 300 epochs and takes 76 hrs with one V100 GPU as the original paper reported. Our model converges within 50 epochs and takes 48 hrs with the same type of GPU.
For the sampling complexity, DiffLinker finished sampling linkers for 43 PROTAC fragment pairs (100 linkers for each pair) in  $132\mathrm{min}$  while our model takes  $761\mathrm{min}$  with the same NVIDIA 1080 GPU.