SlowGuess/NeurIPS_2023_OCR huggingface dataset:
According to [37], diffusion models such as SIR are analogous to message passing neural networks (MPNNs [11]), where each node's state is only updated based on the states of its neighboring nodes. However, the operational architecture must be designed to enable each node to aggregate both its own features and those of its neighbors, and update its state via nonlinear activation as shown in Equation 26. Therefore, we propose using MPNNs to remodel the SIR diffusion process. Denoting  $P_{\Omega}^{i}(t) \equiv P(x_{t}^{i} = \Omega)$  as the probability that node  $i$  is in state  $\Omega \in \{S, I, R\}$  at time  $t$  in the SIR diffusion process. It can be observed that this process is structurally equivalent to:
$$
\left\{ \begin{array}{r c l} m _ {\Omega} ^ {i} (t + 1) & = & \sum_ {j \in N (i)} M _ {t} \left(h \left(P _ {\Omega} ^ {i} (t)\right), h \left(P _ {\Omega} ^ {j} (t)\right), e _ {i j}\right) \\ P _ {\Omega} ^ {i} (t + 1) & = & U _ {t} \left(h \left(P _ {\Omega} ^ {i} (t)\right), m _ {\Omega} ^ {i} (t + 1)\right) \\ h \left(P _ {\Omega} ^ {i} (t)\right) & = & \sigma \left(\mathbf {W} _ {\Omega} P _ {\Omega} ^ {i} (t) + \mathbf {b} _ {\Omega}\right) \end{array} \right. \tag {14}
$$
where  $M_{t}$  is the aggregate function and  $U_{t}$  represents the node status update function.  $\sigma(\cdot)$  is a nonlinear activation function.  $e_{ij}$  represents the edge between nodes i and j. Additionally, since  $h(P_{\Omega}^{i}(t))$  already contains nonlinear transformations, more complex forms of transformation are not necessary and  $U_{t}$  can be defined as:  $U_{t}(a,b) = a + b$ . To sum up, Equation 14 can be simplified as:
$$
P _ {\Omega} ^ {i} (t + 1) = h \left(P _ {\Omega} ^ {i} (t)\right) + \sigma \left(\sum_ {j \in N (i)} h \left(P _ {\Omega} ^ {j} (t)\right)\right) \tag {15}
$$
We can achieve this using a residual block composed of graph convolutional networks, which allows us to easily fit the SIR model:
$$
\left\{ \begin{array}{l} \mathbf {h} _ {i, t} ^ {(0)} = \mathbf {S N} \left(\mathbf {U} x _ {i} ^ {t}\right) \text {w i t h} \mathbf {U} \in \mathbb {R} ^ {C \times M} \\ g (\mathbf {h}) = \sigma_ {g} \left(\mathbf {D} ^ {- 1 / 2} \mathbf {A D} ^ {- 1 / 2} \cdot \mathbf {h} \cdot \mathbf {w} + \mathbf {b}\right) \\ \mathbf {h} _ {\mathbf {i}, \mathbf {t}} ^ {(1 + 1)} = \mathbf {h} _ {\mathbf {i}, \mathbf {t}} ^ {(1)} + \sigma \left(\mathbf {S N} \left(\mathbf {B N} \left(g \left(\mathbf {h} _ {\mathbf {i}, \mathbf {t}} ^ {(1)}\right)\right)\right)\right) \end{array} \right. \tag {16}
$$
Among them,  $\mathbf{U}$  is a linear transformation,  $\mathbf{h}_{i,t}^{(l)}$  denotes the representation of the  $l$ -th layer of the network,  $\mathbf{D}$  is the degree matrix,  $\mathbf{SN}(\cdot)$  and  $\mathbf{BN}(\cdot)$  represent spectral normalization and batch normalization, respectively.  $\sigma(\cdot)$  and  $\sigma_g(\cdot)$  are nonlinear activation functions<sup>6</sup>. In fact,  $f_w$  and  $\mathbf{F}^7$  in Equation 13 are the SN  $(\mathbf{U}(\cdot))$  and residual blocks in Equation 16.
<sup>6</sup>We set  $\sigma (\cdot)$  and  $\sigma_g(\cdot)$  to Mish and LeakyReLU respectively.
${}^7\mathbf{F} = \xi +f_\theta (\xi)$  , where  $f_{\theta}(\xi) = \sigma$  (SN (BN  $(\xi))$
3.4.2 Reversibility of residual blocks
We now discuss the invertibility of Equation 13 as reconstructed from Equation 16. Equation 13 can be written as:
$$
\left\{ \begin{array}{l} \mathbf {S N} (\mathbf {U} (\mathbf {X} _ {\mathbf {0}})) = f _ {w} (\mathbf {X} _ {\mathbf {0}}) = \xi ; \\ \xi + \sigma (\mathbf {S N} (\mathbf {B N} (g (\xi)))) = \xi + f _ {\theta} (\xi) = \mathbf {F} (\xi) = \mathbf {Y} _ {\mathbf {T}}. \end{array} \right. \tag {17}
$$
Lemma 3.1 Denoting the Lipschitz constants of  $f_w$  and  $f_\theta$  to be  $L_w$  and  $L_\theta$ , then  $L_w < 1$  and  $L_\theta < 1$ .
Sketch of Proof. Since  $f_w$  and  $\mathbf{BN}(g(\xi))$  are spectrally normalized, their Lipschitz coefficients are less than 1 [28]. Additionally, since the  $\sigma(\cdot)$  is set to  $Mish(\cdot)$ , it can be determined that the Lipschitz constant of their composite function (i.e.,  $f_\theta$ ) is less than 1.
Theorem 3.2 If  $L_{w} < 1$  and  $L_{\theta} < 1$ , then  $\mathbf{Y}_{\mathbf{T}} = \mathbf{F}(f_w(\mathbf{X}_0))$  is reversible.
Sketch of Proof.  $\mathbf{Y}_{\mathbf{T}} = \mathbf{F}(f_w(\mathbf{X}_0))$  can be written as  $\left\{ \begin{array}{l}\mathbf{X_0} = \xi +\mathbf{X_0} - f_w(\mathbf{X_0})\\ \xi = \mathbf{Y_T} - f_\theta (\xi) \end{array} \right.$ . Since  $f_{w} < 1$  and  $f_{\theta} < 1$ , constructing the iteration according to the Banach fixed point theorem [4],  $\mathbf{Y}_{\mathbf{T}} = \mathbf{F}(f_w(\mathbf{X}_0))$  is invertible.
In practical applications, we typically use multiple residual blocks to form a residual network. This approach is employed to improve the receptive field of the GCN and alleviate the problem of node smoothing caused by multi-layer graph convolution. The multi-layer reversible residual network that we designed is depicted in Figure 2, where  $\mathbf{D}\mathbf{P}$  represents dropout. If the dropout rate is  $r$  and the  $i$ -th layer residual block is  $\mathbf{F}_{\mathbf{i}}$ , then we have  $\mathbf{F}_{\mathbf{i}}(\xi) = \mathbf{D}\mathbf{P}(\xi + f_{w}(\xi))$ .
Theorem 3.3 If dropout-rate  $= 0.5$ ,  $L_{w} < 1$ , and  $L_{\theta} < 1$ , the residual network  $\mathbf{Y}_{\mathbf{T}} = (\mathbf{F}_1 \circ \mathbf{F}_2 \circ \ldots \circ \mathbf{F}_n)(f_w(\mathbf{X}_0))$  is reversible.
Sketch of Proof. Denoting this multilayer residual network as  $F$ , the upper bound of  $L_{F}$  is the product of the Lipschitz constants of each function [13]. Additionally, with a dropout rate of  $r$ , the Lipschitz constants of the functions will be limited to  $(1 - r)$  times their original values. Using these two conclusions, we can calculate the upper bound of the Lipschitz constant of a multilayer reversible network with  $L_{F} \leq 1$  when  $r = 0.5$ .
Datasets. The diffusion of information occurs in a broad range of network types, and DDMSL was evaluated on five distinct realistic datasets: Karate [48], Jazz [12], Cora-ML [27], Power Grid [45], and PGP [5]. The detailed parameters of these networks are provided in the Appendix F.2. Following previous research [43, 9, 22], we conducted SIR and SI diffusion simulations on each dataset by randomly selecting  $10\%$  of the nodes as source nodes at the initial moment, and stopping the simulation when approximately  $50\%$  of the nodes were infected<sup>8</sup> (For PGP dataset, simulation stopped at  $30\%$  infection rate). We randomly divided each generated dataset into a training set, a validation set and a test set in the ratio of  $8:1:1$ .
Evaluation Metrics. Our objective consists of two components: source localization and information diffusion paths recovery. The source localization task is a binary classification task, and thus evaluated using four metrics: Precision (PR), which denotes the proportion of nodes predicted as sources that are true sources; Recall (RE), which represents the proportion of actual source nodes that are correctly predicted; F1 score, the harmonic mean of PR and RE; and ROC-AUC (AUC), quantifying the model's ability to classify accurately. To evaluate the performance of the recovering information diffusion paths, we adopted the Mean Squared Error (MSE) error in [8]:  $MSE = \frac{1}{NT} \sum_{t=0}^{T-1} \left\| \mathbf{X}_t - \hat{\mathbf{X}}_t \right\|^2$ , where  $\hat{X}_t$  is the ground truth. This metric is solely computed for infected nodes, with nodes in the recovered and susceptible states being marked as 0.
While some previous studies [9, 44] commenced source localization when the infection rate reached  $30\%$ , we set it to  $50\%$  to consider more complex scenarios.
4.2 Baseline algorithms and experimental settings
Six algorithms for source location were selected to compare with DDMSL: DDMIX [8] employs a Variational Autoencoder (VAE) to reconstruct the outbreak propagation process; LPSI [44] aggregates information to learn source detection through label propagation; GCNSI [9] uses graph convolutional network to learn manually formulated node features, enabling the discrimination of multiple source nodes; OJC [52] locates the source node by minimising the eccentricity of the infection; NETSLEUTH [30] filters multi-source nodes based on minimum description distance, but works solely on the SI diffusion model; and SLVAE [22] deploys generative models to learn the distribution of source nodes. Appendix F.1 provides the experimental settings and tuned hyperparameters for these algorithms, while implementation details on DDMSL are shown in Appendix E.
4.3 Experimental results
while comparing DDMSL with other algorithms on different diffusion models, we have focused on the widely used SIR and SI models, which are effective in capturing the dynamics of various information diffusion processes found in nature. It is noteworthy that DDMSL's flexibility and robustness enable it to handle real-world scenarios where the diffusion process may undergo significant variations by simply designing the corresponding state transfer matrices<sup>9</sup>.
Table 1: Performance comparison (mean of five rounds) of source localization under the SIR model. The best results are highlighted in bold, while the underlined indicates the best of the five baselines. Statistical significance from paired t-test depicts: \*\*\*, \*\*,  $*$  for p-value  $< 0.01$ , 0.05, 0.1 respectively.
Karate	Jazz	Cora MI	Power Grid	PGP
Methods	PR	RE	F1	AUC	PR	RE	F1	AUC	PR	RE	F1	AUC	PR	RE	F1	AUC	PR	RE	F1	AUC
DDMSL	0.708	0.736	0.722	0.853	0.817	0.881	0.848	0.930	0.894	0.867	0.880	0.928	0.833	0.879	0.855	0.930	0.856	0.903	0.879	0.943
GCNSI	0.275	0.410	0.329	0.671	0.301	0.363	0.330	0.641	0.247	0.273	0.260	0.591	0.165	0.182	0.173	0.540	0.554	0.543	0.549	0.748
LPSI	0.211	0.393	0.274	0.646	0.400	0.098	0.158	0.543	0.246	0.026	0.048	0.509	0.193	0.012	0.022	0.503	0.518	0.437	0.474	0.696
SLVAE	0.552	0.400	0.464	0.696	0.750	0.576	0.651	0.778	0.815	0.721	0.765	0.852	0.908	0.719	0.803	0.856	0.817	0.721	0.766	0.851
OJC	0.178	0.265	0.213	0.594	0.147	0.161	0.154	0.535	0.114	0.114	0.114	0.508	0.109	0.109	0.109	0.505	0.128	0.128	0.128	0.516
DDMIX	0.289	0.234	0.258	0.308	0.215	0.197	0.205	0.238	0.162	0.273	0.204	0.250	0.333	0.253	0.287	0.346	0.172	0.194	0.182	0.213
Improve.	28.3%	84.3%	55.7%	22.5%	8.9%	52.9%	30.1%	19.6%	9.7%	20.2%	15.0%	8.9%	-8.3%	22.2%	6.6%	8.7%	4.8%	25.3%	14.8%	10.8%
Significance	***	***	***	***	**	***	**	*	***	*	**	*	***	***	***	***	***	***	***	***
Table 2: Performance comparison (mean of five rounds) of source localization under the SI model. The best results are highlighted in bold, while the underlined indicates the best of the six baselines. Statistical significance from paired t-test depicts: ***, **, * for p-value < 0.01, 0.05, 0.1 respectively.
Karate	Jazz	Cora MI	Power Grid	PGP
Methods	PR	RE	F1	AUC	PR	RE	F1	AUC	PR	RE	F1	AUC	PR	RE	F1	AUC	PR	RE	F1	AUC
DDMSL	0.706	0.980	0.798	0.972	0.782	0.853	0.813	0.914	0.790	0.908	0.845	0.941	0.763	0.966	0.852	0.966	0.754	0.887	0.815	0.928
GCNSI	0.357	0.456	0.401	0.687	0.366	0.426	0.394	0.676	0.321	0.354	0.337	0.636	0.335	0.325	0.330	0.639	0.487	0.370	0.421	0.665
LPSI	0.339	0.414	0.351	0.681	0.474	0.097	0.156	0.544	0.494	0.207	0.291	0.592	0.343	0.277	0.306	0.609	0.453	0.284	0.349	0.623
SLVAE	0.591	0.477	0.503	0.733	0.888	0.579	0.691	0.785	0.841	0.728	0.780	0.856	0.815	0.780	0.797	0.880	0.844	0.633	0.723	0.810
OJC	0.267	0.396	0.318	0.663	0.120	0.127	0.123	0.517	0.125	0.125	0.125	0.514	0.178	0.178	0.178	0.544	0.118	0.118	0.118	0.510
DDMIX	0.253	0.377	0.303	0.654	0.244	0.133	0.172	0.212	0.220	0.222	0.221	0.247	0.345	0.235	0.280	0.340	0.189	0.186	0.187	0.207
NetSleuth	0.239	0.339	0.279	0.634	0.216	0.252	0.233	0.580	0.217	0.229	0.223	0.569	0.206	0.216	0.211	0.562	0.200	0.210	0.205	0.558
Improve.	19.4%	105.5%	58.5%	32.5%	-11.9%	47.3%	17.6%	16.3%	-6.0%	24.8%	8.3%	9.9%	-6.3%	23.8%	7.0%	9.8%	-10.6%	40.2%	12.7%	14.5%
Significance	***	***	***	***	***	***	***	***	***	***	***	***	*	*	**	**	***	***	***	***
Source Location. Table 1 shows the performance of DDMSL in SIR diffusion mode, where it outperforms all baseline algorithms regarding almost all metrics. Notably, DDMSL leads the best of the baseline algorithms by approximately  $25\%$  and  $14\%$  on F1 score and AUC, respectively. Meanwhile, Table 2 shows similar results in the SI diffusion mode. While SLVAE has a higher precision than DDMSL, DDMSL surpasses SLVAE by approximately  $21\%$  and  $17\%$  in terms of F1 score and AUC, indicating that DDMSL is more accurate in identifying source nodes. Moreover, the performance of DDMSL varies across datasets and is mainly influenced by the network's typologies (Refer to Table 6) and the scale of information diffusion. The performance of DDMSL in solving the inverse problem generally has a negative correlation with the average clustering coefficient and density of the network. The larger these two parameters are, the more complex the scenario of information diffusion is, and the accuracy of DDMSL in detecting source points decreases, which aligns with our intuition.
9Please refer to Appendix E for measuring the state transition matrices for different propagation models.
(a) MSE reconstruction error in SIR mode.
(b) MSE reconstruction error in SI mode.
Figure 3: Comparisons of DDMSL and DDMIX on reconstructed information diffusion processes.
Reconstructing information diffusion paths. Both DDMSL and DDMIX can reconstruct the evolution of information diffusion, but DDMIX can only recover the states of susceptible nodes  $(S)$  and infected nodes  $(I)$ , whereas DDMSL can reconstruct nodes of all states. To facilitate fair comparison, we calculate MSE loss for node reconstruction in states  $S$  and  $I$ , as shown in Figure 3. DDMSL consistently outperforms DDMIX on all datasets, especially in SI diffusion mode where DDMSL's average reconstruction loss is  $69\%$  lower than DDMIX's. Unlike DDMIX, which recovers the entire graph state, DDMSL allows for fine-grained node-level reconstruction.
Additional experiments. DDMSL represents a source detection algorithm relying on diffusion models, necessitating an assessment of its aptitude for generalization. Concurrently, we assessed its computational complexity while also incorporating experiments utilizing two real diffusion datasets. We have documented the results of these experiments in Appendix G.
Visualization. To further showcase the efficacy of DDMSL in solving the inverse problem of information diffusion on complex networks, we designed a visualization that demonstrates source localization and the propagation evolution process. Due to space constraints, we have presented the visualization in Appendix H.
Table 3: The results of the ablation study.
Karate	Jazz	Cora MI	Power Grid	PGP
Methods	PR	RE	F1	AUC	PR	RE	F1	AUC	PR	RE	F1	AUC	PR	RE	F1	AUC	PR	RE	F1	AUC
DDMSL	0.708	0.736	0.722	0.853	0.817	0.881	0.848	0.930	0.894	0.867	0.880	0.928	0.833	0.879	0.855	0.930	0.856	0.903	0.879	0.943
DDMSL(a)	0.264	0.910	0.379	0.827	0.254	0.683	0.367	0.741	0.277	0.626	0.384	0.722	0.459	0.605	0.522	0.706	0.366	0.857	0.513	0.846
DDMSL(b)	0.655	0.850	0.718	0.860	0.648	0.822	0.723	0.889	0.834	0.856	0.845	0.915	0.818	0.875	0.845	0.919	0.811	0.888	0.848	0.922
DDMSL(c)	0.339	0.185	0.236	0.592	0.931	0.111	0.1943	0.556	0.817	0.043	0.08	0.521	0.998	0.2756	0.432	0.638	0.997	0.656	0.792	0.823
To evaluate the effectiveness of each component, we conducted ablation experiments on DDMSL in SIR diffusion mode. One essential component of DDMSL is the reversible residual network. We removed this component and replaced each reversible residual block with a GCN network, denoted as DDMSL(a). Additionally, DDMSL is also supervised by  $L_{\text{constant}}$  on the propagation rules. The model without the propagation rule supervision module is denoted as DDMSL(b). Finally, to assess the impact of GCN (Graph Convolutional Network) in reversible residual blocks, DDMSL(c) represents the model after removing the GCN modules from residual blocks. We ran the three variants on five datasets and compared the results,
Figure 4: MSE error in ablations.
which are shown in Table 3 and Figure 4. As can be seen in table 3, we have these observations: (1) removing the residual module (DDMSL(a)) leads to a significant degradation in model performance, with the F1 score and AUC decreasing by  $49\%$  and  $17\%$ , respectively. Although the GCN network is also a form of MPNNs, we noticed that the output of the inference network composed of GCNs were very similar, indicating node feature oversmoothing, and further highlighting the effectiveness of the reversible residual network module; (2)  $L_{constraint}$  is also effective, which contributes to  $5\%$  and  $2\%$  performance improvement in terms of F1 score and AUC, respectively. Similar results can be obtained in Figure 4 regarding the task of reconstructing the information diffusion processes; and (3) upon the drop of the GCNs from the reversible residual network (DDMSL(c)), a noteworthy deterioration model performance was observed, manifesting as a substantial decrease in F1 scores and AUC by  $59\%$  and  $32\%$  respectively. This compellingly signifies the indispensable role of GCN in acquiring the diffusion mode of the SIR model through the process of learning. Similar results can be obtained in Figure 4 regarding the task of reconstructing the information diffusion processes.
In this paper, we introduced a reversible residual network block based on the relationship between diffusion phenomena and message passing neural networks, while ensuring the reversibility of the network by limiting its Lipschitz coefficient. Using this, we constructed a discrete denoising diffusion model (DDMSL) which can locate the source of graph diffusion and restore the diffusion paths. Extensive experiments on five real datasets have demonstrated the effectiveness of DDMSL and its constituent modules. Our work offers insights into how to calculate the distribution of solutions to graph diffusion inverse problems based on the information propagation laws on complex networks.
Solving the inverse problem of graph diffusion plays a crucial role in many social operations, including controlling the spread of infectious diseases, rumors, and computer viruses. It provides valuable insights on enhancing source detection performance and fills the gap in methods for recovering diffusion evolution. However, our work has some limitations. For instance, DDMSL requires the prior knowledge about propagation models, such as infection rate and recovery rate. Although we can infer these parameters from existing observation data [32, 21], it limits the application of DDMSL in situations with insufficient observations of propagation conditions. Our future research will focus on reducing the dependence of DDMSL on prior conditions.
This work was supported in part by the National Key R&D Program of China under Grant No. 2021YFC3300302, the National Natural Science Foundation of China (Grant No. 62202089, 72192832 and U22A2004), the General project of Liaoning Provincial Department of Education (Grant No. LJKZ0005), the Shanghai Rising-Star Program (Grant No. 23QA1403100), the Natural Science Foundation of Shanghai (Grant No. 21ZR1421900), and the Fundamental Research Funds for the Central Universities (Grant No. N2319004).
[53] Linhe Zhu, Gui Guan, and Yimin Li. Nonlinear dynamical analysis and control strategies of a network-based sis epidemic model with time delay. Applied Mathematical Modelling, 70:512-531, 2019.
[54] Arkaitz Zubiaga, Ahmet Aker, Kalina Bontcheva, Maria Liakata, and Rob Procter. Detection and resolution of rumours in social media: A survey. ACM Computing Surveys (CSUR), 51(2):1-36, 2018.
A Broader impacts
Overall, our work offers valuable insights into how to limit the spread of malicious information. For example, by tracing the spread of infectious diseases, we can identify potential contacts of infected individuals, effectively containing the outbreak. However, it is important to consider the potential negative implications of this approach for society, such as compromising the privacy of individuals living with infectious diseases like HIV. Addressing these concerns should be important.
B Derivation of forward and backward processes
Derivation of forward processes. Extending Equation 4 by means of the Markov property:
$$
\begin{array}{l} q \left(\boldsymbol {x} _ {t} ^ {i} \mid \boldsymbol {x} _ {0} ^ {i}\right) = \sum_ {\boldsymbol {x} _ {1: t - 1} ^ {i}} \prod_ {k = 1} ^ {t} q \left(\boldsymbol {x} _ {k} ^ {i} \mid \boldsymbol {x} _ {k - 1} ^ {i}\right) \\ = \sum_ {\boldsymbol {x} _ {1: t - 1} ^ {i}} \prod_ {k = 1} ^ {t} \boldsymbol {x} _ {k - 1} ^ {i} Q _ {k} ^ {i} \boldsymbol {x} _ {\boldsymbol {k}} ^ {\boldsymbol {i}} ^ {T} \\ = \sum_ {\boldsymbol {x} _ {1: t - 1} ^ {i}} \boldsymbol {x} _ {0} ^ {i} Q _ {1} ^ {i} \boldsymbol {x} _ {1} ^ {i ^ {T}} \dots \boldsymbol {x} _ {k - 1} ^ {i} Q _ {k} ^ {i} \boldsymbol {x} _ {k} ^ {i ^ {T}} \dots \boldsymbol {x} _ {t - 1} ^ {i} Q _ {t} ^ {i} \boldsymbol {x} _ {t} ^ {i ^ {T}} \tag {18} \\ = \boldsymbol {x} _ {0} ^ {i} Q _ {1} ^ {i} \left(\sum_ {\boldsymbol {x} _ {1} ^ {i}} \boldsymbol {x} _ {1} ^ {i} ^ {T} \boldsymbol {x} _ {1} ^ {i}\right) \dots \left(\sum_ {\boldsymbol {x} _ {t - 1} ^ {i}} \boldsymbol {x} _ {t - 1} ^ {i} ^ {T} \boldsymbol {x} _ {t - 1} ^ {i}\right) Q _ {t} ^ {i} \boldsymbol {x} _ {t} ^ {i} ^ {T} \\ = \pmb {x} _ {0} ^ {i} Q _ {1} ^ {i} I Q _ {2} ^ {i} \dots I Q _ {k} ^ {i} I \dots I Q _ {t} ^ {i} \pmb {x} _ {\pmb {t}} ^ {i T} \\ = \boldsymbol {x} _ {0} ^ {i} \bar {Q} _ {t} ^ {i} \boldsymbol {x} _ {\boldsymbol {t}} ^ {\boldsymbol {i} ^ {T}} \sim \operatorname {C a t} \left(\boldsymbol {x} _ {t} ^ {i}; \boldsymbol {p} = \boldsymbol {x} _ {0} ^ {i} \bar {Q} _ {t} ^ {i}\right) \\ \end{array}
$$
Where  $I$  is the identity matrix.
Derivation of backward processes. The detailed derivation of Equation 6 is as follows:
$$
\begin{array}{l} q \left(\boldsymbol {x} _ {t - 1} ^ {i} \mid \boldsymbol {x} _ {t} ^ {i}, \boldsymbol {x} _ {0} ^ {i}\right) = \frac {q \left(\boldsymbol {x} _ {t} ^ {i} \mid \boldsymbol {x} _ {t - 1} ^ {i} , \boldsymbol {x} _ {0} ^ {i}\right) q \left(\boldsymbol {x} _ {t - 1} ^ {i} \mid \boldsymbol {x} _ {0} ^ {i}\right)}{q \left(\boldsymbol {x} _ {t} ^ {i} \mid \boldsymbol {x} _ {0} ^ {i}\right)} \\ = \frac {q \left(\boldsymbol {x} _ {t} ^ {i} \mid \boldsymbol {x} _ {t - 1} ^ {i}\right) q \left(\boldsymbol {x} _ {t - 1} ^ {i} \mid \boldsymbol {x} _ {0} ^ {i}\right)}{q \left(\boldsymbol {x} _ {t} ^ {i} \mid \boldsymbol {x} _ {0} ^ {i}\right)} \\ = \frac {\boldsymbol {x} _ {t - 1} ^ {i} Q _ {t} ^ {i} \boldsymbol {x} _ {t} ^ {\boldsymbol {i} ^ {T}} \cdot \boldsymbol {x} _ {0} ^ {i} \bar {Q} _ {t - 1} ^ {i} \boldsymbol {x} _ {t - 1} ^ {\boldsymbol {i}} {} ^ {T}}{\boldsymbol {x} _ {0} ^ {i} \bar {Q} _ {t} ^ {i} \boldsymbol {x} _ {t} ^ {\boldsymbol {i} ^ {T}}} \\ = \frac {\left(\boldsymbol {x} _ {t} ^ {i} Q _ {t} ^ {i T} \boldsymbol {x} _ {\boldsymbol {t} - 1} ^ {i} {} ^ {T}\right) \cdot \left(\boldsymbol {x} _ {0} ^ {i} \bar {Q} _ {t - 1} ^ {i} \boldsymbol {x} _ {\boldsymbol {t} - 1} ^ {i} {} ^ {T}\right)}{\boldsymbol {x} _ {0} ^ {i} \bar {Q} _ {t} ^ {i} \boldsymbol {x} _ {\boldsymbol {t}} ^ {i T}} \tag {19} \\ = \frac {\left(\boldsymbol {x} _ {t} ^ {i} Q _ {t} ^ {i ^ {T}}\right) \odot \left(\boldsymbol {x} _ {0} ^ {i} \bar {Q} _ {t - 1} ^ {i}\right) \left(\boldsymbol {x} _ {\boldsymbol {t} - \boldsymbol {1}} ^ {i ^ {T}}\right)}{\boldsymbol {x} _ {0} ^ {i} \bar {Q} _ {t} ^ {i} \boldsymbol {x} _ {\boldsymbol {t}} ^ {i ^ {T}}} \\ \sim \operatorname {C a t} \left(\pmb {x} _ {t - 1} ^ {i}; \pmb {p} = \frac {\left(\pmb {x} _ {t} ^ {i} Q _ {t} ^ {i T}\right) \odot \left(\pmb {x} _ {0} ^ {i} \bar {Q} _ {t - 1} ^ {i}\right)}{\pmb {x} _ {0} ^ {i} \bar {Q} _ {t} ^ {i} \pmb {x} _ {t} ^ {i T}}\right) \\ \end{array}
$$
From the Bayesian formula, it follows that: