SlowGuess/NeurIPS_2023_OCR huggingface dataset:
According to the above theorem, global_MATCHing can robustly separate the clients' initial dictionaries into global and local parts, provided that the aggregated error in the initial dictionaries is below a threshold. Specific initialization schemes that can satisfy the condition of Theorem 1 include Algorithm 1 from Agarwal et al. (2013) and Algorithm 3 from Arora et al. (2015). We also remark that since the constructed graph is a DAG, the shortest path problem can be solved in time linear in the number of edges, which is  $\mathcal{O}\left(r_1 + r_N + \sum_{i=1}^{N-1} r_i r_{i+1}\right)$ , via a simple labeling algorithm (see, e.g., (Ahuja et al., 1988, Chapter 4.4)). Since we need to solve the shortest path problem  $r^g$  times, this brings the computational complexity of Algorithm 2 to  $\mathcal{O}(r^g N r_{\max}^2)$ , where  $r_{\max} = \max_i r_i$ .
Given the initial local and global dictionaries, the clients progressively refine their estimates by applying  $T$  rounds of local_update (Algorithm 3). At a high level, each client runs a single iteration of a linearly convergent algorithm  $\mathcal{A}_i$  (see Definition 2), followed by an alignment step that determines the global atoms of the updated dictionary using  $\mathbf{D}^{g,(t)}$  as a "reference point". Notably, our implementation of local_update is adaptive to different DL algorithms. This flexibility is indeed intentional to provide a versatile meta-algorithm for clients with different DL algorithms.
Algorithm 3 local_update
1: Input:  $\mathbf{D}_i^{(t)} = \left[ \begin{array}{ll}\mathbf{D}^{g,(t)} & \mathbf{D}_i^{l,(t)} \end{array} \right],\mathbf{Y}_i$
2:  $\mathbf{D}_i^{(t + 1)} = \mathcal{A}_i\left(\mathbf{Y}_i,\mathbf{D}_i^{(t)}\right) / /$  One iteration of a linearly-convergent algorithm.
3: Initialize  $S$  as an empty set and  $\mathbf{P} \in \mathbb{R}^{r^g \times r^g}$  as an all-zero matrix.
4: for  $j = 1, \dots, r^g$  do
5: Find  $k^{*} = \arg \min_{k}d_{2}\left(\left(\mathbf{D}^{g,(t)}\right)_{(j)},\left(\mathbf{D}_{i}^{(t + 1)}\right)_{(k)}\right)$ .
6: Append  $k^*$  to  $\mathcal{S}$ .
7: Set  $(i,i)$ -th entry of  $\mathbf{P}$  to sign  $\left(\left\langle \left(\mathbf{D}^{g,(t)}\right)_{(j)}, \left(\mathbf{D}_i^{(t + 1)}\right)_{(k^*)} \right\rangle\right)$ .
8: end for
9: Output:  $\mathbf{D}_i^{g,(t + 1)} = \left(\mathbf{D}_i^{(t + 1)}\right)_{(\mathcal{S})}\mathbf{P}$  and  $\mathbf{D}_i^{l,(t + 1)} = \left(\mathbf{D}_i^{(t + 1)}\right)_{([r_i]\backslash \mathcal{S})}$ .
In this section, we show that our proposed meta-algorithm provably solves PerDL under suitable initialization, identifiability, and algorithmic conditions. To achieve this goal, we first present the definition of a linearly-convergent DL algorithm.
Definition 2. Given a generative model  $\mathbf{Y} = \mathbf{D}^{*}\mathbf{X}^{*}$ , a DL algorithm  $\mathcal{A}$  is called  $(\delta, \rho, \psi)$ -linearly convergent for some parameters  $\delta, \psi > 0$  and  $0 < \rho < 1$  if, for any  $\mathbf{D} \in \mathbb{R}^{d \times r}$  such that  $d_{1,2}(\mathbf{D}, \mathbf{D}^{*}) \leq \delta$ , the output of one iteration  $\mathbf{D}^{+} = \mathcal{A}(\mathbf{D}, \mathbf{Y})$ , satisfies
$$
d _ {2, (j)} \left(\mathbf {D} ^ {+}, \mathbf {D} ^ {*}\right) \leq \rho d _ {2, (j)} \left(\mathbf {D}, \mathbf {D} ^ {*}\right) + \psi , \quad \forall 1 \leq j \leq r. \tag {9}
$$
One notable linearly convergent algorithm is introduced by (Arora et al., 2015, Algorithm 5); we will discuss this algorithm in more detail in the appendix. Assuming all clients are equipped with linearly convergent algorithms, our next theorem establishes the convergence of PerMA.
Theorem 2 (Convergence of PerMA). Suppose  $\{\mathbf{D}_i^*\}_{i=1}^N$  are  $\mu$ -incoherent (Assumption 1) and  $\beta$ -identifiable (Assumption 2). Suppose, for every client  $i$ , the DL algorithm  $\mathcal{A}_i$  used in local_update (Algorithm 3) is  $(\delta_i, \rho_i, \psi_i)$ -linearly convergent with  $4\sum_{i=1}^N \delta_i \leq \min \left\{\sqrt{2 - 2\frac{\mu}{\sqrt{d}}}, \beta \right\}$ . Assume the initial dictionaries  $\{\mathbf{D}_i^{(0)}\}_{i=1}^N$  satisfy:
$$
d _ {1, 2} \left(\frac {1}{N} \sum_ {i = 1} ^ {N} \mathbf {D} _ {i} ^ {g, (0)}, \mathbf {D} ^ {g *}\right) \leq \min  _ {1 \leq i \leq N} \delta_ {i}, \quad d _ {1, 2} \left(\mathbf {D} _ {i} ^ {l, (0)}, \mathbf {D} _ {i} ^ {l *}\right) \leq \delta_ {i}, \quad f o r \quad i = 1, \dots , N. \tag {10}
$$
Then, for every  $t \geq 0$ , PerMA (Algorithm 1) satisfies
$$
d _ {1, 2} \left(\mathbf {D} ^ {g, (t)}, \mathbf {D} ^ {g *}\right) \leq \left(\frac {1}{N} \sum_ {i = 1} ^ {N} \rho_ {i}\right) d _ {1, 2} \left(\mathbf {D} ^ {g, (0)}, \mathbf {D} ^ {g *}\right) + \frac {1}{N} \sum_ {i = 1} ^ {N} \psi_ {i}, \tag {11}
$$
$$
d _ {1, 2} \left(\mathbf {D} _ {i} ^ {l, (t)}, \mathbf {D} _ {i} ^ {l *}\right) \leq \rho_ {i} d _ {1, 2} \left(\mathbf {D} ^ {l, (0)}, \mathbf {D} _ {i} ^ {l *}\right) + \psi_ {i}, \quad \text {f o r} \quad 1 \leq i \leq N. \tag {12}
$$
The above theorem sheds light on a number of key benefits of PerMA:
Relaxed initial condition for weak clients. Our algorithm relaxes the initial condition on the global dictionaries. In particular, it only requires the average of the initial global dictionaries to be close to the true global dictionary. Consequently, it enjoys a provable convergence guarantee even if some of the clients do not provide a high-quality initial dictionary to the server.
Improved convergence rate for slow clients. During the course of the algorithm, the global dictionary error decreases at an average rate of  $\frac{1}{N}\sum_{i=1}^{N}\rho_i$ , improving upon the convergence rate of the slow clients.
Improved statistical error for weak clients. A linearly convergent DL algorithm  $\mathcal{A}_i$  stops making progress upon reaching a neighborhood around the true dictionary  $\mathbf{D}_i^*$  with radius  $O(\psi_i)$ . This type of guarantee is common among DL algorithms (Arora et al., 2015; Liang et al., 2022) and often corresponds to their statistical error. In light of this, PerMA improves the performance of weak clients (i.e., clients with weak statistical guarantees) by borrowing strength from strong ones.
In this section, we showcase the effectiveness of Algorithm 1 using synthetic and real data. All experiments are performed on a MacBook Pro 2021 with the Apple M1 Pro chip and 16GB unified memory for a serial implementation in MATLAB 2022a. Due to limited space, we will only provide the high-level motivation and implication of our experiments. We defer implementation details and comparisons with the existing methods to the appendix.
In this section, we validate our theoretical results on a synthetic dataset. We consider ten clients, each with a dataset generated according to the model 1. The details of our construction are presented in the appendix. Specifically, we compare the performances of two strategies: (1) independent strategy, where each client solves DL without any collaboration, and (2) collaborative strategy, where clients collaboratively learn the ground truth dictionaries by solving PerDL via the proposed meta-algorithm PerMA. We initialize both strategies using the same  $\{\mathbf{D}_i^{(0)}\}_{i = 1}^N$ . The initial dictionaries are obtained via a warm-up method proposed in (Liang et al., 2022, Algorithm 4). For a fair comparison between independent and collaborative strategies, we use the same DL algorithm ((Liang et al., 2022, Algorithm 1)) for different clients. Note that in the independent strategy, the clients cannot separate global
from local dictionaries. Nonetheless, to evaluate their performance, we collect the atoms that best align with the true global dictionary  $\mathbf{D}^{g*}$  and treat them as the estimated global dictionaries. As can be seen in Figure 2, three out of ten clients are weak learners and fail to recover the global dictionary with desirable accuracy. On the contrary, in the collaborative strategy, all clients recover the same global dictionary almost exactly.
Figure 2: PerMA improves the accuracy of the recovered global dictionary for all clients, even if some (three out of ten) are weak learners.
5.2 Training with Imbalanced Data
In this section, we showcase the application of PerDL in training with imbalanced datasets. We consider an image reconstruction task on MNIST dataset. This dataset corresponds to a set of handwritten digits (see the first row of Figure 3). The goal is to recover a single concise global dictionary that can be used to reconstruct the original handwritten digits as accurately as possible. In particular, we study a setting where the clients have imbalanced label distributions. Indeed, data imbalance can drastically bias the performance of the trained model in favor of the majority groups, while hurting its performance on the minority groups (Leevy et al., 2018; Thabtah et al., 2020). Here, we consider a setting where the clients have highly imbalanced datasets, where  $90\%$  of their samples have the same label. More specifically, for client  $i$ , we assume that  $90\%$  of the samples correspond to the handwritten digit " $i$ ", with the remaining  $10\%$  corresponding to other digits. The second row of Figure 3 shows the effect of data imbalance on the performance of the recovered dictionary on a single client, when the clients do not collaborate. The last row of Figure 3 shows the improved performance of the recovered dictionary via PerDL on the same client. Our experiment clearly shows the ability of PerDL to effectively address the data imbalance issue by combining the strengths of different clients.
Figure 3: PerMA improves training with imbalanced datasets. We consider the image reconstruction task on the imbalanced MNIST dataset using only five atoms from a learned global dictionary. The first row corresponds to the original images. The second row is based on the dictionary learned on a single client with an imbalanced dataset. The third row shows the improved performance of the learned dictionary using our proposed method on the same client.
Global Atoms Reconstruction
Local Atoms Reconstruction
Figure 4: PerMA effectively separates the background from moving objects in video frames. Here we reconstruct the surveillance video frames using global and local dictionaries learned by PerMA. We reconstruct the frames using only 50 atoms from the combined dictionaries.
5.3 Surveillance Video Dataset
As a proof of concept, we consider a video surveillance task, where the goal is to separate the background from moving objects. Our data is collected from Vacavant et al. (2013) (see the first column of Figure 4). As these frames are taken from one surveillance camera, they share the same background corresponding to the global features we aim to extract. The frames also exhibit heterogeneity as moving objects therein are different from the background. This problem can indeed be modeled as an instance of PerDL, where each video frame can be assigned to a "client", with the global dictionary capturing the background and local dictionaries modeling the moving objects. We solve PerDL by applying PerMA to obtain a global dictionary and several local dictionaries for this dataset. Figure 4 shows the reconstructed background and moving objects via the recovered global and local dictionaries. Our results clearly show the ability of our proposed framework to separate global and local features.
2We note that moving object detection in video frames has been extensively studied in the literature and typically solved very accurately via different representation learning methods (such as robust PCA and neural network modeling); see (Yazdi and Bouwmans, 2018) for a recent survey. Here, we use this case study as a proof of concept to illustrate the versatility of PerMA in handling heterogeneity, even in settings where the data is not physically distributed among clients.
6 Social Impact, Limitations and Future Directions
Our novel approach for personalized dictionary learning presents a versatile solution with immediate applications across various domains, such as video surveillance and object detection. While these applications offer valuable benefits, they also bring to the forefront ethical and societal concerns, particularly concerning privacy, bias, and potential misuse. In the context of video surveillance, deploying object detection algorithms may inadvertently capture private information, leading to concerns about violating individuals' right to privacy. However, it is important to emphasize that our proposed algorithm is specifically designed to focus on separating unique and common features within the data, without delving into the realm of personal information. Thus, it adheres to ethical principles by ensuring that private data is not processed or used without explicit consent. Bias is another critical aspect that necessitates careful consideration in the deployment of object detection algorithms. Biases can manifest in various forms, such as underrepresentation or misclassification of certain groups, leading to discriminatory outcomes. Our approach acknowledges the importance of mitigating biases by solely focusing on the distinction between common and unique features, rather than introducing any inherent bias into the learning process. Furthermore, the potential misuse of object detection algorithms in unauthorized surveillance or invasive tracking scenarios raises valid concerns. As responsible researchers, we are cognizant of such risks and stress that our proposed algorithm is meant to be deployed in a controlled and legitimate manner, adhering to appropriate regulations and ethical guidelines.
Even though our meta-algorithm PerMA enjoys strong theoretical guarantees and practical performance, there are still several avenues for improvement. For instance, the theoretical success of PerMA, especially the Global Matching step, relies on an individual initial error of  $O(1 / N)$ . In other words, the initial error should decrease as the number of clients grows. As a future work, we plan to relax such dependency via a more delicate analysis. We also note that imposing an upper bound on the initial error is not unique to our setting, as virtually all existing algorithms for classical (non-personalized) dictionary learning require certain conditions on the initial error. On the other hand, once the assumption on the initial error is satisfied, our meta-algorithm achieves a final error with the same dependency on  $d$  (the dimensionality of the data) and  $n$  (the number of samples) as the state-of-the-art algorithms for classical dictionary learning (Agarwal et al. (2016), Arora et al. (2015)). Remarkably, this implies that personalization is achieved without incurring any additional cost on  $d$  or  $n$ , making PerMA highly efficient and competitive in its performance.
S.F. is supported, in part, by NSF Award DMS-2152776, ONR Award N00014-22-1-2127, and MICDE Catalyst Grant. R. A. K. is supported in part by NSF CAREER 2144147.
Agarwal, A., Anandkumar, A., Jain, P., and Netrapalli, P. (2016). Learning sparsely used overcomplete dictionaries via alternating minimization. SIAM Journal on Optimization, 26(4):2775-2799.
Agarwal, A., Anandkumar, A., and Netrapalli, P. (2013). Exact recovery of sparsely used overcomplete dictionaries. stat, 1050:8-39.
Ahuja, R. K., Magnanti, T. L., and Orlin, J. B. (1988). Network flows.
Arora, S., Ge, R., Ma, T., and Moitra, A. (2015). Simple, efficient, and neural algorithms for sparse coding. In Conference on learning theory, pages 113-149. PMLR.
Arora, S., Ge, R., and Moitra, A. (2014). New algorithms for learning incoherent and overcomplete dictionaries. In Conference on Learning Theory, pages 779-806. PMLR.
Bhagoji, A. N., Chakraborty, S., Mittal, P., and Calo, S. (2019). Analyzing federated learning through an adversarial lens. In International Conference on Machine Learning, pages 634-643. PMLR.
Chatterji, N. and Bartlett, P. L. (2017). Alternating minimization for dictionary learning with random initialization. Advances in Neural Information Processing Systems, 30.
Shi, N. and Kontar, R. A. (2022). Personalized pca: Decoupling shared and unique features. arXiv preprint arXiv:2207.08041.
Spielman, D. A., Wang, H., and Wright, J. (2012). Exact recovery of sparsely-used dictionaries. In _Conference on Learning Theory_, pages 37-1. JMLR Workshop and Conference Proceedings.
Sun, J., Qu, Q., and Wright, J. (2016). Complete dictionary recovery over the sphere i: Overview and the geometric picture. IEEE Transactions on Information Theory, 63(2):853-884.
T Dinh, C., Tran, N., and Nguyen, J. (2020). Personalized federated learning with moreau envelopes. Advances in Neural Information Processing Systems, 33:21394-21405.
Thabtah, F., Hammoud, S., Kamalov, F., and Gonsalves, A. (2020). Data imbalance in classification: Experimental evaluation. Information Sciences, 513:429-441.
Tošić, I. and Frossard, P. (2011). Dictionary learning. IEEE Signal Processing Magazine, 28(2):27-38.
Tropp, J. A. (2006). Just relax: Convex programming methods for identifying sparse signals in noise. IEEE transactions on information theory, 52(3):1030-1051.
Vacavant, A., Chateau, T., Wilhelm, A., and Lequievre, L. (2013). A benchmark dataset for outdoor foreground/background extraction. In Computer Vision-ACCV 2012 Workshops: ACCV 2012 International Workshops, Daejeon, Korea, November 5-6, 2012, Revised Selected Papers, Part I 11, pages 291-300. Springer.
Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D., and Khazaeni, Y. (2020). Federated learning with matched averaging. arXiv preprint arXiv:2002.06440.
Yazdi, M. and Bouwmans, T. (2018). New trends on moving object detection in video images captured by a moving camera: A survey. Computer Science Review, 28:157-177.
Yue, X., Nouiehed, M., and Al Kontar, R. (2022). Gifair-fl: A framework for group and individual fairness in federated learning. INFORMS Journal on Data Science.
Zhao, P. and Yu, B. (2006). On model selection consistency of lasso. The Journal of Machine Learning Research, 7:2541-2563.
Zhao, R., Li, H., and Liu, X. (2021). A survey of dictionary learning in medical image analysis and its application for glaucoma diagnosis. Archives of Computational Methods in Engineering, 28(2):463-471.
A Further Details on the Experiments
In this section, we provide further details on the numerical experiments reported in Section 5.
A.1 Details of Section 5.1
In Section 5.1, we generate the synthetic datasets according to Model 1 with  $N = 10$ ,  $d = 6$ ,  $r_i = 6$ , and  $n_i = 200$  for each  $1 \leq i \leq 10$ . Each  $\mathbf{D}_i^*$  is an orthogonal matrix with the first  $r^g = 3$  columns shared with every other client and the last  $r_i^l = 3$  columns unique to themselves. Each  $\mathbf{X}_i^*$  is first generated from a Gaussian-Bernoulli distribution where each entry is non-zero with a probability 0.2. Then,  $\mathbf{X}_i^*$  is further truncated, where all the entries  $(\mathbf{X}_i^*)_{(j,k)}$  with  $|\mathbf{X}_i^*)(_{j,k})| < 0.3$  are replaced by  $(\mathbf{X}_i^*)(_{j,k}) = 0.3 \times \mathrm{sign}((\mathbf{X}_i^*)(_{j,k}))$ .
We use the orthogonal DL algorithm (Algorithm 4) introduced in (Liang et al., 2022, Algorithm 1) as the local DL algorithm for each client. This algorithm is simple to implement and comes equipped with a strong convergence guarantee (see (Liang et al., 2022, Theorem 1)). Here  $\mathrm{HT}_{\zeta}(\cdot)$  denotes the hard-thresholding operator at level  $\zeta$ , which is defined as:
$$
(\mathrm {H T} _ {\zeta} (\mathbf {A})) _ {(i, j)} = \left\{ \begin{array}{l l} \mathbf {A} _ {(i, j)} & \text {i f} \quad | \mathbf {A} _ {(i, j)} | \geq \zeta , \\ 0 & \text {i f} \quad | \mathbf {A} _ {(i, j)} | <   \zeta . \end{array} \right.
$$
Specifically, we use  $\zeta = 0.15$  for the experiments in Section 5.1.  $\mathrm{Polar}(\cdot)$  denotes the polar decomposition operator, which is defined as  $\mathrm{Polar}(\mathbf{A}) = \mathbf{U}_{\mathbf{A}}\mathbf{V}_{\mathbf{A}}^{\top}$ , where  $\mathbf{U}_{\mathbf{A}}\mathbf{\Sigma}_{\mathbf{A}}\mathbf{V}_{\mathbf{A}}^{\top}$  is the Singular Value Decomposition (SVD) of  $\mathbf{A}$ .
Algorithm 4 Alternating minimization for orthogonal dictionary learning (Liang et al. (2022))
1: Input:  $\mathbf{Y}_i, \mathbf{D}_i^{(t)}$
2: Set  $\mathbf{X}_i^{(t)} = \mathrm{HT}_{\zeta}\left(\mathbf{D}^{(t);T}\mathbf{Y}_i\right)$
3: Set  $\mathbf{D}_i^{(t + 1)} = \mathrm{Polar}\left(\mathbf{Y}_i\mathbf{X}_i^{(t)\top}\right)$
4: return  $\mathbf{D}_i^{(t + 1)}$
For a fair comparison, we initialize both strategies using the same  $\{\mathbf{D}_i^{(0)}\}_{i = 1}^N$ , which is obtained by iteratively calling Algorithm 4 with a random initial dictionary and shrinking thresholds  $\zeta$ . For a detailed discussion on such an initialization scheme we refer the reader to Liang et al. (2022).
A.2 Details of Section 5.2
In section 5.2, we aim to learn a dictionary with imbalanced data collected from MNIST dataset (LeCun et al., 2010). Specifically, we consider  $N = 10$  clients, each with 500 handwritten images. Each image is comprised of  $28 \times 28$  pixels. Instead of randomly assigning images, we construct dataset  $i$  such that it contains 450 images of digit  $i$  and 50 images of other digits. Here client 10 corresponds to digit 0. After vectorizing each image into a  $784 \times 1$  one-dimension signal, our imbalanced dataset contains 10 matrices  $\mathbf{Y}_i \in \mathbb{R}^{784 \times 500}$ ,  $i = 1, \ldots, 10$ .
We first use Algorithm 4 to learn an orthogonal dictionary for each client, using their own imbalanced dataset. For client  $i$ , given the output of Algorithm 4 after  $T$  iterations  $\mathbf{D}_i^{(T)}$ , we reconstruct a new signal  $\mathbf{y}$  using the top  $k$  atoms according to the following steps: first, we solve a sparse coding problem to find the sparse code  $\mathbf{x}$  such that  $\mathbf{y} \approx \mathbf{D}_i^{(T)}\mathbf{x}$ . This can be achieved by Step 2 in Algorithm 4. Second, we find the top  $k$  entries in  $\mathbf{x}$  that have the largest magnitude:  $\mathbf{x}_{(\alpha_1,1)}, \mathbf{x}_{(\alpha_2,1)}, \dots, \mathbf{x}_{(\alpha_k,1)}$ . Finally, we calculate the reconstructed signal  $\tilde{\mathbf{y}}$  as
$$
\tilde {\mathbf {y}} = \sum_ {j = 1} ^ {k} \mathbf {x} _ {(\alpha_ {h}, 1)} \left(\mathbf {D} _ {i} ^ {(T)}\right) _ {\alpha_ {h}}.
$$
The second row of Figure 3 is generated by the above procedure with  $k = 5$  using the dictionary learned by Client 1. The third row of Figure 3 corresponds to the reconstructed images using the output of PerMA.
A.3 Details of Section 5.3
Our considered dataset in section 5.3 contains 62 frames, each of which is a  $480 \times 640 \times 3$  RGB image. We consider each frame as one client ( $N = 62$ ). After dividing each frame into  $40 \times 40$  patches, we obtain each data matrix  $\mathbf{Y}_i \in \mathbb{R}^{576 \times 1600}$ . Then we apply PerMA to  $\{\mathbf{Y}_i\}_{i=1}^{62}$  with  $r_i = 576$  for all  $i$  and  $r^g = 30$ . Consider  $\mathbf{D}_i^{(T)} = \left[ \begin{array}{cc} \mathbf{D}^{g,(T)} & \mathbf{D}_i^{l,(T)} \end{array} \right]$ , which is the output of PerMA for client  $i$ . We reconstruct each  $\mathbf{Y}_i$  using the procedure described in the previous section with  $k = 50$ . Specifically, we separate the contribution of  $\mathbf{D}^{g,(T)}$  from  $\mathbf{D}_i^{l,(T)}$ . Consider the reconstructed matrix  $\tilde{Y}_i$  as
$$
\tilde {\mathbf {Y}} _ {i} = \left[ \begin{array}{l l} \mathbf {D} ^ {g, (T)} & \mathbf {D} _ {i} ^ {l, (T)} \end{array} \right] \left[ \begin{array}{l} \mathbf {X} _ {i} ^ {g} \\ \mathbf {X} _ {i} ^ {l} \end{array} \right] = \underbrace {\mathbf {D} ^ {g , (T)} \mathbf {X} _ {i} ^ {g}} _ {\tilde {\mathbf {Y}} _ {i} ^ {g}} + \underbrace {\mathbf {D} _ {i} ^ {l , (T)} \mathbf {X} _ {i} ^ {l}} _ {\tilde {\mathbf {Y}} _ {i} ^ {l}}
$$
The second column and the third column of Figure 4 correspond to reconstructed results of  $\tilde{\mathbf{Y}}_i^g$  and  $\tilde{\mathbf{Y}}_i^{l}$  respectively. We can see clear separation of the background (which is shared among all frames) from the moving objects (which is unique to each frame).