SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Zeyang Zhang $^{1*}$ , Xin Wang $^{1}$ , Ziwei Zhang $^{1}$ , Zhou Qin $^{2}$ , Weigao Wen $^{2}$ , Hui Xue $^{2}$ , Haoyang Li $^{1}$ , Wenwu Zhu $^{1}$
<sup>1</sup>Department of Computer Science and Technology, BNRist, Tsinghua University, <sup>2</sup>Alibaba Group zy-zhang20@mails.tsinghua.edu.cn, {xin_wang, zwzhang}@tsinghua.edu.cn, {qinzhou.qinzhou, weigao.wen, hui.xueh}@alibaba-inc.com, lihy18@mails.tsinghua.edu.cn, wwzhu@tsinghua.edu.cn
Dynamic graph neural networks (DyGNNs) currently struggle with handling distribution shifts that are inherent in dynamic graphs. Existing work on DyGNNs with out-of-distribution settings only focuses on the time domain, failing to handle cases involving distribution shifts in the spectral domain. In this paper, we discover that there exist cases with distribution shifts unobservable in the time domain while observable in the spectral domain, and propose to study distribution shifts on dynamic graphs in the spectral domain for the first time. However, this investigation poses two key challenges: i) it is non-trivial to capture different graph patterns that are driven by various frequency components entangled in the spectral domain; and ii) it remains unclear how to handle distribution shifts with the discovered spectral patterns. To address these challenges, we propose Spectral Invariant Learning for Dynamic Graphs under Distribution Shifts (SILD), which can handle distribution shifts on dynamic graphs by capturing and utilizing invariant and variant spectral patterns. Specifically, we first design a DyGNN with Fourier transform to obtain the ego-graph trajectory spectrums, allowing the mixed dynamic graph patterns to be transformed into separate frequency components. We then develop a disentangled spectrum mask to filter graph dynamics from various frequency components and discover the invariant and variant spectral patterns. Finally, we propose invariant spectral filtering, which encourages the model to rely on invariant patterns for generalization under distribution shifts. Experimental results on synthetic and real-world dynamic graph datasets demonstrate the superiority of our method for both node classification and link prediction tasks under distribution shifts.
Dynamic graph neural networks (DyGNNs) have achieved remarkable success in many predictive tasks over dynamic graphs [1, 2]. Existing DyGNNs exhibit limitations in handling distribution shifts, which naturally exist in dynamic graphs due to multiple uncontrollable factors [3, 4, 5, 6]. Existing work on out-of-distribution generalized DyGNNs focuses on handling distribution shifts in the time domain. For example, DIDA [7] utilizes dynamic graph attention to mask the graph trajectories to capture the invariant patterns on dynamic graphs, which assumes that in the time domain, the distribution shift is observable and the invariant and variant patterns can be easily disentangled.
However, there exist cases that the distribution shift is unobservable in the time domain while observable in the spectral domain, as shown in Figure 1. The shift in frequency components can
*This work was done during the author's internship at Alibaba Group
Figure 1: An illustration example: the graph dynamics from different frequency components are entangled in the temporal domain, while it is much easier to distinguish different frequency components by masking the spectrums in the spectral domain. In this case, the frequency components in the invariant spectrums determine the node labels, while the relationship between the variant spectrums and labels is not stable under distribution shifts.
be clearly observed in the spectral domain, while these components are indistinguishable in the time domain. Moreover, in real-world applications, the observed dynamic graphs usually consist of multiple mixed graph structural and featural dynamics from various frequency components [8, 9, 10].
To address this problem, in this paper, we study the problem of handling distribution shifts on dynamic graphs in the spectral domain for the first time, which poses the following two key challenges: i) it is non-trivial to capture different graph patterns that are driven by various frequency components entangled in the spectral domain, and ii) it remains unclear how to handle distribution shifts with the discovered spectral patterns.
To tackle these challenges, we propose Spectral Invariant Learning for Dynamic Graphs under Distribution Shifts (SILD<sup>3</sup>). Our proposed SILD model can effectively handle distribution shifts on dynamic graphs by discovering and utilizing invariant and variant spectral patterns. Specifically, we first design a DyGNN with Fourier transform to obtain the ego-graph trajectory spectrums so that the mixed graph dynamics can be transformed into separate frequency components. Then we develop a disentangled spectrum mask that leverages the amplitude and phase information of the ego-graph trajectory spectrums to obtain invariant and variant spectrum masks so that graph dynamics from various frequency components can be filtered. Finally, we propose an invariant spectral filtering that discovers the invariant and variant patterns via the disentangled spectrum masks, and minimize the variance of predictions with exposure to various variant patterns. As such, SILD is able to exploit invariant patterns to make predictions under distribution shifts. Experimental results on several synthetic and real-world datasets, including both node classification and link prediction tasks, demonstrate the superior performance of our SILD model compared to state-of-the-art baselines under distribution shifts. To summarize, we make the following contributions:
- We propose to study distribution shifts on dynamic graphs in the spectral domain, to the best of our knowledge, for the first time.
- We propose Spectral Invariant Learning for Dynamic Graphs under Distribution Shifts (SILD), which can handle distribution shifts on dynamic graphs in the spectral domain.
- We employ DyGNN with Fourier transform to obtain the node spectrums, design a disentangled spectrum mask to obtain invariant and variant spectrum masks in the spectral domain, and propose the invariant spectral filtering mechanism so that SILD is able to handle distribution shifts.
- We conduct extensive experiments on several synthetic and real-world datasets, including both node classification and link prediction tasks, to demonstrate the superior performance of our method compared to state-of-the-art baselines under distribution shifts.
3The codes are available at Github.
2 Problem Formulation and Notations
Dynamic Graphs A dynamic graph can be represented as  $\mathcal{G} = (\{\mathcal{G}^t\}_{t=1}^T)$ , where  $T$  represents the total number of time stamps, and each  $\mathcal{G}^t = (\mathcal{V}^t, \mathcal{E}^t)$  corresponds to a graph snapshot at time stamp  $t$  with the node set  $\mathcal{V}^t$  and the edge set  $\mathcal{E}^t$ . For simplicity, we also represent a graph snapshot as  $\mathcal{G}^t = (\mathbf{X}^t, \mathbf{A}^t)$ , which includes the node feature matrix  $\mathbf{X}^t$  and the adjacency matrix  $\mathbf{A}^t$ . We further denote a random variable of  $\mathcal{G}^t$  as  $\mathbf{G}^t$ . The prediction task on dynamic graphs aims to utilize past graph snapshots to make predictions, i.e.,  $p(\mathbf{Y}^t | \mathbf{G}^{1:t})$ , where  $\mathbf{G}^{1:t} = \{\mathbf{G}^1, \mathbf{G}^2, \ldots, \mathbf{G}^t\}$  denotes the graph trajectory, and the label  $\mathbf{Y}^t$  represent the node properties or the links at time  $t + 1$ . For brevity, we take node-level prediction tasks as an example in this paper. Following [7], the distribution of graph trajectory can be factorized into ego-graph trajectories, such that  $p(\mathbf{Y}^t | \mathbf{G}^{1:t}) = \prod_v p(\mathbf{y}_v^t | \mathbf{G}_v^{1:t})$ .
Distribution Shifts on Dynamic Graphs The common optimization objective for prediction tasks on dynamic graphs is to learn an optimal predictor with empirical risk minimization (ERM), i.e.  $\min_{\theta}\mathbb{E}_{(y^t,\mathcal{G}_v^{1:t})\sim p_{tr}(\mathbf{y}^t,\mathbf{G}_v^{1:t})}\mathcal{L}(f_\theta (\mathcal{G}_v^{1:t}),y^t)$ , where  $f_{\theta}$  is a learnable dynamic graph neural networks. Under distribution shifts, however, the optimal predictor trained with ERM and the training distribution may not generalize well to the test distribution, since the risk minimization objectives under two distributions are different due to  $p_{tr}(\mathbf{Y}^t,\mathbf{G}^{1:t})\neq p_{te}(\mathbf{Y}^t,\mathbf{G}^{1:t})$ . The distribution shift on dynamic graphs is complex that may originate from temporal distribution shifts [11, 6, 12, 13, 14] as well as structural distribution shifts [15, 16, 17]. For example, trends or community structures can affect interaction patterns in co-author networks [18] and recommendation networks [19], i.e., the distribution of ego-graph trajectories may vary through time and structures.
Following out-of-distribution (OOD) generalization literature [7, 15, 11, 20, 21, 22], we make the following assumptions of distribution shifts on dynamic graphs:
Assumption 1 For a given task, there exists a predictor  $f(\cdot)$ , for samples  $(\mathcal{G}_v^{1:t}, y^t)$  from any distribution, there exists an invariant pattern  $P_I^t(v)$  and a variant pattern  $P_V^t(v)$  such that the following conditions are satisfied: 1) the invariant patterns are sufficient to predict the labels,  $y_v^t = f(P_I^t(v)) + \epsilon$ , where  $\epsilon$  is a random noise, 2) the observed data is composed of invariant and variant patterns,  $P_I^t(v) = \mathcal{G}_v^{1:t} \backslash P_V^t(v)$ , 3) the influence of the variant patterns on labels is shielded by the invariant patterns,  $\mathbf{y}_v^t \perp \mathbf{P}_V^t(v) \mid \mathbf{P}_I^t(v)$ .
In the next section, inspired by [16], we give a motivation example to provide some high-level intuition before going to our formal method.
3 Motivation Example
Here we introduce a toy dynamic graph example to motivate learning invariant patterns in the spectral domain. We assume that the invariant and variant patterns lie in the 1-hop neighbors, i.e., each node has an invariant subgraph and a variant subgraph. For simplicity, we focus on the number of neighbors, i.e., each node  $v$  has an invariant subgraph related degree  $\mathbf{d}_{v,1} \in \mathbb{R}^{T \times 1}$  and a variant subgraph related degree  $\mathbf{d}_{v,2} \in \mathbb{R}^{T \times 1}$ . Only the former determines the node label, i.e.,  $y_{v} = \mathbf{g}^{\top}\mathbf{d}_{v,1}$ . Note that invariant and variant subgraphs are not observed in the data. We further assume a one-dimensional constant feature for each node, which is set as 1 without loss of generality.
For the model in the spatial-temporal domain, we adopt sum pooling as one-layer graph convolution, i.e., the message passing for each node and time is  $\mathbf{h}_v = \sum_{u\in \mathcal{N}_v}1 = \mathbf{d}_{v,1} + \mathbf{d}_{v,2}$ . We further adopt a mask  $\mathbf{m}\in \mathbb{R}^{T\times 1}$  to filter patterns in the temporal domain and make predictions by a linear classifier, i.e.,  $\hat{y}_v = \mathbf{w}^\top (\mathbf{m}\odot \mathbf{h}_v)$ , where  $\mathbf{w}\in \mathbb{R}^{T\times 1}$  denotes the learnable parameters. Then, the empirical risk in the training dataset  $D_{tr}$  is  $R_{tr}(\mathbf{w}) = \frac{1}{|D_{tr}|}\sum_{v\in D_{tr}}(\hat{y}_v - y_v)^2$ . We have the following proposition.
Proposition 1 For any mask  $\mathbf{m} \in \mathbb{R}^{T \times 1}$ , for the optimal classifier in the training data  $\mathbf{w}^* = \arg \min_{\mathbf{w}} R_{tr}(\mathbf{w})$ , if  $\|\mathbf{m} \odot \mathbf{w}^*\|_2 \neq 0$ , there exist OOD nodes with unbounded error, i.e.,  $\exists v$  s.t.  $\lim_{||\mathbf{d}_{v,2}|| \to \infty} (\hat{y}_v - y_v)^2 = \infty$ .
The proposition 1 shows that a classifier trained with masks and empirical risk minimization has unbounded risks in testing data under distribution shifts as the classifier uses variant patterns to make predictions. Next, we show that under mild conditions, an invariant linear classifier in the spectral
domain can solve this problem. Denote  $\Phi \in \mathbb{C}^{T\times T}$  as the Fourier bases, where  $\Phi_{k,t} = \frac{1}{\sqrt{T}} e^{-j\frac{2\pi kt}{T}}$ . Denote  $\mathbf{z}_v = \Phi \sum_{u\in \mathcal{N}_v}1$  as the spectral representation after a linear message-passing. The prediction is  $\hat{y}_v = \mathbf{w}^{\mathrm{H}}(\mathbf{m}\odot \mathbf{z}_v)$ , where  $\mathbf{m}\in \mathbb{C}^{T\times 1}$  is the mask to filter the spectral patterns,  $\mathbf{w}\in \mathbb{C}^{T\times 1}$  is a linear classifier, and  $(\cdot)^{\mathrm{H}}$  denotes Hermitian transpose. We have the following proposition.
Proposition 2 If  $\left(\overline{\Phi\mathbf{d}_{v,1}}\odot \Phi \mathbf{d}_{v,1}\right)\odot \left(\overline{\Phi\mathbf{d}_{v,2}}\odot \Phi \mathbf{d}_{v,2}\right) = \mathbf{0},\forall \mathbf{d}_{v,1},\mathbf{d}_{v,2}$ , then  $\exists \mathbf{m}\in \mathbb{C}^{T\times 1}$  such that the optimal spectral classifier in the training data has bounded error, i.e., for  $\mathbf{w}^* = \arg \min_{\mathbf{w}}R_{tr}(\mathbf{w})$ $\exists \epsilon >0,\forall v,\lim_{||\mathbf{d}_{\mathbf{v},2}||\to \infty}(\hat{y}_v - y_v)^2 < \epsilon$
The proposition 2 shows that if the frequency bandwidths of invariant and variant patterns do not have any overlap, there exists a spectral mask such that a linear classifier trained with empirical risk minimization in the spectral domain will have bounded risk in any testing data distribution. This example motivates us to capture invariant and variant patterns in the spectral domain, which is not feasible in the spatial-temporal domain.
In this section, we introduce our method named Spectral Invariant Learning for Dynamic Graphs under Distribution Shifts (SILD) to handle distribution shifts in dynamic graphs, including three modules, dynamic graph neural networks with Fourier transform, disentangled spectrum mask, and invariant spectral filtering. The framework of our method is shown in Figure 2.
4.1 Dynamic Graph Neural Network with Spectral Transform
Dynamic Graph Trajectories Modeling Each node on the dynamic graph has its ego-graph trajectory evolving through time that may determine the node properties or the occurrence of future links. Following [23, 24, 25], we adopt a message-passing network for each graph snapshot to aggregate the neighborhood information at the current time, i.e.,
$$
\mathbf {m} _ {u \rightarrow v} ^ {t} \leftarrow \operatorname {M S G} \left(\mathbf {h} _ {u} ^ {t}, \mathbf {h} _ {v} ^ {t}\right), \mathbf {h} _ {v} ^ {t} \leftarrow \operatorname {A G G} \left( \right.\left\{\mathbf {m} _ {u \rightarrow v} ^ {t} \mid u \in \mathcal {N} ^ {t} (v) \right\}, \mathbf {h} _ {v} ^ {t} \left. \right\}), \tag {1}
$$
where 'MSG' and 'AGG' denote message and aggregation functions,  $\mathbf{m}_{u\rightarrow v}^{t}$  is the message from node  $u$  to node  $v$ ,  $\mathbf{h}_u^t$  is the node embedding for node  $u$  at time  $t$ ,  $\mathcal{N}^t (v) = \{u\mid (u,v)\in \mathcal{E}^t\}$  is node  $v$ 's neighborhood at time  $t$ . To model the high-order neighborhood information, we can stack multiple message-passing layers. In this way, the node embedding along time  $\{\mathbf{h}_u^t\}_{t = 1}^T$  summarizes the evolution of node  $u$ 's ego-graph trajectories. We denote  $\mathbf{H}\in \mathbb{R}^{T\times N\times d}$  as the ego-graph trajectory signals for all nodes on the dynamic graph, where  $T$  denotes the total time length,  $N$  denotes the number of nodes and  $d$  denotes the hidden dimensionality.
Spectral Transform As some patterns on dynamic graphs are unobservable in the time domain, while observable in the spectral domain, we transform the summarized ego-graph trajectory signals  $\mathbf{H}$  into the spectral domain via Fourier transform for each node and hidden dimension, i.e.,
$$
\boldsymbol {\Phi} _ {k, t} = \frac {1}{\sqrt {T}} e ^ {- j \frac {2 \pi k t}{T}}, \mathbf {Z} = \boldsymbol {\Phi} \mathbf {H}, \tag {2}
$$
where  $\Phi \in \mathbb{C}^{K\times T}$  denotes the Fourier bases,  $K$  denotes the number of frequency components, and  $\mathbf{Z}\in \mathbb{C}^{K\times N\times d}$  denote the node embeddings along frequency components in the spectral domain, and  $\mathbf{Z}_{k,n,m} = \sum_{t = 1}^{T}\Phi_{k,t}\mathbf{H}_{t,n,m}$ . By choosing the Fourier bases, our spectral transform has the following advantages: 1) we can use fast Fourier transform (FFT) [26] to accelerate the computation. The computation complexity of Eq. (2) can be reduced from  $O(NdT^2)$  to  $O(NdT\log T)$ . 2) Each basis has clear semantics, e.g.,  $\mathbf{Z}_k$  denotes the node embeddings at the  $k$ -th frequency component in the spectral domain. In this way, we can observe how the nodes on the dynamic graph evolve in different frequency bandwidths. 3) Fourier transform is able to capture global and periodic patterns [27, 28], which are common in real-world dynamic graphs, e.g., the interactions on e-commerce networks may result from seasonal sales or product service cycles.
Figure 2: The framework of our proposed method SILD. Given a dynamic graph evolving through time, the dynamic graph neural networks with spectral transform first obtain the ego-graph trajectory spectrums in the spectral domain. Then the disentangled spectrum mask leverages the amplitude and phase information of the ego-graph trajectory spectrums to obtain invariant and variant spectrum masks. Last, invariant spectral filtering discovers the invariant and variant patterns via the disentangled spectrum masks, and minimizes the variance of predictions with exposure to various variant patterns, to help the model exploit invariant patterns to make predictions under distribution shifts.
4.2 Disentangled Spectrum Mask
To capture invariant patterns in the spectral domain, we propose to explicitly learn spectrum masks to disentangle the invariant and variant patterns. The embeddings in the spectral domain contain the amplitude information as well as the phase information for each node
$$
\operatorname {A m p} (\mathbf {Z}) = \left(\operatorname {I m a g} ^ {2} (\mathbf {Z}) + \operatorname {R e a l} ^ {2} (\mathbf {Z})\right) ^ {\frac {1}{2}}, \phi (\mathbf {Z}) = \arctan \frac {\operatorname {I m a g} (\mathbf {Z})}{\operatorname {R e a l} (\mathbf {Z})}, \tag {3}
$$
where  $\operatorname{Real}(\cdot)$  and  $\operatorname{Imag}(\cdot)$  denote the real and imaginary part of the complex number, i.e.,  $\mathbf{Z} = \operatorname{Real}(\mathbf{Z}) + j\operatorname{Imag}(\mathbf{Z})$ ,  $j$  denotes the imaginary unit,  $\operatorname{Amp}(\mathbf{Z}) \in \mathbb{R}^{K \times N \times d}$  and  $\phi(\mathbf{Z}) \in \mathbb{R}^{K \times N \times d}$  denote the amplitude and phase information respectively. For brevity, the tensor operators in Eq. (3) are all element-wise, e.g.,  $(\operatorname{Imag}^2(\mathbf{Z}))_{i,j,k} = (\operatorname{Imag}(\mathbf{Z}_{i,j,k}))^2$ . Then, we obtain the spectrum masks by leveraging both the amplitude and phase information
$$
\mathbf {M} = \operatorname {M L P} (\operatorname {R e a l} (\mathbf {Z}) | | \operatorname {I m a g} (\mathbf {Z})), \mathbf {M} _ {I} = \operatorname {s i g m o i d} (\mathbf {M} / \tau), \mathbf {M} _ {V} = \operatorname {s i g m o i d} (- \mathbf {M} / \tau), \tag {4}
$$
where MLP denotes multi-layer perceptrons,  $\tau$  is the temperature,  $\mathbf{M}_I\in [0,1]^K$  and  $\mathbf{M}_V\in [0,1]^K$  denote the spectrum mask for invariant and variant patterns, and  $||$  represents the concatenation of the embeddings. In this way, the invariant and variant masks have a negative relationship, and each node can have its own spectrum mask. As the phase information includes high-level semantics in the original signals [29, 30, 31, 32, 33], we keep the phase information unchanged to reduce harm in the fine-grained semantic information for the graph trajectories, and filter the spectrums by the learned disentangled masks in terms of amplitudes,
$$
\mathbf {Z} _ {I} = \left(\mathbf {M} _ {I} \odot \operatorname {A m p} (\mathbf {Z})\right) \odot \left(\cos \phi (\mathbf {Z}) + j \sin \phi (\mathbf {Z})\right), \mathbf {Z} _ {V} = \left(\mathbf {M} _ {V} \odot \operatorname {A m p} (\mathbf {Z})\right) \left(\cos \phi (\mathbf {Z}) + j \sin \phi (\mathbf {Z})\right), \tag {5}
$$
where  $\mathbf{Z}_I$  and  $\mathbf{Z}_V$  denote the summarized invariant and variant patterns in the spectral domain. For node classification tasks, we can directly adopt the spectrums for the classifier to predict classes. For link prediction tasks, we can utilize inverse fast Fourier transform (IFFT) to transform the embeddings into the temporal domain for future link prediction
$$
\mathbf {H} _ {I} ^ {\prime} = \boldsymbol {\Phi} ^ {\mathrm {H}} \mathbf {Z} _ {I}, \mathbf {H} _ {V} ^ {\prime} = \boldsymbol {\Phi} ^ {\mathrm {H}} \mathbf {Z} _ {V}, \tag {6}
$$
where  $(\cdot)^{\mathrm{H}}$  is Hermitian transpose,  $\mathbf{H}_I^{\prime}$  and  $\mathbf{H}_V^{\prime}$  denote the filtered invariant and variant patterns that are transformed back into the temporal domain respectively.
4.3 Invariant Spectral Filtering
Under distribution shifts, the variant patterns on dynamic graphs have varying relationships with labels, while the invariant patterns have sufficient predictive abilities with regard to labels. We propose invariant spectral filtering to capture the invariant and variant patterns in the spectral domain, and help the model focus on invariant patterns to make predictions, thus handling distribution shifts. We take node classification tasks for an example as follows.
Let  $\mathbf{Z}_I\in \mathbb{C}^{K\times N\times d}$  and  $\mathbf{Z}_V\in \mathbb{C}^{K\times N\times d}$  be the filtered invariant and variant spectrums in the spectral domain. Then we can utilize the invariant and variant node spectrums to calculate the task loss
$$
\mathcal {L} _ {I} = l \left(f _ {I} \left(\mathbf {Z} _ {I}\right), \mathbf {Y}\right), \mathcal {L} _ {V} = l \left(f _ {V} \left(\mathbf {Z} _ {V}\right), \mathbf {Y}\right), \tag {7}
$$
where  $f_{I}(\cdot)$  and  $f_{V}(\cdot)$  are the classifiers for invariant and variant patterns respectively,  $\mathbf{Y}$  is the labels, and  $l$  is the loss function. The task loss is utilized to capture the patterns with the predictive abilities of labels. Recall in Assumption 1, the influence of variant patterns on labels is shielded given invariant patterns as the invariant patterns have sufficient predictive abilities w.r.t labels, and thus the model's predictions should not change when being exposed to different variant patterns and the original invariant patterns. Inspired by [15, 7], we calculate the invariance loss by
$$
\mathcal {L} _ {I N V} = \operatorname {V a r} \left(\left\{\mathcal {L} _ {m} \mid \tilde {\mathbf {z}}: \tilde {\mathbf {z}} \in \mathcal {S} \right\}\right), \tag {8}
$$
where  $\mathcal{L}_m\mid \tilde{\mathbf{z}}$  denotes the mixed loss to measure the model's prediction ability with exposure to the specific variant pattern  $\tilde{\mathbf{z}}\in \mathbb{C}^{K\times d}$  that is sampled from a set of variant patterns  $S$ . We adopt all the node embeddings in  $\mathbf{Z}_V$  to construct the set of variant patterns  $S$ . Inspired by [34, 15], we calculate the mixed loss as
$$
\mathcal {L} _ {m} \mid \tilde {\mathbf {z}} = l \left(f _ {I} \left(\mathbf {Z} _ {I}\right) \odot \sigma \left(f _ {V} (\tilde {\mathbf {z}})\right), \mathbf {Y}\right), \tag {9}
$$
where  $\sigma$  denotes the sigmoid function. Then, the final training objective is
$$
\min  _ {\theta , f _ {I}} \mathcal {L} _ {I} + \lambda \mathcal {L} _ {I N V} + \min  _ {f _ {V}} \mathcal {L} _ {V}, \tag {10}
$$