SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Xinyi  $\mathbf{W}\mathbf{u}^{1,2}$
Amir Ajorlou²
Zihui Wu3
Ali Jadbabaie<sup>1,2</sup>
$^{1}$ Institute for Data, Systems and Society (IDSS), MIT
$^{2}$ Laboratory for Information and Decision Systems (LIDS), MIT
<sup>3</sup>Department of Computing and Mathematical Sciences (CMS), Caltech
{xinyiwu,ajorlou,jadbabai}@mit.edu zwu2@caltech.edu
Oversmoothing in Graph Neural Networks (GNNs) refers to the phenomenon where increasing network depth leads to homogeneous node representations. While previous work has established that Graph Convolutional Networks (GCNs) exponentially lose expressive power, it remains controversial whether the graph attention mechanism can mitigate oversmoothing. In this work, we provide a definitive answer to this question through a rigorous mathematical analysis, by viewing attention-based GNNs as nonlinear time-varying dynamical systems and incorporating tools and techniques from the theory of products of inhomogeneous matrices and the joint spectral radius. We establish that, contrary to popular belief, the graph attention mechanism cannot prevent oversmoothing and loses expressive power exponentially. The proposed framework extends the existing results on oversmoothing for symmetric GCNs to a significantly broader class of GNN models, including random walk GCNs, Graph Attention Networks (GATs) and (graph) transformers. In particular, our analysis accounts for asymmetric, state-dependent and time-varying aggregation operators and a wide range of common nonlinear activation functions, such as ReLU, LeakyReLU, GELU and SiLU.
Graph neural networks (GNNs) have emerged as a powerful framework for learning with graph-structured data [4, 8, 9, 13, 19, 32, 38] and have shown great promise in diverse domains such as molecular biology [45], physics [1] and recommender systems [40]. Most GNN models follow the message-passing paradigm [12], where the representation of each node is computed by recursively aggregating and transforming the representations of its neighboring nodes.
One notable drawback of repeated message-passing is oversmoothing, which refers to the phenomenon that stacking message-passing GNN layers makes node representations of the same connected component converge to the same vector [5, 18, 19, 24, 26, 31, 41]. As a result, whereas depth has been considered crucial for the success of deep learning in many fields such as computer vision [16], most GNNs used in practice remain relatively shallow and often only have few layers [19, 38, 42]. On the theory side, while previous works have shown that the symmetric Graph Convolution Networks (GCNs) with ReLU and LeakyReLU nonlinearities exponentially lose expressive power, analyzing the oversmoothing phenomenon in other types of GNNs is still an open question [5, 26]. In particular, the question of whether the graph attention mechanism can prevent oversmoothing has not been settled yet. Motivated by the capacity of graph attention to distinguish the importance of different edges in the graph, some works claim that oversmoothing is alleviated in Graph Attention Networks (GATs), heuristically crediting to GATs' ability to learn adaptive node-wise aggregation operators via the attention mechanism [25]. On the other hand, it has been empirically observed that similar to
the case of GCNs, oversmoothing seems inevitable for attention-based GNNs such as GATs [31] or (graph) transformers [34]. The latter can be viewed as attention-based GNNs on complete graphs.
In this paper, we provide a definitive answer to this question — attention-based GNNs also lose expressive power exponentially, albeit potentially at a slower exponential rate compared to GCNs. Given that attention-based GNNs can be viewed as nonlinear time-varying dynamical systems, our analysis is built on the theory of products of inhomogeneous matrices [14, 33] and the concept of joint spectral radius [30], as these methods have been long proved effective in the analysis of time-inhomogeneous markov chains and ergodicity of dynamical systems [2, 14, 33]. While classical results only apply to generic one-dimensional linear time-varying systems, we address four major challenges arising in analyzing attention-based GNNs: (1) the aggregation operators computed by attention are state-dependent, in contrast to conventional fixed graph convolutions; (2) the systems are multi-dimensional, which involves the coupling across feature dimensions; (3) the dynamics are nonlinear due to the nonlinear activation function in each layer; (4) the learnable weights and aggregation operators across different layers result in time-varying dynamical systems.
Below, we highlight our key contributions:
- As our main contribution, we establish that oversmoothing happens exponentially as model depth increases for attention-based GNNs, resolving the long-standing debate about whether attention-based GNNs can prevent oversmoothing.
- We analyze attention-based GNNs through the lens of nonlinear, time-varying dynamical systems. The strength of our analysis stems from its ability to exploit the inherently common connectivity structure among the typically asymmetric state-dependent aggregation operators at different attentional layers. This enables us to derive rigorous theoretical results on the ergodicity of infinite products of matrices associated with the evolution of node representations across layers. Incorporating results from the theory of products of inhomogeneous matrices and their joint spectral radius, we then establish that oversmoothing happens at an exponential rate for attention-based GNNs from our ergodicity results.
- Our analysis generalizes the existing results on oversmoothing for symmetric GCNs to a significantly broader class of GNN models with asymmetric, state-dependent and time-varying aggregation operators and nonlinear activation functions under general conditions. In particular, our analysis can accommodate a wide range of common nonlinearities such as ReLU, LeakyReLU, and even non-monotone ones like GELU and SiLU. We validate our theoretical results on six real-world datasets with two attention-based GNN architectures and five common nonlinearities.
Oversmoothing problem in GNNs Oversmoothing is a well-known problem in deep GNNs, and many techniques have been proposed in order to mitigate it practically [6, 15, 20, 21, 29, 43, 47]. On the theory side, analysis of oversmoothing has only been carried out for the graph convolution case [5, 18, 26, 41]. In particular, by viewing graph convolutions as a form of Laplacian filter, prior works have shown that for GCNs, the node representations within each connected component of a graph will converge to the same value exponentially [5, 26]. However, oversmoothing is also empirically observed in attention-based GNNs such as GATs [31] or transformers [34]. Although some people hypothesize based on heuristics that attention can alleviate oversmoothing [25], a rigorous analysis of oversmoothing in attention-based GNNs remains open [5].
Theoretical analysis of attention-based GNNs Existing theoretical results on attention-based GNNs are limited to one-layer graph attention. Recent works in this line include Brody et al. [3] showing that the ranking of attention scores computed by a GAT layer is unconditioned on the query node, and Fountoulakis et al. [11] studying node classification performance of one-layer GATs on a random graph model. More relevantly, Wang et al. [39] made a claim that oversmoothing is asymptotically inevitable in GATs. Aside from excluding nonlinearities in the analysis, there are several flaws in the proof of their main result (Theorem 2). In particular, their analysis assumes the same stationary distribution for all the stochastic matrices output by attention at different layers. This is typically not the case given the state-dependent and time-varying nature of these matrices. In fact, the main challenge in analyzing multi-layer attention lies in the state-dependent and time-varying
nature of these input-output mappings. Our paper offers novel contributions to the research on attention-based GNNs by developing a rich set of tools and techniques for analyzing multi-layer graph attention. This addresses a notable gap in the existing theory, which has primarily focused on one-layer graph attention, and paves the way for future research to study other aspects of multi-layer graph attention.
Let  $\mathbb{R}$  be the set of real numbers and  $\mathbb{N}$  be the set of natural numbers. We use the shorthands  $[n] := \{1, \dots, n\}$  and  $\mathbb{N}_{\geq 0} := \mathbb{N} \cup \{0\}$ . We denote the zero-vector of length  $N$  by  $\mathbf{0} \in \mathbb{R}^N$  and the all-one vector of length  $N$  by  $\mathbf{1} \in \mathbb{R}^N$ . We represent an undirected graph with  $N$  nodes by  $\mathcal{G} = (A, X)$ , where  $A \in \{0, 1\}^{N \times N}$  is the adjacency matrix and  $X \in \mathbb{R}^{N \times d}$  are the node feature vectors of dimension  $d$ . Let  $E(\mathcal{G})$  be the set of edges of  $\mathcal{G}$ . For nodes  $i, j \in [N]$ ,  $A_{ij} = 1$  if and only if  $i$  and  $j$  are connected with an edge in  $\mathcal{G}$ , i.e.,  $(i, j) \in E(\mathcal{G})$ . For each  $i \in [N]$ ,  $X_i \in \mathbb{R}^d$  represents the feature vector for node  $i$ . We denote the degree matrix of  $\mathcal{G}$  by  $D_{\mathrm{deg}} = \mathrm{diag}(A\mathbf{1})$  and the set of all neighbors of node  $i$  by  $\mathcal{N}_i$ .
Let  $\| \cdot \| _2,\| \cdot \|_\infty ,\| \cdot \| _F$  be the 2-norm,  $\infty$  -norm and Frobenius norm, respectively. We use  $\| \cdot \|_{\mathrm{max}}$  to denote the matrix max norm, i.e., for a matrix  $M\in \mathbb{R}^{m\times n}$ $\| M\|_{\mathrm{max}}\coloneqq \max_{ij}|M_{ij}|$  . We use  $\leq_{\mathrm{ew}}$  to denote element-wise inequality. Lastly, for a matrix  $M$  , we denote its  $i^{th}$  row by  $M_{i}$  . and  $j^{th}$  column by  $M_{j}$
3.2 Graph attention mechanism
We adopt the following definition of graph attention mechanism. Given node representation vectors  $X_{i}$  and  $X_{j}$ , we first apply a shared learnable linear transformation  $W \in \mathbb{R}^{d \times d'}$  to each node, and then use an attention function  $\Psi : \mathbb{R}^{d'} \times \mathbb{R}^{d'} \to \mathbb{R}$  to compute a raw attention coefficient
$$
e _ {i j} = \Psi \left(W ^ {\top} X _ {i}, W ^ {\top} X _ {j}\right)
$$
that indicates the importance of node  $j$ 's features to node  $i$ . Then the graph structure is injected into the mechanism by performing masked attention, where for each node  $i$ , we only compute its attention to its neighbors. To make coefficients easily comparable across different nodes, we normalize  $e_{ij}$  among all neighboring nodes  $j$  of node  $i$  using the softmax function to get the normalized attention coefficients:
$$
P _ {i j} = \operatorname {s o f t m a x} _ {j} \left(e _ {i j}\right) = \frac {\exp \left(e _ {i j}\right)}{\sum_ {k \in \mathcal {N} _ {i}} \exp \left(e _ {i k}\right)}.
$$
The matrix  $P$ , where the entry in the  $i^{th}$  row and the  $j^{th}$  column is  $P_{ij}$ , is a row stochastic matrix. We refer to  $P$  as an aggregation operator in message-passing.
3.3 Attention-based GNNs
Having defined the graph attention mechanism, we can now write the update rule of a single graph attentional layer as
$$
X ^ {\prime} = \sigma (P X W),
$$
where  $X$  and  $X'$  are the input and output node representations, respectively,  $\sigma(\cdot)$  is a pointwise nonlinearity function, and the aggregation operator  $P$  is a function of  $XW$ .
As a result, the output of the  $t^{th}$  graph attentional layers can be written as
$$
X ^ {(t + 1)} = \sigma \left(P ^ {(t)} X ^ {(t)} W ^ {(t)}\right) \quad t \in \mathbb {N} _ {\geq 0}, \tag {1}
$$
where  $X^{(0)} = X$  is the input node features,  $W^{(t)} \in \mathbb{R}^{d' \times d'}$  for  $t \in \mathbb{N}$  and  $W^{(0)} \in \mathbb{R}^{d \times d'}$ . For the rest of this work, without loss of generality, we assume that  $d = d'$ .
The above definition is based on single-head graph attention. Multi-head graph attention uses  $K \in \mathbb{N}$  weight matrices  $W_{1},\ldots ,W_{K}$  in each layer and averages their individual single-head outputs [11, 38]. Without loss of generality, we consider single graph attention in our analysis in Section 4, but we note that our results automatically apply to the multi-head graph attention setting since  $K$  is finite.
3.4 Measure of oversmoothing
We use the following notion of oversmoothing, inspired by the definition proposed in Rusch et al. [31]
Definition 1. For an undirected and connected graph  $\mathcal{G}$ ,  $\mu : \mathbb{R}^{N \times d} \to \mathbb{R}_{\geq 0}$  is called a node similarity measure if it satisfies the following axioms:
1.  $\exists c\in \mathbb{R}^d$  such that  $X_{i} = c$  for all node  $i$  if and only if  $\mu (X) = 0$  ,for  $X\in \mathbb{R}^{N\times d}$
2.  $\mu (X + Y)\leq \mu (X) + \mu (Y),$  for all  $X,Y\in \mathbb{R}^{N\times d}$
Then oversmoothing with respect to  $\mu$  is defined as the layer-wise convergence of the node-similarity measure  $\mu$  to zero, i.e.,
$$
\lim  _ {t \rightarrow \infty} \mu (X ^ {(t)}) = 0. \tag {2}
$$
We say oversmoothing happens at an exponential rate if there exists constants  $C_1, C_2 > 0$ , such that for any  $t \in \mathbb{N}$
$$
\mu \left(X ^ {(t)}\right) \leq C _ {1} e ^ {- C _ {2} t}. \tag {3}
$$
We establish our results on oversmoothing for attention-based GNNs using the following node similarity measure:
$$
\mu (X) := \| X - \mathbf {1} \gamma_ {X} \| _ {F}, \text {w h e r e} \gamma_ {X} = \frac {\mathbf {1} ^ {\top} X}{N}. \tag {4}
$$
Proposition 1.  $\| X - \mathbf{1}\gamma_{X}\|_{F}$  is a node similarity measure.
The proof of the above proposition is provided in Appendix B. Other common node similarity measures include the Dirichlet energy [5, 31].<sup>2</sup> Our measure is mathematically equivalent to the measure  $\inf_{Y = \mathbf{1}c^{\top}, c \in \mathbb{R}^{d}}\left\{\|X - Y\|_{F}\right\}$  defined in Oono and Suzuki [26], but our form is more direct to compute. One way to see the equivalence is to consider the orthogonal projection into the space perpendicular to  $\operatorname{span}\{\mathbf{1}\}$ , denoted by  $B \in \mathbb{R}^{(N-1) \times N}$ . Then our definition of  $\mu$  satisfies  $\|X - \mathbf{1}\gamma_{x}\|_{F} = \|BX\|_{F}$ , where the latter quantity is exactly the measure defined in [26].
3.5 Assumptions
We make the following assumptions (in fact, quite minimal) in deriving our results:
A1 The graph  $\mathcal{G}$  is connected and non-bipartite.
A2 The attention function  $\Psi (\cdot ,\cdot)$  is continuous.
A3 The sequence  $\{\| \prod_{t = 0}^{k}|W^{(t)}|\|_{\max}\}_{k = 0}^{\infty}$  is bounded.
A4 The point-wise nonlinear activation function  $\sigma(\cdot)$  satisfies  $0 \leq \frac{\sigma(x)}{x} \leq 1$  for  $x \neq 0$  and  $\sigma(0) = 0$ .
We note that all of these assumptions are either standard or quite general. Specifically, A1 is a standard assumption for theoretical analysis on graphs. For graphs with more than one connected components, the same results apply to each connected component. A1 can also be replaced with requiring the graph  $\mathcal{G}$  to be connected and have self-loops at each node. Non-bipartiteness and self-loops both ensure that long products of stochastic matrices corresponding to aggregation operators in different graph attentional layers will eventually become strictly positive.
The assumptions on the GNN architecture A2 and A4 can be easily verified for commonly used GNN designs. For example, the attention function LeakyReLU $(a^\top[W^\top X_i||W^\top X_j])$ ,  $a \in \mathbb{R}^{2d'}$  used in the GAT [38], where  $\lceil \cdot \rceil \cdot \rceil$  denotes concatenation, is a specific case that satisfies A2. Other architectures that satisfy A2 include GATv2 [3] and (graph) transformers [37]. As for A4, one way to
<sup>1</sup>We distinguish the definition of oversmoothing and the rate of oversmoothing. This parallels the notion of stability and its rate in dynamical systems.
2In fact, our results are not specific to our choice of node similarity measure  $\mu$  and directly apply to any Lipschitz node similarity measure, including the Dirichlet energy under our assumptions. See Remark 2 after Theorem 1.
satisfy it is to have  $\sigma$  be 1-Lipschitz and  $\sigma(x) \leq 0$  for  $x < 0$  and  $\sigma(x) \geq 0$  for  $x > 0$ . Then it is easy to verify that most of the commonly used nonlinear activation functions such as ReLU, LeakyReLU, GELU, SiLU, ELU, tanh all satisfy A4.
Lastly, A3 is to ensure boundedness of the node representations' trajectories  $X^{(t)}$  for all  $t \in \mathbb{N}_{\geq 0}$ . Such regularity assumptions are quite common in the asymptotic analysis of dynamical systems, as is also the case for the prior works analyzing oversmoothing in symmetric GCNs [5, 26].
4 Main Results
In this section, we lay out a road-map for deriving our main results, highlighting the key ideas of the proofs. The complete proofs are provided in the Appendices.
We start by discussing the dynamical system formulation of attention-based GNNs in Section 4.1. By showing the boundedness of the node representations' trajectories, we prove the existence of a common connectivity structure among aggregation operators across different graph attentional layers in Section 4.2. This implies that graph attention cannot fundamentally change the graph connectivity, a crucial property that will eventually lead to oversmoothing. In Section 4.3, we develop a framework for investigating the asymptotic behavior of attention-based GNNs by introducing the notion of ergodicity and its connections to oversmoothing. Then utilizing our result on common connectivity structure among aggregation operators, we establish ergodicity results for the systems associated with attention-based GNNs. In Section 4.4, we introduce the concept of the joint spectral radius for a set of matrices [30] and employ it to deduce exponential convergence of node representations to a common vector from our ergodicity results. Finally, we present our main result on oversmoothing in attention-based GNNs in Section 4.5 and comment on oversmoothing in GCNs in comparison with attention-based GNNs in Section 4.6.
4.1 Attention-based GNNs as nonlinear time-varying dynamical systems
The theory of dynamical systems concerns the evolution of some state of interest over time. By viewing the model depth  $t$  as the time variable, the input-output mapping at each graph attentional layer  $X^{(t + 1)} = \sigma (P^{(t)}X^{(t)}W^{(t)})$  describes a nonlinear time-varying dynamical system. The attention-based aggregation operator  $P^{(t)}$  is state-dependent as it is a function of  $X^{(t)}W^{(t)}$ . Given the notion of oversmoothing defined in Section 3.4, we are interested in characterizing behavior of  $X^{(t)}$  as  $t\to \infty$ .
If the activation function  $\sigma(\cdot)$  is the identity map, then repeated application of (1) gives
$$
X ^ {(t + 1)} = P ^ {(t)} \dots P ^ {(0)} X W ^ {(0)} \dots W ^ {(t)}.
$$
The above linear form would enable us to leverage the rich literature on the asymptotic behavior of the products of inhomogeneous row-stochastic matrices (see, e.g., [14, 33]) in analyzing the long-term behavior of attention-based GNNs. Such a neat expansion, however, is not possible when dealing with a nonlinear activation function  $\sigma(\cdot)$ . To find a remedy, let us start by observing that element-wise application of  $\sigma$  to a vector  $y \in \mathbb{R}^d$  can be written as
$$
\sigma (y) = \operatorname {d i a g} \left(\frac {\sigma (y)}{y}\right) y, \tag {5}
$$
where  $\mathrm{diag}\left(\frac{\sigma(y)}{y}\right)$  is a diagonal matrix with  $\sigma(y_i) / y_i$  on the  $i^{th}$  diagonal entry. Defining  $\sigma(0) / 0 \coloneqq \sigma'(0)$  or 1 if the derivative does not exist along with the assumption  $\sigma(0) = 0$  in A4, it is easy to check that the above identity still holds for vectors with zero entries.
We can use (5) to write the  $i^{th}$  column of  $X^{(t + 1)}$  as
$$
X _ {\cdot i} ^ {(t + 1)} = \sigma \left(P ^ {(t)} \left(X ^ {(t)} W ^ {(t)}\right) _ {\cdot i}\right) = D _ {i} ^ {(t)} P ^ {(t)} \left(X ^ {(t)} W ^ {(t)}\right) _ {\cdot i} = D _ {i} ^ {(t)} P ^ {(t)} \sum_ {j = 1} ^ {d} W _ {j i} ^ {(t)} X _ {\cdot j} ^ {(t)}, \tag {6}
$$
where  $D_{i}^{(t)}$  is a diagonal matrix. It follows from the assumption on the nonlinearities A4 that
$$
\operatorname {d i a g} (\mathbf {0}) \le_ {\mathrm {e w}} D _ {i} ^ {(t)} \le_ {\mathrm {e w}} \operatorname {d i a g} (\mathbf {1}).
$$
We define  $\mathcal{D}$  to be the set of all possible diagonal matrices  $D_{i}^{(t)}$  satisfying the above inequality:
$$
\mathcal {D} := \{\operatorname {d i a g} (\mathbf {d}): \mathbf {d} \in \mathbb {R} ^ {N}, 0 \le_ {\mathrm {e w}} \mathbf {d} \le_ {\mathrm {e w}} \mathbf {1} \}.
$$
Using (6) recursively, we arrive at the following formulation for  $X_{\cdot i}^{(t + 1)}$ :
$$
X _ {i} ^ {(t + 1)} = \sum_ {j _ {t + 1} = i, (j _ {t}, \dots , j _ {0}) \in [ d ] ^ {t + 1}} \left(\prod_ {k = 0} ^ {t} W _ {j _ {k} j _ {k + 1}} ^ {(k)}\right) D _ {j _ {t + 1}} ^ {(t)} P ^ {(t)} \dots D _ {j _ {1}} ^ {(0)} P ^ {(0)} X _ {j _ {0}} ^ {(0)}. \tag {7}
$$
4.2 Common connectivity structure among aggregation operators across different layers
We can use the formulation in (7) to show the boundedness of the node representations' trajectories  $X^{(t)}$  for all  $t \in \mathbb{N}_{\geq 0}$ , which in turn implies the boundedness of the input to graph attention in each layer,  $X^{(t)}W^{(t)}$ .
Lemma 1. Under assumptions A3-A4, there exists  $C > 0$  such that  $\| X^{(t)}\|_{\max} \leq C$  for all  $t \in \mathbb{N}_{\geq 0}$ . For a continuous  $\Psi (\cdot ,\cdot)^3$ , the following lemma is a direct consequence of Lemma 1, suggesting that the graph attention mechanism cannot fundamentally change the connectivity pattern of the graph.
Lemma 2. Under assumptions A2-A4, there exists  $\epsilon >0$  such that for all  $t\in \mathbb{N}_{\geq 0}$  and for any  $(i,j)\in E(\mathcal{G})$ , we have  $P_{ij}^{(t)}\geq \epsilon$ .
One might argue that Lemma 2 is an artifact of the continuity of the softmax function. The softmax function is, however, often favored in attention mechanisms because of its trainability in back propagation compared to discontinuous alternatives such as hard thresholding. Besides trainability issues, it is unclear on a conceptual level whether it is reasonable to absolutely drop an edge from the graph as is the case for hard thresholding. Lemma 2 is an important step towards the main convergence result of this work, which states that all the nodes will converge to the same representation vector at an exponential rate. We define the family of row-stochastic matrices satisfying Lemma 2 below.
Definition 2. Let  $\epsilon >0$ . We define  $\mathcal{P}_{\mathcal{G},\epsilon}$  to be the set of row-stochastic matrices satisfying the following conditions:
1.  $\epsilon \leq P_{ij}\leq 1$ $if(i,j)\in E(\mathcal{G})$
2.  $P_{ij} = 0$  if  $(i,j)\notin E(\mathcal{G})$
4.3 Ergodicity of infinite products of matrices
Ergodicity, in its most general form, deals with the long-term behavior of dynamical systems. The oversmoothing phenomenon in GNNs defined in the sense of (2) concerns the convergence of all rows of  $X^{(t)}$  to a common vector. To this end, we define ergodicity in our analysis as the convergence of infinite matrix products to a rank-one matrix with identical rows.
Definition 3 (Ergodicity). Let  $B \in \mathbb{R}^{(N-1) \times N}$  be the orthogonal projection onto the space orthogonal to span  $\{\mathbf{1}\}$ . A sequence of matrices  $\{M^{(n)}\}_{n=0}^{\infty}$  is ergodic if
$$
\lim  _ {t \rightarrow \infty} B \prod_ {n = 0} ^ {t} M ^ {(n)} = 0.
$$
We will take advantage of the following properties of the projection matrix  $B$  already established in Blondel et al. [2]:
1.  $B\mathbf{1} = 0$
2.  $\| Bx\| _2 = \| x\| _2$  for  $x\in \mathbb{R}^N$  if  $x^{\top}\mathbf{1} = 0$
3. Given any row-stochastic matrix  $P \in \mathbb{R}^{N \times N}$ , there exists a unique matrix  $\tilde{P} \in \mathbb{R}^{(N-1) \times (N-1)}$  such that  $BP = \tilde{P}B$ .