SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Zhiyu Zhang*
zhiyuz@seas.harvard.edu
Ioannis Ch. Paschalidis
yannisp@bu.edu
Motivated by the challenge of nonstationarity in sequential decision making, we study Online Convex Optimization (OCO) under the coupling of two problem structures: the domain is unbounded, and the comparator sequence  $u_{1}, \ldots, u_{T}$  is arbitrarily time-varying. As no algorithm can guarantee low regret simultaneously against all comparator sequences, handling this setting requires moving from minimax optimality to comparator adaptivity. That is, sensible regret bounds should depend on certain complexity measures of the comparator relative to one's prior knowledge. This paper achieves a new type of such adaptive regret bounds leveraging a sparse coding framework. The complexity of the comparator is measured by its energy and its sparsity on a user-specified dictionary, which offers considerable versatility. For example, equipped with a wavelet dictionary, our framework improves the state-of-the-art bound [JC22] by adapting to both  $(i)$  the magnitude of the comparator average  $\| \bar{u} \| = \| \sum_{t=1}^{T} u_{t} / T \|$ , rather than the maximum  $\max_{t} \| u_{t} \|$ ; and  $(ii)$  the comparator variability  $\sum_{t=1}^{T} \| u_{t} - \bar{u} \|$ , rather than the uncentered sum  $\sum_{t=1}^{T} \| u_{t} \|$ . Furthermore, our analysis is simpler due to decoupling function approximation from regret minimization.
Nonstationarity is prevalent in sequential decision making, which poses a critical challenge to the vast majority of existing approaches developed offline. Consider weather forecasting for example  $\left[\mathrm{SBG}^{+}21\right]$ . A meteorologist typically starts from the governing physical equations and simulates them online using high performance computing; the imperfection of this physical model can lead to time-varying patterns in its forecasting error. Alternatively, a machine learning scientist may build a data-driven model from historical weather datasets, but its online deployment is subject to distribution shifts. If the structure of such nonstationarity can be exploited in our algorithm, then we may expect better forecasting performance. This paper investigates the problem from a theoretical angle – we aim to improve nonstationary online decision making by incorporating temporal representations.
Concretely, we study Online Convex Optimization (OCO), which is a repeated game between us (the player) and an adversarial environment  $\mathcal{E}$ . In each (the  $t$ -th) round, with a mutually known Lipschitz constant  $G$ :
1. We make a prediction  $x_{t} \in \mathbb{R}^{d}$  based on the observations before the  $t$ -th round.
2. The environment  $\mathcal{E}$  reveals a convex loss function  $l_{t}:\mathbb{R}^{d}\to \mathbb{R}$  dependent on our prediction history  $x_{1},\ldots ,x_{t};l_{t}$  is  $G$  -Lipschitz with respect to  $\| \cdot \| _2$
3. We suffer the loss  $l_{t}(x_{t})$
The game ends after  $T$  rounds, and then, our total loss is compared to that of an alternative sequence of predictions  $u_{1},\ldots ,u_{T}\in \mathbb{R}^{d}$ . Without knowing the time horizon  $T$ , the environment  $\mathcal{E}$  and the
*Work done at Boston University. Future versions available at https://arxiv.org/abs/2301.13349.
comparator sequence  $\{u_t\}_{t\in \mathbb{Z}}$ , our goal is to achieve low unconstrained dynamic regret
$$
\operatorname {R e g r e t} _ {T} \left(u _ {1: T}\right) := \sup  _ {\varepsilon} \left[ \sum_ {t = 1} ^ {T} l _ {t} \left(x _ {t}\right) - \sum_ {t = 1} ^ {T} l _ {t} \left(u _ {t}\right) \right]. \tag {1}
$$
Fixing any comparator  $\{u_t\}_{t \in \mathbb{Z}}$ : if such an expression can be upper-bounded by a sublinear function of  $T$ , then asymptotically, in any environment, we perform at least as well as the  $\{u_t\}_{t \in \mathbb{Z}}$  sequence.
The above setting deviates from the most standard setting of OCO [Haz16, Ora19] in two ways.
- Structure 1. The domain  $\mathbb{R}^d$  is unbounded.
- Structure 2. The comparator is allowed to be time-varying.
While the latter has been studied extensively in the literature (since [Zin03]) to account for nonstationarity, most existing approaches require a time-invariant bounded domain to set the hyperparameters properly, which, to some extent, limits the amount of nonstationarity they can handle. One might argue that most practical problems have a finite range, which could be heuristically estimated from offline datasets. However, such a heuristic is not robust in nature, as underestimates will invalidate the theoretical analysis, and overestimates will make the regret bound excessively conservative. It is thus important to study the more challenging unconstrained dynamic setting<sup>2</sup> combining the two problem structures, where algorithms cannot rely on pre-selected range estimates at all.
Taking a closer look at their analysis, it is perhaps a little surprising that these two problem structures share a common theme, despite being studied mostly separately. In either the unconstrained static setting [MO14, OP16, CO18] or the bounded dynamic setting [Zin03, HW15, ZLZ18], the standard form of minimax optimality [ABRT08, AABR09, RS14b] becomes vacuous, as it is impossible to guarantee that  $\sup_{u_{1:T}}\mathrm{Regret}_T(u_{1:T})$  is sublinear in  $T$ . Circumventing this issue relies on comparator adaptivity<sup>3</sup> – instead of only depending on  $T$ , any appropriate regret upper bound, denoted by  $\mathrm{Bound}_T(u_{1:T})$ , should also depend on the comparator  $u_{1:T}$  through a certain complexity measure. Intuitively, despite the intractability of hard comparators, nonvacuous bounds can be established against “easy ones”. A total loss bound then follows from the oracle inequality
$$
\sum_ {t = 1} ^ {T} l _ {t} \left(x _ {t}\right) \leq \inf  _ {u _ {1: T}} \left[ \sum_ {t = 1} ^ {T} l _ {t} \left(u _ {t}\right) + \operatorname {B o u n d} _ {T} \left(u _ {1: T}\right) \right]. \tag {2}
$$
A crucial observation is that the complexity of  $u_{1:T}$  is not uniquely defined: one could imagine bounding  $\mathrm{Regret}_T(u_{1:T})$  by many different non-comparable functions of  $u_{1:T}$ . Essentially, this complexity measure serves as a Bayesian prior: choosing it amounts to assigning different priorities to different comparators  $u_{1:T} \in \mathbb{R}^{d \times T}$ . The associated algorithm guarantees lower  $\mathrm{Bound}_T(u_{1:T})$  against comparators with higher priority, and due to Eq.(2), the total loss of our algorithm is low if some of these high priority comparators actually achieve low loss  $\sum_{t=1}^{T} l_t(u_t)$ . Such a Bayesian reasoning highlights the importance of versatility in this workflow: in order to place an arbitrary application-dependent prior, we need a versatile algorithmic framework that adapts to a wide range of complexity measures. This leads to the limitations of existing results, discussed next.
To our knowledge, [JC22] is the only existing work that considers our setting. Two unconstrained dynamic regret bounds are presented based on three statistics of the comparator sequence, the maximum range  $M \coloneqq \max_{t} \| u_{t} \|_{2}$ , the norm sum  $S \coloneqq \sum_{t=1}^{T} \| u_{t} \|_{2}$  and the path length  $P \coloneqq \sum_{t=1}^{T-1} \| u_{t+1} - u_{t} \|_{2}$ . First, using a 1D unconstrained static algorithm as a simple range scalar, the paper achieves [JC22, Lemma 10]
$$
\operatorname {R e g r e t} _ {T} \left(u _ {1: T}\right) \leq \tilde {O} \left(\sqrt {(M + P) M T}\right). \tag {3}
$$
It is known that unconstrained OCO algorithms can also handle time-varying (but not necessarily bounded) domains in a black-box manner [Cut20, Section 4].
<sup>3</sup>In general, adaptivity means achieving near minimax optimality simultaneously for many restricted subclasses of the problem, where minimax optimality is well-defined [Joh19, Chapter 6].
4The prior can be selected on the fly, depending on the observation history. This brings key practical benefits: Appendix E discusses how an empirical forecaster based on domain knowledge or deep learning could be "robustified" using our framework.
Table 1: Comparison in almost static environments. Each row improves the previous row (omitting logarithmic factors), c.f., Appendix A.
Algorithm	P-dependent bound	K-switching regret	Example 1	Example 2
ADER [ZLZ18] (meta-expert OGD)	\(\tilde{O}\left(\sqrt{(D+P)DT}\right)\)	\(\tilde{O}\left(D\sqrt{(1+K)T}\right)\)	N/A	\(\tilde{O}(T^{3/4})\)
[JC22, Algorithm 6] (range scaling)	\(\tilde{O}\left(\sqrt{(M+P)MT}\right)\)	\(\tilde{O}\left(M\sqrt{(1+K)T}\right)\)	\(\tilde{O}(T)\)	\(\tilde{O}(T^{3/4})\)
[JC22, Algorithm 2] (centered MD)	\(\tilde{O}\left(\sqrt{(M+P)S}\right)\)	\(\tilde{O}\left(\sqrt{(1+K)MS}\right)\)	\(\tilde{O}(T^{3/4})\)	\(\tilde{O}(T^{3/4})\)
Ours (Haar OLR)	\(\tilde{O}\left(\|\bar{u}\|_2\sqrt{T}+\sqrt{PS}\right)\)	\(\tilde{O}\left(\|\bar{u}\|_2\sqrt{T}+\sqrt{KE}\right)\)	\(\tilde{O}(\sqrt{T})\)	\(\tilde{O}(\sqrt{T})\)
Then, by developing a customized mirror descent approach, most of the effort is devoted to improving  $MT$  to  $S$  [JC22, Theorem 4], i.e., adapting to the magnitude of individual  $u_{t}$ .
$$
\operatorname {R e g r e t} _ {T} \left(u _ {1: T}\right) \leq \tilde {O} \left(\sqrt {(M + P) S}\right). \tag {4}
$$
Despite the strengths of these results and their nontrivial analysis, a shared limitation is that both bounds depend explicitly on the path length  $P$ . Intuitively, it means that good performance is only guaranteed in almost static environments: in the typical situation of  $S = \Theta (T)$ , these bounds are only sublinear when  $P = o(T)$ , which rules out important persistent dynamics such as periodicity. Moreover, even the second bound still depends on  $MS$  instead of a finer characterization of each individual  $u_{t}$ 's magnitude. That is, the mission of removing  $M$  is not fully accomplished yet.
The goal of this paper is to extend comparator adaptivity to a wider range of complexity measures. For almost static environments in particular, quantitative benefits will be obtained from specific instances of this general approach.
The contributions of this paper are twofold.
1. First, we present an algorithmic framework achieving a new type of unconstrained dynamic regret bounds. It is based on a conversion to vector-output Online Linear Regression (OLR): given a dictionary  $\mathcal{H}$  of orthogonal feature vectors spanning the sequence space  $\mathbb{R}^{dT}$ , we use an unconstrained static OCO algorithm to linearly aggregate these feature vectors, which are themselves time-varying prediction sequences. Such a procedure guarantees
$$
\operatorname {R e g r e t} _ {T} \left(u _ {1: T}\right) \leq \tilde {O} \left(\sqrt {E \cdot \operatorname {S p a r s i t y} _ {\mathcal {H}}}\right), \tag {5}
$$
where  $E = \sum_{t=1}^{T} \| u_t \|^2_2$  is the energy of the comparator  $u_{1:T}$ , and Sparsity $_{\mathcal{H}}$  measures the sparsity of  $u_{1:T}$  on the dictionary  $\mathcal{H}$ . Both  $E$  and Sparsity $_{\mathcal{H}}$  are unknown beforehand.
Compared to [JC22], the main advantage of this framework is its versatility. Prior knowledge on the transform domain can be incorporated by picking  $\mathcal{H}$ , and favorable algorithmic properties can be conveniently inherited from static online learning.
2. Our second contribution is quantitative: although [JC22] is specifically crafted to handle almost static environments, we show that equipped with a Haar wavelet dictionary, our framework actually guarantees better bounds (Table 1) in this setting, which is a surprising finding to us.
- With the comparator average  $\bar{u} := \sum_{t=1}^{T} u_t / T$  and the first order variability  $\bar{S} := \sum_{t=1}^{T} \| u_t - \bar{u} \|_2$ , our Haar wavelet algorithm guarantees
$$
\operatorname {R e g r e t} _ {T} \left(u _ {1: T}\right) \leq \tilde {O} \left(\| \bar {u} \| _ {2} \sqrt {T} + \sqrt {P \bar {S}}\right).
$$
The significance of this issue could be seen through an analogy to (static  $D$ -bounded domain) gradient adaptive OCO: although there are algorithms achieving the already adaptive  $O\left(D\sqrt{G\sum_{t=1}^{T}\|g_t\|_2}\right)$  static regret bound, the hallmark of gradient adaptivity is the so-called "second-order bound"  $O\left(D\sqrt{\sum_{t=1}^{T}\|g_t\|_2^2}\right)$ , popularized by ADAGRAD [DHS11]. In a rough but related sense, we aim to achieve "second order comparator adaptivity", which is only manifested in the less studied dynamic regret setting.
For conciseness, we omit  $u_{1:T}$  in the notation. Throughout this paper, the regularity parameters on the RHS of the regret bound generally depend on  $u_{1:T}$ . A list of these parameters is presented in Appendix A, including their relations.
It improves Eq.(4) by (i) a better dependence on the comparator magnitude  $(\sqrt{MS} \to \|\bar{u}\|_2\sqrt{T})$ ; and (ii) decoupling the bias  $\bar{u}$  from the characterization of variability  $(\sqrt{PS} \to \sqrt{P\bar{S}})$ .
- With the number of switches  $K \coloneqq \sum_{t=1}^{T-1} \mathbf{1}[u_{t+1} \neq u_t]$  and the second order variability  $\bar{E} \coloneqq \sum_{t=1}^{T} \| u_t - \bar{u} \|_2^2$ , the same Haar wavelet algorithm guarantees an unconstrained switching regret bound
$$
\operatorname {R e g r e t} _ {T} \left(u _ {1: T}\right) \leq \tilde {O} \left(\left\| \bar {u} \right\| _ {2} \sqrt {T} + \sqrt {K \bar {E}}\right),
$$
which improves the existing  $\tilde{O}\left(\sqrt{(1 + K)MS}\right)$  bound resulting from Eq.(4) and  $P = O(KM)$ .
Due to the local property of wavelets, our algorithm runs in  $O(d\log T)$  time per round, matching that of the baselines. As for the regret, our bounds are never worse than the baselines, and in two examples corresponding to  $\| \bar{u}\| _2\ll M$  and  $S\ll S$ , they reduce to clearly improved rates in  $T$ . Furthermore, our analysis follows from the generic regret bound Eq.(5) and the wavelet approximation theory, providing an intriguing connection between disparate fields.
The paper concludes with an application in fine-tuning time series forecasters, where unconstrained dynamic OCO is naturally motivated. Due to limited space, this is deferred to Appendix E, with experiments that support our theoretical results.
Our paper addresses the connection between unconstrained OCO and dynamic OCO. Although they both embody the idea of comparator adaptivity, unified studies have been scarce.
Unconstrained OCO To obtain static regret bounds in OCO, Online Gradient Descent (OGD) [Zin03] is often the default approach. With learning rate  $\eta$ , it guarantees  $O(\eta^{-1}\| u\|_2^2 + \eta T)$  regret with respect to any unconstrained static comparator  $u \in \mathbb{R}^d$ , and the optimal choice in hindsight is  $\eta = O(\| u\|_2 / \sqrt{T})$ . Without the oracle knowledge of  $\| u\|_2$ , it is impossible to tune  $\eta$  optimally. To address this issue, a series of works (also called parameter-free online learning) [SM12, MA13, GY14, MO14, OP16, FKMS17, CO18, FRS18, MK20, CLW21, ZCP22] developed vastly different strategies to achieve the oracle optimal rate  $O(\| u\| \sqrt{T})$  up to logarithmic factors. That is, the algorithm performs as if the complexity measure  $\| u\|$  is known beforehand.
There is certain flexibility in the choice of the norm  $\| \cdot \|$ :  $L_{1}$  and  $L_{2}$  norm bounds were presented in [SM12], while Banach norm bounds were developed by [FRS18, CO18]. Historically, the connection between the  $L_{1}$  norm and sparsity has powered breakthroughs in batch data science, including LASSO [Tib96] and compressed sensing [CRT06]. However, the parallel path in online learning remains less studied: while the sparsity implication of the  $L_{1}$  norm adaptive bounds has been discussed in the literature [SM12, Ger13, vdH19], there is in general a lack of downstream investigations with concrete benefits. In this paper, we show that the sparsity of the comparator can be naturally associated to the structural simplicity of a nonstationary environment.
Dynamic OCO Comparing against dynamic sequences is a classical research topic. It is clear that one cannot go beyond linear regret in the worst case, therefore various notions of complexity should be introduced.
- The closest topic to ours is the universal dynamic regret, where the regret bound adapts to the complexity of an arbitrary  $u_{1:T}$  on a bounded domain with  $L_p$ -diameter  $D$ . In the most common framework, the complexity measure is an  $L_{p,q}$  norm of the difference sequence  $\{u_{t+1} - u_t\}$ , such as the  $L_{p,1}$  norm, i.e., the path length  $P = \sum_{t=1}^{T-1} \| u_{t+1} - u_t \|_p$  [HW01]. Omitting the dependence on the dimension  $d$  (thus also the choice of  $p$ ), the optimal bound under convex Lipschitz losses is  $\tilde{O}\left(\sqrt{(D + P)DT}\right)$  [Zin03, HW15, JRSS15, GS16, ZLZ18], while the accelerated rate  $\tilde{O}(P^{2/3}T^{1/3} \vee 1)$  can be achieved with strong convexity [BW21, BW22]. Improvements have been
Motivated by nonparametric statistics, the  $L_{p,q}$  norm  $\left(\sum_{t=1}^{T-1} \|u_{t+1} - u_t\|_p^q\right)^{1/q}$  with  $q > 1$  is associated to more homogeneous measures of comparator smoothness [BW19, BW21].
Further omitting the dependence on the diameter  $D$
studied under the additional smoothness assumption [ZZZZ20, ZZZZ21]. These bounds subsume results in switching (a.k.a., shifting) regret, where the complexity of  $u_{1:T}$  is measured by its number of switches  $K$ , as  $P$  is dominated by  $DK$ .
A notable exception is the dynamic model framework from [HW15, ZLZ18]. Still considering a bounded domain, it takes a collection of dynamic models as input, which are mappings from the domain to itself. Then, the complexity of a comparator  $u_{1:T}$  is measured by how well it can be reconstructed by the best dynamic model in hindsight. Essentially, the use of temporal representations is somewhat similar to the dictionary in our framework. The important difference is that instead of using the best feature (or the best convex combination of the features) to measure the comparator, we use linear combinations of the features – this allows handling unconstrained domains through subspace modeling.
- Besides the universal dynamic regret, there are other notions of dynamic regret that do not induce oracle inequalities like Eq.(2), including (i) the restricted dynamic regret [YZJY16, ZYY+17, BW19, BW20, BZW21], which depends on the complexity of certain offline optimal comparators; and (ii) regret bounds that depend on the functional variation  $\sum_{t=1}^{T-1} \max_x |l_t(x) - l_{t+1}(x)|$  [BGZ15, CWW19]. They are not as relevant to our purpose, due to being vacuous on unbounded domains under the linear losses – this is an important setting in our investigation.
Unconstrained (universal) dynamic regret To our knowledge, [JC22] is the only work studying the universal dynamic regret without a bounded domain, whose contributions have been summarized in our Introduction. Here we survey some negative results in the literature.
- The restricted dynamic regret is a special case of the universal dynamic regret, therefore lower bounds for the former apply to the latter as well. For convex Lipschitz losses [YZJY16] and strongly convex losses [BW19], any algorithm should suffer the dynamic regret of  $\Omega(P)$  and  $\Omega(P^2)$ , respectively.
- For dynamic OCO on bounded domains, a recurring analysis goes through the notion of strong adaptivity [DGSS15]: one first achieves low static regret bounds on every subinterval of the time horizon  $[1: T]$ , and then assembles these local bounds appropriately to bound the global dynamic regret  $\left[\mathrm{ZYZ}^{+} 18, \mathrm{Cut20}, \mathrm{BW21}, \mathrm{BW22}\right]$ . Following this route in the unconstrained setting appears to be challenging, as [JC22, Section 4] showed that (a natural form of) strong adaptivity cannot be achieved there.
Additional discussions of related works are deferred to Appendix B, including the more general problem of online nonparametric regression, the more specific problem of parametric time series forecasting, and other orthogonal uses of sparsity in online learning.
For two integers  $a \leq b$ ,  $[a:b]$  is the set of all integers  $c$  such that  $a \leq c \leq b$ . The brackets are removed when on the subscript, denoting a tuple with indices in  $[a:b]$ . Treating all vectors as column vectors,  $\operatorname{span}(A)$  represents the column space of a matrix  $A$ .  $\log$  is natural logarithm when the base is omitted, and  $\log_+( \cdot ) := 0 \lor \log ( \cdot )$ . polylog denotes a poly-logarithmic function of its input. 0 represents a zero vector whose dimension depends on the context.
2 The general sparse coding framework
This section presents our sparse coding framework, achieving the generic sparsity adaptive regret bound Eq.(5). The key idea is to view online learning on the sequence space  $\mathbb{R}^{dT}$ , rather than the default domain  $\mathbb{R}^d$ . Despite its central role in signal processing (e.g., the Fourier transform), such a view is (in our opinion) under-explored by the online learning community. $^{10}$
Notably, [BW19, BW20] creatively employed wavelet techniques to detect change-points of the environment, which, to the best of our knowledge, is the only existing use of wavelets in the online learning literature.
10Possibly due to the emphasis on the static regret: the sequence  $u_{1:T}$  collapses into a time-invariant  $u$ , which is contained in  $\mathbb{R}^d$
To begin with, we follow the conventions in online learning [Haz16, Ora19] to linearize convex losses. Consider that instead of the full loss function  $l_{t}$ , we only observe its subgradient  $g_{t} \in \partial l_{t}(x_{t})$  at our prediction  $x_{t}$ . By using the linear loss  $\langle g_t,\cdot \rangle$  as a surrogate, we can still upper bound the regret Eq.(1) due to  $l_{t}(x_{t}) - l_{t}(u) \leq \langle g_{t},x_{t} - u\rangle$ . The linear loss problem is also called Online Linear Optimization (OLO), where each observation  $g_{t}$  is a  $d$  dimensional vector satisfying  $\| g_t\| _2\leq G$ .
Now, consider the length  $T$  sequences of predictions  $x_{1:T}$ , gradients  $g_{1:T}$  and comparators  $u_{1:T}$ . Let us flatten everything and treat them as  $dT$  dimensional vectors, concatenating per-round quantities in  $\mathbb{R}^d$ . These are called signals. The comparator statistics could be more succinctly represented using vector notations, e.g., the energy  $E = \sum_{t=1}^{T} \| u_t \|^2_2 = \| u_{1:T} \|^2_2$ .
Our framework requires a dictionary matrix  $\mathcal{H} \in \mathbb{R}^{dT \times N}$ , possibly revealed online, whose columns are  $N$  nonzero feature vectors. We write  $\mathcal{H}$  in an equivalent block form as  $[h_{t,n}]_{1 \leq t \leq T, 1 \leq n \leq N}$ , where each block  $h_{t,n} \in \mathbb{R}^{d \times 1}$ . The accompanied linear transform  $u = \mathcal{H} \hat{u}$  relates a signal  $u \in \mathbb{R}^{dT}$  to a coefficient vector  $\hat{u} \in \mathbb{R}^N$  (if it exists). Adopting the convention in signal processing, we will call  $\mathbb{R}^{dT}$  the time domain, and  $\mathbb{R}^N$  the transform domain. In general, symbols without hat refer to time domain quantities, while their transform domain counterparts are denoted with hat.
Summarizing the above, we consider the following concise interaction protocol. Despite its parametric appearance, our main focus is on the nonparametric regime, where the dictionary size  $N$  scales with the amount of data  $T$ .
Vector-output OLR with linear losses In the  $t$ -th round, our algorithm observes a  $d$ -by-N feature matrix  $\mathcal{H}_t \coloneqq [h_{t,n}]_{1 \leq n \leq N}$ , linearly combines its columns into a prediction  $x_t \in \mathbb{R}^d$ , receives a loss gradient  $g_t \in \mathbb{R}^d$ , and then suffers the linear loss  $\langle g_t, x_t \rangle$ . We assume that  $\| h_{t,n} \|_2 \leq 1$ ,  $\sum_{t=1}^{T} \| h_{t,n} \|_2^2 \geq 1$  and  $\| g_t \|_2 \leq G$ . The performance metric is the unconstrained dynamic regret defined in Eq.(1).
2.2 Main result