# Regression Coefficient Derivation via Fractional Calculus Framework

Authors: Muath Awadalla; Yves Yannick Yameni Noupoue; Yucel Tandogdu; Kinda Abuasbeh
Journal: Journal of Mathematics
 (2022)

---

## Abstract

This study focuses on deriving coefficients of a simple linear regression model and a quadratic regression model using fractional calculus. The work has proven that there is a smooth connection between fractional operators and classical operators. Moreover, it has also been shown that the least squares method is classically used to obtain coefficients of linear and quadratic models that are viewed as special cases of the more general fractional derivative approach which is proposed.


---

## Body

## 1. Introduction

Initial concepts of fractional calculus (FC) can be traced back to seventeenth century when Isaac Newton, Leibniz, and L’ Hospital discussed preliminary ideas that shed light to future developments on FC. A considerable time elapsed before mathematicians returned to discuss the idea of FC. In 1819, Lacroix [1] mentioned the derivative of arbitrary order. Euler and Fourier also mentioned the derivatives of an arbitrary order. The first applications are made by Abel [2] in 1823. Ross [3] provided a historical track on the fundamental theory of FC.The new research trend in FC since last century focuses on the investigation of its application in real life problems. In this regard, researchers have produced thousands of articles from various branches of sciences in which they showed how some problems defined in the classical calculus could be converted into FC problems. Another class of researchers in the fields consists of those who not only defined fractional calculus approach of solving problems but who also managed to show in their works that the FC might be more efficient than its classical counterpart in solving problems. In either case, it is common for researchers to back up their FC finding with similar results from classical calculus.Fractional differential equations are a part of FC in which a substantial amount of work has been undertaken aiming to prove the power of the method over the classical differential equations. Works carried out along these lines with applications in Biology [4, 5], Physics [6], and Finance [7] are just a few to mention. Some more recent work on FC worth to mention are found, for instance, in [8], where the authors investigated bending of the beam using fractional differential equations. Sumelka et al. [9] reformulated the classical Euler–Bernoulli beam theory using fractional calculus. Stempin and Sumelka [10] studied the bending analysis of nanobeams aiming to improve the space-fractional Euler–Bernoulli beam (s-FEBB) theory. Sidhardh et al. [11]. presented their findings on their studies of the analytical and finite element formulation of a geometrically nonlinear and fractional-order nonlocal model of a Euler–Bernoulli beam. The size-dependent bending behavior of nanobeams is studied by Oskouie et al. [12] employing the Euler–Bernoulli beam theory, where the nonlocal effects are obtained via fractional calculus.Solving methods of fractional differential equations are not necessarily like classical differential equations. Therefore, researchers prefer to define a fractional model to a problem and compare it with an existing classical model of the same problem. Such approach has drawn our attention, and the investigation of the existence of a smooth transition between FC and classical calculus is undertaken in this study.In this work, we have chosen to investigate the derivation of the coefficients of simple linear and quadratic regression using FC. It is has become evident that the solution of the fractional approach offers a more general solution. Hence, the classical solution becomes a special case of the fractional solution. The aim of the work is to lay a foundation for the application of fractional calculus in statistics. That is to investigate to which extend fractional calculus can be used directly for estimating parameters of a statistical model. Hence, we mainly focus on proving that the fractional model is well defined for solving the problem.


## 2. Deriving the Coefficients of Linear and Quadratic Regressions

Classical approaches of linear and quadratic regressions are discussed in this section. Indeed, these are two models of data fitting with many applications in various branches of sciences. In general, these models are used to fit data collected from experiments in which there is a predictor, also known as the independent variableX and a dependent or response variable Y. Dataset to be used should be made of pairs xi,yii=1n, where n is the sample size. A simple linear model is defined by(1)Y=b0+b1x,where b1 is the slope and b0 is the y intercept. Equation (1) is written provided that there is a deterministic relationship between the independent variable X and the response variable Y. However, in practice, such relationship does not exists as experimentally collected data are random in nature and contain random errors. A least squares technique is used in minimizing the error when fitting a linear model to available data. Statistically, the simple linear model is a best fit to the dataset. Then, (1) becomes(2)Y=b0+b1x+ε,where the variable ε represents the random errors in the process. Moreover, it is assumed that Eε=0 and Varε=σ. Introducing estimated coefficients of b0 and b1 denoting them by B0 and B1, respectively, the fitted model becomes(3)Y^=B0+B1x.Recalling the pairs of experimental dataxi,yii=1n, and denoting those fitted by the model xi,Yii=1n, it follows that the random error can be expressed as(4)yi−EYi=yi−EB0+B1X=εi.CoefficientsB0 and B1 computed using the least squared error method are those that best fit the model given by (1). Given the sum of squared error (SSE) between the observed and the fitted data points is denoted by(5)SSE=∑j=1nyj−y^j2=∑j=1nyj−B0−B1xi2.It is shown in [13] that the simple optimization of (5) helps to find the optimal value of B0 and B1 as(6)B1=∑j=1nxj−x¯yj−y¯∑j=1nxj−x¯2B0=y¯−B1x¯.With a similar approach, the optimal coefficients for a quadratic model are computed. Given an experimental datasetxi,yii=1n, where its scatter plot exhibits an upward or downward concavity which is an indication that a quadratic model is appropriate. The deterministic quadratic model is defined as(7)Y=b0+b1x+b2x2,with b2≠0, otherwise, the model becomes linear.Since in practice, experimental datasets are to be fitted, and a random error term is added. Hence, (7) becomes(8)Y=b0+b1x+b2x2+ε.The variableε represents the randomness in the process, with Eε=0 and Varε=σ. Letting B0,B1,andB2 to be the estimates of b0,b1,andb2, respectively, the fitted model becomes(9)Y^=B0+B1x+B2x2.Recalling the pairs of experimental dataxi,yii=1n, and corresponding fitted model pairs xi,Yii=1n, the random error can be expressed as(10)yi−EYi=yi−EB0+B1X+B2X2=εi.CoefficientsB0, B1, and B2 computed using the least squares error method are those that best fit the model given by (7). The sum of squared errors (SSE) between the observed and the fitted data points is denoted by(11)SSE=∑j=1nyj−y^j2=∑j=1nyj−B0−B1xi−B2xi22.Optimal values of the coefficientsB0, B1, and B2 are easily derived from the following system of 3 equations obtained from (11).(12)∑j=1nxj4∑j=1nxj3∑j=1nxj2∑j=1nxj3∑j=1nxj2∑j=1nxj∑j=1nxj2∑j=1nxjnB2B1B0=∑j=1nxj2yj∑j=1nxjyj∑j=1nyj.Results obtained in the current section are based on Newtonian calculus. In the next section, analogue results are built using FC.


## 3. Deriving the Coefficients of Fractional Linear and Quadratic Regressions

This section presents similar results to those obtained in the previous section based on FC. We prefer to define and use FC tools whenever they are needed. We consider the linear model problem defined by (1) following the SSE setting (5). The aim is to use fractional partial fraction derivative to compute optimal coefficients of the fitted model.Definition 1 (see [3]).
The Riemann–Liouville fractional integral of orderq>0 for a function g:0,+∞⟶ℝ is defined as(13)IRL0+qgt=1Γq∫0tt−sq−1gsds.
Provided that the right-hand side of the integral is point-wise defined on0,+∞, and Γ is the gamma function. Γυ=∫0∞e−ttυ−1dt,∀υ>0.Definition 2 (see [3]).
The Caputo derivative of orderq>0 for a function g:0,+∞⟶ℝ is defined as(14)DC0+qgt=∫0tt−sn−q−1gnsΓn−qds,n−1<q<n,q∈ℝ,gnt,q∈N,where n=q+1,q is the integer part of q.Definition 3.
Given a function of two variablesf, the fractional partial derivative of order q with respect to one variable is defined as ∂q/∂xqfx,y=I0,x1−q∂/∂xfx,y.Lemma 1.
We considerq such that n−1<q<n , with n∈ℕ. Moreover, let a∈ℝ; the fractional integral ∫0aa−s−qsds=a2−q/1−q2−q is a well-defined number.Proof of Lemma1.
It is done through integration by parts. Indeed, settingu=s,dv=a−s−α is a good hint to go through to the solution.
Minimization of the SSE defined by (5) is performed in order to obtain the best coefficient for B0 and B1 using fractional derivative. The results are given by some theorems and their proofs are as follows:Theorem 1.
We consider the simple linear model defined by (1), and the best coefficients of the fitted model (5) using fractional derivative are given by B1=∑i=1nxiyi−2−qnx¯y¯/1/2−q∑i=1nxi2−n2−qx¯2 and B0=2−qy¯−B1x¯, where q represents the fractional order of the derivative.Proof of Theorem1.
Fractional derivative is used. In fact, the minimization of the SSE, (5), using the partial fractional derivative is written as follows:(15)∂q∂B0qSSEB0,B1=∂q∂B0q∑j=1nyj−B0−B1xj2=I0,B01−q∂q∂B0qSSEB0,B1=I0,B01−q−2∑j=1nyj−B0−B1xj=−21Γ1−q∫0B0B0−s−q∑j=1nyj−s−B1xjds=−2Γ1−q∫0B0B0−s−q∑j=1nyjds−∫0α^B0−s−q∑j=1nsds−∫0B0B0−s−q∑j=1nB1xjds=−2Γ1−q∑j=1nyj∫0B0B0−s−qds−n∫0B0B0−s−qsds−B1∑j=1nxj∫0B0B0−s−qds=−2Γ1−q∑j=1nyjB01−q1−q−nB02−q1−q2−q−B1∑j=1nxjB01−q1−q=−2Γ1−qB01−q1−q∑j=1nyj−nB02−q−B1∑i=1nXi=0,
implying thatB0=2−qy¯−B1x¯. The other coefficient is derived similarly. Indeed, let the partial derivative be computed with respect to B1. Then,(16)∂q∂B1qSSEB0,B1=∂q∂B1q∑j=1nyj−B0−B1xj2=I0,B11−q∂α∂B1αSSEB0,B1=I0,B11−q−2∑j=1nxjyj−B0xj−B1xj2=−21Γ1−q∫0B1B1−s−q∑j=1nxjyj−B0xj−sxj2ds=−2Γ1−q∫0B1B1−s−q∑j=1nxjyjds−∫0B1B1−s−q∑j=1nB0xjds−∫0B1B1−s−q∑j=1nsxi2ds=−2Γ1−q∑j=1nxjyj∫0B1B1−s−qds−∑j=1nB0xj∫0B1B1−s−qsds−∑j=1nxj2∫0B1B1−s−qsds=−2Γ1−q∑i=1nxjyjB11−q1−q−B0∑j=1nxjB11−q1−q−∑j=1nxj2B12−q1−q2−q=−2Γ1−qB11−q1−q∑j=1nxjyj−B0∑j=1nxj−∑j=1nxj2B12−q=0.
Last equation leads to the following∑i=1nxjyj=B0∑j=1nxj−∑j=1nxj2B1/2−q, in which B0=2−qy¯−B1x¯ is plugged in to obtain a compact form B1 as follows:(17)∑j=1nxjyj=2−qy¯∑j=1nxj−2−qB1x¯∑j=1nxj−∑j=1nxj2B12−q,∑j=1nxjyj−2−qy¯∑j=1nxj=2−qB112−q2∑i=1nxj2−x¯∑j=1nxj,∑j=1nxjyj−2−qnx¯y¯=2−qB112−q2∑j=1nxj2−nx¯2,∑j=1nxjyj−2−qnx¯y¯=2−qB112−q2∑j=1nxj2−nx¯2.
The last equation leads toB1=∑i=1nxiyi−2−qnx¯y¯/1/2−q∑i=1nxi2−n2−qx¯2. This ends the proof of Theorem 1.
Given the SSE of the quadratic model defined by (11), FC tools can be used to compute coefficients B0, B1, and B2 of the fitted model. Theorem and proof of this assertion are given below.Theorem 2.
Given the quadratic model (7), the Least square-based coefficients of the fitted model (9) computed using fractional derivative that would best fit the model are as follows:(18)B2=2−q−∑j=1nB1xj3+B0xj2−xj2yj∑j=1nxj4,B1=2−q−∑j=1nB2xj3+B0xj−xjyj∑j=1nxj2,B0=2−q−∑j=1nB2xj2+B1xj2−yjn.Proof of Theorem2.
The proof is done by the minimization of the SSE, (11) using fractional derivative. To ease computational procedure, the SSE is expanded as follows:(19)SSE=∑j=1nyj−y^j2=∑j=1nyj−B0−B1xj−B2xj22=∑j=1nB2xj2+B1xj+B0−yj2=∑j=1nB22xj4+2B2xj2B1xj+B0−yj+B1xj+B0−yj2=B22∑j=1nxj4+2B2∑j=1nB1xj3+B0xj2−xj2yj+∑j=1nB1xi+B0−yj2.
Applying the partial fractional derivative to the expanded SSE, equation (19) leads to(20)∂q∂B2qSSEB0,B1,B2=I0,B21−q2B2∑j=1nxj4+2∑j=1nB1xj3+B0xj2−xj2yj=2Γ1−q∑j=1nxj4∫0B2B2−s−qsds+∑j=1nB1xj3+B0xj2−xj2yj∫0B2B2−s−qds=2Γ1−qB21−q1−q∑j=1nxj4B22−q+∑j=1nB1xj3+B0xj2−xj2yj=0.
It follows from (20) that(21)B2=2−q−∑j=1nB1xj3+B0xj2−xj2yj∑j=1nxj4.
Applying similar methods to those given by (19) and (20), the remaining coefficients are given as follows:(22)B1=2−q−∑j=1nB2xj3+B0xj−xjyj∑j=1nxj2,(23)B0=2−q−∑j=1nB2xj2+B1xj−yjn.
This ends the proof of Theorem2.
It is clear that the estimated coefficients of the quadratic model (see equations (21)–(23)) are interconnected, making it difficult to compute any of them independently. In this regard, equations (21)–(23) are arranged to form a system of equations, (24) to express and compute the coefficients simultaneously.(24)B2∑j=1nxj42−q+B1∑j=1nxj3+B0∑j=1nxj2=∑j=1nxj2yjB2∑j=1nxj3+B1∑j=1nxj2−q+B0∑j=1nxj2=∑j=1nxjyj,B2∑j=1nxj2+B1∑j=1nxj+B0∑j=1n12−q=∑j=1nyj
Equation (24) is written in a matrix form as(25)12−q∑j=1nxj4∑j=1nxj3∑j=1nxj2∑j=1nxj312−q∑j=1nxj∑j=1nxj2∑j=1nxj2∑j=1nxj12−qnB2B1B0=∑j=1nxj2yj∑j=1nxjyj∑j=1nyj.
Goodness of fit model measures how explainable is the target data through the model. As coefficient of determination,R2 is a popular goodness-of-fit metric used in regression analysis. It is computed using the sum of squared regression (SSR) error also known as explainable error and the total sum of squared deviation of the data from its mean.Definition 4.
We consider a dataset inxi,yi format, where xi is the predictor and yi the response variable. Let yi^ be the fitted response variable through a linear regression model. The coefficient of determination of the model is computed as follows:(26)R2=∑y^i−y¯2∑yi−y¯2=1−∑yi−y^i2∑yi−y¯2.
In practice,R2∈0,1 with a value of 0 means that the model does not explain the target variable, whereas a value of 1 means that the model is perfect, thus explaining the entire variation in the target dataset.
The square root of theR2 is the absolute value of a new metric called the correlation coefficient and is denoted R or r. In practice, instead of using (26) to compute R2, one can just compute the square of the correlation coefficient. Computed R∈−1,+1 with a value of 0 means expressing no correlation between xi and yi; whereas values of -1 and 1 means perfect negative and perfect positive correlation, respectively.
LetB1 be the slope of the linear regression model defined by (3). Let Sx, Sy be the standard deviations of xi and yi, respectively. There exists a relationship between B1 and r defined by(27)r=B1.SxSy.
SinceSx>0 and Sy>0, it follows from (27) that r and B1 play similar role in determining whether the linear relationship between xi and yi is positive or negative. Besides their common relationship, B1 determines the steepness of the regression line. Unlike R2 and r that are restricted to the closed intervals [0, 1] and [−1, 1], respectively, B1 is unitless and its magnitude solely depend on the range of the datasets xi and yi.


## 4. Experiment and Comparison

This section aims to simulate the formulae established in previous sections. Two datasets are considered.The first dataset which was retrieved from the Kaggle database [14] which consists of two variables, the predictor (X-variable) is the number of years of working experience of employees, whereas the response variable Y is their yearly salaries in USD. The scatter plot of the dataset shows a linear trend (see Figure 1(a)). Next to Figure 1(a) is the Figure 1(b), which represents three different models aiming to fit the dataset. The blue line is the linear regression taken in the classical sense, whereas the grey and brown lines represent fractional models taken, respectively, for q = 0.98 and q = 1.01. These lines were taken to illustrate the behavior of the proposed model. A global trend of the error rate of the classical approach and the fractional approach is shown in Figure 1(c). In fact, all possible values of the fraction in the interval [0.9–1.3] were considered using a step h = 0.001 between two values. This led to a total of 400 possible values. Each value was used in building the fractional model, and the fitting error rate was computed using the formula found in [15]. It was observed that when the fractional order of derivative coincides with the classical first derivative, both methods produce the same error rate. This is evident that the proposed method is well defined and that there exists a smooth transition from the fractional to the classical approach in this case.Figure 1
(a) Scatter plot, (b) fitting lines, and (c) error graph of linear regression.
(a)(b)(c)When the fractional order of derivative coincides with the classical approach, the correlation coefficient of the model isr=0.978 and the coefficient of determination is R2=0.957; standard deviations are Sx=2.8379 and Sy=27414.43. The slope of the model is B1=9450. These information leads to the conclusion that more than 95% of variation in salary depends on the number of years of experience. The relationship is positive, which means salary increases with years of experience.The second dataset which was retrieved from the United Nation database [16] consists of two variables; the predictor (X-variable) is the indexes of years between 1970 and 1995, and the response variable Y is the Bosnia and Herzegovina population within those years. The scatter plot of the dataset shows somewhat a quadratic trend (see Figure 2(a)). Beside is Figure 2(b), which represents three different models fitted to the dataset. The blue curve is the classical quadratic regression, whereas the grey and brown curves represent fractional models taken, respectively, for q = 0.98 and q = 1.001. These curves were meant to illustrate the behavior of the proposed model. A global trend of the error rate of the classical approach and the fractional approach is shown in Figure 2(c). In fact, all possible values of the fraction in the interval [0.9–1.1] were considered using a step h = 0.001. This resulted in a total of 200 possible values of q. Each q value was used in building the fractional model, and the error rate of the model was computed using the formula found in [15]. It is observed that when the fractional order of derivative coincides with the classical first derivative, both methods produce the same error rate. This is evident that the proposed method is well defined, and similar to the linear case, a smooth transition from the fractional to the classical case is observable.Figure 2
(a) Scatter plot, (b) fitting lines, and (c) error graph of quadratic regression.
(a)(b)(c)There are many prospective applications of the proposed method. In particular, the fractional derivative approach proposed in this work appears as a powerful alternative to estimate the fuzzy coefficients in a fuzzy linear model. Several authors have investigated coefficient computation of the linear model with a crisp predictor but fuzzy response [17–19]. In a book chapter on fuzzy linear regression [17], authors proposed a Monte Carlo approach to build the confidence interval of fuzzy coefficients and their estimations. The obtained coefficients were then written as a triangular fuzzy number. More interesting fact is that the Monte Carlo simulation was time consuming as it required about 30 min to complete the process. Based on experimental studies, we believe that two fractional derivatives with fractions from either side of 1 would be efficient enough to build such fuzzy coefficients. This is not in the scope of the current work, but the said approach will be investigated in forthcoming work.


## 5. Conclusion and Future Work

In this work, we investigated computation of the coefficients of simple linear regression and quadratic regression using fractional derivative as a result expressions for the regression coefficients of the linear and quadratic models which is established under the fractional approach. Their solutions investigated and applied to two different datasets. Moreover, a smooth transition was observed between the classical model and the fractional model as both models coincide when the fractional order of derivativeq = 1. Another interesting fact that deserves in depth investigation in future work is to determine if the proposed fractional method can be a powerful alternative tool to Monte Carlo and other approaches used in estimating fuzzy coefficients in fuzzy regression analysis. The results of this work lay a foundation of further studies to investigate other statistical methods using FC approach.For the future work, researchers can investigate a modified estimation for the regression coefficients by using a different type of fractional derivative such as Caputo, Psi-Caputo, and Hadamard fractional derivatives and trying to compare the accuracy of each estimator via the statistical methods.



---

*Source: 1144296-2022-04-09.xml*