# The Modeling of Time Series Based on Least Square Fuzzy Cognitive Map
## 3 The Learning of Least Square Fuzzy Cognitive Map  
Different from traditional learning approaches for FCMs, the fuzzy weight matrix is obtained with the least square method in this study. Compared with the traditional method, learning the FCM with least square method is a one-time solution of the matrix equation rather than multi-iteration stochastic searching. The learning of the FCM least square method is abbreviated to LSFCM. The activation function \(f(x)=\frac{1}{1+e^{-\lambda x}}\) is a sigmoid function, let us consider the function,  
\[A_{(t+1)i}=\frac{1}{1+e^{-\lambda\sum_{j=1}^{c}A_{tj}w_{ji}}} \tag{6}\] \[\Rightarrow e^{-\lambda\sum\limits_{j=1}^{c}A_{tj}w_{ji}}=\left(A_{(t +1)i}\right)^{-1}-1\] \[\Rightarrow -\lambda\sum\limits_{j=1}^{c}A_{tj}w_{ji}=\ln\Bigl{(}\left(A_{(t +1)i}\right)^{-1}-1\Bigr{)} \tag{7}\]  
where \(t\) is the discrete time, \(t=~{}1,~{}2,~{}3,\ldots n-1\), and \(c\) is the number of concepts. We have,  
\[\sum\limits_{j=1}^{c}A_{tj}w_{ji}=-\lambda^{-1}\ln\Bigl{(}\left(A_{(t+1)i} \right)^{-1}-1\Bigr{)} \tag{8}\]  
\(0<A_{i(t+1)}<\). Let \(Y_{(t+1)i}\triangleq-\lambda^{-1}\ln\Bigl{(}\left(A_{(t+1)i}\right)^{-1}-1 \Bigr{)}\), then,  
\[\sum\limits_{j=1}^{c}A_{tj}w_{ji}=Y_{(t+1)i} \tag{9}\]  
The primitive function is \(\left(A_{t},A_{(t+1)i}\right)\to A_{i(t+1)}=f\Bigl{(}A_{(t)}\Bigr{)}\) and now we have \(\left(A_{t},Y_{(t+1)i}\right)\to Y_{(t+1)i}=\phi(A_{t})\); the least square method is used to estimate \(w_{ji}\). The sum of squares to be minimized, the fitness function, is described as follows,\[J=arg\underset{w}{\min}\frac{1}{2}\sum_{i=1}^{c}\sum_{l=1}^{n-1}\left(Y_{(l+1)i}- \sum_{j=1}^{c}A_{ij}w_{ji}\right)^{2} \tag{10}\]  
The state values of concepts at the current time moment are described by \(X\), \(X=\left[A_{1}\ A_{2}\ \cdots\ A_{n-1}\right]^{T}\); it results in a set of concepts' state values at the next time moment \(\overline{Y}=\left[A_{2}\ A_{3}\ \cdots\ A_{n}\right]^{T}\), and \(Y_{i}=-\lambda^{-1}\ln\left(1./\overline{Y}_{i}-1\right)\). Then, the fitness function can be written in matrix form,  
\[\min:\frac{1}{2}\sum_{i=1}^{c}\left(Y_{i}-XW_{i}\right)^{2} \tag{11}\]  
where \(Y\) is a \((n-1)\times c\) matrix, \(Y_{i}\) is the \(i\)th column of \(Y\), \(W_{i}\) is the \(i\)th column of \(W\). Then we can solve the estimated \(W_{i}\) as  
\[W_{i}^{*}=arg\underset{W_{i}}{\min}\frac{1}{2}\|Y_{i}-XW_{i}\|^{2}+\beta\|W_{ i}\|^{2} \tag{12}\]  
The term \(\|W_{i}\|^{2}\) is the \(L_{2}\)-norm of \(W_{i}\) which reduces the collinearity of data, \(\beta\geq 0\). Let \(\frac{\partial I_{i}}{\partial W_{i}}=0\), then \(-X^{\mathrm{T}}(Y_{i}-XW_{i})=0\), we obtain the solution,  
\[W_{i}^{*}=\left(X^{\mathrm{T}}X+\beta I\right)^{-1}X^{\mathrm{T}}Y_{i} \tag{13}\]  
The vector \(W_{i}\) is one column of weight matrix \(W\). We the estimated \(W\) as,  
\[W^{*}=\left(X^{\mathrm{T}}X+\beta I\right)^{-1}X^{\mathrm{T}}Y \tag{14}\]  
We can obtain the estimated values of \(W\) by solving (14); there is no iteration and result obtainable at one stroke. Note that the values of the \(W^{*}\) may not be all in the interval \([-1,1]\), when the least square method is used to learn the FCM. Owing to \(Y=-\lambda^{-1}\ln\left(1./\overline{Y}-1\right)\), (14) can be rewritten as follows,  
\[\Rightarrow e^{-\lambda\sum\limits_{j=1}^{c}A_{tj}w_{ji}}=\left(A_{(t +1)i}\right)^{-1}-1\]0  
It is shown from the formulas that the values of \(W^{*}\) are linearly proportional to the shape parameter \(\lambda\), viz., \(W^{*}\propto\lambda^{-1}\). Suppose \(\lambda=1\), then \(W^{1}=\left(X^{\mathrm{T}}X+\beta I\right)^{-1}X^{\mathrm{T}}\left(-\ln\left( 1./\overline{Y}-1\right)\right)\); it is not sure that the values of elements of the matrix are all in the interval \([-1,1]\). In order to obtain the suitable estimated weight matrix, let \(\lambda_{1}=\max\left\{\left|w_{ij}^{1}\right|\right\}\), \(i,j=1,2,\cdots c\) and \(\lambda\geq\lambda_{1}\), then \(W^{*}\) can be calculated by (15) and the values of elements of the matrix are all in the interval \([-1,1]\).