# Fast Minimum Error Entropy for Linear Regression
## 3 Methodology
\begin{split} H(x)&=H(\xi)-\int\frac{1}{2}\varphi( \xi)\bigg{[}[\frac{\kappa_{3}(x)}{3!\sigma^{3}}H_{3}(\frac{\xi-\mu}{\sigma})]^ {2}+\\ &[\frac{\kappa_{4}(x)}{4!\sigma^{4}}H_{4}(\frac{\xi-\mu}{\sigma}) ]^{2}+\\ & 2\frac{\kappa_{3}(x)}{3!\sigma^{3}}H_{3}(\frac{\xi-\mu}{\sigma} )\frac{\kappa_{4}(x)}{4!\sigma^{4}}H_{4}(\frac{\xi-\mu}{\sigma})\bigg{]} \mathrm{d}\xi\\ &-\int\varphi(\xi)\bigg{[}\log\frac{1}{\sqrt{2\pi}\sigma}+\log \exp(-\frac{\xi^{2}}{2})\bigg{]}*\\ &\quad\bigg{[}\frac{\kappa_{3}(x)}{3!\sigma^{3}}H_{3}(\frac{\xi- \mu}{\sigma})+\frac{\kappa_{4}(x)}{4!\sigma^{4}}H_{4}(\frac{\xi-\mu}{\sigma}) \bigg{]}\mathrm{d}\xi\\ =& H(\xi)-\frac{\kappa_{3}^{2}(x)}{18\sigma^{6}}- \frac{\kappa_{3}^{2}(x)}{32\sigma^{8}}\\ &\log\frac{1}{\sqrt{2\pi}\sigma}\int\varphi(\xi)\bigg{[}\frac{ \kappa_{3}(x)}{3!\sigma^{3}}H_{3}(\frac{\xi-\mu}{\sigma})+\\ &\frac{\kappa_{4}(x)}{4!\sigma^{3}}H_{4}(\frac{\xi-\mu}{\sigma}) \bigg{]}\mathrm{d}\xi+\frac{1}{2\ln 2}\int\varphi(\xi)\xi^{2}*\\ &\quad\bigg{[}\frac{\kappa_{3}(x)}{3!\sigma^{3}}H_{3}(\frac{\xi- \mu}{\sigma})+\frac{\kappa_{4}(x)}{4!\sigma^{3}}H_{4}(\frac{\xi-\mu}{\sigma}) \bigg{]}\mathrm{d}\xi\\ =&\frac{1}{2}\log 2\pi\mathrm{e}\sigma^{2}-\frac{ \kappa_{3}^{2}(x)}{18\sigma^{6}}-\frac{\kappa_{4}^{2}(x)}{32\sigma^{8}}\end{split}  \tag{25}\]  
So, Equation (14) holds. For the linear regression, \(e_{i}=y_{i}-w^{\mathrm{T}}\mathbf{x}_{i}\), the standard deviation of the error \(e_{w}\) is in Equation (26):  
\[\sigma^{2}(e_{w})=Ep(E_{w}^{2})-[Ep(E_{w})]^{2}=Ep(E_{w}^{2})-(\bar{y}-\mathbf{w}^ {\mathrm{T}}\bar{\mathbf{x}})^{2}, \tag{26}\]where \(g\) is the mean of \(y\) and \(\bar{x}\) donates the mean of \(x\). To simplify the calculation of \(\kappa_{4}(e_{w})\), let \(e_{w}\) become a random variable with 0 mean:  
\[e_{w_{i}}=y_{i}-\mathbf{w}^{\mathrm{T}}\mathbf{x}_{i}-\bar{y}+\mathbf{w}^{\mathrm{T}}\bar{x}. \tag{27}\]  
Notice that the transformation in Equation (27) does not change the standard deviation and the entropy of \(e_{w}\) as \(\sigma^{2}(e+c)=\sigma^{2}(e)\) and \(H(e+c)=H(e)\) when \(c\) is a constant number. In the rest of the paper, to keep clarity, \(\sigma^{2}(\cdot)\) is replaced with \(var(\cdot)\). Then, for linear regression, its entropy of the error can be expressed as Equation (28):  
\[ \begin{split} H(e_{w})=&\frac{1}{2}\log 2\pi \mathrm{e}+\frac{1}{2}\log var(e_{w})\\ &-\frac{\kappa_{3}^{2}(e_{w})}{2*3!var^{3}(e_{w})}-\frac{\kappa_{ 4}^{2}(e_{w})}{2*4!var^{4}(e_{w})},\end{split}  \tag{28}\]  
where  
\[\kappa_{3}(e_{w})=Ep[(y-\mathbf{w}^{\mathrm{T}}\mathbf{x}-\bar{y}+\mathbf{w}^{\mathrm{T}} \bar{\mathbf{x}})^{3}], \tag{29}\]  
\[\kappa_{4}(e_{w})=Ep[(y-\mathbf{w}^{\mathrm{T}}\mathbf{x}-\bar{y}+\mathbf{w}^{\mathrm{T}} \bar{\mathbf{x}})^{4}]-3. \tag{30}\]  
To minimize \(H(e_{w})\), the derivative of \(H(e_{w})\) with respect to \(w\) is calculated in Equation (31):  
\[ \begin{split}\frac{\partial H(e_{w})}{\partial\mathbf{w}}=& \frac{1}{2 var(e_{w})\ln 2}\frac{\partial var(e_{w})}{\partial\mathbf{w}}- \frac{1}{2*3!var^{6}(e_{w})}*\\ &\big{[}2 var^{3}(e_{w})\kappa_{3}(e_{w})\frac{\partial\kappa_{3}( e_{w})}{\partial\mathbf{w}}-\\ & 3\kappa_{3}^{2}(e_{w})var^{2}(e_{w})\frac{\partial var(e_{w})}{ \partial\mathbf{w}}\big{]}\\ -&\frac{1}{2*4!var^{8}(e_{w})}\big{[}2 var^{4}(e_{w })\kappa_{4}(e_{w})\frac{\partial\kappa_{4}(e_{w})}{\partial\mathbf{w}}-\\ & 4\kappa_{4}^{2}(e_{w})var^{3}(e_{w})\frac{\partial var(e_{w})}{ \partial\mathbf{w}}\big{]},\end{split}