# Fast Minimum Error Entropy for Linear Regression
## 1 Introduction  
Entropy serves as a pivotal metric for quantifying the inherent uncertainty within a system, finding ubiquitous application across a multitude of domains including dimension reduction [1, 2], parameter estimation [3, 4], identification [5, 6], feedback control [7, 8], and abnormal detection [9, 10, 11]. The minimum error entropy (MEE) criterion stands out as a notable framework aimed at minimizing the entropy associated with estimation errors, thereby mitigating uncertainty within the estimation model. Particularly pertinent to linear regression estimation quadaries, MEE diverges from conventional least square methodologies by not only considering the variance of prediction errors, but also incorporating higher-order cumulants, rendering it adept at handling non-Gaussian noise distributions [12, 13]. Given the prevalence of non-Gaussian noise in real-world scenarios, the efficacy of MEE has been duly validated across various applications, encompassing adaptive filtering [14, 15, 16], face recognition [17, 18], sparse system identification [19, 20, 21], stochastic control systems [22, 23], and visible light communication [24]. Furthermore, the convergence properties of MEE have been meticulously examined in the prior literature [25, 26, 27]. In-depth analyses delved into the convergence dynamics of fixed-point MEE methodologies [28], while investigations into the interplay betweenMEE and Minimum Mean-Squared Error (MMSE) [29], as well as its relationship with maximum correntropy [30], have been extensively explored. Noteworthy extensions to the conventional MEE framework include the proposal of kernel MEE variants [31], elucidations on regularization strategies for MEE implementations [32], and the development of semi-supervised and distributed adaptations [33]. Moreover, within the errors-in-variables (EIV) modeling realm, robustness analyses of MEE methodologies have been expounded upon [34], alongside the proposition of novel methodologies such as the minimum total error entropy approach [35]. The computational demands associated with MEE criterion stem from its time complexity, which scales quadratically as \(O(n^{2})\). Central to MEE's computational load is the necessity for a double summation operation during the calculation of the gradient pertaining to the error probability density function (PDF), particularly when employing Parzen windowing [36] and certain types of Kernel functions. Moreover, the selection of an inappropriate Kernel function or its parameters can detrimentally impact the accuracy of the resulting PDF. To solve the problem, efforts have been made to optimize the gradient descent MEE approach, as evidenced by the normalization of the step size in [37], leveraging the power of the input entropy. Additionally, innovative methodologies such as the utilization of a quantization operator, as proposed in [38], have been instrumental in mitigating computational complexity. By mapping the error onto a series of real-valued code words, this approach effectively transforms the double summation operation into a singular form, consequently reducing the algorithm's computational overhead. This paper presents a novel approach, the fast minimum error entropy estimation (FMEE) algorithm, tailored specifically for linear regression tasks, leveraging the polynomial expansion technique applied to the error Probability Density Function (PDF). In contrast to the traditional minimum error entropy (MEE) methodology employing Parzen windowing for error PDF derivation, FMEE adopts the Gram-Charlier expansion method to approximate the error PDF, resulting in a notable reduction in time complexity from \(O(n^{2})\) to \(O(n)\). Notably, FMEE obviates the need for Kernel functions or their associated parameters. The proposed algorithm entails several key steps: firstly, the derivation of the PDF approximation for a random variable via the Gram-Charlier expansion, followed by the simplification of the entropy of the random variable facilitated by the orthogonality of the Hermite polynomials.