# Estimation of Distribution Algorithms with Fuzzy Sampling for Stochastic Programming Problems
## 2 Estimation of Distribution Algorithms  
EDAs were firstly introduced in [44] as a new population-based method, and have been extensively studied in the field of global optimization [26; 44]. Despite the fact that EDAs were firstly proposed for combinatorial optimization, many studies have been performed applying them to continuous optimization. The primary difference between EDAs is the aspect of building the probabilistic model. Generally, in continuous optimization there are two considerable branches: one is based on the Gaussian distribution model [25; 26; 45; 46; 47; 48; 49; 50; 51], and the other on the histogram model [47; 52; 53; 54; 55; 56; 57; 58]. The first is the most widely used and has been studied extensively. The main steps of general EDAs are stated in Algorithm 1. ```
1:\(g\gets 0\)
2:\(P_{g}\leftarrow\) Generate and evaluate \(M\) random individuals (the initial population). 3:repeat
4:\(P_{g}^{s}\leftarrow\) Select \(m(\leq M)\) individuals from \(P_{g}\) according to a selection method. 5:\(D_{g}(x)=p_{g}(x|P_{g}^{s})\leftarrow\) Estimate the joint probability distribution of the selected individuals. 6:\(P_{g+1}\leftarrow\) Generate \(M\) individuals using \(D_{g}(x)\), and evaluate them. 7:\(g\gets g+1\). 8:until a stopping criterion is met. ```  
**Algorithm 1** Pseudo-code for EDA approach  
In the case of adapting a Gaussian distribution model \(D_{g}(x)\) in Algorithm 1, it has the form of a normal density with a mean \(\hat{\mu}\) and a covariance matrix \(\Sigma\). The earliest proposed EDAs were based on simple univariate Gaussian distributions, such as the Marginal Distribution Algorithm for continuous domains (\(UMDA_{c}^{G}\)) and Population-Based Incremental Learning for continuous domains(\(PBIL_{c}\)) [26; 45]. In these, all variables are taken to be completely independent of each other, and the joint density function is  
\[f_{l}(\mathbf{x};\mathbf{\Theta}^{l})=\prod_{i=1}^{n}f_{l}(\mathbf{x_{i}};\mathbf{\Theta_{i} ^{l}}), \tag{2}\]  
where \(\Theta^{l}\) is a set of local parameters. Such models are simple and easy to implement with a low computational cost, but they fail with high dependent variable problems. For this problem, many EDAs based on multivariate Gaussian models have been proposed, which adapt the conventional maximum likelihood-estimated multivariate Gaussian distribution, such as Normal IDEA [46; 47], EMNA\({}_{global}\)[26], and EGNA [25; 26]. These methods have the same performance, since they are based on the same multivariate Gaussian distribution, and there is no significant difference between them [26]. However, in these methods the dependence between variables is taken, so they have a poor exploitative ability and the computational cost increases exponentially with the problem size [59]. To address this problem, various extensions of these methods have been introduced, which depend on scaling \(\Sigma\) after estimating the maximum likelihood according to certain criteria to improve the exploration quality. This has been done in methods such as EEDA [48], SDR-AVS-IDEA [50], and CT-AVS-IDEA [49]. The EDA with Model Complexity Control (EDA-MCC) method was introduced to control the high complexity of the multivariate Gaussian model without losing the dependence between variables [43]. Since the univariate Gaussian model has a simple structure and limited computational cost, it has difficulty solving nonseparable problems. On other hand, the multivariate Gaussian model can solve nonseparable problems, but it usually has difficulty as a result of its complexity and cost. In the EDA-MCC method, the advantages of the univariate and multivariate Gaussian models are combined according to certain criterion and by applying two main strategies:  
* **Weakly Dependent Variable Identification (WI). ** In this strategy, the correlation coefficients between variables are calculated to measure how much they are dependent.