# Improving Multi-Instance Multi-Label Learning by Extreme Learning Machine
## 3 The Proposed Approach MIMLELM
### Determination of the Number of Clusters  
The primary important task for MIMLELM is to transform MIML into SIML. Unlike MIMLSVM, which performs the transformation through a clustering process with a user-specified parameter for the number of clusters, we utilize AIC [39], a model selection criterion, to automatically determine the number of clusters.  
_AIC_ is founded on information theory. It offers a relative estimation of the information lost when a given model is used to represent the process that generates the data. For any statistical model,the general form of \(AIC\) is \(AIC=-2 ln(L)+2K\), where \(L\) is the maximized value of the likelihood function for the model and \(K\) is the number of parameters in the model. Given a set of candidate models, the one of the minimum \(AIC\) value is preferred [39].  
Let \(M_{k}\) be the model of the clustering result with \(k\) clusters \(C_{1}\), \(C_{2}\), \(\ldots\), \(C_{k}\), where the number of samples in \(C_{i}\) is \(m_{i}\). \(X_{i}\) denotes a random variable indicating the \(PD\) value between any pair of micro-clusters in \(C_{i}\). Then, under a general assumption commonly used in the clustering community, \(X_{i}\) follows a Gaussian distribution with \((\mu_{i},\sigma_{i}^{2})\), where \(\mu_{i}\) is the expected \(PD\) value between any pair of micro-clusters in \(C_{i}\), and \(\sigma_{i}^{2}\) is the corresponding variance. That is, the probability density of \(X_{i}\) is:  
\[p(X_{i})=\frac{m_{i}}{m}\cdot\frac{1}{\sqrt{2\pi\sigma_{i}}}\exp(-\frac{1}{2 \sigma_{i}^{2}}(X_{i}-\mu_{i})^{2}) \tag{1}\]  
Let \(x_{ij}\) (1\(\leq\)\(j\)\(\leq\)\(C_{m_{i}}^{2}\)) be an observation of \(X_{i}\); the corresponding log-likelihood w.r.t the data in \(C_{i}\) is:  
\[\ln L(C_{i}|\mu_{i},\sigma_{i})=\ln\prod_{j=1}^{C_{m_{i}}^{2}}p(X_{i}=x_{ij}) =\sum_{j=1}^{C_{m_{i}}^{2}}(\ln\frac{1}{\sqrt{2\pi\sigma_{i}}}-\frac{1}{2 \sigma_{i}^{2}}(x_{ij}-\mu_{i})^{2}+\ln\frac{m_{i}}{m}) \tag{2}\]  
Since the fact that the log-likelihood for all clusters is the sum of the log-likelihood of the individual clusters, the log-likelihood of the data w.r.t \(M_{k}\) is:  
\[\ln L(M_{k}|\mu_{1},\mu_{2},\ldots,\mu_{k},\sigma_{1},\sigma_{2},\ldots,\sigma _{k})=\sum_{i=1}^{k}\ln L(C_{i}|\mu_{i},\sigma_{i})=\sum_{i=1}^{k}\sum_{j=1}^{ C_{m_{i}}^{2}}(\ln\frac{1}{\sqrt{2\pi\sigma_{i}}}-\frac{1}{2\sigma_{i}^{2}}(x_{ ij}-\mu_{i})^{2}+\ln\frac{m_{i}}{m}) \tag{3}\]  
Further, take the MLE (maximum likelihood estimate) of \(\sigma_{i}^{2}\), _i.e._: \(\sigma_{i}^{2}\)\(=\)\(\frac{1}{C_{m_{i}}}\sum\limits_{j=1}^{C_{m_{i}}^{2}}(x_{ij}-\mu_{i})^{2}\), into Equation (3); we obtain that:  
\[\begin{split}&\ln L(M_{k}|\mu_{1},\mu_{2},\ldots,\mu_{k},\sigma_{1}, \sigma_{2},\ldots,\sigma_{k})=-\frac{\sum\limits_{i=1}^{k}C_{m_{i}}^{2}}{2} \ln(2\pi)\\ &-\frac{\sum\limits_{i=1}^{k}C_{m_{i}}^{2}\ln(\widehat{\sigma_{i }^{2}})}{2}-\frac{\sum\limits_{i=1}^{k}C_{m_{i}}^{2}}{2}+\sum\limits_{i=1}^{k} C_{m_{i}}^{2}\ln m_{i}-\ln m\sum\limits_{i=1}^{k}C_{m_{i}}^{2}\end{split} \tag{4}\]  
Finally, in our case, the number of independent parameters \(K\) is \(2k\). Thus, \(AIC\) of the model \(M_{k}\) is:  
\[AIC_{M_{k}}=\ln(2\pi m^{2}e)\sum\limits_{i=1}^{k}C_{m_{i}}^{2}+\sum\limits_{i= 1}^{k}C_{m_{i}}^{2}\ln(\widehat{\sigma_{i}^{2}})-2\sum\limits_{i=1}^{k}C_{m_{ i}}^{2}\ln m_{i}+4k \tag{5}\]