# Comparative Analysis of Exemplar-Based Approaches for Students' Learning Style Diagnosis Purposes
## 2 Methods
### Bayesian Approach and Bayesian Inference in Generative Models
In the Markov chain Monte Carlo (MCMC) method, when conditional distributions are not known exactly, a \"guide\" is proposed, and the proposal is accepted or rejected using feedback from model [17]. In other words, instead of using a predetermined guide distribution, as this is done in the case of importance sampling, random walking over \(Z\) is performed. \(Z\) is such that the proportion of samples \(z^{*}\) is proportional to \(\mathrm{p}(z^{*}|x)\). Randomly walking, we take correlated samples from a Markov chain, and a new sample is based on the feedbackfrom the previous sample. In MCMC it is presumed that the posterior distribution is stationary [17]. Having a prior distribution of latent variables, generative models predict the likelihood of observations (posterior distribution), given a latent variable. The Monte Carlo approach is used to generate a hypothetical posterior distribution based on \"guess parameters\". Having a prior over the parameters and some likelihood function, it is possible to combine those to compute the posterior of the parameters (according to [PERSON] theorem):  
\[\mathrm{p}(\theta|X)=\frac{\mathrm{p}(X|\theta)\mathrm{p}(\theta)}{\mathrm{p}( X)} \tag{3}\]  
Here, \(\mathrm{p}(\theta|X)\) is the posterior, \(\mathrm{p}(X|\theta)\) is likelihood, \(\mathrm{p}(\theta)\) is prior, and \(\mathrm{p}(X)\) is a normalizing constant. MCMC is applied when direct sampling from joint posterior distribution is difficult. It uses Gibbs sampling to estimate the model parameters, i.e., evaluation of the conditional posterior distribution of each variable conditioned on the other variables. In other words, Gibbs sampling is used to obtain a sample from the posterior distribution with multiple variables. In Gibbs sampling the conditional probability of one axis given all the others must be computed, i.e., sampling from all of the conditional distributions for all the model's parameters must be possible. Practical difficulty is faced in that this is rarely possible unless only simple models are used. In Gibbs sampling the parameter value simulated from its posterior distribution in one iteration step is used as the conditional value in the next step [18]. Repeating the process provides the result of an approximate random sample to be drawn from the posterior distribution. Sampling begins by creating a preliminary clustering to generate start values for the parameters [18]. The start values may be determined through a qualified guess or using neutral values. Clustering is preferred since the Markov chains converge faster when the start values are closer to their target values. A non-hierarchical clustering is used with an iterative algorithm that minimizes the sum of distances from each object to its cluster centroid, over all clusters. This algorithm moves objects between clusters until the sum cannot be decreased further. The result is a set of clusters that are compact and well-separated [18]. As Gibbs sampling is slow, collapsed Gibbs sampling is used. Its idea is to sample from a distribution with one of the conditioned variables (one of the full conditionals) integrated out. Thus, in summary, in Bayesian updating without a prior conjugate, the Gibbs update on a randomly chosen subset of the new full data set is performed since previous data points are dependent on the new data (for example, Latent Dirichlet allocation (LDA) uses Gibbs sampling algorithm [19]). The size of the subset may be adjusted to achieve an appropriate trade-off between speed and accuracy [16]. Theoretically, the data are generated by the following process: first, \(\mathrm{p}(\theta)\) is sampled, and then the observables x from a distribution which depends on \(\theta\) are sampled, i.e., \(\mathrm{p}(\theta,X)=\mathrm{p}(\theta)\times\mathrm{p}(x|\theta)\). Note that \(\mathrm{p}(x|\theta)\) can take a variety of parametric forms [20]. For classical Bayesian inference, assumption about a likelihood function for the data must be done, and then parameters may be estimated.