# Hamiltonian Monte Carlo with Random Effect for Analyzing Cyclist Crash Severity
## 2 Method
### Finite Mixture Model  
Here, instead of employing subjective assignment for our random effect, we implemented an objective method for assigning the random effect attribute. Thus, it is important to discuss the methodological approach of coming up with an objective attribute. The method of the finite mixture model (FMM) assumes that the dataset arises from data components. In the model-based method, instead of working with distance measures in K-means, for instance, we would work with a set of dimensional data. A multivariate Gaussian distribution might be assumed for each component of \(f_{k}(x_{i};\theta_{k})\) by parameters of \(N(\mu_{k},\sum_{k})\). The estimated clusters with multivariate Gaussian distribution would get the shape of ellipsoid with the means being centered at \(\mu_{k}\), while volume, shape and orientation would be determined by the covariance matrix of \(\sum_{k}\). The covariance matrices' parameters would be identified by Eigen-decomposition [16]:  
\[\sum_{k}=\lambda_{k}D_{k}A_{k}D_{k}^{\intercal} \tag{1}\]  
where \(\lambda_{k}\) is scalar with the eigenvalue, \(A_{k}\) is the diagonal matrix, with the diagonal elements proportional to the corresponding eigenvalues, and \(D_{k}\) is an orthogonal matrix accommodating the eigenvector. These parameters determine volume, shape and orientation of the ellipsoid based on combinations of setting constrains on \(\sum_{k}\), and based on various components of \(\sum_{k}\). The finite mixture model could be written as follows:  
\[f(x_{i};\varphi)=\sum_{k=1}^{U}\pi_{k}f_{k}(x_{i};\varphi_{k})=\sum_{k=1}^{U} \pi_{k}f_{k}\Big{(}x_{i};\mu_{k},\sum_{k}\Big{)},\text{ where }\pi_{k}>0,\text{ and }\sum_{k=1}^{U}\pi_{k}=1 \tag{2}\]  
The parameters of the mixture model are \(\varphi=\{\mu_{i},\dots,\mu_{U};\sum_{1},\dots,\sum_{U}\}\), and \(f_{k}\) is the density for observation \(x_{i}\) with parameters vectors of \(\theta_{k}\). We know that the density of multivariate normal density of \(f_{k}\) as \(\varphi_{k}\) would be written as:  
\[\varphi_{k}(x_{i}|\mu_{k},\sum_{k})=\frac{exp\Big{(}-0.5(x_{i}-\mu_{k})^{ \intercal}\sum_{k}^{-1}(x_{i}-\mu_{k})\Big{)}}{det(2\pi\sum_{k})^{0.5}} \tag{3}\]  
The expectation maximization (EM) algorithm is an iterative method for finding maximum likelihood, where the model depends on the unobserved latent variable. For the EM algorithm, the complete data are \(x_{i}=(y_{i},z_{i})\), where \(z_{i}\) is an unobserved proportion of data, while the \(y_{i}\) is observed. Now, the E step is [17]:  
\[\hat{z}_{ik}=\frac{\pi_{k}f_{k}(y_{i}|\varphi_{k})}{\sum_{j=1}^{U}\pi_{j}f_{j }(y_{i}|\varphi_{j})} \tag{4}\]  
The M part would be employed to maximize the log likelihood equation by considering the fixed parameter of \(z_{ik}\) instead of \(\hat{z}_{ik}\) from above as  
\[\mathcal{L}(\varphi_{k},\tau_{k},z_{ik}|x)=\sum_{i=1}^{n}\sum_{k=1}^{U}z_{ik} log[\pi_{k}f_{k}(y_{i}|\varphi_{k})] \tag{5}\]  
Model Parameters Estimation  
The model implementations, in general, largely depend on Equations (4) and (5). First, to start the process, it is required to have initial values for cluster allocations and the covariance matrix. Those could be assigned by methods such as the K-means technique. Now, the single matrix of covariance for all classes would be passed to the next algorithm for the finite mixture model process. The objective function is to maximize the likelihood of \(f(x_{i};\varphi)\) in Equation (2) by EM algorithm. The EM algorithm would be employed to alternate between the two steps of \"E\" and \"M\". In the \"E\" step, the complete data, or the probability density function (PDF) of multivariate distribution, would be estimated based on Equation (3). Then, the obtained values would be used in the M step for determining parameters, maximizing the log likelihood, which is based on estimated parameters in E step. In summary, the E step is based on Equation (4) and M is based on Equation (5).