Hence, the K-mean clustering (K-MC) technique [15; 16] and the Gaussian mixture model (GMM) [17; 18] are sequentially used to initialise the HMM process. Since the K-MC and GMM techniques are embedded in the Gaussian emission process of HMM, the latter is referred to as an ensemble hidden Markov model (EHMM) in this article. Therefore, this article evaluates the performance of PCA and MRE when combined with EHMM (PCA-EHMM and MRE-EHMM). Likewise, the performance of PCA and MRE is verified when combined with the conventional HMM (PCA-HMM and MRE-HMM), demonstrating the necessity of the initialisation. Moreover, the performance of the proposed MRE-EHMM and PCA-EHMM techniques is validated with other classic ML models such as random forest (RF), linear regression (LR), decision trees (DT), and K-nearest neighbour (KNN). The performances are documented for different performance metrics such as recall/sensitivity (\(\mathcal{S}_{e}\)), specificity (\(\mathcal{S}_{p}\)), precision (\(\mathcal{P}\)), and F1-score (\(\mathcal{F}\)1) using the credit card transactions dataset of European cardholders gathered within two days in September 2013 ([[https://www.kaggle.com/mlg-ulb/creditcardfraud](https://www.kaggle.com/mlg-ulb/creditcardfraud)]([https://www.kaggle.com/mlg-ulb/creditcardfraud](https://www.kaggle.com/mlg-ulb/creditcardfraud)), accessed on 21 July 2024). The key contributions and significance of this article are outlined as follows. While various studies have proposed the use of HMM for predicting credit card fraud, these works often lack clarity regarding the feature extraction techniques employed and their integration with the HMM. Feature extraction is crucial for enhancing the model's predictive accuracy [11]. In this article, two feature extraction techniques are proposed, which are tailored for use with the EHMM to effectively predict fraudulent credit card transactions. The EHMM approach presented here distinguishes itself from the conventional HMM models used in credit card fraud detection in two significant ways: (1) HMM is systematically integrated with K-MC and GMM techniques to form an EHMM, enhancing the model's predictive capability, and (2) two feature extraction techniques (PCA and MRE) are introduced and can be seamlessly integrated with the EHMM to further improve its performance. Moreover, the results obtained from applying these techniques demonstrate high reliability and are reproducible for real-time credit card transaction monitoring. This article is expected to contribute to reducing financial losses for cardholders and institutions while bolstering confidence in online banking involving credit card transactions. The remaining part of this article is organised as follows. Section 2 briefly reviews some of the recent works on the prediction of credit card frauds using HMMs. In Section 3, the dataset used for result verification is discussed. The PCA and MRE feature extraction techniques are explained in Section 4. Section 5 discusses the EHMM in detail while also explaining its training process. In Section 6, the metrics used in documenting the performances of the proposed feature extraction techniques are briefly explained. Section 7 discusses the obtained results from the PCA-EHMM and MRE-EHMM prediction techniques. In addition, this section highlights the performance difference between the EHMM and conventional HMM approaches, and also compares the PCA-EHMM and MRE-EHMM techniques to other existing ML techniques in the literature. The article is concluded with some final remarks in Section 8.