In a legume species recognition system proposed by [10], an accuracy of 76% was achieved on 5400 images with a specificity of 99.10%, edging out the deep learning models used in the study by more than 10%. In the same study, VGG-16 was also used. It managed to achieve an accuracy of 93.79%; however, it was outperformed by Inception-v3. In a study conducted by [11], the accuracy achieved for SVM was 100%. It utilizes the dataset obtained from the UCI Machine Learning Repository. The model has been tested using 10-fold cross-validation to obtain the average accuracy of the model. It is on par with C4.5 algorithm in terms of accuracy, which is 100%, however it loses out on processing time as it is slower by 6.67 s. In the study conducted by [9], the SVM algorithm did not perform well, as it only scored 58.3% in accuracy when trained on eigen features with real dimensions. It scored 83% when trained on eigen features with virtual dimensions, and 86.1% when trained on parametric features (contrast, skewness, kurtosis, entropy, mean, standard deviation, energy, correlation, and homogeneity). It achieved its highest accuracy when trained on histogram features. Despite the progress made with machine learning, there remain opportunities for further improvement. For example, deep learning allows for automatic feature extraction and can achieve a higher accuracy when using a large dataset. In a study conducted by [13], the highest accuracy achieved was 84% with VGG-16, followed by Inception-v3 with 82%. Both models were trained on raw images only. In another comparison with models trained on contrast-enhanced images, Inception-v3 achieved an accuracy of 88%, followed by VGG16 with 84%. In another study by [12], the accuracy achieved by their Mobilenet-V2_GAP_flatten_fc model was 97.20% in their testing dataset which contained five classes. According to the literature, there is a scarcity of research on classifying mushroom species, particularly within the context of Malaysia. The majority of existing studies primarily concentrate on distinguishing mushrooms as either edible or inedible. Other than that, the majority of current research studies utilize a text-based dataset which is based on hypothetical scenarios of where certain mushrooms can be found. This study aims to develop a novel machine learning and image processing-based system for recognizing mushrooms species in Malaysia. This can be divided into two stages: first, curating a unique dataset tailored to Malaysia's ecology, and second, introducing a deep learning model for recognizing mushroom species using an image-based dataset. Additionally, this study also compares the performance of the traditional convolutional neural network architecture against a novel transformer architecture with the same task, under the same parameters. The experiment shows promising results and proves that deep learning can be used to identify different species of mushrooms using an image-based dataset. The ViT-L/16 model is particularly promising as it has the highest accuracy among all other models tested. The model also exhibits the highest recall and precision which is important for it to be practically applicable. Although there is always room for improvement, this paper can serve as a basis for future researchers to conduct more in-depth studies in this area.