# Machine-Learning-Based Password-Strength-Estimation Approach for Passwords of Lithuanian Context
## 4 Password-Strength-Estimation Model
### The Results of the Experimental Investigation  
Data classification tasks are widely used in various fields, especially when machine learning models must be created. There exist many types of classification algorithms [29; 30], such as neural networks, decision-tree-based, probabilistic algorithms, deep learning algorithms [31], or transformers [32]. The performed related works analysis showed that traditional machine learning algorithms are still frequently used because of the simplicity in the model training or retraining processes and because of their speed when the new dataset item is fed to the model when the model is used in a real environment. Our experimental investigation examined five widely used machine learning algorithms: Naive Bayes, linear, decision tree, kNN, and SVM. The effectiveness of these methods has already been proven in terms of speed. All selected classification methods have been trained using the combined dataset constructed from English and Lithuanian passwords. It is necessary to mention that in total, four datasets have been analyzed because some features differ according to the one similarity measure that has been used (Fuzz, Levenshtein Jaro, Levenshtein Jaro Winkler, and Levenshtein Ratio). In the training process, cross-validation has been used, where the k-fold number is equal to 10, and the stratified option has been selected. The accuracy, precision, recall, and F1 scores of each model have been calculated. Figure 6 presents the results of each model's performance in terms of accuracy. As we can see, the lowest accuracy was obtained using the Naive Bayes algorithm. Better results are obtained when the Levenshtein Ratio similarity measure is chosen (61%). The linear and SVM algorithms perform similarly and reach 70% accuracy. There is not much influence on the results of the different similarity measures used. The results of the kNN (about 75%) and decision tree algorithms (about 77%) are higher than those of other methods. The best accuracy performance is obtained by the decision tree algorithm, where accuracy reaches 78%. The experimental investigation has shown that the Levenshtein Ratio similarity measure allows us to obtain slightly better accuracy for all algorithms analyzed compared to the other similarity measures used. Figure 7 presents a deeper analysis of the results of the decision tree algorithm. We can see the results of the precision, recall, and F1 scores of the decision tree algorithm using the Levenshtein Ratio similarity measures. It is obvious that the results of Class 0 (too guessable) are the worst in terms of all the measurements of the machine learning models. Figure 6: The performance of each machine learning model in terms of accuracy. This class had a small number of dataset items (Figure 4) because it is not common for users to create and use such simple passwords. The confusion matrix of the decision tree model using the Levenshtein Ratio similarity measure is presented in Figure 8. As we can see, the majority of Class 0 (too guessable) data items have not been predicted correctly, and usually dataset items have been considered as Class 1 (very guessable). Class 4 (very unguessable) has been predicted the best, where the model has made the smallest number of incorrect predictions. The prediction of Classes 2 and 3 have also been correctly predicted in a high ratio. As was mentioned, to evaluate the machine learning model's effectiveness in simulating a real environment, two separate subsets of international (9999 passwords) and Lithuanian (16,269) passwords have been hidden from the model's training and fed to the highest-accuracy-obtained model (decision tree). Figure 9 presents the results of the model's accuracy. As we can see, the accuracy of the model using testing subsets differs slightly from that of the trained model. In the case of the international-password subset,  
Figure 8: The confusion matrix of the decision tree model using the Levenshtein Ratio similarity measure.