# Comparing Commit Messages and Source Code Metrics for the Prediction Refactoring Activities
## 3 Research Methodology and Conduction
### Feature Extraction  
#### 3.4.1 Text-Based Model  
Feature extraction includes extracting keywords from commits; these extracted features are used to build a training dataset. For feature extraction, we have used a word embedding library from Keras, which provides the indexes for each word. Word embedding helps to extract information from the pattern and occurrences of words. It is an advanced method that goes beyond traditional feature extraction methods from NLP to decode the meaning of words, providing more relevant features to our model for training. Word embedding is represented by a single n-dimensional vector where similar words occupy the same vector. To accomplish this, we have used pretrained GloVe word embedding. The GloVeword embedding technique is efficient since the vectors generated by using this technique are small in size, and none of the indexes generated are empty, reducing the curse of dimensionality. On the other hand, other feature extraction techniques such as n-grams, TF-IDF, and bag of words generate very huge feature vectors with sparsity, which causes memory wastage and increases the complexity of algorithm.  
Steps followed to convert text into word embedding: We converted the text into vectors by using tokenizer function from Keras, then converted sentences into numeric counterparts and applied padding to the commit messages with shorter length. Once we had the padded number sequence representing our commit message corpus, we then compared it with the pretrained GloVe word embedding and created the embedding matrix that has words from commit and respective values for each GloVe embedding. After these steps, we have word embeddings for all words in our corpus of commit messages.  
**Text-Based Model Building.** Model building and training: To build the model with commit messages as input in order to predict the refactoring type (see Figure 3), we used Keras functional API after we obtained the word embedding matrix. We followed the following steps:  
* We created a model with an input layer of word embedding matrix, LSTM layer, which provided us with a final dense layer of output.
* For the LSTM layer, we used 128 neurons; for the dense layer, we have five neurons since there are five different refactoring classes.
* We have Softmax as an activation function in the dense layer and categorical_crossentropy as the loss function.
* As shown in Table 3, we also performed parameter hypertuning in order to choose the values of activation function, optimizer, loss function, number of nodes, hidden layers, epoch, number of dense layers, etc. The dataset and source code of this experiments is available on GitHub [[https://github.com/smilevo/refactoring-metrics-prediction](https://github.com/smilevo/refactoring-metrics-prediction)]([https://github.com/smilevo/refactoring-metrics-prediction](https://github.com/smilevo/refactoring-metrics-prediction)) (accessed on 20 September 2021).
* We trained this model on 70% of data with 10 epochs.
* After checking training accuracy and validation accuracy, we observed that this model is not overfitting.
* To test the model with only commit messages as input, we used 30% of data, and we used the evaluate function from the Keras API to test the model on test dataset and visualized model accuracy and model loss.