# Decision Support Using Machine Learning Indication for Financial Investment
## 4 Development
### Logistic Regression  
The regression method basically performs the prediction of _Yi_ from the knowledge of _xi_. There are several regression techniques that estimate the relationship between variables, for example: logistic, linear, non-linear, and ridge, among others [46]. Linear regression refers to the relationship between variables, where the conditional expectation of \(Y\), given \(X\) = \(x\), is a linear function of _x_[66]. In a model based on simple or multiple linear regression, the dependent variable \(Y\) is a variable of a continuous nature. However, it can also be qualitative, where it is represented by two or more categories. In this case, the least squares optimization method does not provide a reasonable estimator. The ridge model solves a regression model where the loss function is the linear least square's function and regularization is given by the l2-norm and has built-in support for multi-variate regression [67]. Therefore, logistic regression offers a more appropriate approximation,  
Figure 5: Decision Tree Classification Metrics.  
allowing the use of regression models to calculate or predict the probability of a given event [68].  
The Logistic Regression algorithm was implemented using [PERSON]'s LogisticRegression. To define the parameters of the algorithm, GridSearchCV was used, which was also used for the same purpose in the ANNs and Decision Tree. Three parameters were chosen to be tested with GridSearchCV, namely: class_weight, max_iter and solver. The class_weight is a parameter used to work with databases that are unbalanced, which is the case in this work. The max_iter is basically the number of \"epochs\", like the parameter defined in the ANNs. The solver is basically the algorithm to be used for optimization.  
After running GridSearchCV, the best parameters for the Logistic Regression were generated, being class_weight set to \"None\", max_iter set to 100 and solver set to \"newtoncg\". After setting the Logistic Regression settings, the tests were performed using the 2014-2019 database. The Accuracy obtained with the model was 64%. Data classified as \"1\" obtained an accuracy of 0.67, recall of 0.89 and F1-score of 0.76. However, data classified as \"0\" obtained an accuracy of 0.41, recall of 0.14, and F1-score of 0.21. After performing the training and testing using the 2014-2019 database, training and testing was performed using the 1998-2019 database, which obtained the classification metrics shown in Figure 7.  
In the metrics presented in Figure 7, one can see that the Accuracy obtained with the model was 66%. Data classified as \"1\" obtained an accuracy of 0.69, recall of 0.67 and F1-score of 0.68. Data classified as \"0\" obtained an accuracy of 0.62, recall of 0.64 and F1-score  
Figure 6: Decision Tree Confusion Matrix.  
Figure 7: Logistic Regression Classification Metrics.  
of 0.63. Similarly to the ANNs and Decision Tree, when performing the comparison of the metrics obtained with the 1998-2019 database, compared to those obtained with the 2014-2019 database, the improvement in the metrics of the data classified as \"0\" is noted. Figure 8 shows the Confusion Matrix obtained with the Decision Tree.  
In the Confusion Matrix presented in Figure 8, it is shown that the data labeled as \"1\" are 67% correctly classified. The data labeled as \"0\", on the other hand, have a 64% correct classification.