# Data Analytics for Optimizing and Predicting Employee Performance
## 3 Methods and Results
### Data Modeling
#### 3.4.3 Optimizing the Models  
The second step of our data modeling is to optimize the results previously obtained as well, since this can be achieved for each of our algorithms. Based on related works on these kinds of methods, we will discover in the next chapter of the current paper how a hyperparameter can achieve a better performance in general, what kind of methods exist in order to achieve this, and what are the solutions chosen to apply to our research. Hyperparameter tuning is an important factor for machine learning algorithms and their associated processes, and it is also recognized as the best practice for maximizing the results of a successful implementation. This method varies among the different machine learning algorithms due to the variety of parameters existing for each of them, such as discrete, continuous, or even categorical parameters. Nonetheless, this is seen as an expensive implementation, as it also engages the evaluation of multiple combinations and demands a remarkable number of computational resources. Generally, building a productive model is time-consuming, which leads to calls for distinguishing the best model obtained using hyperparameter tuning [40].  
Over the last decades, grid search has been the classical approach used for parameter optimization when using machine learning techniques, which introduces a comprehensive search throughout a predefined subset of an algorithm's hyperparameter space used for the learning process. While there are also proposed alternative methods, such as random search, Bayesian optimization or even gradient optimization, grid search remains the most popular one, due to its parallelization capability, lack of difficulty in terms of running it, and flexibility in low-dimensional spaces. Regardless of its simplicity, this method is still expensive in terms of the computational resources needed, mainly when using multiple hyperparameters at different levels, fact that also highlights the exponential growth of the computing cost [41; 42].  
On another note, there are some other types of optimization in terms of parameters that do not work in the standard way described above. One of these techniques is known as Optuna, and it is a Python library that is dedicated to working on hyperparameter optimization, with origins in Preferred Networks, which is a Japanese organization. This open-source instrument provides a complex and automated way to search the optimal hyperparameters by applying fine-tuning steps to an objective function. In addition, this comes with a very user-friendly design and demonstrates adaptability as well over-diversified machine learning frameworks.  
Regarding the way Optuna works, it firstly defines the search space by having users illustrate the range and type of hyperparameters that are going to be used for optimization. Secondly, it also defines the object function that will be in charge of evaluating the model performance, considering a set of hyperparameters. Lastly, Optuna undergoes optimization, by handling iterative evaluations, and updates of the probabilistic model, while the search space will progressively narrow down until adequate parameters are identified [43].  
Lastly, since this is a relatively new approach considered for this type of optimization, we can underline some advantages of these methods, such as efficiency, improvedperformances, automation, and reproducibility. While other standard methods are already well-known and generally applied in similar research, Optuna can actually reduce the time and resources needed for this process; it explores, in a systematic way, the hyperparameter space, which increases the chances of discovering an optimal structure and also provides data mining analysts and developers with the opportunity to focus more on the model, rather than on manually tuning it, since this will mostly be undertaken automatically.