Validation and testing on a balanced dataset

## Practice full name

Perform validation and testing over a perfectly balanced dataset

## About

Once the model has been built and trained, it must be rigorously tested and validated according to specific techniques. This practice wants testing and validation activities to be carried out on a balanced dataset, therefore it may be necessary to execute data balancing activities where necessary. Having an unbalanced dataset could give unrealistic results during the testing and validation phases.

## What the crowd is saying

Is this practice best or bad?

In the opinion of the participants, this practice has a great tendency for which it can be considered as best.

Looking at more details, there is much more confidence in the answers that classify the practice as best.

## Involved topics

### What is validation?

Validation is a procedure where a trained model is assessed with a testing data set. Validation is done after model training and it intends to locate an ideal model with the best execution. Some validation techniques are: K-fold Cross-Validation Method, Random Subsampling Validation Method and Leave-One-Out Cross-Validation Method.

K-fold Cross-Validation is used to detect the overfitting or fluctuations in the training data that is selected and learned as concepts by the model. It repeats cross validation process many times with different splits of the sample data in to K-parts.

With Random Subsampling Validation data is randomly partitioned into dis-joint training and test sets multiple times means multiple sets of data are randomly chosen from the dataset and combined to form a test dataset while remaining data forms the training dataset.

With Leave-One-Out Cross-Validation all the data except one record is used for training and that one record is used later only for testing [1].

### What is testing?

Testing identifies explicitly which part of the code fails and provides a relatively coherent coverage measure (e.g., lines of code covered). It helps to assure quality, whether the software works according to requirements, and to identify defects and flaws during development and in production. Data Scientists and Machine Learning Engineers train models by feeding them with examples and setting parameters. The model training logic produces the behavior: this make it difficult to understand the reasons behind a model’s low performance, interpret the results, and assure that our model will work even when there is a change in the input data [2].

### What is a balanced dataset?

A balanced dataset is a dataset where each output class is represented by the same number of input samples. Balancing can be performed using a specific technique: the most popular are oversampling and undersampling [3].

Oversampling is a technique wherein the number of training samples of the minority class is increased so as to balance the number of samples of both majority and minority class. Undersampling refers to removing training samples from the data which belong to that class of the target feature that has the majority over the other class [4].

## Sources

1. What is Model Validation
2. How to Test Machine Learning Models
3. How to balance a dataset in Python
4. Imbalanced Data and How to Balance It

**Other useful links**

* Challenges of Testing Machine Learning Based Systems

## How to contribute

* You can use the "comments" section of the wiki. You can find it at the bottom of the article page that you want to edit. Suggest here your edits, the administrator will evaluate whether to accept them and report them on the page.

* You can send an email to *abatefrancesco98@gmail.com* setting as subject the page you want to edit.

Any interesting information will be reported on this page.