Overfitting is a problem that arises in Machine Learning models when the model is trained too well and learns the details and noise in the training data set instead of the true underlying patterns. This causes the model to be unable to generalize to unseen data and will not be able to accurately predict. To avoid overfitting, one should use Cross-Validation and create an additional holdout set. This holdout set should be 10% of the original dataset and is used to validate the model's performance. Additionally, it is important to compare the distributions of the train and test sets to ensure that they do not differ drastically.