Techniques to Overcome Overfitting With Small Datasets Choose simple models. ... Remove outliers from data. ... Select relevant features. ... Combine several models. ... Rely on confidence intervals instead of point estimates. ... Extend the dataset. ... Apply transfer learning when possible. Aug 26, 2019