Kaggle competitions are diverse but, for the most part, they deal with one of the following problems:
Classification;
Regression;
Computer vision;
Image processing;
Natural language processing.
The choice of the algorithm depends on the problem that you’re dealing with. It is tempting to solve everything by using only neural networks and deep learning, and quite often they do guarantee good results. But not all the time. Sometimes sticking to ANNs is just inefficient. So don’t be afraid of simple solutions to simple problems and choose a machine learning technique wisely.