The breakthrough came in 2012 with the ImageNet competition, where Ilya, along with Jeff Hinton and Alex Krizhevsky, created AlexNet, showcasing the potential of deep learning and leading to a revolution in the field.