—Machine learning has grown in popularity and effectiveness over the last decade. It has become possible to solve complex problems, especially in artiﬁcial intelligence, due to the effectiveness of deep neural networks. While numerous books and countless papers have been written on deep learning, new researchers want to understand the ﬁeld’s history, current trends and envision future possibilities. This review paper will summarise the recorded work that resulted in such success and address patterns and prospects. Index Terms —Deep Learning, Multitask Learning, Self- supervised learning, Contrastive Learning, Meta-Learning, Ader- serial Learning. I. I NTRODUCTION Deep learning is an umbrella term for a wide variety of machine learning techniques that learn from large amounts of data. The deep learning algorithms are essentially artiﬁcial neural networks that learn from data over and over again, each time ﬁne-tuning the task a little more. We refer to “deep learning” because neural networks have multiple layers that offer complex learning. In “deep learning,” the word “deep” refers to the number of layers through which the data is processed. Deep learning allows machines to make the best decision possible for a wide variety of complex, diverse, unstructured and interconnected data sets. We present some of the formal deﬁnitions of deep learning: Deﬁnition 1: According to [1], “Deep learning is about learning feature hierarchies with features from higher levels of the hierarchy formed by the composition of lower level features.” Deep learning (also known as deep structured learn- ing or hierarchical learning) is learning data representations, as opposed to task-speciﬁc algorithms. Learning can be super- vised, semi-supervised or unsupervised. Deﬁnition 2: It’s deep if it has more than one stage of non- linear feature transformation [2]. Deﬁnition 3: Deep Learning builds a system by assembling parameterised modules into a (possibly dynamic) computation graph and training it to perform a task by optimising the parameters using a gradient-based method [3]. Deep learning is considered a subset of machine learning and artiﬁcial intelligence. The Venn diagram in Figure 1 is showing the relationships between deep learning and other approaches in AI. Due to its superior performance and accuracy, deep learning has become increasingly popular. Figure 2 shows a Google Trend graph that displays an increasing interest over timein the term “Deep Learning”. The Turing Award, presented for excellence in artiﬁcial intelligence research, was awarded to three of deep learning’s most inﬂuential architects, Yann LeCun, Geoffrey Hinton, and Yoshua Bengio. The members of this trio and many of their colleagues have developed the algorithms, systems, and techniques behind the recent AI- fueled wave of products and services, which they expect will dominate the future. This review paper is organised as follows: In section 2, we provide a development panorama and timeline of deep learning in two episodes of its ﬁrst emergence as artiﬁcial neural networks and reemergence in the revolution of deep learning. Section 3 presents a discussion and future research directions followed by a conclusion and