# Making Sense of Machine Learning: A Review of Interpretation Techniques and Their Applications
## 2 Unveiling the Evolution of AI: From Symbol Manipulation to Deep Learning Networks
### From Brain Cell Models to Deep Learning Networks  
The development of AI has not been a linear climb up a ladder of success. There have been several \"AI winters\" when interest has faded. The first occurred in the 1970s due to the failure to meet the unrealistic expectations that had been raised for the field. The second AI winter occurred in the late 1980s and early 1990s, and was caused in part by the fact that symbolic manipulation systems, which were based on the Lisp programming language, were not able to handle the complexity of real-world problems. These systems were limited by their reliance on handcrafted rules and lack of ability to learn from data.  
Parallel to the symbolic computational stream, there was a stream of research based on computation models of brain cells and a model of learning (Hebbian learning). In 1943 [PERSON] and [PERSON] introduced the \"perceptron\", an idealized representation of a neuron cell that sums its inputs and gives an output signal when the sum exceeds a threshold. The first implementation was a machine built by [PERSON] in 1958 at the Cornell Aeronautical Laboratory.  
In 1969 [PERSON] and [PERSON] showed, in their book _Perceptrons_, that it was impossible for these classes of network to learn an XOR function, leading to an AI winter for the computational approach. However two advances (rediscovered several times) overcame this limitation leading to the \"multilayer perceptron\" (MLP) models (variously called neural networks and deep learning). The MLP model consists of layers of \"neurons\" fully connected to the neurons in the next layer, with the possibility of some connections skipping layers.  
These overcame previous constraints through the use of continuous activation functions, instead of the perceptrons step function, and the \"backpropagation\" of error terms through the network. Backpropogation can be seen to be the chain rule for differentiation, coupled with the fact that the needed information is always available at each neuron via the established connections. [PERSON]'s theorem [PERSON] (1969) shows the universal function approximation capability of MLPs, establishing their significance as general regression and classification tools.  
The MLP model was gradually expanded to architectures aimed at particular problem areas. The convolutional neural network (CNN), for example, was particularly targeted at image problems. There has been a transformative impact of Deep Learning (DL) in text, speech, and image applications. We can now add protein folding to that list. AlphaFold can predict the structure of a protein to an accuracy comparable with the results of laborious experimental determinations and is considered a correct solution that scientists can rely on with some confidence [PERSON] et al. (2014).  
The evolution depicted in this section embodies AI's transformative narrative--from its rule-driven inception to the intricate landscapes of DL. While early AI systems thrived on explicit rules, the difficulty of interpretation for contemporary AI raises multifaceted ethical and practical concerns. In response, techniques such as model distillation, feature visualization, and attention mechanisms are being explored as promising avenues for enhancing transparency and reliability. By embracing these methods, AI practitioners may usher in a new era of systems that inspire trust and understanding, bridging the gap between human users and the intricacies of machine intelligence.