# Causality Mining in Natural Languages Using Machine and Deep Learning Techniques: A Survey
## 2 Machine Learning Techniques  
Machine learning approaches are commonly divided into supervised learning (SL) and unsupervised learning (USL). The SL is based on labeled data, which contains some valuable information for the model. Text classification/mining is a common task in SL and is most frequently used in CM. Though, manually labeling the data is more cost-effective and time-consuming. Therefore, the absence of enough labeled data forms the major bottleneck to SL. On the contrary, USL mines the key feature knowledge from unlabeled data, making it much easier to gain training data. Though, the discovery performance of USL techniques is usually lower as compared to SL. Taxonomy of the most common shallow ML and DL algorithms used in CM is presented in Figure 2.  
ML algorithms help progress the learning performance, simplify the learning process, and increase the possibilities of diverse applications. However, since the 80s, most studies relied on finding explicitly marked causality and cause-effect event pairs in domain-specific corpora, annotated manually by ruled-based/non-statistical techniques. Hence, at the beginning of the 2000s, there was sudden wholesale transformation. The paradigm of manual CM is shifted to ML approaches, which replaced the non-statistical approaches or enhanced with automatic feature engineering. Over time, researchers progressively began to account for implicit, heterogeneous, and ambiguous constructs through careful feature extraction by using a massive amount of textual labeled and domain-independent datasets to automatically extract implicit patterns in the text, demonstrating that ML approaches could potentially perform superior to purely linguistics-based approaches. In this section, a few distinctive ML techniques and frameworks are discussed. Table 1 summarizes the primary approaches, contributions, and limitations of linguistics-based and simple cue-pattern based ML approaches for CM. Similarly, Figure 4 represent the processing levels of shallow ML techniques, which consist of different parts, including Target Data Sources, Pre-Processing, Manual Features Designing and Extraction ((Principal Component Analysis (PCA), PPCA (Probabilistic Principle Component Analyzers), ICA (Independent Component Analysis), and GAM (generalized additive models)), Shallow Machine Learning Networks (Supervised Learning, Unsupervised Learning), Training and Testing, Prediction, and Performance Evaluation.  
\begin{table}
\begin{tabular}{c c c c c c c} \hline
**SNo** & **Reference** & **Description** & **Pattern/Structure** & **Applications** & **Data Corpus** & **Languages** & **Limitations** \\ \hline \multirow{3}{*}{1.} & \multirow{3}{*}{[7]} & Improved version of C4.5 decision tree is used [44]. & A pattern of causative verbs NP\({}_{1}\)-verb-NP\({}_{2}\) are used. & Question Answering. & Domain-independent text. LATIMES section of the TREC 9 text group. & \multirow{3}{*}{English} & \multirow{3}{*}{\
\begin{tabular}{c} Only considered \\ marked and explicit cues. \\ \end{tabular} } \\  & & & & & & \\ \hline \multirow{3}{*}{3.} & \multirow{3}{*}{[30]} & Decision trees are used over POS-tagged data, and WordNet is used for mining semantic relations. & WordNet and POS-tagging features based features & Knowledge acquisition for decision making. & SemEval 2010 Task \# (7954 instances for training and 2707 for testing) & \multirow{3}{*}{
\begin{tabular}{c} Cost much \\ more time in \\ feature \\ extraction. \\ \end{tabular} } \\  & & & & & \\ \hline \multirow{3}{*}{4.} & \multirow{3}{*}{[45]} & Syntactic parser for NP\({}_{1}\)-verb-NP\({}_{2}\) relation and WordNet knowledge base are used. & & & & Lack of ambiguity resolution and use of small dataset. \\ \cline{1-1} \cline{3-3} \cline{5-6} \end{tabular}
\end{table}
Table 1: Summary of linguistics and Cue phrase-based ML approaches.