# Toward a Multi-Column Knowledge-Oriented Neural Network for Web Corpus Causality Mining
###### Abstract
### Linguistics Background of Source Corpus

This part discusses the linguistic background of causality and the AltLexes ([[https://github.com/chridey/altlex](https://github.com/chridey/altlex)]([https://github.com/chridey/altlex](https://github.com/chridey/altlex)), accessed on 3 May 2021) dataset. About 12% of the Pine Discourse Tree Bank (PDTB) is labeled as causal, and around 26% is implicit [58]. In addition, there exists another type of implicit relation called \"AltLex\", which represents causality and is marked as an open and infinite class of causality. The generalization of \"AltLex\" is extended with an open class of markers [22]. Some examples in the \"AltLexes\" dataset are not present in the explicit relations of PDTB including ambiguous causal verbs, e.g., \"COVID-19 made many countries affected\" and partial prepositional phrases, e.g., \"He has made aircraft with the idea of a new deep neural technology\". In the first example, the term \"made\" has numerous meanings and is employed to express causation. However, in the second example, the causal relationship expression is not clear. According to our analysis, the parallel data constructed has 1164 causal connectives and about 7627 non-causal connections. Furthermore, their intersection has 155 types of connectives, which are hybrid. It shows their reliance on a causal set of 12.6%, and reliance on non-causal sets is 1.8% [22]. Several implicit and heterogeneous relationships are discovered as a result of the analysis. In this case, prior approaches have several demerits to making an expert system. \begin{table}
\begin{tabular}{p{56.9 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline
**References** & **Description** & **Features** & **Benefits** & **Drawbacks** \\ \hline
[50] & A novel model with an event context word extension mechanism & FFNN + BK & Targeted implicit causalities social media tweets & Results in info loss due to opinionated posts \\ \hline
[51] & TCDF model & Considered multi-attention-based CNN with a causal support stage & Centred on implicit observational time series data & Executes rather worse on short time series \\ \hline
[26] & A novel K-CNN for causality extraction & Combined DOC and KOC. & Focused on implicit causality & Over-fitting problem \\ \hline
[53] & SCITE with multi-head self-attention [30] & They formulate causality as a sequence tagging & ✓ & ✓ \\ \hline
[54] & A deep neural based MCDN approach & Using word and segment level information & Focused on implicit and ambiguous & They only looked at causality within sentences and ignored multi-level sentences \\ \hline
[56] & Generative approach & Using encoder–decoder and pointer networks to extract causality & Focused on implicit causality & Limited to financial dataset \\ \hline \end{tabular}
\end{table} 
Table 1: Cont. ### Input Sentence Representations

The input sentence (\(N\)) contains '\(n\prime\) tokens, \(N=\{n_{1},n_{2},\ldots,n_{i-1},n_{i}\}\). Where '\(n_{i}\)' is the filter token in the sentence at '\(i\)' position. Further, each sentence is formatted to L, AL, and BL. The purpose is to generate sentence level '\(y\)' prediction, where '\(y\)' is the input sentence label shown in Equation (1). For the parallel corpus feature in our model, we employ a pair

Figure 1: Applied MCKN architecture. This contains three columns using convolutional word filters: the first column processes the BL (e1) part of the input sentence, the second column deals with the L part of the input sentence, and the third column targets the AL (e2) part of the input sentence. of simple and English Wikipedia sentences, although it still only takes a single sentence as input [43]. \[y=\left\{ \begin{array}{ll}1,&\left|\text{{Causal sentence}}\\ 0,&\left|\text{{Not}}-\text{{Causal sentence}}\right.\end{array} \right. \tag{1}\]

Motivated by [30], each token/word in the input sentence can be denoted by summing the corresponding token embedding, position embedding, and segment embedding.