# Toward a Multi-Column Knowledge-Oriented Neural Network for Web Corpus Causality Mining
###### Abstract
By limiting \(f_{j}\) and \(w_{i+j-1}\) (word embedding) to unit vectors, the resultant value of \(m_{i}\) becomes the cosine similarity between \(f\) and \(w_{kgram}\). The goal of cosine similarity in feature maps is to achieve equal importance of \"wf\" with different lengths by creating the same scale for all convolutional window sizes, while the conventional method will obtain a higher number for the wider window size. The most specific feature map is generated using max-pooling for each filter to further aggregate the convolutional results. The pooling procedure for each feature map is shown in Equation (3). The largest cosine similarity provides strong cues for the presence of cue phrases and keywords in the text, which is why the feature map maximum value is obtained. \[p=max\big{\{}m_{1},m_{2},m_{3},\ldots,m_{n_{1}-k+1}\big{\}} \tag{3}\]

About 900 \"wf\" are produced by Algorithm 1, which is thought to be highly dimensional and has limited training data. These \"wf\" for causality mining provide a large number of features, some of which may be redundant and irrelevant. In order to enhance the performance of the model, we used \"wf\" clustering and selection [26]. ### Segments and Connective Level Processing

The proposed model presents a novel method to mine causal relationships within a single sentence at the connective and segment level using 3 KCs and RN. The input connective L and segments BL and AL can be denoted as \(Z_{L}\in R_{I_{Lxd}}\), \(Z_{BL}\in R_{J_{BLxd}}\), and \(Z_{AL}\in R_{J_{ALx,cl}}\) input format. Where, \(J_{BL}\), \(J_{AL}\), and \(J_{L}\) are the token lengths in each segment and connective. Each channel is responsible for parsing \(Z_{BL}\), \(Z_{L}\), and \(Z_{AL}\) into a set of objects. Unlike [26,58], MCKN convolves them through a 1D convolutional layer into different window sizes for \"\(k\)\" feature maps of size \(J_{BL\times 1}\), \(J_{L\times 1}\), and \(J_{AL\times 1}\), where \"\(k\)\" is the sum of \"wf\". After convolution, each segment's and connective feature maps are rescaled into a \(k\)-dimensional vector via a max-pooling layer, and dimensionality reduction is then implemented by further reducing the dimensionality. Finally, we create a set of objects in Equation (4). \[\left\{o^{BL},o^{L},o^{AL}\right\}\in R^{k} \tag{4}\]

In addition, because RN works with objects, we created four object pairs in Equation (5). \[ObjectPair= \begin{vmatrix}o^{BL};o^{L}\\ o^{L};o^{AL}\\ o^{BL};o^{AL}\\ o^{AL};o^{BL}\end{vmatrix}  \tag{5}\]

The \"\(;\)\" is now an operator that concatenates object feature vectors. We can simplify it using the notation in Equation (6), where '\(*\)' represents a pair-wise operation. For causality candidates, \(BL*L\) and \(L*AL\) determine the relationship between the cause-effect event and \(L\), while \(BL*AL\) and \(AL*BL\) infer the direction of causality. \[o^{BL}*L=\left[o^{BL};o^{L}\right]o^{L}*AL=\left[o^{L};o^{AL}\right]o^{BL}*AL= \left[o^{BL};o^{AL}\right]o^{AL}*BL=\left[o^{AL};o^{BL}\right] \tag{6}\]

As a result, the simplified form of the object is represented by the Equation (7). \[Op= \begin{vmatrix}o^{BL}*\mathrm{L}\\ o^{L}*\mathrm{AL}\\ o^{BL}*\mathrm{AL}\\ o^{AL}*\mathrm{BL}\end{vmatrix}  \tag{7}\]

Here \(Op\in R^{4}\times(2k+2 dg)\) is the matrix representation of object pairs. More generally speaking, by changing the architecture in a mathematical formulation, we were able to derive the final representation (Final_rep \(\in\) R\({}^{\mathrm{d}dg}\)) at the segment and connective levels in Equation (8). \[Final\_rep=f_{\hat{\Psi}}\big{(}\sum g_{\theta}(Op)\big{)} \tag{8}\]

At the segments and connective level, MCKN transforms segments and connectives into object pairs and then integrates these object pairs for pair-wise inference to discover the relationship between segments and connectives. ### Causality Identification

The applied model discovers causality in each sentence by passing \"Final_rep\" to FFN.