# Linked Data Triples Enhance Document Relevance Classification
## 6 Experimental Description
### Experiments on Energy Hub
#### 6.2.1 Implementation  
The LDA topic model is generated from the training data. The probability topics distributions of each document in the training set, and important keywords of each topic are identified. This pre-trained topic model is then applied on the validation and test sets. The named entities are parsed through spaCy, and the Topic-Entity triples are constructed (as described in Figure 2).  
Raw text from each document is split into list of sentences with '.' delimiter. These sentences are encoded using a pre-trained InferSent model, with a batch size of 64, 300-dimensional word embeddings, and 2048-dimensional Long-Short Term Memory (LSTM) encoding. Each sentence is represented as a 4096-dimensional embedded vector.  
Each entry from training, validation, and test sets contains document text, the LDA generated probability distribution of topics from the document, the named entities, the Topic-Entity triples, and vector representation of sentences. As the Energy-Hub data is biased with more \"relevant\" examples, down-sampling the collection to match the number of \"irrelevant\" articles makes the learning set more balanced, so additional irrelevant news articles are sourced and added to the training set. This balancing results in a training set with more or less equal numbers of relevant/irrelevant articles to present to the learning algorithm. The balanced training set has 1382 documents, where 691 (exactly half) are relevant, and the remainder are irrelevant.  
Text attribute requires additional pre-processing, like stop word removal, filtering special characters, and case conversion. Stop-words are identified from the natural language tool kit ( NLTK ([[https://www.nltk.org/](https://www.nltk.org/)]([https://www.nltk.org/](https://www.nltk.org/)), accessed on 10 March 2021)) and removed from the text. Special characters and digits are filtered and the text folded to lower case to maintain consistency. Topics, entities, and triples are composed as a list of elements, which are later converted to text, like sentences.  
All models were built using the Keras ([[https://keras.io/](https://keras.io/)]([https://keras.io/](https://keras.io/)), accessed on 10 March 2021) API in Tensorflow ([[https://www.tensorflow.org/guide/keras](https://www.tensorflow.org/guide/keras)]([https://www.tensorflow.org/guide/keras](https://www.tensorflow.org/guide/keras)), accessed on 10 March 2021). The experiments were conducted in Google Colaboratory ([[https://colab.research.google](https://colab.research.google)]([https://colab.research.google](https://colab.research.google)).  
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Dataset** & **Training Samples \#** & **Test Samples \#** & **Average Length** & **\# Classes** \\ \hline Energy Hub & 2176 & 544 & 1574 & 2 \\ Reuters & 1828 & 457 & 140 & 2 \\
20 News Groups & 11,314 & 7532 & 284 & 20 \\ \hline \hline \end{tabular}
\end{table}
Table 1: Features of the three datasets used in these experiments. Energy-Hub is an in-house dataset, while Reuters and 20 News groups are widely used standard news classification datasets.  
com/, accessed on 10 March 2021) with 12,288 Gigabytes (GB) of RAM. The TensorFlow models were built on a Tesla P100 GPU with 16,280 MB memory, CUDA version 10.1, and driver version 418.67. The Adam optimizer with a learning rate of \(2\times 10^{-5}\) was found to be the best fit for the data. Sparse categorical cross-entropy is considered as the loss function with accuracy as the optimization metric. The same set of hyper-parameters are used for all models.  
GloVe and sentence embedding models are trained for 300 epochs with a batch size of 32 and early stopping using generated training and validation sets. BERT models are trained for 3 epochs with a batch size of 4.