# Neural Embeddings for the Elicitation of Jurisprudence Principles: The Case of Arabic Legal Texts
## 4 Methodology
### Experiments
#### 4.3.3 Experiment 3: Using Paragraph-Vector with Negative Sampling  
Based on the results of the second experiment, which were encouraging for the PV-DBOW, we used the same model for our third experiment but with new values for the hyperparameters. The results changed significantly when the following parameters were changed, as shown in Table 2: window size, negative sample, dibow_words, and sample. For our task, we needed the model to learn the word embeddings only from those context words that could be indicative of a JP, while excluding domain-relevant words that are common or most frequent in Sharia legal texts (so-called noise words).  
Negative sampling is a technique that allowed us to achieve this by having each training sample modify only a small percentage of the weights rather than all of the weights. This was accomplished by setting common or most frequent words as negative (those for which we want the network to deliver an output of 0). We assigned the value to 20 to select the common or frequent words as negative and only update the weights for the positive words that were relevant to the five major JPs.  
The sample parameter value was also changed to represent the probability of keeping the word in the vocabulary: smaller values for the sample mean that the most frequent words are less likely to be retained. In our task, the most frequent words were general words that may occur in any Arabic legal text (Sharia text). Thus, we increased this value to ensure that only rare words indicative of the JPs remained. Finally, the PV-DBOW is not concerned with the word ordering and ignores the context words in the input. Thus, the word vectors were activated to be trained simultaneously with DBOW document vector. Table 2 presents the values of the parameters used in this experiment.  
To evaluate how well the model was capable of eliciting the JP associated with a specific JC, two JCs were chosen randomly for each JP. Using the model, the inference vector was generated for each of the two JCs and the 30 most similar JCs were analyzed by domain experts to evaluate their relevance to the JP (Figure 1 shows a diagram for the evaluation process).  
\begin{table}
\begin{tabular}{l c} \hline \hline
**Hyperparameter Name and Description** & **Setting** \\ \hline dm: Distributed bag-of-words (PV-DBOW) is employed & 0 \\ \hline vector\_size: Dimensionality of the feature vectors & 300 \\ \hline Negative: “Noise words” should be drawn & 20 \\ \hline min\_count: Ignores all words with a total frequency \textless{}5 & 5 \\ \hline epochs: Number of iterations (epochs) over the corpus & 30 \\ \hline dibow\_words: Trains word vectors simultaneously with DBOW document & 1 \\ vector training & \\ \hline window: The word context & 3 \\ \hline sample: The threshold for configuring which higher-frequency words are & 0 \\ randomly downsampled & 0 \\ \hline hs: Negative sampling will be used & 0 \\ \hline \hline \end{tabular}
\end{table}
Table 2: Hyperparameter Values.  
The results depicted in Figure 2 show that the model was able to find the JCs most similar to the input JC regardless of the legal topic of the JC. Thus, the similarity produced was based on the shared JP. Interestingly, the model can detect the relevance of JCs based on words that are indicative of a JP.