## 3 Methods
### Statistical Content Analysis  
Statistical topic models based on Latent Dirichlet Allocation (LDA) ([PERSON] _et al._ 2003 [15]) have been applied to analyze the content of textual logs in order to identify the various topics/concepts that exist in them at different levels of thematic abstraction. LDA represents each topic as a distribution over words  
Figure 2: Flowchart of the decision engine. and allows for mixed memberships of documents to several topics. LDA is very effective in identifying the various thematically coherent topics that exist in document collections and producing a soft clustering solution of the documents. In addition, LDA's iterative nature allows it to scale to document collections containing millions of documents ([PERSON] _et al_. 2007 [16]). ```
Input: Documents D, number of topics \(n\), partition parameter \(m\), anomaly threshold \(k\) Output: Set of anomalous topics \(B\)=ComputeBagOfWords(\(D\)) {Computes the statistical frequencies of word co-occurrences in the corpora, Section 3.1} \(T\)=LDA(\(B\),\(n\)) {Cluster bag-of-words into \(n\) topics, Section 3.1} \(T1\)=Rank(\(T\)) {Rank topics based on document co-occurrence, Section 3.1} \(Test\) = Partition(\(T1\),\(m\)) {Partition into typical and test topic sets, Section 3.3}  FindContext(\(Test\)) {Find context from semantic measures, Section 3.2} \(R\)=Decision(\(Test\),\(k\)) {Pick \(k\) lowest context score topics, Section 3.3} Output \(R\)
```  
**Algorithm 1** The Semantic Anomaly Detection Algorithm  
LDA's generative model is illustrated in Figure 3. Topics are assumed to be distributions over words. A Dirichlet distribution with parameter \(\alpha\) is assumed as the prior over all documents. From Dirichlet(\(\alpha\)) each document is drawn as a multinomial distribution \(\theta\). The dimensionality of this distribution is determined by the number of topics. For each word within a document we draw a topic from this multinomial distribution. We subsequently go to the corresponding topic distribution and draw a word from it. Ultimately a document is assumed to be a mixture of topic distributions. In training the model the objective is to learn the topic distributions along with the underlying multinomial distributions \(\theta\) representing the documents. In LDA the resulting topics can be represented by the most likely words occurring in them. For example, Figure 4 shows some of the most important terms identified from the top four topics from the Enron email dataset, which can be used as a summary of the identified topic. Figure 3: LDA’s generative model illustrated with 3 topics and four words. The documents in the text logs were converted to the bag of words format after tokenization, noise removal and removal of stop words. Then the following two steps were carried out:  
1. Clustering: The text log was divided into k-Clusters using the LDA model. The values of k and other parameters (\(\theta\), \(\alpha\), _etc._) were decided based on the size of the dataset and our understanding of the nature of the dataset. The top 10 most likely words were extracted as a representative summary of each topic. 2. Ranking: The topics were ordered based on their co-usage in documents. LDA assumes every document to have been created by a mixture of topic distributions. We obtain an ordering of topics based on the assumption that topics that appear together are similar to each other and should have a low relative difference in their rankings. First, the topic distributions for each topic was calculated. Then, for each pair of topics _i.e._, \((P,Q)\), the symmetrized KL divergence between topic distributions \(P\) and \(Q\) was calculated. Equation 1 shows the divergence measure between the probability distributions \(P\) and \(Q\). Equation 2 shows the symmetrized KL divergence measure (henceforth \(SD\)), which has the properties of being symmetric and non-negative (Equations 3 and 4 ). The symmetrized KL divergence was subjected to dimensionality reduction and the first  
Figure 4: Results from Enron dataset.