## 3 Methodology
The use of semantic-based similarity measures would not be feasible for our system for the same reasons described before. Rather than comparing documents (news reports) with each other, our system compares documents with the centroid vectors representing each cluster. This same approach was followed in [4, 7, 17, 23]. The cluster centroid is represented by an index of the stemmed versions of all the terms (excluding  
Figure 1: The Categorisation Process  
stop-words) that appear in those documents which are members of that cluster. The weight of each term within the cluster centroid index is set to be the average weight of that term within the documents in that cluster. More specifically:  
\[w_{t,c}=\frac{\sum_{d\in D_{c}}(w_{t,d})}{|D_{c}|}\]  
where \(w_{t,c}\) refers to the weight of term \(t\) within cluster \(c\), \(D_{c}\) refers to the collection of documents in cluster \(c\), and \(w_{t,d}\) refers to the weight of term \(t\) within document \(d\). In Section 2, we described different clustering techniques, and how these techniques can be divided into Hierarchical Clustering and Partitional Clustering techniques. Since we do not require hierarchies, we did not consider the use of Hierarchical Clustering approaches since they have been reported to be slower than Partitional Clustering approaches [23]. Our clustering component is derived from the k-means Clustering algorithm, which is also used in [7,12,15,16,26], but adapted to work in an incremental environment. The k-means variation used is similar to the ones described in [14,15,20]. In this approach, each incoming document is compared to existing clusters, and if the cluster-document similarity is above a pre-defined threshold (in our setup, this is set to 0.4), the document is placed within that cluster. In our system, each news report is placed within the first encountered cluster with which it has a similarity that is higher than the pre-defined threshold. If no cluster is found to have a similarity with the document that exceeds the similarity threshold, a new cluster is created with that document as its first member. This is different from the usual approach found in literature where a document is clustered with the cluster with which it has the highest similarity. In our environment, the event-clusters being created are highly specific--they are quite far apart in the vector space. Therefore, generally, each document has a relatively low similarity with unrelated clusters. We also performed some tests to compare the results between our categorisation system and a system where the document is clustered with the most similar threshold, and have found that the difference in results is negligible. Since our system operates in an incremental environment, the terms' document occurrence frequencies of the terms are updated continuously. According to [PERSON] [16], cluster reorganisation would eventually be needed in such incremental environments. In our system, the news reports are clustered into event clusters. Since events are usually quite distinct from each other, the clusters should be quite distinct from each other, and cluster reorganisation would not be necessary. During the operation of our clustering system, news reports are being continuously received and clustered. This means that the amount of clusters is constantly growing. For example, in the operational setup of our clustering system that downloads news reports from 9 different news sources, there were 1369 news report clusters created during July 2012, and 10,562 news report clusters were created during 6 months of operation (mid-February till mid August 2012). When a news report describing a new event is received, it is compared with the existing clusters, and when no \"similar\" cluster is found, it is clustered on its own in a new clusters.