# Outlier Detection and Prediction in Evolving Communities
## 5 Evaluation Results
### Outlier Score Distribution  
In this set of experiments, we tried to verify that a node to which we assigned a high outlier score would be a member of hardly any communities. We used the traditional definition of an outlier as not belonging to any community and investigated whether our outlier score reflects this. The outlier score we propose is a metric of outlierness, or how much of an outlier a node is; it ranges from 0 for totally central nodes to to 1 for theoretically total outliers, respectively. The intuitive decisive threshold for outlier assignment lies at the halfway mark of this range; for example, a node with an outlier score less than 0.5 is closer to totally central than to a total outlier. This can be adjusted based on the context.  
Thus, to confirm the validity of the outlier score, we studied the distribution of outlier scores among nodes in total (Figure 3) and then separately for nodes that were members of at least one community (Figure 4) and for nodes that did not belong to any communities (Figure 5). Our results confirm that, based on the traditional definition, most outliers have high outlier scores and vice versa. Thus, we can conclude that our proposed outlier score was able to determine the community membership of each node based on its connectivity and its content.  
Figure 3: Outlier scores of all nodes: StackExchange (**left**) and MovieLens (**right**).  
Figure 2: Distribution of outlier scores for different weight values (\(w\)): StackExchange (**left**) and MovieLens (**right**).  
For StackExchange, Figure 3 (left) shows that most outlier scores are gathered around values near 0.50 or higher (81.30% of nodes have outlier scores over 0.40). This reflects a graph in which most nodes tend to be outliers, a usual phenomenon in forum networks, where a few users share the most interactions and the vast majority uses the interface only to ask a specific question when needed. In contrast, for MovieLens, Figure 3 (right) depicts more evenly distributed scores with a preferred direction towards smaller outlier scores, reflecting nodes that are less likely to be outliers. Indeed, 78.60% of nodes have an outlier score of 0.50 or less. This can be explained in part based on dataset construction, as we selected only those movie ratings which included content information (tags). It is obvious that users who are involved in commenting on a movie and not just rating it are likely to be more active in the network than other users.  
When studying community members for both datasets (Figure 4), we observed that most of the nodes have low outlier score values; thus, they are less likely to be assigned as outliers. Specifically, 71.90% of community nodes in StackExchange and 88.50% of nodes in MovieLens had outlier scores less than 0.50. On the contrary, in Figure 5 the outlier scores are quite high for most nodes that are not members of any community.  
However, our proposal for defining an outlier score characterizes a percentage of nodes as outliers or not in all cases, despite their degree of community membership. This is because the content of the network is taken into account. We argue that the definition of an outlier should not rely on structural criteria. A node which is a member of a community but with quite different labels assigned to it and probably fewer connections to other members can be considered to have outlying behavior. In the same way, a node with fewer connections but quite similar content with a specific community, e.g., two forum users that did not exchange answers or comments between them but used similar sets of labels in their interactions, should not be considered an outlier despite not being a member of it.