Determining Topics in Link Traversals through Graph Based Association Modeling

Information networks have long been a technique to understand how a subject moves through a complex maze of information. May it be a user on a computer, a packet in a network, or hub on a street, these networks provide us with valuable information about where a subject is going and why it is going there. In this paper we begin at looking at two such networks. One is a network of citation graphs taken from the high energy physics citations from ArXiv. The second dataset involves a novel game, Wikispeedia in which a user is given a starting and ending node and has to find the shortest path between the two. We compare and contrast these two networks and provide an indepth analysis of both. We conclude that one of the datasets are more conducive to a predictive task in which we can begin to learn about human understanding. The directed path graph in the WIkispeedia dataset paths lends itself well to tree structures we explore two distinct tree based models and assess the strengths and weakness of these models. In addition we perform a brief runtime analysis and provide ways to optimize these models for large datasets. While it is important to look at the data and find correlations and relationships, we show that it is just as important to understand the story the raw data tell us and set expectations about what can and can’t be assumed. Correlation doesn’t imply causation and by the end of the paper we begin to see why this is the case. 1. DATASET SELECTION The project began with the evaluation of two separate network based datasets. Here we describe the two networks that we evaluated for the project and what motivated our selection of one over the other. 1.1 ArXiv Citation Graph ArXiv HEP-TH is a citation graph based on papers from high energy physical theory. The data was gathered from the e-print website arXiv and spans the dates from January 1993 to April 2003, a total of 124 months[6][4]. The dataset namely contains a list of 352542 edges and 27770 nodes in the format x y indicating that paper x cited paper y. In addition for each paper we are provided with the time the paper was submitted to arvix. The dataset also contains meta information about the paper which includes its citation index, the author, date, email, and a brief summary of the paper. Our preliminary analysis of the dataset began with looking at graph of complete data set, Fig.1. There is a large community in the center and some small communities around. First of all we wanted to make sure if the center community is a single large community. We wanted to find the importance of papers which resulted to this community, was it seminal, and did it touch many physics sub fields? Moreover, we wanted to find out more about those smaller communities and sub-areas in high energy physics such as particle physics, quatam field theory, or unparticle physics. In Fig.2 edge of the graph is enlarged which shows some of these communities. Figure 1: graph of complete data set Figure 2: an edge of the graph of complete data set 1.1.1 Clique Analysis We utilized a clique percolation method by finding cliques of 3 and 4 nodes. After extracting these cliques, we found out that there are 1480565 cliques with 3 nodes and 4129820 cliques with four nodes and the computation is order of n with considerable comparison and computation in each iteration. So it was required to select some part of the dataset for our experiment which could be ran in our computers with the limited time. We selected first 60000 edges of the dataset. To make sure our results are reasonable we sketched degree count of both graphs which are shown as Fig.3. Obviously these selection of data resulted in more nodes with small degree; however, still there are several nodes with high Figure 3: Degree count for full set and selected subset degrees and there are only 70 nodes in the full data set with degree of higher than 300. Hence if we separate the nodes which got out of their community due to these reduction of data set, we can comprehend the results. Next we ran the community detection algorithm and there were 394 communities of size 4, which meant they did not have any common 3 node clique with other communities. We discarded these nodes since they must be result of subsetting or they may be recent papers which were not cited in this data set. For the rest of the nodes number of communities based on their size is depicted as Fig.4. Figure 4: Number of communities by their size We categorized these sizes into 3 classes, very small communities having at most 10 nodes, moderate communities having less than 100 nodes and large communities having more than 100 nodes. As expected there was only one community with high number of nodes, 3235, among total nodes of 8247. There were 485 nodes in the moderate class and the rest in the small one. The interesting point of this analysis is for the moderate class, they must be a subcategory which was explored completely or left due to lack of technology or facilities, or paper from a not well known research center which was cited only locally. Our goal was by using the available metadata and find these moderate fields and predict whether a paper would be highly cited or not. 1.1. 2 Limitations of Dataset Since this data set is for citation, the newer papers would not have been cited and hence would be out of communities. We discarded these nodes by ignoring small communities. Moreover, it had metadata for each paper, including author, email, date of publication, and abstraction. However, it did not have subcategory which would be highly useful for us for the predictive task that we wanted to look. We then took another network graph in the same vein as this one to compare different kinds of direct path graphs. Figure 5: Graph of a complete path