## 4 Experiments
### Underlying Systems for Overlapping Clustering  
ICRT works as a post-processor for a clustering system so that performance of the system in issue-based clustering can be improved. We thus implement several types of systems that employ typical article similarity measures to conduct overlapping clustering (i.e., partitioning articles into _non-exclusive_ clusters, as an article may discuss multiple issues), and investigate how they can be improved by collaborating with ICRT. As summarized in Table 2, these underlying clustering systems work on two kinds of contents: (1) _references_ (citations) in each article, and (2) _title_ and _abstract_ of each article, which are publicly available and employed by many systems for the clustering of scientific articles (e.g., [26]). Two clustering systems that work on _references_ employ bibliographic coupling (_BC_, see Equation (1)) and _DescriptiveBC_ (see Equation (3)) as their similarity measures. As noted in Section 2.1, BC is a typical measure that performed well in the retrieval and clustering of scholarly articles [13; 16]. _DescriptiveBC_ employed titles of the references to improve BC [16]. The clustering systems that work on _titles_ and _abstracts_ (TA) of articles employ several kinds of inter-article similarity measures, including (1) PubMed-based similarity (_PMS_), (2) _OK_, and (3) those derived by topic modeling. The textual data is reprocessed by removing stop words (by a stop word list available at [32]) and mapping synonymous words to the same ID (by the MetaMap tool \begin{table}
\begin{tabular}{c c c c c c c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multicolumn{2}{c}{**\# Clusters Having X Articles**} & \multicolumn{2}{c}{**\# Articles Belonging to X Clusters**} & \multicolumn{2}{c}{**\# Articles Published in Year X**} \\ \cline{2-10}  & X = 2 & X = 3 & X \(\geq\) 4 & X = 1 & X = 2 & X \(\geq\) 3 & X \(\leq\) & X in & X \(\geq\) \\  & Articles & Articles & Articles & Clusters & Clusters & Clusters & **2005** & **[2006,2010]** & **2011** \\ \hline (1) \textless{chemical}, & 6719 & 3140 & 3448 & 2669 & 1614 & 4167 & 1215 & 3233 & 4002 \\ gene\textgreater{} & (50.49\%) & (23.60\%) & (25.91\%) & (31.59\%) & (19.10\%) & (49.31\%) & (14.38\%) & (38.26\%) & (47.36\%) \\ \hline (2) \textless{chemical}, & 1338 & 479 & 634 & 3176 & 1235 & 813 & 1840 & 1919 & 1465 \\ disease\textgreater{} & (54.59\%) & (19.54\%) & (25.87\%) & (60.80\%) & (23.64\%) & (15.56\%) & (35.22\%) & (36.73\%) & (28.05\%) \\ \hline (3) \textless{gene}, & 414 & 94 & 63 & 902 & 188 & \multirow{2}{*}{60 (5.22\%)} & 170 & 446 & 534 \\ disease\textgreater{} & (72.50\%) & (16.46\%) & (11.04\%) & (78.43\%) & (16.35\%) & (16.35\%) & (14.78\%) & (38.78\%) & (46.44\%) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Statistical characteristics of the three datasets in the experiment. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Similarity Measures**} & \multicolumn{2}{c}{**Content**} & \multicolumn{2}{c}{**Overlapping Clustering**} \\ \cline{2-5}  & \multicolumn{2}{c}{**References**} & \multicolumn{2}{c}{**Tites \& Abstracts**} & \multicolumn{1}{c}{**Fuzzy Clustering**} & \multicolumn{1}{c}{**Thresholding**} \\ \hline (1) Bibliographic Coupling: _BC_ & \(\surd\) & & & \(\surd\) & \(\surd\) \\ (2) _DescriptiveBC_ & \(\surd\) & & & \(\surd\) & \(\surd\) \\ (3) _PubMed-based similarity: PMS_ & & \(\surd\) & \(\surd\) & \(\surd\) \\ (4-1) OK: OK-_TA_ & & \(\surd\) & \(\surd\) & \(\surd\) \\ (4-2) OK: _OK-RT_ & \(\surd\) & & \(\surd\) & \(\surd\) \\ (5-1) Topic Modeling: _TM-TA_ & \(\surd\) & & & \(\surd\) \\ (5-2) Topic Modeling: _TM-RT_ & \(\surd\) & & & \(\surd\) \\ (5-3) Topic Modeling: _TM-TA-FC_ & & \(\surd\) & \(\surd\) & \(\surd\) \\ (5-4) Topic Modeling: _TM-RT-FC_ & \(\surd\) & & \(\surd\) & \(\surd\) \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Systems for overlapping clustering with various inter-article similarity measures. available at [33]).