BACKGROUND
there is growing recognition that mirnas regulate various diseases and biological processes  as evidenced by the rapidly growing body of literature related to mirnas .
there are manually curated repositories such as mirbase  <cit>  and mir2disease  <cit>  that catalog mirnas in several organisms as well as summarize their associations with diseases and other biological processes.
however, it is generally accepted that manual curation is unable to keep up with the rapidly growing genomic information  <cit> .
for instance, mirbase has not been updated since  <dig> and mir2disease has not been updated since  <dig>  it is therefore imperative to devise automated methods that can keep pace with the functional information which is deposited in the biomedical literature with respect to mirnas.
information retrieval  is a key component of text mining  <cit> .
it consists of three types of models: set-theoretic , probabilistic, and algebraic .
documents in each case are retrieved based on boolean logic, probability of relevance to the query, and the degree of similarity to the query, respectively.
the concept of literature-based discovery was introduced by swanson  <cit>  and has since been extended to many different areas of research.
in the gene space, several approaches have focused on mining both explicit associations based on co-occurrence  <cit> , as well as implicit associations based on higher order co-occurrence and indirect relationships  <cit> .
several ir approaches have focused on mining mirna specific associations.
mircancer  <cit> , mirsel  <cit>  and mirtex  <cit>  use co-occurrence and sentence level natural language processing to automatically extract direct relationships from text between mirnas and genes or diseases.
while useful, these tools may miss mirna interactions where direct relationships were not explicitly stated.
in such cases, automated extraction of semantic relationships would be useful to associate genes and mirnas based on shared biological processes.
also, explicit relationships such as those based on co-occurrence count between mirnas and genes may be harder to prioritize if they have the exact same score.
in contrast, semantic associations that take into account other relationships could be useful for prioritization of mirna and gene associations  <cit> .
aside from exploring mirna to gene associations, semantic analysis could be useful for other research scenarios.
for example, investigators may want to prioritize candidate mirnas for specific diseases or phenotypes.
alternatively, investigators may want to understand the functional pathways shared between different mirnas.
to address these needs, we developed and evaluated an lsi based text mining approach.
previously, we applied lsi to extract functional relationships amongst genes  <cit>  as well as relationships between genes and transcription factors  <cit>  from medline abstracts.
lsi uses singular value decomposition   <cit> , which is a dimensionality reduction technique that decomposes the original term-by-document weighted frequency matrix into a new set of factor matrices that can be used to represent both terms and documents in lower-dimensional subspace.
previously, we demonstrated that lsi can extract both explicit  and implicit  semantic relationships amongst genes.
in addition, lsi allows genes to be prioritized based on keyword queries as well as gene-abstract queries with better accuracy than term co-occurrence methods  <cit> .
here, we applied this approach to mirnas and demonstrate its utility to prioritize, cluster and functionally annotate mirnas.
the accompanying web based tool, mirna literature network , available at http://bioinfo.memphis.edu/mirlin, provides an automated framework for interactively extracting and discovering functional information on human mirnas based on up to date biomedical literature.
methods
mirna document collection
for  <dig> human mirnas indexed in the mirbase repository,  <dig> different abstract collections were built.
firstly, a curated collection limited to manually assigned abstracts was constructed.
a total of  <dig> unique abstracts  cross referenced in the linkouts from mirbase as well as entrez gene  <cit>  were collected.
these citations  have been assigned either by professional staff at the national library of medicine, or by the scientific research community via gene reference into function  portal, or by curators of mirbase.
since these abstracts are manually curated, they are expected to have a very high precision for tagging correct citations to mirnas but at the same time the number of citations referenced for each mirna is a small proportion of the total number of relevant citations in medline for that mirna, resulting in low recall.
in order to increase the information content for the mirnas, a retrieved collection was built by querying the pubmed repository.
a single mirna can be referenced in the literature in several spelling variants e.g., mir19a, mir-19a, microrna19a, microrna-19a etc.
for each mirna, all such tentative synonyms with and without hyphens were constructed, and a pubmed query with the form ‘ synonym # <dig> or synonym # <dig> or...or synonym #n’ was submitted using the ncbi efetch utility for retrieving relevant citations that have at least one synonym present in either title or abstract.
further restrictions were added to the query to limit the search to abstracts relevant to humans and mirnas.
a total of  <dig> unique citations were retrieved.
the two collections were merged to get  <dig> unique citations.
we further filtered the nonspecific citations by removing pmids that referred to  <dig> or more mirnas.
typically, these citations described sequencing experiments which mentioned a large number of mirnas without substantive biological or mechanistic information.
this threshold of  <dig> was derived as the smallest right outlier in the distribution of numbers of mirnas linked to each unique citation.
the outlier calculation was based on the iqr .
the iqr is q <dig>  – q <dig> .
the designated outliers were >q3+ <dig> ∗i
q
r. post filtering,  <dig> unique citations and  <dig> active mirnas  remained in the collection, which comprised of less than half of the original number of  <dig> mirnas.
thus a large number of mirnas were excluded from our collection because they lacked a specific citation.
the number of citations assigned to the active mirnas ranged from  <dig>  to  <dig>  the average and median number of citations in the collection were  <dig> and  <dig>  respectively.
for each of  <dig> active mirnas, a mirna document was created by concatenating the titles and abstracts of all citations referenced by the mirna.
construction of the lsi model
the outline of the lsi approach used in this study is depicted in fig.
<dig>  sixty eight thousand five hundred ninety-six terms  were parsed from the collection of  <dig> mirna documents using text to matrix generator software  <cit> .
all punctuation  and capitalization were ignored and, in addition, articles and other common, non-distinguishing words were discarded using the stop list from cornell’s smart project repository  <cit> .
a term-by-mirna matrix was created where the entries of the matrix were log-entropy weighted frequencies of terms across the mirna document collection.
term weighting schemes are typically employed in order to normalize the matrix and discount the effect of common terms while at the same time increasing the importance of terms that are better delineators between mirna documents.
each matrix entry a
ij is transformed into a product of a local component  and global component :
<dig> lij=log <dig>
fig.
<dig> overview of latent semantic indexing.
in a vector-space model, the semantic structure of a document is represented as a vector  in word space, and the degree of similarity between documents is calculated by the cosine of the angle between document vectors.
the vectors consist of weighted terms, which are a function of the frequency of the terms in and across all documents in the collection.
a variant of the vector space model, called latent semantic indexing, improves retrieval by applying singular value decomposition  to create a subspace in which text documents are represented as vectors.
the components in the subspace may be regarded as a concept derived from the word usage patterns in the document.
hence, the relevant documents are retrieved based on the degree of conceptual similarity between the documents
<dig> gi=1+∑jpijlog2pijlog2n
<dig> pij=fij∑jfij
where f
ij is the frequency of the i
th term in the j
th mirna-document, p
ij is the probability of the i
th term occurring in the j
th mirna-document and n is the number of mirna documents in the collection.
the log-entropy weighting scheme is based on information-theoretic concepts and takes into account the distribution of terms over mirna documents and has been found to be more useful in extracting implied relationships  <cit> .
singular value decomposition   <cit>  was applied to the term-by-mirna log-entropy weighted frequency matrix.
a data matrix a with n rows  and m columns , where n>>m, can be construed as n term row vectors in m-dimensional mirna space and m mirna-document column vectors in n-dimensional term space.
svd transforms the two sets of vectors into a new r-dimensional orthogonal space in which the maximum variation is expressed along the first dimension axis, as much variation independent of that is expressed along an axis orthogonal to the first, and so on.
the new set of axes may reveal the true dimensionality of the data if the dataset is not inherently m-dimensional.
the svd is formulated as:
<dig> a=usv′
where ′ indicates transpose of the matrix obtained by permuting the modes, i.e., transforming rows into columns and vice versa, u is n×r, s is r×r, and v is m×r .
both u and v are orthogonal, i.e., u
u
′=i and v
v
′=i where i is the identity matrix.
s is a diagonal matrix with non-negative and non-increasing entries σ
<dig> σ
<dig> ...,σ
r which are known as singular values.
r is the rank of the matrix, which is the number of linearly independent rows or columns of a. it is however, known from observation, for most practical datasets, r=m.
the third matrix v is written as a transpose so that the rows of both matrices u and v correspond to terms and mirnas, respectively.
the rows of a can be interpreted as term coordinates in an m-dimensional space.
the axes of this space can be interpreted as rows of i .
the svd transforms the term coordinates to rows of u and the axes to the rows of s
v
′.
the matrix v
′ acts as the rotation matrix for the original axes and the diagonal of matrix s contains the scaling factor for each axis.
the u matrix can now be construed as a new transformed dataset whose rows still correspond to the original n terms but the mirnas are transformed into r eigen mirnas  that are a linear combination of the original mirnas.
svd is symmetric in the sense that a decomposition on the rows  can be transformed into a decomposition on the columns :
<dig> a′=vsu′
a
′ reverses the roles of terms and mirnas.
v plays the role originally played by u and u plays the role originally played by v. since s is diagonal, s=s
′.
the svd transforms the mirna coordinates to rows of v and the axes to the rows of s
u
′.
the matrix u
′ acts as the rotation matrix for the original axes and the diagonal of matrix s contains the scaling factor for each axis.
the v matrix can now be construed as a new transformed dataset whose rows still correspond to the original m mirnas but the terms are transformed into r eigen terms  that are a linear combination of the original terms.
the new scaled and rotated axes and the coordinates tend to better fit the data than the original axes and coordinates.
the singular values in s determine the relative importance of each axis.
the first few axes capture the maximum variation in the data and the subsequent ones less so.
only the first k  factors corresponding to k largest singular values may be used to represent the data.
there are two potential benefits of performing this truncation.
firstly, for large datasets , this translates into savings in memory space as well as analysis time, as vectors in k dimensions can be compared in less time than vectors in m dimensions.
secondly, svd reveals the true dimensionality present in the data, where the bulk of the information content in the original m-dimensional data may be captured in a lower dimensional manifold, after axis rotation and scaling.
an appropriate choice for k  can be made by assessing the contribution of each of the singular values as a measure of the amount of variation captured in each dimension, and then calculating the entropy of the contributions that might be indicative of what percentage of the total number of factors may be needed  <cit> .
the contribution c
i of each of r singular values σs can be calculated as:
<dig> ci=σi2∑i=1rσi <dig>
and the entropy of the r contributions calculated as:
<dig> e=−1logr∑k=1rcklogck
entropy measures the amount of disorder in the set of variations captured in the r dimensions.
the magnitude of the entropy may vary from  <dig>  to  <dig> .
k is calculated as e×r.
for the term-by-mirna matrix, k was computed to be  <dig>
the association between any pair of entities  can be calculated as the cosine of the angle between the respective k-dimensional vectors.
the association scores can theoretically fall between − <dig> and  <dig>  but in practice were observed to occur between − <dig>  and  <dig>  a higher association score between a pair of entities indicates a stronger relationship in literature.
evaluation
information gain calculation
information gain  for each mirna was calculated as
<dig> #ofcitationsretrievedfrompubmed−#ofcitationsinmirbaseandentrezgene#ofcitationsinmirbaseandentrezgene
gold standards
mir2disease was used for evaluating lsi performance.
it is a comprehensive database containing descriptions of more than  <dig> diseases and their associated mirnas.
auc calculation
the term-to-mirna and mirna-to-term prioritizations were evaluated against gold standards by generating receiver operating characteristics  curves which display recall and false positive rates at each rank.
the area under the curve  can be used as a measure of ranking quality  <cit> .
the auc will have the value of  <dig> for perfect ranking ,  <dig>  for randomly generated ranking, and  <dig> for the worst possible ranking .
cohesion calculation
the cohesion for a set of mirnas was calculated as described in  <cit> .
given a set of n mirnas for a disease, n aucs were calculated.
each mirna was treated as a query and the rest of the n− <dig> mirnas were treated as gold standard.
the set of all mirnas  in mir2disease were prioritized against the query mirna using the cosine between the mirna vectors as the similarity measure, and an auc was calculated.
the median auc out of n aucs was treated as the cohesion.
if a set of mirnas for a disease are closely related, then the mirnas in the set would ideally have high cosine association with each other compared to remaining mirnas that are not in the set, signifying a highly cohesive set.
RESULTS
mirna literature landscape
the annual number of publications related to mirnas is growing exponentially.
this trend is observed in curated databases such as mirbase and entrez gene, as well as in pubmed using “mirna” keyword search .
overall,  <dig>  times more citations were retrieved from the pubmed search than the number designated in curated databases.
to collect more abstracts for the growing number of mirnas, we designed an automated search strategy as described in “methods”.
out of  <dig> mirnas found in mirbase, while all had at least one manually designated citation in either mirbase or entrez gene, only  <dig> had at least one citation retrieved from pubmed.
our pubmed search did not identify abstracts for nearly  <dig> % of the mirnas in the curated databases.
for the aforementioned  <dig> mirnas with at least one retrieved citation, the recall values for more than  <dig> % of mirnas were between  <dig>  and  <dig>  when using the curated citations as gold standard .
it is however important to note that our pubmed search strategy retrieved  <dig>  % of all abstracts in curated databases .
this result suggests that there may be discrepancies in the curated databases for assignment of citations to mirnas.
on the other hand, it is possible that our search strategy misses important aliases for some mirnas, thus affecting the recall performance.
next, we calculated the information gain, as described in methods, for each of the  <dig> mirnas.
<dig> mirnas showed positive information gain and  <dig> mirnas showed a negative information gain .
only  <dig> mirnas showed an information gain greater than  <dig>  based on these results, we concluded that merging citations from mirbase and entrez gene with pubmed retrieved citations would allow for the best coverage and information gain for building the lsi model.
fig.
<dig> distribution of recall  and information gain  metrics for curated and retrieved citations for mirnas.
curated citations were collected from mirbase and entrez gene.
the retrieved citations were obtained by querying pubmed using a compound search term, which included the mirna symbol and its aliases.
recall for a mirna was computed as the fraction of curated citations present amongst the retrieved citations for that mirna.
information gain for a mirna was calculated as the ratio between the number of additional citations retrieved, and the number of curated citations for that mirna
once the abstract collection was updated and filtered for all mirnas, an lsi model was built using a total of  <dig> citations for the remaining  <dig> human mirnas, as described in the “methods”.
figure  <dig> shows the first three dimensions of the normalized scaled term vectors  and mirna vectors  in lsi space.
both term and mirna vectors are comparable with each other as they share the same coordinate space.
we found that while term vectors span a broad area, the mirna vectors are more concentrated.
the limited distribution of the mirna vectors suggests that the documents share many terms and that mirnas are functionally quite similar.
additional file 1: figure s <dig> shows the distribution of normalized singular values.
the first factor captured a little more  <dig> % of the variance  of the term-by-mirna matrix.
for this study, we used the top  <dig>  factors out of  <dig> factors, which comprised  <dig> % of the total information content.
fig.
<dig> distribution of term vectors  and mirna vectors  across the first three lsi dimensions.
each point on the plots represents a single term or mirna.
for each vector, the magnitude of each axis component was scaled by the corresponding singular value and the scaled vector was then normalized to unit length
evaluation of the lsi model
lsi is a robust approach to extract both explicit and implicit relationships between terms and mirnas directly from the biomedical literature.
in this study, the performance of the lsi model was evaluated based on three different use-case scenarios as described below.
mirna ranking by term query
a typical use-case involves ranking relevant mirnas based on their association with keyword queries.
a query may consist of a single word  such as “cancer” or a combination of words such as “head and neck squamous cell carcinoma”.
a binary query vector q
<dig> of a length equal to the total number of terms is created, with 1’s corresponding to the query terms and 0’s for the remaining terms in the dictionary.
a term query q is constructed by projecting q
<dig> onto the u
k matrix as q0′u
k, which is the weighted sum of k-dimensional term vectors corresponding to the query terms in the u
k matrix.
the mirnas are prioritized by calculating the cosines of the term query with each of the scaled k-dimensional mirna vectors in the v
s
k matrix.
to evaluate the lsi model, we used the mir2disease knowledge base as the gold standard.
since mir2disease was last updated in  <dig>  an lsi model specifically for this gold standard was generated using only publications dating to  <dig> or earlier.
for each disease, the full name  served as the query.
figure  <dig> shows the distribution of aucs for different query term lengths.
a full list of diseases and their respective aucs are included in additional file 2: table s1a.
the aucs for  <dig>  of all disease queries were above  <dig> .
generally, single word queries performed somewhat better than multiword queries.
this result is expected as summing various term vectors could make the query ambiguous, and the high ranked mirna vectors may actually be close to the composite query vector but only remotely related to any of the constituent terms of the composite query.
in addition, disease categories which included more than  <dig> mirnas generally resulted in lower aucs.
this may be due to the fact that some mirnas may have multiple roles and molecular functions, thereby lowering their relative ranking against a single disease query.
lastly, these results may be in part due to discrepancies in the annotations by the curators of mir2disease database.
fig.
<dig> distribution of area under the receiver operating characteristic curve  for term-to-mirna  and mirna-to-term  rankings.
for term-to-mirna rankings, the terms constituting a given disease name  were used as the query, the query length refers to the number of terms, and the mirnas associated with the disease were utilized as the gold standard.
for mirna-to-term rankings, the mirnas associated with a given disease were used as the query, the query length refers to the number of mirnas, and the terms constituting the disease name were utilized as the gold standard.
for both types of rankings, the auc values are shown stratified across various query lengths
to address the latter issue, we also tested the performance of the lsi model using a different gold standard  list of mirnas for nine different diseases or physiologies.
the gold standard ii mirnas were determined by manual examination of recent review papers on each topic .
the number of mirnas in each disease category ranged from  <dig> to  <dig>  the lsi model for evaluation was built using a collection of abstracts up to  <dig>  importantly, we found that the average auc for the nine disease queries was  <dig>  .
these results were substantially higher than those achieved by using mir2disease categories as gold standards, suggesting that mir2disease database may have errors.
term ranking by mirna query
another use-case for researchers would be to functionally annotate groups of mirnas.
this is relevant to genomic experiments which generally yield many differentially expressed mirnas.
here, the mirnas are treated as the query and the relevant terms are rank ordered.
mir2disease was used to select groups of mirnas that were assigned to specific diseases.
to evaluate the performance of the lsi model, the top  <dig> ranked terms associated with the group of mirnas were compared to the disease descriptors in mir2disease database.
a threshold of  <dig> terms was chosen because it would be impractical for users to consider the entire prioritized list of  <dig> terms and also to reduce the computational burden.
the list of diseases and their respective term aucs are available in additional file 2: table s1b.
the aucs for  <dig> diseases could not be obtained as none of the constituent terms in the names of these diseases were found amongst the top  <dig> ranked terms.
among the queries which returned at least one disease term in the top  <dig> ranked terms,  <dig>  queries produced an auc above  <dig> .
surprisingly, the average auc for the gold standard ii list was  <dig>  and none of the disease queries produced and auc above  <dig>  .
these results suggest that the top  <dig> terms extracted from the lsi model may be related to other topics  than only diseases.
mirna ranking by mirna query
a third use-case is prioritization of mirnas in response to a mirna query.
to evaluate the lsi model, we calculated the cohesion, as described in methods, amongst the group of mirnas assigned to specific diseases in mir2disease  or by subject matter experts .
the intent was to determine how well the lsi cosine similarity measure captures the real world clustering of related mirnas.
if a set of mirnas are involved in a disease, then the mirnas in the set should ideally have high cosine association with each other compared to remaining mirnas that are not in the set.
figure  <dig> shows the distribution of cohesions for  <dig> mirna disease groups having at least two mirnas.
the median cohesion for the lsi model was  <dig> , compared to the median cohesion of  <dig>  for a co-occurrence method, in which the similarity measure between mirnas was designated as the number of shared abstracts.
for  <dig>  diseases, the cohesions derived via the lsi model were significantly higher than chance when compared with the cohesions derived via the co-occurrence model  .
in contrast, the median cohesion using lsi was only marginally better than that produced via the co-occurrence method using the gold standard ii set,  <dig>  and  <dig>  respectively .
these results suggest that as the body of literature grows, mirnas may be associated with many different pathways and functions beyond just specific diseases.
fig.
<dig> distribution of cohesions for  <dig> diseases’ mirna groups in mir2disease.
the cohesions for the lsi model were compared with those from the co-occurrence model.
in the lsi model, the similarity between any two mirnas was calculated as the cosine of the angle between their vectors in truncated lsi space.
in the co-occurrence model, the similarity between any two mirnas was designated as the number of shared citations
clustering and functional annotation of mirnas
a major advantage of lsi is that the semantic relationships amongst all mirna documents may be measured in lower dimensional  space.
therefore, the cosine values between mirnas may be used as a similarity score to cluster functionally related mirnas.
in addition, using the lsi model, mirna clusters may be annotated using the top ranked terms as demonstrated above.
cosines were calculated for all mirna pairs and a mirna-mirna cosine matrix was generated.
the matrix was transformed into an adjacency  matrix using a cosine threshold of  <dig> percentile of all pair-wise cosines .
the adjacency matrix was truncated to include only  <dig> mirnas that were part of the largest connected component.
the graph was clustered using ukmeans algorithm  <cit>  with k =  <dig> to generate  <dig> mutually connected clusters .
for each cluster, the lsi model was queried using all of its mirnas and then the top  <dig> terms were extracted for each cluster.
fig.
<dig> clustering and functional annotation of mirnas based on lsi derived associations.
ukmeans clustering of the maximally connected component  of the mirna graph, in which an edge is assigned if the cosine value is above 99th percentile of all pair-wise cosine values.
the functional annotations for each mirna cluster were selected from amongst the top  <dig> ranked terms obtained via querying the truncated lsi space with the mirna cluster
the terms were manually examined and used to label each cluster in fig.
<dig>  for instance, the largest cluster containing  <dig> mirnas is associated with alzheimer disease.
this number is slightly different from the number  of alzheimer related mirnas in mir2disease database.
interestingly, the largest mir2disease group of mirnas  was associated with hepatocellular carcinoma.
it is important to note that the top nine mir2disease categories, containing between  <dig> to  <dig> mirnas, were all associated with some form of cancer.
this suggests that there is a large bias in the mirna databases as of  <dig>  by comparison, we found that the lsi-based clusters contained smaller number of mirnas that were associated with more specific terms, which were functionally aligned.
these results indicate that lsi based clustering allows for more robust functional clustering and more specific functional annotation beyond simply assigning mirnas to diseases.
mirlin web tool
we developed a publicly available web-tool  to provide access to the lsi model, which contains the most recent and comprehensive collection of mirna abstracts in medline .
the user can query the model with any combination of terms or mirnas.
when querying with terms, the tool ranks all mirnas in the collection with respect to semantic associations to the query.
alternatively, a mirna query may be used to compute associations with both mirnas and terms.
the output of the tool is a ranked list of mirnas and terms based on the degree of association  to the query.
selected mirnas and terms can be visualized as a network graph, where the nodes represent the selected mirnas and terms and the edges represent cosine values above  <dig> .
multiple nodes can be selected from the graph display to retrieve their shared abstracts, if applicable.
the abstracts are displayed with the selected terms and mirnas highlighted for convenience.
fig.
<dig> screenshot of mirna literature network  web tool.
mirlin enables users to prioritize mirnas and terms according to queries.
specific mirnas or terms can be selected  and displayed as a graph .
single or multiple nodes on the graph may be selected to view the abstracts associated with them in the lower panel
for benchmarking, we compared the performance of our web tool with two existing web tools, mircancer  <cit>  and miriad  <cit> .
while both tools are disease focused, mirlin is more flexible and can accept any type of query.
also, both tools rely on databases with binary associations between mirnas and diseases.
in contrast, mirlin ranks mirnas based on the functional relevancy to the query and also enables a genome-wide network view of mirnas with multiple associations to one another.
additional file 2: tables s4a and s4b compare the results from the  <dig> web tools for ‘choriocarcinoma’ and ‘meningioma’ queries.
for the ‘choriocarcinoma’ query, mircancer listed  <dig> mirnas  while miriad listed  <dig> additional mirnas .
importantly, mirlin retrieved all  <dig> mirnas within the top  <dig> ranked mirnas .
we manually evaluated the  <dig> additional mirnas retrieved by mirlin.
we found that mirnas hsa-mir-378a, hsa-mir-371b, hsa-mir-371a, hsa-let-7g, hsa-mir- <dig>  hsa-mir- <dig> and hsa-mir-15a were co-mentioned with the query term in the same abstracts, but not in the same sentences.
it appears that these mirnas were found to be differentially expressed in choriocarcinoma cell lines.
one mirna hsa-mir- <dig> was co-mentioned with the query term in the same sentence that suggests a direct link.
interestingly, hsa-mir- <dig> association with choriocarcinoma appeared to be indirect via its association with hsa-mir- <dig>  in addition, the abstract for hsa-mir- <dig> in mirlin was different from the other two web tools, suggesting that our abstract retrieval approach is slightly different than the other two methods.
lastly, hsa-mir- <dig> did not appear to be related to choriocarcinoma or any other type of cancer, thus appears to be a false discovery.
for the ‘meningioma’ query, mircancer retrieved  <dig> mirnas  and miriad retrieved  <dig> additional mirnas .
only two meningioma related mirnas overlapped between miriad and mircancer.
in comparison, mirlin retrieved all but one  amongst the top  <dig> ranked mirnas .
moreover, mirlin identified two additional mirnas .
manual examination found that hsa-mir- <dig> is in fact negatively associated with meningioma, where the citation explicitly negates its involvement in meningioma.
this result reveals a shortcoming of our method, which does not take into account negations and other parts of speech that are considered in nlp based approaches.
lastly, manual examination did not find an association between hsa-mir- <dig> and meningioma, albeit it is associated with other types of cancer.
discussion
we have developed an lsi based approach to prioritize, cluster and functionally annotate mirnas.
lsi enables representation of mirnas and terms as vectors in low dimensional space that can be compared against each other.
lsi provides an advantage over co-occurrence based methods as semantic associations between entities take into account not only the entities being compared but also indirect associations amongst all other related entities in the collection.
several choices were made in the construction of the model that affects its performance.
the rationale behind the choices and the potential ramifications of the alternatives are discussed below.
while building the mirna document collection, citations that referenced more than  <dig> mirnas were filtered out.
manual examination of citations revealed that certain high throughput screening papers were associated with many mirnas but these papers did not describe any functional information about the specific mirnas.
for instance, many citations described sequencing experiments that identified several mirnas.
inclusion of such citations in the model would create strong semantic associations between pairs of mirnas that are otherwise remotely related.
better automated methods are needed to identify and filter such abstracts that do not describe any functional relationships.
our results suggest that parsing of terms from mirna documents still needs improvement.
we found that many of the top  <dig> terms associated with groups of mirnas were indeed too specific, relating to gene symbols or non-standard abbreviations used in the papers.
for the current lsi model, only designated stopwords were removed prior to factorization.
automated methods may need to be investigated that can filter out additional non-useful terms.
stemming of the terms to their roots may also be useful in terms of reducing the dictionary size, although strategies for expanding the roots to the most relevant expansion will need to be devised once the terms are to be used for functional annotation.
currently, the selection of interesting functional annotation terms is still manual but could be automated by restricting to mesh  <cit> , go  <cit>  and kegg  <cit> .
however, this filtering strategy may result in loss of interesting terms such as gene or transcription factor names or phrases like ‘acaa-deletion’ that may indirectly link the mirnas to a physiology or a biological function or a disease.
several other methods may need to be investigated in the future to improve the performance of the lsi approach.
for instance, different types of normalization methods for the term-by-mirna matrix, in addition to the log-entropy method, may need to be investigated  <cit> .
in the current study, an entropy based method was used to select k highest magnitude singular values.
other strategies have been discussed in the literature that may improve performance  <cit> .
the web tool currently displays top  <dig> mirnas and  <dig> terms in response to a query.
automated methods, such as one used for determining the singular value threshold, may also be useful in devising a prioritization threshold for cosines.
finally, adding collections for other model organisms such as mouse, rat etc.
will make a more comprehensive text mining database for mirnas.
CONCLUSIONS
all together, we have demonstrated that an lsi based approach provides a robust and automated method to interrogate the large amount of literature that is accumulating with respect to mirnas.
the approach enables rapid prioritization of mirnas in relation to keyword or mirna queries.
furthermore, the lsi based approach allows for global clustering of all mirnas based on functional information in the literature and provides a method for annotating groups of mirnas with highly specific terms and concepts.
additional files
additional file  <dig> figures s <dig>  s <dig>  and s <dig>  ’s10-s <dig> pdf’ contains supplementary figures  <dig>   <dig> and  <dig> in separate pages.
additional file  <dig> tables s1a, s1b, s1c, s2a, s2b, s <dig>  s4a and s4b.
microsoft excel  <dig> workbook ’s11-s <dig> xlsx’ contains supplementary tables 1a, 1b, 1c, 2a, 2b,  <dig>  4a and 4b in separate tabs.
from 13th annual mcbios conference memphis, tn, usa.
3- <dig> may 2016
the authors would like to thank the university of memphis high performance computing facility for providing the needed computational resources for this study.
declarations
this article has been published as part of bmc bioinformatics volume  <dig> supplement  <dig>  2016: proceedings of the 13th annual mcbios conference.
the full contents of the supplement are available online at http://bmcbioinformatics.biomedcentral.com/articles/supplements/volume-17-supplement- <dig>
funding
this work and its publication was supported in part by the memphis research consortium and the university of memphis center for translational informatics.
availability of data and materials
the lsi model described in the paper can be accessed through the web tool mirna literature network  at http://bioinfo.memphis.edu/mirlin.
authors’ contributions
sr and rh designed the research and wrote the manuscript.
sr and bm conducted data analysis.
bc developed the web tool.
rh supervised the research and assisted with interpretation of results.
all authors read and approved the final manuscript.
competing interests
the authors declare that they have no competing interests.
consent for publication
not applicable.
ethics approval and consent to participate
not applicable.

 there is growing recognition that mirnas regulate various diseases and biological processes as evidenced by the rapidly growing body of literature related to mirnas. there are manually curated repositories such as mirbase and mir2disease that catalog mirnas in several organisms as well as summarize their associations with diseases and other biological processes. however, it is generally accepted that manual curation is unable to keep up with the rapidly growing genomic information. for instance, mirbase has not been updated since and mir2disease has not been updated since it is therefore imperative to devise automated methods that can keep pace with the functional information which is deposited in the biomedical literature with respect to mirnas. information retrieval is a key component of text mining. it consists of three types of models: set-theoretic, probabilistic, and algebraic. documents in each case are retrieved based on boolean logic, probability of relevance to the query, and the degree of similarity to the query, respectively. the concept of literature-based discovery was introduced by swanson and has since been extended to many different areas of research. in the gene space, several approaches have focused on mining both explicit associations based on co-occurrence, as well as implicit associations based on higher order co-occurrence and indirect relationships. several ir approaches have focused on mining mirna specific associations. mircancer, mirsel and mirtex use co-occurrence and sentence level natural language processing to automatically extract direct relationships from text between mirnas and genes or diseases. while useful, these tools may miss mirna interactions where direct relationships were not explicitly stated. in such cases, automated extraction of semantic relationships would be useful to associate genes and mirnas based on shared biological processes. also, explicit relationships such as those based on co-occurrence count between mirnas and genes may be harder to prioritize if they have the exact same score. in contrast, semantic associations that take into account other relationships could be useful for prioritization of mirna and gene associations. aside from exploring mirna to gene associations, semantic analysis could be useful for other research scenarios. for example, investigators may want to prioritize candidate mirnas for specific diseases or phenotypes. alternatively, investigators may want to understand the functional pathways shared between different mirnas. to address these needs, we developed and evaluated an lsi based text mining approach. previously, we applied lsi to extract functional relationships amongst genes as well as relationships between genes and transcription factors from medline abstracts. lsi uses singular value decomposition, which is a dimensionality reduction technique that decomposes the original term-by-document weighted frequency matrix into a new set of factor matrices that can be used to represent both terms and documents in lower-dimensional subspace. previously, we demonstrated that lsi can extract both explicit and implicit semantic relationships amongst genes. in addition, lsi allows genes to be prioritized based on keyword queries as well as gene-abstract queries with better accuracy than term co-occurrence methods. here, we applied this approach to mirnas and demonstrate its utility to prioritize, cluster and functionally annotate mirnas. the accompanying web based tool, mirna literature network, available at http://bioinfo.memphis.edu/mirlin, provides an automated framework for interactively extracting and discovering functional information on human mirnas based on up to date biomedical literature. methods mirna document collection for human mirnas indexed in the mirbase repository, different abstract collections were built. firstly, a curated collection limited to manually assigned abstracts was constructed. a total of unique abstracts cross referenced in the linkouts from mirbase as well as entrez gene were collected. these citations have been assigned either by professional staff at the national library of medicine, or by the scientific research community via gene reference into function portal, or by curators of mirbase. since these abstracts are manually curated, they are expected to have a very high precision for tagging correct citations to mirnas but at the same time the number of citations referenced for each mirna is a small proportion of the total number of relevant citations in medline for that mirna, resulting in low recall. in order to increase the information content for the mirnas, a retrieved collection was built by querying the pubmed repository. a single mirna can be referenced in the literature in several spelling variants e.g., mir19a, mir-19a, microrna19a, microrna-19a etc. for each mirna, all such tentative synonyms with and without hyphens were constructed, and a pubmed query with the form ‘ synonym # or synonym # or...or synonym #n’ was submitted using the ncbi efetch utility for retrieving relevant citations that have at least one synonym present in either title or abstract. further restrictions were added to the query to limit the search to abstracts relevant to humans and mirnas. a total of unique citations were retrieved. the two collections were merged to get unique citations. we further filtered the nonspecific citations by removing pmids that referred to or more mirnas. typically, these citations described sequencing experiments which mentioned a large number of mirnas without substantive biological or mechanistic information. this threshold of was derived as the smallest right outlier in the distribution of numbers of mirnas linked to each unique citation. the outlier calculation was based on the iqr. the iqr is q – q. the designated outliers were >q3+ ∗i q r. post filtering, unique citations and active mirnas remained in the collection, which comprised of less than half of the original number of mirnas. thus a large number of mirnas were excluded from our collection because they lacked a specific citation. the number of citations assigned to the active mirnas ranged from to the average and median number of citations in the collection were and respectively. for each of active mirnas, a mirna document was created by concatenating the titles and abstracts of all citations referenced by the mirna. construction of the lsi model the outline of the lsi approach used in this study is depicted in fig. sixty eight thousand five hundred ninety-six terms were parsed from the collection of mirna documents using text to matrix generator software. all punctuation and capitalization were ignored and, in addition, articles and other common, non-distinguishing words were discarded using the stop list from cornell’s smart project repository. a term-by-mirna matrix was created where the entries of the matrix were log-entropy weighted frequencies of terms across the mirna document collection. term weighting schemes are typically employed in order to normalize the matrix and discount the effect of common terms while at the same time increasing the importance of terms that are better delineators between mirna documents. each matrix entry a ij is transformed into a product of a local component and global component: lij=log fig. overview of latent semantic indexing. in a vector-space model, the semantic structure of a document is represented as a vector in word space, and the degree of similarity between documents is calculated by the cosine of the angle between document vectors. the vectors consist of weighted terms, which are a function of the frequency of the terms in and across all documents in the collection. a variant of the vector space model, called latent semantic indexing, improves retrieval by applying singular value decomposition to create a subspace in which text documents are represented as vectors. the components in the subspace may be regarded as a concept derived from the word usage patterns in the document. hence, the relevant documents are retrieved based on the degree of conceptual similarity between the documents gi=1+∑jpijlog2pijlog2n pij=fij∑jfij where f ij is the frequency of the i th term in the j th mirna-document, p ij is the probability of the i th term occurring in the j th mirna-document and n is the number of mirna documents in the collection. the log-entropy weighting scheme is based on information-theoretic concepts and takes into account the distribution of terms over mirna documents and has been found to be more useful in extracting implied relationships. singular value decomposition was applied to the term-by-mirna log-entropy weighted frequency matrix. a data matrix a with n rows and m columns, where n>>m, can be construed as n term row vectors in m-dimensional mirna space and m mirna-document column vectors in n-dimensional term space. svd transforms the two sets of vectors into a new r-dimensional orthogonal space in which the maximum variation is expressed along the first dimension axis, as much variation independent of that is expressed along an axis orthogonal to the first, and so on. the new set of axes may reveal the true dimensionality of the data if the dataset is not inherently m-dimensional. the svd is formulated as: a=usv′ where ′ indicates transpose of the matrix obtained by permuting the modes, i.e., transforming rows into columns and vice versa, u is n×r, s is r×r, and v is m×r. both u and v are orthogonal, i.e., u u ′=i and v v ′=i where i is the identity matrix. s is a diagonal matrix with non-negative and non-increasing entries σ σ...,σ r which are known as singular values. r is the rank of the matrix, which is the number of linearly independent rows or columns of a. it is however, known from observation, for most practical datasets, r=m. the third matrix v is written as a transpose so that the rows of both matrices u and v correspond to terms and mirnas, respectively. the rows of a can be interpreted as term coordinates in an m-dimensional space. the axes of this space can be interpreted as rows of i. the svd transforms the term coordinates to rows of u and the axes to the rows of s v ′. the matrix v ′ acts as the rotation matrix for the original axes and the diagonal of matrix s contains the scaling factor for each axis. the u matrix can now be construed as a new transformed dataset whose rows still correspond to the original n terms but the mirnas are transformed into r eigen mirnas that are a linear combination of the original mirnas. svd is symmetric in the sense that a decomposition on the rows can be transformed into a decomposition on the columns: a′=vsu′ a ′ reverses the roles of terms and mirnas. v plays the role originally played by u and u plays the role originally played by v. since s is diagonal, s=s ′. the svd transforms the mirna coordinates to rows of v and the axes to the rows of s u ′. the matrix u ′ acts as the rotation matrix for the original axes and the diagonal of matrix s contains the scaling factor for each axis. the v matrix can now be construed as a new transformed dataset whose rows still correspond to the original m mirnas but the terms are transformed into r eigen terms that are a linear combination of the original terms. the new scaled and rotated axes and the coordinates tend to better fit the data than the original axes and coordinates. the singular values in s determine the relative importance of each axis. the first few axes capture the maximum variation in the data and the subsequent ones less so. only the first k factors corresponding to k largest singular values may be used to represent the data. there are two potential benefits of performing this truncation. firstly, for large datasets, this translates into savings in memory space as well as analysis time, as vectors in k dimensions can be compared in less time than vectors in m dimensions. secondly, svd reveals the true dimensionality present in the data, where the bulk of the information content in the original m-dimensional data may be captured in a lower dimensional manifold, after axis rotation and scaling. an appropriate choice for k can be made by assessing the contribution of each of the singular values as a measure of the amount of variation captured in each dimension, and then calculating the entropy of the contributions that might be indicative of what percentage of the total number of factors may be needed. the contribution c i of each of r singular values σs can be calculated as: ci=σi2∑i=1rσi and the entropy of the r contributions calculated as: e=−1logr∑k=1rcklogck entropy measures the amount of disorder in the set of variations captured in the r dimensions. the magnitude of the entropy may vary from to. k is calculated as e×r. for the term-by-mirna matrix, k was computed to be the association between any pair of entities can be calculated as the cosine of the angle between the respective k-dimensional vectors. the association scores can theoretically fall between − and but in practice were observed to occur between − and a higher association score between a pair of entities indicates a stronger relationship in literature. evaluation information gain calculation information gain for each mirna was calculated as #ofcitationsretrievedfrompubmed−#ofcitationsinmirbaseandentrezgene#ofcitationsinmirbaseandentrezgene gold standards mir2disease was used for evaluating lsi performance. it is a comprehensive database containing descriptions of more than diseases and their associated mirnas. auc calculation the term-to-mirna and mirna-to-term prioritizations were evaluated against gold standards by generating receiver operating characteristics curves which display recall and false positive rates at each rank. the area under the curve can be used as a measure of ranking quality. the auc will have the value of for perfect ranking, for randomly generated ranking, and for the worst possible ranking. cohesion calculation the cohesion for a set of mirnas was calculated as described in. given a set of n mirnas for a disease, n aucs were calculated. each mirna was treated as a query and the rest of the n− mirnas were treated as gold standard. the set of all mirnas in mir2disease were prioritized against the query mirna using the cosine between the mirna vectors as the similarity measure, and an auc was calculated. the median auc out of n aucs was treated as the cohesion. if a set of mirnas for a disease are closely related, then the mirnas in the set would ideally have high cosine association with each other compared to remaining mirnas that are not in the set, signifying a highly cohesive set.  mirna literature landscape the annual number of publications related to mirnas is growing exponentially. this trend is observed in curated databases such as mirbase and entrez gene, as well as in pubmed using “mirna” keyword search. overall, times more citations were retrieved from the pubmed search than the number designated in curated databases. to collect more abstracts for the growing number of mirnas, we designed an automated search strategy as described in “methods”. out of mirnas found in mirbase, while all had at least one manually designated citation in either mirbase or entrez gene, only had at least one citation retrieved from pubmed. our pubmed search did not identify abstracts for nearly % of the mirnas in the curated databases. for the aforementioned mirnas with at least one retrieved citation, the recall values for more than % of mirnas were between and when using the curated citations as gold standard. it is however important to note that our pubmed search strategy retrieved % of all abstracts in curated databases. this result suggests that there may be discrepancies in the curated databases for assignment of citations to mirnas. on the other hand, it is possible that our search strategy misses important aliases for some mirnas, thus affecting the recall performance. next, we calculated the information gain, as described in methods, for each of the mirnas. mirnas showed positive information gain and mirnas showed a negative information gain. only mirnas showed an information gain greater than based on these results, we concluded that merging citations from mirbase and entrez gene with pubmed retrieved citations would allow for the best coverage and information gain for building the lsi model. fig. distribution of recall and information gain metrics for curated and retrieved citations for mirnas. curated citations were collected from mirbase and entrez gene. the retrieved citations were obtained by querying pubmed using a compound search term, which included the mirna symbol and its aliases. recall for a mirna was computed as the fraction of curated citations present amongst the retrieved citations for that mirna. information gain for a mirna was calculated as the ratio between the number of additional citations retrieved, and the number of curated citations for that mirna once the abstract collection was updated and filtered for all mirnas, an lsi model was built using a total of citations for the remaining human mirnas, as described in the “methods”. figure shows the first three dimensions of the normalized scaled term vectors and mirna vectors in lsi space. both term and mirna vectors are comparable with each other as they share the same coordinate space. we found that while term vectors span a broad area, the mirna vectors are more concentrated. the limited distribution of the mirna vectors suggests that the documents share many terms and that mirnas are functionally quite similar. additional file 1: figure s shows the distribution of normalized singular values. the first factor captured a little more % of the variance of the term-by-mirna matrix. for this study, we used the top factors out of factors, which comprised % of the total information content. fig. distribution of term vectors and mirna vectors across the first three lsi dimensions. each point on the plots represents a single term or mirna. for each vector, the magnitude of each axis component was scaled by the corresponding singular value and the scaled vector was then normalized to unit length evaluation of the lsi model lsi is a robust approach to extract both explicit and implicit relationships between terms and mirnas directly from the biomedical literature. in this study, the performance of the lsi model was evaluated based on three different use-case scenarios as described below. mirna ranking by term query a typical use-case involves ranking relevant mirnas based on their association with keyword queries. a query may consist of a single word such as “cancer” or a combination of words such as “head and neck squamous cell carcinoma”. a binary query vector q of a length equal to the total number of terms is created, with 1’s corresponding to the query terms and 0’s for the remaining terms in the dictionary. a term query q is constructed by projecting q onto the u k matrix as q0′u k, which is the weighted sum of k-dimensional term vectors corresponding to the query terms in the u k matrix. the mirnas are prioritized by calculating the cosines of the term query with each of the scaled k-dimensional mirna vectors in the v s k matrix. to evaluate the lsi model, we used the mir2disease knowledge base as the gold standard. since mir2disease was last updated in an lsi model specifically for this gold standard was generated using only publications dating to or earlier. for each disease, the full name served as the query. figure shows the distribution of aucs for different query term lengths. a full list of diseases and their respective aucs are included in additional file 2: table s1a. the aucs for of all disease queries were above. generally, single word queries performed somewhat better than multiword queries. this result is expected as summing various term vectors could make the query ambiguous, and the high ranked mirna vectors may actually be close to the composite query vector but only remotely related to any of the constituent terms of the composite query. in addition, disease categories which included more than mirnas generally resulted in lower aucs. this may be due to the fact that some mirnas may have multiple roles and molecular functions, thereby lowering their relative ranking against a single disease query. lastly, these results may be in part due to discrepancies in the annotations by the curators of mir2disease database. fig. distribution of area under the receiver operating characteristic curve for term-to-mirna and mirna-to-term rankings. for term-to-mirna rankings, the terms constituting a given disease name were used as the query, the query length refers to the number of terms, and the mirnas associated with the disease were utilized as the gold standard. for mirna-to-term rankings, the mirnas associated with a given disease were used as the query, the query length refers to the number of mirnas, and the terms constituting the disease name were utilized as the gold standard. for both types of rankings, the auc values are shown stratified across various query lengths to address the latter issue, we also tested the performance of the lsi model using a different gold standard list of mirnas for nine different diseases or physiologies. the gold standard ii mirnas were determined by manual examination of recent review papers on each topic. the number of mirnas in each disease category ranged from to the lsi model for evaluation was built using a collection of abstracts up to importantly, we found that the average auc for the nine disease queries was. these results were substantially higher than those achieved by using mir2disease categories as gold standards, suggesting that mir2disease database may have errors. term ranking by mirna query another use-case for researchers would be to functionally annotate groups of mirnas. this is relevant to genomic experiments which generally yield many differentially expressed mirnas. here, the mirnas are treated as the query and the relevant terms are rank ordered. mir2disease was used to select groups of mirnas that were assigned to specific diseases. to evaluate the performance of the lsi model, the top ranked terms associated with the group of mirnas were compared to the disease descriptors in mir2disease database. a threshold of terms was chosen because it would be impractical for users to consider the entire prioritized list of terms and also to reduce the computational burden. the list of diseases and their respective term aucs are available in additional file 2: table s1b. the aucs for diseases could not be obtained as none of the constituent terms in the names of these diseases were found amongst the top ranked terms. among the queries which returned at least one disease term in the top ranked terms, queries produced an auc above. surprisingly, the average auc for the gold standard ii list was and none of the disease queries produced and auc above. these results suggest that the top terms extracted from the lsi model may be related to other topics than only diseases. mirna ranking by mirna query a third use-case is prioritization of mirnas in response to a mirna query. to evaluate the lsi model, we calculated the cohesion, as described in methods, amongst the group of mirnas assigned to specific diseases in mir2disease or by subject matter experts. the intent was to determine how well the lsi cosine similarity measure captures the real world clustering of related mirnas. if a set of mirnas are involved in a disease, then the mirnas in the set should ideally have high cosine association with each other compared to remaining mirnas that are not in the set. figure shows the distribution of cohesions for mirna disease groups having at least two mirnas. the median cohesion for the lsi model was, compared to the median cohesion of for a co-occurrence method, in which the similarity measure between mirnas was designated as the number of shared abstracts. for diseases, the cohesions derived via the lsi model were significantly higher than chance when compared with the cohesions derived via the co-occurrence model. in contrast, the median cohesion using lsi was only marginally better than that produced via the co-occurrence method using the gold standard ii set, and respectively. these results suggest that as the body of literature grows, mirnas may be associated with many different pathways and functions beyond just specific diseases. fig. distribution of cohesions for diseases’ mirna groups in mir2disease. the cohesions for the lsi model were compared with those from the co-occurrence model. in the lsi model, the similarity between any two mirnas was calculated as the cosine of the angle between their vectors in truncated lsi space. in the co-occurrence model, the similarity between any two mirnas was designated as the number of shared citations clustering and functional annotation of mirnas a major advantage of lsi is that the semantic relationships amongst all mirna documents may be measured in lower dimensional space. therefore, the cosine values between mirnas may be used as a similarity score to cluster functionally related mirnas. in addition, using the lsi model, mirna clusters may be annotated using the top ranked terms as demonstrated above. cosines were calculated for all mirna pairs and a mirna-mirna cosine matrix was generated. the matrix was transformed into an adjacency matrix using a cosine threshold of percentile of all pair-wise cosines. the adjacency matrix was truncated to include only mirnas that were part of the largest connected component. the graph was clustered using ukmeans algorithm with k = to generate mutually connected clusters. for each cluster, the lsi model was queried using all of its mirnas and then the top terms were extracted for each cluster. fig. clustering and functional annotation of mirnas based on lsi derived associations. ukmeans clustering of the maximally connected component of the mirna graph, in which an edge is assigned if the cosine value is above 99th percentile of all pair-wise cosine values. the functional annotations for each mirna cluster were selected from amongst the top ranked terms obtained via querying the truncated lsi space with the mirna cluster the terms were manually examined and used to label each cluster in fig. for instance, the largest cluster containing mirnas is associated with alzheimer disease. this number is slightly different from the number of alzheimer related mirnas in mir2disease database. interestingly, the largest mir2disease group of mirnas was associated with hepatocellular carcinoma. it is important to note that the top nine mir2disease categories, containing between to mirnas, were all associated with some form of cancer. this suggests that there is a large bias in the mirna databases as of by comparison, we found that the lsi-based clusters contained smaller number of mirnas that were associated with more specific terms, which were functionally aligned. these results indicate that lsi based clustering allows for more robust functional clustering and more specific functional annotation beyond simply assigning mirnas to diseases. mirlin web tool we developed a publicly available web-tool to provide access to the lsi model, which contains the most recent and comprehensive collection of mirna abstracts in medline. the user can query the model with any combination of terms or mirnas. when querying with terms, the tool ranks all mirnas in the collection with respect to semantic associations to the query. alternatively, a mirna query may be used to compute associations with both mirnas and terms. the output of the tool is a ranked list of mirnas and terms based on the degree of association to the query. selected mirnas and terms can be visualized as a network graph, where the nodes represent the selected mirnas and terms and the edges represent cosine values above. multiple nodes can be selected from the graph display to retrieve their shared abstracts, if applicable. the abstracts are displayed with the selected terms and mirnas highlighted for convenience. fig. screenshot of mirna literature network web tool. mirlin enables users to prioritize mirnas and terms according to queries. specific mirnas or terms can be selected and displayed as a graph. single or multiple nodes on the graph may be selected to view the abstracts associated with them in the lower panel for benchmarking, we compared the performance of our web tool with two existing web tools, mircancer and miriad. while both tools are disease focused, mirlin is more flexible and can accept any type of query. also, both tools rely on databases with binary associations between mirnas and diseases. in contrast, mirlin ranks mirnas based on the functional relevancy to the query and also enables a genome-wide network view of mirnas with multiple associations to one another. additional file 2: tables s4a and s4b compare the results from the web tools for ‘choriocarcinoma’ and ‘meningioma’ queries. for the ‘choriocarcinoma’ query, mircancer listed mirnas while miriad listed additional mirnas. importantly, mirlin retrieved all mirnas within the top ranked mirnas. we manually evaluated the additional mirnas retrieved by mirlin. we found that mirnas hsa-mir-378a, hsa-mir-371b, hsa-mir-371a, hsa-let-7g, hsa-mir- hsa-mir- and hsa-mir-15a were co-mentioned with the query term in the same abstracts, but not in the same sentences. it appears that these mirnas were found to be differentially expressed in choriocarcinoma cell lines. one mirna hsa-mir- was co-mentioned with the query term in the same sentence that suggests a direct link. interestingly, hsa-mir- association with choriocarcinoma appeared to be indirect via its association with hsa-mir- in addition, the abstract for hsa-mir- in mirlin was different from the other two web tools, suggesting that our abstract retrieval approach is slightly different than the other two methods. lastly, hsa-mir- did not appear to be related to choriocarcinoma or any other type of cancer, thus appears to be a false discovery. for the ‘meningioma’ query, mircancer retrieved mirnas and miriad retrieved additional mirnas. only two meningioma related mirnas overlapped between miriad and mircancer. in comparison, mirlin retrieved all but one amongst the top ranked mirnas. moreover, mirlin identified two additional mirnas. manual examination found that hsa-mir- is in fact negatively associated with meningioma, where the citation explicitly negates its involvement in meningioma. this result reveals a shortcoming of our method, which does not take into account negations and other parts of speech that are considered in nlp based approaches. lastly, manual examination did not find an association between hsa-mir- and meningioma, albeit it is associated with other types of cancer. discussion we have developed an lsi based approach to prioritize, cluster and functionally annotate mirnas. lsi enables representation of mirnas and terms as vectors in low dimensional space that can be compared against each other. lsi provides an advantage over co-occurrence based methods as semantic associations between entities take into account not only the entities being compared but also indirect associations amongst all other related entities in the collection. several choices were made in the construction of the model that affects its performance. the rationale behind the choices and the potential ramifications of the alternatives are discussed below. while building the mirna document collection, citations that referenced more than mirnas were filtered out. manual examination of citations revealed that certain high throughput screening papers were associated with many mirnas but these papers did not describe any functional information about the specific mirnas. for instance, many citations described sequencing experiments that identified several mirnas. inclusion of such citations in the model would create strong semantic associations between pairs of mirnas that are otherwise remotely related. better automated methods are needed to identify and filter such abstracts that do not describe any functional relationships. our results suggest that parsing of terms from mirna documents still needs improvement. we found that many of the top terms associated with groups of mirnas were indeed too specific, relating to gene symbols or non-standard abbreviations used in the papers. for the current lsi model, only designated stopwords were removed prior to factorization. automated methods may need to be investigated that can filter out additional non-useful terms. stemming of the terms to their roots may also be useful in terms of reducing the dictionary size, although strategies for expanding the roots to the most relevant expansion will need to be devised once the terms are to be used for functional annotation. currently, the selection of interesting functional annotation terms is still manual but could be automated by restricting to mesh, go and kegg. however, this filtering strategy may result in loss of interesting terms such as gene or transcription factor names or phrases like ‘acaa-deletion’ that may indirectly link the mirnas to a physiology or a biological function or a disease. several other methods may need to be investigated in the future to improve the performance of the lsi approach. for instance, different types of normalization methods for the term-by-mirna matrix, in addition to the log-entropy method, may need to be investigated. in the current study, an entropy based method was used to select k highest magnitude singular values. other strategies have been discussed in the literature that may improve performance. the web tool currently displays top mirnas and terms in response to a query. automated methods, such as one used for determining the singular value threshold, may also be useful in devising a prioritization threshold for cosines. finally, adding collections for other model organisms such as mouse, rat etc. will make a more comprehensive text mining database for mirnas.  all together, we have demonstrated that an lsi based approach provides a robust and automated method to interrogate the large amount of literature that is accumulating with respect to mirnas. the approach enables rapid prioritization of mirnas in relation to keyword or mirna queries. furthermore, the lsi based approach allows for global clustering of all mirnas based on functional information in the literature and provides a method for annotating groups of mirnas with highly specific terms and concepts. additional files additional file figures s s and s ’s10-s pdf’ contains supplementary figures and in separate pages. additional file tables s1a, s1b, s1c, s2a, s2b, s s4a and s4b. microsoft excel workbook ’s11-s xlsx’ contains supplementary tables 1a, 1b, 1c, 2a, 2b, 4a and 4b in separate tabs. from 13th annual mcbios conference memphis, tn, usa. 3- may 2016 the authors would like to thank the university of memphis high performance computing facility for providing the needed computational resources for this study. declarations this article has been published as part of bmc bioinformatics volume supplement 2016: proceedings of the 13th annual mcbios conference. the full contents of the supplement are available online at http://bmcbioinformatics.biomedcentral.com/articles/supplements/volume-17-supplement- funding this work and its publication was supported in part by the memphis research consortium and the university of memphis center for translational informatics. availability of data and materials the lsi model described in the paper can be accessed through the web tool mirna literature network at http://bioinfo.memphis.edu/mirlin. authors’ contributions sr and rh designed the research and wrote the manuscript. sr and bm conducted data analysis. bc developed the web tool. rh supervised the research and assisted with interpretation of results. all authors read and approved the final manuscript. competing interests the authors declare that they have no competing interests. consent for publication not applicable. ethics approval and consent to participate not applicable.
the documents were parsed and a weighted term-by-mirna frequency matrix was created, which was subsequently factorized via singular value decomposition to extract pair-wise cosine values between the term and mirna vectors in reduced rank semantic space. micrornastext mininglatent semantic indexingsingular value decomposition13th annual mcbios conference memphis, tn, usa 3- may https://mcbios.org/issue-copyright-statement© the author 2016 for approximately human mirnas indexed in mirbase, text documents were created by concatenating titles and abstracts of medline citations which refer to the mirnas. lsi modeling of medline abstracts provides a robust and automated method for mirna related knowledge discovery. the latest collection of mirna abstracts and lsi model can be accessed through the web tool mirna literature network at http://bioinfo.memphis.edu/mirlin. the amount of scientific information about micrornas is growing exponentially, making it difficult for researchers to interpret experimental results. lsi enables derivation of both explicit and implicit associations between entities based on word usage patterns. in this study, we present an automated text mining approach using latent semantic indexing for prioritization, clustering and functional annotation of mirnas. finally, term ranking by querying the lsi space with a group of mirnas enabled annotation of the clusters with functionally related terms.
 the amount of scientific information about micrornas is growing exponentially, making it difficult for researchers to interpret experimental results. in this study, we present an automated text mining approach using latent semantic indexing for prioritization, clustering and functional annotation of mirnas.  for approximately human mirnas indexed in mirbase, text documents were created by concatenating titles and abstracts of medline citations which refer to the mirnas. the documents were parsed and a weighted term-by-mirna frequency matrix was created, which was subsequently factorized via singular value decomposition to extract pair-wise cosine values between the term and mirna vectors in reduced rank semantic space. lsi enables derivation of both explicit and implicit associations between entities based on word usage patterns. using mir2disease as a gold standard, we found that lsi identified keyword-to-mirna relationships with high accuracy. in addition, we demonstrate that pair-wise associations between mirnas can be used to group them into categories which are functionally aligned. finally, term ranking by querying the lsi space with a group of mirnas enabled annotation of the clusters with functionally related terms.  lsi modeling of medline abstracts provides a robust and automated method for mirna related knowledge discovery. the latest collection of mirna abstracts and lsi model can be accessed through the web tool mirna literature network at http://bioinfo.memphis.edu/mirlin. electronic supplementary material the online version of this article contains supplementary material, which is available to authorized users. keywords micrornastext mininglatent semantic indexingsingular value decomposition13th annual mcbios conference memphis, tn, usa 3- may https://mcbios.org/issue-copyright-statement© the author 2016 