BACKGROUND
the rapid growth of next generation sequencing and transcriptomic studies, particularly on the causative agents of infectious diseases, requires accurate genome annotations to confidently analyze the sequencing data and identify and compare functions, pathways and networks.
there are many resources available for genome annotation and most rely on transferring annotations from model organism or protein family databases that vary greatly in content and quality  <cit> .
for microbial genomes, there are very few model organism databases containing manual annotations based on experimental evidence in the current literature.
therefore, when microbial genomes are reannotated or new gene functions are identified by subsequent experiments, the new updates are rarely incorporated into public sequence databases.
since the manual annotation of genomes using controlled vocabularies and evidence codes is a time-consuming task  <cit> , text mining solutions that link evidence in the literature to annotations in genome databases are needed  <cit> .
one recent example is text2genome, which extracts dna sequences from pubmed central  and maps them to model organism databases  <cit> .
significantly, this study was the first to mine text in supplementary files in the open access  subset.
the authors found dna sequences in 20% of the oa articles and then requested permission to mine the full text from over  <dig> publisher websites .
a related project called pubmed2ensembl links millions of articles to thousands of genes from  <dig> eukaryotic species using six data sources containing gene to literature links  <cit> .
many other projects have shown that text mining improves the links between literature and biological databases such as the protein data bank and gene expression omnibus  <cit>  or uniprot and the european nucleotide archive  <cit> .
in this latter study, the authors noted the existence of accession number ranges but did not attempt to expand or quantify the regions.
many other tools have been developed to extract information from biological texts and are reviewed in  <cit> .
most of the text mining applications discussed in these reviews focus on innovative efforts to extract genes, functions and interactions from model eukaryotic organisms.
for microbial genomes with very few functionally characterized proteins, locus tags are often associated with structural and functional annotations in the literature.
structural annotations may include revised gene starts, novel genes or mobile regions based on either computational or experimental evidence.
functional annotations may include the assignment of new definitions, gene names and functions.
therefore, a typical role filled by model organism databases is to update annotations by linking genes to experimental evidence in the literature, and text mining tools are often used to assist in the process of manual curation  <cit> .
another option for curators is to use a full text database like pmc to search for articles citing a specific gene or locus tag in the full text.
however, finding the locus tag within the article requires searching through the entire text and linked tables.
to facilitate these types of automated searches, we developed an r package to mine locus tags from text, tables and supplements in the oa subset.
we demonstrate the capabilities of these tools by mining locus tags from ten microbial genomes.
our main objectives are to  improve access to structured data in order to extract locus tags from rows that are linked to column names, captions and subheadings,  identify locus tags pairs marking the start and end of a region and then list genes mentioned indirectly within the range, and  search for all locus tags in supplementary tables and publications outside the oa subset in burkholderia pseudomallei k <dig> for comparison.
this comprehensive set of locus tags in b. pseudomallei is used to highlight deficiencies in current annotations and suggest future microbial gene mining efforts.
implementation
searching for a single locus tag in pmc is straightforward, for example, enter “rv3874” in the search box and this will return  <dig> articles .
these full text articles are part of two groups in pmc, an open access subset that are available for text mining and another set that are merely free to read.
the oa subset are available as xml files that can be downloaded using automated queries to either the ftp site or open archives initiative service and articles may be searched by adding the open access filter .
searching for all locus tags in pmc requires other steps that are outlined in the flowchart in figure  <dig>  we selected ten species  and searched entrez genome to find the strain listed as the “reference genome, community selected”.
we downloaded gff <dig> files from the ncbi genomes ftp site to retrieve an ordered list of locus tag identifiers from all features including cds, pseudogenes and rnas.
we used the locus tag prefix and first digit from the gff <dig> file to build wildcard searches and find pmc articles with a matching locus tag .
we also restricted the number of spurious matches by limiting the results to articles with the genus name in the title or abstract.
for example, this query was used for yersinia pestis co92:  and  and open access.
in some cases, the results returned a warning that the wildcard search used only the first  <dig> variations and therefore we lengthened the root word to include two digits.
for each publication, we passed the pmc id to the open archives initiative service and downloaded the xml version of the full text article.
we parsed the xml into text by splitting the document into main sections, and each section was further divided into complete sentences.
we parsed xml tables using rowspan and colspan attributes to correctly position and repeat cell values and then joined column names and cell values into a single delimited list to preserve the table structure in a single row.
we extracted tags from both main text and tables by matching a prefix followed by four digits  and optional suffixes.
we then expanded locus tag pairs marking the start and end of a region such as an operon or genomic island using the ordered list of tags in the gff <dig> file.
we saved the pmc id, locus tag, section title or table caption, full sentence or table row, and a flag indicating if the tag was mentioned indirectly within a range.
we manually checked all ranges with ten or more locus tags to ensure valid range expansions.
finally, we searched for additional locus tags in b. pseudomallei from supplementary tables and from full text articles outside the oa subset.
a complete description of the r functions listed in figure  <dig> is available in the supplementary text in additional file  <dig>  the r code is also available on github  for further community development.
RESULTS
locus tags in reference genomes
there are over  <dig>  million articles in pmc and  <dig>  articles  are included in the open access subset .
the number of oa publications is increasing rapidly each year  and 56% of pmc articles published in  <dig> are open and available for text mining.
we searched for articles from ten microbial genomes  with locus tag mentions by using the locus tag prefix as a wildcard pattern and found  <dig> total articles in pubmed and  <dig> articles in pmc, ranging from  <dig> articles in burkholderia pseudomallei to  <dig> publications in pseudomonas aeruginosa .
in order to find the most relevant articles in pmc, we also matched the genus name in the title or abstract and limited the results to the oa subset, which ranged from  <dig> articles in yersinia pestis to  <dig> in mycobacterium tuberculosis.
we downloaded the xml documents from  <dig> oa publications and extracted locus tags within the full text and tables .
the wildcard searches returned  <dig> papers without a locus tag, simply because the truncated pattern required by pmc returns the papers while the extraction step fails to return a valid locus tag string.
most of these articles matched a strain name sharing the same prefix as a locus tag ; more problematic are strain names like vc <dig> and vc <dig> in vibrio cholerae that also match a valid tag name and are false positives in the results.
we identified  <dig>  locus tags in main text and  <dig>  tags in tables .
we expanded locus tag ranges and identified another  <dig>  tags mentioned indirectly within a range.
the complete list of all  <dig> publications and  <dig>  tag mentions are available by genome in additional file 3: table s3-s <dig>  the majority of tags in b. pseudomallei were part of ranges , while the number of tags mentioned indirectly within other genomes included 5% from francisella, 8% from helicobacter, 14% from chlamydia and listeria, and 20-23% from the other five genomes.
we corrected  <dig> matches to locus tag pairs that were not part of a valid region .
most matches were to interaction pairs such as rv2158c-rv0631c from pmc2649132: “some edges in the sos response  were common to paths from cell wall proteins and gyrase”.
other matches included network paths, primer names, comparisons and ranges spanning the origin of replication such as rv3913-rv0017c.
in this case, the parser returned  <dig> tags between rv0017c and rv <dig> instead of the  <dig> tags between rv <dig> and rv0017c.
we also corrected six large range expansions that were the result of typographical errors in the published articles.
we noted a few cases where ranges should be expanded but the current parser did not detect them automatically, for example, some tables list the start and end of a region in different columns in a table.
we matched  <dig>  unique locus tags to identifiers in the refseq gff <dig> files , which is 42% of all refseq genes in the ten genomes.
for comparison, the entrez gene database contains only  <dig> genes linked to pubmed articles .
the locus tag mining provided eight times as many gene-literature links than currently available.
we identified  <dig> putative locus tags that were not found in the refseq gff <dig> files.
almost all the putative tags were due to the inconsistent use of a “c” suffix on the minus strands in strains of campylobacter, francisella and mycobacterium and could be resolved.
in a few cases, new locus tags such as bpss <dig> in b. pseudomallei were identified but not found in any gene database.
overall, the total number of locus tags with mentions in the oa subset ranged from 19% in v. cholerae to 82% in m. tuberculosis.
locus tags in burkholderia pseudomallei
in order to better estimate the fraction of locus tags indexed by oa publications, we also checked supplementary tables and other publications in b. pseudomallei .
there were  <dig> open access articles in pmc containing  <dig> direct mentions and  <dig> tags within ranges .
there were another  <dig> free articles in pmc with  <dig> direct mentions and  <dig> tags within ranges .
there were  <dig> articles in pmc matching the locus tag and the genus name burkholderia anywhere in the full text.
these articles included very few mentions as expected  and over half the tags were from two tables listing type vi secretion system homologs in b. pseudomallei.
we also identified seven pmc articles not found in the search results, including five with tags in supplementary tables only.
for example, the study by schell et al.
<cit>  lists  <dig> virulence genes in a zipped document file in the supplement, so these virulence genes are not even available using web searches.
the  <dig> supplementary tables contained the majority of all locus tags  and included  <dig>  total mentions from  <dig> publications.
the supplements included  <dig> word tables,  <dig> pdf tables,  <dig> excel files, four html tables, three zipped files and two genbank files .
the pmcsupp function in the r package was used to read all file types directly into data frames in r, except for pdf tables that were loaded as a vector of text and required additional code to reformat the table structure.
we included the genbank files from the genome reannotation  <cit>  since these  <dig> locus tags included operon groups, novel proteins and revised start coordinates for  <dig> proteins.
finally, we searched pubmed for any articles not included in pmc.
we identified  <dig> pubmed articles matching the b. pseudomallei locus tag prefix in the abstract.
nine of these articles have the full text available from the publisher and we extracted  <dig> mentions.
we found another  <dig> pubmed articles containing  <dig> total mentions from our own reference collection, although there are likely many other publications in this group that have not been identified.
overall, we retrieved  <dig> total articles and extracted  <dig>  total mentions .
there were  <dig> b. pseudomallei locus tags mentioned in  <dig> or more publications .
seven of these proteins are annotated as hypothetical proteins in entrez gene and another three are marked as putative proteins in uniprot, so nearly half of the highly cited proteins are hypothetical or putative proteins in one of these two major protein databases.
a few of the hypothetical proteins are reviewed below.
bpss <dig> is mentioned in  <dig> different publications and was first designated in  <dig> as the burkholderia intracellular motility a protein or bima, which is required for actin-based motility  <cit> .
bpsl <dig> is the burkholderia lethal factor  <dig>  and includes a known structure in the protein data bank   <cit> .
bpsl <dig> and bpss <dig> are burkholderia oca-like adhesin proteins or boab and boaa  <cit> .
bpss <dig> is named the burkholderia pseudomallei adhesion a protein or bpaa  <cit> .
it was noted by adler et al.
that the new annotation gives bpss <dig> that same gene name as an unrelated type v two-partner secreted bpaa found in some australian strains of b. pseudomallei <cit> .
bpss <dig> is a homolog of cycle inhibiting factors  and also has a structure in pdb  <cit> .
bpss <dig> is part of the type vi secretion system and included in a large gene cluster bpss1496-bpss <dig>  <cit>  with two other hypothetical proteins in table  <dig>
b. pseudomallei
locus tag and the number of times each tag was mentioned directly within the text or indirectly within a range
discussion
since the number of oa publications is increasing rapidly , tools that automatically link gene identifiers to recent articles in full text databases could improve microbial gene annotation in many ways.
within the document, the locus tags could be highlighted and linked to protein databases.
within a protein database, the links to publications containing the locus tag and the specific sentence or table row could be provided, along with the context of the mention such as a section title or table caption .
the mentions could also be viewed as tracks in genome browsers or processed further to summarize structural and functional annotations.
structural annotations include revised gene starts, novel genes, doubtful coding regions and mobile regions.
in b. pseudomallei, over half of refseq genes have alternate starts in the genemark, glimmer or prodigal predictions available in the same genomes ftp directory at ncbi.
therefore, finding verified start coordinates in the primary literature based on either experimental or computational evidence would be very useful.
prodigal has been recommended for burkholderia genomes due to their high gc content, and dunbar et al.
<cit>  includes gene start revisions for  <dig> inconsistent ortholog sets.
however, the locus tags and coordinates reported for b. pseudomallei are from strain 1710b.
since many of these protein sequences have 100% similarity to the corresponding protein sequences of k <dig>  locus tags in closely related strains would be another valuable resource to improve annotations.
in addition, the b. pseudomallei genome reannotation by nandi et al.
<cit>  included  <dig> refseq proteins with new start coordinates.
other useful sources of structural annotations in the primary literature include novel genes and doubtful coding regions.
the reannotation by nandi et al.
identified  <dig> novel genes and  <dig> doubtful cdss in the supplementary tables  <cit> .
one of the novel genes included bpsl1057f <dig> and the protein reportedly increased actin stress fiber formation in transfected cells.
since these novel genes are only found in the literature, they are often missed by tagging systems based solely on dictionary lookups.
we extracted locus tags based on pattern searches, which returned  <dig> additional tags not found in the refseq gff <dig> files .
functional annotations include gene names, definitions and less often terms from controlled vocabularies describing functions and other characteristics.
protein definitions and gene names are critical for comparative analyses since they are the most commonly used source of information transfer  <cit> .
clearly, public sequence databases and annotation service providers have failed to keep up with the increasing number of publications, and as illustrated in table  <dig>  many commonly cited locus tags are still listed as hypothetical proteins.
for example, bpss <dig> is mentioned in  <dig> different publications and was first identified as a burkholderia intracellular motility a protein  from stevens et al.
in  <dig>  <cit> .
there are also  <dig> papers in pmc matching bima and burkholderia; however, the gene name bima is not included in any public sequence database for strain k <dig> including ncbi, uniprot and ensembl.
this gene name is also missing from annotations provided by img  <cit> , rast  <cit> , and specialized databases, such as patric  <cit>  or burkholderia.com  <cit>  as well as the reannotation by nandi et al.
<cit> .
in  <dig>  the national institute of allergy and infectious diseases funded eight bioinformatic resource centers to provide access to pathogen genomes  <cit> .
as part of this effort, curated burkholderia annotations were available from the now defunct pathema database at jcvi  <cit>  and bima was correctly identified in this database.
however, these were not propagated to the other databases.
in this study, we focused only on locus tags.
however, there are many other gene identifiers that should be extracted.
in fact, many tables and text sources list gene names by default, and only use a locus tag if a gene name was not assigned by refseq or other annotation source.
for example, this sentence in bartpho et al.
<cit>  is typical: “further confirmation of the presence of some selected virulence genes; flic, bsaq, rpos, bpsl <dig>  bpss <dig>  bpsl <dig> and bpss <dig> was also performed”.
the gene names flic, bsaq, and rpos correspond to bpsl <dig>  bpss <dig>  and bpsl <dig> respectively in the refseq gff <dig> file; therefore, extracting gene names from oa publications will definitely improve microbial genome annotations.
in an effort to obtain the most accurate annotations for b. pseudomallei genomes, we are continuing to develop r scripts to extract these gene names.
there are many challenges in extracting gene identifiers from the literature, and some groups like the ucsc genocoding project are actively trying to mine articles outside the oa subset to expand access to human gene and sequence mentions  <cit> .
at least for microbial genomes, and b. pseudomallei in particular, we believe that freely available supplementary materials and locus tags mentioned indirectly within ranges are important sources for acquiring gene annotations.
other sources including gene names, accession numbers and coordinates should also be collected before proceeding with future efforts to summarize functions, interactions and pathways.
CONCLUSIONS
only  <dig> b. pseudomallei locus tags are mentioned directly in the main text and tables of the oa subset and are indexed and available for searching in pmc.
this represents 3% of the total number of b. pseudomallei locus tags mentioned in the literature, since most locus tags are available in supplementary tables or within ranges.
both of these are valuable annotation sources and we developed queries and tools in the pmcxml package to improve access to these data sources.
due to the rapid growth of oa submissions, extracting gene and locus tags from the literature would clearly benefit efforts to improve microbial genome annotation.
the next challenge will involve developing the data mining algorithms needed to automatically summarize the gene mentions to identify names and functions of experimentally characterized proteins such as virulence factors and antibiotic resistance genes directly from the literature database.
if successful, this would help to convert a full text database into a functional gene annotation database first envisioned by bourne  <cit> , and would provide a valuable reference for most microbial genomes that do not have recent or updated annotations available in public sequence databases.
availability and requirements
project name: pmcxml
project home page:https://github.com/cstubben/pmcxml
operating system: platform independent, however loading supplementary tables requires a number of unix dependencies  to read zip, word tables and pdf files
programming language: r
other requirements: package dependencies include stringr and gdata from cran, genomes from bioconductor and genomes <dig> from github
license: gpl
any restrictions to use by non-academics: none
competing interests
the authors declare that no competing interests exist.
authors’ contributions
cjs wrote the r code to mine locus tags and wrote the manuscript.
jfc defined the data mining task applied to b. pseudomallei genomes and edited the manuscript.
both authors read and approved the final manuscript.
supplementary material
additional file 1: table s1-s2
summary of ncbi searches in pubmed and pmc using locus tag prefixes and a summary of gene-pubmed links in the entrez gene database.
click here for file
additional file 2
a description of r functions in the pmcxml package and r code used to parse locus tags.
click here for file
additional file 3: table s3-s13
a list of  <dig> publications from ten reference genomes and  <dig>  locus tag mentions in the pmc open access subset.
click here for file
additional file 4: table s14
locus tag pairs detected as ranges that should not be expanded.
click here for file
additional file 5: table s15-s19
a list of  <dig> publications from burkholderia pseudomallei k <dig> and  <dig>  locus tags mentions in the scientific literature.
the publications include counts of locus tags in full text, tables or supplements.
click here for file
additional file 6: table s20
a list of  <dig> supplementary tables from burkholderia pseudomallei with counts of locus tags mentions, file types and formatting notes.
click here for file
acknowledgements
this work was supported in part through dtra grant cbs119924543-7049-basic to jfc.
jian song provided helpful comments on earlier drafts.

 the rapid growth of next generation sequencing and transcriptomic studies, particularly on the causative agents of infectious diseases, requires accurate genome annotations to confidently analyze the sequencing data and identify and compare functions, pathways and networks. there are many resources available for genome annotation and most rely on transferring annotations from model organism or protein family databases that vary greatly in content and quality. for microbial genomes, there are very few model organism databases containing manual annotations based on experimental evidence in the current literature. therefore, when microbial genomes are reannotated or new gene functions are identified by subsequent experiments, the new updates are rarely incorporated into public sequence databases. since the manual annotation of genomes using controlled vocabularies and evidence codes is a time-consuming task, text mining solutions that link evidence in the literature to annotations in genome databases are needed. one recent example is text2genome, which extracts dna sequences from pubmed central and maps them to model organism databases. significantly, this study was the first to mine text in supplementary files in the open access subset. the authors found dna sequences in 20% of the oa articles and then requested permission to mine the full text from over publisher websites. a related project called pubmed2ensembl links millions of articles to thousands of genes from eukaryotic species using six data sources containing gene to literature links. many other projects have shown that text mining improves the links between literature and biological databases such as the protein data bank and gene expression omnibus or uniprot and the european nucleotide archive. in this latter study, the authors noted the existence of accession number ranges but did not attempt to expand or quantify the regions. many other tools have been developed to extract information from biological texts and are reviewed in. most of the text mining applications discussed in these reviews focus on innovative efforts to extract genes, functions and interactions from model eukaryotic organisms. for microbial genomes with very few functionally characterized proteins, locus tags are often associated with structural and functional annotations in the literature. structural annotations may include revised gene starts, novel genes or mobile regions based on either computational or experimental evidence. functional annotations may include the assignment of new definitions, gene names and functions. therefore, a typical role filled by model organism databases is to update annotations by linking genes to experimental evidence in the literature, and text mining tools are often used to assist in the process of manual curation. another option for curators is to use a full text database like pmc to search for articles citing a specific gene or locus tag in the full text. however, finding the locus tag within the article requires searching through the entire text and linked tables. to facilitate these types of automated searches, we developed an r package to mine locus tags from text, tables and supplements in the oa subset. we demonstrate the capabilities of these tools by mining locus tags from ten microbial genomes. our main objectives are to improve access to structured data in order to extract locus tags from rows that are linked to column names, captions and subheadings, identify locus tags pairs marking the start and end of a region and then list genes mentioned indirectly within the range, and search for all locus tags in supplementary tables and publications outside the oa subset in burkholderia pseudomallei k for comparison. this comprehensive set of locus tags in b. pseudomallei is used to highlight deficiencies in current annotations and suggest future microbial gene mining efforts. implementation searching for a single locus tag in pmc is straightforward, for example, enter “rv3874” in the search box and this will return articles. these full text articles are part of two groups in pmc, an open access subset that are available for text mining and another set that are merely free to read. the oa subset are available as xml files that can be downloaded using automated queries to either the ftp site or open archives initiative service and articles may be searched by adding the open access filter. searching for all locus tags in pmc requires other steps that are outlined in the flowchart in figure  we selected ten species and searched entrez genome to find the strain listed as the “reference genome, community selected”. we downloaded gff files from the ncbi genomes ftp site to retrieve an ordered list of locus tag identifiers from all features including cds, pseudogenes and rnas. we used the locus tag prefix and first digit from the gff file to build wildcard searches and find pmc articles with a matching locus tag. we also restricted the number of spurious matches by limiting the results to articles with the genus name in the title or abstract. for example, this query was used for yersinia pestis co92: and and open access. in some cases, the results returned a warning that the wildcard search used only the first variations and therefore we lengthened the root word to include two digits. for each publication, we passed the pmc id to the open archives initiative service and downloaded the xml version of the full text article. we parsed the xml into text by splitting the document into main sections, and each section was further divided into complete sentences. we parsed xml tables using rowspan and colspan attributes to correctly position and repeat cell values and then joined column names and cell values into a single delimited list to preserve the table structure in a single row. we extracted tags from both main text and tables by matching a prefix followed by four digits and optional suffixes. we then expanded locus tag pairs marking the start and end of a region such as an operon or genomic island using the ordered list of tags in the gff file. we saved the pmc id, locus tag, section title or table caption, full sentence or table row, and a flag indicating if the tag was mentioned indirectly within a range. we manually checked all ranges with ten or more locus tags to ensure valid range expansions. finally, we searched for additional locus tags in b. pseudomallei from supplementary tables and from full text articles outside the oa subset. a complete description of the r functions listed in figure  is available in the supplementary text in additional file the r code is also available on github for further community development.  locus tags in reference genomes there are over million articles in pmc and articles are included in the open access subset. the number of oa publications is increasing rapidly each year and 56% of pmc articles published in are open and available for text mining. we searched for articles from ten microbial genomes with locus tag mentions by using the locus tag prefix as a wildcard pattern and found total articles in pubmed and articles in pmc, ranging from articles in burkholderia pseudomallei to publications in pseudomonas aeruginosa. in order to find the most relevant articles in pmc, we also matched the genus name in the title or abstract and limited the results to the oa subset, which ranged from articles in yersinia pestis to in mycobacterium tuberculosis. we downloaded the xml documents from oa publications and extracted locus tags within the full text and tables. the wildcard searches returned papers without a locus tag, simply because the truncated pattern required by pmc returns the papers while the extraction step fails to return a valid locus tag string. most of these articles matched a strain name sharing the same prefix as a locus tag; more problematic are strain names like vc and vc in vibrio cholerae that also match a valid tag name and are false positives in the results. we identified locus tags in main text and tags in tables. we expanded locus tag ranges and identified another tags mentioned indirectly within a range. the complete list of all publications and tag mentions are available by genome in additional file 3: table s3-s the majority of tags in b. pseudomallei were part of ranges, while the number of tags mentioned indirectly within other genomes included 5% from francisella, 8% from helicobacter, 14% from chlamydia and listeria, and 20-23% from the other five genomes. we corrected matches to locus tag pairs that were not part of a valid region. most matches were to interaction pairs such as rv2158c-rv0631c from pmc2649132: “some edges in the sos response were common to paths from cell wall proteins and gyrase”. other matches included network paths, primer names, comparisons and ranges spanning the origin of replication such as rv3913-rv0017c. in this case, the parser returned tags between rv0017c and rv instead of the tags between rv and rv0017c. we also corrected six large range expansions that were the result of typographical errors in the published articles. we noted a few cases where ranges should be expanded but the current parser did not detect them automatically, for example, some tables list the start and end of a region in different columns in a table. we matched unique locus tags to identifiers in the refseq gff files, which is 42% of all refseq genes in the ten genomes. for comparison, the entrez gene database contains only genes linked to pubmed articles. the locus tag mining provided eight times as many gene-literature links than currently available. we identified putative locus tags that were not found in the refseq gff files. almost all the putative tags were due to the inconsistent use of a “c” suffix on the minus strands in strains of campylobacter, francisella and mycobacterium and could be resolved. in a few cases, new locus tags such as bpss in b. pseudomallei were identified but not found in any gene database. overall, the total number of locus tags with mentions in the oa subset ranged from 19% in v. cholerae to 82% in m. tuberculosis. locus tags in burkholderia pseudomallei in order to better estimate the fraction of locus tags indexed by oa publications, we also checked supplementary tables and other publications in b. pseudomallei. there were open access articles in pmc containing direct mentions and tags within ranges. there were another free articles in pmc with direct mentions and tags within ranges. there were articles in pmc matching the locus tag and the genus name burkholderia anywhere in the full text. these articles included very few mentions as expected and over half the tags were from two tables listing type vi secretion system homologs in b. pseudomallei. we also identified seven pmc articles not found in the search results, including five with tags in supplementary tables only. for example, the study by schell et al. lists virulence genes in a zipped document file in the supplement, so these virulence genes are not even available using web searches. the supplementary tables contained the majority of all locus tags and included total mentions from publications. the supplements included word tables, pdf tables, excel files, four html tables, three zipped files and two genbank files. the pmcsupp function in the r package was used to read all file types directly into data frames in r, except for pdf tables that were loaded as a vector of text and required additional code to reformat the table structure. we included the genbank files from the genome reannotation since these locus tags included operon groups, novel proteins and revised start coordinates for proteins. finally, we searched pubmed for any articles not included in pmc. we identified pubmed articles matching the b. pseudomallei locus tag prefix in the abstract. nine of these articles have the full text available from the publisher and we extracted mentions. we found another pubmed articles containing total mentions from our own reference collection, although there are likely many other publications in this group that have not been identified. overall, we retrieved total articles and extracted total mentions. there were b. pseudomallei locus tags mentioned in or more publications. seven of these proteins are annotated as hypothetical proteins in entrez gene and another three are marked as putative proteins in uniprot, so nearly half of the highly cited proteins are hypothetical or putative proteins in one of these two major protein databases. a few of the hypothetical proteins are reviewed below. bpss is mentioned in different publications and was first designated in as the burkholderia intracellular motility a protein or bima, which is required for actin-based motility. bpsl is the burkholderia lethal factor and includes a known structure in the protein data bank. bpsl and bpss are burkholderia oca-like adhesin proteins or boab and boaa. bpss is named the burkholderia pseudomallei adhesion a protein or bpaa. it was noted by adler et al. that the new annotation gives bpss that same gene name as an unrelated type v two-partner secreted bpaa found in some australian strains of b. pseudomallei. bpss is a homolog of cycle inhibiting factors and also has a structure in pdb. bpss is part of the type vi secretion system and included in a large gene cluster bpss1496-bpss with two other hypothetical proteins in table  b. pseudomallei locus tag and the number of times each tag was mentioned directly within the text or indirectly within a range discussion since the number of oa publications is increasing rapidly, tools that automatically link gene identifiers to recent articles in full text databases could improve microbial gene annotation in many ways. within the document, the locus tags could be highlighted and linked to protein databases. within a protein database, the links to publications containing the locus tag and the specific sentence or table row could be provided, along with the context of the mention such as a section title or table caption. the mentions could also be viewed as tracks in genome browsers or processed further to summarize structural and functional annotations. structural annotations include revised gene starts, novel genes, doubtful coding regions and mobile regions. in b. pseudomallei, over half of refseq genes have alternate starts in the genemark, glimmer or prodigal predictions available in the same genomes ftp directory at ncbi. therefore, finding verified start coordinates in the primary literature based on either experimental or computational evidence would be very useful. prodigal has been recommended for burkholderia genomes due to their high gc content, and dunbar et al. includes gene start revisions for inconsistent ortholog sets. however, the locus tags and coordinates reported for b. pseudomallei are from strain 1710b. since many of these protein sequences have 100% similarity to the corresponding protein sequences of k locus tags in closely related strains would be another valuable resource to improve annotations. in addition, the b. pseudomallei genome reannotation by nandi et al. included refseq proteins with new start coordinates. other useful sources of structural annotations in the primary literature include novel genes and doubtful coding regions. the reannotation by nandi et al. identified novel genes and doubtful cdss in the supplementary tables. one of the novel genes included bpsl1057f and the protein reportedly increased actin stress fiber formation in transfected cells. since these novel genes are only found in the literature, they are often missed by tagging systems based solely on dictionary lookups. we extracted locus tags based on pattern searches, which returned additional tags not found in the refseq gff files. functional annotations include gene names, definitions and less often terms from controlled vocabularies describing functions and other characteristics. protein definitions and gene names are critical for comparative analyses since they are the most commonly used source of information transfer. clearly, public sequence databases and annotation service providers have failed to keep up with the increasing number of publications, and as illustrated in table  many commonly cited locus tags are still listed as hypothetical proteins. for example, bpss is mentioned in different publications and was first identified as a burkholderia intracellular motility a protein from stevens et al. in. there are also papers in pmc matching bima and burkholderia; however, the gene name bima is not included in any public sequence database for strain k including ncbi, uniprot and ensembl. this gene name is also missing from annotations provided by img, rast, and specialized databases, such as patric or burkholderia.com as well as the reannotation by nandi et al.. in the national institute of allergy and infectious diseases funded eight bioinformatic resource centers to provide access to pathogen genomes. as part of this effort, curated burkholderia annotations were available from the now defunct pathema database at jcvi and bima was correctly identified in this database. however, these were not propagated to the other databases. in this study, we focused only on locus tags. however, there are many other gene identifiers that should be extracted. in fact, many tables and text sources list gene names by default, and only use a locus tag if a gene name was not assigned by refseq or other annotation source. for example, this sentence in bartpho et al. is typical: “further confirmation of the presence of some selected virulence genes; flic, bsaq, rpos, bpsl bpss bpsl and bpss was also performed”. the gene names flic, bsaq, and rpos correspond to bpsl bpss and bpsl respectively in the refseq gff file; therefore, extracting gene names from oa publications will definitely improve microbial genome annotations. in an effort to obtain the most accurate annotations for b. pseudomallei genomes, we are continuing to develop r scripts to extract these gene names. there are many challenges in extracting gene identifiers from the literature, and some groups like the ucsc genocoding project are actively trying to mine articles outside the oa subset to expand access to human gene and sequence mentions. at least for microbial genomes, and b. pseudomallei in particular, we believe that freely available supplementary materials and locus tags mentioned indirectly within ranges are important sources for acquiring gene annotations. other sources including gene names, accession numbers and coordinates should also be collected before proceeding with future efforts to summarize functions, interactions and pathways.  only b. pseudomallei locus tags are mentioned directly in the main text and tables of the oa subset and are indexed and available for searching in pmc. this represents 3% of the total number of b. pseudomallei locus tags mentioned in the literature, since most locus tags are available in supplementary tables or within ranges. both of these are valuable annotation sources and we developed queries and tools in the pmcxml package to improve access to these data sources. due to the rapid growth of oa submissions, extracting gene and locus tags from the literature would clearly benefit efforts to improve microbial genome annotation. the next challenge will involve developing the data mining algorithms needed to automatically summarize the gene mentions to identify names and functions of experimentally characterized proteins such as virulence factors and antibiotic resistance genes directly from the literature database. if successful, this would help to convert a full text database into a functional gene annotation database first envisioned by bourne, and would provide a valuable reference for most microbial genomes that do not have recent or updated annotations available in public sequence databases. availability and requirements project name: pmcxml project home page:https://github.com/cstubben/pmcxml operating system: platform independent, however loading supplementary tables requires a number of unix dependencies to read zip, word tables and pdf files programming language: r other requirements: package dependencies include stringr and gdata from cran, genomes from bioconductor and genomes from github license: gpl any restrictions to use by non-academics: none competing interests the authors declare that no competing interests exist. authors’ contributions cjs wrote the r code to mine locus tags and wrote the manuscript. jfc defined the data mining task applied to b. pseudomallei genomes and edited the manuscript. both authors read and approved the final manuscript. supplementary material additional file 1: table s1-s2 summary of ncbi searches in pubmed and pmc using locus tag prefixes and a summary of gene-pubmed links in the entrez gene database. click here for file additional file 2 a description of r functions in the pmcxml package and r code used to parse locus tags. click here for file additional file 3: table s3-s13 a list of publications from ten reference genomes and locus tag mentions in the pmc open access subset. click here for file additional file 4: table s14 locus tag pairs detected as ranges that should not be expanded. click here for file additional file 5: table s15-s19 a list of publications from burkholderia pseudomallei k and locus tags mentions in the scientific literature. the publications include counts of locus tags in full text, tables or supplements. click here for file additional file 6: table s20 a list of supplementary tables from burkholderia pseudomallei with counts of locus tags mentions, file types and formatting notes. click here for file acknowledgements this work was supported in part through dtra grant cbs119924543-7049-basic to jfc. jian song provided helpful comments on earlier drafts.
we propose to utilize the open access subset of pubmed central as a gene annotation database and have developed an r package called pmcxml to automatically mine and extract locus tags from full text, tables and supplements. we mined locus tags from oa publications in ten microbial genomes and extracted tags mentioned in sentences in main text and rows in tables. we also searched for locus tags in supplementary tables and publications outside the oa subset in burkholderia pseudomallei k for comparison. there were publications containing locus tags and 83% of mentions were from supplementary materials and 9% from publications outside the oa subset. b. pseudomallei locus tags within the full text and tables of oa publications represent only a small fraction of the total mentions in the literature. for microbial genomes with very few functionally characterized proteins, the locus tags mentioned in supplementary tables and within ranges like genomic islands contain the majority of locus tags. significantly, the functions in the r package provide access to additional resources in the oa subset that are not currently indexed or returned by searching pmc. the scientific literature contains millions of microbial gene identifiers within the full text and tables, but these annotations rarely get incorporated into public sequence databases. we identified locus tag pairs marking the start and end of a region such as an operon or genomic island and expanded these ranges to add another tags.
 the scientific literature contains millions of microbial gene identifiers within the full text and tables, but these annotations rarely get incorporated into public sequence databases. we propose to utilize the open access subset of pubmed central as a gene annotation database and have developed an r package called pmcxml to automatically mine and extract locus tags from full text, tables and supplements.  we mined locus tags from oa publications in ten microbial genomes and extracted tags mentioned in sentences in main text and rows in tables. we identified locus tag pairs marking the start and end of a region such as an operon or genomic island and expanded these ranges to add another tags. we also searched for locus tags in supplementary tables and publications outside the oa subset in burkholderia pseudomallei k for comparison. there were publications containing locus tags and 83% of mentions were from supplementary materials and 9% from publications outside the oa subset.  b. pseudomallei locus tags within the full text and tables of oa publications represent only a small fraction of the total mentions in the literature. for microbial genomes with very few functionally characterized proteins, the locus tags mentioned in supplementary tables and within ranges like genomic islands contain the majority of locus tags. significantly, the functions in the r package provide access to additional resources in the oa subset that are not currently indexed or returned by searching pmc. 