Background: Most biomedical corpora have not been used outside of the lab that created them, despite the fact that the availability of the gold-standard evaluation data that they provide is one of the rate-limiting factors for the progress of biomedical text mining. Data suggest that one major factor affecting the use of a corpus outside of its home laboratory is the format in which it is distributed. This paper tests the hypothesis that corpus refactoring -changing the format of a corpus without altering its semantics -is a feasible goal, namely that it can be accomplished with a semiautomatable process and in a time-effcient way. We used simple text processing methods and limited human validation to convert the Protein Design Group corpus into two new formats: WordFreak and embedded XML. We tracked the total time expended and the success rates of the automated steps.
The authors of REF also conclude that the format of a corpus is a major obstacle that hinders reuse of the corpus outside of the lab that has developed it.
Corpus Refactoring: a Feasibility Study
Journal of Biomedical Discovery and Collaboration
Journal of Biomedical Discovery and Collaboration
Computer Science Medicine