cometadata/datacite-titles-descriptions-related-identifiers huggingface dataset:
DOI: 10.5281/zenodo.5055686
title: PGxMine

description: This describes the output files for the PGxMine project. The code for this viewer is available in the PGxMine Github repo if you want to run it independently. Each file is a tab-delimited file with a header, no comments and no quoting. You likely want <strong>pgxmine_collated.tsv</strong> if you just want the list of chemical/variant assocations. If you want the supporting sentences, look at <strong>pgxmine_sentences.tsv</strong>. You can use the <em>matching_id</em> column to connect the two files. If you want to dig further and are okay with a higher false positive rate, look at <strong>pgxmine_unfiltered.tsv</strong>. <strong>pgxmine_collated.tsv:</strong> This contains the chemical/variant associations with citation counts supporting them. It contains the normalized chemical, variant and where appropriate gene names with identifiers for PharmGKB, dbSNP and Entrez. <strong>pgxmine_sentences.tsv:</strong> This contains the supporting sentences for the chemical/variant associations in the collated file. Each row is a single supporting sentence for one association. This file contains information on the source publication (e.g. journal, publication date, etc), the actual sentence and the chemical/variant association extracted. <strong>pgxmine_unfiltered.tsv:</strong> This is the combined raw output of the createKB.py script across all of PubMed, Pubmed Central Open Access and PubMed Central Author Manuscript Collection. It contains every predicted relation with a prediction score above 0.5. So this may contain many false positives. Each row contain information on the publication (e.g. journal, publication date, etc) along with the sentence and the specific chemical/variant association.
DOI: 10.35111/fjaq-y976
title: TDT5 Topics and Annotations

description: <h3>Introduction</h3><br>
<p>TDT5 Topics and Annotations was developed by the Linguistic Data Consortium (LDC) and includes about 10,000 topic relevance judgments and other associated information for the TDT5 2004 evaluation topics.</p><br>
<p>This release contains complete relevance judgments, including the results of adjudication, in which discrepancies between system submissions and LDC annotations are reviewed and relevance judgments updated. This release also contains answer keys for the link detection task.</p><br>
<p>The TDT5 corpora were created by Linguistic Data Consortium with support from the DARPA TIDES (Translingual Information Detection, Extraction and Summarization) Program. The multilingual news text corresponding to this publication can be found in <a href="http://catalog.ldc.upenn.edu/LDC2006T18" rel="nofollow">TDT5 Multilingual News Text (LDC2006T18)</a>.</p><br>
<h3>Data</h3><br>
<p>A total of 250 topics, numbered 55001 - 55250, were annotated by LDC using a search guided annotation technique. Details of the annotation process are described in the annotation task definition.</p><br>
<p>Approximately 25% of the topics are monolingual English (ENG), 25% are monolingual Mandarin Chinese (MAN), 25% are monolingual Arabic (ARB), and 25% are multilingual:</p><br>
<table><br>
<tbody><br>
<tr><br>
<td>63</td><br>
<td>ENG</td><br>
</tr><br>
<tr><br>
<td>62</td><br>
<td>MAN</td><br>
</tr><br>
<tr><br>
<td>62</td><br>
<td>ARB</td><br>
</tr><br>
<tr><br>
<td>35</td><br>
<td>ARB ENG MAN</td><br>
</tr><br>
<tr><br>
<td>21</td><br>
<td>ENG MAN</td><br>
</tr><br>
<tr><br>
<td>7</td><br>
<td>ARB ENG</td><br>
</tr><br>
<tr><br>
<td>250</td><br>
<td>total</td><br>
</tr><br>
</tbody><br>
</table><br>
<p>Broken down by language and counting both mono- and multi-lingual topics:</p><br>
<table><br>
<tbody><br>
<tr><br>
<td>126</td><br>
<td>ENG</td><br>
</tr><br>
<tr><br>
<td>118</td><br>
<td>MAN</td><br>
</tr><br>
<tr><br>
<td>104</td><br>
<td>ARB</td><br>
</tr><br>
</tbody><br>
</table><br>
<h3>Samples</h3><br>
<p>For an example of the data in this corpus, please review this <a href="desc/addenda/LDC2006T19.txt">sample (TXT)</a> from the link detection files.</p><br>
<h3>Updates</h3><br>
<p>None at this time.</p></br>
Portions © 2004, 2006 Trustees of the University of Pennsylvania
DOI: 10.35111/afrz-9s73
title: Arabic Broadcast News Transcripts

description: <h3>Introduction</h3><br>
<p>Arabic Broadcast News Transcripts was developed by the Linguistic Data Consortium (LDC) and consists of 10 hours of transcribed speech from Voice of America satellite radio news broadcasts in Arabic recorded by LDC between June 2000 and January 2001. The corresponding speech files are available in <a href="../../LDC2006S46">Arabic Broadcast News Speech (LDC2006S46)</a>.</p><br>
<p>This work was undertaken in the Networking Data Centers (NetDC) project (MLIS-5017, NSF IIS-9982201) in conjunction with the <a href="http://www.elda.org/en/">European Language Resources Association</a> (ELRA). ELRA transcribed 22.5 hours of Arabic broadcast data from Radio Orient (France) that is available in <a href="http://catalog.elra.info/product_info.php?products_id=13">NetDC Arabic BNSC (Broadcast News Speech Corpus) (ELRA-S0157)</a>. The goal of the NetDC project was to improve the infrastructure for language resources by designing and implementing new modes of cooperation between LDC and ELRA.</p><br>
<h3>Data</h3><br>
<p>The character encoding is entirely in ASCII; Buckwalter transliteration is used for rendering the Arabic text content. Time alignment and structural markup are rendered via "pseudo-SGML" tags, which are presented one tag per line, with the first character of the line being an open angle bracket.</p><br>
<p>The lines of transcription text (i.e. the speech and annotation content between the time-stamp tags) all begin with a single space character, and present exactly one token per line. A "token" may be a spoken Arabic word, a punctuation mark, or a single Arabic word enclosed by "(%" and ")", which represents an annotation of a non-speech condition or event (e.g. "music", "noise", "laugh", etc).</p><br>
<h3>Updates</h3><br>
<p>None at this time.</p><br>
<h3>Samples</h3><br>
<p>Please view this <a href="desc/addenda/LDC2006T20.txt">transcript sample</a>.</p></br>
Portions © 2000, 2001, 2002, 2005, 2006 Trustees of the University of Pennsylvania
DOI: 10.35111/77zm-gw29
title: Levantine Arabic Conversational Telephone Speech

description: <h3>Introduction</h3> <p>This database contains 982 Levantine Arabic speakers taking part in spontaneous telephone conversations in Colloquial Levantine Arabic. A total of 985 conversation sides are provided (there are three speakers who each appear in two disctinct conversations). The average duration per side is between 5 and 6 minutes.</p> <p>This corpus was collected and transcribed in 2004 by Appen Pty Ltd, Sydney, Australia.</p> <h3>Samples</h3> <p>For an example of the data into this corpus, please listen to this <a href="./desc/addenda/LDC2007S01.wav" rel="nofollow">audio sample</a> (wav format). </p> </br>
Portions © 2007 Trustees of the University of Pennsylvania
DOI: 10.35111/6z57-s580
title: ARL Urdu Speech Database, Training Data

description: <h3>Introduction</h3><br>
<p>ARL Urdu Speech Database, Training Data is a collection of recorded speech from 200 adult native Urdu speakers from Pakistan and Northern India. The recordings in this release were collected by Appen Pty Ltd, Sydney, Australia in 2006. The U.S. Army Research Laboratory (ARL) provided this corpus to the Linguistic Data Consortium for distribution.</p><br>
<p>Urdu is an Indo-Aryan language spoken throughout South Asia that developed under the Mughal Empire and Delhi Sultinate between 1200 AD and 1800 AD. It has Persian, Turkish and Arabic influences, but in fact is a dialect of Hindustani. The word "Urdu" refers to the standardized register of Hindustani, but there are many non-standard idiolects as well. Urdu is the twentieth most spoken language in the world. It is the native language of over 60 million people, it is the offical language of Pakistan, and it is one of India's national languages. Urdu is also spoken in Afghanistan.</p><br>
<p>The distribution of speaker dialects in the corpus is as follows:</p><br>
<table border="1" cellpadding="2"><br>
<tbody><br>
<tr><br>
<td><strong>Accent</strong></td><br>
<td><strong>Number of Speakers</strong></td><br>
</tr><br>
<tr><br>
<td>South Sindh</td><br>
<td>29</td><br>
</tr><br>
<tr><br>
<td>North Sindh</td><br>
<td>30</td><br>
</tr><br>
<tr><br>
<td>South Punjab</td><br>
<td>27</td><br>
</tr><br>
<tr><br>
<td>North Punjab</td><br>
<td>29</td><br>
</tr><br>
<tr><br>
<td>Captial Area</td><br>
<td>29</td><br>
</tr><br>
<tr><br>
<td>North West Regions</td><br>
<td>30</td><br>
</tr><br>
<tr><br>
<td>Baluchistan</td><br>
<td>26</td><br>
</tr><br>
</tbody><br>
</table><br>
<p> </p><br>
<p>The database is divided into two parts, a training set containing approximately 80% of the data and a test set comprised of 20% of the data. This release consists of approximately 80% of the complete dataset (training and test).</p><br>
<h3>Data</h3><br>
<p>Each speaker was presented with 400 prompts to read: sentences, place names, and person names. Two microphones set at different distances to the speaker were used for the recordings. The recorded speech was stored in raw format files with headers stored in separate directories.</p><br>
<p>Each utterance was transcribed in the corresponding label file for each recording. The transcriptions were encoded in UTF-8. Punctuation was omitted and numbers were written out in full.</p><br>
<h3>Update</h3><br>
<p>Earlier versions were missing the content list file. This is now available as part of the complete download file.</p><br>
<p>09/14/18 - The test data for this corpus was originally held back and is now available as part of the download. New downloads after the indicated date will contain the full corpus.</p><br>
<h3>Samples</h3><br>
<p>For an example of the data in this corpus, please listen to this following audio <a href="desc/addenda/LDC2007S03.wav" rel="nofollow">sample</a> (.wav format)</p></br>
Portions © 2006 U.S. Army Research Laboratory, © 2007 Trustees of the University of Pennsylvania
DOI: 10.35111/18ns-0a21
title: CSLU: Yes/No Version 1.2

description: <h3>Introduction</h3> <p>This file contains documentation for CSLU:Yes/No Version 1.2, Linguistic Data Consortium (LDC) catalog number LDC2007S05 and isbn 1-58563-445-X. </p><p>CSLU: Yes/No Version 1.2 is a collection of answers to yes/no questions from various telephone speech corpora created by the Center for Spoken Language Understanding, Oregon Health and Science University (CSLU). The corpus contains approximately 20,000 examples of roughly 18,000 speakers saying "yes" or "no" in response to various questions. </p><p>Each speech file in the corpus has a corresopnding orthographic transcription following the CSLU Labeling Conventions. In cases where a transcription did not already exist, the utterance was run through a speech recognizer to automatically obtain the transcription. </p><p>The data were collected from both analog and digital phone lines. The analog data were recorded using a Gradient Technologies analog-to-digital conversion box. These files were recorded as 16-bit, 8 khz and stored in a linear format. The digital data were recorded with the CSLU T1 digital data collection system. These files were sampled at 8 khz 8-bit and stored as ulaw files. All of the data use the RIFF standard file format. This file format is 16-bit linearly encoded. </p><h3>Samples</h3> <p>For a sample of the audio in this corpus, please listen to this <a href="./desc/addenda/LDC2007S05.wav" rel="nofollow">sample</a> . </p> </br>
Portions © 1996, 1998, 2000, 2002 Center for Spoken Language Understanding, Oregon Health and Science University, © 2007 Trustees of the University of Pennsylvania
DOI: 10.35111/0vwp-xn48
title: CSLU:  Foreign Accented English Release 1.2

description: <h3>Introduction</h3> <p>This file contains documentation on CSLU: Foreign Accented English Release 1.2, Linguistic Data Consortium (LDC) catalog number LDC2006S38 and isbn 1-58563-392-5.</p> <p>CSLU: Foreign Accented English Release 1.2 consists of continuous speech in English by native speakers of 22 different languages: Arabic, Cantonese, Czech, Farsi, French, German, Hindi, Hungarian, Indonesian, Italian, Japanese, Korean, Mandarin Chinese, Malay, Polish, Portuguese (Brazilian and Iberian), Russian, Swedish, Spanish, Swahili, Tamil and Vietnamese. The corpus contains 4925 telephone-quality utterances, information about the speakers' linguistic backgrounds and perceptual judgments about the accents in the utterances. The speakers were asked to speak about themselves in English for 20 seconds. Three native speakers of American English independently listened to each utterance and judged the speakers' accents on a 4-point scale: negligible/no accent, mild accent, strong accent and very strong accent. This corpus is intended to support the study of the underlying characteristics of foreign accent and to enable research, development and evaluation of algorithms for the identification and understanding of accented speech. Some of the files in this corpus are also contained in <a href="http://catalog.ldc.upenn.edu/LDC2005S26" rel="nofollow">CSLU: 22 Languages Corpus, LDC2005S26.</a> </p> <h3>Samples</h3> <p>For an example of the data in this corpus, please listen to this <a href="./desc/addenda/LDC2007S08.wav" rel="nofollow">audio sample</a>. </p> </br>
Portions © 2000-2002 Center for Spoken Language Understanding, Oregon Health & Science University, © 2007 Trustees of the University of Pennsylvania
DOI: 10.35111/3mfr-b569
title: CSLU: Apple Words and Phrases

description: <h3>Introduction</h3> <p>Apple Words and Phrases Version 1.3 contains approximately 69.5 hours of speech from 3008 telephone calls placed on analog and digital phone systems. Apple Computer, Inc. supported the development of this data and also supplied the list of words and phrases collected. Callers responded to questions and repeated a list of phrases as they were prompted.</p> <h3>Data</h3> <p>Subjects calling the analog system (998 callers) were employees of Apple Computer, Inc. and were solicited through interoffice email within the company. Subjects calling the digital system (2010 callers) were responding to USEnet postings or newspaper advertisements placed in several cities across the United States. Each subject called the CSLU data collection system by dialing a toll-free number. The analog data were collected via a Worldport Pod on an Apple Quadra A/V. The digital data were collected with the CSLU T1 digital data collection system.</p> <p>Callers were prompted to answer certain questions including, What is your native language? In which city and state did you spend most of your childhood? What time is it now? What day is today? Callers were also instructed to repeat various comnand and control type phrases, including "play previous message again", "make a meeting for today", "quit", "who is at work", "what is the area code for this state", "hello, what are my messages", "help", "please send a car from the city", "delete my email tomorrow", "read this text", "erase all information", "record extended phonebook", "transfer all calls to home at twelve o'clock", "record urgent message" and "find the operator".</p> <p>Each recorded utterance was listened to by a human verifier to determine if the speaker adequately followed the directions. If an utterance contained extraneous words or excessive noise, it was not included in the corpus.</p> <h3>Samples</h3> <ul> <li><a href="./desc/addenda/LDC2007S13_analog.wav" rel="nofollow">Analog</a></li> <li><a href="./desc/addenda/LDC2007S13_digital.wav" rel="nofollow">Digital</a></li> </ul> </br>
Portions © 2000-2002 Center for Spoken Language Understanding, Oregon Health & Science University, © 2007 Trustees of the University of Pennsylvania
DOI: 10.35111/q5tn-8096
title: CSLU:  Kids` Speech Version 1.1

description: <h3>Introduction</h3><br>
<p>CSLU: Kids' Speech Version 1.1 , Linguistic Data Consortium (LDC) catalog number LDC2007S18 and isbn 1-58563-395-X, is a collection of spontaneous and prompted speech from 1100 children between Kindergarten and Grade 10 in the Forest Grove School District in Oregon. All children -- approximately 100 children at each grade level -- read approximately 60 items from a total list of 319 phonetically-balanced but simple words, sentences or digit strings. Each utterance of spontaneous speech begins with a recitation of the alphabet and contains a monologue of about one minute in duration. This release consists of 1017 files containing approximately 8-10 minutes of speech per speaker. Corresponding word-level transcriptions are also included.</p><br>
<p>This corpus was developed to facilitate research about the characteristics of children's speech at different ages and to train and evaluate recognizers for use in language training and other interactive tasks involving children, including to train recognizers used in language development with deaf children.</p><br>
<h3>Data</h3><br>
<p>Data collection was performed using the <a href="http://www.cslu.ogi.edu/toolkit/" rel="nofollow">CSLU Speech Toolkit</a> and two computers running Windows NT 4.0. Each computer was manned by a CSLU staff member who monitored progress and helped the child with any difficulties. The average time at the computer was 20 minutes, yielding approximately 8-10 minutes of speech digitized at 16 bits and 16kHz using Soundblaster 16 PnP audio cards with head-mounted microphones.</p><br>
<p>The prompted speech, consisting of 200 isolated words and 10 numeric strings, was presented as text appearing below an animated character that produced accurate visible speech synchronized with recorded prompts. A text prompt was also displayed. The child then reproduced the prompted word. Once the prompted speech collection was completed, the experimenter then asked the subject a series of questions designed to elicit spontaneous speech (i.e "Tell me about your favorite movie").</p><br>
<h3>Samples</h3><br>
<p>For an example of the speech in this corpus, please listen to this <a href="desc/addenda/LDC2007S18.wav">sample of spontaneous speech</a>.</p></br>
Portions © 2001-2002 Center for Spoken Language Understanding, Oregon Health & Science University, © 2007 Trustees of the University of Pennsylvania
DOI: 10.35111/xchx-cr96
title: Levantine Arabic Conversational Telephone Speech, Transcripts

description: <h3>Introduction</h3> <p>This database contains 982 Levantine Arabic speakers taking part in spontaneous telephone conversations in Colloquial Levantine Arabic. A total of 985 conversation sides are provided (there are three speakers who each appear in two disctinct conversations). The average duration per side is between 5 and 6 minutes.</p> <p>This corpus was collected and transcribed in 2004 by Appen Pty Ltd, Sydney, Australia.</p> <h3>Samples</h3> <p>For an example of the data in this corpus, please examine this <a href="./desc/addenda/LDC2007T01.jpg" rel="nofollow">screen capture</a> of the transcription data. </p> </br>
Portions © 2007 Trustees of the University of Pennsylvania
DOI: 10.35111/jn2g-zd52
title: English Chinese Translation Treebank v 1.0

description: <h3>Description</h3><br>
<p>This release of English Chinese Translation Treebank v. 1.0 consists of 146,300 words in 325 files of individual news stories from Xinhua News Agency (corresponding to the Xinhua data in Chinese Treebank 5.0 <a href="../../../LDC2005T01">LDC2005T01</a>) that are translated into English, part-of-speech tagged and treebanked. The files were compressed using gzip.</p><br>
<p>The source files for the treebank annotation contain the final updated translation of these files. Translation errors that prevented complete treebank annotation have been corrected. This translation and annotation were completed in October 2004 and supersede any earlier translation.</p><br>
<p>This publication was compiled under <a href="http://www.nsf.gov/" rel="nofollow">National Science Foundation</a> Grant #IIS-0325646.</p><br>
<h3>Samples</h3><br>
<p>For an example of the data in this publication, please view this <a href="desc/addenda/LDC2007T02.jpg" rel="nofollow">sample</a>.</p></br>
Portions © 1994-1998 Xinhua News Agency, © 2004, 2007 Trustees of the University of Pennsylvania
DOI: 10.35111/7gs1-8e36
title: ISI Arabic-English Automatically Extracted Parallel Text

description: <p>This distribution contains a corpus of Arabic-English parallel sentences, which were extracted automatically from two monolingual corpora: Arabic Gigaword Second Edition (LDC2006T02) and English Gigaword Second Edition (LDC2005T12). The data was extracted from news articles published by Xinhua News Agency and Agence France Presse and was obtained using the automatic parallel sentence identification method described in the following publication: Dragos Stefan Munteanu, Daniel Marcu, 2005. Machine Translation Performance by Exploiting Non-parallel Corpora, Computational Linguistics, 31(4):477-504</p><br>
<p>The corpus contains 1,124,609 sentence pairs; the word count on the English side is approximately 31M words. The sentences in the parallel corpus preserve the form and encoding of the texts in the original Gigaword corpora.</p><br>
<p>For each sentence pair in the corpus the authors provide the names of the documents from which the two sentences were extracted, as well as a confidence score (between 0.5 and 1.0), which is indicative of their degree of parallelism. The parallel sentence identification approach is designed to judge sentence pairs in isolation from their contexts, and can therefore find parallel sentences within document pairs which are not parallel. The fact that two documents share several parallel sentences does not necessarily mean the documents are parallel.</p><br>
<p>In order to make this resource useful for research in Machine Translation (MT), the authors made efforts to detect potential overlaps between this data and the standard test and development data sets used by the MT community. The NIST 2002-2005 MT evaluation data sets contain several articles from Xinhua News Agency and Agence France Presse. Sentence pairs in this distribution that have a 7-gram overlap with a sentence pair in a NIST MT evaluation set or sentence pairs coming from documents whose names are similar to those in the NIST MT sets are marked with a negative confidence score.</p><br>
<h3>Samples</h3><br>
<p>For an example of the data in this publication, please examine this <a href="desc/addenda/LCD2007T08_para.jpg" rel="nofollow">image of text data</a>.</p></br>
Portions © 1994-2004 Agence France Presse, © 1995-2004 Xinhua News Agency, © 2005, 2006, 2007 Trustees of the University of Pennsylvania