cometadata/datacite-titles-descriptions-related-identifiers huggingface dataset:
DOI: 10.35111/4w31-m996
title: RST Discourse Treebank

description: <h3>Introduction</h3><br>
<p>Rhetorical Structure Theory (RST) Discourse Treebank was developed by researchers at the Information Sciences Institute (University of Southern California), the US Department of Defense and the Linguistic Data Consortium (LDC). It consists of 385 Wall Street Journal articles from the <a href="http://catalog.ldc.upenn.edu/LDC99T42" rel="nofollow">Penn Treebank</a> annotated with discourse structure in the RST framework along with human-generated extracts and abstracts associated with the source documents.</p><br>
<p>In the RST framework (Mann and Thompson, 1988), a text's discourse structure can be represented as a tree in four aspects: (1) the leaves correspond to text fragments called <em>elementary discourse units</em> (the mininal discourse units); (2) the internal nodes of the tree correspond to contiguous text <em>spans</em>; (3) each node is characterized by its <em>nuclearity</em>, or essential unit of information; and (4) each node is also characterized by a <em>rhetorical relation</em> between two or more non-overlapping, adjacent text spans. </p><br>
<h3>Data</h3><br>
<p>The data in this release is divided into a training set (347 documents) and a test set (38 documents). All annotations were produced using a discourse annotation tool that can be downloaded from <a href="http://www.isi.edu/~marcu/discourse" rel="nofollow">http://www.isi.edu/~marcu/discourse</a>.</p><br>
<p>Human-generated material in the corpus includes (1) long and short abstracts for 30 documents that were intended to convey the essential information and the main topic of the article, respectively; and (2) long, short and informative extracts for 180 documents, some of which were created from scratch and some of which were derived from the humanly-producted abstracts indicated above.</p><br>
<h3>Samples</h3><br>
<p>Please view this <a href="desc/addenda/LDC2002T07.txt">sample</a>.</p><br>
<h3>Updates</h3><br>
<p>There are no updates at this time.</p></br>
Portions © 1987-1989 Dow Jones & Company, Inc., © 1995, 1999, 2002 Trustees of the University of Pennsylvania
DOI: 10.35111/bpbe-jj95
title: Korean English Treebank Annotations

description: <h3>Introduction</h3> <p>This file contains documentation on the Korean English Treebank Annotations, Linguistic Data Consortium (LDC) catalog number LDC2002T26 and ISBN 1-58563-236-8. </p><p>This corpus consists of 33 texts originally written in Korean and translated into English for the purpose of language training in a military setting. The conversations are not authentic dialogues but were constructed for pedagogical purposes. The texts were made available for linguistic research by the Defense Language Institute (DLI). They were delivered on paper to the Institute for Research in Cognitive Science (IRCS) at the University of Pennsylvania, where they were converted to digital form using the KSC 5601 character set encoding (also known as KS X 1001 Wansung). </p><p>Both the Korean and English texts are presented with complete Treebank annotation which was done manually at IRCS, including syntactic constituent bracketing and part-of-speech (POS) tagging. Further documentation about the parsing and POS specifications used in these annotations can be found on the <a href="http://www.cis.upenn.edu/~xtag/koreantag/" rel="nofollow">Korean NLP web site</a>. </p><h3>Data</h3> <p>There are 66 data files: 33 for Korean and 33 for English. The text files mostly contain sets of question and answer sentences. A full, unannotated sentence is presented first, on a single line with an initial semi-colon character ";" -- the first token on such lines (the string preceding the first space character on the line) is a sentence-identifier tag that matches the English and Korean versions of the sentence. The parsed/POS-tagged annotation of the sentence follows on subsequent lines. </p><h3>Updates</h3> <p>There are no updates at this time. </p></br>
Portions (c) 2001-2002 CoGenTex, Inc., Trustees of the University of Pennsylvania
DOI: 10.35111/jcsy-gv04
title: CALLHOME Egyptian Arabic Transcripts Supplement

description: <h3>Introduction</h3><br>
<p>The CALLHOME Egyptian Arabic Transcripts Supplement corpus was produced by Linguistic Data Consortium (LDC), catalog number LDC2002T38 and ISBN 1-58563-244-9.</p><br>
<p>This publication contains transcripts for 20 CALLHOME Egyptian Arabic telephone conversations. These 20 conversations are published as CALLHOME Egyptian Arabic Speech Supplement <a href="http://catalog.ldc.upenn.edu/LDC2002S37" rel="nofollow">LDC2002S37</a>. These conversations had originally been held in reserve for future NIST HUB5 Non-English evaluations, but are being "re-tasked" to provide additional material for general use.</p><br>
<h3>Data</h3><br>
<p>There are 40 data files. Each of the 20 calls has transcripts in two formats: .txt and .scr.</p><br>
<p>The .txt files are transcript files containing the Romanized orthographic forms that were used in the original transcription process. These forms also serve as the head-words in the associated Egyptian Colloquial Lexicon <a href="http://catalog.ldc.upenn.edu/LDC99L22" rel="nofollow">LDC99L22</a>.</p><br>
<p>The .scr files are transcript files rendered in Arabic script orthography, using the ISO 8859-6 character set; these files were derived from the .txt files by replacing each word token with its Arabic script counterpart (which is also provided in the Egyptian Colloquial Arabic Lexicon). These files have been formatted to avoid problems of bi-directional text: line-feed characters are used to separate ASCII content from Arabic script content in each utterance.</p><br>
<h3>Samples</h3><br>
<p>Please follow these links for sample transcripts: <a href="desc/addenda/LDC2002T38.txt" rel="nofollow">txt</a> | <a href="desc/addenda/LDC2002T38_scr.gif" rel="nofollow">scr</a></p><br>
<h3>Updates</h3><br>
<p>There are no updates at this time.</p></br>
Portions © 1996-2002 Trustees of the University of Pennsylvania.
DOI: 10.35111/cdb8-c507
title: 1997 HUB5 Arabic Transcripts

description: <h3>Introduction</h3><br>
<p>The 1997 HUB5 Arabic Transcripts corpus was produced by the Linguistic Data Consortium (LDC), catalog number LDC2002T39 and ISBN 1-58563-245-7.</p><br>
<p>This publication contains transcripts for twenty CALLHOME Egyptian Arabic telephone conversations. These 20 conversations were used in NIST's 1997 HUB5 Non-English evaluation, and are published as 1997 HUB5 Arabic Evaluation <a href="http://catalog.ldc.upenn.edu/LDC2002S22" rel="nofollow">LDC2002S22</a>.</p><br>
<h3>Data</h3><br>
<p>There are 40 data files. Each of the 20 calls has transcripts in two formats: .txt and .scr.</p><br>
<p>The .txt files are transcript files containing the Romanized orthographic forms that were used in the original transcription process. These forms also serve as the head-words in the associated Egyptian Colloquial Arabic Lexicon <a href="http://catalog.ldc.upenn.edu/LDC99L22" rel="nofollow">LDC99L22</a>.</p><br>
<p>The .scr files are transcript files rendered in Arabic script orthography, using the ISO 8859-6 character set; these files were derived from the .txt files by replacing each word token with its Arabic script counterpart (which is also provided in the CALLHOME Arabic Lexicon). These files have been formatted to avoid problems of bi-directional text: line-feed characters are used to separate ASCII content from Arabic script content in each utterance.</p><br>
<h3>Samples</h3><br>
<p>Please follow these links for sample transcripts: <a href="desc/addenda/LDC2002T39.txt" rel="nofollow">txt</a> | <a href="desc/addenda/LDC2002T39_scr.gif" rel="nofollow">scr</a></p><br>
<h3>Updates</h3><br>
<p>There are no updates at this time.</p></br>
Portions © 1996-2002 Trustees of the University of Pennsylvania
DOI: 10.35111/8c95-0c55
title: 2000 HUB5 English Evaluation Transcripts

description: <h3>Introduction</h3><br>
<p>2000 HUB5 English Evaluation Transcripts was developed by the Linguistic Data Consortium (LDC)  and consists of transcripts of 40 English telephone conversations used in the 2000 HUB5 evaluation sponsored by <a href="http://nist.gov/itl/iad/mig/">NIST</a> (National Institute of Standards and Technology). </p><br>
<p>The Hub5 evaluation series focused on conversational speech over the telephone with the particular task of transcribing conversational speech into text. Its goals were to explore promising new areas in the recognition of conversational speech, to develop advanced technology incorporating those ideas and to measure the performance of new technology.</p><br>
<h3>Data</h3><br>
<p>This release contains transcripts in .txt format for the 40 source speech data files used in the evaluation: (1) 20 unreleased telephone conversations from the Swtichboard studies in which recruited speakers were connected through a robot operator to carry on casual conversations about a daily topic announced by the robot operator at the start of the call; and (2) 20 telephone conversations from <a href="http://catalog.ldc.upenn.edu/LDC97S42" rel="nofollow">CALLHOME American English Speech</a> which consists of unscripted telephone conversations between native English speakers.</p><br>
<p>The corresponding speech files are availalbe in 2000 HUB5 English Evaluation Speech (<a href="../../../LDC2002S09">LDC2002S09</a>).</p><br>
<h3>Samples</h3><br>
<p>Please view this <a href="desc/addenda/LDC2002T43.txt">sample</a>.</p><br>
<h3>Updates</h3><br>
<p>There are no updates at this time.</p></br>
Portions © 1997, 2000, 2002 Trustees of the University of Pennsylvania
DOI: 10.35111/z5x4-4x59
title: Grassfields Bantu Fieldwork: Dschang Lexicon

description: <h3>Introduction</h3><br>
<p>Grassfields Bantu Fieldwork: Dschang Lexicon was produced by Linguistic Data Consortium (LDC) catalog number LDC2003L01 and ISBN 1-58563-255-4.</p><br>
<p>The data contains a lexicon of the language <a href="http://www.ethnologue.com/show_language.asp?code=BAN" rel="nofollow">Y&eacute;mba (Bamileke Dschang)</a>, a Bamileke (Grassfields Bantu) language spoken by 300,000+ people in Southwestern Cameroon.</p><br>
<h3>Data</h3><br>
<p>There are two sets of pages in HTML format: one set is 28 pages, the other is 105. In the first set, one page lists every word in the lexicon that begins with a given letter (or letter combination). In the second set, one page lists approximately 25 entries, unless there are fewer than 25 entries beginning with a given letter. The second set is more convenient for use on slower machines.</p><br>
<p>Each entry under a headword has links to two or more sound files. Two speakers were recorded; a given entry generally contains links to utterances of the headword by both speakers, as well as a laryngograph recording of the headword by the second speaker. Nouns may also have links to pronunciations of the plurals, and verbs may have links to pronunciations of the imperative and infinitive forms. These ancillary forms will in general also have laryngograph recordings.</p><br>
<p><strong>Recorded:</strong> May 1997, Dschang, recording studio of SIL Cameroon, Yaound&eacute;</p><br>
<p><strong>Digitized, Labelled and Segmented:</strong> 1997-1998 Phonetics Laboratory, University of Edinburgh</p><br>
<p><strong>Annotated:</strong> 1998-2002 LDC, University of Pennsylvania</p><br>
<p><strong>Sponsorship:</strong></p><br>
<table><br>
<tbody><br>
<tr><br>
<td>SIL Cameroon</td><br>
</tr><br>
<tr><br>
<td>Economic and Social Research Council (UK) Grant R000235540</td><br>
</tr><br>
<tr><br>
<td>National Science Foundation (US) Grant 9983258</td><br>
</tr><br>
<tr><br>
<td>National Science Foundation (US) TalkBank Project Grant BCS-998009, KDI, SBE</td><br>
</tr><br>
<tr><br>
<td>Linguistic Data Consortium</td><br>
</tr><br>
</tbody><br>
</table><br>
<h3>Updates</h3><br>
<p>There are no updates available at this time.</p><br>
<p>Due to a problem with the way a number of images are rendered by the Windows operating system, the release is recommended to be viewed in the Unix operating system, which displays the icons correctly.</p><br>
<h3>Note</h3><br>
<p>The cost of the first 100 copies of this publication (not counting the copies distributed to LDC members) is covered by NSF Grant Number 9983258, and therefore free of charge to qualified researchers; a $30 shipping and handling fee applies. After these first 100 copies are distributed, additional copies will be available under the normal terms and costs.</p></br>
Portions © 2003 Steven Bird, © 2003 Trustees of the University of Pennsylvania
DOI: 10.35111/hpva-ym97
title: Korean Telephone Conversations Lexicon

description: <h3>Introduction</h3><br>
<p>Korean Telephone Conversations Lexicon was produced by Linguistic Data Consortium (LDC) catalog number LDC2003L02 and ISBN 1-58563-265-1.</p><br>
<p>Korean Telephone Conversations Lexicon consists of 25,251 words, and contains separate fields with phonological, morphological, and frequency information for each word.</p><br>
<p>The lexicon covers the tokens occurring in 100 telephone conversations transcribed and published as <a href="../../../LDC2003T08">Korean Telephone Conversations Transcripts</a>. The token coverage is 100%. The corresponding speech is published as<a href="../../../LDC2003S03"> Korean Telephone Conversations Speech</a>.</p><br>
<h3>Data</h3><br>
<p>The lexicon contains five tab-separated information fields:</p><br>
<ol><br>
<li>orthographic form in Hangul (head-word), encoded in the KSC-5601 (Wansung) system</li><br>
<li>orthographic form in Yale romanization</li><br>
<li>pronunciation</li><br>
<li>frequency of the word in Korean Telephone Conversations Transcripts</li><br>
<li>morphological analysis of the word</li><br>
</ol><br>
<p>Please follow this link for a sample page from the lexicon: <a href="desc/addenda/LDC2003L02.txt" rel="nofollow">txt</a> | <a href="desc/addenda/LDC2003L02.gif" rel="nofollow">gif</a>.</p><br>
<h3>Updates</h3><br>
<p>There are no updates available at this time.</p></br>
Portions © 2003 Trustees of the University of Pennsylvania.
DOI: 10.35111/3aqw-nc22
title: 2001 Communicator Evaluation

description: <h3>Introduction</h3> <p>2001 Communicator Evaluation was produced by Linguistic Data Consortium (LDC) catalog number LDC2003S01 and ISBN 1-58563-259-7. </p><p>The original goals of the Communicator program were to support the creation of speech-enabled interfaces that scale gracefully across modalities, from speech-only to interfaces that include graphics, maps, pointing and gesture. The original vision of the Communicator systems included the ability of a user, during one 10-minute session, to plan a three-leg trip, with the three flights/legs on three different days, with rental car and hotel in each of the two "away" cities, plus dictating/sending a voice-mail message. </p><p>The actual research that led to the data collections in 2000 and 2001 explored ways to construct better spoken-dialogue systems, with which users interact via speech-alone to perform relatively complex tasks such as travel planning. During 2000 and 2001 two large data sets were collected, in which users used the Communicator systems built by the research groups to do travel planning. The researchers improved their systems intensively during the ten months between the two data collections. This distribution consists of all the data from the 2001 collection. </p><p>All the Communicator implementations used a common software architecture, called Galaxy-II, which was designed by a research team at MIT and adapted for Communicator in collaboration with a team at MITRE. The architecture supported detailed logging of the interaction between users and the systems. </p><p>For possible updated information about the Communicator project and the data distributions, please visit the NIST <a href="http://www.nist.gov/speech/tests/communicator/index.htm" rel="nofollow">website</a>. </p><h3>Data</h3> <p>The following sites participated in this project: ATT, BBN, Carnegie Mellon University, IBM, Lucent Bell Labs, MIT, SRI and University of Colorado at Boulder. </p><p>All audio files have been converted into SPHERE format; there are 53394 sphere files, totalling approximately 102 hours of audio. All sphere files are one-channel, 8KHz, but the sample coding and format, while consistent for all files belonging to one site, is not consistent across sites (for example, some sites provided pcm, while others provided u-law data). The documentation included in this distribution is replicated exactly as received from NIST and from the participating sites. </p><h3>Updates</h3> <p>There are no updates available at this time. </p> <p>NIST and DARPA have an Interagency Agreement by which funds are transferred to NIST. The funds to support NIST's DARPA Communicator Role were transferred under ARPA Order No. G270.</p></br>
Portions © 2003, Trustees of the University of Pennsylvania <br><br>NIST and DARPA have an Interagency Agreement by which funds are transferred to NIST. The funds to support NIST's DARPA Communicator Role were transferred under ARPA Order No. G270.
DOI: 10.35111/req3-vx07
title: Grassfields Bantu Fieldwork: Dschang Tone Paradigms

description: <h3>Introduction</h3><br>
<p>Grassfields Bantu Fieldwork: Dschang Tone Paradigms was produced by Linguistic Data Consortium (LDC) catalog number LDC2003S02 and ISBN 1-58563-254-6.</p><br>
<p>The data contains tone paradigms of the language <a href="http://www.ethnologue.com/show_language.asp?code=ybb" rel="nofollow">Y&eacute;mba (Bamileke Dschang)</a>, a Bamileke (Grassfields Bantu) language spoken by 300,000+ people in Southwestern Cameroon.</p><br>
<h3>Data</h3><br>
<p>There are 45 paradigm pages in html format. Each page lists 32 utterances, varying across subject, verb, and object. Each utterance has one to three links to recordings in .wav format, as well as a laryngograph recording (also in .wav format). Phonetic transcription has been done for every utterance tonological transcription has been done for a little more than half.</p><br>
<p><strong>Recorded:</strong> June 1997, Dschang, Western Province, and recording studio of SIL Cameroon, Yaound&eacute;</p><br>
<p><strong>Digitized, Labelled and Segmented:</strong> 1997-1998 Phonetics Laboratory, University of Edinburgh</p><br>
<p><strong>Transcribed and Annotated:</strong> 1998-2002 LDC, University of Pennsylvania</p><br>
<p><strong>Sponsorship:</strong></p><br>
<table><br>
<tbody><br>
<tr><br>
<td>SIL Cameroon</td><br>
</tr><br>
<tr><br>
<td>Economic and Social Research Council (UK) Grant R000235540</td><br>
</tr><br>
<tr><br>
<td>National Science Foundation (US) Grant 9983258</td><br>
</tr><br>
<tr><br>
<td>National Science Foundation (US) TalkBank Project Grant BCS-998009, KDI, SBE</td><br>
</tr><br>
<tr><br>
<td>Linguistic Data Consortium</td><br>
</tr><br>
</tbody><br>
</table><br>
<h3>Updates</h3><br>
<p>There are no updates available at this time.</p><br>
<p>Due to a problem with the way a number of images are rendered by the Windows operating system, the CD is recommended to be viewed in the Unix operating system, which displays the icons correctly.</p><br>
<h3>Note</h3><br>
<p>The cost of the first 100 copies of this publication (not counting the copies distributed to LDC members) is covered by NSF Grant Number 9983258, and therefore free of charge to qualified researchers a $30 shipping and handling fee applies. After these first 100 copies are distributed, additional copies will be available under the normal terms and costs.</p></br>
Portions © 2003 Steven Bird, © 2003 Trustees of the University of Pennsylvania
DOI: 10.35111/6bre-jj92
title: Korean Telephone Conversations Speech

description: <h3>Introduction</h3><br>
<p>Korean Telephone Conversations Speech was produced by Linguistic Data Consortium (LDC) catalog number LDC2003S03 and ISBN 1-58563-263-5.</p><br>
<p>The telephone conversations in this corpus were originally recorded as part of the CALLFRIEND project. The CALLFRIEND Korean telephone speech was collected by Linguistic Data Consortium primarily in support of the Language Identification (LID) project, sponsored by the U.S. Department of Defense. The calls were later transcribed for use in other projects.</p><br>
<p>This publication consists of 100 telephone conversations, 49 of which were published in 1996 as <a rel="nofollow">CALLFRIEND Korean</a>, while the rest of 51 are previously unexposed calls.</p><br>
<p>All 100 conversations have been transcribed and are published as <a href="../../../LDC2003T08">Korean Telephone Conversations Transcripts</a>.</p><br>
<p>The recorded conversations are between native speakers of Korean and last up to 30 minutes, of which the transcribed speech covers between 15 to 18 minutes. All speakers were aware that they were being recorded. They were given no guidelines concerning what they should talk about. Once a caller was recruited to participate, he/she was given a free choice of whom to call. Most participants called family members or close friends. All calls originated in either the United States or Canada.</p><br>
<h3>Data</h3><br>
<p>There are 100 speech files, totalling approximately 44 hours of audio. All speech files are in sphere format (shorten-compressed), recorded in two-channel ulaw with a sampling rate of 8 KHz.</p><br>
<h3>Samples</h3><br>
<p>Please listen to this <a href="desc/addenda/LDC2003S03.sph">audio sample</a>.</p><br>
<h3>Updates</h3><br>
<p>There are no updates available at this time.</p></br>
Portions © 2003 Trustees of the University of Pennsylvania.