cometadata/datacite-titles-descriptions-related-identifiers huggingface dataset:
DOI: 10.35111/wqck-a633
title: CALLHOME Spanish Dialogue Act Annotation

description: <h3>Introduction</h3> <p>The CALLHOME Spanish Dialogue Act Annotation Corpus, Linguistic Data Consortium (LDC) catalog number LDC2001T61 and ISBN 1-58563-197-3, was developed under Project CLARITY. The goal of CLARITY was to glean discourse information from unrestricted conversational speech using shallow, corpus-based analysis. The annotation was carried out at <a href="http://www.is.cs.cmu.edu" rel="nofollow">Interactive Systems Labs</a> at Carnegie Mellon University. </p><h3>Data</h3> <p>This publication used a three-level coding scheme to manually tag the <a href="http://catalog.ldc.upenn.edu/LDC96T17" rel="nofollow">LDC CALLHOME Spanish Transcripts</a>. The three levels of the coding scheme are: </p><ol> <li>a dialogue act level consisting of a tag set extended from DAMSL and Switchboard; </li> <li>a dialogue game level featuring short sequences of dialogue acts </li> <li>a genre level similiar to topical segments. All available (120) dialogues have been annotated. </li> </ol><p>Dialogue games are short sequences of dialogue acts such as question/answer pairs. Genres can be storytelling, discussion, planning, etc. Segmentation takes topics into account as well. Genres, games, and dialogue acts are annotated by type. Genres are additionally annotated for activities and topics (on a 0-5 scale), for the central object or person being discussed (who or what category), and contain a short synopsis of the segment. </p><p>All available 120 CALLHOME Spanish dialogues have been annotated. The dialogue act annotation scheme is a further development of the SwitchBoard DAMSL tagset. Dialogue games are short sequences of dialogue acts such as question/answer pairs. Genres can be storytelling, discussion, planning etc. and the segmentation takes topic into account as well. Genres, games and dialogue acts are annotated for their type. Genres are additionally annotated for activities and topics (on a 0-5 scale), for the central object or person being discussed (who or what category) and contain a short gist of the segment. </p><p>An example of the tagging from one conversation is presented below. </p> &lt;?xml version="1.0" encoding="iso-8859-1"?&gt; Sm, eso es para eso, de seguro. No importa. No importa. Bueno aqum, la Zaida esta estudiando tambiin en la universidad con la Liana. Y qui estudia, mama, qui estan estudiando. [background speech] Estan estudiando Sociales. Ciencias Sociales. Ah, para maes- para maestra de Sociales. Sm <h3>Updates</h3> There are no updates at this time.</br>
DOI: 10.35111/4sr4-3r57
title: CETEMpublico

description: <h3>Introduction</h3><br>
<p>CETEMPublico Version 1.7 (Corpus de Extractos de Textos Electronicos MCT/Publico), produced by the Linguistic Data Consortium (LDC) as catalog number LDC2001T62 with ISBN 1-58563-206-6, is a corpus of newspaper texts from the Portuguese daily newspaper Publico, compiled for purposes of research and development in natural language processing (NLP) by the Computational Processing of Portuguese Project, under an agreement between Publico and the Portuguese Ministry of Science and Technology (MCT).</p><br>
<h3>Data</h3><br>
<p>The corpus includes the text of approximately 2,600 editions of Publico, produced between 1991 and 1998, and amounting to approximately 180 million words. CETEMPublico Version 1.7 contains 1,504,258 extracts (CETEMPublico Version 1.0 had 1,567,625). Version 1.7 was created in Oslo on August 6, 2001 and uses SGML tagging. The corpus is in 196 compressed text files, with names in the form cetemXXX.gz, from cetem001.gz to cetem196.gz.</p><br>
<p>This corpus was designed to assist researchers who develop computer programs processing the Portuguese language and who would need raw material for their work. In addition, the authors wished for the corpus to be useful to everyone who studies the Portuguese language and wishes to verify their hypotheses in previously organized text material. The online and the CQP versions are meant for such users, who are, in any case, also welcome to get it on CD in order to process the corpus locally, possibly by means of the corpus processing system of their choice.</p><br>
<p>More detailed information is available at <a href="http://www.linguateca.pt/cetempublico/" rel="nofollow">http://www.linguateca.pt/cetempublico</a>.</p><br>
<h3>Updates</h3><br>
<p>There are no updates at this time.</p></br>
DOI: 10.35111/7t9x-0z23
title: Chinese-English Translation Lexicon Version 3.0

description: <h3>Introduction</h3><br>
<p>2002 Chinese-English Translation Lexicon Version 3.0 was developed by the Linguistic Data Consortium (LDC).</p><br>
<p>In 1999, responding to urgent demand for a Chinese-English bilingual wordlist to support various projects, LDC quickly solicited entries from both in-house and Internet resources and compiled two versions of Chinese-English wordlists, "ldc_ce_dict.1.0.gb" (henceforth Version 1) and "ldc_ce_dict.2.0.txt" (henceforth Version 2).</p><br>
<p>Version 1, with 24,298 entries, was relatively small and its coverage was unbalanced. Version 2, created as an experiment, is impractical for translingual information processing. Many of its entries were created by reversing source and target language fields in various English-to-Chinese wordlists; as a result many entries are not really words. The increasing demand for richer lexical resources led to the present release, "ldc_cedict.gb.Version 3" (henceforth Version 3).</p><br>
<h3>Data</h3><br>
<p>What's New in Version 3</p><br>
<p>The total number of Chinese headwords in this release is 54,170.</p><br>
<p>In terms of coverage, Version 3 is a superset of Version 1 and the LDC's Mandarin pronunciation lexicon (Version 3/Version 4). The pronunciation lexicon has a total of 44,404 entries, or 43,968 unique Chinese character strings (i.e. with pronunciation removed). There are still 553 entries from the pronunciation lexicon not found in Version 3. We were unable to provide accurate translations for these head words for various reasons: they may be very technical; they don't make sense unless their source is re-examined; they may have segmentation errors; or they may be rare words for which appropriate translations could not be found due to limited time and resources.</p><br>
<p>Version 3 also left out less than 40 entries from Version 1. Most of these are rare single-character words whose translations cannot be verified for accuracy.</p><br>
<h3>Format</h3><br>
<p>There is one data file, the lexicon itself. Within the lexicon, each entry is in this format:</p><br>
<p>head_word_in_Chinese_characters /gloss 1/gloss 2/.../gloss n/ For example:</p><br>
<p>&ordm;&ordm;&Oacute;&iuml; /Chinese language/Chinese/ &Oacute;&cent;&Icirc;&Auml; /English language/English/ (A Chinese-capable browser is needed to see this properly. You may need to change your browser's character set to see Simplified Chinese characters.)</p><br>
<h3>Updates</h3><br>
<p>There are no updates at this time.</p></br>
Portions © 2002 Trustees of the University of Pennsylvania
DOI: 10.35111/7vzm-mb15
title: Buckwalter Arabic Morphological Analyzer Version 1.0

description: <h3>Introduction</h3><br>
<p>Buckwalter Arabic Morphological Analyzer Version 1.0 is used for annotating Arabic text with part of speech tags.</p><br>
<h3>Data</h3><br>
<p>The data consists primarily of three Arabic-English lexicon files: prefixes (299 entries), suffixes (618 entries), and stems (82,158 entries representing 38,600 lemmas). The lexicons are supplemented by three morphological compatibility tables used for controlling prefix-stem combinations (1,648 entries), stem-suffix combinations (1,285 entries), and prefix-suffix combinations (598 entries). The actual code for morphology analysis and POS tagging is contained in a Perl script. The documentation consists of a readme file with a description of the lexicon files, the morphological compatibility tables, the morphology analysis algorithm, a summary of stem morphological categories, and a table with the author's Arabic transliteration system.</p><br>
<h3>Updates</h3><br>
<p>There has been a case mismatch in the manner by which six files were named in the data, compared with their names in the documentation and the script, which caused the analyzer to crash on case sensitive systems. This problem has been remedied and you can now download the fixed version of the analyzer.</p><br>
<h3>Licensing</h3><br>
<p>Buckwalter Arabic Morphological Analyzer Version 1.0  is released under the <a href="https://www.gnu.org/licenses/old-licenses/gpl-2.0-faq.html" rel="nofollow"> GNU General Public License version 2</a>. Organizations interested in licensing the lexicon and/or morphological analyzer for commercial use should contact: QAMUS LLC 448 South 48th St. Philadelphia, PA 19143 ATTN: Tim Buckwalter email: info@qamus.org</p><br>
<h3>Note</h3><br>
<p>This corpus is free of charge as a web download distribution; a request must be submitted to ldc@ldc.upenn.edu to obtain the data. Note that there is a $100 charge if requested on a CD-ROM.</p></br>
Portions © 2002 QAMUS LLC (www.qamus.org), © 2002 Trustees of the University of Pennsylvania
DOI: 10.35111/d6h3-pf89
title: Translanguage English Database (TED) Speech

description: <h3>Introduction</h3><br>
<p>Translanguage English Database (TED) Speech consists of recordings of presentations made  by native English and non-native English speakers at the <a href="http://dblp.uni-trier.de/db/conf/interspeech/eurospeech1993.html">Third European Conference on Speech Communication and Technology, EUROSPEECH 1993</a> in Berlin, Germany. This is a joint publication with the <a href="http://www.elra.info/" rel="nofollow">European Language Resources Association (ELRA)</a> sponsored in part by <a href="http://www.nsf.gov/" rel="nofollow">National Science Foundation</a> Grant No. IIS-9982201. The data set is released by ELRA as TED Translanguage English Database (<a href="http://catalog.elra.info/product_info.php?products_id=743">ELRA-S0031</a>). </p><br>
<h3>Data</h3><br>
<p>The audio recordings contain 188 speakers presenting academic papers for approximately 15 minutes each. Transcripts for 39 of the recordings are available in Translanguage English Database (TED) Transcripts <a href="http://catalog.ldc.upenn.edu/LDC2002T03" rel="nofollow">LDC2002T03</a> and in Translanguage English Database (TED) Transcripts database <a href="http://catalog.elra.info/product_info.php?products_id=637">ELRA-S0120</a>.</p><br>
<h3>Samples</h3><br>
<p>Please listen to this <a href="desc/addenda/LDC2002S04.sph">audio sample</a>.</p><br>
<h3>Updates</h3><br>
<p>There are no updates available at this time.</p></br>
Portions © 1993-2002 University of Munich, LIMSI-CNRS, ELRA, © 2002 Trustees of the University of Pennsylvania
DOI: 10.35111/ydsv-hw57
title: Switchboard-2 Phase III Audio

description: <h3>Introduction</h3> <p>The Switchboard-2 Phase III Audio corpus was produced by the Linguistic Data Consortium; catalog number LDC2002S06 and ISBN number 1-58563-222-8. This release contains speech data files ONLY, along with documentation describing speaker information (sex, age, education, city and state where raised), call information (date, time, call duration, Personal Identification Numbers, topic), and audit information (channel quality, background noise). The data files are not compressed. </p><p>The Switchboard-2 Phase III collection was focused primarily in the American South. The collection commenced on October 21, 1997 and was completed on January 1, 1998. The project's goal was to target native speakers of English in the American South, balanced by gender, to participate in (10+) five to six minute conversations on a variety of telephone (land line) handsets. </p><h3>Data</h3> <p>The speech data was collected for research, development, and evaluation of automatic systems for speech-to-text conversion, talker identification, language identification and speech signal detection purposes. </p><p>During the collection period, the LDC collected a total of 2,728 calls, or 5,456 sides, from 640 participants (292 Male, 348 Female), under varied environmental conditions. </p><p>Each speech file consists of a 1,024-byte ASCII-formatted Sphere header, followed by two-channel interleaved mu-law sample data. The mu-law samples represent the actual digital data transmission from the telephone service provider (MCI), as captured separately for each side of the telephone conversation by the LDC's telephone collection platform. The header also indicates the caller_pin, callee_pin, topic_id. </p><p>The speech files are named according to the following pattern: </p><p>sw_NNNNN.sph </p><p>where the five-digit string "NNNNN" represents the conversation-id; this string is used to identify all speech files and to identify the calls in the associated data base tables that provide information about the calls and participants (i.e. callstat.tbl, master.tbl). </p><p>Other documentation files available on the publication are: </p><table> <tr> <td>0readme.1st</td> <td>Field information for all database tables</td> </tr> <tr> <td>swb_callaudit.tbl</td> <td>Audit results for each channel</td> </tr> <tr> <td>swb_callaudit.txt</td> <td>Document describing audit table</td> </tr> <tr> <td>swb_callstats.tbl</td> <td>Information about recorded calls</td> </tr> <tr> <td>swb_callstats.txt</td> <td>Document describing callstats table</td> </tr> <tr> <td>swb_callsubjects.tbl</td> <td>Demographic information</td> </tr> <tr> <td>swb_callsubjects.txt</td> <td>Document describing callsubjects table</td> </tr> <tr> <td>topics.txt</td> <td>List of proposed call topics</td> </tr> </table><p>There are a total of 2,657 data files (=~ 222 hours of audio) </p><h3>Updates</h3> <p>No updates are available at this time. </p> </br>
Portions © 1997-2002 Trustees of the University of Pennsylvania
DOI: 10.35111/p7pz-x179
title: 2000 HUB5 English Evaluation Speech

description: <h3>Introduction</h3><br>
<p>2000 HUB5 English Evaluation was developed by the Linguistic Data Consortium (LDC) and consists of approximately 11 hours of English conversational telephone speech used in the 2000 HUB5 evaluation sponsored by <a href="http://nist.gov/itl/iad/mig/">NIST</a> (National Institute of Standards and Technology). </p><br>
<p>The Hub5 evaluation series focused on conversational speech over the telephone with the particular task of transcribing conversational speech into text. Its goals were to explore promising new areas in the recognition of conversational speech, to develop advanced technology incorporating those ideas and to measure the performance of new technology.</p><br>
<h3>Data</h3><br>
<p>The source data consists of conversational telephone speech collected by LDC: (1) 20 unreleased telephone conversations from the Swtichboard studies in which recruited speakers were connected through a robot operator to carry on casual conversations about a daily topic announced by the robot operator at the start of the call; and (2) 20 telephone conversations from <a href="http://catalog.ldc.upenn.edu/LDC97S42" rel="nofollow">CALLHOME American English Speech</a> which consists of unscripted telephone conversations between native English speakers.</p><br>
<p>The audio files are two channel interleaved mulaw in sphere format. The sphere headers have been modified from the original evaluation data by the addition of sample checksums to the CALLHOME data files. A documentation table contains information on the speech segments.</p><br>
<p>Corresponding transcripts are available in 2000 HUB5 English Evaluation Transcripts (<a href="../../../LDC2002T43">LDC2002T43</a>).</p><br>
<h3>Samples</h3><br>
<p>Please listen to this <a href="desc/addenda/LDC2002S09.wav">audio sample</a>.</p><br>
<h3>Updates</h3><br>
<p>There are no updates at this time.</p></br>
Portions © 1997, 2000, 2002 Trustees of the University of Pennsylvania
DOI: 10.35111/py2a-6706
title: 1998 HUB5 English Evaluation

description: <h3>Introduction</h3><br>
<p>1998 HUB5 English Evaluation was developed by the Linguistic Data Consortium (LDC) and consists of English conversational telephone speech used in the 1998 HUB5 evaluation sponsored by <a href="http://nist.gov/itl/iad/mig/">NIST</a> (National Institute of Standards and Technology). </p><br>
<p>The Hub5 evaluation series focused on conversational speech over the telephone with the particular task of transcribing conversational speech into text. Its goals were to explore promising new areas in the recognition of conversational speech, to develop advanced technology incorporating those ideas and to measure the performance of new technology. Further information about the evaluation can be found on the <a href="http://www.itl.nist.gov/iad/mig/tests/ctr/1998/">NIST HUB5 website</a> and in The 1998 HUB-5E Evaluation Plan for Recognition of Conversational Speech over the Telephone in English, included in this release.</p><br>
<h3>Data</h3><br>
<p>The source data consists of conversational telephone speech collected by LDC: (1) 20 telephone conversations from Swtichboard-2 Phase 1 (<a href="../../../LDC98S75">LDC98S75</a>) in which recruited speakers were connected through a robot operator to carry on casual conversations about a daily topic announced by the robot operator at the start of the call; and (2) 20 telephone conversations from <a href="http://catalog.ldc.upenn.edu/LDC97S42" rel="nofollow">CALLHOME American English Speech</a> which consists of unscripted telephone conversations between native English speakers.</p><br>
<p>The audio files are two channel interleaved mulaw in sphere format. The sphere headers have been modified from the original evaluation data by the addition of sample checksums to the CALLHOME data files.</p><br>
<p>Corresponding transcripts are available in 1998 HUB5 English Transcripts (<a href="../../../LDC2003T02">LDC2003T02</a>).</p><br>
<h3>Samples</h3><br>
<p>Please listen to this <a href="desc/addenda/LDC2002S10.wav">audio sample</a>.</p><br>
<h3>Updates</h3><br>
<p>There are no updates at this time.</p></br>
Portions © 1997 Trustees of the University of Pennsylvania
DOI: 10.35111/qfs3-p755
title: 1997 HUB4 English Evaluation Speech and Transcripts

description: <h3>Introduction</h3> <p>The 1997 HUB4 English Evaluation, Linguistic Data Consortium (LDC) catalog number LDC2002S11 and ISBN 1-58563-227-9, is part of an ongoing series of periodic evaluations conducted by NIST. These evaluations provide an important contribution to the direction of research efforts and the calibration of technical capabilities. They are intended to be of interest to all researchers working on the general problem of conversational speech recognition. To this end, the evaluation was designed to be simple, to focus on core speech technology issues, to be fully supported, and to be accessible. </p><p>The purpose of this evaluation is to foster research on the problem of accurately transcribing broadcast news speech and to measure objectively the state of the art. </p><p>Additional documentation is available at the 1997 NIST Evaluation Plan for Broadcast News <a href="http://www.nist.gov/speech/tests/bnr/hub4e_97/hub4e_97.htm" rel="nofollow">webpage</a>. </p><h3>Data</h3> <p>This year the entire test set is contained in a single waveform file. The SPHERE-formatted waveforem file h4e_97.sph is located in the h4e_evl directory of this CD-ROM. The waveform file contains 334 Mbytes of sphere data, which represents approximately three hours of concatenated radio and television broadcast news stories. The transcript file contains a rough number of 30,600 total tokens and 4,800 unique tokens. </p><h3>Updates</h3> <p>There are no updates at this time. </p> <h3>Additional Copyright Information and Disclaimers</h3> C-SPAN: Washington Journal USC : Radio Marketplace ABC : Prime Time CNN : Morning News CNN : World View NPR : Morning Edition PRI : The World<p>"NPR Programs or excerpts of Programs contained herein are from 1996 broadcasts of "All Things Considered(R)" and "Morning Edition(R)" and are copyrighted works owned by National Public Radio, Inc. The Programs are used with permission of National Public Radio, Inc. Any unauthorized duplication or other use is strictly prohibited." </p><p>"RESTRICTED RIGHTS LEGEND: INFORMATION FROM THE USC PROGRAM 'MARKETPLACE' CONTAINED HEREIN IS THE PROPERTY OF USC RADIO AND THE UNIVERSITY OF SOUTHERN CALIFORNIA AND IS PROTECTED BY COPYRIGHT. USE, DUPLICATION OR DISCLOSURE BY YOU IS SUBJECT TO THE RESTRICTIONS SET FORTH IN THE USER AGREEMENT AND ATTACHED TO THE COMPUTER READABLE MEDIA PROVIDED TO YOU BY THE LINGUISTIC DATA CONSORTIUM OF THE UNIVERSITY OF PENNSYLVANIA. COPYRIGHT 1996 UNIVERSITY OF SOUTHERN CALIFORNIA, AND IS DSITRIBUTED TO PUBLIC RADIO STATIONS NATIONWIDE BY PRI-PUBLIC RADIO INTERNATIONAL. MARKETPLACE IS MADE POSSIBLE BY GE, THE CORPORATION FOR PUBLIC RADIO, AND PUBLIC RADIO STATIONS NATIONWIDE."</p></br>
 Portions (c) 1997 Trustees of the University of Pennsylvania, (c) 1996 C-SPAN, (c) 1996 American Broadcasting Company, (c) 1996 USC Radio of the University of Southern California, (c) 1996 National Public Radio, Inc., (c) 1996 Public Radio International, (c) 1996 Cable News Network, Inc. All Rights Reserved.
DOI: 10.35111/9ydy-sh84
title: 2001 HUB5 Mandarin Evaluation

description: <h3>Introduction</h3><br>
<p>2001 HUB5 Mandarin Evaluation is part of an ongoing series of periodic evaluations conducted by NIST. These evaluations provide an important contribution to the direction of research efforts and the calibration of technical capabilities. They are intended to be of interest to all researchers working on the general problem of conversational speech recognition. To this end the evaluation was designed to be simple, to focus on core speech technology issues, to be fully supported, and to be accessible.</p><br>
<p>The evaluation was held from February 21 - March 12, 2001. The systems were to produce character-level transcripts and character-level confidence scores for the complete set of evaluation test material.</p><br>
<p>Additional information about the evaluation is available from this NIST <a href="https://www.nist.gov/itl/iad/mig/speaker-recognition">website</a>.</p><br>
<h3>Data</h3><br>
<p>The test data comes from unexposed Mandarin CALLHOME Conversations, stored in sphere format. There are 20 sphere files encoded in two-channel interleaved mulaw for a total of 441,990,656 bytes (421 Mbytes) or eight hours of sphere data. These conversations were transcribed and time-marked by speaker turn, by the LDC.</p><br>
<p>An included documentation table contains information on the speech segments to be processed as follows: ...</p><br>
<h3>Updates</h3><br>
<p>There are no updates at this time.</p></br>
Portions © 1997-2002 Trustees of the University of Pennsylvania