cometadata/datacite-titles-descriptions-related-identifiers huggingface dataset:
DOI: 10.35111/d2fq-1e93
title: Korean Telephone Conversations Complete Set

description: <h3>Introduction</h3> <p>Korean Telephone Conversations Complete Set was produced by Linguistic Data Consortium (LDC) catalog number LDC2003P01 and ISBN 1-58563-267-8. </p><p>The complete set of Korean Telephone Conversations consists of the following: </p><ol> <li> <a rel="nofollow">Korean Telephone Conversations Speech</a> </li> <li> <a rel="nofollow">Korean Telephone Conversations Transcripts</a> </li> <li> <a rel="nofollow">Korean Telephone Conversations Lexicon</a> </li> </ol><p>The Korean telephone conversations were originally recorded as part of the Callfriend project. The Callfriend Korean telephone speech was collected by Linguistic Data Consortium primarily in support of the Language Identification (LID) project, sponsored by the U.S. Department of Defense. The calls were later transcribed for use in other projects. </p><p><a rel="nofollow">Korean Telephone Conversations Speech</a> consists of 100 telephone conversations, 49 of which were published in 1996 as <a rel="nofollow">CALLFRIEND Korean</a>, while the remaining 51 are previously unexposed calls. </p><p>The recorded conversations are between native speakers of Korean and last up to 30 minutes, of which the transcribed speech covers between 15 and 18 minutes. All speakers were aware that they were being recorded. They were given no guidelines concerning what they should talk about. Once a caller was recruited to participate, he/she was given a free choice of whom to call. Most participants called family members or close friends. All calls originated in either the United States or Canada. </p><p><a rel="nofollow">Korean Telephone Conversations Transcripts</a> consists of 100 text files, totalling approximately 190K words and 25K unique words. </p><p>All files are in Korean orthography: orthographic Korean characters are in Hangul, encoded in KSC5601 (Wansung) system. </p><p>Please follow this link for a sample transcript: <a href="desc/addenda/LDC2003T08.txt" rel="nofollow">txt</a> | <a href="desc/addenda/LDC2003T08.gif" rel="nofollow">gif</a>. </p><p><a rel="nofollow">Korean Telephone Conversations Lexicon</a> covers the tokens occurring in the Korean Telephone Conversations Transcripts. </p><p>The lexicon contains five tab-separated information fields: </p><ol> <li>orthographic form in Hangul (head-word), encoded in the KSC-5601 (Wansung) system </li> <li>orthographic form in Yale romanization </li> <li>pronunciation </li> <li>frequency of the word in Korean Telephone Conversations Transcripts </li> <li>morphological analysis of the word </li> </ol><p>Please follow this link for a sample page from the lexicon: <a href="desc/addenda/LDC2003L02.txt" rel="nofollow">txt</a> | <a href="desc/addenda/LDC2003L02.gif" rel="nofollow">gif</a>. </p> </br>
Portions © 2003 Trustees of the University of Pennsylvania.
DOI: 10.35111/naha-fq64
title: 2001 HUB5 Mandarin Transcripts

description: <h3>Introduction</h3><br>
<p>2001 HUB5 Mandarin Transcripts was developed by the Linguistic Data Consortium (LDC).</p><br>
<p>This publication contains transcripts for twenty CALLHOME Mandarin telephone conversations. These twenty conversations were used in NIST's 2001 HUB5 Non-English evaluation, and are published as 2001 HUB5 Mandarin Evaluation (<a href="../../../LDC2002S12">LDC2002S12</a>).</p><br>
<h3>Data</h3><br>
<p>There are 20 data files in .txt format.</p><br>
<p>The .txt files are transcript files rendered in Mandarin script orthography, containing the orthographic forms that were used in the original transcription process. These forms also serve as the head-words in the associated CALLHOME Mandarin Lexicon (<a href="http://catalog.ldc.upenn.edu/LDC96L15" rel="nofollow">LDC96L15</a>).</p><br>
<p>Please follow these links for a sample transcript: <a href="desc/addenda/LDC2003T01.txt" rel="nofollow">Mandarin script</a> | <a href="desc/addenda/LDC2003T01.gif" rel="nofollow">GIF format</a>.</p><br>
<h3>Updates</h3><br>
<p>There are no updates at this time.</p></br>
Portions © 2003 Trustees of the University of Pennsylvania.
DOI: 10.35111/qxsf-4869
title: 1998 HUB5 English Transcripts

description: <h3>Introduction</h3><br>
<p>1998 HUB5 English Transcripts was developed by the Linguistic Data Consortium and consists of transcripts of 40 English telephone conversations used in the 1998 HUB5 evaluation sponsored by <a href="http://nist.gov/itl/iad/mig/">NIST</a> (National Institute of Standards and Technology). </p><br>
<p>The Hub5 evaluation series focused on conversational speech over the telephone with the particular task of transcribing conversational speech into text. Its goals were to explore promising new areas in the recognition of conversational speech, to develop advanced technology incorporating those ideas and to measure the performance of new technology. Further information about the evaluation can be found on the <a href="http://www.itl.nist.gov/iad/mig/tests/ctr/1998/">NIST HUB5 website</a>.</p><br>
<h3>Data</h3><br>
<p>This release contains transcripts in .txt format for the 40 source speech data files used in the evaluation: (1) 20 telephone conversations from Swtichboard-2 Phase 1 (<a href="../../../LDC98S75">LDC98S75</a>) in which recruited speakers were connected through a robot operator to carry on casual conversations about a daily topic announced by the robot operator at the start of the call; and (2) 20 telephone conversations from <a href="http://catalog.ldc.upenn.edu/LDC97S42" rel="nofollow">CALLHOME American English Speech</a> which consists of unscripted telephone conversations between native English speakers.</p><br>
<p>The corresponding speech data is released as 1998 HUB5 English Evaluation (<a href="http://catalog.ldc.upenn.edu/LDC2002S10" rel="nofollow">LDC2002S10</a>).</p><br>
<h3>Sample</h3><br>
<p>Please follow this link for a sample transcript <a href="desc/addenda/LDC2003T02.txt" rel="nofollow">example</a>.</p><br>
<h3>Updates</h3><br>
<p>There are no updates at this time.</p></br>
Portions © 1996, 1997, 2003 Trustees of the University of Pennsylvania
DOI: 10.35111/cs3c-x004
title: 1997 HUB5 German Transcripts

description: <h3>Introduction</h3> <p>The 1997 HUB5 German Transcripts corpus was produced by the Linguistic Data Consortium (LDC), catalog number LDC2003T03 and ISBN 1-58563-247-3. </p><p>This publication contains transcripts for 20 CALLHOME German telephone conversations. These twenty conversations were used in NIST's 1997 HUB5 Non-English evaluation, and are published as 1997 HUB5 German Evaluation (<a href="http://catalog.ldc.upenn.edu/LDC2002S24" rel="nofollow">LDC2002S24</a>). </p><h3>Data</h3> <p>There are 20 data files in .txt format. </p><p>The .txt files are transcript files containing the orthographic forms that were used in the original transcription process. These forms also serve as the head-words in the associated Callhome German Lexicon (<a href="http://catalog.ldc.upenn.edu/LDC97L18" rel="nofollow">LDC97L18</a>). </p><p>Please follow this link for a <a href="desc/addenda/LDC2003T03.txt" rel="nofollow">sample transcript</a>. </p><h3>Updates</h3> <p>There are no updates at this time. </p> </br>
Portions © 2003 Trustees of the University of Pennsylvania.
DOI: 10.35111/mt5q-ek34
title: 1997 HUB5 Spanish Transcripts

description: <h3>Introduction</h3> <p>The 1997 HUB5 Spanish Transcripts corpus was produced by the Linguistic Data Consortium (LDC), catalog number LDC2003T04 and ISBN 1-58563-248-1. </p><p>This publication contains transcripts for 20 Callhome Spanish telephone conversations. These 20 conversations were used in NIST's 1997 HUB5 Non-English evaluation, and are published as 1997 HUB5 Spanish Evaluation, (<a href="http://catalog.ldc.upenn.edu/LDC2002S25" rel="nofollow">LDC2002S25</a>). </p><h3>Data</h3> <p>There are 20 data files in .txt format. </p><p>The .txt files are transcript files containing the orthographic forms that were used in the original transcription process. These forms also serve as the head-words in the associated CALLHOME Spanish Lexicon (<a href="http://catalog.ldc.upenn.edu/LDC96L16" rel="nofollow">LDC96L16</a>). </p><p>Please follow this link for a <a href="desc/addenda/LDC2003T04.txt" rel="nofollow">sample transcript</a>. </p><h3>Updates</h3> <p>There are no updates at this time. </p> </br>
Portions © 2003 Trustees of the University of Pennsylvania.
DOI: 10.35111/vfdx-p575
title: Arabic Treebank: Part 1 v 2.0

description: <h3>Introduction</h3> <p>Arabic Treebank: Part 1 v 2.0 was produced by Linguistic Data Consortium (LDC) catalog number LDC2003T06 and ISBN 1-58563-261-9. This publication is part one of a a corpus of one million words of Arabic Treebank, designed to support language research and development of language technology for Modern Standard Arabic. </p><h3>Data</h3> <p>The Penn Arabic Treebank, which is part of the DARPA TIDES project, started in the Fall of 2001 with the objective of performing human and computer annotations of a large Arabic machine-readable text corpus (for project background please see <a href="http://www.ldc.upenn.edu/Projects/TIDES/Arabic/data/POS/POStest.html" rel="nofollow">POStest.html</a>). As in previous Penn Treebanks, two different kinds of information need to be produced by two different (human and computer) processes. The Arabic Treebank project consists therefore of two distinct phases: </p><ol> <li>Part-of-Speech (POS) tagging - divides the text into lexical tokens, and gives relevant information about each token such as lexical category, inflectional features, and a gloss </li> <li>Arabic Treebanking (ArabicTB) - characterizes the constituent structures of word sequences, provides categories for each non-terminal node, and identifies null elements, co-reference, traces, etc. </li> </ol><p>Both tasks started in November 2001 with an initial pilot consisting of 734 files representing roughly 166K words of written Modern Standard Arabic newswire from the Agence France Presse corpus. </p><p>The target of this publication is to provide a description of a written Modern Standard Arabic text corpus. The source data consists of Agence France Presse (AFP) newswire, spanning from July through November of 2000. This publication includes 734 stories representing 140,265 words (168,123 tokens after clitic segmentation in the Treebank). </p><h3>Updates</h3> <p>There are no updates available at this time. </p> </br>
Portions © 2000 Agence France-Presse, © 2002 Trustees of the University of Pennsylvania
DOI: 10.35111/brh9-dy85
title: Arabic Treebank: Part 1 - 10K-word English Translation

description: <h3>Introduction</h3><br>
<p>Arabic Treebank: Part 1 - 10K-word English Translation was produced by Linguistic Data Consortium (LDC) catalog number LDC2003T07 and ISBN 1-58563-262-7. The purpose of this corpus of 10K Arabic words translated into English is to support the development of data-driven approaches to natural language processing, machine translation, human language technologies, cross-lingual information retrieval, and other forms of linguistic research on Modern Standard Arabic in general.</p><br>
<h3>Data</h3><br>
<p>The project targets the translation of a written Modern Standard Arabic corpus from the Agence France Presse (AFP) newswire archives for July 2000 (the files are dated 20000715*). The corpus consists of 49 source stories, which is a subset of the 734 stories published in <a href="http://catalog.ldc.upenn.edu/LDC2003T06" rel="nofollow">Arabic Treebank: Part 1 v 2.0</a>. These 49 source files consist of 418 paragraphs and 9,981 words.</p><br>
<p>The source data and the translations are stored in SGML format. The files have been validated using the DTD provided in the corpus. Please follow these links for an example of an <a href="desc/addenda/LDC2003T07_Arabic.jpg" rel="nofollow">Arabic source file</a> and the <a href="desc/addenda/LDC2003T07_English.txt" rel="nofollow">English translation</a>.</p><br>
<p>The stories have been translated at paragraph level and verified/corrected by different annotators. In general, the translation between Arabic and English has been aligned at sentence-to-sentence level. However, we noticed that an Arabic sentence can be translated into multiple English sentences (16 occurrences), and two Arabic sentences can be translated into a single English sentence (two occurrences). For 18 paragraphs out of the total of 418 in the corpus, only paragraph-to-paragraph alignment is provided.</p><br>
<h3>Updates</h3><br>
<p>There are no updates available at this time.</p></br>
Portions © 2000 Agence France-Presse, © 2003 Trustees of the University of Pennsylvania
DOI: 10.35111/92vj-wg93
title: Korean Telephone Conversations Transcripts

description: <h3>Introduction</h3><br>
<p>Korean Telephone Conversations Transcripts was produced by Linguistic Data Consortium (LDC) catalog number LDC2003T08 and ISBN 1-58563-264-3.</p><br>
<p>The telephone conversations on which these transcripts are based were originally recorded as part of the CALLFRIEND project. The CALLFRIEND Korean telephone speech was collected by Linguistic Data Consortium primarily in support of the Language Identification (LID) project, sponsored by the U.S. Department of Defense. The calls were later transcribed for use in other projects.</p><br>
<p>This publication consists of 100 transcribed telephone conversations in Korean. The corresponding speech is published as <a href="../../../LDC2003S03">Korean Telephone Conversations Speech</a>. The Korean orthographic forms from the 100 trascription files serve as the head-words in the associated <a href="../../../LDC2003L02">Korean Telephone Conversations Lexicon</a>.</p><br>
<p>The recorded conversations are between native speakers of Korean and last up to 30 minutes, of which the transcribed speech covers between 15 to 18 minutes. All speakers were aware that they were being recorded. They were given no guidelines concerning what they should talk about. Once a caller was recruited to participate, he/she was given a free choice of whom to call. Most participants called family members or close friends. All calls originated in either the United States or Canada.</p><br>
<h3>Data</h3><br>
<p>There are 100 time aligned text files, totalling approximately 190K words and 25K unique words.</p><br>
<p>All files are in Korean orthography: orthographic Korean characters are in Hangul, encoded in KSC5601 (Wansung) system, also known as EUC-KR or ISO-2022-KR.</p><br>
<p>Please follow this link for a sample transcript: <a href="desc/addenda/LDC2003T08.txt" rel="nofollow">txt</a> | <a href="desc/addenda/LDC2003T08.gif" rel="nofollow">gif</a>.</p><br>
<h3>Updates</h3><br>
<p>There are no updates available at this time.</p></br>
Portions © 2003 Trustees of the University of Pennsylvania.
DOI: 10.35111/msvm-t728
title: SAID

description: <h3>Introduction</h3> <p>SAID (A Syntactically Annotated Idiom Dataset) was produced by Linguistic Data Consortium (LDC) catalog number LDC2003T10 and ISBN 1-58563-268-6. </p><p>The purpose if this corpus is to provide data for investigating the structural configurations in which English idioms are typically found. The assumption was that, since idioms are phrasal lexical items (PLIs), they would therefore have structural properties which are idiosyncratic. In order to study the structural properties of phrasal lexical items, the data is more useful if it is syntactically annotated. </p><h3>Data</h3> <p>The data was originally drawn from four dictionaries of English idioms. </p><p>Only citation forms, suitably adapted for this purpose, were used. The citation files were amalgamated. The rationale for the selection was that these are among the biggest and most comprehensive lists of English idioms. </p><p>There are 13,467 phrasal lexical items in this corpus. </p><p>The analysis of the phrasal lexical items was manual, while the bracketting symmetry was checked computationally. </p><p>In order to facilitate machine manipulation of the annotated data, the manual analysis was converted to PROLOG format. This involved expansions of those PLIs which had optional constituents so that both the case with and the case without the options were made available. </p><p>The files are provided in text format, which each record separated by a carriage return. </p><h3>Sponsorship</h3> The New Zealand Vice Chancellors' Committee <p>The University of Canterbury </p><h3>Updates</h3> <p>There are no updates available at this time. </p> </br>
Portions © 2003 Koenraad Kuiper, Heather McCann, Heidi Quinn, Therese Aitchison, Kees van der Veer
DOI: 10.35111/wbcc-y063
title: Message Understanding Conference (MUC) 6

description: <h3>Introduction</h3><br>
<p>Message Understanding Conference (MUC) 6 was produced by Linguistic Data Consortium (LDC) catalog number LDC2003T13 and ISBN 1-58563-239-2.</p><br>
<p>In the 1990s, the MUC evaluations funded the development of metrics and statistical algorithms to support government evaluations of emerging information extraction technologies. Additional information from NIST can be found at <a href="http://www.itl.nist.gov/iaui/894.02/related_projects/muc" rel="nofollow">http://www.itl.nist.gov/iaui/894.02/related_projects/muc</a>.</p><br>
<h3>Data</h3><br>
<p>This corpus contains the 318 annotated Wall Street Journal articles, the scoring software and the corresponding documentation used in the MUC6 evaluation. Both the <a href="http://catalog.ldc.upenn.edu/LDC96T10" rel="nofollow">MUC 6 Additional News Text</a> and the MUC 6 corpus are necessary in order to replicate the evaluation. All the materials are published as received from the corpus creators, without any quality control being done at the LDC (the only difference is that the files have been uncompressed).</p><br>
<h3>Samples</h3><br>
<p>Please view this <a href="desc/addenda/LDC2003T13.txt">text sample</a>.</p><br>
<h3>Updates</h3><br>
<p>August 20th, 2003: What was formerly published as MUC VI Text Collection (<a href="http://catalog.ldc.upenn.edu/LDC96T10" rel="nofollow">LDC1996T10</a>) was renamed as MUC 6 Additional News Text, because LDC96T10 consists only of additional training materials.</p></br>
Portions © 1986-1994 Dow Jones & Company, Inc. <br><br>RESTRICTED RIGHTS LEGEND: INFORMATION FROM THE WALL STREET JOURNAL AND/OR THE DOW JONES NEWS SERVICE CONTAINED HEREIN IS THE PROPERTY OF DOW JONES & COMPANY, INC. AND IS PROTECTED BY COPYRIGHT. USE, DUPLICATION OR DISCLOSURE BY YOU IS SUBJECT TO THE RESTRICTIONS SET FORTH IN THE USER AGREEMENT DELIVERED TO YOU BY THE LINGUISTIC DATA CONSORTIUM OF THE UNIVERSITY OF PENNSYLVANIA. COPYRIGHT 1986-1994 DOW JONES & COMPANY, INC. ALL RIGHTS RESERVED.
DOI: 10.35111/7v71-fh28
title: SummBank 1.0

description: <h3>Introduction</h3><br>
<p>SummBank 1.0 contains the data created for the Summer 2001 Johns Hopkins University Workshop which focused on text summarization in a cross-lingual information retrieval framework. The goal was to gather a corpus of original documents and summaries for use as gold standards by the documents summarization community.</p><br>
<p>The source of the data consists of 18,147 aligned bilingual (Cantonese and English) article pairs from the Information Services Department of the Hong-Kong Special Administrative Region of the People's Republic of China, which were published by the LDC in 2000 as <a href="http://catalog.ldc.upenn.edu/LDC2000T46" rel="nofollow">Hong Kong News Parallel Text</a>.</p><br>
<h3>Data</h3><br>
<p>This release contains 40 news clusters in English and Chinese, 360 multi-document, human-written non-extractive summaries, and nearly two million single document and multi-document extracts created by automatic and manual methods. MEAD was the summarizer that was reimplemented and upgraded during the workshop; versions of the software are available from the <a href="http://www.summarization.com/mead" rel="nofollow">MEAD website</a>.</p><br>
<p>This distribution includes roughly two million text files, totalling approximately 13GB uncompressed. The text files are encoded either as utf-8 for English or GB or Big-5 for Chinese.</p><br>
<h3>Updates</h3><br>
<p>Additional information, updates, bug fixes may be available on the <a href="http://www.summarization.com/summbank" rel="nofollow">SummBank website</a>.</p></br>
Portions © 1997-2000 The Government of the Hong Kong Special Administrative Region (HKSAR), © 2000, 2003 Trustees of the University of Pennsylvania
DOI: 10.35111/ge24-bd53
title: FORM2 Kinematic Gesture

description: <h3>Introduction</h3><br>
<p>FORM2 Kinematic Gesture was produced by the Linguistic Data Consortium.</p><br>
<p>FORM is a gesture annotation scheme designed to capture the kinematic information in gesture from videos of speakers. This publication is a detailed database of gesture-annotated videos stored in the Anvil and FORM file formats. FORM encodes the "phonetics" of gesture by giving geometric descriptions of location and movement of the right and left arms. Other kinematic information such as effort and shape are also recorded.</p><br>
<h3>Data</h3><br>
<p>There are a total of 24 data files: eight movie files, eight Anvil files, and eight Form files.</p><br>
<p>The movie files represent 12 minutes of audio and video recordings excerpted from a lecture given by Brian MacWhinney on January 24, 2000 at Carnegie Mellon University. These video recordings were chosen because they are part of the NSF-funded Talkbank project.</p><br>
<p>The video format is as follows:</p><br>
<table><br>
<tbody><br>
<tr><br>
<td colspan="40%">Size</td><br>
<td colspan="60%">360 x 240 pixels</td><br>
</tr><br>
<tr><br>
<td colspan="40%">Compression</td><br>
<td colspan="60%">H.261</td><br>
</tr><br>
<tr><br>
<td colspan="40%">Data rate</td><br>
<td colspan="60%">696 K/sec</td><br>
</tr><br>
<tr><br>
<td colspan="40%">Video rate</td><br>
<td colspan="60%">29.82 fps</td><br>
</tr><br>
<tr><br>
<td colspan="40%">Audio rate</td><br>
<td colspan="60%">48.000 kHz</td><br>
</tr><br>
<tr><br>
<td colspan="40%">Audio format</td><br>
<td colspan="60%">8-bit stereo</td><br>
</tr><br>
</tbody><br>
</table><br>
<p>The gesture annotations were created using the FORM 2.0 tag set. The Anvil annotation files used in their creation, augmented with FORM 1.0 data, are also included. (FORM1 data will be the subject of a separate publication to be released in the near future). FORM1 values that are not included in the FORM2 spec are not included in the publication. A full description of the FORM tag set with explanations of each value can be found in the documentation.</p><br>
<h3>Sponsorship</h3><br>
<p>This research was conducted using funding from the following grant sources: ISLE - 9910603 NSF: TalkBank (via subcontract from Carnegie Mellon University) - BCS-998009 and BCS-9978056 NSF: Discourse and Gesture - EIA98-09209</p><br>
<h3>Updates</h3><br>
<p>There are no updates available at this time.</p><br>
<h3>Note</h3><br>
<p>The cost of the first 50 copies of this publication (not counting the copies distributed to LDC members) is covered by the sponsoring grants, and therefore free of charge to qualified researchers; a $30 shipping and handling fee applies. After these first 50 copies are distributed, additional copies will be available for the production cost of $500.</p></br>
Portions © 2003 Trustees of the University of Pennsylvania, © 2000 Brian MacWhinney