cometadata/datacite-titles-descriptions-related-identifiers huggingface dataset:
DOI: 10.35111/j3yz-jw79
title: Chinese Treebank 5.0

description: <h3>Introduction</h3><br>
<p>Chinese Treebank 5.0 was produced by Linguistic Data Consortium (LDC) catalog number LDC2005T01 and ISBN 1-58563-323-2.</p><br>
<p>The Penn Chinese Treebank is an ongoing project that started in the summer of 1998. The goal of the project is to create a 500,000-word corpus of Chinese text with syntactic bracketing. Chinese Treebank 1.0 was first published in 2000, and it was later corrected and released in 2001 as <a href="http://catalog.ldc.upenn.edu/LDC2001T11" rel="nofollow">Chinese Treebank 2.0</a>. Another updated version was released in 2004 as <a href="http://catalog.ldc.upenn.edu/LDC2004T05" rel="nofollow"> Chinese Treebank 4.0</a>. More information about the project is available on the <a href="https://www.cs.brandeis.edu/~llc/page2/page2.html">Chinese Treebank</a> website.</p><br>
<p>The content used in this corpus comes from the following newswire sources:</p><br>
<table><br>
<tbody><br>
<tr><br>
<td colspan="20%">698 articles</td><br>
<td>Xinhua (1994-1998)</td><br>
</tr><br>
<tr><br>
<td colspan="20%">55 articles</td><br>
<td>Information Services Department of HKSAR (1997)</td><br>
</tr><br>
<tr><br>
<td colspan="20%">132 articles</td><br>
<td>Sinorama magazine, Taiwan (1996-1998 & 2000-2001)</td><br>
</tr><br>
</tbody><br>
</table><br>
<h3>Data</h3><br>
<p>Chinese Treebank 5.0 contains 507,222 words, 824,983 Hanzi, 18,782 sentences, and 890 data files.</p><br>
<p>All files are GB encoded. The format of Chinese Treebank 5.0 is the same as the Penn English Treebank. All files have been annotated at least twice. The first pass was done by one annotator, and the resulting files were checked by a second annotator (second pass). Some files were also double-blind annotated and then adjudicated to create gold standard files.</p><br>
<p>The corpus provides four versions of files: bracketed, raw, segmented and postagged. The raw, segmented and postagged versions are generated from the bracketed version and so do not reflect the previous annotation stages. The bracketed files are sequentially named as follows: chtb_nnnn.fid, where nnnn is a sequential file number.</p><br>
<h3>Samples</h3><br>
<p>To see an example of Gold Standard file, please examine this <a href="../../../Catalog/desc/addenda/LDC2005T01_gold.fid" rel="nofollow">sample</a>.</p><br>
<h3>Updates</h3><br>
<p>The 5.1 update contains corrections to errors found in the earlier version. Specifically, sentences which had more than one top-level node have been modified. Additionally, some GB-encoded white spaces have been converted to ASCII. The 5.1 package is available as an additional download to all those who have licensed CTB5.0.</p></br>
 Portions © 1994-1998 Xinhua News Agency, © 1996-2001 Sinorama Magazine, © 1997 The Government of the Hong Kong Special Administrative Region, © 2001, 2004, 2005 Trustees of the University of Pennsylvania
DOI: 10.35111/12tq-vy11
title: Arabic Treebank: Part 1 v 3.0 (POS with full vocalization + syntactic analysis)

description: <h3>Introduction</h3> <p>To support the development of data-driven approaches to natural language processing (NLP), human language technologies, automatic content extraction (topic extraction and/or grammar extraction), cross-lingual information retrieval, information detection, and other forms of linguistic research on Modern Standard Arabic in general, the LDC was sponsored to develop an Arabic Treebank of 1,000,000 words. This corpus is a re-release of part one of that project, with the addition in Version 3.0 of improved morphological/part-of-speech annotation (including full vocalization and case endings). </p> <h3>Data</h3> <p>The project targets the description of a written Modern Standard Arabic corpus from the Agence France Presse (AFP) newswire archives for July-November 2000 (files dated 20000/7/15 to 2000/11/15). This corpus includes 734 stories representing 145,386 words (166,068 tokens after clitic segmentation in the Treebank; the number of Arabic tokens is 123,796). For this work, annotators must be native speakers of Arabic, and they must understand enough linguistics to check morphosyntactic analysis and build syntactic structures.</p> <h3>Samples</h3> <p> To see an example of this corpus, please examine the following samples: </p><ul> <li> <a href="desc/addenda/LDC2005T02.gif" rel="nofollow">Source</a> </li> <li> <a href="desc/addenda/LDC2005T02_pos.txt" rel="nofollow">POS</a> </li> <li> <a href="desc/addenda/LDC2005T02_treebank.txt" rel="nofollow">Treebank</a> </li> </ul> </br>
Portions © 2000 Agence France Presse, Portions © 2005 Trustees of the University of Pennslyvania
DOI: 10.35111/9qxc-x068
title: Arabic CTS Levantine Fisher Training Data Set 3, Transcripts

description: <h3>Introduction</h3><br>
<p>Arabic CTS Levantine Fisher Training Data Set 3 Transcripts provides the transcription for the speech contained in <a href="http://catalog.ldc.upenn.edu/LDC2005S07">Arabic CTS Levantine Fisher Training Data Set 3, Speech (LDC2005S07)</a>.</p><br>
<p>This training speech release consists of 322 conversations, representing a total of approximately 50 hours of Levantine Arabic speech.</p><br>
<p>The Fisher telephone conversation collection protocol was created at LDC to address a critical need of developers trying to build robust automatic speech recognition (ASR) systems. Previous collection protocols, such as CALLFRIEND and Switchboard-II and the resulting corpora, have been adapted for ASR research but were in fact developed for language and speaker identification respectively. Although the CALLHOME protocol and corpora were developed to support ASR technology, they feature small numbers of speakers making telephone calls of relatively long duration with narrow vocabulary across the collection. CALLHOME conversations are challengingly natural and intimate. Under the Fisher protocol, a very large number of participants each make a few calls of short duration speaking to other participants, whom they typically do not know, about assigned topics. This maximizes inter-speaker variation and vocabulary breadth although it also increases formality.</p><br>
<p>Previous protocols such as CALLHOME, CALLFRIEND and Switchboard relied upon participant activity to drive the collection. Fisher is unique in being platform driven rather than participant driven. Participants who wish to initiate a call may do so; however the collection platform initiates the majority of calls. Participants need only answer their phones at the times they specified when registering for the study.</p><br>
<p>To encourage a broad range of vocabulary, Fisher participants are asked to speak on an assigned topic which is selected at random from a list, which changes every 24 hours and which is assigned to all subjects paired on that day. Some topics are inherited or refined from previous Switchboard studies while others were developed specifically for the Fisher protocol.</p><br>
<h3>Samples</h3><br>
<p>Please examine this <a href="../../../Catalog/desc/addenda/LDC2005T03.txt" rel="nofollow">sample</a> for an example of this corpus. The file is UTF_8 encoded text.</p></br>
Portions © 2003-2005 Trustees of the University of Pennsylvania
DOI: 10.35111/6a17-c826
title: Multiple-Translation Arabic (MTA) Part 2

description: <h3>Introduction</h3> <p>Multiple-Translation Arabic (MTA) Part 2, Linguistic Data Consortium (LDC) catalog number LDC2005T05 and ISBN 1-58563-328-3, was produced by LDC. </p><p>To support the development of automatic means for evaluating translation quality, LDC was sponsored to solicit four sets of human translations for a single set of Arabic source materials. LDC was also asked to produce translations from various commercial-off-the-shelf-systems (COTS, including commercial Machine Translation (MT) systems as well as MT systems available on the Internet). This corpus contains two sets of COTS outputs and one output set from a TIDES 2003 MT Evaluation participant, which is representative for the state-of-the-art research systems. </p><p>To determine if automatic evaluation systems such as BLEU track human assessment, LDC also performed human assessment on the two COTS outputs and the TIDES research system. The corpus includes the assessment results for one of the two COTS systems, the assessment result for the TIDES research system, and the specifications used for conducting the assessments. </p><h3>Source Data Selection:</h3> <ul> <li>Xinhua News Service (Xinhua): 50 news stories </li> <li>Agence France Presse (AFP): 50 news stories </li> </ul><p>(total: 100 stories)</p> <p>There are 100 source files and 700 translation files. All source data were drawn from January and February 2003 collection of Xinhua Arabic data and AFP Arabic data. </p> <p>The story selection from the two newswire collections was controlled by story length: all selected stories contain between 700 and 1,500 Arabic characters. </p> <p>The overall count of Arabic words (excluding markup), by source, is shown in the following table: </p><ul type="none"> <li>AFP 7,528 </li> <li>Xinhua 7,551 </li> <li>------------- </li> <li> 15,079 <h3>Samples</h3> <p>For samples from this corpus, see this screen shot of the Arabic source <a href="/Catalog/desc/addenda/LDC2005T05_source.jpg" rel="nofollow">file</a> and its <a href="/Catalog/desc/addenda/LDC2005T05_trans.txt" rel="nofollow">translation</a>. </p> <p>Please contact <a rel="nofollow">Xiaoyi Ma</a> with any questions regarding this corpus. </p>   </li> </ul></br>
Portions © 2003 Xinhua News Agency, © 2003 Agence France Press, © 2004-2005 Trustees of the University of Pennsylvania
DOI: 10.35111/9n1n-0q43
title: Chinese News Translation Text Part 1

description: <h3>Introduction</h3> <p>Chinese News Translation Text Part 1 was produced by Linguistic Data Consortium (LDC) catalog number LDC2005T06 and ISBN 1-58563-329-1. </p><p>To support the development of automatic machine translation systems, the LDC was sponsored to solicit English translations for a single set of Chinese source materials. </p><p>The source Chinese text and its English translations were selected and translated in different LDC projects during the time period of February 2003 to January 2005. A total of about 474K Chinese characters were selected from two sources, namely Xinhua and AFP, and translation services were provided by seven translation agencies. Each Chinese news story was translated once. </p><p>All stories and its translations were created for TIDES Machine Translation as training data, following roughly the same guidelines and procedures. </p><h3>Samples</h3> To see an example of this corpus, please examine this <a href="desc/addenda/LDC2005T06.txt" rel="nofollow">translation</a> file.  </br>
Portions © 2002-2004 Xinhua News Agency, 2002-2004 Agence France-Presse, © 2005 Trustees of the University of Pennsylvania
DOI: 10.35111/9nye-wg76
title: ACE Time Normalization (TERN) 2004 English Training Data v 1.0

description: <h3>Introduction</h3> <p>This file contains documentation on the ACE Time Normalization (TERN) 2004 English Training Data v 1.0, Linguistic Data Consortium (LDC) catalog number LDC2005T07 and ISBN 1-58563-331-3. </p><p>This release contains the English training data prepared for the 2004 Time Expression Recognition and Normalization (TERN) Evaluation, sponsored by the Automatic Content Extraction (ACE) program. The evaluation was held in August 2004 and a workshop in September 2004. Evaluation participants received this data for training purposes, and it is now being released for general use. </p><p> The annotation specifications for this corpus were developed under DARPA's Translingual Information Detection Extraction and Summarization (TIDES) program, with continuing support from ACE. </p><p> The purpose of this corpus and the TERN evaluation is to advance the state of the art in the automatic recognition and normalization of natural language temporal expressions. In most language contexts such expressions are indexical. For example, with "Monday," "last week," or "three months starting October 1," one must know the narrative reference time in order to pinpoint the time interval being conveyed by the expression. In addition, for data exchange purposes, it is essential that the identified interval be rendered according to an established standard, i.e., normalized. Accurate identification and normalization of temporal expressions is in turn essential for the temporal reasoning being demanded by advanced NLP applications such as question answering, information extraction, and summarization. </p><h3>Samples</h3> <p>Please examine this <a href="desc/addenda/LDC2005T07.txt" rel="nofollow">sample</a> to see an example of the corpus. </p><h3>Updates</h3> <p>Additional information, updates, bug fixes may be available in the LDC catalog entry for this corpus at <a href="http://catalog.ldc.upenn.edu/LDC2005T07" rel="nofollow">LDC2005T07</a>. </p> <p>"The World" is a co-production of Public Radio International and the British Broadcasting Corporation and is produced at WGBH Boston.</p></br>
Portions © 1998 Los Angeles Times-Washington Post News Service, Inc., © 1998, 2000 American Broadcasting Corporation, © 1998, 2000 Cable News Network, LP, LLLP, © 1998, 2000 The Associated Press, © 1998, 2000 New York Times, © 1998, 2000 National Broadcasting Company, Inc., ©1998, 2000 Public Radio International, © 2005 Trustees of the University of Pennsylvania <br><br>"The World" is a co-production of Public Radio International and the British Broadcasting Corporation and is produced at WGBH Boston.
DOI: 10.35111/7snd-y397
title: Discourse Graphbank

description: <h3>Introduction</h3><br>
<p>Discourse Graphbank aimed to define a descriptively adequate data structure for representing discourse coherence structures. This project also investigated the impact of discourse coherence structures on other linguistic processes and natural language applications (e.g. anaphora resolution, summarization and information retrieval), and developed and tested discourse parsing algorithms.</p><br>
<h3>Data</h3><br>
<p>The source data consists of 135 texts from Assoicated Press and Wall Street Journal newswire data annotated with coherence relations. That source material was taken from LDC's <a href="../../../LDC93T3A">TIPSTER</a> releases.</p><br>
<h3>Samples</h3><br>
<p>A <a href="../../../desc/addenda/LDC2005T08.jpg" rel="nofollow">screenshot</a> of the output of the annotator tool has been provided as an example of this corpus.</p></br>
Portions © 1988-1990 Associated Press, © 1986-1989 Dow Jones & Company, Inc., © 2005 Trustees of the University of Pennsylvania
DOI: 10.35111/28bx-hc14
title: Chinese English News Magazine Parallel Text

description: <h3>Introduction</h3> <p>This file contains documentation on the Chinese English News Magazine Parallel Text, Linguistic Data Consortium (LDC) catalog number LDC2005T10 and ISBN 1-58563-333-X. </p><p>This corpus contains Chinese news stories and their English translations LDC collected via Sinorama Magazine, Taiwan, from 1976 to 2004. It totals 6,366 story pairs, 365,568 sentence pairs, 20M Chinese characters and 9M English words. The corpus is aligned at sentence level.</p><h3>Data</h3> <p>Sinorama Magazine is published monthly in several languages, including Chinese, English, Japanese. LDC received its 1976 to 2000 publications on a single CD, and its 2001 to 2004 publications via Sinorama's website. </p> <p>The Sinorama Chinese text was encoded in Big5. The data came story aligned but were lack of sentence level alignment. The sentence alignment was done at the LDC using <a href="http://champollion.sourceforge.net" rel="nofollow">Champollion</a> v 1.1. </p> <p>The final data is put in the data directory, which contains subdirectories for Chinese documents, English documents, and the sentence level alignment, identified as "Chinese," "English," and "alignment." </p> <p>The English and Chinese files may contain one or more documents, with each document formatted in SGML as follows: </p><p> [English or Chinese text] </p><p> [English or Chinese text] [English or Chinese text] ... </p><p>Notes: * the </p><p> and tags are always assigned sequential numeric IDs, starting at one. * the tags are always placed on the same line with their contents, and are always separated from the contents by a space. </p> * if an English file and a Chinese file share the same file name, they contain the same documents. * all Chinese text is encoded in Big5. <p>Each alignment file contains the sentence level alignment of multiple documents, each being formatted in SGML as follows: ... </p><p>Notes: * the docid in an English file, its Chinese translation and the ALIGNMENT are the same. * EnglishSegId and ChineseSegId may have none, one, or more than one segment IDs. </p><h3>Samples</h3> <p>The following files provide an example of this corpus: </p><ul> <li> <a href="desc/addenda/LDC2005T10_Chinese.txt" rel="nofollow">Chinese</a> </li> <li> <a href="desc/addenda/LDC2005T10_English.txt" rel="nofollow">English</a> </li> <li> <a href="desc/addenda/LDC2005T10_align.txt" rel="nofollow">Alignment</a> </li> </ul> <p>Portions © 2005 Trustees of the University of Pennsylvania</p></br>
Portions © 1976-2004 Sinorama Magazine <br><br>Portions © 2005 Trustees of the University of Pennsylvania
DOI: 10.35111/stcf-4x49
title: English Gigaword Second Edition

description: <h3>Introduction</h3><br>
<p>English Gigaword Second Edition was produced by Linguistic Data Consortium (LDC) catalog number LDC2005T12 and ISBN 1-58563-350-X. The English Gigaword corpus is a comprehensive archive of newswire text data in English that has been acquired over several years by the LDC. This is the second edition of the English Gigaword corpus.</p><br>
<p>This edition includes all of the contents in the first edition of the English Gigaword corpus (LDC2003T05) as well as new data from July 2002 through Dec 2004. Also, a new newswire source (the Central New Agency of Taiwan, English Service) has been added in this edition.</p><br>
<p>The five distinct international sources of English newswire included in this release are the following:</p><br>
<table><br>
<tbody><br>
<tr><br>
<td colspan="60%">Agence France-Presse, English Service</td><br>
<td colspan="20%">(afp_eng )</td><br>
</tr><br>
<tr><br>
<td colspan="60%">Associated Press Worldstream, English Service</td><br>
<td colspan="20%">(apw_eng)</td><br>
</tr><br>
<tr><br>
<td colspan="60%">Central News Agency of Taiwan, English Service</td><br>
<td colspan="20%">(cna_eng)</td><br>
</tr><br>
<tr><br>
<td colspan="60%">The New York Times Newswire Service</td><br>
<td colspan="20%">(nyt_eng)</td><br>
</tr><br>
<tr><br>
<td colspan="60%">The Xinhua News Agency, English Service</td><br>
<td colspan="20%">(xin_eng)</td><br>
</tr><br>
</tbody><br>
</table><br>
<h3>What's New In The Second Edition</h3><br>
<ul><br>
<li>New newswire data contents from July 2002 to December 2004 have been added for all of the four newswire sources that were represented in the first edition.</li><br>
<li>A new source, the Central News Agency of Taiwan English Service (CNA_ENG), has been added.</li><br>
<li>We have adopted a new naming scheme for filenames and DOC IDs. The new naming scheme represents the source names in a three-letter code and the language name in a three-letter code.</li><br>
<li>Minor formatting improvements (mostly line-wrapping) have been made to some of the data contents originally published in the first edition.</li><br>
</ul></br>
Portions © 1994-1997 and 2001-2004 Agence France-Presse, © 1994-2004 Associated Press, © 1997-2004 Central News Agency of Taiwan, © 1994-2004 New York Times, © 1995-2004 Xinhua News Agency, © 2005 Trustees of the University of Pennsylvania
DOI: 10.35111/a589-6d76
title: CCGbank

description: <h3>Introduction</h3><br>
<p>CCGbank is a translation of the <a href="../../../LDC95T7">Penn Treebank</a> into a corpus of Combinatory Categorial Grammar derivations. It pairs syntactic derivations with sets of word-word dependencies which approximate the underlying predicate-argument structure.</p><br>
<h3>Data</h3><br>
<p>CCGbank contains 99.44% of the sentences in the Penn Treebank, for which it corrects a number of inconsistencies and errors in the original annotation.</p><br>
<h3>Samples</h3><br>
<p>For an example of this corpus, please examine this <a href="desc/addenda/LDC2005T13.html" rel="nofollow">sample</a>.</p><br>
<h3>Update</h3><br>
<p>The current version, 1.1, is a bug fix that supersedes the old package. It is available for download.</p></br>
Portions © 2005 Julia Hockenmaier and Mark Steedman, © 2005 Trustees of the University of Pennsylvania
DOI: 10.35111/vr0r-sb06
title: Chinese Gigaword Second Edition

description: <h3>Introduction</h3> <p>Chinese Gigaword Release Second Edition was produced by Linguistic Data Consortium (LDC) catalog number LDC2005T14 and ISBN 1-58563-353-4. This is a comprehensive archive of newswire text data in Chinese that has been acquired over several years by the LDC. </p><p>This edition includes all of the contents in the first release of the Chinese Gigaword corpus (LDC2003T09) as well as new data collected after the publication of the first edition. Also, a limited number of articles from a new newspaper source (Zaobao) have been added in this edition. </p><p>The three distinct international sources of Chinese newswire included in this edition are the following: </p><table> <tr> <td colspan="60%">Central News Agency, Taiwan</td> <td colspan="20%">(cna_cmn)</td> </tr> <tr> <td colspan="60%">Xinhua News Agency</td> <td colspan="20%">(xin_cmn)</td> </tr> <tr> <td colspan="60%">Zaobao Newspaper</td> <td colspan="20%">(zbn_cmn)</td> </tr> </table><p>The seven-character abbreviations shown above represent both the source name and the language ID ("cmn" for Mandarin Chinese). </p><h3>New In Second Edition</h3> <p>New documents (Xinhua from October 2002 through December 2004 and CNA from January 2003 through December 2004) have been added. </p><p>A new newspaper source (Lianhe Zaobao) has been added. </p><h3>Samples</h3> <p>For an example of this corpus, please review <a href="desc/addenda/LDC2005T14.jpg" rel="nofollow">this screen capture</a> displaying some of the text included. </p> </br>
Portions © 1991-2004 Central News Agency, Taiwan © 1990-2004 Xinhua News Agency, © 2000-2003 SPH AsiaOne, Ltd., © 2005 Trustees of the University of Pennsylvania