cometadata/datacite-titles-descriptions-related-identifiers huggingface dataset:
DOI: 10.35111/wnve-3m88
title: GALE Phase 2 Chinese Broadcast News Transcripts

description: <h3>Introduction</h3><br>
<p>GALE Phase 2 Chinese Broadcast News Transcripts was developed by the Linguistic Data Consortium (LDC) and contains transcriptions of approximately 110 hours of Chinese broadcast news speech collected in 2006 and 2007 by LDC and Hong University of Science and Technology (HKUST), Hong Kong, during Phase 2 of the DARPA GALE (Global Autonomous Language Exploitation) Program.</p><br>
<p>Corresponding audio data is released as GALE Phase 2 Chinese Broadcast News Speech (<a href="http://catalog.ldc.upenn.edu/LDC2013S08" rel="nofollow">LDC2013S08</a>).</p><br>
<p>The source broadcast recordings feature news programs focusing principally on current events from the following sources: Anhui TV, a regional television station in Mainland China, Anhui Province; China Central TV (CCTV), a national and international broadcaster in Mainland China; and Phoenix TV, a Hong Kong-based satellite television station.</p><br>
<h3>Data</h3><br>
<p>The transcript files are in plain-text, tab-delimited format (TDF) with UTF-8 encoding, and the transcribed data totals 1,593,049 tokens. The transcripts were created with the LDC-developed transcription tool, XTrans, a multi-platform, multilingual, multi-channel transcription tool that supports manual transcription and annotation of audio recordings. XTrans is available from the following link, <a href="https://www.ldc.upenn.edu/language-resources/tools/xtrans" rel="nofollow">https://www.ldc.upenn.edu/language-resources/tools/xtrans</a>.</p><br>
<p>The files in this corpus were transcribed by LDC staff and/or by transcription vendors under contract to LDC. Transcribers followed the LDC quick transcription guidelines (QTR) and quick rich transcription specification (QRTR) both of which are included in the documentation with this release. QTR transcription consists of quick (near-)verbatim, time-aligned transcripts plus speaker identification with minimal additional mark-up. It does not include sentence unit annotation. QRTR annotation adds structural information such as topic boundaries and manual sentence unit annotation to the core components of a quick transcript. Files with QTR as part of the filename were developed using QTR transcription. Files with QRTR in the filename indicate QRTR transcription.</p><br>
<h3>Samples</h3><br>
<p>Please view this <a href="desc/addenda/LDC2013T20.jpg" rel="nofollow">text sample</a>.</p><br>
<h3>Updates</h3><br>
<p>None at this time.</p><br>
<h3>Acknowledgement</h3><br>
<p>This work was supported in part by the Defense Advanced Research Projects Agency, GALE Program Grant No. HR0011-06-1-0003. The content of this publication does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.</p></br>
Portions © 2007 Anhui TV, © 2006-2007 China Central TV, © 2006-2007 Phoenix TV, © 2006-2007, 2013 Trustees of the University of Pennsylvania
DOI: 10.35111/enjk-8f86
title: CALLFRIEND Farsi Second Edition Speech

description: <h3>Introduction</h3><br>
<p>CALLFRIEND Farsi Second Edition Speech was developed by the Linguistic Data Consortium (LDC) and consists of approximately 42 hours of telephone conversation (100 recordings) among native Farsi speakers. The calls were recorded in 1995 and 1996 as part of the CALLFRIEND collection, a project designed primarily to support research in automatic language identification. One hundred native Farsi speakers living in the continental United States each made a single telephone call, lasting up to 30 minutes, to a family member or friend living in the United States.</p><br>
<p>This release represents all calls from the collection. LDC released recordings from 60 calls without transcripts in 1996 as CALLFRIEND Farsi (<a href="http://catalog.ldc.upenn.edu/LDC96S50" rel="nofollow">LDC96S50</a>) after 20 of those calls were used as evaluation data in the first <a href="http://www.itl.nist.gov/iad/mig/tests/lre/1996/" rel="nofollow">NIST Language Recognition Evaluation</a> (LRE). Seven of these original 60 calls were deemed unsuitable for transcription and, thus 53 of the original CF Farsi files are included along with 47 new files.</p><br>
<p>Corresponding transcripts are available in CALLFRIEND Farsi Second Edition Speech Transcripts (<a href="http://catalog.ldc.upenn.edu/LDC2014T01" rel="nofollow">LDC2014T01</a>).</p><br>
<h3>Data</h3><br>
<p>All recordings involved domestic calls routed through the automated telephone collection platform at LDC and were stored as 2-channel (4-wire), 8-KHz mu-law samples taken directly from the public telephone network via a T-1 circuit. Each audio file is a <a href="https://xiph.org/flac/" rel="nofollow">FLAC</a>-compressed MS-WAV (RIFF) format audio file containing 2-channel, 8-KHz, 16-bit PCM sample data.</p><br>
<p>This release includes speaker information, including gender, the number of speakers on each channel and call duration.</p><br>
<h3>Samples</h3><br>
<p>Please listen to this <a href="desc/addenda/LDC2014S01.wav" rel="nofollow">audio sample</a>.</p><br>
<h3>Updates</h3><br>
<p>None at this time.</p></br>
Portions © 1995-1996, 2014 Trustees of the University of Pennsylvania
DOI: 10.35111/zd7f-qr83
title: Multi-Channel WSJ Audio

description: <h3>Introduction</h3> <p>Multi-Channel WSJ Audio (MCWSJ) was developed by the <a href="http://www.cstr.ed.ac.uk/" rel="nofollow">Centre for Speech Technology Research </a> at The University of Edinburgh and contains approximately 100 hours of recorded speech from 45 British English speakers. Participants read Wall Street Journal texts published in 1987-1989 in three recording scenarios: a single stationary speaker, two stationary overlapping speakers and one single moving speaker.</p> <p>This corpus was designed to address the challenges of speech recognition in meetings, which often occur in rooms with non-ideal acoustic conditions and significant background noise, and may contain large sections of overlapping speech. Using headset microphones represents one approach, but meeting participants may be reluctant to wear them. Microphone arrays are another option. MCWSJ supports research in large vocabulary tasks using microphone arrays. The news sentences read by speakers are taken from <a href="http://catalog.ldc.upenn.edu/LDC95S24" rel="nofollow">WSJCAM0 Cambridge Read News</a>, a corpus originally developed for large vocabulary continuous speech recognition experiments, which in turn was based on <a href="http://catalog.ldc.upenn.edu/LDC93S6A" rel="nofollow">CSR-1 (WSJ0) Complete</a>, made available by LDC to support large vocabulary continuous speech recognition initiatives. </p> <h3>Data</h3> <p> Speakers reading news text from prompts were recorded using a headset microphone, a lapel microphone and an eight-channel microphone array. In the single speaker scenario, participants read from six fixed positions. Fixed positions were assigned for the entire recording in the overlapping scenario. For the moving scenario, participants moved from one position to the next while reading. </p> <p>Fifteen speakers were recorded for the single scenario, nine pairs for the overlapping scenario and nine individuals for the moving scenario. Each read approximately 90 sentences. </p> <p>The audio data are presented as single channel 16kHz flac compressed wav files.</p> <h3>Samples</h3> <p>Please listen to the below samples.</p> <ul> <li><a href="./desc/addenda/LDC2014S03.olap.wav" rel="nofollow">Overlapping Sample</a></li> <li><a href="./desc/addenda/LDC2014S03.stat.wav" rel="nofollow">Stationary Sample</a></li> <li><a href="./desc/addenda/LDC2014S03.move.wav" rel="nofollow">Moving Sample</a></li> </ul> <h3>Updates</h3> <p>None at this time. </p>  </br>
 Portions © 1987-1989 Dow Jones & Company, Inc., © 2014 University Court of the University of Edinburgh, © 1992-1995, 2014 Trustees of the University of Pennsylvania
DOI: 10.35111/whyr-g327
title: United Nations Proceedings Speech

description: <h3>Introduction</h3><br>
<p>United Nations Proceedings Speech was developed by the <a href="http://www.un.org/">United Nations</a> (UN) and contains approximately 8,500 hours of recorded proceedings in the six official UN languages, Arabic, Chinese, English, French, Russian and Spanish. The data was recorded in 2009-2012 from sessions 64-66 of the <a href="http://www.un.org/en/ga/">General Assembly</a> (GA) and <a href="http://www.un.org/en/ga/first/">First Committee</a> (FC) (Disarmament and International Security), and meetings 6434-6763 of the <a href="http://www.un.org/en/sc/">Security Council</a>.</p><br>
<p>Recordings were made using a customized system following a daily internal circulated instruction from the <a href="http://www.un.org/depts/DGACM/mms.shtml">Meetings Management Section</a>. Most of the subjects and information related to a particular meeting or session are published in a UN Journal which can be found in the following link: <a href="http://www.un.org/en/documents/journal.asp">http://www.un.org/en/documents/journal.asp</a></p><br>
<h3>Data</h3><br>
<p>Data is presented either as mp3 or flac compressed wav and are 16-bit single channel files in either 22,050 or 8,000 Hz organized by committee and session number, then language. The folder labeled "Floor" indicates the microphone used by the particular speaker. Those files may include other languages, for instance, if the speaker's language was not among the six official UN languages.</p><br>
<p>File naming conventions for GA and FC data are in the form of LYY_ZZ_format.format and Security Council data is in the form of LYYYY_ZZ_format.format where L is a one letter language designation, YY is the meeting number, ZZ indicates the audio segment number and format.format is the wav or mp3 designation. Note that not all files are present for every language.</p><br>
<h3>Samples</h3><br>
<p>Please listen to the following samples</p><br>
<ul><br>
<li><a href="desc/addenda/LDC2014S08.flr.wav">Floor</a></li><br>
<li><a href="desc/addenda/LDC2014S08.A.wav">Arabic</a></li><br>
<li><a href="desc/addenda/LDC2014S08.C.wav">Chinese</a></li><br>
<li><a href="desc/addenda/LDC2014S08.E.wav">English</a></li><br>
<li><a href="desc/addenda/LDC2014S08.F.wav">French</a></li><br>
<li><a href="desc/addenda/LDC2014S08.R.wav">Russian</a></li><br>
<li><a href="desc/addenda/LDC2014S08.S.wav">Spanish</a></li><br>
</ul><br>
<h3>Updates</h3><br>
<p>None at this time.</p><br>
<p> </p></br>
Portions © 2009-2012, 2014 United Nations, © 2014 Trustees of the University of Pennsylvania
DOI: 10.35111/t7m9-b537
title: CALLFRIEND Farsi Second Edition Transcripts

description: <h3>Introduction</h3><br>
<p>CALLFRIEND Farsi Second Edition Transcripts was developed by the Linguistic Data Consortium (LDC)and consists of transcripts for approximately 42 hours of telephone conversation (100 recordings) among native Farsi speakers. The calls were recorded in 1995 and 1996 as part of the CALLFRIEND collection, a project designed primarily to support research in automatic language identification. One hundred native Farsi speakers living in the continental United States made a single telephone call, lasting up to 30 minutes, to a family member or friend living in the United States.</p><br>
<p>Corresponding speech data is available as CALLFRIEND Farsi Second Edition Speech (<a href="http://catalog.ldc.upenn.edu/LDC2014S01" rel="nofollow">LDC2014S01</a>).</p><br>
<h3>Data</h3><br>
<p>Transcripts are presented in three formats: romanized transcripts (*asc.txt), Arabic-script transcripts (*ntv.txt) and both romanized and Arabic forms in a simple XML format (*.xml). For the *.txt files, the four main fields on each line (start-offset, end-offset, speaker-label, transcript-text) are separated by tabs. Each file begins with a single comment line containing the file_id string. This is followed immediately by the list of time-stamped segments, in order according to their start-offset values, with no blank lines. The XML form of the transcripts contains both Arabicized and romanized forms for Farsi words.</p><br>
<h3>Samples</h3><br>
<p>Please view the following samples.</p><br>
<ul><br>
<li><a href="desc/addenda/LDC2014T01.asc.txt" rel="nofollow">Romanized Text</a></li><br>
<li><a href="desc/addenda/LDC2014T01.ntv.jpg" rel="nofollow">Arabic Script Text</a></li><br>
<li><a href="desc/addenda/LDC2014T01.xml.jpg" rel="nofollow">XML</a></li><br>
</ul><br>
<h3>Updates</h3><br>
<p>None at this time.</p></br>
Portions © 1995-1996, 2000-2001, 2012-2014 Trustees of the University of Pennsylvania
DOI: 10.35111/ccbq-js63
title: GALE Phase 2 Chinese Broadcast News Parallel Text Part 1

description: <h3>Introduction</h3><br>
<p>GALE Phase 2 Chinese Broadcast News Parallel Text Part 1 was developed by the Linguistic Data Consortium (LDC). Along with other corpora, the parallel text in this release comprised training data for Phase 2 of the DARPA GALE (Global Autonomous Language Exploitation) Program. This corpus contains Chinese source text and corresponding English translations selected from broadcast news (BN) data collected by LDC between 2005 and 2007 and transcribed by LDC or under its direction.</p><br>
<h3>Data</h3><br>
<p>This release includes 30 source-translation document pairs, comprising 198,350 characters of translated material. Data is drawn from 11 distinct Chinese BN programs broadcast by China Central TV, a national and international broadcaster in Mainland China, Jiangsu TV, a regional television station in Mainland China, Jiangsu Province, New Tang Dynasty TV, a broadcaster based in the United States, and Phoenix TV, a Hong-Kong based satellite television station. The broadcast news recordings in this release focus principally on current events.</p><br>
<p>The data was transcribed by LDC staff and/or transcription vendors under contract to LDC in accordance with Quick Rich Transcription guidelines developed by LDC. Transcribers indicated sentence boundaries in addition to transcribing the text. Data was manually selected for translation according to several criteria, including linguistic features, transcription features and topic features. The transcribed and segmented files were then reformatted into a human-readable translation format and assigned to translation vendors. Translators followed the Chinese to English translation guidelines developed by LDC. Bilingual LDC staff performed quality control procedures on the completed translations.</p><br>
<p>Source data and translations are distributed in TDF format. TDF files are tab-delimited files containing one segment of text along with meta information about that segment. Each field in the TDF file is described in TDF_format.text. All data are encoded in UTF-8.</p><br>
<h3>Samples</h3><br>
<p>Please view this <a href="desc/addenda/LDC2014T04.cmn.jpg" rel="nofollow">source sample</a> and <a href="desc/addenda/LDC2014T04.eng.txt" rel="nofollow">translation sample</a>.</p><br>
<h3>Updates</h3><br>
<p>None at this time.</p><br>
<h3>Sponsorship</h3><br>
<p>This work was supported in part by the Defense Advanced Research Projects Agency, GALE Program Grant No. HR0011-06-1-0003. The content of this publication does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.</p></br>
Portions © 2005-2007 China Central TV, © 2006 Jiangsu TV, © 2005 New Tang Dynasty TV, © 2005-2007 Phoenix TV, © 2005-2007, 2014 Trustees of the University of Pennsylvania
DOI: 10.35111/7ez0-x912
title: ETS Corpus of Non-Native Written English

description: <h3>Introduction</h3><br>
<p>ETS Corpus of Non-Native Written English was developed by <a href="https://www.ets.org/" rel="nofollow">Educational Testing Service</a> and is comprised of 12,100 English essays written by speakers of 11 non-English native languages as part of an international test of academic English proficiency, <a href="http://www.ets.org/toefl/ibt/about" rel="nofollow">TOEFL</a> (Test of English as a Foreign Language). The test includes reading, writing, listening, and speaking sections and is delivered by computer in a secure test center. This release contains 1,100 essays for each of the 11 native languages sampled from eight topics with information about the score level (low/medium/high) for each essay.</p><br>
<p>The corpus was developed with the specific task of native language identification in mind, but is likely to support tasks and studies in the educational domain, including grammatical error detection and correction and automatic essay scoring, in addition to a broad range of research studies in the fields of natural language processing and corpus linguistics. For the task of native language identification, the following division is recommended: 82% as training data, 9% as development data and 9% as test data, split according to the file IDs accompanying the data set.</p><br>
<h3>Data</h3><br>
<p>The data is sampled from essays written in 2006 and 2007 by test takers whose native languages were Arabic, Chinese, French, German, Hindi, Italian, Japanese, Korean, Spanish, Telugu, and Turkish. The essays are presented in both original raw and tokenized forms and presented in UTF-8 formatted text files. Also included are the prompts (topics) for the essays and metadata about the test takers' proficiency level.</p><br>
<h3>Samples</h3><br>
<p>Please view this <a href="desc/addenda/LDC2014T06.orig.txt" rel="nofollow">original</a> and <a href="desc/addenda/LDC2014T06.token.txt" rel="nofollow">tokenized</a> samples.</p><br>
<h3>Updates</h3><br>
<p>In July 2014, 1,100 files were added to the corpus, bringing the total number of tokenized and original files to 12,100. All copies distributed after that date contain the full data set.</p></br>
 Portions © 2014 Educational Testing Service, © 2014 Trustees of the University of Pennsylvania
DOI: 10.35111/9hpq-hv57
title: Domain-Specific Hyponym Relations

description: <h3>Introduction</h3><br>
<p>Domain-Specific Hyponym Relations was developed by the Shaanxi Province Key Laboratory of Satellite and Terrestrial Network Technology at <a href="http://en.xjtu.edu.cn/">Xi'an Jiaotung University</a>, Xi'an, Shaanxi, China. It provides more than 5,000 English hyponym relations in five domains including data mining, computer networks, data structures, Euclidean geometry and microbiology. All hypernym and hyponym words were taken from Wikipedia article titles.</p><br>
<p>A hyponym relation is a word sense relation that is an <em>IS-A</em> relation. For example, <em>dog</em> is a hyponym of <em>animal</em> and <em>binary tree</em> is a hyponym of <em>tree structure</em>. Among the applications for domain-specific hyponym relations are taxonomy and ontology learning, query result organization in a faceted search and knowledge organization and automated reasoning in knowledge-rich applications.</p><br>
<h3>Data</h3><br>
<p>The data is presented in XML format, and each file provides hyponym relations in one domain. Within each file, the term, Wikipedia URL, hyponym relation and the names of the hyponym and hypernym words are included. The distribution of terms and relations is set forth in the table below:</p><br>
<table><br>
<tbody><br>
<tr><br>
<td>Dataset</td><br>
<td>Terms</td><br>
<td>Hyponym Relations</td><br>
</tr><br>
<tr><br>
<td>Data Mining</td><br>
<td>278</td><br>
<td>364</td><br>
</tr><br>
<tr><br>
<td>Computer Network</td><br>
<td>336</td><br>
<td>399</td><br>
</tr><br>
<tr><br>
<td>Data Structure</td><br>
<td>315</td><br>
<td>578</td><br>
</tr><br>
<tr><br>
<td>Euclidean Geometry</td><br>
<td>455</td><br>
<td>690</td><br>
</tr><br>
<tr><br>
<td>Microbiology</td><br>
<td>1,028</td><br>
<td>3,533</td><br>
</tr><br>
</tbody><br>
</table><br>
<h3>Samples</h3><br>
<p>Please view this <a href="desc/addenda/LDC2014T07.jpg" rel="nofollow">sample</a>.</p><br>
<h3>Updates</h3><br>
<p>None at this time.</p></br>
 Portions © 2014 Shaanxi Province Key Laboratory of Satellite and Terrestrial Network Technology, © 2014 Trustees of the University of Pennsylvania
DOI: 10.35111/ed7d-z579
title: HyTER Networks of Selected OpenMT08/09 Sentences

description: <h3>Introduction</h3><br>
<p>HyTER Networks of Selected OpenMT08/09 Progress Set Sentences was developed by <a href="http://www.sdl.com/" rel="nofollow">SDL</a> and contains HyTER (Hybrid Translation Edit Rate) networks for 102 selected source Arabic and Chinese sentences from <a href="http://catalog.ldc.upenn.edu/LDC2013T07" rel="nofollow">OpenMT08 and OpenMT09 Progress Set </a>data. HyTER is an evaluation metric based on large reference networks created by an annotation tool that allows users to develop an exponential number of correct translations for a given sentence. Reference networks can be used as a foundation for developing improved machine translation evaluation metrics and for automating the evaluation of human translation efficiency.</p><br>
<h3>Data</h3><br>
<p>The source material is comprised of Arabic and Chinese newswire and web data collected by LDC in 2007. Annotators created meaning-equivalent annotations under three annotation protocols. In the first protocol, foreign language native speakers built English networks starting from foreign language sentences. In the second, English native speakers built English networks from the best translation of a foreign language sentence as identified by NIST (National Institute of Standards and Technology). In the third protocol, English native speakers built English networks starting from the best translation, but those annotators also had access to three additional, independently produced human translations. Networks created by different annotators for each sentence were combined and evaluated.</p><br>
<p>This release includes the source sentences and four human reference translations produced by LDC in XML format, along with five machine translation system outputs representing a variety of system architectures and performance, and the human post-edited output of those systems also presented in XML.</p><br>
<h3>Samples</h3><br>
<p>Please view this <a href="desc/addenda/LDC2014T09.fst.txt" rel="nofollow">FST sample</a> and <a href="desc/addenda/LDC2014T09.ref.jpg" rel="nofollow">Reference XML sample.</a></p><br>
<h3>Updates</h3><br>
<p>None at this time.</p></br>
 Portions © 2007 Agence France Presse, Al-Ahram, Al Hayat, Al-Quds Al-Arabi, Asharq Al-Awsat, An Nahar, Assabah, China Military Online,Chinanews.com, Guangming Daily, Xinhua News Agency, © 2007, 2013, 2014 Trustees of the University of Pennsylvania