cometadata/datacite-titles-descriptions-related-identifiers huggingface dataset:
DOI: 10.35111/3vdt-tm63
title: 2006 NIST Speaker Recognition Evaluation Training Set

description: <h3>Introduction</h3><br>
<p>2006 NIST Speaker Recognition Evaluation Training Set was developed by LDC and NIST (National Institute of Standards and Technology). It contains 595 hours of conversational telephone speech in English, Arabic, Bengali, Chinese, Hindi, Korean, Russian, Thai and Urdu and associated English transcripts used as training data in the NIST-sponsored 2006 Speaker Recognition Evaluation (SRE). The ongoing series of SRE yearly evaluations conducted by NIST are intended to be of interest to researchers working on the general problem of text independent speaker recognition. To this end the evaluations are designed to be simple, to focus on core technology issues, to be fully supported and to be accessible to those wishing to participate.</p><br>
<p>The task of the 2006 SRE evaluation was speaker detection, that is, to determine whether a specified speaker is speaking during a given segment of conversational telephone speech. The task was divided into 15 distinct and separate tests involving one of five training conditions and one of four test conditions. Further information about the test conditions and additional documentation is available in the <a href="docs/LDC2011S09/sre-06_evalplan-v9.pdf" rel="nofollow"> 2006 SRE Evaluation Plan</a>.</p><br>
<h3>Data</h3><br>
<p>The speech data in this release was collected by LDC as part of the <a href="https://www.ldc.upenn.edu/collaborations/past-projects" rel="nofollow">Mixer</a> project, in particular Mixer Phases 1, 2 and 3. The Mixer project supports the development of robust speaker recognition technology by providing carefully collected and audited speech from a large pool of speakers recorded simultaneously across numerous microphones and in different communicative situations and/or in multiple languages. The data is mostly English speech, but includes some speech in Arabic, Bengali, Chinese, Hindi, Korean, Russian, Thai and Urdu.</p><br>
<p>The telephone speech segments are multi-channel data collected simultaneously from a number of auxiliary microphones. The files are organized into three types: two-channel excerpts of approximately 10 seconds, two-channel conversations of approximately 5 minutes and summed-channel conversations also of approximately 5 minutes.</p><br>
<p>The speech files are stored as 8-bit u-law speech signals in separate SPHERE files. In addition to the standard header fields, the SPHERE header for each file contains some auxiliary information that includes the language of the conversation and whether the data was recorded over a telephone line.</p><br>
<p>English language transcripts in .ctm format were produced using an automatic speech recognition (ASR) system.</p><br>
<h3>Samples</h3><br>
<p>For an example of the data contained in this corpus, review this <a href="desc/addenda/LDC2011S09.wav" rel="nofollow">audio sample</a>.</p><br>
<h3>Updates</h3><br>
<p>None at this time.</p></br>
 Portions © 2004-2006, 2011 Trustees of the University of Pennsylvania
DOI: 10.35111/nvpd-ya81
title: Indian Language Part-of-Speech Tagset: Sanskrit

description: <h3> Introduction </h3> <p> Indian Language Part-of-Speech Tagset: Sanskrit, Linguistic Data Consortium (LDC) catalog number LDC2011T04 and isbn 1-58563-575-8, is a corpus developed by Microsoft Research (MSR) India to support the task of Part-of-Speech Tagging (POS) and other data-driven linguistic research on Indian Languages in general. It is created as a part of the <a href="http://research.microsoft.com/en-us/groups/mls/default.aspx" rel="nofollow">Indian Language Part-of-Speech Tagset (IL-POST)</a> project, a collaborative effort among linguists and computer scientists from MSR India, AU-KBC (Anna University, Chennai), Delhi University, IIT Bombay, Jawaharlal Nehru University (Delhi) and Tamil University (Tamilnadu). </p> <p> The goal of the IL-POST project is to provide a common tagset framework for Indian Languages that offers flexibility, cross-linguistic compatibility and resuability across those languages. It supports a three-level hierarchy of Categories, Types and Attributes. The corpus mainly consists therefore of two different levels of information for each lexical token: (a) lexical Category and Types, and (b) set morphological attributes and their associated values in the context. </p> <p> Sanskrit is the classical language of Indian and the oldest documented language of the Indo-European language family. It is also the liturgical language of Hinduism, Buddhism and Jainism and one of the twenty-two official languages of India. The name Sanskrit means refined, consecrated and sanctified. </p> <h3> Data </h3> This corpus contains 3,703 sentences (57,218 words) of manually annotated Sanskrit text selected from the <a href="http://en.wikipedia.org/wiki/Panchatantra" rel="nofollow">Panchatrantra</a> stories, a collection of animal fables in verse and prose dating from the third century BCE. All annotated data is provided in both xml and text files. The xml files are contained in the XML_files folder and the text files in the text_files folder. Each data file contains between 12,000-45,000 words. The XML file contains metadata about the material, such as language, encoding and data size. <h3> Annotation Procedure </h3> <p> The paper, Annotating Sanskrit corpus: adapting IL-POSTS included in this release, contains a detailed description of the annotation methodology.</p><h3>Sample</h3> <h3> Updates </h3> <p> Additional information, updates, bug fixes may be available in the LDC catalog entry for this corpus at <a href="http://catalog.ldc.upenn.edu/LDC2011T04" rel="nofollow">LDC2011T04</a>. </p> </br>
 Portions © 2010 Microsoft Research Labs India, Pvt. Ltd., © 2011 Trustees of the University of Pennsylvania
DOI: 10.35111/bwn1-3f08
title: Arabic Treebank: Part 2 v 3.1

description: <h3>Introduction</h3><br>
<p>Arabic Treebank: Part 2 (ATB2) v 3.1 was developed at the Linguistic Data Consortium (LDC). It consists of 501 newswire stories from Ummah Press with part-of-speech (POS), morphology, gloss and syntactic treebank annotation in accordance with the Penn Arabic Treebank (PATB) Guidelines developed in 2008 and 2009. This release represents a significant revision of LDC's previous ATB2 publication: <a href="http://catalog.ldc.upenn.edu/LDC2004T02" rel="nofollow">Arabic Treebank: Part 2 v 2.0 LDC2004T02</a>.</p><br>
<p>The ongoing PATB project supports research in Arabic-language natural language processing and human language technology development. The methodology and work leading to the release of this publication are described in detail in the documentation accompanying this corpus and in two research papers: <a href="https://www.ldc.upenn.edu/sites/www.ldc.upenn.edu/files/lrec2008-enhancing-arabic-treebank.pdf">Enhancing the Arabic Treebank: A Collaborative Effort toward New Annotation Guidelines </a>and <a href="https://www.ldc.upenn.edu/sites/www.ldc.upenn.edu/files/lrec2010-consistent-flexible-integration.pdf">Consistent and Flexible Integration of Morphological Annotation in the Arabic Treebank</a>.</p><br>
<h3>Data</h3><br>
<p>ATB2 v 3.1 contains a total of 144,199 source tokens before clitics are split, and 169,319 tree tokens after clitics are separated for the treebank annotation. Source texts were selected from Ummah Press news archives covering the period from July 2001 through September 2002.</p><br>
<h3>Sponsorship</h3><br>
<p>This work was supported in part by the Defense Advanced Research Projects Agency, GALE Program Grant No. HR0011-06-1-0003. The content of this publication does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.</p><br>
<h3>Samples</h3><br>
<p>Please view this <a href="desc/addenda/LDC2011T09_full.jpg">sample</a>.</p></br>
 Portions © 2001-2002 Ummah Press, © 2003, 2004, 2008, 2011 Trustees of the University of Pennsylvania
DOI: 10.35111/zxev-1k65
title: Turkish Broadcast News Speech and Transcripts

description: <h3>Introduction</h3><br>
<p>Turkish Broadcast News Speech and Transcripts was developed by <a href="http://www.boun.edu.tr/en-US/Content/About_BU/History.aspx" rel="nofollow">Bogazi&ccedil;i University</a>, Istanbul, Turkey and contains approximatley 130 hours of Voice of America (VOA) Turkish radio broadcasts and corresponding transcripts. This is part of a larger corpus of Turkish broadcast news data collected and transcribed with the goal to facilitate research in Turkish automatic speech recognition and its applications, such as speech retrieval.</p><br>
<p>The VOA material was collected between December 2006 and June 2009 using a PC and TV/radio card setup. The data collected during the period 2006-2008 was recorded from analog FM radio the 2009 broadcasts were recorded from digitial satellite transmissions. A quick manual segmentation and transcription approach was followed.</p><br>
<p>Speech recognition and retrieval experiments using the larger corpus can be found in the following journal article: Ebru Arisoy, Dogan Can, Siddika Parlak, Hasim Sak, and Murat Saraclar, Turkish Broadcast News Speech and Transcripts Transcription and Retrieval, IEEE Transactions on Audio, Speech and Language Processing, 17(5):874-883, July 2009.</p><br>
<p>For more information please visit <a href="http://busim.ee.boun.edu.tr/~speech" rel="nofollow">http://busim.ee.boun.edu.tr/~speech</a> or contact the principal investigator, Murat Sara&ccedil;lar.</p><br>
<h3>Data</h3><br>
<p>The data was recrded at 32 kHz and resampled at 16 kHz. After screening for recording quality, the files were segmented, transcribed, and verified. The segmentation occurred in two steps, an initial automatic segmentation followed by manual correction and annotation which included information such as background conditions and speaker boundaries.</p><br>
<p>The transcription guidelines were adapted from the LDC HUB4 and quick transcription guidelines. An English version of the adapted guidelines is provided with the data <a href="docs/LDC2012S06/RapidTrans.pdf" rel="nofollow">here</a>. The manual segmentations and transcripts were created by native Turkish speakers at Bo?azi&ccedil;i University using <a href="http://trans.sourceforge.net/en/presentation.php" rel="nofollow">Transcriber</a>. The transcriptions are provided in the ISO-8859-9 (Latin5) character set.</p><br>
<h3>Samples</h3><br>
<p>Please follow the links below for samples:</p><br>
<ul><br>
<li><a href="desc/addenda/LDC2012S06.wav" rel="nofollow">Audio</a></li><br>
<li><a href="desc/addenda/LDC2012S06.jpg" rel="nofollow">Transcript</a></li><br>
</ul><br>
<h3>Sponsorship</h3><br>
<p>Funding for this corpus collection effort came from TUBITAK Project 105E102 and Bogazici University Research Fund Project 05HA202.</p><br>
<h3>Updates</h3><br>
<p>None at this time.</p></br>
 Portions © 2012 Murat Saraçlar, Trustees of the University of Pennsylvania
DOI: 10.35111/69ts-ey63
title: Chinese Dependency Treebank 1.0

description: <h3>Introduction</h3><br>
<p>Chinese Dependency Treebank 1.0 was developed by the <a href="http://ir.hit.edu.cn/">Harbin Institute of Technologys Research Center for Social Computing and Information Retrieval</a> (HIT-SCIR). It contains 49,996 Chinese sentences (902,191 words) randomly selected from Peoples Daily newswire stories published between 1992 and 1996 and annotated with syntactic dependency structures.</p><br>
<h3>Data</h3><br>
<p>Ill-formed or short sentences were eliminated from the randomly-selected sentences prior to annotation. The data was segmented and annotated for part of speech (POS), syntactic structures, verb subclasses and noun compounds.Word segmentation and POS tagging were accomplished automatically using statistical models trained on a larger, annotated corpus of Peoples Daily newswire stories. Humans manually annotated the syntactic structures and corrected word segmentation errors. POS tags were not corrected.</p><br>
<p>The data is provided in the format of CoNLL-X and in UTF-8. One line presents information for one word. An empty line indicates the end of a sentence. Each line contains 10 columns separated with a tab.</p><br>
<h3>Samples</h3><br>
<p>Please click follow this <a href="desc/addenda/LDC2012T05.html" rel="nofollow">link</a> for a sample of the data.</p><br>
<h3>Updates</h3><br>
<p>None at this time.</p></br>
 Portions © 1992-1996 Peoples Daily, © 2012 Harbin Institute of Technology, Research Center for Social Computing and Information Retrieval, © 2012 Trustees of the University of Pennsylvania
DOI: 10.35111/633t-z038
title: Arabic Treebank - Broadcast News v1.0

description: <h3>Introduction</h3><br>
<p>Arabic Treebank - Broadcast News v1.0 was developed at the Linguistic Data Consortium (LDC). It consists of 120 transcribed Arabic broadcast news stories with part-of-speech, morphology, gloss and syntactic tree annotation.</p><br>
<p>The ongoing PATB project supports research in Arabic-language natural language processing and human language technology development. The methodology and work leading to the release of this publication are described in detail in the documentation accompanying this corpus.</p><br>
<h3>Data</h3><br>
<p>This release contains 432,976 source tokens before clitics were split, and 517,080 tree tokens after clitics were separated for treebank annotation. The source materials are Arabic broadcast news stories collected by LDC during the period 2005-2008 from the following sources: Abu Dhabi TV, Al Alam News Channel, Al Arabiya, Al Baghdadya TV, Al Fayha, Alhurra, Al Iraqiyah, Aljazeera, Al Ordiniyah, Al Sharqiyah, Dubai TV, Kuwait TV, Lebanese Broadcasting Corp., Oman TV, Radio Sawa, Saudi TV and Syria TV. The transcripts were produced by LDC.</p><br>
<h3>Samples</h3><br>
<p>Please follow this <a href="desc/addenda/LDC2012T07.jpg" rel="nofollow">link</a> for a sample from this corpus.</p><br>
<h3>Sponsorship</h3><br>
<p>This work was supported in part by the Defense Advanced Research Projects Agency, GALE Program Grant No. HR0011-06-1-0003. The content of this publication does not necessarily reflect the position or the policy of the Government, and no official endorsement should be inferred.</p><br>
<h3>Updates</h3><br>
<p>None at this time.</p></br>
 Portions © 2007-2008 Abu Dhabi TV, © 2007 Al Alam News Channel, © 2007-2008 Al Arabiya, © 2008 Al Baghdadya TV, © 2008 Al Fayha, © 2007-2008 Al Iraqiyah, © 2005-2007 Aljazeera, © 2007 Al Ordiniyah, © 2008 Al Sharqiyah, © 2005, 2007-2008 Dubai TV, © 2007 Kuwait TV, © 2008 Oman TV, © 2007 PAC, Ltd., © 2007-2008 Saudi TV, © 2007 Syria TV, © 2012 Trustees of the University of Pennsylvania
DOI: 10.35111/mv82-j246
title: Prague Czech-English Dependency Treebank 2.0

description: <h3>Introduction</h3><p>Prague Czech-English Dependency Treebank (PCEDT) 2.0 was developed by the <a href="http://ufal.mff.cuni.cz/" rel="nofollow"> Institute of Formal and Applied Linguistics</a> at <a href="http://www.cuni.cz/" rel="nofollow">Charles University</a> in Prague, Czech Republic. It is a corpus of Czech-English parallel resources translated, aligned and manually annotated for dependency structure, semantic labeling, argument structure, ellipsis and anaphora resolution. This release updates Prague Czech-English Dependency Treebank 1.0 (<a href="http://catalog.ldc.upenn.edu/LDC2004T25" rel="nofollow">LDC2004T25</a>) by adding English newswire texts so that it now contains over two million words in close to 100,000 sentences. </p><h3>Data</h3><p> The principal new material in PCEDT 2.0 is the inclusion of the entire Wall Street Journal data from Treebank-3 (<a href="http://catalog.ldc.upenn.edu/LDC99T42" rel="nofollow">LDC99T42</a>). Not included from PCEDT 1.0 are the Readers Digest material, the Czech monolingual corpus, and the English-Czech dictionary. </p><p>Each section is enhanced with a comprehensive manual linguistic annotation in the Prague Dependency Treebank style (<a href="http://catalog.ldc.upenn.edu/LDC2006T01" rel="nofollow">LDC2006T01</a>, Prague Dependency Treebank 2.0). The main features of this annotation style are:</p><ul> <li>dependency structure of the content words and coordinating and similar structures (function words are attached as their attribute values)</li> <li>semantic labeling of content words and types of coordinating structures</li> <li>argument structure, including an argument structure (valency) lexicon for both languages</li> <li>ellipsis and anaphora resolution</li> </ul><p>This annotation style is called <strong>tectogrammatical annotation,</strong> and it constitutes the <strong>tectogrammatical layer</strong> in the corpus.</p><p>Please consult the PCEDT <a href="http://ufal.mff.cuni.cz/pcedt2.0/" rel="nofollow">website</a> for more information and documentation.</p><h3>Samples</h3><p>Please follow this <a href="./desc/addenda/LDC2012T08.jpg" rel="nofollow">link</a> for a sample of the data included.</p><h3>Updates</h3><p> None at this time. </p></br>
 Portions © 1987-1989 Dow Jones & Company, Inc., © 2002-2012 Charles University in Prague, Institute of Formal and Applied Linguistics, © 1999, 2004, 2012 Trustees of the University of Pennsylvania
DOI: 10.35111/kjfh-jw12
title: American English Nickname Collection

description: <h3>Introduction</h3> <p>American English Nickname Collection was developed by <a href="http://www.intelius.com/corp/" rel="nofollow">Intelius, Inc</a>. and is a compilation of American English nicknames to given name mappings based on information in US government records, public web profiles and financial and property reports. This corpus is intended as a tool for the quantitative study of nickname usage in the United States such as in demographic and sociological studies. It has multiple potential human language technology applications as well, including entity extraction, coreference resolution, people search, language modeling and machine translation. </p><h3>Data</h3> <p>The American English Nickname Collection contains 331,237 distinct mappings encompassing millions of names. The data was collected and processed through a record linkage pipeline. The steps in the pipeline were (1) data cleaning, (2) blocking, (3) pair-wise linkage and (4) clustering. In the cleaning step, material was categorized, processed to remove junk and spam records and normalized to an approximately common representation. The blocking process utitlized an algorithm to group records by shared properties for determining which record pairs should be examined by the pairwise linker as potential duplicates. The linkage step assigned a score to record pairs using a supervised pairwise-based machine learning model. The clustering step combined record pairs into connected components and further partitioned each connected component to remove inconsistent pairwise links. The result is that input records were partitioned into disjoint sets called profiles, where each profile corresponded to a single person.</p> <p>The material is presented in the form of a comma delimited text file. Each line contains a first name, a nickname or alias, its conditional probability and its frequency. The conditional probability for each nickname is derived from the base data using an algorithim which calculates both the probability for which any alias refers to a given name and a threshold below which the mapping is most likely an error. This threshold eliminates typographic errors and other noise from the data. </p> <h3>Samples</h3> <p>Please view this <a href="desc/addenda/LDC2012T11.jpg" rel="nofollow">image sample</a>.</p> <p> Portions © 2012 Intelius, Inc., © 2012 Trustees of the University of Pennsylvania</p></br>
DOI: 10.35111/m5b6-4m82
title: English Web Treebank

description: <h3>Introduction</h3><br>
<p>English Web Treebank was developed by the Linguistic Data Consortium (LDC) with funding through a gift from Google Inc. It consists of over 250,000 words of English weblogs, newsgroups, email, reviews and question-answers manually annotated for syntactic structure and is designed to allow language technology researchers to develop and evaluate the robustness of parsing methods in those web domains.</p><br>
<h3>Data</h3><br>
<p>This release contains 254,830 word-level tokens and 16,624 sentence-level tokens of webtext in 1174 files annotated for sentence- and word-level tokenization, part-of-speech, and syntactic structure. The data is roughly evenly divided across five genres: weblogs, newsgroups, email, reviews, and question-answers. The files were manually annotated following the sentence-level tokenization guidelines for web text and the word-level tokenization guidelines developed for English treebanks in the <a href="http://projects.ldc.upenn.edu/gale/index.html" rel="nofollow">DARPA GALE</a> project. Only text from the subject line and message body of posts, articles, messages and question-answers were collected and annotated.</p><br>
<p>Weblogs are interactive web sites that display content as discrete entries or posts and allow viewers to comment on entries and engage in discussions. They are typically managed by individuals and use informal or colloquial language. The weblog data in this release was collected by LDC and covers the period 2003-2006.</p><br>
<p>Newsgroups are repositories of online discussions pertaining to a topic or interest area. They consist of threads that in turn contain articles with comments and discussion from group users. The newsgroup data in this release was collected by LDC and covers the period 2003-2006.</p><br>
<p>Email are messages sent to discrete individuals or well defined groups via the TCP-IP Simple Mail Transfer Protocol (SMTP). The email messages in this corpus are a subset of emails sent by Enron Corporation employees during the period 1999-2002. Specifically, those messages are contained in the <a href="http://verbs.colorado.edu/enronsent/" rel="nofollow">Enronsent Corpus</a>, a collection of 96,107 email messages from the sent folders of Enron email users which were processed to remove any content not generated by human users.</p><br>
<p>The reviews in this corpus were gleaned from online reviews of businesses and services on various Google web sites written by individuals. This information was provided to LDC by Google in 2011 the dates of individual reviews are not available.</p><br>
<p>Question-answers are posts from Yahoo!s community-driven question-answering web site, <a href="http://answers.yahoo.com/" rel="nofollow">Yahoo! Answers</a>, where individuals submit and answer questions which may be on any topic. This data was collected in 2011; the dates of individual question-answers were not collected.</p><br>
<h3>Samples</h3></br>
 Portions © 2012 Google Inc., © 2011 Yahoo! Inc., © 2012 Trustees of the University of Pennsylvania