cometadata/datacite-titles-descriptions-related-identifiers huggingface dataset:
DOI: 10.35111/khb2-hh45
title: Hungarian-English Parallel Text, Version 1.0

description: <h3>Introduction</h3> <p>Hungarian-English Parallel Text, Version 1.0 (also known as the "Hunglish Corpus") is a sentence-aligned Hungarian-English parallel corpus consisting of approximately two million sentence pairs. The corpus contains additional language resources for the Hungarian text, including a monolingual corpus, morphological toolset and aligner.</p> <p>Hungarian-English Parallel Text, Version 1.0 is a joint work of the <a href="http://mokk.bme.hu/en/" rel="nofollow">Media Research and Education Center</a> at the <a href="http://www.bme.hu/en" rel="nofollow">Budapest University of Technology and Economics (BUTE)</a> and the <a href="http://www.nytud.hu/depts/corpus/index.html" rel="nofollow">Corpus Linguistics Department</a> at the Hungarian Academy of Sciences <a href="http://www.nytud.hu/eng/index.html" rel="nofollow">Institute of Linguistics</a>. </p>Additional information about this release is available from the <a href="http://mokk.bme.hu/resources/hunglishcorpus/" rel="nofollow">corpus website maintained by BUTE</a>. <h3>File formats, character encoding</h3> <p>This publication is issued on CD as a tarred zip file. Commonly available utilities such as <a href="http://www.gzip.org/" rel="nofollow">Gnu Zip</a> or <a href="http://my.smithmicro.com/win/stuffit.html" rel="nofollow">Stuffit</a> will readily extract this publication from its compressed form.</p> <p>Sentence pair (.bi) files consist of tab-separated, matching sentence pairs. The .bi files do not contain segments where deletion or contraction occurred. They are also filtered based on quality, so the full reconstruction of the raw texts is impossible. Some .bi files were shuffled (sorted alphabetically).</p> <p>Alignment "ladder" (.lad) files preserve the whole of both input texts with ordering, even those segments that were not successfully aligned. In .lad files, every line is tab-separated into two columns. The first is a segment of the Hungarian text. The second is a (supposedly corresponding) segment of the English text. Such segments of the source or target text will generally consist of exactly one sentence on both sides, but can also consist of zero, or more than one, sentence. In the latter case, the special separating token " ~~~ " is placed between sentences. </p> <p>The encoding of the sentence pair and the alignment files is mixed: ISO Latin-2 on the Hungarian side, and ISO Latin-1 on the English side. The overwhelming majority of the texts use compatible subsets of these two encodings, so for viewing, the files can be considered ISO Latin-2 encoded.</p> <p>hu and en are the raw texts used, in ISO Latin-2 and ISO Latin-1 encoding respectively.</p> <h3>Samples</h3> <p>For an example of the data contained in this corpus, please examine this sample screen capture of <a href="./desc/addenda/LDC2008T01.jpg" rel="nofollow">bilingual text</a>. </p> </br>
Portions © 2005 Budapest University of Technology and Economics, © 2005 Hungarian Academy of Sciences Institute of Linguistics, © 2005 Diplomacy and Trade Magazine, © 1996, 2008 Trustees of the University of Pennsylvania
DOI: 10.35111/nh9x-6n14
title: Chinese Proposition Bank 2.0

description: <p>Chinese Proposition Bank 2.0 is a continuation of the <a href="http://verbs.colorado.edu/chinese/cpb" rel="nofollow">Chinese Propostion Bank project</a>, which aims to create a corpus of Chinese text annotated with information about basic semantic propositions. <a href="http://catalog.ldc.upenn.edu/LDC2005T23" rel="nofollow">Chinese Propostion Bank 1.0</a> consists of predicate-argument annotation on 250,000 words from <a href="http://catalog.ldc.upenn.edu/LDC2005T01" rel="nofollow">Chinese Treebank 5.0</a>. Chinese Proposition Bank 2.0 adds predicate-argument annotation on 500,000 words from <a href="http://catalog.ldc.upenn.edu/LDC2007T36" rel="nofollow">Chinese Treebank 6.0</a>. The data sources include newswire from Xinhua News Agency, articles from Sinorama Magazine, news from the website of the Hong Kong Special Administrative Region and transcripts from various Chinese broadcast news programs. </p><h3>Data</h3> <p>This release contains the predicate-argument annotation of 81,009 verb instances (11,171 unique verbs) and 14,525 noun instances (1,421 unique nouns). The annotation of nouns is limited to nominalizations that have a corresponding verb. The general annotation guidelines and the lexical guidelines (called frame files) for each verbal and nominal predicate are included in this release. </p> <table> <tr> <td>Total propositions for verbs:</td> <td>81,009</td> </tr> <tr> <td>Total propositions for nouns:</td> <td>14,525</td> </tr> <tr> <td>Total verbs framed:</td> <td>11,171</td> </tr> <tr> <td>Total framesets:</td> <td>11,776</td> </tr> <tr> <td>Verbs with multiple framesets:</td> <td>474</td> </tr> <tr> <td>Average framesets per verb:</td> <td>1.05</td> </tr> <tr> <td>Total nouns framed:</td> <td>1,421</td> </tr> <tr> <td>Total noun framesets:</td> <td>1,528</td> </tr> <tr> <td>Nouns with multiple framesets:</td> <td>48</td> </tr> <tr> <td>Average framesets per noun:</td> <td>1.08</td> </tr> </table> <h3>Samples</h3> <p>For an example of the data in this corpus, please examine this <a href="./desc/addenda/LDC2008T07.jpg" rel="nofollow">sample image</a>(jpeg) of a parse tree. </p> </br>
Portions © 2000-2001 China Broadcasting System, © 2000-2001 China Central TV, © 2000-2001 China National Radio, © 2000-2001 China Television System, © 1997 The Government of the Hong Kong Special Administrative Region, © 1996-2001 Sinorama Magazine, © 1994-1998 Xinhua News Agency, © 2001, 2004, 2005, 2007, 2008 Trustees of the University of Pennsylvania
DOI: 10.35111/aq26-5c13
title: NomBank v 1.0

description: <h3>Introduction</h3><br>
<p>NomBank is an annotation project at New York University that provides argument structure for instances of common nouns in the Penn Treebank (<a href="http://catalog.ldc.upenn.edu/LDC95T7" rel="nofollow">Treebank-2</a> and <a href="http://catalog.ldc.upenn.edu/LDC99T42" rel="nofollow">Treebank-3</a>). NomBank 1.0 was released by the Nombank project in December 2007. It covers all of the "markable" nouns in the Penn Treebank Wall Street Journal data. Specifically, it includes 114,576 propositions that were derived from looking at a total of 202,965 noun instances and choosing only those nouns who arguments occur in the text. The work of NomBank is related to the <a href="http://verbs.colorado.edu/%7Empalmer/projects/ace.html" rel="nofollow">PropBank </a>project at the University of Colorado. NomBank marks the sets of arguments that cooccur with nouns in <a href="../../../LDC2004T14">Proposition Bank I</a>, just as PropBank records such information for verbs.</p><br>
<p>Related resources and further information about NomBank are available from the <a href="http://nlp.cs.nyu.edu/meyers/NomBank.html" rel="nofollow">NomBank</a> project website.</p><br>
<h3>Data</h3><br>
<p>NomBank v. 1.0 is a human-readable version of NomBank 1.0. It contains data with licenses that are owned or managed by the Linguistic Data Consortium. A license to either Treebank-2 or Treebank-3 is required in order to obtain NomBank v. 1.0.</p></br>
Portions © 1987-1989 Dow Jones & Company, Inc., © 1995, 1999, 2008 Trustees of the University of Pennsylvania
DOI: 10.35111/fjkn-rv50
title: COMNOM v 1.0

description: <h3>Introduction</h3><br>
<p>COMNOM is an automatically enriched version of <a href="http://nlp.cs.nyu.edu/comlex/" rel="nofollow">COMLEX Syntax</a> that was created at New York University as part of the <a href="http://nlp.cs.nyu.edu/meyers/NomBank.html" rel="nofollow">NomBank</a> annotation project. COMLEX resources are distributed by the Linguistic Data Consortium (LDC) and consist of the following: <a href="http://catalog.ldc.upenn.edu/LDC98L21" rel="nofollow">COMLEX English Syntax Lexicon (LDC98L21)</a>, an English dictionary consisting of approximately 38,000 lemmas with detailed information about the syntactic characteristics of each lexical item and subcategorization (complement structures); and <a href="http://catalog.ldc.upenn.edu/LDC96T11" rel="nofollow">COMLEX Syntax Text Corpus Version 2.0 (LDC96T11)</a>.</p><br>
<p>COMNOM adds classes to COMLEX Syntax lexical entries using NOMLEX-PLUS, a dictionary with approximately 8,000 entries. COMNOM collected prepositions from NOMLEX-PLUS sub-categorizations (:VERB-SUBC, :OBJECT, :SUBJECT, etc.), deduced essential complements from them and added them to the existing COMLEX entry.</p><br>
<p>Further information about the methodology used in COMNOM can be found in <a href="http://nlp.cs.nyu.edu/meyers/nombank/those-other-nombank-dictionaries.pdf" rel="nofollow">Meyers, "Those Other NomBank Dictionaries -- Manual for Dictionaries that Come with NomBank"</a>. Related resources and further information about COMNOM and NomBank are available from the <a href="http://nlp.cs.nyu.edu/meyers/NomBank.html" rel="nofollow">Nom Bank</a> project website.</p><br>
<p>A license to COMLEX English Syntax Lexicon (LDC98L21) or COMLEX Syntax Text Corpus Version 2.0 (LDC96T11) is required in order to obtain COMNOM v. 1.0.</p><br>
<h3>Data</h3><br>
<p>This release includes three versions of COMNOM which correspond to the three versions of NOMLEX-PLUS and are characterized by the amount of corpus training that influenced their creation. The data used for training are the Wall Street Journal materials in the Penn Treebanks (<a href="http://catalog.ldc.upenn.edu/LDC95T7" rel="nofollow">Treebank-2</a> and <a href="http://catalog.ldc.upenn.edu/LDC99T42" rel="nofollow">Treebank-3</a>), with annotations from <a href="../../../LDC2004T14">Proposition Bank I</a> and <a href="http://catalog.ldc.upenn.edu/LDC2008T23" rel="nofollow">NomBank 1.0</a>.</p><br>
<p>The three versions are:</p><br>
<ul><br>
<li>COMNOM-clean.1.0 -- contains no information derived from annotated data</li><br>
<li>COMNOM.1.0 -- contains information from the entire annotated corpus</li><br>
<li>COMNOM-training.1.0 -- contains information from annotated data in sections 02-21 of the corpus only.</li><br>
</ul></br>
Portions © 1987-1989 Dow Jones & Company, Inc., © 1996, 1998, 2008 Trustees of the University of Pennsylvania
DOI: 10.35111/0xmx-k439
title: Czech Broadcast Conversation Speech

description: <h3>Introduction</h3> <p> Czech Broadcast Conversation Speech was prepared by researchers at the University of West Bohemia, Pilsen, Czech Republic, and consists of 40 hours of speech recorded from Czech Radio 1 in 2003. Transcripts corresponding to the audio files in this corpus are provided in <a href="http://catalog.ldc.upenn.edu/LDC2009T20" rel="nofollow">Czech Broadcast Conversation MDE Transcripts (LDC2009T20)</a>. These corpora join LDC's other Czech broadcast data sets: <a href="http://catalog.ldc.upenn.edu/LDC2004S01" rel="nofollow">Czech Broadcast News Speech (LDC2004S01)</a>, <a href="http://catalog.ldc.upenn.edu/LDC2004T01" rel="nofollow">Czech Broadcast News Transcripts (LDC2004T01)</a>, <a href="http://catalog.ldc.upenn.edu/LDC2000S89" rel="nofollow">Voice of America (VOA) Czech Broadcast News Audio (LDC2000S89)</a>, and <a href="http://catalog.ldc.upenn.edu/LDC2000T53" rel="nofollow">Voice of America (VOA) Czech Broadcast News Transcripts (LDC2000T53)</a>. </p><p>Czech Broadcast Conversation Speech consists of 72 single channel recordings of Radioforum, a live talk program broadcast by <a href="http://www.rozhlas.cz/radiozurnal/portal/" rel="nofollow">Czech Radio 1 (CRo1)</a> every weekday evening. Its format consists of invited guests (most often politicians) spontaneously answering topical questions posed by one or two interviewers. The number of interviewees in a single program varies from one to three, but typically, one interviewer and two interviewees appear in the program. The material includes passages of interactive dialogue, but longer stretches of monologue-like speech comprise the majority of the collected data. Radioforum also has an interactive segment where listeners call the studio and ask their own questions. That telephony speech was not transcribed in the current release. </p><h3>Data</h3> <p>Individual recordings range from 27 minutes to 36 minutes each. The recordings were collected during the period from February 12, 2003 through June 26, 2003. The signal is mono, sampled at 22.05 kHZ with 16-bit resolution, stored in Windows PCM waveform format. The names of the audio files refer to the broadcast date (rfYYMMDD.wav). </p><p>The table below contains details about the audio files and the transcripts: </p> <table width="25%"> <tr> <td>Number of shows</td> <td> 72 </td> </tr> <tr> <td>Number of word tokens</td> <td> 292.6k </td> </tr> <tr> <td>Number of unique words</td> <td> 30.5k </td> </tr> <tr> <td>Duration of transcribed speech</td> <td> 33.0h </td> </tr> <tr> <td>Total number of speakers</td> <td> 128 </td> </tr> <tr> <td>Male speakers</td> <td> 108 </td> </tr> <tr> <td>Female speakers</td> <td> 20 </td> </tr> </table><h3>Samples</h3> <ul> <li><a href="./desc/addenda/LDC2009S02.wav" rel="nofollow">Speech</a></li> </ul><h3>Sponsorship</h3> <p>The completion of this corpus was facilitated by funding provided by the Ministry of Education of the Czech Republic under projects No. ME909 and 2C06020.</p>  </br>
Portions © 2003 Cesky rozhlas 1 Radiozurnal, © 2009 Trustees of the University of Pennsylvania
DOI: 10.35111/6a5x-dv17
title: CSLU: S4X Release 1.2

description: <h3>Introduction</h3> <p> CSLU: S4X Release 1.2, Linguistic Data Consortium (LDC) catalog number LDC2009S03 and isbn 1-58563-523-5, was created by the Center for Spoken Language Understanding, Oregon Health and Science University (CSLU). The corpus consists of 36 speakers (22 male, 14 female) uttering 11 specified words. </p><p>The speakers repeated the following words six times on each of four channels: startrek, supernova, tektronix, generation, nebula, processing, singularity, 71523, abracadabra, sungeeta and computer. The four channels used were office phone, home phone, carbon microphone telephone and speaker phone. Each speech file has a corresponding time-aligned phoneme-level transcription (achieved using automatic forced alignment) and an automatically-generated world-level transcription. </p><p>Humans reviewed each utterance in two passes and classified it as good, bad, noisy or different. The results of this verification process are included in the /docs directory. </p><h3>Data</h3> <p>The data was recorded with the CSLU T1 digital data collection system. Each utterance is recorded as a separate file. These files were sampled at 8 khz 8-bit and stored as ulaw files. All of the data use the RIFF standard file format. This file format is 16-bit linearly encoded. </p> <h3>Samples</h3> <p>For an example of the data in this corpus, please listen to this recording of a subject speaking the word 'computer': <a href="./desc/addenda/LDC2009S03.wav" rel="nofollow">SD-1030-computer-t3-67</a>. </p> </br>
Portions © 1996, 1998, 2000, 2002 Center for Spoken Language Understanding, Oregon Health and Science University, © 2009 Trustees of the University of Pennsylvania
DOI: 10.6073/pasta/cdfb122158c50b8f114308a240040714
title: Soil Chemistry and Moisture in Macrosystems Biodiversity Project at Harvard Forest 2012

description: Patterns of biodiversity, such as the increase toward the tropics and the peaked curve during ecological succession, are fundamental phenomena for ecology. Such patterns have multiple, interacting causes, but temperature emerges as a dominant factor across organisms from microbes to trees and mammals, and across terrestrial, marine, and freshwater environments. However, there is little consensus on the underlying mechanisms, even as global temperatures increase and the need to predict their effects becomes more pressing. The purpose of this project is to generate and test theory for how temperature impacts biodiversity through its effect on biochemical processes and metabolic rate. A combination of standardized surveys in the field and controlled experiments in the field and laboratory measure diversity of three taxa -- trees, invertebrates, and microbes -- and key biogeochemical processes of decomposition in seven forests distributed along a geographic gradient of increasing temperature from cold temperate to warm tropical. Soil chemistry (TN, TC, NH4-N, NO3-N, and pH) and moisture measurements were taken from soil cores from an array of 21 1m2 subplots and processed by the University of Oklahoma Institute for Environmental Genomics as part of a macrosystems biodiversity and latitude project supported by the National Science Foundation under Cooperative Agreement DEB#1065836.
DOI: 10.35111/gs5s-gg06
title: Japanese Web N-gram Version 1

description: <h3>Introduction</h3> <p>Japanese Web N-gram Version 1, Linguistic Data Consortium (LDC) catalog number LDC2009T08 and isbn 1-58563-510-3, was created by Google Inc. It consists of Japanese "word" n-grams and their observed frequency counts generated from over 255 billion tokens of text. The length of the n-grams ranges from unigrams to seven-grams.</p> <p>The n-grams were extracted from publicly accessible web pages that were crawled by Google in July 2007. This data set contains only n-grams that appear at least 20 times in the processed sentences. Less frequent n-grams were simply discarded. Those web pages requiring user authentication, pages containing "noarchive" or "noindex" meta tags, and pages under other special restrictions were excluded from the final release. While the aim was to process only Japanese pages, the corpus may contain some pages in other languages due to language detection errors. This dataset will be useful for research in areas such as statistical machine translation, language modeling and speech recognition, among others. </p> <h3>Data</h3> <p>Before the n-grams were collected, the web pages were converted into UTF-8 encoding, normalized into Unicode Normalization Form KC (see below), and split into sentences. Ill-formed sentences were filtered out, and the remaining sentences were segmented into "words". </p><p>All strings were normalized into Unicode Normalization Form KC (NFKC), which is described in <a href="http://www.unicode.org/unicode/reports/tr15/" rel="nofollow">http://www.unicode.org/unicode/reports/tr15/</a>. Japanese strings were normalized according to the following rules: </p><ul> <li>Full-width letters/digits were converted to ASCII letters/digits</li> <li>Half-width katakana were converted to full-width katakana</li> <li>Glyphs for Roman digits were converted to ASCII characters </li> <li>Certain Japanese-specific symbols were converted </li> </ul><p>The vocabulary was restricted to "words" that appeared at least 50 times in the processed sentences. </p> <p>Statistical information about the corpus is set forth in the following table:</p> <table> <tr> <th>Data size</th> <td>The total compressed data size is about 26GB.</td> </tr> <tr> <th>Number of tokens: </th> <td>255,198,240,937</td> </tr> <tr> <th>Number of sentences: </th> <td> 20,036,793,177</td> </tr> <tr> <th> Number of unique unigrams: </th> <td> 2,565,424</td> </tr> <tr> <th> Number of unique bigrams: </th> <td> 80,513,289</td> </tr> <tr> <th>Number of unique trigrams: </th> <td> 394,482,216</td> </tr> <tr> <th>Number of unique 4-grams: </th> <td> 707,787,333</td> </tr> <tr> <th>Number of unique 5-grams: </th> <td> 776,378,943</td> </tr> <tr> <th>Number of unique 6-grams: </th> <td> 688,782,933</td> </tr> <tr> <th>Number of unique 7-grams: </th> <td> 570,204,252</td> </tr> </table><h3>Samples</h3> <p><a href="./desc/addenda/LDC2009T08_Bigram.png" rel="nofollow">Japanese Bigram</a> <a href="./desc/addenda/LDC2009T08_Trigram.png" rel="nofollow">Japanese Trigram</a> </p>  </br>
Portions © 2007 Google Inc., © 2009 Trustees of the University of Pennsylvania
DOI: 10.35111/eqds-c312
title: REFLEX Entity Translation Training/DevTest

description: <h3>Introduction</h3><br>
<p>REFLEX Entity Translation Training/DevTest was developed by the Linguistic Data Consortium for the <a href="http://projects.ldc.upenn.edu/ace/" rel="nofollow">Automatic Contact Extraction (ACE)</a> program. This release constitutes the complete set of training data and development test data for the 2007 REFLEX Entity Translation evaluation sponsored by the National Institute of Standards and Technology (NIST) and consists of approximately 67.5k words of newswire and weblog text for each of three languages: English, Chinese and Arabic. The data set is made up of 22.5k words of English data, 22.5k words of Chinese data, and 22.5k words of Arabic data translated into each of the other two languages and annotated for entities and TIMEX2 extents and normalization.</p><br>
<p>Entity Annotation. The annotations identify seven types of entities: Person, Organization, Location, Facility, Weapon, Vehicle and GeoPolitical Entity. Each type is further divided into subtypes (for instance, Person subtypes include Individual, Group and Indefinite). Annotators tagged all mentions of each entity within a document, whether named, nominal or pronominal. For every mention, the annotator identified the maximal extent of the string that represents the entity and labeled the head of each mention. Nested mentions were also captured. Each entity was classified according to its type and subtype. Each entity mention was further tagged according to its class such as specific, generic, attributive, negatively quantified or under specified. Annotators also reviewed the entire document to group mentions of the same entity together; they also labeled cases of metonymy, where the name of one entity is used to refer to another entity (or entities) related to it.</p><br>
<p>TIMEX2 Annotation. <a href="http://timexportal.wikidot.com/timex2" rel="nofollow">TIMEX2</a> annotation of events and temporal relations fulfills two objectives. The first is the interpretation of expressions that refer to time. Such expressions tell when something happened, or how long something lasted, or how often something occurs. Such expressions also often require knowledge of the temporal context in order to truly understand them. A second objective is the normalization of temporal expressions. This facilitates interoperability between systems. Problems occur, for example, when a programmer in France encodes "October sixteenth 1962" as "1962.10.16" and one in the U.S. encodes it as "10/16/1962". It will appear as if two different dates are being referenced. The standards presented here require that the same meaning is always encoded in the same way.</p><br>
<h3>Sample</h3><br>
<p>Please use this <a href="desc/addenda/LDC2009T11.jpg" rel="nofollow">link</a> for a sample.</p></br>
 Portions © 2000, 2003 Agence France-Presse, © 2003 The Associated Press, © 2000 Al Hayat, © 2000, 2002 An Nahar, © 1994-1998, 2000, 2003 Xinhua News Agency, © 1994-2009 Trustees of University of Pennsylvania