Omkar2118/datasets huggingface dataset:
BillSum
Full name: BillSum

BillSum is the first dataset for summarization of US Congressional and California state bills.

The BillSum dataset consists of three parts: US training bills, US test bills and California test bills. The US bills were collected from the Govinfo service provided by the United States Government Publishing Office (GPO). The corpus consists of bills from the 103rd-115th (1993-2018) sessions of Congress. The data was split into 18,949 train bills and 3,269 test bills. For California, bills from the 2015-2016 session were scraped directly from the legislature’s website; the summaries were written by their Legislative Counsel.

The BillSum corpus focuses on mid-length legislation from 5,000 to 20,000 character in length. The authors chose to measure the text length in characters, instead of words or sentences, because the texts have complex structure that makes it difficult to consistently measure words. The range was chosen because on one side, short bills introduce minor changes and do not require summaries. While the CRS produces summaries for them, they often contain most of the text of the bill. On the
other side, very long legislation is often composed of several large sections.

Source: [BillSum: A Corpus for Automatic Summarization of US Legislation](https://arxiv.org/pdf/1910.00523v2.pdf)

Introducing paper: BillSum: A Corpus for Automatic Summarization of US Legislation
Business Scene Dialogue

The Japanese-English business conversation corpus, namely **Business Scene Dialogue** corpus, was constructed in 3 steps:

1. selecting business scenes,
2. writing monolingual conversation scenarios according to the selected scenes, and
3. translating the scenarios into the other language.

Half of the monolingual scenarios were written in Japanese and the other half were written in English. The whole construction process was supervised by a person who satisfies the following conditions to guarantee the conversations to be natural:

- has the experience of being engaged in language learning programs, especially for business conversations
- is able to smoothly communicate with others in various business scenes both in Japanese and English
- has the experience of being involved in business

The BSD corpus is split into balanced training, development and evaluation sets. The documents in these sets are balanced in terms of scenes and original languages. In this repository we publicly share the full development and evaluation sets and a part of the training data set.

Source: [BSD](https://github.com/tsuruoka-lab/BSD)

Introducing paper: Designing the Business Conversation Corpus
X-WikiRE
Full name: X-WikiRE

X-WikiRE is a new, large-scale multilingual relation extraction dataset in which relation extraction is framed as a problem of reading comprehension to allow for generalization to unseen relations. 

Source: [X-WikiRE: A Large, Multilingual Resource for Relation Extraction as Machine Comprehension](https://arxiv.org/pdf/1908.05111v2.pdf)

Introducing paper: X-WikiRE: A Large, Multilingual Resource for Relation Extraction as Machine Comprehension
ProofWriter
Full name: ProofWriter

The ProofWriter dataset contains many small rulebases of facts and rules, expressed in English. Each rulebase also has a set of questions (English statements) which can either be proven true or false using proofs of various depths, or the answer is “Unknown” (in open-world setting, OWA) or assumed negative (in closed-world setting, CWA).

The dataset includes full proofs with intermediate conclusions, which models can try to reproduce.

The dataset supports various tasks:

- Given rulebase + question, what is answer + proof (w/intermediates)?
- Given rulebase, what are all the provable implications?
- Given rulebase + question without proof, what single fact can be added to make the question true?

Source: [Allen AI](https://allenai.org/data/proofwriter)

Introducing paper: ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language
Open PI
Full name: Open PI

**Open PI** is the first dataset for tracking state changes in procedural text from arbitrary domains by using an unrestricted (open) vocabulary. The dataset comprises 29,928 state changes over 4,050 sentences from 810 procedural real-world paragraphs from WikiHow.com.
The state tracking task assumes new formulation in which just the text is provided, from which a set of state changes (entity, attribute, before, after) is generated for each step, where the entity, attribute, and values must all be predicted from an open vocabulary.

Source: [Allen Institute of AI](https://allenai.org/data/openpi)

Introducing paper: A Dataset for Tracking Entities in Open Domain Procedural Text
hasPart KB
Full name: hasPart KB

This dataset is a new knowledge-base (KB) of hasPart relationships, extracted from a large corpus of generic statements. Complementary to other resources available, it is the first which is all three of: accurate (90% precision), salient (covers relationships a person may mention), and has high coverage of common terms (approximated as within a 10 year old’s vocabulary), as well as having several times more hasPart entries than in the popular ontologies ConceptNet and WordNet. In addition, it contains information about quantifiers, argument modifiers, and links the entities to appropriate concepts in Wikipedia and WordNet.

Source: [Allen Institute for AI](https://allenai.org/data/haspartkb)

Introducing paper: Do Dogs have Whiskers? A New Knowledge Base of hasPart Relations
SciDocs
Full name: SciDocs

SciDocs evaluation framework consists of a suite of evaluation tasks designed for document-level tasks.

Source: [Allen Institute for AI](https://github.com/allenai/scidocs)

Introducing paper: SPECTER: Document-level Representation Learning using Citation-informed Transformers
GenericsKB
Full name: GenericsKB

The **GenericsKB** contains 3.4M+ generic sentences about the world, i.e., sentences expressing general truths such as "Dogs bark," and "Trees remove carbon dioxide from the atmosphere." Generics are potentially useful as a knowledge source for AI systems requiring general world knowledge. The GenericsKB is the first large-scale resource containing naturally occurring generic sentences (as opposed to extracted or crowdsourced triples), and is rich in high-quality, general, semantically complete statements. Generics were primarily extracted from three large text sources, namely the Waterloo Corpus, selected parts of Simple Wikipedia, and the ARC Corpus. A filtered, high-quality subset is also available in GenericsKB-Best, containing 1,020,868 sentences.

Source: [Allen Institute for AI](https://allenai.org/data/genericskb)

Introducing paper: GenericsKB: A Knowledge Base of Generic Statements