praj-17/QA-MCQ-Open-ended huggingface dataset:
Title: Sichuan
Context: Sichuan was China's most populous province before Chongqing became a directly-controlled municipality; it is currently the fourth most populous, after Guangdong, Shandong and Henan. As of 1832, Sichuan was the most populous of the 18 provinces in China, with an estimated population at that time of 21 million. It was the third most populous sub-national entity in the world, after Uttar Pradesh, India and the Russian Soviet Federative Socialist Republic until 1991, when the Soviet Union was dissolved. It is also one of the only six to ever reach 100 million people (Uttar Pradesh, Russian RSFSR, Maharashtra, Sichuan, Bihar and Punjab). It is currently 10th.

question: What are the three most populist provinces in China? 

answer: Guangdong, Shandong and Henan

question: What was the population of Sichuan in 1832? 

answer: 21 million

question: What is the tenth most populous sub-national entity in the world? 

answer: Sichuan

question: How many sub-national entities have surpassed 100 million people in total? 

answer: six

question: In what year was the Soviet Union disbanded? 

answer: 1991
Title: Sichuan
Context: Garzê Tibetan Autonomous Prefecture and Ngawa Tibetan and Qiang Autonomous Prefecture in western Sichuan are populated by Tibetans and Qiang people. Tibetans speak the Khams and Amdo Tibetan, which are Tibetic languages, as well as various Qiangic languages. The Qiang speak Qiangic languages and often Tibetic languages as well. The Yi people of Liangshan Yi Autonomous Prefecture in southern Sichuan speak the Nuosu language, which is one of the Lolo-Burmese languages; Yi is written using the Yi script, a syllabary standardized in 1974. The Southwest University for Nationalities has one of China's most prominent Tibetology departments, and the Southwest Minorities Publishing House prints literature in minority languages. In the minority inhabited regions of Sichuan, there is bi-lingual signage and public school instruction in non-Mandarin minority languages.

question: What are the largest minorities of western Sichuan? 

answer: Tibetans and Qiang

question: What languages do Tibetans in Sichuan speak? 

answer: Khams and Amdo Tibetan

question: What language do the Yi people speak? 

answer: Nuosu

question: In what year was the language of the Yi people standardized? 

answer: 1974

question: Which University houses one of China's most prominent Tibetology departments? 

answer: Southwest University for Nationalities
Title: Unicode
Context: Unicode is a computing industry standard for the consistent encoding, representation, and handling of text expressed in most of the world's writing systems. Developed in conjunction with the Universal Coded Character Set (UCS) standard and published as The Unicode Standard, the latest version of Unicode contains a repertoire of more than 120,000 characters covering 129 modern and historic scripts, as well as multiple symbol sets. The standard consists of a set of code charts for visual reference, an encoding method and set of standard character encodings, a set of reference data files, and a number of related items, such as character properties, rules for normalization, decomposition, collation, rendering, and bidirectional display order (for the correct display of text containing both right-to-left scripts, such as Arabic and Hebrew, and left-to-right scripts). As of June 2015[update], the most recent version is Unicode 8.0. The standard is maintained by the Unicode Consortium.

question: What was Unicode published as? 

answer: The Unicode Standard

question: What was unicode developed in conjunction with? 

answer: Universal Coded Character Set (UCS)

question: What is the most recent version of Unicode? 

answer: Unicode 8.0

question: Who maintains the Unicode Standard?  

answer: Unicode Consortium

question: Unicode contains how many thousands of characters in its repertoire?  

answer: 120,000 characters
Title: Unicode
Context: Unicode can be implemented by different character encodings. The most commonly used encodings are UTF-8, UTF-16 and the now-obsolete UCS-2. UTF-8 uses one byte for any ASCII character, all of which have the same code values in both UTF-8 and ASCII encoding, and up to four bytes for other characters. UCS-2 uses a 16-bit code unit (two 8-bit bytes) for each character but cannot encode every character in the current Unicode standard. UTF-16 extends UCS-2, using one 16-bit unit for the characters that were representable in UCS-2 and two 16-bit units (4 × 8 bits) to handle each of the additional characters.

question: What are the most commonly used encodings of Unicode? 

answer: UTF-8, UTF-16 and the now-obsolete UCS-2

question: What does UTF-8 use in terms of bytes?  

answer: UTF-8 uses one byte for any ASCII character

question: What does UTF-16 expand?  

answer: UCS-2

question: What type of code does UCS-2 use? 

answer: 16-bit code unit

question: How are two 16-bit units used?  

answer: (4 × 8 bits
Title: Unicode
Context: Unicode has the explicit aim of transcending the limitations of traditional character encodings, such as those defined by the ISO 8859 standard, which find wide usage in various countries of the world but remain largely incompatible with each other. Many traditional character encodings share a common problem in that they allow bilingual computer processing (usually using Latin characters and the local script), but not multilingual computer processing (computer processing of arbitrary scripts mixed with each other).

question: What is the aim of Unicode? 

answer: transcending the limitations of traditional character encodings

question: Traditional character encodings don't allow what type of computer processing? 

answer: multilingual computer processing

question: What is multilingual computer processing? 

answer: (computer processing of arbitrary scripts mixed with each other

question: Bilingual computer processing uses what characters? 

answer: usually using Latin characters

question: What is the major limitation of traditional character encodings? 

answer: wide usage in various countries of the world but remain largely incompatible with each other
Title: Unicode
Context: The first 256 code points were made identical to the content of ISO-8859-1 so as to make it trivial to convert existing western text. Many essentially identical characters were encoded multiple times at different code points to preserve distinctions used by legacy encodings and therefore, allow conversion from those encodings to Unicode (and back) without losing any information. For example, the "fullwidth forms" section of code points encompasses a full Latin alphabet that is separate from the main Latin alphabet section because in Chinese, Japanese, and Korean (CJK) fonts, these Latin characters are rendered at the same width as CJK ideographs, rather than at half the width. For other examples, see Duplicate characters in Unicode.

question: What were the first 256 code points of Unicode made identical to?  

answer: ISO-8859-1

question: Why were the first 256 code points made identical to ISO-8859-1? 

answer: to make it trivial to convert existing western text

question: What does the "fullwidth forms" section of code points encompass? 

answer: a full Latin alphabet that is separate from the main Latin alphabet

question: What are the CJK languages referenced? 

answer: Chinese, Japanese, and Korean
Title: Unicode
Context: In 1996, a surrogate character mechanism was implemented in Unicode 2.0, so that Unicode was no longer restricted to 16 bits. This increased the Unicode codespace to over a million code points, which allowed for the encoding of many historic scripts (e.g., Egyptian Hieroglyphs) and thousands of rarely used or obsolete characters that had not been anticipated as needing encoding. Among the characters not originally intended for Unicode are rarely used Kanji or Chinese characters, many of which are part of personal and place names, making them rarely used, but much more essential than envisioned in the original architecture of Unicode.

question: When was a surrogate character mechanism implemented in Unicode 2.0? 

answer: 1996

question: Why was a surrogate character mechanism implemented? 

answer: so that Unicode was no longer restricted to 16 bits

question: What are some characters not originally intended for Unicode?  

answer: rarely used Kanji or Chinese characters

question: Why are some foreign characters rarely used?  

answer: many of which are part of personal and place names

question: Why was the Unicode 2.0 codespace increased?  

answer: allowed for the encoding of many historic scripts
Title: Unicode
Context: Each code point has a single General Category property. The major categories are: Letter, Mark, Number, Punctuation, Symbol, Separator and Other. Within these categories, there are subdivisions. The General Category is not useful for every use, since legacy encodings have used multiple characteristics per single code point. E.g., U+000A <control-000A> Line feed (LF) in ASCII is both a control and a formatting separator; in Unicode the General Category is "Other, Control". Often, other properties must be used to specify the characteristics and behaviour of a code point. The possible General Categories are:

question: What are the General Categories of Unicode? 

answer: Letter, Mark, Number, Punctuation, Symbol, Separator and Other

question: What type of use is the General Category not helpful for? 

answer: not useful for every use, since legacy encodings have used multiple characteristics per single code point

question: What is the General Category in Unicode? 

answer: Other, Control

question: How many General Category properties does each code point have? 

answer: a single General Category property

question: What exists within the main Unicode categories?  

answer: subdivisions
Title: Unicode
Context: Code points in the range U+D800–U+DBFF (1,024 code points) are known as high-surrogate code points, and code points in the range U+DC00–U+DFFF (1,024 code points) are known as low-surrogate code points. A high-surrogate code point (also known as a leading surrogate) followed by a low-surrogate code point (also known as a trailing surrogate) together form a surrogate pair used in UTF-16 to represent 1,048,576 code points outside BMP. High and low surrogate code points are not valid by themselves. Thus the range of code points that are available for use as characters is U+0000–U+D7FF and U+E000–U+10FFFF (1,112,064 code points). The value of these code points (i.e., excluding surrogates) is sometimes referred to as the character's scalar value.

question: What are code points in the range U+D800-U+DBFF known as?  

answer: high-surrogate code points

question: What are code points in the range U+DC00-U+DFFF known as?  

answer: low-surrogate code points

question: what is another name for a high-surrogate code point?  

answer: leading surrogate

question: what is another name for a low-surrogate code point?  

answer: trailing surrogate

question: What is known as a character's scalar value?  

answer: the range of code points that are available for use as characters
Title: Unicode
Context: The set of graphic and format characters defined by Unicode does not correspond directly to the repertoire of abstract characters that is representable under Unicode. Unicode encodes characters by associating an abstract character with a particular code point. However, not all abstract characters are encoded as a single Unicode character, and some abstract characters may be represented in Unicode by a sequence of two or more characters. For example, a Latin small letter "i" with an ogonek, a dot above, and an acute accent, which is required in Lithuanian, is represented by the character sequence U+012F, U+0307, U+0301. Unicode maintains a list of uniquely named character sequences for abstract characters that are not directly encoded in Unicode.

question: What is an ogonek?  

answer: a dot above

question: How does Unicode encode characters? 

answer: associating an abstract character with a particular code point

question: How are some abstract characters represented in Unicode? 

answer: a sequence of two or more characters
Title: Unicode
Context: All graphic, format, and private use characters have a unique and immutable name by which they may be identified. This immutability has been guaranteed since Unicode version 2.0 by the Name Stability policy. In cases where the name is seriously defective and misleading, or has a serious typographical error, a formal alias may be defined, and applications are encouraged to use the formal alias in place of the official character name. For example, U+A015 ꀕ YI SYLLABLE WU has the formal alias yi syllable iteration mark, and U+FE18 ︘ PRESENTATION FORM FOR VERTICAL RIGHT WHITE LENTICULAR BRAKCET (sic) has the formal alias presentation form for vertical right white lenticular bracket.

question: What policy guaranteed that characters have a unique and immutable name? 

answer: Name Stability policy

question: What happens when a name is defective or misleading? 

answer: a formal alias may be defined

question: How long has this name immutability been guaranteed?  

answer: since Unicode version 2.0
Title: Unicode
Context: Unicode is developed in conjunction with the International Organization for Standardization and shares the character repertoire with ISO/IEC 10646: the Universal Character Set. Unicode and ISO/IEC 10646 function equivalently as character encodings, but The Unicode Standard contains much more information for implementers, covering—in depth—topics such as bitwise encoding, collation and rendering. The Unicode Standard enumerates a multitude of character properties, including those needed for supporting bidirectional text. The two standards do use slightly different terminology.

question: Who was Unicode developed in conjunction with? 

answer: International Organization for Standardization

question: What does Unicode share a character repertoire with? 

answer: the Universal Character Set

question: What includes topics like bitwise encoding, collation, and rendering? 

answer: The Unicode Standard

question: What do the two standards differ in?  

answer: slightly different terminology
Title: Unicode
Context: The Consortium first published The Unicode Standard (ISBN 0-321-18578-1) in 1991 and continues to develop standards based on that original work. The latest version of the standard, Unicode 8.0, was released in June 2015 and is available from the consortium's website. The last of the major versions (versions x.0) to be published in book form was Unicode 5.0 (ISBN 0-321-48091-0), but since Unicode 6.0 the full text of the standard is no longer being published in book form. In 2012, however, it was announced that only the core specification for Unicode version 6.1 would be made available as a 692-page print-on-demand paperback. Unlike the previous major version printings of the Standard, the print-on-demand core specification does not include any code charts or standard annexes, but the entire standard, including the core specification, will still remain freely available on the Unicode website.

question: What does the print on demand, core version not include?  

answer: any code charts or standard annexes

question: When was the Unicode Standard first published?  

answer: 1991

question: When was the latest version, Unicode 8.0, released? 

answer: June 2015

question: What was the last major version of Unicode to be published in book form?  

answer: Unicode 5.0

question: What year was it announced that only the core specification for Unicode would be printed in physical form? 

answer: 2012
Title: Unicode
Context: The Unicode Roadmap Committee (Michael Everson, Rick McGowan, and Ken Whistler) maintain the list of scripts that are candidates or potential candidates for encoding and their tentative code block assignments on the Unicode Roadmap page of the Unicode Consortium Web site. For some scripts on the Roadmap, such as Jurchen, Nü Shu, and Tangut, encoding proposals have been made and they are working their way through the approval process. For others scripts, such as Mayan and Rongorongo, no proposal has yet been made, and they await agreement on character repertoire and other details from the user communities involved.

question: Michael Everson, Rick McGowan, and Ken Whistler make up what group? 

answer: Unicode Roadmap Committee

question: What does the Unicode Roadmap Commmittee do?  

answer: maintain the list of scripts that are candidates or potential candidates for encoding

question: What proposal has been made for the Mayan script?  

answer: no proposal has yet been made

question: Where does the Unicode Roadmap Committee post information on these scripts? 

answer: Unicode Consortium Web site
Title: Unicode
Context: Unicode defines two mapping methods: the Unicode Transformation Format (UTF) encodings, and the Universal Coded Character Set (UCS) encodings. An encoding maps (possibly a subset of) the range of Unicode code points to sequences of values in some fixed-size range, termed code values. The numbers in the names of the encodings indicate the number of bits per code value (for UTF encodings) or the number of bytes per code value (for UCS encodings). UTF-8 and UTF-16 are probably the most commonly used encodings. UCS-2 is an obsolete subset of UTF-16; UCS-4 and UTF-32 are functionally equivalent.

question: How many mapping methods does Unicode define? 

answer: two

question: What are the two mapping methods that Unicode defines? 

answer: Unicode Transformation Format (UTF) encodings, and the Universal Coded Character Set (UCS) encodings

question: What do numbers in the names of the encodings indicate?  

answer: the number of bits per code value (for UTF encodings) or the number of bytes per code value

question: What are the most commonly used encodings?  

answer: UTF-8 and UTF-16

question: What does UCS stand for? 

answer: Universal Coded Character Set
Title: Unicode
Context: The UCS-2 and UTF-16 encodings specify the Unicode Byte Order Mark (BOM) for use at the beginnings of text files, which may be used for byte ordering detection (or byte endianness detection). The BOM, code point U+FEFF has the important property of unambiguity on byte reorder, regardless of the Unicode encoding used; U+FFFE (the result of byte-swapping U+FEFF) does not equate to a legal character, and U+FEFF in other places, other than the beginning of text, conveys the zero-width non-break space (a character with no appearance and no effect other than preventing the formation of ligatures).

question: What does BOM stand for?  

answer: Unicode Byte Order Mark

question: What specifies the BOM? 

answer: UCS-2 and UTF-16

question: what is the code point of the BOM? 

answer: U+FEFF

question: What is U+UFFE the result of?  

answer: byte-swapping U+FEFF
Title: Unicode
Context: The same character converted to UTF-8 becomes the byte sequence EF BB BF. The Unicode Standard allows that the BOM "can serve as signature for UTF-8 encoded text where the character set is unmarked". Some software developers have adopted it for other encodings, including UTF-8, in an attempt to distinguish UTF-8 from local 8-bit code pages. However RFC 3629, the UTF-8 standard, recommends that byte order marks be forbidden in protocols using UTF-8, but discusses the cases where this may not be possible. In addition, the large restriction on possible patterns in UTF-8 (for instance there cannot be any lone bytes with the high bit set) means that it should be possible to distinguish UTF-8 from other character encodings without relying on the BOM.

question: What is the UTF-8 standard?  

answer: RFC 3629

question: Byte order marks are forbidden in protocols using what standard?  

answer: UTF-8

question: Why is it possible to distinguish UTF-8 from other protocols? 

answer: the large restriction on possible patterns
Title: Unicode
Context: In UTF-32 and UCS-4, one 32-bit code value serves as a fairly direct representation of any character's code point (although the endianness, which varies across different platforms, affects how the code value manifests as an octet sequence). In the other encodings, each code point may be represented by a variable number of code values. UTF-32 is widely used as an internal representation of text in programs (as opposed to stored or transmitted text), since every Unix operating system that uses the gcc compilers to generate software uses it as the standard "wide character" encoding. Some programming languages, such as Seed7, use UTF-32 as internal representation for strings and characters. Recent versions of the Python programming language (beginning with 2.2) may also be configured to use UTF-32 as the representation for Unicode strings, effectively disseminating such encoding in high-level coded software.

question: How is UTF-32 widely used?  

answer: internal representation of text in programs

question: What programming language uses UTF-32 as internal representation of characters?  

answer: Seed7

question: what version of python can be used with UTF-32?  

answer: 2.2
Title: Unicode
Context: Unicode includes a mechanism for modifying character shape that greatly extends the supported glyph repertoire. This covers the use of combining diacritical marks. They are inserted after the main character. Multiple combining diacritics may be stacked over the same character. Unicode also contains precomposed versions of most letter/diacritic combinations in normal use. These make conversion to and from legacy encodings simpler, and allow applications to use Unicode as an internal text format without having to implement combining characters. For example, é can be represented in Unicode as U+0065 (LATIN SMALL LETTER E) followed by U+0301 (COMBINING ACUTE ACCENT), but it can also be represented as the precomposed character U+00E9 (LATIN SMALL LETTER E WITH ACUTE). Thus, in many cases, users have multiple ways of encoding the same character. To deal with this, Unicode provides the mechanism of canonical equivalence.

question: What combinations does unicode contain in normal use? 

answer: most letter/diacritic combinations

question: How is the latin small letter e represented in Unicode? 

answer: U+0065

question: How is the accent added to the small latin e?  

answer: U+0301

question: What precomposed character represents the small latin e with an accent?  

answer: U+00E9
Title: Unicode
Context: The CJK ideographs currently have codes only for their precomposed form. Still, most of those ideographs comprise simpler elements (often called radicals in English), so in principle, Unicode could have decomposed them, as it did with Hangul. This would have greatly reduced the number of required code points, while allowing the display of virtually every conceivable ideograph (which might do away with some of the problems caused by Han unification). A similar idea is used by some input methods, such as Cangjie and Wubi. However, attempts to do this for character encoding have stumbled over the fact that ideographs do not decompose as simply or as regularly as Hangul does.

question: The CJK ideographs comprise simpler elements called what in English? 

answer: radicals

question: Why have ideographs been unable to be simplified like Hangul? 

answer: ideographs do not decompose as simply or as regularly

question: What would be the benefit of Unicode decomposing  ideographs? 

answer: greatly reduced the number of required code points