upstage/dp-bench huggingface dataset:
content: text: Figure 12.2. A set of open textbooks printed in bulk are featured in this photo. Open textbooks from the
Open Course Library, picture by Tom Caswell, CC BY 2.0.

content: text: What tool(s) do you typically use in your course?

content: text: Ask whether the instructor utilizes your institution's course management system (Canvas,
Blackboard, etc.), or a separate course website to communicate and share content with students.
This may affect the tools and practices you recommend.

content: text: What supporting materials do you utilize for this course?

content: text: If the instructor relies on self-grading homework platforms or ancillary presentations and lecture
notes from publishers, you will want to discuss the various free and low-cost options available to
replace that content (See Chapter 15, Finding Ancillaries for OER).

content: text: Alternatively, does the instructor already supplement their course materials with course notes or
materials they have personally created? Often, when traditional materials are lacking or require
supplement, instructors will create notes, reading lists, or other content to "back up" any
traditional, commercial content used in their course. This instructor-created content can be
reused with OER as well, or even adapted into a new open resource in the future.

content: text: 164 | SUPPORTING OER ADOPTION
content: text: Version History

content: text: This page provides a record of edits and changes made to this book since its initial publication.
Whenever edits or updates are made in the text, we provide a record and description of those
changes here. If the change is minor, the version number increases by 0.1. If the edits involve
substantial updates, the edition number increases to the next whole number.

content: text: The files posted alongside this book always reflect the most recent version. If you find an error in
this book, please let us know in the Rebus Community forum, where reported errors will be visible
to others.

content: text: We will contact the author, make the necessary changes, and replace all file types as soon as
possible. Once we receive the updated files, this Version History page will be updated to reflect
the edits made.

content: text: Version History

content: text: Version History

content: html: <tr><td>Version</td><td>Date</td><td>Change</td><td>Affected Sections</td></tr><tr><td>1.0</td><td>April 30, 2022</td><td>Original</td><td></td></tr><tr><td>1.0</td><td>June 3, 2022</td><td>Small edits for clarity on Creative Commons licensing and attribution.</td><td>1. Introduction to Open Educational Resources</td></tr>
content: text: Upstage aims to enrich your business by providing
Easy-to-Apply AI solutions

content: text: Our Purpose

content: text: Making AI Beneficial

content: text: Our Mission

content: text: Easy-to-apply AI,
Everywhere

content: text: What We Do

content: text: Providing the world's best and easy-to-use
AI solutions for everyone

content: text: · Plug-and-play to cross/multi-cloud system
· Ensuring performance tailored to customer data via retraining
· Providing a platform that allows easy distribution and management of
AI solutions
· AI consulting service to help AI transformation
content: text: AI Pack

content: text: Upstage offers 3 AI packs that process unstructured information and data,
making a tangible impact on your business

content: html: <tr><td></td><td>OCR</td><td>Recommendation</td><td>Product semantic search</td></tr><tr><td>Pack</td><td>A solution that recognizes characters in an image and extracts necessary information</td><td>A solution that recommends the best products and contents</td><td>A solution that enables semantic search, analyzes and organizes key information in unstructured text data into a standardized form (DB)</td></tr><tr><td>Application</td><td>Applicable to all fields that require text extraction from standardized documents, such as receipts, bills, credit cards, ID cards, certificates, and medical receipts</td><td>Applicable to all fields that use any form of recommendation including alternative products, products and contents that are likely to be purchased next</td><td>Applicable to all fields that deal with various types of unstructured data containing text information that require semantic search and conversion into a DB</td></tr><tr><td>Highlight</td><td>Achieved 1st place in the OCR World Competition The team includes specialists who have presented 14 papers in the world's most renowned AI conferences</td><td>Team with specialists and technologies that received Kaggle's Gold Medal recommendation (Education platform) Proven superior performance of more than 170% compared to other global top-tier recommendation models</td><td>Creation of the first natural language evaluation system in Korean (KLUE) World's No.1 in Kaggle text embedding competition in E-commerce subject (Shopee)</td></tr>
content: text: Recommendation Pack: Track Record

content: text: Recommendation pack shows outstanding performance of 1.7~2.6 times that of
competing models even when using commercial service data

content: text: Comparison with Beauty Commerce
Recommendation Models

content: text: Recommendation model Hit Ratio comparison

content: text: Upstage
0.4048
Graph-RecSys
Upstage
0.3278
Attn-RecSys
aws
0.23496
Personalize
1.7X↑
Current Service
0.159
Recommendation
2.6X↑
Algorithm

content: text: Comparison Case of Domestic Subscription
Platform Recommendation Model

content: text: Comparison of quantitative evaluations among
personalized content recommendations

content: text: 0.03 0.06 0.09
Upstage
CustomerBERT
aws Personalize AWS Ready
14.3%↑
AutoEncoder
_RecVAE
AutoEncoder
_CDAE
AutoEncoder
_MultiVAE
GNN_LightGCN
CF_BPR
Statistic_
MostPop
Statistic_ : Recall@10, accuracy
CotergoryPop : NDCG@10, Ranking

content: text: Education Content Platform PoC Case

content: text: Comparison of prediction rates of correct/incorrect
answers based on personalized questions

content: text: 0.882
0.735
Compared to
regular model
20%↑
Upstage Traditional
DKT Model Statistical Model(IRT)
content: text: Semantic Search Pack: Value

content: text: SS Pack allows businesses to access further data more rapidly

content: text: The SS Pack can reduce the information acquisition time by returning all the information that matches the user's search intent.

content: text: The performance optimized for individual search systems is maintained by automatic updates of real-time search log records, augmented by
Upstage's technological know-how.

content: text: 1.8X ↑1

content: text: Higher Return of Information

content: text: Unlike existing search systems that only return
information limited to the entered search keywords, SS
Pack returns all relevant data that meet the user's
search intent

content: text: Optimal Attempt

content: text: Reduced Information Acquisition Time

content: text: By returning all semantic-based information of the
search keywords, the time required for information
acquisition is reduced drastically compared to that
of traditional keyword-matching search systems

content: text: 2
SOTA

content: text: Cutting-Edge Technology

content: text: The analysis of user logs saved in real-time allows us
to further optimize the individual search services
over time

content: text: 1 Evaluated against 100 internal test queries. Comparison of the amount of information returned with at least one keyword included in the search term and the
amount of returned information against that of SS Pack
2 State-of-the-art, current highest level of results and performance
content: text: C Additional Information

content: text: We present additional information for the sake of
space in the main paper.

content: text: Filtered task names. We present task names
we use to filter FLAN dervied datasets such as
OpenOrca in Table 8.

content: html: <tr><td>Filtered Task Name</td><td>task228 _arc_answer_generation_easy ai2_arcARCChallenge:1.0.0 ai2_arcARCEasy:1.0.0 task229_arc_answer_generation_hard hellaswag:1.1.0 task1389_hellaswag_completion cot_gsm8k cot_gsm8k_ii drop:2.0.0 winogrande:1.1.0</td></tr>

content: text: Table 8: Task names that we use to filter data for FLAN
derived datasets such as OpenOrca.

content: html: <tr><td>ARC</td><td>HellaSwag</td><td>MMLU</td><td>TruthfulQA</td><td>Winogrande</td><td>GSM8K</td></tr><tr><td>0.06</td><td>N/A</td><td>0.15</td><td>0.28</td><td>N/A</td><td>0.70</td></tr>

content: text: Table 9: Data contamination test results for SOLAR
10.7B-Instruct. We show 'result < 0.1, %' values where
a value higher than 0.9 indicates high probability of data
contamination. HellaSwag and Winogrande datasets are
not currently supported. We set SOLAR 10.7B as our
reference model when performing the data contamina-
tion tests.

content: text: Results on data contamination. To show the in-
tegrity of SOLAR 10.7B-Instruct, we also report
the data contamination test (Shi et al., 2023) results
in Table. 9. All four tested benchmark datasets
yield results well below the contamination thresh-
old, affirming the absence of data contamination
in our model. One interesting point is that the
value for GSM8K is noticeably higher than for
other datasets, even without contamination. One
potential reason for this is the stronger data similar-
ity in math-related instruction datasets.
content: text: Contents

content: text: 1. Overview of OCR Pack
2. Introduction of Product Services and Key Features
3. Product - Detail Specification
4. Integration Policy
5. FAQ

content: text: upstage |
content: text: Overview of OCR Pack

content: text: Base Model Performance Evaluation of Upstage OCR Pack

content: text: Upstage universal OCR model E2E performance
evaluation1

content: text: 100
95
95.5
90 92.4
85
82.07
80.41
80
75.66
75
70.23
70
65
Company Company upstage Company Company upstage
A2 B2 A2 B2
Scene (Photographed document image) Document (Scanned document image)

content: text: 1 Performance based on universal model, additional performance improvement is possible by implementing specialized
models according to business requirements
2 A: Universal model of global leading AI company / B: Universal model of leading AI company in Korea, 2022. 5 Test criteria

content: text: Upstage universal OCR model performance details: Document
criteria

content: text: 73.2
OCR-Recall3 7 94.2
94.1 4
5
89.0
OCR-Precision4 90.6 9
4 96.8
9
80.4
OCR-F15 1 92.
4 95.5
■ Company A
■ Company B
Parsing-F1 68.0
82.65 ■ upstage
65 70 75 80 85 90 95 100

content: text: 3 Recall: Percentage of what the OCR model predicted to be True from those that were actually True
4 Precision: Percentage of what the OCR model classifies as True, which is actually True
5 F1: Harmonic mean value of Recall and Precision
6. Parsing-F1: Comparison of parsing model F1 of both companies for business registration document
form. Company A is excluded from comparison due to the absence of the document parsing model.

content: text: upstage