Andrea Seveso commited on
Commit
0e13466
·
1 Parent(s): 3a3f5a6

Change about

Browse files
Files changed (1) hide show
  1. src/about.py +7 -3
src/about.py CHANGED
@@ -28,15 +28,19 @@ TITLE = """<h1 align="center" id="space-title">👩‍🏫Invalsi Leaderboard
28
  # What does your leaderboard evaluate?
29
  INTRODUCTION_TEXT = """
30
  Welcome into <a href="https://crispresearch.it/"><b>CRISP Bicocca</b></a> Invalsi Leaderboard page!
31
- On this page you will find the Invalsi Leaderboard: The Invalsi tests are tests prepared by the Invalsi Institute, administered to primary and secondary school students to assess their knowledge about mathematics and the Italian language.
32
-
33
- We structured a benchmark for assessing Italian capabilities of both open and close source Large Language Models, using 10 Invalsi tests, including all different grades of difficulty. (Paper Link)
34
  """
35
 
36
  # Which evaluations are you running? how can people reproduce what you have?
37
  LLM_BENCHMARKS_TEXT = f"""
38
  ## INVALSI Benchmark
39
 
 
 
 
 
40
  ### Data Collection
41
  We have collected from public sources 58 unique tests, divided into 141 unique units, with 2114 questions and 2808 unique items. Some questions are subdivided into multiple items, each requiring a specific answer.
42
  The data has been gathered from the <a href="https://www.gestinv.it/Index.aspx"><b>Gestinv</b></a> database.
 
28
  # What does your leaderboard evaluate?
29
  INTRODUCTION_TEXT = """
30
  Welcome into <a href="https://crispresearch.it/"><b>CRISP Bicocca</b></a> Invalsi Leaderboard page!
31
+ We adapted the INVALSI benchmark for automated LLM evaluation, which involves rigorous adaptation of the test format to suit automated processing while retaining the essence of the original tests. In this leaderboard, we provide a detailed assessment of current LLMs, offering a crucial reference point for the academic community.
32
+ Researchers are invited to submit their models for ongoing evaluation, ensuring the benchmark remains a current and valuable resource.
33
+ For more information on the benchmark, please refer to our arXiv paper <a href="https://arxiv.org/abs/SOON"><b>here</b></a> and read the "About" section.
34
  """
35
 
36
  # Which evaluations are you running? how can people reproduce what you have?
37
  LLM_BENCHMARKS_TEXT = f"""
38
  ## INVALSI Benchmark
39
 
40
+ The INVALSI (National Institute for the Evaluation of the Education and Training System) test has been essential to Italy's educational assessment framework since the 2005-2006 academic year. It aims to evaluate students' competencies across various subjects, including Italian language and mathematics, at multiple stages of the education system in a standardised manner. The test's primary objective is to assess students' linguistic proficiency, a crucial skill that schools are tasked with developing as per national educational guidelines.
41
+
42
+ Since the test covers a wide range of linguistic and comprehension skills, using it to evaluate LLMs can provide a detailed view of a model's proficiency in handling real-world, nuanced language tasks designed for human learners. The test's structured and standardised nature makes it an excellent benchmark for comparing different LLMs with questions culturally and contextually relevant to Italian speakers. However, our findings are relevant across all languages since they assess various general capabilities, such as word formation and text comprehension abilities. Also, since it is designed for different educational stages, it provides varied levels of complexity and challenge. This aspect can gauge an LLM's capability at different difficulty levels, reflecting its potential scalability and adaptability across simpler to more complex linguistic tasks.
43
+
44
  ### Data Collection
45
  We have collected from public sources 58 unique tests, divided into 141 unique units, with 2114 questions and 2808 unique items. Some questions are subdivided into multiple items, each requiring a specific answer.
46
  The data has been gathered from the <a href="https://www.gestinv.it/Index.aspx"><b>Gestinv</b></a> database.