DOI: 10.23695/gp75-6148
title: NyLLex v2
title: NyLLex v2

description: I. IDENTIFYING INFORMATION

Title*
NyLLex v 2.0

Subtitle

      A Novel Resource of Swedish Words Annotated with Reading Proficiency Level

Created by*

      Daniel Holmer (daniel.holmer@liu.se), Evelina Rennes
      (evelina.rennes@liu.se)

License(s)*
CC BY 4.0

Abstract*

      NyLLex is a lexical resource derived from books published by Sweden´s
      largest publisher of easy language texts. The entries are annotated with
      frequency counts distributed over six reading proficiency levels.

Funded by*
Vetenskapsrådet (2020-03580)

Cite as
[1]

Related datasets
[2], [3]

II. USAGE

Key applications
Text complexity analysis

Intended task(s)/usage(s)

      (1) Lexical analysis of easy language texts. (2) Lexical simplification

Recommended evaluation measures
-

Dataset function(s)
-

Recommended split(s)
-

III. DATA

Primary data*
Words (text)

Language*
Swedish

Dataset in numbers*
14983 entries

Nature of the content*

      Each entry in the resource contains a word, its part-of-speech tag
      (SUC-style), and a number of frequencies over different readability
      levels. Multi-word expressions are denoted by multiple words linked by
      underscores.

Format*
Comma-separated values (CSV) with the following columns:

word: a word in its lemma form

POS: a part-of-speech tag in the SUC-format

      level1_freq - level6_freq (six headers): the dispersed frequency of the
      word in the given reading proficiency level

      total_freq: the adjusted frequency for the word across all reading
      proficiency levels

      n_level1 - n_level6 (six headers): raw frequency of the word in the given
      reading proficiency level

      n_total: raw frequency for the word across all reading proficiency levels

Data source(s)*

      The words are collected from 247 easy language books published by
      NyponVilja förlag. The books were OCR-scanned from PDF-format and
      preprocessed by the authors. Unfortunately, the book dataset is not
      publicly available due to copyright reasons.

Data collection method(s)*
See [1]

Data selection and filtering*
See [1]

Data preprocessing*
See [1]

Data labeling*
"See "Format""

Annotator characteristics
-

IV. ETHICS AND CAVEATS

Ethical considerations

      The books contain words that when taken out of context can be seen as
      offensive. The authors have manually removed such entries, but can not
      guarantee that the resource is completely devoid of offensive words.

Things to watch out for
-

V. ABOUT DOCUMENTATION

Data last updated*
20220909

Which changes have been made, compared to the previous version*

      This version contain more entries than described in the original paper.
      This is due to two reasons: 1) An increased number of books available for
      the source material (from 247 to 280). 2) An updated method to filter out
      bad entries due to erraneous OCR-readings from the soruce PDFs. In
      practice, this means that the number of entries (unique words) of the
      resource is signifcantly larger (more than double the number of entries)
      in this version, since entries that only appear once in the source
      material are no longer discarded. However, for the total frequency counts
      for all entries, the difference between this updated version and the paper
      version is only around 2%.

Access to previous versions
-

This document created*
20221219, Daniel Holmer (daniel.holmer@liu.se)

This document last updated*
20230608, Aleksandrs Berdicevskis (aleksandrs.berdicevskis@gu.se)

Where to look for further details
See [1] and https://gitlab.liu.se/danho69/nyllex/

VI. OTHER

Related projects

References

      "[1]. Daniel Holmer and Evelina Rennes. 2022. NyLLex: A Novel Resource of
      Swedish Words Annotated with Reading Proficiency Level. In Proceedings of
      the Thirteenth Language Resources and Evaluation Conference, pages
      1326–1331, Marseille, France. European Language Resources Association.
      https://aclanthology.org/2022.lrec-1.141.pdf 
      [2]. Thomas François, Elena Volodina, Ildikó Pilán, Anaïs Tack. 2016.
      SVALex: a CEFR-graded lexical resource for Swedish foreign and second
      language learners. Proceedings of LREC 2016, Slovenia. 
      [3]. Elena Volodina, Ildikó Pilán, Lorena Llozhi, Baptiste Degryse, and
      Thomas François. 2016. SweLLex: Second language learners’ productive
      vocabulary. In Proceedings of the joint workshop on NLP for Computer
      Assisted Language Learning and NLP for Language Acquisition, pages 76–84,
      Umeå, Sweden. LiU Electronic Press."
description: I. IDENTIFYING INFORMATION

Title*
NyLLex v 2.0

Subtitle

      A Novel Resource of Swedish Words Annotated with Reading Proficiency Level

Created by*

      Daniel Holmer (daniel.holmer@liu.se), Evelina Rennes
      (evelina.rennes@liu.se)

License(s)*
CC BY 4.0

Abstract*

      NyLLex is a lexical resource derived from books published by Sweden´s
      largest publisher of easy language texts. The entries are annotated with
      frequency counts distributed over six reading proficiency levels.

Funded by*
Vetenskapsrådet (2020-03580)

Cite as
[1]

Related datasets
[2], [3]

II. USAGE

Key applications
Text complexity analysis

Intended task(s)/usage(s)

      (1) Lexical analysis of easy language texts. (2) Lexical simplification

Recommended evaluation measures
-

Dataset function(s)
-

Recommended split(s)
-

III. DATA

Primary data*
Words (text)

Language*
Swedish

Dataset in numbers*
14983 entries

Nature of the content*

      Each entry in the resource contains a word, its part-of-speech tag
      (SUC-style), and a number of frequencies over different readability
      levels. Multi-word expressions are denoted by multiple words linked by
      underscores.

Format*
Comma-separated values (CSV) with the following columns:

word: a word in its lemma form

POS: a part-of-speech tag in the SUC-format

      level1_freq - level6_freq (six headers): the dispersed frequency of the
      word in the given reading proficiency level

      total_freq: the adjusted frequency for the word across all reading
      proficiency levels

      n_level1 - n_level6 (six headers): raw frequency of the word in the given
      reading proficiency level

      n_total: raw frequency for the word across all reading proficiency levels

Data source(s)*

      The words are collected from 247 easy language books published by
      NyponVilja förlag. The books were OCR-scanned from PDF-format and
      preprocessed by the authors. Unfortunately, the book dataset is not
      publicly available due to copyright reasons.

Data collection method(s)*
See [1]

Data selection and filtering*
See [1]

Data preprocessing*
See [1]

Data labeling*
"See "Format""

Annotator characteristics
-

IV. ETHICS AND CAVEATS

Ethical considerations

      The books contain words that when taken out of context can be seen as
      offensive. The authors have manually removed such entries, but can not
      guarantee that the resource is completely devoid of offensive words.

Things to watch out for
-

V. ABOUT DOCUMENTATION

Data last updated*
20220909

Which changes have been made, compared to the previous version*

      This version contain more entries than described in the original paper.
      This is due to two reasons: 1) An increased number of books available for
      the source material (from 247 to 280). 2) An updated method to filter out
      bad entries due to erraneous OCR-readings from the soruce PDFs. In
      practice, this means that the number of entries (unique words) of the
      resource is signifcantly larger (more than double the number of entries)
      in this version, since entries that only appear once in the source
      material are no longer discarded. However, for the total frequency counts
      for all entries, the difference between this updated version and the paper
      version is only around 2%.

Access to previous versions
-

This document created*
20221219, Daniel Holmer (daniel.holmer@liu.se)

This document last updated*
20230608, Aleksandrs Berdicevskis (aleksandrs.berdicevskis@gu.se)

Where to look for further details
See [1] and https://gitlab.liu.se/danho69/nyllex/

VI. OTHER

Related projects

References

      "[1]. Daniel Holmer and Evelina Rennes. 2022. NyLLex: A Novel Resource of
      Swedish Words Annotated with Reading Proficiency Level. In Proceedings of
      the Thirteenth Language Resources and Evaluation Conference, pages
      1326–1331, Marseille, France. European Language Resources Association.
      https://aclanthology.org/2022.lrec-1.141.pdf 
      [2]. Thomas François, Elena Volodina, Ildikó Pilán, Anaïs Tack. 2016.
      SVALex: a CEFR-graded lexical resource for Swedish foreign and second
      language learners. Proceedings of LREC 2016, Slovenia. 
      [3]. Elena Volodina, Ildikó Pilán, Lorena Llozhi, Baptiste Degryse, and
      Thomas François. 2016. SweLLex: Second language learners’ productive
      vocabulary. In Proceedings of the joint workshop on NLP for Computer
      Assisted Language Learning and NLP for Language Acquisition, pages 76–84,
      Umeå, Sweden. LiU Electronic Press."