Package: lingx
LingXA library for introducing state-of-the-art metrics on measuring linguistic complexity developed by ContentSide and CRITT at Kent State University.LingX is:A library for calculating some of the psycholinguistics complexity metrics.A library for obtaining helpful metrics for translational process studies.A library for different factors related to the text analysis.A library with extended modules to easily integerate translational studies in CRITT TPR-DB.How does LingX generally work?LingX calculates different token-based and segment-based mono-bilingual complexity metrics. It internaly parses a given text into a dependency grammar graph. Using the graph and other linguistic information such as part-of-speech tagging, it can caculates different psycholinguistics, linguistic and translational process metrics. See the reference section for detailed information.LingX usesStanzastate-of-the-arts NLP library for different language-based tasks. Stanza is a collection of accurate and efficient tools for the linguistic analysis of many human languages. Stanza brings state-of-the-art NLP models to different languages.Quick StartRequirements and InstallationThe project is based on Stanza 1.2.1 and Python 3.6+. If you do not have Python 3.6, install it first. Then, in your favorite virtual environment, simply do:pip install lingxIf you are running project in Jupyter Notebook or Google Colab enviroments run the following command instead:!pip install lingxExample UsageLet's run a simple token-based psycholingual incomplete complexity theory (IDT) metric as a test. All you need to do is to make import related methods and codes as follows:fromlingx.utilsimportdownload_lang_modelsfromlingx.core.lang_modelimportget_nlp_objectfromlingx.utils.lximportget_sentence_lxnlp_en=get_nlp_object("en",use_critt_tokenization=False,package="partut")input="The reporter who the senator who John met attacked disliked the editor."tokens_scores_list,aggregated_score=get_sentence_lx(input,nlp_en,result_format="segment",complexity_type="idt",aggregation_type="sum")print(f"Tokens Scores List =={tokens_scores_list}")print(f"Aggregated Score =={aggregated_score}")This should print the incomplete complexity theory (IDT) metric list with related tokens and aggregated score using aggregated functionsum:Tokens Scores List == [['The', 1], ['reporter', 2], ['who', 3], ['the', 4], ['senator', 3], ['who', 4], ['John', 5], ['met', 2], ['attacked', 2], ['disliked', 2], ['the', 3], ['editor', 1], ['.', 0]]Aggregated Score == 32TutorialsWe provide a set of quick tutorials to get you started with the library:Tutorial 1: BasicsTutorial 2: Getting Incomplete Dependency Theory(=IDT) MetricTutorial 3: Getting Dependency Locality Theory(=DLT) MetricTutorial 4: Getting Combined IDT and DLT MetricTutorial 5: Getting Left-Embededness MetricTutorial 6: Getting Nested Nouns Distance MetricTutorial 7: Getting Bilingual Complexity Ratio MetricTutorial 8: Reading and Working with CRITT TPR-DBThe tutorials explain how the base metrics can be obtained. Let us know if anything is unclear.CRITT Translation Process Database (TPR-DB)The CRITT Translation Process Database (TPR-DB) is released underCreative Commons License (CC BY-NC-SA). Note that the availableEN-ZH_IMBst18database in this github belongs to CRITT TPR-DB.Citing LingXPlease cite:@inproceedings{Zou2021
  title={Syntactic Complexity and Translation Performance in English-to-Chinese Sight Translation},
  author={Zou, Longhui and Mirzapour, Mehdi and Jacquenet, Hélène},
  booktitle={Applied Linguistics and Professional Practice 2021},
  year={2021},
  publisher={Translational Data Analytics Institute, The Ohio State University}
}For IDT-based and DLT-based complexities, please citethis paper:@incollection{mirzapour2020,
  title={Measuring Linguistic Complexity: Introducing a New Categorial Metric},
  author={Mirzapour, Mehdi and Prost, Jean-Philippe and Retor{\'e}, Christian},
  booktitle={Logic and Algorithms in Computational Linguistics 2018 (LACompLing2018)},
  pages={95--123},
  year={2020},
  publisher={Springer}
}ContactPlease email your questions or comments toMehdi Mirzapour.LicenseLingX is licensed under the following MIT License (MIT) Copyright © 2021 ContentSide and CRITT at Kent State University.