Package: docassemble-textstat
TextstatModified from the original by Jonathan Pyle to remove the Pyphen
dependency because it is a GPL library and textstat is MIT licensed.Textstat is an easy to use library to calculate statistics from text. It helps determine readability, complexity, and grade level.Photo byPatrick TomassoonUnsplashUsage>>>importtextstat>>>test_data=("Playing games has always been thought to be important to ""the development of well-balanced and creative children; ""however, what part, if any, they should play in the lives ""of adults has never been researched that deeply. I believe ""that playing games is every bit as important for adults ""as for children. Not only is taking time out to play games ""with our children and other adults valuable to building ""interpersonal relationships but is also a wonderful way ""to release built up tension.")>>>textstat.flesch_reading_ease(test_data)>>>textstat.smog_index(test_data)>>>textstat.flesch_kincaid_grade(test_data)>>>textstat.coleman_liau_index(test_data)>>>textstat.automated_readability_index(test_data)>>>textstat.dale_chall_readability_score(test_data)>>>textstat.difficult_words(test_data)>>>textstat.linsear_write_formula(test_data)>>>textstat.gunning_fog(test_data)>>>textstat.text_standard(test_data)>>>textstat.fernandez_huerta(test_data)>>>textstat.szigriszt_pazos(test_data)>>>textstat.gutierrez_polini(test_data)>>>textstat.crawford(test_data)The argument (text) for all the defined functions remains the same -
i.e the text for which statistics need to be calculated.InstallYou can install textstat either via the Python Package Index (PyPI) or from source.Install using pippipinstalltextstatInstall using easy_installeasy_installtextstatInstall latest version from GitHubgitclonehttps://github.com/shivam5992/textstat.gitcdtextstat
pipinstall.Install from PyPIDownload the latest version of textstat fromhttp://pypi.python.org/pypi/textstat/You can install it by doing the following:tarxfztextstat-*.tar.gzcdtextstat-*/
pythonsetup.pybuild
pythonsetup.pyinstall# as rootLanguage supportBy default functions implement algorithms for english language.
To change language, use:textstat.set_lang(lang)The language will be used for syllable calculation and to choose
variant of the formula.Language variantsAll functions implementen_USlanguage. Some of them has also variants
for other languages listed below.Functionendeesfritnlplruflesch_reading_ease✔✔✔✔✔✔✔gunning_fog✔✔Spanish-specific testsThe following functions are specifically designed for spanish language.
They can be used on non-spanish texts, even though that use case is not recommended.>>>textstat.fernandez_huerta(test_data)>>>textstat.szigriszt_pazos(test_data)>>>textstat.gutierrez_polini(test_data)>>>textstat.crawford(test_data)Additional information on the formula they implement can be found in their respective docstrings.List of FunctionsSyllable Counttextstat.syllable_count(text)Returns the number of syllables present in the given text.Uses the Python modulePyphenfor syllable calculation.Lexicon Counttextstat.lexicon_count(text,removepunct=True)Calculates the number of words present in the text.
Optionalremovepunctspecifies whether we need to take
punctuation symbols into account while counting lexicons.
Default value isTrue, which removes the punctuation
before counting lexicon items.Sentence Counttextstat.sentence_count(text)Returns the number of sentences present in the given text.The Flesch Reading Ease formulatextstat.flesch_reading_ease(text)Returns the Flesch Reading Ease Score.The following table can be helpful to assess the ease of
readability in a document.The table is anexampleof values. While the
maximum score is 121.22, there is no limit on how low
the score can be. A negative score is valid.ScoreDifficulty90-100Very Easy80-89Easy70-79Fairly Easy60-69Standard50-59Fairly Difficult30-49Difficult0-29Very ConfusingFurther reading onWikipediaThe Flesch-Kincaid Grade Leveltextstat.flesch_kincaid_grade(text)Returns the Flesch-Kincaid Grade of the given text. This is a grade
formula in that a score of 9.3 means that a ninth grader would be able to
read the document.Further reading onWikipediaThe Fog Scale (Gunning FOG Formula)textstat.gunning_fog(text)Returns the FOG index of the given text. This is a grade formula in that
a score of 9.3 means that a ninth grader would be able to read the document.Further reading onWikipediaThe SMOG Indextextstat.smog_index(text)Returns the SMOG index of the given text. This is a grade formula in that
a score of 9.3 means that a ninth grader would be able to read the document.Texts of fewer than 30 sentences are statistically invalid, because
the SMOG formula was normed on 30-sentence samples. textstat requires at
least 3 sentences for a result.Further reading onWikipediaAutomated Readability Indextextstat.automated_readability_index(text)Returns the ARI (Automated Readability Index) which outputs
a number that approximates the grade level needed to
comprehend the text.For example if the ARI is 6.5, then the grade level to comprehend
the text is 6th to 7th grade.Further reading onWikipediaThe Coleman-Liau Indextextstat.coleman_liau_index(text)Returns the grade level of the text using the Coleman-Liau Formula. This is
a grade formula in that a score of 9.3 means that a ninth grader would be
able to read the document.Further reading onWikipediaLinsear Write Formulatextstat.linsear_write_formula(text)Returns the grade level using the Linsear Write Formula. This is
a grade formula in that a score of 9.3 means that a ninth grader would be
able to read the document.Further reading onWikipediaDale-Chall Readability Scoretextstat.dale_chall_readability_score(text)Different from other tests, since it uses a lookup table
of the most commonly used 3000 English words. Thus it returns
the grade level using the New Dale-Chall Formula.ScoreUnderstood by4.9 or loweraverage 4th-grade student or lower5.0–5.9average 5th or 6th-grade student6.0–6.9average 7th or 8th-grade student7.0–7.9average 9th or 10th-grade student8.0–8.9average 11th or 12th-grade student9.0–9.9average 13th to 15th-grade (college) studentFurther reading onWikipediaReadability Consensus based upon all the above teststextstat.text_standard(text,float_output=False)Based upon all the above tests, returns the estimated school
grade level required to understand the text.Optionalfloat_outputallows the score to be returned as afloat. Defaults toFalse.Spanish-specific testsÍndice de lecturabilidad Fernandez-Huertatextstat.fernandez_huerta(text)Reformulation of the Flesch Reading Ease Formula specifically for spanish.
The results can be interpreted similarlyFurther reading onThis blog postÍndice de perspicuidad de Szigriszt-Pazostextstat.szigriszt_pazos(text)Adaptation of Flesch Reading Ease formula for spanish-based texts.
Attempts to quantify how understandable a text is.Further reading onThis blog postFórmula de comprensibilidad de Gutiérrez de Polinitextstat.gutierrez_polini(text)Returns the Guttiérrez de Polini understandability index.Specifically designed for the texts in spanish, not an adaptation.
Conceived for grade-school level texts.
Scores for more complex text are not reliable.Further reading onThis blog postFórmula de Crawfordtextstat.crawford(text)Returns the Crawford score for the textReturns an estimate of the years of schooling required to understand the text.
The text is only valid for elementary school level texts.Further reading onThis blog postContributingIf you find any problems, you should open anissue.If you can fix an issue you've found, or another issue, you should open
apull request.Fork this repository on GitHub to start making your changes to the master
branch (or branch off of it).Write a test which shows that the bug was fixed or that the feature works as expected.Send a pull request!Development setupIt is recommended you use avirtual environment, orPipenvto keep your development work isolated from your
systems Python installation.$gitclonehttps://github.com/<yourname>/textstat.git# Clone the repo from your fork$cdtextstat
$pipinstall-rrequirements.txt# Install all dependencies$# Make changes$python-mpytesttest.py# Run tests