Package: text-quality
Text QualityA package to determine the quality of a a digitized text, from a handwritten script or scanned print (HTR/OCR output).The current pipeline is tuned on (historic) Dutch language, and will not perform well on other languages.
However, theunderlying modelhas been used for other (Germanic) languages, and can be adapted and applied to texts of other languages and time periods.ExamplesGood quality (not necessarily perfect):Van
Malacca den 29 maart 1.
door zoo veel ruijmer handen te hebben,
[…]
Siac van waar op den 5=e deeser,
na onse verschijde adhortaties, is over
eeen gekomen
zoo meede van SiacBad quality:uijtkoops --
winst suijverevense versis
e ee
,, 19
1 oe
na aftrek van
5 p:s C: Commiss:s
t 1a per 't geheel t p=s lb. off @'t geheeke
[…]What's MissingPipelines for languages other than historic DutchAutomatic training procedure for creating and update pipelinesAdditional features such as publication year.Seethis notebookfor a semi-automated pipeline creation process.How to use text_qualityAfterinstallation, use theclassify_text_quality.pyscript to classify PageXML or plain text files.
For instance, if you want to classify all*.xmlfiles in thepages/directory, use the--globargument:classify_text_quality.py--glob"page/*.xml"--outputclassifications.csv--output-scoresPer input file, one output line is returned in CSV table format, along with the classification result:Good qualityMedium qualityBad qualityAll supported parameters:$classify_text_quality.py--helpusage: Classify the quality of a (digitized) text. [-h] [--input [FILE ...]] [--pagexml [FILE ...]] [--pagexml-glob PATTERN] [--output FILE] [--output-scores]options:-h, --help            show this help message and exit--output FILE, -o FILEOutput file; defaults to stdout.--output-scores       Output scores and text statistics.Input:--input [FILE ...], -i [FILE ...]Plain text file(s) to classify. Use '-' for stdin.--pagexml [FILE ...]  Input file(s) in PageXML format.--pagexml-glob PATTERN, --glob PATTERNA pattern to find a set of PageXML files, e.g. 'pagexml/*.xml'.NotesThe pipeline might emit warnings like this:UserWarning: X does not have valid feature names, but MLPClassifier was fitted with feature namesThis is due to the internals of theScikit-Learn Pipeline object, and can safely be ignored.The dependencies are pinned to specific versions.
While this prevents implicit updated even for patch-level updated of required libraries, it prevents misleading warnings emitted by varying Scikit-Learn versions.
Hence, requirement dependecies can be changed manually, if you are aware of these issues.The project setup is documented inproject_setup.md. Feel free to remove this document (and/or the link to this document) if you don't need it.InstallationTo install thetext_qualitypackage:pipinstall-Utext-qualityAlternatively, install the package from GitHub repository:gitclonehttps://github.com/LAHTeR/htr-quality-classifier.gitcdhtr-quality-classifier
python3-mpipinstall-U.DocumentationReadthedocsSoftware ArchitectureThis diagram shows the class design of thetext_qualitypackage.ContributingIf you want to contribute to the development of text_quality,
have a look at thecontribution guidelines.CreditsLogic and implementation are based onNautilus-OCR.This package was created withCookiecutterand theNLeSC/python-template.Badges(Customize these badges with your own links, and checkhttps://shields.io/orhttps://badgen.net/to see which other badges are available.)fair-software.eu recommendations(1/5) code repository(2/5) license(3/5) community registry(4/5) citation(5/5) checklisthowfairisOther best practicesStatic analysisCoverageDocumentationGitHub ActionsBuildCitation data consistencySonarCloudMarkDown link checker