Package: metalex
MetaLex Tool (tool for lexicographers and metalexicographers)metalexis general tool in AGPL License forlexicographicsandmetalexicographicsactivities.
For current development version of this tool, seeMetaLex/v2.0Usagemetalex proceeds in this way (open github project page to see the graphic)1-> Enhance image quality
2-> OCR
3-> Enhance quality of OCR text
4-> Codification parser of articles dictionaries
5-> Text codification (XML, HTML, LMF, TEI)
6-> Tools for metalexicographic analysisThis is an example of this process used with metalexI am a metalexicographer or linguist and I have paper dictionaries.
I want to perform a diachronic study of the evolution of the wording of
definitions in a collection of dictionaries available from period A to period B.Traditionally or at best, the contemporary metalexicographer (according to our point of view) would apply the following methodology :1- Scanning of printed materials (Scan) and enhance its qualities
2- Optical reading of the pictures (Ocrisation) = extract articles content
3- Manual Error Corrections  of text articles
4- Marking of the articles with regular standard
5- Metalexographical analysis / decryption of articlesmetalex through its modules operates in the same way by successively executing  each of these tasks automatically.1)  MetaLex enhances the quality of dictionary images
    **metalex.ocrtext.normalizeImage.EnhanceImages().filter(f.DETAIL)**
2) MetaLex extract from dictionary images all dictionary articles
    **metalex.ocrtext.make_ocr.image_to_text()**
3) MetaLex corrects dictionary articles
    **metalex.ocrtext.make_text_well()**
4) MetaLex marking dictionary articles depending of some standard
    **metalex.xmlised.put_xml('tei') or MetaLex.xmlised.put_xml('lmf')**
5) MetaLex generates some metalexicographics analysis of part of content dictionary
    **metalex.xmlised.handleStat()**Some other more complex processes can be done !RequirementsMetaLex is developed inPython 2.7environment, the following packages are required :We can install all extra package dependencies manuallysudoapt-getinstallbuild-essentiallibssl-devlibffi-devpython-devsudopipinstallCythonsudoapt-getinstalllibtesseract-devlibleptonica-devlibjpeg-devzlib1g-devlibpng-devsudoapt-getinstalltesseract-ocr-allsudoapt-getinstallpython-html5libsudoapt-getinstallpython-lxmlsudoapt-getinstallpython-bs4sudopipinstallpillowsudopipinstall--no-cache-dir-Ipillowsudopipinstallhttp://effbot.org/downloads/Imaging-1.1.7.tar.gzsudopipinstalltermcolorsudoCPPFLAGS=-I/usr/local/includepipinstalltesserocrOr follow these steps from github project pagesudo./config.sh# Install linux package dependenciessudopipinstall-rrequirements.txt# Install python module dependenciesor merely use (the preferred option is to begin with the extra linux packages)sudopipinstallmetalexHow to run MetaLex ?Run the help commandmetalex-h---------------------------------------------------------------|********************||**************||******************|---------------------------------------------------------------metalexisgeneraltoolforlexicographicsandmetalexicographicsactivitiesoptionalarguments:-h,--helpshowthishelpmessageandexit-v,--versionshowprogram'sversionnumberandexit-pPROJECTNAME,--projectPROJECTNAMEDefinedmetalexprojectname-cauthorcommentcontributors,--confprojectauthorcommentcontributorsDefinedmetalexconfigurationforthecurrentproject-i[IMAGEFILE],--dicimage[IMAGEFILE]Inputoneormultipledictionaryimage(s)file(s)forcurrentmetalexproject--dldDOWNLOADDownloadocropymodelfromGithubforcurrentmetalexproject-o{ocropy,tesserocr},--ocrtype{ocropy,tesserocr}OCRtypetouseforcurrentmetalexproject-m{modeldef,},--model{modeldef,}OCRLSTMmodeltouseforcurrentmetalexproject-dIMAGESDIR,--imagedirIMAGESDIRInputfoldernameofdictionaryimagefilesforcurrentmetalexproject--imgalgactiontypevalueSetalgorithmforenhancingdictionaryimagefilesforcurrentmetalexproject(actiontypemustbe:contrastorbrightorfilter)-rFILERULE,--fileruleFILERULEDefinedfilerulesthatweusetoenhancequalityofOCRresult-lLANG,--langLANGSetlanguageforopticalcharactersrecognitionandothersmetalextreatment-x{xml,lmf,tei},--xml{xml,lmf,tei}Definedoutputresulttreatmentofmetalex-s,--saveSaveoutputresultofthecurrentprojectinfiles-t,--terminalShowresultofthecurrenttreatmentintheterminal------------------------------------------------------------------------------metalexproject:specialThanktoBillformetalex-vagrantversion------------------------------------------------------------------------------Build the file rules of the project.MetaLex takes files using specific structure to enhance output text of OCR data (from dictionary image files).\Wfor word replacement,\Cfor character replacement and\Rfor regular expression replacement. The spaces between headers are used to to describe remplacement.\START
\MetaLex\project_name\type_of_project\lang\author\date
\W
/t'/t
/{/f.
/E./f.
\C
/i'/i
\R
/a-z+/ij
\ENDIf you want to use ocropy OCR, please download its models first : It is save at$home/metalex/models.metalex--dldmodelDefRun your project with the default parameters except dictionary images data and save results. You must create a folder containing dictionary image files such astest-files/images/.# [OCRopy OCR]    We defined a folder containing dictionary images for current processmetalex-d'test-files/images'-oocropy-mmodeldef-s# [Tesserocr OCR] Or you can define a single dictionary image filemetalex-i'test-files/images/LarClasIll_1911_gay-Trouin.jpg'-otesserocr-mmodeldef-sRun your project with your own set of parameters and save resultsmetalex-p'projectname'-c'author''comment''contributors'-d'test-files/images'-r'test-files/file_Rule.dic'-l'fra'-otesserocr-mmodeldef-sOUTPUT :For the first command (without parameters), the result in the console will produce this.NB :With parameters, error and warning messages will disappear.Open github project page to see the graphixContributorsSpecial thank toBillforMetaLex-vagrantversion for windows, Mac OS 6, LinuxReferencePlease don’t forget to cite this work :@article{Mboning-Elvis,
    title  ={Quand le TAL s'empare de la métalexicographie : conception d'un outil pour le métalexicographe},
    author ={Mboning, Elvis},
    url    ={https://github.com/Levis0045/MetaLex},
    date   ={2017-06-20},
    shool  ={Université de Lille 3},
    year   ={2017},
    pages  ={12},
    keywords ={métalexicographie, TAL, fouille de données, extraction d'information, lecture optique, lexicographie, Xmlisation, DTD}}