Package: info.gianlucacosta.cervantes
CervantesExtract a compact Spanish dictionary from Wikcionario, with eleganceIn Spanish literature - and all over the world - the works ofMiguel de Cervantesare considered anobra maestrabecause of their stylistic 🦋elegance, witty remarks andhumanisticdepth of thought......which is why I wish to dedicate this project to his memory: more precisely, this is my library for creatinga customized, Wiktionary-based corpus of the Spanish language.Cervantesis a type-checked library for Python, built on top ofWikiPrism, focusing on:ParsinganXML dumpofWikcionarioand extractingSpanish termsfrom each wiki pageClassifyingeach term according to a set ofgrammar categoriesProvidinga Spanish-related Dictionary, backed by a SQLite db, that can be used for custom analysis viaSQL queriesDespite its sophisticatedregex-based engine, Cervantes has a minimalist programming interface; furthermore, it is designed to be a core plugin ofJardinero- which makes it extremely simple to use, via a web-application user interface.No matter the scenario, it is essential to explore the SQL schema of its underlying database: for details, please consult the sections below.InstallationTo installCervantes, just run:pipinstallinfo.gianlucacosta.cervantesor, if you are using Poetry:poetryaddinfo.gianlucacosta.cervantesExtracting a custom dictionary from WikcionarioOnce Cervantes is installed in your Python environment, you can import it just like any other Python library......or you can run it as an extension module within Jardinero's infrastructure! 🥳Just make sure that bothJardineroand Cervantes are installed, then run:python-OO-minfo.gianlucacosta.jardineroinfo.gianlucacosta.cervantesYou will then be able to create a dictionary and perform SQL queries via Jardinero's web user interface.Cervantes also supports Jardinero'sdeveloper mode: in that case, the system will refer to your local copy of Wikcionario - whichmustbe a BZ2 archive residing at the following address:http://localhost:8000/eswiktionary-latest-pages-articles.xml.bz2Usually, you can make this URL available by running:python-mhttp.serverfrom within the directory containing yourWikcionario dump file.For a more detailed explanation about the developer mode, please refer to Jardinero's documentation.Database schemaEvery single table in the database created by Cervantes has two fields:entry(TEXT NOT NULL) - denoting the term within the dictionarypronunciation(TEXT) - the IPA pronunciation, with an ASCII apostrophe character (and not a more sophisticated Unicode symbol) before the syllable having the primary stressGiven the nature of the extraction process, there are no foreign keys enforcing consistency between tables (for example, betweenverbsandverb_forms) - but one can still perform JOINs according to one's needs.Table: prepositionsFieldTypeRequiredPrimary keyentryTEXT**pronunciationTEXTTable: interjectionsFieldTypeRequiredPrimary keyentryTEXT**pronunciationTEXTTable: conjunctionsFieldTypeRequiredPrimary keyentryTEXT**pronunciationTEXTTable: adverbsFieldTypeRequiredPrimary keyentryTEXT**pronunciationTEXTkindTEXT*Table: verbsFieldTypeRequiredPrimary keyentryTEXT**pronunciationTEXTkindTEXT*Table: pronounsFieldTypeRequiredPrimary keyentryTEXT**pronunciationTEXTkindTEXT*Table: articlesFieldTypeRequiredPrimary keyentryTEXT**pronunciationTEXTkindTEXT*Table: adjectivesFieldTypeRequiredPrimary keyentryTEXT**pronunciationTEXTreference_entryTEXT*Table: nounsFieldTypeRequiredPrimary keyentryTEXT**pronunciationTEXTgenderTEXT**number_traitTEXTreference_entryTEXT*Table: verb_formsFieldTypeRequiredPrimary keyentryTEXT**pronunciationTEXTinfinitiveTEXT**modeTEXT**tenseTEXT*personTEXT*The APIEven though this library is designed forJardinero, one can still use its functions in other Python programs - mostly in a custom subclass ofWikiPrism'sPipelineStrategy.As a matter of fact, one just needs a few functions from theinfo.gianlucacosta.cervantesnamespace:extract_terms(page: Page) -> list[SpanishTerm]: given a page, returns a list of Spanish terms from the page - and whose types can be imported from theinfo.gianlucacosta.cervantes.termsnamespace. In WikiPrism's model, this function is aTermExtractor[SpanishTerm]create_sqlite_dictionary(connection: Connection) -> SpanishSqliteDictionary: given a SQLite connection, creates aSqliteDictionary(from WikiPrism) that will become its owner and that can be used to read and write Spanish terms. Consequently, it is aSqliteDictionaryFactory[SpanishTerm]Parting thoughtsCervantesis a project that I created because I definitely needed to further explore Spanish morphology: actually, it was the initial kernel ofJardinero, which I later refactored into a separate library, as well asWikiPrism.Since it relies on a very dynamic source like Wikcionario, and despite the carefully-crafted parsing regular expressions, its output cannot be 100% accurate.Furthermore, it focuses on the linguistic aspects that I felt more appealing according to my own needs - which means that I had to discard information during the parsing, or even to include aspects that may be unnecessary in a different context.Consequently... feel free to experiment, and maybe to create your own library! ^__^Actually, one can even adopt Cervantes's patterns to create a linguistic module for Jardinero dedicated to another language! 🤩Further referencesJardinero- Python/TypeScript React web app for exploring natural languagesWikiPrism- Parse wiki pages and create dictionaries, fast, with PythonEos-core- Type-checked, dependency-free utility library for modern PythonMiguel de Cervantes