Package: info.gianlucacosta.jardinero
JardineroExtensible web application for exploring natural languagesIntroductionNatural languagesare assublimeas exquisite flowers in a garden - and from such a naturalistic simile stems the name of this web application:Jardinero, meaninggardener.I definitely needed a tool to perform morphological analysis over the Spanish language - that is, I wanted to find an answer to questions like:Why some Spanish words end with -tad, whereas others end with -dad? What are the differences between them, in terms of both morphology and cardinality?To solve this mystery - and several more - I decided to create Jardinero, a web application extracting my compact SQLite Spanish dictionary fromWikcionario, ready for custom SQL queries.While developing the project, I felt it would be nice to extend the approach to any language, thus creating the whole open source architecture consisting of:Eos-core- type-checked, dependency-free utility library for modern PythonWikiPrism- library for parsing wiki pages and creating dictionariesCervantes- WikiPrism-based library extracting a compact Spanish dictionary from WikcionarioJardinero: hybrid Python/TypeScript web application, with a Flask backend and a React frontend communicating via websocketsAs a core aspect, the architecture can be easilyextendedby creating Python modules and packages namedlinguistic modules.Main featuresJardinero's user interface enables users to:create a SQLite dictionary from a wiki file- whose URL depends on the current linguistic moduleperform queries- in SQL or even in a custom DSL - upon the internal dictionaryre-create the dictionary, especially when the data source gets frequent updatesPresentation on SpeakerDeckTo explore in detail how the overall architecture works, as well as the purpose and the creation process of its components, please consultmy presentation on SpeakerDeck:The making of Jardinero.RequirementsJardinero requires at leastPython 3.10- available atPython's official websiteor via your operating system's package manager.InstallationYou can install Jardinero just like any other PyPI package for your Python distribution:pipinstallinfo.gianlucacosta.jardineroRunning JardineroJardinero requires alinguistic module- for example,Cervantes, dedicated to the Spanish language:pipinstallinfo.gianlucacosta.cervantesJardinero should preferably be run with Python's-OOand-mcommand-line arguments:python-OO-minfo.gianlucacosta.jardinero<linguisticmodule>which, in the case of Cervantes, becomes:python-OO-minfo.gianlucacosta.jardineroinfo.gianlucacosta.cervantesThen, you can just point any browser tohttp://localhost:7000/Running in developer modeBy omitting the-OO(and even the-O) flag, Jardinero will start indeveloper mode- which enables additional aspects:Flask running withfile watchingenabledMore fine-grainedloggingHTTP redirectionto the frontend development serverPython's__debug__global variable set totrue- for example, in this case, Cervantes downloads from localhost and not from Wikcionario's official websiteFor simplicity, Jardinero's TOML project includes auxiliary scripts:Install the frontend as an NPM package:poetryrunpoeinstall-frontendAfter that, to start the frontend server during development, you can run:poetryrunpoestart-frontendAlternatively, for better debugging introspection, you can always runyarn starton the related project - to start Webpack's dev serverPython's static HTTP server, serving files from your$HOME/Downloadsdirectory:poetryrunpoestart-staticThe above command lines can be further simplified if you add the following alias to your shell configuration - especially.profilefor Bash:aliaspoe='poetry run poe'Once the above commands have been issued, you can just start Jardinero in development mode:python-minfo.gianlucacosta.jardinero<linguisticmodule>and finally open your browser to the usual address -http://localhost:7000/Extending JardineroJardinero is designed to be extensible! I created it to explore the nuances of the Spanish language, but it can support arbitrary combinations of parameters:source wiki URL- provided it points to a BZ2-compressed fileterm-extraction algorithmfrom each wiki pageSQL schemain the SQLite dbIt is definitely up to your needs and creativity! 😊Yourlinguistic modulecan be just a Python module (or a package) - within the current Python module search path - containing these functions:get_wiki_url: a() -> strfunction returning the URL of a BZ2-compressed XML wiki file, which in turn should have the format described inWikiPrismdocumentationextract_terms: a(Page) -> list[TTerm]function, extracting a list of terms from a given wiki pagecreate_sqlite_dictionary: a(Connection) => SqliteDictionary[TTerm]function creating an instance of a WikiPrismSqliteDictionaryfrom the given SQLite connection. In particular,it is the Dictionary that actually responds to queries, so you might want to design your own DSL via a custom subclass.The exact meaning ofTTermdepends on your linguistic model: to explore a real-world example, please refer toCervantes- my library dedicated to the analysis of the Spanish language.Final thoughtsJardinero's core point is the web UI for creating and querying custom dictionaries, as well as its extensible engine.Of course, there are limitations: if you need advanced features like pagination, charts, and even more analysis tools, you can still run Jardinero to create your custom SQL db, that will be stored at:$HOME/.jardinero/<module name>/dictionary.dbThen, you can also use your favorite database explorer - such as the excellent, open sourceDB Browser for SQLite.Further referencesThe making of Jardinero-Story of a software engineer who wanted to learn SpanishCervantes- Extract a compact Spanish dictionary from Wikcionario, with eleganceWikiPrism- Parse wiki pages and create dictionaries, fast, with PythonEos-core- Type-checked, dependency-free utility library for modern PythonSpecial thanksRainbow loaderfromPreloaders.net