Package: pylade
PyLaDepyladeis a lightweight language detection tool written in Python. The tool provides a ready-to-use command-line interface, along with more complex scaffolding for customized tasks.The current version ofpyladeimplements theCavnar-Trenkle N-Gram-based approach. However, the tool can be further expanded with customized language identification implementations.InstallationUsageTrain a model on a training setEvaluate a model on a test setDetect language of a text using a trained modelCustom implementations and corporaDevelopmentTestingGenerating documentation with SphinxNotesReferencesInstallationYou can install using pip:$pipinstallpyladeUsageFor a quick use, simply give the following command from terminal:$pylade"Put text here"enDone!If you want to get deeper and use some more advanced features, please keep reading.Note:you can obtain more information about each of the following commands using the--helpflag.Train a model on a training set$pylade_train\training_set.csv\--implementationCavnarTrenkleImpl\--corpus-readerTwitterCorpusReader\--outputmodel.json\--train-args'{"limit": 5000, "verbose": "True"}'--train-argsis a dictionary of arguments to be passed to thetrain()method of the chosen implementation (CavnarTrenkleImplin the example above). For an accurate description of the arguments please refer to thetrain()method docstring.NOTE: to define a new training set, you can check the format of the filetests/test_files/training_set_example.csv.Evaluate a model on a test set$pylade_eval\test_set.csv\--modelmodel.json\--implementationCavnarTrenkleImpl\--corpus-readerTwitterCorpusReader\--outputresults.json\--eval-args'{"languages": ["it", "de"], "error_values": 8000}'--eval-argsis a dictionary of arguments to be passed to theevaluate()method of the chosen implementation (CavnarTrenkleImplin the example above). For an accurate description of the arguments please refer to theevaluate()method docstring.Detect language of a text using a trained model$pylade\"Put text here"\--modelmodel.json\--implementationCavnarTrenkleImpl\--outputdetected_language.txt\--predict-args'{"error_value": 8000}'--predict-argsis a dictionary of arguments to be passed to thepredict_language()method of the chosen implementation (CavnarTrenkleImplin the example above). For an accurate description of the arguments please refer to thepredict_language()method docstring.Custom implementations and corporaDifferent language detection approaches can be implemented creating new classes that inherit from theImplementationclass. This class should be considered as an interface whose methods are meant to be implemented by the inheriting class.Customized corpus readers can be created the same way, inheriting from theCorpusReaderinterface instead.DevelopmentTestingYou can install development requirements using Poetry (poetry install). This will also install requirements needed for testing.To run tests, just runtoxfrom the package root folder.Generating documentation with SphinxPyLaDe's documentation is generated using Sphinx. If you want to update the docs, you can install the necessary dependencies with Poetry:$poetryinstall--withdocsDocumentation files are automatically generated from code docstrings. To rebuild the documentation to take changes into consideration, just run the following:$cddocs$makehtmlNotesThe default model (data/model.json) has been trained usinglimit = 5000. This value provides a good balance between computational performance and accuracy. Please note that this might change if you use your own data to train a new model.ReferencesCavnar, William B., and John M. Trenkle. "N-gram-based text categorization."Ann Arbor MI48113.2 (1994): 161-175.