Package: kotki
kotkiFast language translations without using the cloud using Neural Machine techniques. Written in C++. Based onBergamot.Uses language models borrowed from the Mozilla extension'Firefox Translations'.100% FOSS,Linux onlyQuick startRequirementsFor Ubuntu:sudoaptupdate&&sudoaptupgrade
sudoaptinstall-ycmakeccachebuild-essentialgitpkg-configrapidjson-devpybind11-devlibyaml-cpp-devpython3-devpython3-virtualenvlibopenblas-devlibpcre2-devlibprotobuf-devprotobuf-compilerlibsqlite3-devPythonpip install kotki -vInstall language translation modelsProgrammaticallyimportkotkikotki.scan()# auto-find language translation models# kotki.scan("/path/to/registry.json")  # or supply the path# English -> Germankotki.translate("Whenever I am at the office, I like to drink coffee.","ende")'Wann immer ich im büro bin, trinke ich gerne kaffee.'# Bulgarian -> Englishkotki.translate("Румънците получиха дълго чакани новини: пенсиите и минималната заплата ще бъдат увеличени от 2023 г.","bgen")'Romanians have received long-awaited news: pensions and minimum wages will be increased from 2023'# Dutch -> English>>>kotki.translate("Auto begeeft het nadat man benzine steelt in Breda, blijkt dieselauto te zijn","nlen")'Car breaks after man steals gas in Breda, turns out to be diesel car'# English -> Polish>>>kotki.translate("I am going outside to buy some Pierogi.","enpl")'Jadę na zewnątrz, żeby kupić Pierogi.'CLI$ kotki-cli --help
Usage: kotki-cli [OPTIONS]

  Translate some text.

Options:
  -i, --input TEXT         Text to translate  [required]
  -m, --model TEXT         Model names. Use -l to list. Leave empty to guess
                           the input language automatically.
  -r, --registry FILENAME  Path to registry.json. Leave empty for auto-
                           detection of translation models.
  -l, --list               List available models.
  -d, --debug              Print debug log.
  --help                   Show this message and exit.Self-hosted web-interfaceExample:kotki.kroket.io$ kotki-web --help
Usage: kotki-web [OPTIONS]

  Exposes kotki via HTTP web interface and provide an API.

Options:
  -h, --host TEXT          bind host (default: 127.0.0.1)  [required]
  -p, --port INTEGER       bind port (default: 7000)  [required]
  -d, --debug              run Quart web-framework in debug
  -r, --registry FILENAME  Path to registry.json. Leave empty for auto-
                           detection of translation models.
  --help                   Show this message and exit.C++Link againstkotki-lib(CMake target, seesrc/demo/for reference).#include<string>#include"kotki/kotki.h"usingnamespacestd;intmain(intargc,char*argv[]){auto*kotki=newKotki();kotki->scan();// auto loadedModels = kotki->listModels();  // show currently loaded language modelscout<<kotki->translate("This should work, in theory.","ende");// English to Germanreturn0;}whyKotki is aimed at developers who "just want to translate some text" in their C++ or Python applications without
too much headache, as other translation frameworks are often big, difficult to compile, non-performant, etc.Producing libkotkilibkotki.soorlibkotki.aVia CMakeInstallmarian-lite(and its dependencies) manually
(and if you are lazy, you can let kotki download the dependencies
automatically via-DVENDORED_LIBS=ON- though your mileage may vary).STATIC- Produce static binary (TODO: doesn't work yet)SHARED- Produce shared binaryBUILD_DEMO- Produce example demo application(s)cmake-DBUILD_DEMO=ON-DSTATIC=OFF-DSHARED=ON-Bbuild.
make-Cbuild-j6
sudomake-Cbuildinstall# install into /usr/local/...Via debian packagingsudoaptinstall-ydebhelper
dpkg-buildpackage-b-ucLibrary usage (CMake)cmake_minimum_required(VERSION3.16)find_package(kotkiREQUIRED)target_link_libraries(my_appPRIVATEkotki::kotki-lib)ModelsThe translation models are borrowed from the
MozillaFirefox Translationsextension.You need
to manually download these models.They are conveniently packaged into a single archive that can be
downloaded over atkotki/releases.Extract to~/.config/kotki/models/for automatic detection:mkdir-p~/.config/kotki/models/
wgethttps://github.com/kroketio/kotki/releases/download/v0.4.5/kotki_models_0.3.3.zip
unzipkotki_models_0.3.3.zip-d~/.config/kotki/modelsOr supply your own pathscan("/path/to/registry.json").Performance / footprintTranslations arefast- Translating a simple sentence is generallyunder10ms(except the first time, due to model loading). Translation models are loaded on-demand.
This means that model loading does not happen duringscan()but during the first use
oftranslate()- which typically takes (only)100ms(per model). So if you have
a project that uses bothtranslate('foo', 'enfr')andtranslate('foo', 'fren')- you'll be using 2 models (and consequently~50MBworth of RAM during the duration of your program).Note that translations are done synchronously (and thus are 'blocking').AcknowledgementsThis project was made possible through the combined effort of all researchers
andpartnersin the Bergamot project (Jerin Philip, et al). Thetranslation modelsare prepared as part of the Mozilla project. The translation engine used isbergamot-translatorwhich is based onmarian.Bergamot-TranslatorKotki differs from Bergamot-Translator. The changes are specified below:Removed async/blocking worker poolsRemoved async/callback style translationsRemoved code related to parsing of HTMLWork from a single JSON config file (registry.json)Dynamically generate marian configs 'on-the-fly'Simplified the example C++ CLI program (src/demo/kotki.cpp).Switch frommarian-devtomarian-liteSimplified Python bindingsSimplified the build system (cleaned up various CMakeLists.txt)Introduced automatic use ofccachefor compilationsSupply CMake configs for kotki (and its dependencies)Supply debian packaging for kotki (and its dependencies)Removed support for Apple, Microsoft, WASM (rip)Removed usage of proprietary libraries like CUDA, Intel MKLRemoved unit testsRemoved CI/CD definitionsIntroduced new dependency: rapidjsonDoxygen, and other documentation removedLicenseMPL 2.0