Package: pl-itn
pl_itnInverse Text Normalization is an NLP task of changing the spoken form of a phrase to written form, for example:one two three -> 1 2 3pl_itnis an opensource Polish ITN Python library and REST API for practical applications.This project is an implementation ofNeMo Inverse Text Normalizationfor Polish.Table of contentsPrerequisitesSetupDockerUsagegRPC serviceBuilding custom grammarsDocumentationContributingLicenseReferencesPrerequisitesForpyniniA standards-compliant C++17 compiler (GCC >= 7 or Clang >= 700)The compatible recent version of OpenFst built with the grm extensions (seedeps/install_openfst.md)SetupMake sure to first install prerequisites, especially OpenFST.Install from PyPIpipinstallpl_itnBuild from sourcepipinstall.Editable install for developmentpipinstall-e.[dev]DockerTo build docker image containing pl_itn library usepl_itn_lib.dockerfilefile.To build docker image with gRPC service usegrpc_service.dockerfilefile.dockerbuild-t<IMAGE:TAG>-f<DOCKERFILE>.UsageConsole appusage:pl_itn[-h](-tTEXT|-i)[--taggerTAGGER][--verbalizerVERBALIZER][--configCONFIG][--log-level{debug,info}]InverseTextNormalizationbasedonFiniteStateTransducers

options:-h,--helpshowthishelpmessageandexit-tTEXT,--textTEXTInputtext-i,--interactiveIfused,demowillprocessphrasesfromstdininteractively.--taggerTAGGER--verbalizerVERBALIZER--configCONFIGOptionallyprovideyamlconfigwithtaggerandverbalizerpaths.--log-level{debug,info}returnastepbackvalue.pl_itn-t"jest za pięć druga"jest01:55

pl_itn-t"drugi listopada dwa tysiące osiemnastego roku"2listopada2018rokuPython>>>frompl_itnimportNormalizer>>>normalizer=Normalizer()>>>normalizer.normalize("za pięć dwunasta")'11:55'DockerExisting docker image containing pl_itn library is required. For build command refer toDockersection.dockerrun--rm-it<IMAGE:TAG>--helpgRPC ServicegRPC service methods are described ingrpc_service/pl_itn_api/api.protofile. Docker container is suggested approach for running the service. For build command refer toDockersection.
Service within container serves on port 10010.Example of building the image and starting the service.dockerbuild-tpl_itn_service:test-fgrpc_service.dockerfile.
dockerrun-p10010:10010pl_itn_service:testBuilding custom grammarsCustom grammars can be built usingbuild_grammar/build_grammar.pyscript.There are three demo grammars available:not declined cardinal numbers (e.g. "jeden", "dwa", "trzy")declined cardinal numbers (e.g. "jednego", "dwóch", "trzech")ordinal numbers (e.g. "pierwszy", "druga", "trzecie")Normalization types can be included and excluded from the grammar through the config file, which is set by default tobuild_grammar/grammar_config.yaml.# cardinals_basic_forms: True# cardinals_declined: True# ordinals: True$python3build_grammar/build_grammar.py--grammars-dirall

$pl_itn\--taggerall/tagger.fst\--verbalizerall/verbalizer.fst\-t"Jeden trzech piąta"135# cardinals_basic_forms: True# cardinals_declined: False# ordinals: True$python3build_grammar/build_grammar.py--grammars-dircardinals_basic_ordinals

$pl_itn\--taggercardinals_basic_ordinals/tagger.fst\--verbalizercardinals_basic_ordinals/verbalizer.fst\-t"Jeden trzech piąta"1trzech5# cardinals_basic_forms: True# cardinals_declined: False# ordinals: False$python3build_grammar/build_grammar.py--grammars-dironly_basic_cardinals

$pl_itn\--taggeronly_basic_cardinals/tagger.fst\--verbalizeronly_basic_cardinals/verbalizer.fst\-t"Jeden trzech piąta"1trzechpiątaSeeDocumentationfor more details.DocumentationContributingLicenseRerencesK. Gorman. 2016. Pynini: A Python library for weighted finite-state grammar compilation. In Proc. ACL Workshop on Statistical NLP and Weighted Automata, 75-80.Y. Zhang, E. Bakhturina, K. Gorman, and B. Ginsburg. 2021. NeMo Inverse Text Normalization: From Development To Production.