Package: icenlpy
IceNLPy🐍 A Python wrapper for IceNLPEver wanted to useIceNLPwith the ease of any other Python package? Now you can!IceNLPyis a Python wrapper for the Java-based IceNLP toolkit for Icelandic. Although originally made for fun, the package is fully functional and can be used in real-world projects.UsageIn Python:>>>fromicenlpyimporticetagger,iceparser,tokenizer>>>text="Hann er mjög virtur málfræðingur að norðan. Hvað segirðu um það?">>>tokens=tokenizer.split_into_sentences(text)>>>tokens['Hann er mjög virtur málfræðingur að norðan .','Hvað segirðu um það ?']>>>tagged=icetagger.tag_text(tokens)>>>tagged['Hann fpken er sfg3en mjög aa virtur lkensf málfræðingur nken að aa norðan aa . .\n','Hvað fshen segirðu sfg2en um ao það fpheo ? ?\n']>>>parsed=iceparser.parse_text(tagged)>>>parsed[[[NPHannfpken][VPbersfg3en][NP[AP[AdvPmjögaa]virturlkensf]málfræðingurnken][AdvPaðaanorðanaa]..],[[NPHvaðfshen][VPsegirðusfg2en][PPumao[NPþaðfpheo]]??]]>>>forsentinparsed:...print(f"Sentence text:{sent.text}\n")...print(sent.view)Sentencetext:Hannermjögvirturmálfræðinguraðnorðan.S0+-NP+-fpken:'Hann'+-VPb+-sfg3en:'er'+-NP+-AP+-AdvP+-aa:'mjög'+-lkensf:'virtur'+-nken:'málfræðingur'+-AdvP+-aa:'að'+-aa:'norðan'+-.:'.'Sentencetext:Hvaðsegirðuumþað?S0+-NP+-fshen:'Hvað'+-VP+-sfg2en:'segirðu'+-PP+-ao:'um'+-NP+-fpheo:'það'+-?:'?'CLI:The package also includes a basic command line tool to use the tools from the terminal. The tokenizer command is shown below:Input:$icenlpytokenizer--helpOutput:usage:icenlpytokenizer[-h][-iINPUT][-oOUTPUT][-of{1,2}][-t][input_text]positionalarguments:input_textDirecttexttoprocess

optionalarguments:-h,--helpshowthishelpmessageandexit-iINPUT,--inputINPUTInputfilepath-oOUTPUT,--outputOUTPUTOutputfilepath-of{1,2},--output-format{1,2}Thedesiredoutputformat.1foronetokenperline,2foronesentenceperline.-t,--tree-viewPrinttheoutputinatreeviewformat.OnlyappliestoIceParser.


$icenlpytokenizer"Hann er mjög virtur málfræðingur að norðan."-of1Hann
er
mjög
virtur
málfræðingur
að
norðan
.FeaturesThe package is designed to be as simple as possible, and integrate well into established workflows in NLP projects which use purely Python-implemented packages, e.g.GreynirEnginefor Icelandic orspaCyfor other languages. Dependencies are kept to a minimum outside the Python standard library and the IceNLP Java library itself.With this in mind the package is distributed with the original IceNLP JAVA clusters, and interacts with them using simple I/O calls via thesubprocessmodule. This is simple and lightweight, but has not been tested for specifically for performance or scalability.As of pre-release version 0.2.1, the package supports the following features:Tokenization via thetokenizermodulePart-of-speech tagging via theicetaggermoduleParsing via theiceparsermoduleAlong with the built-in features of IceNLP, the package also includes a simple way to convert IceNLP output to a more human-readable format, and dedicated object types to better manipulate the output in external pipelines. As such, it emulates the functionality of more modern NLP toolkits, particularly GreynirEngine.Future version of the package may include additional features, such as named entity recognition and lemmatization.AboutIceNLP, a groundbreaking NLP library for Icelandic was developed by Hrafn Loftsson from 2004 to 2007, with sporadic modifications and expansions in the mean time by students at the University of Reykjavík and the University of Iceland. The library is written in Java and sports a variety of NLP tools, including:TokenizationPart-of-speech taggingNamed entity recognitionParsingLemmatizationMorphological analysisAlthough today considered a legacy application and well outside the realm of state of the art technology, IceNLP is still maintained and has both practical and educational value. With this in mind, a simple way to implement the library in Python was developed, which is the basis for this Python package.PrerequisitesThis package runs on Python 3.8 or newer. Furthermore, as the package uses unmodified IceNLP source code, you will need to have Java installed on your system. For further information, see theIceNLP documentation.InstallationpipinstallicenlpyTo edit the source code, clone the repository and install the package in editable mode:gitclonehttps://github.com/hinrikur/IceNLPycdIceNLPy# [ Activate your virtualenv here if you have one ]pipinstall-e.AcknowledgementsIceNLP is developed and maintained by Hrafn Loftsson and the University of Reykjavík. The IceNLPy Python wrapper was developed by Hinrik Hafsteinsson.