Package: twol
twol: Compiler and other tools for two-level morphologyThese tools can be installed to your computer from PyPi by normal commands, such aspip3 install --user twolFor more instructions, see TWO Wikihttps://github.com/koskenni/twol/wikiNOTE:the programs are under development and some of them may not
work as expected, furthermore, the command-line parameters may still
change.  Thus, you should check the names and the meanigs of the
parameters by using the "-h" option.This repository contains various tools forSimplified Two-level
morphologywhich is a revised form of the original two-level
morphology as implemented in hfst-twolc (seehttps://github.com/hfst/hfst/wiki/HfstTwolc).  The tools are
implemented in Python and many of them use the HFST finite-state
transducer tools, especially its Python version (seehttps://github.com/hfst/python).The tools in this repository include:A compilertwol.pyortwol-compwhich reads in a set of
examples and a grammar file containing two-level rules.  The
compiler parses the rules, compiles them and tests them against the
examples. The compiler can write the compiled rules as binary
finite-state transducers into a file which can be used with the
HFST command line tools.Methods for aligning words or stems. These are useful for defining
underlying representations of lexical entries.  Morphophonemes in
the entries are a result of the alignment process.Documentation of the methods and the programs.  The source text for
documentation is in the docs directory and a human readable set of
interlinked documents is available at Readthedocs
(https://pytwolc.readthedocs.io)twol-comp: Compiler and rule tester for two-level rulesThe compiler is based on a well-chosen set of examples against which
the rules will be immediately tested.  Rules have no significance
before we have the examples.  One way to produce a set of two-level
examples is the alignment programs which are described later on in
this file.  The idea of simplified two-level model  is described inhttps://pytwolc.readthedocs.io/en/latest/intro.htmland the use of thetwol-compprogram is described inhttps://pytwolc.readthedocs.io/en/latest/compiletest.htmland some
other chapters there.Letter by letter alignment of wordsMethods for careful letter by letter alignment for e.g. cognate words
in historical linguistics or when matching different stems of a words
with each other. Alignment adds zero symbols where necessary in order
to match words or stems that differ in length. Alignment is
particularly important in two-level morphology because the alignment
determines what morphophonemes there will be.In the present context, alignment is the process of inserting some
zero symbols in the words so that the letters or phonemes in the
corresponding positions of the words are phonologically as similar as
possible, e.g. a Finnish word "kieli" and an Estonian word "keel"
could be aligned by inserting a zero symbol 'Ø':k i e l i
k e e l ØNow there are pairs of identical phonemes (k:k, e:e and l:l) and one
modification of a vowel (i:e) and the deletion of a word-final vowel
(i:Ø).There are stand-alone Python 3 programs which can be used from
command-line for aligning individual words:twol-alignerandmetrics.pywith which you can compare
cognate words of two languages. The latter reads an alphabet
definition and writes a weighted finite-state transducer (WFST) which
the former program needs for the concrete alignment.twol-multialigncompares two or more corresponding words or
morphs and aligns them.There is a suite of stand-alone programs for building morphophonemic
representations of morphemes.  The input consists of inflected word
forms given as a table where individual cells contain the word forms
where morph boundaries are indicated.  The forms with the same stem
are given as a row of the table and different forms correspond to the
columns of the table.  The programs are:twol-table2wordsreads in a table in CSV (Comma Separated
Values) format and writes it in a one word form per line CSV format.twol-words2zerofilledreads in the output of the above program
and aligns the morphs, i.e. stems of the same lexeme with each other
and alternate forms of affixes of the same grammatical form with each
other.  Aligned result is a table where the morphs include the
optimally inserted zeros as an additional column in the CSV format
file.twol-zerofilled2rawreads in the output of the above program
and produces an additional column which contains raw morphophonemic
forms of each morpheme.twol-raw2namedreads in the output of the above program and a
table of user-given shorter names for some raw morphophonemes and
writes out the examples as two-level symbol pairs, one example per
line.  The examples now consist of a sequence of symbol pairs where
the first component of a pair is the morphophoneme and the second
component is the surface character.  This file is used by the
two-level compiler in conjunction with the rules which the linguist
now can start to design.More information on these programs can be found at:https://pytwolc.readthedocs.io/en/latest/morphophon.htmland by
starting the programs with a--helpoption.LicensesThe programs aligner.py, charmetric.py and multialign.py are written
by Kimmo Koskenniemi alone and he has the copyright to these
programs. The programs are free software according to the GNU General
Public License Version 3, 29 June 2007, see LICENSE.txt in this
repository orhttps://www.gnu.org/licenses/gpl-3.0.en.htmlfor the
full text of the license.The file tyveb-n-stems.text is derived from a file available at the
Institute of Estonian Language (IEL)https://www.eki.ee/tarkvara/perlmorf/tyvebaas.pmf.  The license for
the file can be seen athttps://www.eki.ee/eki/licence.html