Package: dafsa
DAFSA is a library for computingDeterministic Acyclic Finite State
Automata(also known as “directed acyclic word graphs”, or DAWG). DAFSA are data
structures derived fromtriesthat allow to represent a set of sequences (typically character strings
orn-grams) in the form of a directed acyclic graph with a single
source vertex (thestartsymbol of all sequences) and at least one
sink edge (finalsymbols, each pointed to by one or more sequences).
In the current implementation, a trait of each node expresses whether it
can be used a sink.The primary difference between DAFSA and tries is that the latter
eliminates suffix and infix redundancy, as in the example of Figure 1
(from the linked Wikipedia article) storing the set of strings"tap","taps","top", and"tops". Even though DAFSAs
cannot be used to store precise frequency information, given that
multiple paths might reach the same terminal node, they still allow to
estimate the sampling frequency; being acyclic, they can also reject any
sequence not included in the training. Fuzzy extensions will allow to
estimate the sampling probability of unobserved sequences.Trie vs. DAFSAThis data structure is a special case of a finite state recognizer that
acts as a deterministic finite state automaton, as it recognizes all and
only the sequences it was built upon. Frequently used in computer
science for the space-efficient storage of sets of sequences without
common compression techniques, such as dictionary or entropy types, or
without probabilistic data structures, such as Bloom filters, the
automata generated by this library are intended for linguistic
exploration, and extend published models by allowing to approximate
probability of random observation by carrying information on the weight
of each graph edge.Installation and usageThe library can be installed as any standard Python library withpip, and used as demonstrated in the following snippet:In any standard Python environment,dafsacan be installed with:$pipinstalldafsaAcondapackage is also available, and can be installed (from thetresoldichannel) with:$condainstall-ctresoldidafsaDetailed instructions on how to use the library can be found in theofficial
documentation.
For most purposes, it is enough to pass a list of sequences to theDAFSAobject:>>>fromdafsaimportDAFSA>>>print(DAFSA(["dib","tip","tips","top"]))DAFSAwith8nodesand9edges(4insertedsequences)+--#0: 0(#1/4:<d>/1|#4/4:<t>/3) [('t', 4), ('d', 1)]+--#1: n(#2/1:<i>/1) [('i', 2)]+--#2: n(#3/1:<b>/1) [('b', 3)]+--#3: F() []+--#4: n(#5/3:<i>/2|#8/3:<o>/1) [('i', 5), ('o', 8)]+--#5: n(#6/2:<p>/2) [('p', 6)]+--#6: F(#3/2:
/1) [('s', 3)]+--#8: n(#3/1:<p>/1) [('p', 3)]Full documentation is availableat
ReadTheDocs.io.ShowcaseBasic example:First exampleOutput can be textual, GML, DOT, or (via dot and third-party
software) PNG, PDF, ASCII-art and Unicode-art:DNA exampleG                                A
                             +---------------------+          +----------+
                             |                     v          |          v
    #====#  C   +---+  G   +---+  C   +---+  G   +---+  A   +---+  T   +---+  A   #===#
+-- H 0  H ---> | 5 | ---> | 6 | ---> | 7 | ---> | 8 | ---> | 9 | ---> | 3 | ---> H 4 H
|   #====#      +---+      +---+      +---+      +---+      +---+      +---+      #===#
|     |    A                                                             ^
| G   +-----------+                                                      |
|                 v                                                      |
|   +----+  G   +---+  A   +---+  T                                      |
+-> | 20 | ---> | 1 | ---> | 2 | ----------------------------------------+
    +----+      +---+      +---+G                                A
                             ┌─────────────────────┐          ┌──────────┐
                             │                     ▼          │          ▼
    ╔════╗  C   ┌───┐  G   ┌───┐  C   ┌───┐  G   ┌───┐  A   ┌───┐  T   ┌───┐  A   ╔═══╗
┌── ║ 0  ║ ───▶ │ 5 │ ───▶ │ 6 │ ───▶ │ 7 │ ───▶ │ 8 │ ───▶ │ 9 │ ───▶ │ 3 │ ───▶ ║ 4 ║
│   ╚════╝      └───┘      └───┘      └───┘      └───┘      └───┘      └───┘      ╚═══╝
│     │    A                                                             ▲
│ G   └───────────┐                                                      │
│                 ▼                                                      │
│   ┌────┐  G   ┌───┐  A   ┌───┐  T                                      │
└─▶ │ 20 │ ───▶ │ 1 │ ───▶ │ 2 │ ────────────────────────────────────────┘
    └────┘      └───┘      └───┘With or without single-path joining:Phoneme exampleReduced Phoneme exampleChangelogVersion 0.6:Documentation improvements following JOSS reviewFixed bug where node finality was not considered in minimizationVersion 0.5.1:Minor changes in preparation for submission (including tagged
release)Version 0.5:Improvements in speed, particularly in the__eq__()method ofDAFSANodeand the_minimize()method ofDAFSA. The
computation of a DAFSA for the contents of/usr/share/dict/wordsin the test machine (99,171 sequences) is now performed in under 8
minutes.Added code from Daciuk’s packages in an extra directory, along with
notes on licenseVersion 0.4:Full documentation for existing codeAdded GML, PDF, and SVG exportAllow to access all options from command-lineVersion 0.3:Allow to join transitions in single sub-pathsAllows to export a DAFSA as anetworkxgraphPreliminary documentation atReadTheDocsVersion 0.2.1:Added support for segmented dataVersion 0.2:Added support for weighted edges and nodesAdded DOT export and Graphviz generationRefined minimization method, which can be skipped if desired
(resulting in a standard trie)Added examples in the resources, also used for test dataVersion 0.1:First public release.RoadmapAfter 1.0:Preliminary generation of minimal regular expressions matching the
contents of a DAFSAConsider adding support for empty transitions (or depend on the user
aligning those)Work on options for nicer graphviz output (colors, widths, etc.)Community guidelinesWhile the author can be contacted directly for support, it is
recommended that third parties use GitHub standard features, such as
issues and pull requests, to contribute, report problems, or seek
support.Contributing guidelines, including a code of conduct, can be found in
the CONTRIBUTING.md file.Author and citationThe library is developed by Tiago Tresoldi (tresoldi@shh.mpg.de).The author has received funding from the European Research Council (ERC)
under the European Union’s Horizon 2020 research and innovation
programme (grant agreement No.ERC Grant
#715618,Computer-Assisted Language Comparison.If you usedafsa, please cite it as:Tresoldi, Tiago (2020). DAFSA, a a library for computing
Deterministic Acyclic Finite State Automata. Version 1.0. Jena.
Available at:https://github.com/tresoldi/dafsaIn BibTeX:@misc{Tresoldi2020dafsa,author={Tresoldi, Tiago},title={DAFSA, a a library for computing Deterministic Acyclic Finite State Automata. Version 1.0.},howpublished={\url{https://github.com/tresoldi/dafsa}},address={Jena},year={2020},}