danieldk/dictomaton — pull request #5 by renekrie (2014-09-25 20:15:49+00:00)
Title: Resume traversal in PerfectHashDictionary / Guava version / CharSequence for DictionaryBuilder.add() 

* Updated guava-testlib version to avoid issues with truth dependency (not in Maven central, see https://groups.google.com/forum/#!topic/guava-discuss/IkX0FhaPtKA for discussion)
* DictionaryBuilder now accepts adding more general CharSequence instead of String and uses CharSequence internally
* PerfectHashDictionary has new methods getStateInfo(...). They return a StateInfo object for a given CharSequence input. The StateInfo object contains the hashcode (similar to number()) but also enough information to resume the traversal where it left off when finishing the sequence. This is to avoid having to re-traverse the automaton from the initial state when checking for multiple sequences in the dictionary where one sequence is a substring of the other.
For example, given a PerfectHashDictionary we might want to lookup sequences "aaa" and "aaabbbb". Instead of having to traverse the automaton for both sequences we can now first do a lookup for the sub-sequence "aaa":
```java
StateInfo info = dict.getStateInfo("aaa");
// we can now check if we are in a final state and get the hashcode via
if (info.isInFinalState()) {
hash = info.getHash()
}

// we can then test if we are in a known state and resume the lookup:
if (info.isInKnownState()) {
StateInfo info2 = dict.getStateInfo("bbbb".substring("aaa".length()), info);
// the lookup would now only have to traverse for the sequence "bbbb" but return the
// StateInfo for the complete sequence "aaabbbb"
}
```
This feature is handy for looking up sub-sequences of text token sequences in a dictionary.

See PerfectHashDictionaryTest for further examples.