Package: vlmc
Variable Length Markov Model (VLMC)Implementation of Variable Length Markov Chains (VLMC) for Python.
Suffix tree building is done top-down using thePeres-Shieldorder estimation method.
It is written in Rust with Python Bindings.ContentsInstallationCompiling from sourceUsagefitsuffixcountsdistributioncontextsFutureInstallationPre-built packages for many Linux, Windows, and OSX systems are available
inPyPIand can be installed with:pipinstallvlmcOn uncommon architectures, you may need to firstinstall Cargobefore runningpip install vlmc.Compilation from sourceIn order to compile from source you will need toinstall Rust/Cargoandmaturinfor the python bindings.
Maturin is best used within a Python virtual environment:# activate your desired virtual environment first, then:pipinstallmaturin
gitclonehttps://github.com/antonio-leitao/vlmc.gitcdvlmc# build and install the package:maturindevelop--releaseUsageimportvlmctree=vlmc.VLMC(max_depth,alphabet_size,n_jobs=-1)Parameters:max_depth: Maximum depth of tree. Subsequences whose length exceed themax_depthwill not be considered nor counted.alphabet_size: Total number of symbols in the alphabet. This number has to be bigger than the highest integer encountered, else it will cause runtime errors.n_jobs: Number of subprocesses to spawn when running the vlmc. Choose-1for using all available processes.fitNotefit method returnsNoneand notself. This is by design as to not expose the rust object to python.data=[[1,2,3],[2,3],[1,0,1],[2]]tree.fit(data)Arguments:data: List of lists containing sequences of discrete values to fit on. Values are assumed to be integers form0toalphabet_size. List is expected to be two dimensional.get_suffixGiven a sequence, returns the longest suffix that is present in the VLMC.suffix=tree.get_suffix(sequence)Arguments:sequence: list of integers representing a sequence of discrete varaibles.Returns:suffix: longest suffix of sequence that is present in the VLMC.get_countsGets the total number of occurences of a given sequence of integers.
Will throw aKeyErrorif the sequence is not a tree node. Consider usingget_suffixto make sure to get a tree node.counts=tree.get_counts(sequence)Arguments:sequence: list of integers representing a sequence of discrete varaibles.Returns:counts: integerget_distributionGets the vector of probabilities over the entire alphabet for the given sequence.
Will throw aKeyErrorif the sequence is not a tree node. Consider usingget_suffixto make sure to get a tree node.probabilities=tree.get_distribution(sequence)Arguments:sequence: list of integers representing a sequence of discrete variables.Returns:probabilities: list of floats representing the probability of observing a specific state (index) as the next symbol.get_contextscontexts=tree.get_contexts()Returns:contexts: list of relevant contexts according to the Peres-Shield tree prunning method. Contexts are ordered by length.TODOParalelizationAfter experimentation the best possible idea for paralelization would be to create different hashmaps for each sunsequence length.
Hashmaps are then joined from longest to smallest.
The hashmap atmax_depth + 1can be discarded after.
Could be very fast depending on merging algo.