Package: pydivsufsort
pydivsufsort: bindings to libdivsufsortpydivsufsortprebuildslibdivsufsortas a shared library and
includes it in a Python package with bindings. Wheels are built for Linux, macOS and Windows (32 and 64 bits) usingcibuildwheeland GitHub Actions. Basically, you should be able to install it withpip install pydivsufsorton any system and it should work out of the box. If it doesn't, please create an issue.Features:bindings todivsufsortthat return numpy arrayshandle string, bytes and almost any integer data type (e.g.int64) and not onlycharalgorithms work even for non char inputsadditional string algorithms coded in CythonInstallationOn Linux, macOS and Windows:python -m pip install pydivsufsortWe provide precompiled wheels for common systems usingcibuildwheel, and a source distribution for Unix systems. Manual compilation on Windows might require some tweaking, please create an issue.FeaturesAll methods support string, bytes and numpy array inputs, including datatypes greater thanuint8_t(e.g.uint64_t). Below are the signatures of all methods exposed bypydivsufsort. To import a method, just dofrom pydivsufsort import method_name. All methods are documented in the docstrings. You can display them withhelp(method_name).A nicer interface to reuse computations lazily is provided in WonderString but currently undocumented. Please create an issue if you are interested.Methods exposed from libdivsufsortdivsufsort(string): suffix arraybw_transform(string): Burrows-Wheeler transforminverse_bw_transform(idx, string): inverse Burrows-Wheeler transformsa_search(string, suffix_array, pattern): search for a pattern in a suffix arrayAdditional string algorithmskasai(string, suffix_array=None): LCP array computation (lazily computes the suffix array if not provided)lcp_segtree(string, suffix_array=None, lcp=None): build a segment tree for LCP queries (lazily computes the suffix array and LCP array if not provided)lcp_query(segtree, queries): query a segment tree for LCP queries. Queries are pairs of indices.levenshtein(string1, string2): Levenshtein distancemost_frequent_substrings(lcp, length, limit=0, minimum_count=1): most frequent substrings. See the docstring for details.common_substrings(string1, string2, limit=25): common substrings between two strings.min_rotation(string): minimum rotation of a stringExample usagefrompydivsufsortimportdivsufsort,kasaistring_inp="banana$"string_suffix_array=divsufsort(string_inp)string_lcp_array=kasai(string_inp,string_suffix_array)print(string_suffix_array,string_lcp_array)# [6 5 3 1 0 4 2] [0 1 3 0 0 2 0]# You can also convert the string input to integers firstimportnumpyasnpint_inp=np.unique(np.array(list(string_inp)),return_inverse=True)[1]int_suffix_array=divsufsort(int_inp)int_lcp_array=kasai(int_inp,int_suffix_array)print(int_suffix_array,int_lcp_array)# [6 5 3 1 0 4 2] [0 1 3 0 0 2 0]DevelopmentYou can install locally withpip install -e .A useful command to iterate quickly when changing Cython code ispython setup.py build_ext --inplace && pytest -sProfilingProfiling can be activated with the environment variablePROFILE:PROFILE=1 python setup.py build_ext --inplace && pytest -sHere is an example with line_profiler (requirespip install "line_profiler<4"):import line_profiler
from pydivsufsort import common_substrings
from pydivsufsort.stringalg import (
    _common_substrings,
    repeated_substrings,
)

s1 = "banana" * 10000
s2 = "ananas" * 10000

func = common_substrings
profile = line_profiler.LineProfiler(func)
profile.add_function(_common_substrings)
profile.add_function(repeated_substrings)
profile.runcall(func, s1, s2, limit=15)
profile.print_stats()TestingpytestTechnical details (for performance tweaks)libdivsufsortis compiled in both 32 and 64 bits, asthe 32 bits version is faster.pydivsufsortautomatically chooses to use the 32 bits version when possible (aka when the input size is less than2**31-1).For best performance, use contiguous arrays. If you have a sliced array, pydivsufsort converts it automatically withnumpy.ascontiguousarray.The precompiled libraries use OpenMP. You can disable it by setting the env variableOMP_NUM_THREADS=1, and it will yield the same performance as the version compiled without OpenMPThe originallibdivsufsortonly supports char as the base type.pydivsufsortcan handle arrays of any integer type (even signed), by encoding each element as multiple chars, which makes the computation slower. If your values use an integer type that is bigger than required, but they span over a small contiguous range,pydivsufsortwill automatically change their type (see#6).AcknowledgementsYuta Morifor writinglibdivsufsortSean Lawfor initiating this project and contributingCitingIf you have used this software in a scientific publication, please cite it using the following BibLaTeX code:@software{pydivsufsort,
  author       = {Louis Abraham},
  title        = {pydivsufsort},
  year         = 2023,
  publisher    = {Zenodo},
  doi          = {10.5281/zenodo.7932458},
  url          = {https://github.com/louisabraham/pydivsufsort}
}