Package: hermetrics
hermetricsPython library for distance and similarity metricsInstallFrom pypi:pipinstallhermetricsOverviewHermetrics is a library designed for use in experimentation with string metrics. The library features a base classMetricwhich is highly configurable and can be used to implement custom metrics.Based onMetricare some common string metrics already implemented to compute thedistancebetween two strings. Some common edit distance metrics such asLevenshteincan be parametrized with different costs for each edit operation, althought have been only thorougly tested with costs equal to 1. Also, the implemented metrics can be used to compare any iterable in addition to strings, but more tests are needed.A metric has three main methodsdistance,normalized_distanceandsimilarity. In general thedistancemethod computes the absolute distance between two strings, whereasnormalized_distancecan be used to scale the distance to a particular range, usually (0,1), and thesimilaritymethod being normally defined as (1-normalized_distance).The normalization of the distance can be customized overriding the auxiliary methods for its computation. Those methods aremax_distance,min_distanceandnormalize.MetricclassMetricis a base class that can receive as arguments six specific functions to be used as methods for the metric being implemented. The class constructor just assign the functions received as parameters to the class methods. If ypu omit some parameter then a default method is used, which allows you to implement metrics without the need to rewrite some of the functionality that is common among metrics.classMetric:"""Class for metric implementations"""def__init__(self,distance=None,max_distance=None,min_distance=None,normalize=None,normalized_distance=None,similarity=None,name='Generic'):"""Class constructor - receives a function for distance or similarity evaluation"""self.name=nameself.distance=distanceorself.distanceself.max_distance=max_distanceorself.max_distanceself.min_distance=min_distanceorself.min_distanceself.normalize=normalizeorself.normalizeself.normalized_distance=normalized_distanceorself.normalized_distanceself.similarity=similarityorself.similarityDefault methodsDescription of default methods for theMetricclass.In general a method of a metric receives three parameters:source. The source string or iterable to compare.target. The target string or iterable to compare.cost=1. If a number, the unit cost for any edit operations. If a tuple, the cost for edit operations in the following order (deletion, insertion, substitution, transposition).distanceThedistancemethod computes the total cost of transforming thesourcestring on thetargetstring. The default method just return 0 if the strings are equal and 1 otherwise.max_distanceReturns the maximum value of the distance betweensourceandtargetgiven a specificcostfor edit operations. The default method just return 1 givensourceandtargetdon't have both length=0, in that case just return 0.min_distanceReturn 0.normalizeThis method is used to scale a value between two limits, usually those obtained bymax_distanceandmin_distance, to the (0,1) range. Unlike the other methods,normalizedoesn't receive the usual arguments (source,targetandcost), instead receive the following:x. The value to be normalized.low=0. The minimum value for the normalization, usually obtained withmin_distancemethod.high=1. The maximum value for the normalization, usually obtained withmax_distancemethod.normalized distanceScale the distance betweensourceandtargetfor specificcostto the (0,1) range usingmax_distance,min_distanceandnormalize.similarityComputes how similar aresourceandtargetgiven a specificcost. By default defined as 1 -normalized_distanceso the result is also in the (0,1) range.MetricsFor the time being the following metrics have been implemented:HammingThe Hamming distance count the positions where two strings differ. Normally the Hamming distance is only defined for strings of equal size but in this implementation strings of different size can be compared counting the difference in size as part of the distance.fromhermetrics.hammingimportHammingham=Hamming()ham.distance('abcd','abce')# 1ham.normalized_distance('abcd','abce')# 0.25ham.similarity('abcd','abce')# 0.75LevenshteinLevenshtein distance is usually known as "the" edit distance. It is defined as the minimum number of edit operations (deletion,insertionandsubstitution) to transform the source string into the target string. The algorithm for distance computation is implemented using the dynamic programming approach with the full matrix construction, althought there are optimizations for time and space complexity those are not implemented here.fromhermetrics.levenshteinimportLevenshteinlev=Levenshtein()lev.distance('ace','abcde')# 2lev.normalized_distance('ace','abcde')# 0.4lev.similarity('ace','abcde')# 0.6# With cost=2lev.distance('ace','abcde',2)# 4lev.normalized_distance('ace','abcde',2)# 0.4lev.similarity('ace','abcde',2)# 0.6# With different costs for deletion, insertion and substituionlev.distance('ace','abcde',(1,1.25,1.5))# 2.5lev.normalized_distance('ace','abcde',(1,1.25,1.5))# 0.3571lev.similarity('ace','abcde',(1,1.25,1.5))# 0.6429OSA (Optimal String Alignment)TheOSAdistance is based on theLevenshteindistance but counting thetranspositionas a valid edit operation with the restriction that no substring can be transposed more than once.fromhermetrics.osaimportOsaosa=Osa()osa.distance('abcd','abdc')# 1osa.normalized_distance('abcd','abdc')# 0.25osa.similarity('abcd','abdc')# 0.75# With different costs for deletion, insertion, substituion and transpositionosa.distance('ace','abcde',(0.75,1,1.25,1.5))# 2osa.normalized_distance('ace','abcde',(0.75,1,1.25,1.5))# 0.3478osa.similarity('ace','abcde',(0.75,1,1.25,1.5))# 0.6522Damerau-LevenshteinTheDamerau-Levenshteindistance is likeOSAbut without the restriction on the number of transpositions for the same substring.fromhermetrics.damerau_levenshteinimportDamerauLevenshteindam=DamerauLevenshtein()dam.distance('abcd','cbad')# 2dam.normalized_distance('abcd','cbad')# 0.5dam.similarity('abcd','cbad')# 0.5# With different costs for deletion, insertion, substituion and transpositiondam.distance('ace','abcde',(0.75,1,1.25,1.5))# 2dam.normalized_distance('ace','abcde',(0.75,1,1.25,1.5))# 0.3478dam.similarity('ace','abcde',(0.75,1,1.25,1.5))# 0.6522JaccardTheJaccardindex considers the strings as abag-of-charactersset and computes the cardinality of the intersection over the cardinality of the union. The distance function forJaccardindex is already normalized.fromhermetrics.jaccardimportJaccardjac=Jaccard()jac.distance('abcd','abce')# 0.4jac.similarity('abcd','abce')# 0.6Dice (Sorenson-Dice)Is related toJaccardindex in the following manner:fromhermetrics.diceimportDicedic=Dice()dic.distance('abcd','abce')# 0.25dic.similarity('abcd','abce')# 0.75JaroJarodistance is based on thematchingcharacters present on two strings and the number of transpositions between them. Amatchingoccurs when a character of a string is present on the other string but in a position no further away that certain threshold based on the lenght of the strings. TheJarodistance is normalized.fromhermetrics.jaroimportJarojar=Jaro()jar.distance('abcd','abe')# 0.278jar.similarity('abcd','abe')# 0.722Jaro-WinklerExtension ofJarodistance with emphasis on the first characters of the strings, so strings that havematchingcharacters on the beginning have more similarity than those that havematchingcharacters at the end. This metric depends on an additional parameterp(with0<=p<=0.25and defaultp=0.1) that is a weighting factor for additional score obtained for matching characters at the beginning of the strings..fromhermetrics.jaro_winklerimportJaroWinklerjaw=JaroWinkler()jaw.distance('abcd','abe')# 0.222jaw.similarity('abcd','abe')# 0.778jaw.similarity('abcd','abe',p=0.05)# 0.750jaw.similarity('abcd','abe',p=0.15)# 0.806jaw.similarity('abcd','abe',p=0.25)# 0.861To DoUse **kwargs instead of cost tuplesWeighted LevenshteinShow matrix for Levenshtein like distancesAllow variable maximun prefix length in Jaro-WinklerImplement backtracking of operationsMore metricsType hints?