Package: pylotiger
Compute TIGER Rates for Phylogenetic DataThis is a pure Python implementation of the TIGER rates (Cummins and McInerney 2011), which can be computed for all kinds of phylogenetic data. The creation of the package was inspired by a study ofSyrjänen et al. (2021)in which the authors present a new implementation of TIGER rate computation for linguistic purposes. Since the implementation by Syrjänen et al. does not come along as a package, but rather as a commandline tool, whose application from within Python scripts is not straightforward, I decided to reimplement the rate computation based on the description in both publications in the form of a small Python package without further dependencies.WorkflowIn order to compute TIGER rates for your phylogenetic characters, your characters should be stored in the form of a nested Python dictionary in which each character is represented by a key, which links to a dictionary representing the character states. The character states themselves are represented by a dictionary with taxonomic units (language varieties, species) as key and character states being represented as values in the form of a list, allowing for polymorphous characters per taxonomic unit and for missing data, which would be passed as an empty list. In this form, character states can be conveniently stored in JSON format, such as the following examples from Cummins and McInerney,A = CTTAA, andB = AGGGG:{"A":{"1":["C"],"2":["T"],"3":["T"],"4":["A"],"5":["A"]},"B":{"1":["A"],"2":["G"],"3":["G"],"4":["G"],"5":["G"]}}With your characters stored in this form, you can then convert them to set partitions:>>>frompylotigerimportget_set_partitions>>>characters={"A":{"1":["C"],"2":["T"],"3":["T"],"4":["A"],"5":["A"]},"B":{"1":["A"],"2":["G"],"3":["G"],"4":["G"],"5":["G"]}}>>>taxa=["1","2","3","4","5"]>>>set_partitions=get_set_partitions(characters,taxa)The output is a dictionary with a character as a key and the corresponding set partition as a value. Set partitions consist offrozensetsin which the taxonomic units (represented by numbers 1 to 5 here) are grouped.>>>set_partitions{'A':{frozenset({'1'}),frozenset({'4','5'}),frozenset({'2','3'})},'B':{frozenset({'1'}),frozenset({'2','3','4','5'})}}Since our implementation allows for polymorphous characters, it is possible that different set partitions contain the same taxonomic unit. If state information for a given character is missing, the corresponding taxonomic unit would not appear in any of the set partitions.Having computed the set partitions, we can compare them with each other. Cummins and McInerney report that comparing A with B yields the score 0.5, while comparing B with A yields 1.>>>frompylotigerimportget_partition_agreement_score>>>get_partition_agreement_score(set_partitions["A"],set_partitions["B"])0.5>>>get_partition_agreement_score(set_partitions["B"],set_partitions["A"])1In order to compute the rates, we now use the exemplary data from the study of Syrjänen et al. (2021).{"X":{"a":["A"],"b":["B"],"c":["A"],"d":["A"],"e":["C"]},"Y":{"a":["D"],"b":["D"],"c":["E"],"d":["E"],"e":["F"]},"Z":{"a":["G"],"b":["H"],"c":["H"],"d":["H"],"e":["H"]}}Here, the authors report rates of 0.5835 for character X and Y, and 0.667 for character Z.>>>frompylotigerimportget_rates>>>characters={"X":{"a":["A"],"b":["B"],"c":["A"],"d":["A"],"e":["C"]},"Y":{"a":["D"],"b":["D"],"c":["E"],"d":["E"],"e":["F"]},"Z":{"a":["G"],"b":["H"],"c":["H"],"d":["H"],"e":["H"]}}>>>taxa=["a","b","c","d","e"]>>>set_partitions=get_set_partitions(characters,taxa)>>>rates=get_rates(set_partitions)>>>forcharin"XYZ":...print(char,"{0:.4f}".format(rates[char]))X0.5833Y0.5833Z0.6667Note that in this version, the rate computation is not optimized with respect to repeating character values. As a result, the current implementation may take some time when being used to investigate datasets with many characters.Computing the Corrected TIGER RatesThe TIGER rates are supposed to test the tree-likeness of a given dataset. In the form presented by Cummins and McInerney,invariant characterswhich have the same character state across all taxonomic units are not explicitly excluded from the data. This may be considered as problematic, since invariant characters do not provide any specific information about the subgrouping. In the original partition agreement score, however, they will always yield the score 1 when compared toanyother set partition, as we can easily see when setting our character "X" in the previous example to reflect identical character states and comparing it with the other characters.>>>characters={"X":{"a":["A"],"b":["A"],"c":["A"],"d":["A"],"e":["A"]},"Y":{"a":["D"],"b":["D"],"c":["E"],"d":["E"],"e":["F"]},"Z":{"a":["G"],"b":["H"],"c":["H"],"d":["H"],"e":["H"]}}>>>taxa=["a","b","c","d","e"]>>>set_partitions=get_set_partitions(characters,taxa)>>>forcharin"YZ":...print("X vs.",char,"{0:.4f}".format(get_partition_agreement_score(set_partitions["X"],set_partitions[char])))Xvs.Y1.0000Xvs.Z1.0000Conversely, if a characters has distinct states for all taxonomic units, allotherset partitions in the data will receive a score of 1 when compared to it, as we can see easily when modifying character "X" in our example to have different states for all taxonomic units.>>>characters={"X":{"a":["A"],"b":["B"],"c":["C"],"d":["D"],"e":["E"]},"Y":{"a":["D"],"b":["D"],"c":["E"],"d":["E"],"e":["F"]},"Z":{"a":["G"],"b":["H"],"c":["H"],"d":["H"],"e":["H"]}}>>>taxa=["a","b","c","d","e"]>>>set_partitions=get_set_partitions(characters,taxa)>>>forcharin"YZ":...print(char,"vs. X","{0:.4f}".format(get_partition_agreement_score(set_partitions[char],set_partitions["X"])))Yvs.X1.0000Zvs.X1.0000One might argue that this kind of behavior is consistent with the original rate calculation, but when working with concrete language data, I found that the inclusion of characters which are not informative for the subgrouping may at times skew the scores, giving the impression that a dataset is highly tree-like, only because it contains many characters with singleton character states. For this reason, this implementation of TIGER rates provides a corrected version of the partition agreement score, which differs from the original partition agreement score in three respects:singletons (character states that occur only once in the data) are excluded from the compatibility calculation,invariant character states (which are identical for all characters) are excluded from the calculation, andinstead of dividing the matches (where partition A includes the partition B) by the number of sets in partition B, we divide by the number of intersections, which allows to catch those cases in which a character state is compatible with two partitions.With respect to the handling of 1 and 2, we have two possibilities: we can set the score to 0 if we encounter a case of a character that consists only of singleton states or has only invariant states, or we can ignore it from the calculation.To compute the corrected TIGER scores, all one needs to do is to import thecorrected_pas()function and to pass it as the argumentpartition_functo theget_rates()function.>>>frompylotigerimportcorrected_pas>>>characters={"X":{"a":["A"],"b":["B"],"c":["C"],"d":["D"],"e":["E"]},"Y":{"a":["D"],"b":["D"],"c":["E"],"d":["E"],"e":["E"]},"Z":{"a":["G"],"b":["G"],"c":["G"],"d":["H"],"e":["H"]}}>>>taxa=["a","b","c","d","e"]>>>set_partitions=get_set_partitions(characters,taxa)>>>forxin"XYZ":...foryin"XYZ":...pas1=get_partition_agreement_score(set_partitions[x],set_partitions[y])...pas2=corrected_pas(set_partitions[x],set_partitions[y])...print("{0}|{1}|{2:.2f}|{3}".format(x,y,pas1,"{0:.2f}".format(pas2)ifpas2isnotNoneelse"None"))X|X|1.00|NoneX|Y|0.00|NoneX|Z|0.00|NoneY|X|1.00|NoneY|Y|1.00|1.00Y|Z|0.50|0.33Z|X|1.00|NoneZ|Y|0.50|0.33Z|Z|1.00|1.00As can be seen from this example, we exclude all comparisons with character X, for which the function yieldsNone. When computing TIGER rates, these characters will be excluded from the rate calculation. Furthermore, the calculation of scores for character Y (states: DDEEE) and Z (states: GGGHH) also shows a crucial difference, in so far, as we identify three possible links between character states in Z and character states in Y (G=abc → D=ab, G=abc → E=cde, and H=de → E=cde), of which only one is compatible (G → D). As a result, the score is set to 0.33 (1/3) instead of 0.5, as in the calculation of the original TIGER partition agreement rates.