# Disjoint Tree Mergers for Large-Scale Maximum Likelihood Tree Estimation
## 3 Results
### Experiment 3: Evaluating Fast DTM Pipelines in Comparison to ML Codes on Ultra-Large Datasets  
Because TreeMerge did not offer an accuracy advantage over GTM and it is slower than GTM and Constrained-INC, for our experiment evaluating scalability to 10K and 50K sequences, we only examined the DTM pipelines using GTM and Constrained-INC. We compared these two pipelines (using the same starting tree and decomposition strategy) to all three ML methods, allowing all methods up to one week of running time, given 16 cores and using 64 GB of memory. Tables 6 and 7 show results on the RNASim10K and RNASim50K datasets, respectively, and are separately discussed. \begin{table}
\begin{tabular}{l c c} \hline \hline  & **Method** & **Running Time (h)** & **Tree Error** \\ \hline FastTree & 2.0 & 10.8 \\ IQ-TREE & 74.6 & 10.9 \\ RAXML & 168 * & 12.3 \\ \hline DTMs, FastTree starting tree & & \\ Constrained-INC & 4.3 & 10.5 \\ GTM & 3.5 & **10.1** \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Results on the RNASim10k datasets The average running time (in hours, assuming use of 16 cores) and tree error (FN) rates over 10 replicates. Each method was given 64 GB. The asterisk (*) for RAxML running time indicates that it had not completed by that time, and we used the best scoring tree that it found in that time period. The most accurate result is boldfaced. \begin{table}
\begin{tabular}{l c c} \hline \hline  & **Method** & **Running Time (h)** & **Tree Error** \\ \hline FastTree & 0.03 & 50.9\% \\ IQ-TREE & 2.5 & 30.2\% \\ RAxML & 24 * & **24.9\%** \\ \hline DTMs, FastTree starting tree & & \\ Constrained-INC & 1.0 & 42.4\% \\ GTN & 1.0 & 42.4\% \\ TreeMerge & 2.5 & 42.5\% \\ \hline DTMs, IQ-TREE starting tree & & \\ Constrained-INC & 3.1 & 28.6\% \\ GTM & 3.1 & 28.4\% \\ TreeMerge & 3.9 & 28.5\% \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Results on the 1000M1-HF datasets. The tree error rates (FN) and running time for existing ML methods and DTM pipelines on the 1000M1-HF model condition (averaged across five replicates). The DTM pipelines use the starting tree as a guide tree with two options for guide trees (FastTree or IQ-TREE), and decompose to maximum subset size 500. If a pipeline requires a distance matrix, as is the case with Constrained-INC and TreeMerge, the topological distance matrix is obtained from the starting tree. The asterisk (*) for RAxML running time indicates that it had not completed by that time, and we used the best scoring tree that it found in that time period. The most accurate result is boldfaced. On the RNASim10K datasets (Table 6), FastTree completed in two hours, IQ-TREE completed in just under 75 h, and RAxML did not complete within the allowed running time (one week). We report the FN error rate for the best ML tree that was found by RAxML before it timed out. The comparison between the three ML methods shows that FastTree and IQ-TREE are tied for the best accuracy (10.8% and 10.9% error, respectively), followed by RAxML at 12.3%. The GTM pipeline achieves 10.1% error and the Constrained-INC pipeline achieves 10.5% error, and both complete quickly (3.5 h for GTM and 4.3 h for Constrained-INC). Thus, on these datasets, the two DTM pipelines are more accurate than all of the ML methods tested and are also much faster than either IQ-TREE or RAxML. The results on the RNASim50K datasets (Table 7) are somewhat different. First, IQ-TREE fails to return a tree (failing due to a memory allocation problem) and, while RAxML does not fail, the tree it returns has 100% topological error. FastTree, as before, achieved good accuracy, with an error rate of 8.0%, and it does so in just 8.3 h. The GTM pipeline also completed and returned a tree with good accuracy (7.5% error), improving on the FastTree starting tree, and it did so in 14.1 h. However, Constrained-INC failed to return a tree on these datasets due to a memory allocation problem.