DGCF: A Distributed Greedy Clustering Framework for Largescale Genomic Sequences Clustering is a very fundamental while timeconsuming compute operation in biological sequence analysis. New sequencing technologies such as NGS and 3GS have dramatically increased both the dataset size and the length of a single read sequence. However, existing tools lack scalability for handling largescale datasets as well as long sequences. A feasible solution to this problem is to use parallel and distributed systems. The efficient deployment of such systems, however, requires high parallelism in both software implementations as well as algorithmic optimizations. In this paper, we propose DGCF, a Distributed Greedy Clustering Framework which is capable to handle largescale datasets and long sequences. Our framework adopts a greedy clustering strategy which overlaps communication with computation among many distributed computing nodes. We also design and implement a sparse suffix array (SSA)based alignment algorithm that can support long sequences. Experiments show that our framework achieves nearlinear speedups on a distributed memory cluster.

Gclust: A Parallel Clustering Tool for Microbial Genomic Data Abstract   The accelerating growth of the public microbial genomic data imposes substantial burden on the research community that uses such resources. Building databases for nonredundant reference sequences from massive microbial genomic data based on clustering analysis is essential. However, existing clustering algorithms perform poorly on long genomic sequences. In this article, we present Gclust, a parallel program for clustering complete or draft genomic sequences, where clustering is accelerated with a novel parallelization strategy and a fast sequence comparison algorithm using sparse suffix arrays (SSAs). Moreover, genome identity measures between two sequences are calculated based on their maximal exact matches (MEMs). In this paper, we demonstrate the high speed and clustering quality of Gclust by examining four genome sequence datasets. Gclust is freely available for noncommercial use at  https:github.comniulabgclust . We also introduce a web server for clustering useruploaded genomes at  http:niulab.scgrid.cngclust .