# Presentation: 005190

## Bioinformatics:Infrastructure for High-Throughput Automated Analysis of Genomes.

**Natalia Maltsev, Elizabeth Marland, Gong Xin Yu, **
**Alex Rodrigez, Dinanath Sulakhe, Praveen Chandromohan, Rich Lusk, Luke Ulrich, Ramya Krishnamurthy**

***MCS ***
***Argonne National Laboratory***

## Biology in a Nutshell(for people with little knowledge but infinite intelligence)

**slide “borrowed” from Nagiza Samatova (ORNL)**

-  Genome consists of genes
-  **Gene**  **Protein**: Object description Object instantiation
-  **Protein ********Functions**
-  **Enzymes**: proteins that catalyze biochemical reactions
**Pathway**: sequence of reactions
-  **Network** (directed graph): set of pathways with metabolites as vertices and enzymes as edges

-  **Genome** (ROM): assembly code on how to build proteins
-  Instructions: A, C, T, G
-  3 variables  amino acid

**Notes:**
As I said at the beginning, biology is a very fascinating area. It will constantly challenge you with its complexity and will encourage you with its simplicity once you discover it. One of the complexities that I found almost frustrating is a terminology. That’s why I decided to give this very brief background information, or “Biology on a Palm” from the computer scientist perspective. I assume that you have very little knowledge but infinite intelligence. As you know, now we have so-called “Post-Genome Era”. So you can see how much we are behind. Some people had been working in this area for the whole “Genome Era” but we decided to join them only now. But it is NOT late. Because all the advances made during Genome Period have actually made this area even more complex. Now it challenges us with lots of computational problems that even Terflop Machines are not capable of managing.

## General Project Architecture

**Sequence Analysis module**
**In G-Wiz**

**Stages of analysis:**
-  Determine components of the system (assign functions to the genes)
-  Establish relationships between components (static model)
- Develop dynamic model of the system

## Goals of the Project

- Development of:
-  Integrated Computational infrastructure Gwiz/GADU
- Tools and algorithms for
- High-throughput genetic sequence analysis (assignments of functions to the genes in sequenced genomes)
- Metabolic Reconstructions from sequence data (static models)

## Bioinformatics: Tasks for Q4 2002

- To Ensure the robustness of dataflow in GADU System
- Design a working plan for Chimerization of GADU
- To use grid-based backend for high-throughput genome analysis
- Develop Tools and Algorithms for High-throughput genome analysis (in collaboration with ORNL, Nagiza Samatova)

##               Data Acquisition Module

- Gadu searches once a week through major public sequence databases (NCBI, TrEMBL, JGI, and more).
- User is notified via email about new or updated genomes.
- Gadu’s interactive web-based user interface allows the user to decide which genomes should be acquired and analyzed.
- A user log containing the latest upload activity is available.

**                **_**Data Analysis Module**_
- Submit genomes to bioinformatics tools and algorithms :
- Blast – comparative pair wise sequence analysis.
- Blocks – based on HMM and multiple aligned ungapped segments corresponding to most highly
- conserved regions of proteins.
- Pfam – based on HMM and multiple sequence alignments covering common protein  domains.
- Clustering algorithms.
- SVM-based algorithms.
- Scale by parallel submission of jobs to HPC nodes (Chiba City).
- 256 nodes, 500 MHz, 512 MB RAM.
- All steps of genome analysis are monitored to ensure correct outputs.

_**Data Storage Module**_
**Divided into two phases...**
**1. Before Data Analysis:** Raw data downloaded from the public database sites (NCBI, TIGR, JGI etc..) is stored into Oracle relational database.
**2. After Data Analysis:** Data after the analysis (results of running the various tools like Blast, Pfam, and Blocks) are parsed and stored into the database.

## Accomplishments
In collaboration with Zach Miller and Miron Livny (UWisc), GADU-Condor pipeline was established:
Grid-FTP was Installed 
GADU submits Blast jobs to Condor via Grid-FTP.
100 genomes were submitted to Condor for Blast analysis.

Remaining Condor Tasks
Automate pipeline between GADU and Condor for periodic automated blast submissions.
Add analysis pipeline for InterPro and other bioinformatics tools
Add Condor pools at UofC to GADU

- In collaboration with Zach Miller and Miron Livny (UWisc), GADU-Condor pipeline was established:
- Grid-FTP was Installed
- GADU submits Blast jobs to Condor via Grid-FTP.
- 100 genomes were submitted to Condor for Blast analysis.
- _Remaining Condor Tasks_
- Automate pipeline between GADU and Condor for _periodic automated_ blast submissions.
- Add analysis pipeline for InterPro and other bioinformatics tools
- Add Condor pools at UofC to GADU

## Computational Requirements – still need more...

**A. Conventional tools. We will run each of them on approx 900,000 sequences**
**Blast **
**4000 sequences (one average bacterial genome) x 18 nodes (36 cpu) -> 4 hrs (550 MHz) on one cpu for 1000 sequences ~144 hrs**
**Pfam ****-- 100 sec/sequence**
**Blocks --**** 55 sec/seq **
**B. This set of tools we will run on the order of 30,000 families. And also it will be put into an interactive workbench**
**1. ClustalW (40 seq) **
-         **CPU Speed: 2.2 Ghz**
**        ****Time for MSA => 12.5 seconds**
**        ****Total cycles needed: 12.5 s * (2.2e9) = ~27e9 cycles**
**        ****Time for Bootstrapping => 9 s**
**        ****Total cycles needed: 9 s * (2.2e9) = ~18.2e9 cycles**
**2. T_coffee (40 seq, aligning via clustalw and lalign)**
-         **CPU Speed: 2.2 Ghz**
**        ****Time for MSA => 11 min 22 s = 682 s**
**        ****Total cycles needed: 682 s * (2.2e9) = 1.5e12 cycles**
**3. MAST (searching against database with PSSM)**
-         **CPU Speed: 2.2 Ghz**
**        ****Time for search: 26 min = 1560 s**
**        ****Total cycles needed: 1560 s * (2.2e9) = 3.432e12 cycles **

## Problems: Data storage requirements

- _Stats for genome acquisition_
- Flat file size  ~ 1 kb – 15 Mb
- Current qty. Of genomes   100
- Future qty. Of genomes  635
- Process time for one genome: < 1 minute

_**Stats for genome analysis**_
**Avg. Genome size ~ 4000 sequences (genes)**
**Blast - 4000 seq. X 18 nodes (36 cpu, 500 MHz) ******** 4 hrs**
**            ****One cpu for 4000 sequences ********144 hrs!!!**
**Pfam**** ****-- 100 sec/seq  x 4000 seq x 18 nodes  ******** 3 hours**
**Blocks**** --**** 55 sec/seq x 4000 seq x 18 nodes ******** 1.5 hours**

_**Stats for genome storage**_
- Time varies for each tool:
**Blast ~ 1 hour (depends on size)**
**Pfam ~ 10 minutes**
**Blocks ~ 10 minutes**

_**Total Time for Completion of One Genome**_
**The following average is based on 18 nodes usage:**
**Average ******** Data Acquisition   ******** 1 minute**
**                    ****Data Analysis       ******** 4 hours**
**                    **_**Data Storage          **__****__** 1 hour     .           **_
**                    ****Total time              ******** 5 hours**

_**Output file size**_** (for 1 avg. genome of 4000 genes)  ****There are 100 genomes!**
**	****	****	****	**_**Blast**_**	****	**_**Blocks**_**	****	**_**Pfam**_
**Temporary Storage****	****	****0.8GB****	****	****28.1MB****	****	****10MB**
**Permanent Storage****	****	****0.8GB****	****	****8MB****	****	****10MB**

## Plans for Q1 2003 for GADU

**To run a prototype of GADU under Chimera**
**Add additional distrubuted resources to GADU : **
**UofC Condor pool**
**DataGrid cluster**
**TeraGrid (Jazz?)**
**Develop approaches for recording of the data provenance**
**Find solutions for temporary and permanent data storage problem**

## Slide 11

##  Sources of Protein Families

**Base(WIT3)**

- Subunits
- DB (ANL)

- COGs

- BLAST

- Hobacgen

**HMM-Based**
**PhyloBLOCKS**

** ****Sources of Protein Families**

**SVM -Based**
**Classification**
**Characterization**
**of the Protein Families**

**Motivation:** The current
- resolution of tools such
- as BLOCKs and Pfam are
- unable discriminate closely
- related  homologous sequences

**Motivation: **
- To develop a library of Blocks
- HMM profiles corresponding
- to particular enzymatic
- functions or evolutionary
- versions of enzymes

**Output: **
- 1. A library of SVM Models for
- identification of certain
- enzymatic functions
- andd computational tools for
- predictions of protein
- functions based on these
- models

**Applications:**
- 1.Identification of conserved
- amino acid residues
- responsible for the
- functionality of a protein
- sequence.
- 2. Automated class-
- ification and predicting
- functions of proteins

**Applications:**
- 1. Identification,
- Classification and
- characterization of
- Proteins utilizing refined
- BLOCKs.
- 2. Phylogenetic
- analysis using BLOCKs
- distribution to identify
- cases of
- convergent/divergent
- Evolution of enzymes

**Output:**
- Refined BLOCKs
- specific for particular
- Enzymatic functions

**Tools Development: Algorithms for Automated Assignment of functions to Genes and Evolutionary Analysis of Protein Families**

- GongXin Yu, Praveen Chandromohan,
-        Luke Ulrich, Liz Marland,
-          Natalia Maltsev – ANL
-      Nagiza Samatova - ORNL

## PhyloBLOCKS

- Base

- BLAST

- Habacgen
- Families

- COGS

- Selected Set of
- Sequences
- for Particular
- Protein Families

- CLUSTALW :
-  MSA and
- create tree files

- View tree files and user selection
- of functional groups

- Schema
- for creating BLOCKS from selected functional groups.

- CLUSTALW – MSA
- designated groups and then create BLOCKS using BLOCKMaker

- 1. Run LAMA to verify uniqueness
- of BLOCKS between the groups
- BLAST using consensus
- for additional verification.
- 2. Add BLOCKS to existing ANL DB

**Protein Families**

## SVMMER

**SVM- based Classification and Characterization Schema (in collaboration with Nagiza Samatova, ORNL)**

- http://compbio.mcs.anl.gov/svmmer/

## SVM for Protein Identification

**STATISTICS**

-  **Size of Training Set / Test Set : 1.5 GB each. (Huge Data Set as a Whole)**
** ****Each node in Chiba City : 500 MHz CPU 512 MB  RAM**
** ****Approx Run Time for 1 Functional Group in a node:**
**Learning Phase :  4 hours**
**Classification Phase : 2 hours**
**Total runtime for 2500 groups in 50 Nodes of Chiba (dual processors) approximates to 6 days!**

## Bioinformatics Report: Tasks for Q4 2002

- To Ensure the robustness of dataflow in GADU System (more than a 100 genomes were processed in GADU) – **DONE – no problem**
- Design a working plan for Chimerization of GADU – DONE (thanks to Nika and Mike Wilde)
- To use grid-based backend for genome analysis – DONE – Condor at UofW (thanks to Miron Livny and Zach Miller)
- Develop Tools and Algorithms for High-throughput genome analysis – on-going... (in collaboration with ORNL, Nagiza Samatova)
**PhyloBlock is available at **_[http://compbio.mcs.anl.gov/ulrich/phyloblock/](http://compbio.mcs.anl.gov/ulrich/phyloblock/)_
**SVMMER (in collaboration with Nagiza Samatova, ORNL) is available at: ****http://compbio.mcs.anl.gov/svmmer/**** **

## Milestones for Q1 2003

**GADU infrastructure development:**
**Development of a Chimera-ized prototype of GADU**
**Creating/enhancing a database framework for metadata and annotations**
**Utilizing large distributed resources**
**Condor (UofW – Blast, InterPro, UofC – Pfam, Blocks)**
**DataGrid**
**DOE ScienceGrid**
**Other?**
**Solving GADU space management problems**
- Tools development (in collaboration with ORNL)
**Development of algorithms for automated construction of specific HMM libraries for PhyloBlock**
**Enhancement of SVMmer algorithm with feature selection**