# Presentation: 292519

## GLU: New software for the analysis of genetic data, large and small

- Kevin Jacobs
- CGEMS Operations & Analytical LeadBioInformed LLC

**Copyright © 2008 BioInformed LLC**

## GLU?

- Genotype Library & Utilities
- GLU provides command-line tools for
- the management of large amounts of SNP genotype data
- check genotype quality
- test for association between SNP markers and phenotypes
- Designed for large scale GWAS
- Ample for smaller data sets        	TaqMan  OPA

**Copyright © 2008 BioInformed LLC**

## Genotype Library

- Scalable genotype data storage and management
- Manage a trillion genotypes (1012)
- One hundred thousand samples (105)
- Ten million of genotypes (107)
- Queries on data and metadata
- Transformations
- Import
- Export

**Copyright © 2008 BioInformed LLC**

##  GLUtilities

- Data conversion
- Data management
- Genotype quality control
- Association testing
- Tag SNP selection
- Genomic metadata

**Copyright © 2008 BioInformed LLC**

## Key innovations

- Compressed binary genotype storage
- Use of streaming and stackable data transformations that can avoid fully materializing data sets in main memory
- Integration with a high-level scripting language for easy customization and extension
- All text input files can be transparently read and written using data compression
- Designed to support for parallel processing and distributed computing (future release)

**Copyright © 2008 BioInformed LLC**

## Extensible

- Core is implemented in Python, an ideal “glue” language.  Easy to learn, easy to modify.
- Small amount of performance critical code in C
- 38,000 lines of Python
-   3,800 lines of C (optional)
- Integrates open source libraries
- HDF5, high-performance binary files from the high-energy physics community
- SQLite, embedded SQL database
- NumPy, Matlab-like vector language
- SciPy, a vast library of mathematical scientific tools
- R, for plotting and advanced numerical routines
- Powerful command-line interface, intended for analysts and developers
- GUI coming...  though not soon enough.

**Copyright © 2008 BioInformed LLC**

## Release Plans

- GLU is currently used by
- CGF, Harvard, Mayo/Moffit
- Soon GLU 1.0 will be released as free open source software
- Development will be opened to all
- Continued investment by CGF
- World-wide participation
- Still much work to do
- The curse of all 1.0 software
- Documentation and tutorial
- Installation needs to be streamlined
- Windows support

**Copyright © 2008 BioInformed LLC**

## Platform Support

- Release 1.0
- Linux 			AMD x86_64
- Linux 			Intel  x86_64
- Mac OS X		Intel  x86_64
- Release 1.1
- Windows XP/Vista 	Intel 32
- Upon request
- Sun Solaris 		Intel/Sparc
- Linux 			Intel 32 bit
- Windows Vista 	Intel x86_64

**Copyright © 2008 BioInformed LLC**

## Data formats

- Genotype triples
- Genotype matrices
- sdat: samples in rows
- ldat: loci in rows
- Capable of efficiently storing
- Biallelic SNPs, multi-allelic SNPs, indels, microsatellites, etc.
- Formats
- triples: 	tdat, tbat, PrettyBase
- ldat: 	ldat, lbat, HapMap, PLINK tped,     				PLINK bed, WTCCC
- sdat: 	sdat, sbat, PLINK ped, PLINK t-bed, 			Merlin/Mach, STRUCTURE, PHASE

**Copyright © 2008 BioInformed LLC**

## Data formats: triples

- Sequence of
- Sample identifier
- Locus identifier
- Genotype
- KM1421	rs65213	AA
- KM1422	rs65213
- KM1422	rs131312	CT
- KM1421	rs131312 	AC
- KM1422	rs5551212 	TT
- KM1423	rs131312	AA

**Copyright © 2008 BioInformed LLC**

## Data formats: ldat

- Header with sample names followed by rows of locus name and genotypes:
- ldat	S1	S2	S3	S4
- L1	AA	AT		AA
- L2	CT	CC	TT
- L3	TT	TT	TT	TT
- L4
- L5	CC	AC	CA	TT

**Copyright © 2008 BioInformed LLC**

## Data formats: sdat

- Header with sample names followed by rows of locus name and genotypes:
- sdat	L1	L2	L3	L4	L5
- S1	AA	CT	TT		CC
- S2	AT	CC	TT		AC
- S3		TT	TT		CA
- S4	AA		TT		TT

**Copyright © 2008 BioInformed LLC**

## Binary formats

- Text files are easy for humans to read, but impractical for large data sets
- Microsoft Excel allows 255 columns, 65,000 rows
- Beyond which specialized software is necessary
- Example: CGEMS breast cancer scan on NHS
- 2,332 subjects, 546,646 loci, 1.27 billion genotypes
- 	3,652 MB (24  bits/geno) uncompressed text format
- 	   673 MB (4.4 bits/geno) compressed text format
-    235 MB (1.5 bits/geno) GLU compressed binary format
-    313 MB (2.0 bits/geno) PLINK binary format (.bed)
- GLU’s format is 25% smaller
- Example: HapMap build 22
- 5,250 MB in HapMap bulk data format
-    174 MB in GLU lbat format

**Copyright © 2008 BioInformed LLC**

## Binary representations

- In memory:
- 2 bits of storage per bialleleic SNP genotype versus 8 or 64 or more bits for non-binary
- NHS: 303 MB RAM vs. 1.2 GB RAM
-         (1⁄2 of a CD-ROM vs. 1⁄4 of a DVD)
- On-disk:
- 2 bits of storage per biallelic SNP genotype on disk uncompressed,
- <2 bits of storage per SNP genotype on disk compressed,
- versus 24 bits uncompressed test or 2-5 bits compressed text
- Loading is >10x faster, saving is >5x faster
- Efficient range queries

**Copyright © 2008 BioInformed LLC**

## New formats?

- GLU defines several new formats
- Improvements
- GLU text formats can represent non-SNPs
- GLU binary formats are based on HDF5, a data storage format developed for high-energy physics and used widely
- GLU binary files also store subject and locus metadata
- Community focus
- Document and publish formats for comment
- Add support to PLINK, R-Genetics

**Copyright © 2008 BioInformed LLC**

##  GLUtilities

- Data conversion
- Data management
- Quality control
- Association testing
- Genomic metadata
- Tag SNP selection

**Copyright © 2008 BioInformed LLC**

## Data import

- Most data input and output conversations are possible from any GLU module.
- Some more complex ones are handled by the “convert” module.
- Importing Illumina genotypes from the “Locus by DNA” report format
- Importing Affymetrix genotypes

**Copyright © 2008 BioInformed LLC**

## Data management 

- glu ginfo: query genotype metadata
- Samples: 	family id, subject id, parents, sex
- Loci:	name, chromosome, location, strand, model
- $ **glu ginfo subjects_STUDY.lbat**
- GLU module: ginfo
- Filename    : subjects_STUDY.lbat
- Format      : ldat
- sample count: 10180
- locus  count: 27383
- model  count: 6
- (278 million genotypes, for those that are counting)

**Copyright © 2008 BioInformed LLC**

## Data management

- glu transform:
- import, export, merge, and genotype data among several common formats and standards
- Filter based on powerful criteria for inclusion, exclusion
- Rename and adjust sample and locus metadata
- Transpose:
**$ glu transform project.ldat –o project.sdat**
- Merge:
**$ glu transform taqman.sdat fluidigm.ldat –o project.lbat**
- Subset:
**$ glu transform --excludesamples=naughty.txt **
**                ****--renameloci=special.map**
**                ****project.lbat –o subset.phase**

**Copyright © 2008 BioInformed LLC**

## Data management

- glu split: Split genotypes into many subsets
- By sample group, by locus group, and into chunks
**$ glu split –-samplegroups=ccstatus.txt project.ldat**
- project_*CASE*.ldat, project_*CONTROL*.ldat, project_*UNKNOWN*.ldat
**$ ****glu split –-locusgroups=map.txt project.ldat**
- project_*chr1*.ldat, project_*chr2*.ldat, ..., project_*chrX*.ldat
**$ ****glu split --maxrows=1000 project.ldat**
- project_*part1*.ldat, project_*part2*.ldat, ..., project_*n*.ldat
- Possible to specify sample groups, locus groups, and maxrows simultaneously: project_CASE_chr1_part4.ldat

**Copyright © 2008 BioInformed LLC**

## qc.completion

- Estimation of assay completion
- Proportion of non-missing genotypes to total genotypes
- By sample or group of samples
- By locus or group of loci
**$ glu qc.completion project.lbat**
- Sample  Total:  2610, Empty:  0, Dropped:  0, Informative:  2610
- Locus   Total:    60, Empty:  0, Dropped:  0, Informative:    60
- GLOBAL GENOTYPE COMPLETION RATE:  138348 / 156600 = 88.345%
- MISSING GENOTYPES BY SAMPLE
- Rank  Sample                           N / Total      %
-   ----  -------------------------  -----------------  ------
-      1  CG-P02-10201                     1 /      60    1.67
-      2  CG-P02-10405                     1 /      60    1.67
-      3  CG-P02-11648                     1 /      60    1.67
- ...
-    488  CG-P02-14188                    57 /      60   95.00
-    489  CG-P02-14485                    57 /      60   95.00
-    490  CG-P02-14536                    57 /      60   95.00
-    491  CG-P02-14645                    57 /      60   95.00

**Copyright © 2008 BioInformed LLC**

##   Rank  Locus                        N / Total      %

-   Rank  Locus                        N / Total      %
-   ----  -----------------------  ---------------  ------
-      1  rs116013                  1436 /    2610   55.02
-      2  rs74566                   2148 /    2610   82.30
-      3  rs177295                  2215 /    2610   84.87
-      4  rs109649                  2251 /    2610   86.25
-      5  rs102675                  2256 /    2610   86.44
-      6  rs65509                   2259 /    2610   86.55
-      7  rs49279                   2261 /    2610   86.63
-      8  rs102623                  2263 /    2610   86.70
-      9  rs79443                   2264 /    2610   86.74
-     10  rs114546                  2274 /    2610   87.13
-     42  rs119319                  2323 /    2610   89.00
-     43  rs81215                   2323 /    2610   89.00
-     44  rs47319                   2325 /    2610   89.08
-     45  rs70698                   2327 /    2610   89.16
-     46  rs71871                   2327 /    2610   89.16
-     47  rs127872                  2338 /    2610   89.58
-     48  rs109470                  2344 /    2610   89.81
-     49  rs49118                   2392 /    2610   91.65
-     50  rs128487                  2419 /    2610   92.68
-     51  rs2496                    2428 /    2610   93.03
-     52  rs110422                  2439 /    2610   93.45
-     53  rs108667                  2442 /    2610   93.56
-     54  rs19081                   2446 /    2610   93.72
-     55  rs70336                   2449 /    2610   93.83
-     56  rs123166                  2452 /    2610   93.95
-     57  rs44885                   2454 /    2610   94.02
-     58  rs110607                  2456 /    2610   94.10
-     59  rs71094                   2465 /    2610   94.44
-     60  rs40211                   2484 /    2610   95.17

**Copyright © 2008 BioInformed LLC**

## qc.hwp

- Test for deviations from Hardy-Weinberg proportions
- Exact test for biallelic SNPs
- Asymptotic test for general markers
**$ glu qc.hwp data.ldat**
-  Exact       Asymptotic
-   Rank  Locus            hom1s  hets   hom2s    n    p-value     p-value
-   ----  ---------------  -----  -----  -----  -----  ----------  ----------
-      1  rs4546145            0    297    229    526  0.00000000  0.00000000
-      2  rs11575404          16     18    506    540  0.00000000  0.00000000
-      3  rs7599592           15     17    513    545  0.00000000  0.00000000
-      4  rs2164560            0    243    282    525  0.00000000  0.00000000
- ...
-   4104  rs11210896           6     76    463    545  0.14895419  0.15773081
-   4105  rs11753972           6    137    403    546  0.14895798  0.13031716
-   4106  rs2291369            6    137    403    546  0.14895798  0.13031716
-   4107  rs7244046           53    258    234    545  0.14912755  0.13415849
- ...

**Copyright © 2008 BioInformed LLC**

## Other QC tools

- qc.maf: Estimate allele frequencies
- qc.hets: Estimate sample heterozygosity
- Useful to detect certain types of genotyping errors
- Serves as a sex checker when restricted to only sex-linked loci
- A crude test for population stratification
- qc.mendel_check: Mendelian checking
- Check for genotypes inconsistent with Mendelian inheritance among related individuals

**Copyright © 2008 BioInformed LLC**

## qc.concordance

- Genotype concordance checker
- Compares two genotype data sets for concordance
- Accommodates:
- Differently named samples and markers
- strand and allele coding differences
- Can determine optimal allele remapping
- Reports:
- Count of concordant genotypes
- Counts of discordances for
- heterozygote  heterozygote,
- homozygote  heterozygote,
- heterozygote  homozygote
- homozygote  homorozygote,
- Hardy-Weinberg statistics
- Allele remapping applied

**Copyright © 2008 BioInformed LLC**

## qc.dupcheck

- Detection and verification of known and detection of unknown duplicate samples
- Used to detect a multitude of inconsistencies, informatic and laboratory
- Counts concordant genotypes for all possible pairs of samples
- Sorts results in categories
- Expected duplicated, found
- Expected duplicated, not found
- Unexpected duplicated
- Can be used on both microsatellite and SNP data

**Copyright © 2008 BioInformed LLC**

## Analytical tools

- Fitting generalized linear models to test for phenotype/genotype association
- Discrete and continuous outcomes
- Genetic models: genotype, trend, dominant, recessive, missing, and interactions
- Arbitrary covariates
- Linkage disequilibrium (LD) estimation
- TagZilla: a high-performance SNP tagger
- Can to augment SNPs from set panels with a optimal tag SNPs
- Uses flexible criteria incorporating design scores from major genotyping vendors
- Provides detailed genomic coverage information

**Copyright © 2008 BioInformed LLC**

## Copyright © 2008 BioInformed LLC

**Copyright © 2008 BioInformed LLC**

## Copyright © 2008 BioInformed LLC

**Copyright © 2008 BioInformed LLC**

## Copyright © 2008 BioInformed LLC

**Copyright © 2008 BioInformed LLC**

## genedb

- Embedded SQL database of HapMap, Illumina, and Affymetrix annotations
- genedb.annoate
- Input is a file containing a list of SNPs
- Ouput adds columns for chromosome, physical location, and gene neighborhood
- genedb.find_snps
- Input is a list of genes or genomic regions
- Output is all of the SNPs within or near that region

**Copyright © 2008 BioInformed LLC**

## What’s next?

- GLU 1.0 release (soon)
- _[http://](http://cgf.nci.nih.gov/development/tooldev.html)__[cgf.nci.nih.gov/development/tooldev.html](http://cgf.nci.nih.gov/development/tooldev.html)_
- Tutorials
- GLU at the DCEG molecular epi course
- GLU at ENAR
- Short course
- From you:
- Feedback
- Feature requests
- Bug reports

**Copyright © 2008 BioInformed LLC**

## Acknowledgements

- GLU would not exist without the vision, support and help of
- Meredith Yeager
- Nick Orr
- Peter Kraft
- Bob Welch
- Stephen Chanock
- CGF Lab and Bioinformatics Groups
- CGEMS Analysis Working Group
- SAIC-Frederick
- DCEG

**Copyright © 2008 BioInformed LLC**