BioRuby 1.0 &the BioRuby shell
Toshiaki Katayama<k@bioruby.org>
Univ of Tokyo, JapanBOSC2006 in Fortaleza, Brazil (Aug 5 2006) / Copyright (C) 2006 Toshiaki Katayama

kToshiaki Katayama@bioruby.org

fromlab Tokyo branchwhereI'm developing ...
• KEGG API
SOAP/WSDL based web service
http://www.genome.jp/kegg/soap/• KEGG DAS
GMOD/GBrowse based genome DB
http://das.hgc.jp/

to get here

11 hours

http://www.lib.utexas.edu/maps/

wait for 8 hours

40 overbookedanother 24 hours

10 hourslost baggage

cancelled

no refund...

wait 7 hours

6 hours w/ 2 stops

BrazilAIRPASS75 hours to getto this paradise!

and took
138 hours(6days)for myluggage...

BioRubyfacts• What: BioPerl for Ruby language• When: started in Nov 2000
• Who: about 10 developers(3 Japanese core developers + others)• Where: originally in KEGG lab(now hosted at open-bio.org)

Why Ruby?
(Do you know Ruby?)

Ruby is ...
• object oriented• scripting language• born in Japan• 10 years old

Ruby in five E's• Everything is an object• Elegant blocks of inline joy• Exploring with reflection• Extending at runtime• Extensive standard librarycited from http://media.rubyonrails.org/presentations/fisl6.pdf

but now known as ...
• the language needed to run Rails

Ruby on Rails - http://rubyonrails.org/

history

2000

2000
• BioRuby project was started• some in-house Perl modules wereported to Ruby

• BioPerl, BioPython, BioJava werealready existed

2001• Jun 21: bioruby-0.2.tar.gz• Sequence model• GenBank, PROSITE, KEGG parsers• DBGET (entry retrieval system)• FASTA (summarize results)• developed a genome browser

2001• BOSC in Copenhagen2001• visited Ewan@EBIBioRuby what?!%$@

2002• BioHackathon #1, #2• Tucson - O'Reilly• South Africa - Electric GeneticsOBDA• Open Bio* Database Access• BioRegistry• BioFlat• BioFetch• BioSQL

2003• BioHackathon #3• Singapore - Apple
GBrowse (Bio::Graphics), Chado, BioMOBY, ...

• BOSC2003• Australia - BioRuby w/ KEGG API• BioRuby 0.5• applications, web services

2004• BOSC2004• Glasgow• BioRuby w/ pathway + genome• BioRuby 0.6• fairly stable2005• Funded by IPA• ChemRuby launched• pandemic of Rails

2006• BioRuby 1.0.0 is out!

DLs 05-06

500000

2000000
1500000
1000000

1.0M
0.5M

0Hits

1.5M

2005
1714223038

2010

46

54

Month

62

70

78

8694

2015

1.5M DLs in 2015

anywayBioRuby 1.0• Released: Feb 2006
• New features:• interactive shell• unit tests• documents• bug fixes (some bugs are not yet fixed ;-)purposebioinfo + cheminfo
genomemicroarray

pathwaydrug

new methods

automation
Innovation & BusinessArchitecture Inc.

medical genome
BioRuby

ChemRuby

BioRuby + ChemRuby
Bio::Sequence
Bio::GenBank,Bio::EMBL etc.
RefSeqLIGAND

EMBL

DNA(ゲノム,遺伝子)タンパク質配列DB

GenBank

GenPept

GENES

UniProt
UCSC

KEGG
DAS

EnsemblBio::Reference

文献情報

gene

COM
POUND

PubChem

酵素,化合物DB

Bio::Flat(BDB)
Bio::Fetch(HTTP)
Bio::SQL(RDB)
Bio::DAS(REST CGI)ENZYME

各種フォーマット

Bio::KEGG::API(SOAP/WSDL)

interactioncompound
PATHWAY

PubMedE-Utils(CGI)

Library to handle various
biological/medical databases
and applications is required
Bio::Pathway

化学計算ソフトウェア

PDB

タンパク質立体構造

Bio::BLAST,Bio::HMMER etc.

HMMER

遺伝子発現

解析ソフトウェア

TINKERbiological data formats
Sequences (GenBank, UniProt,......)MMEILRGSPALSAFRINKLLARFQAARLPVHNIYAEYVHFADLNAPLNDDEHAQLERLLKYGPALASHAPQGKLLLVTPRPGTISPWSSKATDIAHNCGLQQVNRLERGVAYYIEAGTLTNEQWQQVTAELHDRMMETVFFALDDAEQLFAHHQPTPVTSVDLLGQGRQALIDANLRLGLALAEDEIDYLQDAFTKLGRNPNDIELYMFAQANSEHCRHKIFNADWVIDGEQQPKSLFKMIKNTFETTPDHVLSAYKDNAAVMEGSEVGRYFADHETGRYDFHQEPAHILMKVETHNHPTAISPWPGAATGSGGEIRDEGATGRGAKPKAGLVGFSVSNLRIPGFEQPWEEDFGKPERIVTALDIMTEGPLGGAAFNNEFGRPALNGYFRTYEEKVNSHNGEELRGYHKStructures (PDB,......)HEADERTITLE
TITLE
TITLE:ATOMATOMATOMATOMATOM

LIGASE
15-DEC-04
1VQ3CRYSTAL STRUCTURE OF PHOSPHORIBOSYLFORMYLGLYCINAMIDINE
2 SYNTHASE, PURS SUBUNIT (EC 6.3.5.3) (TM1244) FROM
3 THERMOTOGA MARITIMA AT 1.90 A RESOLUTION
12345NCACOCBHIS AHIS AHIS AHIS A
HIS A-3-3-3-3-370.636
70.43669.09868.68470.49228.97929.95430.65731.30829.234

8.146
7.050
7.163
6.214
5.702

1.00 32.07
1.00 28.18
1.00 26.01
1.00 29.86
1.00 33.52

NCCOC

XML (DAS, KGML,......)

Expressions (GEO,.......)<?xml version="1.0" standalone="yes"?><!DOCTYPE DASGFF SYSTEM "http://www.biodas.org/dtd/dasgff.dtd"><DASGFF><GFF version="1.01" href="http://das.hgc.jp/cgi-bin/das/eco/features?segment=eco%3A2671072%2C2721071"><SEGMENT id="eco" start="2671072" stop="2721071" version="1.0"><FEATURE id="EC:1.14.12.17/7199" label="1.14.12.17"><TYPE id="enzyme:KEGG" category="enzyme">enzyme:KEGG</TYPE><METHOD id="enzyme">enzyme</METHOD><START>2683857</START><END>2685047</END>

Affymetrix:CompositeSequence:HG_U95Av2:AFFX-HUMGAPDH/M33197_5_at
HUMGAPDH/M33197_5_at
IPB000173
1.4.1.16
M33197
ENSG00000111640 7
Affymetrix:CompositeSequence:HG_U95Av2:AFFX-HUMGAPDH/M33197_M_at
HUMGAPDH/M33197_M_at
IPB000173
1.4.1.16
M33197
ENSG00000111640 7
Affymetrix:CompositeSequence:HG_U95Av2:AFFX-HUMGAPDH/M33197_3_at
HUMGAPDH/M33197_3_at
IPB000173
1.4.1.16
M33197
ENSG00000111640 7

AFFXAFFXAFFX-chemical data formats
Linear (SMILES, InChI,......)
CC1C(C(CC(O1)OC2C(C(C(OC2OC3=C4C=C5C=C3OC6=C(C=C(C=C6)C(C(C(=O)NC(C(=O)NC5C(=O)NC7C8=CC(=C(C=C8)O)C9=C(C=C(C=C9C(NC(=O)C(C(C1=CC(=C(O4)C=C1)Cl)O)NC7=O)C(=O)O)O)O)CC(=O)N)NC(=O)C(CC(C)C)NC)O)Cl)CO)O)O)(C)N)O

Network (MDL, Tinker,......)
ISISHOST03240423012D 15XML (CML, CDXML,......)<?xml version="1.0" encoding="UTF-8" ?><!DOCTYPE CDXML SYSTEM "http://www.cambridgesoft.com/xml/cdxml.dtd" ><CDXML
CreationProgram="ChemDraw 7.0"
Name="hypericin.mol"
BoundingBox="67.44 97.35 301.63 315.25"

4 00 0-0.04140.1586-0.7621-0.25170.6759-0.25170.1724-0.6414-0.04140.98971 2 1 00 0
1 3 1 00 0
1 4 1 00 0
1 5 2 00 0
M END

1.00000
999 V2000
0.0000 P
0 00.0000 O0 00.0000 O
0 00.0000 O
0 00.0000 O0 00.00000
3000000000900000000000000000000

Binary (CDX,.......)
00000000: 0d6a 4344 3031 3030 0403 0201 0000 0000
00000010: 0000 0000 0000 0000 0000 0000 0300 0e00
00000020: 0000 4368 656d 4472 6177 2037 2e30 0800
00000030: 0f00 0000 6879 7065 7269 6369 6e2e 6364
00000040: 7800 0332 0008 00ff ffff ffff ff00 0000
00000050: 0000 00ff ff00 0000 00ff ffff ff00 0000
00000060: 00ff ff00 0000 00ff ffff ff00 0000 00ff.jCD0100........
................
..ChemDraw 7.0..
....hypericin.cd
x..2............
................
................

00000

ChemRuby
find similarstructurewith KEGG
Connect
seamlessly..

BioRubygene,
expression,
pathway# 枯草菌のパスウェイ106枚中の１枚（アミノ酸合成経路）に# 栄養源枯渇時の遺伝子発現データを色でマッピング# process KEGG Expression data here...

serv = Bio::KEGG::API.new
list = serv.list_pathways(org)list.each do |path|pathway = path.entry_id
fg_list = Array.new
bg_list = Array.new
genes = serv.get_genes_by_pathway(pathway)genes.each do |gene|fg_list << "#000000"
bg_list << hash[gene] || "#cccccc"
end
url = serv.color_pathway_by_objects(pathway, genes, fg_list, bg_list)
serv.save_image(url)endprogress

BioRuby dev in 2005
GUI: Rails (web)
CUI: Shell (term)
BioRuby/ChemRuby}

New!

Unit tests

Documents

Ruby : String, RegExp, I/O, Test::Unit, SOAP

BioRuby shell• newly implemented in BioRuby 1.0• interactive CUI for BioRuby libraryfeatures• persistent object (across sessions)• history (with timestamps)• plugins (to extend functionality)• script generation(interactive prototyping toreproduce the procedure)Library vs Shell (CUI based on 'irb')•for biologist (!= programmer)•uniﬁed interface

TAMWTDI vs TIOOWTDI
too#!/usr/bin/env rubyrequire 'bio'# sequence generated from string
dna = Bio::Sequence::NA.new("atgc")# sequence obtained via KEGG API
kegg = Bio::KEGG::API.new
entry = kegg.bget("hsa:217")
gene = Bio::KEGG::GENES.new(entry)
dna = gene.naseq# sequence obtained via OBDA BioFetch
obda = Bio::Fetch.newentry = obda.fetch("gb:AF237819")
gb = Bio::GenBank.new(entry)
dna = gb.naseq# sequence obtained via EMBOSS seqret
entry = Bio::EMBOSS.entret("gb:AF237819")
gb = Bio::GenBank.new(entry)
dna = gb.naseq# sequence obtained from local flat file
Bio::FlatFile.auto("file.txt") do |ff|
ff.each do |entry|
dna = entry.naseq
end
end% bioruby
bioruby> dna = seq("atgc")
bioruby> dna = seq("hsa:217")
bioruby> dna = seq("gb:AF237819")
bioruby> flatfile("file.txt") do |entry|
bioruby+dna = seq(entry)
bioruby+ end

to obtain sequenceas a library #1• Generate sequence from string#!/usr/bin/env rubyrequire 'bio'# sequence generated from string
dna = Bio::Sequence::NA.new("atgc")

as a library #2• Obtain sequence from KEGG#!/usr/bin/env rubyrequire 'bio'# sequence obtained via KEGG API
kegg = Bio::KEGG::API.new
entry = kegg.bget("hsa:217")
gene = Bio::KEGG::GENES.new(entry)
dna = gene.naseq

as a library #3• Obtain sequence from BioFetch#!/usr/bin/env rubyrequire 'bio'# sequence obtained via OBDA BioFetch
obda = Bio::Fetch.newentry = obda.fetch("gb:AF237819")
gb = Bio::GenBank.new(entry)
dna = gb.naseq

as a library #4• Obtain sequence from EMBOSS#!/usr/bin/env rubyrequire 'bio'# sequence obtained via EMBOSS seqret
entry = Bio::EMBOSS.entret("gb:AF237819")
gb = Bio::GenBank.new(entry)
dna = gb.naseq

as a library #5• Obtain sequence from EMBOSS#!/usr/bin/env rubyrequire 'bio'# sequence obtained from local flat file
Bio::FlatFile.auto("file.txt") do |ff|
ff.each do |entry|
dna = entry.naseq
endendas a shell• Unified interface% biorubybioruby> dna = seq("atgc")
bioruby> dna = seq("hsa:217")
bioruby> dna = seq("gb:AF237819")
bioruby> dna = seq("file.txt")
bioruby> flatfile("file.txt") do |entry|
bioruby+dna = seq(entry)
bioruby+ end

list of shell commands
seq

sequence retrieval

keggapi

access KEGG API serverent

database entry retrieval

keggdbslist databases in KEGG

obj

parsed entry retrieval

keggorgs

list organisms in KEGG

seqstatsequence statistics

keggpathways list pathways in KEGG

aminoacids

list of amino acids

binfo

database update info

nucleicacids list of nucleic acids

bfind

search KEGG databases

codontables translation tablebget

retrieve KEGG entry

flatparse

parse flatfile entry

bconv

convert to KEGG ID

flatfile

open flatfile w/ iteration

obdadbslist OBDA databases

flatauto

auto detect DB format

biofetch

OBDA BioFeth

flatindex

create index on flatfile

psort

predict localization

flatsearchsearch indexed flatfile

blast

execute BLAST searchflatfasta

convert to fasta format

fasta

execute FASTA searchlist of util commands
lslist of objects

scriptsave procedure in filerm

delete object

pager

set pager used by dispsavefile

save object in fileconfigcd

change directory:echo

toggle for irb echo

dirlist of files:colortoggle for color mode

pwd

print working directory:messageset opening message

head

peep in file or object:splashset opening splash

dispdisplay file or object

demo

foldfolding text

:sequence

sequence demo

fillfilling text

:entryentry retrieval demo

htmlseq

generate colored sequence:pdb

PDB demo

doublehelix display animated sequence
midifile

generate DNA music:webbind to Rails

obtain gene sequence
sequence manipulation
mapping to pathway
genome music

DEMO - TOC

other enhancements

1.0 features• BioRuby shell• documentations• unit tests

documentation

docs• new English tutorials• API refs http://bioruby.org/rdoc/• doc site http//bioruby-doc.org/• guideline doc for developer

Bundled tutorials
Thank you, Pjotr!
(according to Google..)I miss you.

Japanese Ed.
doubled！

English Ed.
NEW！

APIs on the web

http://bioruby.org/rdoc/
RDoc formatted
documentations
by Jan and Ryan!

http://bioruby-doc.org/

English doc siteby Trevor!

README.DEV• guidelines for code contribution• license• coding style• name spaces• unit tests• autoloads

mentioned in books

Unit tests• added >1000 tests for reliability

miscautoload
30x faster to ﬁre up!
% repeat 10 { time ruby -r bio -e 0 }------------------------------------------------------------0.6.4:
0.74s user 0.16s system
1.082 total
0.7.0 (9/10):
0.01s user 0.01s system
0.033 total

1.082 sec
→ 0.033 sec

BioRuby on Rails

road to open-bio 2.0

BOSC2005@Detroit

BioPerlachieved!

what direction to go?
Nature (2002) 417:119

web services

Ruby and WS

SOAP/WSDL in Java
1. download and install Axis library(...snip...)2. generate Java classes from WSDL
% java -classpath axis.jar:jaxrpc.jar:commons-logging.jar:commons-discovery.jar: saaj.jar: wsdl4j.jar:.
org.apache.axis.wsdl.WSDL2Java -p keggapi http://soap.genome.jp/KEGG.wsdl% javac -classpath axis.jar:jaxrpc.jar:wsdl4j.jar:. keggapi/KEGGLocator.java% jar cvf keggapi.jar keggapi/*

3. codeimport keggapi.*;class GetGenesByPathway {public static void main(String[] args) throws Exception {KEGGLocator locator = new KEGGLocator();KEGGPortType serv = locator.getKEGGPort();String query = args[0];String[] results = serv.get_genes_by_pathway(query);}for (int i = 0; i < results.length; i++) {System.out.println(results[i]);}}4. compile and execute(...snip...)SOAP/WSDL in Ruby
1. code#!/usr/bin/env rubyrequire 'soap/wsdlDriver'wsdl = "http://soap.genome.jp/KEGG.wsdl"
serv = SOAP::WSDLDriverFactory.new(wsdl).create_driverserv.generate_explicit_type = true
puts serv.get_enzymes_by_pathway(ARGV.shift)

2. execute% ruby enzymes.rb map00020#!/usr/bin/env rubyrequire 'bio'serv = Bio::KEGG::API.new
puts serv.get_enzymes_by_pathway(ARGV.shift)WSs in combination#!/usr/bin/env rubyrequire 'bio'
kegg = Bio::KEGG::API.new
seq = kegg.get_aaseqs("hsa:aldh2")

ddbj = Bio::DDBJ::XML::Blast.new
res = ddbj.searchSimple("blastp", "SWISS", seq)
hit = Bio::Blast::Report.new(res).hits.firstncbi = Bio::NCBI::SOAP::EUtils.new
ent = ncbi.run_eFetch("db" => "protein","id" => hit)ebi:= Bio::EBI::SOAP::Dbfetch.newKEGG API access log

Road to BioRuby 2.0• 1.1: ASAP (need to fix serious bug :)• 1.2: BioRuby with ActiveRecord• 1.3: BioRuby <=> ChemRuby:• 1.9: BioRuby on Rails• 2.0: BioRuby on web service

Acknowledgments• M. Nakao : Kazusa DNA Inst.
• N. Goto : Osaka Univ.
• N. Tanaka : Kyoto Univ. (ChemRuby)• T. Katayama : Tokyo Univ.
• + contributors, committers on theInternet

Questions?
• If you have, please ask me veeeeerysloooooowly, clearly, word by word.

• Q: ___or type in here___http://bioruby.org/