Package: pyensemblrest
pyEnsemblRestis a simple Python wrapper around the EnsEMBL REST APILicensepyEnsemblRest - A wrapper for the EnsEMBL REST APICopyright (C) 2013-2016, Steve MosspyEnsemblRest is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.pyEnsemblRest is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.You should have received a copy of the GNU General Public License
along with pyEnsemblRest.  If not, see <http://www.gnu.org/licenses/>.InstallationUsing pipSimply type:pipinstallpyensemblrestFrom sourceClone the pyEnsemblRest then install package from source:gitclonehttps://github.com/pyOpenSci/pyEnsemblRest.gitcdpyEnsemblRestsudopythonsetup.pyinstallUsageTo import an setup a new EnsemblRest object you should do the following:fromensemblrestimportEnsemblRestensRest=EnsemblRest()EnsemblRest() istance points tohttp://rest.ensembl.org/. In order to use EnsemblGenome, you can import a different object:fromensemblrestimportEnsemblGenomeRestensGenomeRest=EnsemblGenomeRest()Or, as an alternative, you can give a different base url during EnsemblRest class instantiation:fromensemblrestimportEnsemblRestensGenomeRest=EnsemblRest(base_url='http://rest.ensemblgenomes.org')To use a custom EnsEMBL REST server you should setup the EnsemblRest as the precedent way:fromensemblrestimportEnsemblRest# setup rest object to point to localhost server. The 3000 stands for REST default portensRest=EnsemblRest(base_url='http://localhost:3000')You may also provide proxy server settings in the form of a dict, as follows:fromensemblrestimportEnsemblRest# setup rest object to point to a proxy serverensRest=EnsemblRest(proxies={'http':'proxy.address.com:3128','https':'proxy.address.com:3128'})EnsEMBL has a rate-limit policy to deal with requests. You can do up to 15 requests per second. You could wait a little during your requests:fromtimeimportsleep# sleep for a second so we don't get rate-limitedsleep(1)Alternatively this library verifies and limits your requests to 15 requests per second. Avoid to run different python processes to get your data, otherwise you will be blacklisted by ensembl team. If you have to do a lot or requests, consider to use POST supported endpoints, or contact ensembl team to add POST support to endpoints of your interest.GET endpointsEnsemblRest and EnsemblGenomeRest class methods are not defined in libraries, so you cannot see docstring using help() method on python or ipython terminal. However you can see all methods available forensemblandensemblgenomesrest server once class is instantiate. To get help on a particular method, please refer to ensembl help documentation on different endpoints in theensemblandensemblgenomesrest service. Please note that endpoints onensemblmay be different fromensemblgenomesendpoints.
If you look, for example, atsequenceendpoint documentation, you will find optional and required parameters. Required parameters must be specified in order to work properly, otherwise you will get an exception. Optional parameters may be specified or not, depending on your request. In all cases parameter name are the same used in documentation. For example to get data usingsequenceendpoint, you must specify at least required parameters:seq=ensRest.getSequenceById(id='ENSG00000157764')In order to mask sequence and to expand the 5’ UTR you may set optional parameters using the same name described in documentation:seq=ensRest.getSequenceById(id='ENSG00000157764',mask="soft",expand_5prime=1000)Multiple values for a certain parameters (for GET methods) can be submitted in a list. For example, to get the same results ofcurl'http://rest.ensembl.org/overlap/region/human/7:140424943-140624564?feature=gene;feature=transcript;feature=cds;feature=exon'-H'Content-type:application/json'as described inoverlap regionGET endpoint, you can use the following function:data=ensRest.getOverlapByRegion(species="human",region="7:140424943-140624564",feature=["gene","transcript","cds","exon"])POST endpointsPOST endpoints can be used as the GET endpoints, the only difference is that they support parameters in python list in order to perform multiple queries on the same ensembl endpoint. The parameters name are the same used in documentation, for example we can use thePOST sequenceendpoint in such way:seqs=ensRest.getSequenceByMultipleIds(ids=["ENSG00000157764","ENSG00000248378"])where the example value{ "ids" : ["ENSG00000157764", "ENSG00000248378" ] }is converted in the non-positional argumentids=["ENSG00000157764","ENSG00000248378" ]. As the previous example, we can add optional parameters:seqs=ensRest.getSequenceByMultipleIds(ids=["ENSG00000157764","ENSG00000248378"],mask="soft")Change the default Output formatYou can change the default output format by passing a supportedContent-typeusing
thecontent_typeparameter, for example:plain_xml=ensRest.getArchiveById(id='ENSG00000157764',content_type="text/xml")For a complete list of supportedContent-typeseeSupported MIME Typesfrom
ensembl REST documentation. You need also to check if the sameContent-typeis supported in the EnsEMBL endpoint description.Rate limitingSometime you can be rate limited if you are querying EnsEMBL REST services with more than one concurrent processes, or bysharing ip addresses. In such case, you can have a message like this:ensemblrest.exceptions.EnsemblRestRateLimitError:EnsEMBLRESTAPIreturneda429(TooManyRequests):Youhavebeenrate-limited;waitandretry.TheheadersX-RateLimit-Reset,X-RateLimit-LimitandX-RateLimit-Remainingwillinformyouofhowlongyouhaveuntilyourlimitisresetandwhatthatlimitwas.Ifyougetthisresponseandhavenotexceededyourlimitthencheckifyouhavemadetoomanyrequestspersecond.(Ratelimithit:Retryafter2seconds)Even if this library tries to do 15 request per seconds, you should avoid to run multiple
EnsEMBL REST clients. To deal which such problem without interrupting your code, try
to deal with the exception; For example:# import required modulesimportosimportsysimporttimeimportlogging# get ensembl REST modules and exceptionfromensemblrestimportEnsemblRestfromensemblrestimportEnsemblRestRateLimitError# An useful way to defined a logger lever, handler, and formatterlogging.basicConfig(format='%(asctime)s-%(name)s-%(levelname)s-%(message)s',level=logging.INFO)logger=logging.getLogger(os.path.basename(sys.argv[0]))# setup a new EnsemblRest objectensRest=EnsemblRest()# Get a request and deal with retry_after. Set a maximum number of retries (don't# try to do the same request forever or you will be banned from ensembl!)attempt=0max_attempts=3whileattempt<max_attempts:# update attempt countattempt+=1try:result=ensRest.getLookupById(id='ENSG00000157764')# exit while on successbreak# log exception and sleep a certain amount of time (sleeping time increases at each step)exceptEnsemblRestRateLimitError,message:logger.warn(message)time.sleep(ensRest.retry_after*attempt)finally:ifattempt>=max_attempts:raiseException("max attempts exceeded (%s)"%(max_attempts))sys.stdout.write("%s\n"%(result))sys.stdout.flush()Methods listHere is a list of all methods defined. Methods called byensRestinstance are specific toensemblrest server, while methods called viaensGenomeRestinstance are specific ofensemblgenomesrest server.To access theArchiveendpoints you can use the following methods:printensRest.getArchiveById(id="ENSG00000157764")printensRest.getArchiveByMultipleIds(id=["ENSG00000157764","ENSG00000248378"])To access theComparative Genomicsendpoints you can use the following methods:printensGenomeRest.getGeneFamilyById(id="MF_01687",compara="bacteria")printensGenomeRest.getGeneFamilyMemberById(id="b0344",compara="bacteria")printensGenomeRest.getGeneFamilyMemberBySymbol(symbol="lacZ",species="escherichia_coli_str_k_12_substr_mg1655",compara="bacteria")# Change the returned content type to "Newick" formatprintensRest.getGeneTreeById(id='ENSGT00390000003602',nh_format="simple",content_type="text/x-nh")printensRest.getGeneTreeMemberById(id='ENSG00000157764')printensRest.getGeneTreeMemberBySymbol(species='human',symbol='BRCA2')printensRest.getAlignmentByRegion(species="taeniopygia_guttata",region="2:106040000-106040050:1",species_set_group="sauropsids")printensRest.getHomologyById(id='ENSG00000157764')printensRest.getHomologyBySymbol(species='human',symbol='BRCA2')To access theCross Referencesendpoints you can use the following methods:printensRest.getXrefsById(id='ENSG00000157764')printensRest.getXrefsByName(species='human',name='BRCA2')printensRest.getXrefsBySymbol(species='human',symbol='BRCA2')To access theInformationendpoints you can use the following methods:printensRest.getInfoAnalysis(species="homo_sapiens")printensRest.getInfoAssembly(species="homo_sapiens",bands=1)#bands is an optional parameterprintensRest.getInfoAssemblyRegion(species="homo_sapiens",region_name="X")printensRest.getInfoBiotypes(species="homo_sapiens")printensRest.getInfoComparaMethods()printensRest.getInfoComparaSpeciesSets(methods="EPO")printensRest.getInfoComparas()printensRest.getInfoData()printensGenomeRest.getInfoEgVersion()printensRest.getInfoExternalDbs(species="homo_sapiens")printensGenomeRest.getInfoDivisions()printensGenomeRest.getInfoGenomesByName(name="campylobacter_jejuni_subsp_jejuni_bh_01_0142")#This response is very heavy#print ensGenomeRest.getInfoGenomes()printensGenomeRest.getInfoGenomesByAccession(division="U00096")printensGenomeRest.getInfoGenomesByAssembly(division="GCA_000005845")printensGenomeRest.getInfoGenomesByDivision(division="EnsemblPlants")printensGenomeRest.getInfoGenomesByTaxonomy(division="Arabidopsis")printensRest.getInfoPing()printensRest.getInfoRest()printensRest.getInfoSoftware()printensRest.getInfoSpecies(division="ensembl")printensRest.getInfoVariation(species="homo_sapiens")# Restrict populations returned to e.g. only populations with LD data. It is highly recommended# to set a filter and to avoid loading the complete list of populations.printensRest.getInfoVariationPopulations(species="homo_sapiens",filter="LD")To access theLinkage Disequilibriumendpoints you can use the following methods:printensRest.getLdId(species="human",id="rs1042779",population_name="1000GENOMES:phase_3:KHV",window_size=500,d_prime=1.0)printensRest.getLdPairwise(species="human",id1="rs6792369",id2="rs1042779")printensRest.getLdRegion(species="human",region="6:25837556..25843455",population_name="1000GENOMES:phase_3:KHV")To access theLookupendpoints you can use the following methods:printensRest.getLookupById(id='ENSG00000157764')printensRest.getLookupByMultipleIds(ids=["ENSG00000157764","ENSG00000248378"])printensRest.getLookupBySymbol(species="homo_sapiens",symbol="BRCA2",expand=1)printensRest.getLookupByMultipleSymbols(species="homo_sapiens",symbols=["BRCA2","BRAF"])To access theMappingendpoints you can use the following methods:printensRest.getMapCdnaToRegion(id='ENST00000288602',region='100..300')printensRest.getMapCdsToRegion(id='ENST00000288602',region='1..1000')printensRest.getMapAssemblyOneToTwo(species='human',asm_one='NCBI36',region='X:1000000..1000100:1',asm_two='GRCh37')printensRest.getMapTranslationToRegion(id='ENSP00000288602',region='100..300')To access theOntologies and Taxonomyendpoints you can use the following methods:printensRest.getAncestorsById(id='GO:0005667')printensRest.getAncestorsChartById(id='GO:0005667')printensRest.getDescendantsById(id='GO:0005667')printensRest.getOntologyById(id='GO:0005667')printensRest.getOntologyByName(name='transcription factor complex')printensRest.getTaxonomyClassificationById(id='9606')printensRest.getTaxonomyById(id='9606')printensRest.getTaxonomyByName(name="Homo%25")To access theOverlapendpoints you can use the following methods:printensRest.getOverlapById(id="ENSG00000157764",feature="gene")printensRest.getOverlapByRegion(species="human",region="7:140424943-140624564",feature="gene")printensRest.getOverlapByTranslation(id="ENSP00000288602")To access theRegulationendpoints you can use the following method:printensRest.getRegulatoryFeatureById(species="homo_sapiens",id="ENSR00001348195")To access theSequencesendpoints you can use the following methods:printensRest.getSequenceById(id='ENSG00000157764')printensRest.getSequenceByMultipleIds(ids=["ENSG00000157764","ENSG00000248378"])printensRest.getSequenceByRegion(species='human',region='X:1000000..1000100')printensRest.getSequenceByMultipleRegions(species="homo_sapiens",regions=["X:1000000..1000100:1","ABBA01004489.1:1..100"])To access theTranscript Haplotypesendpoints you can use the following methods:printensRest.getTranscripsHaplotypes(species="homo_sapiens",id="ENST00000288602")To access theVEPendpoints you can use the following methods:printensRest.getVariantConsequencesByHGVSnotation(species="human",hgvs_notation="AGT:c.803T>C")printensRest.getVariantConsequencesById(species='human',id='COSM476')printensRest.getVariantConsequencesByMultipleIds(species="human",ids=["rs116035550","COSM476"])printensRest.getVariantConsequencesByRegion(species='human',region='9:22125503-22125502:1',allele='C')printensRest.getVariantConsequencesByMultipleRegions(species="human",variants=["21 26960070 rs116645811 G A . . .","21 26965148 rs1135638 G A . . ."])To access theVariationendpoints you can use the following methods:printensRest.getVariationById(id="rs56116432",species="homo_sapiens")printensRest.getVariationByMultipleIds(ids=["rs56116432","COSM476"],species="homo_sapiens")To access theVariation GA4GHendpoints you can use the following methods:printensRest.searchGA4GHCallSet(variantSetId=1,pageSize=2)printensRest.getGA4GHCallSetById(id="1:NA19777")printensRest.searchGA4GHDataset(pageSize=3)printensRest.getGA4GHDatasetById(id="6e340c4d1e333c7a676b1710d2e3953c")printensRest.getGA4GHVariantsById(id="1:rs1333049")printensRest.searchGA4GHVariants(variantSetId=1,referenceName=22,start=17190024,end=17671934,pageToken="",pageSize=1)printensRest.searchGA4GHVariantsets(datasetId="6e340c4d1e333c7a676b1710d2e3953c",pageToken="",pageSize=2)printensRest.getGA4GHVariantsetsById(id=1)printensRest.searchGA4GHReferences(referenceSetId="GRCh38",pageSize=10)printensRest.getGA4GHReferencesById(id="9489ae7581e14efcad134f02afafe26c")printensRest.searchGA4GHReferenceSets()printensRest.getGA4GHReferenceSetsById(id="GRCh38")