Package: pubchem-api-crawler
PubChem API CrawlerThis package provides a python client for crawling chemical compounds and their properties onPubChem.InstallationYou can install the PubChem API Crawler directly with pip :pip install pubchem-api-crawlerOr you can clone the project from github and install it locally usingpoetrywithpoetry installNotebooksExample notebooks showing how to use the library are available in thenotebooksdirectory. To run the notebooks, runpoetry run jupyter laband select the notebook in the browser window.Molecular Formula SearchThe main entry point for PubChem API Crawler is theMolecular Formula Search function of Pubchemwhich lets you retrieve compounds given a molecular formula search input.For example, if you wanted to find all compounds on PubChem containing carbon, hydrogen, aluminium and bore, you would use :frompubchem_api_crawlerimportMolecularFormulaSearchdf=MolecularFormulaSearch().search(["C1-","H1-","B1-","Al1-"],allow_other_elements=False,properties=["MolecularFormula","CanonicalSMILES"])CIDMolecularFormulaCanonicalSMILES0168084494CH5AlB2[BH].[BH].C[Al]1163556649C16H14AlB[B]CCC1=C2CCC=CC2=C(C3=CC=CC=C31)[Al]2161576177C27H30AlB[H+].[B-](C1=CC=CC=C1)(C2=CC=CC=C2)(C3=CC=CC=C3)C4=CC=CC=C4.C[Al](C)C3160352291C6H15AlB[B].CC[Al](CC)CC4159123289C10H28AlB2[B](C)C.[B](C)C.CCCC.C[Al]C5158802573C11H29AlBB(C)(C)C.CCCC.CC[Al]CC6158250967C3H9AlB[B].C[Al](C)C7158044531C2H6AlB[B].C[Al]C8157093180C3H9AlBB(C)(C)C.[Al]9156888304C12H14AlB[B]C1=CC=CC=C1C2CCCCC2[Al]10129859217C2H6AlB[B].C[Al]C11129657578C2H6AlB[B-].C[Al+]C12129657197CH3AlB2[B-].[B-].C[Al+2]1359992955C7H9AlB[BH2].C1=CC=C(C=C1)C[Al]1422996618C12H30AlBB(CC)(CC)CC.CC[Al](CC)CC1519734271C8H18AlB[B-].CC(C)C[Al+]CC(C)C16155575130C8H8AlB[B]C1=CC(=C(C=C1C)[Al])CMolecular Formula Search InputThe valid inputs for Molecular Formula Search are describedhere.The general MF query syntax consists of a series of valid atomic symbols
(please consult your periodical chart), each optionally followed by either
a number or a range.
The generic range syntax is "[atomic symbol][low count]-[high count]",
repeated for every specified element. Elements may be written in
arbitrary order.

Examples:
1. C7-8:	represents compounds with seven or eight carbons.
2. C-7:	represents compounds with up to seven carbons.
3. C7-:	represents compounds with seven or more carbons
4. C or C1:	represents compounds with exactly one carbon
5. C-:	represents any number of carbons, including none.The search input must be provided as a list of[atomic symbol][low count]-[high count]strings to the search method.Note: specifying an open ended high count (i.e. C2-) does not seem to work correctly on PubChem. It is recommended to always specify a high count (i.e. C2-500).Molecular Formula Search OptionsBy default, the molecular formula search will return the cids of the matching compounds. Optionally, a list of properties can also be requested. The list of valid compound properties which can be requested is availablehere.Aditionnaly, theallow_other_elementsoption lets you choose to allow other elements to be present in addition to those specified.Request timeouts on the REST APIREST requests made to PubChem time out after 30s. Therefore, searches that are too broad will timeout on the server and raise an error. To overcome this limitation, it is possible to use PubChem's Async REST API. If your search request times out, you should retry it via the Async REST API with the_async=Trueparameter :df=MolecularFormulaSearch().search(["C1-","H1-"],allow_other_elements=False,properties=["MolecularFormula","CanonicalSMILES"],_async=True)Experimental Properties AnnotationsWhen using PubChem's REST API, you can only retrieve computed compound properties (list is availablehere).If you want to retrieve experimental properties annotations, you can use theAnnotationsclass of PubChem API Crawler. The list of annotation headings (and their types) for which PubChem has any data is availablehere.PubChem API Crawler offers two ways to get annotations. You can get annotations for specific compounds individually by giving their cids. But there are no batch methods to fetch annotations on PubChem, so this requires sending a REST request per compound, which can be quite slow if you want to get properties for a lot of compounds. The alternative is to get all the data that PubChem has for a given annotation heading.Getting annotations for a specific compoundTheget_compound_annotationsmethod will get a specific annotation heading for the given cids (if heading is unspecified, it will get theExperimental Propertiessection).frompubchem_api_crawlerimportAnnotationsAnnotations().get_compound_annotations(356,heading='Heat of Combustion')Heat of CombustionValueHeat of CombustionReferenceCID01,302.7 kg cal/g mol wt at 760 mm Hg and 20 °CWeast, R.C. (ed.) Handbook of Chemistry and Physics. 69th ed. Boca Raton, FL: CRC Press Inc., 1988-1989., p. D-278356Getting all annotations for a specific headingTheget_annotationsmethod will get all available data on PubChem for a given heading.frompubchem_api_crawlerimportAnnotationsAnnotations().get_annotations("Autoignition Temperature")SourceNameSourceIDURLValueReferenceCID0Hazardous Substances Data Bank (HSDB)30https://...270 °C (518 °F)Fire Protection Guide to Hazardous Materials. ...45101Hazardous Substances Data Bank (HSDB)35https://...928 °F (498 °C)National Fire Protection Association;  Fire Protection Guide ...2412Hazardous Substances Data Bank (HSDB)37https://...871 °F (466 °C)National Fire Protection Association;  Fire Protection Guide ...25373Hazardous Substances Data Bank (HSDB)39https://...772 °F (411 °C)Fire Protection Guide to Hazardous Materials. ...78354Hazardous Substances Data Bank (HSDB)40https://...867 °F (463 °C)National Fire Protection Association;  Fire Protection Guide ...176Rate limitsYou should first check therate limitsthat PubChem imposes on requests to its API. On top of those dynamic request throttling policies, you shouldnot send more than 5 requests per second to the PubChem REST API.By default, PubChem API Crawler sets a rate limit of 5 calls per 3 seconds on REST API calls. These settings can be modified either by setting environment variablesRATE_LIMIT_CALLS(integer) andRATE_LIMIT_PERIOD(integer, in seconds) or by creating a.envfile in your working directory where those variables are set.If you enable logging for the PubChem API Crawler namespace with log level set toDEBUG, the library will report request throttling status in the logs after each request.LogsEnable logging before calling the library's functions to see debugging and info messages.importlogginglogger=logging.getLogger('pubchem_api_crawler')logger.setLevel(logging.DEBUG)ch=logging.StreamHandler()ch.setFormatter(logging.Formatter('%(asctime)s-%(name)s-%(levelname)s-%(message)s'))logger.addHandler(ch)