Package: chemxor
ChemXorPrivacy Preserving AI/ML for Drug DiscoveryOverviewChemXor is an open source library thta provides a set of pre-tuned model architectures for evaluating FHE(Fully homomorphic encryption) encrypted inputs. These models can be trained as normal Pytorch models. It also provides convenient functions to quickly query and host these models as a service with strong privacy guarantees for the end-user. It is built on top of TenSEAL and Pytorch.What is Fully Homomorphic Encryption (FHE)?A cryptosystem that supports arbitrary computation on ciphertexts is known as fully homomorphic encryption (FHE). Such a scheme enables the construction of programs for any desirable functionality, which can be run on encrypted inputs to produce an encryption of the result. Since such a program need never decrypt its inputs, it can be run by an untrusted party without revealing its inputs and internal state. Fully homomorphic cryptosystems have great practical implications in the outsourcing of private computations. (Wikipedia)Why do you need FHE for Machine Learning?Using FHE, one can compute on encrypted data, without learning anything about the data. This enables novel privacy preserving interactions between actors in the context of machine learning.TutorialsLook at the notebook directory for tutorialsGetting StartedChemxor is available on PyPi and can be installed using pip.pipinstallchemxorModel selectionAt the moment, one can choose from 3 pre-tuned models.OlindaNetZero : Slimmest model with one convolution and 3 linear layersOlindaNet: Model with two convolutions and 4 linear layersOlindaOneNet: Model with four convolutions and 4 linear layersThese models accept a 32 x 32 input and can be configured to produce a signle or multiple outputs.fromchemxor.modelsimportOlindaNetZero,OlindaNetOne,OlindaNet# model for regressionmodel=OlindaNetZero(output=1)The model is a normal Pytorch Lightning module which is compatible with Pytorch NN module.Dataset PreparationChemXor provides two generic Pytorch Lightning Datamodules (Regression, Classification) that can be used to train and evaluate the models. These Datamodules expects raw data as CSV files with two columns (target, SMILES).fromchemxor.dataimportOlindaCDataModule,OlindaRDataModuledm_regression=OlindaRDataModule(csv_path="path/to/csv")# Use the threshold value to automatically create categorical# classes from the target column of the CSVdm_classification=OlindaCDataModule(csv_path="path/to/csv",threshold=[0.5])The DataModules will take care of converting the smiles input to 32 x 32 images.Model TrainingIt is recommended to use a Pytorch Lightning trainer to train the models. Although a normal Pytorch training loop can also be used.importpytorch_lightningaspl# Save the best 3 checkpoints based on validation losscheckpoint_callback=pl.callbacks.ModelCheckpoint(dirpath="path/to/save/checkpoints",save_top_k=3,monitor="VAL_Loss",)trainer=pl.Trainer(callbacks=[checkpoint_callback],accelerator="auto")trainer.fit(model=model,datamodule=data_module)FHE modelsAfter training, the models can be wrapped using their specific FHE wrappers to process FHE inputs. FHE wrappers will take care of Tenseal context parameters and keys management.fromchemxor.modelsimportOlindaNetZero,OlindaNetOne,OlindaNetfromchemxor.modelsimportFHEOlindaNetZero,FHEOlindaNetOne,FHEOlindaNetmodel=OlindaNetZero(output=1)model.load("path/to/checkpoint")fhe_model=FHEOlindaNetZero(model=model)FHE inputs evaluationThe Datamodules can generate Pytorch dataloaders that produce encrypted inputs for the model.fromchemxor.dataimportOlindaCDataModule,OlindaRDataModuledm_regression=OlindaRDataModule(csv_path="path/to/csv")dm_regression.setup("test")enc_data_loader=dm_classification.enc_dataloader(context=fhe_model.enc_context)enc_sample=next(iter(enc_data_loader))Also, the FHE models are partitioned to control multiplicative depth. So, the forward function is modified to accept a step parameter. For testing, The FHE model can be evaluated locally as follows:fromchemxor.utilsimportprepare_fhe_inputoutput=enc_sampleforstepinrange(fhe_model.steps):output=fhe_model(output,step)dec_out=output.decrypt()output=prepare_fhe_input(dec_out,fhe_model.pre_process[step],fhe_model.enc_context)# final decryted outputdecrypted_output=output.decrypt()This process can automated using a utility function provided by ChemXorfromchemxor.utilsimportevaluate_fhe_modeldecrypted_output=evaluate_fhe_model(fhe_model,enc_sample,decrypt=True)Serve modelsFHE Models can be served in the form of a Flask app as follows:fromchemxor.serviceimportPartitionNetServerfhe_model_server=PartitionNetServer(fhe_model)if__name__=="__main__":fhe_model_server.run()ChemXor's Pre defined Models can also be served using the CLIchemxorserveolida|olinda_zero|olinda_oneQuery modelschemxorquery[modelurl][moleculeSMILES]DevelopingWe use poetry to manage project dependecies. Use poetry to install project in editable mode.poetryinstallLicenseThis project is licensed under GNU AFFERO GENERAL PUBLIC LICENSE Version 3.
The direct and indirect dependecies of the project are licensed as follows:NameVersionLicenseAuthorBabel2.10.1BSD LicenseArmin RonacherCacheControl0.12.11Apache Software LicenseEric LarsonFaker13.12.0MIT Licensejoke2kFlask2.1.2BSD LicenseArmin RonacherGitPython3.1.27BSD LicenseSebastian Thiel, Michael TrierJinja23.1.2BSD LicenseArmin RonacherMarkdown3.3.7BSD LicenseManfred Stienstra, Yuri takhteyev and Waylan limbergMarkupSafe2.1.1BSD LicenseArmin RonacherPillow9.1.1Historical Permission Notice and Disclaimer (HPND)Alex Clark (PIL Fork Author)PyJWT2.3.0MIT LicenseJose PadillaPyNaCl1.4.0Apache License 2.0The PyNaCl developersPyYAML5.4.1MIT LicenseKirill SimonovPygments2.12.0BSD LicenseGeorg BrandlSQLAlchemy1.4.36MIT LicenseMike BayerSecretStorage3.3.2BSD LicenseDmitry ShachnevSend2Trash1.8.0BSD LicenseAndrew SenetarWerkzeug2.1.2BSD LicenseArmin Ronacherabsl-py1.0.0Apache Software LicenseThe Abseil Authorsaiofiles0.6.0Apache Software LicenseTin Tvrtkovicaiohttp3.8.1Apache Software LicenseUNKNOWNaiosignal1.2.0Apache Software LicenseNikolay Kimanyconfig0.10.1MIT LicenseSatoru SATOHanyio3.6.1MIT LicenseAlex Grönholmargcomplete1.12.3Apache Software LicenseAndrey Kislyukargon2-cffi21.3.0MIT LicenseUNKNOWNargon2-cffi-bindings21.2.0MIT LicenseHynek Schlawackarrow1.2.2Apache Software LicenseChris Smithascii-magic1.6MIT LicenseLeandro Baronease3.22.1GNU Lesser General Public License v2 or later (LGPLv2+)UNKNOWNasgiref3.5.2BSD LicenseDjango Software Foundationasync-timeout4.0.2Apache Software LicenseAndrew Svetlovandrew.svetlov@gmail.comattrs21.4.0MIT LicenseHynek Schlawackautodp0.2Apache Software LicenseYu-Xiang Wangbackcall0.2.0BSD LicenseThomas Kluyverbackports.cached-property1.0.1MIT LicenseAleksei Stepanovbandit1.7.4Apache Software LicensePyCQAbcrypt3.2.0Apache Software LicenseThe Python Cryptographic Authority developersbeautifulsoup44.11.1MIT LicenseLeonard Richardsonbinaryornot0.4.4BSD LicenseAudrey Roy Greenfeldblack22.3.0MIT LicenseŁukasz Langableach5.0.0Apache Software LicenseUNKNOWNcachetools4.2.4MIT LicenseThomas Kemmercachy0.3.0MIT LicenseSébastien Eustacecertifi2022.5.18.1Mozilla Public License 2.0 (MPL 2.0)Kenneth Reitzcffi1.15.0MIT LicenseArmin Rigo, Maciej Fijalkowskicfgv3.3.1MIT LicenseAnthony Sottilechardet4.0.0GNU Library or Lesser General Public License (LGPL)Mark Pilgrimcharset-normalizer2.0.12MIT LicenseAhmed TAHRI @Ousretchemxor0.1.0GPLv3Ersilia Open Source Initiativecleo0.8.1MIT LicenseSébastien Eustaceclick8.1.3BSD LicenseArmin Ronacherclikit0.6.2MIT LicenseSébastien Eustacecolorlog6.6.0MIT LicenseSam Clementscookiecutter1.7.3BSD LicenseAudrey Roycoverage6.4Apache Software LicenseNed Batchelder and 152 otherscrashtest0.3.1MIT LicenseSébastien Eustacecryptography37.0.2Apache Software License; BSD LicenseThe Python Cryptographic Authority and individual contributorscycler0.11.0BSD LicenseThomas A Caswelldarglint1.8.1MIT Licenseterrencepreillydebugpy1.6.0Eclipse Public License 2.0 (EPL-2.0); MIT LicenseMicrosoft Corporationdecorator5.1.1BSD LicenseMichele Simionatodefusedxml0.7.1Python Software Foundation LicenseChristian Heimesdistlib0.3.4Python Software Foundation LicenseVinay Sajipdnspython2.2.1ISCBob Halleydparse0.5.1MIT LicenseJannis Gebauerdynaconf3.1.8MIT LicenseBruno Rochaemail-validator1.2.1CC0 1.0 Universal (CC0 1.0) Public Domain DedicationJoshua Taubererentrypoints0.4MIT LicenseThomas Kluyvereva1.0.1UNKNOWNMicrosoft Research EVA compiler teamfactory-boy3.2.1MIT LicenseMark Sandstromfastapi0.66.1MIT LicenseSebastián Ramírezfastjsonschema2.15.3BSD LicenseMichal Horejsekfilelock3.7.0Public DomainBenedikt Schmittflake84.0.1MIT LicenseTarek Ziadeflake8-annotations2.9.0MIT LicenseS Co1flake8-bandit3.0.0MIT LicenseTyler Winceflake8-black0.3.3MIT LicensePeter J. A. Cockflake8-bugbear22.4.25MIT LicenseŁukasz Langaflake8-docstrings1.6.0MIT LicenseSimon ANDRÉflake8-import-order0.18.1GNU Lesser General Public License v3 (LGPLv3); MIT LicenseAlex Stapletonflake8-polyfill1.0.2MIT LicenseIan Cordascofonttools4.33.3MIT LicenseJust van Rossumforbiddenfruit0.1.4GNU General Public License v3 or later (GPLv3+); MIT LicenseLincoln de Sousafrozenlist1.3.0Apache Software LicenseUNKNOWNfsspec2022.1.0BSD LicenseUNKNOWNgensim4.2.0LGPL-2.1-onlyRadim Rehurekgevent21.8.0MIT LicenseDenis Bilenkogitdb4.0.9BSD LicenseSebastian Thielgoogle-auth2.6.6Apache Software LicenseGoogle Cloud Platformgoogle-auth-oauthlib0.4.6Apache Software LicenseGoogle Cloud Platformgraphql-core3.2.1MIT LicenseChristoph Zwerschkegreenlet1.1.2MIT LicenseAlexey Borzenkovgriddify0.0.1MIT LicenseMiquel Duran-Frigolagrpcio1.46.3Apache Software LicenseThe gRPC Authorsh110.13.0MIT LicenseNathaniel J. Smithhtml5lib1.1MIT LicenseUNKNOWNhttptools0.4.0MIT LicenseYury Selivanovidentify2.5.1MIT LicenseChris Kuehlidna3.3BSD LicenseKim Daviesimportlib-metadata4.11.4Apache Software LicenseJason R. Coombsimportlib-resources5.7.1Apache Software LicenseBarry Warsawiniconfig1.1.1MIT LicenseRonny Pfannschmidt, Holger Krekelipykernel6.13.0BSD LicenseIPython Development Teamipython7.34.0BSD LicenseThe IPython Development Teamipython-genutils0.2.0BSD LicenseIPython Development Teamipywidgets7.7.0BSD LicenseIPython Development Teamitsdangerous2.1.2BSD LicenseArmin Ronacherjedi0.18.1MIT LicenseDavid Halterjeepney0.8.0MIT LicenseThomas Kluyverjinja2-time0.2.0MIT LicenseRaphael Pierzinajmespath0.10.0MIT LicenseJames Saryerwinniejoblib1.1.0BSD LicenseGael Varoquauxjson50.9.8Apache Software LicenseDirk Prankejsonschema4.5.1MIT LicenseJulian Bermanjupyter-client7.3.1BSD LicenseJupyter Development Teamjupyter-core4.10.0BSD LicenseJupyter Development Teamjupyter-server1.17.0BSD LicenseJupyter Development Teamjupyterlab3.4.2BSD LicenseJupyter Development Teamjupyterlab-pygments0.2.2BSDJupyter Development Teamjupyterlab-server2.14.0BSD LicenseUNKNOWNjupyterlab-widgets1.1.0BSD LicenseJupyter Development Teamkedro0.18.1Apache Software License (Apache 2.0)Kedrokedro-viz4.6.0Apache Software License (Apache 2.0)Kedrokeyring23.5.0MIT License; Python Software Foundation LicenseKang Zhangkiwisolver1.4.2BSD LicenseUNKNOWNlap0.4.0BSD (2-clause)UNKNOWNlark-parser0.12.0MIT LicenseErez Shinanllvmlite0.38.1BSDUNKNOWNlockfile0.12.2MIT LicenseOpenStackloguru0.5.3MIT LicenseDelganlogzero1.7.0MIT LicenseChris Hagermatplotlib3.5.2Python Software Foundation LicenseJohn D. Hunter, Michael Droettboommatplotlib-inline0.1.3BSD 3-ClauseIPython Development Teammccabe0.6.1MIT LicenseIan Cordascomistune0.8.4BSD LicenseHsiaoming Yangmpmath1.2.1BSD LicenseFredrik Johanssonmsgpack1.0.3Apache Software LicenseInada Naokimultidict6.0.2Apache Software LicenseAndrew Svetlovmypy0.942MIT LicenseJukka Lehtosalomypy-extensions0.4.3MIT LicenseThe mypy developersnames0.3.0MIT LicenseTrey Hunnernbclassic0.3.7BSD LicenseJupyter Development Teamnbclient0.6.3BSD LicenseJupyter Development Teamnbconvert6.5.0BSD LicenseJupyter Development Teamnbformat5.4.0BSD LicenseJupyter Development Teamnest-asyncio1.5.5BSD LicenseEwald R. de Witnetworkx2.8.2BSD LicenseAric Hagbergnglview3.0.3MIT LicenseAlexander S. Rose, Hai Nguyennltk3.7Apache Software LicenseNLTK Teamnodeenv1.6.0BSD LicenseEugene Kalininnotebook6.4.11BSD LicenseJupyter Development Teamnotebook-shim0.1.0BSD LicenseJupyter Development Teamnox2022.1.7Apache Software LicenseAlethea Katherine Flowersnumba0.55.2BSD LicenseUNKNOWNnumpy1.22.4BSD LicenseTravis E. Oliphant et al.oauthlib3.2.0BSD LicenseThe OAuthlib Communityonnx1.11.0Apache License v2.0ONNXpackaging21.3Apache Software License; BSD LicenseDonald Stufft and individual contributorspandas1.3.4BSD LicenseThe Pandas Development Teampandocfilters1.5.0BSD LicenseJohn MacFarlaneparso0.8.3MIT LicenseDavid Halterpastel0.2.1MIT LicenseSébastien Eustacepathspec0.9.0Mozilla Public License 2.0 (MPL 2.0)Caleb P. Burnspbr5.9.0Apache Software LicenseOpenStackpep5170.12.0MIT LicenseThomas Kluyverpexpect4.8.0ISC License (ISCL)Noah Spurrier; Thomas Kluyver; Jeff Quastpickleshare0.7.5MIT LicenseVille Vainiopip-tools6.6.2BSD LicenseVincent Driessenpkginfo1.8.2MIT LicenseTres Seaver, Agendaless Consultingplatformdirs2.5.2MIT LicenseUNKNOWNplotly5.8.0MITChris Ppluggy1.0.0MIT LicenseHolger Krekelpoetry1.1.13MIT LicenseSébastien Eustacepoetry-core1.0.8MIT LicenseSébastien Eustacepoyo0.5.0MIT LicenseRaphael Pierzinapre-commit2.19.0MIT LicenseAnthony Sottileprometheus-client0.14.1Apache Software LicenseBrian Brazilprompt-toolkit3.0.29BSD LicenseJonathan Slendersprotobuf3.19.13-Clause BSD LicenseUNKNOWNpsutil5.9.1BSD LicenseGiampaolo Rodolaptyprocess0.7.0ISC License (ISCL)Thomas Kluyverpy1.11.0MIT Licenseholger krekel, Ronny Pfannschmidt, Benjamin Peterson and otherspyDeprecate0.3.2MITJiri Borovecpyarrow6.0.0Apache Software LicenseUNKNOWNpyasn10.4.8BSD LicenseIlya Etingofpyasn1-modules0.2.8BSD LicenseIlya Etingofpycodestyle2.8.0MIT LicenseJohann C. Rochollpycparser2.21BSD LicenseEli Benderskypydantic1.9.1MIT LicenseSamuel Colvinpydocstyle6.1.1MIT LicenseAmir Rachumpyflakes2.4.0MIT LicenseA lot of peoplepylev1.4.0BSD LicenseDaniel Lindsleypymbolic2021.1MIT LicenseAndreas Kloecknerpynndescent0.5.7BSDLeland McInnespyparsing3.0.9MIT LicenseUNKNOWNpyrsistent0.18.1MIT LicenseTobias Gustafssonpytest7.1.2MIT LicenseHolger Krekel, Bruno Oliveira, Ronny Pfannschmidt, Floris Bruynooghe, Brianna Laugher, Florian Bruhin and otherspytest-cov3.0.0MIT LicenseMarc Schlaichpytest-mock3.7.0MIT LicenseBruno Oliveirapython-dateutil2.8.2Apache Software License; BSD LicenseGustavo Niemeyerpython-dotenv0.20.0BSD LicenseSaurabh Kumarpython-json-logger2.0.2BSD LicenseZakaria Zajacpython-multipart0.0.5Apache Software LicenseAndrew Dunhampython-slugify6.1.2MIT LicenseVal Neekmanpytools2022.1.7MIT LicenseAndreas Kloecknerpytorch-lightning1.6.3Apache Software LicenseWilliam Falcon et al.pytz2022.1MIT LicenseStuart Bishoppyzmq23.0.0BSD License; GNU Library or Lesser General Public License (LGPL)Brian E. Granger, Min Ragan-Kelleyrdkit-pypi2022.3.2.1BSD-3-ClauseChristopher Kuennethregex2022.4.24Apache Software LicenseMatthew Barnettrequests2.27.1Apache Software LicenseKenneth Reitzrequests-oauthlib1.3.1BSD LicenseKenneth Reitzrequests-toolbelt0.9.1Apache Software LicenseIan Cordasco, Cory Benfieldrope0.21.1GNU Lesser General Public License v3 or later (LGPLv3+)Ali Gholami Rudirsa4.8Apache Software LicenseSybren A. Stüvelsafety1.10.3MIT Licensepyup.ioscikit-learn1.1.1new BSDUNKNOWNscipy1.7.3BSD LicenseUNKNOWNsemver2.13.0BSD LicenseKostiantyn Rybnikovsetuptools-scm6.4.2MIT LicenseRonny Pfannschmidtshellingham1.4.0ISC License (ISCL)Tzu-ping Chungsix1.16.0MIT LicenseBenjamin Petersonsmart-open6.0.0MIT LicenseRadim Rehureksmmap5.0.0BSD LicenseSebastian Thielsniffio1.2.0Apache Software License; MIT LicenseNathaniel J. Smithsnowballstemmer2.2.0BSD LicenseSnowball Developerssoupsieve2.3.2.post1MIT LicenseUNKNOWNstarlette0.14.2BSD LicenseTom Christiestevedore3.5.0Apache Software LicenseOpenStackstrawberry-graphql0.114.0MIT LicensePatrick Arminiosyft0.6.0Apache-2.0OpenMinedsympy1.9BSD LicenseSymPy development teamtenacity8.0.1Apache Software LicenseJulien Danjoutenseal0.3.11Apache-2.0OpenMinedtensorboard2.9.0Apache Software LicenseGoogle Inc.tensorboard-data-server0.6.1Apache Software LicenseGoogle Inc.tensorboard-plugin-wit1.8.1Apache 2.0Google Inc.terminado0.15.0BSD LicenseUNKNOWNtext-unidecode1.3Artistic License; GNU General Public License (GPL); GNU General Public License v2 or later (GPLv2+)Mikhail Korobovthreadpoolctl3.1.0BSD LicenseThomas Moreautinycss21.1.1BSD LicenseUNKNOWNtoml0.10.2MIT LicenseWilliam Pearsontomli2.0.1MIT LicenseUNKNOWNtomlkit0.10.2MIT LicenseSébastien Eustacetoposort1.7Apache Software License"Eric V. Smith"torch1.11.0BSD LicensePyTorch Teamtorch-tb-profiler0.4.0BSD LicensePyTorch Teamtorchani2.2MITXiang Gaotorchmetrics0.8.2Apache Software LicensePyTorchLightning et al.torchvision0.12.0BSDPyTorch Core Teamtornado6.1Apache Software LicenseFacebooktqdm4.64.0MIT License; Mozilla Public License 2.0 (MPL 2.0)UNKNOWNtraitlets5.2.1.post0BSD LicenseUNKNOWNtypeguard2.13.3MIT LicenseAlex Grönholmtyping-extensions4.2.0Python Software Foundation LicenseUNKNOWNumap-learn0.5.3BSDUNKNOWNurllib31.26.9MIT LicenseAndrey Petrovuvicorn0.17.6BSD LicenseTom Christieuvloop0.16.0Apache Software License; MIT LicenseYury Selivanovvirtualenv20.14.1MIT LicenseBernat Gaborwatchgod0.8.2MIT LicenseSamuel Colvinwcwidth0.2.5MIT LicenseJeff Quastwebencodings0.5.1BSD LicenseGeoffrey Sneddonwebsocket-client1.3.2Apache Software Licenseliriswebsockets10.3BSD LicenseAymeric Augustinwidgetsnbextension3.6.0BSD LicenseIPython Development Teamyarl1.7.2Apache Software LicenseAndrew Svetlovzipp3.8.0MIT LicenseJason R. Coombszope.event4.5.0Zope Public LicenseZope Foundation and Contributorszope.interface5.4.0Zope Public LicenseZope Foundation and Contributors