Package: kazu
Kazu - Biomedical NLP FrameworkWelcome to Kazu (Korea AstraZeneca University), a python biomedical NLP framework built in collaboration with Korea University,
designed to handle production workloads.This library aims to simplify the process of using state of the art NLP research in production systems. Some of the
research contained within are our own, but most of it comes from the community, for which we are immensely grateful.If you want to use Kazu, please cite ourEMNLP 2022 publication!
(citation link)Please click here for theweb live demo(Swagger UI) from http://kazu.korea.ac.kr/Please click here for the TinyBERN2 training and evaluation codeQuickstartInstallPython version 3.9 or higher is required (tested with Python 3.11).Either:pip install kazuor download the wheel from the release page and install locally.If you intend to useMypyon your own codebase, consider installing Kazu using:pip install kazu[typed]This will pull in typing stubs for kazu's dependencies (such astypes-requestsforRequests)
so that mypy has access to as much relevant typing information as possible when type checking your codebase. Otherwise (depending on mypy config), you may see errors when running mypy like:.venv/lib/python3.10/site-packages/kazu/steps/linking/post_processing/xref_manager.py:10: error: Library stubs not installed for "requests" [import]Getting the model packFor most functionality, you will also need the Kazu model pack. This is tied to each release, and can be found on therelease page. Once downloaded,
extract the archive and:export KAZU_MODEL_PACK=<path to the extracted archive>Kazu is highly configurable (usingHydra), although it comes preconfigured with defaults appropriate for most literature processing use cases.
To make use of these, and process a simple document:importhydrafromhydra.utilsimportinstantiatefromkazu.data.dataimportDocumentfromkazu.pipelineimportPipelinefromkazu.utils.constantsimportHYDRA_VERSION_BASEfrompathlibimportPathimportos# the hydra config is kept in the model packcdir=Path(os.environ["KAZU_MODEL_PACK"]).joinpath("conf")@hydra.main(version_base=HYDRA_VERSION_BASE,config_path=str(cdir),config_name="config")defkazu_test(cfg):pipeline:Pipeline=instantiate(cfg.Pipeline)text="EGFR mutations are often implicated in lung cancer"doc=Document.create_simple_document(text)pipeline([doc])print(f"{doc.get_entities()}")if__name__=="__main__":kazu_test()DocumentationFind our docs hereLicenseLicensed underApache 2.0.Kazu includes elements under compatible licenses (full licenses are in relevant files or as indicated):Some elements are a modification of code licensed under MIT by Explosion.AI - see the READMEhere.The doc build process (conf.py's linkcode_resolve function) uses code modified from pandas, in turn modified from numpy. SeePANDAS_LICENSE.txtandNUMPY_LICENSE.txtElements of the model distillation code are inspired by or modified from Huawei Noah's Ark LabTinyBERTand DMIS-Lab'sBioBERT.
See the details in dataprocessor.py, models.py and tiny_transformer.py.PLSapbertModel is inspired by the code fromsapbert, licensed under MIT. See the file for details, and see theSapBertsection below regarding use of the model.GildaUtils in the string_normalizer.py file is modified fromGilda. See the file for full details
including the full BSD 2-Clause license.The AbbreviationFinderStep uses KazuAbbreviationDetector, which is a modified version ofSciSpacy's abbreviation finding algorithm, licensed under Apache 2.0 - see the files for full details.The JWTAuthenticationBackend Starlette Middleware in jwtauth.py is originally fromstarlette-jwt, licensed under BSD 3-Clause.The AddRequestIdMiddleware Starlette Middleware in req_id_header.py is modified from 'CustomHeaderMiddleware' in theStarlette Middleware docs.
This is licensed under BSD 3-Clause along with the rest of Starlette.The kazu-jvm folder includes files like gradelw and gradelw.bat distributed by gradle under Apache 2.0 - see the files for details.kazu/data/data.pycontainsAutoNameEnum, which isAutoNamefrom
thePython Enum Docslicensed underZero-Clause BSD.Dataset licencesFor the version of each ontology currently in use, please see the 'data_origin' field in kazu/conf/ontologiesUnderCreative Commons Attribution-Share Alike 3.0 Unported LicenceChemblChEMBL data is fromhttp://www.ebi.ac.uk/chemblCLOCLO data is fromhttp://www.ebi.ac.uk/ols/ontologies/cloUBERONUBERON data is fromhttp://www.ebi.ac.uk/ols/ontologies/uberonUnderCreative Commons Attribution 4.0 Unported LicenseMONDOMONDO data is fromhttp://www.ebi.ac.uk/ols/ontologies/mondoCELLOSAURUSCELLOSAURUS data is fromhttps://www.cellosaurus.org/Gene OntologyGene Ontology data is fromhttp://purl.obolibrary.org/obo/go.owlOther licenced datasets and modelsHPOThis service/product uses the Human Phenotype Ontology (version information). Find out more athttp://www.human-phenotype-ontology.orgFreely licenced underhttps://hpo.jax.org/app/licenseSebastian Köhler, Michael Gargano, Nicolas Matentzoglu, Leigh C Carmody, David Lewis-Smith,
Nicole A Vasilevsky, Daniel Danis, Ganna Balagura, Gareth Baynam, Amy M Brower,
Tiffany J Callahan, Christopher G Chute, Johanna L Est, Peter D Galer, Shiva Ganesan,
Matthias Griese, Matthias Haimel, Julia Pazmandi, Marc Hanauer, Nomi L Harris,
Michael J Hartnett, Maximilian Hastreiter, Fabian Hauck, Yongqun He, Tim Jeske, Hugh Kearney,
Gerhard Kindle, Christoph Klein, Katrin Knoflach, Roland Krause, David Lagorce, Julie A McMurry,
Jillian A Miller, Monica C Munoz-Torres, Rebecca L Peters, Christina K Rapp, Ana M Rath,
Shahmir A Rind, Avi Z Rosenberg, Michael M Segal, Markus G Seidel, Damian Smedley,
Tomer Talmy, Yarlalu Thomas, Samuel A Wiafe, Julie Xian, Zafer Yüksel, Ingo Helbig,
Christopher J Mungall, Melissa A Haendel, Peter N Robinson,The Human Phenotype Ontology in 2021,Nucleic Acids Research, Volume 49, Issue D1, 8 January 2021, Pages D1207–D1217,https://doi.org/10.1093/nar/gkaa1043OPEN TARGETSOpen Targets datasets are kindly provided bywww.opentargets.org, which are free for commercial use caseshttps://platform-docs.opentargets.org/licenceOchoa, D. et al. (2021). Open Targets Platform: supporting systematic drug–target identification and prioritisation. Nucleic Acids Research.https://doi.org/10.1093/nar/gkaa1027STANZAThe Stanza framework:Peng Qi, Yuhao Zhang, Yuhui Zhang, Jason Bolton and Christopher D. Manning. 2020. Stanza: A Python Natural Language Processing Toolkit for Many Human Languages. In Association for Computational Linguistics (ACL) System Demonstrations. 2020.https://arxiv.org/abs/2003.07082Biomedical NLP models are derived from:Yuhao Zhang, Yuhui Zhang, Peng Qi, Christopher D. Manning, Curtis P. Langlotz.
Biomedical and Clinical English Model Packages in the Stanza Python NLP Library,
Journal of the American Medical Informatics Association. 2021.https://doi.org/10.1093/jamia/ocab090SCISPACYBiomedical scispacy models are derived fromMark Neumann, Daniel King, Iz Beltagy, Waleed Ammar
ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing
Proceedings of the 18th BioNLP Workshop and Shared Task
ACL 2019https://www.aclweb.org/anthology/W19-5034SAPBERTKazu uses a distilled form of SAPBERT, fromFangyu Liu, Ehsan Shareghi, Zaiqiao Meng, Marco Basaldella, Nigel Collier
Self-Alignment Pretraining for Biomedical Entity Representations
ACL 2021https://aclanthology.org/2021.naacl-main.334/SETHKazu's SethStep uses Py4j to call the SETH mutation finder.Thomas, P., Rocktäschel, T., Hakenberg, J., Mayer, L., and Leser, U. (2016).SETH detects and normalizes genetic variants in textBioinformatics (2016)http://dx.doi.org/10.1093/bioinformatics/btw234OpsinKazu's OpsinStep uses Py4j to call OPSIN: Open Parser for Systematic IUPAC nomenclature.Daniel M. Lowe, Peter T. Corbett, Peter Murray-Rust, and Robert C. Glen
Chemical Name to Structure: OPSIN, an Open Source Solution
Journal of Chemical Information and Modeling 2011 51 (3), 739-753
DOI:10.1021/ci100384d