Package: pubmed-types
pubmed-types (v0.2.0)IntroductionA complete implementation of the XML schema for PMC Open Access articles and Pubmed
article sets (citations).This package helps to parse PubMed XML data into Pydantic models. This validates the
input xml data and provides typehints for working with the complex XML structures
present in PubMed data.Most Recent ChangesBreaking Change:Theparse_pubmed_xmlis replaced bypmc_articleandpubmed_article_set.More test coveragePubmed Articles can now parse MathMLRestructured code to separate outjats(pmc open access articles) andpubmed(pubmed article set)One unit test with 99% coverageAddedCHANGELOG.mdWhy do I need this?PubMed keeps track of 10s of millions of research data, and a complex XML structure is
used to store it. Parsing XML on its own is challenging enough. Add to it the feature
rich data inside of each citation, and you will find yourself with hours or days of
navigating the XML structure.The approach here was to autogenerate Pydantic classes to parse the XML using thexsdata-pydantictool. This approach has the benefit of making sure every piece of data
is parsed properly, and an error is thrown is something is missing or incorrect. Instead
of using dictionaries to hold the data, Pydantic classes have the benefit of providing
type hints with tab completion for IDEs, making it easier to navigate the complex
structure of the citation data.How do I use it?It is possible to usexsdata-pydanticand the autogenerated classes directly to parse
an XML file, but we provide a convenience function to easily open PubMed XMl citations
and PMC open access articles.Example 1: A PMC Open Access Articleimporttarfileimporturllib.requestasrequestfromcontextlibimportclosingfrompathlibimportPathfrompubmed_typesimportpmc_article# Input file source and output file destinationsource=("ftp://ftp.ncbi.nlm.nih.gov"+"/pub/pmc/oa_bulk/oa_comm/xml"+"/oa_comm_xml.incr.2023-03-21.tar.gz")destination=Path("downloads")destination.mkdir(exist_ok=True)# 1. Get an open access article dataset from the FTP serverwithclosing(request.urlopen(source))asurl:withtarfile.open(fileobj=url,mode="r:gz")asfr:fr.extractall(destination)# 2. Parse the filefile_path=destination.joinpath("PMC009xxxxxx").joinpath("PMC9970662.xml")full_text=pmc_article(file_path)# 3. Print out the article titleprint(f"Title:{full_text.front.article_meta.title_group.article_title.content[0]}")Output:Title:Lactateasamyokineandexerkine:driversandsignalsofphysiologyandmetabolismExample 2: A Pubmed baseline citation fileimportgzipimporturllib.requestasrequestfromcontextlibimportclosingfrompathlibimportPathfrompubmed_typesimportpubmed_article_set# Input file source and output file destinationsource="ftp://ftp.ncbi.nlm.nih.gov"+"/pubmed/updatefiles"+"/pubmed23n1168.xml.gz"destination=Path("downloads").joinpath("pubmed23n1168.xml")destination.parent.mkdir(exist_ok=True)# 1. Get a pubmed citation daily update file from the FTP serverwithclosing(request.urlopen(source))asurl:withgzip.GzipFile(fileobj=url,mode="rb")asfr:withopen(destination,mode="wb")asfw:fw.write(fr.read())# 2. Parse the filearticle_set=pubmed_article_set(destination)# 3. Get the number of citations in the fileprint(f"Number of citations:{len(article_set.pubmed_article)}")print(f"{article_set.pubmed_article[0].medline_citation.article.article_title.content[0]}")Output:Numberofcitations:2543APatentandPatternMother.FAQWhy does it take so long to parse a pubmed citation setThere is a lot of data, and the schema is deep and complex.Why are the return structures so complicated?The return structures are a direct reflection of the XML format defined by the NLM. In
the future some utility classes might be made for common components (title, authors,
etc), but for now this is intended to be an unbiased way of parsing the XML.