Package: grobid-tei-xml
grobid_tei_xml: Python parser and transforms for GROBID-flavor TEI-XMLThis is a simple python library for parsing the TEI-XML structured documents
returned byGROBID, a machine learning
tool for extracting text and bibliographic metadata from research article PDFs.TEI-XML is a standard format, and there exist other libraries to parse entire
documents and work with annotated text. This library is focused specifically on
extracting "header" metadata from document (eg, title, authors, journal name,
volume, issue), content in flattened text form (full abstract and body text as
single strings, for things like search indexing), and structured citation
metadata.Quickstartgrobid_tei_xmlworks with Python 3, using only the standard library. It does
not talk to the GROBID HTTP API or read files off disk on it's own, but see
examples below. The library is packaged onpypi.org.Install usingpip, usually within avirtualenv:pip install grobid_tei_xmlThe main entry points are the functionsprocess_document_xml(xml_text)andprocess_citation_xml(xml_text)(orprocess_citation_list_xml(xml_text)for
multiple citations), which return python dataclass objects. The helper method.to_dict()can be useful for, eg, serializing these objects to JSON.Usage ExamplesRead an XML file from disk, parse it, and print to stdout as JSON:importjsonimportgrobid_tei_xmlxml_path="./tests/files/small.xml"withopen(xml_path,'r')asxml_file:doc=grobid_tei_xml.parse_document_xml(xml_file.read())print(json.dumps(doc.to_dict(),indent=2))Userequeststo download a PDF from the web, submit to GROBID (via HTTP API),
parse the TEI-XML response withgrobid_tei_xml, and print some metadata
fields:importrequestsimportgrobid_tei_xmlpdf_resp=requests.get("https://arxiv.org/pdf/1802.01168v3")pdf_resp.raise_for_status()grobid_resp=requests.post("https://cloud.science-miner.com/grobid/api/processFulltextDocument",files={'input':pdf_resp.content,'consolidate_Citations':0,'includeRawCitations':1,},timeout=60.0,)grobid_resp.raise_for_status()doc=grobid_tei_xml.parse_document_xml(grobid_resp.text)print("title: "+doc.header.title)print("authors: "+", ".join([a.full_nameforaindoc.header.authors]))print("doi: "+str(doc.header.doi))print("citation count: "+str(len(doc.citations)))print("abstract: "+doc.abstract)Userequeststo submit a "raw" citation string to GROBID for extraction,
parse the response withgrobid_tei_xml, and print the structured output to
stdout:importrequestsimportgrobid_tei_xmlraw_citation="Kvenvolden K.A. and Field M.E. 1981. Thermogenic hydrocarbons in unconsolidated sediment of Eel River Basin, offshore northern California. AAPG Bulletin 65:1642-1646"grobid_resp=requests.post("https://cloud.science-miner.com/grobid/api/processCitation",data={'citations':raw_citation,'consolidateCitations':0,'includeRawCitations':1,},timeout=10.0,)grobid_resp.raise_for_status()citation=grobid_tei_xml.parse_citation_xml(grobid_resp.text)print(citation)See Alsogrobid_client_python:
Python client and CLI tool for making requests to GROBID via HTTP API. Returns
TEI-XML; could be used with this library (grobid_tei_xml) for parsing into
python object or, eg, JSON.GROBID Documentations2orc-doc2json: Python library
from AI2 which includes a similar Python library for extracting both
bibliographic metadata and (structured) full text from GROBID TEI-XML. Has nice
features like resolving references to bibliography entry.delb: more flexible/powerful interface
to TEI-XML documents. would be a better tool for working with structured text
(body, abstract, etc)"Parsing TEI XML documents with
Python"(2019): blog post about basic parsing of GROBID TEI-XML files into Pandas
DataFramesLicenseThis library is available under the permissive MIT License. SeeLICENSE.txtfor a copy.