Package: pent
Mini-language driven parser for structured numerical (or other) data
in free textCurrent Development Version:Most Recent Stable Release:Info:Do you have structured numerical data stored as text?Does the idea of writing regex to parse it fill you with loathing?pentcan help!Say you have data in a text file that looks like this:$vibrational_frequencies
18
    0        0.000000
    1        0.000000
    2        0.000000
    3        0.000000
    4        0.000000
    5        0.000000
    6      194.490162
    7      198.587114
    8      389.931897
    9      402.713910
   10      538.244274
   11      542.017838
   12      548.246738
   13      800.613516
   14     1203.096114
   15     1342.200360
   16     1349.543713
   17     1885.157022What’s the most efficient way to get that list of floats
extracted into anumpyarray?
There’s clearly structure here, but how to exploit it?It would work to import the text into a spreadsheet, split columns appropriately,re-export just the one column to CSV,
and import to Python from there,
but that’s just exhausting drudgery if there are dozens of files involved.Automating the parsing via a line-by-line string search would work fine
(this is howcclibimplements its data imports), but a new line-by-line
method is needed for every new kind of dataset,
and any time the formatting of a given dataset changes.It’s nottoohard towrite regexthat will parse it, but because of the mechanics of regex group captures
you have to write two patterns: one to capture the entire block, including the header
(to ensure other, similarly-formatted data isn’t also captured); and then one toiterate line-by-lineover just the data block to extract the individual values. And, of course, one has to actuallywrite(and proofread, and maintain) the regex.pentprovides a better way.The data above comes fromthis file,C2F4_01.hess. Withpent, the data can be pulled intonumpyin just a couple
of lines, without writinganyregex at all:>>>data=pathlib.Path("pent","test","C2F4_01.hess").read_text()>>>prs=pent.Parser(...head=("@.$vibrational_frequencies","#.+i"),...body=("#.+i #!..f")...)>>>arr=np.array(prs.capture_body(data),dtype=float)>>>print(arr)[[[0.][0.][0.][0.][0.][0.][194.490162][198.587114][389.931897][402.71391][538.244274][542.017838][548.246738][800.613516][1203.096114][1342.20036][1349.543713][1885.157022]]]The result comes out as a length-one list of 2-D matrices, since the search pattern
occurs only once in the data file. The single 2-D matrix is laid out as a
column vector, because the data runs down the column in the file.pentcan handle larger, more deeply nested data as well.
Takethis 18x18 matrixwithinC2F4_01.hess, for example.
Here, it’s necessary to pass aParseras thebodyof anotherParser:>>>prs_hess=pent.Parser(...head=("@.$hessian","#.+i"),...body=pent.Parser(...head="#++i",...body="#.+i #!+.f"...)...)>>>result=prs_hess.capture_body(data)>>>arr=np.column_stack([np.array(_,dtype=float)for_inresult[0]])>>>print(arr[:3,:7])[[0.468819-0.0067710.020586-0.382690.017874-0.05449-0.044552][-0.0067190.022602-0.0161830.010997-0.0333970.014422-0.01501][0.020559-0.0161840.066859-0.0336010.014417-0.0728360.045825]]The need for the generator expression, the[0]index intoresult,
and the composition vianp.column_stackarises
due to the manner in whichpentreturns data from a nested match like this.
See thedocumentation,
in particularthis example,
for more information.The grammar of thepentmini-language is designed to be flexible enough that
it should handle essentially all well-formed structured data, and even some data
that’s not especially well formed. Some datasets will require post-processing of the
data structures generated bypentbefore they can be pulled intonumpy(see, e.g.,this test,
parsingthis data block).Beta releases available onPyPI:pip install pentFull documentation is hosted atRead The Docs.Source onGitHub.  Bug reports,
feature requests, andParserconstruction help requests
are welcomed at theIssuespage there.Copyright (c) Brian Skinn 2018-2019License: The MIT License. SeeLICENSE.txtfor full license terms.