Package: scielo-clea
CleaThis project is an XML front matter metadata reader for documents
thatalmostfollows theSciELO Publishing Schema,
extracting and sanitizing the values regarding the affiliations.InstallationOne can install Clea with either:pip install scielo-clea          # Minimal
pip install scielo-clea[cli]     # Clea with CLI (recommended)
pip install scielo-clea[server]  # Clea with the testing/example server
pip install scielo-clea[all]     # Clea with both CLI and the serverActually all these commands installs everything,
only the dependencies aren't the same.
The first is an installation with minimal requirements,
intended for use within Python, as an imported package.Running the command line interfaceThe CLI is a way to use Clea as an article XML to JSONL converter
(one JSON output line for each XML input):clea -o output.jsonl article1.xml article2.xml article3.xmlThe same can be done withpython -m cleainstead ofclea.
The output is the standard output stream.
Seeclea --helpfor more information.Running the testing serverYou can run the development server using the flask CLI.
For example, for listening at 8080 from every host:FLASK_APP=clea.server flask run -h 0.0.0.0 -p 8080In a production server with 4 worker processes for handling requests,
you can, for example:Install gunicorn (it's not a dependency)Rungunicorn -b 0.0.0.0:8080 -w 4 clea.server:appClea as a libraryA simple example to see all the extracted data is:fromcleaimportArticlefrompprintimportpprintart=Article("some_file.xml")pprint(art.data_full)That's a dictionary of lists with all the "raw" extracted data.
The keys of that dictionary can be directly accessed,
so one can avoid extracting everything from the XML
by getting just the specific items/attributes
(e.g.art["journal_meta"][0].data_fullorart.journal_meta[0].data_fullinstead ofart.data_full["journal_meta"][0]).
These items/attributes are always lists, for example:art["aff"]: List ofclea.core.Branchinstancesart["sub_article"]: List ofclea.core.SubArticleinstancesart["contrib"][0]["contrib_name"]: List of stringsWhere theart["contrib"][0]is aBranchinstance,
and all such instances behave in the same way
(there's no nested branches).
That can be seen as another way to navigate in the former dictionary,
the last example should return the same list one would get withart.data_full["contrib"][0]["contrib_name"],
but without extracting everything else
that appears in theart.data_fulldictionary.More simple stuff that can be done:len(art.aff)# Number of <aff> entrieslen(art.sub_article)# Number of <sub-article>art.contrib[0].data_full# Data from the first contributor as a dict# Something like {"type": ["translation"], "lang": ["en"]},# the content from <sub-article> attributesart["sub_article"][0]["article"][0].data_full# A string with the article title, accessing just the desired contentart["article_meta"][0]["article_title"][0]AllSubArticle,ArticleandBranchinstances
have thedata_fullproperty and thegetmethod,
the latter being internally used for item/attribute getting.
Their behavior is:Branch.getalways returns a list of stringsArticle.get("sub_article")returns a list ofSubArticleArticle.get(...)returns a list ofBranchSubArticlebehaves likeArticleThe extracted information is not exhaustive!
Its result should not be seen as a replacement of the raw XML.One of the goals of this library was
to help on creating a tabular data from a given XML
with as many rows as required
to have a pair of a matching<aff>and<contrib>in each row.
These are theArticlemethods/properties that does that matching:art.aff_contrib_inner_gen()art.aff_contrib_full_gen()art.aff_contrib_innerart.aff_contrib_fullart.aff_contrib_inner_indicesart.aff_contrib_full_indicesThe most useful ones are probably the last ones,
which return a list of pairs (tuples) of indices (ints),
so one can use a(ai, ci)result
to access the(art.aff[ai], art.contrib[ci])pair,
unless the index is-1(not found).
The ones with the_gensuffix are generator functions
that yields a tuple with twoBranchentries (orNone),
the ones without a suffix return a list of merged dictionaries
in an almost tabular format (dictionary of lists of strings).
Each list regarding these elements for these specific elements
should usually have at most one string,
but that's not always the case even for these specific elements,
then one should be careful when using thedataproperty.Theinnerandfullin the names
regards toINNER JOINandFULL OUTER JOINfrom SQL,
meaning the unmatched elements
(all<aff>and<contrib>unreferred nodes)
are discarded in the former strategy,
whereas they're forcefully matched withNonein the latter.To print all the extracted data from a XML
including the indices of matching<aff>and<contrib>pairs
performed in theFULL OUTER JOINsense,
similar to the test server response:pprint({**article.data_full,"aff_contrib_pairs":article.aff_contrib_full_indices,})