Package: pys2
PyS2: Python Library for the Semantic Scholar APIA python library for theSemantic Scholar (S2) APIwith typedpydanticobjects
and various nifty functionalities.For more information, check out thedocumentationInstall PyS2PyS2 is supported on Python 3.6+.
The recommended way to install PyS2 is via pip.pipinstallpys2To install the latest development version of PyS2 run the following instead:pipinstall--upgradehttps://github.com/mirandrom/pys2/archive/master.zipExamplesObtaining anS2PaperObjectTo scrape a paper from the S2 API, you first need an S2 paper identifier.
This can be found at the end of the URL of a paper on Semantic Scholar.For example, thispaperhas the S2 identifier8d8844106e7bc83d49ea3544ab2dfc74cd8f258aThe S2 identifier can also be specified based on a paper's identifier from
other platforms. For example, thesame paper on arxivalso has the S2 identifierarXiv:1407.5648. The convention for different platforms is described on
theS2 API page, as well as in the
PyS2 documentation forget_paper.
In fact, we will use this method to scrape the paper above with the two
different identifiers and show that they indeed give the same paper.imports2pid="8d8844106e7bc83d49ea3544ab2dfc74cd8f258a"pid2="arXiv:1407.5648"paper=s2.api.get_paper(paperId=pid)paper2=s2.api.get_paper(paperId=pid2)assertpaper==paper2Using an API KeyBe aware of the rate limit (100 requests per 5 minute window) for the
public API. Depending on the nature of your use-case (e.g. research),
you may apply for theData Partners API Accessto obtain an API key allowing you to scrape papers at a much faster rate.
If you share your code, be careful to keep the API key separate.If you have an API key, it's really easy to use in one of two ways.Using theapi_keyargumentpaper=s2.api.get_paper(paperId=pid,api_key=API_KEY)Using a customSessionfromrequestsimportSessionsession=Session()session.headers={'x-api-key':API_KEY}paper=s2.api.get_paper(paperId=pid,session=session)The same approaches can be used for the PyS2 functionget_authorcovered below.Get all the Papers of an AuthorIn this example we'll get all the papers ofBill Gateswho was an S2AuthorIdof144794037. This will also allow us to compute hish-index.ObtainS2AuthorObjectSimply pass theAuthorIdto
the PyS2 functionget_author:imports2author=s2.api.get_author(authorId="144794037")And just like that, we now have anS2Authorinstance from which we
can extract their papers, stored asS2AuthorPaperinstances. However,
this object contains limited information and so we must useget_paperto obtain theS2Paperinstances which contain
the complete information for a paper.Obtain MultipleS2PaperObjectsBecause we are performing multiple requests, we can includeretriesandwaitarguments toget_paperto work around rate-limiting. The default values of 2 and
150 are conservative but work well for the public API. Lastly, we can specify
thatS2Paperinstances returned include references or citations
(S2Reference) that are not indexed by Semantic Scholar, e.g. if we
want to attempt recovering them in a different way.paperIds=[p.paperIdforpinauthor.papers]papers=[]forpidinpaperIds:paper=s2.api.get_paper(paperId=pid,retries=2,wait=150,params=dict(include_unknown_references=True))papers+=[paper]Now we have a list of Bill Gates' papers and everything we need to compute
hish-index, namely the citations for each of his papers.Computingh-indexTheh-index is defined as the maximum value ofhsuch that an author has
publishedhpapers that have each been cited at leasthtimes.n_citations=sorted([len(p.citations)forpinpapers],reverse=True)forn_papers,n_citedinenumerate(n_citations):ifn_cited<n_papers:h_index=n_papers-1breakWhich gives us anh-index 12 for Bill Gates!Working locally withs2.dbThes2.dbAPI makes it easy to save and retrieve yourS2PaperandS2Authorobjects through a dict-like interface.froms2.db.jsonimportJsonS2PaperDB,JsonS2AuthorDB# path of directory where S2Papers will be saved as jsonss2paper_json_dir="pdb"# if the directory does not exist, it is created# otherwise, previously saved S2Papers become accessiblepdb=JsonS2PaperDB(s2paper_json_dir)# lets save Bill's papers from the previous exampleforpinpapers:pdb[p.paperId]=p# now lets delete pdb and recover Bill's papersdelpdbpdb=JsonS2PaperDB(s2paper_json_dir)forpinpapers:p2=pdb[p.paperId]assertp2==p# we can do the same for S2Author objectsadb=JsonS2AuthorDB("adb")adb[author.authorId]=author# note that setting a value requires the key to be equal to the# S2 identifier of the object, but this behaviour can be disabledadb=JsonS2AuthorDB("adb",enforce_id=False)adb["billy"]=author