Package: arxivloader
arXiv LoaderThis tool is a wrapper of thearXiv APIthat allows you to retrieve metadata of articles published on arXiv aspandas.DataFrame.Please abide by theTerms of Usageof the arXiv API.Installationpip install arxivloaderUsagePlease consult thearXiv API documentationfor help in constructing a valid query string.Searching by keywordTo search for a keyword the query needs to start withsearch_query=followed by a prefix and the search word.Possible prefixes arePrefixExplanationtiTitleauAuthorabsAbstractcoCommentsjrJournal ReferencecatSubject CategoryrnReport NumberidarXiv IDallAll of the abovePlease have a look at thearXiv API documentationfor details.import arxivloader

keyword = "DustPy"
prefix = "all"
query = "search_query={pf}:{kw}".format(pf=prefix, kw=keyword)
columns = ["id", "title", "authors"]

df = arxivloader.load(query, columns=columns)
print(df)idtitleauthors02207.00322v2DustPy: A Python Package for Dust Evolution in Protoplanetary DisksSebastian Markus Stammler; Tilman Birnstiel12110.04007v1The formation of wide exoKuiper belts from migrating dust trapsE. Miller; S. Marino; S. M. Stammler; P. Pinilla; C. Lenz; T. Birnstiel; Th. HenningSearching by idTo search for a specific arXiv ID the query needs to start withid_list=followed by a comma-separated list of arXiv IDs:import arxivloader

IDs = ["1909.04674", "1909.10526"]
query = "id_list={}".format(",".join(IDs))
columns = ["id", "title", "authors"]

df = arxivloader.load(query, columns=columns)

print(df)idtitleauthors01909.04674v1The DSHARP Rings: Evidence of Ongoing Planetesimal Formation?Sebastian M. Stammler; Joanna Drazkowska; Til Birnstiel; Hubert Klahr; Cornelis P. Dullemond; Sean M. Andrews11909.10526v1Including Dust Coagulation in Hydrodynamic Models of Protoplanetary Disks: Dust Evolution in the Vicinity of a Jupiter-mass PlanetJoanna Drazkowska; Shengtai Li; Til Birnstiel; Sebastian M. Stammler; Hui LiFiltering specific articles by keywordsIf both,search_query=andid_list=are present, the given arXiv articles are filtered by the give key word.import arxivloader

keyword = "DSHARP"
prefix = "ti"
IDs = ["1909.04674", "1909.10526"]
query = "search_query={pf}:{kw}&id_list={ids}".format(pf=prefix, kw=keyword, ids=",".join(IDs))
columns = ["id", "title", "authors"]

df = arxivloader.load(query, columns=columns)

print(df)idtitleauthors01909.04674v1The DSHARP Rings: Evidence of Ongoing Planetesimal Formation?Sebastian M. Stammler; Joanna Drazkowska; Til Birnstiel; Hubert Klahr; Cornelis P. Dullemond; Sean M. AndrewsSearching by dateIt is possible to only retrieve entries in a specified date window.This query selects all publications that have been submitted toastro-ph.EPon July 1st 2022 between 8am and 1pm.import arxivloader

prefix = "cat"
cat = "astro-ph.EP"
submittedDate = "[20220701080000+TO+20220701130000]"
query = "search_query={pf}:{cat}+AND+submittedDate:{sd}".format(pf=prefix, cat=cat, sd=submittedDate)
columns = ["id", "title", "authors", "published"]

df = arxivloader.load(query, columns=columns, sortBy="submittedDate", sortOrder="ascending")
print(df)idtitleauthorspublished02207.00273v1Whistler Waves As a Signature of Converging Magnetic Holes in Space PlasmasWence Jiang; Daniel Verscharen; Hui Li; Chi Wang; Kristopher G. Klein2022-07-01 08:55:5412207.00322v2DustPy: A Python Package for Dust Evolution in Protoplanetary DisksSebastian Markus Stammler; Tilman Birnstiel2022-07-01 10:25:59Searching by categoryIt is possible to search large number of articles by category. Please be responsible with the traffic this query causes.import arxivloader

keyword = "astro-ph.EP"
prefix = "cat"
query = "search_query={pf}:{kw}".format(pf=prefix, kw=keyword)
columns = ["id", "title", "primary_category", "categories", "published"]

df = arxivloader.load(query, columns=columns, sortBy="submittedDate", sortOrder="descending", num=1000, page_size=100)

print(df.head(5))idtitleprimary_categorycategoriespublished02210.11357v1The Key Factors Controlling the Seasonality of Planetary Climatephysics.ao-phphysics.ao-ph; astro-ph.EP2022-10-20 15:45:4312210.11305v1On the origin of the dichotomy of stellar activity cyclesastro-ph.SRastro-ph.SR; astro-ph.EP2022-10-20 14:34:3322210.11207v1$\texttt{KOBEsim}$: a Bayesian observing strategy algorithm for planet detection in radial velocity blind-searchastro-ph.EPastro-ph.EP; astro-ph.IM2022-10-20 12:33:0332210.11103v1Lower-than-expected flare temperatures for TRAPPIST-1astro-ph.SRastro-ph.SR; astro-ph.EP2022-10-20 08:55:4742210.10909v1TOI-3884 b: A rare 6-R$_{\oplus}$ planet that transits a low-mass star with a giant and likely polar spotastro-ph.EPastro-ph.EP2022-10-19 22:19:15Optionsarxivloader.load()has several keyword arguments:KeywordDefault valueDescriptionnum10Maximum total number of entries to be retrieved.start0Starting index of query.page_size10The entries are retrieved in pages. The maximum allowed page size is 30000.delay3.Delay in seconds between page requests.sortBy"relevance"Possible values:"relevance","lastUpdatedDate","submittedDate".sortOrder"descending"Possible values:"descending","ascending".columns["id", "title", "summary", "authors", "primary_category", "categories", "comments", "updated", "published", "doi", "links"]List of the columns thepandas.DataFrameshould contain.timeout10.Timeout in seconds for a single page.verbosity2Level of verbosity.The default options are usually good enough.Thedelayhas to be at least three seconds to be fair with the load on the arXiv API.It can happen that the arxiv API does not respond for a query.timeoutwill set the time after whicharxivloaderassumes a failed attempt and will retry at most five times. Please note, thattimeoutneeds to be larger than the arXiv API takes to process the query, which depends onpage_size. Consider two minutes for ten thousand entries in a page.Ifverbosityis0,arxivloaderwill not display anything on screen. Ifverbosityis1,arxivloaderwill print out the number of retrieved entries at the end of execution. Ifverbosityis2,arxivloaderwill additionally show a progess bar.AcknowledgementsThank you to arXiv for use of its open access interoperability.