Package: ai-scholar-toolbox
ai-scholar-toolboxThe python package provides an efficient way to get statistics of a scholar on Google Scholar given academic information of the scholar.Install packagepipinstallai-scholar-toolboxDownload Browser Binary and Browser DriverBy default, our package uses Chromium binary file. Please take care the compatibility between the binary file and the browser driver. Also, if your OS is not linux based or you install the browser in a directory other than default directory, please refer toSelenium Chrome requirementswhen instantiating the browser driver.Download:Download Chrome browser driverInstall:Linuxsudoaptupdate
sudoaptupgrade
wgethttps://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
sudoaptinstall./google-chrome-stable_current_amd64.debGet Started in ai-scholar-toolboxInstantiate aScholarSearchobject. This will download 78k dataset to the local machine automatically.fromScholarSearchimportScholarSearchscholar_search=ScholarSearch()Set attributes for the class:# set the similarity ratio of comparing two strings when searching on Google Scholar webpage. If not given, default is 0.8.scholar_search.similarity_ratio=0.8# set the path of browser driver.scholar_search.driver_path='../../chromedriver'# required: setupscholar_search.setup()Optional: In case that you want to get responses of a list of scholars, the class methodget_profiles()is implemented for you to load (could be multiple) json data files.# optionalscholar_search.get_profiles(['../review_data/area_chair_id_to_profile.json','../review_data/reviewer_id_to_profile.json'])Search candidate scholars by matching a specific query:If you want to input the information of a scholar on OpenReview and get related google scholar information, you can pass in a pythondictionarywith necessary features based on the OpenReview scholar profile page (for instance, fromZhijing's OpenReview profile). Note that filling in more information as recommended below will get a better search result.TODO: change the dict to be of a certain person.# keys that are required:# scholar_info_dict['content']['gscholar']: the link to Google Scholar profile in the OpenReview webpage. If cannot be found, you can either choose not to include it or pass in an empty string.# scholar_info_dict['content']['history']: the most updated history of the scholar in the OpenReview webpage. Previous history is not needed.# scholar_info_dict['content']['relations']: all relations that the scholar list in the OpenReview webpage. We recommend to list all the relations here. Only name is needed.# scholar_info_dict['content']['expertise']: all keywords that the scholar label their academic research field. We recommend to list all the expertise keywords here. Only keyword is needed.# Most recommended:scholar_info_dict={"profile":{"id":"~Zhijing_Jin1",# most important information to use"content":{"gscholar":"https://scholar.google.com/citations?user=RkI8h-wAAAAJ","history":[# second most important information to use{"position":"PhD student","institution":{"domain":"mpg.de","name":"Max-Planck Institute"}}],"relations":[{"name":"Bernhard Schoelkopf"},{"name":"Rada Mihalcea"},{"name":"Mrinmaya Sachan"},{"name":"Ryan Cotterell"}],"expertise":[{"keywords":["causal inference"]},{"keywords":["computational social science"]},{"keywords":["social good"]},{"keywords":["natural language processing"]}]}}}# Minimum required but least recommended:scholar_info_dict={"profile":{"id":"~Zhijing_Jin1","content":{}}}Then, you can pass the dictionary to the methodget_scholar()to get possible candidates.# query: python dictionary that you just generated.resp=scholar_search.get_scholar(query=scholar_info_dict,simple=True,top_n=3,print_true=True)respAlternatively, if you just want to input the name of a scholar and get possible google scholar candidates, you can pass the name as a string directly to the function as the following:# query: python str, the name of the scholar.resp=scholar_search.get_scholar(query='Zhijing Jin',simple=True,top_n=3,print_true=True)respSearch AlgorithmsThe algorithm can be explained as follows if the input query is a python dictionary:defget_candidates(openreview_dict,top_n_related):ifgs_sidinopenreview_dict:ifgs_sidin78k_scholar:returndict(78k_scholar.loc[78k_scholar[‘gs_sid’]==gs_sid])else:response=search_directly_on_google_scholar_by_gssid(gs_sid)returnresponseelse:name,email_suffix,position,organization,relations=extract_name_from_openreview_dict(openreview_dict)response_78k=search_scholar_on_78k(name)response_gs=search_scholar_on_google_scholar(name,email_suffix,position,organization,relations)response=select_final_candidates(response_78k,response_gs,top_n_related=top_n_related)returnResponseStatistics SummaryOur 78k dataset has 78,066 AI scholars in total. Please check our78k AI scholar datasetfor more details.Given all the chairs and reviewers in OpenReview (664 in total), our package achieves 93.02% precision, 85.11% recall, and 88.89% F1-score on a random subset of 50 scholars that don't havegs_sidincluded in the input dict.FAQTODO: add contentSupportIf you have any questions, bug reports, or feature requests regarding either the codebase or the models released in the projects section, please don't hesitate to post on ourGithub Issues page.LicenseThe package is licensed under the MIT license.TODO: check licenses