Package: sesg
sesgSeSG (Search String Generator) python package repository.SeSG is a tool developed to help Systematic Literature Review researchers, specifically at the step of building a search string.InstallationYou can install withpip,poetry, or any other package manager:poetryaddsesgUsageFor a more extensive example, please refer tothis repository.Generating a search stringfromdataclassesimportdataclassfromrandomimportsamplefromsesg.search_stringimport(SimilarWordsFinder,create_enrichment_text,generate_search_string,set_pub_year_boundaries,)fromsesg.topic_extractionimportcreate_docs,extract_topics_with_bertopicfromtransformersimportBertForMaskedLM,BertTokenizer@dataclassclassStudy:title:strabstract:strkeywords:strGS:list[Study]=[]QGS:list[Study]=sample(GS,len(GS)//3)defmain():docs=create_docs([{"title":s.title,"abstract":s.abstract,"keywords":s.keywords,}forsinQGS])enrichment_text=create_enrichment_text([{"title":s.title,"abstract":s.abstract,}forsinQGS])similar_words_finder=SimilarWordsFinder(enrichment_text=enrichment_text,bert_model=BertForMaskedLM.from_pretrained("bert-base-uncased"),bert_tokenizer=BertTokenizer.from_pretrained("bert-base-uncased"),)topics=extract_topics_with_bertopic(docs,kmeans_n_clusters=2,umap_n_neighbors=5,)search_string=generate_search_string(topics,n_words_per_topic=5,n_similar_words_per_word=1,similar_words_finder=similar_words_finder,)search_string=f"TITLE-ABS-KEY({search_string})"search_string=set_pub_year_boundaries(search_string,min_year=2010,max_year=2020)print(search_string)# TITLE-ABS-KEY((("antipatterns") AND ("detection" OR "management") AND ("bdtex") AND ("approach" OR "algorithm") AND ("smurf")) OR (("code" OR "pattern") AND ("detection" OR "management") AND ("design" OR "software") AND ("software" OR "computer") AND ("learning" OR "translation"))) AND PUBYEAR > 1999 AND PUBYEAR < 2018  # noqa: E501if__name__=="__main__":main()Assessing the quality of a search stringimporttriofromsesg.evaluationimportEvaluationFactory,Studyfromsesg.scopusimportInvalidStringError,Page,ScopusClientAPI_KEYS:list[str]=[]GS:list[Study]=[]QGS:list[Study]=[]asyncdefmain():string='TITLE-ABS-KEY("machine learning" and "code smell") AND PUBYEAR > 2010 AND PUBYEAR < 2020'# noqa: E501evaluation_factory=EvaluationFactory(gs=GS,qgs=QGS)client=ScopusClient(API_KEYS)entries:list[Page.Entry]=[]try:asyncforpageinclient.search(string):entries.extend(page.entries)exceptInvalidStringError:print("Invalid string")evaluation=evaluation_factory.evaluate([e.titleforeinentries])print(evaluation.start_set_recall)# 0.7if__name__=="__main__":trio.run(main)CreditsThis project is a continuation ofLeo Fuchs'work. Most of my work in this project consisted in refactoring the codebase, adding tests, improving the documentation and optimizing the performance, along with the addition of some new features.HighlightsBelow you can find the major improvements over the original project:AddedBERTopicas a topic extraction strategy.Improvedsnowballingperformance by 100x~120x (thanks torapidfuzzandmultiprocessing).Improvedscopus searchperformance by 30x~40x (thanks tohttpxandEduardo Mendes'help).Improvedsearch string generationperformance by ~1.5x (thanks to a caching system).Improvedcode qualityby adopting the use oflintandformattingtools. Also, addedtype hintsto try to catch errors before runtime.Addedteststo prevent bugs when refactoring or adding new features.Addeddocsto help users and contributors.ContributingYou can contribute in many ways, such ascreating issuesandsubmitting pull requests. If you wish to contribute with code, please read thecontributor guide.LicenseThis project is licensed under the terms of theGPL-3.0-only license.