Package: pyAutoSummarizer
pyAutoSummarizerpyAutoSummarizer - An Extractive and Abstractive Summarization Library Powered with Artificial Intelligence.IntroductionpyAutoSummarizer is a sophisticated Python library developed to handle the complex task of text summarization, an essential component of NLP (Natural Language Processing). The library implements several advanced summarization algorithms, both extractive and abstractive. Extractive summarization algorithms focus on identifying and extracting key sentences or phrases from the original text to form the summary. Among the techniques utilized by pyAutoSummarizer areTextRank,LexRank,LSA(Latent Semantic Analysis), andKL-Sum. In the domain of deep learning, pyAutoSummarizer incorporatesBART(Bidirectional and Auto-Regressive Transformers) and the use ofT5(Text-to-Text Transfer Transformer) model, which is known for its versatility in handling a range of language tasks including summarization. Furthermore, pyAutoSummarizer also utilizesPEGASUS(Pre-training with Extracted Gap-sentences for Abstractive Summarization) and the OpenAI's GPT (Generative Pretrained Transformer), specifically thechatGPTmodel for abstractive summarization. Unlike extractive techniques, abstractive summarization involves generating new sentences, offering a summary that maintains the essence of the original text but may not use the exact wording.pyAutoSummarizer stands out for its proficient preprocessing capabilities that pave the way for high-quality text summarization. Recognizing the importance of text normalization, the library offers a range of text cleansing and standardization features. It can convert text tolowercase, ensuring uniformity across the data. Additionally, it canremove accents,remove special characters, andremove numbers, which helps mitigate the text's noise. It also offers the functionality toremove custom words, enabling users to tailor their preprocessing needs. Notably, pyAutoSummarizer supportsstopwordsremoval across various languages, including Arabic, Bengali, Bulgarian, Chinese, Czech, English, Finnish, French, German, Greek, Hebrew, Hind, Hungarian, Italian, Japanese, Korean, Marathi, Persia, Polish, Portuguese-br, Romanian, Russian, Slovak, Spanish, Swedish, Thai, and Ukrainian. The library provides flexibility in sentence segmentation, allowing sentences to be split based onpunctuation,character count, orword count.To evaluate the quality of the summaries generated, pyAutoSummarizer integrates various metrics such asRouge-N,Rouge-L, andRouge-S, which compare the overlap of n-grams, longest common subsequence, and skip-bigram between the generated summary and the reference summary respectively. Additionally, it employsBLEU(Bilingual Evaluation Understudy), andMETEOR(Metric for Evaluation of Translation with Explicit ORdering).UsageInstallpipinstallpyAutoSummarizerTry it inColab:Extractive SummarizationExample 01: TextRank             (Colab Demo)Example 02: LexRank              (Colab Demo)Example 03: LSA                  (Colab Demo)Example 04: KL-Sum               (Colab Demo)Example 05: BART (Deep Learning) (Colab Demo)Example 06: T5 (Deep Learning)   (Colab Demo)Abstractive Summarization.Example 01: chatGPT (Deep Learning) (Colab Demo) Requires the user to have anAPI key(https://platform.openai.com/account/api-keys)Example 02: PEGASUS (Deep Learning) (Colab Demo)OtherspyBibX- A Bibliometric and Scientometric Python Library Powered with Artificial Intelligence Tools