Package: pynlp-lib
PyNLP Lib (or PyNLPL, pronounced "pineapple")PyNLP Lib is an open source Python Natural Language Processing library that provides functionality for both web and local development. It offers a wide range of functionality from text analysis to audio transcription to (planned) language generation. Also called "PyNLPL" (actually this was taken too), "PyNLP-L", "PyNLP-Lib", "PyNLPLib" (the official package name), or "PyNLP Library". This package would have been named PyNLP if that name wasn't taken by a third party wrapper library for Stanford NLP.IF YOU ARE LOOKING FOR THE STANFORD NLP PACKAGE GO TO THE OFFICIAL STANFORD NLP PYTHON PACKAGE-Stanza.PyNLP Lib README NavigationPyNLP Library InstallationUsage for PyNLPLExternal Docs for PyNLP-Lib ToolingPyNLP-Lib FunctionalityPyNLP-L module breakdownPyNLP Lib Online/Web API BackendsLocal Backends for PyNLPLRoadmap for PyNLP Lib DevelopmentTimeline for PyNLP Lib so farContribution GuidelinesHow to Test LocallyPyNLP Library InstallationPyNLP-Lib can be installed from pip with the linepip install pynlp-libPyNLP-L UsageEVEN THOUGH WE INSTALL PYNLP-LIB, WE MUSTimport pynlpl! DASHES ARE FORBIDDEN IN IMPORTSThe following code snippets assume you are using a.envfile with your API keys for these online backends stored there under the shown keynames. (Deepgram key stored underdeepgram_key, The Text API key stored undertextapi_key)Transcribing an Audio File on the Web with Deepgram:from pynlpl.web import audio
from dotenv import load_dotenv
import os, asyncio, json

load_dotenv()
deepgram = audio.Deepgram(os.getenv("deepgram_key"))

async def main():
    # Initializes the Deepgram SDK
    # Open the audio file of https://www.youtube.com/watch?v=sQuFl0PSoXo
    # download with youtube_dl script, found on
    # github here: https://gist.github.com/ytang07/9b8317f268ffcf97cd47950aa7f94282 
    with open("./tests/Watch a professional software engineer live code a web scraper.mp3", 'rb') as audio:
        # ...or replace mimetype as appropriate
        source = {'buffer': audio, 'mimetype': 'audio/mp3'}
        response = await deepgram.transcription.prerecorded(source, {'punctuate': True})
        print(json.dumps(response, indent=4))

asyncio.run(main())Online Text Analysis with The Text API:from pynlpl.web import text
from dotenv import load_dotenv
import os

load_dotenv()
text_API = text.TheTextAPI(os.getenv("textapi_key"))

test_text = """The Text API is a comprehensive text processing and sentiment analysis API created by Yujian Tang. PyNLP-Lib or PyNLPL is an open source NLP library for Python. PyNLP-L aims to coalesce many different NLP backend tools and offer a high level API to use them. This test example shows how we can use the online text processing capabilities of PyNLP-L."""

def summarize_test():
    res = text_API.summarize(text=test_text)
    assert "most positive sentences" in res

def most_common_phrases_test():
    res = text_API.most_common_phrases(text=test_text)
    assert "most common phrases" in res

def least_common_phrases_test():
    res = text_API.least_common_phrases(text=test_text)
    assert "least common phrases" in res

def ner_test():
    res = text_API.ner(text=test_text)
    assert "ner" in res

def most_positive_sentences_test():
    res = text_API.most_positive_sentences(text=test_text)
    assert "most positive sentences" in res

def most_negative_sentences_test():
    res = text_API.most_negative_sentences(text=test_text)
    assert "most negative sentences" in res

def summarize_test():
    res = text_API.summarize(text=test_text)
    assert "summary" in res

def kw_test():
    res = text_API.sentences_with_keywords(kws =["PyNLP"], text=test_text)
    assert "\"PyNLP\":" in res

def similarity_by_sentences_test():
    res = text_API.similarity_by_sentences(texts=[test_text, test_text])
    assert any(x in res for x in ["doc1 cleaned", "doc2 cleaned", "repeat sentences"])

def test():
    summarize_test()
    kw_test()
    most_positive_sentences_test()
    most_negative_sentences_test()
    similarity_by_sentences_test()
    most_common_phrases_test()
    least_common_phrases_test()
    ner_test()

test()External Documentation for PyNLP LibThis section includes external documentation for the tools used in PyNLP Lib.The Text APIIncluded in BetaResources:Build an AI Text SummarizerBuild an AI Content Moderation SystemText Sentiment AnalysisBest Way to do Named Entity Recognition (NER) PythonNLP: What is Text Polarity?NLP Stop Words and How to Use ThemExample Projects:What are the Most Common Phrases on YouTube?Black Friday: How Does Twitter Feel?Using NLP to Analyze Obama HeadlinesUse NLP to get Insights from TwitterDeepgramIncluded in BetaResources for the SDK:Create Readable Transcripts for Your PodcastsTopic DetectionExample Projects:Play the Piano with Your VoiceTorchAudioComing in 2023Resources:Local Speech Recognition with PyTorch TorchAudioExample Projects:spaCyComing late 2022NLTKComing late 2022Stanford NLP/StanzaComing in 2023 (flex add)DeepSpeechComing late 2022Resources:DeepSpeech Local Speech RecognitionExample Projects:Microsoft TextComing in 2023Microsoft AudioComing in 2023Google TextComing in 2023Google AudioComing in 2023Amazon TextComing in 2023Amazon AudioComing in 2023PyNLP-Lib functionalityPyNLPL is the comprehensive module for NLP in Python. It is an open source NLP module with multiple backends. Currently, PyNLP Lib is maintained by the team atThe Text API.As of the August 2022 release, PyNLP Lib includes functionality for online text and audio processing. SeeRoadmapfor planned future functionality. Ideally, we will add Natural Language Generation, Natural Language Understanding, Optical Character Recognition, and Conversational AI backends as well as additional backends for the existing text/audio features through 2023.PyNLP-L module breakdownPyNLP Lib has two high level modules -webandlocal. Thewebmodule provides access to the web APIs that are used as the backend of PyNLPL. Thelocalmodule provides access to tools that allow you to do NLP on your device.Inside of the modules are individual backends. As of the beta release (0.1.0), thewebbackend containstextandaudiosubmodules. Each of these submodules containclassesfor different backends.web.textcurrently hasThe Text APIwith future plans to extend to include Google, Amazon, and Microsoft Cloud products.web.audiocurrently hasDeepgramwith future plans to extend to include Google, Amazon, and Microsoft Cloud products.PyNLP Lib Online/Web API BackendsCurrent online backends are Deepgram (audio) and The Text API (text)Planned online backends include: Google Cloud, Azure, MicrosoftLocal Backends for PyNLPLLocal backends planned include: spaCy, NLTK, Stanford NLP, and DeepspeechRoadmap for PyNLP Lib DevelopmentThis roadmap assumes no one helps add to this open source library! However, we'd LOVE help, so please feel free to contribute!August 2022 - Initial Public Beta Release (0.1.0)September 2022 - Add Deepspeech for local audio transcription (0.2.0)October 2022 - Add spaCy backend for local text analysis (0.3.0)November 2022 - Add NLTK backend for local text analysis  (0.4.0)December 2022 - Add TorchAudio for local audio transcription (0.5.0)January 2023 - Add Google Cloud Natural Language AI for online text analysis (0.6.0)February 2023 - Add Azure Text Analysis for online Text Analysis (0.7.0)March 2023 - Add an online Translation API (0.8.0)April 2023 - Add Google Online Speech Transcription for audio transcription (0.9.0)May 2023 - Add an online Text Generation API (0.10.0)June 2023 - Add Amazon Transcribe for online audio transcription (0.11.0)July 2023 - Add an online Conversational AI API (0.12.0)August 2023 - Add an online OCR API, upgrade version for official release (1.0.0)Timeline for PyNLP Lib Development so farAugust 2022 - Initial Beta ReleaseContribution GuidelinesRemember to update requirements inpyproject.tomlRemember to update version inpyproject.tomlHow to Test LocallyRunpython -m buildin the folder that containspyproject.tomlthis will produce adistfolder with a.whlfile, copy the relative pathRunpip install <path to whl>You have to runpip uninstall pynlp-libbetween tests of each update to remove it from the cacheHow to Create a Pull Request