Package: nlp-profiler
NLP Profiler||||||A simple NLP library that allows profiling datasets with one or more text columns.NLP Profiler returns either high-level insights or low-level/granular statistical information about the text when given a dataset and a column name containing text data, in that column.In short: Think of it as using thepandas.describe()function or runningPandas Profilingon your data frame, but for datasets containing text columns rather than the usual columnar datasets.Table of contentsCommunity/Chat/Communication:What do you get from the library?RequirementsGetting startedDemo and presentationsInstallationUsageDeveloper guideNotebooksScreenshotsCredits and supportersChangesLicenseContributingWhat do you get from the library?Input a Pandas dataframe series as an input parameter.You get back a new dataframe with various features about the parsed text per row.High-level: sentiment analysis, objectivity/subjectivity analysis, spelling quality check, grammar quality check, ease of readability check, etc...Low-level/granular: number of characters in the sentence, number of words, number of emojis, number of words, etc...From the above numerical data in the resulting dataframe descriptive statistics can be drawn using thepandas.describe()on the dataframe.See screenshots under theJupytersection and also underScreenshotsfor further illustrations.Under the hood it does make use of a number of libraries that are popular in the AI and ML communities, but we can extend it's functionality by replacing or adding other libraries as well.A simplenotebookhave been provided to illustrate the usage of the library.Please join theGitter.im communityand say "hello" to us, share your feedback, have a fun time with us.Note:this is a new endeavour and it may have rough edges i.e. NLP_Profiler in its current version is probably NOT capable of doing many things. Many of these gaps are opportunities we can work on and plug, as we go along using it. Please provide constructive feedback to help with the improvement of this library. We just recently achieved this withscaling with larger datasets.RequirementsPython 3.6.x or higher.Dependencies described in therequirements.txt.High-level including Grammar checks:faster processorhigher RAM capacityworking disk-space of 1 to 3 GBytes (depending on the dataset size)(Optional)Jupyter Lab (on your local machine).Google Colab account.Kaggle account.Grammar check functionality:Internet accessJava 8 or higherGetting startedDemo and presentationsLook at a short demo of the NLP Profiler library at one of these:or you find the rest of thetalk hereor here forslidesor you find the rest of thetalk here or here forslidesInstallationFor Conda/Miniconda environments:condaconfig--setpip_interop_enabledTrue
pipinstall"spacy >= 2.3.0,<3.0.0"# in case spacy is not presentpython-mspacydownloaden_core_web_sm### now perform any of the below pathways/optionsFrom PyPi:pipinstall-Unlp_profilerFrom the GitHub repo:pipinstall-Ugit+https://github.com/neomatrix369/nlp_profiler.git@masterFrom the source (only for development purposes), seeDeveloper guideUsageimportnlp_profiler.coreasnlpprofnew_text_column_dataset=nlpprof.apply_text_profiling(dataset,'text_column')orfromnlp_profiler.coreimportapply_text_profilingnew_text_column_dataset=apply_text_profiling(dataset,'text_column')SeeNotebookssection for further illustrations.Developer guideSeeDeveloper guideto know how to build, test, and contribute to the library.NotebooksAfter successful installation of the library, RESTART Jupyter kernels or Google Colab runtimes for the changes to take effect.SeeNotebooksfor usage and further details.ScreenshotsSeeScreenshotsCredits and supportersSeeCREDITS_AND_SUPPORTERS.mdChangesSeeCHANGELOG.mdLicenseReferlicensing(and warranty) policy.ContributingContributions are Welcome!Please have a look at theCONTRIBUTINGguidelines.Please share it with the wider community (and get credited for it)!Go to theNLP page