Saravanan666/pypi-org huggingface dataset:
Package: data-science-common
Data Science Common (dsc)UNDER CONSTRUCTION:
A simple Python library for facilitating analysis.When performing data analysis and other data intensive tasks, one often finds many examples where they use same code
or workflow repeatedly.  The next obvious step is to write resuable piece.  This project is an attempt to collect
all of these reusable pieces on a central place and to facilitate collaboration.This library contains methods to help with:String manipulationDate manipulationData analysisDatabase connections and queriesCloud computingThis is an experimental library and will possibly be broken into smaller pieces in the future.Installationpipinstalldata-science-commonUsage exampleA few motivating and useful examples of how your product can be used. Spice this up with code blocks and potentially more screenshots.For more examples and usage, please refer to theWiki.Development setupDescribe how to install all development dependencies and how to run an automated test-suite of some kind. Potentially do this for multiple platforms.makeinstallRelease History0.1.7refactored code layout and updated build to pyproject.toml paradigm0.1.6updating documentation and deployment infrastructure0.1.5CHANGE: added packagedsc.util.dateCHANGE: added packagedsc.util.dbCHANGE: added packagedsc.util.errorCHANGE: added packagedsc.util.objCHANGE: added packagedsc.util.pdCHANGE: added packagedsc.util.str0.1.4CHANGE: adding CI/CD0.1Initial releaseMetaRyan Kelley –kelleyrw@gmail.comDistributed under the Apache License, Version 2.0. SeeLICENSEfor more information.https://github.com/kelleyrwContributingFork it (https://github.com/kelleyrw/data_science_common/fork)Create your feature branch (git checkout -b feature/fooBar)Commit your changes (git commit -am 'Add some fooBar')Push to the branch (git push origin feature/fooBar)Create a new Pull Requestrequirements for developmentblackisortpandoc
Package: data-science-helper-utility
data-science-helper xD1. Organización de las librerias==============================├── README.md          <- README principal para los desarrolladores que usen este proyecto.
│
├── config             <- Archivos de configuracion.
│
├── docs               <- Documentación del proyecto, ver detalles.
│
├── src                <- Código principal para el uso del proyecto.
│   ├── __init__.py    <- Convierte al folder en un modulo de Python.
│   │
│   ├── model          <- Contiene scripts de los modelos.
│   │
│   ├── helper_cache.py                   <- Exploración para entender los datos y sus disponibilidad.
│   │
│   ├── helper_classification_model.py    <- Seleccionar, ordenar, agrupar, remover, etc. los datos para alcanzar 
│   │
│   ├── helper_clean.py                   <- Scripts para generación del modelo y afinamiento de parámetros.
│   │
│   ├── helper_clustering.py              <- Scripts para evaluación de resultados establecidos al inicio del proyecto.
│   │
│   ├── helper_dataframe.py               <- Scripts para el despliegue y pase a producción.
│   │
│   ├── helper_feature_selection.py       <- Scripts para evaluación de resultados establecidos al inicio del proyecto.
│   │
│   ├── helper_functions.py               <- Scripts para evaluación de resultados establecidos al inicio del proyecto.
│   │
│   ├── helper_general.py                 <- Scripts para evaluación de resultados establecidos al inicio del proyecto.
│   │
│   ├── helper_output.py                  <- Scripts para evaluación de resultados establecidos al inicio del proyecto.
│   │
│   ├── helper_plot.py                    <- Scripts para evaluación de resultados establecidos al inicio del proyecto.
│   │
│   ├── helper_text_mining.py             <- Scripts para evaluación de resultados establecidos al inicio del proyecto.
│   │
│   └── helper_transformers.py            <- Scripts para el despliegue y pase a producción.
│   
├── enviroment.yml              <- Archivo con listado de los paquetes necesarios para reproducir el entorno de análisis.
│
├── 00.create_env.bat           <- Ejecutable para crear el entorno virtual con los parámetros del archivo "enviroment.yml".
│
└── 01.update_enviroment.bat    <- Ejecutable para actualizar el archivo "enviroment.yml" antes de ser compartido.2. data-science-helper documentacion.Este proyecto contiene las librerias de uso general que la dirección OSEE ha implementado, las librerias sirven principalmente para limpiar, procesar, transformar, agregar, etc. los datos que maneja la dirección.Version: 0.0.79Date: Agostos 31, 2022Enlace de descarga: https://URL_PAQUETE/data-science-helper-utility/-/archive/master/data-science-helper-utility-master.tarPara ver el detalle de cada una de las funciones debe dirigirse al sisguiente enlace:Documentación detallada
Package: data-science-kit
Data Science Kit - KitbagKITBAG is a helpful library where you can find basic data science functions under three main headings. Three main headings are given below. The titles are given below;Measurement UnitsArithmetic MeanGeometric MeanHarmonic MeanMedianModeVarianceStandart DeviationFind Minimum Value In ListFind Maximum Value In ListKurtosisSkewnewssQuantilesRange Of ChangeCovarianceCorrelationExploratory Data AnalysisCopy DatasetShow Head & Tail ValuesStructural InformationVariable TypesObject To CategoricalObservation Values SizeVariable Values SizeSize Of DatasetDimension Of DataFrameDimension Of SeriesVariable Names Of DatasetDescriptive Statistics Of DatasetDescriptive Statistics Of Numerical VariableDescriptive Statistics Of Categorical VariableSelect Categorical VariablesSelect Numerical VariablesFrequency Of Categorical VariablesCategorical Variables Percentage Of DatasetFind Unique Categorical VariablesFrequency Unique Categorical VariablesMeasure Operations Of Single Categorical Variables With All Numerical VariablesMeasure Operations Of Single Categorical Variables Single Numerical VariablesMeasure Operations Of Multiple Categorical Variables Single Numerical VariablesCreate And Sort Rank VariablesSelect Range Row By IndexData OperationsMissing ValuesIs There Any NAN ValuesTotal NAN ValuesPercentage NAN ValuesDrop NAN Values In RowDrop NAN Values In ColumnFill NAN Values ColumnFill NAN Values All DatasetFil NAN Values KNNFil NAN Values EMOutlier ValuesSelect Lower Outlier Values - According to the specified ColumnSelect Upper Outlier Values - According to the specified ColumnSelect All Outlier Values - According to the specified ColumnDelete Outlier ValuesFill Mean Value All Outlier ValuesFill Suppression Method All Outlier ValuesLocal Outlier FactorLocal Outlier Factor Suppression MethodVariables StandartizationNumerical Variables StandartizationNumerical Variables NormalizationNumerical Variables TransformationNumerical Variables BinarizeNumerical Variables To Categorical VariablesEncoding OperationsOrdinal EncoderLabel EncoderWhere EncoderOnehot Encoder
Package: data-science-toolbox
data_science_toolbox=====================Various code to aid in data science projects for tasks involving data cleaning,
ETL, EDA, NLP, viz, feature engineering, feature selection, model training and validation etc.InstallationUsing pipYou can install using the pip package manager by runningpip install data-science-toolboxProject Organization├── README.md              
├── data_science_toolbox   <- Project source code
│   │
│   ├── gists              <- Code gists with commonly used code (change to root
│   │                         directory, connect to database, profile data, etc)
│   ├── data_checks        <- Code for data checks and assertions
│   ├── io                 <- Code for input/output utilities
│   ├── etl                <- For building reproducible ETL pipelines, including data
│   │                         checks and transformers
│   ├── ml                 <- Machine Learning utility code (feature engineering, etc) 
│   ├── pandas             <- Pandas related utility code
│   │   ├── analysis                  
│   │   ├── cleaning
│   │   ├── engineering
│   │   ├── text    
│   │   ├── datetime     
│   │   ├── optimization       
│   │   └── profiling   
│   ├── project_utils.py   <- For project specific utilities
│   │
│   ├── text               <- Code for dealing with text. Includes distributed loading of text corpus, 
│   │                         entity statement extraction, sentiment analysis, pii removal etc.
│   └── __init__.py        <- Makes data_science_toolbox a Python module               
├── tests                  <- Pytest unit tests 
├── dist                   <- tars and whls of version builds
├── LICENSE
├── poetry.lock
└── pyproject.toml
Package: data-science-toolkit
Data Science ToolkitData Science Toolkit (DST) is a Python library that helps implement data science related project with ease.Simple Demofromdata_science_toolkit.dataframeimportDataFramefromdata_science_toolkit.modelimportModeldata=DataFrame()data.load_dataset('iris')y=data.get_column('target')data.drop_column('target')# decision tree modelmodel=Model(data_x=data.get_dataframe(),data_y=y,model_type='dt',training_percent=0.8)# train the modelmodel.train()# get all classification evaluation metricsmodel.report()#get the cross validationmodel.cross_validation(5)DocumentationMore information can be found on theDST documentation site.ContributingContrubution and suggestions are welcome via GitHub Pull Requests.MaintainershipWe're actively enhacing the repo with new algorithms.How to cite
Package: datascience-workbench
Data Science WorkbenchData Science Workbench is a Python library to handle common tasks in a data science project.
It gives a shortcut to processing, storing and preparing data.InstallationUse the package managerpipto install.pipinstalldatascience-workbenchUsagefromworkbenchimportListsaswlfromworkbenchimportTimeaswtTodoadd unit testsContributingPull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.Please make sure to update tests as appropriate.LicenseMIT
Package: datascientists
datascientistsQuick Data Science tools, providing data processing and visualization in Hypotheses Testing, Regression, and etc.Dev VersionTry latest version, visitTest PyPI.Best PracticesBest Practices fordatascientistsvisualization requirenodejsandjupyter.Best Practices for Python venvIf you need to isolate the Python environment, visitvenv — Creation of virtual environmentsBest Practices for nodejsInstall nvm (Node Version Manager) from githubCheck your nvm version:nvm-vOnce you installed nvm, usingnvmto install node:nvminstallnodeTo list all local nodes:nvmlistTo choose your node version:
For example, using v16.0.0nvmusenodev16.0.0You can list available versions usingls-remote:nvmls-remoteBest Practices for JupyterlabInstall jupyterlabpipinstalljupyterlabSet jupyterlab and install extensions for plotly:# (OS X/Linux)exportNODE_OPTIONS=--max-old-space-size=4096# Jupyter widgets extensionjupyterlabextensioninstall@jupyter-widgets/jupyterlab-manager--no-build# jupyterlab renderer supportjupyterlabextensioninstalljupyterlab-plotly--no-build# FigureWidget supportjupyterlabextensioninstallplotlywidget--no-build# Build jupyterlab (must be done to activate extensions since --no-build is used above)jupyterlabbuild# Unset NODE_OPTIONS environment variable# (OS X/Linux)unsetNODE_OPTIONSSee if you have installed the three extensions correctly:jupyterlabextensionlistYou should see:@jupyter-widgets/jupyterlab-managerjupyterlab-plotlyplotlywidget
Package: datascienv
Data Science Environment Setup in single lineThis package helps to setup your Data Science environment in single line.Developed by Ashish Patel(c) 2020.datascienvdatascienv is a python package offering a single line Data Science Environment setup.InstallationDependenciesdatascienvis tested to work under Python 3.7+ and greater. The dependency requirements are based on thedatascienvpackage update release:pandas(latest) -https://pandas.pydata.org/numpy(latest) -https://numpy.org/install/scipy(latest) -https://www.scipy.org/scikit-learn(latest) -https://scikit-learn.org/joblib(latest) -https://joblib.readthedocs.io/en/latest/statmodels(latest) -https://www.statsmodels.org/stable/index.htmlmatplotlib(latest) -https://matplotlib.org/seaborn(latest) -https://seaborn.pydata.org/xgboost(latest) -https://xgboost.ai/sponsorsimbalanced-learn(latest) -https://imbalanced-learn.org/bokeh(latest) -https://docs.bokeh.org/en/latest/Boruta(latest) -https://github.com/scikit-learn-contrib/boruta_pyjupyter(latest) -https://jupyter.org/spyder(latest) -https://www.spyder-ide.org/mlxtend(latest) -http://rasbt.github.io/mlxtend/lightgbm(lightgbm) -https://lightgbm.readthedocs.io/en/latest/catboost(latest) -https://catboost.ai/pycaret(latest) -https://pycaret.org/Installationdatascience is currently available on the PyPi's repository and you can install it via pip:pipinstall-UdatascienvIf you prefer, you can clone it and run the setup.py file. Use the following commands to get a copy from GitHub and install all dependencies:gitclonehttps://github.com/ashishpatel26/datascienv.gitcddatascienv
pipinstall.Or install using pip and GitHub:pipinstall-Ugit+https://github.com/ashishpatel26/datascienv.gitWarnings:If you find this type of warning then ignore that warning.
Package: datascope_tools
datascope-toolsconsists of a collection of scripts, tools, and
ideas that we have collected over the years at
[Datascope Analytics](http://datascopeanalytics.com) that help with
our workflow and some very basic command-line data analysis.This is somewhat inspired by
[bit.ly’s data_hacks](https://github.com/bitly/data_hacks)[scripts] datascope-toolspair-with: simple script to manipulate.hg/hgrcfiles when
pairing with other peopledecrapify: simple script to remove junk files that commonly occur when
we’re writing code or experimenting with scripts
Package: data-scout
Data ScoutThis package provides the tools to quickly setup a scalable and readable data preparation pipeline that can be run on different platforms. Currently only vanilla Python is available, but PySpark should be available soon. There is also a user interface available here:Data Scout serverthat allows you to create data pipelines in a visual editor and then export them as either a JSON file, or just plain old simple Python that can be used anywhere.InstallationThe easiest and quickest way to install Data Scout is through PyPi, just execute the following command:pipinstalldata-scoutExecuting a JSON pipelinePipeline definitions can be given as JSON files or directly as Python commands. To execute a JSON definition, your code would look somewhat as follows:fromdata_scout.executorimportPandasExecutorfromdata_scout.scoutimportScoutscout=Scout()executor=PandasExecutor({"source":"CSV","kwargs":{"filename":"test.csv","delimiter":",","encoding":"utf-8","has_header":True}},[{"transformation":"data-convert","kwargs":{"field":"column1","to":"int"}}],scout)executor()This will load a CSV file and convert the column named "column1" to an integer using Pandas as a backend.DevelopmentFor development purposes, install the package using the following command:pipinstall-e.[dev]TestingThere are some unit tests available. The unit tests are written using the Nose2 framework. The setup.py script should have already installed Nose2, so now you may run the tests as follows:nose2-v
Package: datascraping-article
#### This python package I have created to scrape the latest articles title of recent 30 days. Besides that it will scrape and collect the date as well as the subtitle.
* Importantly, this task was conducted with a steadfast commitment to the ethical guidelines governing web scraping.
Additionally this would be helpful in finding the the keywords and the trends in data science community with in the month after doing the analysis.
Also on enhancing the skills for text generation using RNN LSTMs.
Package: datascribe
datascribe: useful tools to help you document and describe data processing and modellingdatascribe has been developed to assist data scientists in documenting and describing data analysis, processing and modelling.Vision for datascribeAssist users in documenting the dataset they are using.Assist users in documenting the preprocessing and subsequent analysis they perform on datasets.Provide summary text, images and tables in an editable format (markdown and Microsoft Word) tool.Help users document research and analyses well so that it is transparent and repeatable.Features:Creation of basic summary paragraphs with supporting markdown tables or table images to describe initial dataset information.Implementation of key feature engineering tools fromsklearnin tandem with producing a log of steps taken to process and analyse the dataset.Ability to produce a workflow diagram viagraphvizto help visualise the data processing and modelling workflow.How to explore datascribeInstalling the virtual environmentDetails of the conda virtual environment are available here:binder/environment.ymlOpen the repo in a terminal (Mac/Linux) or anaconda prompt (Windows)Navigate to the correct directory.Create the environment with the following command:condaenvcreate-fbinder/environment.ymlActivate the environment with the following command:condaactivatedatascribeIt is strongly recommended that you install a conda environment to avoid dependency conflicts.DependenciesThis project relies on the following external dependencies:Graphviz: Used for creating visualizations in the workflow.Installing DependenciesGraphvizMake sure you have Graphviz installed on your system. You can download it from theofficial Graphviz websiteor install it using your package manager:Linux (e.g., Ubuntu)sudoapt-getinstallgraphviz
Package: datascrub
DataScrubDataScrub is a Python package that provides powerful data cleaning and preprocessing capabilities for pandas DataFrames. It offers a collection of functions and utilities to facilitate data cleaning tasks, handling missing values, standardizing data formats, and more. With DataScrub, you can streamline your data preparation process and ensure the quality and consistency of your datasets.FeaturesClean text data in pandas DataFramesHandle missing values with customizable actionsPerform scaling normalization on numerical columnsSplit and expand data in specified columnsConvert columns to datetime format and format them as 'YYYY-MM-DD'Translate text columns to English using Google TranslateInstallationDataScrub can be easily installed using pip. Simply run the following command:pipinstalldatascrubMake sure you have Python 3.7 or above installed on your system.UsageTo use DataScrub in your Python projects, import the package and create an instance of theDataCleanclass:fromdatascrubimportDataCleanimportpandasaspd# Create a DataFramedata=pd.read_csv("data.csv")# Create an instance of DataCleancleaner=DataClean(data)# Call the available methods to clean and preprocess your datacleaned_data=cleaner.prep(clean='all',missing_values={},perform_scaling_normalization_bool=False,explode={},parse_date={},translate_column_names={})TheDataCleanclass takes a pandas DataFrame or a file path as input. You can then use the various methods available in the class to clean and preprocess your data.Refer to thedocumentationfor detailed information on available methods and usage examples.ContributingContributions to DataScrub are welcome! If you encounter any bugs, have suggestions for improvements, or would like to add new features, please open an issue or submit a pull request on theGitHub repository.LicenseThis project is licensed under the MIT License. See theLICENSEfile for more information.
Package: datasdk
datasdkDescriptionThis is a template used to start python projects.It can be used as if clonning the git repository, or using degit:degit https://github.com/nuxion/python-template <your-project-name>After that runrename.shscript to refactor folders and names to the new one:./rename.sh <your-project-name>FeaturesMakefile for common tasksSphinxpytest.gitignorefor python + nodejs projects + emacs + vim + vscode (using topal gitignore generation tool)jupyter as optional dependency + jupytextlinting toolsReferenceshttps://waylonwalker.com/hatch-version/Licensedatasdkis distributed under the terms of theMITlicense.
Package: data-serialize
data_serializeThis library allows reading and writing binary to stringInstallationpip3 install data_serializeserialize and deserialize#importdata_serializeimportnumpyasnpdeftest_feature():val1=data_serialize.Int64List(value=[1,2,3]*20)val2=data_serialize.FloatList(value=[1,2,3]*20)val3=data_serialize.BytesList(value=[b'The china',b'boy'])featrue=data_serialize.Features(feature={"item_0":data_serialize.Feature(int64_list=val1),"item_1":data_serialize.Feature(float_list=val2),"item_2":data_serialize.Feature(bytes_list=val3)})example=data_serialize.Example(features=featrue)# 序列化serialize=example.SerializeToString()print(serialize)# 反序列化example=data_serialize.Example()example.ParseFromString(serialize)print(example)deftest_numpyobject():a=np.random.randint(0,21128,size=(10,),dtype=np.int64)b=np.random.rand(3,4)c=np.asarray(b'The china')val1=data_serialize.NumpyObject(header='',dtype=str(a.dtype),shape=list(a.shape),int64=a.reshape((-1,)).tolist(),)val2=data_serialize.NumpyObject(header='',dtype=str(b.dtype),shape=list(b.shape),float64=b.reshape((-1,)).tolist(),)val3=data_serialize.NumpyObject(header='',dtype=str(c.dtype),shape=list(c.shape),bytes=c.tobytes(),)example=data_serialize.NumpyObjectMap(numpyobjects={"item_0":val1,"item_1":val2,"item_2":val3})# 序列化serialize=example.SerializeToString()print(serialize)# 反序列化example=data_serialize.NumpyObjectMap()example.ParseFromString(serialize)print(example)test_feature()test_numpyobject()
Package: dataserve
PythonDataServeA module for serving up python data in a stand-alone process.AboutPythonDataServe is a simple library that allows you to serve data from a different python process.  This eliminates the need to load large data files every time you want to run your program.The library includes a very simple HTTP Server for basic data types and a Proxy Server for python objects.  Both servers implement a similar methodology, allowing the user to define functions on the server side which the client can call with a data payload to and get a return response.  This is a remote-procedure-call type of implementation which also makes the library an excellent tool for doing multiprocessing across computers.The library also contains a DataContainer utility class to encapsulate pickling, gzipping, loading and saving functionality.  This simplifies some common data operations.The only external requirement is the commonly used "requests" library.DocumentationFor the latest documentation, seeReadTheDocs.  Alternatively, you can view the markup files directly in thedocsfolder of this project.You can see example implementations in thetestsfolder.
Package: datasesame
This is the loader module and CLI tool forDataSesame. DataSesame is the easiest way to find open tabular datasets for all your data science projects.You can find ready-to-go CSV datasets in the DataSesame databasehere. All datasets can be downloaded in your browser using the download link.Alternatively, you can:Download a dataset into your terminal's current working directoryDirectly load a dataset into your scripts/notebooks (as a Pandas DataFrame)Install from PyPi:$pipinstalldatasesameA. Download a dataset by itsIDinto your CWD using the CLI interface, e.g.:$datasesame42B. Load a dataset as a Pandas DataFrames in Python, e.g.:importdatasesameasdsdf=ds.load(42)# returns a Pandas DataFrameprint(df.head(5))Sadly this dataset does not reveal the meaning of life, but it might make for interesting data science / ML...
Package: dataset-builder-for-segmentation
This is a library with one module to help you build a tf.data Dataset pipeline for the purpose of semantic image segmentation.
Refer ‘demo.ipynb’ athttps://github.com/abhimanyu911/segmentation_pipeline_builderfor instructionsChange Log0.0.1 (02/11/2021)First Release0.0.2 (02/11/2021)Second Release0.0.3 (02/11/2021)Third Release: Option to repeat() shuffling for multiple epochs enabled.