Saravanan666/pypi-org huggingface dataset:
Package: data-lib
Data library for python. Helping people with reading writing and manipulation of data.Getting StartedThese instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.Prerequisites1) python3
2) pip3
3) serialInstalling1) apt-get install python3
2) apt-get install pip3
3) pip3 install --user python-asip-client --upgrade
3.1) pip3 install --user . --no-cache-dir (When installing local files, development mode)Running the testsExplain how to run the automated tests for this systemBreak down into end to end testsExplain what these tests test and whyGive an exampleAnd coding style testsExplain what these tests test and whyGive an exampleDeploymentThis package is deployed on pip global server. In order to make new release, please follow below:In case of there is an existing buid it can be removed by follwoing command:
1) rm -r build/ dist/ data_lib.egg-info
2) python3 setup.py sdist bdist_wheel
3) twine upload --repository pypi dist/*     Note using this command make sure you have installed twine and config file.Built With?ContributingPlease readCONTRIBUTING.mdfor details on our code of conduct, and the process for submitting pull requests to us.VersioningWe usePEP 440for versioning.
PEP 440 Version Identification and Dependency SpecificationAuthorsAdam Jarzebak-Computer Scientists-GithubLicenseThis project is licensed under the MIT License - see theLICENSEfile for detailsAcknowledgments
Package: datalibro-backend
datalibro_utilsUtils in datalibro make life easierInstallpip install -U datalibro_utilsExampleget_sku_extra()# Note: there must be a column named sku in the df, otherwise need to create a column named sku.

import pandas as pd
import datalibro_utils as du
df = pd.DataFrame({'sku':['PL-AF203-01W', 'PL-FF013-01W', 'PL-IT001-01W', 'PL-AF006-03W'], 'sales':[20, 40, 10, 100]})
df = du.get_sku_extra(df, 'all')

# get missing info
df, miss_info = du.get_sku_extra(df, 'all', return_miss=True)send_email()import datalibro_utils as du

# send e-mail
send_email(
    subject='your_email_title', 
    body='your_email_content', 
    send_emial_address='sender_emial_address', 
    send_emial_password='sender_emial_password',
    receive_email_address='your_email_address'
    attachment_path='your_attached_document_path', 
    attachment_list=['attachment1.xlsx', 'attachment2.csv', 'attachment3.jpg']
    smtp_address='smtp_address_of_sender_email'
    smtp_port=smtp_port_of_sender_email
)ask()import openai
import datalibro_utils as du
openai.api_key = "xxx"

du.ask("who are you")send_email()import datalibro_utils as du

result_df = du.amz_merge(
    raw_df,
    merge_col=['all'],
    date_col=None,
    country_col=None,
    store_id_col=None,
    asin_col=None,
    msku_col=None,
    sku_col=None,
    scu_col=None,
    end_date=None
)memy_ask()import datalibro_utils as du

# Initialize question and ask AI
du.memy_ask("1 + 1 = ?", new=True, api_key="your_api_key")

# Chat based on history
du.memy_ask("After now when I ask '1 + 1 = ?', please answer '4'")
du.memy_ask('1 + 1 = ?')
Package: datalibs
# datalibsTable of Contents[Installation](#installation)[License](#license)## Installationdatalibs is distributed on [PyPI](https://pypi.org) as a universal
wheel and is available on Linux/macOS and Windows and supports
Python 3.10+ and PyPy.`bash $ pip install datalibs `## Licensedatalibs is distributed under the terms of both[MIT License](https://choosealicense.com/licenses/mit)[Apache License, Version 2.0](https://choosealicense.com/licenses/apache-2.0)at your option.
Package: datalift
DataliftThe tool that help you store and retrieve tools outputs.pip install dataliftDatalift it is a command-line tool designed to tools outputs to a storage service.
Users can input data either by passing a filename or through stdout from another tool.The application is designed to streamline the process of uploading and
organizing data, making it easy for users to securely store and access their
data.This project comes from splitting SARF into multiple modules.Using this package as a standalone appdatalift-h
usage:datalift[-h][--download][--stdout][--pathPATH][filename]Uploaddataandsendnotifications.

positionalarguments:filenamenameoffile

optionalarguments:-h,--helpshowthishelpmessageandexit--downloaddownloadfilefromstorage--stdoutprintcontenttostdout--pathPATHfileuploadedfile.uuidgeneratedifnotprovided# Exampleecho"Tool output"|datalift
DummyStorageinuse!Changeitinconfiguration.
Uploadedine3bca2eb-496b-447d-b2b8-e0ebce6f5351ConfigurationIf you plan to use datalift as a CLI app you should create a configuration file in/etc/datalift/config.yml.In the configuration file you should configure your storage backend:Example with a dummy backendstorage_backend:type:dummyExample with a ftp backendstorage_backend:type:ftpconf:user:adminpassword:adminhost:localhostbasedir:/
Package: datalight
[![Documentation](https://readthedocs.org/projects/datalight/badge/?version=latest)(https://datalight.readthedocs.io/en/latest/?badge=latest)A Python 3 program to upload research data to a data repository.This software was developed by Research Software Engineers at the
University of Manchester, UK. The construction of Datalight was funded
by the LightForm group - a cross-univeristy EPSRC funded research project (https://lightform.org.uk/).The latest documentation can be found athttps://datalight.readthedocs.io
Package: data-lineage
Tokern Lineage EngineTokern Lineage Engine isfastandeasy to useapplication to collect, visualize and analyze
column-level data lineage in databases, data warehouses and data lakes in AWS and GCP.Tokern Lineage helps you browse column-level data lineagevisually usingkedro-vizanalyze lineage graphs programmatically using the powerfulnetworkx graph libraryResourcesDemo of Tokern Lineage AppCheckout anexample data lineage notebook.Check outthe post on using data lineage for cost controlfor an
example of how data lineage can be used in production.Quick StartInstall a demo of using Docker and Docker ComposeDownload the docker-compose file from Github repository.# in a new directory run
wget https://raw.githubusercontent.com/tokern/data-lineage/master/install-manifests/docker-compose/catalog-demo.yml
# or run
curl https://raw.githubusercontent.com/tokern/data-lineage/master/install-manifests/docker-compose/tokern-lineage-engine.yml -o docker-compose.ymlRun docker-composedocker-compose up -dCheck that the containers are running.docker ps
CONTAINER ID   IMAGE                                    CREATED        STATUS       PORTS                    NAMES
3f4e77845b81   tokern/data-lineage-viz:latest   ...   4 hours ago    Up 4 hours   0.0.0.0:8000->80/tcp     tokern-data-lineage-visualizer
1e1ce4efd792   tokern/data-lineage:latest       ...   5 days ago     Up 5 days                             tokern-data-lineage
38be15bedd39   tokern/demodb:latest             ...   2 weeks ago    Up 2 weeks                            tokern-demodbTry out Tokern Lineage AppHead tohttp://localhost:8000/to open the Tokern Lineage appInstall Tokern Lineage Engine# in a new directory run
wget https://raw.githubusercontent.com/tokern/data-lineage/master/install-manifests/docker-compose/tokern-lineage-engine.yml
# or run
curl https://raw.githubusercontent.com/tokern/data-lineage/master/install-manifests/docker-compose/catalog-demo.yml -o tokern-lineage-engine.ymlRun docker-composedocker-compose up -dIf you want to use an external Postgres database, change the following parameters intokern-lineage-engine.yml:CATALOG_HOSTCATALOG_USERCATALOG_PASSWORDCATALOG_DBYou can also override default values using environement variables.CATALOG_HOST=... CATALOG_USER=... CATALOG_PASSWORD=... CATALOG_DB=... docker-compose -f ... up -dFor more advanced usage of environment variables with docker-compose,refer to docker-compose docsPro-tipIf you want to connect to a database in the host machine, setCATALOG_HOST: host.docker.internal # For mac or windows
#OR
CATALOG_HOST: 172.17.0.1 # LinuxSupported TechnologiesPostgresAWS RedshiftSnowflakeComing SoonSparkSQLPrestoDocumentationFor advanced usage, please refer todata-lineage documentationSurveyPlease take thissurveyif you are a user or considering using data-lineage. Responses will help us prioritize features better.
Package: dataliner
DataLiner - Data processing package for PythonA dataprocessing package for data preprocess and feature engineering.Please feel free to send pull requests for bug fix, improvements or new preprocessing methods!Installation! pip install datalinerDocumentationhttps://shallowdf20.github.io/dataliner/preprocessing.htmlQuick StartTrain data from Kaggle Titanic is used in this example.https://www.kaggle.com/c/titanic/dataimportpandasaspdfromsklearn.pipelineimportmake_pipelineimportdatalinerasdldf=pd.read_csv('train.csv')target_col='Survived'X=df.drop(target_col,axis=1)y=df[target_col]process=make_pipeline(dl.DropNoVariance(),dl.DropHighCardinality(),dl.BinarizeNaN(),dl.ImputeNaN(),dl.TargetMeanEncoding(),dl.DropHighCorrelation(),dl.StandardScaling(),dl.DropLowAUC(),)process.fit_transform(X,y)
Package: datalines
DataLinesA package to prevent reinventing the wheel every time data needs to be loaded or processed. Aims at operating with the least number of dependencies (for high compatibility and lightweight).Table of contentsDataLinesTable of contentsDocsContentsReferencesDocsPackage documentation resources are listed belowdatalinespackageContentsThe contents of this folder are described belowItem NameDescriptionLICENSELicense:GNU LGPL v3pyproject.tomlProject build systemsetup.pyProject metadataconda-env-full.ymlAnaconda environment (for development and build systems)srcfolderThe main source codeReferencesPackaging Python projects:packaging.python.orgProject page onPyPI
Package: datalinker
This package helps you to share the variables/data across the different Python files in the project/package. Once you set the variables in any Python file (.py) and if you want to access or fetch it back in any other Python file in any location/folder then you can do it easily with this package. You just need to import the datalinker library in all the Python files to use this feature. It is very simple to use this library.Supported data structure:This library supports all the Python in-built data structuresStringNumberListTupleDictionarySetsAdvantages:We need not send arguments/parameters while calling the functions/methodNeed not return any variables/values from the function/method once you set the variables using push()Easy to maintain the variables across the Python filesNeed not use variables as globalEliminates extra headache to maintain the variable and its namesInstallationRun the following to installpip install datalinkerUsageImport and create an instanceimport datalinker
data = datalinker.DataLinker().run()Setting the dataWhile pushing/setting the data you have to provide a unique variable name so that you can access it while fetching it back with the same variable name.data.push(variable_name, variable_value)Getting the dataWhatever variable name you have provided in push() can be fetched using pop() as belowdata.pop(variable_name)Example:The following example will set the variable "message" in the test_1.py and retrieve it test_2.py.test_1.py#!/usr/bin/python
# Import datalinker package
import datalinker

# Create an object using datalinker library
data = datalinker.DataLinker().run()

import test_2

if __name__ == '__main__':
	# Setting the data
	data.push("message", "Hello World!!")

	# Calling test_2.py without any arguments
	test_2.print_msg()test_2.py#!/usr/bin/python
# Import datalinker package
import datalinker

# Create an object using datalinker library
data = datalinker.DataLinker().run()

# Observe that no argument has been received in the function definition
def print_msg():
    # Getting back the data
    msg_data = data.pop("message")
    print(f"The message is: {msg_data}")Now, change the path to current directory and run the test_1.pypython test_1.pyOutput:The message is: Hello World!!NOTE:Its case-sensitive hence be careful while pushing and popping the variables namesSuppose if have not set the data and if you try to access that using the pop() method then it will return "None"Once the code execution completes then the variables will be destroyed
Package: dataloader-fetcher
DataloaderA python module for loading datasets fromScikit-learnTorchvisionIt supports loadingWineIrisMNIST datasetsDefaults to loading IRIS.This module also performs pre-processing relevant to each data set type.
It handles categorical data for tabular sets like Iris and it performs transforms for images in the computer vision dataset like MNIST.InstallUsing a package manager$ poetry add dataloader-fetcherExample usagefetcher = DataloaderFetcher()
train_loader = fetcher.train_loader(name="Iris")
test_loader  = fetcher.test_loader(name="Iris")
Package: data-loader-xyz
Version actuelleversion = 1.0.0UtilisationLe projet pourrait etre utile pour:Travailler un projet sur un template bien défini (git clone)créer un package dans pypi (make all)Passer la partie ci dans gitLab (ajouter gitlab_ci.yaml et faite réference au Makefile)Pour utiliser le template.git clone https://github.com/Proxia-ai/template_vierge.gitPrise en mainDefinir les parametre projet dans configs.?.yamlTravailler avec /.data pour stoquer les dataAppeler d'autres asset en se basant sur ce projet, il va vous falloir juste de mettre le nouveau Yaml dans configsvous pourriez ajouter d'autre dossier comme par exemple .secrets (selon la nature du projet)Contenu du projetLe template contient deja un exemple:le dossier .data (facultatif) :où on stoke les donnéela réference à ce dossier est dans configs/config.inile dossier configs :init.py : utile pour en faire une librairie interne surtout lors du build (à ne pas toucher)base_config.py: il contient les réference vers les fichiers de configuration d'une maniére génerique ( à toucher avec précaution)config.ini : il contient l'emplacement des dossier et fichier utile pour le projet. Il faut mettre des liens indépendante de l'emplacement machine?_config.yaml: il contient les parametres du projet. Il ne faut pas toucher à la partie meta-information. Le reste dépond du projetdocs : Il contient l'ensemble de la documentation qui se génere automatiquement à partir des docs string dans les différents scripts?/src : Il contient le travail çàd le source code du projetinit.py: à ne pas toucher utile lors du buildtests/?: Un dossier qui contient l'ensemble des tests unitaires à en faire sur les données qui sont dans .datautils : Il contient l'enseble des fonction génerique au projetinit.py : à ne pas supprimer.gitignore : contient l'enseble des dossiers et fichiers à ignorer (à toucher selon le projet).pylintrc : L'ensemble des regles à verifier lors de l'étape lintage du codeLICENCE : Contient une licence Standard, utile pour pypimain.py : Le script de lancement par defaut si pas d'utilisation des source. mettez juste le nom du projet dans la console et il va lancer ce script tout seulMakefile : Ensemble d'instruction utile pour la partie CiREADME.md : documentation pour bien utiliser le projet à lirerequirements.txt : Il est recommandé d'extraire ce fichier requirements.txt avant un push gitsetup.cfg : fichier utile pour le buildsetup.py: fichier d'installation du projetNB:Dans cette version il faut placer les dépendances dans setup.py si vous voulez builder un package. Cette partie n'est encore automatiquepour le build dans pypi, il faut créer un fichier de réferences qui contient les identifiants et les réference d'authentificationUtilisation en tant que packageexemple:pip install -i https://test.pypi.org/simple/ template-vierge==0.0.3.35L'upgrade du package de l'autre coté sera comme ça:pip install -i https://test.pypi.org/simple/ template-vierge --upgrade --no-cache-diril faut aussi pense à mettre le dossier site-package accessible à tous les users:sudo chmod -R ugo+rX /home/zied/PycharmProjects/pythonProject_test/venv/lib/python3.10/site-packages
Package: data-locality-system
# data-locality-systemDLS is a library and pattern that tries to keep data and processing near together for latency and performance.# UsageSplit up your API request handler into a series of methods with each method being responsible for some API call or processing.AutomaticallyOpenTracing wraps all your spansRequests are routed to servers which are favourableRequests run in parallel using threads.# UsageRun each DLS service using Gunicorn.
Package: datalog
# DataLog: a library for interacting with PicoLog hardwareThis is a Python library to interface with [PicoLog ADC hardware](https://www.picotech.com/data-logger/adc-20-adc-24/precision-data-acquisition).It provides a Python class to interact with the `ADC24`, and data structuresto represent its outputs. The `ADC24` is implemented, but the library isdesigned to be easily extended for other PicoLog (and potentially non-PicoLog)hardware. It may work with the `ADC20` without modification, but this is nottested.## PrerequisitesThe following requirements must be met before `DataLog` is installed:* Python 3.5+* PicoLog ADC 20/24 hardware and driver (`libpicohrdl`)The PicoLog ADC driver can be obtained from[PicoTech](https://www.picotech.com/downloads/) and must be installed andworking before using this library.## InstallationInstallation is handled by `setup.py`. This is most easily handled by `pip`:```bashpip3 install datalog```This installs the Python package dependencies automatically. If the above command doesn't make sense, try reading[this introduction to pip](https://packaging.python.org/tutorials/installing-packages/).## UseThe documentation should help to get you started. Compile it with:```bashcd docmake html```The main functionality is implemented in `datalog/adc/hrdl/picolog.py` and thisclass can be used on its own.### Site-specific useWe use this library at the [University of Glasgow observatory](http://www.astro.gla.ac.uk/observatory/weather/Observatory_weather/Observatory_weather.htm)for our magnetometer. We have to calibrate the raw counts coming from thePicoLog unit, as well as upload this data to our FTP server. The set of scriptswe use can be found [here](https://github.com/acrerd/magnetometer).## ContributingI welcome contributions to the codebase - just open a pull request!Sean Leaveyhttps://github.com/SeanDS/
Package: datalogd
datalogd is a data logging daemon service which uses a source/filter/sink plugin architecture to
allow extensive customisation and maximum flexibility. There are no strict specifications or
requirements for data types, but typical examples would be readings from environmental sensors such
as temperature, humidity, voltage or the like.The user guide and API documentation can be read online atRead the Docs. Source code is available onGitLab.Custom data sources, filters, or sinks can be created simply by extending an existing~datalogd.DataSource,~datalogd.DataFilter, or~datalogd.DataSinkpython
class and placing it in a plugin directory.Data sources, filters, and sinks can be arbitrarily connected together with a connection digraph
described using theDOT graph description language.Provided data source plugins include:LibSensorsDataSource- (Linux) computer motherboard sensors for temperature, fan speed,
voltage etc.SerialDataSource- generic data received through a serial port device. Arduino code for
acquiring and sending data through its USB serial connection is also included.RandomWalkDataSource- testing or demonstration data source using a random walk algorithm.ThorlabsPMDataSource- laser or light power measurement using the Thorlabs PM100 or PM400
power meter.PicoTC08DataSource- thermocouple or other sensor measurements using the Pico Technologies
TC-08 USB data logger.Provided data sink plugins include:PrintDataSink- print to standard out or standard error streams.FileDataSink- write to a file.LoggingDataSink- simple output to console using python logging system.InfluxDB2DataSink- InfluxDB 2.x database system specialising in time-series data.MatplotlibDataSink- create a plot of data using matplotlib.PyqtgraphDataSink- plot incoming data in realtime in a pyqtgraph window.Provided data filter plugins include:SocketDataFilter- bridge a connection over a network socket.KeyValDataFilter- selecting or discarding data entries based on key-value pairs.TimeStampDataFilter- adding timestamps to data.AggregatorDataFilter- aggregating multiple data readings into a fixed-size buffer.CSVDataFilter- format data as a table of comma separated values.PolynomialFunctionDataFilter- apply a polynomial function to a value.FlowSensorCalibrationDataFilter- convert a pulse rate into liquid flow rate.CoolingPowerDataFilter- calculate power dissipation into a liquid using temperatures and
flow rate.See the Data Logging Recipes section in the documentation for examples of how to link various data
sources, filters, and sinks to make something useful.
Package: datalogy
# datalogy[![Build Status](https://secure.travis-ci.org/michaeljoseph/datalogy.png)](http://travis-ci.org/michaeljoseph/datalogy)
[![Stories in Ready](https://badge.waffle.io/michaeljoseph/datalogy.png?label=ready)](https://waffle.io/michaeljoseph/datalogy)
[![pypi version](https://badge.fury.io/py/datalogy.png)](http://badge.fury.io/py/datalogy)
[![# of downloads](https://pypip.in/d/datalogy/badge.png)](https://crate.io/packages/datalogy?version=latest)
[![code coverage](https://coveralls.io/repos/michaeljoseph/datalogy/badge.png?branch=master)](https://coveralls.io/r/michaeljoseph/datalogy?branch=master)![datalogy](https://github.com/michaeljoseph/datalogy/raw/master/resources/datalogy.jpg)A collection of python tools that facilitate the obtaining, scrubbing
and exploring of data.## scrapeExtract HTML elements using an XPath query or CSS3 selector.### Example usagecurl -shttp://en.wikipedia.org/wiki/List_of_sovereign_states|scrape -be ‘table.wikitable > tr > td > b > a’## random-sampleOutput lines from stdin to stdout with a given probability, for a given
duration, and with a given delay between lines.### Example usageseq 100 | random-sample -r 20% -d 1000## TestingInstall development requirements:pip install -r requirements.txtRun the tests with:nosetestsLint the project with:flake8 datalogy tests## API documentationTo generate the documentation:cd docs && PYTHONPATH=.. make singlehtml