Saravanan666/pypi-org huggingface dataset:
Package: datahub-py
Datahub-pyPip version of Datahub coreEmitter focused for now, with buffered queue if RabbitMQ isn't running on the local network, rabbitMQ auto-connect, buffer delivery on connection.Service auto-discovery on the entire wifi network via /specs endpointProject Integrationfrom datahub_py import Project
import config

project = Project(config)
project.beacon.emit("my_queue", { "hello": "world" })

project.beacon.helper.get_local_ip_address()
project.beacon.helper.find_emitters_and_consumers()
project.beacon.helper.find_on_ip("192.168.0.3")
project.beacon.helper.get_available_port()
project.beacon.helper.get_rabbitmq_local_ip()
project.beacon.callService(address, port, endpoint, params)Build & deploy from sourcepip install -e .
  python setup.py sdist
  twine upload dist/datahub_py-0.0.14.tar.gz
  twine upload dist/datahub_py-{version}.tar.gz

  pip uninstall datahub_py && pip install datahub_py
Package: datahub-sdk
Introdatahub-python-sdk is a Python client implemented most of the APIs to control DataHub Service.Homepagedatahub-python-sdk’s documentation.ExamplesfrommacacaimportDataHub,DataHubSDKdatahub=DataHub(hostname='127.0.0.1',port='9200')datahub.switch_scenes(hub='sample',pathname='test1',data={'currentScene':'scene1'})datahub.switch_all_scenes(hub='sample',data={'currentScene':'default'})ChangelogDetails changes for each release are documented in theHISTORY.rst.ContributingSee CONTRIBUTING.rstLicenseMIT
Package: datahubtool
DatahubToolToolkit for batch management of data inDuraMAT Datahub, like upload and delete.NOTE: onlyauthorised users(API Key required, shown on user page of Datahub) can manage its owned project data in Datahub.Installationpip install datahubtoolPackage overviewHere is a high level overview of the important functions of the package.'get_local_file_names': Get the names of local files to upload in a given path'get_Datahub_file_names': Get all existing files's name in a given package in Datahub'upload_files': Upload files to Datahub'delete_Datahub_files': Delete files in DatahubAuthorsBaojie Li (LBNL)
Package: datahugger
Datahugger - Where DOI :open_hands: DataDatahugger is a tool to download scientific datasets, software, and code from a large number of repositories based on their DOI(wiki)or URL. With Datahugger, you can automate the downloading of data and improve the reproducibility of your research. Datahugger provides a straightforwardPython interfaceas well as an intuitiveCommand Line Interface(CLI).Supported repositoriesDatahugger offers support for more than377 generic and specific (scientific) repositories(and more to come!).We are still expanding Datahugger with support for more repositories. You can
help byrequesting support for a repositoryin the issue
tracker. Pull Requests are very welcome as well.InstallationDatahugger requires Python 3.6 or later.pip install datahuggerGetting startedDatahugger with PythonLoad a dataset (or any digital asset) from a repository with thedatahugger.get()function. The first argument is the DOI or URL,
and the second is the folder name to store the dataset (it will be
created if it does not exist).The following code loads dataset10.5061/dryad.mj8m0into
the folderdata.importdatahugger# download the dataset to the folder "data"datahugger.get("10.5061/dryad.mj8m0","data")For an example of how this can integrate with your work, see theexample workflow notebookorDatahugger with command lineThe command line functiondatahuggerprovides an easy interface to download data. The first
argument is the DOI or URL, and the second argument is the name of the folder to store the dataset (will be
created if it does not exist).datahugger10.5061/dryad.mj8m0data%datahugger10.5061/dryad.mj8m0data
Collecting...
NestTemperatureData.csv:100%|████████████████████████████████████████|607k/607k
README_for_NestTemperatureData.txt:100%|██████████████████████████████████████|2.82k/2.82k
ExternalTemps.csv:100%|██████████████████████████████████████|1.06k/1.06k
README_for_ExternalTemps.txt:100%|██████████████████████████████████████|2.82k/2.82k
InternalEggTempData.csv:100%|██████████████████████████████████████████|664/664
README_for_InternalEggTempData.txt:100%|██████████████████████████████████████|2.82k/2.82k
SoilSimulation_Output.csv:100%|████████████████████████████████████████|229M/229M
README_for_SoilSimulation_[...].txt:100%|██████████████████████████████████████|2.82k/2.82k
Datasetsuccessfullydownloaded.Tip:On some systems, you have to quote the DOI or URL. For example:datahugger "10.5061/dryad.mj8m0" data.Tips and tricksNo need to struggle with DOIs versus DOI URLs. They both work (and more). Example: The values10.5061/dryad.x3ffbg7m8,doi:10.5061/dryad.x3ffbg7m8,https://doi.org/10.5061/dryad.x3ffbg7m8, andhttps://datadryad.org/stash/dataset/doi:10.5061/dryad.x3ffbg7m8all point to the same dataset.Do not republish the dataset when uploading your data to a scientific data repository. These storage resources can be used better :)LicenseMITContactPlease feel free to reach out with questions, comments, and suggestions. Theissue trackeris a good starting point. You can also email me atjonathandebruinos@gmail.com.
Package: dataicer
dataicer -Ice (save) your data and high level objects for use later.Do you have complex classes or objects that you want to save to disk and reinstate
later? Do you want to use a data structures natural save methods? Do you want it
to be easy and manageable, capturing key information so you can come back and load
your data again later if you need to?dataicercan help you with all this. Build on top ofjsonpickle,dataicerallows you to create a central handler (just for a directory at the moment) where
Python objects can be saved injsonformat. However, whilejsonformat might
be ok for small objects or simple types it is not great fornumpy.ndarrayorpandas.DataFrameorxarray.Datasetcomplex structures. Complex structures also
come with their own way of saving information anddataicerleverages this on top
ofjsonpickleto create portable and recreatable saved Python state.InstallationInstallation usingpipvia the source directory.pip install .or install from PyPipip install digirockUsageFirst, create a newDirectoryHandlerclass. This points at the archive folder
you want to use.If you have speical classes you need to pickle they need a special handler. Dataicer includes handlers fornumpy.ndarray,xarray.Dataarrayandxarray.Datasetandpandas.DataFrame. Handlers are unique to theDirectoryHandlerinstance.from dataicer import DirectoryHandler, get_numpy_handlers, get_pandas_handlers, get_xarray_handlers

handlers = get_pandas_handlers()
handlers.update(get_xarray_handlers())

dh = DirectoryHandler("my_archive", handlers, mode="w")Numpy arrays can be saved in single column"txt","npy"binary, or"npz"compressed.
Xarray structures can only be saved as"nc"netcdf.
Pandas DataFrames can be saved as"h5"hdf5 or"csv"text files.Objects are then passed to theicefunction of theDirectoryHandleras keyword arguments.import numpy as np
import xarry as xr
import pandas as pd

dh.ice(nparr=np.zeros(10), df=pd.DataFrame(data={"a":[1, 2, 3]}), xarrds=xr.tutorial.scatter_example_dataset())dataicerwill create the directorymy_archiveand place three files identified via a uuid
in the directory for each object. There is also a JSON file with the key name containing all
the meta information for the object saved and ameta.jsonfile which contains information
about the system state at the time the archive was created.Thedeicecommand can be used to reload all of the arguments into a dictionary.state = dh.deice()
state["nparr"]

    array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])
Package: dataify
Module providing common Datasets for PyTorch.Home-page:https://github.com/neurospin-deepinsight/dataifyAuthor:
dataify developersAuthor-email:antoine.grigis@cea.frLicense: CeCILL-B
Description: Module providing common Datasets for PyTorch.Platform: OS Independent
Classifier: Development Status :: 5 - Production/Stable
Classifier: Environment :: Console
Classifier: Environment :: X11 Applications :: Qt
Classifier: Operating System :: OS Independent
Classifier: Programming Language :: Python
Classifier: Topic :: Scientific/Engineering
Classifier: Topic :: Utilities
Package: dataiku-api-client
API client for Dataiku Data Science StudioFor more information, see:https://doc.dataiku.com/dss/latest/python-api/rest-api-client/index.htmlChangelog12.5.2 (2024-02-26)Initial release for DSS 12.5.212.5.0 (2024-01-23)Initial release for DSS 12.5.012.4.2 (2024-01-15)Initial release for DSS 12.4.212.4.0 (2023-12-06)Initial release for DSS 12.4.012.3.2 (2023-11-16)Initial release for DSS 12.3.212.2.3 (2023-10-12)Initial release for DSS 12.2.312.1.0 (2023-06-29)Initial release for DSS 12.1.012.0.1 (2023-06-29)Initial release for DSS 12.0.112.0.0 (2023-06-13)Initial release for DSS 12.0.011.4.1.1 (2023-04-06)Initial release for DSS 11.4.1.111.4.1 (2023-04-06)Initial release for DSS 11.4.111.4.0 (2023-03-16)Initial release for DSS 11.4.011.1.0 (2022-10-04)Initial release for DSS 11.1.011.0.0 (2022-07-12)Initial release for DSS 11.0.09.0.0 (2021-05-07)Initial release for DSS 9.08.0.0 (2020-08-14)Initial release for DSS 8.07.0.0 (2020-07-21)Initial release for DSS 7.05.1.0 (2019-03-01)Initial release for DSS 5.15.0.0 (2018-07-26)Initial release for DSS 5.04.3.0 (2018-06-01)Initial release for DSS 4.34.2.1 (2018-04-30)Initial release for DSS 4.24.1.0 (2017-11-10)Initial release for DSS 4.14.0.0 (2017-01-10)Initial release for DSS 4.03.1.4 (2017-01-03)Pypi packaging fixes3.1.3 (2017-01-03)Fix user creation3.1.2 (2016-08-23)Add proper Pypi tags3.1.0 (2016-08-22)Initial release for DSS 3.1
Package: dataiku-scoring
# Dataiku ML Scoring Python libraryThis repository contains the Python library to run exported Dataiku models outside of Dataiku.To use this library, you need to first export your model to Python from Dataiku.Changelog12.5.2 (2024-02-26)Initial release for DSS 12.5.212.5.0 (2024-01-23)Initial release for DSS 12.5.012.4.2 (2024-01-15)Initial release for DSS 12.4.212.4.0 (2023-12-06)Initial release for DSS 12.4.012.3.2 (2023-11-21)Initial release for DSS 12.3.212.3.1 (2023-11-21)Initial release for DSS 12.3.112.2.3 (2023-10-16)Initial release for DSS 12.2.312.1.0 (2023-07-06)Initial release for DSS 12.1.012.0.1 (2023-07-04)Initial release for DSS 12.0.111.4.4 (2023-07-04)Initial release for DSS 11.4.411.4.0 (2023-03-16)Initial release for DSS 11.411.1.0 (2022-10-20)Initial release for DSS 11.1
Package: dataimport
data-importimportdata files via import-hook trickeryInstallation$ pip install dataimportUsage>>>importdataimport.test.csvascsv_file>>>print(list(csv_file))[['col1','col2','col3','col4'],['1','2','3','4'],['5','6','7','8'],['9','10','11','12'],['13','14','15','16'],['17','18','19','20']]csvandjsonloaders are included, but adding or replacing a loader is easy!>>>importdataimport>>>importpandas>>>dataimport.register('csv',pandas.read_csv)>>>importdataimport.test.csvasdatatable>>>print(datatable)col1col2col3col4012341567829101112313141516417181920
Package: data-inclusion-schema
Schéma de l'offre d'insertionCe dépôt contient le schéma de l'offre d'insertion. Il est issu de la collaboration d'acteurs de l'insertion.ContexteLa démarche a été initiée par l'équipeDora, portée le GIP Plateforme de l'inclusion.Il a été constaté l'existence d'une multitude de portails en ligne ou simulateurs, mais pas de langage commun pour décrire l’offre d’insertion et peu de partage des informations. La diffusion des aides et des dispositifs en est diminuée. Des informations sur les aides ou dispositifs deviennent obsolètes. Les efforts de saisie et de développement sont multipliés.Ce schéma a été créé avec la collaboration des acteurs de l'insertion afin d'harmoniser les données et de permettre l'intéropérabilité entre les différents acteurs et leurs outils.Le schémaLe schéma définit les structures de l'insertion et devraient à terme inclure les services proposés par ces structures. Il comprend les informations essentielles permettant  de décrire l'offre d'insertion et la réutilisation par les acteurs de l'insertion.Le schéma est pour le moment formalisé dans le format JSON schema. Nous veillions néanmoins à maintenir la possibilité de représenter les données au format csv.La documentation détaillée du schéma est disponibleici.Consolidationdata.inclusion réalise une consolidation et publie la base consolidée, aux formats json et csv, surdata.gouv.fr.Aide à la saisieLes structures qui partagent leurs données via l'outilDorasont incluses par défaut dans la base consolidée.ContribuerNous organisons des ateliers régulièrement pour définir et améliorer le schéma de données. Vous pouvez librement participer aux prochains ateliers en vous inscrivantici.Vous pouvez également créer unnouveau ticketsur le dépôt ouvert du schéma pour proposer une modification ou une amélioration ou poser une question.
Package: datainfo
Changed the README format from Markdown to rST.DescriptionThis code can be used to explore data with the following format:HDF4HDF5netCDF and netCDF4IDL save fileDependenciesPython 3.6h5pypyhdfscipy.ionetCDF4How to installOpen up a terminal,pip3 install datainfoHow to useFor example, we have a HDF4 file named “MCD43C3.A2015337.005.2015365205333.hdf”.
Open up a terminal, type in:lss MCD43C3.A2015337.005.2015365205333.hdfThe variable information will be displayed as the following:+ HDF4
Albedo_BSA_Band1 ----- : Dataset  (3600, 7200)
Albedo_BSA_Band2 ----- : Dataset  (3600, 7200)
Albedo_BSA_Band3 ----- : Dataset  (3600, 7200)
Albedo_BSA_Band4 ----- : Dataset  (3600, 7200)
Albedo_BSA_Band5 ----- : Dataset  (3600, 7200)
Albedo_BSA_Band6 ----- : Dataset  (3600, 7200)
Albedo_BSA_Band7 ----- : Dataset  (3600, 7200)
Albedo_BSA_nir ------- : Dataset  (3600, 7200)
Albedo_BSA_shortwave - : Dataset  (3600, 7200)
Albedo_BSA_vis ------- : Dataset  (3600, 7200)
Albedo_WSA_Band1 ----- : Dataset  (3600, 7200)
Albedo_WSA_Band2 ----- : Dataset  (3600, 7200)
Albedo_WSA_Band3 ----- : Dataset  (3600, 7200)
Albedo_WSA_Band4 ----- : Dataset  (3600, 7200)
Albedo_WSA_Band5 ----- : Dataset  (3600, 7200)
Albedo_WSA_Band6 ----- : Dataset  (3600, 7200)
Albedo_WSA_Band7 ----- : Dataset  (3600, 7200)
Albedo_WSA_nir ------- : Dataset  (3600, 7200)
Albedo_WSA_shortwave - : Dataset  (3600, 7200)
Albedo_WSA_vis ------- : Dataset  (3600, 7200)
BRDF_Quality --------- : Dataset  (3600, 7200)
Local_Solar_Noon ----- : Dataset  (3600, 7200)
Percent_Inputs ------- : Dataset  (3600, 7200)
Percent_Snow --------- : Dataset  (3600, 7200)
-
Package: data-inspector
Data InspectorData Inspector is an open-source python library that brings 15++ types of different functions to make EDA, data cleaning easier.Author: Kazi Amit HasanProject Description:Data Inspector brings 15++ essential exploratory data analysis, data cleaning automations to make a dataset understandable. This is a perfect tool to get started with you data.Installation:pip install data-inspectorPackage available athttps://pypi.org/project/data-inspector/Available automation:Line plot :line_plot(data, x_data, y_data, x_label="", y_label="", title="")Skew feature:plot_skewed_feature(data, column)Showing data distribution:show_distribution(data, column)Scatter plot:plot_scatter(data,x_data, y_data)Correlation plot:plot_correlation(data)Create histogram:histogram(data,column, x_label, y_label, title)Create bar plot:plot_bar(data, column, xlabel, ylabel, title)Create boxplots of all features:box_plot(data)Checking dataset's shape:datasetShape(data)Get dataset's diagnostic plots:diagnostic_plots(data, variable)Divide numerical and categorical features:divideFeatures(data)Fill NaN values:fillNan(data, column, value)Get pearson's correlation between two variables:get_correlation(column_1, column_2, data)Plotting kde plots:plot_cont_kde(data, var)Automatic calculating the missing values and their percentage along with visualization :calculating_missing_values(data)Regression plot with 95% CI :plot_regplot(data,x_data, y_data)Tutorial:Link:https://github.com/AmitHasanShuvo/data-inspector/blob/main/notebook/example%20notebook.ipynbColab link:https://colab.research.google.com/drive/1mj9gz2XyQprSYdKMUKlKkJ9Qi8XmleHW?usp=sharingSome visualizations:Available at:https://github.com/AmitHasanShuvo/data-inspectorHow to cite:@online{data-inspector,title={data-inspector},url={https://pypi.org/project/data-inspector/},urldate={2021-08-21},publisher={Kazi Amit Hasan}}Future Works:Add some automations for time series data.How to contribute:Any contribution would be highly appreciated. Kindly go through the guidelines for contributing in github.Change Log0.0.1 (20/08/2021)First Release0.0.2 (20/08/2021)Minor updates0.0.3 (20/08/2021)Minor updates0.0.4 (20/08/2021)Minor updates0.0.5 (20/08/2021)Minor updates0.0.6 (20/08/2021)Minor updates0.0.8 (21/08/2021)Minor updates1.1 (21/08/2021)Minor updates1.5.4 (29/08/2021)Regression plot functions added
Package: datainventory
Data Inventory is a database designed for managing machine learning models. It is also suitable for storing images and videos.RequirementsPython 3.7or newer is required.InstallationThere are a few ways to installData Inventory.Install the latest release from PyPIpipinstalldatainventoryInstall from sourcegitclonehttps://github.com/iot-spectator/datainventory.gitcddatainventorypipinstall.
Package: data-inventory-tool
With this tool you can inspect various data sources and generate data
inventory tables. Data inventory table essentially contains list of tables
and table columns of a specified data source.Inventory tables are designed to be easily readable and editable using a
spreadsheet program.Here is an example how you can generate inventory table from a CSV file:inventory-table inspect cvs data.cvsHow to installIn order to install this tool, first you need to installPython. When you
have Python installed, then install data inventory tool like this:pip install data-inventory-toolGenerating inventory tablePostgreSQLFor PostgreSQL first you need to install PostgreSQL driver:pip install psycopg2-binaryThen generate inventory table like this:inventory-table inspect sql postgresql://user:password@host:port/dbMySQLFor MySQL first you need to install MySQL driver:pip install mysqlclientThen generate inventory table like this:inventory-table inspect sql mysql+mysqldb://user:password@host:port/dbTroubleshootingIf you get errors like this:Error: Database driver not found: No module named 'psycopg2'.
See: https://docs.sqlalchemy.org/en/13/dialects/index.htmlThat means, you don’t have a required database driver installed.
Package: dataIO
Welcome to the dataIO DocumentationdataIOprovides lots of utility method working on IO in betweenstring,bytes,file,json,pickle,compress and decompress. Most of the cases you can do your workin single line of code.dataIOwritten in pure python, no other dependency required.Now it support:gzipjsonpickletext fileQuick links:GitHub HomepageOnline DocumentationPyPI downloadInstallIssue submit and feature requestAPI reference and source codeInstalldataIOis released on PyPI, so all you need is:$pipinstalldataIOTo upgrade to latest version:$pipinstall--upgradedataIO
Package: dataip
dataipdataip is used to extract some information based on your IP address.Installationinstall using pippipinstalldataipUsageimportdataip
print(dataip.getIP())this will print your current IP addressFunctionsdataip supports the following commandsdataip.getIP()dataip.continent()dataip.continentCode()dataip.country()dataip.countryCode()dataip.region()dataip.regionCode()dataip.city()dataip.zip()dataip.latitude()dataip.longitude()dataip.timezone()dataip.offset()dataip.currencyCode()dataip.isp()dataip.org()dataip.asname()Othersyou can also calldataip.info()to learn about the project ordataip.debug()to learn about some common errors and their solutionsmade byAditya Khemka
Package: dataiter
Python Classes for Data ManipulationDataiter currently includes the following classes.DataFrameis a class for tabular data similar to R'sdata.frameorpandas.DataFrame. It is under the hood a dictionary of NumPy arrays
and thus capable of fast vectorized operations. You can consider this to
be a light-weight alternative to Pandas with a simple and consistent
API. Performance-wise Dataiter relies on NumPy and Numba and is likely
to be at best comparable to Pandas.ListOfDictsis a class useful for manipulating data from JSON
APIs. It provides functionality similar to libraries such as
Underscore.js, with manipulation functions that iterate over the data
and return a shallow modified copy of the original.attd.AttributeDictis used to provide convenient access to dictionary keys.GeoJSONis a simple wrapper class that allows reading a GeoJSON
file into aDataFrameand writing a data frame to a GeoJSON file. Any
operations on the data are thus done with methods provided by the data
frame class. Geometry is read as-is into the "geometry" column, but no
special geometric operations are currently supported.Installation# Latest stable versionpipinstall-Udataiter# Latest development versionpipinstall-Ugit+https://github.com/otsaloma/dataiter# Numba (optional)pipinstall-UnumbaDataiter optionally usesNumbato speed up certain operations. If
you have Numba installed and importing it succeeds, Dataiter will use it
automatically. It's currently not a hard dependency, so you need to
install it separately.Documentationhttps://dataiter.readthedocs.io/If you're familiar with either dplyr (R) or Pandas (Python), the
comparison table in the documentation will give you a quick overview of
the differences and similarities.https://dataiter.readthedocs.io/en/latest/comparison.html