Saravanan666/pypi-org huggingface dataset:
Package: datatables-mongoengine
DataTables with MongoEngineTheDataTablesManagerclass can be used instead of the default MongoEngineQuerySetclass to add adatatablesmethod for returning results as required by the
jQuery plugin DataTables.Installationpip install datatables-mongoengineExampleHere's an example for Flask.models.pyfrom mongoengine import Document, StringField, ListField
from datatables_mongoengine import DataTablesManager


class Links(Document):
    """The MongoEngine ODM class for the links collection."""

    meta = {
        "queryset_class": DataTablesManager
    }
    name = StringField()
    category = StringField()
    link = StringField()
    group = ListField()routes.pyfrom flask import request, g, jsonify

from app import app
from app.models import Links


@app.route("/ajax/links", methods=["POST"])
def ajax_links():
    """Get results from MongoDB for DataTables."""

    data = request.get_json()
    custom_filter = {
        'group': g.user.group
    }

    results = Links.objects.datatables(data, **custom_filter)
    return jsonify(results)Note that you can inject any filter you want server-side, like I do above to make sure
the results all match the current user's group.app.js$(document).ready( function () {
    $('#example').DataTable({
        processing: true,
        serverSide: true,
        ajax: {
            url: '/ajax/links',
            dataSrc: 'data',
            type: 'POST',
            contentType: 'application/json',
            data: function (d) {
                return  JSON.stringify(d)
            }
        },
        columns: [
            { data: 'name'},
            { data: 'category'},
            { data: 'link'}
        ],
    });
Package: datatableview-advanced-search
django-datatable-view-adv-queryThis is a small repo to allow user to use an advanced search in the search window.For example this is now possible:(name="Foo Man" OR name=Bar) AND modified_date >= 12/25/2017Learn morehttps://github.com/icmanage/django-datatable-view-adv-query>.Build Process:Update the__version_info__inside of the application. Commit and push.Tag the release with the version.git tag <version> -m "Release"; git push --tagsBuild the releaserm -rf dist build *egg-info; python setup.py sdist bdist_wheelUpload the datatwine upload dist/*
Package: datatableview-user-columns
Django Datatable view user columns repositoryThis is a small repo to allow user based custom columns.Learn morehttps://github.com/icmanage/django-datatable-view-user-columnsBuild Process:Update the__version_info__inside of the application. Commit and push.Tag the release with the version.git tag <version> -m "Release"; git push --tagsBuild the releaserm -rf dist build *egg-info; python setup.py sdist bdist_wheelUpload the datatwine upload dist/*
Package: datatap
The visual data management platform fromZensors.Join for free atapp.datatap.dev.The dataTap Python library is the primary interface for using dataTap's rich data management tools. Create datasets, stream annotations, and analyze model performance all with one library.DocumentationFull documentation is available atdocs.datatap.dev.Features⚡ Begin training instantly🔥 Works with all major ML frameworks (Pytorch, TensorFlow, etc.)🛰️ Real-time streaming to avoid large dataset downloads🌐 Universal data format for simple data exchange🎨 Combine data from multiples sources into a single dataset easily🧮 Rich ML utilities to compute PR-curves, confusion matrices, and accuracy metrics.💽 Free access to a variety of open datasets.Getting Started (Platform)To begin, select a dataset from the dataTap repository.Then copy the starter code based on your library preference.Paste the starter code and start training.Getting Started (API)Install the client library.pipinstalldatatapRegister atapp.datatap.dev. Then, go toSettings > Api Keysto find your personal API key.exportDATATAP_API_KEY="XXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXX"Start using open datasets instantly.fromdatatapimportApiapi=Api()coco=api.get_default_database().get_repository("_/coco")dataset=coco.get_dataset("latest")print("COCO: ",dataset)Data Streaming ExampleimportitertoolsfromdatatapimportApiapi=Api()dataset=(api.get_default_database().get_repository("_/wider-person").get_dataset("latest"))training_stream=dataset_version.stream_split("training")forannotationinitertools.islice(training_stream,5):print("Received annotation:",annotation)More ExamplesDocumented SamplePytorch Jupyter NotebookSupport and FAQQ. How do I resolve a missing API Key?If you see the errorException: No API key available. Either provide it or use the [DATATAP_API_KEY] environment variable, then the dataTap library was not able to find your API key. You can find your API key onapp.datatap.devunder settings. You can either set it as an environment variable or as the first argument to theApiconstructor.Q. Can dataTap be used offline?Some functionality can be used offline, such as the droplet utilities and metrics. However, repository access and dataset streaming require internet access, even for local databases.Q. Is dataTap accepting contributions?dataTap currently uses a separate code review system for managing contributions. The team is looking into switching that system to GitHub to allow public contributions. Until then, we will actively monitor the GitHub issue tracker to help accomodate the community's needs.Q. How can I get help using dataTap?You can post a question in theissue tracker. The dataTap team actively monitors the repository, and will try to get back to you as soon as possible.
Package: data-tap
data-tapA python wrapper that connects to multiple 3rd party resources for AWS.Google AnalyticsThe GoogleAnalyticsTap connects to the v4 GA api and returns data from Google Analytics.batch_dates:'bool.enablesbatchingofqueriesbydate'start_date:'str."yesterday","today","3_days_ago"or"2020-01-01"etc...'end_date:'str."yesterday","today","3_days_ago"or"2020-01-01"etc...'metrics:-expression:'str.metricname'dimensions:-name:'str.dimensionname'pageSize:'int.pagesize,defaultsto1000'view_ids:-'str.alistofviewtoiterateover'fromdatetimeimportdatetimeimportpandasaspdfromdata_tap.google_tap.google_analyticsimportGoogleAnalyticsTapga_tap=GoogleAnalyticsTap(config_file='',creds_file='',auth_method='',service_account_email='')dfA=pd.DataFrame()forreportinga_tap.run_query():dfB=pd.DataFrame(report)dfA=dfA.append(dfB,ignore_index=True)dfA.to_json(f"{datetime.utcnow()}.json")Google Search ConsoleThe GoogleSearchConsoleTap connects to the v3 GSC api and returns data from Google Search Console.
Functionality to generate an authentication token is available.batch_dates:'bool.enablesbatchingofqueriesbydate'start_date:'str."yesterday","today","3_days_ago"or"2020-01-01"etc...'end_date:'str."yesterday","today","3_days_ago"or"2020-01-01"etc...'dimensions:-'str.dimensionname'metrics:-'str.metricname'search_types:-'str.availablegscsearchtypes'site_urls:-'str.websiteurlsinquestion'row_limit:'int.maxrowlimitis25000'fromdatetimeimportdatetimeimportpandasaspdfromdata_tap.google_tap.google_seach_consoleimportGoogleSeachConsoleTapgsc_tap=GoogleSeachConsoleTap(config_file='',creds_file='.pickle')dfA=pd.DataFrame()forreportingsc_tap.run_query():dfB=pd.DataFrame(report)dfA=dfA.append(dfB,ignore_index=True)dfA.to_json(f"{datetime.utcnow()}.json")
Package: data-tasks
data-tasksSmall library to run data related scriptsSetting up Your data-tasks Development EnvironmentFirst you'll need to install:Git.
On Ubuntu:sudo apt install git, on macOS:brew install git.GNU Make.
This is probably already installed, runmake --versionto check.pyenv.
Follow the instructions in pyenv's README to install it.
TheHomebrewmethod works best on macOS.
TheBasic GitHub Checkoutmethod works best on Ubuntu.
Youdon'tneed to set up pyenv's shell integration ("shims"), you canuse pyenv without shims.Docker Desktop.
On Ubuntu followInstall on Ubuntu.
On macOS followInstall on Mac.Then to set up your development environment:git clone https://github.com/hypothesis/data-tasks.git
cd data-tasks
make services
make helpReleasing a New Version of the ProjectFirst, to get PyPI publishing working you need to go to:https://github.com/organizations/hypothesis/settings/secrets/actions/PYPI_TOKENand add data-tasks to thePYPI_TOKENsecret's selected
repositories.Now that the data-tasks project has access to thePYPI_TOKENsecret
you can release a new version by justcreating a new GitHub release.
Publishing a new GitHub release will automatically triggera GitHub Actions workflowthat will build the new version of your Python package and upload it tohttps://pypi.org/project/data-tasks.Changing the Project's Python VersionsTo change what versions of Python the project uses:Change the Python versions in thecookiecutter.jsonfile. For example:"python_versions":"3.10.4, 3.9.12",Re-run the cookiecutter template:make templateCommit everything to git and send a pull requestChanging the Project's Python DependenciesTo change the production dependencies in thesetup.cfgfile:Change the dependencies in the.cookiecutter/includes/setuptools/install_requiresfile.
If this file doesn't exist yet create it and add some dependencies to it.
For example:pyramid
sqlalchemy
celeryRe-run the cookiecutter template:make templateCommit everything to git and send a pull requestTo change the project's formatting, linting and test dependencies:Change the dependencies in the.cookiecutter/includes/tox/depsfile.
If this file doesn't exist yet create it and add some dependencies to it.
Use tox'sfactor-conditional settingsto limit which environment(s) each dependency is used in.
For example:lint: flake8,
format: autopep8,
lint,tests: pytest-faker,Re-run the cookiecutter template:make templateCommit everything to git and send a pull request
Package: datatc
data-traffic-controlWhhrrrr... Voooooosh... That's the sound of your data coming and going exactly where it belongsWhat is it?data traffic controlis a python package that automates every-day interactions with your data files.
Data files are the building materials we work withevery day, all day. Working with them should beeffortless.Main FeaturesNavigate your data directories with ease, without having to memorize long file paths.Load data files without having to think about it (or looking it up the syntax yet again).Be certain of how a datafile was generated, and rest easy knowing you could reproduce it.Vizualize how all of the files produced by your data experiments are related, and inspect how each one was produced.Installationpip install datatcDocumentationThe official documentation is hosted onreadthedocs.io.
Package: dataterkinigempabumidiindoensia
latest-indonesia-earthquakeThis package will get the latest earthquake indonesia from BMKG | Meteorology Climatology and Geophysics Agency##HOW IT WORK
This package will scrape fromBMKGto get latest earthquake happened in indonesiaThis package will use beautifulsoup4 and requests, to produce output in the form of JSON that is ready to be used in web or mobile applications
Package: datathief
Small utility for retrieving data from figures. Inspired by theJava package of the same name.InstallationThe usual:pip install datathief.UsageUnlike the Java DataThief package and similar online tools, here the usermanuallyannotates the figure with the data points of their choosing. This makes it more transparent how the data are being read and makes the results more reproducible. However, it might be annoying for a large amount of data.If you want to extract a lot of data, or extract data from a continuous line, you are better off using the original Java DataThief package, or one of themanyonlinetoolsthat do exactly this.To use this tool, first annotate the plot by adding asingle pixelat the start and end of the x-axis in a specified color that does not exist anywhere else in the image (default color: pure blue). Do the same for the y-axis (default color: pure red). Then one pixel for each data point you wish to extract (default color: pure green). This function will then return the x and y coordinates of each data point. It will warn you if too many or too few pixels are detected.For example, running this code:importdatathiefasdtfilename='du_fig1a_annotated.png'xlim=[-10,20]ylim=[0,15]data=dt.datathief(filename,xlim=xlim,ylim=ylim)On this input (NB, you might need to zoom in to see the individual pixels):Extracts the data for this plot:See the examples folder for more information. (Figure courtesy Du et al.,https://www.medrxiv.org/content/10.1101/2020.02.19.20025452v4)More questions? Emailinfo@sciris.org.
Package: datatoaster
# datatoaster[![Build Status](https://travis-ci.org/abc612008/datatoaster.svg?branch=master)](https://travis-ci.org/abc612008/datatoaster) [![codecov](https://codecov.io/gh/abc612008/datatoaster/branch/master/graph/badge.svg)](https://codecov.io/gh/abc612008/datatoaster) [![license](https://img.shields.io/github/license/abc612008/datatoaster.svg)](https://github.com/abc612008/datatoaster/blob/master/LICENSE)datatoaster is a Python 3 library that can convert raw data to chart data so that it can used to generate charts easily. If you like this project or find it useful please consider giving me a GitHub star. Much appreciated! :)Suggestions are welcome!## InstallationYou can install it via pip```$ pip install datatoaster```or manually install by```$ python3 setup.py install```You can run unit tests by```$ python3 ./tests/tests.py```## ExamplesGiven a data set, I want to know how many percent of my users paid money (per OS).```python>>> data = [{"PaidAmount": 0, "OS": "iOS"}, {"PaidAmount": 0, "OS": "Android"}, {"PaidAmount": 301, "OS": "Android"}, {"PaidAmount": 0, "OS": "Windows"}, {"PaidAmount": 14, "OS": "Windows"}, {"PaidAmount": 56, "OS": "iOS"}, {"PaidAmount": 2, "OS": "Windows"}, {"PaidAmount": 0, "OS": "Windows"}]>>> datatoaster.DataSet(data) \.set_x(lambda i: i["OS"]) \.set_y(datatoaster.DataSet.PercentageWithinGroup) \.add_constraint(lambda i: i["PaidAmount"] != 0) \.get_result(){'Windows': 0.5, 'iOS': 0.5, 'Android': 0.5}```I can now know 50% users in each OS paid money to me! I can also use this data to draw chart.For more examples, you can run `python3 ./demo/server.py`.## DocumentationDataSet.set_x(DataSet.Single / function): Set X valueDataSet.Single: equivalent to (lambda i:"")function: data_item -> desired_x_valueDataSet.set_y(DataSet.NumberOfAppearance/DataSet.Percentage/DataSet.PercentageWithinGroup/function): Set Y valueNumberOfAppearance: Y value will be the number of appearance of X.Percentage: Y value will be the percentage of appearance of X over the whole data set.PercentageWithinGroup: Y value will be the percentage of appearance of X over the group.function: [data_item] -> y_value / [y_values] Y value will be what your function returns.DataSet.get_result(): Get the final resultExample 1: {"x":y, "x2":y2}Example 2: {"x":[y11,y12], "x2":[y21,y22]}DataSet.add_constraint(function, is_single): Add a constraintfunction: data_item -> boolean (indicates whether the value should be used)is_single: indicates if the constraint applies to "whole data set"(Only makes difference when .SetY(Percentage)), defaults to False.It may seem confusing. You can refer to [unit tests](https://github.com/abc612008/datatoaster/blob/master/tests/tests.py) or [demo](https://github.com/abc612008/datatoaster/blob/master/demo/server.py) for more examples.## Licensesdatatoaster is released under the MIT License. See [LICENSE](https://github.com/abc612008/datatoaster/blob/master/LICENSE) for more information.
Package: data-to-html-connosieurofdoom
data_to_htmlLibrary to convert pandas Dataframes to different html elementsTable:Input:1. Pandas Dataframe (df): The dataframe containing the data to be converted to html table (Required), 
2. column order (col): A list item containing the column names of the dataframe in the order in which they are to be displayed. (Default: None (Returns the columns in the order as in the dataFrame)) | (Optional)
3. The rest is same as pandas to_html: (Refer link)[https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_html.html]Ouput:1. Returns an html formatted table from the dataframeSample:dtl=table(dtl)Select:Input:1. Pandas DataFrame (df): The data frame to be converted to html Select | (Required)
2. text (text): Column to be displayed between the option tage | (Required)
3. value (value): Column for the value attribute in the option tag | (Required)
4. id (id): Value for the id attribute in the select tag | (Optional) | (Default None)
5. classes (classes): Values for the classes attribute for html| Type : List |(Optional) ! (Default None)
6. placeholder (placeholder): Value for placeholder attribute | (Optional) | (Default None)
7. additional_attribute: Any additional attributes | (Optional) | (Default "")
8. multiple: Optional | Default False
9. onchange : Optional | Default NoneOutput:1.	A html format select statement with the options as the column values of the dataframesdtl=select(dtl,text="col1",value="col2",id="id",classes=["class1","class2"],placeholder="data-placeholder")
Package: datatonic-pipeline-components
Datatonic Pipeline ComponentsDatatonic Pipeline Components (DTPC) is a set of Kubeflow (KFP) components that can be run on any KFP pipeline execution backend. The components can be composed together into pipelines using the Kubeflow Pipelines SDK.What is this?Datatonic Pipeline Components is a library of reusableKubeflow Pipelinecomponents. These components have been designed and open sourced to make pipeline
development easier and more enjoyable. The components are well tested and the containers
on which they are built are scanned for vulnerabilities so that you can have confidence
in their performance and security.InstallationInstall using pip:pip install datatonic-pipeline-componentsComponentsHere we list out the components that are available via the library. See theHow to usesection
for an example of using a component in a pipeline.load_huggingface_torch - Loads a PyTorch model, its configuration, and its tokenizer from HuggingFace and saves the artifacts in Google Cloud Storage.Detailed Documentation.load_huggingface_tensorflow - Loads a TensorFlow model, its configuration, and its tokenizer from HuggingFace and saves the artifacts in Google Cloud Storage.Detailed Documentation.upload_pytorch_model - Takes as input a trained PyTorch model artifact stored in GCS and
outputs a Vertex Model uploaded in Vertex Model Registry and ready to serve predictions
in an Endpoint or via Batch Predictions.Detailed Documentation.xgboost_shap_gpu - Computes feature attributions of an XGBoost model using GPU accelerated SHAP.Detailed Documentation.gpt_tokenize - Generates a tokenised training and validation dataset from a given dataset.Detailed Documentation.How to useInclude any components in the library in your pipelines using the following pattern:fromkfp.dslimportpipelineimportdatatonic_pipeline_componentsasdtpc@pipelinedefmy_pipeline():dtpc.load_huggingface_tensorflow(model_class_name="TFAutoModel",config_name="AutoConfig",tokenizer_class_name="AutoTokenizer",model_name="bert-base-cased",)The following figure illustrates that the library can be installed from PyPI, its components
can be included in your pipeline code, and the container images upon which the components
are built will be pulled from the corresponding dtpipelinecomponents dockerhub repository:ContributingWe are an open-source project and welcome contributions. This may be in the form of new components, bugfixes or better documentation.Seeherefor our guide on how to contribute.
Package: datatoolbox
DATATOOLBOXThis is a python package for handling global datasets. It contains the following features:Augumented pandas DataFrames adding meta dataAutomatic unit conversion and dataframe based computationsID based data structureThis package is under development and serves as a collection of tools around various data analysis packages.
The package is developed in-house and supported by Climate Analytics gGmbH and therefore mainly used in the context of climate change mitigation.The included csv-based git database structure allows multi-user access to unified and version-controlled data sets.
Data access is locally controlled by define data IDs and globally via dataset-IDs.Authors:Andreas GeigesJonas HörschGaurav GantiDependenciespandasnumpygitpythonopenscm-unitspint==0.11pycountryfuzzywuzzytqdmmatplotlibopenpyxlpyam-iamc<=0.8.0hdx-python-countrynetworkx>=2.4.0xarraydeprecatedpyarrowInstallation via pipUsing pip:

pip install datatoolbox

Using conda

conda install datatoolboxRead the docshttps://datatoolbox.readthedocs.io/en/latest/TestingFrom the root directory, run:pytest
Package: data-toolkit
ML & data helper code!Jakub Langr (c) 2021This is a CLI utility for speeding up basic Data Engineering/Data Science tasksusage: dt [-h] [-d] [-q] [-v]
          {cai,config,ec2,gcp,gh,hist,load,monitor,nf,py,run,s3,sftp,sg,ssh,td,viz}
          ...

ML & data helper code!

optional arguments:
  -h, --help            show this help message and exit
  -d, --debug           full application debug mode
  -q, --quiet           suppress all console output
  -v, --version         show program's version number and exit

sub-commands:
  {cai,config,ec2,gcp,gh,hist,load,monitor,nf,py,run,s3,sftp,sg,ssh,td,viz}
    cai                 Ops with Codex AI from OpenAI.
    config              Operations with config.
    ec2                 AWS EC2 helper commands.
    gcp                 GCP VM Instances helper commands.These are currently
                        in alpha and some may not work
    gh                  All Github related operations
    hist                Operations on CLI hist
    load                Appends defaults: from zshrc.txt to ~/.bashrc or
                        ~/.zshrc or tmux conf
    monitor             Monitor lack of GPU activity. When there is none, runs
                        -job
    nf                  Displays the number of files
    py                  (WIP/Pre-Alpha) Execute a Python command across all
                        files in current dir.
    run                 Operations with snippets
    s3                  Operations with s3 buckets
    sftp                Operations to easily work with remote servers /
                        devices
    sg                  AWS Security Groups helper functions
    ssh                 Operations on the SSH config
    td                  Manage TODOs using Google Keep
    viz                 Basic viz using streamlit for image comparisons

Usage: dt command [args] [kwargs]The screen above can be called using:dt -hFor example, to run to list the running ec2 isntances rundt ec2 ls --profile <profile>Installation$ pip install data-toolkitTips and examplesPersonally I find most helpful the following commands:dt ec2 lsShows running EC2 instances with a range of parameters depending on the width of your terminal.dt sg lsShows an overview of all security groups in the current region.dt sg showShows detail of either all or one security group in the current region.dt ssh updateUpdates the SSH config with with the IP addresses of the running EC2 instances (matches them by name/tag).When I'm travelling I like to use the following commands:dt sg update -n <sg_name>  -m "22:jakub 4g"Which updates the<sg_name>security group to allow SSH from the current IP address and port 22 and sets the description to "jakub 4g".Hope this helps!DevelopmentThis project includes a number of helpers in theMakefileto streamline common development tasks.Environment SetupThe following demonstrates setting up and working with a development environment:### create a virtualenv for development

$ make virtualenv

$ source env/bin/activate


### run dt cli application

$ dt --help


### run pytest / coverage

$ make test
Package: datatool-python
datatool-pythonA python library that contains a number of useful functions and approaches
for a data scientist. It includes time measurement tool, map-reduce,
clasterization on CUDA,LCSand so on. Look through the source code for details.Installpip install datatool-pythonUse examplefromtimeimportsleepfromdatatool.timeimportMeterwithMeter()astm:sleep(1.5)print(tm.duration())# Prints the measured duration of the body in with-statement (around 1.5)