Saravanan666/pypi-org huggingface dataset:
Package: datashader
Turn even the largest data into images, accuratelyBuild StatusCoverageLatest dev releaseLatest releasePythonDocsSupportWhat is it?Datashader is a data rasterization pipeline for automating the process of
creating meaningful representations of large amounts of data. Datashader
breaks the creation of images of data into 3 main steps:ProjectionEach record is projected into zero or more bins of a nominal plotting grid
shape, based on a specified glyph.AggregationReductions are computed for each bin, compressing the potentially large
dataset into a much smalleraggregatearray.TransformationThese aggregates are then further processed, eventually creating an image.Using this very general pipeline, many interesting data visualizations can be
created in a performant and scalable way. Datashader contains tools for easily
creating these pipelines in a composable manner, using only a few lines of code.
Datashader can be used on its own, but it is also designed to work as
a pre-processing stage in a plotting library, allowing that library
to work with much larger datasets than it would otherwise.InstallationDatashader supports Python 3.8, 3.9, 3.10, and 3.11 on Linux, Windows, or
Mac and can be installed with conda:conda install datashaderor with pip:pip install datashaderFor the best performance, we recommend using conda so that you are sure
to get numerical libraries optimized for your platform. The latest
releases are avalailable on the pyviz channelconda install -c pyviz datashaderand the latest pre-release versions are avalailable on the
dev-labelled channelconda install -c pyviz/label/dev datashader.Fetching ExamplesOnce you've installed datashader as above you can fetch the examples:datashader examples
cd datashader-examplesThis will create a new directory calleddatashader-exampleswith all the data
needed to run the examples.To run all the examples you will need some extra dependencies. If you
installed datashaderwithin a conda environment, with that
environment active run:conda env update --file environment.ymlOtherwise create a new environment:conda env create --name datashader --file environment.yml
conda activate datashaderDeveloper InstructionsInstall Python 3minicondaoranaconda, if you don't
already have it on your system.Clone the datashader git repository if you do not already have it:git clone git://github.com/holoviz/datashader.gitSet up a new conda environment with all of the dependencies needed
to run the examples:cd datashader
conda env create --name datashader --file ./examples/environment.yml
conda activate datashaderPut the datashader directory into the Python path in this
environment:pip install --no-deps -e .Learning moreAfter working through the examples, you can find additional resources linked
from thedatashader documentation,
including API documentation and papers and talks about the approach.Some Examples
Package: datashakereviewsapi
datashakereviewsapi: python API-wrapper for DATASHAKE reviewsPython API-wrapper for DATASHAKE reviews API (https://www.datashake.com/review-scraper-api)
This module makes it easier to schedule jobs and fetch the results
Official web API documentation:https://api.datashake.com/#reviewsYou need to have datashake API key to use this moduleInstallationpipinstalldatashakereviewsapiUsage examplesInitiate API instancefromdatashakereviewsapiimportDatashakeReviewAPI# Initiate API instance with your API key from DATASHAKEapi=DatashakeReviewAPI('your_datashake_reviews_scraper_api_key')Schedule a single job with a URL to review page.
DATASHALE API takes several hours to crawl the page and collect the results.response=api.schedule_job('https://uk.trustpilot.com/review/store.playstation.com')# save job_id for querying the results laterfirst_job_id=response['job_id']Get the job results - reviewsreviews=api.get_job_reviews(first_job_id)Schedule another job with a reference to the first one - get delta (new reviews) onlyresponse2=api.schedule_job('https://uk.trustpilot.com/review/store.playstation.com',previous_job_id=first_job_id)Create a job list (one row in the example) and schedule jobs for all the urls from the listjobs_list=pd.DataFrame(columns=['Website','url','latest_job_id','status','last_crawl','latest_schedule_message'])jobs_list['url']=['https://uk.trustpilot.com/review/store.playstation.com']updated_job_list=api.schedule_job_list(jobs_list)And ultimately - fetch the reviews  and save them to a csv file, reschedule all jobs in the jobs list# Plug-n-Play block to schedule/update jobs and get/save results# The prerequisite for running the snippet is existence of two CSV files with the following structure:# jobs_list.csv columns: ['Website', 'url', 'latest_job_id', 'status', 'last_crawl', 'latest_schedule_message']# reviews_list.csv columns: ['job_id', 'source_name', 'id', 'name', 'date', 'rating_value',#                           'review_text', 'url', 'profile_picture', 'location', 'review_title',#                           'verified_order', 'reviewer_title', 'language_code', 'meta_data']# Code block refresh review jobs and review resultsjobs_list_filepath='job_list.csv'reviews_list_filepath='reviews_list.csv'df_jobs=pd.read_csv(jobs_list_filepath,index_col='id')df_reviews=pd.read_csv(reviews_list_filepath,index_col='unique_id')df_jobs_new,df_reviews_new=api.get_job_list_reviews(df_jobs,df_reviews)df_jobs_new.to_csv(jobs_list_filepath,encoding='utf-8-sig')df_reviews_new.to_csv(reviews_list_filepath,encoding='utf-8-sig')# Codes block to reschedule review jobsdf_jobs=pd.read_csv(jobs_list_filepath,index_col='id')df_jobs_new=api.schedule_job_list(df_jobs)df_jobs_new.to_csv(jobs_list_filepath,encoding='utf-8-sig')
Package: datashaper
datashaperThis project provides a collection of utilities for doing lightweight data wrangling.There are two goals of the project:Create a shareable client/server schema for serialized wrangling instructionsMaintain an implementation of a basic wrangling engine (based on Arquero) and in the case of python implemented in PandasBuildingYou need to installpoetrypython package manager.Run:poetry installUsageThis project is intended to be used as a library for lightweight data wrangling. In the examples folder there is aNotebookwhich provides several examples of how to create data wrangling pipelines and how to read json specifications that can be generated by the js implementation.Example of joining two tables:fromdatashaper.pipelineimportPipelineimportdatashaper.typesastypesimportpandasaspd# id   name# 1    bob# 2    joe# 3    janeparents=pd.DataFrame({"id":[1,2,3],"name":['bob','joe','jane']})# id   kid# 1    billy# 1    jill# 2    kaden# 2    kyle# 3    moekids=pd.DataFrame({"id":[1,1,2,2,3],"kid":['billy','jill','kaden','kyle','moe']})pipeline=Pipeline()pipeline.add_dataset('parents',parents)pipeline.add_dataset('kids',kids)pipeline.add(Step(verb=Verb.join,input="parents",output="output",args={"other":"kids","on":["id"]}))# id   name    kid# 1    bob     billy# 1    bob     jill# 2    joe     kaden# 2    joe     kyle# 3    jane    moeresult=pipeline.run()ContributingThis project welcomes contributions and suggestions. Most contributions require you to agree to a
Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us
the rights to use your contribution. For details, visithttps://cla.opensource.microsoft.com.When you submit a pull request, a CLA bot will automatically determine whether you need to provide
a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions
provided by the bot. You will only need to do this once across all repos using our CLA.This project has adopted theMicrosoft Open Source Code of Conduct.
For more information see theCode of Conduct FAQor
contactopencode@microsoft.comwith any additional questions or comments.TrademarksThis project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft
trademarks or logos is subject to and must followMicrosoft's Trademark & Brand Guidelines.
Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship.
Any use of third-party trademarks or logos are subject to those third-party's policies.
Package: datashare-network-client
datashare network client libraryThis is the client library for the protocol described in the EPFL paper:DATASHARENETWORK A Decentralized Privacy-Preserving Search Engine for Investigative JournalistsThis is a work in progress.Testing$maketestDatabaseTo run the migrations ondsnet.db:$/path/to/alembicupgradeheadIf you change the models, then please run :$alembicrevision--autogenerate-m"migration description"It will generate a new migration step file inmigrations/versionsthat you can add with your commit.
Package: datashare-network-server
Datashare network serverThis is the http server for the protocol described in the EPFL paper:DATASHARENETWORK A Decentralized Privacy-Preserving Search Engine for Investigative JournalistsThis is a work in progress.Requirementspsycopg2requires the installation ofpython3-dev(see psycopg prerequisites here)Running the server locally$makeinstall
$makerun
pipenvrunuvicorndsnetserver.main:app
INFO:Startedserverprocess[9120]INFO:Waitingforapplicationstartup.
INFO:Applicationstartupcomplete.
INFO:Uvicornrunningonhttp://127.0.0.1:8000(PressCTRL+Ctoquit)DevelopingWe usepipenvto package/develop the server. We will use it to install all dependencies :$makeinstallDatabaseYou need to create first the database. You can bootstrap it with :$psql-U<admin_user>-h<psql_host><bootstrap_db.sqlThen to run the migrations :$/path/to/alembicupgradeheadTesting$maketest
Package: datashare-network-tokenserver
datashare-network-tokenserverA server to issue tokens based on blind signatureIt is based on springlab@EPFLSScredbased itself onpetliband itelf based onopenssl.It is supposed to be secured behind an SSO/OAuth2 server.ConfigurationConfiguration is provided with environment variables :TOKEN_SERVER_REDIS_URL: redis url (default:redis://redis)TOKEN_SERVER_REDIS_TTL: time to live for commitments internal parameters (default 30s)TOKEN_SERVER_SKEY: master secret key for the server encoded inmsgpackhex stringEndpointsAll endpoints should be secured with HTTPS (TLS).GET /publickeyreturns the server public keyPOST /commitments?uid=foo&number=3parameters :number: (int) number of token to generateuid: (string) user idreturns a commitment list msg pack encodedPOST /pretokens?uid=fooparameters :uid: (string) user idpayload: list of pretokens msg pack encodedreturns a token list msg pack encoded
Package: datasheet
Doc atreadthedocsVersion 1.2.0Adds a new Feature: Headings (only those that also show up in theTOC) can now be clicked to hide their contentsVersion 1.1.3Automatically creates containing folder on instantiationVersion 1.1.2Has a new css template for pathfinderVersion 1.1.1Fixes a bug in gated cache, that wasn’t caught by a test before (sorry)Version 1.1.0Changes the default CSS to look way nicer, borrowed fromhttps://gist.github.com/killercup/5917178adds stand alone modeadds Layout classes.Removes tuple interpretation by Sheet.__lshift__, use the Layouts insteadVersion 1.0.1set matplotlibs rendering back-end to agg before importing pyplot, so thatthe library does not depend on tkinterVersion 1.0 - release
Package: datashell
If you have a statistics background, you’re probably familiar with R.
And if you’ve been using R for a while, the thing you start to
appreciate most is that once you have that terminal open, you can get
down to business pretty much immediately. Dumping functionality as
diverse as numerical optimization, linear regression and the cumulative
distribution function for a Poisson distribution all into the global
namespace isprobablynot a good idea, but boy is it useful for quick
data exploration. That’s what Data Shell does for Python.Install withpip3installdatashelldatashell-installOpen up an IPython-based data shell for Python 3 by typingdatashellinto your terminal. For inline plotting, usedatashell-qtinstead.Pro tip:alias these shells to something shorter. For example, putalias dash=datashellandaliasdashi=datashell-qtinto your~/.bashrcor wherever your shell customizations live.Convenience functionsCurrently, it loads convenience functions frommath,random,numpy,scipy.stats,statsmodels,sympyas well aspandas.All functions are lazy-loaded, so startup time is not much different
than a regular IPython terminal.Data shell does a star import of various packages into the global
namespace, but also keeps them available under their respective
namespaces, so you can access functionality both ways.To give just one example, once you’re in your IPython data shell, a
linear regression on a dataset in your working directory is simply:ols('y ~ x',data=tables.test).fit().summary()Behind the scenes, this will loadstatsmodels.formulas.api.olsto
perform a linear regression, andtables.testwill loadtest.csv.Datashell can also be used in (non-interactive) scripts:fromdatashellimport*diff(2*x**2)(Though at some point you’ll probably want to clean things up and do
proper imports.)Data autoloaderData shell also includes a Pandas autoloader for CSV files: you can
access a Pandas DataFrame of./subdir/myfile.csvfromtables.subdir.myfile.Useful shortcutsfrommath:ceil,floor,log,factorial,sinand pretty much anything you’d find on a good calculatorfromrandom:shuffle,choice,sampleand friendsfromsympy:expand,factor,simplifyto simplify
mathematical expressions,diffto differentiate,integrateto
integrate (many one-letter variables are also predefined: a-e, o-s
and u-z)fromscipy.optimize:minimizefromscipy.stats:describe,itemfreq,relfreq,kurtosis,mode,moment,skew,pearsonr,spearmanrand othersfromscipy.stats.contingency:expected_freq,marginsfromscipy.stats.distributions:cdf,pdf,ppf,sf,rvsand various other functions on statistical
distributions from normal to gammafromstatsmodels.api:datasetsandfamilies(for use with
generalized linear models)fromstatsmodels.formula.api:olsandgls
Package: datasherlock
# Sherlock## RequirementsPackage requirements are handled using pip. To install them do` pip install-rrequirements.txt `## TestsTesting is set up using [pytest](http://pytest.org) and coverage is handled
with the pytest-cov plugin.Run your tests withpy.testin the root directory.Coverage is ran by default and is set in thepytest.inifile.
To see an html output of coverage openhtmlcov/index.htmlafter running the tests.
Package: datashuttle
DataShuttleDatashuttle is a work in progress and is currently in alpha release v0.1.0.Datashuttle includes tools for automated generation and transfer of neuroscience project folders formatted to theNeuroBlueprintspecification.Manage files across multiple data-collection computers by synchronising all data to with a centrally stored project.Simplify data transfers by selecting only a sub-set of data to move (e.g. specific subjects, sessions or datatypes)See theDataShuttle Documentationto get started or join theZulip chatto discuss any questions, comments or feedback.InstallationDataShuttle is hosted onPyPIand can be installed with pip.pip install datashuttleDatashuttle required Rclone for data transfers. The easiest way to install Rclone is usingMiniconda:conda install -c conda-forge rcloneSeethe Rclone websitefor alternative installation methods.NeuroBlueprint Folder TreeDataShuttle project folders are managed according toNeuroBlueprint.└── project_name/
    └── raw_data/
        ├── sub-001/
        │   └── ses-001/
        │       ├── ephys/
        │       └── behav/
        │       └── anat/
        └── sub-002/
            └── ses-001/
            │   ├── behav/
            │   └── imaging/
            └── ses-002/
                └── behav/
                └── anat/└── project_name/
    └── rawdata/[test_utils.py](tests%2Ftest_utils.py)
        ├── sub-001  /
        │   └── ses-001/
        │       ├── ephys
        │       └── behav
        │       └── anat
        └── sub-002/
            ├── ses-001/
            │   ├── behav
            │   └── imaging
            └── ses-002/
                └── behav
                └── anat
Package: datasift
This is the official Python library for accessing DataSift APIs.Getting StartedTo get started choose one of our quick start guides:STREAM Quick StartPYLON for Facebook Topic Data Quick StartInstallationInstall withpip:pip install datasiftInstall from source:git clone https://github.com/datasift/datasift-python.git
cd datasift-python
python setup.py installUsageFull documentation is available athttp://datasift.github.io/datasift-pythonSupported Operating EnvironmentThis version of the client library has been tested, and is known to work against the following language versions and Operating Systems:Language versionsPython 2.6Python 2.7Python 3.4Python 3.5pypy 2.7Operating SystemsLinuxBSDOS XWindows 7/8/10 (only with python 2.x) (upstream bug here: <http://twistedmatrix.com/trac/ticket/7626>)Style GuideThis code should conform to pep8, with the exception of E501, the 79 character line limit.It can be style checked with thepep8module by runningpep8--show-source--ignore=E501datasift/*.pyRequirementsSupports Python 2.6, 2.7 and 3.3, as well as pypy.
Usesrequests,autobahn,six,twisted,pyopensslanddateutil.On non-Windows systems, thetwistedandpyopensslpackages require the Python and OpenSSL development headers before they can be installed. These are usually provided by distribution package managers in a separate package to the main packages.Windows users may need to use Python 2.7 due to compatability issues with theTwisted libraryLicenseAll code contained in this repository is Copyright 2012-Present MediaSift Ltd.This code is released under the BSD license. Please see the LICENSE file for
more details.Contributorswoozyking - patches for Historics/prepareglyph from rackspace - assisting with Twisted SSL codeChangelogSee CHANGELOG file
Package: datasift-beta
This is the official Python library for accessingDataSift.
See the examples folder for some simple example usage.Supported Operating EnvironmentThis version of the client library has been tested, and is known to work against the following language versions and Operating Systems:Language versionsPython 2.6Python 2.7Python 3.3Python 3.4pypy 2.7Operating SystemsLinuxBSDOS XWindows 7/8InstallationInstall withpip:pip install datasift-betaInstall from source:git clone git@github.com:datasift/datasift-python.git
cd datasift-python
python setup.py installStyle GuideThis code should conform to pep8, with the exception of E501, the 79 character line limit.It can be style checked with thepep8module by runningpep8--show-source--ignore=E501datasift/*.pyRequirementsSupports Python 2.6, 2.7 and 3.3, as well as pypy.
Usesrequests,autobahn,sixandtwisted.LicenseAll code contained in this repository is Copyright 2012-Present MediaSift Ltd.This code is released under the BSD license. Please see the LICENSE file for
more details.Changelogv.2.0.7 Small bugfix for livestreaming (2014-05-09)v.2.0.6 Added windows support for livestreaming (2014-05-09)v.2.0.5 Added support for all 3 outputs in pull (2014-04-29)v.2.0.4 Upgraded the output mapper date logic to be more robust (2014-03-18)v.2.0.3 Fixed secure websockets on OpenSSL >1.0 (2014-02-20)v.2.0.2 Improved reconnection logic for livestreaming (2014-02-20)v.2.0.1 Fixed disconnect problems in livestreaming on slow streams (2014-02-19)v.2.0.0 Ground up rewrite. (2014-02-14)v.0.5.7 Fixed issues with buffers on reconnections (2013-06-28)v.0.5.6 Fixed broken tests, multistreaming, reconnection timeout (2013-05-03)v.0.5.5 Introduced automatic reconnection after 65 seconds of no data or ticks. Solves ‘silent disconnect’ issue (2013-03-06)v.0.5.4 Removed checks for existance of deprecated ‘volume_info’ field in historics/prepare response (2013-01-18)v.0.5.3 Added missing Historic sample size into historic/prepare requests (2012-12-03)v.0.5.2 Patch for the missing availability info after Historic/prepare [woozyking](https://github.com/woozyking) (2012-11-28)v.0.5.1 Removed unit tests that are no longer required (2012-08-30)v.0.5.0 Added support for Historic queries and Push delivery (2012-08-27)v.0.4.0 Fixed issues with SSL timeouts & low throughput streams (2012-08-08)v.0.3.0 Added SSL support and fixed a reconnection bug (2012-05-16)The SSL support is enabled by default and can be disabled by passing false as
the third parameter to the User constructor, or calling enableSSL(false) on
the User object.v.0.2.0 Fixed the handling of error messages in streams (2012-05-04)v.0.1.1 Initial release (2012-03-09)
Package: datasift_preview_grabber
A script to fetch datasift preview stats for a particular hash, over a particular date range.Quick StartIf you have Python and virtualenv installed, the simplest way to get set up is to
clone or download all the source files, cd into the directory and then run these
commands for first time::$ virtualenv .env
$ source .env/bin/activate
$ python setup.py installThat will install everything you need to run the script.Running the ScriptUsage: datasift_preview_grabber <start_date> <end_date> <stream_hash> <datasift-username> <datasift-apikey>

Where:
    start_date and end_date are in the format yyyy-mm-ddThe script will split the date range you give it into individual days (a limitation
of the Datasift Preview service as it stands).  Each day costs 20DPU currently so
don’t go mental with your date range!For each day, the script creates a preview job with Datasift.  The script waits for
each job to finish, which can take a while.  When all the jobs are finished, the
results of them are simply printed to stdout.Developing the Script FurtherIf you want to enhance the script in some way, install the libraries from the
test_requirements.txt in the virtualenv you created earlier and make sure the
tests all pass before adding something new::$ pip install -r test_requirements.txt
$ nosetestsIf it’s useful, send it back to us in the form of a pull request on Github!
Package: data-significance
data_significanceRidge and Logistic regression with a fast implementation of statistical significance test.Prerequisite:1, python >= 3.6 developer version. We suggest install anaconda (https://www.anaconda.com) to include all required packages.2, numpy >= 1.19;3, pandas >= 1.1.4;4, gcc >= 4.2;5, gsl-2.6:https://ftp.gnu.org/gnu/gsl(please don't use other versions, such as 2.7)Install:python setup.py installTest:python -m unittest tests.regressionUsage:Call the regression function in python code as follows:beta, se, zscore, pvalue = data_significance.ridge(X, Y, alpha, alternative, nrand, verbose).Input:X: explanatory matrix, numpy matrix in C-contiguous order (last-index varies the fastest).Y: response variable, numpy matrix in C-contiguous order (last-index varies the fastest).The row dimension of X and Y should be the same. Y input allows multiple columns, with each column as one input response variable. The function will return the same number of output variables in output matrices (beta, se, zscore, pvalue).alpha: penalty factor in ridge regression (>= 0). If alpha is 0, we will use regular ordinary least square.alternative: one-tailed or two-tailed statistical test, with three options: 1, two-sided; 2, greater; 3, less.nrand: number of randomizations (>=0). If nrand = 0, we will use student t-test for the statistical test. Otherwise, we will use permutation test.verbose: 1 or 0. Report intermediate results.Output:beta: regression coefficient.se: standard error.zscore: beta/se.pvalue: statistical significance from permutation test (nrand>0) or student t-test (nrand=0).