Saravanan666/pypi-org huggingface dataset:
Package: dataplane
dataplane-python-packageThe data engineering library to build robust, reliable and on time data pipelines in Python.To use with Dataplane Data PlatformInstall Python Package on Dataplane:https://learn.dataplane.app/installing-python-packagesRecipes to use Dataplane Python Package:https://recipes.dataplane.app/office-365/sharepoint-apiRun tests in repo rootpytest-sFor a specific testpytest-ssrc/dataplane/data_storage/test_sharepoint_upload.py
pytest-s-ktest_sharepoint.pyTroubleshooting errorsIf an error occurs, try removing .pytest_cache and__pycache__- can happen after devcontainers buildfind.|grep-E"(/__pycache__$|\.pyc$|\.pyo$)"|xargsrm-rf
rm-rf/home/vscode/.local/lib/python3.10/site-packages/### S3 / Minio for testing in VS code devcontainersGo tohttp://localhost:9011Login with: username: admin and password: hello123bucket: dataplanebucket
Package: dataplate
DataPlate - Python Data Access APIInstallationpip install dataplateDescriptionThis client communicates with DataPlate platform servers from within your Data-Science development environment
For more info:DataPlateSee also our JupyterLab extension:DataPlate-LabUsage using our open-sourceFirst:InstallDataPlate Portal Web serviceand navigate to "API Documentation" for usage instructionsMore details:DataPlate() constructor accepts the following parameters:env - Environment to retrieve the Data from ('dev' or 'prd').
access_key - Alternative method for supplying your access key.
dataplate_ur - Alternative method for supplying DataPlate Portal URI.Get the access key from Dataplate Web-service portal (Navigate in Menu to "Private access key"):This example shows how to run a query, and return results as Pandas DataFrame object:from dataplate.client import DataPlate

dataplate = DataPlate()
df = dataplate.query_to_df('''
SELECT * FROM myTable WHERE `date`='20200218' AND hour=12
''')For more instructions, please refer to theDataPlate Github
Package: dataplate-lab
dataplate-labDataPlate ML Platform Jupyter Lab extension for easy notebook operationsThis extension is composed of a Python package nameddataplate-labfor the server extension and a NPM package nameddataplate-labfor the frontend extension.RequirementsDataPlate Platform (server onPremise/Saas) is requiredJupyterLab >= 3.0dataplateInstallTo install the extension, execute:pipinstalldataplate-labUninstallTo remove the extension, execute:pipuninstalldataplate-labTroubleshootIf you are seeing the frontend extension, but it is not working, check
that the server extension is enabled:jupyterserverextensionlistIf the server extension is installed and enabled, but you are not seeing
the frontend extension, check the frontend extension is installed:jupyterlabextensionlistContributingDevelopment installNote: You will need NodeJS to build the extension package.Thejlpmcommand is JupyterLab's pinned version ofyarnthat is installed with JupyterLab. You may useyarnornpmin lieu ofjlpmbelow.# Clone the repo to your local environment# Change directory to the dataplate-lab directory# Install package in development modepipinstall-e.# Link your development version of the extension with JupyterLabjupyterlabextensiondevelop.--overwrite# Server extension must be manually installed in develop modejupyterserverextensionenabledataplate-lab# Rebuild extension Typescript source after making changesjlpmbuildYou can watch the source directory and run JupyterLab at the same time in different terminals to watch for changes in the extension's source and automatically rebuild the extension.# Watch the source directory in one terminal, automatically rebuilding when neededjlpmwatch# Run JupyterLab in another terminaljupyterlabWith the watch command running, every saved change will immediately be built locally and available in your running JupyterLab. Refresh JupyterLab to load the change in your browser (you may need to wait several seconds for the extension to be rebuilt).By default, thejlpm buildcommand generates the source maps for this extension to make it easier to debug using the browser dev tools. To also generate source maps for the JupyterLab core extensions, you can run the following command:jupyterlabbuild--minimize=FalseDevelopment uninstall# Server extension must be manually disabled in develop modejupyterserverextensiondisabledataplate-lab
pipuninstalldataplate-labIn development mode, you will also need to remove the symlink created byjupyter labextension developcommand. To find its location, you can runjupyter labextension listto figure out where thelabextensionsfolder is located. Then you can remove the symlink nameddataplate-labwithin that folder.Packaging the extensionSeeRELEASE
Package: dataplatform-path-generator
Path generator for dataplatform files in cloud storageThis library generates the path to write files to in cloud storage, partitioned into the prefix, source, optionally type, and then finally the day. Arguments:prefix: "the prefix of the cloud storage location of your files, usually the top level bucketsource: the type of files to output.date: the execution date in isoformattype(Optional): a subset of source, if the source has multiples typesExample usage:from pathgenerator import path

output_path = path.generate_path('gs://dataplatform-raw', 'salesforce', "2022-03-11T12:12:12+00:00", "account")Outputgs://dataplatform-raw/source=salesforce/type=account/year=2022/month=3/day=11
Package: dataplicity
## About
Dataplicity agent is a Python based script which allows remote connection to your device via dataplicity.com.It is packaged and delivered using a PEX file for convenience, but you can see the source code here.Please feel free to raise issues, fork the code etc.## How it works
When you install the Dataplicity Agent on your device, it will opportunistically establish and maintain a secure HTTPS connection to the Dataplicity IoT Router.When you connect to the Remote Shell via the Dataplicity website, or to the redirected web interface via your device’s Wormhole URL, your connection will be routed between your browser and your device via our IoT Router.In practice, this means that you can access the devices covered by Dataplicity anywhere that they have a viable internet connection. The traffic is routed using encrypted websocket connections, and is robust enough to be used in instances where the internet coverage is flaky. Because the device itself is the originator of the connection, traditional impediments to remote access (such as NAT, firewalls and dynamic IP addressing) are no longer an issue.For more information, seehttps://docs.dataplicity.com## License
Dataplicity agent is licensed under a modified-BSD license. If you have any thoughts on different types of licensing, please raise an issue.
Package: dataplot
dataplotProvides plotters useful in datascience.Installation$pipinstalldataplotRequirementsattrs
hintwith>=0.0.1
lazyr>=0.0.10
matplotlib
numpy
pandas
scipySee AlsoGithub repositoryhttps://github.com/Chitaoji/dataplot/PyPI projecthttps://pypi.org/project/dataplot/LicenseThis project falls under the BSD 3-Clause License.Historyv0.0.1Initial release.
Package: dataplug
DataPlugA simple graph data manager, in other words: on the fly schemaless multi-model data client with timeseries wannabees.Inspired by InfluxDB, ElasticSearch and other cool stuffs that do not cover a little thing: Graphization !Main requirements for devsDataplug supports the last updates of Arango and its python driver.Python driver for Arangoversion > 4ArangoDBversion >= 3.3
A multi-model no-sql graph databaseInstallationpipinstalldataplugQuick startimportdataplugserver_config={"host":"localhost","port":7144,"username":"root","password":"autoGeneRatEd"}# Creating a node, locallyA=dataplug.Node(domain="db1",collection="collection1",data={"name":"NODE_A","value":3.14},client_config=server_config)# Saving it into databaseifA.upsave():print(A.key())# Creating another nodeB=dataplug.Node(domain="db1",collection="collection2",data={"name":"NODE_B","value":1.41},client_config=server_config)# Saving it into databaseB.upsave()# Creating an edge between these nodesedgeAB=dataplug.Edge("db1",A,B)# Adding information to the edgeedgeAB.add_field("strength","high")# Saving it into databaseedgeAB.upsave()Update your node dataThat replaces data totally with newdata dictionnary:node.data = newdataThat searches of similar node in the database and updates/adds data:node.sync()To append/update data with newdata use:node.data.update(newdata)Testingpytest -v tests
Package: dataplunger
DataPlunger===========```________________________________________________<                 COWSAY SAY                     ><      "Give me a well thought-out plan,         >< a good plunger... and I'll set your data free.">------------------------------------------------\   ^__^\  (oo)\_______(__)\       )\/\||----w |||     ||```A prototype for an ETL workflow. Hooray!dev-branch![dev-branch](https://travis-ci.org/mattmakesmaps/DataPlunger.png?branch=development)master![master](https://travis-ci.org/mattmakesmaps/DataPlunger.png?branch=master)
Package: dataplus
datasyncA tool written in Python to synchronize databases in a simple wayThe format of the configuration file is as follows:[sourcedb]
host = *.*.*.*
user = yourusername
password = yourpassword
port = 1521
database = databasename
charset = AL32UTF8
timestamp = UPDATETIME
[targetdb]
host = *.*.*.*
user = yourusername
password = yourpassword
port = 3306
database = databasename
charset = utf8mb4
timestamp = UPDATETIME
tablelist = tablename1
            tablename2
            tablename3
Package: datapoint
Datapoint for PythonA Python module for accessing weather data via the Met
Office’s open data API known as
`Datapoint`.Disclaimer: This module is in no way part of the datapoint
project/service. This module is intended to simplify the use of
Datapoint for small Python projects (e.g school projects). No support
for this module is provided by the Met Office and may break as the
Datapoint service grows/evolves. The author will make reasonable efforts
to keep it up to date and fully featured.ChangelogExplicitly state the use of semantic versioning inREADME.md.Addelements()function toTimestep.Remove night/day indication from weather codes which have them.Change the logic used to calculate the closest timestep to a datetime. The closest timestep to the datetime is now used. Add a new function,Forecast.at_datetime(target)to do this.Forecast.now()has been changed to use this new function. The old behaviour is deprecated and available usingForecast.now_old().Forecast.future()has been changed to use this new function. The old behaviour is deprecated and available usingForecast.future_old().Check if keys are returned from datapoint api inManager.py. Do not attempt to read the values from the dict if they are not there.Installation$pipinstalldatapointYou will also require aDatapoint APIkey fromhttp://www.metoffice.gov.uk/datapoint/API.FeaturesList forecast sitesGet nearest forecast site from latitiude and longitudeGet the following 5 day forecast types for any siteDaily (Two timesteps, midday and midnight UTC)3 hourly (Eight timesteps, every 3 hours starting at midnight UTC)Get observation sitesGet observations for any siteContributing changesPlease feel free to submit issues and pull requests.LicenseGPLv3.
Package: datapoints
data_connectorОписаниеsignal - элементарный тип данных. Кроме значения, хранит также время последнего чтения / записи, код качества, единицу измерения, пределы измерения.channel - объект для связи signal с периферийными значениями.driver - управляет объектами channel.datapoint - объект для храненения данных. Связывает signal и channel.РазработкаУстановить виртуальное окружениеpoetryinstallОпубликовать пакетpoetrybuild&&poetrypublish
Package: datapools
datapoolsAwesome datapools created by openlicenseaiInstall it from PyPIpipinstalldatapoolsUsagefromdatapoolsimportcrawl...awaitcrawl(datapool_id=1)$python3-mdatapools--id=1#or$datapoolsDocumentationDocumentation is available atdocs.openlicense.aiDevelopmentRead theCONTRIBUTING.mdfile.
Package: dataportal
dataportal공공데이터포털 api를 이용하여 데이터를 다운로드 하기 위한 라이브러리입니다.xml의 경우OrderedDict형태로 데이터를 불러오고 json의 경우json형태로 데이터를 불러옵니다.Guide사용법은 매우 간단합니다.
(예시를 위해 기상청 일기도 api를 활용하였습니다.)아래와 같이 response부분과 데이터 다운로드 부분을 분리하여 할수도 있고url = 'http://apis.data.go.kr/1360000/WthrChartInfoService'
serviceKey = 'Test'
operation = 'getSurfaceChart'
params = {'code' : 12, 'time' : '20211118'}

dataportal = DataportalApi(url, serviceKey, operation, params)
dataportal.get_request_query()
api_out = dataportal.parse_response()
print(api_out.head())아래와 같이 모든 과정을 한번에 처리할 수도 있습니다.url = 'http://apis.data.go.kr/1360000/WthrChartInfoService'
serviceKey = 'Test'
operation = 'getSurfaceChart'
params = {'code' : 12, 'time' : '20211118'}

dataportal = DataportalApi(url, serviceKey, operation, params)
api_out = dataportal.load(verbose=False)
print(api_out.head())Requiresxmltodict>=0.12.0requests>=2.25.1urllib3>=1.26.4
Package: data-portal-explorer
Data Portal ExplorerThis package contains tools to extract metadata from CKAN instances which can
be used to study data portals as infrastructures from the perspective of
social/cultural research.Documentation:https://dataportalexplorer.readthedocs.io.CreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History[0.1.12] - 2024-02-09FixedUpdate requirements to supported versions of packages.[0.1.11] - 2021-10-28FixedAdd missing documentation about logging configuration.[0.1.10] - 2021-10-21FixedAdd missing destination path to the usage documentation.[0.1.9] - 2021-10-11AddedGitHub action.[0.1.8] - 2021-10-11FixedTravis/Pypi authentication.[0.1.7] - 2021-09-29FixedPypi authentication.[0.1.6] - 2021-09-29AddedUsage documentation.ChangedPin requirements.[0.1.5] - 2020-05-13AddedUsage documentation.ChangedAdoptKeep a Changelogformat for this document.Pin requirements.SecurityUpgrade bleach,https://nvd.nist.gov/vuln/detail/CVE-2020-6816.[0.1.4] - 2020-01-22FixedFix setup.py URL.[0.1.3] - 2020-01-22
—AddedCommand line documentation.[0.1.1] - 2020-01-22FixedSyntax in README.[0.1.0] - 2020-01-22First release on PyPI.
Package: dataporten-auth
Dataportenis an authentication, authorization and API platform
for higher and lower education and research in Norway. Dataporten
offers authentication of users to applications using OAuth 2.0 and
the OpenID Connect.Dataporten-auth is a client for dataporten that depends onpython-social-auth.InstallationInstall withpipor by downloading the source and runningsetup.py.UsageThere needs to exist an entry for your site/app at dataporten. Log
in todataporten’s dashboardand create an application.You’ll need to set the client id and client secret in the
settings of your app/site, while the application at dataporten
will need one or more redirect uris.Both the name of the settings and the redirect uris depend on the
plugins used. Add at least one of the plugins below. Seepython-social-auth’s documentationfor how.The redirect uri is of the form<type>://<domainpath>/<suffix>/,
where<type>is one ofhttporhttps,<domainpath>is
the domain name of your site and an optional path, and the<suffix>is
plugin-dependent. See the examples.You can have several redirect-uris, and you will be needing at least
one per plugin used.dataporten.psa.DataportenOauth2Plugin namedataporten.psa.DataportenOauth2SettingsClient id:SOCIAL_AUTH_DATAPORTEN_KEYClient secret:SOCIAL_AUTH_DATAPORTEN_SECRETScopes neededuseridandprofile, this is the default.Username generated:Unique, alphanumeric string. You might want to let users
change this generated username. The plugin only cares that a
username exists and won’t change the username back.Redirect-uri ends with/complete/dataporten/Example redirect uri:http://127.0.0.1/complete/dataporten/dataporten.psa.DataportenEmailOauth2Plugin namedataporten.psa.DataportenEmailOauth2SettingsClient id:SOCIAL_AUTH_DATAPORTEN_EMAIL_KEYClient secret:SOCIAL_AUTH_DATAPORTEN_EMAIL_SECRETScopes neededemail, this must be explicitly allowed in the dashboard.Username generated:From email-addressRedirect-uri ends with/complete/dataporten_email/Example redirect uri:https://supersites.exmaple.net/mysite/complete/dataporten_email/dataporten.psa.DataportenFeideOauth2Plugin namedataporten.psa.DataportenEmailOauth2SettingsClient id:SOCIAL_AUTH_DATAPORTEN_FEIDE_KEYClient secret:SOCIAL_AUTH_DATAPORTEN_FEIDE_SECRETScopes neededuserid-feide, this must be explicitly allowed in the dashboard.Username generated:From the Feide attributeeduPersonPrincipalName, which looks
like an email address.Redirect-uri ends with/complete/dataporten_feide/Example redirect uri:https://example.com/cheatsheet/complete/dataporten_feide/DemoGet the source codeInstall dependencies:pip install-rrequirements/demo.txtMake an application at dataportenEdit the settings-file to setSOCIAL_AUTH_DATAPORTEN_KEYandSOCIAL_AUTH_DATAPORTEN_SECRETSet three redirect-uris, all starting withhttp://127.0.0.1:8000Runpython mange.py runserverVisithttp://127.0.0.1:8000in a fresh browser. Log out doesn’t work (yet),
so to reset, delete the filedb.sqlite3and runpython mange.py runserveragainOptionally, you can add other python-social-auth plugins as well.Version:0.1.1
Package: datapot
Open source tool for machine learning on semi-structured
data that creates numeric object-feature matrix from JSON. The idea of
Datapot is to make the process of data preparation and feature
extraction automatic, easy and effective.UsageInstall Datapot:$gitclonehttps://github.com/bashalex/datapot.git$cddatapot$pipinstall.Tocreate a Datapotobject simply write the following:>>>importdatapotasdp>>>data=dp.DataPot()DataPot has two main methods:fit()transform()Methodfit(self, data, limit)goes through the first N objects (N =
limit), passes the possible features to Transformers. Each Transformer
evaluates if a feature from current field or a number of fields can be
created. As a result a dict of features and Transformers is created.To applyfit()to JSON file:>>>f=open('data/matches_test.jsonlines','r')>>>data.fit(f,limit=100)>>>dataDataPotclassinstance-numberoffeatureswithouttransformation:806-numberofnewfeatures:315featurestotransform:(u'players.0.gold_t',[ComplexTransformer])(u'picks_bans.0.is_pick',[BoolToIntTransformer])(u'players.0.kills_log.0.unit',[TfidfTransformer])(u'players.1.xp_t',[ComplexTransformer])(u'picks_bans.1.is_pick',[BoolToIntTransformer])(u'players.1.kills_log.0.unit',[TfidfTransformer])...Methodtransform(self, data, verbose)generates a pandas. DataFrame
with new features that were detected on the fit() call. If parameter
verbose is true, progress description is printed during the feature
extraction.>>>df=data.transform(f,verbose=False)fittransformers...OKnumofnewfeatures:315ExamplesLook formore examplesof using Datapot with
different datasets and more Transformer specific.FeaturesDatapot provides many ways of extracting features from JSON-s.Data types that can be processed: - Boolean - Numerical array (transform
array to their sum divided by average length of array in training set) -
Time series (сalculate descriptive statistical properties of a given
time series) - Timestamp (date, time, day of week, day of month etc.) -
Text (bag of words tf-idf, word2vec) - Categorial (one-hot encoding,
dimension reduction)AuthorsAlex BashYuriy MokriyNikita SavelyevMichal RozenwaldPeter RomovDatapot is a course work project ofthe Faculty of Computer
Scienceofthe Higher School of
Economics.
Package: data-prep-helper
"data_prep_helper" PackageThis is a helper package for preparing and combining data from a variety of sourcesas described in the Overview section below.Installationpip install data_prep_helperOverviewThis is a helper package for a variety of functionsAllows you to read data form a variety of sourcesA single csv fileA single xls fileA folder containing multiple csv fileA weblink to a csvA pandas dataframeAllows you to specify multiple data sources.Helper functions to clean the dataHelper functions to combine the data for final consumptionExamplesCheck out theexamplesfolder for details on usageVersion History0.0.3Fixed link to internal imports0.0.2Fixed package name to remove - and replace with _PEP8 format0.0.1Initial Release
Package: datapreprep
This is a Data pre processing package where you can Treat
1) Missing Values using Traditional method - Mean , Median, Mode,Knn Method
2) Outlier Treatment using - IQR , Zscore
3) Feature Scaling using - Standard Scalar , Min Max Scalar, Robust Scalar, Max absolute scalar.——————Missing Value Treatment———————————Mean   -   treat_mean(dataframe)
Median  -  treat_median(dataframe)
Mode  -  treat_mode(dataframe)
KNN -    treat_knn(dataframe,int) # int specify nearest neighbour by default 1———Get information of a dataframe ———————————info(dataframe)————————–Outlier Treatment—————————————————–IQR —  ot_iqr(dataframe,column_name)Zscore– ot_zscore(dataframe,column_name)————————————–Feature Scaling—————————————Standard Scalar —   f_standardscalar(dataframe)Min Max Scalar —  f_minmax(dataframe)Robust Scalar —-  f_robustscalar(dataframe)Max absolute Scalar —   f_maxabs(dataframe)——————Data Visualization———————————————————————-bar(df)
heatmap(df)
matrix(df)
dendrogram(df)
geoplot(df)——————————You can also use the GUI version of our package————————————————–We’ll love it if give it a try——————-https://share.streamlit.io/mohammed-muzzammil/data_pre_processing/main/st1.py————————-More Information on our Website—————————————–https://mohammed-muzzammil.github.io/datapreprepsChange Log0.0.1 (22/11/2020)First Release0.0.5 (29/11/2020)-Fifth RealeaseAdded Data Visualization0.07 (9/12/2020)Advance Missing value Treatment Method
Package: dataPreprocess
Use this library to get an out of the box solution for all text pre-processing related problems.
A wide range of text processing methods have been added to this library and I will keep on adding more methods.
The use of this library is very simple and intuitive and makes it very easy to clean the data for your NLP/NLU/Machine learning pipelines.Pre-requiste steps to be completed before use:Download the nltk stopwords with the following command on your python interpreter:>>>nltk.download('stopwords')Install spacy en_core_web_sm with the following command on your machine's terminal:$python3-mspacydownloadenInstall pathlib package with the command:$pipinstallpathlibMake sure that the python version is 3.x and aboveSample demonstration of some of the methods in the library:>>>fromdataPreprocess.preprocessimportPreprocess>>>text="<br> This is   the firt     line. And this    is the 23   secodn lie. </br>">>>Data_preprocessor=Preprocess()>>>clean_text=Data_preprocessor.strip_html_tags(text)>>>clean_text=Data_preprocessor.text_lowercase(clean_text)>>>clean_text=Data_preprocessor.correct_spellings(clean_text)>>>clean_text=Data_preprocessor.remove_stopwords(clean_text)>>>clean_text=Data_preprocessor.remove_whitespace(clean_text)>>>clean_text=Data_preprocessor.remove_numbers(clean_text)>>>clean_text=Data_preprocessor.correct_spellings(clean_text)As demonstrated above, the methods of the library can be used in series without any hassle. It also takes out the headache of matching
the input format requirements of various libraries that are otherwise available online by different contributers.
PS: I have written the code from scratch and not copy pasted the code of the other contributers.Note that despite supporting various functions, this library is very fast. That means that this adding this library to your production
pipeline will not hold you back at all ;)Right now this library supports 21 different functions to clean your text right out of the box.
The list of functions is as follows:text_lowercasetext_uppercaseremove_numbersremove_punctuationremove_whitespaceremove_stopwordsstem_wordslemmatize_wordspos_taggingNERremove_emojiremove_emoticonsemoticon_to_wordsremove_urlsremove_htmlcorrect_spellingsRemove_special_charExpand_contractionsremove_accented_charsconvert_number_towordsremove_freqwordsThe functionalities of the methods listed above is pretty self-explanatoryThis library is still in development phase and I will keep adding more and more functions to it other than just text cleaning.