Saravanan666/pypi-org huggingface dataset:
Package: pydat
## DoubleArrayTrieDoubleArrayTrie(DAT) support prefix search & exact search & multiple pattern match for python implemented by c++### What’s differentNow greedy search is supported:
functionsearchonly returns the longest subsequence(greedy search)
functionmaximum_searchreturns all subsequencesword: she sher shers he
to_search: sherssearch:
{‘entities’: [{‘begin’: 0, ‘end’: 5, ‘entity’: ‘shers’}]}maximum_search:
{‘entities’: [{‘begin’: 0, ‘end’: 3, ‘entity’: ‘she’}, {‘begin’: 0, ‘end’: 4, ‘entity’: ‘sher’}, {‘begin’: 0, ‘end’: 5, ‘entity’: ‘shers’}, {‘begin’: 1, ‘end’: 3, ‘entity’: ‘he’}]}### C++#include “double_array_trie.h”DoubleArrayTrie dat;
// load from file
dat.load_file(“test.txt”)
// or add word
dat.add_word(“he”)
dat.add_word(“she”)
// make base & check
dat.make()
// greedy_search
vector<string> index_s = dat.search(“ushers”)
// maximum_search
vector<string> index_s = dat.maximum_search(“ushers”)### Python# install
pip install pydat# use
dat = pydat.Dat()
dat.load_file(“test.txt”)
dat.add_word(“he”)
dat.make()
# search
dat.search(“ushers”)
# greedy_search
dat.maximum_search(“ushers”)### About DAT
![](https://linux.thai.net/~thep/datrie/double.gif)Definitioncheck[base[s] + c] = s
base[s] + c = tReference: [An implementation of Double-Array Trie](https://linux.thai.net/~thep/datrie/datrie.html#Double)### Update Log#### 2018.09.19
- optimize the algorithm#### 2018.08.29
- support greedy_search#### 2018.08.08
- support python3
Package: pydata
# pydataPyData location commmand[![Build Status](https://travis-ci.org/takezyou/pydata.svg?branch=master)](https://travis-ci.org/takezyou/pydata)[![Coverage Status](https://coveralls.io/repos/github/takezyou/pydata/badge.svg?branch=master)](https://coveralls.io/github/takezyou/pydata?branch=master)## Installation```pip install pydata```## Usage```usage: pydata [-l]Location list PyDataoptional arguments:-h, --help  show this help message and exit-l          You can see lists of all PyData location.```
Package: pydataapi
py-data-api - Data API Client for Pythonpy-data-api is a client for Data API ofAurora Serverless.
Also, the package includes SQLAlchemy Dialects and DB API 2.0 Client.FeaturesSQLAlchemy DialectsDB API 2.0 compatible clientPEP 249Support Database EnginesMySQLPostgreSQLWhat's AWS Aurora Serverless's Data API?https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/data-api.htmlThis project is an experimental phase.Warning: Some interface will be changed.How to installpydataapi requires Python 3.6.1 or later$pipinstallpydataapiExamplefromtypingimportListfromsqlalchemyimportColumn,Integer,Stringfromsqlalchemy.ext.declarativeimportdeclarative_basefrompydataapiimportDataAPI,ResultclassPets(declarative_base()):__tablename__='pets'id=Column(Integer,primary_key=True,autoincrement=True)name=Column(String(255,collation='utf8_unicode_ci'),default=None)database:str='test'resource_arn:str='arn:aws:rds:us-east-1:123456789012:cluster:serverless-test-1'secret_arn:str='arn:aws:secretsmanager:us-east-1:123456789012:secret:serverless-test1'defexample_driver_for_sqlalchemy():fromsqlalchemy.engineimportcreate_engineengine=create_engine('mysql+pydataapi://',connect_args={'resource_arn':'arn:aws:rds:us-east-1:123456789012:cluster:dummy','secret_arn':'arn:aws:secretsmanager:us-east-1:123456789012:secret:dummy','database':'test'})result=engine.execute("select * from pets")print(result.fetchall())defexample_simple_execute():data_api=DataAPI(resource_arn=resource_arn,secret_arn=secret_arn,database=database)result:Result=data_api.execute('show tables')print(result.scalar())# PetsContributing to pydataapiWe are waiting for your contributions topydataapi.How to contributehttps://koxudaxi.github.io/py-data-api/contributingRelated projectslocal-data-apiDataAPI Server for localhttps://github.com/koxudaxi/local-data-apiPyPihttps://pypi.org/project/pydataapiSource Codehttps://github.com/koxudaxi/py-data-apiDocumentationhttps://koxudaxi.github.io/py-data-apiLicensepy-data-api is released under the MIT License.http://www.opensource.org/licenses/mit-license
Package: pydataassist
PyDataUtilsFeaturesTODORequirementsTODOInstallationYou can installPyDataUtilsviapipfromPyPI:$pipinstallpydatautilsUsagePlease see theCommand-line Referencefor details.ContributingContributions are very welcome.
To learn more, see theContributor Guide.LicenseDistributed under the terms of theMIT license,PyDataUtilsis free and open source software.IssuesIf you encounter any problems,
pleasefile an issuealong with a detailed description.CreditsThis project was generated from@cjolowicz'sHypermodern Python Cookiecuttertemplate.
Package: pyDataBank
pyDataIPImport resource files by selecting them one by one or in a batch with regex or with open dialog window
Associate these files or filesets to a key for an easy call
The residual keys (others than files, fileset and dialogs) are stored as it in datapak in the others sectionAn example of use of both classes is shown below:resources = DataFiles(datafiles_settings)
datapack = resources.generateDataPack()
files_as_dict = datapack.getFileDict()
fileset_as_dict = datapack.getFilesetDict()
all_files_as_list = datapack.getFileList()
others = datapack.getOthers()Where two examples of datafiles_settings given as a yml file are shown below:Example1 - 
    files:
        parent: "C:/folder"
        depth: -1
        caseSensitive: true
        file1: 'fi.*_1\.txt'
        file2: 'file2'
    fileset:
        parent: "C:/folder2"
        depth: 0
        caseSensitive: true
        texts_set: '.*\.txt'
        images_set2: .*\.(jpg|png|tif|tiff)

    Example2 - 
    dialogs:
        images:
            tip: 'provide image file'
            type: 'png'
            set: true           # default value is True
        text:
            tip: 'provide txt file'
            type: 'txt'
            set: false
    mykey1: myvalue1
    mykey2:
       - item1
       - item2The resulting loaded resources due to the "files" section could be:files_as_dict = {'file1': 'path/to/file_1.txt', 'file2': 'path/to/myfile2.txt}
fileset_as_dict = {'texts_set': ['mydoc.txt'], 'images_set2': ['image1.png', 'image2.jpg']}The "others" section of databank in example 2 would be:
others = {'mykey1' : 'myvalue1', 'mykey2': ['item1', 'item2']}And the dialogs settings pops open a dialog box which creates a fileset for 'image' and a file dictionary for 'text'DataFiles classThe DataFiles class manages resource files thanks to 2 dictionaries (files and fileset), one is a key/value per file,
the other one is a key/value per file set (a group of files)The way resource files are retrieved is based on regex. It uses Finder class ofpyFileFindermodule to do so.
It is also possible to request the pop up of an open dialog box to select files.generateDataPack method returns a DataPack object. DataPack will be discussed in the next section.See more example in thetest classDataPack classThe DataPack class is a container for 2 types of data: file paths and fileset paths.
It manages 2 dictionaries.
The first one is {name: path} with name as a short name for the file and path the file path
The second one is {name: [paths]} with name as a short name for the fileset and [paths] the list of file paths
Then it allows calling files by their shortname or get a list of all files referenced in the 2 dictionaries.It offers convenient methods toadd files or filesets with or without a short name for these resources. A unique short name is always given to added resources.get files, fileset or the list of all files either contained in files or in fileset
Package: py-database-gilaxy04
py_databaseexemple codeimport py_database
from py_database import serverdb = py_databasedb.danger.newdb("test")
action = db.action("test")
action.add("test","text")
print(action.all())user = db.user("g7y7y75nnv4oth7777888777777om77g8ju")
user.register_user("test","test")server.run()py_database
Package: pydatabrary
Python Databrary API WrapperThis is a Python wrapper arounddatabraryAPIInstallationRun the following to install:pipinstallpydatabraryUsageDatabrary APIfrompybraryimportPybrary# Start a Databrary sessionpb=Pybrary.get_instance('USERNAME','PASSWORD')# You need to have permissions to the volume, to interact with itvolume_info=pb.get_volume_info(1)print(volume_info)Databrary CurationGenerate templatesThis will generate for you template files where you can curate your new volumefrompybraryimportCuration# The number of records that you need for your ingestpayload={'participant':0,'pilot':0,'exclusion':0,'condition':0,'group':0,'task':0,'context':0,}records=Curation.generate_records(categories=payload,output='/PATH/TO/OUTPUT/CSV')# Generate an asset csv file from a local folderassets=Curation.generate_assets('PATH/TO/FOLDER/',output='/PATH/TO/OUTPUT/CSV')# Value is the number of containers you wish to generatecontainers=Curation.generate_containers(value=2,output='/PATH/TO/OUTPUT/CSV')Read CSV filesAfter you edit your CSV files you will have to pass them toCurationto validate them and
generate the JSON file needed for your ingestOnly the containers file is required. if you provide assets and records files,Curationwill populate
asset and record ids found in the container from the provided files.Note:pybrarywill ignore duplicate keys, so make sure to have unique ids for your rowsfrompybraryimportCurationvolume=Curation.from_files(volume_name="TEST",containers_file='PATH/TO/CONTAINER/FILE',assets_file='PATH/TO/ASSET/FILE',records_file='PATH/TO/RECORD/FILE')Generate the ingest filefrompybraryimportCurationvolume=Curation.from_files(volume_name="TEST",containers_file='PATH/TO/CONTAINER/FILE',assets_file='PATH/TO/ASSET/FILE',records_file='PATH/TO/RECORD/FILE')Curation.to_json(volume,'/PATH/TO/JSON/OUTPUT')Duplicate existing volumeGet your volume from databraryfrompybraryimportCurationvolume=Curation.from_databrary(volume_id=1,username='YOUR_USERNAME',password='YOUR_PASSWORD')# You can edit your volume programmatically or dump your volume in# csv files that you can edit with your text editorCuration.to_files(volume,containers_output='PATH/TO/CONTAINER/FILE',assets_output='PATH/TO/ASSET/FILE',records_output='PATH/TO/RECORD/FILE')# once your are done you can import your filesGenerate the SQL query to execute on the server (Please contact Databray admin)frompybraryimportCuration# Source is the source volume# Target is the target volumeCuration.generate_sql_query(volume_id=1)The SQL query will create a script in/tmp/volume_id.shthat need to be executed prior
to the ingest, it will copy the volume's asset into a staging folder.Note:The staging folder on the server (where files are copied) and the path
of the file in the final ingest file must matchDevelopmentTestpytestInstall Locallypipinstall-e.Buildpython-mbuildPublishtwineuploaddist/*TODOsGenerate documentation and add examples.Check for duplicate records and asset id.check if the record measures are valid.Fix utils methods.
Package: pydataclasses
Python Data Classes support Python 2 and 3 with many human-friendly features.This project has been tested in Python 2.7 and Python 3.4+.
It is not related to thedataclassesmodule introduced in Python 3.7.Feature SupportPython Data Classes generally supportuntyped data, typed data, and typed data listarbitrary attribute names in the dictionary inputno python attribute naming conflict or pollutionlazy and auto object creation on-the-flylazy loading of the dict input (default: enabled)synchronous updates of the dict input (default: disabled)Getting StartedInstall the Python Data Classespip install pydataclassesCreate the Python Data ClassesfrompydataclassesimportDataClassclassTT(DataClass):attr=NoneclassONCE(DataClass):data=NoneclassTWICE(ONCE):tear=TTtears=[TT]def__init__(self,*args,**kwargs):super(TWICE,self).__init__(*args,**kwargs)self.pair=TWICEself.pairs=[TWICE]Play with the Python Data Classesold_data='hello'new_data='world'old_dict={'data':old_data}new_dict={'data':new_data}# lazy and auto object creation on-the-flytwice=TWICE()asserttwice.pairs[1].tear.attrisNonetwice.pairs[1].tear.attr=old_dataassertold_data==twice.pairs[1].tear.attr# lazy loading of the dict input (default: enabled)twice=TWICE(old_dict)assertold_data==twice.dataassertold_dict==twice.__as_dict__(dict,0)# synchronous updates of the dict input (default: disabled)assertold_dict!=new_dicttwice=TWICE(old_dict,__sync__=True)twice.data=new_dataassertold_dict==new_dictBest PracticesPleaseread and refer topydataclasses.utils.JSONDataas an exampleput a tailored subclass betweenpydataclassesand your projectwrite (or copy frompydataclasses) as many tests as possible for this subclassfrompydataclassesimportDataClassclassProjectDataClass(DataClass):defas_dict(self,dict_class=OrderedDict):returnself.__as_dict__(dict_class,0)Release History1.0.1 (2019-05-04)Bump the version again1.0.1 (2019-05-04)Bump the version1.0.0 (2019-05-04)Support the annotations of native dataclasses0.0.2 (2018-08-28)Fix the python packaging issuesAdd the tox, pylint, pytest, coverage tools0.0.1 (2018-08-26)Birthday
Package: pydatafabric
SHINSEGAE DataFabric Python PackageThis is highly site dependent package. Resources are abstracted into package structure.UsageGet pandas dataframe from parquet file in hdfsfrompydatafabric.yeimportparquet_to_pandaspandas_df=parquet_to_pandas(hdfs_path)Save pandas dataframe as parquet in hdfsfrompydatafabric.yeimportget_sparkfrompydatafabric.yeimportpandas_to_parquetspark=get_spark()pandas_to_parquet(pandas_df,hdfs_path,spark)# we need spark for this operationspark.stop()Work with sparkfrompydatafabric.yeimportget_sparkspark=get_spark()# do with spark sessionspark.stop()Work with spark-bigquery-connector# SELECTfrompydatafabric.gcpimportbq_table_to_pandaspandas_df=bq_table_to_pandas("dataset","table_name",["col_1","col_2"],"2020-01-01","cust_id is not null")# INSERTfrompydatafabric.gcpimportpandas_to_bq_tablepandas_to_bq_table(pandas_df,"dataset","table_name","2022-02-22")Send slack messagefrompydatafabric.yeimportslack_sendtext='Hello'username='airflow'channel='#leavemealone'slack_send(text=text,username=username,channel=channel)# Send dataframe as textdf=pd.DataFrame(data={'col1':[1,2],'col2':[3,4]})slack_send(text=df,username=username,channel=channel,dataframe=True)Get bigquery clientfrompydatafabric.gcpimportget_bigquery_clientbq=get_bigquery_client(project="emart-datafabric")bq.query(query)IPython BigQuery Magicfrompydatafabric.gcpimportimport_bigquery_ipython_magicimport_bigquery_ipython_magic()query_params={"p_1":"v_1","dataset":"common_dev",}%%bq--params$query_paramsSELECTc_1FROM{dataset}.user_logsWHEREc_1=@p_1Use NES CLInes input_notebook_url -p k1 v1 -p k2 v2 -p k3 v3Use github utilfrompydatafabric.yeimportget_github_utilg=get_github_util# query graphqlres=g.query_gql(graph_ql)# get file in github repositorybyte_object=g.download_from_git(github_url_path)Installation$pipinstallpydatafabric--upgradeIf you would like to install submodules for Emart Inc.$pipinstallpydatafabric[emart]--upgrade
Package: py-data-framework
py-analyticsThis repo contains a python framework with Data Enginner, Data Scientist and 3rd party integration tools capabilitiesInstallationpy-test-utility can be installed via pippipinstallpy-analyticstdd_utility - moduleclass load_csv(csv,schema)Contains methods to extract the equivalent json from csv with nested and repeated records structuresArgscsvpath and file name of the csvmandatorynested fields shall be separated by a dot "."  (i.e. item.id, item.quantity)orderitem.iditem.quantitydelivery.addressdelivey.postcodeA0001item15address1e13bpitem21item33A0002item44address4e13bpitem14item32schemapath and schema file name of the table schemarequired if the CSV contain nested and repeated recordsjson format i.e.[{"mode":"NULLABLE","name":"order","type":"STRING"},{"fields":[{"mode":"NULLABLE","name":"id","type":"STRING"},{"mode":"NULLABLE","name":"quantity","type":"STRING"}],"mode":"REPEATED","name":"item","type":"RECORD"},{"fields":[{"mode":"NULLABLE","name":"address","type":"STRING"},{"mode":"NULLABLE","name":"postcode","type":"STRING"}],"mode":"NULLABLE","name":"delivery","type":"RECORD"}]Methodsto_json()if successfuls return the json extracted from the csvto_new_line_delimiter_file(output_file_name)return 0 if successfulscreate new line delimiter "output_file_name" fileUsage>>>fromdata_prepimporttdd_utilityastu>>>mockdata_csv=tu.load_csv(...csv="path/to/filename/file.csv",...schema="path/to/schema/schema.json")# initialise the object>>>mockdata_json=mockdata_csv.to_json()# return the equivalent json>>>mockdata_json=mockdata_csv.to_new_line_delimiter_file(output="path/output_file_name.json")# return output_file_name
Package: pydata-google-auth
pydata-google-authis a package providing helpers for authenticating to Google APIs.InstallationInstall latest release version via conda$condainstallpydata-google-auth--channelconda-forgeInstall latest release version via pip$pipinstallpydata-google-authInstall latest development version$pipinstallgit+https://github.com/pydata/pydata-google-auth.gitUsageUse thepydata_google_auth.get_user_credentials()function to
authenticate to Google APIs with user credentials.importpydata_google_authcredentials=pydata_google_auth.get_user_credentials(['https://www.googleapis.com/auth/cloud-platform'],)# Use the credentials in other libraries, such as the Google BigQuery# client library.fromgoogle.cloudimportbigqueryclient=bigquery.Client(project='YOUR-PROJECT-ID',credentials=credentials)See thepydata-google-auth documentationfor more details.
Package: pyDatahubDriver
# PyDataHub Driver (Official Datahub Python Package)[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![OpenFaaS](https://img.shields.io/badge/openfaas-serverless-blue.svg)](https://www.openfaas.com)## What is it?KAPSARC Datahub makes it easy for model developers to main multiple versions of their model data. This is an official python package for the same.## InstallationYou can install using the pip package manager by runningpip install pyDatahubDriverAlternatively, you could install directly from Github:pip installhttps://github.com/kapsarc/pyDatahub/archive/master.zip## From sourceDownload the source code by cloning the repository or by pressing [‘Download ZIP’](https://github.com/kapsarc/pyDatahub/archive/master.zip) on this page. Install by navigating to the proper directory and running## API DocumentationThe documentation for KAPSARC Datahub API can be found [‘here’](https://apps.kapsarc.org/datahub/apidoc).## Configurationimport datahub
config = datahub.Config()
config.domain = “domain name”
config.api_key = “api_key”## TODOUpdate commitDelete commitRepo statsCreate repoDelete commitDelete repoCreate a branchDelete a branch
Package: pydatalake-gen2
Python client for Azure Data Lake Gen2 REST APIIt's simple implementation of Data Lake Gen2 REST api. Some of methods still not works properly.Installationpip3installpydatalake-gen2Usagefromazure.datalake.gen2.clientimportDataLakeGen2Clientclient=DataLakeGen2Client(ACCOUNT_NAME,STORAGE_KEY)# Create fylesystem (aka container)headers=client.create_filesystem('/testfilesystem')# List available filesystems (containers)filesystems=client.list_filesystem()# Get files from containerfiles=client.list_path('testfilesystem',directory="/optional/folder")# Create directorypath=client.create_path('testfolder','testfolder2',resource='directory')# Rename (move) filefiles=client.rename_file('/testfolder/file1.txt','/testfolder2/file2.txt')This is really early version of client. So, please fell free to contribute!
Package: pyDatalog
WarningThis package is not maintained.  Use at your own risk.
Consider usingIDP-Z3instead.DescriptionpyDatalogadds the logic programming paradigm to Python's toolbox, in a pythonic way.
You can now run logic queries on databases or Python objects, and use logic clauses to define python classes.
In particular,pyDatalogcan be used as a query language:it can perform multi-database queries (from memory datastore, 11 relational databases, and noSQL database with
appropriate connectors)it is more expressive than SQL, with a cleaner syntax;it facilitates re-use of SQL code snippet (e.g. for frequent joins or formula);Datalog = SQL + recursivityDatalog is a truly declarative language derived from Prolog, with strong academic foundations.  It complements Python
very well for:managing complex sets of related information (e.g. in data integration or the semantic web).simulating intelligent behavior (e.g. in games),performing recursive algorithms (e.g. in network protocol, code and graph analysis, parsing)solving discrete constraint problems.As simple as ExcelDatalog excels at accelerated development: Datalog programs are often shorter than their Python equivalent,
and Datalog statements can be specified in any order, as simply as formula in a spreadsheet.More info
Package: pydatalogger
PyDataloggerAboutDatalogger for SBC machines like BeagleBone, RaspberryPi or similar.Supported sensor typesBME280BMP180DHT11DS18B20SDS011HX711ABE-DELTA-SIGMA-ADCSetupgit clone https://github.com/daq-tools/PyDatalogger.git
cd PyDatalogger

virtualenv --python=python3 .venv3
source .venv3/bin/activate
python setup.py developUsage$ pydatalogger --help

Usage:
  pydatalogger info
  pydatalogger readings [--config=<config>] [--target=<target>]... [--dry-run] [--debug]
  pydatalogger --version
  pydatalogger (-h | --help)

Options:
  --config=<config>             Configuration file for runtime settings
  --target=<target>             Data output target
  --version                     Show version information
  --dry-run                     Skip publishing to MQTT bus
  --debug                       Enable debug messages
  -h --help                     Show this screen

Examples:

  # Display readings in JSON format
  pydatalogger readings

  # Publish readings to MQTT broker on localhost
  pydatalogger readings --target=mqtt://localhost/testdrive

  # Publish readings to MQTT broker on remote host
  pydatalogger readings --target=mqtt://daq.example.org/testdriveCreditsRPi-Beeloggerby Markus Hies, seeBeelogger Version 2Hiverize-SensorbeutenbyHiverizeluftdaten-pythonfor luftdaten.info sensor networkBERadiobyHiveeyes
Package: pydatamail
Manage your emails with PythonThepydatamailis a python module to apply data science principles to email processing. It stores the emails in anSQLdatabase and generatespandas.DataFrameobjects for futher processing and plotting.InstallationInstall thepydatamailpackage usingpip:pip install pydatamailOr alternatively usingconda:conda install -c conda-forge pydatamailPython interfaceImport thepygmailfilermodulefrom pydatamail DatabaseInterface, get_from_pie_plot, get_labels_pie_plot, get_number_of_email_plot, Message, email_date_converterThe individual components are briefly explained below:DatabaseInterface-SQLalchemybased interface forSQLdatabase to store emailsget_from_pie_plot- plot a pie chart of the distribution of emails sendersget_labels_pie_plot- plot a pie chart of the distribution of email labelsget_number_of_email_plot- plot the number of incoming emails over timeMessage- abstract class to implementpydatamailcompatible email adaptersemail_date_converter- convert the email date to pythondatetimedates.
Package: pydatamailbox
pydatamailboxpydatamailis an unoffical client for the Ewon's datamailbox web APIs. The Talk2M Service "DataMailbox" allows easy retrieval of Ewon historical data. It uses the great http clientrequestsResourcesOffical ewon pageDocumentationHistory0.2.3AddewonIdsfilter onsyncdataAdd authentication with token form t2m0.2.2Addgetdatamethod0.2.1Add global request timeout0.2.0Add support for m2web api0.1.0First version