Saravanan666/pypi-org huggingface dataset:
Package: datapane
Docs|Examples|Note: Datapane is no longer actively maintained.We've made the difficult decision to stop driving this project and therefore we will no longer actively respond to issues or pull requests. If you would like to take over maintaining this project independently, please let us know so we can add a link to your forked project here.The final release,0.17.0, has been optimized to focus on free local report saving, with unused features and analytics removed, and libraries updated to allow usage for as long as possible.Thank You.OverviewBuild interactive reports in seconds using Python.Datapane makes it simple to build interactive reports in seconds using Python.Import Datapane's Python library into your script or notebook and build reports programmatically by wrapping components such as:Pandas DataFramesPlots from Python visualization libraries such as Bokeh, Altair, Plotly, and FoliumMarkdown and textFiles, such as images, PDFs, JSON data, etc.Interactive forms which run backend Python functionsDatapane reports are interactive and can also contain pages, tabs, drop downs, and more. Once created, reports can be exported as HTML, shared as standalone files, or embedded into your own application, where your viewers can interact with your data and visualizations.GalleryCheck out example reports:https://github.com/datapane/examplesGetting StartedCheck out ourQuickstartto build a report in 3m.Installing DatapaneThe best way to install Datapane is through pip or conda.pip$ pip3 install -U datapaneconda$ conda install -c conda-forge "datapane>=0.17.0"Datapane also works well in hosted Jupyter environments such as Colab or Binder, where you can install as follows:!pip3 install --quiet datapaneExamples📊 Share plots, data, and more as reportsCreate reports from pandas DataFrames, plots from your favorite libraries, and text.importaltairasaltfromvega_datasetsimportdataimportdatapaneasdpdf=data.iris()fig=(alt.Chart(df).mark_point().encode(x="petalLength:Q",y="petalWidth:Q",color="species:N"))view=dp.Blocks(dp.Plot(fig),dp.DataTable(df))dp.save_report(view,path="my_app.html")🎛 Layout using interactive blocksAdd dropdowns, selects, grid, pages, and 10+ other interactive blocks....view=dp.Blocks(dp.Formula("x^2 + y^2 = z^2"),dp.Group(dp.BigNumber(heading="Number of percentage points",value="84%",change="2%",is_upward_change=True),dp.BigNumber(heading="Simple Statistic",value=100),columns=2),dp.Select(dp.Plot(fig,label="Chart"),dp.DataTable(df,label="Data")),)dp.save_report(view,path="layout_example.html")Next StepsQuickstart- build a report in 3mView ExamplesRead the documentation
Package: datapane-components
Cloud|Docs|Examples|Roadmap|Forum|DiscordWelcome to the Datapane Component RepositoryHi there! 👋This repository contains our collection of reusable components for use in your Datapane Reports and Apps.For UsersRunpip install datapane-componentsIn your code, import the libraryimportdatapaneasdpimportdatapane_componentsasdc...For ContributorsClone this repoInstall the dependencies viapoetry installMake changesSubmit your PRThis package is released on a regular time-based schedule.ResourcesWe're here to help you get up and running with Datapane. Check out theDatapane quickstart repoto get started, or visit any of the resources below.Join DiscordSign up for a free accountRead the documentationAsk a questionWhat makes Datapane special?Static generation:Sharing an app shouldn't require deploying an app. Render a standalone HTML bundle which you can share or host on the web.API-first and programmatic:Programmatically generate apps from inside of Spark, Airflow, or Jupyter. Schedule updates to build real-time dashboards.Dynamic front-end components: Say goodbye to writing HTML. Build apps from a set of interactive components, like DataTables, tabs, and selects.
Package: dataparcel
DataverkDocs:https://dataverk.readthedocs.io/en/latest/Relaterte  prosjekterurlbeskrivelsego-jek feastFeast - Feature Store for Machine LearningContinuumIO/intakeIntake: A general interface for loading datafrictionlessdata.ioFrictionless datagit lfsGithub Large File Storagedvc.orgData Science Version Controlquilt (github)Quilt - Version and deploy dataPython package in S3CLI tool for creating a Python Package Repo i S3pypiserver based on bottleMinimal PyPI server
Package: dataparrots-test
Copyright (c) 2021 The Python Packaging AuthorityPermission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
Package: data-parser
Parse raw logs to tabular formatThis package helps to parse new line delimited logs to tabular formats. The user provides the regex, file path and column names, and a dataframe will be returned.Depending on the supplied mode (local/spark), a pandas dataframe or a spark dataframe will be returned.FeaturesLocal modeRegex matching is done using multiprocessing.Glob searching for files.Lazy evaluation of files. This allows larger than memory datasets to be parsed, but note that upon parsing, the resultant pandas dataframe must be able to fit in memory.InstallationPurely for local usage (No pyspark)pip install data-parserBoth local and pysparkpip install data-parser[pyspark]Usage - Local (Pandas)fromdata_parserimportDataSource# Bind 9: Feb  5 09:12:11 ns1 named[80090]: client 192.168.10.12#3261: query: www.server.example IN Adns=DataSource(path='/path/to/dnsdir/*.txt',# Glob patterns supportedmode='local')# Pandas dataframe is returneddns_df=dns.parse(regex='^([A-Z][a-z]{2})\s+(\d+) (\d{2}\:\d{2}\:\d{2}) (\S+).+client ([^\s#]+)#(\d+)',col_names=['month','day','time','nameserver','query_ip','port'],on_error='raise')Usage - Spark (Pyspark)fromdata_parserimportDataSource# Bind 9: Feb  5 09:12:11 ns1 named[80090]: client 192.168.10.12#3261: query: www.server.example IN Adns=DataSource(path='/path/to/dns/log',mode='spark')# Spark dataframe is returneddns_df=dns.parse(regex='^([A-Z][a-z]{2})\s+(\d+) (\d{2}\:\d{2}\:\d{2}) (\S+).+client ([^\s#]+)#(\d+)',col_names=['month','day','time','nameserver','query_ip','port'],on_error='raise')
Package: data-partitioner
Data PartitionerSimple project that can be used to consistently partition a data set
into two parts - a test set and a training set. There are also helpful
methods that provide a way to partition into more groups of elements.InstallationThe easiest way to install this module is to install it viapip:$ pip install data_partitionerUsageUsing this module is dead simple. The main module (DatasetSuplier)
offers two methods that return the training set (training_set()) or
the test set (test_set()). Both of these methods are consitent, so
no matter how many times you call them on the same object, they will
return the same set of elements back.You have two configuration options you can specify:training_percent- the percent of the dataset used for the
training set. It defaults to0.8.partitioning_function- the function that’s used to partition the
dataset.It defaults todata_partitioner.pseudorandom_function, which will
randomly assign every element of the dataset to either the test set
or the training set.Another useful existing option you can set it to isdata_partitioner.LinearFakeRandomFunction, which will make sure
that no elements in the training set come after any elements of the
test set.You can also manually write this callable, which will take one
parameter as input - the index of the element currently considered.Examplefrom data_partitioner import DatasetSuplier

dataset = [
    ('Alice', 10, 23, 401),
    ('Bob', 20, 40, 812),
    ('Christine', 41, 92, 533),
    ('Dave', 843, 12, -5),
    ('Elizabeth', 682, 33, -7),
    ('Fred', 95, 642, 34),
]
suplier = DatasetSuplier(dataset)

for iteration in range(100):
    for element in suplier.training_set():
        do_train(element[1])
for element in suplier.test_set():
    do_evaluate(element[1])
Package: datapatch
datapatchA Python library for defining rule-based overrides on messy data. Imagine, for example,
trying to import a dataset in each row is associated with a country - which have been
entered by humans. You might find country names likeNorthkorea, orGreet Britainthat you want to normalise.datapatchcreates a mechanism to build a flexible lookup
table (usually stored as a YAML file) to catch and repair these data issues.InstallationYou can installdatapatchfrom the Python package index:pipinstalldatapatchExampleGiven a YAML file like this:countries:normalize:truelowercase:trueasciify:trueoptions:-match:Frankreichvalue:France-match:-Northkorea-Nordkorea-Northern Korea-NKorea-DPRKvalue:North Korea-contains:Britainvalue:Great BritainThe file can be used to apply the data patches against raw input:fromdatapatchimportread_lookups,LookupExceptionlookups=read_lookups("countries.yml")countries=lookups.get("countries")# This will apply the patch or default to the original string if none exists:forrowiniter_data():raw=row.get("Country")row["Country"]=countries.get_value(raw,default=raw)Extended optionsThere's a host of options available to configure the application of the data
patches:countries:# If you mark a lookup as required, a value that matches no options will# throw a `datapatch.exc:LookupException`.required:true# Normalisation will remove many special characters, remove multiple spacesnormalize:false# By default normalize perform transliteration across alphabets (Путин -> Putin)# set asciify to false if you want to keep non-ascii alphabets as isasciify:falseoptions:-match:Francoisvalue:France# This is a shorthand for defining options that have just one `match` and# one `value` defined:map:Luxemborg:LuxembourgLux:LuxembourgResult objectsYou can also have more details associated with a result and access them:countries:options:-match:Frankreich# These can be arbitrary attributes:label:Francecode:FRThis can be accessed as a result object with attributes:fromdatapatchimportread_lookups,LookupExceptionlookups=read_lookups("countries.yml")countries=lookups.get("countries")result=countries.match("Frankreich")print(result.label,result.code)assertresult.capitalisNone,result.capitalLicensedatapatchis licensed under the terms of the MIT license, which is included asLICENSE.
Package: data-persistence-repository
Data Persistence RepositoryData Persistence Repositoryis a Python package providing a portable implementation of the repository pattern for a
data persistence layer. For the moment there is only SQL support but it is planned to introduce more database types.
This package facilitates interaction with SQL databases through an abstracted interface, ensuring clean and maintainable
database interaction in Python applications.FeaturesAbstract repository interface for streamlined data layer interactions.Async and sync SQL repository implementation with SQLAlchemy.Context manager for robust database session management.Methods for CRUD operations (Create, Read, Update, Delete).Support for filtering and bulk operations.InstallationInstallData Persistence Repositoryfrom PyPI:pipinstalldata_persistence_repositoryUsageBasic usage example (synchron):fromsqlalchemy.ormimportSessionfromdata_persistence_repository.sql_repositoryimportSqlRepositoryclassYourOwnRepository(SqlRepository):defspecial_query(self,session:Session):# execute a special query# Initialize the repositoryrepo=YourOwnRepository("sqlite:///your_database.db")# Use the repository within a managed contextwithrepo.start_session()assession:# Perform add, get, delete operations hererepo.special_query(session)Basic usage example (asynchron):fromsqlalchemy.ext.asyncioimportAsyncSessionfromdata_persistence_repository.sql_repository_asyncimportAsyncSqlRepositoryclassYourOwnRepository(AsyncSqlRepository):asyncdefspecial_query(self,session:AsyncSession):# execute a special queryawaitsession.execute(...)# Initialize the repositoryrepo=AsyncSqlRepository("sqlite:///your_database.db")# Use the repository within a managed contextasyncwithrepo.start_session()assession:# Perform add, get, delete operations hereawaitrepo.special_query(session)Replace"sqlite:///your_database.db"with your actual database URL.RequirementsPython 3.xDependencies listed inrequirements.txt.Running TestsExecute tests using pytest:python-mpytesttestsContributingContributions are welcomed! For substantial changes, please open an issue first to discuss what you'd like to change.
Please make sure to update tests accordingly. Also, each PR must have tests.LicenseMITFor more information on the repository pattern and its benefits in data persistence, please
visitRepository Design Pattern Information.
Package: dataphile
DataphileDataphile is a high-level python package for both data analysis and data processing. It started as
a central repository of common tasks and capabilities used by the author, but has now evolved into
something others might find useful. Seecomponentsbelow.Figure: Demonstration of Dataphile'sAutoGUIfeature.InstallationTo install Dataphile for general purposes use Pip:pip install dataphileIf you are using Anaconda, install using the above call to pipfrom inside your environment.
There is not as of yet a separate conda package.DocumentationDocumentation will be available atdataphile.readthedocs.io.
Currently, development of additional features is a priority, but this is a great place for
contributing to the project.ContributionsContributions are welcome  in the form of  suggestions for additional features,  pull requests with
new features or  bug fixes, etc. If you find  bugs or have questions, open anIssuehere. If and
when the project grows, a  code of conduct will be provided along side  a more comprehensive set of
guidelines for contributing; until then, just be nice.Road Mapadditional command line toolsMany additional  command line tools  are planned for future  releases including tools  that expose
database queries and filters.  Generally, just a massive extension of  the UNIX philosophy whereby
we can compose several functions together with pipes to make unique workflows.data acquisitionOne of  the motivations for this  package was to  provide an easy-to-use, high-level  interface to
collecting scientific  data from an externel  device (e.g., over  USB). This, along side  a simple
live  data visualization  feature would  go a  long  way for  high school  and university  student
laboratory courses to both aquire and analyze  their data using all open-source tools right inside
of aJupyter Notebook.documentation and package managementA quickstart guide along with full documentation of all components needs to be built using Sphinx.
Package: data-pipe
data_pipeFeaturesCross Thread Message Pipe:lock-freethread-safeexponential-back-offMessage Transfer End Points:any combination of: sync, trio, curio, asyncioBenchmarkPerformance on local CPU:cross-threadsync -> sync     @ 1.0 micros/object
sync -> curio    @ 5.0 micros/object
sync -> asyncio  @ 6.0 micros/object
sync -> trio     @ 7.0 micros/objectcross-processsync -> sync     @ 0.5 micros/object
sync -> curio    @ 0.7 micros/object
sync -> asyncio  @ 0.7 micros/object
sync -> trio     @ 0.7 micros/objectInstallTo install python package:sudo pip install data_pipeUsagecross-process, cross-frameworkrpc:basic_trunk.pybasic_trunk_test.pycross-process, cross-frameworkqueue:native_ruptor.pyxany_ruptor_perf.py
Package: datapipe-app
datapipe-appdatapipe-appimplements two aspects to make everydatapipepipeline to work as
an application:REST API + debug UI based of FastAPIdatapipeCLI toolCommon usageCommon pattern to usedatapipe-appis to createapp.pywith the following code:from datapipe_app import DatapipeApp

from pipeline import ds, catalog, pipeline

app = DatapipeApp(ds, catalog, pipeline)Wherepipelineis a module that defines common elements:ds,catalogandpipeline.REST API + UIDatapipeAppinherits fromFastApiapp and can be started with server likeuvicorn.uvicorn app:appUIREST APIAPI documentation can be found at/api/v1alpha1/docssub URL.CLIdatapipeCLI tool implements useful operations.rundatapipe run --pipeline appDoes full run of a specific pipeline.table listdatapipe table listLists all tables in pipeline.table reset-metadatadatapipe table reset-metadata TABLEResets metadata for a specific table: setsupdated_ts,processed_ts,hashto0.
Package: datapipe-core
Datapipedatapipeis a realtime incremental ETL library for Python application.DevelopmentAt the moment these branches are active:master- current development state, will be promoted to0.13.xseries
release once readyv0.13- current stable versionv0.11- legacy stable version (v0.12was skipped due reasons)Version compatibilityAt the moment, the datapipe library is under active development. Versions:v0.*.*It should be expected that each minor version is not backward compatible with
the previous one. That is,v0.7.0is not compatible withv0.6.1. Dependencies
should be fixed to the exact minor version.After stabilization and transition to the major versionv1.*.*, the common
rules will apply: all versions with the same major component are compatible.
Package: data-pipeline-api
pyDataPipelineWelcome to pyDataPipeline a Python api to interact with the Fair Data Pipeline.Full documention of the pyDataPipeline is avaialable athttps://www.fairdatapipeline.org/pyDataPipeline/InstallationpyDataPipeline can be installed from PyPi:pip3 install data-pipeline-apiOr from the Repository:git clone https://github.com/FAIRDataPipeline/pythonFDP.git

git checkout dev

pip3 install -e .NB. PyDataPipeline requires Python3.Example submission_scriptAssume FDP_CONFIG_DIR, storage_locations and objects have been set by CLI toolimport os
import fairdatapipeline as pipeline

token = os.environ['FDP_LOCAL_TOKEN']
config_dir = os.environ['FDP_CONFIG_DIR']
config_path = os.path.join(config_dir, 'config.yaml')
script_path = os.path.join(config_dir, 'script.sh')

handle = pipeline.initialise(token, config_path, script_path)

pipeline.finalise(token, handle)SEIRS Model ExampleThe SEIRS Model Example is available at:https://www.fairdatapipeline.org/pyDataPipeline/examples/SEIRS.html
Package: data_pipeline_avro_util
# Data Pipeline Avro UtilWhat is it?-----------The Data Pipeline Avro utility package provides a Pythonic interfacefor reading and writing Avro schemas. It also provides an enum classfor metadata that we've found useful to include in our schemas.Download and Install---------------------------```git clone git@github.com:Yelp/data_pipeline_avro_util.gitpip install data_pipeline_avro_util```Tests-----Running unit tests```make test```Usage-----Using Avro Schema Builder::```from data_pipeline_avro_util.avro_builder import AvroSchemaBuilderfrom data_pipeline_avro_util.data_pipeline.avro_meta_data import AvroMetaDataKeysavro_builder = AvroSchemaBuilder()avro_builder.begin_record(name="test_name",namespace="test_namespace",doc="test_doc")avro_builder.add_field(name = "key1",typ = "string",     # datatype of this field is stringdoc="test_doc1",metadata={AvroMetaDataKeys.PRIMARY_KEY: 1     # first primary key})avro_builder.add_field(name = "key2",typ = "string",doc="test_doc2")record_json = avro_builder.end()print record_json{"type": "record","namespace": "test_namespace","name": "test_name","doc": "test_doc","fields": [{"type": "string", "doc": "test_doc1", "name": "key1", "pkey": True},{"type": "string", "doc": "test_doc2", "name": "key2"}]}```Disclaimer-------We're still in the process of setting up this package as a stand-alone. There may be additional work required to run code and integrate with other applications.License-------Data Pipeline Avro Util is licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0Contributing------------Everyone is encouraged to contribute to Data Pipeline Avro Util by forking the Github repository and making a pull request or opening an issue.Documentation-------------The full documentation is atTODO (DATAPIPE-2030|abrar): upload servicedocs to public server.History-------0.1.0 (2015-01-29)++++++++++++++++++* First release.
Package: data-pipeline-tooling
IntroductionTODO: Give a short introduction of your project. Let this section explain the objectives or the motivation behind this project.Getting StartedTODO: Guide users through getting your code up and running on their own system. In this section you can talk about:Installation processSoftware dependenciesLatest releasesAPI referencesBuild and TestTODO: Describe and show how to build your code and run the tests.ContributeTODO: Explain how other users and developers can contribute to make your code better.If you want to learn more about creating good readme files then refer the followingguidelines. You can also seek inspiration from the below readme files:ASP.NET CoreVisual Studio CodeChakra Core
Package: datapiper
Datapiper provides a flexible easy-to-use library for constructing and running
simple data batch processing pipelines.Give Datapiper your list of data processing callables and it will construct a
runnable data pipeline for you.If you instantiate the pipe with a (iterable) data source, you get a generator
that reads from a source and outputs processed data for you:>>> operations = [lambda context, data: data+1]
>>> datasource = [1,2,3]
>>> p = Piper(operations, source=datasource)
>>> print p
pipe: source > <lambda>
>>> [r for r in p]
[2,3,4]If you instead instantiate it with a (callable) data sink, you get a coroutine
that accepts data from a producer and delivers processed data to a sink:>>> operations = [lambda context, data: data+1]
>>> results = []
>>> def datasink(data):
...    results.append(data)
>>> p = Piper(operations, sink=datasink)
>>> print p
pipe: <lambda> > sink
>>> for v in (1,2,3):
...    p.send(v)
...
>>> results
[2,3,4]The context parameter passed to the data operations callables is meant for
sharing state between them. It can be initialized to desired value(s) by passing
it to the Piper class as a (optional) keyword argument. The context parameter can
be anything; a dictionary is recommended.Please see the tests for more examples.History0.1.0 (2017-10-31)First release on PyPI.
Package: datapkg
Introductiondatapkg is a python library and command line tool for working withData
Packagesand interacting with data hubs likeCKAN.For more information visit the documentation at:http://packages.python.org/datapkg/InstallationSee doc/install.rst or the online documentation.LicenseSee doc/license.rstFor DevelopersBuilding the documentation.You need sphinx (python-sphinx) to do this. Now you can build the docs:python setup.py build_sphinxRunning tests. We suggest you use nose:nosetests datapkg/tests/To exclude tests depend on access to the internet:nosetests -a \!__external__ datapkg/tests
Package: datapkggui
Descriptiondatapkggui is a graphical frontend for the datapkg tool (http://okfn.org/projects/datapkg/). It is aiming
to become the standard CKAN (http://ckan.net/) graphical tool for searching, fetching and distributing open data sets.It is a 10 weeks project started on 2011-09-26 and developed by
Daniel Graziotin <d@danielgraziotin.it> (http://task3.cc) during a stage
performed at the Fondazione Bruno Kessler (http://fbk.eu).Please take a look at docs/report.{odt, pdf} for many other info.WebsiteThe official website of the project ishttp://task3.cc/projects/datapkggui.
The source code is hosted on GitHub (http://github.com/dgraziotin/datapkggui)