Saravanan666/pypi-org huggingface dataset:
Package: data_dispenser
==============data-dispenser==============.. image:: https://travis-ci.org/catherinedevlin/data-dispenser.svg?branch=master:alt: Travis build statusGiven a source of rowlike data, acts as a generator of OrderedDicts.Usage::src = Source('mydata.csv')for row in src:print(row)data-dispenser thus serves as a single API for a variety of data sources.* Free software: MIT licenseData source types supported...........................* file names / paths* open file objects* pymongo Collection objects* strings interpretable as data* URLs beginning with http:// or https://Will work most reliably against filenames with extensions that indicatethe data format; otherwise data-dispenser may guess the input format wrong.Data input formats supported............................* csv* yaml (requires ``pyyaml``)* json* pickle* ``eval``-able Python* xls* xml (experimental)* HTML with ``<table>``sMultiple files..............File paths with wildcards will beeffectively concatenated into one large data source.Load limits...........Large data sources could overwhelm your system's memory.  Passing a ``limit``keyword to the ``Source`` constructor limits the rows returned from eachsource.  For file paths with wildcards, the limit applies to each filesource, not to the number of file sources.Code----https://pypi.python.org/pypi/data_dispenserSource and bug tracker----------------------https://github.com/catherinedevlin/data-dispenserHistory-------0.1.0 (2014-05-21)++++++++++++++++++* First release on PyPI.0.1.1 (2014-05-23)++++++++++++++++++* Fixed bugs in handling non-listlike YAML files0.2.0 (2014-07-14)++++++++++++++++++* Support .xls* Support URLs* Support wildcards0.2.1 (2014-27-14)++++++++++++++++++* Support .html0.2.2 (2014-27-14)++++++++++++++++++* Fix bug with .html tables with <thead>0.2.3 (2014-09-14)++++++++++++++++++* Fix for URLs with no file extension0.2.4 (2014-09-14)++++++++++++++++++* Better error messages on parse failure0.2.5 (2015-02-01)++++++++++++++++++* Directly interpret SQLAlchemy sources0.2.5.1 (2015-02-04)++++++++++++++++++++* Emit SQLAlchemy tables in order required by foreign keys
Package: data-displays
data_displaysAsynchronus loading bars and moreExample:from data_diplays import data

if __name__ == "__main__":
	from os import get_terminal_size
	from time import sleep

	data.bar.create("Download progress", duration=10, length=get_terminal_size()[0], lineno=1, size=250, unit="kb")
	data.bar.create("Working", duration=5, length=50, lineno=3, size=125, unit="bytes")
	data.bar.create("Decoding text", duration=5, length=25, lineno=5, size=2000)
	sleep(3) # wait one second
	data.bar.create("Uploading data", duration=10, length=15, lineno=7)
	data.wait() # Wait until all displays are complete (This is absolutely necessary. It must be called after all the displays are created.)
	print("Done!")
Package: datadivecsv
datadivecsv: AnÃ¡lisis Exploratorio de Datos para CSVEl mÃ³dulo datadivecsv proporciona una herramienta sencilla y eficaz para realizar un anÃ¡lisis exploratorio bÃ¡sico de datos contenidos en archivos CSV. Este mÃ³dulo es ideal para obtener una rÃ¡pida comprensiÃ³n de los datos, especialmente Ãºtil en las etapas iniciales de cualquier proyecto de anÃ¡lisis de datos.CaracterÃ­sticasLectura de Archivos CSV: Capacidad para leer archivos CSV desde el directorio actual o subdirectorios especÃ­ficos (datasets, notebooks).NormalizaciÃ³n de Nombres de Columnas: OpciÃ³n para convertir nombres de columnas a snake_case, facilitando el manejo de los datos.VerificaciÃ³n y EliminaciÃ³n de Duplicados: Chequea y elimina filas duplicadas en el conjunto de datos para asegurar la precisiÃ³n del anÃ¡lisis.Resumen del DataFrame: Proporciona un resumen exhaustivo del DataFrame, incluyendo las primeras y Ãºltimas filas, una muestra aleatoria de filas, informaciÃ³n del DataFrame, estadÃ­sticas descriptivas, valores faltantes, y mÃ¡s.VisualizaciÃ³n de Datos: Genera histogramas para variables numÃ©ricas y mapas de calor de correlaciÃ³n, ademÃ¡s de anÃ¡lisis de variables categÃ³ricas.UsoPara utilizar datadivecsv, simplemente importa la funciÃ³n execute_analysis y pÃ¡sale el nombre del archivo CSV que deseas analizar. AquÃ­ tienes un ejemplo bÃ¡sico de uso:fromdatadivecsvimportexecute_analysisdf=execute_analysis('tu_archivo.csv',normalize_cols=True,check_duplicates=True,show_summary_flag=True)ParÃ¡metros de execute_analysisfile_name: Nombre del archivo CSV a analizar.normalize_cols(opcional): Booleano para normalizar los nombres de las columnas a snake_case (por defecto True).check_duplicates(opcional): Booleano para verificar y eliminar duplicados (por defecto True).show_summary_flag(opcional): Booleano para mostrar un resumen del DataFrame (por defecto True).InstalaciÃ³nPuedes instalar este paquete usando pip:pipinstalldatadivecsvContribuirLas contribuciones son bienvenidas. Por favor, revisa lasissuesen GitHub para ver cÃ³mo puedes contribuir.LicenciaEste proyecto estÃ¡ bajo la Licencia MIT.
Package: datadm
dataDM💬📊DataDM is your private data assistant. A conversational interface for your data where you can load, clean, transform, and visualize without a single line of code. DataDM is open source and can be run entirely locally, keeping your juicy data secrets fully private.Demohttps://github.com/approximatelabs/datadm/assets/916073/f15e6ab5-8108-40ea-a6de-c69a1389af84Note: Demo above isGPT-4, which sends the conversation to OpenAI's API. To use in full local mode, be sure to selectstarchat-alpha-cudaorstarchat-beta-cudaas the model. This will use the StarChat model, which is a bit less capable but runs entirely locally.⚠️ LLMs are known to hallucinate and generate fake results. So, double-check before trusting their results blindly!⇒Try it now! Hosted public environment is live! (Click Here)⇐Don't put any sensitive data in the public environment, use the docker image or colab notebook for your private conversations.Join ourdiscordto join the community and share your thoughts!FeaturesPersistent Juptyer kernel backend for data manipulation during conversationRun entirely locally, keeping your data privateNatural language chat, visualizations/plots, and direct download of data assetsEasy to use docker-images for one-line deploymentLoad multiple tables directly into the chatSearch for data and load CSVs directly from githubOption to use OpenAI's GPT-3.5 or GPT-4 (requires API key)WIP: GGML based mode (CPU only, no GPU required)WIP: Rollback kernel state when undousingcriu(re-execute all cells)TODO: Support for more data sources (e.g. SQL, S3, PySpark etc.)TODO: Export a conversation as a notebook or htmlThings you can ask DataDMLoad data from a URLClean data by removing duplicates, nulls, outliers, etc.Join data from multiple tables into a single output tableVisualize data with plots and chartsAsk whatever you want to your very own private code-interpreterQuickstartYou can use docker, colab, or install locally.1. Docker to run locallydockerrun-eOPENAI_API_KEY={{YOUR_API_KEY_HERE}}-p7860:7860-itghcr.io/approximatelabs/datadm:latestFor local-mode using StarChat model (requiring a CUDA device with at least 24GB of RAM)dockerrun--gpusall-p7860:7860-itghcr.io/approximatelabs/datadm:latest-cuda2. Colab to run in the cloud3. Use as a python package⚠️ datadm used this way runs LLM generated code in your userspaceFor local-data, cloud-model mode (no GPU required) - requires an OpenAI API key$pipinstalldatadm
$datadmFor local-mode using StarChat model (requiring a CUDA device with at least 24GB of RAM)$pipinstall"datadm[cuda]"$datadmSpecial Thanksstarchat-beta(starcoderwithdatabricks-dollyandOpenAssistant/oasst1)GuidanceHuggingFaceOpenAIContributionsContributions are welcome! Feel free to submit a PR or open an issue.CommunityJoin theDiscordto chat with the teamCheck out our other projects:sketchandapproximatelabs
Package: data-documenter
Data DocumenterA package to generate documentation frompanderaschema usingmkdocs material.Installationpipinstalldata-documenterQuick startCreate schema usingpanderafrompanderaimportDataFrameSchema,Columnschema=DataFrameSchema(title="Title of dataset",description="Description of dataset",columns={"COLUMN_NAME":Column(dtype="str",nullable=False,unique=True,description="Description of column",title="Column Name"),},)Create documentation using schemafromdata_documenter.pandera_pluginimportcreate_documentationdocs=create_documentation(schema,docs_path='docs_folder',title='page title',filename='index.md')Run serverdocs.run()Stop serverdocs.stop()It is better to usemkdocscommand instead ofrun,stopfor better control of processes. See alternative usage.cdpath/to/folder
mkdocsserveBuild and deploy documentationSeemkdocsdocumentation.Alternative usageCreate pandera schemafrompanderaimportDataFrameSchema,Columnschema=DataFrameSchema(title="Title of dataset",description="Description of dataset",columns={"COLUMN_NAME":Column(dtype="str",nullable=False,unique=True,description="Description of column",title="Column Name"),},)Create new folder for documentation server.fromdata_documenter.metadocsimportMetaDocsdocs=MetaDocs(docs_path='my_docs')docs.new()This will simply runmkdocs new my_docsand replacemkdocs.ymlfile.Save documentation for schema in a markdown file.fromdata_documenter.pandera_pluginimportpandera_to_markdowntext_markdown=pandera_to_markdown(schema,title='Homepage')docs.save_markdown(text_markdown,filename='index.md')See generated documentation.cd my_docs
mkdocs serveBuild and deploySeemkdocsdocumentation.
Package: datadog-checks-base
Datadog Checks BaseOverviewThis package provides the Python bits needed by theDatadog Agentto run Agent-based Integrations (also known asChecks).This package is used in two scenarios:When used from within the Python interpreter embedded in the Agent, it
provides all the base classes and utilities needed by any Check.When installed in a local environment with a regular Python interpreter, it
mocks the presence of a running Agent so checks can work in standalone mode,
mostly useful for testing and development.Please refer to thedocsfor details.InstallationChecks fromintegrations-corealready
use the toolkit in a transparent way when you run the tests but you can
install the toolkit locally and play with it:pipinstalldatadog-checks-baseTroubleshootingNeed help? ContactDatadog support.
Package: datadog-custom-logger
Datadog Custom LoggerUsageStepsInstall pip packagepipinstalldatadog-custom-loggerImport packagefromdatadog_custom_loggerimportDatadogCustomLogHandlerInitialize the handlerdatadog_custom_handler=DatadogCustomLogHandler(level=logging.INFO)💡Note: if the level is set to logging.WARNING, you won't be able to see info or debug level logs.Hierarchy:- debug (logging.DEBUG)
- info (logging.INFO)
- warning (logging.WARNING)
- error (logging.ERROR)Attach the handlerlogging.basicConfig()logger=logging.getLogger()logger.addHandler(datadog_custom_handler)logging.getLogger().setLevel(logging.INFO)Now simply log the logs# This statement won't be logged because the .setLevel() is set to logging.INFO.# Please check the logging hierarchy for more.logging.debug("This is debug level code")# This will be logged as per .setLevel()logging.info("This is info level logs")Complete example:
Package: datadog-deployer
A simple command line utility to deploy datadog monitors from code.dumpdatadog-deployer dumpdumps the current datadog monitor definitions to thefilenameOptions:--accountTEXTname of the Datadog account.--filenamePATHto dump the monitors to.--helpShow this message and exit.deploydatadog-deployer deployDeploys the datadog monitor definitions fromfilename. Compares the monitors with the deployed monitors and
determines for each monitor whether to insert or update. Monitors defined in datadog which are not in defined in the file are
left, unless–force-deleteis specified.Options:--accountTEXTname of the Datadog account.--filenamePATHto deploy monitors from.--verboseshowing change details that are applied.--dry-runonly show changes that would be applied.--force-deletemonitors in datadog not defined in the file.--helpShow this message and exit.ExampleDump the current monitor definitions:$ datadog-deployer dump --filename dd-monitors.yamlUpdate the monitor definitions:$ datadog-deployer deploy --filename dd-monitors.yaml
INFO: 5 inserts, 1 updates, 2 unmanaged and 33 unchanged.
INFO: "VPN connectivity" not defined in file. use --force-delete to delete.
INFO: "Invalid objects in Oracle" not defined in file. use --force-delete to delete.If you want to delete unmanaged monitors, type:$ datadog-deployer deploy --filename dd-monitors.yaml --force-delete
INFO: 0 inserts, 0 updates, 2 deletes and 39 unchanged.File formatsThe file ~/.datadog.ini is a Python configuration file from which the Datadog connection parameters are read.  At least it will
need theapi_keyandapi_appattributes in the section DEFAULT:[DEFAULT]
api_key=a77aaaaaaaaaaaaaaaaaaaaa
app_key=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbOther attributes you can add are:proxies,api_host,statsd_host,statsd_port,statsd_socket_path,cacert,mute
Package: datadog-export
A simple command line utility to export datadog metrics.credentialsThe file ~/.datadog.ini is a Python configuration file from which the Datadog connection parameters are read.  At least it will
need theapi_keyandapi_appattributes in the section DEFAULT::[DEFAULT]
api_key=a77aaaaaaaaaaaaaaaaaaaaa
app_key=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb
Package: datadog-exporter
A simple command line utility to export datadog metrics.eventsto export datadog events that where reported in the last 24 hours, type:datadog-exporter events \
    --iso-datetime \
    --pretty-print \
    --start-time -24hmetric namesto export all available database metric names, type:datadog-exporter namesmetricsto export specific metrics from the last 24 hours, type:datadog_export metrics \
    --iso-datetime \
    --pretty-print \
    --start-time -2h \
    --window 30m \
    'docker.cpu.system{*}'The--windowoption allows you to influence the resolution of the values returned.credentialsAdd your Datadog Application and API key in the file$HOME/.datadog.iniin the
section DEFAULT:[DEFAULT]
api_key=a77aaaaaaaaaaaaaaaaaaaaa
app_key=bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbAlternatively, you can set the environment variableDATADOG_API_KEYandDATADOG_APP_KEY.
Package: datadog-http-handler
Python logging module allowing you to log directly to Datadog via https.
Currently support only for python 3.6+.InstallationInstallpythonInstallpipcurlhttps://bootstrap.pypa.io/get-pip.py-oget-pip.pypythonget-pip.pyInstall datadog-http-handlerpipinstalldatadog-http-handlerUsageThe typical usage of this library to output to datadogfromdatadog_http_handlerimportDatadogHttpHandlerlogger=DatadogHttpHandler(api_key=os.getenv('DATADOG_API_KEY',''),service='test',host='your_hostname',logger_name='example',tags={'env':'test','user':'Tim the Enchanter'}).loggerlogger.info('Hello World')Since sending logs to datadog via http can issue in a request failure for a myriad of reasons, it would be
good to know if the failure accord.  By setting raise_failure=True when creating the DatadogHttpHandler
object, an exception will now be thrown if the status code of the response is not 200.  See below:importtracebackfromdatadog_http_handlerimportDatadogHttpHandlerlogger=DatadogHttpHandler(api_key=os.getenv('DATADOG_API_KEY',''),raise_exception=True,service='test',host='your_hostname',logger_name='example',tags={'env':'test','user':'Tim the Enchanter'}).loggertry:logger.info('Hello World')exceptExceptionase:print(traceback.format_exc())LICENSEMITAUTHORSNRCCUA Software EngineersSee also the list ofcontributorswho participated in this project.ACKNOWLEDGEMENTSBryan Cusatis- NRCCUA Architecture Team LeadTim Reichard- NRCCUA Architecture Team MemberGrant Evans- NRCCUA DevOps Engineering Lead
Package: datadog-log
Datadog-logWrite out JSON formatted logs in the format and with
theattributesthat Datadog expectsInstallationpip install datadog-logSetupThe simplest way is to useinit_logging()which will log to stdout.If you want finner control then you can useDatadogFormatterdirectly. Seeinit_loggingfor example usage.LoggingUse logging as normal:importloggingimportdatadoglogdatadoglog.init_logging()log=logging.getLogger("example")log.setLevel(logging.INFO)log.info("Some info",extra={"extra":["json"]})The above would output json like below (but as a single line, shown 'pretty' here for readability):{"message":"Some info","extra":["json"],"timestamp":"2021-09-18T21:14:07.707371+00:00","status":"info","logger":{"name":"example","method_name":"<stdin>.<module>","thread_name":"MainThread"}}
Package: datadog-logger
DataDog LoggerNote: This library does not send logs to Datadog's Log Management product. See
Datadog's documentation for how to configure log collection. Seehttps://docs.datadoghq.com/logs/log_collection/A Pythonlogging.Handlerfor sending log messages to DataDog as Events in
the Events Explorer.Installationpip install datadog-loggerUsageThe simplest way to enable logging to DataDog is to use thelog_error_eventshelper, which will cause alllogging.ERRORand
higher messages to be sent to DataDog:importdatadogfromdatadog_loggerimportlog_error_eventsimportlogging# Authenticate with DataDogdatadog.initialize(api_key="api-key",app_key="app-key")# Note, a normal STDOUT handler will not be configured if this is not# called firstlogging.basicConfig()log_error_events(tags=["tag1:value","tag2:value"],mentions=["@devs","@slack"])logging.error("Oh no!")This will enable the handler on the root logger;tagsandmentionsmay both beNone, in which case none will be included. The handler
will have a log level oflogging.ERROR, meaning onlyERRORandCRITICAL(or any higher custom levels) will be sent to DataDog.A specific logger may also be specified by name:log_error_events("some.logger")DetailsTheHandlermay also be created and added to a logger manually:importdatadogfromdatadog_loggerimportDatadogLogHandlerimportloggingdatadog.initialize(api_key="api-key",app_key="app-key")datadog_handler=DatadogLogHandler(tags=["some:tag"],mentions=["@some-mention"],level=logging.WARNING)# Enable STDOUT logginglogging.basicConfig()# The root loggerlogger=logging.getLogger()logger.addHandler(datadog_handler)# This will be logged, because the level is WARNINGlogger.warning("Watch out!")# This will notlogger.info("Lets not log everything to DataDog")When the event is created, themsgof the log record will be passed
as the title of the event. Normally, this would be the string passed
into the logging call, with any%substitutions applied.The text of the event will be the formatted record. If the log
record includesexc_info(i.e. because you calledlogging.exception, or passedexc_infoto the log function), then
the full stack trace will be included in the event text.Any@mentionspassed via thementionsconstructor argument will be
appended to the end of the text, so this library can be used to alert,
via DataDog, on particular log messages.
Package: datadog-muted-alert-checker
To check muted alert for production in datadog and send alert to opsgenie if anyHow To Use this module in your local hostTo use the script you should clone this repositoryMake sure you already have python 3 installed on your local hostCreate new spreadsheet for the report and get the id of the spreadsheetPut the id of spreadsheet into the script that have been marked “redacted”Export OPSGENIE_API_KEY variable to your env and add opsgenie api key value in that variable like example below.ExportOPSGENIE_API_KEY=`apikeyfromopsgenie`ExportDATADOG_API_KEY=`apikeyfromdatadog`ExportDATADOG_APP_KEY=`appkeyfromdatadog`Install python dependencies requirements to your local hostpipinstall-rrequirements.txtExecute this main python scriptpython3package/main.py
Package: datadog-pandas
Datadog PandasPython library for querying Datadog and accessing the results asPandasdata structures.This is mostly intended for use inJupyternotebooks.ExampleEvaluate an instant query at a single point in time:>>>fromdatadog_pandasimportquery>>>>>>dd_options={"api_key":"XXX","app_key":"XXX"}>>>p=query.DatadogMetrics(dd_options)>>>p.query('sum:caseflow.ct.attachments.requests{*}.rollup(sum, 86400)','2020-05-10T00:00:00Z','2020-05-12T00:00:00Z')InstallationLatest release viapip:pipinstalldatadog-pandas[--user]via Git:gitclonehttps://github.com/LemontechSA/datadog-pandas.git;cddatadog-pandas
python3setup.pyinstall[--user]LicenceLicenced under theMIT License. SeeLICENSEfor details.
Package: datadog-serverless-utils
DataDog Serverless UtilsUtilities for integration between serverless execution environments and DataDogError Tracking for Serverless Execution EnvironmentsServerless execution environments (e.g. Cloud Functions) are finished by the cloud provider as soon as the execution
finished. Sometimes, the DataDog tracer(ddtrace)does not
have a chance to send the collected traces especially when it ends abruptly due to an unhandled exception.This library offers a decorator to make sure ddtrace can send the telemetry data before the execution ends.SetupMake sure ddtrace config is correctly set up as mentioned on thedocs.UsageDecorate your function as follows and voilà!from datadog_serverless_utils import datadog_serverless


@datadog_serverless
def main():
    ...A return value can be specified for the case when the decorated function raises an unhandled exception via the parametererror_return_value:@datadog_serverless(error_return_value="my_value")
def main():
    ...
Package: datadraw
DataDraw is a python function library class to produce many kinds of automated,presentable, data-driven 2-D plots / charts / graphics / image annotations in SVG.It's an all-python server-side solution, producing SVG results that can beembedded directly into html web pages or saved as .svg image files.No javascript, css, or svg knowledge is required.Axis and legend rendering, tooltips, linkouts, automatic ranging, basic statistics,draw primitives.  Handles numeric, log, categorical, and datetime data types.Leverages SVG's fonts, colors, transparency, image handling, and other aspects.Works well within frameworks such as Flask or Django.Limited interactivity / reactivity use-cases.Tested on linux python 3.6 and 3.9 using recent browser versions; believed to be thread-safe.