Saravanan666/pypi-org huggingface dataset:
Package: ddtrace-asgi
ddtrace-asgiUnofficialddtraceintegration for ASGI apps and frameworks.Should work seamlessly for any ASGI web framework, e.g. Starlette, FastAPI, Quart, etc.Note: This project is in alpha stage.RequirementsPython 3.6+.ddtracemust be installed to use theddtrace-runcommand.TheDatadog Agentmust be installed and running for traces to be effectively sent to Datadog APM.Installationpipinstallddtrace-asgiQuickstartTo automatically send traces toDatadog APMon each HTTP request, wrap your ASGI application aroundTraceMiddleware:# app.pyfromddtrace_asgi.middlewareimportTraceMiddlewareasyncdefapp(scope,receive,send):assertscope["type"]=="http"headers=[[b"content-type",b"text/plain"]]awaitsend({"type":"http.response.start","status":200,"headers":headers})awaitsend({"type":"http.response.body","body":b"Hello, world!"})app=TraceMiddleware(app,service="asgi-hello-world",tags={"env":"local"},)Then useddtrace-runwhen serving your application. For example, if serving with Uvicorn:ddtrace-runuvicornapp:appFor more information on usingddtrace, please see the officialdd-trace-pyrepository.ExamplesStarlettefromddtrace_asgi.middlewareimportTraceMiddlewarefromstarlette.applicationsimportStarletteapp=Starlette()app.add_middleware(TraceMiddleware,service="my-starlette-app")FastAPIfromddtrace_asgi.middlewareimportTraceMiddlewarefromfastapiimportFastAPIapp=FastAPI()app.add_middleware(TraceMiddleware,service="my-fastapi-app")API ReferenceTracingMiddlewareclassTracingMiddleware:def__init__(self,app,tracer=None,service="asgi",tags=None,distributed_tracing=True):...An ASGI middleware that sends traces of HTTP requests to Datadog APM.Parametersapp- An [ASGI] application.tracer-(optional)ATracerobject. Defaults to the globalddtrace.tracerobject.service-(optional)Name of the service as it will appear on Datadog.tags-(optional)Default tags attached to all request spans. Either a dictionary, or a string of comma-separated tags (e.g."env:staging, app:shop"). See alsoTagging.distributed_tracing-(optional)Whether to enabledistributed tracing.ChangelogAll notable changes to this project will be documented in this file.The format is based onKeep a Changelog.0.3.0 - 2019-11-15AddedAllow to set default tags on all request spans. (Pull #24)0.2.2 - 2019-11-03AddedNow ships with binary distribution (wheel) in addition to source distribution (sdist). (Pull #16)0.2.1 - 2019-10-31FixedImprove resilience to ASGI protocol violations. (Pull #11)0.2.0 - 2019-10-30ChangedThetracer,serviceanddistributed_tracingparameters ofTraceMiddlewareare now keyword-only. (Pull #10)AddedThetracerforTraceMiddlewareis now the globalddtrace.tracerby default. (Pull #10)0.1.0 - 2019-10-23Initial release.AddedAddTracingMiddleware.
Package: ddtruss
Data-driven solver for truss structuresThis repository provides a Python implementation of the data-driven solver for truss structures described inKirchdoerfer, T., & Ortiz, M. (2016). Data-driven computational mechanics. Computer Methods in Applied Mechanics and Engineering, 304, 81-101.Some notebook examples can be found inexamples.Documentation is availablehere.InstallationTo installddtruss, you are invited to usepipand its associated optionspip install -U ddtrussTestingTo run theddtrussunit tests, check out this repository and typepytestLicenseddtrussis published under theMIT license.
Package: ddui
# Airflow's DataDriver plugin## from Pandas' dataframes to Airflow pipelines#### WHY :In a machine learning project, there is a recurring problemwith the difference between local interactive modeling source codeand production pipelines source code.It is very error prone and, as a consequence, time consuming because weswitch constantly between experimentation and production.The Datadriver project aims to solve this issue by making the glue code **based on Pandas and sklearn**for modelization, **and on Airflow** for automation, scheduling, and monitoring of trainingand predicting pipelines.#### Plugin description**Datadriver UI (ddui)** is the Airflow's plugin we developed to track our models.Combined with the Datadriver's API (pyddapi), it offers a DAG view to track machine learning workflow (or dataflow).More specifically, it shows the **Output** of any Airflow's Task with a lot of metrics andcharts :- choose a DAG to track![img/ddui_titan1.png](img/ddui_titan1.png)- select a task to see charts and describe metrics on the output_table![img/ddui_titan3.png](img/ddui_titan3.png)- look at histograms to verify if columns are correct (distributions, number of NAs,unique values, etc...)![img/ddui_titan2.png](img/ddui_titan2.png)## Getting startedgit clone git_url_of_this_project && cd this_projectlocal install :pip install -e .ddui installdocker install :./run_docker.sh## Package modulesddui/dash_app -> the application defined like a Dash application, with callbacks and event handeling. It is imported in plugin.py laterdash_components -> html custom components like a Panel or an Alert Divorm -> function to access the Airflow metastore and retrieve DAGs list and infosplot -> functions using plotly, they return a Graph objectplugin -> defines the DataDriverUI plugin that implements Airflow's Plugin interface https://airflow.apache.org/plugins.html#interfaceviews -> a FlaskAdminView that implements Dash too, to have the ability to include plotly charts in Airflow###### dependencies graph![pydeps ddui](img/dependencies_analysis.png)## Developer setupThere is an existing DAG in tests/dags that mocks the behavior of Datadriver's API, butwithout any dependency to pyddapi.You can use it to develop the User Interface, using the script located in tests/dev_tools.cd tests/dev_toolspython run_webserver.pyIt runs the Airflow's webserver, and it overrides the AIRFLOW__CORE__DAGS_FOLDER to look into tests/dags.### Setup your virtual envvirtualenv venvsource venv/bin/activatepip install -e .pip install -r ci/tests_requirements.txtddui install
Package: ddump
ddumpddump(Data Dump)数据转存工具。主要解决以下问题：数据库的增量转存API数据转存的通用模式数据的本地文件组织方案本工具定位是数据下载，文件目录和文件名的组织方式以实现增量下载和减少下载量为首要目标，读取是否方便为次要目标。用户可能需要根据自己的使用习惯，将数据转存成其它格式，如导入到数据库等为何使用文件存储，而不用数据库没有表结构的情况下，to_sql保存格式的效率很低，提前准备表结构又麻烦金融类数据特殊，并不需要随机访问。全量加载或按日期加载都是更常用的方法数据备份分享时，文件更方便为何采用Parquet文件格式csv格式，文本格式，读写慢，容易丢失精度pickle格式，只能在Python下使用HDF5格式，强大灵活，跨语言parquet格式，列式存储，支持直接读取文件夹。跨语言，常用于大数据处理安装pip install ddump -ihttps://pypi.tuna.tsinghua.edu.cn/simple--upgrade # 国内镜像下载pip install ddump -ihttps://pypi.org/simple--upgrade # 国外官方源下载开发pip install -e .数据库转存请访问数据库转存文档API转存请访问API转存文档数据库工具在开发本项目时，提炼了一个数据库ORM工具，它是对sqlalchemy的进一步封装，简单易用，可以直接映射已经存在的表。使用方法仿照聚宽的数据接口。fromddump.db.toolimportDbTooldb=DbTool(url="mysql+pymysql://user:pasword@127.0.0.1:3306/tushare?charset=utf8")db.show_tables()db.describe('FDT_STK_AUDIT')q=db.query(db.FDT_STK_AUDIT).limit(10)df=db.run_query(q)df样例参考examples，内有常见的几个库的调用示例，欢迎大家提供更多的案例
Package: ddup
ddupCompare images in two image lists and find dulplicate ones.InstallpipinstallddupUsageUse it in command lineddup{--list1img1img2img3|--path1imglist_path}\[{--list2img1img2img3|--path2imglist_path}]\[--outoutput_dir][--log]Compare result will be sved inddup_output.jsonExample 1Compare images in list file1 to those in list file2 and save results in
the specified fodler.ddup--path1imglist1.txt--path2imglist2.txt--out/mnt/StorageInput for Ex1path1imglist1.txt/mnt/Storage/test1/000001.jpg
/mnt/Storage/test1/000002.jpgpath2imglist2.txt/mnt/Storage/test2/000001.jpg
/mnt/Storage/test2/000002.jpg
/mnt/Storage/test2/000003.jpgout/mnt/StorageOutput for Ex1hash1.hdf5Store the hashes of images in list1 in .hdf5 format.hash2.hdf5Store the hashes of images in list2 in .hdf5 format.ddup_output.jsonStore the comparision results in json format. Each image in list1
will correspond to one or more images in list2 if they are similar.{"/mnt/Storage/test1/000001.jpg":["/mnt/Storage/test2/000001.jpg"],"/mnt/Storage/test1/000002.jpg":["/mnt/Storage/test2/000002.jpg","/mnt/Storage/test2/000003.jpg"]}Example 2Compare a list of images to themselves.You can give a single list, or give two same lists.ddup--list11.jpg2.jpg3.jpg4.jpg
ddup--list11.jpg2.jpg3.jpg4.jpg--list21.jpg2.jpg3.jpg4.jpgInput for Ex2["1.jpg","2.jpg","3.jpg","4.jpg"]Output for Ex2hash1.hdf5Store thephashes of images in list1 in .hdf5 format.ddup_output.jsonStore the comparision results in json format.For self comparison, similar images will be orginized into groups
with the first image in the group be the key and the whole group be the
value.{"1.jpg":["1.jpg","2.jpg","3.jpg"]}Parameterslist1andpath1are considered asinput1.One and only oneof them must be provided.list2andpath2are considered asinput2.None or oneof them can be provided.If none of them are provided,input1will be compared withitself.If one of them is provided,input1will be compared withinput2.list1Directly give paths of several images.--list11.jpgimg/2.jpgimg2/3.jpgpath1Path of the first image list file.If there are many images to be compared with, an image list file can be
provided instead.Image paths in the list should beAbsolute path.--path1imglist1.txtlist2Same aslist1path2Same aspath1out[optional]To specify a folder to save the results files.The folder will be created if it does not exist.Default is folderddup_outputin current path.--log[optional]With this option added, the program will print detail log for each thread and
each dulplicate image pair.
This may cause message flush on screen so it is recommended to pipe
it in to a log file.
Package: ddv-logging
DDV LoggingPython logging formatters, filters, and convenience methods.GitLab:https://gitlab.com/davidevi/ddv-loggingPyPi:https://pypi.org/project/ddv-logging/Features:Can enable printing of logs tostdoutCan indent logs based on position in stack traceCan colour the log levelCan filter out modules from the log output based on verbosity levelsUsage ExampleThe code:importloggingfromddv.loggingimportlog_to_stdoutlogger=logging.getLogger(__name__)classA(object):def__init__(self):self.logger=logging.getLogger(__name__+".A")self.logger.debug("Creating instance of A")self.b=B()self.A1()defA1(self):self.logger.info("A1 has been called")classB(object):def__init__(self):self.logger=logging.getLogger(__name__+".B")self.logger.debug("Creating instance of B")self.c=C()self.B1()defB1(self):self.logger.info("B1 has been called")classC(object):def__init__(self):self.logger=logging.getLogger(__name__+".C")self.logger.debug("Creating instance of C")self.C1()defC1(self):self.logger.info("C1 has been called")defmain():verbosity_filters={# Verbosity 0 will display none1:["__main__.A"],# Verbosity 1 will display A2:["__main__.B"],# Verbosity 2 will display A and B3:["__main__.C"]# Verbosity 3 will display A, B, and C}log_to_stdout(logging_level=logging.DEBUG,enable_colours=True,enable_indentation=True,verbosity_filters=verbosity_filters,verbosity_level=2,)logger.info("Main has been called")A()logger.warning("Execution complete")if__name__=="__main__":main()The output:Verbosity levels could come from the command line:program --arg1 --arg2 -vvvWhere the number ofvdictates the verbosity level.You could simply use the length of-vvvto determine verbosity level.
Package: ddv-settings
DDV SettingsModule for easily reading and exporting settings from filesExamplesettings.ini[a]
b = c
d = e

[f]
g = h
j = kCode that loads settings fromsettings.iniand uses them:import os
import sys
from ddv.settings import read, export

read("settings.ini")
export(sys.modules[__name__], True, "TEST")

print("Variable TEST_A_B == c")
print(TEST_A_B)

print("Env Var TEST_F_G == h")
print(os.environ.get("TEST_F_G"))Output:Variable TEST_A_B == c
c
Env Var TEST_F_G == h
h
Package: ddwrtdb
Simple CLI to search on DD-WRT router database and retrieve router information.You can export all the results to any tabular format supported by therows
library(such as CSV, XLS, XLSX, HTML an
SQLite).To install, just run:pip installhttps://github.com/turicas/rows/archive/develop.zippip install ddwrtdbIf you want support forXLS, also run:pip install ddwrtdb[xls]If you want support forXLSX, also run:pip install ddwrtdb[xlsx]The command-line interface is pretty simple to use:read the README.
Package: ddx-python
DDX-PythonRust bindings and utils for use in PythondecimalPython bindings forrust_decimal. Supports comparison, addition, subtraction, multiplication, division, and remainder. Constructor only supportsstr.Usage:$ python
>>> from ddx_python.decimal import Decimal
>>> d1 = Decimal("8631989.747461568422879160")
>>> d2 = Decimal("12354.867325583148587639999525")
>>> d3 = Decimal("8644344.6147871515714668000000")
>>> d1+d2 == d3
True
>>> d1+d2
Decimal('8644344.614787151571466800000')
Package: dd-yandex-pay
DD Yandex PayWrapper forYandex Pay API.For a detailed description of how to work with the client, read theusagesection in thedocumentation.You can leave any suggestions for the development of the project indiscussionson GitHub.If you come across a bug, please open anissue.If you would like to contribute to the development of the project, please read thecontributionsection in the documentation.
Package: de1
de1Curated collection of DE1's favorite kedro utilities.EmptyPartitionedDataSetFor those times when data is not yet available in a particular folder, or if no data is a valid value.Particularly useful when doing sub-node parallelization.empty_json_collection:
    type: de1.empty.EmptyPartitionedDataSet
    path: data/02_intermediate/json_collection
    dataset: json.JSONDataSetLazyPartitionedDataSetFor when the data is too big to calculate all at once, and requires at least some clean-up in the process.lazy_json_collection:
    type: de1.lazy.LazyPartitionedDataSet
    path: data/02_intermediate/json_collection
    dataset: json.JSONDataSetPDFDataSetA dataset that usespdfplumberto extract text and tables from pdf files.Data gets returned as aPDFPageobject.invoice_pdf:
    type: de1.pdf.PDFDataSet
    filepath: data/01_raw/invoice.pdfZipFileDataSetA dataset that extracts a single file from a zip file and returns the bytes.
By default will return a byte array, but a dataset can be passed in to change unzip behavior.invoice_pdf:
    type: de1.zip.ZipFileDataSet
    filepath: data/01_raw/invoice.zip
    filename: invoice.pdf
Package: de2en
de2ende2en and gen_cmatInstall itpipinstallde2en# poetry add de2en# pip install git+https://github.com/ffreemt/de2en# poetry add git+https://github.com/ffreemt/de2en# git clone https://github.com/ffreemt/de2en && cd de2enUse itfromde2enimportde2en
Package: de405
This ephemeris has been the basis for the Astronomical Almanac of the
United States Naval Observatory since 2003 and powers the Jet Propulsion
Laboratory’s online HORIZONS service. Coordinates and velocities are
provided for the major planets, the Sun, and Earth’s Moon.Name:DE405 (May 1997)Years:1600 through 2200Planets:YesSun/Moon:YesNutations:YesLibrations:YesReport:Standish (1998) [PDF]Size:54 MBThe accuracy of this ephemeris is good; the position of the Earth or
Mars, for example, should be accurate to within 2 km.  But the more
recentDE421ephemeris provides
greater accuracy, especially with respect to the Moon, and you should
use it instead if you are planning a space mission.  For missions to
Mercury or Venus,DE423will be
an even better choice.SeeDE406if you are interested
in a similar ephemeris for dates far in the past or future, orDE422if you want high accuracy over a
long time period (and have enough disk space).To compute using this ephemeris in Python, see thejplephempackage.
Package: de406
This is the long-period ephemeris which is used by the Jet Propulsion
Laboratory’s online HORIZONS service for dates far in the past or
future.Name:DE406 (May 1997)Years:-3000 through 3000Planets:YesSun/Moon:YesNutations:NoLibrations:NoReport:Standish (1998) [PDF]Size:190 MBThis ephemeris is likeDE405but
covers a longer time period.  To keep its files from being too large,
its size was reduced by permitting up to 1 meter of interpolation error
for the position of the Moon, and 25 meters for all other solar system
bodies.  Since DE405 itself is often only accurate to within a few
kilometers for planetary positions, the difference was not important for
many users.  You can get much higher accuracy from the more recent
long-term ephemerisDE422but at
the cost of three times the RAM and disk space.To compute using this ephemeris in Python, see thejplephempackage.
Package: de421
This is a recent short-period ephemeris published by the Jet Propulsion
Laboratory.  It requires only 27 MB of storage and is specially accurate
with respect to the position of Earth’s Moon.Name:DE421 (February 2008)Years:1900 through 2050Planets:YesSun/Moon:YesNutations:YesLibrations:YesReport:Folkner, Williams, Boggs (2009) [PDF]Size:27 MBThe JPL called this ephemeris is a “significant advance” over
predecessors likeDE405/DE406and cited accuracies that are in
many cases ten times greater, such as giving the position of Venus
within 200m and the positions of Earth and Mars within 300m over the
last decade.  Note that even greater accuracies are achieved, for
Mercury and Venus in particular (but not for the Moon), byDE423which also has the advantage of
covering a 400-year period instead of only 150 years.Greater accuracy can also be expected from the long-term ephemerisDE422since it incorporates more
spacecraft observations than DE421.  It also covers a period of 6000
years, making it useful to astronomy historians.  But as it requires a
half-gigabyte of disk space, some users may prefer DE421.To compute using this ephemeris in Python, see thejplephempackage.
Package: de422
This is the most recent long-period ephemeris published by the Jet
Propulsion Laboratory.  While requiring more than half a gigabyte of
space, it achieves quite high accuracy.Name:DE422 (September 2009)Years:-3000 through 3000Planets:YesSun/Moon:YesNutations:YesLibrations:YesReport:Jones, Fomalont, Dhawan, Romney, Folkner, Lanyi, Border, Jacobson (2010) [PDF]Size:531 MBThis ephemeris incorporates ranging data from recent spacecraft
missions, including the Cassini mission to Saturn, which results in an
accuracy for Saturn of a few milliarcseconds for positions over the past
decade.  If this ephemeris is too large for your application, take a
look atDE421andDE423as alternatives.To compute using this ephemeris in Python, see thejplephempackage.
Package: de423
This is the most recent short-period ephemeris published by the Jet
Propulsion Laboratory.  It achieves especially high accuracy for Mercury
and Venus since it was prepared for the MESSENGER mission, and should be
able to predict the coordinates of those planets within a few
milliarcseconds.Name:DE423 (February 2010)Years:1800 through 2200Planets:YesSun/Moon:YesNutations:YesLibrations:YesReport:Folkner (2010) [PDF]Size:36 MBWhile lunar missions will want to use the slightly older ephemerisDE421because of the extra
accuracy that it provides for the Moon, general purpose users will
probably find this ephemeris more useful, especially as it covers a full
400 years instead of only 150 years.Similar accuracy can be expected from the long-term ephemerisDE422which also covers a period of 6000
years, making it useful to astronomy historians.  But as it requires a
half-gigabyte of disk space, some users may prefer DE423.To compute using this ephemeris in Python, see thejplephempackage.
Package: de9im
As part of my continuing education about the theory and methods underlying
Shapely, GEOS, JTS, and the OGC’s Simple Features specs, I’ve written a small
package of utilities for working with DE-9IM matrices and patterns:http://bitbucket.org/sgillies/de9im/. Shapely provides the standard (these are
probably my favorite OGC standards) predicates as geometry class methods:>>> from shapely.wkt import loads
>>> p = loads('POLYGON ((1.0 0.0, 0.0 -1.0, -1.0 0.0, 0.0 1.0, 1.0 0.0))')
>>> q = loads('POLYGON ((3.0 0.0, 2.0 -1.0, 1.0 0.0, 2.0 1.0, 3.0 0.0))')
>>> p.disjoint(q)
False
>>> p.intersects(q)
True
>>> p.touches(q)
Truebut what if you wanted to test whether the features touched at exactly one
point only? A “side hug”, you might say. Instead of computing the intersection
and checking its geometry type, you can use the de9im package to define a
DE-9IM matrix pattern and test it against the relation matrix for the two
features. The0in the pattern below requires that the intersection of the
boundaries of the features be a 0-dimensional figure. In other words: a point:>>> from de9im import pattern
>>> side_hug = pattern('FF*F0****')
>>> im = p.relate(q)
>>> print im
FF2F01212
>>> side_hug.matches(im)
TrueOne may also use familiarly named patterns:>>> from de9im.patterns import touches
>>> repr(touches)
"DE-9IM or-pattern: 'FT*******||F**T*****||F***T****'"
>>> touches.matches(im)
True
Package: deab
DEAB - Data Exploratory Analysis Block
This package aim at facilitate the step DEA in datascience.
The main features include:
- Plotly Dask app interface for interactive data viz
- Modulable: io, plot, etc… will be separate modules
- Customable: user provided io/plot-function can be executed.
- Dockerized: webapp will be dockerized for the reproducibility and ease of use.
Package: dead
deaddead simple python dead code detectioninstallationpipinstalldeadcliConsult the help for the latest usage:$dead--helpusage: dead [-h] [--files FILES] [--exclude EXCLUDE] [--tests TESTS]optional arguments:-h, --help         show this help message and exit--files FILES      regex for file inclusion, default: ''--exclude EXCLUDE  regex for file exclusion, default '^$'--tests TESTS      regex to mark files as tests, default'(^|/)(tests?|testing)/'run thedeadutility from the root of a git repository.as a pre-commit hookSeepre-commitfor instructionsSample.pre-commit-config.yaml:-repo:https://github.com/asottile/deadrev:v1.5.2hooks:-id:deadhow it worksfind all files in a repository usinggit ls-filesand filtering:only include files matched by the--filesregexexclude files matched by the--excluderegexonly include files identified aspythonbyidentifyclassify test files by the--testsregexast parse each filesearch for definitions and referencesreport things which do not have referencesfalse positivesI wrote this in ~15 minutes on an airplane, it's far from perfect but often
finds things.  Here's a few things it's not good at:functions which implement an interface are often marked as unusedmetaclass magic is often marked as unused (enums, model classes, etc.)suppressingdeadThe# dead: disablecomment will telldeadto ignore
any line which has reportedly dead code.is this project dead?maybe.
Package: deadbeats
DEADBEATSAn easy to use Slack messaging library for research.UsagefromdeadbeatsimportDEADBEATS# set environment variables as below# SLACK_ACCESS_TOKEN=xxxx-xxxxxxxxxxxxx-xxxxxxxxxxxxxx-xxxxxxxxxxxxxxxxxxxxxxxx (Get your own Slack API access token)# SLACK_CHANNEL_ID=deadbeats (set slack channel id whatever you like!)# or you can set configurations manually.DEADBEATS.set_access_token("SLACK_ACCESS_TOKEN")DEADBEATS.set_channel_id("SLACK_CHANNEL_ID")# `DEADBEATS.wrap` sends a message at the beginning and the end of the wrapped function.# `DEADBEATS.wrap` catch every errors and raise it after sending a error message.@DEADBEATS.wrapdefmain():# A simple "heartbeating" message.DEADBEATS.ping()# Start threading!# All subsequent messages will be sent to the thread.DEADBEATS.start_thread()# You can add extra information like below.params={"loss":0.5,"val_loss":1.6,"acc":100.0}DEADBEATS.ping(text="message whatever you like",params=params,additional="info",huga="huga")# If you want to stop threading, you can use this method.# This method reset "thread_ts" of a instance variable, which is a id of thread.DEADBEATS.reset_thread()With PyTorch LightningfromdeadbeatsimportDEADBEATSclassMyModel(pl.LightningModule):...defon_train_start(self):DEADBEATS.start_thread()...defvalidation_epoch_end(self,outputs):avg_loss=torch.stack([x['val_loss']forxinoutputs]).mean()DEADBEATS.ping(val_loss=avg_loss,current_epoch=self.current_epoch)return{'val_loss':avg_loss}...# custom training function@DEADBEATS.wrapdeffit(self,trainer):trainer.fit(self)messages like belowThis library is named after the wonderful work ofMori Calliope,DEAD BEATS, and inspired byhugginface/knockknock.