Saravanan666/pypi-org huggingface dataset:
Package: distkv
Welcome toDistKV!DistKV is a master-less distributed key-value storage system. It
circumvents the CAP theorem by assuming that keys are usually only changed
by one node. It is resistant to partitioning and intended to be always-on;
it might delay – but will not lose – updates even in a partitioned network.DistKV comes with several batteries included:Basic user management, pattern-based ACLsStrong typing, code- and/orJSON Schema-basedData manglingBackground code executionSeamless recovery even if only one master is runninga MQTT 3.1 back-end that stores persistent data in DistKV,
based on hbmqttAPIDistKV offers an efficient msgpack-based interface to access data and to
change internal settings. Most configuration is stored inside DistKV
itself.Stored data arenotforced to be strings or binary sequences, but can
be anything thatMsgPacksupports. Keys to storage are multi-level and
support string, integer/float, and list keys.Non-FeaturesDistKV does not support data partitioning. Every node stores the whole
data set and can instantly deliver mostly-uptodate data.DistKV does not have a disk-based storage backend; periodic snapshots and
event logs can be used to quickly restore a system, if necessary.StatusDistKV is mostly stable. There are a lot of corner cases that don’t
have tests yetTODOs:
* some services (esp. command line tools and runners) are under-tested
* there’s no good API for errorsChangelog0.41: the message monitor can do multiple subpaths and only reports initial-load-complete once0.40: use asyncscope for running subsystems in a reasonable way0.35: allow forgetting nodes (if they have no data attached)0.30: major API refactoring: paths are now separate objectsTODOupdate the whole ecosystem to anyio 2.0 (asyncclick asyncscope …)clean up some of the more egregious command line mistakescreate a page for showcase-ing subprojects (distinv knx owfs akumuli …)improve Home Assistant integration
Package: distkv-hass
DistHASS is a link between Home Assistant and DistKV.It willset up standard config for Home Assistant in DistKVhave command-line support to register (or not) devicesPrinciple of OperationHome Assistant talks via MQTT. DistKV has a quite versatile MQTT adapter.
Thus we can store Home Assistant’s entitiy configuration, plus their state,
plus the commands Home Assisant issues to get things to change their state,
in DistKV.MQTT can only transmit binary data. DistHASS thus creates a few codecs that
support binary data (translating them to on/off), integers/floats, and JSON.Thus, bottom to top:Mosquitto on port 51883basic installation, does not need persistence or retained messagesSerf would work, but it imposesadditional delaysDistKV, using Mosquitto as a backbonea special user with conversion rules for JSON etc.DistMQTTretained messages are stored in DistKVtransparent channels to forward MQTT messages unmodified, if requiredanything else is broadcast as a DistKV messageuses port 1883Home Assistant, or anything else MQTTish for that mattermust use MQTT 3.11If you have devices that only can use MQTT 3.1, you can teach them to talk
directly to Mosquitto, via a transparent range.SetupRundistkv client hass init-iAdd a DistKV user for Home Assistant and set itsconvparameter tohassco:distkv client auth user param NAME conv hasscoStart DistMQTT:distkv:
    server:
        host: '127.0.0.1'
        port: 27586
        auth: "password name=NAME password=PASSWORD"
    topic: [mqtt, msg]
    transparent:
    - [home,ass,event]
    - [home,ass,state]
    retain: [home, ass]
listeners:
    default:
        max-connections: 500
        type: tcp
    local-tcp:
        bind: 127.0.0.1:1883
    remote-tcp:
        bind: 10.107.3.18:1883
timeout-disconnect-delay: 2
plugins: ['auth_anonymous']
auth:
    allow-anonymous: trueIf you have devices that cannot talk via DistMQTT, modify thetransparentlist to include the MQTT prefixes you currently
use.Modify Home Assistant’s MQTT integration to do autodiscovery.
If you’ve set it up via the GUI, the file you need is.storage/core.config_entries. Find the entry with"domain": "mqtt"and modify itsdataentry to read:"data": {
    "broker": "127.0.0.1",
    "discovery": true,
    "discovery_prefix": "home/ass/dyn",
    "port": 1883,
},Restart Home Assistant.Rundistkv client hass add light foo bar.A new light should show up in the Home Assistant GUI.You can try to turn it on, but it will go off by itself a second or two
later because there’s no device yet.Rundistkv client hass state light foo bar True.This command changes the state manually.In Home Assistant, the light turns on.Adding an actual device that monitors the light’s DistKV command entry / its
[Dist]MQTT command topic, actually affects the hardware, and changes the
state, is your job.
Package: distkv-inv
DistInv is a link between DistKV and your network infrastructure.DistInv knows about hosts, ports, VLANs, (sub)networks, cables.It storeswhich VLAN a network is inwhich network a host belongs towhich port is plugged into whatwhich groups a host belongs towhere a host is locatedThis allows you torecord which systems are where (surprisingly difficult …)auto-generate (part of) your Ansible inventorydisplay which VLANs need to be configured on a switch portgenerate a network link path from A to Bauto-update DNS entries when hosts are added, moved, or modified
Package: distkv-knx
DistKNX is a link between KNX buses and DistKV.It willquery and monitor inputs as specifiedwrite values that it reads from them to some DistKV entrymonitor a DistKV entry and write any updates to KNXwork with DistKV’s runner system, either centrally or distributedWarningCurrently this will only work with MoaT’s fork of XKNX because we’re using
AnyIO and some other improvements.
Package: distl
simplified and condensed distributionspip install distlIMPORTANT:distlis currently still under development, is not yet well-tested, and is subject to significant API changes.  Please keep posted until an official release is ready.Read thelatest documentation on readthedocsorbrowse the current documentation.distlprovides a python object-interface on top of several distribution (random variable) functions inscipy.statsand allows for:serialization of distributions (so they can be saved to disk or pickled and sent to processors within MPI)support for units and wrappingconversion between different types of distributionsmath between distributions, handling covariances from multivariate distributions wherever possibleplotting convenience functionsGetting StartedDependenciesdistlrequires the following dependencies:python 2.7+ or 3.6+scipy 1.0+numpy 1.10+and the following optional dependencies:matplotlib 2.2+ (required for plotting distributions)corner(required for plotting multivariate distributions and distribution collections)astropy 1.0+ (required for units support)dill (required for saving/loading Function distributions)You can see theTravis testing matrixfor
details on what exact versions have been tested and ensured to work.  If you run
into any issues with dependencies, pleasesubmit an issue.InstallationTo install the latest release via pip:pipinstalldistlTo install from source locally for a single user:pythonsetup.pybuild
pythonsetup.pyinstall--userOr to install globally:pythonsetup.pybuild
sudopythonsetup.pyinstallImportNow from within python we can import thedistlpackage:importdistland then create, sample from, and plot our first distribution:g=distl.gaussian(10,1)print(g.sample())print(g.sample(10))g.plot(show=True)Documentation and API DocsRead thelatest documentation on readthedocsorbrowse the current documentation.ContributorsKyle ConroyContributions are welcome!  Feel free to file an issue or fork and create a pull-request.
Package: distlib
What is it?Distlib is a library which implements low-level functions that relate to
packaging and distribution of Python software. It is intended to be used as the
basis for third-party packaging tools. The documentation is available athttps://distlib.readthedocs.io/Main featuresDistlib currently offers the following features:The packagedistlib.database, which implements a database of installed
distributions, as defined byPEP 376, and distribution dependency graph
logic. Support is also provided for non-installed distributions (i.e.
distributions registered with metadata on an index like PyPI), including
the ability to scan for dependencies and building dependency graphs.The packagedistlib.index, which implements an interface to perform
operations on an index, such as registering a project, uploading a
distribution or uploading documentation. Support is included for verifying
SSL connections (with domain matching) and signing/verifying packages using
GnuPG.The packagedistlib.metadata, which implements distribution metadata as
defined byPEP 643,PEP 566,PEP 345,PEP 314andPEP 241.The packagedistlib.markers, which implements environment markers as
defined byPEP 508.The packagedistlib.manifest, which implements lists of files used
in packaging source distributions.The packagedistlib.locators, which allows finding distributions, whether
on PyPI (XML-RPC or via the “simple” interface), local directories or some
other source.The packagedistlib.resources, which allows access to data files stored
in Python packages, both in the file system and in .zip files.The packagedistlib.scripts, which allows installing of scripts with
adjustment of shebang lines and support for native Windows executable
launchers.The packagedistlib.version, which implements version specifiers as
defined byPEP 440, but also support for working with “legacy” versions and
semantic versions.The packagedistlib.wheel, which provides support for building and
installing from the Wheel format for binary distributions (seePEP 427).The packagedistlib.util, which contains miscellaneous functions and
classes which are useful in packaging, but which do not fit neatly into
one of the other packages indistlib.* The package implements enhanced
globbing functionality such as the ability to use**in patterns to
specify recursing into subdirectories.Python version and platform compatibilityDistlib is intended to be used on and is tested on Python versions 2.7 and 3.6 or later,
pypy-2.7 and pypy3 on Linux, Windows, and macOS.Project statusThe project has reached a mature status in its development: there is a comprehensive
test suite and it has been exercised on Windows, Ubuntu and macOS. The project is used
by well-known projects such aspipandcaniusepython3.This project was migrated from Mercurial to Git and from BitBucket to GitHub, and
although all information of importance has been retained across the migration, some
commit references in issues and issue comments may have become invalid.Code of ConductEveryone interacting in the distlib project’s codebases, issue trackers, chat
rooms, and mailing lists is expected to follow thePyPA Code of Conduct.
Package: distlock
DistLockDistributed locks with Redis and PythonWhy DistLock ?This library implements the DistLock algorithm introduced by@antirez, Due to the origin fork fromredlockhas been inactive since 2015, Then this repo was born with additional implementations:Bug fixPython3 syntax improvementsNew tweak for adaptationYet another ...There are already a few redis based lock implementations in the Python world, e.g.retools,redis-lock.However, these libraries can only work withsingle-masterredis server. When the Redis master goes down, your application has to face a single point of failure. We can't rely on the master-slave replication, because Redis replication is asynchronous.This is an obvious race condition with the master-slave replication model :Client A acquires the lock into the master.The master crashes before the write to the key is transmitted to the slave.The slave gets promoted to master.Client B acquires the lock to the same resource A already holds a lock for. SAFETY VIOLATION!A quick introduction to the DistLock algorithmTo resolve this problem, the Distlock algorithm assume we haveNRedis masters. These nodes are totally independent (no replications). In order to acquire the lock, the client will try to acquire the lock in all the N instances sequentially. If and only if the client was able to acquire the lock in the majority ((N+1)/2)of the instances, the lock is considered to be acquired.The detailed description of the DistLock algorithm can be found in the Redis documentation:Distributed locks with Redis.APIsThedistlock.DistLockclass shares a similar API with thethreading.Lockclass in the  Python Standard Library.Basic UsagefromdistlockimportDistLock# By default, if no redis connection details are# provided, DistLock uses redis://127.0.0.1:6379/0lock=DistLock("distributed_lock")lock.acquire()do_something()lock.release()With Statement / Context ManagerAs withthreading.Lock,distlock.DistLockobjects are context managers thus support theWith Statement. This way is more pythonic and recommended.fromdistlockimportDistLockwithDistLock("distributed_lock"):do_something()Specify multiple Redis nodesfromdistlockimportDistLockwithDistLock("distributed_lock",connection_details=[{'host':'xxx.xxx.xxx.xxx','port':6379,'db':0},{'host':'xxx.xxx.xxx.xxx','port':6379,'db':0},{'host':'xxx.xxx.xxx.xxx','port':6379,'db':0},{'host':'xxx.xxx.xxx.xxx','port':6379,'db':0},]):do_something()Theconnection_detailsparameter expects a list of keyword arguments for initializing Redis clients.
Other acceptable Redis client arguments  can be found on theredis-py doc.Reuse Redis clients with the DistLockFactoryUsually the connection details of the Redis nodes are fixed.DistLockFactorycan help reuse them, create multiple DistLocks but only initialize the clients once.fromdistlockimportDistLockFactoryfactory=DistLockFactory(connection_details=[{'host':'xxx.xxx.xxx.xxx'},{'host':'xxx.xxx.xxx.xxx'},{'host':'xxx.xxx.xxx.xxx'},{'host':'xxx.xxx.xxx.xxx'},])withfactory.create_lock("distributed_lock"):do_something()withfactory.create_lock("another_lock"):do_something()
Package: distlock-client
Distribute lock system’s client.Installpip install distlock-clientUsageUse Lock apis.from distlock_client import Lock
from distlock_client import get_app_unique_name
from .services import do_something

def view01(request):
    lockName = "view01_lock"
    appid = get_app_unique_name()
    server = "http://127.0.0.1/"
    lock = Lock(appid, server)

    locked = lock.safe_acquire(lockName, 60)
    if locked:
        try:
            do_something()
        finally:
            lock.safe_release(lockName)Use with statement.from distlock_client import Lock
from distlock_client import distlock
from distlock_client import get_app_unique_name
from .services import do_something

def view01(request):
    lockName = "view01_lock"
    appid = get_app_unique_name()
    server = "http://127.0.0.1/"
    lock = Lock(appid, server)
    with distlock(lock, lockName) as locked:
        if locked:
            do_something()MiscAvailable servers.django-distlock-server [python, django, database backend]django-distlock-server-redis [python, django, redis backend]
Package: distlre
# DistLRE (dist-el-ar-ee)A lightweight Python package to distribute commands on remote hosts via SSH and to execute them locally in parallel.Supports running a local commands:```pythonfrom distlre.distlre import DistLRE, Task, RemoteHostexecutor = DistLRE(local_threads=1)task = Task(command='ls ~', meta='META', time_limit=10, memory_limit=10)future = executor.submit(task)executor.execute_tasks()executor.wait()print(future.result().output)```Or runs command on a remote host:```pythonimport getpasspassword = getpass.getpass("Password to connect to [localhost]:")executor = DistLRE(remote_hosts=[RemoteHost('localhost', port=31415, password=password)])task = Task(command='ls ~', meta='META', time_limit=10, memory_limit=10)other_task = Task(command='cd ~', meta='META', time_limit=10, memory_limit=10)future = executor.submit(task)other_future = executor.submit(other_task)executor.execute_tasks()executor.wait()print(future.result().output)```# Install`pip3 install DistLRE`
Package: dist_matrix
Distance MatricesFast Numba-enabled CPU and GPU computations of 1D Earth Mover's (scipy.stats.wasserstein_distance) and Euclidean distances.10000 x 10000weighted Wasserstein distance matrix in ~15 s on an NVIDIA GeForce RTX 2060 GPU. GPU tends to be ~2x faster than parallelized CPU on Intel(R) Core(TM) i7-10750H CPU @ 2.60GHz (6 physical cores).Installationcondapipconda install -c sgbaird dist_matrixpip install dist_matrixTo best reflect the development workflow, you can clone the repository and install viaflit:gitclonehttps://github.com/sparks-baird/dist-matrix.gitcddist-matrixcondainstallflit# alternatively, `pip install -e .`flitinstall--pth-file# --pth-file flag is a Windows-compatible local installation; you can edit the source without reinstallingUsageYou can compute distance matrices (more efficient per distance calculation) or access the lower-level single distance calculation.Distance MatricesThere is a GPU version (dist_matrix.cuda_dist_matrix_full) as well as a CPU version (dist_matrix.njit_dist_matrix_full).importnumpyasnpfromdist_matrix.cuda_dist_matrix_fullimportdist_matrixasgpu_dist_matrix# from dist_matrix.njit_dist_matrix_full import dist_matrix as cpu_dist_matrixn_features=10n_rows=100U,V,U_weights,V_weights=np.random.rand(4,n_rows,n_features)distances=gpu_dist_matrix(U,V=V,U_weights=U_weights,# optionalV_weights=V_weights,# optionalmetric="wasserstein",# "euclidean")Single Distance CalculationsSeemetrics.py. Note that these lower-level functions are not GPU-accelerated.importnumpyasnpfromdist_matrix.utils.metricsimportwasserstein_distance,euclidean_distancen_features=10u,v,u_weights,v_weights=np.random.rand(4,n_features)presorted,cumweighted,prepended=[False,False,False]em_dist=wasserstein_distance(u,v,u_weights,v_weights,presorted,cumweighted,prepended)eucl_dist=euclidean_distance(u,v)See AlsoElement Mover's Distances viachem_wasserstein(based onElM2D)
Package: dist-meta
Parse and create Python distribution metadata.DocsTestsPyPIAnacondaActivityQAOtherInstallationdist-metacan be installed from PyPI or Anaconda.To install withpip:$python-mpipinstalldist-metaTo install withconda:First add the required channels$condaconfig--addchannelshttps://conda.anaconda.org/conda-forge$condaconfig--addchannelshttps://conda.anaconda.org/domdfcodingThen install$condainstalldist-meta
Package: distnav
IntroductionThis repository contains related code for the RSS 2021paper"Move Beyond Trajectories: Distribution Space Coupling for Crowd Navigation" by Muchen Sun, Francesca Baldini, Pete Trautman and Todd Murphey.TutorialsWe provide a Jupyter notebook tutorials for our algorithm. You can find them under the "notebooks" directory.Tutorial 1: distribution space coupling in one-dimensional space: In the first tutorial, we will build from scratch on a one-dimensional two-agents toy example to show how DistNav optimization works. We will show both the analytical solution with numerical integration and approximated solution with sampling and Monte-Carlo integration, and how they can match with
each other. You can find a copy of the notebook in Google Colaboratoryhere.
Package: distnet
distnetVisualizer for tcp connections. Ports 53 and 43 should accept connections for full functionality.Requirements:Package was built for python3.4 or above. Please usepython3Installation:pip3 install distnetpip3 install git+https://github.com/p0licat/distnetUsage:python3 -m distnet -hpython3 -m distnet --visual -c
Package: distobjects
py-distobjectspy-distobjects(Python Distributed Objects) is a Python library to easily map Objects to Caching Systems like Redis.UsageSimply define classes with minimum schema and connect to a backendCreate objects from those classes and use them as if they are regualer objectsEven Concurrent Access from multiple hosts will work as long as conntected to same backendYou can update the object from any Host/Process, you can access the updated values instently from other Hosts/Processes by just reading attribute without explicit refresh/readingWhy? when i can use redis directlySame reson as why we use ORMs for Database access when we can use SQL directlyNo need to worry about keys, values, Serialization, and Deserialization etc. all over the code.It just makes it easily organize/maintain the code by abstracting some functionsCreating object in host1>>>fromdistobjectsimportRedisBackend,DObject>>>fromdistobjects.fieldsimportTextField>>>importredis>>>>>>r=redis.Redis(host='redis-server1',port=6379,db=5)>>>redis_backend=RedisBackend(client=r)>>>classMyStudent(DObject):...classMeta:...backend=redis_backend...first_name=TextField()...last_name=TextField()...>>>student1=MyStudent("1")>>>student1.first_name="Harry">>>student1.last_name="James"Reading object in host2>>>fromdistobjectsimportRedisBackend,DObject>>>fromdistobjects.fieldsimportTextField>>>importredis>>>>>>r=redis.Redis(host='redis-server1',port=6379,db=5)>>>redis_backend=RedisBackend(client=r)>>>classMyStudent(DObject):...classMeta:...backend=redis_backend...first_name=TextField()...last_name=TextField()...>>>student=MyStudent("1")>>>student<__main__.MyStudentobjectat0x105bc29d0>>>>student.first_name'Harry'>>>student.last_name'James'
Package: distogram
DistoGram is a library that allows to compute histogram on streaming data, in
distributed environments. The implementation follows the algorithms described in
Ben-Haim’sStreaming Parallel Decision TreesGet StartedFirst create a compressed representation of a distribution:importnumpyasnpimportdistogramdistribution=np.random.normal(size=10000)# Create and feed distogram from distribution# on a real usage, data comes from an event streamh=distogram.Distogram()foriindistribution:h=distogram.update(h,i)Compute statistics on the distribution:nmin,nmax=distogram.bounds(h)print("count:{}".format(distogram.count(h)))print("mean:{}".format(distogram.mean(h)))print("stddev:{}".format(distogram.stddev(h)))print("min:{}".format(nmin))print("5%:{}".format(distogram.quantile(h,0.05)))print("25%:{}".format(distogram.quantile(h,0.25)))print("50%:{}".format(distogram.quantile(h,0.50)))print("75%:{}".format(distogram.quantile(h,0.75)))print("95%:{}".format(distogram.quantile(h,0.95)))print("max:{}".format(nmax))count: 10000
mean: -0.005082954640481095
stddev: 1.0028524290149186
min: -3.5691130319855047
5%: -1.6597242392338374
25%: -0.6785107421744653
50%: -0.008672960012168916
75%: 0.6720718926935414
95%: 1.6476822301131866
max: 3.8800560034877427Compute and display the histogram of the distribution:hist=distogram.histogram(h)df_hist=pd.DataFrame(np.array(hist),columns=["bin","count"])fig=px.bar(df_hist,x="bin",y="count",title="distogram")fig.update_layout(height=300)fig.show()InstallDistoGram is available on PyPi and can be installed with pip:pip install distogramPlay With MeYou can test this library directly on thislive notebook.PerformancesDistogram is design for fast updates when using python types. The following
numbers show the results of the benchmark program located in the examples.On a i7-9800X Intel CPU, performances are:InterpreterOperationNumpyReq/spypy 7.3updateno6563311pypy 7.3updateyes111318CPython 3.7updateno436709CPython 3.7updateyes251603On a modest 2014 13” macbook pro, performances are:InterpreterOperationNumpyReq/spypy 7.3updateno3572436pypy 7.3updateyes37630CPython 3.7updateno112749CPython 3.7updateyes81005As you can see, your are encouraged to use pypy with python native types. Pypy’s
jit is penalised by numpy native types, causing a huge performance hit. Moreover
the streaming phylosophy of Distogram is more adapted to python native types
while numpy is optimized for batch computations, even with CPython.CreditsAlthough this code has been written by following the aforementioned research
paper, some parts are also inspired by the implementation fromCarson Farmer.Thanks toJohn Belmontefor his help on
performances and accuracy improvements.