Saravanan666/pypi-org huggingface dataset:
Package: dlg-nifty-components
dlg-nifty-componentsdlg-nifty-components contains a collection of cpu and gpu nifty gridding/degridding implementations for radio interferometry datasets.InstallationThere are multiple options for the installation, depending on how you are intending to run the DALiuGE engine, directly in a virtual environment (host) or inside a docker container. You can also install it either from PyPI (latest released version).Install it from PyPIPrerequisitesThe following packages are required before installation such that dlg-nifty-components can compile it's dependencies:g++python3-devnumpycuda-libraries-devDALiuGE Engine Python App in Docker ContainerFor development purposes it is preferable to install and run dlg-nifty-components as a python app for fast recompilation of dynamically linked binaries. To do this you must first install a cuda version in daliuge-common or daliuge-engine followed by python-dev, numpy and g++. daliuge-nifty-components will then compile and install wagg for cuda acceleration.daliuge-common/Dockerfile.devRUNaptinstall-ywgetgnupg2software-properties-common
RUNmkdir-p/code&&cd/code&&\wgethttps://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/cuda-ubuntu2004.pin&&\mvcuda-ubuntu2004.pin/etc/apt/preferences.d/cuda-repository-pin-600&&\apt-keyadv--fetch-keyshttps://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/7fa2af80.pub&&\add-apt-repository"deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2004/x86_64/ /"&&\aptupdate

RUNDEBIAN_FRONTEND=noninteractiveapt-y--no-install-recommendsinstall\cuda-minimal-build-11-2cuda-libraries-11-2cuda-libraries-dev-11-2&&\ln-s/usr/local/cuda-11.2/usr/local/cuda&&\ln-s/usr/local/cuda/targets/x86_64-linux/lib/usr/local/cuda/lib&&\ln-s/usr/local/cuda/targets/x86_64-linux/include/usr/local/cuda/includedaliuge-engine/Dockerfile.devRUNaptinstall-ypython3-devg++run_engine.sh# append configured nvidia-docker arguments here, e.g.DOCKER_OPTS=$DOCKER_OPTS--gpus=all--privilegeddlg-nifty-components may be installed before or after DALiuGE engine is running:dockerexec-tdaliuge-enginebash-c'pip install --prefix=$DLG_ROOT/code dlg_nifty_components'EAGLE PaletteAn EAGLE .palette file can be conveniently generated locally using command:bash./build_palatte.shUsagePythonFor example the MS2DirtyApp component will be available to the engine when you specifyfromdaliuge_component_niftyimportMS2DirtyAppMS2DirtyApp('a','a')in the AppClass field of a Python Branch component. The EAGLE palette associated with these components are also generated and can be loaded directly into EAGLE. In that case all the fields are correctly populated for the respective components.DALiuGE Docker AppOptionallyb uild the container image for use as a daliuge docker app:dockerbuild-tdlg-nifty-components-f./Containerfile.
Package: dlg-paletteGen
DALiuGE palette generator toolThis is the palette generator of theDALiuGEsystem.It processes a file or a directory of source files or an installed python module and produces a DALiuGE compatible palette file containing the information required to use the identified functions, classes and methods to construct logical graphs inEAGLE.
For more information please refer to thedocumentation.Install it from PyPIpipinstalldlg_paletteGenUsage$python-mdlg_paletteGen

or

$dlg_paletteGen-h

or

$dlg-paletteGen-h

usage:dlg_paletteGen[-h][-mMODULE][-tTAG][-c][-r][-s][-v]idirofile

ThisisthepalettegeneratoroftheDALiuGEsystem.

Itprocessesafileoradirectoryofsourcefilesand
producesaDALiuGEcompatiblepalettefilecontainingthe
informationrequiredtousefunctionsandcomponentsingraphs.
Formoreinformationpleaserefertothedocumentation
https://daliuge.readthedocs.io/en/latest/development/app_development/eagle_app_integration.html#automatic-eagle-palette-generation

positionalarguments:idirinputdirectorypathorfilenameofileoutputfilename

optionalarguments:-h,--helpshowthishelpmessageandexit-mMODULE,--moduleMODULEModuleloadpathname-tTAG,--tagTAGfiltercomponentswithmatchingtag-cCmode,ifnotsetPythonwillbeused-r,--recursiveTraversesub-directories-s,--parse_allTrytoparsenonDAliuGEcompliantfunctionsandmethods-v,--verboseincreaseoutputverbosity
Package: dlgpu-assistant
Deep Learning AssistantThis library offers some tools to make the process of creating and training an ANN eassier and faster.For projects and libraries like this visit myGitHub ProfileInstallingpip install dlgpu_assistantimport dlgpu_assistant
from dlgpu_assistant.dl_tools import DlAssistant, DlTools, DlLoggerInstall Requirestensorflow-gpu==2.9.0
pandas==1.4.3
numpy==1.23.1
scikit-learn==1.1.2
Package: dl-hf-model
dl-hf-modelDownload and cache a huggingface model given a url, save to models dirInstall itpipinstalldl-hf-model# pip install git+https://github.com/ffreemt/dl-hf-model# poetry add git+https://github.com/ffreemt/dl-hf-model# git clone https://github.com/ffreemt/dl-hf-model && cd dl-hf-modelUse itfromdl_hf_modelimportdl_hf_model
Package: dlhn
dlhndlhn is a Python CLI script to download my comments and submissions
from the Hacker News API
and generate a static HTML archive with a Jinja2 templateFeaturesDownload comments and submissions from the Hacker News APIArchive comment trees and submissions as JSONArchive comment trees and submissions as static HTML
with a Jinja2 HTML templateAggressively cache entries that couldn’t have changed
with a two-layer caching system that includes requests_cache
and a sqlite databaseInstallationInstall dlhn with pip:pipinstalldlhn# pip install -e git+https://github.com/westurner/dlhn#egg=dlhnUsageCall dlhn:dlhn-udlhntestuser-oindex.html--expire-newerthan14dOptionally, create a repo for e.g. GitHub Pages and add aMakefile:# hnlog MakefileUSERNAME:=dlhntestuserdefault:backupinstall:pipinstall-egit+https://github.com/westurner/dlhn#egg=dlhnbackup:@# items with a cachetime newer than 14d ago may need to be pulled again
    @# because they may not be locked yet (cachetime != item_time)dlhn-u'$(USERNAME)'-oindex.html--expire-newerthan14dbackup-nocache:dlhn-u'$(USERNAME)'-oindex.htmlcommit:gitadd./index.html./index.html.json./dlhn.sqlite&&\gitcommit-m":books: Updated index.html, index.html.json, and dlhn.sqlite"push:gitpushall:backupcommitpushAnd pass USERNAME as an arg when callingmake:makeallUSERNAME=dlhntestuserReferencesHacker News Guidelines:https://news.ycombinator.com/newsguidelines.htmlHacker News API docs:https://github.com/HackerNews/APIdlhntestuserSubmissions:https://news.ycombinator.com/submitted?id=dlhntestuserComments:https://news.ycombinator.com/threads?id=dlhntestuserLicenseBSD Licensedlhn archives pubicly available comments and submissions
from the Hacker News API
for noncommercial use to make searching one-page with ‘Ctrl-F’ easy.Credits@westurnerHistory0.2.5 (2019-04-17)Update README.rst0.2.4 (2019-04-17)Update docs with pip package nameUpdate HISTORY.rst0.2.3 (2019-04-17)Add a setup.pyAdd a few testsFirst release on PyPI.0.1.0 (2017-11-28)Simple python script
Package: dlhub-cli
[![Build Status](https://travis-ci.org/DLHub-Argonne/dlhub_cli.svg?branch=master)](https://travis-ci.org/DLHub-Argonne/dlhub_cli)[![CoverageStatus](https://coveralls.io/repos/github/DLHub-Argonne/dlhub_cli/badge.svg?branch=master)](https://coveralls.io/github/DLHub-Argonne/dlhub_cli?branch=master)[![PyPI version](https://badge.fury.io/py/dlhub-cli.svg)](https://badge.fury.io/py/dlhub-cli)[![DocumentationStatus](https://readthedocs.org/projects/dlhub-cli/badge/?version=latest)](https://dlhub-cli.readthedocs.io/en/latest/?badge=latest)A Command Line Interface for [DLHub](https://dlhub.org)Source Code:https://github.com/DLHub-Argonne/dlhub_cli## Installation` pip install dlhub_cli `## Development
After cloning this repository, run` pip install-e. `## Documentation
[Read the Docs](https://dlhub-cli.readthedocs.io/en/latest/)## Project Support
This material is based upon work supported by Laboratory Directed Research and Development (LDRD) funding from Argonne National Laboratory, provided by the Director, Office of Science, of the U.S. Department of Energy under Contract No. DE-AC02-06CH11357.
Package: dlhub-kernel-gateway
Jupyter Kernel Gateway is a web server that supports different mechanisms for
spawning and communicating with Jupyter kernels, such as:A Jupyter Notebook server-compatible HTTP API used for requesting kernels
and talking theJupyter kernel protocolwith the kernels over WebsocketsA HTTP API defined by annotated notebook cells that maps HTTP verbs and
resources to code to execute on a kernelThe server launches kernels in its local process/filesystem space. It can be
containerized and scaled out using common technologies liketmpnb,Cloud Foundry, andKubernetes.
Package: dlh-utils
DLH_utilsA Python package produced by the Linkage Development team from the Data Linkage Hub at Office for National Statistics (ONS) containing a set of functions used to expedite and streamline the data linkage process.It's key features include:it's scalability to large datasets, usingsparkas a big-data backendprofiling and flagging functions used to describe and highlight issues in datastandardisation and cleaning functions to make data comparable ahead of linkagelinkage functions to derive linkage variables and join data together efficientlyPlease log an issue on the issue board or contact any of the active contributors with any issues or suggestions for improvements you have.Installation stepsDLH_utils supports Python 3.6+. To install the latest version, simply run:pipinstalldlh_utilsOr, if using CDSW, in a terminal session run:pip3installdlh_utilsThe -U argument can be used to upgrade the package to its newest version:pip3install-Udlh_utilsDemoFor a worked demonstration notebook of these functions being applied within a data linkage context, head over to ourseparate demo repositoryCommon issuesWhen using the jaro/jaro_winkler functions the error "no module called Jellyfish found" is thrownThese functions are dependent on the Jellyfish package and this may not be installed on the executors used in your spark session.
Try submitting Jellyfish to your sparkcontext via addPyFile() or by setting the following environmental variables in your CDSW engine settings (ONS only):PYSPARK_DRIVER_PYTHON = /usr/local/bin/python3.6PYSPARK_PYTHON = /opt/ons/virtualenv/miscMods_v4.04/bin/python3.6Using the cluster functionThe cluster function uses Graphframes, which requires an extra JAR file dependency to be submitted to your spark context in order for it to run.We have published a graphframes-wrapper package on Pypi that contains this JAR file. This is included in the package requirements
as a dependency.If outside of ONS and this dependency doesn't work, you will need to submit graphframes' JAR file dependency to your spark context. This can be found here:https://repos.spark-packages.org/graphframes/graphframes/0.6.0-spark2.3-s_2.11/graphframes-0.6.0-spark2.3-s_2.11.jarOnce downloaded, this can be submitted to your spark context by adding this parameter to your SparkSession config:spark.conf.set('spark.jars',path_to_jar_file)ThanksThanks to all those in the Data Linkage Hub, Data Engineering and Methodology at ONS that have contributed towards this repository.Any questions?If you need any additional help, or have any feedback on the package, please contact Jenna Hart (Jenna.Hart@ons.gov.uk) or the Data Linkage Hub atLinkage.Hub@ons.gov.uk.
Package: dlib2mediapipe
Package to convert Dlib to MediapipeFunctionality of the ConverterConverts given input to 468 3-D facial landmark pointsConverts 468 3-D facial landmark points to 2-D facial landmark pointsConverts the facial landmark points to dlib's formatUsageMake sure you have Python installed in your system.Run Following command in the CMD.pip install dlib2mediapipeRun the following Script.python test.pyNoteWe have tried to implement all the functionality, it might have some bugs also so please do ignore it or please try to solve that bug.
Package: dlin
dlin-tracerDescribe piece cycles on a Rubik's Cube.Usage: As a Python PackageIf you would like to use the tracer as a python package, please follow the
following instructions.InstallingTo install, runpip install dlinUsingThedlin.tracerfunction converts a scramble
(set of moves defined byWCA regulation 12aseparated by spaces)
into a python dictionary describing cycles on the cube.To use, runfromdlinimporttracerThe following is example code to get the tracing for a scramble.described_cycles=tracer("F' L2 B2 F2 D' F2 D F2 U R2 U' L' B L' R' F2 R2 B2 U2 Fw'")Here is an example of what could be contained indescribed_cycles{"edge":[{"type":"cycle","buffer":"UF","targets":["LD","LF","RU","LU"],"orientation":0,"parity":0,},{"type":"cycle","buffer":"UB","targets":["RB","DR"],"orientation":1,"parity":0,},{"type":"cycle","buffer":"FR","targets":["DB","DF"],"orientation":1,"parity":0,},],"corner":[{"type":"cycle","buffer":"UFR","targets":["UBL","RUB","RDB","FDR","UFL","FDL"],"orientation":1,"parity":0,},{"type":"misoriented","location":"DBL","targets":[],"orientation":-1,"parity":0,},],"scramble":"F' L2 B2 F2 D' F2 D F2 U R2 U' L' B L' R' F2 R2 B2 U2 Fw'","rotation":["z"],}Usage: Command Line ScriptRundlin-trace [scramble]Make sure to surround the scramble in quotation marks. This will output a json,
with similar structure to the dictionary shown above.
Package: dlinfo
Python wrapper for libc’s dlinfoInstallpipinstalldlinfo# orpipenvinstalldlinfoUsage>>>fromdlinfoimportDLInfo>>>lib=ctypes.cdll.LoadLibrary(ctypes.util.find_library('c'))>>>dlinfo=DLInfo(lib)>>>dlinfo.path'/lib/x86_64-linux-gnu/libc.so.6'
Package: dlink-scraper
D-Link ScraperThis is a library that gets information from your D-Link DWR-921 (and possible others) LTE router admin page using web scrapingThis package containslibrarydlinkscraper- which you can use to scrape data from D-Link DWR-921 LTE router
(I don't know if it will work on others, I have only this one :P)CLI tooldlinkscraperthat you can use to get info from consoledlinkscraper-duckdns-updatescript, which updates your DuckDNS IP
(and it's also the reason why I created this package :D)Library usage examplefromdlinkscraperimportDLinkdl=DLink('http://192.168.1.1')# Change this if yours has different IPdl.login('admin',<PASSWORD>)dl.get_main_site()print(dl.isp_name)# You should always log out if you don't want to get "admin is already logged in" laterdl.logout()CLI usageJust rundlinkscraperfrom terminal and follow the instructionsDuckDNS script usageProvide all required variables listed in--help. Example:dlinkscraper-duckdns-update--token<YOUR_DUCKDNS_TOKEN>--domain<YOUR_DOMAIN>-p<ROUTER_PASSWORD>
Package: dlio-profiler-py
DLIO Profiler v0.0.4A multi-level profiler for capturing application functions and low-level system I/O calls from deep learning workloads.Requirements for profilerPython > 3.7pybind11Requirements for analyzerbokeh>=2.4.2pybind11zindex_pypandas>=2.0.3dask>=2023.5.0distributednumpy>=1.24.3pyarrow>=12.0.1rich>=13.6.0python-intervals>=1.10.0.post1matplotlib>=3.7.3Build DLIO Profiler with pipUsers can easily install DLIO profiler using pip. This is the way most python packages are installed.
This method would work for both native python environments and conda environments.From sourcegitclonegit@github.com:hariharan-devarajan/dlio-profiler.gitcddlio-profiler# You can skip this for installing the dev branch.# for latest stable version use master branch.gitcheckouttags/<Release>-b<Release>pipinstall.From GithubDLP_VERSION=dev
pipinstallgit+https://github.com/hariharan-devarajan/dlio-profiler.git@${DLP_VERSION}For more build instructions checkhereUsagefrom dlio_profiler.logger import dlio_logger, fn_interceptor
    log_inst = dlio_logger.initialize_log(logfile=None, data_dir=None, process_id=-1)
    dlio_log = fn_interceptor("COMPUTE")

    # Example of using function decorators
    @dlio_log.log
    def log_events(index):
        sleep(1)

    # Example of function spawning and implicit I/O calls
    def posix_calls(val):
        index, is_spawn = val
        path = f"{cwd}/data/demofile{index}.txt"
        f = open(path, "w+")
        f.write("Now the file has more content!")
        f.close()
        if is_spawn:
            print(f"Calling spawn on {index} with pid {os.getpid()}")
            log_inst.finalize() # This need to be called to correctly finalize DLIO Profiler.
        else:
            print(f"Not calling spawn on {index} with pid {os.getpid()}")

    # NPZ calls internally calls POSIX calls.
    def npz_calls(index):
        # print(f"{cwd}/data/demofile2.npz")
        path = f"{cwd}/data/demofile{index}.npz"
        if os.path.exists(path):
            os.remove(path)
        records = np.random.randint(255, size=(8, 8, 1024), dtype=np.uint8)
        record_labels = [0] * 1024
        np.savez(path, x=records, y=record_labels)

    def main():
        log_events(0)
        npz_calls(1)
        with get_context('spawn').Pool(1, initializer=init) as pool:
            pool.map(posix_calls, ((2, True),))
        log_inst.finalize()


    if __name__ == "__main__":
        main()For this example, as the DLIO_PROFILER_CPP_INIT do not pass log file or data dir, we need to setDLIO_PROFILER_LOG_FILEandDLIO_PROFILER_DATA_DIR.
By default the DLIO Profiler mode is set to FUNCTION.
Example of running this configurations are:# the process id, app_name and .pfw will be appended by the profiler for each app and process.
    # name of final log file is ~/log_file-<APP_NAME>-<PID>.pfw
    DLIO_PROFILER_LOG_FILE=~/log_file
    # Colon separated paths for including for profiler
    DLIO_PROFILER_DATA_DIR=/dev/shm/:/p/gpfs1/$USER/dataset:$PWD/data
    # Enable profiler
    DLIO_PROFILER_ENABLE=1For more example checkExamples.
Package: dlipower
DESCRIPTIONThis is a python module and a script to mange the
Digital Loggers Web Power switch.The module provides a python class named
PowerSwitch that allows managing the web power
switch from python programs.When run as a script this acts as a command
line utility to manage the DLI Power switch.SUPPORTED DEVICESThis module has been tested against the following
Digital Loggers Power network power switches:ProSwitchWebPowerSwitch IIWebPowerSwitch IIIWebPowerSwitch IVWebPowerSwitch VEthernet Power Controller IIIExampleimportdlipowerprint('Connecting to a DLI PowerSwitch at lpc.digital-loggers.com')switch=dlipower.PowerSwitch(hostname="lpc.digital-loggers.com",userid="admin")print('Turning off the first outlet')switch.off(1)print('The powerstate of the first outlet is currently',switch[0].state)print('Renaming the first outlet as "Traffic light"')switch[0].name='Traffic light'print('The current status of the powerswitch is:')print(switch)ConnectingtoaDLIPowerSwitchatlpc.digital-loggers.comTurningoffthefirstoutletThepowerstateofthefirstoutletiscurrentlyOFFRenamingthefirstoutletas"Traffic light"Thecurrentstatusofthepowerswitchis:DLIPowerSwitchatlpc.digital-loggers.comOutletHostnameState1TrafficlightOFF2killerrobotON3BuitenverlictiON4MeetingRoomLiOFF5BrocadeLVM123ON6ShoretelABC123ON7Shortel24V-TON8Shortel24V-TON
Package: dlist-top
DList.top Python clientOfficialdlist.topgateway client for Python.Installationpip install dlist_topSetupTo get your token please refer to theDList.top documentation.Usagefromdlist_topimportClientdlist=Client(token='YOUR DLIST TOKEN')dlist.connect()@dlist.on('rate')defon_rate(data):print(data)@dlist.on('vote')defon_vote(data):print(data)
Package: dlk
A Deep Learning ToolKitThis project is WIP.Read the DocsInstallpip install dlk

or 
git clone this repo and do

python setup.py installWhat's this?Provide a templete for deep learning (especially for nlp) training and deploy.Provide parameters search.Provide basic architecture search.Provide some basic modules and models.Provide reuse the pretrained model for predict.More Feature is CommingTasks supportNLPClassificationPair ClassificationRegressionPair RegressionSequence LabelingSpan ClassificationRelation ExtractionToken RerankMRC SQuADTranslationSummaryCVClassificationGenerate models.Distill structure.Ensemble models for NLU(and check how to do this in NLG)Training StrategyAdversarial Training(FGM/PGD/FreeLB)Schedule Loss(you can control the loss schedule)Schedule MultiTask Loss(you can control the loss schedule for each task)Focal LossOnline service by triton.Data Augment.Support LightGBM. Will split to another package.Make most complexity modules like Beam Search, CRF to be scriptable.Add UnitTestParserTokenizerConfigLink
Package: dlkinematics
Deep Learning KinematicsDifferentiable Forwad Kinematics for TensorFlow and KerasSupported Joint Types:FixedRevoluteContiniousPrismaticFloating (not coverd by unit tests)Planar (not coverd by unit tests)InstallationInstall from PyPi$ pip install dlkinematicsInstall from source$ pip install -e git+https://github.com/lumoe/dlkinematics.git@main#egg=DLKinematicsUsage:importtensorflowastffromdlkinematics.urdfimportchain_from_urdf_filefromdlkinematics.dlkinematicsimportDLKinematics# Load URDFchain=chain_from_urdf_file('data/human.urdf')# Create DLKinematicsdlkinematics=DLKinematics(chain,base_link="human_base",end_link="human_spine_2",batch_size=2)# Joint configuartionthetas=tf.Variable([1.,2.,3.,4.],dtype=tf.float32)# Forward passwithtf.GradientTape()astape:result=dlkinematics.forward(thetas)print(result)print(tape.gradient(result,thetas))As Keras Layerfromdlkinematics.training_utilsimportForwardKinematicsfromtensorflowimportkerasimporttensorflowastfmodel=keras.Sequential()FK_layer=ForwardKinematics(urdf_file='path/to/urdf',base_link='link0',end_link='linkN',batch_size=2)model.add(FK_layer)# Output shape of FK_layer is (batch_size, 4, 4)Run testsThe tests use ROS packages to validate the result of the dlkinematics module.Build the docker image for tests:$ docker build -t dlkinematics_tests .Start the container in the root folder of the project:$ docker run -it -v $PWD:/work dlkinematics_tests python3 -m pytestExecute all tests:$ docker run -it -v $PWD:/work dlkinematics_tests python3 -m pytestExecute only a single testfile:$ docker run -it -v $PWD:/work dlkinematics_tests python3 -m pytest tests/test_prismatic.py
Package: dlkit
IntroductionThis documentation for the Digital Learning Toolkit (DLKit), like the toolkit itself, is still under development. It currently covers only a small handful of the 180 service packages and over 10,000 educational service APIs that have been defined by MIT’s Office of Digital Learning and its collaborators to date.The DLKit codebase is generated from the Open Service Interface Definitions (OSIDs), an extensive and growing suite of interface contract specifications that describe the integration points among the core services and components that make up modern educational systems.Note that this documentation is intended for API consumers. However, at the heart of DLKit is an integration stack that is even more closely aligned with the OSID specifications. This has been designed to allow third parties to extend the library with alternative or additional implementations of any of the defined services for the purposes of service integration, technology migration, service adaptation, etc. Documentation written for service implementers and system integrators, including implementation notes and compliance information, will be provided elsewhere.The completeOSID specificationprovides more details.Currently, DLKit works with Python 2.7, 3.4, 3.5, and 3.6.If you are interested in learning more about the DLKit Python libraries documented here, please contactdlkit-info@mit.eduRunning DLKitTutorialThisJupyter notebookis useful for learning about how to usedlkit.Example ProjectsFor help understandingdlkitand how to configure it, you can also
refer to an existing project, likeqbank.Runtime ConfigTo usedlkit, you need to include a runtime config in andlkit_configsdirectory:dlkit_configs
  |-configs.py
  |-registry.py
your_app
  |-app.pyExample / defaultdlkit_configfiles are included in this repo
and used for testing.Building This LibraryNote that this library is mostly generated by thedlkit_buildersrepo,
which can be foundhere.Hand-written modules are:primordiumaws_adapterfilesystem_adapterhandcarrecordstests/authz_implstests/dlkittests/functionaltests/learning/handcar_teststests/othertests/recordstests/utilities