Saravanan666/pypi-org huggingface dataset:
Package: dlq-redeem
dlq-redeemAWS SQS dead letter queue (DLQ) message inspection and cleanup automation toolInstallationpipinstalldlq-redeemWhat is it about?dlq-redeemallows you to go through, interactively, DLQ messages and decide if you want te reprocess them.Based on type of message it will either move it back to its source SQS or invoke lambda function in case message was an EventBridge event.
Package: dlrepo
dlrepodlrepois an artifact repository. It supports storing build artifacts
(binary packages, documentation, vm images, container images, etc.) in
a structured file system tree. It exposes an HTTP API to upload files, delete
them, add metadata, etc.dlrepodoes not use an external database. It
does de-duplication of artifacts by the use of file system hard links.installationpipinstalldlrepoOr, asrooton Debian testing:curl-Lhttps://repo.diabeteman.com/static/key.asc>/etc/apt/trusted.gpg.d/repo-diabeteman-com.ascecho"deb https://repo.diabeteman.com/products/dlrepo/all/main/0.x/deb/ /">/etc/apt/sources.list.d/dlrepo.list
aptupdate
aptinstalldlrepodevelopment quickstartsudoaptinstalllibldap2-devlibsasl2-devpython3-devpython3-pippython3-venvsassc
gitclonehttps://git.sr.ht/~rjarry/dlrepocddlrepo
makelinttests
makerundocumentationdlrepo.7dlrepo-cli.1dlrepo-config.5dlrepo-acls.5dlrepo-api.7dlrepo-layout.7contributingAnyone can contribute to dlrepo:Clone the repository.Patch the code.Make some tests.Ensure that your code is properly formatted with black (make format).Ensure that the linters are happy (make lint).Ensure that everything works as expected.Ensure that you did not break anything.If applicable, update unit tests.If adding a new feature, please consider adding new tests.Do not forget to update the docs.Once you are happy with your work, you can create a commit (or several
commits). Follow these general rules:Limit the first line (title) of the commit message to 60 characters.Use a short prefix for the commit title for readability withgit log --oneline.Use the body of the commit message to actually explain what your patch does
and why it is useful.Address only one issue/topic per commit.If you are fixing a ticket, use appropriatecommit trailers.If you are fixing a regression introduced by another commit, add aFixes:trailer with the commit id and its title.There is a great reference for commit messages in theLinux kernel documentation.Before sending the patch, you should configure your local clone with sane
defaults:git config format.subjectPrefix "PATCH dlrepo"
git config sendemail.to "~rjarry/dlrepo@lists.sr.ht"And send the patch to the mailing list:gitsendemail--annotate-1Wait for feedback. Address comments and amend changes to your original commit.
Then you should send a v2:gitsendemail--in-reply-to=$first_message_id--annotate-v2-1Once the maintainer is happy with your patch, they will apply it and push it.resourcesBrowsesource codeSubmit patches & questions to~rjarry/dlrepo@lists.sr.htFile or browsetickets
Package: dlroWolleH
dlroWolleH!dlroWolleHFeaturesHelloWorld('print')InstallationpipinstalldlroWolleHUsagefromdlroWolleHimportHelloWorldHelloWorld('print')You can also try a different input:HelloWorld('input')TheHelloWorld('something')will be executed assomething('Hello World!').FAQWhy?Why not?What can I do with this?Who knows?
Package: dls-backup-bl
A tool for backing up the configuration of the following devices on
a Beamline:Motion ControllerVME pmacsgeobrickszebrasTerminal serversACSMoxaConfluencehttps://confluence.diamond.ac.uk/x/WoxKBgPyPIpip install dls_backup_blSource codehttps://github.com/dls-controls/dls_backup_blDocumentationhttps://epics-containers.github.io/dls_backup_blReleaseshttps://github.com/dls-controls/dls_backup_bl/releasesHow to UseAll examples here use the beamline i16. Substitute i16 with the beamline name you
are working with.Run these tools on a beamline workstation or provide the
beamline name with the command line parameter--beamlinei16.First create a configuration file that describes the devices on the
beamline. The following command will allow you to view and edit the
list of devices that the backup process will manage:dls-backup-guiThis will open a GUI that will allow you to configure the devices.Once the configuration is complete you can launch the full backup with:dls_backup_blNote that you can backup a subset of devices like this:dls_backup_bl --devices BL16I-MO-STEP-01 BL16I-MO-STEP-02More HelpBoth tools have help which describes the command line options:dls-backup-gui --help
dls_backup_bl --helpThis confluence page has much more detailedhttps://confluence.diamond.ac.uk/x/WoxKBgBackup FilesAll the backup files are stored in/dls_sw/work/motion/Backups/BL16IThe device description file is/dls_sw/work/motion/Backups/BL16I/BL16I-backup.jsonThere are also log files in this directory. Plus a subfolder for each class
of backup device.The backup folder is a git
repository and all backups are incremental only. The full history of
backups can be retrieved with git commands.Seehttps://epics-containers.github.io/dls_backup_blfor more detailed documentation.
Package: dls-bluesky-core
This module stores Bluesky functionality that is common to multiple techniques or groups within the Diamond Light Source
organisation, such that it can be imported by instances of BlueAPI, or built upon within more focussed plan
repositories to reduce duplication.The ‘plans’ package contains functions that describe a full operation which performs an experiment and captures data,
and may wish to be available to instances of BlueAPI to allow common experiment types to be maintained centrally.
The ‘stubs’ package contains modular partial instructions that may act as a building block for constructing plans, for
which the implementation is common: e.g. querying APIs, standard handling of metadata
The ‘tasks’ package contains instructions that are not sufficient to run a full experiment but are useful utilities for
providing functionality to instances of BlueAPI: e.g. moving a motor.PyPIpip installdls-bluesky-coreSource codehttps://github.com/DiamondLightSource/dls-bluesky-coreDocumentationhttps://DiamondLightSource.github.io/dls-bluesky-coreReleaseshttps://github.com/DiamondLightSource/dls-bluesky-core/releasesThe module built from this repository is intended to either act directly as a source of plans for an instance of
BlueAPI by being a planFunctions source in the config of an instance:worker:env:sources:-kind:planFunctionsmodule:dls_bluesky_core.plans-kind:planFunctionsmodule:dls_bluesky_core.tasksOr else contributing functionality that may be common with other plan repositories within Diamond.importdls_bluesky_core.stubsascpsdeftechnique_specific_plan(*args,**kwargs):yield fromcps.common_diamond_setup()Seehttps://DiamondLightSource.github.io/dls-bluesky-corefor more detailed documentation.
Package: dls-dodal
Ophyd devices and other utils that could be used across DLS beamlinesPyPIpip installdls-dodalSource codehttps://github.com/DiamondLightSource/dodalDocumentationhttps://DiamondLightSource.github.io/dodalReleaseshttps://github.com/DiamondLightSource/dodal/releasesSeehttps://DiamondLightSource.github.io/dodalfor more detailed documentation.
Package: dls-mainiac
Convenient Python helper classes for cli Main programs.Intended advantages:consistent command lineconsistent loggingInstallationpip install git+https://gitlab.diamond.ac.uk/kbp43231/dls-mainiac.git

dls-mainiac --versionSummaryTBDDocumentationSeehttps://www.cs.diamond.ac.uk/dls-mainiacfor more detailed documentation.Building and viewing the documents locally:git clone git+https://gitlab.diamond.ac.uk/kbp43231/dls-mainiac.git
cd dls-mainiac
virtualenv /scratch/$USER/venv/dls-mainiac
source /scratch/$USER/venv/dls-mainiac/bin/activate
pip install -e .[dev]
make -f .dls-mainiac/Makefile validate_docs
browse to file:///scratch/$USER/venvs/dls-mainiac/build/html/index.htmlTopics for further documentation:TODO list of improvementschange log
Package: dls-marco-score-ispyb
dls_marco_score_ispby ia a script which runs the marco scoring tool
for images passed to it in a list and writes the scores into ispyb database
using ispby API.There are two requirements for it tow work:the connection to the database has to be establishedthe images for which the scoring is calculate need to be added to the database table called ispyb.BLSampleImage.See comments in ticket VMXI-488 for more details.Once the local database connection is established and the images added
to the database the script can be used as follows:import dls_marco_score_ispyb.score as mscrystal_images =
[‘imgae1’, ‘image2’]ms.run(crystal_images)FeaturesRun function takes up to 3 parameters. The first one is the list of image names
(absolute path) and is an obligatory parameter. The second is the path
to the ispyb config file (ispyb_config_path), this is a optional parameter with the default
path to the local host (DEFAULT_ISPYB_CONFIG=dls_marco_score_ispby/config.conf)
The third parameter is the path to the marco model, this is also
an optional parameter, the default value is the path to the model
packaged up with the module (DEFAULT_MODEL_DIR=dls_marco_score_ispby/savedmodel).InstallationIn order to install dls_marco_score_ispyb it do:pip install dls_marco_score_ispybNote one of the modules used by dls_marco_score_ispyb - tensorflow - is a bit picky in terms of the version of
pip used to install it. Make sure you have the latest version of pip installed.ContributeThe source code of dls_marco_score_ispyb is on gitlab
- Source Code:https://gitlab.diamond.ac.uk/controls/python3/dls_marco_score_ispybSupportIf you are having issues, please contact Urszula Neuman aturszula.neuman@diamond.ac.uk
Package: dls-multiconf
Python class for runtime configuration coordinating multiple services.Converting from old Configurator usage:
- in project.yaml depedencies add dls_multiconf
- from src delete configurators directory
- from tests delete test_configurator.py
- change all xchembku_lib.configurators.configurators to dls_multiconf_lib.multiconfs
- change all Configurators to Multiconfs
- change all xchembku_configurators_set_default to multiconfs_set_default
- change all configurator to multiconf
- change all chimpflow_datafaces_get_default to xchembku_datafaces_get_default
- change all “xchembku_lib.xchembku_multiconfs.yaml” to MulticonfThingTypes.YAML
- add from dls_multiconf_lib.constants import ThingTypes as MulticonfThingTypesTable of contentsViewing the docshttps://diamondlightsource.github.io/dls_multiconf
Package: dls-pairstream
Simple data streaming protocolIntended advantages:agnostic of type of underlying transport mechanismmeta/data message envelopereusable across projectsInstallationpip install git+https://gitlab.diamond.ac.uk/kbp43231/dls-pairstream.git

dls-pairstream --versionDocumentationSeehttps://www.cs.diamond.ac.uk/dls-pairstreamfor more detailed documentation.Building and viewing the documents locally:git clone git+https://gitlab.diamond.ac.uk/kbp43231/dls-pairstream.git
cd dls-pairstream
virtualenv /scratch/$USER/venv/dls-pairstream
source /scratch/$USER/venv/dls-pairstream/bin/activate
pip install -e .[dev]
make -f .dls-pairstream/Makefile validate_docs
browse to file:///scratch/$USER/venvs/dls-pairstream/build/html/index.htmlTopics for further documentation:TODO list of improvementschange log
Package: dls-pmacanalyse
A library and CLI for verifying that a delta tau motor controller
configuration matches its motion area definition.PyPIpip install dls_pmacanalyseSource codehttps://github.com/dls-controls/dls_pmacanalyseDocumentationhttps://dls-controls.github.io/dls_pmacanalyseReleaseshttps://github.com/dls-controls/dls_pmacanalyse/releasesSeehttps://dls-controls.github.io/dls_pmacanalysefor more detailed documentation.
Package: dls-pmaccontrol
dls_pmac_controlThe dls_pmac_control package is a python application which provides a GUI front
end to control the DeltaTau PMAC, Power PMAC and Geobrick motor control systems.Sourcehttps://github.com/DiamondLightSource/dls-pmac-controlPyPIpip install dls-pmaccontrolDocumentationhttps://diamondlightsource.github.io/dls-pmac-controlReleaseshttps://github.com/DiamondLightSource/dls-pmac-control/releasesSeehttps://diamondlightsource.github.io/dls-pmac-controlfor more detailed documentation.
Package: dls-pmaclib
The dls_pmaclib library brings together a number of useful modules that are used with the various PMAC tools.DocumentationFull documentation is available athttp://dls_pmaclib.readthedocs.ioSource CodeAvailable fromhttp://github.com/dls-controls/dls_pmaclibInstallationTo start using this project:git clone https://github.com/dls-controls/dls_pmaclibContributingSeeCONTRIBUTINGLicenseAPACHE License. (seeLICENSE)
Package: dls-powerpmacanalyse
dls_powerpmacanalyseA library and CLI for analysing/comparing/downloading Power PMAC
configurations.UsageHelp:dls-powerpmac-analyse -hLaunch GUI:dls-powerpmac-analyse -gContributingSeeCONTRIBUTINGLicenseAPACHE License. (seeLICENSE)
Package: dls-python3-skeleton
This skeleton module (inspired byjaraco/skeleton) is a generic Python project structure
which provides a means to keep tools and techniques in sync between multiple
Python projects.PyPIpip installdls-python3-skeletonSource codehttps://github.com/dls-controls/dls-python3-skeletonDocumentationhttps://dls-controls.github.io/dls-python3-skeletonReleaseshttps://github.com/dls-controls/dls-python3-skeleton/releasesIt integrates the following tools:Pipenv for version managementPre-commit with black, flake8, isort and mypy for static analysisPytest for code and coverageSphinx for tutorials, how-to guides, explanations and reference documentationGitHub Actions for code and docs CI and deployment to PyPI and GitHub PagesIf you use VSCode, it will run black, flake8, isort and mypy on saveTheskeletonbranch of this module contains the source code that can be
merged into new or existing projects, and pulled from to keep them up to date.
It can also serve as a working example for those who would prefer to
cherry-pick.Themasterbranch contains the
docs and a command line tool to ease the adoption of this skeleton into new:dls-python3-skeleton new /path/to/be/createdand existing projects:dls-python3-skeleton existing /path/to/existing/repoSeehttps://dls-controls.github.io/dls-python3-skeletonfor more detailed documentation.
Package: dls-python-copier-template-example
python_copier_template_exampleAn expandedhttps://github.com/DiamondLightSource/python-copier-templateto illustrate how it looks with all the options enabled.This is where you should write a short paragraph that describes what your module does,
how it does it, and why people should use it.Sourcehttps://github.com/DiamondLightSource/python-copier-template-examplePyPIpip install dls-python-copier-template-exampleDocumentationhttps://diamondlightsource.github.io/python-copier-template-exampleReleaseshttps://github.com/DiamondLightSource/python-copier-template-example/releasesThis is where you should put some images or code snippets that illustrate
some relevant examples. If it is a library then you might put some
introductory code here:frompython_copier_template_exampleimport__version__print(f"Hello python_copier_template_example{__version__}")Or if it is a commandline tool then you might put some example commands here:python -m python_copier_template_example --versionSeehttps://diamondlightsource.github.io/python-copier-template-examplefor more detailed documentation.
Package: dls-toolkit
DLS ToolkitDescriptionA set of my common tools.Installationpip install dls-toolkitEnvironment Variables# Google Cloud

GOOGLE_APPLICATION_CREDENTIALS - Path to the Google Cloud Service Account JSON file.
    Needs access to:
        - Google Drive

# AWS

AWS_ACCESS_KEY_ID - AWS Access Key
AWS_SECRET_ACCESS_KEY - AWS Secret Access Key
AWS_REGION - AWS Region
    Needs access to:
        - BillingLicenseThis project is licensed under theMIT License.ContactGitHub
Package: dlsym
dlsym – A cross-platform symbol locatordlsymallows Python C extension modules to use symbols present in already
loaded C libraries, without having to actually link these libraries.  As a
simple example, usingpybind11:double(*my_atan2)(double,double);my_atan2=reinterpret_cast<decltype(my_atan2)>(py::module::import("dlsym").attr("dlsym")("atan2").cast<uintptr_t>());Obviously, linking againstlibmto get access toatan2is not
particularly difficult, but this approach also allows one to use e.g.numpy-provided BLAS/LAPACK functions which are available after importingnumpy(regardless of whether the underlying implementation is OpenBLAS,
MKL, or something else),fftwfunctions after importingpyfftw, orTcl/Tkfunctions after importingtkinter(see tests for examples).The main goal here is to simplify the compilation of such extension modules on
machines where the C libraries may not be present by default, but where they
can be “requested” by declaring aninstall_requireson the corresponding
Python package.Note that the path to the shared library is not actually passed as an argument
todlsym(unlike the POSIXdlsym(3).  This is because the symbol search
on Windows has to enumerate all loaded modules anyways, as one cannot just
pass a module that transitively loads the symbol.  On POSIX, we thus follow
the same strategy for consistency (but enumerating all extension modules insys.modulesinstead).The path of the library where the symbol is actually defined is logged at the
INFO level.I learned this trick from Matthew Brett’s original implementation for loadingtkinterintomatplotlib.
Package: dlt2json
dlt2jsonConvert aGENIVI's dlt file to json format.This is a draft version and only supports log type for now.Control, app trace and network trace payload is encoded in hexadecimal.DLT message formatHeader0..3    header pattern DLT14..7    4 bytes epoch time (little endian)8..11   4 bytes microseconds (little endian)12..15  4 bytes ECU ID16      1 byte header type contains some configuration datause extended headerMSB firstwith ECU IDwith session IDwith timestampversion number, 0x117      1 byte message counter18..19  2 bytes lentgh of complete message without header20..23  4 bytes ECU ID again24..27  4 bytes session number28..31  4 bytes timestamp since system start in 0.1 milliseconds32      1 byte message info contains some configuration data toomessage type (log, app trace, network trace, control)message type info (debug, error, ...)33      1 byte number of arguments34..37  4 bytes application ID38..41  4 bytes context idpayload0..4    4 bytes argument type5..6    2 argument size....    argumentinstall$pythonsetup.pyinstallUsageFor a quick test, downloadtestfile.dltand run$dlt2jsontestfile.dlt[{"000":{"seconds":1305029670,"microSeconds":828120,"ecu":"ECU","sid":"N/A","timestamp":979535935,"app":"APP","ctx":"CON","type":"control","info":"request","args":1,"payload":"11 00 00 00 04 72 65 6d 6f"}},...{"030":{"seconds":1305029677,"microSeconds":562582,"ecu":"ECU","sid":"N/A","timestamp":"N/A","app":"LOG","ctx":"TES4","type":"log","info":"info","args":2,"payload":"0 Hello world"}},...{"104":{"seconds":1305029689,"microSeconds":47104,"ecu":"ECU","sid":"N/A","timestamp":"N/A","app":"N/A","ctx":"N/A","type":"N/A","info":"N/A","args":"N/A","payload":"91 01 00 00 1a 00 00 00 0c 00 48 65 6c 6c 6f 20 77 6f 72 6c 64 00"}}]
Package: dltkai
DLTK open-source SDK (Python)DLTK renders a comprehensive spectrum of solutions that can be accessed by users on-demand from our pool of transformational technologies.InstallationDLTK SDK requires Python 3.5 + .Fork the repo on GitHubDownload the project code files with:git clone https://github.com/dltk-ai/dltkai-sdk.gitGo to the project directorycd dltkai-sdk/pythonInstall all the required packages:pip install .Usageimportdltkaiasdlresponse=dl.NaturalLanguage.sentiment_detect('I am feeling good.')print(response)or you can also checkout some samples by running test.py:python test.pyFor more details, visithttps://dltk.ai/LicenseThe content of this project itself is licensed underGNU LGPL, Version 3 (LGPL-3)
Package: dltk-ai
DLTK SDK (Python)DLTK renders a comprehensive spectrum of solutions that can be accessed by users on-demand from our pool of transformational technologies.InstallationDLTK SDK requires Python 3.5 + . Go tohttps://dltk.ai/and create an app. On creation of an app, you will get an API Key.importdltk_aic=dltk_ai.DltkAiClient('API Key')response=c.sentiment_analysis('I am feeling good.')print(response)For more details, visithttps://dltk.ai/LicenseThe content of this project itself is licensed underGNU LGPL, Version 3 (LGPL-3)
Package: dlt-metabase-source
dlt-metabase-sourceParent tablesStateful tables: these get replaced on each load'stats', 'cards', 'collections', 'dashboards', 'databases', 'metrics', 'pulses',
'tables', 'segments', 'users', 'fields'Append (event) tables: these endpoints buffer a small event window, you need to merge it afterwardsto do - add time filter parameters to only load filtered requested data.'activity', 'logs'some of these tables have sub-tablesto join the parent table to the sub table, use the joinparent.dlt_id = child.parent_dlt_idUsageoptionally Create a virtual environmentpython3 -m venv ./dlt_metabase_env4
source ./dlt_metabase_env4/bin/activateinstall librarypip install dlt-metabase-sourceIf the library cannot be found, ensure you have the required python version as per thepyproject.tomlfile.
(3.8+)You can run the snippet file below to load a sample data set.
You would need to add your target credentials first.python run_load.pyFirst, import the loading method and add your credentialsfrom dlt_metabase_source import load


# target credentials
# example for bigquery
creds = {
  "type": "service_account",
  "project_id": "zinc-mantra-353207",
  "private_key_id": "example",
  "private_key": "",
  "client_email": "example@zinc-mantra-353207.iam.gserviceaccount.com",
  "client_id": "100909481823688180493"}
  
# or example for redshift:
# creds = ["redshift", "database_name", "schema_name", "user_name", "host", "password"]Metabase credentialsurl='http....com',
user='example@ai',
password='dolphins',Now, you can use the code below to do a serial load:mock_data=Trueflag below will load sample data.Remove or set to False themock_dataflag to enable loading your data.# remove some tables from this list of you only want some endpoints
tables=['activity', 'logs', 'stats', 'cards', 'collections', 'dashboards', 'databases', 'metrics', 'pulses',
                 'tables', 'segments', 'users', 'fields']
                 
load(url=url,
         user=user',
         password=password,
         target_credentials=creds,
         tables=tables,
         schema_name='metabase',
         mock_data=True)or, for parallel load, create airflow tasks for each table like so:for table in tables:
    load(url=url,
         user=user',
         password=password,
         target_credentials=creds,
         tables=[table],
         schema_name='metabase',
         mock_data=True)If you want to do your own pipeline or consume the source differently:from dlt_metabase_source import MetabaseSource, MetabaseMockSource

prod = MetabaseSource(url='http....com',
         user='example@ai',
         password='dolphins')
              
dummy = PersonioSourceDummy()

sample_data = dummy.tasks() 

for task in tasks:
    print(task['table_name'])
    for row in task['data']
        print(row)