Saravanan666/pypi-org huggingface dataset:
Package: opencv-transforms
opencv_transformsThis repository is intended as a faster drop-in replacement forPytorch's Torchvision augmentations. This repo uses OpenCV for fast image augmentation for PyTorch computer vision pipelines. I wrote this code because the Pillow-based Torchvision transforms was starving my GPU due to slow image augmentation.RequirementsA working installation of OpenCV.Tested with OpenCV version 3.4.1, 4.1.0Tested on Windows 10 and Ubuntu 18.04. There is evidence that OpenCV doesn't work well with multithreading on Linux / MacOS, for examplenum_workers >0in a pytorchDataLoader. I haven't run into this issue yet.Installationopencv_transforms is now a pip package! Simply usepip install opencv_transformsUsageBreaking change! Please note the import syntax!from opencv_transforms import transformsFrom here, almost everything should work exactly as the originaltransforms.Example: Image resizingimportnumpyasnpimage=np.random.randint(low=0,high=255,size=(1024,2048,3))resize=transforms.Resize(size=(256,256))image=resize(image)Should be 1.5 to 10 times faster than PIL. See benchmarksPerformanceMost transformations are between 1.5X and ~4X faster in OpenCV. Large image resizes are up to 10 times faster in OpenCV.To reproduce the following benchmarks, download theCityscapes dataset.An example benchmarking file can be found in the notebookbencharming_v2.ipynbI wrapped the Cityscapes default directories with a HDF5 file for even faster reading.The changes start to add up when you compose multiple transformations together.TODOInitial commit with all currently implemented torchvision transformsCityscapes benchmarksMake theresampleflag onRandomRotation,RandomAffineactually do somethingSpeed up augmentation in saturation and hue. Currently, fastest way is to convert to a PIL image, perform same augmentation as Torchvision, then convert back to np.ndarray
Package: opencv-utf-8
Opencv读写图像对utf-8路径（包括中文路径）的支持Support opencv reading and writing image using utf-8 paths.InstallInstall using pip:pip install opencv-utf-8or:pip install -i https://pypi.org/simple/ opencv-utf-8Usageimportcv2_extimportcv2img=cv2_ext.imread(r'E:\壁纸\pc-twoup-weibu.jpg')cv2.imshow('',img)cv2.waitKey()cv2_ext.imwrite(r'E:\壁纸\pc-twoup-weibu_中文.tif',img)Thanks.Good luck bro.
Package: opencv.wrap
opencv.wrapA collection of decorators for opencv and helper functions for multiple opencv tasks.Built with ❤︎ and ☕ by [@rishi23root](https://github.com/rishi23root/)rishi23root/opencv.wrap/FeaturesTotalTimeTakenCalculateFpsMirrorFrameConvertCOLORReadCamAndShowFramesReadCamAddDetectShowFramesReadCamAddDetectShowFrames_videoadditional Utilsface detectionhand detectioneye detection (yet to be added)
Package: opencypher
# opencypher[OpenCypher](https://www.opencypher.org/) AST and Builder APIOpenCypher defines an _abstract syntax tree_ based on the published OpenCypher[EBNF grammar](https://s3.amazonaws.com/artifacts.opencypher.org/cypher.ebnf)and a builder-oriented API for constructing Cypher queries.OpenCypher leans heavily on Python 3.7 `dataclasses` and `typing`.## Setuppip install opencypher## UsageThe core API exposes a fluent builder interface for constructing queries and patterns:from opencypher.api import match, nodequery = match(node("person", "Person").rel_in().node("pet", "Pet")).ret("person","pet",)print(query)  #  MATCH (person:Person)-[]->(pet :Pet) RETURN person, petThe builder supports chaining patterns and chaining clauses; queries may terminate on eithera return statement (`.ret()`) or on any updating clause (e.g. `create()`, `delete()`, `merge()`,and so forth.from opencypher.api import match, nodequery = match(node("alice", "Person", {"name": "Alice"}),).match(node("bob", "Person", {"name": "Bob"}),).merge(node("bob").rel_in(types="IS_FRIENDS_WITH").node("alice"),).merge(node("alice").rel_in(types="IS_FRIENDS_WITH").node("bob"),)The resulting `Cypher` query object integrates with -- but does not depend on -- the[Neo4J Python driver](https://github.com/neo4j/neo4j-python-driver):from neo4j import GraphDatabasedriver = GraphDatabase.driver("bolt://localhost:7687", auth=("neo4j", "password"))with driver.session() as session:session.run(str(query), dict(query))## CaveatsSome compromises have been made with respect to the completeness of the AST:1. The grammar for expressions (and literals, atoms, etc.) has been deliberately simplifiedand reduces to `str` in many cases. The expression grammar is likely to get more completeover time.2. Some forms of argumementation (e.g. `Set` items) are not easy to construct using the builderAPI (although these remain available within the AST).3. Several top level query clauses are not yet implemented, including:-  `CALL`-  `WITH`4. Parameters do not automatically generate unique identifiers/prefixes. Parameter names will bederived from variable names where known, but no fallback exists yet for anonyomous pattern terms.5. Parameters do not support numeric values (`$1`); symbolic names (`$foo`) *are* supported.
Package: opendal
Apache OpenDAL™ Python BindingDocumentation:mainThis package intends to build a native python binding for Apache OpenDAL.InstallationpipinstallopendalUsagefs service example:importopendalop=opendal.Operator("fs",root="/tmp")op.write("test.txt",b"Hello World")print(op.read("test.txt"))print(op.stat("test.txt").content_length)Or using the async API:importasyncioasyncdefmain():op=opendal.AsyncOperator("fs",root="/tmp")awaitop.write("test.txt",b"Hello World")print(awaitop.read("test.txt"))asyncio.run(main())s3 service example:importopendalop=opendal.Operator("s3",root="/tmp",bucket="your_bucket_name",region="your_region")op.write("test.txt",b"Hello World")print(op.read("test.txt"))print(op.stat("test.txt").content_length)Or using the async API:importasyncioasyncdefmain():op=opendal.AsyncOperator("s3",root="/tmp",bucket="your_bucket_name",region="your_region")awaitop.write("test.txt",b"Hello World")print(awaitop.read("test.txt"))asyncio.run(main())DevelopmentSetup virtualenv:python-mvenvvenvActivate venv:sourcevenv/bin/activateInstallmaturin:pipinstallmaturinBuild bindings:maturindevelopRun some tests:maturindevelop-Etest# To run `test_write.py` and use `fs` operatorOPENDAL_TEST=fsOPENDAL_FS_ROOT=/tmppytest-vktest_writeBuild API docs:maturindevelop-Edocs
pdoc-t./templateopendalLicense and TrademarksLicensed under the Apache License, Version 2.0:http://www.apache.org/licenses/LICENSE-2.0Apache OpenDAL, OpenDAL, and Apache are either registered trademarks or trademarks of the Apache Software Foundation.
Package: opendap-downloader
OPeNDAP downloaderLibrary to facilitate data downloads from OPeNDAP services configured with a file size limitation.This module will check if your requested file has a size under or above the file size limitation. If the size is under this limit, the file will be downloaded. Otherwise, the module will split the file (over time coordinates) in many files respecting the size limitation and download them. Ifoutput-fileoption is given, all the downloaded files will be concatenated into one and then be removed.InstallationWithpip:pip install opendap-downloaderUsageCommand Line Interface (CLI)Check-out usage and examples:> opendap-downloader --help
Usage: opendap_downloader.py [OPTIONS]

  Downloads OPeNDAP dataset as NetCDF files taking into account the server
  data query limit.

  Example:

    opendap-downloader --dataset-url https://nrt.cmems-du.eu/thredds/dodsC/METOFFICE-GLO-SST-L4-NRT-OBS-SST-V2 --variable analysed_sst --variable sea_ice_fraction --temporal-subset 2021-01-01 2021-01-02 --geographical-subset 0.0 0.1 0.0 0.1

    opendap-downloader -u https://nrt.cmems-du.eu/thredds/dodsC/METOFFICE-GLO-SST-L4-NRT-OBS-SST-V2 -v analysed_sst -v sea_ice_fraction -t 2021-01-01 2021-01-02 -g 0.0 0.1 0.0 0.1  

Options:
-u, --dataset-url TEXT          The full OPeNDAP dataset URL  [required]
  -v, --variable TEXT             Specify dataset variables
  -g, --geographical-subset <FLOAT RANGE FLOAT RANGE FLOAT RANGE FLOAT RANGE>...
                                  The geographical subset as minimal latitude,
                                  maximal latitude, minimal longitude and
                                  maximal longitude
  -t, --temporal-subset <DATETIME DATETIME>...
                                  The temporal subset as start datetime and
                                  end datetime
  -d, --depth-range <FLOAT RANGE FLOAT RANGE>...
                                  The depth range in meters, if depth is a
                                  dataset coordinate
  -o, --output-path PATH          The destination path for the downloaded
                                  files. Default is the current directory
                                  [required]
  -f, --output-file PATH          Concatenate the downloaded data in the given
                                  file name (under the output path)
  -l, --limit INTEGER             Specify the download size limit (in MB) of
                                  the Opendap server if it can't be provided
                                  by the message error
  --confirmation                  Print dataset metadata and ask for
                                  confirmation before download
  --login TEXT
  --password TEXT
  --help                          Show this message and exit.ContributeFollowthese instructions.
Package: opendap-protocol
[![Build Status](https://travis-ci.com/MeteoSwiss/opendap-protocol.svg?branch=master)](https://travis-ci.com/MeteoSwiss/opendap-protocol) [![codecov](https://codecov.io/gh/MeteoSwiss/opendap-protocol/branch/master/graph/badge.svg)](https://codecov.io/gh/MeteoSwiss/opendap-protocol) [![Documentation Status](https://readthedocs.org/projects/opendap-protocol/badge/?version=latest)](https://opendap-protocol.readthedocs.io/en/latest/?badge=latest) [![PyPI version](https://badge.fury.io/py/opendap-protocol.svg)](https://badge.fury.io/py/opendap-protocol)A pure Python implementation of the OPeNDAP server protocol.This module allows you to serve arbitrary data structures through the web
framework of your choice as OPeNDAP data objects. It implements just the bare
minimum of the DAP 2.0 protocol: DDS, DAS, and DODS responses and slicing. Array
data needs to be supplied asnumpy.ndarrayor asdask.array.Array.History0.1.0 (2020-03-27)First release on PyPI.
Package: opendaq
openDAQSDK that bridges the gap between data acquisition devices, in an era where incompatibility is the norm.openDAQ is a software development kit that allows for easy integration of data acquisition devices into its ecosystem.
It provides a simple generic API that allows users to connect to Data Acquisition (DAQ) devices that are compliant with
openDAQ and any of its supported open standards. With the SDK, users can configure properties of devices and read their
measurement data. It also contains a framework for developing custom signal processing blocks that process and analyse
signal data. In a world where DAQ devices do not readily communicate, openDAQ provides a common language for all to speak
while also embracing existing standards such as MQTT and XCP via Ethernet.InstallationThe package is available as binary wheels for Python 3.8-3.12. It has been built and tested for:64-bit Windowsx86_64 Linux (built on manylinux 2014, requires GLIBC >= 2.17)On supported systems you can install it with:pipinstallopendaqQuick-startYou can start the application by creating a new instance:importopendaq# Create a fresh openDAQ(TM) instance that we will use# for all the interactions with the openDAQ(TM) SDKinstance=opendaq.Instance()In order to list all the available devices, run:fordevice_infoininstance.available_devices:print("Name:",device_info.name,"Connection string:",device_info.connection_string)When using just a reference module without being connected to a physical device, the expected output will beName: Device 0 Connection string: daqref://device0
Name: Device 1 Connection string: daqref://device1More examples can be found underhttps://docs.opendaq.com/manual/.
Package: open-darts
openDARTSDARTS is a scalable parallel modeling framework and aims to accelerate the simulation performance while capturing multi-physics processes in geo-engineering fields such as hydrocarbon, geothermal, CO2 sequestration and hydrogen storage.InstallationopenDARTS is available for Python 3.7 to 3.11.pipinstallopen-dartsTo build openDARTS please check ourwiki.TutorialsCheck some our tutorials and a crash-course on openDARTShere.Also to get started take a look at the different examples inmodels.DocumentationFor more information about how to get started visit thedocumentation.Advanced examplesMore advanced examples of complex simulations with openDARTS can be found inopen-darts-models.LegalPlease refer toLICENSE.mdfor more information about the licensing of openDARTS.For developersCheck ourwiki.How to citeHelp and supportCheck our wiki section abouthow to contribute.
Package: open_data
A Python wrapper for the Microsoft Open Data API.Usage>>> from open_data import OpenData
>>> od = OpenData()
>>> od.catalog()
['bls', 'dc', ... ]>>> od.catalog('dc')
['BankLocations', 'TrafficCameras', 'ZoningPermits', ... ]>>> data = od.query('dc', 'BankLocations')
>>> print data
{'all': ['of', 'your', 'data']}>>> # Data returned in XML format.
... od.query('dc', 'BankLocations', format='xml')>>> # JSON-P data is also available.
... od.query('dc', 'BankLocations', callback='my_callback')CopyrightCopyright (c) 2011 Code for America Laboratories.See LICENSE for details.
Package: opendatagen
⬜️ Open Datagen ⬜️Open Datagenis a Data Preparation Tool designed to build Controllable AI SystemsIt offers improvements for:RAG: Generate large Q&A datasets to improve your Retrieval strategies.Evals: Create unique, “unseen” datasets to robustly test your models and avoid overfitting.Fine-Tuning: Produce large, low-bias, and high-quality datasets to get better models after the fine-tuning process.Guardrails: Generate red teaming datasets to strengthen the security and robustness of your Generative AI applications against attack.Additional FeaturesUse external sources to generate high-quality synthetic data (Local files, Hugging Face datasets and Internet)Data anonymizationOpen-source model support + local inferenceDecontaminationTree of thoughtMultimodality (Text, Audio and Image)(SOON) No-code dataset generationInstallationpipinstall--upgradeopendatagenSetting up your API keysexportOPENAI_API_KEY='your_openai_api_key'#(using openai>=1.2)exportMISTRAL_API_KEY='your_mistral_api_key'exportTOGETHER_API_KEY='your_together_api_key'exportANYSCALE_API_KEY='your_anyscale_api_key'exportELEVENLABS_API_KEY='your_elevenlabs_api_key'exportSERPLY_API_KEY='your_serply_api_key'#Google Search APIUsageExample: Generate a low-biased FAQ dataset based on Wikipedia contentfromopendatagen.templateimportTemplateManagerfromopendatagen.data_generatorimportDataGeneratoroutput_path="opendatagen.csv"template_name="opendatagen"manager=TemplateManager(template_file_path="faq_wikipedia.json")template=manager.get_template(template_name=template_name)iftemplate:generator=DataGenerator(template=template)data,data_decontaminated=generator.generate_data(output_path=output_path,output_decontaminated_path=None)where faq_wikipedia.json ishereContributionWe welcome contributions to Open Datagen! Whether you're looking to fix bugs, add templates, new features, or improve documentation, your help is greatly appreciated.AcknowledgementsWe would like to express our gratitude to the following open source projects and individuals that have inspired and helped us:Textbooks are all you need(Read the paper)Evol-Instruct Paper(Read the paper) byWizardLM_AITextbook GenerationbyVikParuchuriConnectIf you need help for your Generative AI strategy, implementation, and infrastructure, reach us onLinkedin:@Thomas.
Twitter:@thoddnn.
Package: opendatahub
OpenDataLab Python SDKIMPORTANT: OpenDataHub SDK status: WIP, which will not ensure the necessary compatibility of OpenAPI and SDK. As a result, please use the SDK with released version later.Please wait for the released version!!!OpenDataHub Python SDK is a python library to accessOpendatalaband use open datasets.It provides:A pythonic way to access opendatalab resources by Opendatalab OpenAPI.An convient CLI toolodl(Opendatalab for short) to access open datasets.Rich information about the open datasets.Installationpip3 install opendatahubUsage:Anaccountis needed to access to opendatalab service.
Please visitoffical websitto get the account username and password first.Loginlogin with opendatalab username and passwordodl login -u aaa@email.com -p ****orfromopendatahubimportODLodl=ODL.auth(username,pasword)Logoutlogout current opendatalab accountodl logoutorfromopendatahubimportODLodl.logout()lslist all dataset by offset, limitodl lsorfromopendatahubimportODLodl.list_datasets()infoshow dataset info in json formatodl info cocoorfromopendatahubimportODLodl.get_info()searchsearch dataset by nameodl search cocoorfromopendatahubimportODLodl.search('coco')downloaddownload dataset into local pathodl download --name coco --root /home/XXX/cocoorfromopendatahubimportODLodl.download(name='coco',root='/home/XXX/coco ')DocumentationMore information can be found on thedocumentation site
Package: opendatalab-python-sdk
OpenDataLab Python SDK 0.0.4Features:download dataset from offical website:https://opendatalab.com/preview dataset info onlineUsage:installpip install opendatalab-python-sdk==0.0.4download dataset into local pathscript:
$ python -m opendatalab.download --help    
    Tool for download opendatalab public dataset.  

  Options:
    --name        TEXT    Name of the public dataset.
    --format      TEXT    Format, avaiable options: source, standard.
    --root        TEXT    Data root path, default: current working path.
    --thread      INTEGER Number of thread for download, default: 10.
    --limit_speed INTEGER Download limit speed: KB/s, 0 is unlimited, default: 0

example:
1). download dataset source compressed
    $ python -m opendatalab.download --name coco --format source --root /home/XXX/coco

2). download standard format compressed
    $ python -m opendatalab.download --name coco --format standard --root /home/XXX/coco --thread   8 --limit_speed 5000SupportMore see:https://opendatalab.com/docs/3
Package: opendatalake
## OpenDataLake [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)Since it is all about data, this are data wrappers for common datasets which load the data into a common structure.There are handlers for several datasets.
To get you started quickly.## InstallSimply install it via pip.`bash pip install opendatalake `## ClassificationHere are all classification datasets.
They are loaded as a generator spitting out a feature(-vector/image) and a one-hot-encoded label.[Named Folders (Foldername = Label)](opendatalake/classification/named_folders.py)[MNIST](opendatalake/classification/mnist.py)ImageNet [TODO][Cifar10/Cifar100](opendatalake/classification/cifar.py)[LFW (named folders)](opendatalake/classification/named_folders.py)PASCAL VOC [TODO]Places [TODO]## SegmentationHere are all segmentation datasets.
They are loaded as a generator spitting out a feature(-vector/image) and segmentation(-vector/image).[Coco (WIP)](opendatalake/segmentation/coco.py)CamVid [TODO]Cityscapes [TODO]## DetectionHere are all detection datasets.
They are loaded as a generator spitting out a feature(-vector/image) and a detections(-vector/image).
A detection is either a detection_2d, detection_2.5d or detection_3d object defined [here](opendatalake/detection/utils.py).[Bosch TLR](opendatalake/detection/bosch_tlr.py)[Coco (WIP)](opendatalake/detection/coco.py)[Kitti Detection (incl. 3d)](opendatalake/detection/kitti_detection.py)[Pascal Voc 3d (WIP)](opendatalake/detection/pascal_voc_3d.py)[Udacity Autti (WIP)](opendatalake/detection/udacity_autti.py)[Udacity Crowdai (WIP)](opendatalake/detection/udacity_crowdai.py)## UnlabeledHere are all unlabeled datasets.
They are loaded as a generator spitting out a feature(-vector/image).[Image Folder](opendatalake/unlabeled/image_folder.py)
Package: opendata-pipeline
TODO:Add A LOT more PRINT statementsAdd commentsAdd documentation (README and docs site)The latter will be necesarry once we move to dockerfiles and actionsAdd tests 😅including CLI testsUse arcgis package for geocodingUse batch geocoding (had problem with Token... can register as anonymous user?)- [x] Use Socrata package (register API key) for data fetching from datasets published on SocrataUserequestspackage for data fetching from datasets published on odataUse github python package to keep config.yaml updated after successful runsCan also use to update JS datafiles at end of analysis (see below)Just used requests and api directlyThese should be very small and generated by pandas analysis of the dataresults should be in a github release (data files) (can zip them)Use GH CLI in bash script because pre-installed in ActionsWe can then just use the OctoKit JS package to point to the LINKS of the files and when you click on them it will download themthen web page to enable file downloads and show some graphs (basic --> records over time for each dataset)what charting frameowkr to use?Need an action to update the frontend codebase with the new dataStore in JSON format-  [ ] add website to socrata keyMake a container to run the whole pipeline (so no downloads for users)Host on GHCRMAKE OUR OWN UNIQUE IDENTIFIERS FOR ALL DATASETS COMBINEDSAME COLUMN NAME IN ALL DATASETS, THEN DON'T HAVE TO PROVIDE IDENTIFIER COLUMN IN config.yamlAlso allows for better merging of datasets (i.e. records + drugs + geo)- [ ] DO we want to publish a Web API as well?- [ ] Then weould need DBNo Windows support due to drug extraction tool usageI think, if my math is right, we can do ~20 minutes / day of actions... (2,000 minutes per month limit for free)*** make a note it is very important to often PULL to stay updated with the CONFIG
Package: opendataschema
Open Data Schema Python clientInstallpipinstallopendataschemaUsageNote: theschema.jsonfile can be given as a file path or an URL.opendataschemaschema.jsonlist
opendataschemaschema.jsonshow
opendataschemaschema.jsonshow--name<schema_name>
opendataschemaschema.jsonshow--versionsConfigurationEnvironment variables:CATALOG_SCHEMA: customise the URL or file path of the JSON schema to use to validate the catalog JSON fileGitHub API Rate limitingBecause of to therate limiting strategyof GitHub API, you may encounter agithub.GithubException.RateLimitExceededException. To avoid that, generate a private access token in yourtokens settings page, and set theGITHUB_ACCESS_TOKENenvironment variable.