Saravanan666/pypi-org huggingface dataset:
Package: datumaro
Dataset Management Framework (Datumaro)A framework and CLI tool to build, transform, and analyze datasets.VOC dataset                                  ---> Annotation tool
     +                                     /
COCO dataset -----> Datumaro ---> dataset ------> Model training
     +                                     \
CVAT annotations                             ---> Publication, statistics etc.Getting startedLevel UpFeaturesUser manualDeveloper manualContributingFeatures(Back to top)Dataset reading, writing, conversion in any direction.CIFAR-10/100(classification)CityscapesCOCO(image_info,instances,person_keypoints,captions,labels,panoptic,stuff)CVATImageNetKitti(segmentation,detection,3D raw/velodyne points)LabelMeLFW(classification,person re-identification,landmarks)MNIST(classification)Open ImagesPASCAL VOC(classification,detection,segmentation,action_classification,person_layout)TF Detection API(bboxes,masks)YOLO(bboxes)Other formats and documentation for them can be foundhere.Dataset buildingMerging multiple datasets into oneDataset filtering by a custom criteria:remove polygons of a certain classremove images without annotations of a specific classremoveoccludedannotations from imageskeep only vertically-oriented imagesremove small area bounding boxes from annotationsAnnotation conversions, for instance:polygons to instance masks and vice-versaapply a custom colormap for mask annotationsrename or remove dataset labelsSplitting a dataset into multiple subsets liketrain,val, andtest:random splittask-specific splits based on annotations,
which keep initial label and attribute distributionsfor classification task, based on labelsfor detection task, based on bboxesfor re-identification task, based on labels,
avoiding having same IDs in training and test splitsSampling a datasetanalyzes inference result from the given dataset
and selects the ‘best’ and the ‘least amount of’ samples for annotation.Select the sample that best suits model training.sampling with Entropy based algorithmDataset quality checkingSimple checking for errorsComparison with model inferenceMerging and comparison of multiple datasetsAnnotation validation based on the task type(classification, etc)Dataset comparisonDataset statistics (image mean and std, annotation statistics)Model integrationInference (OpenVINO, Caffe, PyTorch, TensorFlow, MxNet, etc.)Explainable AI (RISE algorithm)RISE for classificationRISE for object detectionCheckthe design documentfor a full list of features.
Checkthe user manualfor usage instructions.Contributing(Back to top)Feel free toopen an Issue, if you
think something needs to be changed. You are welcome to participate in
development, instructions are available in ourcontribution guide.Telemetry data collection noteTheOpenVINO™ telemetry libraryis used to collect basic information about Datumaro usage.To enable/disable telemetry data collection please see theguide.
Package: datumaro-headless
Dataset Management Framework (Datumaro)A framework and CLI tool to build, transform, and analyze datasets.VOC dataset                                  ---> Annotation tool
     +                                     /
COCO dataset -----> Datumaro ---> dataset ------> Model training
     +                                     \
CVAT annotations                             ---> Publication, statistics etc.Getting startedFeaturesUser manualDeveloper manualContributingFeatures(Back to top)Dataset reading, writing, conversion in any direction.CIFAR-10/100(classification)CityscapesCOCO(image_info,instances,person_keypoints,captions,labels,panoptic,stuff)CVATImageNetKitti(segmentation,detection,3D raw/velodyne points)LabelMeLFW(classification,person re-identification,landmarks)MNIST(classification)Open ImagesPASCAL VOC(classification,detection,segmentation,action_classification,person_layout)TF Detection API(bboxes,masks)YOLO(bboxes)Other formats and documentation for them can be foundhere.Dataset buildingMerging multiple datasets into oneDataset filtering by a custom criteria:remove polygons of a certain classremove images without annotations of a specific classremoveoccludedannotations from imageskeep only vertically-oriented imagesremove small area bounding boxes from annotationsAnnotation conversions, for instance:polygons to instance masks and vice-versaapply a custom colormap for mask annotationsrename or remove dataset labelsSplitting a dataset into multiple subsets liketrain,val, andtest:random splittask-specific splits based on annotations,
which keep initial label and attribute distributionsfor classification task, based on labelsfor detection task, based on bboxesfor re-identification task, based on labels,
avoiding having same IDs in training and test splitsSampling a datasetanalyzes inference result from the given dataset
and selects the ‘best’ and the ‘least amount of’ samples for annotation.Select the sample that best suits model training.sampling with Entropy based algorithmDataset quality checkingSimple checking for errorsComparison with model inferenceMerging and comparison of multiple datasetsAnnotation validation based on the task type(classification, etc)Dataset comparisonDataset statistics (image mean and std, annotation statistics)Model integrationInference (OpenVINO, Caffe, PyTorch, TensorFlow, MxNet, etc.)Explainable AI (RISE algorithm)RISE for classificationRISE for object detectionCheckthe design documentfor a full list of features.
Checkthe user manualfor usage instructions.Contributing(Back to top)Feel free toopen an Issue, if you
think something needs to be changed. You are welcome to participate in
development, instructions are available in ourcontribution guide.Telemetry data collection noteTheOpenVINO™ telemetry libraryis used to collect basic information about Datumaro usage.To enable/disable telemetry data collection please see theguide.
Package: datup
datupHow is it work?import datup as dtTo Instance the Classins = dt.Datup("aws_acces_key_id","aws_secret_access_key","datalake")Is necessary invoke all time the instance object?Yes, is necessary invoke all time the instance object since, the functions must
write in /tmp/ log file as default. In documentation will find how to use the class
and change default values.Example of a method with the callable instancedt.check_data_format(ins,dataframe)Can I test my updates?Yes, there is a file called _testing.ipynb where you can test your changes. The variables,
must be initialized always for modularity.Where can I find some documentation?You can find the documentation in the next address:http://18.234.95.22/Documentation as the library is on beta mode, for that reason write tocristhianpo@datup.aifor
whatever bug
Package: daty
DatyDatyis a free cross-platform advanced Wikidata editor adhering toGNOME Human Interface Guidelines, intended to enable better editing workflow and faster deployment of requested user features.
UseDatyto search, select, read, batch edit items, script actions, share, visualize proposed changes and bots.Datyis written in Python 3 and it usesGTK+ 3.0python bindings for interface organization and drawing.It has a progressive layout thanks tolibhandyand usespywikibotas a backend.Current statusThe development of the current version (1.0α) has been made possible thanks to a sponsorship provided byWikimedia CH.Endorse the project onWikidatato help the development get funds.An overview of what you will find in the stable 1.0 version is available at project presentationpage.Search and open entities through elastic search;Search and open entities with triplets (broken in the current version);Read entities and follow their values;Mobile view;Open entities search;Property In-page search;Edit statements;Mass-edit statements.InstallationWindowsYou can download the installerhere.GNU/LinuxFlatpakArchlinuxThe packagedaty-githas been published onAUR.Ubuntu Disco (19.04 unstable)# apt install python3-gi gir1.2-gtk-3.0 python3-pip libhandy-0.0-0
# pip3 install pywikibot datyOthersProvided you have already installed on your system* pygobject >= 3.20
* Gtk >= 3.20
* libhandy >= 0.0.4
* pywikibot >= 3.0you can install daty fromPypi.Mac OSHardware or contributors needed.Building from sourceOption 1: with GNOME BuilderOpen GNOME Builder, click the "Clone..." button, paste the repository url.
Clone the project and hit thebutton to start building Daty.Option 2: with Flatpak Builder# Clone Daty repository
git clone https://gitlab.gnome.org/World/Daty.git
cd daty
# Add Flathub repository
flatpak remote-add flathub --if-not-exists https://dl.flathub.org/repo/flathub.flatpakrepo
# Install the required GNOME runtimes
flatpak install flathub org.gnome.Platform//3.30 org.gnome.Sdk//3.30
# Start building
flatpak-builder --repo=repo ml.prevete.Daty flatpak/ml.prevete.Daty.json --force-clean
# Create the Flatpak
flatpak build-export repo ml.prevete.Daty
flatpak build-bundle repo ml.prevete.Daty.flatpak ml.prevete.Daty
# Install the Flatpak
flatpak install ml.prevete.Daty.flatpakOption 3: with PypiProvided you have installed* pygobject >= 3.20
* Gtk >= 3.20
* libhandy >= 0.0.4
* pywikibot >= 3.0You just need to enter in the cloned directory and run$ sudo python3 setup.py install
$ datyDocumentationSphinx documentation for the project can be built running$ python3 setup.py build_sphinxYou can then read the main page of the built html documentation directing your browser todoc/build/html/index.html.It will be made directly available online after the code will stabilize.AboutThis program is licensed underGNU Affero General Public License v3 or laterbyPellegrino Prevete.If you find this program useful, considerendorsingthe project, offering me abeer, a newcomputeror a part time remotejobto help me pay the bills.
Package: dau
Django Auto UserDjango Auto User (DAU) is a Django (Python) library for the automatic creation of users.InstallationUse the package managerpipto install Django Auto User.pipinstalldauUsageAdd 'dau' to INSTALLED APPS after 'django.contrib.admin' in your settings.py:INSTALLED_APPS=[...'django.contrib.admin','dau',...]Then, just add DJANGO_AUTO_USER settings to your settings.py file as:DJANGO_AUTO_USER=[{'username':'admin','email':'admin@admin.com','password':'adminpass','is_superuser':True,'is_staff':True,'is_active':True}]Make your migrations with:pythonmanage.pymakemigrations
pythonmanage.pymigrateContributingPull requests are welcome. For major changes, please open an issue first to discuss what you would like to change.
Please make sure to update tests as appropriate.LicenseMIT
Package: daub
🖌️ daubOpinionated pdf renderer⚙️ Installpipinstall--userdaubOr, to install from source:gitclonegit@github.com:verillious/daub.git
pipinstall--user-edaub/🖊️ UsageFrom the command line:Usage:daub[OPTIONS]MARKDOWN_FILEOpinionatedpdfrenderer

Options:-o,--outputPATHOverridetheoutputpdffilepath.--cssPATHOverridethedefaultstylesheet.--versionShowtheversionandexit.--helpShowthismessageandexit.💪 FeaturesConverts a markdown file to a pdf fileOptionally takes in a custom css file for stylingBy default tries to emulate theminimal-github-theme🔍 AlternativesRequiresWeasyprintwhichonly supports Windows 11:md2pdfmarkdown2pdfmarkdown-to-pdfresume.md(RequiresChromeorChromium)rinohtype(Doesn't support css)amd2pdf(Requireswkhtmltopdf)🙏 Credits🍪 This package was created withCookiecutterand theverillious/cookiecutter-pythonproject template.
Package: daum_openapi
Python library for Daum Open Data API.Daum OPEN API :http://dna.daum.net/apisInstallationpip install daum_openapiFeaturesJSON response keys to class member variables.Convert XML response to JSON and then keys to class member variables.Usagesearch : blog>>> from daum_openapi.search import search
>>> sc = search(API_KEY)
>>> result =  sc.blog(u'coldplay', 10, 1, 'date', 'xml')
>>> {
>>>  "channel":
>>>  {
>>>    "result":"10",
>>>    "pageCount":"800",
>>>    "title":"Search Daum Open API"
>>>    "item": [
>>>              {"title":"Coldplay - viva la vida", "description":"viva la vida new song"},
>>>              {"title":"YELLOW", "description":"Yellow lyrics"}
>>>            ]
>>>
>>>  }
>>> }
>>> result.channel.result  //"10"
>>> result.channel.pageCount //"800"
>>> result.channel.title  //"Search Daum Open API"
>>> result.channel.item[0].title  //"Coldplay - viva la vida"
>>> result.channel.item[1].description   //"Yellow lyrics"
Package: dauricum
dauricumA work-in-progress Python 3.10+ obfuscator with many obfuscation methods.This obfuscator is built on ast. Inspired byjargonaut.Note that this is a proof-of-concept and a work in progress.FeaturesMBA ExpressionsIn OutlineControl FlowTry CatchTry Catch NormalizerOpaqueRenamer (Classes, fields, functions)String ObscureExamplesCheck out examples folderHow toInstallationInstall with pippipinstalldauricumInstall from githubgitclonehttps://github.com/Maxdsdsdsd/dauricum.gitcddauricumbuild.batUseObfuscate .py filepy-mdauricum-inputexample1-unobf.py-outputexample1-obf-0.4.py--mba-expression--mba-expression-modetrue--in-outline--control-flow--try-catch--try-catch-modetrue--try-catch-iter3--try-normalizer--try-normalizer-iter5--rename--rename-mode3--opaque--opaque-iter5--string-obscureGet helppy-mdauricum--help
Package: dauth
DAuthPython FastAPI ABAC Realization.Getting startedRequirements:redis- for cache supportdatabases- for database supportInstall:pipinstalldauthAfter installing import main functions byfromdauthimportauthLibrary represents functiondefPolicy(subject_call:Callable,resource_type:Any,method:str,check_callback:Callable,database:Optional[Database]=None,cache:Optional[Redis]=None)subject_callback- Is function that library put in FastAPIDepends. Usually function returns user which tries to work withresourceresource_type- Isstrwhich on which User tries to get accessmethod- Is API method by which working endpointcheck_callback- Is function that realize Policy's check. Function take arguments:check_callback(subject,resource_type,item_id,method,db,cache)subjectis result ofDepends(subject_callback)resource_typeis argument ofPolicy()item_id(by default '*')is providing by FastAPI decorator@app.get(/test/{item_id})methodis argument ofPolicy()dbis database connectioncacheis redis connectiondatabase- Is a database connecioncache- Is a Redis connectionExamplesSimple usagefromfastapiimportFastAPI,Dependsfromdauthimportauthapp=FastAPI()defis_admin(subject,resource_type,item_id,method,db,cache):if'admin'notinsubject.scopes:raiseauth.DENY@app.get("/test")# function get_user_auth returns User's objectdeftest(user=Depends(auth.Policy(get_user_auth,'test','get',is_admin))):return{"message":"Good"}@app.get("/test_with_db")# function get_database returns Databases connectiondeftest_db(user=Depends(auth.Policy(get_user_auth,'test','get',is_admin,database=Depends(get_database)))):return{"message":"Good"}@app.get("/test_with_cache")# function get_cache returns Redis connectiondeftest_cache(user=Depends(auth.Policy(get_user_auth,'test','get',is_admin,cache=Depends(get_cache)))):return{"message":"Good"}Developed byDenVilk
Package: dautomation
# requirements_dev.txt we use for the testing
It makes it easier to install and manage dependencies for development and testing, separate from the dependencies required for production.# difference between requirements_dev.txt and requirements.txtrequirements.txt is used to specify the dependencies required to run the production code of a Python project, while requirements_dev.txt is used to specify the dependencies required for development and testing purposes.# tox.ini
We use if for the testing in the python package testing against different version of the python## how tox works tox enviornment creation
1. Install depedencies and packages
2. Run commands
3. Its a combination of the (virtualenvwrapper and makefile)
4. It creates a .tox# pyproject.toml
it is being used for configuration the python project it is a alternative of the setup.cfg file. its containts configuration related to the build system
such as the build tool used package name version author license and dependencies# setup.cfg
In summary, setup.cfg is used by setuptools to configure the packaging and installation of a Python projec# Testing python applicationtypes of testing1. Automated testing
2. Manual testingMode of testing1. Unit testing
2. Integration testsTesting frameworkspytestunittestrobotframeworkseleniumbehavedoctest# check with the code style formatting and syntax(coding standard)pylintflake8(it is best because it containt 3 library pylint pycodestyle mccabe)pycodestyle
Package: davat
Davat(دوات)A very simple python library for normalizing and cleaning Persian text.Text NormalizingText CleaningPython 3.x supportUsage>>>importdavat>>>sample_text="بِسْمِ اللَّهِ الرَّحْمنِ الرَّحِيمِ">>>davat.normalize(sample_text)'بسم الله الرحمن الرحیم'>>>sample_text="""این یك متن تست است که حروف عربي ، کشیـــــده'اعداد 12345' و... دارد     که می خواهیم آن را نرمالایز کنیم .""">>>print(davat.normalize(sample_text))اینیکمتنتستاستکهحروفعربی،کشیده«اعداد۱۲۳۴۵»و…داردکهمی‌خواهیمآنرانرمالایزکنیم.>>>sample_text="""... متنی برای برسی تابع تمیز کردن متن... که #هشتگ_ها را خیلی عاااااللللییییی!!!! تبدیل به متن عادی می‌کند!... منشن‌ها @mh_salari و لینک‌ها www.mh-salari.ir را حذف می‌کند.... حروف غیر فارسی  a b c d و اموجی‌ها :( 🐈‍ را حذف می‌کند... علائم دستوری/نگارشی ?!٫ را حذف نمی‌کند... و ...... http://localhost:8888... """>>># davat.clean(...#     text: str,...#     remove_links=True,...#     remove_mentions=True,...#     remove_hashtags=False,...#     remove_hashtag=True,...#     remove_underline=True,...#     remove_emojis=True,...#     normalize_persian=True,...#     remove_punctuations=False,...#     fix_multiple_punctuations=True,...#     remove_3dots=False,...#     remove_non_persian_letters=True,...#     remove_extraspaces=True,...#)...>>>text=davat.clean(sample_text)>>>print(text)متنیبرایبرسیتابعتمیزکردنمتنکههشتگ‌هاراخیلیعااللیی!تبدیلبهمتنعادیمی‌کند!منشن‌هاولینک‌هاراحذفمی‌کند.حروفغیرفارسیواموجی‌هاراحذفمی‌کندعلائمدستوری/نگارشی؟!،راحذفنمی‌کندو…InstallationThe latest stable version of Davat can be installed throughpip:pip install davatThanks to:Persian poems corpusHazmparsivar
Package: davbackup
Downloads an entire tree from a DAV server. This can be used to make backups from an owncloud/nextcloud server.UsageThe following output of the--helpoption shows the usage:$ python3 davbackup.py --help
usage: davbackup.py [-h] [--config FILE] [--destdir DIR] [--start RMT_DIR]

Download content from DAV server.

optional arguments:
  -h, --help            show this help message and exit
  --config FILE, -c FILE
                        Specify a config file other than the default.
  --destdir DIR, -d DIR
                        Specify destination directory.
  --start RMT_DIR, -s RMT_DIR
                        Download only the specified subtree.Configuration fileA sample configuration file is provided with the namedavbackup.json.sample. Rename and edit to suit your needs.The configuration is a json file whose fields are self explanatory, except perhaps thenbackupsfield, explained below. Note that the password is base64 encoded. This makes it obscure, not secure, but the program is meant to be run on a trusted machine:{
    "protocol": "https",
    "server": "nextcloud.samplesite.com",
    "username": "fred",
    "password": "YWJyYWNhZGFicmE=",
    "davstart": "/remote.php/dav/files/fred",
    "nbackups": 4
}The fieldnbackupsis the number of backups copy to keep. When the local directory exists already, it is presumed to contain a previous backup, and it is renamed with an extension .001, if a directory with a .001 extension exists, it is renamed to .002 and so on until the specified number of backups is reached. Directories that would have a higher number than the number of backups are simply deleted.InstallingUse the provided setup.py utility to install:$ python3 setup.py installOr simply usepipto install from the the Pypi site:$ pip install davbackupRunningOnce you have a valid configuration file as described above, just run:$ davbackup --config <configfile> --destdir <localdir>You can use the arguments described above.
Package: davebot
D.A.V.E.Meet DAVE, the screenwriting bot.
DAVE generates formatted screenplays from other screenplays.Installpip install davebotUseStanley moduleGenerate screenplaysDAVE.nlp.Stanleyuses all the modules below (except web scraping) to generate formatted screenplays (PDF and plaintext).Example:fromDAVE.nlp.StanleyimportStanleycharacters=['HAL','DAVE','Stanley Kubrick','Discovery One','Arthur C. Clarke','The Sentinel']source=['2001-A-Space-Odyssey.txt','2012.txt']director=Stanley(source,characters,destination='output',title='2013',author='DAVE')director.direct(length=100)DAVE will create two files in thedestinationdirectory:<title>.pdfand<title>.txt.Scraper moduleScraping ScreenplaysDAVE.scraper.DiscoveryOnescrapes from IMSDB using BeautifulSoupfromDAVE.scraperimportDiscoveryOne,SentinelDiscoveryOne.crawl(path,directory='.',pattern=GENRE):ParseDAVE.scraper.Sentineland parses screenplays into subsections: headings, transitions, actions, parentheticals, and dialogue, to be further prepared into Markov models.Sentinel.parse(*sources,destination='.',write=True)Or, parse recursively usingSentinel.parse_directorySentinel.parse_directory(source='.',destination='parsed_categories',genre='All')`NLP moduleSerialize markov modelsDAVE.nlp.HALgenerates Markov chains from the parsed categories made byDAVE.scraper.Sentineland serializes the models into JSON.To create the Markov models:fromDAVE.nlp.HALimportHALHAL.generate_models(source='parsed_categories',destination='.')InspirationA Space OdysseyInspiration from the name D.A.V.E. came from Stanley Kubrick's masterpiece film2001: A Space Odyssey. The names of this package's modules all reflect characters, creators, texts, and important people involved in A Space Odyssey.SunspringThe original inspiration for this project comes from a short science fiction film:Sunspring.Description (from Ars Technica):In the wake of Google's AI Go victory, filmmaker Oscar Sharp turned to his technologist collaborator Ross Goodwin to build a machine that could write screenplays. They created "Jetson" and fueled him with hundreds of sci-fi TV and movie scripts. Building a team including Thomas Middleditch, star of HBO's Silicon Valley, they gave themselves 48 hours to shoot and edit whatever Jetson decided to write.Click the image above to see the full video.
Package: dave-misc
Medium multiplyA small demo library for a Medium publication about publishing libraries.Installationpip install dave-miscGet startedHow to match some pattern with string:fromregx_entitiesimportRegxEntities# Instantiate a Multiplication objectregx=RegxEntities()#Can added regular custom expressionsregx.add_entity("otp",r"[0-9]{4}",priority=True)# Call the multiply methodresult=regx.get_matchs("the otp is 0204")
Package: daves-utilities
# daves_utilitiesThe MIT License (MIT)
Copyright (c) 2022, David KuchelmeisterPermission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.