Saravanan666/pypi-org huggingface dataset:
Package: hadoop-protoseq
ProtoSeqProtoSeq is a python library that allows working with sequences of specified protobuf messages. The sequence of protobuf messages is stored as a sequence of pairs:size of message in bytes – 4 bytes (int);protobuf message bytes.This sequence format is a flexible storage format similar toHadoop SequenceFilethat allows to process files with multiprocessing (e.g. with Hadoop) if extra index is provided.This repository is an example how to work with binary data using Hadoop Streaming.Quick StartInstall package with pip:pip install protoseq.This is an example program that reads file in protoseq format, saves it to temprorary file and prints protobufs in human readable format.#!/usr/bin/env python3# -*- coding: utf-8 -*-importsysimportaddress_pb2fromtempfileimportTemporaryFilefromprotoseq.readerimportProtobufSequenceReaderfromprotoseq.writerimportProtobufSequenceWriterwithTemporaryFile(mode='wb+')asf_out:reader=ProtobufSequenceReader(address_pb2.Address,sys.stdin.buffer)writer=ProtobufSequenceWriter(f_out)forrecordinreader:writer.write(record)f_out.seek(0)reader=ProtobufSequenceReader(address_pb2.Address,f_out)forrecordinreader:print(record)Hererecordis an instance ofaddress_pb2.Address.This program needs anaddress_pb2.pyfile –generated sourcesfor python.address_pb2.pycan be changed to your own protobuf.Hadoop Streaming ExampleHere is an example of Map-Reduce program (map-only) that copies a file in HDFS.There are some dependencies we need to run the MR job:$treemapreduce
mapreduce
├──hadoop-streaming-protoseq.jar
├──streaming
│├──address_pb2.py
│└──mapper.py
└──streaming-env-py37.tar.gz1directory,5filesYou can get all these files just runningmake allcommand insideexample directory:hadoop-streaming-protoseq.jar– ProtoSeq library for Hadoop Streaming;streaming-env-py37.tar.gz– environment with python3 and installed ProtoSeq package;streaming/mapper.py– mapper stage for job;streaming/address_pb2.py– generated protobuf sources for python.You supposed to havecondaandconda packto preparestreaming-env-py37.tar.gzfor streaming.To run MR program we need to execute command:${HADOOP}jar${HADOOP_STREAMING}\-Dmapred.job.name="Example: Copy proto file"\-Dmapred.reduce.tasks=0\-Dstream.map.input='rawbytes'\-Dstream.map.input.writer.class='org.apache.hadoop.streaming.io.RawBytesOutputReader'\-Dstream.map.output='rawbytes'\-Dstream.map.output.reader.class='org.apache.hadoop.streaming.io.RawBytesOutputReader'\-files"streaming/mapper.py"\-libjars"hadoop-streaming-protoseq.jar"\-archives"streaming-env-py37.tar.gz#env"\-inputformat"com.github.vbugaevskii.hadoop.streaming.protobuf.ProtobufSequenceInputFormat"\-outputformat"com.github.vbugaevskii.hadoop.streaming.protobuf.ProtobufSequenceOutputFormat"\-mapper"env/bin/python streaming/mapper.py"\-input"/tmp/v.bugaevskii/addresses.protoseq"\-output"/tmp/v.bugaevskii/protoseq_copy"
Package: hadoop-ranger-audit
hadoop_ranger_audit===================Generates a stand alone web based report for Hadoop Apache Ranger policies.Performs reverse lookups against Active Directory to match users with Ranger resources that have beenprovision with AD Groups.Currently provides a report the following policies:HDFSHIVEKNOXYARNInstallation from GIT source:-----------------------------Clone the repo::$ git clone https://github.com/jackscodemonkey/hadoop_ranger_auditCreate a new virtual environment::$ virtualenv -p python3 hadoop_ranger_audit_envActivate the new environment::$ source hadoop_ranger_audit_env/bin/activateYour command prompt should now have the environment prefix showing::$ (hadoop_ranger_audit_env):Quickly install hadoop_ranger_audit into your new environment via `setuptools`_Change to the directory where you downloaded this package and run::$ (hadoop_ranger_audit_env)python setup.py installUsage:------hadoop_ranger_audit.py::Switch                  | Description                       | Required======================================================================--ranger_url            | Ranger API URL                    | Yes--ranger_user           | Ranger API Username               | Yes--ranger_password       | Ranger API Password               | Yes--ad_controller         | Active Directory Server Hostname  | Yes--ad_user               | Read Only AD user - basic account | Yes--ad_password           | AD user password                  | Yes--ad_search_base        | DN root to start search from      | Yes--output_file           | Output HTML file                  | Yes--cluster_name          | Name of cluster for the report    | Yes-h / --help             | Print command help                | NoRequirements^^^^^^^^^^^^.. include:: requirements.txtCompatibility-------------hadoop_ranger_audit is written and tested in Python 3.6.Licence-------Licensed under the MIT license.Authors-------`hadoop_ranger_audit` was written by `Marcus Robb <marcus.robb@initworx.com>`_... _`setuptools`: http://pypi.python.org/pypi/setuptools
Package: hadoop-yarn-rest-api
This is Python Library for Hadoop YARN REST api.This project follows the [Hadoop YARN REST Api](https://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/ResourceManagerRest.html) document.We have implemented the following APIs.Cluster Information APICluster Metrics APICluster Scheduler APICluster Applications APICluster Application Statistics APICluster Application APICluster Application Attempts APICluster Nodes APICluster Node APIUsageInstall it:pip install hadoop-yarn-rest-apiUsage:from hadoop.yarn import Yarn
import xml.etree.ElementTree as ET
import json
import xml.dom.minidom

def main():
    yarn = Yarn("http://localhost:8088", 'json')

    response_obj = yarn.cluster_information()
    response_obj = yarn.cluster_metrics()
    response_obj = yarn.cluster_scheduler()
    response_obj = yarn.cluster_applications()
    response_obj = yarn.cluster_applications({"limit":100})
    response_obj = yarn.cluster_appstatistics()
    response_obj = yarn.cluster_appstatistics({"states":"accepted,running,finished","applicationTypes":"mapreduce"})
    response_obj = yarn.cluster_application("job_id")
    response_obj = yarn.cluster_application_attempts("job_id")
    response_obj = yarn.cluster_nodes()
    response_obj = yarn.cluster_nodes({"states":"RUNNING"})  # NEW, RUNNING, UNHEALTHY, DECOMMISSIONING, DECOMMISSIONED, LOST, REBOOTED, SHUTDOWN
    response_obj = yarn.cluster_node("node_id")

    if yarn.response_type == 'json':
        print(json.dumps(response_obj, indent=4, sort_keys=True))
    elif yarn.response_type == 'xml':
        print(xml.dom.minidom.parseString(ET.tostring(response_obj)).toprettyxml())
    else:
        print(response_obj)

if __name__ == '__main__':
    main()
Package: hadopy
Hadopy - Easy parallel map-reduce command line toolIf you want to map reduce parallel but hadoop is overkill,
with Hadopy you can run map reduce in python.InstallingTo get Hadopy, either install from PyPi:$pipinstallhadopyor clone this github project and install:$pipinstall.UsageHadopy was programmed with ease-of-use in mind. To run it use one of the following command:Linux / MacOS$catexample.txt|hadopy--mapper"python mapper.py"--reducer"python reducer.py"Windows$typeexample.txt|hadopy--mapper"python mapper.py"--reducer"python reducer.py"For more information use$hadopy--help
Package: hadrian
[![Build Status](https://secure.travis-ci.org/dstegelman/hadrian.png?branch=develop)](http://travis-ci.org/dstegelman/hadrian)What?!Hadrian - A collection of plugable Django apps and utilities that I use in my projects.  I find these apps and scripts useful, either use it or don’t.Utilties/AppsISO Country ChoicesGravatar library/template tagsStarter fabric fileUnique_slugify utiltiyFile field widget that displays the currently uploaded fileMiddleware to require login on every pageCustom tagging application that uses slugs (Deprecated, use django-taggit instead)Installationpip install hadrianWant to use the gravatar template tags?Add hadrian.contrib.gravatar to installed appsload gravatar_tagsCall get_gravatar or show_gravatar ( Get is just the image URL, while show provides the img tag wrapper )Use it or don’t.Hadrian (Latin: Publius Aelius Trajanus Hadrianus Augustus 24 January 76 – 10 July 138), commonly known as Hadrian and after his apotheosis Divus Hadrianus, was Roman Emperor from 117 to 138. He is best known for building Hadrian’s Wall, which marked the northern limit of Roman territory in Britain. In Rome, he re-built the Pantheon and constructed the Temple of Venus and Roma. In addition to being emperor, Hadrian was a humanist and was philhellene in all his tastes. He was the third of the so-called Five Good Emperors.
Package: hadro
HadroDB is a database storage engine forOpteryx.LicenseStatusHadroDB is in alpha. Alpha means different things to different people, to us, being alpha means:Interfaces may be significantly changedThe data file format is unstableExpected functionality is missingThings that worked yesterday, don't work todayThe results of the system may be unreliableAs such, we really don't recommend using HadroDB anywhere where your data matters.
Package: hadrodb
HadroDBHadroDB is a Document Store, being developed as a data store forOpteryx.HadroDB is based onCaskDBwhich is based onBitcask.LicenseStatusHadroDB is in alpha. Alpha means different things to different people, to us, being alpha means:Interfaces may be significantly changedThe data file format is unstableExpected functionality is missingThings that worked yesterday, don't work todayThe results of the system may be unreliableAs such, we really don't recommend using HadroDB anywhere where your data matters.
Package: hadroid
=============Hadroid Bot=============Installation============Hadroid is on PyPI:.. code-block:: console$ pip install hadroidConfiguration=============Copy the default configuration file:.. code-block:: console$ cp hadroid/config.py myconfig.pyEdit the ``myconfig.py`` and set the ``GITTER_PERSONAL_ACCESS_TOKEN`` to yourGitter API token.Point to your config with environmental variable:.. code-block:: console$ export HADROID_CONFIG=myconfig.pyUsage=====The bot is run using the ``hadroid`` command,which allows for launching bot threads that can listenon the Gitter channels for incoming commands or execute periodic tasks.Run the bot on Gitter~~~~~~~~~~~~~~~~~~~~Bot will listen for messages and reply back on a single channel.This can be either an orgazation channel, repository or a private one-on-onechat with a user. The bot needs to be already in the room, or at least oneprivate message needs to be send to the bot.First, run the main bot "server" application:.. code-block:: console$ hadroid runKeep this session alive an in another session have the bot join some channels:.. code-block:: console$ hadroid --helpFor CRON commands, run a CRON daemon in a separate shell:.. code-block:: console$ hadroid start stream <your_github_username>$ hadroid start cron <your_github_username>Deployment==========To deploy Hadroid you can use the provided `fabile.py<http://docs.fabfile.org/en/latest/>`_ (tested on commonly used VPS vanillainstances of Ubuntu 16.04 and Debian 8) in the following manner:.. code-block:: console$ # fabtools is a helper library for Fabric$ pip2 install --user fabric fabtools$ # "bootstrap" has to run once for each machine you plan to deploy$ fab -U root -H my-server.xyz bootstrap$ # You should run "deploy" everytime you change the config as well$ fab -U root -H my-server.xyz deploy:config_path=/path/to/your/config.py$ # To manage the bot you can use the "start/stop/restart" commands:$ fab -U root -H my-server.xyz startTake a look and modify the fabfile if your remote machine doesn't play well.
Package: haesleinhuepf-pyqode.core
This is a fork of the original [PyQode](https://github.com/pyQode/pyqode.core)
deployed to pypi by [haesleinhuepf](https://github.com/haesleinhuepf)
under the name [haesleinhuepf-pyqode.core](https://pypistats.org/packages/haesleinhuepf-pyqode.core).Bugfix maintenance onlyAboutpyqode.coreis the core framework of thepyQodeproject.It contains the base classes and a set of extensions (modes/panels/managers)
needed to develop a specialised code editor.It also provides a basic generic code editor that you can use as a fallback
when there is no specialised editor for a given language.Issue trackerWikiAPI referenceContributingChangelogScreenshotsRequirementspyqode.core depends on the following libraries:Python 2 (>=2.7) or Python 3 (>= 3.2)PyQt5 or PyQt4 or PySidepygmentspyqode.qtfutureqtawesome (optional)InstallationYou need to install PyQt or PySide by yourself. Note that you should preferPyQt5 on Mac OSX(retina screen support, better integration).Then you can install pyqode.core usingpip:$ pip install pyqode.core --upgradeTestingpyqode.core has a test suite and measure its coverage.To run the tests, just runpython setup.py testTo measure coverage, run:python setup.py test -a "--cov pyqode"To check for PEP8 warnings, install pytest-pep8 and run:python setup.py test -a "--pep8 -m pep8"To run a single test, use-a"--test_file_path.py::test_function", e.g.:python setup.py test -a "-- test/test_api/test_code_edit.py::test_set_plain_text"Testing MatrixWe test the following combinations on Travis-CI:PyQt4PyQt5GNU/Linux - Python 2.7yesnoGNU/Linux - Python 3.4yesyes
Package: haesleinhuepf-pyqode.python
This is a fork of the original [PyQode](https://github.com/pyQode/pyqode.python)
containing a modification from [znqi fixing jedi compatibility](https://github.com/znqi/pyqode.python)
and deployed to pypi by [haesleinhuepf](https://github.com/haesleinhuepf)
under the name [haesleinhuepf-pyqode.python](https://pypistats.org/packages/haesleinhuepf-pyqode.python).Bugfix maintenance onlyAboutpyqode.pythonaddspythonsupport topyQode(code completion,
calltips, …).Issue trackerWikiAPI referenceContributingChangelogScreenshotsFeatures:calltips mode (usingJedi)code completion provider (usingJedi)code folding modeauto indent modeon the fly code checkers (frosted (fork of PyFlakes), PEP8)a customisable python specific syntax highlightera pre-configured QPythonCodeEdit (with the corresponding Qt Designer plugin)LicensepyQode is licensed under theMIT license.Requirementspyqode.python depends on the following libraries:python 2.7 or python 3 (>= 3.2)pyqode.corejedipep8frosteddocutilsInstallation$ pip install pyqode.python --upgradeTestingpyqode.core has a test suite and measure its coverage.To run the tests, just runpython setup.py testTo measure coverage, run:python setup.py test -a "--cov pyqode"To check for PEP8 warnings, install pytest-pep8 and run:python setup.py test -a "--pep8 -m pep8"To run a single test, use-a"--test_file_path.py::test_function", e.g.:python setup.py test -a "-- test/test_api/test_code_edit.py::test_set_plain_text"Testing MatrixWe test the following combinations on Travis-CI:PyQt4PyQt5GNU/Linux - Python 2.7yesnoGNU/Linux - Python 3.4yesyes
Package: hafapiserver
haf plugin web serverThe plugin web server of hafHow to get it?by pip tooltsbx-mw#pipinstallhafapiserverby srctsbx-mw#gitclonehttps://github.com/hautof/haf-plugin-webserver./tsbx-mw#pythonsetup.pyinstallusageusing as haf paramspython-mhafrun-ws=Trueusing as haf config{"run":{"web_server":{"host":"","port":"","run":true}}
Package: hafdh-nadhar
hafdh-nadharPackage Python that blur human representations from an imageInstallationpdminstall--devBuild and publishpdmpublish--password<API_TOKEN>UsageFrom Python codefromhafdh_nadhar.hafdhimporthafdh_imghafdh_img(img_path="/path/to/img.jpg")From CLIhafdh"/path/to/img.jpg"ExempleBefore:After:
Package: haferml
Hafer MLThe HomemAde FramEwoRk for Machine Learning (HAFER ML) is a minimal and unambitious framework for your machine learning projects, with reproducibility in mind.Hafer is oat in German.Installpip install hafermlThis will leave out many dependencies. To install all,pip install "haferml[all]"The extras options:all: everythingaws: required if one needs AWSdocs: required to build the docsThe IdeaThe Hafer ML frameworks is designed to be a minimal framework for data scientists. Hafer ML uses config files and saves the artifacts at every step for reproducibility.on whimsical
Package: hafez
Divan Hafez Poems & OmenThis library gives you access to the Hafez Poems.
To get more information about this poet, you can checkhttps://en.wikipedia.org/wiki/Hafez.Poems can be accessed as following:By the number of poem,By searching a query,Random poem (known as Omen orFalin Persian)InstallationThis package can be installed:usingpip:pipinstallhafezusingMakefileon a cloned/forked repo:makeinstallusingpipon a cloned/forked repo:pipinstall-e.--upgrade--upgrade-strategyonly-if-neededMethodshafez.total_poems()->intIt returns `int`, the total number of poems available in the package.hafez.get_poem(poem_id: int)->dictIt returns the poem in a dictionary format (see Poem Data Structure)poem_id: int: It is a number between1andhafez.total_poems().hafez.omen()orhafez.fal()->dictIt returns a random poem in a dictionary format (see Poem Data Structure)hafez.search(qeury: str)->listIt returns a list of poems in a dictionary format (see Poem Data Structure)query: str: It is a string to search within the verses of the Divan Hafezhafez.download_all_audio(force: boolean = False)->intIt downloads all the audio files.force: boolean = False: If it is set toFalse, it only downloads the file if not previously downloaded.
If set toTrue, it will force to re-download and replace the existing file.hafez.download_audio(poem: int)->intIt downloads the audio file of the given `poem`. It returns `1`, once the downloaded is completed.hafez.get_audio(poem: int=1, download: boolean = True)->strIt returns the absolute path of the audio file related to the given  `poem`.poem: int=1: It is the poem number.download: boolean = True: Ifdownload=True, it downloads the audio file if not existing.Poem's Data Structure{"id":1,"poem":[],"interpretation":"","mp3":"https://..."}Example# it returns the total number of poems# returns: inttotal_number_of_poems=hafez.total_poems()# get a poem by ID# returns: dictpoem_5=hafez.get_poem(5)# get a random poem - omen (fal)# returns: dictomen=hafez.omen()# same as: hafez.fal()# search within the verses of poems# returns: list[dict]search_result=hafez.search("حافظ")DatabaseThe poems of Divan Hafez are extracted from an open-source database as below:Source:https://github.com/mahmoud-eskandari/HafezFaalDatabaseVocal AudiosThe vocal audio files are done by Ms. Modares Zadeh, and it is available below:Source:https://avayemastan.deklame.net/hafez/DemoYou can access the demo version deployed on Streamlit server at:https://divan-hafez.streamlit.app/Developer(s)Kaveh Bakhtiyari -Website|Medium|LinkedIn|GitHubContributionFeel free to join the open-source community and contribute to this repository.
Package: hafezpoem
Hafez PoemGive you random poem of hafezYou can use this for factor of restaurants and other stuffUSAGE:fromhafezpoemimportget_random_falfal=get_random_fal()print(fal)RESULT:fal="Ù…Ø·Ø±Ø¨ Ø¹Ø´Ù‚ Ø¹Ø¬Ø¨ Ø³Ø§Ø² Ùˆ Ù†ÙˆØ§ÙŠÙŠ Ø¯Ø§Ø±Ø¯\nÙ†Ù‚Ø´ Ù‡Ø± Ù†ØºÙ…Ù‡ Ú©Ù‡ Ø²Ø¯ Ø±Ø§Ù‡ Ø¨Ù‡ Ø¬Ø§ÙŠÙŠ Ø¯Ø§Ø±Ø¯"Licensehafezpoem is published under theMIT license.
Package: ha-franklin
ha-franklinTable of ContentsBackgroundUsageConfigurationRunning the MonitorHome AssistantBackgroundI wanted a non-toy test example of using theha-mqtt-discoverablemodule.ha-franklinwill monitor CUPSD print queues, and present a binary sensor to Home Assistant over MQTT showing whether the printer is printing.I use this to turn the smart switch for the HP 4050N in the basement on and off so that by the time I walk downstairs from my office after printing something, Home Assistant has turned on the power to the printer and the job has at least started printing.UsageConfigurationCreate a config file (yaml) with a list of dictionaries in it. Each dictionary should have the following keys:mqtt_server: DNS name or a raw IP.mqtt_user: the_mqtt_usermqtt_password: the_mqtt_passwordname: Franklin@cupsdunique_id: printername-cupsdcupsd_queue_name: Queue_name_on_cupsd_servercupsd_server: cupsd.example.comcheck_interval: 10The easiest way to create a configuration file is to start by copyingconfig/config-example.yamland editing it to fit.Running the MonitorI recommend usingdocker,nerdctlorpodmanto run the tooling in a container.docker run -v "$(pwd)/config":/config --rm unixorn/ha-franklin ha-cupsd-monitor-queues --settings-file /config/config.yamlHome AssistantThe container will create a set of MQTT topics using Home Assistant's MQTT discoverability protocol so that your print queue's printing status shows up in Home Assistant.I set up two automations - one to turn on the peanut plug my HP 4050N is plugged into when jobs appear in the cupsd queue when the sensor turns to on, and a second that turns it off once the sensor switches back to off for ten minutes.I give it ten minutes for a couple of reasons:First, because CUPSD will report the queue as done printing when all the postscript has been spooled to the printer. Depending on the complexity of the print job, it may take a minute or two to print the last few pages of the job, even though cupsd considers it complete. If the power gets turned off too soon, you can lose the last page or two of the job, and more annoyingly, cause a printer jam if the power cuts off while a page is moving through the paper path.Secondly, because although I print rarely, when I do, I typically print several things within a few minutes and I'd prefer to not toggle the printer on and off more than is strictly necessary.
Package: hafsqlpublish
haf plugin sql publishThe plugin sql publish of haf, now only support mysqlHow to get it?by pip tooltsbx-mw#pipinstallhafsqlpublishby srctsbx-mw#gitclonehttps://github.com/hautof/haf-plugin-sqlpublish./tsbx-mw#pythonsetup.pyinstallusagenoticefirst of all , you must create an new database by the haf-plugin-sqlpublish/haf_publish.sql script file.using as haf paramspython-mhafrun--sql-publish=True--sql-publish-db=192.168.0.101:6606@root:test123@haf_publishusing as haf config"sql_publish":{"id":1,"sql_name":"upload","publish":true,"host":"192.168.1.2","port":3306,"username":"root","password":"test123","database":"haf_publish","protocol":"mysql"}design
Package: hafweb
haf-webmanagerhaf web managerhafhttps://github.com/tsbxmw/hafhow to runpython3 -m hafweb run -ss=root:root@localhost:3306/haf_publishnow support mysql as an api serverapisidurldescriptionothers1/api/v1/mainget all tests in the db2/api/v1/main/todayget all today test3/api/v1/main?date_time=2019-04-08get the 20190408's tests4/api/v1/suite?main_id=123get the main_id's suite5/api/v1/summary?suite_id=123get the summary of id6/api/v1/case?suite_id=123get the case of suite7/api/v1/expect?id=123get the expect of id8/api/v1/ids?id=123get the ids of id9/api/v1/request?id=123get the request of id10/api/v1/response?id=123get the response of id11/api/v1/sqlinfo?id=123get the sqlinfo of id12/api/v1/sqlinfo/checklist?id=123get the sqlinfo checklist of id13/api/v1/sqlinfo/config?id=123get the sqlinfo config of id14/api/v1/sqlinfo/script?id=123get the sqlinfo script of id15/api/v1/case/detail?id=123get the detail of case idManager of HAF based on vue & python flaskcreate/edit caserun/pasue/stop casegenerate report of case resultdashboard of resultsresults analysisimport case (from json, xlsx, yml, py)export case (to json, xlsx, yml, py)export report