Saravanan666/pypi-org huggingface dataset:
Package: huffman-project
huffman-projectA python package to compress data based on character frequencyThis repository contains files for huffman package and test files for it intests/directory.Installation instructionsClone the repo:pip3installhuffman-projectInstallation instructions (dev)Clone the repo:gitclonegit@github.com:M4verickFr/huffman-project.gitThen install the dependencies and install the package:pip3install-rrequirements.txt
pip3install-e.UsageImport huffman package.fromhuffmanimportHuffmanThen use methods compress or comrpessFile.h=Huffman()h.compress("Bonjour!!")h.compressFile(pathFile)For examples of use, see the test folder.Unit Test CaseYou can run Unit Test Case withpython3-munittesttests/HuffmanTestCase.py
python3-munittesttests/TreeTestCase.pyAll tests must be passedLicenseThis project is under the Apache 2.0 License.
Package: huff-puff
Huffman Coding Python PackageThis Python package implements Huffman coding for file compression and decompression.InstallationYou can install the package using pip:pipinstallhuff-puffUsageCompression (huff.py)To compress a file using Huffman coding:fromhuffimportcompresscompressedFile=compress("input.txt")Decompression (huff.py)To decompress a file previously compressed using Huffman coding:fromhuffimportdecompressdecompressedFile=decompress("input_compressed.bin")Huffman Functions (huffman.py)For detailed information about the Huffman coding functions:fromhuffmanimportmake_frequency_dict,make_heap,merge_nodes,make_codes,get_encoded_text,decode_text//UsethesefunctionsasneededwithinyourcodeLicenseThis project is licensed under theMIT License.
Package: hufr
🤗 RedactionsHuggingFace Redactions(hufr) redacts personal identifiable information from text using pretrained language models from the HuggingFace model repository. This packge wraps token classification models to streamline the redaction of personal identifiable information from free text. This project is not associated with the official HuggingFace organization, just a fun side project for this individual contributor.InstallationTo install this package, runpip install hufrUsageSee below for an example snippet to load a specific token classification library from the HuggingFace model zoo:fromhufr.modelsimportTokenClassificationTransformerfromhufr.redactimportredact_textfromtransformers.tokenization_utils_baseimportBatchEncodingmodel_path="dslim/bert-base-NER"model=TokenClassificationTransformer(model=model_path,tokenizer=model_path)text="Hello! My name is Rob"redact_text(text,redaction_map={'PER':'<PERSON>'},model=model)>`"Hello! My name is <PERSON>"`If you don't want to instantiate a model and supply a specific token classification model, then you can simply rely on the repository defaults for a quick and simple redaction:fromhufr.redactimportredact_texttext="Hello! My name is Rob"redact_text(text)To get the predicted entity for each word in the original text:fromhufr.redactimportredact_texttext="Hello! My name is Rob"redact_text(text,return_preds=True)>"Hello! My name is <PERSON>",['O','O','O','O','PER']By default, personal identifiable information is predicted by thedslim/bert-base-NERmodel where entities are mapped to redactions using the following mapping table:'PER':'<PERSON>','MIS':'<OTHER>','ORG':'<ORGANIZATION>','LOC':'<LOCATION>'
Package: hug_authentication_ldap
hug_authentication_ldapEasy to use LDAP based authentication support for hug:importhugimporthug_authentication_ldapauthentication=hug_authentication_ldap.basic('myldap.server.net','uid={user_name},ou=people')@hug.get(requires=authentication)defsay_hello(hug_user):return'Hello{}!'.format(hug_user.name)Or, for general reusable LDAP password verification within hug:importhugimporthug_authentication_ldapldap_check=hug_authentication_ldap.verify('myldap.server.net','uid={user_name},ou=people')@hug.get()defcheck(user_name,password):ifldap_check(user_name,password):returnTruereturnFalseInstalling hug_authentication_ldapInstalling hug_authentication_ldap is as simple as:pip3installhug_authentication_ldap--upgradeIdeally, within a virtual environment.What is hug_authentication_ldap?An extension for hug that provides LDAP based authentication supportThanks and I hope you find hug_authentication_ldap helpful!~Timothy Crosley
Package: hugchat-api
Huggingchat apiYou Star You Win A BILLION$$$(maybe,it's not impossible that it would happen)This is my first pypi project. Experienced some annoying moments, but i managed to do it anywaypipinstallhugchat-api[!NOTE]VERSIONS BEFOREv0.0.1.6ARE DEPRECATED!  PLEASE UPDATE TO THE LASTEST.For documentation, please seeWikiLastest ChangeLast update: 2023-09-30Fix:summarizeapi removed causing endless loopFeat! :( Testing, not sure if it's working 🤨 ) System prompt injectionrm:Open-Assistantis removed fromListBotsFix:Chatflowerror not shown inMessagehence hard to debug from logUpdate: Change multiple type hints toUnionto adapt lower python version#87ScreenshotsNormal ChatWith Search WebTerminal UsageStart uppython-mhugchat_api.terminal_cli-uyour_emailParamsDescriptions-uLogin Email-pUse password or not (optional)-fIgnore the saved cookie and login--debugOpen cli in with debug log type, log file can be found with the name:terminal_cli.logCommandsUse/+commandto execute:CommandsDescriptionsq/exitExit the programlsList all conversationscdcd into the chosen conversationnewCreate a new conversationrmdelete the chosen conversationoldPrint out the conversation's historieswebSwitch 'Search Web' enable optionproProvide prompt forSystemAnything not start with/will be seen as chat message.Example:(None) > /ls
#* Conversations established:
#
#       0. [649471fa525d2d2474973871] - Hello there! How can I help you? Let me know if you need something specific done.
#       1. [64946fb2525d2d247497382c] - Hi there! How can I assist you?

(None) > /cd 0
(647e09ccabd9de3d82d6fba0) > hi
#(user): hi
#(HFBot): ...
(647e09ccabd9de3d82d6fba0) > /web
#WEB_SEARCH is set to `True`
(647e09ccabd9de3d82d6fba0) > hi
# ...(steps about web search)
#(HFBot): ...
Package: hugdatafast
hugdatafastThe elegant integration of huggingface/datasets and fastai, and some handy data transformation for huggingface/datasets.🎓Documentation:https://hugdatafast.readthedocs.io/en/latest/Installpip install hugdatafastFurture PlanI would like to merge this library to fastai and huggingface/datasets respectively. But I may have no time for it. You're welcome to pr this library to the two libraries.The implemenatation ofCombineTransformworks but might be too complexed to extend, hope HuggingFace or someone come up with some great ideas.Currently, it is designed to work with the dataset part of huggingface/datasets, I may also integrate the part of metric or not.Quick Intro
Package: huge
hugeFind huge additions in Git history.Installation$ pip install hugeCommand line options$huge--helpUsage: huge [OPTIONS]Options:--branch TEXT          Which branch to scan. By default it will scan thecurrently active branch.  [default: HEAD]--num-entries INTEGER  How many top entries to show.  [default: 20]--cutoff INTEGER       Cutoff (bytes) below which to ignore entries.[default: 1000000]--help                 Show this message and exit.Example output$ huge

scanning the history ...
100%|████████████████████████████████████████████████████████████████████| 2493/2493 [00:59<00:00, 41.56it/s]


  size (MB)  path                                                    commit
-----------  ------------------------------------------------------  ----------------------------------------
   43.0599   test/dectests/decrimp2/result/decrimp2_CO2H2.out        c375b575a6310f056a4dd07f9327b3d217b0ea4b
   43.0269   test/dectests/decsosrimp2/result/decsosrimp2_CO2H2.out  3bfa9d4d59e7523921cd66244e2d8f2fb488fbd1
   10.7648   test/dectests/large_decrimp2/dens.restart               3bfa9d4d59e7523921cd66244e2d8f2fb488fbd1
   10.7648   LSDALTON/test/dectests/large_decrimp2/dens.restart      c45c4467bf72141bf073fcb584eed0786095c339
   10.7648   test/dectests/large_decrimp2/overlapmatrix              3bfa9d4d59e7523921cd66244e2d8f2fb488fbd1
   10.7648   test/dectests/large_decrimp2/lcm_orbitals.u             3bfa9d4d59e7523921cd66244e2d8f2fb488fbd1
   10.7648   test/dectests/large_decrimp2/fock.restart               3bfa9d4d59e7523921cd66244e2d8f2fb488fbd1
   10.7648   test/dectests/large_decrimp2/cmo_orbitals.u             3bfa9d4d59e7523921cd66244e2d8f2fb488fbd1
   10.7648   LSDALTON/test/dectests/large_decrimp2/overlapmatrix     c45c4467bf72141bf073fcb584eed0786095c339
   10.7648   LSDALTON/test/dectests/large_decrimp2/lcm_orbitals.u    c45c4467bf72141bf073fcb584eed0786095c339
   10.7648   LSDALTON/test/dectests/large_decrimp2/fock.restart      c45c4467bf72141bf073fcb584eed0786095c339
   10.7648   LSDALTON/test/dectests/large_decrimp2/cmo_orbitals.u    c45c4467bf72141bf073fcb584eed0786095c339
    2.53132  src/LSint/lsdalton_dftd_pars.h                          3bfa9d4d59e7523921cd66244e2d8f2fb488fbd1
    2.53132  LSDALTON/LSint/lsdalton_dftd_pars.h                     c45c4467bf72141bf073fcb584eed0786095c339
    1.9964   test/ref/n2_energy_densfitFCK3_old.ref                  3bfa9d4d59e7523921cd66244e2d8f2fb488fbd1
    1.9964   LSDALTON/test/ref/n2_energy_densfitFCK3_old.ref         c45c4467bf72141bf073fcb584eed0786095c339
    1.83527  test/ref/decmp2_geoopt_FO.ref                           3bfa9d4d59e7523921cd66244e2d8f2fb488fbd1
    1.83527  LSDALTON/test/ref/decmp2_geoopt_FO.ref                  c45c4467bf72141bf073fcb584eed0786095c339
    1.80867  src/FraME/FraME.tar.gz                                  35a9ce3cc06da8c635fdef44d3d1689e7946fc60
    1.80866  src/FraME/FraME.tar.gz                                  12651386cc1d7d29876179fd22629e29703a3f0a
Package: hugedict
hugedicthugedict provides a drop-in replacement for dictionary objects that are too big to fit in memory. hugedict's dictionary-like objects implementtyping.Mappingandtyping.MutableMappinginterfaces using key-value databases (e.g., RocksDB) as the underlying storage. Moreover, they are friendly with Python's multiprocessing.DocumentationInstallationFrom PyPI (using pre-built binaries):pipinstallhugedictTo compile the source, run:maturin build -rinside the project directory. You needRust,Maturin, CMake and CLang (to buildRust-RocksDB).FeaturesCreate a mutable mapping backed by RocksDBfromfunctoolsimportpartialfromhugedict.preludeimportRocksDBDict,RocksDBOptions# replace [str, str] for the types of keys and values you want# as well as deser_key, deser_value, ser_valuemapping:MutableMapping[str,str]=RocksDBDict(path=dbpath,# path (str) to db fileoptions=RocksDBOptions(create_if_missing=create_if_missing),# whether to create database if missing, check other optionsdeser_key=partial(str,encoding="utf-8"),# decode the key from memoryviewdeser_value=partial(str,encoding="utf-8"),# decode the value from memoryviewser_value=str.encode,# encode the value to bytesreadonly=False,# open database in read only modesecondary_mode=False,# open database in secondary modesecondary_path=None,# when secondary_mode is True, it's a string pointing to a directory for storing data required to operate in secondary mode)Load huge data from files into RocksDB in parallel:from hugedict.prelude import rocksdb_load. This function creates SST files in parallel, ingests into the db and (optionally) compacts them.Cache a function when doing parallel processingfromhugedict.preludeimportParallelpp=Parallel()@pp.cache_func("/tmp/test.db")defheavy_computing(seconds:float):time.sleep(seconds)returnseconds*2output=pp.map(heavy_computing,[0.5,1,0.7,0.3,0.6],n_processes=3)Create dictionary backed by Sqlite:hugedict.sqlite.SqliteDictChain multiple dictionaries:hugedict.chained_mapping.ChainedMappingCache a dictionary so previously accessed keys are stored in memory:hugedict.cachedict.CacheDictor calledhugedict.types.HugeMapping.cache
Package: hugegraph-python
hugegraph-pythonA Python SDK for Apache HugeGraph, maintained bycvte research data-mining team.This project currently in alpha testing, will open source the code soon.InstallationPyPihttps://pypi.org/project/hugegraph-python/pip3installhugegraph-pythonInstall from sourcerelease soonExamplesfromhugegraph.connectionimportPyHugeGraphclient=PyHugeGraph("127.0.0.1","8080",user="admin",pwd="admin",graph="hugegraph")"""system"""print(client.get_graphinfo())print(client.get_all_graphs())print(client.get_version())print(client.get_graph_config())"""schema"""schema=client.schema()schema.propertyKey("name").asText().ifNotExist().create()schema.propertyKey("birthDate").asText().ifNotExist().create()schema.vertexLabel("Person").properties("name","birthDate").usePrimaryKeyId().primaryKeys("name").ifNotExist().create()schema.vertexLabel("Movie").properties("name").usePrimaryKeyId().primaryKeys("name").ifNotExist().create()schema.edgeLabel("ActedIn").sourceLabel("Person").targetLabel("Movie").ifNotExist().create()print(schema.getVertexLabels())print(schema.getEdgeLabels())print(schema.getRelations())"""graph"""g=client.graph()g.addVertex("Person",{"name":"Al Pacino","birthDate":"1940-04-25"})g.addVertex("Person",{"name":"Robert De Niro","birthDate":"1943-08-17"})g.addVertex("Movie",{"name":"The Godfather"})g.addVertex("Movie",{"name":"The Godfather Part II"})g.addVertex("Movie",{"name":"The Godfather Coda The Death of Michael Corleone"})g.addEdge("ActedIn","12:Al Pacino","13:The Godfather",{})g.addEdge("ActedIn","12:Al Pacino","13:The Godfather Part II",{})g.addEdge("ActedIn","12:Al Pacino","13:The Godfather Coda The Death of Michael Corleone",{})g.addEdge("ActedIn","12:Robert De Niro","13:The Godfather Part II",{})res=g.getVertexById("12:Al Pacino").label# g.removeVertexById("12:Al Pacino")print(res)g.close()"""gremlin"""g=client.gremlin()res=g.exec("g.V().limit(10)")print(res)Any questions contactming@apache.orgTODOdocumentutcimore api
Package: hug_explainable
Hug Explainable===============|PyPI version| |Build Status| |Coverage Status| |License| |Join the chatat https://gitter.im/timothycrosley/hug\_explainable|Provides an on demand context manager that makes it easy to profile andexplain code blocks / paths within hug.Installing hug\_explainable===========================Installing hug\_explainable is as simple as:.. code:: bashpip3 install hug_explainable --upgradeIdeally, within a virtual environment.Why hug\_explainable?=====================Provides an on demand context manager that makes it easy to profile andexplain code blocks / paths within hug.--------------Thanks and I hope you find hug\_explainable helpful!~Timothy Crosley.. |PyPI version| image:: https://badge.fury.io/py/hug_explainable.svg:target: http://badge.fury.io/py/hug_explainable.. |Build Status| image:: https://travis-ci.org/timothycrosley/hug_explainable.svg?branch=master:target: https://travis-ci.org/timothycrosley/hug_explainable.. |Coverage Status| image:: https://coveralls.io/repos/timothycrosley/hug_explainable/badge.svg?branch=master&service=github:target: https://coveralls.io/github/timothycrosley/hug_explainable?branch=master.. |License| image:: https://img.shields.io/github/license/mashape/apistatus.svg:target: https://pypi.python.org/pypi/hug_explainable/.. |Join the chat at https://gitter.im/timothycrosley/hug\_explainable| image:: https://badges.gitter.im/Join%20Chat.svg:target: https://gitter.im/timothycrosley/hug_explainable?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge&utm_content=badge
Package: huggingbutt
HuggingButtClient library to download and publish reinforcement learning environments/agents on the huggingbutt.com hubInstallationThe base dependency mlagents-envs has been update to version 1.0.0. However, this version may rise an errorIssue 5996(on Windows 10). So I specified mlagents-envs==0.30.0, NumPy==1.21.2 in this project. I did not found this issue on MacOS.Create a new python environment using anaconda/miniconda. Only Python 3.9!!condacreate-nhbpython==3.9.18activate the new python environment.condaactivatehbinstall huggingbutt from pypipipinstallhuggingbutt==0.0.5or from source codegitclone-bv0.0.5https://github.com/huggingbutt/huggingbutt.gitcdhuggingbutt
python-mpipinstall.If there is no error message printed during the installation, congratulations, you have successfully installed this package. Next, you need to apply an access token from the official websitehttps://huggingbutt.com.Register an account and login, just do as shown in the image below.Click new token button generate a new token. This access token is mainly used to restrict the download times of each user, as the server cost is relatively high.Congratulations, you now have an access token!Just put the generated token in the task code and you're gooooood to go.Here is a simple training code:fromhuggingbuttimportEnv,Agent,set_access_tokenif__name__=='__main__':set_access_token('YOUR_TOKEN')env=Env.get("huggingbutt/juggle",'mac',startup_args=['--time_scale','10'])agent=Agent(env=env,algorithm='PPO',policy='MlpPolicy',batch_size=64)agent.learn(total_timesteps=10000)agent.save()env.close()Inference:fromhuggingbuttimportEnv,Agent,set_access_tokenif__name__=='__main__':set_access_token('YOUR_TOKEN')env=Env.get("huggingbutt/juggle",'mac',startup_args=['--time_scale','1'])agent=Agent.get(15,env)obs=env.reset()foriinrange(100):act,_status_=agent.predict(obs)obs,reward,done,info=env.step(act)ifdone:obs=env.reset()env.close()todoSupport more types learning environment, such as native game wrapped by python, pygame, class gym...Develop a framework, user can customize the observation, action and reward of the environment developed under this framework. I hope this makes it easier for everyone to iterate the environment's agent.There are still many ideas, I will add them later when I think about them...
Package: huggingface
Note:VERSION needs to be formatted following the MAJOR.MINOR.PATCH convention
(we need to follow this convention to be able to retrieve versioned scripts)Simple check list for release from AllenNLP repo:https://github.com/allenai/allennlp/blob/master/setup.pyTo create the package for pypi.Change the version in __init__.py, setup.py as well as docs/source/conf.py.Commit these changes with the message: “Release: VERSION”Add a tag in git to mark the release: “git tag VERSION -m’Adds tag VERSION for pypi’ ”
Push the tag to git: git push –tags origin masterBuild both the sources and the wheel. Do not change anything in setup.py between
creating the wheel and the source distribution (obviously).For the wheel, run: “python setup.py bdist_wheel” in the top level directory.
(this will build a wheel for the python version you use to build it).For the sources, run: “python setup.py sdist”
You should now have a /dist directory with both .whl and .tar.gz source versions.Then change the SCRIPTS_VERSION back to to “master” in __init__.py (but don’t commit this change)Check that everything looks correct by uploading the package to the pypi test server:twine upload dist/* -r pypitest
(pypi suggest using twine as other methods upload files via plaintext.)
You may have to specify the repository url, use the following command then:
twine upload dist/* -r pypitest –repository-url=https://test.pypi.org/legacy/Check that you can install it in a virtualenv by running:
pip install -ihttps://testpypi.python.org/pypidatasetsUpload the final version to actual pypi:
twine upload dist/* -r pypiCopy the release notes from RELEASE.md to the tag in github once everything is looking hunky-dory.Update the documentation commit in .circleci/deploy.sh for the accurate documentation to be displayed
Update the version mapping in docs/source/_static/js/custom.jsUpdate README.md to redirect to correct documentation.
Package: huggingface-hub-storj-patch
Monkey patch for HuggingFace Hub to download Git-LFS blobs from StorjThis patch aims to demonstrate the transfer speed that can be achieved withhuggingface_hubPython library when utilizing the power of theStorj Decentralized Cloud Storage.HuggingFace Hub stores all large files in Git-LFS.When thehuggingface_hubPython library requests to download such a file, the download request is redirected to the Git-LFS CDN hosted atcdn-lfs.huggingface.co.This monkey patch modifies thehuggingface_hublibrary to redirect Git-LFS downloads to the Storj Linksharing service hosted atlink.storjshare.io.PrerequisitesThe Git-LFS blobs for the respective AI model must be replicated to a Storj bucket and shared it with theStorj Linksharing Service.We have already replicated the Git-FLS blobs of theStarCodermodel to a Storj bucket and shared it:https://link.storjshare.io/raw/juzlwaj7ovnst5gtkv2km3rkriha/lfs-huggingfaceIf you want to use another AI model, you need to use your own Storj bucket and then configure the patch to use it. SeeConfigurationfor more details.InstallationFirst, install the patch module:pipinstallhuggingface-hub-storj-patchThen add the following import statement at the top, before any other import, of your Python script:importhuggingface_hub_storj_patchNow you can run your script. If the patch is applied successfully, you will see it printing the URLs from which thehuggingface_hublibrary is downloading.ConfigurationThese environment variables can configure the behavior of the patch.HF_HUB_NO_STORJIf set totrue, downloads won't be redirected to the Storj Linksharing Service as if the patch is not applied.HF_HUB_STORJ_PARALLELISMConfigures how many parallel download connections are open to the Storj Linksharing Service. The default value is16.HF_HUB_STORJ_URL_PREFIXConfigures the URL to the shared Storj bucket that replicates the Git-LFS blobs of the AI model. The default value is the bucket that replicates the StarCoder model:https://link.storjshare.io/raw/juzlwaj7ovnst5gtkv2km3rkriha/lfs-huggingface
Package: huggingfaceinference
huggingfaceinferenceSimple inference pipelines using hugging transformers library and finetuned tiny models. Will be highly useful on scenarios where we need to optimize storage and memory cost with a compensation in accuracy.Installationpip install huggingfaceinferenceImplementationTinyGramA simple grammatical and spelling resolver using huggingface transformers. The Dataset (around 50k) is retrieved from kaggle and corrupted using random substution of letters in words for finetuning. The base model used wasgoogle/t5-efficient-tiny-nl32as it is highly compact and efficient(<250 MB).(was modeled on fp32 to reduce drop in efficiency).fromhuggingfaceinference.pipelineimportTinyGramtg=TinyGram()tg.gramcorrector("What do you think I shold be doing",n=3)""" Output: ['What do you think I need to be doing','what do you think I should be doing','what do you think I will be doing']"""# The above pipeline is built by finetuing "google/t5-efficient-tiny-nl32" model on sentences which are corrupted by random noising.Knowledge GraphA simple tool to create knowledge graphs in NLP using a pre-trained model modelled on custom dataset created using spaCy library.fromhuggingfaceinference.pipelineimportKnowledgeGraphkg=KnowledgeGraph()kg.get_graph(text)""" [{'tend': 'O'},{'##ul': 'SRC'},{'##kar': 'SRC'},{'plays': 'SRC'},{'for': 'REL'},{'india': 'REL'}]"""# The above pipeline is built by finetuing "google/t5-efficient-tiny-nl32" model on sentences which are corrupted by random noising.