Saravanan666/pypi-org huggingface dataset:
Package: haxor-game
haxor-game-pythonHow good are you at haxoring?InfoQuickly ported haxor game from C++ to Python in 20 minutes.C++ version:https://github.com/adavila0703/haxor-gameInspired by:https://github.com/UnrealCourse2019/TripleXCourse:https://www.udemy.com/course/unrealcourse/Speed code videohttps://youtu.be/YT_DG3B8PYgInstallpip install haxor-gameRunpython -m haxor
Package: haxor-news
haxor-newsTo view the latestREADME,docs, andcodevisit the GitHub
repo:https://github.com/donnemartin/haxor-newsTo submit bugs or feature requests, visit the issue tracker:https://github.com/donnemartin/haxor-news/issuesChangelog0.4.2 (2017-04-08)Bug Fixes#94- Update
toprompt-toolkit1.0.0, which includes a number of performance
improvements (especially noticeable on Windows) and bug fixes.0.4.1 (2016-05-30)Bug Fixes#62- Fixprompt-toolkitv1.0.0 hanging while autocompleting the hn view
command. Revert back toprompt-toolkitv0.52. This bug only
happens on Windows.0.4.0 (2016-05-30)Features#52- Addexitandquitcommands, which can be used instead ofctrl-d.#53- Allow
clicking of urls in some terminals to open in a browser.Bug Fixes#36- Fix
crash caused by Unicode comments on Windows.#59- Update
toprompt-toolkit1.0.0, which includes a number of performance
improvements (especially noticeable on Windows) and bug fixes.Fix some comments and docstrings.Updates#48,#50-
Update latest monthly hiring post ids.#56-
Update packaging dependencies based on semantic versioning.FixConfigdocstrings.UpdateREADME:Update introAdd Hacker News discussion ofhaxor-newsUpdate comments discussion and examplesUpdate TODOFix urls based on redirectsRemove buggy Codecov graphAdd note about OS X 10.11 pip installation issueAdd Gemnasium dependencies management.Update links inCONTRIBUTING.0.3.1 (2016-04-10)Initial release.
Package: hayaku
Hayaku - lightweight Python utility to generate Dockerfile and distribute tiny python apps without including sources[![Build Status](https://travis-ci.org/msoedov/hayaku.svg?branch=master)](https://travis-ci.org/msoedov/hayaku)## Instalation```shellpip install hayaku```## Why?Disclaimer this repo being made just for fun in certain circumstances and have non opinionated approach for doing things.The purpose of this is to wrap your existing tooling written in python into containers and then ship/distribute it without complicatedworkflow around git repo's.## Approach0. Autodetect of required pip packages which needs to be installed1. Encode py sources with base64/zip/bzip2. Put encoded data into of generated `Dockerfile` ENV (According to POSIX we can put up to 256Mb into env variable)```shellENV PY_LIB "{body}"```3. Extract it during docker build```shellRUN python -c "import os,base64;b=os.getenv('PY_LIB');b=base64.b64decode(b);print(b.decode('utf-8'))" | tee app.py```4. So the final generated Dockerfile will looks like this```shellhayaku app.pyFROM python:3.6MAINTAINER Erlich BachmanWORKDIR /appRUN pip install requestsENV PY_LIB "aW1wb3J0IGJhc2U2NAppbXBvcnQgb3MKZnJvbSB...."RUN python -c "import os,base64;b=os.getenv('PY_LIB');b=base64.b64decode(b);print(b.decode('utf-8'))" | tee app.pyCMD python app.py```which you can ship as a file, build and push image to private docker registry, etc...## Seriosly ?It's up to you. It worked for my limited usecase and I hope it might be helpfull for somebody else. As it said in disclamer: this repo being made just for fun in certain circumstances and have non opinionated approach for doing such things.
Package: hayasaka
hayasakaHayasaka saves the rendered HTML as an image.Use casesSave the slides (an HTML file) usingreveal.jsas an image⚠️To Run hayasaka, you currently need to have theFirefoxbrowser installed (Google Chrome will be supported in the future).Installation$pipinstallhayasakaUsage📸Generate an image from URL$hayasaka-clihttps://ftnext.github.io/2022_slides/pyconjp/python_and_star.htmlpython_and_star.png📸Generate an image from a local HTML file path$hayasaka-cli../2022_slides/build/revealjs/pyconjp/python_and_star.htmlpython_and_star.pngDevelopment environment$gitclonegit@github.com:ftnext/hayasaka.git
$cdhayasaka

$python3.11-mvenvvenv--upgrade-deps
$sourcevenv/bin/activate(venv)$pipinstall-e'.[dev]'(venv)$tasktest
Package: hayde
haydeHaydeInstallation$python3-mpipinstall--user--upgradehaydeUsage$hayde-h
usage:Hayde[-h]--config-fileCONFIG_FILE[--version]Hayde

optionalarguments:-h,--helpshowthishelpmessageandexit--config-fileCONFIG_FILE--versionshowprogram'sversionnumberandexit
Package: hayes2009
hayes2009hayes2009is a small Python 3.7+ module that exposes phonological features
from:Hayes, B. 2009. Introductory Phonology. John Wiley & Sons.Seehayes2009.pyfor information about how this data has been
processed.It exposes two tables: anenglish-only table and a largeruniversaltable.
As far as I can tell, phones that occur in English are given the same
specification in both tables but both tables are provided for completeness.willfix/wontfixPlease read before filing an issue on the issue trackerIwillcorrect the following bugs:Typographical errors in the Hayes' tablesIwill notwork on or accept PRs for any the following "feature requests":Additional feature specifications from other sourcesAlternative names for the features"Fancy" (i.e., tuple-based) feature specificationsReverse look-up (i.e., finding phones that match a certain feature
specification)GotchasPhones are encoded inNFC.Hayes uses the traditional "single-storey"ɡrather thangfor the
voiced velar stop.Hayes includes many contour segments, but these are indicated with the tie
bars (e.g.,t͡s).Hayes uses SHOUTYCASE for the major place features likeLABIAL.LicenseThis library is distributed under an Apache 2.0 license. Please seeLICENSE.txtfor detalis.Authorhayes2009was written byKyle Gorman.
Package: hayloft
HayloftUI tool for LLM frameworks to make easy prompt/completion tracking, store and comparison of different sessions.InstallationUsageLlamaIndexBabyAGIInstallationInstall package with pippip install hayloftUsageStart hayloft serverhayloft startTrace logs of your script onhttp://localhost:7000LlamaIndexInstallllama-index, createexample.pyfile as below. Putexamplesfolder from llama_index repo near the file.importosos.environ["OPENAI_API_KEY"]='YOUR_OPENAI_API_KEY'fromllama_indeximportSimpleDirectoryReader,VectorStoreIndexfromhayloft.llama_indeximportgrab_logsgrab_logs()documents=SimpleDirectoryReader("examples/paul_graham_essay/data").load_data()index=VectorStoreIndex.from_documents(documents)query_engine=index.as_query_engine()query_engine.query("What did the author do growing up?")Or you can start live sessions from hayloft ui, just modify code like hereimportosos.environ["OPENAI_API_KEY"]='YOUR_OPENAI_API_KEY'fromllama_indeximportSimpleDirectoryReader,VectorStoreIndexfromhayloft.llama_indeximportlistendefagent(query:str):documents=SimpleDirectoryReader("examples/paul_graham_essay/data").load_data()index=VectorStoreIndex.from_documents(documents)query_engine=index.as_query_engine()query_engine.query(query)listen(agent)Start this scriptpython example.pyBabyAGICloneBabyAGI forkrepo, setup virtual env and install all dependenciesgit clone git@github.com:eturchenkov/babyagi-hayloft.git && cd babyagi-hayloft
pip install -r requirements.txtAdjust config in .env file and startbabyagi.pyscriptpython babyagi.py
Package: haystack-bm25
Rank-BM25: A two line search engineA collection of algorithms for querying a set of documents and returning the ones most relevant to the query. The most common use case for these algorithms is, as you might have guessed, to create search engines.So far the algorithms that have been implemented are:Okapi BM25BM25LBM25+BM25-AdptBM25TThese algorithms were taken fromthis paper, which gives a nice overview of each method, and also benchmarks them against each other. A nice inclusion is that they compare different kinds of preprocessing like stemming vs no-stemming, stopword removal or not, etc. Great read if you're new to the topic.InstallationThe easiest way to install this package is throughpip, usingpipinstallrank_bm25If you want to be sure you're getting the newest version, you can install it directly from github withpipinstallgit+ssh://git@github.com/dorianbrown/rank_bm25.gitUsageFor this example we'll be using theBM25Okapialgorithm, but the others are used in pretty much the same way.InitalizingFirst thing to do is create an instance of the BM25 class, which reads in a corpus of text and does some indexing on it:fromrank_bm25importBM25Okapicorpus=["Hello there good man!","It is quite windy in London","How is the weather today?"]tokenized_corpus=[doc.split(" ")fordocincorpus]bm25=BM25Okapi(tokenized_corpus)# <rank_bm25.BM25Okapi at 0x1047881d0>Note that this package doesn't do any text preprocessing. If you want to do things like lowercasing, stopword removal, stemming, etc, you need to do it yourself.The only requirements is that the class receives a list of lists of strings, which are the document tokens.Ranking of documentsNow that we've created our document indexes, we can give it queries and see which documents are the most relevant:query="windy London"tokenized_query=query.split(" ")doc_scores=bm25.get_scores(tokenized_query)# array([0.        , 0.93729472, 0.        ])Good to note that we also need to tokenize our query, and apply the same preprocessing steps we did to the documents in order to have an apples-to-apples comparisonInstead of getting the document scores, you can also just retrieve the best documents withbm25.get_top_n(tokenized_query,corpus,n=1)# ['It is quite windy in London']And that's pretty much it!
Package: haystack-client
Haystack bindings for Python OpenTracing APIThis is Haystack's client library for Python that implementsOpenTracingFurther information can be found onopentracing.ioUsing this librarySee examples in /examples directory. See opentracingusagefor additional information.It is important to consider the architecture of the application. In order for the tracer to manage spans properly, an appropriateScopeManagerimplementation must be chosen.  In most environments, the defaultThreadLocalScopeManagerwill work just fine. In asynchronous frameworks, theContextVarsScopeManageris a better choice.First initialize the tracer at the application level by supplying a service name and span recorderimportopentracingfromhaystackimportHaystackAgentRecorderfromhaystackimportHaystackTracertracer=HaystackTracer("a_service",HaystackAgentRecorder())opentracing.set_global_tracer(tracer)Starting a span can be done as a managed resource usingstart_active_span()withopentracing.tracer.start_active_span("span-name")asscope:do_stuff()or finish the span on your own terms withspan=opentracing.tracer.start_span("span-name")do_stuff()span.finish()Note:If there is a Scope, it will act as the parent to any newly started Spanunless the programmer passesignore_active_span=Trueatstart_span()/start_active_span()time or specified parent context explicitly usingchildOf=parent_contextCustom propagation headersIf necessary, default propagation headers can be replaced with custom ones by specifying custom propagator options. Register the new propagator with the tracer once configured.prop_opts=PropagatorOpts("X-Trace-ID","X-Span-ID","X-Parent-Span","X-baggage-")opentracing.tracer.register_propagator(opentracing.Format.HTTP_HEADERS,TextPropagator(prop_opts))LoggingAll modules define their logger vialogging.getLogger(__name__)So in order to define specific logging format or level for this library usegetLogger('haystack')or configure the
root logger.How to configure build environmentCreate a python3 virtual environment, activate it and thenmake bootstrapRunning the example codemake exampleHow to Release this libraryCreate a new release in github specifying a semver compliant tag greater than the current release version.
Package: haystack-duckduckgo
haystack-duckduckgoUse theDuckDuckGosearch engine with your haystack pipeline or agents.Table of ContentsInstallationUsageLicenseInstallationpip install haystack-duckduckgoUsageThis package allows you to use theDuckDuckGosearch api, which does not require an API key and can be used free of charge.Use it in HaystacksWebSearchcomponent:fromhaystack_duckduckgoimportDuckDuckGoAPIfromhaystack.nodes.search_engineimportWebSearch#Build the DuckDuckGo providersearch_engine_provider=DuckDuckGoAPI()#Initialize the `WebSearch` node with the providerws=WebSearch(api_key=None,search_engine_provider=search_engine_provider)#Use the `WebSearch` noderesults=ws.run(query="python programming language")[0]print(results)Use it in a HaystackWebRetrieverfromhaystack_duckduckgoimportDuckDuckGoAPIfromhaystack.nodes.retriever.webimportWebRetrieverfromhaystackimportPipeline#Build the DuckDuckGo providersearch_engine_provider=DuckDuckGoAPI()#Build the pipelinewr=WebRetriever(api_key=None,search_engine_provider=search_engine_provider)p=Pipeline()p.add_node(wr,name="web_retriever",inputs=["Query"])#Run the pipelineresults=p.run(query="python")print(results)Licensehaystack-duckduckgois distributed under the terms of theMITlicense.
Package: haystack-entailment-checker
Haystack Entailment CheckerCustom node for theHaystack NLP framework.
Using aNatural Language Inferencemodel, it checks whether a lists of Documents/passages entails, contradicts or is neutral with respect to a given statement.Live Demo: Fact Checking 🎸 Rocks!How it worksThe node takes a list of Documents (commonly returned by aRetriever) and a statement as input.Using a Natural Language Inference model, the text entailment between each text passage/Document (premise) and the statement (hypothesis) is computed. For every text passage, we get 3 scores (summing to 1): entailment, contradiction and neutral.The text entailment scores are aggregated using a weighted average. The weight is the relevance score of each passage returned by the Retriever, if availaible. It expresses the similarity between the text passage and the statement.Now we have a summary score, so it is possible to tell if the passages confirm, are neutral or disprove the user statement.empirical consideration: if in the first N passages (N<K), there is strong evidence of entailment/contradiction (partial aggregate scores >threshold), it is better not to consider (K-N) less relevant documents.Installationpipinstallhaystack-entailment-checkerUsageBasic examplefromhaystackimportDocumentfromhaystack_entailment_checkerimportEntailmentCheckerec=EntailmentChecker(model_name_or_path="microsoft/deberta-v2-xlarge-mnli",use_gpu=False,entailment_contradiction_threshold=0.5)doc=Document("My cat is lazy")print(ec.run("My cat is very active",[doc]))# ({'documents': [...],# 'aggregate_entailment_info': {'contradiction': 1.0, 'neutral': 0.0, 'entailment': 0.0}}, ...)Fact-checking pipeline (Retriever + EntailmentChecker)fromhaystackimportDocument,Pipelinefromhaystack.nodesimportBM25Retrieverfromhaystack.document_storesimportInMemoryDocumentStorefromhaystack_entailment_checkerimportEntailmentChecker# INDEXING# the knowledge base can consist of many documentsdocs=[...]ds=InMemoryDocumentStore(use_bm25=True)ds.write_documents(docs)# QUERYINGretriever=BM25Retriever(document_store=ds)ec=EntailmentChecker()pipe=Pipeline()pipe.add_node(component=retriever,name="Retriever",inputs=["Query"])pipe.add_node(component=ec,name="EntailmentChecker",inputs=["Retriever"])pipe.run(query="YOUR STATEMENT TO CHECK")Acknowledgements 🙏Special thanks goes to@davidberenstein1957, who contributed to the original implementation of this node, in theFact Checking 🎸 Rocks!project.
Package: haystack-human-tool
Haystack Human In Loop ToolHuman in Loop tool forhaystackAgents.  Uses human guidance to refine queries and improve performance.InstallationPython pip:python3 -m pip install haystack-human-tool. This package will attempt to install the dependencies (farm-haystack>=1.15.0, redis, haystack-memory, bleach)Python pip (skip dependency installation: Usepython3 -m pip install haystack-human-tool --no-depsUsing git:pip install git+https://github.com/rolandtannous/haystack-human-tool.git@main#egg=haystack-human-toolUsageTo use the human in the loop tool in your agent, you need three core components:haystack-memory: This is the memory library for haystack agents. It provides access to working and sensory memories.HumanInLoopNode: This node adds human in the loop to the agent as a tool. It allows the agent to seek guidance from the human user in order to refine the user query.sensory-memory: This is an in-memory implementation that mimics a human's brief sensory memory, lasting only for the duration of one interaction.fromhaystack.agentsimportAgent,Toolfromhaystack.nodesimportPromptNodefromhaystack_memory.memoryimportMemoryRecallNodefromhaystack_memory.utilsimportMemoryUtilsfromhaystack_human_tool.humaninloopimportHumanInLoopNodefromhaystack_human_tool.prompt_templatesimportagent_template# Initialize the memory and the memory tool so the agent can retrieve the memorysensory_memory=[]working_memory=[]memory_node=MemoryRecallNode(memory=working_memory)memory_tool=Tool(name="Memory",pipeline_or_node=memory_node,description="Your memory. Always access this tool first to remember what you have learned.")human_node=HumanInLoopNode(sensory_memory=sensory_memory)human_tool=Tool(name="Human",pipeline_or_node=human_node,description="Access this tool to refine your query. Ask the human to rephrase the question or to explain what or who the human is referring to. Use the human's answer to rephrase your query for use as input to other tools")prompt_node=PromptNode(model_name_or_path="text-davinci-003",api_key="<YOUR_OPENAI_KEY>",max_length=1024,stop_words=["Observation:"])memory_agent=Agent(prompt_node=prompt_node,prompt_template=agent_template)memory_agent.add_tool(memory_tool)memory_agent.add_tool(human_tool)# Initialize the utils to save the query and the answers to the memorymemory_utils=MemoryUtils(working_memory=working_memory,sensory_memory=sensory_memory,agent=memory_agent)result=memory_utils.chat("<Your Question>")ExampleAn Example can be found in theexamples/folder. It contains the usage for all memory types.
To open the example in colab, click on the following links:Human in Tool powered agent:
Package: haystack-hybrid-embedding
haystack-hybrid-embeddingRecently, Pinecone announced support forSparse-dense embeddings, allowing for hybrid vector search (both semantic and keyword search).haystackis a fantastic NLP framework that does not yet support hybrid vectors forRetrievers.This little library helps temporarily bridge the gap!Installation$pipinstallhaystack-hybrid-embeddingUsagefromhaystack_hybrid_embeddingimportSpladeEmbeddingEncoderfromhaystack_hybrid_embedding.pineconeimportPineconeHybridDocumentStore,SparseDenseRetrieverdocument_store=PineconeHybridDocumentStore(...)retriever=SparseDenseRetriever(sparse_encoder=SpladeEmbeddingEncoder(),alpha=0.8,...)ReplacingEmbeddingRetrieverSimply replace your imports ofPineconeDocumentStoreandEmbeddingRetriever/MultihopEmbeddingRetriever.1,2c1,3< from haystack.document_stores.pinecone import PineconeDocumentStore< from haystack.nodes import EmbeddingRetriever, MultihopEmbeddingRetriever---> from haystack_hybrid_embedding import SpladeEmbeddingEncoder> from haystack_hybrid_embedding.pinecone import PineconeHybridDocumentStore, SparseDenseRetriever, SparseDenseMultihopRetriever>4c5< document_store = PineconeDocumentStore(...)---> document_store = PineconeHybridDocumentStore(...)6c7< retriever = EmbeddingRetriever(---> retriever = SparseDenseRetriever(7a9,10>     sparse_encoder=SpladeEmbeddingEncoder(),>     alpha=0.8,ConfigThere are only two additional parameters exposed onSparseDenseRetrieveroverEmbeddingRetriever:sparse_encoderembeds both queries and documents into sparse vectorsalphacontrols the weighting between the sparse and dense vectors (0is all sparse, and1is all dense)
Package: haystack-lemmatize-node
LemmatizationLemmatization is a text pre-processing technique used in natural language processing (NLP) models to break a word down to its root meaning to identify similarities. For example, a lemmatization algorithm would reduce the word better to its root word, or lemme, good.This node can be placed within a pipeline to lemmatize documents returned by a Retriever, prior to adding them as context to a prompt (for a PromptNode or similar).
The process of lemmatizing the document content can potentially reduce the amount of tokens used by up to 30%, without drastically affecting the meaning of the document.Before Lemmatization:After Lemmatization:InstallationClone the repo to a directory, change to that directory, then perform apip install '.'.  This will install the package to your Python libraries.UsageInclude it in your pipeline - example as follows:importloggingimportrefromdatasetsimportload_datasetfromhaystack.document_storesimportInMemoryDocumentStorefromhaystack.nodesimportPromptNode,PromptTemplate,AnswerParser,BM25Retrieverfromhaystack.pipelinesimportPipelinefromhaystack_lemmatize_nodeimportLemmatizeDocumentslogging.basicConfig(format="%(levelname)s-%(name)s-%(message)s",level=logging.WARNING)logging.getLogger("haystack").setLevel(logging.INFO)document_store=InMemoryDocumentStore(use_bm25=True)dataset=load_dataset("bilgeyucel/seven-wonders",split="train")document_store.write_documents(dataset)retriever=BM25Retriever(document_store=document_store,top_k=2)lfqa_prompt=PromptTemplate(name="lfqa",prompt_text="Given the context please answer the question using your own words. Generate a comprehensive, summarized answer. If the information is not included in the provided context, reply with 'Provided documents didn't contain the necessary information to provide the answer'\n\nContext:{documents}\n\nQuestion:{query}\n\nAnswer:",output_parser=AnswerParser(),)prompt_node=PromptNode(model_name_or_path="text-davinci-003",default_prompt_template=lfqa_prompt,max_length=500,api_key="sk-OPENAIKEY",)lemmatize=LemmatizeDocuments()# you can pass the `base_lang=XX` argument here too, where XX is a language as listed here: https://pypi.org/project/simplemma/pipe=Pipeline()pipe.add_node(component=retriever,name="Retriever",inputs=["Query"])pipe.add_node(component=lemmatize,name="Lemmatize",inputs=["Retriever"])pipe.add_node(component=prompt_node,name="prompt_node",inputs=["Lemmatize"])query="What does the Rhodes Statue look like?"output=pipe.run(query)print(output['answers'][0].answer)CaveatsSometimes lemmatization can be slow for large document content, but in the world of AI where we can potentially wait 30+ seconds for an LLM to respond (hello GPT-4), what's a couple more seconds?
Package: haystack_queryparser
haystack-queryparser====================Converts arbitrarily complicated user entered query strings to a haystack query object.###Usage```pythonfrom haystack_queryparser import ParseSQ```Also provides or_parser and and_parser which can be directly used with a query```pythonparser = ParseSQ()sq_object = parser.parse(query)```takes a `AND` or `OR` operator to use as default optionally.###InputInput should be a string.This the query.###OutputOutput is a `SQ(haystack.query.SQ)` object.This can be passed to `SearchQuerySet.filter` and the	query will be applied###TestTo run the test you need to be in the django environment.So you can do something like this:```$ python manage.py shell>>> import haystack_queryparser.tests as test>>> tests.main()test_operators (modules.haystack_queryparser2.tests.SimpleTest) ... oktest_parse (modules.haystack_queryparser2.tests.SimpleTest) ... oktest_parse_with_new_default (modules.haystack_queryparser2.tests.SimpleTest) ... ok----------------------------------------------------------------------Ran 3 tests in 0.004sOK```
Package: haystack-redis
A Whoosh storage engine using redis for persistence. A HaystackSearchBackendsubclass is also provided. Normally theSTORAGEkey could
just be set but Haystack 2.0.0alpha is only aware offileandrambackends.This is especially useful for small sites hosted on Heroku, which does not allow
writing to local disk. TheREDISTOGO_URLenvironment variable is read,
falling back to the localhost default port.UsageConfigure your Haystack connections insettings.py:import tempfile
HAYSTACK_CONNECTIONS = {
    'default': {
        'ENGINE': 'haystack_redis.RedisEngine',
        'PATH': tempfile.gettempdir(),
    },
}