Saravanan666/pypi-org huggingface dataset:
Package: semantic-segmentation-augmentations
semantic_segmentation_augmentationsData augmentation is a regularisation technique that generates new
training samples from the original dataset by applying colour or
geometric transformations. Although this technique has been applied in
other computer vision fields, such as image classification or object
detection, the application of this technique in semantic segmentation is
not yet widespread.This library groups some data augmentation techniques for semantic
augmentation, such asCutOutorCutMix.InstallTo install the library, run:pipinstallsemantic_segmentation_augmentationsHow it worksThis data augmentation techniques are defined asfastaicallbacks.
Given this fact, you need to train your models usingfastai’s API to
use them.As these techniques modifies the input image changing some pixels
regions for something else, those callbacks have been defined as the
union of two subcomponents: theHoleMakerTechnique,
that defines how to make the hole (how to select the region to replace)
and theHolesFilling,
that defines how to fill the region defined below (the way to fill the
region gives the name to the technique used).TheHoleMakerTechniquecan be replaced in order to change the behavior of the selection of the
region to replace. Doing so, you can use a CutOut technique that select
the region randomly or based on the information bounded in that region.You can also define your custom techniques defining how to fill a hole.
You just need to extend theHolesFillingclass and define thebefore_bacth(remember: we are usingcallbacksfromfastai) abstract method. You will have two methods to simplify
the process: themake_holefunction, that uses the selectedHoleMakerTechniqueto make the hole and returns two slices (the region boundaries) and thefill_holefunction, that fills the hole with something.How to use itIn order to use these techniques, you just need to define theLearnerfromfastaiwith theCallbacksthat represent the techniques that
you want to use.For example, if you want to create anU-netLeanerwithresnet18backbone and use the CutOut technique withHoleMakerRandomas region selection techinique (which is the default one), you just need
to import it:fromsemantic_segmentation_augmentations.holesfillingimportCutOutRandomAnd add it to your learner:learner=unet_learner(dls,resnet18,cbs=CutOutRandom(hole_maker=HoleMakerRandom()))If you want to change the default behaviour of the CutOutRandom
technique (i. e. to select the region with aHoleMakerAttentiontechnique), you just need to define the technique as follows:learner=unet_learner(dls,resnet18,cbs=CutOutRandom(hole_maker=HoleMakerAttention(attention_threshold=0.25,hole_size=(150,150))))Finally, you can apply geometrical augments to the selected regions
(using theAlbumentationsaugments). Maybe this augmnets are not
useful with theCutOuttechniques, but with theCutMixones. In
order to do so, you just need to implement aRegionModifierwith a list with the augments to apply:learner=unet_learner(dls,resnet18,cbs=CutMixRandom(holes_num=1,modifier=RegionModifier(A.Compose([A.Blur(),A.GaussNoise()])),hole_maker=HoleMakerRandom((250,250)),p=1))
Package: semantic-similarity-cli
semantic-similarity-cliSummary:semantic-similarity-cli is a command-line tool leveraging NLP models to compute semantic similarity directly from your terminal. All computations are done locally.How It WorksOn first use, the tool automatically downloads and installs the Instructor embedding model. This model is then used to compute the semantic similarity of text inputs.InstallationInstall using pip:pipinstallsemantic-similarity-cliUsageCompare the similarity of input text to a given phrase:semsim"Lorem ipsum"Find the most similar commits to a specified sentence:gitlog--oneline-n100|semsim"refactor rest controllers"|sort-n-r|headFilter commits based on a minimum similarity score (e.g., 0.6):gitlog--oneline-n100|semsim"refactor rest controllers"|awk'$1 >= 0.6'LicenseMIT
Package: semantics-nlu
SemanticsA Python library for modeling the semantics of natural language.IntroductionSemanticsis a Python library for representing, storing, and manipulating the
semantic content of natural language, in a format designed to closely mirror
the structure and behavior of natural language.Semanticscan be used as a
knowledge graph, but it is capable of much more.PatternsOne of the core features ofSemanticsis its pattern matching capabilities. A
natural language utterance can be modeled as a pattern, which is then used to
update or query the knowledge graph. Patterns can even be attached to the graph
and triggered by later updates, potentially supporting automated reasoning
capabilities.In contrast to the syntax of predicates in propositional logic,Semanticspatterns
are designed to closely mirror the structure of the natural language utterances they
represent. This means that the output of a natural language parser can be mapped with
minimal effort toSemanticspatterns. A pattern is itself a graph, essentially
comprising a detailed grammar tree of the utterance it represents. Searching the
knowledge graph is simply a matter of aligning the structure of the pattern with
a subgraph of the knowledge graph.EvidenceAnother core feature ofSemanticsis its use of accumulated evidence to determine
the truth/falsehood of statements. This is in contrast to more rigid knowledge graph
designs which are brittle in the face of conflicting or contradictory assertions. Many
knowledge graphs are all-or-nothing, following the assertions made by their users
without question, ignoring the possibility of mistakes or uncertainty.Semanticsis
built from the ground up with the assumption thatallknowledge is subject to error,
revision, and uncertainty.More to ComeSemanticsis currently very much a work in progress. There is a lot to be done
before the first production-ready release.
Package: semantic-split
Semantic-SplitA Python library to chunk/group your text based on semantic similarity - ideal for pre-processing data for Language Models or Vector Databases. LeveragesSentenceTransformersandspaCy.Why?Better Context:Providing more relevant context to your prompts enhances the LLM's performance (arXiv:2005.14165[cs.CL]). Semantic-Split groups related sentences together, ensuring your prompts have relevant context.Improved Results:Short, precise prompts often yield the best results from LLMs (arXiv:2004.04906[cs.CL]). By grouping semantically similar sentences, Semantic-Split helps you craft such efficient prompts.Cost Savings:LLMs like GPT-4 charge costs per token and have a token limit (e.g., 8K tokens). With Semantic-Split, you can make your prompts shorter and more meaningful, leading to potential cost savings.Real world example:Imagine you're building an application where users ask questions about articles:A. We want to only add parts in the artcile that are relevant to our query (for better results).B. We want to be able to query the article quickly (pre-processing).We want to pre-process the article - so each query is fast (point B). So we split it into semantic chunks usingsemantic-splitand store it in aVector DBas embeddings.Each time the user asks something we calculate the embedding for their question and find the top 3 similar chunks in our Vector DB.We add those 3 chunks to our pompt, to get better results for our user's questions.As you can see, in part1, which involves semantic sentence splitting (grouping), is crucial. If we don't split or group the sentences semantically, we risk losing essential information. This can diminish the effectiveness of our Vector DB in identifying the most suitable chunks. Consequently, we may end up with poorer context for our prompts, negatively impacting the quality of our responses.InstallTo use most of the functionality you will need to install some pre-requisistsSpacy sm dataset:python -m spacy download en_core_web_smpoetry installSee examplesExamplesSentence Split by Semantic Similarityfrom semantic_split import SimilarSentenceSplitter, SentenceTransformersSimilarity, SpacySentenceSplitter

    text = """
      I dogs are amazing.
      Cats must be the easiest pets around.
      Robots are advanced now with AI.
      Flying in space can only be done by Artificial intelligence."""

    model = SentenceTransformersSimilarity()
    sentence_splitter = SpacySentenceSplitter()
    splitter = SimilarSentenceSplitter(model, sentence_splitter)
    res = splitter.split(text)Result:[["I dogs are amazing.", "Cats must be the easiest pets around."],["Robots are advanced now with AI.", "Flying in space can only be done by Artificial intelligence."]]Testspoetry run pytest
Package: semantic-synth
Semantic SynthSo you've decided to build out a RAG (Retrieval Augmented Generation) tool. Great! Now you want to build a vector search index for a critical production task.Before you push into prodcution though, you'd like to be able to test how well the search index itself is working.We have also released a 520k sample dataset for you to run tests. Access at:Huggingface HubCurrent methods for testingHaving reviewed multiple different platforms that work on semantic search correctness, the primary idea is to collect some passage from the dataset, and generate questions on it using an LLM, then get the LLM to answer it, and generate different types of metrics, such as faithfulness, correctness, etc.This is an expensive operation owing to the multiple LLM calls, not to mention would not be conducive for very large scale data testing or continous monitoring.How does this package work?Given any text, we generate keywords using YAKE library. This effectively makes the test self supervised, without need for expensive LLM calls for synthetic generationPhilosophySemantic search mainly works on finding different latent meanings between queries and vectors in a search index. Therefore, it only makes sense that it should be highly effective at finding passages that contain key phrases in a documentHowever, there are multiple steps of possible loss. First is the chunking strategy. Then comes your vector embedding model. Post which comes the capability of the vector index you are using (these indices are often based on approximation algorithms and can be quite lossy)For academic purposes, premade datasets are good enough, but how do you run these estimates on your own data? That is where Semantic Synth comes inif a vector search index works well, it will be able to find phrases in your passages effectively. If not, maybe there's more to look at, such as your chunking strategy, or your embedding model itselfWho is this package for?This package is specifically for testing the retrieval capacity of your vector search index, for testing statistical metrics such as precision, recall, F1 scores, etc.Currently we support the generation of synthetic search terms for your content so that you can perform search and calculate accuracy metrics. We are working on adding a full fledged testing suite.Why did we build this?This package was built to perform research on how different chunking strategies affect vector search accuracy.UsageWARNING: Please note that this is an alpha release and is only suitable for testing, not for productionInstallationpipinstallsemantic-synthCodefromsemantic_synth.datagenimportKeywordDatasetGeneratortext="""<Insert text here>"""gen=KeywordDatasetGenerator()#To get single text responseprint(gen.generate(content=text))#For dataset as dataframecontent=['<Insert text1 here>','<Insert text2 here>,'<Insert text3 here>']print(gen.generate_as_df(content=content))
Package: semantic-text-similarity
semantic-text-similarityan easy-to-use interface to fine-tuned BERT models for computing semantic similarity. that's it.This project contains an interface to fine-tuned, BERT-based semantic text similarity models. It modifiespytorch-transformersby abstracting away all the research benchmarking code for ease of real-world applicability.ModelDatasetDev. CorrelationWeb STS BERTSTS-B0.893Clinical STS BERTMED-STS0.854InstallationInstall with pip:pip install semantic-text-similarityor directly:pip install git+https://github.com/AndriyMulyar/semantic-text-similarityUseMaps batches of sentence pairs to real-valued scores in the range [0,5]fromsemantic_text_similarity.modelsimportWebBertSimilarityfromsemantic_text_similarity.modelsimportClinicalBertSimilarityweb_model=WebBertSimilarity(device='cpu',batch_size=10)#defaults to GPU predictionclinical_model=ClinicalBertSimilarity(device='cuda',batch_size=10)#defaults to GPU predictionweb_model.predict([("She won an olympic gold medal","The women is an olympic champion")])Moreexamples.NotesYou will need a GPU to apply these models if you would like any hint of speed in your predictions.Model downloads are cached in~/.cache/torch/semantic_text_similarity/. Try clearing this folder if you have issues.AcknowledgementClinical models in this project were submitted to the 2019 N2C2 Shared Task Track 1.
Implementation and model training in this project was supported by funding from the Mark Dredze Lab at Johns Hopkins University.
Package: semantic-tree
Semantic TreeThis package provide a semanticTree command, which generates a visualization of the semantic neighborhood of a work in a Gensim Word2vec vector space.InstallationThis package requires thatGourceis installed locally. On Ubuntu, it can be apt installed.$aptinstallgourceSemanticTree itself can be installed from PyPI withpip.pip install -U semantic_treeNoteThis project has been set up using PyScaffold 3.2.3. For details and usage
information on PyScaffold seehttps://pyscaffold.org/.
Package: semantic-version-check
SEMANTIC-VERSION-CHECKERCheck if a ‘version’ is a valid Semantic Version.Code:https://github.com/boromir674/semantic-version-checkDocs:https://semantic-version-check.readthedocs.io/en/master/PyPI:https://pypi.org/project/semantic-version-check/CI:https://github.com/boromir674/semantic-version-check/actions/Featuressemantic_version_checkpython packageCLI that can be usefull for quick checks, a script or in a CI pipelineSemantic Version format check, using Regular ExpressionsTested against multipleplatformsandpythonversionsDevelopmentHere are some useful notes related to doing development on this project.Test Suite, usingpytest, located intestsdirParallel Executionof Unit Tests, on multiple cpu’sDocumentation Pages, hosted onreadthedocsserver, located indocsdirAutomation, usingtox, driven by singletox.inifileCode CoveragemeasuringBuild Command, using thebuildpython packagePypi Deploy Command, supporting upload to bothpypi.organdtest.pypi.orgserversType Check Command, usingmypyLintCheckandApplycommands, usingisortandblackCI Pipeline, running onGithub Actions, defined in.github/Job Matrix, spanning differentplatform’s andpython version’sPlatforms:ubuntu-latest,macos-latestPython Interpreters:3.6,3.7,3.8,3.9,3.10Parallel Jobexecution, generated from thematrix, that runs theTest SuitePrerequisitesYou need to havePythoninstalled.QuickstartUsingpipis the approved way for installingsemantic_version_check.python3-mpipinstallsemantic_version_checkOne Use Case for the semantic_version_check is to invoke its cli, through a consoleand do SemVer check on a single input string.Open a console and run:check-semantic-version1.0.0echo$?echo"Exit code is 0 meaning operation succeeded"check-semantic-version1.3echo$?echo"Exit code is 1, meaning operation failed"LicenseGNU Affero General Public License v3.0LicenseFree software: GNU Affero General Public License v3.0
Package: semantic-versioning
semantic_versioningAnother semantic versioning package, but better! It follows semantic versioning described inPEP440.InstallationUse the package managerpipto installsemantic_versioning.pipinstallsemantic_versioningUsageUsevalidatefunction to check if versions in strings are correctly formatted.fromsemantic_versioningimportSemanticVersion# use to validate (returns True)assertSemanticVersion.validate("3.4.5-dev1")Useparsefunction to load a SemanticVersion object from a string.fromsemantic_versioningimportSemanticVersionSemanticVersion.parse("1.2.4")# SemanticVersion(major=1, minor=2, patch=3)SemanticVersion.parse("1.2a3.dev6")# SemanticVersion(major=1, minor=2, pre_release_type="a", pre_release=3, dev_release=6)SemanticVersion.parse("1-5-6",separator="-")# SemanticVersion(major=1, minor=5, patch=3)SemanticVersion.parse("1.0.1.2.5")# SemanticVersion(major=1, minor=0, patch=1, versions=[2,5])The SemanticVersion class is comparable and will correctly take pre/dev/post releases into account.fromsemantic_versioningimportSemanticVersionSemanticVersion.parse("1.2.3")<SemanticVersion.parse("1.2.4")# TrueSemanticVersion.parse("1.2.3")<SemanticVersion.parse("1.2")# TrueSemanticVersion.parse("1.2.dev0")<SemanticVersion.parse("1.2.dev1")# TrueSemanticVersion.parse("1.2.dev0")<SemanticVersion.parse("1.2")# TrueSemanticVersion.parse("1.2a1")<SemanticVersion.parse("1.2b0")# TrueSemanticVersion.parse("1.2a1")<SemanticVersion.parse("1.2.post2")# TrueContributingPull requests are welcome. For major changes, please open an issue first
to discuss what you would like to change.Please make sure to update tests as appropriate.LicenseMIT
Package: semantix-genai-inference
Semantix GenAI InferenceA python client library to help you interact with the Semantix GenAI Inference API.InstallationIf you're using pip, just install it from the latest release:$ pip install semantix-genai-inferenceElse if you want to run local, clone this repository and install it with poetry:$ poetry build
$ poetry installUsageTo use it:First, make sure you have a valid API key. You can get one atSemantix Gen AI HubSet an environment variable with your api secret:$ export SEMANTIX_API_SECRET=<YOUR_API_SECRET>
$ semantix-ai --helpConfiguring with semantix.yamlBefore using the ModelClient, you need to configure the library with asemantix.yamlfile. This file should be placed in the same directory where your application is executed. Thesemantix.yamlfile should contain the necessary API keys and other configuration options for the models you want to use.Here's an example of asemantix.yamlfile:providers:semantixHub:serverId:"YOURINFERENCESERVERIDHERE"version:"v0"apiSecret:"YOURSEMANTIXGENAIHUBAPITOKEN"cohere:apiKey:"YOURCOHEREAPIKEY"generate:model:"command"version:"v1"Replace the placeholders with your actual API keys and other configuration options.Using ModelClientTheModelClientclass allows you to interact with different models. To create a model client, you need to specify the type of model you want to use. The available options are "alpaca", "llama2", and "cohere".Here's an example of how to create a model client and generate text using the Alpaca model:fromsemantix_genai_inferenceimportModelClient# Create an Alpaca model clientclient=ModelClient.create("alpaca")# Generate text using the Alpaca modelprompt="Once upon a time"generated_text=client.generate(prompt)print(generated_text)You can replace "alpaca" with "llama2" or "cohere" to use the Llama2 or Cohere models, respectively.DEV - Publish to pypi$ poetry config pypi-token.pypi <YOUR_PYPI_TOKEN>
$ poetry build
$ poetry publishDEV - Bump version$ poetry version patch | minor | major | premajor | preminor | prepatch | prereleaseSee more atPoetry version command docsDEV - Commit message semanticsSee atConventional Commits
Package: semantix-genai-serve
Semantix GenAI ServeSemantix GenAI Serve is a library designed for users of Semantix GenAI Hub to create their own servers running AI models. This library provides an easy-to-use interface for serving AI models using the scalable infrastructure of Semantix GenAI Hub.FeaturesEasy integration with Hugging Face Transformers modelsSupport for serving Seq2Seq modelsCustomizable server settingsBuilt-in support for GPU accelerationInstallationTo install the Semantix GenAI Serve library, run the following command:pipinstallsemantix-genai-serveUsageBasic ExampleHere's a simple example of how to use the Semantix GenAI Serve library to serve a Hugging Face Transformers model:fromsemantix_genai_serveimportSemantixTorchKservefromsemantix_genai_serve.huggingfaceimportServeAutoSeq2SeqLMclassMyModel(ServeAutoSeq2SeqLM):defpredict(self,payload,headers):# Implement your custom inference logic herepassmodel=MyModel(checkpoint="facebook/bart-large-cnn")model.start_server()Customizing Server SettingsYou can customize the server settings by passing additional arguments to theSemantixTorchKserveconstructor:model=MyModel(checkpoint="facebook/bart-large-cnn",name="my_model",base_cache_dir="/path/to/cache",force_local_load=True)name: The name of the predictor (default: "predictor")base_cache_dir: The base directory for caching models (default: "/mnt/models"). Do not change that when deploying to Semantix GenAI Hub.force_local_load: If set toTrue, the model will be loaded from the local cache directory instead of downloading from Hugging Face's model hub (default:False). Do not change that when deploying to Semantix GenAI Hub.Implementing Custom Inference LogicTo implement custom inference logic, you need to override thepredictmethod in your model class:classMyModel(ServeAutoSeq2SeqLM):defpredict(self,payload,headers):# Implement your custom inference logic hereinput_example=payload["input_example"]# do something for inference# your output must be a dictionary so it gets automatically converted to JSONreturn{"response":"output"}ContributingWe welcome contributions to the Semantix GenAI Serve library! If you have any suggestions, bug reports, or feature requests, please open an issue on our GitHub repository.LicenseSemantix GenAI Serve is released under theMIT License.
Package: semaphore-sms
semaphore-smsAPI wrapper forSemaphore SMS Gateway. The documentation for their API can be foundhere.InstallationThis library should be able to support Python versions >= 3.7.Install from PyPi usingpipviapipinstallsemaphore-smsDon't have pip installed? Refer tothis documentationUsage‼️ For all these calls, you can see sample responses in tests/test_semaphore_client.py ‼️Initializing the clientfromsemaphore_smsimportSemaphoreClientclient=SemaphoreClient(api_key='your_api_key',sender_name'sender_name',)Note:SemaphoreClientwill try to pullapi_keyfromos.environif not supplied. If it still can't a value then, aSemaphoreExceptionwill be raisedSending SMSFor sendinga basic single message,client.send(message='Your text here',recipients=['09980101010',])To send one messageto several numbers in bulk, simply provide multiple numbers underrecipientsclient.send(message='Your text here',recipients=['09980101010','09980202020',])Sending outpriority messagesandOTP messageswill look very similarclient.priority(message='Your text here',recipients=['09980101010','09980202020',])client.otp(message='Your OTP is:{otp}',# refer to Semaphore's docs for detailsrecipients=['09980101010',],code=123456,# refer to Semaphore's docs for details)Fetching messageshttps://semaphore.co/docs#retrieving_messagesclient.messages(limit=100,# optionalpage=1,# optionalstart_date='2023-11-06',# optionalend_date='2023-12-25',# optionalnetwork='smart',# optionalstatus='succcess'# optional)Retrieving account informationhttps://semaphore.co/docs#retrieving_accountaccount_information=client.accounttransactions=client.transactions(limit=100,# optionalpage=1# optional)sender_names=client.sender_names(limit=100,# optionalpage=1# optional)users=client.users(limit=100,# optionalpage=1# optional)Again, you can see sample responses in tests/test_semaphore_client.pyTestsClone the repo and runPYTHONPATH=srcpython-munittestContributingBug reports and pull requests are welcome on GitHub athttps://github.com/sbenemerito/semaphore-sms/LicenseThe library is available as open source under the terms of theMIT License.