Saravanan666/pypi-org huggingface dataset:
Package: ra-flatfile-importer
RA Flatfile importerUsageThe primary usage of this tool is to validate and load flatfile data (JSON) into OS2mo, for example fixtures generated
usingra-flatfile-importer.Usage: python -m ra_flatfile_importer [OPTIONS] COMMAND [ARGS]...

  OS2mo Flatfile importer.

  Used to validate and load flatfile data (JSON) into OS2mo.

Options:
  --help  Show this message and exit.

Commands:
  schema    Generate JSON schema for valid files.
  upload    Validate the provided JSON file and upload its contents.
  validate  Validate the provided JSON file.On a development machine with the OS2mo stack running, the following will upload a previously-generated flatfile:python -m ra_flatfile_importer upload \
  --mo-url=http://localhost:5000 \
  --client-secret=603f1c82-d012-4d04-9382-dbe659c533fb \
  --auth-server=http://localhost:8081/auth \
  --json-file=mo.jsonThe tool has various other commands too, such as producing the validation schema for the flat file format:python -m ra_flatfile_importer schema --indent=4Which yields:{
    "title": "MOFlatFileFormatImport",
    "description": "Flatfile format for OS2mo.\n\nEach chunk in the list is send as bulk / in parallel, and as such 
                    entries\nwithin a single chunk should not depend on other entries within the same chunk.\n\nMinimal 
                    valid example is [].",
    "type": "object",
    "properties": {
        "chunks": {
            ...
        },
        ...
    }
}Or for validating whether a file is invalid:python -m ra_flatfile_importer validate < mo.jsonVersioningThis project usesSemantic Versioningwith the following strategy:MAJOR: Incompatible changes to existing commandline interfaceMINOR: Backwards compatible updates to commandline interfacePATCH: Backwards compatible bug fixesThe fileformat is versioned directly, and the version is exported in the file itself.LicenseThis project:MPL-2.0Dependencies:pydantic:MITThis project usesREUSEfor licensing. All licenses can be found in theLICENSES folderof the project.
Package: rafmetrics
rafMetricsMetrics for evaluating the performance and complexity of computer programs.Written Material:herePresentation:hereShort Material:herePart of Bachelor Thesis @ CSFull Material:herePresentation (ro):hereShort demo:linkComponents:rafComputingThe ML-based tool implementing the process of automatically tailoring a suitable rComplexity Class for an algorithm.WebMonitoringA tool for monitoring multiple network resources and websites.
Gather data by periodically monitoring specific resources and websites and stores results in database.ResourceManagerMonitors all resources by periodically (timer set default at 1 hour interval) sending requests to existing resources.
Store simple metrics like total time or total requests answer as entries in DB.WebsiteManagerMonitors all websites by periodically (timer set default at 1 hour interval) generating a HAR (HTTP-Archieve data performance file) for loading metrics corresponding to a website, with Chrome using Browsermob-Proxy.
Also parse and store valuable insights resulted from the HAR file into DB.
The service usesspeedprofileengine.WebMonitoring APIProvide an API for interrogating useful metrics from DB.LoginBackend implementation to provide a simple authentication, registration and management for users inside rafMetrics platform.DockerConfigKeeps track of all Docker Compose/Docker Swarm settings and configurations.KubernetesConfigKeeps track of all k8s settingsMySQLDatabase used to store persistent data required by Login and WebMonitoring.
All relations are kept inBoyce-Codd Normal Form.deploy_repo.shSimple script to ensure dockerize and deployment in Kubernetes for all backend componentsmetricsUIFrontend implementation of rafMetrics platform based on Flatlogic Template: React Material Admin — Material-UI Dashboard
Package: rafnixg
Rafnix Guzmán - Personal CardHi there 👋This is my personal cardrafnixg.devHow to use it? 🤔Install using pip:pipinstallrafnixgThen run it:rafnixgExtra Tools 🛠Blog PostsfromrafnixgimportBlogPostsblog_posts=BlogPosts()blog_posts.get_posts()CV ResumefromrafnixgimportResumeresume=Resume()resume.get_resume()
Package: rafraser-theia
TheiaOverviewTheia is a library designed to help out with automating image generation.
Theia currently has excellent functionality to help create.This project is currently in a "pre-alpha" stage. At this stage, I've mostly designed it for personal usage, but I'm working towards making this library more versatile for other users.Future plans for this library include better functionality for creating generative artworks.Getting Startedpipinstallrafraser-theiaMore InfoHandbookPyPiExamplesContributing
Package: raft
To find out what’s new in this version of raft, please seethe changelog.Welcome to Raft!raft is a Python (3.4+) library for managing shell-oriented that is a fork
of the venerable invoke library.  Unlike invoke, raft takes after its
grandmother (fab) and preserves underscores in both command names and
argument names.
subprocesses and organizing executable Python code into CLI-invokable tasks. It
draws inspiration from various sources (make/rake, Fabric 1.x, etc) to
arrive at a powerful & clean feature set.For a high level introduction, including example code, please seeour main
project website; or for detailed API docs, seethe
versioned API website.
Package: raftel
RaftelRaftel is a library to easily plot s2id cell into map and other basic operation related to s2id mapping.
Basicaly a thinly veiled wrapper of the excellents2sphere, the native python implementation of the outstandings2-geometrypackage from Google, and simple mapping function using staticmap library.InstallationInstall directly from pyPI by runningpip install raftelExampleFollow the example in the notebook for basic usage.
Package: raftify
raftify-py⚠️ WARNING: This library is in a very experimental stage. The API could be broken.Python binding ofraftify.Quick guideI strongly recommend to read the basicmemstore example codeto get how to use this library for starters, but here's a quick guide.Define your own log entryDefine the data to be stored in LogEntry and how to serialize and de-serialize it.classSetCommand:def__init__(self,key:str,value:str)->None:self.key=keyself.value=valuedefencode(self)->bytes:returnpickle.dumps(self.__dict__)@classmethoddefdecode(cls,packed:bytes)->"SetCommand":unpacked=pickle.loads(packed)returncls(unpacked["key"],unpacked["value"])Define your application Raft FSMEssentially, the following three methods need to be implemented for theStore.apply: applies a commited entry to the store.snapshot: returns snapshot data for the store.restore: applies the snapshot passed as argument.And also similarly toLogEntry, you need to implementencodeanddecode.classHashStore:def__init__(self):self._store=dict()defget(self,key:str)->Optional[str]:returnself._store.get(key)defapply(self,msg:bytes)->bytes:message=SetCommand.decode(msg)self._store[message.key]=message.valuelogging.info(f'SetCommand inserted: ({message.key}, "{message.value}")')returnmsgdefsnapshot(self)->bytes:returnpickle.dumps(self._store)defrestore(self,snapshot:bytes)->None:self._store=pickle.loads(snapshot)Bootstrap a raft clusterFirst bootstrap the cluster that contains the leader node.logger=Slogger.default()logger.info("Bootstrap new Raft Cluster")node_id=1raft=Raft.bootstrap_cluster(node_id,raft_addr,store,cfg,logger,peers)awaitraft.run()Join follower nodes to the clusterThen join the follower nodes.If peer specifies the configuration of the initial members, the cluster will operate after all member nodes are bootstrapped.logger=Slogger.default()join_ticket=awaitRaft.request_id(raft_addr,peer_addr,logger)node_id=join_ticket.get_reserved_id()raft=Raft.new_follower(node_id,raft_addr,store,cfg,logger,peers)tasks=[]tasks.append(raft.run())awaitraft.join([join_ticket])Manipulate FSM by RaftServiceClientIf you want to operate the FSM remotely, use theRaftServiceClient.client=awaitRaftServiceClient.build("127.0.0.1:60061")awaitclient.propose(SetCommand("1","A").encode())Manipulate FSM by RaftNodeIf you want to operate FSM locally, use the RaftNode interface of the Raft objectraft_node=raft.get_raft_node()awaitraft_node.propose(message.encode())DebuggingRaftify also provides a collection of CLI commands that let you check the data persisted in lmdb and the status of Raft Server.$ raftify_cli debug persisted ./logs/node-1$ raftify_cli debug node 127.0.0.1:60061
Package: rafutils
A few helpful utilities that I find useful.AutovivificationDictionary type that allows for dynamic allocation of nested key values
without prior defintion.>>> a = AutoVivification()
>>> a[1][2] = 3Multiprocessing WrapperThis module provides a set of methods for handling multi-threading
patterns more easily.>>> import user_metrics.utils.multiprocessing_wrapper as mpw
>>> mpw.build_thread_pool(['one','two'],len,2,[])
[2,2]Record TypeDefines the recordtype method which returns a type definition object
that allows for the creation of mutable objects with specified
attributes.  These are essentially identical to collections.namedtuple
except that the attributes may be changed after the namedtuple objects are
created.>>> Point = recordtype('Point', 'x y', default=0)
>>> Point()                 # instantiate with defaults
Point(x=0, y=0)MiscellaneousSome miscellaneous methods exposed by this package.Format TimestampTheformat_timestampmethod takes a timestamp representation (string
or datetime) and convert to a standard string representation.EnumerationTheenummethod implements an enumeration:>>> Numbers = enum('ZERO', 'ONE', 'TWO')
>>> Numbers.ZERO
0
>>> Numbers.ONE
1Build Named TupleGiven a set of types, values, and names builds a named tuple.  This
method expects three lists all of the same length and returns a
dynamically built namedtuple object.  Currently, onlylist,str,int, andfloatcast methods are accepted as members
oftypes.  The mothod is namedbuild_namedtuple.Unpack FieldsUnpacks the attributes and values of a record type or named tuple into
an ordered dictionary.
Package: rag
RagMake creating rest APIs in Django simple.Rag is a simple batteries included tool to make building rest API's using Django fast and easy.UsageQuick StartCommand LineThe command line interface has commands to start a dev server, run development shell, run tests, run Django management commands, create an empty example project, and run a Celery background task worker.ValidationTo validate a field you must start with a root validator:to,am,accepts, oroptional.  All root
validators check that a field is defined before proceeding with the exception ofoptionalwhich will
immediately accept if the field is undefined on the json object.ContributingLauching Dev StackUse docker compose to launch development stack.Dev SetupTo install development requirements.poetry installRunning Dev TestsTo run tests:poetry run poe testTo run a specific test or test file:poetry run poe test -k test_rag.pyTo run a specific test in a suite:poetry run poe test -k "test_validate.py and test_check"To see all print statements of passing tests use the-sflagpoetry run poe test -sTo run tests with auto reload:poetry run poe test-watchTo run tests with auto reload and specific test:poetry run poe test-watch -k test_nameNote: Errors like this might have a root exception with more detail and may mean you are missing migrations:psycopg2.errors.InvalidCursorNameRunning Template Integration TestsFrom project root folder enter poetry shellpoetry shellEnter template foldercd ./rag/templates/largeRun testsrag testRunning All TestsTo run all unit and integration tests:poetry run ./scripts/test_all.py
Package: raga-cli
# Raga CLI (rc) - Data Version Control ToolRaga CLI (rc) is a command-line tool built in Python for managing data version control. It allows you to track changes to datasets and models, create checkpoints, and manage repositories to ensure smooth collaboration and reproducibility in data-related projects.InstallationTo install Raga CLI, make sure you have Python installed, and then run the following command:pipinstallraga-cliUsageRaga CLI provides several commands to help you manage your data version control tasks. Here are the available commands:Create a New Repositoryrcrepocreate-n<repo_name>-tag<model/dataset>This command creates a new repository with the specified name and tags it as a model or dataset repository.Clone an Existing Repositoryrcrepoclone-n<repo_name>Use this command to clone an existing repository by providing the repository name.Show Repository Current Local VersionrcrepoversionDisplays the current local version of the repository.Show Repository InformationrcrepoinfoProvides information about the repository, including its name, type, and current version.Upload Files and Create Checkpointrcput-m"<commit message>"Uploads tracked files or directories to remote storage and creates a checkpoint with the specified commit message.Download Tracked Files or DirectoriesrcgetDownloads files or directories that are tracked by Raga CLI.Download Files or Directories of a Specific Repo Versionrcget-repo-version<repo_version>Downloads files or directories of a particular version of the repository.Additional Flags-h,--help: Prints the usage/help message and exits.-q,--quiet: Suppresses standard output. Exits with 0 if no problems arise; otherwise, exits with 1.-v,--verbose: Displays detailed tracing information.ContributingContributions to Raga CLI are welcome! If you encounter issues or have suggestions for improvements, feel free to open an issue or submit a pull request on theGitHub repository.LicenseThis project is licensed under theMIT License.
Package: ragas-haystack
ragas-haystackTable of Contentsragas-haystackInstallationTestingExamplesLicenseInstallationpip install ragas-haystackFor more information about the Ragas evaluation framework, please refer to theirdocumentation.Testinghatch run testExamplesYou can find code examples showing how to use the Evaluator under theexample/folder of this repo.Licenseragas-haystackis distributed under the terms of theApache-2.0license.
Package: ragas-once
Ragas CLIA one-step Ragas cli tool to evaluate QCAG testsets generated by RAG apps. (Q = Question, C = Contexts, A = Answer, G = Ground_truth)Install with pippipinstallragaslangchain==0.0.354then:pipinstallragas_onceArguments--model: Specifies the model to use for evaluation.Default value is "gpt-3.5-turbo". Langchain compatible.--api_base: Specifies the base URL for the API.Default value is "https://api.openai.com/v1".--api_key: Specifies the API key to authenticate requests.Not required if using psuedo-openai API server, e.g. vLLM, Fastchat, etc.--embeddings: Specifies the Huggingface embeddings model to use for evaluation.Embeddings will runlocally.Will use OpenAI embeddings if not set.Better set if using psuedo-openai API server.--metrics: Specifies the metrics to use for evaluation.Will use Ragas default metrics if not set.Default metrics:["answer_relevancy", "context_precision", "faithfulness", "context_recall", "context_relevancy"]Other metrics:"answer_similarity", "answer_correctness"--dataset: Specifies the path to the dataset for evaluation.Dataset format must meet RAGAS requirements.Will use fiqa dataset as demo if not set.UsageFiqa dataset demo:python3-mragas_once.cli--api_key"YOUR_OPENAI_API_KEY"Evaluate with GPT-4 andBAAI/bge-small-enembeddingsThe huggingface embeddings will run locally, soMake sure your machine works and havesentence-transformersinstalled:pipinstallsentence-transformersThen run:python3-mragas_once.cli--model"gpt-4"--api_key"YOUR_OPENAI_API_KEY"--embeddings"BAAI/bge-small-en"--dataset"path/to/dataset.csv"Prepare DatasetSeeRagas documentation
Package: raga-testing-platform
Raga Testing PlatformTheraga-testing-platformpackage provides a Python client for interacting with the Raga Testing Platform. It allows you to easily create and manage test sessions, datasets, and perform various testing operations.InstallationYou can installraga-testing-platformusing pip:pip install raga-testing-platformUsageTo use the package, import the necessary classes and modules:from raga import Dataset, TestSession, Auth
import pandas as pd
from typing import Optional, List, DictCreating a test DataFrametest_df = pd.DataFrame({
    'column1': [1, 2, 3],
    'column2': ['a', 'b', 'c']
})Defining the Schema classclass Schema:
    def __init__(
        self,
        prediction_id: Optional[str] = None,
        timestamp_column_name: Optional[str] = None,
        feature_column_names: Optional[List[str]] = None,
        metadata_column_names: Optional[List[str]] = None,
        label_column_names: Optional[Dict[str, str]] = None,
        embedding_column_names: Optional[Dict[str, str]] = None,
    ):
        self.prediction_id = prediction_id
        self.timestamp_column_name = timestamp_column_name
        self.feature_column_names = feature_column_names
        self.metadata_column_names = metadata_column_names
        self.label_column_names = label_column_names
        self.embedding_column_names = embedding_column_namesCreating an instance of the Schema classschema = Schema()Creating an instance of the Auth classIt will create auth token for further usages.auth = Auth()This variable stores auth token. You can use it untill expire token.auth.tokenCreating an instance of the TestSession classIn this instanceexperimentwill create and returnexperiment_idexperiment_id = TestSession(token, 1, "my_experiment")This variable storesexperiment_id. You can use it in further experiment.experiment_id.experiment_idCreating an instance of the Dataset classtest_ds = Dataset(token, experiment_id, test_df, schema, "DatasetName")Loading labels from a filetest_ds.load_labels_from_file(
    "/path/to/labels.json",
    "dataset_name",
    "id_column_name",
    "label_column_name",
    "metadata_column_name",
    "category_column_name",
    "category_id_column_name"
)To Debugexport DEBUG=1
Package: ragavi
ragaviRadio Astronomy Gain and Visibility InspectorIntroductionThis library mainly requiresBokehPython casacoreDaskmsDatashaderNodejs>=8- Install build dependencies:**Python casacorecomes as a dependency ofDaskms**
Nodejs is a requirement forBokehand can be installed using the commands$sudoapt-getinstallcurl$curl-sLhttps://deb.nodesource.com/setup_8.x|bash-$apt-getinstall-ynodejsAll python requirements are found in requirements.txtorTo installnodejsin the virtual environment, use:nodeenv, a nodejs virtual environment.
More info can be foundhereCreate nodejs virtual environment with:$nodeenvenvNameand$.envName/bin/activateto switch to environment.InstallationInstallation fromsource,
working directory where source is checked out$pipinstall.This package is available onPYPIvia$pipinstallragaviUsageRagavi currently has two segements:Gain plotterVisibility plotterFor the gains plotter, the name-spaceragavi-visis used. To get help for thisNoteragavinamespace will soon change toragavi-vis$ragavi-gains-hTo use ragavi gain plotter$ragavi-gains-t/path/to/your/tableMultiple tables can be plotted on the same document simply by adding them in a space separated list to the-t/--tableswitch e.g$ragavi-gains-tdelay/table/1/bandpass/table/2flux/table/3For the visibility plotter, the name-spaceragavi-visis used. Help can be obtained by running$ragavi-vis-hTo run ragavi-vis, the arguments--table,--xaxisand--yaxisare basic requirements e.g.$ragavi-vis--ms/my/measurement/set--xaxistime--yaxisamplitudeFor large datasets, it is advisable to supply at least--yminand--ymaxvalues to avoid an extra pass over the data.Change the size (resolution) of the output aggregated image – and resulting html file size – by specifying--canvas-widthand--canvas-heightoptions.A slightly more detailed documentation onragavican be found hereLicenseThis project is licensed under the MIT License - seelicensefor details.ContributeContributions are always welcome! Please ensure that you adhere to our coding standardspep8.
Package: ragbooster
RAGBoosterRAGBooster improves the performance ofretrieval-based large language modelsby learning which data sources are important to retrieve high quality data.We provide an example notebook that shows how we boostRedPajama-INCITE-Instruct-3B-v1, a small LLM with 3 billion parameters to be on par withOpenAI's GPT3.5(175 billion parameters) in a question answering task by using Bing websearch and ragbooster:Demo notebook: Boosting RedPajama-INCITE-Instruct-3B-v1 for question answeringFurthermore, we have an additional example notebook, where we demonstrate how to boost atiny qa modelto get within 5% accuracy on GPT3.5 on a data imputation task:Demo notebook: Boosting minilm-uncased-squad2 for data imputationCore classesAt the core of RAGBooster areRetrievalAugmentedModels, which fetch external data to improve prediction quality. Retrieval augmentation requires two components:Aretriever, which retrieves external data for a prediction sample. We currently only implement aBingRetriever, which queries Microsoft's Bing Websearch API.Agenerator, which generates the final prediction from the prediction sample and the external data. This is typically a large language model. We provide theGeneratorinterface, which makes it very easy to leverage LLMs available via an API, for example from OpenAI.Once you defined your retrieval-augmented model, you can leverageRAGBoosterto boost its performance by learning the data importance of retrieval sources (e.g., domains in the web). This often increases accuracy by a few percent.BackgroundHave a look at our paper onImproving Retrieval-Augmented Large Language Models with Data-Centric Refinementfor detailed algorithms, proofs and experimental results.InstallationRAGBooster is available aspip package, and can be installed as follows:pip install ragboosterInstallation for DevelopmentRequires Python 3.9 andRustto be availableClone the repository:git clone git@github.com:amsterdata/ragbooster.gitChange to the project directory:cd ragboosterCreate a virtualenv:python3.9 -m venv venvActivate the virtualenvsource venv/bin/activateInstall the dev dependencies withpip install ".[dev]"Build the projectmaturin develop --releaseOptional steps:Run the tests withcargo test --releaseRun the benchmarks withRUSTFLAGS="-C target-cpu=native" cargo benchRun linting for the Python code withflake8 pythonStart jupyter withjupyter notebookand run the example notebooks
Package: ragcore
RAG CoreA Retrieval-Augmented Generation library with a CLI interface. Build RAG applications with just a few commands and a configuration file.Supported setupsDatabasesLLMsEmbeddingsDocument typesChroma (local)OpenAIOpenAIPDFPinecone (remote)AzureOpenAIAzureOpenAIFor more details see thedocumentation.InstallationTo install, runpipinstallragcoreor clone and build from sourcegitclonehttps://github.com/daved01/ragcore.gitcdragcore
pipinstall.If everything worked, runningragcore-hshould show you some information aboutragcore.A Simple ExampleTo build an application with OpenAI or AzureOpenAI LLMs and embeddings, and a local database, first set your OpenAIAPI keyas describedhere:exportOPENAI_API_KEY=[yourtoken]Then, create a config fileconfig.yamllike this in the root of your project:database:provider:"chroma"number_search_results:5base_dir:"data/database"splitter:chunk_overlap:256chunk_size:1024embedding:provider:"openai"model:"text-embedding-model"llm:provider:"openai"model:"gpt-model"And finally, create your application using this config file:fromragcoreimportRAGCoreapp=RAGCore()# pass config=<path-to-config.yaml> if not in root# Upload a document "My_Book.pdf"app.add(path="My_Book.pdf")# Now you can ask questionsanswer=app.query(query="What did the elk say?")print(answer.content)# List the document's title and content on which the response is basedfordocinanswer.documents:print(doc.title," | ",doc.content)# List all documents in the databaseprint(app.get_titles())# You can delete by titleapp.delete(title="My_Book")And that's it! For more information, as well as an overview of supported integrations check out thedocumentation.