Saravanan666/pypi-org huggingface dataset:
Package: experiment-collection
Experiment collectionA set of utilities for storing and organizing experimentsInstallpipinstallexperiment-collectionYou can find another versions atreleasesorpypi.Usage_exampleExperiment- structure with experiment data;ExperimentCollectionLocal- local experiment storage;ExperimentCollectionRemote- remote experiment storage;
could combine data from multiply sources.fromexperiment_collectionimportExperiment,ExperimentCollectionRemoteexps=ExperimentCollectionRemote('localhost:50051','main','token')foriinrange(10):exp=Experiment('name_{}'.format(i))exp.set_metrics({'lr':0.1})exp.set_params({'auc':0.5})exps.add_experiment(exp)assertexps.check_experiment(exp)# Delete latest experimentexps.delete_experiment(exp)assertnotexps.check_experiment(exp)View all results.
All metrics and params auto flatten into single columns.exps.get_experiments()name                       time  params_auc  metrics_lr
0  name_0 2020-09-30 23:56:39.932871         0.5         0.1
1  name_1 2020-09-30 23:56:40.216424         0.5         0.1
2  name_2 2020-09-30 23:56:40.620029         0.5         0.1
3  name_3 2020-09-30 23:56:40.916781         0.5         0.1
4  name_4 2020-09-30 23:56:41.240535         0.5         0.1
5  name_5 2020-09-30 23:56:41.567865         0.5         0.1
6  name_6 2020-09-30 23:56:41.861890         0.5         0.1
7  name_7 2020-09-30 23:56:42.177155         0.5         0.1
8  name_8 2020-09-30 23:56:42.507883         0.5         0.1
9  name_9 2020-09-30 23:56:42.818714         0.5         0.1LicenseMIT LicenseCopyright (c) 2020 AsciiShell (Aleksey Podchezertsev)
Package: experiment-collection-server
Experiment collectionA set of utilities for storing and organizing experimentsInstallpipinstallexperiment-collectionYou can find another versions atreleasesorpypi.Usage_exampleExperiment- structure with experiment data;ExperimentCollectionLocal- local experiment storage;ExperimentCollectionRemote- remote experiment storage;
could combine data from multiply sources.fromexperiment_collectionimportExperiment,ExperimentCollectionRemoteexps=ExperimentCollectionRemote('localhost:50051','main','token')foriinrange(10):exp=Experiment('name_{}'.format(i))exp.set_metrics({'lr':0.1})exp.set_params({'auc':0.5})exps.add_experiment(exp)assertexps.check_experiment(exp)# Delete latest experimentexps.delete_experiment(exp)assertnotexps.check_experiment(exp)View all results.
All metrics and params auto flatten into single columns.exps.get_experiments()name                       time  params_auc  metrics_lr
0  name_0 2020-09-30 23:56:39.932871         0.5         0.1
1  name_1 2020-09-30 23:56:40.216424         0.5         0.1
2  name_2 2020-09-30 23:56:40.620029         0.5         0.1
3  name_3 2020-09-30 23:56:40.916781         0.5         0.1
4  name_4 2020-09-30 23:56:41.240535         0.5         0.1
5  name_5 2020-09-30 23:56:41.567865         0.5         0.1
6  name_6 2020-09-30 23:56:41.861890         0.5         0.1
7  name_7 2020-09-30 23:56:42.177155         0.5         0.1
8  name_8 2020-09-30 23:56:42.507883         0.5         0.1
9  name_9 2020-09-30 23:56:42.818714         0.5         0.1LicenseMIT LicenseCopyright (c) 2020 AsciiShell (Aleksey Podchezertsev)
Package: experimenthq
ExperimentHQExperimentHQ is a Python package that allows you to track and manage experiments directly from your Python code, and seamlessly sync the results to a Notion database. With ExperimentHQ, you can easily monitor the performance of your models, log custom metrics, and compare experiments with ease, all from a single, intuitive interface.The purpose of this document is to provide a guide for using ExperimentHQ to track experiments and sync the results to Notion.PrerequisitesBefore you can use ExperimentHQ, you need to set up the following prerequisites:Set up a Notion account.Set up anExperimentHQaccount.Connect you Notion account withExperimentHQCopy your personal API token fromExperimentHQInstall ExperimentHQ.InstallationTo install ExperimentHQ, simply run the following command:pipinstallexperimenthqUsageCreating an ExperimentTo create an experiment and log it to Notion, you can use the following code snippet as an example:importexperimenthqasexexperiment=ex.Experiment(name="My First Experiment",project="Project A",api_key="YOUR_API_KEY")experiment.log_parameter("accuracy",0.85)experiment.log_parameter("loss",0.05)This code creates an experiment with the name "My First Experiment" and tags it with the project "Project A". It then logs two parameters, "accuracy" and "loss", with values 0.85 and 0.05 respectively.Viewing Experiments in NotionTo view the experiments that you've logged with ExperimentHQ in Notion go to the Notion page that you've linked to ExperimentHQ. You should see a table with the experiments that you've logged.ConclusionExperimentHQ provides a simple and intuitive way to track and manage your experiments in Python and sync the results to Notion. By using ExperimentHQ, you can streamline your experiment tracking and make your work more efficient and effective. Try it out today and see the difference it can make in your workflow!
Package: experimentkit
Experiment Kit 🧪Framework for creating and maintaining projects for study and research purposes.Creating a study project is always exciting at first, but almost immediately it can become tedious because of a whole series of arrangements that need to be made in order to experiment without worrying later.
For example, it is very easy to end up withspaghetti-codeand messy directories, where you don't understand where the data are and what the results of an experiment were.This framework aims to assist the user during this boring stages in order to let them time to focus on more valuable things.
Package: experimentmaker
Experiment Maker:Experiment Maker is a software developed to reduce the effort and the time spent when designing prompts
for in-context learning and/or combining multiple prompts in an experimental pipeline.
This package allows you to create custom prompts and pipelines to perform in-context learning with GPT-3.Our system is composed of two complementary components:Prompt Designer: supports users to design promptss.Research project adopted in “Leveraging pre-trained language models for conversational information seeking from text”Pipeline Maker: helps users in creating a pipeline by combining multiple prompts.Pipeline Maker. This tool is part of the Experiment Maker project. It allows to create custom GPT-3 pipeline.
Adopted in “Assisted Knowledge Graph Building Using Pre-Trained Language Models”
This tool also allows to integrate custom python scripts to manipulate results.Demo video:See our tool in actionhttps://youtu.be/5e_XAdI2bPQInstallation:You can choose to install the entire program from pypiExperiment MakerpipinstallexperimentmakerOr, install one of the two components:Prompt DesignerpipinstallpromptdesignerPipeline MakerpipinstallpipelinemakerExecute programs:Experiment Maker:fromexperimentmaker.experimentmakerimportLunchEMLunchEM()To lunch one of the components
- Prompt Designer:frompromptdesigner.PromptDesignerimportLunchPromptDesignerLunchPromptDesigner()Pipeline Maker:frompipelinemaker.experimentmakerimportLunchExperimentLunchExperiment()Custom Modules (filters)You can test our custom modules (written in python) contained in the folder filters-scripts.
To create your own custom filter, you simply need to write a python class with a method, or a python function, called ‘Parse’ that accept a single argument.
The results of a step, or the results of the pipeline are passed as dictionary to the method/function.For example, consider the following example function.
This function receives the results (data variable) and clean the answers by removing unused characters from the text.defParse(self,data):defparseitem(item):item=item.replace('-','',1)item=item.replace("'",'',1)item=item.replace("'",'',1)item=item.strip()returnitemiftype(data)==str:returnparseitem(data)return[parseitem(item)foritemindata]
Package: experiment-manager
Copyright © 2016-2018SoftFIREandTU Berlin. Licensed underApache v2 License.Experiment ManagerThe SoftFIRE Experiment Manager is the main component of the SoftFIRE
middleware. Other managers can register to the experiment manager so
that the Experiment Manager is able to redirect requests from the
experimenter to the appropriate manager. The managers use the gRPC
protocol for communication.The main tasks of the Experiment Manager are:Exposing an API for the dashboard and the SoftFIRE SDKUser authentication and authorizationResource discoveryResource reservationResource provisioningProcessing the experiment definitionProviding experiment monitoringIn the figure below you can see the work flow of the Experiment Manager
interacting with the experimenter and other managers.Technical RequirementsThe Experiment Manager requires Python 3.5 or higher.Installation and configurationYou can install the Experiment Manager using pip:pip install experiment-managerand then start it with theexperiment-managercommand.Or you can run it from source code by cloning the git repository,
installing the dependencies as specified in thesetup.pyfile and
executing theexperiment-managerscript.The experiment manager needs a configuration file present at/etc/softfire/experimen-manager.ini. An example of the configuration
file can be foundhere.Issue trackerIssues and bug reports should be posted to the GitHub Issue Tracker of
this project.What is SoftFIRE?SoftFIRE provides a set of technologies for building a federated
experimental platform aimed at the construction and experimentation of
services and functionalities built on top of NFV and SDN technologies.
The platform is a loose federation of already existing testbed owned and
operated by distinct organizations for purposes of research and
development.SoftFIRE has three main objectives: supporting interoperability,
programming and security of the federated testbed. Supporting the
programmability of the platform is then a major goal and it is the focus
of the SoftFIRE’s Second Open Call.Licensing and distributionCopyright © [2016-2018] SoftFIRE projectLicensed under the Apache License, Version 2.0 (the “License”);you may not use this file except in compliance with the License. You may
obtain a copy of the License athttp://www.apache.org/licenses/LICENSE-2.0Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed
Package: experimentor
Experimentor v0.2.0A flexible package for experiment control and automationExperimentor is a framework that simplifies the task of building instrumentation software in Python. It follows the MVC pattern to the letter. Controllers are drivers, Models implement the logic of the instruments and the experiments themselves, while the View specifies the user interface.Experimentor is opinionated, but allows to be extended mostly by subclassing general classes and overwriting specific
methods.InstallingThe code of this program is the repository that can be found athttps://github.com/aquilesC/experimentor. You may also check the websitePython for the Labfor some updates and comments that may not enter into this documentation.DocumentationTo read more about how to get started with the Experimentor package, check thedocumentation on Read The Docs.Work In ProgressExperimentor is work in progress, meaning that we can introduce backwards-incompatible changes. We will try to give some
warning time prior to this, but we can't guarantee it.Helping with DevelopmentWe welcome contributions from anyone. Please check theContribution Guidelinesto learn how you can contribute to the project.
Package: experiment-replay
Experiment replayDescriptionSimple utils to record the commands you run. It was developped with ML experiments
in mind. Where you often tweak your code just a little before launching an experiment
and when after a few days/weeks you come back you want to know how you achieved such
amazing results you don't remember and it takes a long time to achieve again.This library is extremely simple. You can't run any experiment that is not committed
so you have a commit to know what was changed and why. It also stores the exact command
line you used so that configuration hacking is also remembered. It uses the git commit
message to store that data so it does not require any external tool.Installpip install experiment_replayUsageIt's simple to enable an experiment just do  in yourtrain.pyfile for instanceimportexperiment_replay## My codeif__name__=="__main__":experiment_replay.setup()my_training_loop()Then when you actual run your training let's saypython train.py --batch-size=16.You can then do:python -m experiment_replayto get the list of all the commands you ran with
experiment_replay enabled.Experiments :
Date                       Id     Commit                                   Command             
2019-05-13 14:53:35.410538 472a12 f9dfe80125ea4856ce368270bce3aeb980829b2c python example.pyYou can they replay it withpython -m experiment_replay 472a12
Package: experiment-results-manager
🔬 ERM: Experiment Results ManagerLight-weight alternative tomlflowexperiment tracking that doesn't require kubernetes. Useful tool to compare metrics between training attempts in your model training workflow✨ Features📈 Track plots, metrics, & other data👀 Side-by-side comparison💾 Experiment registry⛅️ Supports S3, GCS, Azure and others (viafsspec)🚀 Examples & DemosQuick and easy:serialize_and_deserialize.ipynbPractical but more involved:compare_runs.ipynbBrowse the registry:browse_registry.ipynb✅ Get StartedInstallationpipinstallexperiment-results-manager\gcsfs\s3fs# install s3fs if you plan to store data in s3# install gcsfs if you plan to store data in google cloud storageBasic Usageimportexperiment_results_manageraserm# Create an experiment runer=erm.ExperimentRun(experiment_id="my_experiment",variant_id="main")# Log relevant dataer.log_param("objective","rmse")er.log_metric("rmse","0.9")er.log_figure(mpl_fig,"ROC Curve")er.log_text("lorem ipsum...","text")# Display the report (if you are in a notebook)html=erm.compare_runs(er)display(HTML(html))# Save to registrysaved_path=erm.save_run_to_registry(er,"s3://erm-registry")Made with ❤️ in Berlin
Package: experiments
# Experiments APITraining a deep neural network requires finding a good combination of model hyperparameters. The process of finding good values for each is called hyperparameter optimization. The number of jobs required for each such experiment typically ranges from the low ones into the hundreds.Individual workflows for optimization vary, but this is typically an ad-hoc manual process including custom job submit scripts or even pen and paper.This project provides an API to support machine learning experiments on Kubernetes. This is done by moving the experiment context into a shared API and standardizing experiment job metadata. This promotes sharing results and tool development. Decoupling parameter space search from job execution further promotes re-use. This project eases job integration with the experiment tracking system by providing a python client library.[![overview figure](docs/images/overview.png)](https://docs.google.com/drawings/d/1CGDVt9finf_QC_H6lAIW9StmYiNOCLoemAmpNRN47tg/edit)## Prerequisitesgitmakepythonkubectl and a connected cluster (minikube or a full cluster)## InstallationTo install the most recent release, run the following:` $ pip install experiments `## DevelopmentTo check out and install the latest development release, run:` $ git clonehttps://github.com/IntelAI/experiments.git$ cd experiments $ pip install . `To test the Experiments API, run the following:` $ pip install-rrequirements_tests.txt $ make test `## Appendix### ConceptsExperimentDescribes a hyperparameter space and how to launch a job for a sample in that space. Has a unique name.OptimizerA program that reads an experiment and creates jobs with different hyperparameter settings. This can be done all in one shot, or the optimizer could be a long-running coordinator that monitors the performance of various samples to direct the hyperparameter optimization process. This program is supplied by the user.ResultEncodes metadata about a single job run for an experiment. For example, a handful of high level metrics per training epoch and a pointer to an output directory on shared storage. There is one result resource per job. Each result has the same name as the job it represents.
Package: experiments-csv
experiments_csv - experiment tracking via CSV filesMotivationYou run an experiments with various input parameters. You check various input combinations and run the experiment on each combination.Suddenly, during one of the runs, the program crashes. You have to restart the experiment for the current and future inputs, but you do not want to repeat it for all previous inputs.Solutionexperiments_csvsaves all the experiment input and output data into a CSV file. When you restart the experiment, it reads the CSV file and notes all the input combinations for which the experiment already completed. It then automatically skips these input combinations.InstallationBasic installation:pip install experiments-csvInstallation with plotting ability:pip install experiments-csv[plotting]UsageSee the demo programs in theexamplesfolder for usage examples. In detail, you should:Write a function for running a single instance of the experiment.
The function may take any number of arguments as inputs.
It should return a dict with any number of arguments as outputs.
For example:def fair_division_algorithm(num_of_agents:int, num_of_items:int, threshold:float, criterion:str):
        # ...
        # Run the fair division algorithm with the given parameters
        # ...
        return {
            "runtime": runtime,
            "max_value": max(values),
            "min_value": min(values)
        }Decide what ranges you want for your input parameters, for example:input_ranges = {
        "num_of_agents": [2, 3, 4],
        "num_of_items": range(2,10),
        "threshold": [0.5],
        "criterion": ["proportionality", "envy-freeness"]
    }Create an Experiment object. It takes as arguments the folder for the experiment results, the experiment file name, and the folder for backups:import experiments_csv
    ex = experiments_csv.Experiment("results/", "results.csv", "results/backups")Run the experiment:ex.run(fair_division_algorithm, input_ranges)This loops over all combinations of inputs in the ranges you passed (the Cartesian product of the ranges in input_ranges), calls the single-instance function, and records the results in the given CSV file. The CSV file will have a column for every input and output (in the example: num_of_agents, num_of_items, threshold, criterion, runtime, min_value, max_value), and a single row for every run.If the experiment stops abruptly due to an error, you can re-run the same code, and it will not repeat the experiments with the combinations of arguments it already completed - it will only run the experiments for the combinations not done yet.If you do want to restart the experiment from scratch, either manually delete the CSV file, or use a new CSV file with a different name, or doex.clear_previous_results()To log the inputs and outputs during run, you can do:ex.logger.setLevel(logging.INFO)PlottingTo plot the results, you can use theplot_resultsfunction, for example:from matplotlib import pyplot as plt
    experiments_csv.plot_results(plt, "results.csv", filter={"algorithm": "abc"}, xcolumn="size", ycolumn="runtime", zcolumn="bits")See thedemo programsfor usage examples.
Package: experiment-utils
Experiment_utilsHelper utils for track and manage Dl experimets with pytorch.Very early stage - just draft for my utils.Installpip install experiment-utilsEditeble install:  git clone https://github.com/ayasyrev/experiment-utils   cd experiment-utils   pip install -e .How to useImport Experiment:fromexperiment_utils.experimentimport*After import you has p (stands for Parameters) and e (Experiment) objects.Name the experiment, later it will be used in logs.p.exp_name='test1'p.exp_name'test1'e.p.exp_name'test1'Load learnere.get_learner()e.p.modelfunctools.partial(<function resnet18 at 0x7fc8d4dd08c0>, num_classes=10)e.learn.model.fcLinear(in_features=512, out_features=10, bias=True)Short notation for learn - le.l.model.conv1Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)Now we can easy change some parameter anr start train with pipeline what yuo neg.from experiment_utils.utils import train_fc, plotp.pipeline = [train_fc, plot]p.lr = 0.001p.epochs = 10e(repeat_times=2)
Package: experinet
experinet#######.. |PyPI-Status| |PyPI-Versions| |Build-Status| |Codecov| |LICENCE|.. .. |experinet_icon| image:: https://github.com/shaypal5/experinet/blob/88d480fd90820ea58c062029ce7e926201794e47/experinet_small.pngSynthetic network theory experiment... code-block:: pythonexperinet.run_experiment({'entry': {'type': 'generate','model': 'ba','d': 2,}}).. contents::.. section-numbering::Installation============.. code-block:: bashpip install experinetFeatures========* Pure python.* Supports Python 3.5+.* Fully tested.Use===TBAContributing============Package author and maintainer is Shay Palachy (shay.palachy@gmail.com); You are more than welcome to approach him for help. Contributions are very welcomed.Installing for development----------------------------Clone:.. code-block:: bashgit clone git@github.com:shaypal5/experinet.gitInstall in development mode, including test dependencies:.. code-block:: bashcd experinetpip install -e '.[test]'Running the tests-----------------To run the tests use:.. code-block:: bashcd experinetpytestAdding documentation--------------------The project is documented using the `numpy docstring conventions`_, which were chosen as they are perhaps the most widely-spread conventions that are both supported by common tools such as Sphinx and result in human-readable docstrings. When documenting code you add to this project, follow `these conventions`_... _`numpy docstring conventions`: https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt.. _`these conventions`: https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txtAdditionally, if you update this ``README.rst`` file,  use ``python setup.py checkdocs`` to validate it compiles.Credits=======Created by Shay Palachy (shay.palachy@gmail.com)... |PyPI-Status| image:: https://img.shields.io/pypi/v/experinet.svg:target: https://pypi.python.org/pypi/experinet.. |PyPI-Versions| image:: https://img.shields.io/pypi/pyversions/experinet.svg:target: https://pypi.python.org/pypi/experinet.. |Build-Status| image:: https://travis-ci.org/shaypal5/experinet.svg?branch=master:target: https://travis-ci.org/shaypal5/experinet.. |LICENCE| image:: https://github.com/shaypal5/experinet/blob/master/mit_license_badge.svg:target: https://github.com/shaypal5/experinet/blob/master/LICENSE.. https://img.shields.io/github/license/shaypal5/experinet.svg.. |Codecov| image:: https://codecov.io/github/shaypal5/experinet/coverage.svg?branch=master:target: https://codecov.io/github/shaypal5/experinet?branch=master
Package: experipy
A framework for writing and running Computational Science experiments.experipyprovides a composable grammar for automating experiment script generation and a tool which can then execute those scripts. Supports PBS and Slurm script based queueing.A Simple Examplefrom experipy.exp       import Experiment
from experipy.grammar   import Executable

exp = Experiment(Executable("echo", ["Hello World"]),
                 expname="test",
                 destdir="results")
exp.run()This will run the programechowith the argumentHello Worldin a directory in/tmp, writing the output and error, along with timing information, to the directoryresults. Directories will be created as needed. A complete example showing how to write an experiment for a Python script can be found intest/runtest.py.Installationexperipycan be installed from PyPI, and has no other dependencies.?> pip install experipyDocumentationFull documentation for experipy can be found athttps://experipy.readthedocs.io.Features In The WorksExpandexperipy.systemto include more standard command line toolsBeef up PBS script options