Saravanan666/pypi-org huggingface dataset:
Package: benchmark
benchmarkis a Python benchmarking framework, similar to Steve Purcell’sunittestin basic structure. It’s as simple as:import benchmark

import math

class Benchmark_Sqrt(benchmark.Benchmark):

    each = 100 # allows for differing number of runs

    def setUp(self):
        # Only using setUp in order to subclass later
        # Can also specify tearDown, eachSetUp, and eachTearDown
        self.size = 25000

    def test_pow_operator(self):
        for i in xrange(self.size):
            z = i**.5

    def test_pow_function(self):
        for i in xrange(self.size):
            z = pow(i, .5)

    def test_sqrt_function(self):
        for i in xrange(self.size):
            z = math.sqrt(i)

class Benchmark_Sqrt2(Benchmark_Sqrt):
    # Subclass the previous benchmark to change input using
    # self.setUp()

    label = "Benchmark Sqrt on a larger range"
    # The above label comes from the class name, this oen
    # comes from the label attribute

    each = 50

    def setUp(self):
        self.size = 750000

if __name__ == '__main__':
    benchmark.main(format="markdown", numberFormat="%.4g")
    # could have written benchmark.main(each=50) if the
    # first class shouldn't have been run 100 times.which produces:Benchmark Report
================

Benchmark Sqrt
--------------

         name | rank | runs |     mean |        sd | timesBaseline
--------------|------|------|----------|-----------|--------------
 pow operator |    1 |  100 | 0.004163 | 0.0001469 |           1.0
sqrt function |    2 |  100 | 0.004584 |  0.000379 | 1.10126499627
 pow function |    3 |  100 | 0.005496 | 0.0002709 | 1.32031909568

Benchmark Sqrt on a larger range
--------------------------------

         name | rank | runs |   mean |       sd | timesBaseline
--------------|------|------|--------|----------|--------------
 pow operator |    1 |   50 | 0.1265 | 0.004772 |           1.0
sqrt function |    2 |   50 | 0.1375 | 0.004729 |    1.08705384
 pow function |    3 |   50 | 0.1649 | 0.003935 | 1.30357975885

Each of the above 450 runs were run in random, non-consecutive order by
`benchmark` v0.1.4 (http://jspi.es/benchmark) with Python 2.7.1
Darwin-11.3.0-x86_64 on 2012-04-17 19:43:01.More examples are available in the example folder or visithttp://jspi.es/benchmarkfor more information.
Package: benchmark-4dn
The repo contains a benchmarking script for some of the CWL workflows used by 4DN-DCIC (https://github.com/4dn-dcic/pipelines-cwl), that returns total space, mem and CPUs required per given input size and a recommended AWS EC2 instance type.Example usage of benchmarking scriptimporting the modulefromBenchmarkimportrunasBmd5app_name='md5'input_json={'input_size_in_bytes':{'input_file':20000}}B.benchmark(app_name,input_json){'aws': {'recommended_instance_type': 't2.xlarge', 'EBS_optimized': False, 'cost_in_usd': 0.188, 'EBS_optimization_surcharge': None, 'mem_in_gb': 16.0, 'cpu': 4}, 'total_size_in_GB': 14.855186462402344, 'total_mem_in_MB': 13142.84375, 'min_CPU': 4}fastqc-0-11-4-1app_name='fastqc-0-11-4-1'input_json={'input_size_in_bytes':{'input_fastq':20000},'parameters':{'threads':2}}B.benchmark(app_name,input_json){'recommended_instance_type': 't2.nano', 'EBS_optimized': False, 'cost_in_usd': 0.006, 'EBS_optimization_surcharge': None, 'mem_in_gb': 0.5, 'cpu': 1}bwa-memapp_name='bwa-mem'input_json={'input_size_in_bytes':{'fastq1':93520000,'fastq2':97604000,'bwa_index':3364568000},'parameters':{'nThreads':4}}B.benchmark(app_name,input_json){'aws': {'cost_in_usd': 0.188, 'EBS_optimization_surcharge': None, 'EBS_optimized': False, 'cpu': 4, 'mem_in_gb': 16.0, 'recommended_instance_type': 't2.xlarge'}, 'total_mem_in_MB': 12834.808349609375, 'total_size_in_GB': 15.502477258443832, 'min_CPU': 4}To use Benchmark in from other places, install it as below.pip install Benchmark-4dnorpip install git+git://github.com/SooLee/Benchmark.gitNote: From0.5.3we have a new function that takes in cpu and memory and returns a sorted list of instance dictionaries.get_instance_types(cpu=1, mem_in_gb=0.5, instances=instance_list(), top=10, rank='cost_in_usd')Keys in each instance dictionary:'cost_in_usd', 'mem_in_gb', 'cpu', 'instance_type', 'EBS_optimized', 'EBS_optimization_surcharge'
Package: benchmarkai-client-lib
A small library for publishing metrics when running a workflow using the Benchmark AI infrastructure.Installationpip install benchmarkai-client-libAPIThe library provides a single method:def emit(metrics: Dict[str, float])PS: This library also works with Python 2, which has no type system.Example usagefrombenchmarkaiimportemitdefmy_fancy_benchmark():importtimebegin=time.time()forepochinrange(100):execute_training()elapsed_time=time.time()-beginemit({"alexnet.batchsize16.latency":elapsed_time})PerformanceAs the author you don't need to worry aboutemit()being slow since it does 2 things:serializesmetricsinto JSONwrites this string into a file (see the internal details)So it will be as slow as these operations can be, which are quite fast.Accepted valuesThe dictionary provided by themetricsparameter is in the form:"metric-name" => valuemetric-nameis astringvalueis afloatRunning testsTo set up the environment to run the tests, the filetest-environment.ymlis provided. This conda environment file
has all dependencies required to run the tests.$ conda env create -f test-environment.yml
$ conda activate benchmark-ai-client-library-tests
$ toxInternal detailsModesThe library can be used in these modes as of now:Stdout printing (this is thedefaultmode)Benchmark AI infrastructureTo control which behavior is desired, the environment variableBENCHMARK_AIcan be set to one of the values:stdout: Thedefaultmode. In this configuration, theemit()method is equivalent to:defemit(metrics):importjsonprint(json.dumps(metrics))fifo: When running inside Benchmark AI infrastructure this is the mode used.Fifo modeIn this mode, theemit()method writes JSON string to a file that was created as anamed pipe. It doesNOTwrite tostdout.Keep in mind that the client library doesNOTcreate thenamed pipefile, this is delegated to the
"daemon process", which will be running alongside the benchmark process in the Benchmark AI infrastructure.The daemon process is responsible to:create thenamed piperead the contents from this pipepost the contents of this pipe to the metrics servers (eg.: Elastic Search, Cloudwatch, etc.)The first time thatemit()is called, it checks for the existence of the file and uses that. If it doesn't exist, it
waits for some time before giving up and throwing an exceptionFifoNotCreatedInTimeError.Fifo mode configurationsThese environment variables control the behavior of the client library:BENCHMARK_AI_FIFO_FILEPATH: The path to the file that will be created as anamed pipe.
Default value:/tmp/benchmark-ai-fifoBENCHMARK_AI_FIFO_MAX_WAIT_TIME: The maximum amount of time that the client library will wait for thenamed pipe fileto be created.
Type isfloat, unit isseconds, default value:10.BENCHMARK_AI_FIFO_WAIT_TIME_STEP: The amount of time between each check that the client library will wait before
checking for the existence of thenamed pipe fileagain.
Type isfloat, unit isseconds, default value:0.5.IMPORTANT: These environment variables are for internal usage of the Benchmark AI infrastructure. They are listed
here for documentation purposes only.
Package: benchmark-django-rest-framework
Benchmark Django Rest Framework is a framework based on Django Rest Framework. It supports a fairly efficient
way to develop Django Rest API. By default, only two-line python codes can create a powerful Django Rest API
view. It supports complex and various HTTP requests to search / add / update / delete the models in a most easy
way. And this framework also has a most important feature that the keywords defined in it are so familiar to
you, if you are good at Django. Because this keywords is same as the keywords or names of functions in Django.
Therefore, you can know how to use the framework quickly.
Package: benchmark-imports
benchmark-importsA CLI tool to record how much time it takes to import each dependency in a Python project. Use it when the import time for your code matters.UsageInstall:python3-mpipinstallbenchmark-importsUse:python3-mbenchmark_importsmy_module_nameFor example, measure import time for numpy:python3-mpipinstallnumpy
python3-mbenchmark_importsnumpyTroubleshootingTo be able to import your module and all its dependencies, the tool has to be installed in the same environment as the tested module and all its dependencies.Keep in mind that the tool actually imports and executes the given module and all its dependencies. So, don't run it on untrusted code or code that can break something at import time.The tool will report you at the end the list of errors that occured in some modules when importing them but were suppressed by other modules. It doesn't mean something is broken. For example, it can indicate that the library has an optional dependency that it tries to import but just ignores if unavailable. However, that means that on some environment the module will be successfully imported, and so the import time may be different.Improving import timeWhen you identified the slow modules, this is what you can do:Decrease coupling. "A little copying is better than a little dependency". For example, if you importnumpyonly to use a single small function (likenumpy.sign), just implement the function by yourself.Use local imports. The best practice is to have imports on the top-level of the file. However, if this is a slow module that is used only in one function which isn't called too often, just move the import into the function body. It won't make the function much slower (well, except when you call it in the first time) because Python caches all imports insys.modules.Use lazy imports. The idea is about the same as with function-local imports: the module will be actually imported and executed only when you try to use it in the first time. It can be achieved either withdeferred-importlibrary orImplementing lazy importssnippet from theimportlibdocumentation.Make type annotations lazyby addingfrom __future__ import annotationsat the beginning of each file (seePEP 563).If something is imported only to be used in type annotations, move the import insideif TYPE_CHECKINGblock.And as a general rule, don't optimize something until you prove it is slow. This is why this tool exists.Module typeFor each reported module, the tool will show you one of the following types:rootis the original module you benchmark (or one of its parent modules).projectis a child module of the root module.dependencyis a direct dependency of one of the modules in the project.transitiveis a dependency of one of the dependencies.
Package: benchmarking-qrc
Benchmarking QRCThis repository contains the code to analysis the role of particle statistics in Quantum Reservoir Computing.InstallationYou can either run this code interactively with Jupyter Notebooks in Google Colab (ADD LINK) or installing the latest version in your local computer by typing in terminal:pipinstallbenchmarking_qrcUsageAlthough, any user is welcome to contribute to the project. The main objective of this repository is to facilitated the transparency and reproducibility of scientific results.From your terminal it is possible to generate the results with:cdsrc/python_scripts
python./memory_capacity42"fermion"1ij3000120030010-s15-e1The cell above measures the memory capacity of a reservoir with 4 fermions. For more details please check the last section in this tutorial (ADD LINK)Package structureTODOHow to citeTODO: Add bibtex link
Package: benchmark-mi
Benchmarking Mutual InformationBMI is the package for estimation of mutual information between continuous random variables and testing new estimators.Documentation:https://cbg-ethz.github.io/bmi/Source code:https://github.com/cbg-ethz/bmiBug reports:https://github.com/cbg-ethz/bmi/issuesPyPI package:https://pypi.org/project/benchmark-miGetting startedWhile we recommend taking a look at thedocumentationto learn about full package capabilities, below we present the main capabilities of the Python package.
(Note that BMI can also be used to test non-Python mutual information estimators.)You can install the package using:$pipinstallbenchmark-miAlternatively, you can use the development version from source using:$pipinstall"bmi @ https://github.com/cbg-ethz/bmi"Note: BMI usesJAXand by default installs the CPU version of it.
If you have a device supporting CUDA, you caninstall the CUDA version of JAX.Now let's take one of the predefined distributions included in the benchmark (named "tasks") and sample 1,000 data points.
Then, we will run two estimators on this task.importbmitask=bmi.benchmark.BENCHMARK_TASKS['multinormal-dense-2-5-0.5']print(f"Task{task.name}with dimensions{task.dim_x}and{task.dim_y}")print(f"Ground truth mutual information:{task.mutual_information():.2f}")X,Y=task.sample(1000,seed=42)cca=bmi.estimators.CCAMutualInformationEstimator()print(f"Estimate by CCA:{cca.estimate(X,Y):.2f}")ksg=bmi.estimators.KSGEnsembleFirstEstimator(neighborhoods=(5,))print(f"Estimate by KSG:{ksg.estimate(X,Y):.2f}")CitingIf you find this code useful in your research, consider citingour manuscript:@misc{beyond_normal2023,
      title={Beyond Normal: On the Evaluation of Mutual Information Estimators}, 
      author={Paweł Czyż and Frederic Grabowski and Julia E. Vogt and Niko Beerenwinkel and Alexander Marx},
      year={2023},
      eprint={2306.11078},
      archivePrefix={arXiv},
      primaryClass={stat.ML}
}
Package: benchmarkstt
AboutThis is a command line tool for benchmarking Automatic Speech Recognition engines.It is designed for non-academic production environments, and prioritises ease of use and relative benchmarking over scientific procedure and high-accuracy absolute scoring.Because of the wide range of languages, algorithms and audio characteristics, no single STT engine can be expected to excel in all circumstances. For this reason, this tool places responsibility on the users to design their own benchmarking procedure and to decide, based on the combination of test data and metrics, which engine is best suited for their particular use case.Usage examplesReturns the number of word insertions, deletions, replacements and matches for the hypothesis transcript compared to the reference:benchmarkstt --reference reference.txt --hypothesis hypothesis.txt --diffcountsReturns the Word Error Rate after lowercasing both reference and hypothesis. This normlization improves the accuracy of the Word Error Rate as it removes diffs that might otherwise be considered errors:benchmarkstt -r reference.txt -h hypothesis.txt --wer --lowercaseReturns a visual diff after applying all the normalization rules specified in the config file:benchmarkstt -r reference.txt -h hypothesis.txt --worddiffs --config confFurther informationThis is a collaborative project to create a library for benchmarking AI/ML applications. It was created in response to the needs of broadcasters and providers of Access Services to media organisations, but anyone is welcome to contribute. The group behind this project is the EBU’sMedia Information Management & AI group.Currently the group is focussing on Speech-to-Text, but it will consider creating benchmarking tools for other AI/ML services.For general information about this project, including themotivationsandguiding principles, please see the projectwikiTo install and start using the tool, go to thedocumentation.
Package: benchmark-templates
AboutWorkflow Templatesare parameterized workflow specifications for theReproducible Open Benchmarks for Data Analysis Platform (ROB). Workflow templates are motivated by the goal to allow users to run pre-defined data analytics workflows while providing their own input data, parameters, as well as their own code modules. Workflow templates are inspired by, but not limited to, workflow specifications for theReproducible Research Data Analysis Platform (REANA).More InformationTheWorkflow Templates Sectionprovides further information about templates and their syntax.Benchmark Templatesare an extension of the base templates. These templates are used by theReproducible Benchmark Engineto run benchmark workflows and maintain benchmark results.
Package: benchmark-utils
Benchmark utilsUtils for benchmark - wrapper over python timeit.Tested on python 3.7 - 3.12InstallInstall from pypi:pip install benchmark_utilsOr install from github repo:pip install git+https://github.com/ayasyrev/benchmark_utils.gitBasic use.Lets benchmark some (dummy) functions.fromtimeimportsleepdeffunc_to_test_1(sleep_time:float=0.1,mult:int=1)->None:"""simple 'sleep' func for test"""sleep(sleep_time*mult)deffunc_to_test_2(sleep_time:float=0.11,mult:int=1)->None:"""simple 'sleep' func for test"""sleep(sleep_time*mult)Let's create benchmark.frombenchmark_utilsimportBenchmarkbench=Benchmark([func_to_test_1,func_to_test_2],)benchoutputBenchmark(func_to_test_1, func_to_test_2)Now we can benchmark that functions.# we can run bench.run() or just:bench()outputFunc name  | Sec/runfunc_to_test_1:0.100.0%func_to_test_2:0.11-8.9%We can run it again, all functions, some of it, exclude some and change number of repeats.bench.run(num_repeats=10)outputFunc name  | Sec/runfunc_to_test_1:0.100.0%func_to_test_2:0.11-9.3%After run, we can prunt results - sorted or not, reversed, compare results with best or not.bench.print_results(reverse=True)outputFunc name  | Sec/runfunc_to_test_2:0.110.0%func_to_test_1:0.1010.2%We can add functions to benchmark as list of functions (or partial) or as dictionary:{"name": function}.bench=Benchmark([func_to_test_1,partial(func_to_test_1,0.12),partial(func_to_test_1,sleep_time=0.11),])benchoutputBenchmark(func_to_test_1, func_to_test_1(0.12), func_to_test_1(sleep_time=0.11))bench.run()outputFunc name  | Sec/runfunc_to_test_1:0.100.0%func_to_test_1(sleep_time=0.11):0.11-8.8%func_to_test_1(0.12):0.12-16.4%bench=Benchmark({"func_1":func_to_test_1,"func_2":func_to_test_2,})benchoutputBenchmark(func_1, func_2)When we run benchmark script in terminal, we got pretty progress thanks to rich. Lets run example_1.py from example folder:BenchmarkIterWith BenchmarkIter we can benchmark functions over iterables, for example read list of files or run functions with different arguments.deffunc_to_test_1(x:int)->None:"""simple 'sleep' func for test"""sleep(0.01)deffunc_to_test_2(x:int)->None:"""simple 'sleep' func for test"""sleep(0.015)dummy_params=list(range(10))frombenchmark_utilsimportBenchmarkIterbench=BenchmarkIter(func=[func_to_test_1,func_to_test_2],item_list=dummy_params,)bench()outputFunc name  | Items/secWe can run it again, all functions, some of it, exclude some and change number of repeats.
And we can limit number of items withnum_samplesargument:bench.run(num_samples=5)MultiprocessingBy default we tun functions in one thread.But we can use multiprocessing withmultiprocessing=Trueargument:bench.run(multiprocessing=True)It will use all available cpu cores.
And we can usenum_workersargument to limit used cpu cores:bench.run(multiprocessing=True, num_workers=2)bench.run(multiprocessing=True,num_workers=2)outputFunc name  | Items/sec0.1022316165981465/0.09319195459829643output1.0970004550158379
Package: benchmarx
BenchmarxBenchmarx provides a convenient interface for benchmarking optimization
methods. This package provides a simple implementation of benchmarking
functions for optimization tasks and provides the ability to use standard
methods from jaxopt as well as to implement your own method. The results
of the experiments are saved in all details in a json file, which can be
used to visualize the data in graphs. Refer to the examples for details.InstallationTo install the latest release of Benchmarx, use the following command:$pipinstallbenchmarxAlternatively, it can be installed from sources with the following command:$pythonsetup.pyinstallExamplesRefer to the examples provided in the repository for details on how to use this package.Gradient Descent on Quadratic problemGradient Descent with custom Line SearchYour own custom methodStochastic Gradient Descent on Quadratic ProblemQuadratic Problem based on real dataNeural Network trainingContributionContributions are welcome! If you have any suggestions or issues, feel free to open an issue or submit a pull request.
Package: benchpress
Quick StartFire up your terminal, and:# Install using PyPi
pip install benchpress --user

# Make the Benchpress binaries available
export PATH=$PATH:$HOME/.local/binSpecify what to benchmark by implementing a Python script that generates commands:import benchpress as bp
from benchpress.suite_util import BP_ROOT

scripts = [
    ('X-ray',  'xraysim',  ["10*10*1", "20*10*1"]),
    ('Bean',   'galton_bean_machine',  ["10000*10", "20000*10"]),
]

cmd_list = []
for label, name, sizes in scripts:
    for size in sizes:
        full_label = "%s/%s" % (label, size)
        bash_cmd = "python {root}/benchmarks/{script}/python_numpy/{script}.py {size}" \
                    .format(root=BP_ROOT, script=name, size=size)
        cmd_list.append(bp.command(bash_cmd, full_label))

# Finally, we build the Benchpress suite, which is written to `--output`
bp.create_suite(cmd_list)And run the script:$ python suites/simple_example.py --output my_benchmark.json
Scheduling 'X-ray/10*10*1': 'python xraysim/python_numpy/xraysim.py --size=10*10*1'
Scheduling 'X-ray/20*10*1': 'python xraysim/python_numpy/xraysim.py --size=20*10*1'
Scheduling 'Bean/10000*10': 'python galton_bean_machine/python_numpy/galton_bean_machine.py --size=10000*10'
Scheduling 'Bean/20000*10': 'python galton_bean_machine/python_numpy/galton_bean_machine.py --size=20000*10'
Writing suite file: my_benchmark.jsonThe result is a JSON fileresults.jsonthat encapsulate the commands that make up the benchmark suite.
Now, usebp-runto run the benchmark suite:$bp-run results.json
Executing 'X-ray/10*10*1'
Executing 'X-ray/20*10*1'
Executing 'Bean/10000*10'
Executing 'Bean/20000*10'Finally, let’s visualize the results in ASCII:$bp-cli results.json
X-ray/10*10*1: [0.013303, 0.013324, 0.012933] 0.0132 (0.0002)
X-ray/20*10*1: [0.108884, 0.105319, 0.105392] 0.1065 (0.0017)
Bean/10000*10: [0.002653, 0.002553, 0.002616] 0.0026 (0.0000)
Bean/20000*10: [0.005149, 0.005088, 0.005271] 0.0052 (0.0001)Or as a bar chart:$bp-chart results.json --output results.pdf
Writing file 'results.pdf' using format 'pdf'.
Package: benchsuite.backends
Benchmarking Suite BackendsThe standard backends for the Benchmarking SuiteDocumentationUser and technical documentation is available athttp://benchmarking-suite.readthedocs.io/.SupportFor bugs, enhancements or support go tohttps://github.com/benchmarking-suite/benchsuite-issues/issues/.LegalThe Benchmarking Suite is released under theApache 2.0license.Copyright © 2014-2018 Engineering Ingegneria Informatica S.p.A. All rights reserved.This work has received partial funding from the European Commission under grant agreements No. FP7-317859 and No. H2020-732258
Package: benchsuite.cli
Benchmarking Suite Command Line InterfaceA command line interface for the Benchmarking SuiteDocumentationUser and technical documentation is available athttp://benchmarking-suite.readthedocs.io/.SupportFor bugs, enhancements or support go tohttps://github.com/benchmarking-suite/benchsuite-issues/issues/.LegalThe Benchmarking Suite is released under theApache 2.0license.Copyright © 2014-2018 Engineering Ingegneria Informatica S.p.A. All rights reserved.This work has received partial funding from the European Commission under grant agreements No. FP7-317859 and No. H2020-732258
Package: benchsuite.core
Benchmarking Suite CoreThe core library of the Benchmarking SuiteDocumentationUser and technical documentation is available athttp://benchmarking-suite.readthedocs.io/.SupportFor bugs, enhancements or support go tohttps://github.com/benchmarking-suite/benchsuite-issues/issues/.LegalThe Benchmarking Suite is released under theApache 2.0license.Copyright © 2014-2018 Engineering Ingegneria Informatica S.p.A. All rights reserved.This work has received partial funding from the European Commission under grant agreements No. FP7-317859 and No. H2020-732258