Saravanan666/pypi-org huggingface dataset:
Package: batch-prediction-pipeline-self
Batch Prediction PipelineCheck outLesson 3on Medium to better understand how we built the batch prediction pipeline.Also, check outLesson 5to learn how we implemented the monitoring layer to compute the model's real-time performance.Install for DevelopmentThe batch prediction pipeline uses the training pipeline module as a dependency. Thus, as a first step, we must ensure that the training pipeline module is published to our private PyPi server.NOTE:Make sure that your private PyPi server is running. Check theUsage sectionif it isn't.Build & publish thetraining-pipelineto your private PyPi server:cdtraining-pipeline
poetrybuild
poetrypublish-rmy-pypicd..Install the virtual environment forbatch-prediction-pipeline:cdbatch-prediction-pipeline
poetryshell
poetryinstallCheck theSet Up Additional ToolsandUsagesections to seehow to set uptheadditional toolsandcredentialsyou need to run this project.Usage for DevelopmentTo start batch prediction script, run:python-mbatch_prediction_pipeline.batchTo compute the monitoring metrics based, run the following:python-mbatch_prediction_pipeline.monitoringNOTE:Be careful to complete the.envfile and set theML_PIPELINE_ROOT_DIRvariable as explained in theSet Up the ML_PIPELINE_ROOT_DIR Variablesection of the main README.
Package: batch-processing-discovery
Batch Processing DiscoveryThis technique takes as input an event log (pd.DataFrame) recording the execution of the activities of a process with enabled, start and end
timestamps, as well as the resource who performed it, and discovers which activity instances have been executed in a batch, and the
characteristics of this batch processing.The discovered characteristics are, for each batch processing:Theactivitybeing executed.Theresourcesinvolved in this batch processing.Thetype of batchprocessing (sequential, concurrent or parallel). In case of more than one type, the most common.Thefrequencyof that activity occurring as part of a batch.Thedistribution of batch sizes, i.e., for each size, the number of activity instances executed as a batch with that size.Thedistribution of durations, i.e., for each batch size, the scaling factor of the duration of the activity instances processed in
that batch. For example, if the activity is processed in a 2-size batch, each activity instance lasts x0.7 what it lasts executed
individually.Thefiring rulesthat better describe the start of the batch.RequirementsPython v3.9.5+PIP v21.1.2+Python dependencies: The packages listed inrequirements.txt.Basic UsageHere we provide a simple example of use with default configuration (seefunction documentationfor more parameters):importpandasaspdfrombatch_processing_discovery.batch_characteristicsimportdiscover_batch_processing_and_characteristicsfrombatch_processing_discovery.configimportDEFAULT_CSV_IDS# Read event logevent_log=pd.read_csv("path/to/event/log.csv.gz")# Discover batch processing activities and their characteristicsbatch_characteristics=discover_batch_processing_and_characteristics(event_log=event_log,log_ids=DEFAULT_CSV_IDS)Discover only batch processing behaviorIn case of being interested only in discovering batch processing behavior, the following example applies (seefunction documentationfor more parameters):importpandasaspdfrombatch_processing_discovery.configimportDEFAULT_CSV_IDSfrombatch_processing_discovery.discoveryimportdiscover_batches# Read event logevent_log=pd.read_csv("path/to/event/log.csv.gz")# Discover batch processing activities and their characteristicsbatched_event_log=discover_batches(event_log=event_log,log_ids=DEFAULT_CSV_IDS)Get batch characteristics with already set batch processing behaviorIn case of being interested only in getting the batch characteristics, based on an event log with already set batch behavior, the following
example applies (seefunction documentationfor more parameters):importpandasaspdfrombatch_processing_discovery.batch_characteristicsimportdiscover_batch_characteristicsfrombatch_processing_discovery.configimportDEFAULT_CSV_IDS# Read event logevent_log=pd.read_csv("path/to/event/log_with_batch_info.csv.gz")# Discover batch processing activities and their characteristicsbatch_characteristics=discover_batch_characteristics(event_log=event_log,log_ids=DEFAULT_CSV_IDS)** No enabled time availableIn case of not enabled time available in the event log, consider
usingthis Python libraryto estimate them.
Package: batch-processor
batch_processorinstallpipinstallbatch_processorquickstartfromthreadingimportThreadfrombatch_processorimportBatchProcessordefbatch_func(batch):return[v+vforvinbatch]defcreate_bulk_request(n,processor):print(n,processor.process(n))processor=BatchProcessor(batch_func,worker_num=2,batch_size=32)threads=[]foriinrange(200):t=Thread(target=create_bulk_request,args=(i,self.processor))t.start()threads.append(t)fortinthreads:t.join()This piece of code generates discrete 200 incomming requests that can be processed in batches whose size is 32. There are 2 workers processing these requests. They take batches and double each integers in the batch, then return results in batches.testrun test casespythontests/test_runner.pygenerate coveragecoverageruntests/test_runner.pygenerate coverage htmlcoveragehtmlbuildbuild the distributionpythonsetup.pysdistbdist_wheelupload to registrypython-mtwineuploaddist/*
Package: batchpy
A python package to run large or small batches of similar calculations and storing the results so no double calculations are performed.
Batches of computation with lots of result data can be performed as results are stored on disk and only loaded in memory when required.
Furthermore, methods for easily retrieving runs with certain parameters are available to ease post-processing and visualizing data.Installationrequires:numpyTo install run:pip install batchpyUsageThequickstartexample in the online documentation gives a good overview of basic usage.
Package: batchqueue
batchqueueThis module provides an extension to the basicqueue.Queueclass in order to
handle situations where incoming data for the queue tends to come in bursts to
be processed as a group.Example Usage:defworker(queue=None):self.assertIsNotNone(queue)whileTrue:items=queue.get_batch()foriinitems:# Do stuffqueue.task_done()# Process a batch any time there's at least a 700ms lull in trafficself.q=BatchQueue(lull_time=700)# turn-on the worker threadthreading.Thread(target=worker,daemon=True,kwargs={'queue':self.q}).start()# send ten task requests to the worker, pausing a random number of# milliseconds (up to 1000)foriteminrange(10):sleep(random.randrange(0,1000)/1000.0)self.q.put(item+1)# block until all tasks are doneself.q.join()
Package: batch-requests
Batch RequestsMake batch http requests.UsageInstall packagepip install batch-requestsorconda install batch-requestsorpip3 install batch-requestsfrombatchimportBatchRequestbatch_request_manager=BatchRequest(['http://httpbin.org/ip','http://httpbin.org/get',])batch_request_manager.run()responses=batch_request_manager.responsesprint(responses)
Package: batchrun
batchrunA simple workflow for launching, tracking, and resuming jobs with varying argument values and parameter sweeps.QuickstartLaunch multiple jobsGiven arunfilecontaining a list of shell commands:sleep1;echo"First!"sleep1;echo"Second!"sleep1;echo"Third!"we can launch them line by line as follows:batchrunlaunchrunfile--n_jobs2Batchrun will run the commands in parallel withn_jobsworkers and keep track of the jobs.
By default, it will write the logs of each command toruns/runfile/logs/<hash of command>, and keep the
metadata inruns/runfile/metadata.json. The metadata file contains the mapping from the commands
to their hashes, and status information about the jobs.Resuming where we left and adding new jobsIn the case of shutdown, batchrun will by default resume the jobs which have not finished.
Similarly, if we add more jobs to the runlist, batchrun will only run the new ones. The way
batchrun tells if a job is new or not is by the command itself. If the command has changed in any
way, e.g., the order of arguments is different, batchrun will assume it's a new job.Create a parameter sweepGiven a specification of a parameter value gridgridspec.yml:program:python3 my_script.pyparameters:alpha:values:-1-2-5beta:values:[0.1,0.25,0.5]gamma:value:100we can generate a list of commands corresponding to all possible parameter value combinations:python3my_script.py--alpha=1--beta=0.1--gamma=100python3my_script.py--alpha=1--beta=0.25--gamma=100python3my_script.py--alpha=1--beta=0.5--gamma=100python3my_script.py--alpha=2--beta=0.1--gamma=100...by running the following command:batchrunsweepgridspec.ymlThe runfile containing the commands by default will be written togridspec.runfile. You can then
launch the sweep using thebatchrun launch gridspec.runfileas detailed before.
Package: batch-run-cmd
batch-run-cmd1 IntroductionSeehttps://github.com/linzhi2013/batch-run-cmd.This is a Python3 package which will read command lines from an input file,
and run each line of command one by one, and check their exit status.More importantly, if one command fails, you can just jump to next command
(-eoption). Furthermore, you can check for the existences of result files
(-Eoption).2 Installationwithpip$ pip install batch-run-cmdThere will be a commandbatch_run_cmdcreated under the same directory as yourpipcommand.3 Usagerunbatch_run_cmd$ batch_run_cmd
usage: batch_run_cmd [-h] [-f <file>] [-e <int> [<int> ...]] [-E <str>]

    To run command line by line, and check each exit status.

    Then optionally check for the existences of result files.

    Each line format:

    echo "Hello world" && echo "Where are you?" ;Result-files: /path/to/resultfile1 /path/to/resultfile2

    Internally, each line will be split by ";Result-files:" delimiter,
    and the part before ";Result-files:" will be treated as a shell command,
    the part after ";Result-files:" will be regardeds as result files.

    Multiple result files are allowed, seperated by blank space(s).

    See https://github.com/linzhi2013/batch-run-cmd for more details.


optional arguments:
  -h, --help            show this help message and exit
  -f <file>             commands in a file.
  -e <int> [<int> ...]  continue On Return Code for a shell command. [[0]]
  -E <str>              continue On: 'a': all files must exist; 'e': at least
                        one file must exist; 'o': omitted, i.e. do not check
                        for existences of files [o]4 CitationCurrently I have no plan to publishbatch-run-cmd.
Package: batch-sen2cor
Batch Sen2corDescription.The package package_name is used to:
Processing:
- batch processing of sentinel 2 image using sen2cor atmospheric correction
Sen2cor:
- the sen2cor aplication fromESAInstallationUse the package managerpipto install package_namepipinstallbatch_sen2corUsageRun the program in the folder where the SENTINEL-2 image is.>>>frombatch_sen2cor.processingimportbatch_sen2cor>>>batch_sen2cor.sentinel_sen2cor()AuthorMateus MirandaLicenseMIT
Package: batch-table-storage
Async Microsoft Azure CosmosDB Table SDK for Python with batching functionalityThis project provides a asynchronous wrapper for the Azure cosmosdb TableService.
The wrapper lets you insert/merge/replace/delete lists of entities usingaioify.
You can give it any number of PartitionKeys, they will be processed in different batches.For documentation on the base package please seeazure-cosmosdb-tablepackage. (https://pypi.org/project/azure-cosmosdb-table/)FeaturesAutomatically split lists of entities into batches based on PartitionKeyAutomatically chunk entities into sub-lists for correct batch sizesBatch Insert Entities asyncBatch Update Entities asyncBatch Merge Entities asyncBatch Delete Entities asyncBatch Insert or Replace Entities asyncBatch Insert or Merge Entities asyncGetting StartedTo install via the Python Package Index (PyPI), type:pip install batch-table-storageMinimum Requirementsazure-cosmosdb-table==1.0.6Code SampleImport the wrapper classfrom batch_table_service import BatchTableServiceInstantiate the serviceservice = BatchTableService(account_name='storageAccount', account_key='x')Create some entitiesentities = [
    {'PartitionKey': 'person', 'RowKey': '1', 'name': 'John'},
    {'PartitionKey': 'pet', 'RowKey': '1', 'name': 'Cat'}
]Import asyncio:import asyncioCreate and commit the batches async:asyncio.run(
    service.batch_insert_entities(table_name='tableA', entities=entities)
)Learn MoreMicrosoft Azure CosmosDB Table SDK for Python:https://pypi.org/project/azure-cosmosdb-table/
Package: batch-tar
batch-tar1 Introductionbatch-taris a tool to tar (and compress) files or directories in batch mode.2 Installationpip install batch-tarThere will be a commandbatch_tarcreated under the same directory as yourpipcommand.3 Usage$ batch_tar

usage: batch_tar [-h] [-l <file>] [-L [<str> [<str> ...]]]
                 [-workdir <directory>] [-tarfile <str>] [-regex <str>]
                 [-compress {z,j}] [-rm] [-p] [--version]

To tar/compress files/directories in batch mode. By Guanliang Meng, see
https://github.com/linzhi2013/batch_tar. version: 0.1

optional arguments:
  -h, --help            show this help message and exit
  -l <file>             path list to be dealed with
  -L [<str> [<str> ...]]
                        basenames of files/subdirecotries in `-workdir`. If
                        `-workdir` was not specified, `-workdir ./` is
                        assumed.
  -workdir <directory>  working directory. must specify `-regex` together
  -tarfile <str>        group all the files/subdirecotries into one file.tar
                        (.bz2 or .gz). only works with `-L` and `-workdir`
  -regex <str>          deal with only the files/subdirecotries in `-l` or
                        `-workdir` whose basenames match the regular
                        expression
  -compress {z,j}       For the subdirecotries, when executing `tar -cf`, also
                        add parameter `z` or `j` for `tar` command. The files
                        will always be gzipped if this option was not set.
  -rm                   delete the files/subdirecotries when tar finishs.
  -p                    only print out the commands, not run. then you can use
                        qsub-sge.pl to submit multiple jobs. Useful to handle
                        a lot of files because I run in single thread mode
                        only! [False]
  --version             show program's version number and exitAuthorGuanliang MENGCitationCurrently I have no plan for publishing this script.
Package: batch-tr
batch_tr[]batch translate long text and list of texts at 10000 chars/sUse Itfrombatch_tr.batch_trimportbatch_trres=batch_tr("very long text "*10000)print(res)# to chinese/zhprint(batch_tr("test me",to_lang="de"))# long listres=batch_tr(("very long text\n"*10000).splitlines())# equally long list of translated text# known to work for to_lang="zh" (default)print(res)
Package: batchutils
batchutilsOverviewMain FeaturesInstallationUsageUse the following function as is.
Replace the paths with[j1.ofile, j2.ofile, ...]b=hb.Batch(default_image='hailgenetics/hail:0.2.77,backend=service_backend,name=f'combine-tsvs')combined=batch_combine2(combine_tsvs_with_headers,combine_compressed_tsvs_with_headers,b,'combine-tsvs-with-headers',['gs://bucket/input1.tsv',...],final_location='gs://bucket/final-file.tsv.gz',branching_factor=25,suffix='.tsv.bgz')final_locationis where the output file is.CiteMaintainerTarjinder Singh @ tsingh@broadinstitute.orgAcknowledgementsRelease Notes
Package: batchx
ContentsThebatchxpackage contains thebxmain application module, that
offers full access to the BatchX gRPC API from Python code.How to use# Import the modulefrombatchximportbx# Connect to the BatchX servers# (environment variables `BATCHX_ENDPOINT` and `BATCHX_TOKEN` expected)bx.connect()# Instantiate service classorg_service=bx.OrganizationService()# Create data requestrequest=org_service.GetOrganizationRequest(organization="batchx")# Call the RPCresponse=org_service.GetOrganization(request);print(response)See alsoBatchX documentation:https://docs.batchx.ioPython protobuf messages:https://developers.google.com/protocol-buffers/docs/reference/python-generated
Package: batcore
BaT CoRe: Baselines and testing framework for code reviewer recommendationsFrameworkThis repository provides a framework for testing code review recommendation algorithms.Model creationRecommenderBaseis a simple interface for a generic recommender algorithm. It has afitmethod that trains model on a given data and apredictmethod that predicts reviewers for the given pulls.BanRecommenderBaseis an in interface derived from RecommenderBase that implements by candidate filtering by their activity and ownership of the pull requestDataset creationDatasetBaseis an interface that encapsulates any preprocessing needed for the dataset with a methodpreprocess.GerritLoaderis a class that deals with the data loader from the PullExtractor tool: loads it, reformats it, and performs some high-level preprocessing (empty data removal, bot removal, alias matching, user encoding)StandardDatasetan implementation of DatasetBase. Receives GerritLoader object and performs all the preprocessing that can be needed for a specific modelSpecialDatasetshas implementations of datasets that are unique to a specific recommenderget_gerrit_datasethelping function for dataset creation. Dataset for a specific model can be created asget_gerrit_dataset(gerritloader_object, model_cls=RecommenderBase_class)StreamLoaderBaseis an interface that encapsulates iteration over data. The interface views data as a temporal stream of events and
yields pairs of consecutive segments of the stream - train (any set of events), test (a pull request event)Testing of modelsTesterBaseis class that hastest_recommendermethod. This method takesRecommenderBaseandStreamLoaderBaseand iterates over the dataset, retrains model on new train data, and calculates its predictions.RecTesteran implementation of TesterBase that calculates metrics standard for recommendation systems (mrr, accuracy, etc)SimulTesteran implementation of TesterBase that tester for project-based metrics on a simulated history (Mirsaeedi, Rigby, 2020). Metrics for the simulated testing can be found inCounter.An example of usage can be found inexample.pyBaselinesFramework contains implementations of the following models (baselines\models)RevFinder,Who Should Review My Code?, Thongtanunam et al., 2015Tie,Who Should Review This Change?, Xin Xia et al., 2015ACRec,Who Should Comment on This Pull Request?, Jiang et al. 2017cHRec,Automatically Recommending Peer Reviewers in Modern Code Review, Zanjani et al., 2015CN,What Can We Learn from Code Review and Bug Assignment?, Yu et al., 2015RevRec,Search-Based Peer Reviewers Recommendation in Modern Code Review, Ouni et al., 2016WRC,Automatically Recommending Code Reviewers Based on Their Expertise: An Empirical Comparison, Hannebauer et al., 2016xFinder,Assigning change requests to software developers, Kagdi et al., 2011
Package: batdata
Introductionbatquantis a package that helps developers, researchers to quickly get Vietnam financial data. Since Vietnam is not well-supported by famous platform like Bloomberg, Reuters, Yahoo, Google, we were inspired to develop our own data package that serves Vietnam data.Recent changesv0.1.0 - 20190215:First versionData available: trading data onlyIncoming: v0.2.0:Add more data:reference prices (ceiling, floor, ref price...)stock info (company name, shareout, list of stocks)Exception handlerReference object to lookup for fieldsInstallationpip install batdataUsageInit data objectimport batdata
vnd = batdata.Vnd() # Data from VND apiGetting data# historical data of HPG from 2018-01-01 to 2018-12-31
data = vnd.hist("HPG","close","20180101","20181231")

jsonData = data.json # getting data as json
dfData = data.df # getting data as pandas DataFrame

# fromTime and toTime can also be python datetime objects
fromTime = datetime.datetime(2018,01,01)
toTime = datetime.datetime(2018,12,31)
hpgHist = vnd.hist("HPG","close",fromTime,toTime)

# historical OHLC data of HPG, MSN
hpgHistWeekly = vnd.hist(["HPG","MSN"],["open","high","low","close"],"20180101","20181231",interval="w")

# last price
hpgLast = vnd.curr("HPG","last")TODOFinancial fields referenceMore data
Package: batframework
batFramework & gamejam ProjectWelcome to thebatFrameworkand the accompanyinggamejamproject. This README provides an overview of both the game framework and the specific game project developed using the framework.batFrameworkThebatFrameworkis a Python game development framework based on pygame, designed to streamline the process of creating 2D games. It provides a set of tools and components to handle scenes, transitions, cutscenes, audio, sprites, and more. The framework is built with flexibility in mind, allowing you to focus on game design while abstracting away low-level details.FeaturesScene managementCutscene supportAudio management (music and sound effects with volume control)Entity, sprite, and animated sprite handlingTransition effectsUtility modules (time management, constants, etc.)No external dependency except for pygameExplore batFrameworkInstall Python (version 3.11 or higher) and the latest stable version of pygame-ce.Clone or download this repository.Explore the framework's modules in thebatFrameworkdirectory and integrate them into your own game project.For more detailed information on how to use the framework, refer to the documentation (if available) or explore the source code in thebatFrameworkdirectory.gamejam ProjectThegamejamproject is a specific game developed using thebatFramework. It serves as an example of how the framework can be used to create a game from scratch.Play the gamejam ProjectInstall Python (version 3.10 or higher) and the latest stable version of pygame-ce.Clone or download this repository.Navigate to thegamejamdirectory.Run the game by executing the main script (e.g.,python main.py).Play the game and have fun!Feel free to explore the code in thegamejamdirectory to see how thebatFrameworkis utilized to create the game. You can modify, extend, or use the project as a starting point for your own games.
Package: bathyreq
BathyReqA Python package for querying public bathymetric data sources.
Currently only digital elevation model (DEM) data from theNOAA National Centers for Environmental Information (NCEI)database is supported.
Future development will focus on implementing other sources, such as theGeneral Bathymetric Chart of the Oceans (GEBCO)and NOAA's newBlue Topoproject.InstallationpipinstallbathyreqUsageDownload bathymetric data for a given area:importbathyreqreq=bathyreq.BathyReq()data,lonvec,latvec=req.get_area(longitude=[-117.43,-117.23],latitude=[32.55,32.75])Download bathymetric data for a given set of longitude/latitude pairs:importbathyreqreq=bathyreq.BathyReq()data=req.get_points(longitude=[-117.43,-117.23],latitude=[32.55,32.75])