Saravanan666/pypi-org huggingface dataset:
Package: processio
Python ProcessIORunning functions with multiprocessing hassle freeA nice package to help you run functions with multiprocessing and get their result.Installationpip install processioUsageRun function in own processimporttimefromprocessioimportProcessIOdefget_company_name():# Do work in own processtime.sleep(5)return'Adapted'defdo_some_work():get_name=ProcessIO(get_company_name)# do stuff or run a while loop to wait for resultwhileget_name.doing_work():print('Waiting for process to finnish')# You can also call .result() and the main thread will wait# for the thread to return your result.company_name=get_name.result()print(company_name)# Outputs -> Adaptedif__name__=='__main__':# <- its required to execute the function maindo_some_work()Main commands# Import the modulefromprocessioimportProcessIO# Start your function with or without argumentsvar=ProcessIO(function,args,kwargs)# Wait for the function to finnish or get the result if its finishedvar.result()Optional commands# Check if your thread is still working on the function.# This will return True if the function is not completed.var.doing_work()Use parseIO to save a lot of time on list parsingimporttimefromprocessioimportParseIOdeflist_parser(list):result=0forlineinlist:result+=get_total_amout(line)returnresultdefdo_some_work():# this will split the list into 4 and run the function on# 4 different processes, that in most cases will almost speed# up the work time by 4.# you can define the number of processes you want to run, but as a# default the module runs on the systems cpu cores - 1parser=ParseIO(list_parser,huge_list)# do stuff or run a while loop to wait for resultwhileparser.doing_work():print('Waiting for processes to finnish')# You can also call .result() and the main thread will wait# for the thread to return your result.result=parser.result()# result comes back as list pr process, so in this case we will get# a list with 4 numbers that we can loop thru.print(result)# Outputs -> [1000, 1000, 1000, 1000] <- exampletotal=0forresinresult:total+=resprint(total)# Outputs -> 4000 <- exampleif__name__=='__main__':# <- its required to execute the function maindo_some_work()TestingUse the following command to run tests.python-munittestthreadit.tests.test_threaditChangelog:See CHANGELOG.md
Package: process-logging
how to use1. define configuration and call set_logger method to start logging process in mainfrom process_logging.logger import Logger, HandlerType

logger = Logger()
log_fime_name = "{:%Y-%m-%d}_info.log".format(datetime.now())
log_path = "./logs/"
trf_handler_args = {"filename": log_path + log_fime_name,
                    "when": "midnight",
                    "interval": 1,
                    "encoding": "utf-8"}
format_def = ("[%(levelname)s] | %(process)d | %(asctime)s | %(message)s", "%Y-%m-%d %H:%M:%S")
handler_def = {HandlerType.StreamHandler: {}, HandlerType.TimedRotateFileHandler: trf_handler_args}

if __name__ == "__main__":
    logger.set_logger(name="logger", level=logging.INFO, handler_def=handler_def, format_def=format_def)2. atfer set _logger, you can logging anywhere in your project with Logger()@cbv(project_router)
class Router:
    def __init__(self):
        self._logger = Logger()

    @project_router.get(PREFIX)
    def test_func(self):
        # logging
        self._logger.logging(name="logger", level=logging.INFO, message="logging test")3. result[INFO] | 4147 | 2023-08-30 10:09:29 | logging test
Package: processmanager
processmanagerA multiprocessing process manager for Python programsIt provides:Simple offloading of work to other processor coresState management for the functions in those processesIt is easy to cancel in-progress work, which can be useful if a user clicks 'Cancel' in a GUILogging of exceptions while running offloaded functionsStreaming of results to main process from generator functionsThis can help smooth out IPC load curve by avoiding an all-at-once result transferOriginal use case:GUI program that calls back-end functions for workGUI has a cancel button that should always work, so:the GUI mainloop should not be blockedthe called functions should intermittently check if the user has pressed the cancel buttonWant to avoid running new work when the user wants to cancelWant to work with generators and multiprocessinge.g. send results constantly to main process instead of all at onceInstallation:pip install processmanagerDocumentation:https://processmanager.readthedocs.ioTested for Python >=3.6.5 on Linux (Ubuntu) and Windows 7/10
Package: process-monitor
Process MonitorA python tool that utilizes redis to keep track of long-running processes.
It triggers a signal every n seconds, so that a health check/monitoring API can view
the last signal timestamp.RequirementsredisConfigurationredis connection parameters to be passed directly to the redis client.UsageConsumerfromprocess_monitorimportMonitorclassMyProcess:def__init__(self):self.monitor=Monitor(args)self.monitor.run_signals_worker('process-name')API - Monitorfromprocess_monitorimportReader@route('/check-process')defcheck_process():monitor_reader=Reader(redis_client=Redis(...))results=monitor_reader.read()# {'process-name': {#     'last_signal': '2021-08-03 22:36:55.457416',#     'last_signal_age': 3,#     'info': {#       'x': 1#     }# }}returnResponse(data=results)ExampleCustom MonitorsKinesis Consumer Monitorfromprocess_monitorimportConsumerMonitorclassMyConsumer:def__init__(self):self.monitor=ConsumerMonitor(args)self.monitor.run_signals_worker('consumer-name')defprocess_record(self,record):self.monitor.received_record()...
Package: process-monitoring-package
Process Monitoring Package (Work in Progress ...)The Process Monitoring package provides functionality to monitor a process by its PID and send email notifications when the process completes.HOMEPAGE:https://github.com/ChengranAA/Process_Monitor_PythonInstallationYou can install the package usingpip:pip install process-monitoring-packageUsageShow PIDTo display the PID (Process ID) of the current script, use theshow_pidfunction:fromprocess_monitoring.show_pidimportshow_pidshow_pid()This will print the PID of the current script to the console.Monitor ProcessTo monitor a specific process by its PID and receive email notifications when it is completed, use themonitor_processfunction:fromprocess_monitoring.monitorimportmonitor_processpid=1234# Replace with the PID of the process you want to monitorsender_email="your_email@gmail.com"receiver_email="recipient_email@gmail.com"password="your_email_password"monitor_process(pid,sender_email,receiver_email,password)Replacepidwith the PID of the process you want to monitor. Provide the necessary email configuration, including the sender's email, receiver's email, and email password. This function will continuously check if the process is running. When the process is completed, an email notification will be sent.EmailI recommend creating a separate Gmail email account and utilizing an app-specific password for this package, as it operates through the Gmail server. To access this option, navigate to the security settings within your Gmail account.see this video for details:https://www.youtube.com/watch?v=Sddnn6dpqk0&t=548sContributingContributions are welcome! If you find any issues or have suggestions for improvements, please feel free to open an issue or submit a pull request on the GitHub repository.LicenseThis package is licensed under the MIT License. See theLICENSEfile for more information.
Package: processnamer
ProcessNamerThis library provides an object that is able toName a processStop a process from it's nameGive information about the current processIf you notice any errors, please notify me via:eliservices.server@gmail.comInstallationpip install processnamerExample use:importsysfromtimeimportsleep#Import the lib:fromprocessnamerimportprocessGame#Create object:prg=processGame()#Display information (stdout or print will cause brokenpipe error):sys.stderr.write("I am "+prg.prcname+", and i have pid "+str(prg.pid)+"\n")#Gather information:                  #Gets the name, but it can do this foe any pidinfo=prg.getName(prg.pid)#Get info for pid of this process#Display information:sys.stderr.write("I am "+info+", and i have pid "+str(prg.pid)+"\n")ifprg.prcname=="python":#If this is the original process#Restart this as process imcool:prg.nameStart("imcool",prg.script)sleep(2)prg.nameStop("imcool")#Stop the second processelse:sleep(10)#Second process waits so that it can be stopped
Package: process-nitta
process-nitta実験データの解析ライブラリ使い方インストールpipinstall-Uprocess-nitta実行例fromprocess_nittaimportInstronSamplesample=InstronSample(file_path="../test/csv/instron.csv",width_mm=4,thickness_μm=100,length_mm=10,speed_mm_per_min=10,)sample.get_result_df()exampleに実行例を置いています。
Package: process-nwb
process_nwbFunctions for preprocessing (ECoG) data stored in the NWB formatInstallationprocess_nwb is availble on PyPIpipinstallprocess-nwbFrom sourceTo install, you can clone the repository andcdinto the process_nwb folder.# use ssh$gitclonegit@github.com:BouchardLab/process_nwb.git# or use https$gitclonehttps://github.com/BouchardLab/process_nwb.git
$cdprocess_nwbIf you are installing into an active conda environment, you can run$condaenvupdate--fileenvironment.yml
$pipinstall-e.If you are installing withpipyou can run$pipinstall-e.-rrequirements.txt
Package: processoptim
Food Process Designprocessoptimis a Python library for food process design.InstallationWaterOptim runs under Python 3.6+. To install it withpip, run the following:pip install processoptimTo upgrade it withpip, run the following:pip install --upgrade processoptimBasic usagefrom processoptim.properties.__properties__ import tomato_sauce
ts = tomato_sauce()
print("density 6% dry matter",ts.rho(0.6),"kg/m3")
print("enthalpy 6% dry matter at 60°C",ts.H(0.6,60),"kJ/kg")
#Tomato paste concentration
from processoptim.opu.__evapo__ import evapo
# n : number of effects
# x0 : initial concentration
# xt : target concentration
# L0 : feed flowrate kg/s
# Ts : steam temperature °C
# S : heat transfer area of each effect m2
# h : Overall heat transfer coeff of each effect W/m2/K
# liq : product to concente, inherit from tomato_sauce class
# N : number of tubes in each effect
# d : tube diameter m
# u : velocity in tubes m/s
# l : tube length m
x0 = 0.06
evapo(n=3,x0=.06,xt=.32,L0=5,Ts=120,S=80,h=2,liq=ts,N=100,d=2.5e-2,u=2,l=5)
Package: processpathway
ProcessPathway--------------ProcessPathway is a nifty little tool that lets you play with image processing algorithms instead of wiring up your testbench all day. It is designed to feed a video camera input (a webcam, usually) through any functions that can digest andreturn a numpy matrix (think OpenCV).How to use==========The `example_threshold.py` example, which - as the name suggests - thresholds the the webcam input, is a good examplefor what a typical ProcessPathway pipeline would look like:```pythonimport cv2from processpathway import LiveProcess```First, import the LiveProcess object from processpathway. You might want to also import whatever processing functionsyou need.```def convert_to_grayscale(_frame):_frame = cv2.cvtColor(_frame, cv2.COLOR_BGR2GRAY)return _framedef threshold(_frame):_, _frame = cv2.threshold(_frame, 128, 255, cv2.THRESH_BINARY)return _framedef reconvert_to_bgr(_frame):_frame = cv2.cvtColor(_frame, cv2.COLOR_GRAY2BGR)return _frame```Write your functions. The only constraint is that they should take and emit a numpy.ndarray image. Of course, that doesnot mean you can't code whatever side effects you need them to generate!```if __name__ == '__main__':processor = LiveProcess(fps=True)processor.bind_process(convert_to_grayscale, threshold, reconvert_to_bgr)processor.initialise_capture_device(0)processor.loop()```Upon call, attach `LiveProcess` to a variable, then bind the processing functions to it, *in the order you want them tobe performed*! Then, bind a capture device and initialise it. Finally, start the loop.Built-in functionalities========================* FPS calculation (use `fps=True` when constructing your `LiveProcess` instance)* Screenshotting (bound to the `s` key, on by default but can be disabled using `screencap=False` in the `LiveProcess`constructor* Automatic clean-up and closedown (using the `esc` key by default)* And lots more...Todos=====* Tests!* Documentation!
Package: process-plot
process-plotCreate plots of resource usage for a process (memory, CPU, etc).Process statistics are polled at a set interval, in a cross-platform manner (i.e. supports Linux, OSX and Windows).UsageInstall the package withpiporpipx:$pipxinstallprocess-plotthen run:$pplotexec-i0.1"sleep 1"PPLOT INFO: Output files will be written to: /user/pplot_out, with basename: 20210921125420PPLOT INFO: Running process as PID: 5379PPLOT INFO: Total run time: 0 hour(s), 00 minute(s), 01.034680 second(s)PPLOT INFO: Plotting results to: pplot_out/20210921125420.pngPPLOT SUCCESS!You will then find the output files in/user/pplot_out, with a plot for the process like:Additional options are available:$pplotexec--helpUsage: pplot exec [OPTIONS] COMMANDExecute a command and profile it.Options:-i, --interval FLOAT            Polling interval (seconds)-t, --timeout FLOAT             Timeout process (seconds)-c, --command-output [hide|screen|file]Mode for stdout/stderr of command  [default:file]-o, --outfolder DIRECTORY       Folder path for output files-n, --basename TEXT             Basename for output files (defaults todatetime)-p, --plot-cols [memory_rss|memory_vms|cpu_percent|cpu_time_user|cpu_time_sys|threads_num|files_num]Columns to plot (comma-delimited)  [default:memory_rss,cpu_percent]--title TEXT                    Plot title (defaults to command)--grid / --no-grid              Add grid to plots  [default: grid]-sw, --size-width FLOAT         Width of plot in cm-sh, --size-height FLOAT        Height of plot in cm-f, --format [png|pdf|svg]      Plot file format-v, --verbose                   Increase verbosity  [x>=0]-q, --quiet                     Quiet mode--help                          Show this message and exit.AcknowledgementsInitially adapted from:https://github.com/jeetsukumaran/Syrupy
Package: processpy
Python Process Manager (processpy)processpy is simple process manager for python.
If you want to run multiple process for the same function, this tool is for you.You can run multiple process of the same function concurrently.You can choose to kill previous running process before running a new process of the same function.You can choose to ignore new process of the same function if it's already running.InstallationpipinstallprocesspyExample (No concurrency and no previous kill)fromprocesspyimportProcessManagerimporttimedefsum(a,b):time.sleep(30)print(a+b)sum_process=ProcessManager(sum,kill_previous=False,concurrent_running=False)sum_process.run({'a':10,'b':20})time.sleep(5)"""The following will not run. Because concurrent run is false and kill previous is also false. So, it will simply return with doing nothing and let the previous run."""sum_process.run({'a':10,'b':20})Example (No concurrency but with previous kill)fromprocesspyimportProcessManagerimporttimedefsum(a,b):time.sleep(30)print(a+b)sum_process=ProcessManager(sum,kill_previous=True,concurrent_running=False)sum_process.run({'a':10,'b':20})time.sleep(5)"""The following will kill the previous unfinished process and run. Because concurrent run is false and kill previous is True. So, it will simply kill the previous unfinished process. If previous one is already finished, nothing to kill."""sum_process.run({'a':10,'b':20})Example (with concurrency)fromprocesspyimportProcessManagerimporttimedefsum(a,b):time.sleep(30)print(a+b)sum_process=ProcessManager(sum,concurrent_running=True)sum_process.run({'a':10,'b':20})time.sleep(5)"""The following will run alongside of the previous process."""sum_process.run({'a':10,'b':20})You can also kill the running process (if concurrent_running=False )sub_process.kill()
Package: process-spectra
process_spectraThis is the repo for the process_spectra package. It is a package designed to process optical spectra of fiber optic sensors based on long period gratings (LPGs). The documentation is written in portuguese, since the project was conceived to improve research at a brazilian university lab (LITel - UFJF). If the contents of the library may be useful to you, and you do not speak portuguese, please send us an e-mail or open an issue.Pacote python feito com o intuito de processar o espectro óptico de sensores ópticos a fibra baseados em grades de periodo longo (LPGs). Com esse pacote é possível fazer a extração de dados de conjuntos grandes de espectros, seguindo uma rotina específica.Instalação:O pacote foi colocado no PyPi, logo é possível instalar pelo pip:pip install process_spectraComo usar:Para usar o pacote, basta criar um objeto da classeMassSpectraData, adicionar passos com os devidos argumentos e rodar. Como um exemplo simples que extrai os vales ressonantes de espectros na pastaspectra:importosimportprocess_spectraasps# Criando a lista de caminhosfiles=os.listdir('spectra')files_complete=[os.path.join('spectra',x)forxinfiles]# Criando o objeto, passando a lista e o nome do arquivo para# salvar as informações extraídasspectra=ps.MassSpectraData(files_complete,'resonant_wavelengths.csv')# Adicionando o passo de extração, com um dicionário de argumentos# para a função do passospectra.add_step(ps.funcs.find_valley,{'prominence':5,'ignore_errors':True})spectra.run(ignore_errors=True)Vale notar que os ignore_errors são passados para evitar que o programa encerre no caso de encontrar algum. Isso é útil quando não se tem certeza da integridade de todos os espectros, visto que se um estiver corrompido, o programa pode travar nele.Na pasta de exemplos estão alguns scripts que foram escritos para e usados em pesquisas, e mostram mais funcionalidades do pacote.DocumentaçãoA documentação não foi publicada em sites, mas foi colocada no código, então é possível encontrar ela através do comando help() do python, passando como argumento uma classe, função ou módulo, ou também lendo do código direto.Também é possível gerar a documentação através do sphinx. Basta clonar / baixar o repositórrio, abrir um cmd na pasta sphinx e rodar o comandomake htmlA documentação será gerada em html, como um site, e pode ser encontrada no caminhosphinx/_build/html/index.html.
Package: process-supervision-torch
"Let’s Verify Step by Step"Implementation of "Improving Mathematical Reasoning with Process Supervision" by OPENAIInstallpip3 install --upgrade process-supervision-torchUsage:GPT4 without tokenizerimporttorchfromprocess_supervision.mainimportGPT4# Usage with random inputstext=torch.randint(0,20000,(1,1024))# Initiliaze the modelmodel=GPT4()output=model(text)print(output)PRMimporttorchfromprocess_supervision.prmimportPRMfromswarms.modelsimportOpenAIChatfromprocess_supervision.generatorimportMathDataGeneratorimportosfromdotenvimportload_dotenvload_dotenv()api_key=os.getenv("OPENAI_API_KEY")# LLM initializationllm=OpenAIChat(openai_api_key=api_key)# Math data generator initializationmath_datagenerator=MathDataGenerator(llm,num_iters=10)# Device initializationdevice=0iftorch.cuda.is_available()else"cpu"# Model initializationprm_model=PRM(model_name="lvwerra/gpt2-imdb-pos-v2",ref_model_name="lvwerra/gpt2-imdb",reward_model_name="lvwerra/distilbert-imdb",device=device,)# Generation argumentsgen_kwargs={"min_length":-1,"top_k":0.0,"top_p":1.0,"do_sample":True,"pad_token_id":prm_model.tokenizer.eos_token_id,}sent_kwargs={"top_k":None,"function_to_apply":"none","batch_size":16}# Sample queriesqueries=["Sample query 1","Sample query 2"]queries=[math_datagenerator.generate_samples(query)forqueryinqueries]# Generate responsesresponses=prm_model.generate_responses(queries,gen_len=10,gen_kwargs=gen_kwargs)# Score responsesscores=prm_model.score_responses(responses,sent_kwargs)# Display resultsforquery,response,scoreinzip(queries,responses,scores):print(f"Query:{query}\nResponse:{response}\nScore:{score}\n")GPT4 + PRMMethodCitation@misc{lightman2023lets,title={Let's Verify Step by Step},author={Hunter Lightman and Vineet Kosaraju and Yura Burda and Harri Edwards and Bowen Baker and Teddy Lee and Jan Leike and John Schulman and Ilya Sutskever and Karl Cobbe},year={2023},eprint={2305.20050},archivePrefix={arXiv},primaryClass={cs.LG}}TodoCreae the PRM reward modelLicenseMIT
Package: process-tests
Tools for testing processes.Free software: BSD 2-Clause LicenseUsagefrom process_tests import ProcessTestCase
from process_tests import TestProcess

class MyTestCase(ProcessTestCase):
    def test_simple(self):
        with TestProcess('mydaemon', 'arg1', 'arg2') as proc:
            with self.dump_on_error(proc.read):
                self.wait_for_strings(proc.read, 10, # wait 10 seconds for process to output lines with these strings
                    'Started',
                    'Working',
                    'Done',
                )FeaturesTODOExampleshttps://github.com/ionelmc/python-redis-lock/blob/master/tests/test_redis_lock.pyhttps://github.com/ionelmc/python-manhole/blob/master/tests/test_manhole.pyhttps://github.com/ionelmc/python-stampede/blob/master/tests/test_stampede.pyhttps://github.com/ionelmc/python-remote-pdb/blob/master/tests/test_remote_pdb.pyTODOtestsdocsRequirementsOS:AnyRuntime:Python 2.6, 2.7, 3.2, 3.3 or PyPySimilar projectsTODOChangelog3.0.0 (2023-11-01)Dropped Python 2 support.Added an optional non-pipe TestProcess mode. You can use file objects for processes that are too verbose for a pipe.Added some tests.2.1.2 (2021-05-02)Fixed another regression caused by theuniversal_newlinesfor Windows.2.1.1 (2020-07-23)Fixed regression caused by theuniversal_newlines(now the internals don’t decode strings).2.1.0 (2020-07-23)Applied the cookiecutter-pylibrary templates.TestProcesswill useuniversal_newlinesby default for the containedsubprocess.Popento make sure line buffering is actually
used. This also fixes warnings on Python 3.2.x (???)Lots of wild stuff.
Package: process-timer
Process TimerInstallationFrom source code:python setup.py installFrom pypi:pip install process-timerUsageimporttimefromprocess_timerimporttimeout@timeout(5)defmytest():print("Start")foriinrange(1,10):time.sleep(1)print(f"{i}seconds have passed")if__name__=='__main__':mytest()Specify an alternate exception to raise on timeout:importtimefromprocess_timerimporttimeout@timeout(5,timeout_exception=StopIteration)defmytest():print("Start")foriinrange(1,10):time.sleep(1)print(f"{i}seconds have passed")if__name__=='__main__':mytest()MultithreadingBy default, timeout-decorator uses signals to limit the execution time
of the given function. This appoach does not work if your function is
executed not in a main thread (for example if it's a worker thread of
the web application). There is alternative timeout strategy for this
case - by using multiprocessing. To use it, just passuse_signals=Falseto the timeout decorator function:importtimefromprocess_timerimporttimeout@timeout(5,use_signals=False)defmytest():print("Start")foriinrange(1,10):time.sleep(1)print(f"{i}seconds have passed")if__name__=='__main__':mytest()WarningMake sure that in case of multiprocessing strategy for timeout, your function does not return objects which cannot
be pickled, otherwise it will fail at marshalling it between master and child processes.AcknowledgementTimeout Decorator - https://github.com/pnpnpn/timeout-decoratorhttp://www.saltycrane.com/blog/2010/04/using-python-timeout-decorator-uploading-s3/https://code.google.com/p/verse-quiz/source/browse/trunk/timeout.pyContributeI would love for you to fork and send me pull request for this project.
Please contribute.LicenseThis software is licensed under theMIT licenseSeeLicense file