Saravanan666/pypi-org huggingface dataset:
Package: threaders
threadersthreaders is a small module to help writeclean threaded code using threading decoratorsand minimize repeating copy-paste actions.InstallingInstall and update using pip:pip install -U threadersA Simple Example:from threaders import threadersimport time@threaders.threader()def function_to_be_threaded(x):“”” :rtype: Thread “””t = time.time()time.sleep(0.5*(x+0.1)/5+0.05)return time.time()-tdef main():# create threadsthreads = []for i in range(10):threads.append(function_to_be_threaded(i))# get first resultprint(threaders.get_first_result(threads))# kill threadst = time.time()for thread in threads:thread.join()print(“all threads terminated: {}”.format(time.time()-t))if __name__ == “__main__”:main()Example with a thread pool:from random import randrangefrom time import sleepimport threadingdelays = [randrange(1, 3) for i in range(50)]print_lock = threading.Lock()def wait_delay(i, d):with print_lock:print(‘{} sleeping for ({})sec’.format(i, d))sleep(d)pool = threaders.ThreadPool(10)for i, d in enumerate(delays):pool.put(wait_delay, i, d)pool.join()
Package: thread-executor
ExecutorFast execute task with python and less mem opsWhy we need Thread Executor?Python threading module is a good structure, it helps developers to folk a thread to run some background tasks.
Python have Queue mechanism to connect thread data.
But what problem??First, threading module folk threads but python not commit late time. Then know your thread can run, but you don't know when? It's oke fast with small traffic but when server high load you will have some problem, high pressure on memory because when you create too many thread cause slowness.waste of timeSecond, when you create and release threads many times, it'll increase memory and CPUs time of system. Sometime, developers did not handle exceptions and release thread. It can put more pressure to the application.waste of resourceHow to resolve problem??This's my resolver.We createexactordynamicnumber of threads. Then usingJob- a unit bring data information toWorkerto process. Workers don't need to release, and you only create 1 time or reset it when you update config.Job bring 2 importance field is:funcandargsand you can call them likefunc(*args)and get all the results and return oncallbackis optional.Your app doesn't need to create and release threads continuouslyEasy to access and using when coding.Disadvance?If you use callback then remembered toadd try catchto handle thread leaked.If queue is full you need to wait for available queue slot. setmax_queue_size=0to avoid this.If you restart your app, all theJobsinQueuethat have not been processed will belost.InstalltionNow it's time you can install lib and experiencepipinstallthread-executorUsage : Interface listsend(job:Job)->None# Push a job to the queuewait()->None# wait for all jobs to be completed without blocking each otherscale_up(number_threads:int)->None# scale up number of threadsscale_down(self,number_threads:int)->None# scale down number of threadsInitialfromexecutor.safe_queueimportExecutor,Jobengine=Executor(number_threads=10,max_queue_size=0)Send Simple Jobimporttimedeftest_exec(*args,**kwargs):time.sleep(3)print(args)return[1,2,3]deftest_exec1(*args,**kwargs):print(kwargs)time.sleep(2)return{"a":1,"b":2,"c":3}engine.send(Job(func=test_exec,args=(1,2),kwargs={},callback=None,block=False))engine.send(Job(func=test_exec1,args=(),kwargs={"time":1},callback=None,block=False))engine.send(Job(func=test_exec1,args=(),kwargs={},callback=None,block=False))engine.send(Job(func=test_exec1,args=(),kwargs={},callback=None,block=False))engine.send(Job(func=test_exec1,args=(),kwargs={},callback=None,block=False))engine.wait()Send Job with callbackdefcall_back(result):print(result)foriinrange(5):engine.send(Job(func=test_exec1,args=(),kwargs={"time":1},callback=call_back,block=False))engine.wait()Thread scale up/downengine.scale_up(3)engine.scale_down(3)
Package: threadfix_api
A Python API wrapper to facilitate interactions withThreadFix.This package implements all API functionality available within ThreadFix 2.2.2 (Apr 30).Quick StartSeveral quick start options are available:Install with pip (recommended):pip install threadfix_apiDownload the latest releaseClone the repository:git clone https://github.com/aparsons/threadfix_api.gitExample# import the packagefromthreadfix_apiimportthreadfix# setup threadfix connection informationhost='http://localhost:8080/threadfix/'api_key='your_api_key_from_threadfix'# instantiate the threadfix api wrappertf=threadfix.ThreadFixAPI(host,api_key)# If you need to disable certificate verification, set verify_ssl to False.# tf = threadfix.ThreadFixAPI(host, api_key, verify_ssl=False)# You can also specify a local cert to use as client side certificate, as a# single file (containing the private key and the certificate) or as a tuple# of both file's path.# cert=('/path/server.crt', '/path/key')# tf = threadfix.ThreadFixAPI(host, api_key, cert=cert)# rock and rollteams=tf.list_teams()ifteams.success:print(teams.data)# Decoded JSON objectforteaminteams.data:print(team['name'])# Print the name of each teamelse:print('Uh Oh! '+teams.message)Supporting information for each method available can be found in thedocumentation.Bugs and Feature RequestsHave a bug or a feature request? Please first search for existing and closed issues. If your problem or idea is not addressed yet,please open a new issue.Copyright and LicenseCopyright 2015Adam ParsonsLicensed under MIT.
Package: threadful
ThreadfulPython Threads - from Dreadful to ThreadfulInstallationpipinstallthreadfulUsagefromthreadfulimportthread@thread# with or without ()defsome_function():time.sleep(10)return" done "# when ready, it sill call these callback functions.some_function().then(lambdaresult:result.strip().then(lambdaresult:print(result))# prints: "done"promise=some_function()# ThreadWithResult[str] objectpromise.result()# Err(None)time.sleep(15)# after the thread is done:promise.result()# Ok(" done ")# alternative to sleep:result=promise.join()# " done " if success, raises if the thread raised an exception@thread()defraises()->str:raisesValueError()promise=raises().catch(lambdaerr:TypeError())promise.join()# raises TypeErrorpromise.result()# Err(TypeError)promise=raises().catch(lambdaerr:"Something went wrong")promise.join()# returns the string "Something went wrong"Licensethreadfulis distributed under the terms of theMITlicense.ChangelogSee CHANGELOG.md
Package: threading2
threading2:  like the standard threading module, but awesomer.This module is designed as a drop-in replacement and extension for the default
“threading” module.  It has two main objectives:implement primitives using native platform functionality where possibleexpose more sophisticated functionality where it can be done uniformlyThe following extensions are currently implemented:ability to set (advisory) thread priorityability to set (advisory) CPU affinity at thread and process levelthread groups for simultaneous management of multiple threadsSHLock class for shared/exclusive (also known as read/write) locksThe following API niceties are also included:all blocking methods take a “timeout” argument and return a success codeall exposed objects are actual classes and can be safely subclassedThis has currently only been tested on WinXP and Ubuntu Karmic; similar
platformsshouldwork OK, and other platformsshouldfall back to using
sensible default behaviour, but I’m not making any guarantees at this stage.Additional planned extensions include:make stack_size a kwarg when creating a threadnative events, semaphores and timed waits on win32native conditions and timed waits on pthreads platformsnative SHLock implementations (SRW on Win Vista+, pthread_rwlock)
Package: threading-manager
threading_manager (v0.0.8)DESCRIPTION_SHORTManage spawned threadsDESCRIPTION_LONGdesigned to working with spawned threadsFeaturesuse different managers for different funcs/methods if neededuse just one decorator to spawn threads from func / methodskeep all spawned threads in list by ThreadItem objectsThreadItem keeps result/exx/is_alive attributes!use wait_all()LicenseSee theLICENSEfile for license rights and limitations (MIT).Release historySee theHISTORY.mdfile for release history.Installationpip install threading-managerImportfromthreading_managerimport*USAGE EXAMPLESSee tests and sourcecode for other examples.1. example1.pyfromthreading_managerimport*count=5time_start=time.time()# define victim ------------------classThreadManager1(ThreadsManager):passclassCls:@ThreadManager1().decorator__to_threaddeffunc1(self,num):time.sleep(1)returnnum*1000# spawn ------------------foriinrange(count):assertCls().func1(i)isNoneassertThreadManager1().count==countThreadManager1().wait_all()assert{item.resultforiteminThreadManager1().THREADS}=={num*1000fornuminrange(count)}ThreadManager1().clear()# spawn ------------------foriinrange(count):assertCls().func1(i)isNone
Package: threadingpg
threadingpgControl PostgreSQL using thread(s).Initialize Controllerimportthreadingpgcontroller=threadingpg.Controller()controller.connect(dbname='database_name',user='user_name',password='password',port=5432)# ...controller.close()Initialize Poolimportthreadingpgcontroller=threadingpg.Pool(dbname='database_name',user='user_name',password='password',port=5432)# ...controller.close()Table and RowTableimportthreadingpgclassMyTable(threadingpg.Table):table_name="mytable"index=threadingpg.Column(data_type=threadingpg.types.serial)name=threadingpg.Column(data_type=threadingpg.types.varchar())# orclassMyTable(threadingpg.Table):def__init__(self)->None:self.index=threadingpg.Column(data_type=threadingpg.types.serial)self.name=threadingpg.Column(data_type=threadingpg.types.varchar())super().__init__("mytable")# important positionCreate/Drop Tablemytable=MyTable()controller.create_table(mytable)controller.drop_table(mytable)RowclassMyRow(threadingpg.Row):def__init__(self,name:str=None)->None:self.name=nameInsert Rowmytable=MyTable()myrow=MyRow("my_row")controller.insert_row(mytable,myrow)# orcontroller.insert_dict(mytable,{"name":"my_row"})Select Rowmytable=MyTable()column_name_list,rows=controller.select(mytable)forrowinrows:myrow=MyRow()myrow.set_data(column_name_list,row)print(f"output:{myrow.name}")# output: my_rowUpdate Rowmytable=MyTable()myrow=MyRow("update_my_row")condition_equal_0=threadingpg.condition.Equal(mytable.index,0)controller.update_row(mytable,myrow,condition_equal_0)Delete Rowmytable=MyTable()delete_condition=threadingpg.condition.Equal(mytable.index,5)controller.delete_row(mytable,delete_condition)ConditionsWheremytable=MyTable()condition_equal_1=threadingpg.condition.Equal(mytable.index,1)condition_equal_2=threadingpg.condition.Equal(mytable.index,2)condition_equal_3=threadingpg.condition.Equal(mytable.index,3)conditions=threadingpg.condition.Or(condition_equal_1,condition_equal_2,condition_equal_3)column_name_list,rows=controller.select(mytable,where=conditions)OrderBymytable=MyTable()orderby_index=threadingpg.condition.OrderBy(mytable.index)orderby_name=threadingpg.condition.OrderBy(mytable.name,True)orderby_conditions=threadingpg.condition.And(orderby_index,orderby_name)column_name_list,rows=controller.select(mytable,order_by=orderby_conditions)TriggerNeed delay each function.mytable=MyTable()channel_name="mych"trigger_name="mytr"function_name="myfn"listner=threadingpg.TriggerListner()# implement 'notify = listner.notify_queue.get()'listner.connect(dbname=dbname,user=user,password=password,port=5432)listner.create_function(function_name,channel_name)listner.create_trigger(mytable,trigger_name,function_name)listner.start_listening()listner.listen_channel(channel_name)# ...listner.unlisten_channel(channel_name)listner.stop_listening()
Package: threadingwebdriver
threadingwebdriverSelenium headless webdriver using two threadpools.ThreadPool(1) for control browser.ThreadPool(custom_number) for read page(ex: get WebElement).Chrome only.Initializeimportthreadingwebdriverdriver=threadingwebdriver.ChromeWebdriver()driver.initialize()CloseClose driver. It will wait tasks of ThreadPools are finish.driver.close()Open URL (Async)url='https://www.google.com/'driver.open_async(url)Open URL (Sync)url='https://www.google.com/'is_open:bool=driver.open(3,url)Get Element (Async)url='https://www.google.com/'driver.open_async(url)timeout=3body_xpath='/html/body'body_xpath_result:WebElementAsyncResult=driver.get_element_xpath_async(timeout,body_xpath)# code...body:WebElement=body_xpath_result.get()Concurrencyimportthreadingwebdriverdriver=threadingwebdriver.ChromeWebdriver()driver.initialize(read_thread_count=3)timeout=2url="Input Your URL"driver.open(timeout,url)p_async=driver.get_element_xpath_async(timeout,'/html/body/div/p')a_async=driver.get_element_xpath_async(timeout,'/html/body/div/a')div_async=driver.get_element_xpath_async(timeout,'/html/body/div')p=p_async.get()a=a_async.get()div=div_async.get()Get Element (Sync)timeout=3body_xpath='/html/body'body:WebElement=driver.get_element_xpath(timeout,body_xpath)Initialize Websocket Listenerasyncdefwebsocket_listener(listener):asyncforeventinlistener:payload_data=event.response.payload_dataprint(payload_data)importthreadingwebdriverdriver=threadingwebdriver.ChromeWebdriver()driver.initialize(websocket_listening_function=websocket_listener)url='https:// Input Your URL'driver.open(3,url)ExceptionsBased on thread order.url1='https://www.google.com/'url2='https://www.github.com/'driver.open_async(url1)driver.open_async(url2)timeout=3body_xpath='/html/body'body_xpath_result:WebElementAsyncResult=driver.get_element_xpath_async(timeout,body_xpath)# Exception: if run 'get_element_xpath_async()' before run 'open_async(url2)'.url1='https://www.google.com/'url2='https://www.github.com/'driver.open_async(url1)timeout=3body_xpath='/html/body'body_xpath_result:WebElementAsyncResult=driver.get_element_xpath_async(timeout,body_xpath)driver.open_async(url2)# Exception: run 'open_async(url2)' if not finish 'get_element_xpath_async()'.
Package: threadio
ThreadIO, is good for capture stdout at multi-thread environment.>>> # standard capture
>>> from threadio import print_to_file
>>> with print_to_file(TextIO()) as fp:
...     print("hello world")
...
>>> fp.getvalue()
'hello world\n'>>> # standard capture (but not stdout, stderr)
>>> from threadio import ThreadIO
>>> fp = ThreadIO(TextIO())
>>> with fp.capture(TextIO()) as fp2:
...     print("hello world", file=fp)
...     with fp.capture(TextIO()) as fp3:
...         print("hello world3", file=fp)
...         # not captured fp3, stored fp2
...         print("fail captured", file=fp2)
...
>>> fp.getvalue()
''
>>> fp2.getvalue()
'hello world\nfail captured\n'
>>> fp3.getvalue()
'hello world3\n'>>> # thread example
>>> from threading import Thread
>>> from threadio import print_to_file
>>> th = Thread(target=print, args=("hello world!",))
>>> with print_to_file(TextIO()) as fp:
...     th.start()
...     th.join()
...     print("HAYO!")
...
hello world!
>>> fp.getvalue() # not capture other thread's print
'HAYO!\n'>>> # note. print_to_xxx function change everything.
>>> a = sys.stdout
>>> with print_to_file(TextIO()): pass
>>> b = sys.stdout
>>> a is b
False
Package: threadit
Python ThreaditRunning functions in threads with no effortA nice package to help you run functions in threads and get their result.This is under development! So please check for changes before updatingInstallationpip install threaditUsageimporttimefromthreaditimportThreadITdefget_company_name():# Do work in threadtime.sleep(5)return'Adapted'defdo_some_work():get_name=ThreadIT(get_company_name)# do stuff or run a while loop to wait for resultmore_thread=ThreadIT(get_company_name)whileget_name.doing_work():print('Waiting for thread to finnish')# You can also call .result() and the main thread will wait# for the thread to return your result.company_name=get_name.result()# this will timeout the thread in 1 second if its not finishedcompany_name2=more_thread.result(timeout=1)print(company_name)# Outputs -> Adaptedprint(company_name2)# Outputs -> NoneMain commands# Import the modulefromthreaditimportThreadIT# Start your function with or without argumentsvar=ThreadIT(function,args,kwargs)# Wait for the function to finnish or get the result if its finishedvar.result()Optional commands# Check if your thread is still working on the function.# This will return True if the function is not completed.var.doing_work()# Terminate the thread. This will noe wait for the function to# finnish and will return None as result if the function is not# done within the timelimit setvar.result(timeout=10)TestingUse the following command to run tests.python-munittestthreadit.tests.test_threaditChangelog:See CHANGELOG.md
Package: threadlet
threadletImproved ThreadPoolExecutorTable of ContentsInstallationLicenseFeaturesBenchmarksInstallationpip install threadletLicensethreadletis distributed under the terms of theMITlicense.FeaturesimporttimeimportthreadingfromthreadletimportTask,Worker,ThreadPoolExecutor,waitdefcalc(x):returnx*2# spawns new thread each time to run function in itfuture=Task.run(calc,2)assertfuture.result()==4# spawns one thread to handle all submitted functionswithWorker()asw:future=w.submit(calc,3)assertfuture.result()==6# "idle_timeout" argument (5 seconds by default):# workers are going to die after doing nothing for "idle_timeout" seconds.withThreadPoolExecutor(4,idle_timeout=1)astpe:assertthreading.active_count()==1fs=set()for_inrange(100):fs.add(tpe.submit(time.sleep,0.1))assertthreading.active_count()==1+4# main thread + 4 workerswait(fs)time.sleep(1)# wait until workers die by timeoutassertthreading.active_count()==1# "min_workers" argument:# amount of workers which are pre-spawned at start and not going to die ever in despite of "idle_timeout".withThreadPoolExecutor(4,min_workers=2,idle_timeout=1)astpe:assertthreading.active_count()==1+2# main thread + 2 pre-spawned workersfs=set()for_inrange(100):fs.add(tpe.submit(time.sleep,0.1))assertthreading.active_count()==1+4# main thread + 4 workerswait(fs)time.sleep(1)# wait until workers die by timeoutassertthreading.active_count()==1+2# as at starting pointBenchmarks+----------------+---------+-----------------------+-----------------------+|Benchmark|default|threadlet|threadlet_simple|+================+=========+=======================+=======================+|max_workers=1|101ms|64.0ms:1.58xfaster|57.7ms:1.76xfaster|+----------------+---------+-----------------------+-----------------------+|max_workers=2|97.6ms|65.2ms:1.50xfaster|55.4ms:1.76xfaster|+----------------+---------+-----------------------+-----------------------+|max_workers=4|103ms|62.8ms:1.63xfaster|56.1ms:1.83xfaster|+----------------+---------+-----------------------+-----------------------+|max_workers=8|95.9ms|63.5ms:1.51xfaster|57.9ms:1.66xfaster|+----------------+---------+-----------------------+-----------------------+|Geometricmean|(ref)|1.56xfaster|1.75xfaster|+----------------+---------+-----------------------+-----------------------+
Package: threadloop
Run Tornado Coroutines from Synchronous Python.fromthreadloopimportThreadLoopfromtornadoimportgen@gen.coroutinedefcoroutine(greeting="Goodbye"):yieldgen.sleep(1)raisegen.Return("%sWorld"%greeting)withThreadLoop()asthreadloop:future=threadloop.submit(coroutine,"Hello")printfuture.result()# Hello World
Package: threadmanager
threadmanagerA thread manager for Python programsIt provides:Centralization of starting and monitoring threadsState management for the functions in the threadsLogging for exceptions and excessive runtimeOriginal use case:GUI program that calls back-end functions for IO-bound workGUI has a cancel button that should always work, so:the GUI mainloop should not be blockedthe called functions should intermittently check if the user has pressed the cancel buttonAvoid running new work threads when the user wants to cancelAllow running callback functions when the program starts working or goes idle.e.g. update a status bar with certain textInstallation:pip install threadmanagerTested for Python >=3.6.5 on Linux (Ubuntu) and Windows 7/10
Package: thread-manager-py
thread-manager-pyPython Thread ManagerInstallpipinstallthread-manager-pyPool Manager UsageYou can find examples intest/test_pool_manager.py.Simple Usageimportosfrompool_managerimportPoolManagerdefcalculate(x):print(f"[{os.getpid()}]  func:{x}\t\t",r:=x**5**2,flush=True)returnrmanager=PoolManager()manager.add_task(calculate,[iforiinrange(2,22)])manager.run_map()manager.add_task(calculate,[iforiinrange(2,22)])manager.add_task(calculate,[iforiinrange(2,22)])manager.run_map()task_result=manager.get_task_result()withPoolManager()asmanager:manager.add_task(calculate,[iforiinrange(2,22)])manager.add_task(calculate,[iforiinrange(2,22)])manager.add_task(calculate,[iforiinrange(2,22)])manager.run_map()Thread Manager UsageYou can find examples intest/test_thread_manager.py.Simple Usageimporttimefromthread_managerimportThreadManager,ThreadArgumentdefprint_something(name:str,number:int):print(name,number)time.sleep(1)thread_manager=ThreadManager(print_something,[ThreadArgument(thread_name=f"Thread:{x}",args=(x,x))forxinrange(1,23)])thread_manager.run()Get Thread Errorerrors=thread_manager.get_errors()has_error=thread_manager.has_error()error_count=thread_manager.get_error_count()foreinerrors:print(e)Simple Usage with decoratorfromthread_managerimportusing_thread@using_threaddefprint_something(number,**kwargs):print(number,kwargs)foriinrange(10):print_something(i,name=f"thread-{i}")Simple Usage with Exception Hookfromthread_managerimportThreadManager,ThreadArgumenterrors=[]deffunc_something(*args):raiseException("test error")deffunc_exception_hook(*args):errors.append(args)thread_manager=ThreadManager(func_something,[ThreadArgument(thread_name=f"Thread:{x}",args=(x,x),kwargs={},)forxinrange(1,23)],except_hook=func_exception_hook)thread_manager.run()
Package: threadmap
Author:Y.OkamuraMulti thread mapExample>>> import threadmap
>>> import time
>>> def testfun(a):
...     print 'start', a
...     time.sleep(1)
...     print 'finish', a
...     return a**2
...
>>> tm = threadmap.ThreadMap()
>>> tm.map(testfun, range(10))LicenseGNU General Public License v3 or later
Package: threadmsg
threadmsgThread safe message queue with function callsTable of contentsInstallExamplesReferencesInstall$ pip3 install threadmsgExamplesimportthreadmsgastm#--------------------------------------------------------------------# Example 1asyncdefmsgThread(ctx):# Check for messagemsg=ctx.getMsg()ifnotmsg:returnprint(msg)# Return a negative number to exit threadreturn-1# Return a positive number and this function will be# called again after that number of secondsreturn3.5# Return zero this function will be called again immediatelyreturn0# Return nothing for inifinite wait, this function will only# be called again if a message is posted or quit is requestedreturn# Create a threadt1=tm.ThreadMsg(msgThread)# Send message to threadt1.addMsg("Hello thread")# Wait for thread to exitt1.join()#--------------------------------------------------------------------# Example 2asyncdefmyThread(ctx,v1,v2):# Verify parametersassert5==v1assert6==v2# Check if this is the first runifnotctx.loops:print('First call')return0# Check if thread is exitingifnotctx.run:print('Thread is exiting')returnprint('Second call')# Exit threadreturn-1# Create a thread, but don't start itt1=tm.ThreadMsg(myThread,(5,6),False)# Start threadt1.start()# Wait for thread to exitt1.join()#--------------------------------------------------------------------# Example 3classfunThread(tm.ThreadMsg):def__init__(self,start=True):super().__init__(self.msgThread,deffk='_funName',start=start)self.callMap={'add':self.add}@staticmethodasyncdefmsgThread(ctx):whilemsg:=ctx.getMsg():awaitctx.mapMsgAsync(None,ctx.callMap,msg)# return nothing so this function is only# called again when there is a messagedefadd(self,a,b):returna+b# Create the threadt1=funThread()defshowReturn(ctx,params,retval,err):print(params,retval,err)# Call add function with callbackt1.call(showReturn,'add',a=1,b=2)# Wait for reply from add functionreply=t1.call('add',a=1,b=2)awaitreply.wait(3)print(reply.getData())# Signal thread to quitt1.stop()# Wait for thread to exitt1.join()# Or call join with True to both signal quit and wait for exit# tw.join(True)ReferencesPythonhttps://www.python.org/piphttps://pip.pypa.io/en/stable/