Saravanan666/pypi-org huggingface dataset:
Package: threadmycode
threadmycodeis a tiny Python library lets you convert your Python methods into asynchronous,
non-blocking methods simply by using a decorator.To enable the theading mode,
First paramater should not be keyword paramater
First paramater of the function must be iterable
Function will be called in thread with first paramter as iterated value of iterableif first paramater is not a iterable then normal execution happensExample# example.pyimportthreadmycodeasmultithreadingimporttimeimportrandomimportsignal# kill all tasks on ctrl-c# signal.signal(signal.SIGINT, multithreading.killall)# or, wait for task to finish on ctrl-c:signal.signal(signal.SIGINT,multithreading.stop)@multithreading.threaditdefhello(count):sleep=3print("Hello%s(sleeping for%ss)"%(count,sleep))time.sleep(3)print("Goodbye%s(after for%ss)"%(count,sleep))if__name__=="__main__":multithreading.threadCount(5)hello(range(10))The output would look something like this:$pythonexample.pyHello0(sleepingfor3s)Hello1(sleepingfor3s)Hello2(sleepingfor3s)Hello3(sleepingfor3s)Goodbye0(afterfor3s)Goodbye1(afterfor3s)Goodbye2(afterfor3s)Goodbye3(afterfor3s)Hello4(sleepingfor3s)Hello5(sleepingfor3s)Hello6(sleepingfor3s)Hello7(sleepingfor3s)Goodbye4(afterfor3s)Goodbye5(afterfor3s)Hello8(sleepingfor3s)Goodbye6(afterfor3s)Hello9(sleepingfor3s)Goodbye7(afterfor3s)Goodbye8(afterfor3s)Goodbye9(afterfor3s)SettingsThe default maximum threads is equal to the # of CPU Cores.This is just a rule of thumb!TheThreadmodule isn't actually using more than one core at a time.You can change the default maximum number of threads using:importthreadmycodeasmultithreadingmultithreading.threadCount(10)
Package: threado
ThreadoA simple way to run multiple threads do the I/OExampleimportstringfromthreado.simple_thread_runnerimportSimpleThreadsRunneractions=list(string.ascii_lowercase)sr=SimpleThreadsRunner()sr.run_threads(5,lambdax:print("Thread output char:"+x),iter_data=actions)# In case the iter_data : Iterator[Any] is an Iterator of large amounts datasr.run_threads(5,lambdax:print("Thread output char:"+x),iter_data=actions,batch_size=20)# Or it can be used separate as producer/consumerthread_count=3for_inactions:sr.q_producer(_)sr.q_consumer(thread_count,lambdax:print("Thread output char:"+x))# File downloadimporturllib.requesturl_list=['https://www.blog.pythonlibrary.org/wp-content/uploads/2012/06/dnd_text-150x93.png','https://www.blog.pythonlibrary.org/wp-content/uploads/2012/06/dnd_file-150x93.png','https://www.blog.pythonlibrary.org/wp-content/uploads/2012/06/panel_smack1-150x91.png']defdownload_url(url:str):pic_name=url.split("/")[-1:][0]urllib.request.urlretrieve(url,pic_name)sr=SimpleThreadsRunner()sr.run_threads(3,download_url,iter_data=url_list)# Simple load testimporttimedefsimple_loadTest(_thread_count,func,load_test_data):s=SimpleThreadsRunner()index=0forindex,datainenumerate(load_test_data,start=1):sr.q_producer(data)_start=time.time()s.q_consumer(func,_thread_count)_end=time.time()print(f"Total transitions:{index}\nTPS={index/(_end-_start)}")Installing threado and Supported Versionsthreado is available on PyPI:$python-mpipinstallthreado
Package: threadpool
WarningThis module isOBSOLETEand is only provided on PyPI to support old    projects that still use it. PleaseDO NOT USE IT FOR NEW PROJECTS!Use modern alternatives like themultiprocessingmodule in the standard library or even an asynchroneous approach withasyncio.A thread pool is an object that maintains a pool of worker threads to performtime consuming operations in parallel. It assigns jobs to the threadsby putting them in a work request queue, where they are picked up by thenext available thread. This then performs the requested operation in thebackground and puts the results in another queue.The thread pool object can then collect the results from all threads fromthis queue as soon as they become available or after all threads havefinished their work. It’s also possible, to define callbacks to handleeach result as it comes in… note::    This module is regarded as an extended example, not as a finished product.    Feel free to adapt it too your needs.
Package: thread_pool
Thread poolThis is a simple thread pool for python(using queue module).UsageFirstly, you should define a callback to deal with your task:def do_work(*args, **kwds):
    # do somethingThen, you can create a thread pool to schedule your tasks:from thread_pool import ThreadPool

# Create thread pool with nums threads
pool = ThreadPool(nums)
# Add a task into pool
pool.add_task(do_work, args, kwds)
# Join and destroy all threads
pool.destroy()
Package: thread-queue
thread-queueA tool to queue tasks and execute them on a fixed number of threadsLargely stolen from the python documentation on QueuesFree software: MIT licenseDocumentation:https://thread-queue.readthedocs.io.FeaturesTODOCreditsThis package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.1.0 (2016-08-02)First release on PyPI.
Package: threadref
Allows threads in Python to create “weak references”
to themselves that detect when the thread is no longer
running, similar to how a weak reference detects when
its referent object is no longer alive.Provides a lightweight way for one or more independent
pieces of code to register per-thread cleanup callbacks
without coordination.VersioningThis library’s version numbers follow theSemVer 2.0.0
specification.Installationpip install threadrefUsageimportthreadrefCreate a reference to the current thread, with a
callback that will fire when the thread exits:reference=threadref.ref(lambdareference:...)Just likeweakref.ref,threadref.refinstances
must still be alive when their referent thread stops
running, or their callback will not be called.Create a finalizer for the current thread, which
will be called when the thread exits:finalizer=threadref.finalize(function,*args,**kwargs)Just likeweakref.finalize,threadref.finalizeinstances remain alive on their own as long as they
need to, so this is a simpler and nicer interface in
the typical case of registering cleanup functions.Detailsthreadref.refandthreadref.finalizewrapweakref.refandweakref.finalize, and the
interface is the same except that they act as if
they are referencing the thread itself instead of
taking a referent argument, and internally they
work by referencing an object saved on a privatethreading.localinstance.
Package: thread-regulator-pjn2work
ThreadRegulatorPython class that allows to control thread execution in time (requests per second) for both constant rate mode, or burst mode. With a notify option that is called after a specific number of executions or a time deltaSample codefromthread_regulatorimportThreadRegulator,safe_sleep,create_regular,create_burstfromthread_regulator.graphsimportPerformanceGraphsfromrandomimportchoicedefdemo_constant_rate():defmy_notifier(stats_dict,arg1,**kwargs):print(arg1,stats_dict)defmy_thread_call(*args,**kwargs):safe_sleep(choice((0.1,0.2,0.3)))returnTruetr=create_regular(users=4,rps=10.0,duration_sec=1.0,executions=15)print(tr)print("="*100)tr.set_notifier(notify_method=my_notifier,every_sec=1,every_exec=8,notify_method_args=("notify_example_arg_1",)).\start(my_thread_call,"arg1","arg2",arg3="my_name",arg4="my_demo")returntrdefdemo_burst_mode():defmy_notifier(arg1,stats_dict):print(arg1,stats_dict)defmy_thread_call(*args,**kwargs):safe_sleep(choice((0.1,0.2,0.3)))returnTruetr=create_burst(users=4,rps=10.0,duration_sec=2.0,req=10,dt_sec=0.5,executions=20)print(tr)print("="*100)tr.set_notifier(notify_method=my_notifier,every_sec=1,every_exec=8,notify_method_args=("notify_example_arg_1",)).\start(my_thread_call,"arg1","arg2",arg3="my_name",arg4="my_demo")returntrdefshow_statistics(tr):print("="*100)print("Statistics:",tr.get_statistics())print(f"Requests start_time jitter:\n{tr.get_execution_dataframe().start_ts.diff().describe()}")print(f"Requests call period:{tr.get_executions_call_period()}")print(f"Should be executed{tr.get_max_executions()}requests, and{tr.get_executions_started()}were executed, and{tr.get_executions_completed()}completed, and{tr.get_executions_missing()}missing.",)print("How many successes over how many requests executed:",tr.get_success_ratio())print("How many successes over how many requests should be executed:",tr.get_success_ratio_overall())defsave_results(tr,filename):pg=PerformanceGraphs()# this will save the results on an Excel file (that can be used to plot graphs as explained on last bullet)pg.collect_data(tr).save_data(filename)if__name__=="__main__":print("RegularMode")show_statistics(demo_constant_rate())print("\n\nBurstMode")tr=demo_burst_mode()show_statistics(tr)save_results(tr,"burst_mode_results.xls")To see the graphical results:Runpython -m thread_regulatorOpen the browserhttp://127.0.0.1:8050/Drag/drop to the browser the .xls file (saved before)
Package: threadsafe-async
Asyncio ↔ Thread Synchronisation libraryThis library provides synchronisation primitives betweenasyncioandthreadsin Python. It allows you to coordinate the execution ofasyncio coroutinesandthreadsusing familiar synchronisation primitives such asEventandChannel.InstallationYou can install this library using pip:pip install threadsafe-asyncUsageEventTo use theEventclass, import it from the library and create an instance:from tsasync import Event

event = Event()You can then use thewait(),set(),clear(), andis_set()methods of theEventinstance to coordinate the execution of asyncio coroutines and threads. For example:async def coroutine(event):
    await event.wait()
    # event is set

def thread(event):
    event.set()You can use thewait()method synchronously from the threadevent.wait()and asynchronously from the coroutine in event loop:await event.wait()Thewait()method blocks until the event is set, while theset()method sets the event and wakes up all tasks waiting for it. Theclear()method clears the event, causing tasks waiting for it to block again. Theis_set()method returnsTrueif the event is set, andFalseotherwise.Notethat you should only call thewait()method from the same thread or asyncio event loop that created theEventinstance. If you try to callwait()from a different thread or event loop, aRuntimeErrorwill be raised.ChannelIt allows you to send and receive data between asyncio coroutines and threads.To use theChannelclass, import it from the library and create an instance:from tsasync import Channel

channel = Channel()You can then use thesend()andreceive()methods of theChannelinstance to communicate between asyncio coroutines and threads:async def consumer_coroutine(channel):
   data = await channel.receive()
   ...

def consumer_thread(channel):
   data = channel.receive()
   ...

async def producer_coroutine(channel):
   ...
   await channel.send(data)

def producer_thread(channel):
   ...
   channel.send(data)Thesend()method sends data to the channel, while thereceive()method receives data from the channel. Thesend()method will block until the receive side of the channel is ready to receive data. Thereceive()method will block until there is data available to be received.Note that thesend()andreceive()methods are will block the main thread if there are using withoutawait.If you want to use a buffered channel with a maximum size, you can pass the buffered argument to theChannelconstructor:channel = Channel(buffered=10)This will create a channel with a buffer of 10 items. If the buffer is full, thesend()method will block until there is room in the buffer.LicenseThis library is licensed under the MIT License. See theLICENSEfile for more information.ContributingContributions are welcome! Please open an issue or pull request on the GitHub repository if you have any suggestions or bug reports.
Package: threadsafevariable
ThreadSafeVariableThe problemImagine you have class like thisclass A(object):
    def __init__(self, x):
        self.x = x

a = A(3)
print(a.x)a.xis a member variable of objecta. It is not thread safe. This means that if one thread changes `a.x``, all other threads see the new value. You can avoid this using a thread local object:import threading

class A(object):    
    def __init__(self, x):
        self.local = threading.local()
        self.local.x

a = A(3)
print(a.local.x)This works but has two downsides:you have changed the syntaxeven if a is accessible from other threads, a.local.x is only defined for the one thread that created the objectThe ThreadSafeVariable module helps solve this problem and preserves the original syntax.The solutionWe change the code in the first example by declaring which variables should the thrad safe:from threadsafevariable import ThreadSafeVariable

class A(object):
    x = ThreadSafeVariable() # we add this line
    def __init__(self, x):
        self.x = x

a = A(3)
print(a.x)We have preserved the syntax for the first example but a.x is now a thread local variable.In addition we can do:iceblock = ThreadSafeVariable.freeze()This will store a snapshot all ThreadSafeVariable(s) of all objects defined in the current thread.
Then for any other thread:ThreadSafeVariable.restore(iceblock)will restore all variables of all objects in this thread to the value stores in iceblock.
Package: threadsnake
ThreadsnakeThreadsnakeis an experimental server side library. It was created as an educational (self-educational) project in order to explain how doesHTTP/1.1works (I'm lying, it was to understand it).importasynciofromthreadsnake.turboimportApplication,HttpRequest,HttpResponseapp=Application(8080)@app.get('/')defmain(app:Application,req:HttpRequest,res:HttpResponse):res.end('Served by threadsnake!')asyncio.run(app.run())The previous example sets a server wich listens at port 8080 and responds to every get resquest to the path /Threadsnake is designed to be lightweight and simple, yet able to serve hypertext and other resources sucessfully. Futhermore, as it was build over theThe Python Standard Library, it has no third party dependencies.FeaturesAs I said, threadsnake is an academic project, not intended to be used as an fully production tool. However, it actually has some interesting capacities out of the box:Static filesSession managementFiles upload and downloadMultipart-form data parsingAuthorization parameters parsing...And much more. In fact, I'm compilinga lot of examplesin order to show how can you use all those options to make a feature-rich and fully functional web server.Installing threadsnakeInstalling through pipThe main way to get threadsnake is usingpip:pip install threadsnakeIf you did it well, thepython package managerwill download and install threadsnake on your system. If not, well... That means something went wrong. Checkthis linkin order to solve the pip installation method. Those other links may be helpful:Installing packagesInstalling Python ModulesDownloading from PypiYes, i know it's redundant, but you always can go topypiand download the packagedirectly(The 'directly' part is the actual link to pypi. However, as all we know, we feel more comfortable by clicking the literal link, so, here you have it:https://pypi.org/project/threadsnake/#files).Downloading from the repository ...if still hereIfMurphyis obsessed with you (Seriously, if you're still reading, it have to be true) and all the above fails, then you have no option than going to the repository atgithub.Documentation and IssuesWow... You're still reading. You have to be some sort of... Anime Archetype.Well, as you can see right now we lack of a proper documentation. You have to understand, it's a new and small project. That said, once I finish uploading the examples, I'm planning to publish thedocumentation in this linkand the bugs trackingto this other link. Till then... you know. Don't let your dreams be dreams. JU5T D0 1T (I don't know anything about trademarks, so i wrote the phrase that way cause i don't want that ancient-greek sneakers-maker goddess to get mad at me).
Package: thread-task
thread_task is built on top ofthreadingand allows to organize tasks and run them parallel.You can:build chains of tasks, which execute tasks sequentially in a single threadbuild parent-child relationships, which allows to construct trees of tasksstop tasks, which also stops the execution of child taskscontinue tasks, which also continues the execution of child tasksrestart finished or stopped tasksjoin tasks, which means waiting until the task and its children are finishedA thread_task is not like a function, it doesn’t return results. Think
of it as an instruction to a reliable but independently acting
person. If feedback is needed, this can be done by callback
functions. Communication between tasks is possible by mutable objects,
which can be red or changed by multiple tasks.The following task types exist:Task: Executes a single callable or a chain of callables.Repeated: Executes a callable multiple times.Periodic: Executes a callable periodicallySleep: Sleeps for a given time, is similar totime.sleep,
but can be stopped and continuedReadthread-task.readthedocs.iofor more details.
Package: thread-worker
thread_worker方便的使用生产者消费者模式 使用多线程+队列支持 正常的一个生产者多个消费者模式支持 优先消费队列模式支持 限流work模式workWorker普通多线程workPriorWorker优先消费队列workLimitWorker限流work使用初始化的话有如下参数def__init__(self,consumer_func,consumer_count,logger,block,timeout):'''@params consumer_func consumer_func default None@params consumer_count Work quantity@params logger You can customize the logger. The default is logging@params block You can set whether the thread ends when the queue is emptyBy default, the thread will end when the queue is empty and the thread task is completedYou can set it to true to enable work to monitor tasks all the time@params timeout The timeout for obtaining data in the queue is 5S by default, that is, if the data is not obtained within 5S, it will end'''consumer_func 消费者函数 默认为None 如果加上了 就认为是先启动消费者再生成任务
consumer_count 是控制有多少个消费者logger 是设置日志输出block 是控制是否是阻塞模式 如果设置为True 那么work线程会一直执行 直到主进程结束 否则 当队列为空且线程任务都完成就将work结束timeout 是从队列获取数据的超时时间 默认是5s 超过5s 如果join是True则work重新再取 join为False则结束当前work生产者消费者模式有两种使用场景一种是先把消费者跑起来然后再生成
可以持续生成持续消费 适合被动代理扫描任务或者说一个主进程控制全局然后生成任务一种是一次性把任务都生成好交给消费者跑适合一次性就生成测试的任务 然后跑就完事了通过work.is_end 方法来阻塞多线程任务定义消费者函数实例一个Work通过实例的Workput数据给消费者函数通过Work.is_end()方法阻塞程序基础版workerfromthread_workerimportWorkerimportrequestsimporttimedefconsumer(iid):url="https://www.baidu.com/{0}".format(iid)resp=requests.get(url=url)print(resp.request.url)# 不需要阻塞w=Worker(consumer,consumer_count=1,block=False)foriidinrange(10):w.put(iid)# 这里通过 is_end 方法来阻塞程序whilenotw.is_end():time.sleep(3)创建一个限流的work与默认的Work一样 只是在创建的时候多了一个limit_time参数通过设置 单位时间limit_time和consumer_count来控制单位时间内执行多少个work默认limit_time是1sfromthread_workerimportLimitWorkerimportrequestsimporttimedefconsumer(iid):url="https://www.baidu.com/{0}".format(iid)resp=requests.get(url=url)print(resp.request.url)# limit_time 是limit_time 秒内有 consumer_count个消费者w=LimitWorker(consumer,consumer_count=1,block=False,limit_time=3)foriidinrange(10):w.put(iid)# 这里通过 is_end 方法来阻塞程序whilenotw.is_end():time.sleep(3)输出是这样的 每3s才会发一个请求即控制频率send: 2022-04-08 10:13:04 904000
https://www.baidu.com/0
send: 2022-04-08 10:13:07 904000
https://www.baidu.com/1
send: 2022-04-08 10:13:10 904000
https://www.baidu.com/2
send: 2022-04-08 10:13:13 904000
https://www.baidu.com/3
send: 2022-04-08 10:13:16 904000
https://www.baidu.com/4
send: 2022-04-08 10:13:19 904000
https://www.baidu.com/5
send: 2022-04-08 10:13:22 904000
https://www.baidu.com/6
send: 2022-04-08 10:13:25 904000
https://www.baidu.com/7
send: 2022-04-08 10:13:28 904000
https://www.baidu.com/8
send: 2022-04-08 10:13:31 904000
https://www.baidu.com/9先把消费者运行起来监听队列再不断生成任务这里的场景时候被动代理或者一些主进程会长期运行的程序中使用与上面不同的是 不需要 Work.is_end()方法阻塞 block也不需要设置 block默认则是True的fromthread_workerimportWorkerimportrequestsimporttimedefconsumer(iid):url="https://www.baidu.com/{0}".format(iid)resp=requests.get(url=url)print(resp.request.url)# block默认就是True的w=Worker(consumer,consumer_count=1)foriidinrange(10):w.put(iid)# 手动阻塞whileTrue:pass
Package: threatbus-cif3
Threat Bus CIFv3 PluginA Threat Bus plugin to push indicators from Threat Bus toCollective Intelligence Framework v3.The plugin uses thecifsdk (v3.x)Python
client to submit indicators received from Threat Bus into a CIFv3 instance.The plugin breaks with the pub/sub architecture of Threat Bus, because CIF does
not subscribe itself to the bus. Instead, the plugin actively contacts a CIF
endpoint.Installationpipinstallthreatbus-cif3ConfigurationConfigure this plugin by adding a section to Threat Bus'config.yamlfile, as
follows:...plugins:cif3:api:host:http://cif.host.tld:5000ssl:falsetoken:CIF_TOKENgroup:everyoneconfidence:7.5tlp:ambertags:-test-malicious...Development SetupThe following guides describe how to set up local, dockerized instances of CIF.Dockerized CIFv3Usedockerized CIFv3to set
up a local CIFv3 environment:Setup a CIFv3 docker containergitclonehttps://github.com/sfinlon/cif-docker.gitcdcif-docker
docker-composebuildEdit the docker-compose.ymlvimdocker-compose.ymlFind the sectioncifin the configuration and edit the following as
appropriate to bind port 5000 to your localhost:cif:...ports:-"5000:5000"...Start the containerdocker-composeup-d# Get an interactive shell in the container:docker-composeexeccif/bin/bash# Become the cif user:sucif# check to see if access tokens were successfully created. Copy the `admin`# token to the CIF config section:cif-tokens# Ping the router to ensure connectivity:cif--pingLicenseThreat Bus comes with a3-clause BSD license.
Package: threatbus-inmem
Threat Bus In-Memory Backbone PluginA very simplistic Threat Bus plugin that provides an in-memory backbone for data
provisioning.Message RoutingMessages are routed based on their data type. Stix2 data, like Indicators and
Sightings are routed via thestix2/topic prefix. Threat Bus-internal messages
like SnapshotRequests and SnapshotEnvelopes are routed via thethreatbus/topic prefix.SubscriptionsApplication plugins can subscribe to topics and topic prefixes.Installationpipinstallthreatbus-inmemConfigurationAdd a placeholder for this plugin's config:...plugins:backbones:inmem:...There is no further configuration needed.LicenseThreat Bus comes with a3-clause BSD license.
Package: threatbus-rabbitmq
Threat Bus RabbitMQ Backbone PluginA Threat Bus plugin to use RabbitMQ as message broker backbone.RabbitMQ PropertiesThe plugin declares one fanout exchange with a configurable name and binds a
single queue to it. Messages are all routed via that exchange. Multiple Threat
Bus instances can bind to it, but per convention must use unique queue names.
The queue name will bethreatbus.<hostname>where<hostname>is the
hostname of the machine running Threat Bus. You can overwrite this behavior in
the configuration file.Messages sent over RabbitMQ are all serialized JSON. Each object must specify atypefield. Serialization, e.g., to STIX-2 objects or to Threat Bus internal
data types, is implemented based on thetypefield.SubscriptionsApplication plugins can subscribe to topics and topic prefixes.Installationpipinstallthreatbus-rabbitmqConfigurationThe plugin requires a RabbitMQ endpoint to start. You can configure endpoint
details in the plugin configuration of the Threat Busconfig.yaml. You can
fine-tune resource behavior with thequeuesection of the config....plugins:backbones:rabbitmq:host:localhostport:5672username:guestpassword:guestvhost:/exchange_name:threatbusqueue:name_suffix:"my_suffix"# this defaults to 'hostname' if left blankname_join_symbol:.# queue will be named "threatbus" + join_symbol + name_suffixdurable:trueauto_delete:falselazy:trueexclusive:falsemax_items:100000# optional. remove property / set to 0 to allow infinite length...LicenseThreat Bus comes with a3-clause BSD license.
Package: threatbus-vast
Threat Bus VAST PluginA Threat Bus plugin that enables communication withVAST.Installationpipinstallthreatbus-vastConfigurationThe plugin uses ZeroMQ to communicate with theVAST bridge. The
plugin serves as a ZeroMQ endpoint for the bridge to connect with. It uses three
endpoints. One for managing subscriptions (and thus snapshot requests). The
other two endpoints are for plain pub-sub operations....plugins:vast:host:"127.0.0.1"manage:13370pub:13371sub:13372...Initially, the bridge only needs to know themanageendpoint. The plugin and
the vast-bridge negotiate all other internals for pub-sub message exchange at
runtime.LicenseThreat Bus comes with a3-clause BSD license.