Package: pesim
pesim: A Minimalist Discrete Event Simulation Library in PythonAboutI write this as the underlying simulation engine of my container yard simulator, with the goals of:Simple interfaces.Cython-ised for speedDO ONE THING WELLSo what is the one thing for a discrete event simulation? Making the events happen on right time and in right order. This is the only thingpesimdo.(I am definitely inspired bySimPy, and I like the way it uses generator andyieldstatement to represent activities. However, for my projects, SimPy is too heavy and too slow. )InstallpipinstallpesimCythonis needed if there is not a corresponding pre-builtwheelfor the targeting platform.DocumentationSeeDocumentation.TutorialA Minimal ExamplefrompesimimportEnvironment,Lock,TIME_PASSEDfromrandomimportuniformdefplayer(self,name:str,ball:Lock):yieldball.acquire(self)# catch the ballprint(name,self.time)# ping!yieldself.time+uniform(5,10),TIME_PASSED# let the ball flyball.release()# so others can catchif__name__=='__main__':ball=Lock()withEnvironment(stop_time=1000)asenv:env.process(player,"ping",ball,loop_forever=True)env.process(player,"pong",ball,loop_forever=True)A Producer-consumer ExampleLet's simulate the classical producer-consumer scenario. Suppose there are two producers and one consumer. Each producer repeatedly producestasksfor every 50-100s, while the consumer also takes 50-100s to process onetask. Let's define thet_j-thtaskproduced by thep_i-th producer as(p_i, t_j)(both count from 0, for example(0,0)is the first task produced by the first produce).pesimis aprocess-based discrete event simulation engine. The basic active components are modelled asprocesses(From now on, we useprocessto represent the process in simulation, andOS processto represent the processes in operation system).  There are two ways to define aprocess:Transforming a generator function into aprocess; orInheriting theProcessbase class.I will show the two ways, respectively. Before starting, let's first import some auxiliary functions and classes.fromcollectionsimportdequefromrandomimportrandintWe will use adeque()list as the task queue, and userandintto randomise task intervals and processing time.Defining ProcessesThe first way to define a process is using a generator function.ConsumerThe consumer process can be defined as such:frompesimimportTIME_FOREVER,TIME_PASSEDclassReason(IntEnum):TaskDone=auto()NewTask=auto()defconsumer(self,task_queue):whileTrue:whiletask_queue:task=task_queue.popleft()print("[{}]Consumer gets task:{}".format(self.time,task))yieldself.time+randint(50,100),Reason.TaskDoneprint("[{}]Consumer completes task:{}".format(self.time,task))reason=yieldTIME_FOREVER,TIME_PASSEDprint("[{}]Consumer activated by reason{}".format(cur_time,reason))The function takes two arguments. The first one isself, which is compulsive and represents the simulationprocess(object of classProcess). The most important attribute of theselfisself.time(of typefloat), which is the current simulated time. All theprocessesin one simulationEnvironment(we will create the environment later) share the same time, while we can access it from anyprocess'stimevalue.From second arguments, there are data that can be passed to the process by users. Here we pass in atask queuecreated bydeque(). To remain simple,pesimitself does not provide any additional data structure to represent shared resources among differentprocesses. All theprocesseswill be executed in a single thread of a singleOS process, so it will be safe to just use any suitable Python object to share information.One can easily find that the function is actually ageneratorfunction, which periodicallyyields a 2-tuple. Theseyieldstatements are the core to represent activities. So, when theprocessneed to do something for a time period, it need toyieldthe execution, and when the predefined amount of time is passed, the execution will be resumed. The first item in the tuple is a time, which is theprocess's supposednext activation timeas afloat, and the second item is areasonas anint. If the activity is completed as expected, theprocesswill be "activated" again at thenext activation time, and the reason is returned by theyieldstatement.The activity may also be interrupted externally with another reason. In this case, theprocesswill be activated at an earlier time point, and the statement's returned value will tell the reason. Another usage of thereasonis to order the events. That is, when multipleprocessesare supposed to be activated at the same time, the one yields theleastreasonwill be awaken first. Here we define two reasons:classReason(IntEnum):TaskDone=auto()NewTask=auto()Note that, thisreasonneedn't to be exactly ofinttype, any type that can be casted tointcan be used. As you can see, here we use theIntEnumto assign values to the reasons automatically.Knowing this, the lineyieldself.time+randint(50,100),Reason.TaskDoneis clear. It randomises a task processing time between 50s and 100s, and pauses theprocessuntil the task is completed. If everything is normal, when re-activated, the reasonReason.TaskDonewill be returned.There is anotheryieldstatement in the code:reason=yieldTIME_FOREVER,TIME_PASSEDThere is also a pre-definedreason:TIME_PASSED, which is an infinite number. Theprocessyields with this reason will always be resumed last. Also,TIME_FOREVERcan be considered as an infinite time. So, when the task queue is emptied, thisyieldstatement will put theprocessinto a forever sleeping, as the consumer does not know when the next task will arrive. But don't worry, thisprocesscan be activated externally by a producer later when a new task is created.Finally, we need to put the code into an infinite loop, so the consumer can keep processing incoming tasks.ProducerThen, the producer is defined as such:defproducer(self,idx,consumer,task_queue,num_task):for_inrange(num_task):task=idx,itask_queue.append(task)print("[{}]P{}creates task:{}".format(self.time,idx,task))ifconsumer.next_activation_time()==TIME_FOREVER:consumer.activate(self.time,Reason.NewTask)yieldself.time+randint(50,100),Reason.NewTaskExcept for theselfandtask_queuearguments, the producerprocessalso takes anidxargument as its index, the consumerprocess, and also the maximum number or tasks to be produced. The logic is simple, after creating a task and inserting it into the task queue, the lineyieldself.time+randint(50,100),Reason.NewTaskmakes theprocesswait for a random time interval between 50s and 100s until it can create another task.Besides, after creating the task and before theprocessbeing paused, the producer will check whether theconsumer's next activation time isTIME_FOREVER. If so, it will externally activate the consumer with the reasonReason.NewTask:ifconsumer.next_activation_time()==TIME_FOREVER:consumer.activate(self.time,Reason.NewTask)(Later we will show another notification way usingSynchronisation Primitives.)Running SimulationNow, let's set up and start the simulation.if__name__=='__main__':frompesimimportEnvironmentenv=Environment()task_queue=deque()c=env.process(consumer,task_queue)foridxinrange(2):env.process(producer,idx,c,task_queue,10)env.start()env.join()env.finish()To run the simulation, we need to create anEnvironmentinstance, and then useEnvironment.process()to transform the producer and consumer functions intoprocess. After that,env.start()starts the simulation,env.join()runs the simulation untilTIME_FOREVER, and after that,env.finish()dose some finishing and cleaning work.If we do not want the simulation to be executed untilTIME_FOREVER, theenv.join()can be replaced withenv.run_until(time). Also, theprocessneeds not to be created before theEnvironmentinstance is started. The following code shows another scenario: one producer and one consumer are started first, and then after 1 hour, another producer will join. Also, we don't want to simulate the scenario until forever; instead, we only simulate it for 24 hours.env=Environment(start=True)task_queue=deque()env.start()c=env.process(consumer,task_queue)p0=env.process(producer,0,c,task_queue,100)env.run_until(3600)p1=env.process(producer,1,c,task_queue,100)env.run_until(24*3600)env.finish()TheEnvironmentcan also be used with thewithstatement. It willstartwhen entering the context andjoin/finishautomatically when exiting. The following code is equivalent to the code above.withEnvironment()asenv:task_queue=deque()c=env.process(consumer,task_queue)p0=env.process(producer,0,c,task_queue,100)env.run_until(3600)p1=env.process(producer,1,c,task_queue,100)Method 2: InheritingProcessClassAprocesscan also been created by inheriting theProcessclass. The above code implements same logics as we shown in last section, except that we now maketask_queuean attribute ofConsumerand create a new methodsubmitto push the task into the queue and check/activate theConsumerprocess.frompesimimportProcessclassConsumer(Process):def__init__(self,env):self.task_queue=deque()super(Consumer,self).__init__(env)defsubmit(self,task):self.task_queue.append(task)ifself.next_activation_time()==TIME_FOREVER:self.activate(self.time,Reason.NewTask)def_wait(self):ifself.task_queue:returnself.time,Reason.NewTaskelse:returnsuper(Consumer,self)._wait()def_process(self):task=self.task_queue.popleft()print("[{}]Consumer gets task:{}".format(self.time,task))yieldself.time+randint(50,100),Reason.TaskDoneprint("[{}]Consumer completes task:{}".format(self.time,task))classProducer(Process):def__init__(self,env,idx,consumer,num_task):self.idx=idxself.consumer=consumerself.num_task=num_tasksuper(Producer,self).__init__(env)def__call__(self):foriinrange(self.num_task):task=self.idx,iprint("[{}]P{}creates task:{}".format(self.time,self.idx,task))self.consumer.submit(task)yieldself.time+randint(50,100),Reason.NewTaskThere are three methods_wait,_processand__call__can be overloaded. If the__call__is overloaded, the code inside will be executed as theprocesslogic. On the other hand, the_waitand_processmethods make creatingprocessesthat do repeated jobs easy. If__call__is not overloaded,_processwill be executed for processing one task, which contains theyieldstatements, and when_processis finished,_waitmethod is called to determine the next job's time, which is also a 2-tuple of time and reason. Similarly,_waitcan just return(TIME_FOREVER, TIME_PASSED)to sleep forever and wait others to awake theprocesslater (this is_wait's default implementation).In this example, use the following code to run the simulation.withEnvironment()asenv:c=Consumer(env)p0=Producer(env,0,c,100)env.run_until(3600)p1=Producer(env,1,c,100)Synchronisation Primitivespesimprovides some basic synchronisation primitives includingLock,Semaphore,ConditionandRLock(re-entrant lock). Note that, all the simulationprocesses' code are running within one thread, so there is no need to use these primitives to avoid data conflicts. Instead, these primitives are designed forcoordinatingdifferentprocesses. For example, if oneprocesswants to do something first, and then pass the task to anotherprocess, wait anotherprocessto do some other work on the task, and then continue the processing, usingConditionmake the thing easy.Let's slightly modify our producer-consumer example:now we want the producer to create a new task after its previous task is processed, instead of waiting for a random time interval.To do so, we can create aConditionfor each task, and make the producer wait on thatCondition. Here is the code:frompesimimportProcess,ConditionclassConsumer(Process):# __init__ and _wait as previousdefsubmit(self,task,condition):self.task_queue.append((task,condition))ifself.next_activation_time()==TIME_FOREVER:self.activate(self.time,Reason.NewTask)def_process(self):task,condition=self.task_queue.popleft()print("[{}]Consumer gets task:{}".format(self.time,task))yieldself.time+randint(50,100),Reason.TaskDonecondition.set(Reason.TaskDone)#reason is optionalprint("[{}]Consumer completes task:{}".format(self.time,task))classProducer(Process):# __init__ as previousdef__call__(self):foriinrange(self.num_task):task=self.idx,iprint("[{}]P{}creates task:{}".format(self.time,self.idx,task))condition=Condition()#by default it is clearedself.consumer.submit(task,condition)yieldcondition.wait(self)Now, theConsumer.submit()takes arguments of not only a task, but also aConditionobject. Once the task is processed, the consumer will callCondition.set(reason)on the corresponding condition.On the producer side, after submit a task, theyieldstatement is nowyieldcondition.wait()Instead of pausing the producerprocessfor a certain amount of time, now, this statement will pause the producer until theconditionis set. If the condition is already set, it will resume theprocessimmediately. TheCondition.wait()needs one argument, which is theprocessthat needs to be activated when the condition is set.Now, let's make one step further. In our current implementation, when the task queue is emptied, the consumer will sleep forever, and it is producers' responsibility to check consumer's state and activate it now and them. This is not elegant at all. Instead, we can use aSemaphoreto do the coordination. Here is the code.frompesimimportProcess,Condition,SemaphoreclassConsumer(Process):def__init__(self,env):self.task_queue=deque()self.sem=Semaphore(0)super(Consumer,self).__init__(env)defsubmit(self,task,condition):self.task_queue.append((task,condition))self.sem.release()def_wait(self):returnself.sem.acquire(self)def_process(self):task,condition=self.task_queue.popleft()print("[{}]Consumer gets task:{}".format(self.time,task))yieldself.time+randint(50,100),REASON_TASK_DONEcondition.set()print("[{}]Consumer completes task:{}".format(self.time,task))classProducer(Process):# __init__ as previousdef__call__(self):foriinrange(self.num_task):task=self.idx,icondition=Condition()self.consumer.submit(task,condition)print("[{}]P{}creates task:{}".format(self.time,self.idx,task))yieldcondition.wait()First, we add an attributesemof typeSemaphoretoConsumer. EverySemaphorehas an internal value, and two methodsacquire()andrelease()are available. Whenever theSemaphore.release()is called, the internal value will increased by 1. On the other hand, whenSemaphore.acquire()is called, there are two situations. If the internal value is equal or less than 0, theprocesswill be paused, until the value because greater than 0. After that, the value will be decreased by 1 again. Otherwise, if the internal value is already greater than 0, it will directly decrease by 1 and does not pause theprocess.In our code, we set thesemwith the initial value of 0. Whenever a task is submitted,sem.release()is called, and before processing the task, inConsumer._wait(), the consumer needs toacquirethesemfirst. In this way, the value ofsemis always the number of tasks in queue, and when the queue is empty, callingsem.acquire()pauses theprocessuntil a new task is inserted.There are alsoLockandRLock(re-entrant lock) that can be used in other scenarios.====================================================Above are all the functionalities provided bypesim.