Package: cross-process-bridge
Cross Process BridgeThis package is a utility for creating an instance of a class in a new process and seamlessly calling its functions from
the original processWhen would I ever want to do that?Good question, I'm glad you asked.Let's say you have a class, that changes thecwdof the program during the course of its actions. If you have another
thread running, which needs a differentcwd, they will interfere with each other because threads share acwdAnother example would be if you have a process running as root, which needs to do something risky that you wouldn't want
a privileged user doing, or just doesn't need high privileges. With this module you could callos.setuid()in the
child process without dropping privileges in the main process.In particular, this module is useful for a situation where the use of your class is intertwined with the rest of your
code to a point which would make it difficult to separate. In that case, simple replace the creation of instances with
transformed instances using this module and every instance will have its own process where its methods execute.So why wouldn't I just use multiprocessing?Themultiprocessingmodule allows running a python function in a separate process usingmultiprocessing.Processlike this:process=multiprocessing.Process(target=func_to_run,args=(args,to,send))process.start()# do other thingsprocess.join()why multiprocessing isn't enoughThe above example will run the functionfunc_to_runwith the given arguments. the program can then do other things and
wait for the process usingprocess.join()when it needs to.However, it only runs thefunc_to_runand then exits.Cross Process ClassThis module allows creating an instance of a class in one process and then seamlessly calling its functions from the
main processExamplelet's say I have a classAwhich has a methodawhich prints the pid of the process in which it is runningclassA:defa(self):print('a',os.getpid())and let's further assume, that I didn't just create this class to check whether what I did worked, and say we just
really want to run an instance of this class in another process for completely unrelated reasonsThe module exports a classCrossProcessBridgewhich must be inherited from in order to run in another processIf, for example I want to run an instance of theAclass in a separate process:I will create a new class, in this caseBwhich will inherit from bothAand fromCrossProcessBridgeimportant- the new class must inherit first from your class -Ain this example - and thenCrossProcessBridgeclassB(A,CrossProcessBridge):passafter creating an instance ofBI will then call thestart()method from theCrossProcessBridgeclass - this will
create another process, and in it create an instance ofAimportantly, not ofB! it will create an instance of
the originalAclassI can now call any methods that exist in theAclass on theBinstance I have created, and they will be called in the
new process.when I am done I can callstop()which will stop the processComplete example:importosfromcross_process_bridgeimportCrossProcessBridgeclassA:@staticmethoddefa():print('a',os.getpid())classB(A,CrossProcessBridge):passdefmain():pid=os.getpid()print(pid)# print the pid of the original processb=B()# create the bridge instanceb.start()# start the process and create an instance in itb.a()# call the `a()` function - will print a different pidb.stop()# stop the processif__name__=='__main__':main()PitfallsMemory spaceAs with anything involving multiprocessing, a significant pitfall is separate memory spaces.for example:classA:defadd_to_list(self,lst):lst.append('a')classB(A,CrossProcessBridge):passdefmain():b=B()b.start()lst=[]b.add_to_list(lst)print(lst)b.stop()if__name__=='__main__':main()in this example theAclass has a methodadd_to_listwhich gets a list and adds an'a'to it.theprintin the line after the function call, will output an empty list because thelstobject in memory in the main
process is not the same list in the child process - it is copied into it when it is passed as a variable but changes to
it will not be reflected in the main process.Themultiprocessing.Managerclass can share simple objects including lists between processes:frommultiprocessingimportManagerfromcross_process_bridgeimportCrossProcessBridgeclassA:defadd_to_list(self,lst):lst.append('a')classB(A,CrossProcessBridge):passdefmain():withB()asb:lst=Manager().list()b.add_to_list(lst)print(lst)if__name__=='__main__':main()in this example, appending'a'to the listisreflected in the main process because the list is a proxy object
handled bymultiprocessing.ManagerMethod call-throughif your class has methods calledstartorstop, they will be called when starting and stopping the process - this is
to allow any setup and teardown you want to do. youcanpass parameters to the start and stop methods, but it is
recommended that they have no parameters for simplicity and because when using thewithkeyword on the class,start()andstop()are called with no parametersOther usagesin addition to the classic usage above, the class can also be used as a context manager using thewithkeyword:importosfromcross_process_bridgeimportCrossProcessBridgeclassA:@staticmethoddefa():print('a',os.getpid())classB(A,CrossProcessBridge):passdefmain():pid=os.getpid()print(pid)withB()asb:b.a()if__name__=='__main__':main()when entering thewithblock thestart()method will be called and when exiting, thestop()method will be called.