Shared memory in multiprocessing by Pawel Putresza
Channel: Engineering Community
Date: 26/11/2021

Following presentation explores the ways we can share data between processes in Python language. Multiprocessing opens up lots of possibilities for efficient CPU usage in a language where GIL would make it impossible. However, with new possibilities comes a new issue of sharing data between independent OS processes. Here we briefly explore our options for inter process communication and then focus on specific use case of sharing a block of memory.

Overview:

How we do things in parallel
Threads and Processes: pros and cons of each
How do we share data between proceses, an overview
Solutions existing so far
shared_memory module
SharedMemory object and how to use it
ShareableList and how to use it
SharedMemoryManager