Title: How to filter a list of objects based on an attribute in parallel, using Pool
Tags: python,oop,parallel-processing,python-multiprocessing

Question: Given a list of objects, I want to be able to reduce that list based on the attributes of the objects. Suppose I have the following class:
class TestClass:
    def __init__(self, x, y):
        self.x = x
        self.y = y

and a list of objects from that class:
N = 10
list_of_objects = []
for i in range(N):
    x = random.randint(0, 10)
    y = random.randint(0, 10)
    tst = TestClass(x, y)
    list_of_objects.append(tst)

I want to reduce list_of_objects such that I only have elements for which self.x > self.y. Since I will have many such objects, and the actual filtering will be much more resource-intensive, I want to parallelize this. I have tried the following, which gives me a list of Nones and objects that match the criterion:
import random
from multiprocessing import Pool

class TestClass:
    def __init__(self, x, y):
        self.x = x
        self.y = y

def filter(test_object):
    if test_object.x > test_object.y:
        return test_object
    else:
        return None

def parallel_process(operation, input, pool):
    result = pool.map(operation, input)
    return result

if __name__ == "__main__":
    N = 10
    list_of_objects = []
    for i in range(N):
        x = random.randint(0, 10)
        y = random.randint(0, 10)
        tst = TestClass(x, y)
        list_of_objects.append(tst)

    process_count = 2
    process_pool = Pool(process_count)
    result = parallel_process(filter, list_of_objects, process_pool)

    print(result)

which returns a list of Nones and objects that match the criterion:
[<__main__.TestClass object at 0x1033f0610>, <__main__.TestClass object at 0x103e9f8d0>, <__main__.TestClass object at 0x103e9fa90>, None, <__main__.TestClass object at 0x103e9fb10>, <__main__.TestClass object at 0x103e9fb50>, None, <__main__.TestClass object at 0x103e9fcd0>, None, <__main__.TestClass object at 0x103e9fd50>]

I could, in principle, then get rid of Nones:
result = [i for i in result if i is not None]

Is this the best way of doing this? By "best" here I mean: (i) doing it in a pythonic and concise way, and (ii) the most efficient way within the confines of (i).

Answer: If you run your program, this is what will happen:
The Pool will create two secondary Processes.
Every item in list_of_objects (instances of TestClass) will be converted to bytes using the Python pickle protocol.  These bytes will be transmitted to the secondary Processes through a Pipe or a Queue.  In the secondary Process, the bytes will be un-pickled to reconstitute objects of TestClass.
The secondary Process will call filter and convert its returned value to bytes using the Python pickle protocol.  These bytes will be transmitted back to the main Process through a Pipe or a Queue.  In the main Process, the bytes will be un-pickled to reconstitute the object that was returned by filter - either a valid instance of TestClass or None.  Finally, as you point out, the main Process will have to perform additional logic to remove the Nones.
This all happens behind the scenes, in the standard library.  This machinery is necessary because Python Processes do not share an address space; in other words, each Python Process acts almost like an independent program, with its own objects and its own memory.  The secondary Processes must receive data from the main Process and return the results somehow.  Python does this using pickle, Pipes and Queues.
Can you make your program faster using this mechanism?  For your simple test program the answer is certainly no.  For your actual program the answer may be yes.  But it seems to me that you are trying to optimize the performance of a program you haven't even written yet.  Why not write the program in a simple, straightforward way and see if the performance is acceptable, before undertaking the difficult task of writing a multiprocessing-based application?