Make sure to keep track of all shared memory objects.