Improve pmap perf by using sets, not lists