Every call to `__array_wrap__` has to check the calling ufunc for membership in this list. Using a set makes that a roughly O(1) operation.