Refactor masks to support non-array masks

Work in progress, do not merge!