Throw an actual error for non-array contents and move the logic down into SimpleSet