I'm concerned about this scaling properly, if you have a small whitelist, it means processing the blacklist is going to be O(mn) over the number of items and the members of the item lists, which is a pretty big number.
How about keeping the whitelist and blacklist seperate, and doing:
foreach( item in item_groups )
    if( item in blacklist && item not in whitelist )
        remove/deactivate item;
That way also, "item in blacklist" (a method probably) could check against both id and category, and so could "item not in whitelist".  So you could blacklist a category and whitelist individuals, or blacklist individuals and whitelist categories, not having to worry about merging the lists somehow.