Correctly detect mutation on serialized columns mapping to binary

Fixes #16701