Take advantage of segment-based hidden flag storage.

To avoid querying this states for every single column and row.

Change-Id: I3e570ae5309c826e809c170ea4635dd37c9b0d43