Actually hide the size label when not in use

It was previously just made transparent, and when it's positioning got out
of sync it would act as an invisible click block.

Change-Id: I47f623b79323d85e90e88204bcb1b2d8f12b60f8
Merge "Actually hide the size label when not in use"