Fix bySegment metrics de-serialization

The prior code prevented blanket unchecked items. Please do not use unchecked unless absolutely necessary.