Remove time deserialization spec for non common attributes