Deserializing Doubly Tagged JSON in Rust
Tags: rust, api
Published: 2019-11-15 19:22:04

I needed to deserialize a JSON data structure that contained versioning and type information.

```json
[
  {
    "provider": "AZURE",
    "version": 1,
    "credentials": {
      "clientId": "",
      "clientSecret": ""
    }
  },
  {
    "provider": "AWS",
    "version": 1,
    "credentials": {
      "accessKeyId": "",
      "secretAccessKey": ""
    }
  }
]
```

My Rust representation had to represent that the datatypes could change with subsequent versions and capture (preferably at the type level) the differences between the sub-structs. I tried a few different patterns for representing this data in Rust but settled on one that seemed be the nicest to work with later.

```rust
pub enum ProviderType {
    AZURE,
    AWS,
}

// How do I deserialize this?
pub enum ProviderConfiguration {
    AzureConfigV1(AzureConfigV1),
    AwsConfigV1(AwsConfigV1),
}

// AZURE
pub struct AzureConfigV1 {
    pub credentials: AzureCredentialsV1,
}

impl AzureConfigV1 {
    pub const TYPE: ProviderType = ProviderType::AZURE;
    pub const VERSION: u16 = 1;
}

pub struct AzureCredentialsV1 {
    pub client_id: String,
    pub client_secret: String,
}

// AWS
pub struct AwsConfigV1 {
    pub credentials: AwsCredentialsV1,
}

impl AwsConfigV1 {
    pub const TYPE: ProviderType = ProviderType::AWS;
    pub const VERSION: u16 = 1;
}

pub struct AwsCredentialsV1 {
    pub access_key_id: String,
    pub secret_access_key: String,
}
``` 

I wanted to be able to deserialize these types with as little effort as possible. Serde provides some pretty great attributes for [deserializing enums](https://serde.rs/enum-representations.html) but none of them covered the combination of version _and_ type going on here. The manual deserialization in the [serde documentation](https://serde.rs/deserialize-struct.html) was pretty confusing to me as someone new to Rust, it's presented without much commentary.

I tried writing `Deserialize` using a `Visitor` like the example, but didn't have access to the original `deserializer` inside the `visit_map`.

```rust
impl<'de> Deserialize<'de> for ProviderConfiguration {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        #[derive(Deserialize)]
        #[rename_all = "camelCase")]
        enum Fields {
            Provider,
            Version
        };

        struct ProviderConfigurationVisitor;

        impl<'de> Visitor<'de> for ProviderConfigurationVisitor {
            type Value = ProviderConfiguration;

            fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
                formatter.write_str("ProviderConfiguration")
            }

            fn visit_map<V>(self, mut map: V) -> Result<ProviderConfiguration, V::Error>
            where
                V: MapAccess<'de>,
            {
                let mut provider = None;
                let mut version = None;
                while let Some(key) = map.next_key()? {
                    match key {
                        Field::Provider => {
                            if provider.is_some() {
                                return Err(de::Error::duplicate_field("provider"));
                            }
                            provider = Some(map.next_value()?);
                        }
                        Field::Version => {
                            if version.is_some() {
                                return Err(de::Error::duplicate_field("version"));
                            }
                            version = Some(map.next_value()?);
                        }
                    }
                }
                let provider = provider.ok_or_else(|| de::Error::missing_field("provider"))?;
                let version = version.ok_or_else(|| de::Error::missing_field("version"))?;
                match (provider, version) {
                    ("AWS", 1) => AwsConfigV1::deserialize(deserializer).map(ProviderConfiguration::AwsConfigV1) // the fn we're in doesn't close over the original deserializer
                }
            }
        }

        const FIELDS: &'static [&'static str] = &["provider", "version"];
        deserializer.deserialize_struct("ProviderConfiguration", FIELDS, ProviderConfigurationVisitor)
    }
}
```

So I thought I may be able to deserialize the whole original value into a helper that only cares about `provider` and `version` then use the resulting helper to deserialize the value as the correct struct.

```rust
impl<'de> Deserialize<'de> for ProviderConfiguration {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        #[derive(Deserialize)]
        struct Helper {
            provider: ProviderType,
            version: u16,
        };

        let helper = Helper::deserialize(deserializer)?;
        match helper {
            Helper {
                provider: AwsConfigV1::TYPE,
                version: AwsConfigV1::VERSION,
            } => Ok(ProviderConfiguration::AwsConfigV1(
                AwsConfigV1::deserialize(deserializer)?,
            )),
            _ => Err(de::Error::duplicate_field("nothing")),
        }
    }
}
```

This didn't work because I couldn't move the `deserializer` and then use it again. This felt like the right track though, and much closer semantically to the way I expected this to work.

I tried a few other pathways that didn't pan out (multiple flattened internally tagged enums), and in trying to figure out why they didn't work eventually stumbled across an [old question](https://users.rust-lang.org/t/serde-json-flatten-nested-internally-tagged-enums/13178/2) that very closely mapped to my use case.

```rust
impl<'de> Deserialize<'de> for ProviderConfiguration {
    fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
    where
        D: Deserializer<'de>,
    {
        #[derive(Deserialize)]
        struct Helper {
            provider: ProviderType,
            version: u16
        };
        
        let v = Value::deserialize(deserializer).map_err(de::Error::custom)?;
        let helper = Helper::deserialize(&v).map_err(de::Error::custom)?;
        match helper {
            Helper {
                provider: AwsConfigV1::TYPE,
                version: AwsConfigV1::VERSION,
            } => Ok(ProviderConfiguration::AwsConfigV1(
                AwsConfigV1::deserialize(&v).map_err(de::Error::custom)?,
            )),
            _ => Err(de::Error::custom("")),
        }
    }
}
```

Perfect! Here's a [playground](https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=f81a042b512f57a966ad0ecc7974d743) with the final working code.