Deconstructing SPF Records with Rust
Tags: rust, dns, spf
Published: 2021-04-08 01:31:26

Hi. If you have visited my blog before you might have seen my series on using the [trust-dns-resolver](https://crates.io/crates/trust-dns-resolver) crate. As part of that series I looked at querying DNS TXT records. During the process I was reminded of SPF records. Something I used to deal with in a previous position. And this got me to thinking about using [SPF records](https://www.easy365manager.com/what-is-spf-sender-policy-framework/) as a way to learn more about working with rust.

Keep in mind I am new to rust and these will just be some of my thoughts as I work through the task I have given myself. This will not be a production project, merely a learning experience. I will not be documenting all the code here. I will give a reference to the git repository at the end of this article.

# TL;DR;
## Some caveats
  - I will not be implementing a full solution to deconstruct an SPF record at this time.
  - Not all edge cases may be covered, though I will do my best to do so.
  - I will initially only be looking at: `redirect`, `include`, `ip4` and `ip6` records contained within the SPF record. 
  - If there is no qualifier present, it is assumed to be `+` which equates to a `Pass`
  - During this phase I will not be looking at using *generics* as I want to understand the basics of using `structs`
  - I will not be writing any code that validates that the SPF record is syntactically correct.
  - The code is going to be fairly ugly and will get polished as I go through.
  - I am not looking at SPF2 (pra/mfrom)

## Some SPF rules I am fairly familiar with, though I could be mistaken. (smile)
- There can only be one `redirect` in a single SPF record.
- The qualifier is optional
- There can be multiple instances of `ip4`, `ip6` and `include`
- There can only be a single `a` and `mx` when they are not followed by a `:` character.
- With the exception of a `redirect` there must an `all` at the end of an SPF record.

# Let's get started
I am not going to start off with how to create a rust project. There is plenty on that out there. And if you read my previous series on [Working with the trust-dns-resolver Crate](http://localhost:1313/series/working-with-the-trust-dns-resolver-crate/), you already know how to get started. In fact this is an extension to that series.

## Structs
Following the basic rules I listed above each *mechanism* can have an optional *qualifier*. This will need to be dealt with. And each *mechanism* can occur multiple times in most cases.  

### The Include Struct
An `include` can be represented in the following ways: (Not an exhaustive list)
  - include:_spf.example.com
  - +include:_spf.otherdomain.com
  - \-include:_spf.somedomain.com

```rust
#[derive(Default, Debug, Clone)]
struct Include {
    qualifier: char,
    txt: String,
}
```
#### Code Explanation
- I will store the `qualifier` as a char, this will allow me to hold `+`,`-`,`~`, or `?`. Alternatively I could create an enum type. But this is over kill since I am just trying to learning at this point. 
- I will store the `_spf.example.com` portion in a `String` which lives on the `heap` in memory.
- To save on some code, I am asking the compiler to generate `Default`, `Debug` and `Clone` for me using the `#[derive]`

### Implementing the Include Struct
Rust is interesting in that it allows us to create functions which are tied to a struct. In my mind this is some what like the struct is acting as an object, and the functions are then used as methods of the struct.

```rust
impl Include {
    fn new(qualifier: char, txt: String) -> Self {
        Self { qualifier, txt }
    }
    fn as_mechanism(&self) -> String {
        let mut txt = String::new();
        if self.qualifier != '+' {
            txt.push(self.qualifier);
        }
        txt.push_str("include:");
        txt.push_str(self.txt.as_str());
        txt
    }
    fn is_pass(&self) -> bool {
        if self.qualifier == '+' {
            true
        } else {
            false
        }
    }
    fn is_fail(&self) -> bool {
        if self.qualifier == '-' {
            true
        } else {
            false
        }
    }
    fn is_softfail(&self) -> bool {
        if self.qualifier == '~' {
            true
        } else {
            false
        }
    }
    fn is_neutral(&self) -> bool {
        if self.qualifier == '?' {
            true
        } else {
            false
        }
    }
}
```

#### Explanation 
The function `new` allows us to instantiate and struct with the passed values. We can call it in the following ways
```rust
let var_a = Include::new('+', "_spf.example.com".as_string());
```
- Single quotes can be used here to denote a type of `char`
- Text enclosed in double-quotes are technically of type `str` and since my function definition expects a `String`, I am transforming the string. This is purely for the example purposes. I could have also easily passed `String::from("_spf.example.com")`  

The `is_*` functions all simple check the value stored in `qualifier` and return a boolean.  

The `as_mechanism()` function is able to reconstruct the spf mechanism and return it as a `String`. If the `qualifier` is **NOT** a `+`, it is appended to the string being created. Otherwise it is omitted.

### The Ip4 Struct
An `ip4` can be presented in the following ways: (not an exhaustive list)
  - ip4:157.55.9.128/25
  - +ip4:ip4:72.14.192.0/18
  - Or as a single host `ip4:X.X.X.X` which would be used as `X.X.X.X/32`

  
```rust
#[derive(Debug, Clone)]
struct Ip4 {
    qualifier: char,
    ip: IpNetwork,
}
```
Here I am only using `Debug` and `Clone` as I am implementing my own `Default` merely for learning purposes. The astute reader might already conclude that this struct is is very similar to `Include` with the only real difference being `String` and `IpNetwork`. IpNetwork is provided by the [ipnetwork](https://crates.io/crates/ipnetwork) crate. I am using this as it provides lots of the basic functions I will need and also many more that would make it useful for other applications.

```rust
impl Default for Ip4 {
    fn default() -> Self {
        Self {
            qualifier: '+',
            ip: IpNetwork::V4("0.0.0.0/0".parse().unwrap()),
        }
    }
}
```

With his `Default` we can call code such as:
```rust
let ip: Ip4 = ip4::default();
```

### Implementing the Ip4 Struct
```rust
impl Ip4 {
    fn new(qualifier: char, ip: IpNetwork) -> Self {
        Self { qualifier, ip }
    }
    fn as_string(&self) -> String {
        self.ip.to_string()
    }
    fn as_mechanism(&self) -> String {
        let mut ip4_string = String::new();
        if self.qualifier != '+' {
            ip4_string.push(self.qualifier);
        }
        ip4_string.push_str("ip4:");
        ip4_string.push_str(self.ip.to_string().as_str());
        ip4_string
    }
    fn as_ip(&self) -> IpNetwork {
        self.ip
    }
    fn is_pass(&self) -> bool {
        if self.qualifier == '+' {
            true
        } else {
            false
        }
    }
    fn is_fail(&self) -> bool {
        if self.qualifier == '-' {
            true
        } else {
            false
        }
    }
    fn is_softfail(&self) -> bool {
        if self.qualifier == '~' {
            true
        } else {
            false
        }
    }
    fn is_neutral(&self) -> bool {
        if self.qualifier == '?' {
            true
        } else {
            false
        }
    }
}
```
As you can see I have had to duplicate a lot of code.... I think `generic` is going to be in the near future.

### The Ip6 Struct

To keep this a little shorter I will just give the basics since it is identical to Ip4

```rust
#[derive(Debug, Clone)]
struct Ip6 {
    qualifier: char,
    ip: IpNetwork,
}

impl Ip6 {
    fn new(qualifier: char, ip: IpNetwork) -> Self {
        Self { qualifier, ip }
    }
}

impl Default for Ip6 {
    fn default() -> Self {
        Self {
            qualifier: '+',
            ip: IpNetwork::V6("FE80::1".parse().unwrap()),
        }
    }
}
```

This is essentially the same as `Ip4` because `IpNetork` is an `enum` of `V4` and `V6`

### Defining the SPF1 Struct
Many of the `mechanisms` in SPF are optional and nicely, rust actually provides the type `Option`. Also a `mechanism` can appear several times, as a `list` and rust provides `Vec` which is a way to store a list of some type.

```rust
#[derive(Default, Debug)]
struct Spf1 {
    source: String,
    include: Option<Vec<Include>>,
    redirect: Option<String>,
    is_redirected: bool,
    a: Option<Vec<A>>,
    mx: Option<Vec<Mx>>,
    ip4: Option<Vec<Ip4>>,
    ip6: Option<Vec<Ip6>>,
    all_qualifier: char,
}
```
#### Explanation of the SPF1 Struct

- source: The original string received from the TXT DNS lookup. This will be deconstructed to populate the SPF struct.
- include: This is an optional list of includes so I am creating a `Vec` of `Include` and then wrapping that inside and `Option` since it might actually be `None`
- `Ip4` and `Ip6` are handle in the same way as `Include`
- redirect: Only ever occurs once in a record. So I am placing it directly within SPF1, but again it is optional, so I wrap it inside an `option`
- is_redirect: Is closely tied to `direct` and simply stores a `bool` value
- all_qualifier: This is a special case. It is only ever not present when the spf record is a `redirect` so I am only ensure that I have a `qualifer` value.

### Implementing SPF1 Struct
```rust
impl Spf1 {
    fn new(str: &String) -> Self {
        Self {
            source: str.clone(),
            include: None,
            redirect: None,
            is_redirected: false,
            a: None,
            mx: None,
            ip4: None,
            ip6: None,
            all_qualifier: '+',
        }
    }
    fn parse(&mut self) {
        let records = self.source.split_whitespace();
        let mut vec_of_includes: Vec<Include> = Vec::new();
        let mut vec_of_ip4: Vec<Ip4> = Vec::new();
        let mut vec_of_ip6: Vec<Ip6> = Vec::new();
        for record in records { // Main loop
            if record.contains("redirect=") {
                let items = record.rsplit("=");
                for item in items {
                    self.redirect = Some(item.to_string());
                    break;
                }
                self.is_redirected = true;
            } else if record.contains("include:") {
                let qualifier_and_modified_str = return_and_remove_qualifier(record, 'i');
                for item in record.rsplit(":") {
                    vec_of_includes.push(Include::new(
                        qualifier_and_modified_str.0,
                        item.to_string(),
                    ));
                    break; // skip the 'include:'
                }
            } else if record.contains("ip4:") {
                let qualifier_and_modified_str = return_and_remove_qualifier(record, 'i');
                if let Some(raw_ip4) = qualifier_and_modified_str.1.strip_prefix("ip4:") {
                    let network: Ip4 =
                        Ip4::new(qualifier_and_modified_str.0, raw_ip4.parse().unwrap());
                    vec_of_ip4.push(network);
                }
            } else if record.contains("ip6:") {
                let qualifier_and_modified_str = return_and_remove_qualifier(record, 'i');
                if let Some(raw_ip6) = qualifier_and_modified_str.1.strip_prefix("ip6:") {
                    let network: Ip6 =
                        Ip6::new(qualifier_and_modified_str.0, raw_ip6.parse().unwrap());
                    vec_of_ip6.push(network);
                }
            } else if record.ends_with("all") {
                self.all_qualifier = return_and_remove_qualifier(record, 'a').0;
            }
        } // End of Main Loop
        if vec_of_includes.len() > 0 {
            self.include = Some(vec_of_includes);
        };
        if vec_of_ip4.len() > 0 {
            self.ip4 = Some(vec_of_ip4);
        };
        if vec_of_ip6.len() > 0 {
            self.ip6 = Some(vec_of_ip6);
        };
    }
    // Code omitted for brevity
}
```
#### Explanation
```rust
let mut spf = Spf1::new(&some_spf_record.to_string());
```

This function takes in a reference, borrows, a string and then internally uses `clone()` to make a copy of the string which then remains valid for the life time of the struct. Is also initialises the variables within the struct.

### Deconstructing time (Finally)
Enter the real work horse in this process. It's currently a little monolithic at the moment. But, it's getting the job done.
```rust
spf.parse();
```

- The first four lines set things up.
  - Split the string in to seperate parts based on white space. Placing them into an `iter()` 
  - Create our mutable `Vec` varibles for `Include`, `Ip4` and `Ip6`
- Next we start the main loop which will parse each `mechanism`
  - redirect:
    - If the current record contains `rediect=` we use `rsplit()` which returns a new `iter()` and access the right side of the split first. We wrap that result into a `Some()` because Spf1.redirect expects an `Option`. Next we `break` from the loop since we don't need any other parts of this iter(). We are also not concerned about the `qualifier` in this case as it is usually omitted.
  - include:
    - Similar to the `redirect` we do an rsplit() but on ":"
    - Create a new `Include` struct and push that value on to the `vec_of_includes` that was defined at the start of `parse()`
    - Call `break` to exist inner loop
  - `ip6` has an issue. I am using `strip_prefix()` because I am unable to use `rsplit()` due to `ip6` being comprised of multiple `:` characters. This is an issue because I am using `strip_prefix()` on `"ip6:"` This *will* fail if there is a `qualifer` present in the string.
    - ie: "+ip6:....". To handle this I have a helper function `return_and_remove_qualifier()` which checks for the presence of a `qualifier`, and returns a `tuple`. The tuple contains the `qualifier` and an updated version of original string with the `qualifier` removed if it was present. This ensure that the `strip_prefix()` function will match correctly.
  - ip4/6:
    - Call `return_and_remove_qualifier(original_string, 'i')`
    - `strip_prefix()` returns `Some()`, so we take a peek inside and conditionally create an `Ip6` struct.
    - Push it on vec_of_ip6, I actually do the same process for `ip4`
  - all;
    - If the the `mechanism` ends with `all` we just store the `qualifier` directly into `self.all_qualifier`
  - When we reach the end of the main loop we then conditionally wrap our various `vec_of.*` structs into their own `Some()`  and assign them to their matching property in `Spf1`

  ## Test Domain and Test Output.
  
  ### Test Domains
  I am using the following records at the time of writing.
  
| Domain | Spf Record |
|--------|------------|
| gmail.com | v=spf1 redirect=_spf.google.com |
| _netblocks.google.com | v=spf1 ip4:35.190.247.0/24 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.192.0/18 ip4:74.125.0.0/16 ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:209.85.128.0/17 ip4:216.58.192.0/19 ip4:216.239.32.0/19 ~all |
| _netblocks2.google.com | v=spf1 ip6:2001:4860:4000::/36 ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:4000::/36 ip6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ~all |
| hotmail.com | v=spf1 ip4:157.55.9.128/25 include:spf.protection.outlook.com include:spf-a.outlook.com include:spf-b.outlook.com include:spf-a.hotmail.com include:_spf-ssg-b.microsoft.com include:_spf-ssg-c.microsoft.com ~all |


### Test Output (gmail.com)
```zsh
List of TXT records found for gmail.com.
TXT Record 1:
globalsign-smime-dv=CDYX+XFHUw2wml6/Gb8+59BsH31KzUr6c1l2BPvqKX8=
TXT Record 2:
v=spf1 redirect=_spf.google.com

Decontructing SPF Record
SPF1: v=spf1 redirect=_spf.google.com

There are no include elements
There are no ip4 networks
There are no ip4 spf records.

Is a redirect: true

redirect: _spf.google.com
mechanism: redirect=_spf.google.com
```

### Test Output (_netblocks.google.com)
```zsh
List of TXT records found for _netblocks.google.com.
TXT Record 1:
v=spf1 ip4:35.190.247.0/24 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.192.0/
18 ip4:74.125.0.0/16 ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:209.85.128.0/17 ip4:216.58.192.0/19 i
p4:216.239.32.0/19 ~all

Decontructing SPF Record
SPF1: v=spf1 ip4:35.190.247.0/24 ip4:64.233.160.0/19 ip4:66.102.0.0/20 ip4:66.249.80.0/20 ip4:72.14.
192.0/18 ip4:74.125.0.0/16 ip4:108.177.8.0/21 ip4:173.194.0.0/16 ip4:209.85.128.0/17 ip4:216.58.192.
0/19 ip4:216.239.32.0/19 ~all

There are no include elements
List of ip4 networks/hosts:
35.190.247.0/24
64.233.160.0/19
66.102.0.0/20
66.249.80.0/20
72.14.192.0/18
74.125.0.0/16
108.177.8.0/21
173.194.0.0/16
209.85.128.0/17
216.58.192.0/19
216.239.32.0/19

List of ip4 mechanisms:
ip4:35.190.247.0/24
ip4:64.233.160.0/19
ip4:66.102.0.0/20
ip4:66.249.80.0/20
ip4:72.14.192.0/18
ip4:74.125.0.0/16
ip4:108.177.8.0/21
ip4:173.194.0.0/16
ip4:209.85.128.0/17
ip4:216.58.192.0/19
ip4:216.239.32.0/19

Is a redirect: false
```

### Test Output (_netblocks2.google.com) / Not being displayed at this time.

```zsh
List of TXT records found for _netblocks2.google.com.
TXT Record 1:
v=spf1 ip6:2001:4860:4000::/36 ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:4000::/3
6 ip6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ~all

Decontructing SPF Record
SPF1: v=spf1 ip6:2001:4860:4000::/36 ip6:2404:6800:4000::/36 ip6:2607:f8b0:4000::/36 ip6:2800:3f0:40
00::/36 ip6:2a00:1450:4000::/36 ip6:2c0f:fb50:4000::/36 ~all

There are no include elements
There are no ip4 networks
There are no ip4 spf records.

Is a redirect: false
```

### Test Output (hotmail.com)

```zsh
List of TXT records found for hotmail.com.
TXT Record 1:
google-site-verification=gqFmgDKSUd3XGU_AzWWdojRHtW3_66W_PC3oFvQVZEw
TXT Record 2:
v=spf1 ip4:157.55.9.128/25 include:spf.protection.outlook.com include:spf-a.outlook.com include:spf-
b.outlook.com include:spf-a.hotmail.com include:_spf-ssg-b.microsoft.com include:_spf-ssg-c.microsof
t.com ~all

Decontructing SPF Record
SPF1: v=spf1 ip4:157.55.9.128/25 include:spf.protection.outlook.com include:spf-a.outlook.com includ
e:spf-b.outlook.com include:spf-a.hotmail.com include:_spf-ssg-b.microsoft.com include:_spf-ssg-c.mi
crosoft.com ~all

Include Mechanisms:
include:spf.protection.outlook.com
include:spf-a.outlook.com
include:spf-b.outlook.com
include:spf-a.hotmail.com
include:_spf-ssg-b.microsoft.com
include:_spf-ssg-c.microsoft.com
List of ip4 networks/hosts:
157.55.9.128/25

List of ip4 mechanisms:
ip4:157.55.9.128/25

Is a redirect: false
```

You can get a copy of the complete code at the time of this writing [here](https://github.com/Bas-Man/learning-rust-trust-dns-resolver/tree/SPF-INITIAL)

I think I will next work on using `generics`

Thanks for reading.