AWSのDHCPセットに複数のnameserverを指定しても意味がない

VPCにはDHCPセットという機能がありそこでnameserverを複数指定できます。
思った挙動と違ったので記事にしました。

## まずはローカルで試してみる

/etc/resolv.confにnameserverを複数指定すると、
DNSリクエストが失敗した時次のnameserverに問い合わせます

docker環境を用意したのでそれを使って確かめます

```
docker-compose up
```
クライアントの/etc/resolv.confはこう
```
nameserver 192.168.0.3
nameserver 192.168.0.22
nameserver 192.168.0.11
```
DNS1は問い合わせ先を知っていて、DNS2は知らないです

そしてclientからnslookupを打つと名前解決してくれます
存在しないDNSだろうがzoneファイルに何もなかろうが関係ないです。

```
# nslookup web.local
;; communications error to 192.168.0.3#53: timed out
;; communications error to 192.168.0.3#53: timed out
;; communications error to 192.168.0.3#53: timed out
;; Got SERVFAIL reply from 192.168.0.22, trying next server
Server:         192.168.0.11
Address:        192.168.0.11#53

Name:   web.local
Address: 192.168.0.100
;; communications error to 192.168.0.3#53: timed out
;; communications error to 192.168.0.3#53: timed out
;; communications error to 192.168.0.3#53: timed out
;; Got SERVFAIL reply from 192.168.0.22, trying next server
```

もちろんcurlも通ります

```
# curl web.local
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
```

## まとめると

環境にもよりますがnameserverは複数指定するとDNSリクエストが失敗した場合次のサーバーに問い合わせます。
options rotateが必要になることもあります。

## DHCPオプションセットでは
一方AWSにDNSをデプロイしてそこからDHCPオプションセットをそのDNSに向き先を変えるとnameserverが複数指定されているのは意味をなさなくなります。

ここにcloudformationがあります。

```yaml
Parameters:
  EnvironmentName:
    Description: An environment name that is prefixed to resource names
    Type: String
  VpcCIDR:
    Description: Please enter the IP range (CIDR notation) for this VPC
    Type: String
    Default: 192.168.0.0/24
  PublicSubnetCIDR:
    Description: Please enter the IP range (CIDR notation) for the public subnet in
      the first Availability Zone
    Type: String
    Default: 192.168.0.0/25
  UbuntuAMI:
    Description: Ubuntu AMI of your region
    Type: String
    Default: ami-0efcece6bed30fd98
  IPADDRDNS1:
    Description: IP address of primary dns server
    Type: String
    Default: 192.168.0.11
  IPADDRDNS2:
    Description: IP address of secondary dns server
    Type: String
    Default: 192.168.0.22
  IPADDRWEB:
    Description: IP address of web server
    Type: String
    Default: 192.168.0.100
  IPADDRCLIENT:
    Description: IP address of client
    Type: String
    Default: 192.168.0.123


Resources:

  VPC:
    Type: AWS::EC2::VPC
    Properties:
      CidrBlock: !Ref VpcCIDR
      EnableDnsSupport: true
      EnableDnsHostnames: true
      Tags:
        - Key: Name
          Value: !Ref EnvironmentName

  InternetGateway:
    Type: AWS::EC2::InternetGateway
    Properties:
      Tags:
        - Key: Name
          Value: !Ref EnvironmentName

  InternetGatewayAttachment:
    Type: AWS::EC2::VPCGatewayAttachment
    Properties:
      InternetGatewayId: !Ref InternetGateway
      VpcId: !Ref VPC

  PublicSubnet:
    Type: AWS::EC2::Subnet
    Properties:
      VpcId: !Ref VPC
      AvailabilityZone: !Select [ 0, !GetAZs '' ]
      CidrBlock: !Ref PublicSubnetCIDR
      MapPublicIpOnLaunch: true
      Tags:
        - Key: Name
          Value: !Sub ${EnvironmentName} Public Subnet (AZ1)

  PublicRouteTable:
    Type: AWS::EC2::RouteTable
    Properties:
      VpcId: !Ref VPC
      Tags:
        - Key: Name
          Value: !Sub ${EnvironmentName} Public Routes

  DefaultPublicRoute:
    Type: AWS::EC2::Route
    DependsOn: InternetGatewayAttachment
    Properties:
      RouteTableId: !Ref PublicRouteTable
      DestinationCidrBlock: 0.0.0.0/0
      GatewayId: !Ref InternetGateway

  PublicSubnetRouteTableAssociation:
    Type: AWS::EC2::SubnetRouteTableAssociation
    Properties:
      RouteTableId: !Ref PublicRouteTable
      SubnetId: !Ref PublicSubnet

  DNSSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: dns-sg
      GroupDescription: Security group to pass DNS traffics
      VpcId: !Ref VPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          CidrIp: 0.0.0.0/0
        - IpProtocol: tcp
          FromPort: 53
          ToPort: 53
          CidrIp: 0.0.0.0/0
        - IpProtocol: udp
          FromPort: 53
          ToPort: 53
          CidrIp: 0.0.0.0/0
      SecurityGroupEgress:
        - IpProtocol: -1
          FromPort: 0
          ToPort: 0
          CidrIp: 0.0.0.0/0

  WebSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: web-sg
      GroupDescription: Security group to pass HTTP traffics
      VpcId: !Ref VPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          CidrIp: 0.0.0.0/0
        - IpProtocol: tcp
          FromPort: 80
          ToPort: 80
          CidrIp: 0.0.0.0/0
      SecurityGroupEgress:
        - IpProtocol: -1
          FromPort: 0
          ToPort: 0
          CidrIp: 0.0.0.0/0

  ClientSecurityGroup:
    Type: AWS::EC2::SecurityGroup
    Properties:
      GroupName: client-sg
      GroupDescription: Security group with ssh rule
      VpcId: !Ref VPC
      SecurityGroupIngress:
        - IpProtocol: tcp
          FromPort: 22
          ToPort: 22
          CidrIp: 0.0.0.0/0
      SecurityGroupEgress:
        - IpProtocol: -1
          FromPort: 0
          ToPort: 0
          CidrIp: 0.0.0.0/0

  KnowingDNS:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: !Ref UbuntuAMI
      InstanceType: t2.micro
      SecurityGroupIds:
        - !Ref DNSSecurityGroup
      SubnetId: !Ref PublicSubnet
      PrivateIpAddress: !Ref IPADDRDNS1
      Tags:
        - Key: Name
          Value: KnowingDNS
      UserData: 
        'Fn::Base64':
          'Fn::Sub': 
            - |
              #!/bin/bash 
              cat << 'eof' > /tmp/named.conf.options 
              options {
                      directory "/var/cache/bind";
              
                      // If there is a firewall between you and nameservers you want
                      // to talk to, you may need to fix the firewall to allow multiple
                      // ports to talk.  See http://www.kb.cert.org/vuls/id/800113
              
                      // If your ISP provided one or more IP addresses for stable 
                      // nameservers, you probably want to use them as forwarders.  
                      // Uncomment the following block, and insert the addresses replacing 
                      // the all-0's placeholder.
              
                      // forwarders {
                      //      0.0.0.0;
                      // };
              
                      //========================================================================
                      // If BIND logs error messages about the root key being expired,
                      // you will need to update your keys.  See https://www.isc.org/bind-keys
                      //========================================================================
                      dnssec-validation auto;
              
                      listen-on port 53 { any; };
                      listen-on-v6 port 53 { any; };
                      allow-query     { any; };
              };
              eof
              
              cat << 'eof' > /tmp/named.conf 
              // This is the primary configuration file for the BIND DNS server named.
              //
              // Please read /usr/share/doc/bind9/README.Debian.gz for information on the 
              // structure of BIND configuration files in Debian, *BEFORE* you customize 
              // this configuration file.
              //
              // If you are just adding zones, please do that in /etc/bind/named.conf.local
              
              include "/etc/bind/named.conf.options";
              include "/etc/bind/named.conf.local";
              include "/etc/bind/named.conf.default-zones";
              
              zone "local" {
                  type master;
                  file "/etc/bind/db.web";
              };
              eof
              
              cat << eof > /tmp/db.web
              ;
              ; BIND data file for local loopback interface
              ;
              \$TTL    604800
              @       IN      SOA     web.local. web.local. (
                                            2         ; Serial
                                       604800         ; Refresh
                                        86400         ; Retry
                                      2419200         ; Expire
                                       604800 )       ; Negative Cache TTL
              ;
              @       IN      NS      web.local.
              web     IN      A       ${IP_ADDR_WEB}
              eof
              sudo apt-get -y update && \
              sudo apt-get install -y bind9 bind9utils dnsutils && \
                  mv /tmp/named.conf.options /etc/bind/ && \
                  mv /tmp/named.conf /etc/bind/ && \
                  mv /tmp/db.web /etc/bind/ && \
                  ls /etc/bind
              sudo systemctl restart named
            - IP_ADDR_WEB: !Ref IPADDRWEB

  UnknowingDNS:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: !Ref UbuntuAMI
      InstanceType: t2.micro
      SecurityGroupIds:
        - !Ref DNSSecurityGroup
      SubnetId: !Ref PublicSubnet
      PrivateIpAddress: !Ref IPADDRDNS2
      Tags:
        - Key: Name
          Value: UnknowingDNS
      UserData: 
        'Fn::Base64':
          'Fn::Sub': 
            - |
              #!/bin/bash
              cat << 'eof' > /tmp/named.conf.options 
              options {
                      directory "/var/cache/bind";
              
                      // If there is a firewall between you and nameservers you want
                      // to talk to, you may need to fix the firewall to allow multiple
                      // ports to talk.  See http://www.kb.cert.org/vuls/id/800113
              
                      // If your ISP provided one or more IP addresses for stable 
                      // nameservers, you probably want to use them as forwarders.  
                      // Uncomment the following block, and insert the addresses replacing 
                      // the all-0's placeholder.
              
                      // forwarders {
                      //      0.0.0.0;
                      // };
              
                      //========================================================================
                      // If BIND logs error messages about the root key being expired,
                      // you will need to update your keys.  See https://www.isc.org/bind-keys
                      //========================================================================
                      dnssec-validation auto;
              
                      listen-on port 53 { any; };
                      listen-on-v6 port 53 { any; };
                      allow-query     { any; };
              };
              eof
              
              cat << 'eof' > /tmp/named.conf 
              // This is the primary configuration file for the BIND DNS server named.
              //
              // Please read /usr/share/doc/bind9/README.Debian.gz for information on the 
              // structure of BIND configuration files in Debian, *BEFORE* you customize 
              // this configuration file.
              //
              // If you are just adding zones, please do that in /etc/bind/named.conf.local
              
              include "/etc/bind/named.conf.options";
              include "/etc/bind/named.conf.local";
              include "/etc/bind/named.conf.default-zones";
              
              zone "local" {
                  type master;
                  file "/etc/bind/db.web";
              };
              eof
              
              cat << eof > /tmp/db.web
              ;
              ; BIND data file for local loopback interface
              ;
              \$TTL    604800
              @       IN      SOA     unknown.local. unknown.local. (
                                            2         ; Serial
                                       604800         ; Refresh
                                        86400         ; Retry
                                      2419200         ; Expire
                                       604800 )       ; Negative Cache TTL
              ;
              @       IN      NS      unknown.local.
              unknown     IN      A       ${IP_ADDR_WEB}
              eof
              EOF
              
              sudo apt-get -y update && \
              sudo apt-get install -y bind9 bind9utils dnsutils && \
                  mv /tmp/named.conf.options /etc/bind/ && \
                  mv /tmp/named.conf /etc/bind/ && \
                  mv /tmp/db.web /etc/bind/ && \
                  ls /etc/bind
              sudo systemctl restart named
            - IP_ADDR_WEB: !Ref IPADDRWEB

  WebInstance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: !Ref UbuntuAMI
      InstanceType: t2.micro
      SecurityGroupIds:
        - !Ref WebSecurityGroup
      SubnetId: !Ref PublicSubnet
      PrivateIpAddress: !Ref IPADDRWEB
      Tags:
        - Key: Name
          Value: Web
      UserData:
        'Fn::Base64': |
            #!/bin/bash 
            sudo apt-get update -y && sudo apt-get install -y nginx
            sudo systemctl restart nginx

  ClientInstance:
    Type: AWS::EC2::Instance
    Properties:
      ImageId: !Ref UbuntuAMI
      InstanceType: t2.micro
      SecurityGroupIds:
        - !Ref ClientSecurityGroup
      SubnetId: !Ref PublicSubnet
      PrivateIpAddress: !Ref IPADDRCLIENT
      Tags:
        - Key: Name
          Value: Client
      UserData:
        'Fn::Base64': |
            #!/bin/bash 
            sudo apt-get -y update && apt-get install -y dnsutils curl

  DhcpOptions12:
    Type: AWS::EC2::DHCPOptions
    Properties:
      DomainNameServers: 
        - !Ref IPADDRDNS1
        - !Ref IPADDRDNS2
      Tags:
        - Key: Name
          Value: knowing

  DhcpOptions21: 
    Type: AWS::EC2::DHCPOptions
    Properties:
      DomainNameServers: 
        - !Ref IPADDRDNS2
        - !Ref IPADDRDNS1
      Tags:
        - Key: Name
          Value: unknowing

#  DhcpOptionsAssociation: 
#    Type: AWS::EC2::VPCDHCPOptionsAssociation
#    Properties:
#      DhcpOptionsId: !Ref DhcpOptions
#      VpcId: !Ref VPC

```

このテンプレートからスタックを作成するとDNSが2台立ちます。

DHCPオプションセットをDNS2を優先に設定します
```
associate-dhcp-options
--dhcp-options-id <value>
--vpc-id <value>
```

作ったVPCにAmazon linuxとUbuntuを起動します。image-idにはAmazon linuxとUbuntuのAMIを指定します。

```
 run-instances
[--image-id <value>]
[--instance-type <value>]
[--security-groups <value>]
[--subnet-id <value>]
```

起動したEC2の中身を見るとVPCにDHCPオプションセットを追加したので/etc/resolv.confが変わっています。

```Amazon linux
$ cat /etc/resolv.conf 
# This is /run/systemd/resolve/resolv.conf managed by man:systemd-resolved(8).
# Do not edit.
#
# This file might be symlinked as /etc/resolv.conf. If you're looking at
# /etc/resolv.conf and seeing this text, you have followed the symlink.
#
# This is a dynamic resolv.conf file for connecting local clients directly to
# all known uplink DNS servers. This file lists all configured search domains.
#
# Third party programs should typically not access this file directly, but only
# through the symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a
# different way, replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.

nameserver 192.168.0.22
nameserver 192.168.0.11
search .
```

```Ubuntu
$ cat /etc/resolv.conf 
# This is /run/systemd/resolve/stub-resolv.conf managed by man:systemd-resolved(8).
# Do not edit.
#
# This file might be symlinked as /etc/resolv.conf. If you're looking at
# /etc/resolv.conf and seeing this text, you have followed the symlink.
#
# This is a dynamic resolv.conf file for connecting local clients to the
# internal DNS stub resolver of systemd-resolved. This file lists all
# configured search domains.
#
# Run "resolvectl status" to see details about the uplink DNS servers
# currently in use.
#
# Third party programs should typically not access this file directly, but only
# through the symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a
# different way, replace this symlink by a static file or a different symlink.
#
# See man:systemd-resolved.service(8) for details about the supported modes of
# operation for /etc/resolv.conf.

nameserver 127.0.0.53
options edns0 trust-ad
search .
```

この状態でnslookupしてもローカルとは違い見つかりません。

```
nslookup web.local
Server:         192.168.0.22
Address:        192.168.0.22#53

** server can't find web.local: NXDOMAIN
```

もちろんcurlも通りません。
```
curl web.local
curl: (6) Could not resolve host: web.local
```

EC2は再起動すると/etc/resolve.confが元に戻るのですが一時的に/etc\resolve.confを編集します。
```
cat << eof | sudo tee /etc/resolv.conf 
nameserver 192.168.0.11
nameserver 192.168.0.22
search .
eof
```
すると通るようになるので単にnameserverに指定したDNSに行かないだけみたいです。

```
nslookup web.local
Server:         192.168.0.11
Address:        192.168.0.11#53

Name:   web.local
Address: 192.168.0.100
```
rotateオプションをnslookupは失敗しますが、curlは通るようになります。この辺はよくわからないです。
```
cat << eof | sudo tee /etc/resolv.conf 
options rotate
nameserver 192.168.0.22
nameserver 192.168.0.11
search .
eof
```

```
 curl web.local
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
```

どちらにせよDHCPオプションにはoptionを指定することができないのでnameserverを複数指定しても最初のDNSサーバーしか見れないです。
EC2のようにDNS指定できるといいですが、lambdaなどのサービスは大変です。

どうしてもということになればRoute 53 ResolverなどでPROVIDED DNSをいじるしかないかなと思います。