# DNS Firewall Based on Machine Learning
## 4 DNS Dataset  
In this section, the DNS dataset creation process is presented. The data collection for non-malicious domains was based on DNS logs, and uses four types of records (A, AAAA, CNAME and MX). The domains were collected from Rapid7 Labs [23], which provide datasets of DNS requests from their Project Sonar. This data are open and provide a structured schema that facilitates the extraction process. A total of 45,000 domains were randomly selected from these lists.  
The malicious domains were collected from the SANS [24] public list. This list contains malicious domains validated using common publicly available virus detectors. A total of 45,000 domains were randomly selected from the list.  
A total of 90,000 domain names constitutes the entry point to the information gathering process using OSINT sources. To do so we implemented a DNS configuration and submitted each name to the resolver. This way we recreate the process as it really is. The DNS server output is used as input for a Python application that processes the domain name and enriches it with useful information. As the output, a CSV file is created. The last column is named \"Class\" and it is the dependent variable classifying each domain as malicious or non-malicious. The dataset is composed of thirty-four features per DNS domain name:  
* Domain-Baseline DNS used to enrich data, e.g., derive features;
* DNSRecordType-DNS record type queried;
* MXDnsResponse-The response from a DNS request for the record type MX;
* TXTDnsResponse-The response from a DNS request for the record type TXT;
* HasSPFInfo-If the DNS response has Sender Policy Framework attribute;
* HasDkimInfo-If the DNS response has Domain Keys Identified Email attribute;
* HasDmarcInfo-If the DNS response has Domain-Based Message Authentication;
* IP-The IP address for the domain;
* DomainInAlexaDB-If the domain is registered in the Alexa DB;
* CommonPorts-If the domain is available on common ports (21, 22, 23, 25, 53, 80, 110, 143, 161, 443, 445, 465, 587, 993, 995, 3306, 3389, 7547, 8080, 8888);
* CountryCode-The country code associated with the IP of the domain;
* RegisteredCountryCode-The country code defined in the domain registration process (WHOIS);
* CreationDate-The creation date of the domain (WHOIS);
* LastUpdateDate-The last update date of the domain (WHOIS);
* ASN-The Autonomous System Number for the domain;
* HttpResponseCode-The HTTP/HTTPS response status code for the domain;
* RegisteredOrg-The organization name associated with the domain (WHOIS);
* SubdomainNumber-The number of subdomains for the domain;
* Entropy-The Shannon Entropy of the domain name;
* EntropyOfSubDomains-The mean value of the entropy for the subdomains;
* StrangeCharacters-The number of characters different from [a-zA-Z] and considering the existence maximum of two numeric integer values;
* TLD-The Top Level Domain for the domain;
* IpReputation-The result of the blocklisted search for the IP;
* DomainReputation-The result of the blocklisted search for the domain;
* ConsoantRatio-The ratio of consonant characters in the domain;
* NumericRatio-The ratio of numeric characters in the domain (_numeric_chars_/_len_ (_domain_));
* SpecialCharRatio-The ratio of special characters in the domain;
* VowelRatio-The ratio of vowel characters in the domain;
* ConsoantSequence-The maximum number of consecutive consonants in the domain;
* VowelSequence-The maximum number of consecutive vowels in the domain;
* NumericSequence-The maximum number of consecutive numbers in the domain;
* SpecialCharSequence-The maximum number of consecutive special characters in the domain;DomainLength-The length of the domain;
* Class-The class of the domain (malicious = 0 and non-malicious = 1);  
The process of data gathering, extraction of information and storage into a structured file is explained in [25]. The source code used, the produced dataset, and the exploratory analysis is also publicly available.