DOI: 10.5281/zenodo.13330073
title: A Dataset of Information (DNS, IP, WHOIS/RDAP, TLS, GeoIP) for a Large Corpus of Benign, Phishing, and Malware Domain Names 2024

description: The dataset contains DNS records, IP-related features, WHOIS/RDAP information, information from TLS handshakes and certificates, and GeoIP information for 368,956 benign domains from Cisco Umbrella, 461,338 benign domains from the actual CESNET network traffic, 164,425 phishing domains from PhishTank and OpenPhish services, and 100,809 malware domains from various sources like ThreatFox, The Firebog, MISP threat intelligence platform, and other sources. The ground truth for the phishing dataset was double-check with the VirusTotal (VT) service. Domain names not considered malicious by VT have been removed from phishing and malware datasets. Similarly, benign domain names that were considered risky by VT have been removed from the benign datasets. The data was collected between March 2023 and July 2024. The final assessment of the data was conducted in August 2024.

The dataset is useful for cybersecurity research, e.g. statistical analysis of domain data or feature extraction for training machine learning-based classifiers, e.g. for phishing and malware website detection.

The dataset was created using software available in the associated GitHub repository nesfit/domainradar-dib.

Data Files





The data is located in the following individual files:



benign_umbrella.json - data for 368,956 benign domains from Cisco Umbrella,

benign_cesnet.json - data for 461,338 benign domains from the CESNET network,

phishing.json - data for 164,425 phishing domains, and

malware.json - data for 100,809 malware domains.



The schema.json file contains a JSON Schema with detailed description of the data entries.


Data Structure



Both files contain a JSON array of records generated using mongoexport (in the MongoDB Extended JSON (v2) format in Relaxed Mode). The following table documents the structure of a record. Please note that:



some fields may be missing (they should be interpreted as nulls), 

extra fields may be present (they should be ignored).












Field name 









Field type 









Nullable 









Description 











domain_name 









String 









No 









The evaluated domain name 











url 









String 









No 









The source URL for the domain name 











evaluated_on 









Date 









No 









Date of last collection attempt 











source 









String 









No 









An identifier of the source 











sourced_on 









Date 









No 









Date of ingestion of the domain name 











dns 









Object 









Yes 









Data from DNS scan 











rdap 









Object 









Yes 









Data from RDAP or WHOIS 











tls 









Object 









Yes 









Data from TLS handshake 











ip_data 









Array of Objects 









Yes 









Array of data objects capturing the IP addresses related to the domain name 











malware_type









String









No









The malware type/family or “unknown” (only present in malware.json) 











DNS data (dns field) 



 
 
 








A 









Array of Strings 









No 









Array of IPv4 addresses 











AAAA 









Array of Strings 









No 









Array of IPv6 addresses 











TXT 









Array of Strings 









No 









Array of raw TXT values 











CNAME 









Object 









No 









The CNAME target and related IPs 











MX 









Array of Objects 









No 









Array of objects with the MX target hostname, priority and related IPs 











NS 









Array of Objects 









No 









Array of objects with the NS target hostname and related IPs 











SOA 









Object 









No 









All the SOA fields, present if found at the target domain name 











zone_SOA 









Object 









No 









The SOA fields of the target’s zone (closest point of delegation), present if found and not a record in the target domain directly 











dnssec 









Object 









No 









Flags describing the DNSSEC validation result for each record type 











ttls 









Object 









No 









The TTL values for each record type 











remarks 









Object 









No 









The zone domain name and DNSSEC flags 











RDAP data (rdap field) 



 
 
 








copyright_notice 









String 









No 









RDAP/WHOIS data usage copyright notice 











dnssec 









Bool 









No 









DNSSEC presence flag 











entitites 









Object 









No 









An object with various arrays representing the found related entity types (e.g. abuse, admin, registrant). The arrays contain objects describing the individual entities. 











expiration_date 









Date 









Yes 









The current date of expiration 











handle 









String 









No 









RDAP handle 











last_changed_date 









Date 









Yes 









The date when the domain was last changed 











name 









String 









No 









The target domain name for which the data in this object are stored 











nameservers 









Array of Strings 









No 









Nameserver hostnames provided by RDAP or WHOIS 











registration_date 









Date 









Yes 









First registration date 











status 









Array of Strings 









No 









The state of the registered object (see RFC 7483, section 10.2.2)











terms_of_service_url 









String 









No 









URL of the RDAP usage ToS 











url 









String 









No 









URL of the RDAP entity 











whois_server 









String 









No 









WHOIS server address 











TLS data (tls field) 



 
 
 








cipher 









String 









No 









TLS cipher suite description according to IANA











protocol 









String 









No 









One of “TLS”, ”TLSv1.2”, ”TLSv1.3” 











certificates 









Array of Objects 









No 









Array of objects representing the certificate chain, the first element is the root certificate 











IP data (elements in the ip_data array) 



 
 
 








ip  









String 









No 









The IP address 











from_record 









String 









No 









The type of the DNS record the address was captured from 











remarks 









Object 









No 









Ping round-trip time, “is alive” flag and rdap/geo/asn evaluation dates 











rdap 









Object 









Yes 









RDAP data, similar to DNS RDAP, see the JSON Schema for details 











geo 









Object 









Yes 









Geolocation data from the GeoLite2 City database (e.g. latitude, longitude, city, country, etc.) 











asn 









Object 









Yes 









Autonomous system data from the GeoLite2 ASN database (ASN, organization, network) 







Acknowledgements

We would like to thank the OpenPhish Team for grating permission to use and publish their dataset. We also thank VirusTotal for providing us access to the API for research purposes.

This dataset includes GeoLite2 data created by MaxMind, available from https://www.maxmind.com.

The research has been supported by the Flow-based Encrypted Traffic Analysis project, no. VJ02010024, granted by the Ministry of the Interior of the Czech Republic and the Smart Information Technology for a Resilient Society project, no. FIT-S-23-8209, granted by Brno University of Technology.