DOI: 10.5281/zenodo.3818004
title: Dataset used for detecting DNS over HTTPS by Machine Learning.

description: <strong> </strong>The dataset consists of three different data sources: DoH enabled Firefox DoH enabled Google Chrome Cloudflared DoH proxy The capture of web browser data was made using the Selenium framework, which simulated classical user browsing. The browsers received command for visiting domains taken from Alexa's top 10K most visited websites. The capturing was performed on the host by listening to the network interface of the virtual machine. Overall the dataset contains almost 5,000 web-page visits by Mozilla and 1,000 pages visited by Chrome. The Cloudflared DoH proxy was installed in Raspberry PI, and the IP address of the Raspberry was set as the default DNS resolver in two separate offices in our university. It was continuously capturing the DNS/DoH traffic created up to 20 devices for around three months. The dataset contains 1,128,904 flows from which is around 33,000 labeled as DoH. We provide raw pcap data, CSV with flow data, and CSV file with extracted features. The CSV with extracted features has the following data fields: - Label (1 - Doh, 0 - regular HTTPS)<br> - Data source<br> - Duration<br> - Minimal Inter-Packet Delay<br> - Maximal Inter-Packet Delay<br> - Average Inter-Packet Delay<br> - A variance of Incoming Packet Sizes<br> - A variance of Outgoing Packet Sizes<br> - A ratio of the number of Incoming and outgoing bytes<br> - A ration of the number of Incoming and outgoing packets<br> - Average of Incoming Packet sizes<br> - Average of Outgoing Packet sizes<br> - The median value of Incoming Packet sizes<br> - The median value of outgoing Packet sizes<br> - The ratio of bursts and pauses<br> - Number of bursts<br> - Number of pauses<br> - Autocorrelation<br> - Transmission symmetry in the 1st third of connection<br> - Transmission symmetry in the 2nd third of connection<br> - Transmission symmetry in the last third of connection The observed network traffic does not contain privacy-sensitive information. The zip file structure is: <pre><code>|-- data | |-- extracted-features...extracted features used in ML for DoH recognition | | |-- chrome | | |-- cloudflared | | `-- firefox | |-- flows...............................................exported flow data | | |-- chrome | | |-- cloudflared | | `-- firefox | `-- pcaps....................................................raw PCAP data | |-- chrome | |-- cloudflared | `-- firefox |-- LICENSE `-- README.md</code></pre> <br> When using this dataset, please cite the original work as follows: <pre><code>@inproceedings{vekshin2020, author = {Vekshin, Dmitrii and Hynek, Karel and Cejka, Tomas}, title = {DoH Insight: Detecting DNS over HTTPS by Machine Learning}, year = {2020}, isbn = {9781450388337}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3407023.3409192}, doi = {10.1145/3407023.3409192}, booktitle = {Proceedings of the 15th International Conference on Availability, Reliability and Security}, articleno = {87}, numpages = {8}, keywords = {classification, DoH, DNS over HTTPS, machine learning, detection, datasets}, location = {Virtual Event, Ireland}, series = {ARES '20} } </code></pre>
description: This work was supported by the European Union's Horizon 2020 research and innovation program under grant agreement No. 833418 and also by the Grant Agency of the CTU in Prague, grant No. SGS20/210/OHK3/3T/20 funded by the MEYS of the Czech Republic.