DOI: 10.5281/zenodo.18700746
title: VPN-nonVPN-Dataset

description: A flow-level dataset of WireGuard tunnel traffic with matched encrypted-side features and application labels

A flow-level dataset derived from paired captures on both sides of a WireGuard VPN tunnel. Each flow combines NFStream-exported pre-tunnel attributes (including application labels from deep packet inspection) with encrypted-side statistics computed from matched WireGuard transport packets.

Dataset summary




Property
Value


Capture sessions
2


Total duration
~80 hours


Total flows
226,454 (122,975 + 103,479)


Columns per flow
123


Unique application names
 81 / 75 per session 


Unique application categories
20 / 19 per session


Protocol split
~47% TCP, ~53% UDP


Capture location
 Budapest, Hungary


VPN endpoint
 Surfshark WireGuard, Prague, Czech Republic


Active devices
10




Repository structure

VPN-nonVPN-Dataset

    data



session1_flows.parquet:   Session 1 flow records 122,975 rows

session2_flows.parquet :  Session 2 flow records 103,479 rows


    code



packet_matching.py :  Phase 3 inner outer packet matching

flow_matching.py:       Phase 3 packet to flow assignment and aggregation


Data files

Both Parquet files share the same 123-column schema. Each row corresponds to one bidirectional flow.




File
Rows
Capture date
 Duration


session1_flows.parquet
122,975 
2025-12-01
 ~48 h


session2_flows.parquet
 103,479
2025-12-04 
 ~32 h




 

Column schema

Each flow record contains 123 columns organized into three groups.

NFStream flow attributes (columns 1-89)

Standard flow fields exported by NFStream, including:



Identifiers: id, expiration_id, src_ip, dst_ip, src_port, dst_port, protocol, ip_version

Timing: bidirectional_first_seen_ms, bidirectional_last_seen_ms, bidirectional_duration_ms, and per-direction equivalents (src2dst_*, dst2src_*)

Volume: bidirectional_packets, bidirectional_bytes, and per-direction equivalents

Packet size statistics: bidirectional_min_ps, bidirectional_mean_ps, bidirectional_stddev_ps, bidirectional_max_ps, and per-direction equivalents

Inter-arrival time statistics: bidirectional_min_piat_ms, bidirectional_mean_piat_ms, bidirectional_stddev_piat_ms, bidirectional_max_piat_ms, and per-direction equivalents

TCP flags: per-flag packet counts (SYN, ACK, FIN, RST, PSH, URG, CWR, ECE) for bidirectional and per-direction

SPLT arrays (first 255 packets per flow):



splt_direction — packet direction sequence

splt_ps — packet size sequence (bytes)

splt_piat_ms — inter-arrival time sequence (milliseconds)


Application labels:



application_name — nDPI-assigned application name (e.g., DNS, TLS.Facebook, QUIC)

application_category_name — nDPI-assigned category (e.g., Web, Network, Chat)

application_is_guessed, application_confidence


DPI metadata: requested_server_name, client_fingerprint, server_fingerprint, user_agent, content_type


Flow assignment metadata (columns 90-94)



flow_id — flow identifier

flow_start_ms, flow_end_ms — flow time boundaries in milliseconds

k5_fwd, k5_rev — forward and reverse 5-tuple keys


Encrypted-side derived features (columns 95-123)

Statistics computed from matched WireGuard tunnel packets (the table summarizes the encrypted-side features added to each flow and maps them to the closest NFStream attributes, supporting structured comparison and feature selection for learning tasks based solely on tunnel-side statistics)










Column name 









Description 









Closest NFStream Column 











matched_packets 









Number of matched packet rows assigned to the flow 









bidirectional_packets 











outer_bytes 









Sum of outer_padded_length over matched packets assigned to the flow 









bidirectional_bytes 











first_matched_time_ms 









Minimum inner_time among assigned packet rows, converted to milliseconds 









bidirectional_first_seen_ms 











last_matched_time_ms 









Maximum inner_time among assigned packet rows, converted to milliseconds 









bidirectional_last_seen_ms 











outer_duration_ms 









Inner-timeline matched duration, computed as last_matched_time_ms - first_matched_time_ms 









bidirectional_duration_ms 











outer_first_matched_time_ms 









Minimum outer_time among assigned packet rows, converted to milliseconds 









bidirectional_first_seen_ms 











outer_last_matched_time_ms 









Maximum outer_time among assigned packet rows, converted to milliseconds 









bidirectional_last_seen_ms 











outer_capture_duration_ms 









Outer-timeline matched duration, computed as outer_last_matched_time_ms - outer_first_matched_time_ms 









bidirectional_duration_ms 











mean_outer_pkt_size 









Mean of outer_padded_length over matched packets assigned to the flow 









bidirectional_mean_ps 











std_outer_pkt_size 









Standard deviation of outer_padded_length over matched packets assigned to the flow 









bidirectional_stddev_ps 











outer_packet_rate 









matched_packets divided by outer_duration_ms in seconds 









bidirectional_packets/ (bidirectional_duration_ms / 1000) 











outer_byte_rate 









outer_bytes divided by outer_duration_ms in seconds 









bidirectional_bytes/ (bidirectional_duration_ms / 1000)  











outer_bytes_in 









Sum of outer_padded_length for matched packets with direction = INBOUND 









dst2src_bytes 











outer_bytes_out 









Sum of outer_padded_length for matched packets with direction = OUTBOUND 









src2dst_bytes 











outer_min_piat_ms_in 









Minimum positive inter-arrival time between consecutive matched outer packets in the INBOUND direction, in milliseconds 









dst2src_min_piat_ms 











outer_mean_piat_ms_in 









Mean positive inter-arrival time between consecutive matched outer packets in the INBOUND direction, in milliseconds 









dst2src_mean_piat_ms 











outer_stddev_piat_ms_in 









Standard deviation of positive inter-arrival times in the INBOUND direction, in milliseconds 









dst2src_stddev_piat_ms 











outer_max_piat_ms_in 









Maximum positive inter-arrival time in the INBOUND direction, in milliseconds 

















dst2src_max_piat_ms 








 











outer_min_piat_ms_out 









Minimum positive inter-arrival time between consecutive matched outer packets in the OUTBOUND direction, in milliseconds 









src2dst_min_piat_ms 











outer_mean_piat_ms_out 









Mean positive inter-arrival time between consecutive matched outer packets in the OUTBOUND direction, in milliseconds 









src2dst_mean_piat_ms 











outer_stddev_piat_ms_out 









Standard deviation of positive inter-arrival times in the OUTBOUND direction, in milliseconds 









src2dst_stddev_piat_ms 











outer_max_piat_ms_out 









Maximum positive inter-arrival time in the OUTBOUND direction, in milliseconds 









src2dst_max_piat_ms 











outer_min_piat_ms 









Minimum positive inter-arrival time pooled across directions, in milliseconds 









bidirectional_min_piat_ms 











outer_mean_piat_ms 









Mean positive inter-arrival time pooled across directions, in milliseconds 









bidirectional_mean_piat_ms 











outer_stddev_piat_ms 









Standard deviation of pooled positive inter-arrival times, in milliseconds 









bidirectional_stddev_piat_ms 











outer_max_piat_ms 









Maximum pooled positive inter-arrival time, in milliseconds 









bidirectional_max_piat_ms 











mean_size_ratio 









Mean of size_ratio over matched packets, where size_ratio = outer_padded_length / inner_length 









No NFStream equivalent (outer-side statistical features) 











std_size_ratio 









Standard deviation of size_ratio over matched packets assigned to the flow 









No NFStream equivalent (outer-side statistical features) 











max_size_ratio 









Maximum observed size_ratio over matched packets assigned to the flow 









No NFStream equivalent (outer-side statistical features) 







Processing code

The code directory contains the scripts used to produce the dataset from raw PCAP captures. These are provided for transparency and reproducibility.

packet_matching.py

Matches inner-side pre-tunnel packets to outer-side encrypted WireGuard transport data packets using time alignment and a padded-length consistency rule.

Requirements: Python 3.8+, tshark (Wireshark CLI)

python code/packet_matching.py \    --inner inner_capture.pcap \    --outer outer_capture.pcap \    --time-tolerance 15 \    --output packet_matches.csv

flow_matching.py

Assigns matched packets to NFStream-exported flows using 5-tuple keys with temporal and capacity constraints, then aggregates encrypted-side statistics per flow.

Requirements: Python 3.8+, pandas, numpy

python code/flow_matching.py \     --packets packet_matches.csv \     --flows nfstream_flows.csv \     --output session_flows

Dataset generation pipeline

The dataset was produced through a three-phase pipeline:



Traffic capture: Paired PCAP captures on the WireGuard tunnel interface (inner, pre-tunnel) and via an inline network TAP (outer, encrypted). NIC offloads were disabled. Nanosecond timestamp precision was enabled.

PCAP cleaning: Inner captures were filtered to retain only TCP and UDP packets and to remove non-initial IPv4 fragments.

Matching, flow export, and aggregation:



packet_matching.py — links inner packets to outer WireGuard transport data packets

NFStream — exports flow records from cleaned inner PCAPs with application labeling via nDPI

flow_matching.py — assigns matched packets to flows and aggregates encrypted-side features



Measurement topology

Traffic was captured on a residential broadband connection in Budapest, Hungary. A GL.iNet Flint 2 (GL-MT6000)  router served as the WireGuard VPN client gateway. A network TAP was placed inline between the router and the ISP router, mirroring encrypted tunnel traffic to a Linux capture host.