DOI: 10.7910/dvn/nktfzm
title: Dataset for Anomaly Detection in a Production Wireless Mesh Community Network

description: Dataset generated gathering data from a production wireless mesh community network. During the data gathering an unprovoked gateway outage is produced. The dataset allows testing unsupervised machine learning methods.
description: We provide 2 files with related datasets:<br>
1) <b>dataset_meshmon.gz</b>: CSV dataset generated gathering data from a production wireless mesh community network. Data is gathered every 5 minutes during the interval 2021-04-13 00:00:00 to 2021-04-16 00:00:00. During the interval [2021-04-14 02:00:00 2021-04-14 17:50:00] there is the failure of a gateway in the mesh (with nodeid 24). <br>
2) <b>dataset_meshmon_rt.pkl.bz2</b>: A python pickle with network topology information of each sample. The pickle file contains a dictionary with adjacency matrices, routing tables, and metrics.<br><br>

The description of the contents of these files are given below.<br>

Data Set Information of file 1)

<b>dataset_meshmon.gz</b> consists of single gzip compressed CSV file. The first line of the file is a header describing the features. The first column is a GMT timestamp of the sample in the format as "2021-03-16 00:00:00". <br>
The rest of the columns provide the comma-separated values of the features collected from each node in the corresponding capture.<br>
A suffix with the nodeid is added to each feature. For instance, the feature having the number of processes of node with nodeid 24 is named as "processes-24"
In total, 63 different nodes showed up during the samples, each being assigned a different nodeid.<br>
Features are of two types: (i) absolute values, for instance, the CPU 1-minute load average, and (ii) counters that are monotonically increased, for instance the number of transmitted packets. We have converted counter-type variables to rates, by dividing the difference between two consecutive samples, over the difference of the corresponding timestamps in seconds, as shown in the following pseudo-code:<br>
<pre>
feature.rate are columns computed from feature as
 feature.rate &lt;- (feature[2:n]-feature[1:(n-1)])/(epoch[2:n]-epoch[1:(n-1)])
 feature.rate &lt;- feature.rate[feature.rate &gt;= 0] # discard samples where the counter is restarted
 where n is the sample number
</pre>

<br>The dataset can be read using the following python code:
<pre>
import pandas as pd
import gzip

with gzip.open("dataset_meshmon.gz", mode='rt') as filehandle:
 df = pd.read_csv(filehandle, index_col=0)
df.index = pd.to_datetime(df.index)

print(df)
Out[21]: 
 processes-23 loadavg-m1-23 ... diff.xp.rate-62
datetime ... 
2021-03-16 00:00:00 50.0 0.04 ... -342.583333
...
[7931 rows x 2387 columns]
</pre>

 Data Set Information of file 2)
<b>dataset_meshmon_rt.pkl.bz2</b> is a python pickle file containing a dictionary with the following data:<br>
- 'date': sample's timestamps as those used in file 1) in python datetime64 data type.<br>
- 'uid': dictionary with nodeid as those used in file 1) and the corresponding order in the matrices below. For instance, an entry {15: 12} means that nodeid 15 is at index 12.
In all matrices described below nodes are sorted by increasing nodeid. All matrices have all nodeids that showed up in any of the captures given in file 1). Thus, all matrices are 63x63.<br>
- 'adj': list of adjacency matrices. <br>
- 'rt': list of routing tables. Each element in the list is the capture given by corresponding 'date' (as in the list of adjacency matrices). Likewise, row is the origin, and column destination. For each origin in the table there is the nodeid of the next node to reach the destination. In the diagonal, and in missing nodes there is -1. Thus, there is a row of '-1' for each missing node in a sample. For instance,<br>
<pre>
 array([[-1, 36, 44, ..., 36, 36, 36],
 [21, -1, 21, ..., 43, 62, 62],
 ...
</pre>
- 'metric': list of matrices with the BMX6 routing protocol metrics. For instance:<br>
<pre>
 array([[128000000000, 54525000, 35651000, ..., 27787000,
 25690000, 54525000],
 ...,
</pre>
As in 'rt', there is '-1' if a node is missing.<br>

<br>The dataset can be read using the following python code:
<pre>
import numpy as np
import pickle
import bz2

with bz2.BZ2File("dataset_meshmon_rt.pkl.bz2", 'rb') as filehandle:
 data = pickle.load(filehandle)

print(data.keys())
Out[10]: dict_keys(['uid', 'date', 'adj', 'rt', 'metric'])
</pre>