# Clustering on the Chicago Array of Things: Spotting Anomalies in the Internet of Things Records
## 3 Methodology
### Testing  
Here, we focus on node-level clustering; each node in the AoT is treated as a separate, independent entity without contributing meaningful information to the other nodes. By restricting our clustering, we can gain a baseline for how individual nodes in the AoT see the world and pick up node-level anomalies. Due to the number of nodes in the set, we chose \(n=4\) nodes in the dataset and performed clustering on them. The following procedures will be observed:  
* Temperature readings will be used to heuristically infer anomalies in the data when directly compared to NOAA data. This comparison will be performed per the mechanisms seen in Section 3.4.
* A node's sensor data will be pruned only to have relevant sensor information present. Reports such as network uptime, individual component IDs, a sensor's location, etc., are outside the scope of this paper.
* A comparison using the DB Index listed in Section 3.4 will be used to determine the efficacy of the clustering algorithms. Again, the lower the score, the better the clustering.  
The four nodes chosen (602f, 3d20, ba3b, 3f54) can be seen in Figure 2. The data for these particular nodes span 2 years (2018-2019). We collected the relevant NOAA data, which were collected from the NOAA GLERL Met Station at Latitude 41.78, Longitude \(-87.76\), just off the coast of Chicago, IL, USA, in Lake Michigan. The NOAA data are not used as part of the clustering algorithms but only to compare clustering success.  
Each node utilizes the same five Metsense sensors: bmp180, htu21d, pr103j2, tmp112, and tsys01. Each of these sensors was part of the node's meteorology sensing stack. These specific sensors were used since they focused on meteorology (related to the NOAA data) rather than internal sensing of the same parameter (temperature). All the other sensing stacks on the node belonged to other sensing ontologies (such as internal monitoring, counting items, air and light quality, etc.) and fell out of the scope of this work. The hardware sheets for each sensor are available at [37]. Once the data were filtered to the four nodes in question with five sensors each, each sensor was run through three clustering algorithms: (a) K-Means, (b) DBSCAN, and (c) H-DBSCAN.  
For the K-Means algorithm, there are two relevant parameters to set: the number of clusters \(k\) and the maximum number of iterations to run the algorithm \(max\_iter\). We set \(k\) to 2 since we are interested in whether or not the data contain anomalies (a binary problem).  
For \(max\_iter\), we set the value to 100. This means the K-Means algorithm updates the \(k\) centroids (and parses over the data) 100 times. This value was the default for the algorithm and was kept for speed of execution.  
For DBSCAN, \(\epsilon\) and \(min\_samples\) are the relevant parameters. The \(min\_samples\) parameter is used to determine a cluster's \"core\" samples. The \(\epsilon\) parameter calculates the distance between elements to determine cluster size. The \(\epsilon\) parameter was set to 3, and \(min\_samples\) was set to 5. The result is a large number of possible clusters. This was performed for computational requirements bounded by the H-DBSCAN algorithm.  
For H-DBSCAN, the \(min\_samples\) parameter is present, but the other required parameter to be set is \(min\_cluster\_size\). These values were set to 5. The result is an algorithm that produces many different clusters. Even with this adjustment, H-DBSCAN failed to cluster the tsys01 sensor on each node, so this information is omitted for this algorithm. Reasons for setting the hyper-parameters in this way are explained in Section 3.6.