# IRONEDGE: Stream Processing Architecture for Edge Applications
## 3 Proposed Architectural Framework
### Case Study of the Ferrovia 4.0 Project
#### 3.2.3 Apache Storm  
Figure 5a,b also reflect the choice of the Stream Processing component, in this case being Apache Storm [17]. The combination of Apache Storm with Apache Kafka, for Stream Processing, has, as already noted, shown previous validation in the literature [35; 36]. The implementations of the generic pipeline, discussed in Section 3.1, are presented for each architectural proposal. The Input Stage step was mapped into a Spout, while all others were mapped into Bolts.  
The generic pipeline was implemented as described below. The processing steps depend on the Data Models, identified by source MQTT/Apache Kafka topic, which define the processing details in each Bolt. Section 4 gives details on a specific dataset and the configurations supplied as part of its Data Model. The mapping between steps and Bolts is:  
1. Input Stage--While following the same sequence, K0-WC uses a KafkaSpout to access the Apache Kafka broker and K1-NC uses an MqttSpout to access the MQTT broker. For K1-NC, the read-time timestamp is inserted. For K0-WC, the write-time is obtained from the Apache Kafka topic;
2. Mapping Stage--A generic Bolt converts Data Reports into Data Points, associating the MQTT/Kafka topic with the Data Model;
3. Windowing and Aggregation Stages--These two stages are implemented as a single Bolt, a WindowedAggregator. The Data Points are grouped in a rolling window by time interval, using the injected timestamps. As a default value, a windowing value of 5 s was selected;
4. Predictive Stage--During the initial development phase of the project, with domain knowledge still being gathered, a more simple approach, the CUSUM [43; 44] method, was selected, to detect changes in accelerator data. This corresponds to the implementation of a CusumBolt. Initially, Events are always transmitted, even when no change is detected, with the current state of the aggregation;
5. Output Stage--For both variations, a KafkaBolt is implemented to inject Events into Apache Kafka topics marked for distribution by Kafka Mirrormaker v2.  
The initial behavior of the Predictive Stage is inherently inefficient and may be optimized through fine-tuning, for example with a threshold mechanism that only sends Data Reports above a certain variance threshold, as seen in [45]. This initial behavior is meant for gathering domain knowledge on the sensor set to be used and stress testing during initial development and is used to feed the required fine-tuning in further phases of the project. Additionally, another important consideration is that of outliers in collected Data Reports resulting from ambient noise and faults in the IoT devices. Initially, these considerations, while important, are considered out of scope for the initial phase of development, where  
Figure 5: Implementation of the Ferrovia 4.0 architectures. (**a**) K0-WC uses Kafka Connect to integrate the MQTT source. (**b**) No Kafka Connect is used by K1-NC, instead consuming the MQTT source directly. Kubernetes implicitly uses Grafana [41] and Prometheus [42] for resource monitoring.  
the purposed architecture was evaluated for an upper maximum processing limit. Once a more significant body of domain knowledge on the set of sensors used has been collected, the Predictive Stage may be further fine-tuned to consider outliers.