# A Unified Vendor-Agnostic Solution for Big Data Stream Processing in a Multi-Cloud Environment
## 2 Related Literature Review
### Architectures, Technology Stacks, and Concrete Implementations  
Ref. [48] proposed an architecture for big data streams which shares some common aspects with MC-BDP, such as a focus on stream processing, reliance on open-source technologies, and evaluation using real-world data emitted by a purpose-built simulator. Among the principal differences is their aim to produce an architecture focused on the IoT domain, while MC-BDP is a reference architecture aimed at the SMEOD market. Their evaluation methodology for the proposed architecture is comparable to that used for MC-BDP's empirical evaluation. A simulator was used to emit smart parking streaming data in real time at different velocities, and the acquisition and processing times were independently measured to understand the overhead introduced by their proposed architecture. The velocities used varied between one and one hundred messages per second, while in MC-BDP's experiments they varied between two and two thousand messages per second, making the latter a more comprehensive test of the prototype under a heavy load. The focus of the experiments and the metrics observed were fundamentally different. They were interested in the speed of data processing while our research monitored performance metrics such as CPU, memory, and network utilisation to understand the overhead introduced by different features of the proposed reference architecture such as its multi-cloud proposal, as well as scalability, fault tolerance, technology agnosticism, the use of windowing to process streaming data, and container co-location. Ref. [49] proposed a specific technology stack for the utilisation of commercial cloud computing resources with minimal risk of vendor lock-in. Although not aimed at big data, their proposal shares some common aspects with MC-BDP, such as the use of container-level virtualisation, an orchestrator, and promotion of multi-cloud deployment as mitigation against the risk of vendor lock-in. In order to verify the viability of their proposed technology stack, a prototype was built whereby a simple browser game was deployed to three commercial providers as a containerised service. Nevertheless, plans on how the prototype was going to be evaluated, as well as a full account of the research's methodology, were not published. The architecture proposed by [50] for the domain of edge cloud computing is an example of a Lambda architecture implementation based on Hadoop for batch processing and Spark for stream processing. Although aimed at very different domains, their research is relevant to the current study as it corroborates the empirical methodology followed in MC-BDP's evaluation. They evaluated the performance of their containerised architecture based on Docker and swarms by measuring CPU and memory utilisation at the container and node levels; disregarding disk I/O, since it is not relevant to stream processing; and monitoring the total processing time. The differences, however, are more significant. Ref. [50]'s architecture was developed for the domain of edge computing, where computation is performed by smaller edge devices before being transferred to the cloud. MC-BDP, on the other hand, was developed for SMEODs and aims to transfer as much of the computation as possible to the cloud to take advantage of its economies of scale. Although [50]'s choice of metrics and experimental setup were similar to MC-BDP's and provided validation for the decisions taken earlier in this study, their experimental work differed considerably in terms of the infrastructure, container allocation, container co-location, velocities, and windowing approach. MC-BDP's evaluation was more extensive, making it an important contribution to the field. A recent contribution by [52] proposed a real-time scheduling algorithm for distributed big data stream processing in a multi-region cloud infrastructure.