# A Unified Vendor-Agnostic Solution for Big Data Stream Processing in a Multi-Cloud Environment
## 2 Related Literature Review
### Reference Architectures
One significant difference between NBDRA and MC-BDP is that the former has a wider scope, applying to big data in general, while the latter applies specifically to the domain of big data streams for the SMEOD market. As a result, the components or roles and fabrics identified by NBDRA are broader and more abstract. As an example, the big data application provider role defined by NBDRA is responsible for processing the data according to domain-specific business logic. In MC-BDP's proposal, a different emphasis was given to this role, represented as the processing code uploaded to the big data framework. One of the main concerns of this study was to investigate the overhead of running technologically agnostic code capable of processing both batch and stream data. Given that development time is expensive, particularly in the context of SMEODs at which MC-BDP is targeted, it was one of the objectives of this study to measure the impact of technology agnosticism on performance. Finally, the data provider and data consumer roles defined by NBDRA were not differentiated in MC-BDP's proposal. Since MC-BDP was designed for stream processing, it used the conceptual model of a directed acyclic graphics, which is common in stream architectures [53; 54]. Thus, the output of a process becomes the input of another, and each node in the process is capable of being both a provider and a consumer of data. The biggest difference between NBDRA and MC-BDP is perhaps that MC-BDP focuses on commercial clouds to provide the infrastructure required for big data processing, whereas NBDRA is neutral in this respect. This difference is explained by MC-BDP's focus on implementers whose presence in the big data arena is facilitated, to a great extent, by cloud computing. The reference architecture proposed by [44] takes an approach similar to our architecture, starting with a thorough review of architectures and technologies for big data, followed by the proposal of a reference architecture and subsequent evaluation. However, their proposal focused on software only, whereas MC-BDP includes both hardware and software. In particular, MC-BDP offers an in-depth look at the advantages of leveraging container-based virtualisation technology to promote vendor lock-in mitigation and facilitate ingress into the big data market through cloud computing. Their proposed reference architecture looked mainly at frameworks for processing batch data. NoSQL databasesand the Hadoop ecosystem are described as relevant big data technologies. However, stream-specific components of the Hadoop ecosystem such as Flink, Spark, and Storm are not listed their research. MC-BDP, on the other hand, focuses on stream processing. Finally, the evaluation of [PERSON]'s reference architecture was performed by analysing the real-world big data implementations of Facebook, LinkedIn, and Oracle and retrofitting them to the reference architecture proposed [44]. While this study did perform a similar investigation of real-world big data implementations [31], its purpose was not to evaluate the MC-BDP proposal, but to gather requirements for it. The evaluation of MC-BDP was performed through a concrete case-study, with both quantitative and qualitative data collected and thoroughly analysed as part of the rigorous scientific process described in Section 4. The generic architecture proposed by [9] has a number of elements in common with MC-BDP, such as its reliance on existing open-source big data frameworks and its focus on stream data processing for providing real-time intelligence. A fundamental difference is that, even though its architecture is focused on stream processing and real-time analytics, it contains a batch layer, implemented as a Hadoop cluster. This makes it an instance of the Lambda architecture, as opposed to MC-BDP, which is purely concerned with big data stream processing.