# A Unified Vendor-Agnostic Solution for Big Data Stream Processing in a Multi-Cloud Environment
## 1 Introduction
### Research Context
A reference architecture is based on best practices and serves as a guide for the design and implementation of concrete architectures. Examples of numerous published domain-specific reference architectures for big data built to address this limitation are: BDWFMSs designed for the domain of data-intensive scientific workflows [8], ref. [9]'s healthcare-related reference architecture, and ref. [10]'s national security-related reference architecture. Approaches to develop an SME big data strategy tend to prioritise simplicity and convenience by focusing on big data as a managed service, whilst overlooking the ominous risk of vendor lock-in. As an example, ref. [11] highlighted the complexity of big data implementations as a significant barrier to SME adoption. They have proposed a big data analytics as a service (BDAaaS) model for the SME market. A similar cloud consumption model, data as a service (DaaS) [12], envisions big data cloud analytics as a crucial strategy to give SMEs a competitive advantage in the market. Both these models, however, are dedicated forms of SaaS which could incur a high risk of vendor lock-in. Ref. [13]'s research findings reveal that a significant barrier to the adoption of big data solutions by British manufacturing SMEs is the high cost of switching solutions. They suggest that these companies need not only appropriate tools, but also appropriate guidance when implementing an architecture for big data [13]. Big data challenges and potential within the SME market have been reviewed [14], while investigation of the opportunities for innovation created through big data has been conducted [15]. However, to date, a high-level vendor-agnostic blueprint to cater to these requirements does not exist. This is an identified gap which our research aims to address, namely, the creation of a reference architecture for big data stream processing targeted at SMEs and therefore specifically aimed at maximising economies of scale via commercial clouds [16; 17]. For illustration, a study carried out by [18] using quantitative and qualitative methods on a target population of over a hundred UK-based companies from different sectors showedthat 71% of respondents saw the risk of vendor lock-in as a possible deterrent to cloud migration. Although their research also indicated a general lack of maturity in the sector and lack of familiarity with the concept of vendor lock-in, it revealed that UK decision-makers were essentially cautious about trusting a single commercial provider with their business-critical data [18]. This study's proposal of a reference architecture for big data using a multi-cloud setup is therefore motivated by its focus on smaller implementers and supported by relevant research in the area. Currently, cloud computing business solutions are being made available to SMEs on a pay-as-you-go model. The cloud has brought with it economies of scale, making it more cost-effective for smaller companies to commission technology as services instead of purchasing hardware and maintaining their in-house IT staff. Such entry cost reduction has enabled more widespread adoption of computing resources to automate processes, satisfying significantly increased data processing needs. Undeniably, more data means greater complexity, but also the potential for more accurate predictions and increased turnover. The profitability of big data initiatives is corroborated by an independent research study of 559 companies from various sectors conducted by the Business Application Research Centre [19]. The findings reveal that the participating companies reported an average eight percent increase in revenue and ten percent decrease in costs as a result of their use of big data [19]. However, most of them own in-house-hosted technologies and are still sceptical about transferring their businesses to the cloud [19]. Given the elevated cost of big data implementations [20; 21], one could speculate that the cloud's economies of scale could potentially raise the reported profits.