# A Unified Vendor-Agnostic Solution for Big Data Stream Processing in a Multi-Cloud Environment
## 5 Conclusions and Future Work
The current study is one such effort, developed to fill the gap for a systematic approach to big data stream processing targeted at a domain whose main preoccupation is with minimising the risk of vendor lock-in while at the same time maximising the cloud's economies of scale. Motivated by a desire to facilitate the adoption of big data stream analytics, this study has proposed a new systematic and unified approach to big data streams. The MC-BDP reference architecture was created to:  
1. Leverage the cloud's economies of scale by promoting an infrastructure hosted on commercial clouds;
2. Move away from the traditional SaaS approach towards a standardised form of PaaS;
3. Mitigate the risk of vendor lock-in by prescribing the use of portable, interoperable, and vendor-agnostic components deployed to multiple clouds; and
4. Alleviate concerns around complexity and skill shortages in the domain by providing a domain-specific reference architecture to guide implementers. This study challenges the accepted belief according to which the most appropriate cloud consumption model for SMEs is SaaS [11; 12; 99; 100; 101]. While acknowledging the concerns around technical complexity and skill shortages which have led previous authors to recommend the SaaS cloud model to SMEs, particularly within the realm of big data, this study places equal importance on the perceived risk of vendor lock-in explored extensively in recent research [18; 32; 102; 103; 104]. The MC-BDP reference architecture, designed specifically for the SMEOD domain, provides a good level of scaffolding to enable implementers to navigate the complexities of big data technology and cloud computing without having to employ a dedicated and highly specialised technical team. The simplification and systematisation provided by the MC-BDP reference architecture enabled this study to break with tradition [11; 12; 99; 100; 101] and recommend a cloud consumption model more desirable than SaaS from a vendor lock-in perspective. This is a significant contribution to the field of big data streams which, up until now, lacked a systematic and thoroughly researched approach to big data based on a consumption model other than SaaS. It is not expected that research on specialised SaaS provisions for SMEs will cease given that concerns around the risk of vendor lock-in are not universally or evenly shared amongst companies--some could find that this is a risk worth taking for the benefit of simplicity. Therefore, works such as [45]'s reference architecture for big data designed from a cost perspective or [56]'s reference architecture based on real-world big data implementations, discussed in Section 2, remain valid and indeed useful within the field. What this study has demonstrated, however, is that rather than searching for a universal architecture for big data to suit all SMEODs, research should be directed towards finding approaches which are flexible and abstract enough to account for the immense variety in requirements and priorities that characterise the domain. The 110 experiments performed under controlled conditions demonstrated that MC-BDP's prototype was satisfactorily scalable and fault-tolerant across clouds, with no considerable difference observed between single and multi-cloud equivalent setups. Container co-location was also found not to significantly affect performance. In terms of technology agnosticism, the overhead introduced by the Beam framework in terms of CPU, memory, and data transfer was considerable. The conclusion reached was that there is indeed a performance cost to technology agnosticism as implemented in MC-BDP's prototype due to its use of a super-framework to provide portability of the code across various stream and batch technologies.