# A Unified Vendor-Agnostic Solution for Big Data Stream Processing in a Multi-Cloud Environment
## 5 Conclusions and Future Work
### Conclusions  
MC-BDP was evaluated as part of a case study involving the Estates and Sustainability departments at Leeds Beckett university using a mixed-methods approach which combinedpost-positivist and interpretivist elements. In its post-positivist evaluation, presented in this paper, MC-BDP was demonstrated to be scalable and fault-tolerant across clouds. Given the cloud's pay-as-you-go model, whereby consumers are only charged for the resources they utilise, being able to scale up or down is particularly important for SMEODs since budgetary constraints demand that resources be allocated sensibly and waste minimised. The relevance of fault tolerance to the domain of SMEs has been identified in the literature as the need for high availability of their production systems [12]. The results of the technology agnosticism experiments revealed that the use of a super-framework such as Beam to provide portability and interoperability of the data processing code did have a cost in terms of performance, measured as, on average, 30% more CPU, 3% more RAM, and 105% more network usage. The significance of these results to researchers and implementers is in being able to take more informed architectural decisions, particularly in the cloud, where the estimated overhead can be translated into projected costs. This study believes that a concern over costs can be generalised to SMEODs in general, and that the overhead measurements obtained empirically are therefore of direct relevance to the field. Of similar relevance are the findings of a direct relationship between the windowing rate and resource utilisation in a distributed big data stream system. Since it was demonstrated that the CPU and network utilisation can be predicted using a simple formula based on the windowing rate and constants derived from empirical observation, implementers now have a more accurate way of projecting the cluster size and the cost of data transfers. It is believed that the relationship derived experimentally is of wider application within the field of big data stream processing using commercial clouds. The container co-location findings confirm that co-location does not significantly affect the performance of CPU-intensive big data stream systems. In the context of cloud computing, this corroboration enables implementers to experiment with a greater number of deployment configurations, from multiple virtual machines with a few containers running in them to a smaller number of more powerful machines hosting a greater number of containers. Factors such as the desired level of fault tolerance (container, node, region, or provider) or projected data transfer costs may play an important part in the decision-making, particularly considering that container-to-container data transfer costs can be reduced or eliminated through co-location. The practical implications of the MC-BDP study conducted at Leeds Beckett University are far-reaching, particularly for SMEODs. These enterprises, which often operate with limited resources, can greatly benefit from the scalability and fault tolerance aspects demonstrated by MC-BDP. The ability to efficiently scale resources in response to demand, a crucial feature in cloud environments, aligns well with the budgetary constraints common to SMEODs. Additionally, the study's insights into the cost-performance trade-offs inherent in using super-frameworks such as Beam provide these organisations with valuable guidance for making cost-effective architectural decisions. This is particularly relevant in the context of cloud computing, where resource utilization directly impacts operational costs. Moreover, the findings regarding container co-location and its minimal impact on performance enable more flexible and cost-efficient deployment strategies, which are crucial for SMEODs striving for high availability and reduced data transfer costs. From a theoretical perspective, the MC-BDP study offers new insights into cloud computing and data processing.