# A Scalable Framework for Sensor Data Ingestion and Real-Time Processing in Cloud Manufacturing
## 7 Conclusions  
Considering the data ingestion processing and real-time analytics in CM, our study confirms that the log-based architecture of the Kafka platform is optimal for robust, scalable, and durable data ingestion. The high throughput and ordered message delivery of the Kafka platform make it particularly effective for real-time monitoring and predictive maintenance, whereas RabbitMQ remains suitable for low-latency small-scale tasks. For data processing, Spark Streaming outperforms Hadoop in latency-sensitive applications, leveraging its in-memory and micro-batch architecture to deliver real-time analytics and actionable insights. This responsiveness is essential for live monitoring and adaptive manufacturing operations, making Spark a preferred solution for dynamic CM environments. Our analysis also highlights the scalability and efficiency of microservices over monolithic architectures. The modular structure of the microservices supports independent deployment, dynamic scaling, and fault tolerance, thereby enabling superior performance under variable loads. Containerization with Docker and orchestration via Kubernetes further enhance flexibility and resilience, ensuring reliable operations in distributed manufacturing scenarios. These choices collectively form a robust and scalable foundation for real-time CM applications, and they are validated through experimental campaigns that demonstrate high throughput, low latency, and adaptability. Future research should focus on deploying this framework in industrial case studies to optimize predictive maintenance, real-time quality control, and resource allocation. Incorporating advanced security measures, such as encryption, Role-Based Access Control, and intrusion detection, is critical for addressing cybersecurity challenges inherent in shared CM environments. Techniques such as blockchain and AI-driven threat intelligence can enhance data integrity and resilience against cyber threats, particularly in sensitive applications, such as AM. Additionally, integrating machine learning models for predictive analytics and adaptive system configurations could further optimize performance and responsiveness in real-world scenarios. In conclusion, the proposed framework integrates Kafka, Spark Streaming, and microservices to address critical CM requirements and achieve high performance, scalability, and resilience. This design establishes a strong foundation for advanced industrial applications with opportunities for future enhancements in security, adaptability, and real-world testing. These advancements have ensured that the framework remains at the forefront of reliable and efficient CM systems in evolving manufacturing environments. Conceptualization, M.P. ; methodology, M.P. and A.P. ; validation, M.P. and A.P. ; resources, G.P. and E.F.; writing--original draft preparation, M.P. and A.P. ; writing--review and editing, M.P. and A.P. ; supervision, M.P. and G.P. ; project administration, G.P. and E.F.; funding acquisition, G.P. and E.F. All authors have read and agreed to the published version of the manuscript. This work was funded by Puglia Region (Italy)--Project \"SMILE&SCALETECH\" (P.O. PUGLIA FESR 2014-2020). The data analyzed in this manuscript are available at [[https://archive.ics.uci.edu/dataset/374/appliances+energy+prediction](https://archive.ics.uci.edu/dataset/374/appliances+energy+prediction)]([https://archive.ics.uci.edu/dataset/374/appliances+energy+prediction](https://archive.ics.uci.edu/dataset/374/appliances+energy+prediction)) (accessed on 1 January 2025). The authors express their gratitude for the administrative and technical assistance provided by the company Advantech S.r.l. in Lecce, Italy. Author [PERSON] was employed by the company Laser Romae S.r.l. in Rome, Italy. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.