# A Scalable Framework for Sensor Data Ingestion and Real-Time Processing in Cloud Manufacturing
## 1 Introduction  
Cloud Manufacturing (CM) represents a significant advancement in the manufacturing sector, combining the power of Cloud Computing (CC) and the Internet of Things (IoT) to create a seamless, interconnected production ecosystem. This innovative approach allows the integration of geographically dispersed manufacturing resources, including equipment, materials, and expertise, into a unified platform. Thus, the CM enables manufacturers to access and utilize resources on demand, similar to how CC services are consumed [1, 2]. This paradigm shift not only enhances operational efficiency, but also promotes flexibility and scalability in manufacturing processes. As illustrated in Figure 1, the pay-per-use model inherent in CM offers manufacturers numerous advantages. This allows companies to access advanced manufacturing capabilities without the need for significant upfront investment in equipment and infrastructure. This model optimizes resource allocation by ensuring that manufacturing assets are utilized more efficiently across different users and projects. Furthermore, it enables smaller enterprises to compete in a more level-playing field with larger corporations by providing access to state-of-the-art manufacturing technologies and expertise. Consequently, CM is poised to revolutionize the manufacturing industry by fostering innovation, reducing costs, and improving overall productivity. The integration of CC and IoT technologies facilitates the development of scalable and adaptable production systems, enabling manufacturers to meet the dynamic demands of contemporary manufacturing effectively. Despite their potential, the current CM frameworks encounter significant challenges related to scalability, resource management, and data security, particularly in decentralized and rapidly evolving environments [3]. Addressing these limitations is essential for enhancing the operational efficiency of CM systems. Recent advancements have highlighted the critical importance of resource optimization and scheduling algorithms for improving overall system efficiency [4]. Advanced scheduling methods, for instance, contribute to optimized resource allocation, reduced production lead times, and improved system performance. However, existing solutions often lack the robustness required to manage the complexities of modern manufacturing, such as volatile demand patterns and integration of diverse heterogeneous resources [5]. To address these challenges, this study proposes an innovative nine-layer framework for CM that incorporates Apache Kafka for high-throughput data ingestion and Apache Spark Streaming for real-time data processing, thereby enhancing operational efficiency and resilience [6]. Additionally, microservice architecture underpins the framework, promoting scalability and reducing latency to facilitate rapid adaptation to market changes, particularly within the manufacturing industry sector [7]. Experimental validation of the framework using sensor data from the University of California-Irvine (UCI) Machine Learning Repository [8] revealed significant improvements in processing time and throughput compared with traditional systems. Notably, RabbitMQ low-latency capabilities, combined with Kafka data durability, optimize the performance for real-time applications. Furthermore, Spark Streaming in-memory processing enables timely insights, making it highly suitable for dynamic data analysis in complex manufacturing scenarios [9]. Therefore, the proposed nine-layer framework enhances operational efficiency, resource optimization, and data security and establishes a resilient solution tailored to the demands of modern industrial applications. This study highlights the potential of this framework to advance CM by presenting practical insights into its performance, scalability, and applicability to contemporary manufacturing ecosystems. The remainder of this paper is organized as follows. Section 2 presents a review of the recent state-of-the-art CM frameworks and related advanced technologies.