# CommC: A Multi-Purpose COMModity Hardware Cluster
###### Abstract
Additionally, Kubernetes enhances fault tolerance by automatically rescheduling Spark executors in case of failures. Its native support for multi-cloud and on-premises environments ensures flexibility in infrastructure choices, while unified logging and monitoring tools streamline performance tracking and debugging. This container-centric approach reduces overhead, ensures portability, and simplifies management compared to provisioning and maintaining dedicated Spark Standalone clusters. Together, K8s and Spark enable robust, scalable, and efficient big data processing pipelines. Compared to Mesos or YARN, Kubernetes benefits from a wider adoption and more active development community, which translates into better support, frequent updates, and rich plugins for networking, storage, and security. While YARN is tailored to Hadoop ecosystems and Mesos is known for generalized cluster management, Kubernetes embraces a broader cloud-native philosophy that lets Spark coexist seamlessly with other containerized workloads, ultimately delivering greater flexibility and efficiency. In CommC's case, this setup allows for running a series of services in the same Kubernetes environment. Running web services on Kubernetes enhances scalability, reliability, and operational efficiency by automating scaling to handle traffic fluctuations, ensuring consistent performance, and optimizing resource use. Its self-healing capabilities replace failed containers and restart unresponsive services, boosting uptime. Kubernetes simplifies deployments with declarative configurations, enabling seamless updates and rollbacks while minimizing disruptions. Moreover, Kubernetes natively provides both Layer 4 (TCP/UDP) load-balancing through its Service abstraction (e.g., ClusterIP, NodePort, and LoadBalancer Services) and Layer 7 (HTTP/HTTPS) load-balancing via the Ingress resource, which ensures even traffic distribution for better performance and resilience. Its platform-agnostic design supports deployments across on-premises, cloud, or hybrid environments, avoiding vendor lock-in. Robust networking features, including service discovery and secure communication,combined with integrated monitoring and logging, make Kubernetes an ideal platform for deploying, managing, and scaling modern web applications. Nonetheless, the architecture that was described in the previous paragraphs comes with some drawbacks, too. Some jobs are inherently difficult to parallelize, and the machines used in this scenario are very limited, so it is of great importance to extract every bit of available performance without sacrificing the ease of managing the system. Our proposed solution for serial performance-oriented tasks is to run Linux containers alongside the full virtual machines. This enables us to have a high-level production orchestration system for managing our cluster and, at the same time, maintain full control over the resources allocated by the nodes participating in it. However, apart from the technological advantages, this approach challenges environmental issues. It is common knowledge that the rapid growth of electronic waste (E-waste) [10] has been noted as an issue by various resources and organizations. The work of Jain et al. (2023) [10] highlights that E-waste generated by the disposal of electronics like laptops, mobile phones, and televisions contains hazardous materials posing significant risks to human health and the environment. The study explores global and Indian E-waste production trends, identifying factors such as industrialization, technological advancements, and poor recycling practices as key contributors. E-waste arises not only from the end of a product's lifespan but also from consumer behavior and technological obsolescence. Despite its rapid growth (3-4% annually), only 15% of E-waste is recycled, highlighting a significant environmental and health concern due to toxic substances [11].