Data Analytics as a Service: A look inside the PANACEA project

The Cloud. The evolution of computing, this has been the hot topic for quite some time; a few years have passed since the beginning of what is commonly known as Cloud computing. During all this years the Cloud has evolved from a few thousand machines to enormous datacenters that interconnect with each other in diverse regions to provide the power required for nowadays connected society. While it is currently used by the general public, Cloud computing has become a norm within enterprise IT. The competition among public Cloud providers is red hot; private Cloud continues to grab increasing shares of IT budgets, and hybrid Cloud strategies are beginning to conquer the enterprise IT world. However, even if it has evolved, the Cloud continues to be physical computers interconnected by physical networks in a physical space; the Free Software Foundation Europe (FSFE) puts it simply: “There is no Cloud, just other people’s computer”. [1] In such setting, PANACEA is born as a solution to the complete set of problems that the Cloud may encounter during its full working life cycle, since the very first use to the continuing grow and decrease through time. The main objective of the project PANACEA is to provide Proactive Autonomic Management of Cloud Resources, which is why it is called PANACEA, in short. The aim is to set a proactive autonomic management of Cloud resources based on machine learning. It presents the main techniques to be used to achieve this goal: two-level architecture, middleware at each node of the overlay network and the virtualization framework. PANACEA is being developed by a consortium of different stakeholders across Europe and Middle East, in this context a series of use cases have been developed by said consortium to ultimately test and verify that the goal of PANACEA is being achieved. Those use cases cover an array of different services and applications in which the cloud is used or could be used in the near future. One of such cases is that of Data Analytics as a Service (DAaaS). Data Analytics or even known as Big Data Analytics is the process of examining large data sets containing a variety of data types, i.e., big data, to uncover hidden patterns, unknown correlations, market trends, customer preferences and other useful business information. The analytical findings can lead to more effective marketing, new revenue opportunities, better customer service, improved operational efficiency, competitive advantages over rival organizations and other business benefits. This kind of process requires a high amount of resources that only the Cloud is able to provide, but as more and more resources are required and the more and more people are using it the probabilities of encountering an issue, big or small, increases drastically. PANACEA aims in this way to provide the availability and performance required by this use case by introducing a set of innovations described in this work. In this particular work we will take a close look to two of these innovations in the DAaaS use case and the steps need to perform and validate said innovations. In it, we review different alternatives to our use case and/or the innovations we are introducing as part of PANACEA, we describe part of what PANACEA is and what the DAaaS use case is composed of and finishing with the steps and testes performed to validate the innovations in the use case. The cornerstone of all of the work described before is making elasticity possible for Hadoop in Cloud; elasticity is the capacity for a system (or a particular cloud layer) autonomously adapts its capacity to workload over time. To be able to do this we performed a study to understand how to make it possible for Hadoop to increase and decrease its resources so then we can then make it possible for Hadoop to do it autonomously; the tests we mentioned before aim to give us enough evidence that we have done it correctly and we have reached the objective of making Hadoop elastic in Cloud. Please bear in mind that the work captured here was done as part of an internship that lasted 4 months, reader discretion is advised.