# Improving Incident Response in Big Data Ecosystems by Using Blockchain Technologies
## 3 Background
### Security Reference Architecture for Big Data Ecosystems  
The term big data refers to an environment that allows the analysis and management of large amounts of data, which can be generated in almost real time and can come from different data sources with different formats; these are the main differences with traditional data processing techniques [40]. In this type of environment, these traditional techniques are not capable to properly handle unstructureddata or high volumes of real-time datasets, but they could be part of the big data ecosystem; for example, both relational and non-relational databases do not provide analytical services they can be used as storage for different kinds of data. However, not all scenarios are suitable for using big data technologies, a deep understanding of the project requirements and the business context is needed to determine if big data is the most appropriate solution. This is crucial since a big data ecosystem is usually a very complex system and its implementation it is not an easy task.  
In order to consider all the components of a big data ecosystem it is necessary to use abstraction. This being so, we have developed an SRA for big data ecosystems [15]. This SRA defines the main components of a big data environment, and highlights the importance of using security patterns as a way to handle threats and vulnerabilities. Our SRA is based on NIST's Reference Architecture for Big Data, which has received the general consensus of the industry and scientific community [41]. However, NIST's proposal is too abstract, not placing enough emphasis in how the components are connected to each other. Figure 1 summarizes the main components of our SRA and how they are connected to each other; it also highlights the importance of defining security requirements in the early stages of the development. The monitor system is the novelty we want to add to the reference architecture as a way to improve its security during the operation phase, however, it does not belong to the SRA as such; this is the main objective of this paper. The definition of the incident response process that covers the operation phase, as well as its possible implementation will be shown in the following sections.  
The system orchestrator (SO) focuses primarily on the definition of the requirements and security solutions that will be implemented in the other components of the SRA. The next component is the big data application provider (BDAP), which is composed of the different services offered by big data (collection, preparation, analysis, visualization and access control). These services are implemented in the next component: the big data framework provider (BDFP), which can be considered as the hardware layer of the big data ecosystem. The BDFP supports the functionalities of the BDAP. To do that, it is usually composed of one or more clusters which are in turn composed of nodes. In addition to the hardware infrastructure, this component provides storage, processing, and other services such as communications or resource management. Finally, the last two components of the SRA are the data producer (DP) and the data consumer (DC), which have a similar function, but at opposite edges of the architecture. On the one hand, the DP is responsible for feeding data to the big data ecosystem, serving as a connection point with data sources, which can be both structured and unstructured. The DC, for its part, is the component that consumes the information generated by the big data ecosystem, serving as a connection point with the end user of the data. In this kind of ecosystem, the data processed usually contains personal data, so all actions performed in these components should be logged in order to improve the control over the ecosystem.  
Figure 1: Main components of the security reference architectures (SRAs) for big data ecosystems.