# On a Certain Research Gap in Big Data Mining for Customer Insights
## 4 Challenges in Big Data Mining for Customer Insights  
Generating insights from big data is a process consisting of two main activities--data management and data analytics. The former encompasses data acquisition, extraction, cleaning, integration, and representation. The latter consists of data modeling, analysis, and interpretation [60]. Data mining techniques are among the most used ones in big data analytics. Hence, it can be assumed that challenges in BDA concern also big data mining, even if not expressed explicitly. In a very general way, the first and foremost challenge of big data analytics is to generate business value [95]. It is also one of the ultimate goals of BDA and big data mining. The other ones are also provision of competitive advantage [5; 11], and generation of new business ideas from big data insights [89]. Obviously, the quality of insights results from a proper orchestration of big data-related resources, that is, data, technology, processes, and people within the framework of organization [4]. Human skills and organizational culture are as important as the technological dimensions of BDA in providing valuable results for the success of organization [89; 149]. The overall success of big data analytics is therefore dependent on such factors as top management support, organizational change, technical infrastructure, the data science skillset, data availability and quality, data security, and privacy [150]. Conceptually, the overall big data challenges can be summarized as presented in Figure 4.  
In the big data lifecycle, the first group of challenges concerns the characteristics of big data itself (the \"Vs\") which in turn affect the issues of big data preprocessing, e.g., integration, cleansing, and transformation. At the data processing stage, the typical tasks of analysis, modeling, mining, etc. must be adjusted to properly address the challenges of the first stage. Moreover, at the stage of presenting the results, the graphical methods must be able to cope with visualizing a huge amount of big data analysis results. The big data management stage extends over all other stages and is associated with challenges such as privacy, security, data ownership, and other ethical issues [76; 152; 153; 154; 155]. Among the features determining data, information, and insights quality the completeness, accuracy, and currency are mentioned to be the most significant [156]. Especially, the last feature is a challenge when applied to big data analysis and mining. Not only the data comes as a stream or flow (the velocity dimension of big data), but also, it must be analyzed/mined in real-time manner to provide value to organizations. We will cover this temporal challenge later. Figure 3: The marketing mix framework for big data management. Source: [PERSON] et al. [97]. With data mining as one of the most important elements of big data analytics, it is not surprising that the DM software is one of the most appreciated tools among various analytical tools used for BDA [157]. However, even the best software will not produce valuable results from garbage data. Hence, among big data mining challenges, the first group includes data inconsistence and incompleteness, scalability, timeliness, and data security. Challenges also concern data capture, storage, searching, sharing, analysis, and visualization [158]. There is a common agreement that before mining the data it is mandatory to consider such issues as validity and reliability of data. The bigger the data quantity is, the bigger the challenge. With the amount of data, discovering dependencies and valuable patterns becomes extremely difficult [87; 118]. The next group of challenges is associated with the mining process itself. The algorithms and techniques used for \"classic\" data mining in, e.g., data warehouses sometimes are not suited to be used with huge amounts of constantly incoming big data. This is so because traditional data mining approaches start with a centralized data repository, able to store and process data.