Data stream classification and big data analytics

Nowadays, the development of computers technology and its recent applications provide access to new types of data, which have not been considered by the traditional data analysts. Two particularly interesting characteristics of such data sets include their huge size and streaming nature. In many fields the quantity of data collected every day has exponentially exploded. It means that it is not possible to store all data which is produced and such a large scale algorithms exceed the processing capacity of computing systems. Moreover, standard data mining methods cannot handle its complex structure and size, fulfill computational cost-effective requirements as even simple computational operations are too costly. On the other hand, this also opens many challenging tasks for data mining community, like developing new types of algorithms, protocols and other computer infrastructure to mine large, massive and high dimensional data; dealing with new types of data complexity; developing new distributed and parallel computation tools; considering new kinds of appropriate privacy and security issues. The above-mentioned requirements and challenges are particularly visible in emerging applications where data are continuously generated at a high rate in a form of data streams. Sensor networks, monitoring, traffic management, telecommunication, or web log analysis are just representatives of such applications where machines working in dynamic environments generate streaming data. Compared to static environments, data items arrive at high rate and algorithms that process them have to fulfill very strict constraints on memory usage, computing time, and on-line scan of incoming instances. Simple gathering incoming examples and training new classifiers from time to time cannot be used in such a scenario. Another very important aspect of learning models from data streams refers to changes in the data distributions and target concepts over time which are usually called concept drifts. Detecting these changes and adapting classifiers to concept drift become one of the main challenges for streaming algorithms. In the last few decades several new algorithms for learning classifiers from evolving data streams have been introduced. However, several research issues still remain open. One of the most important issues is an evaluation of such classifiers. First of all there is a need for new evaluation measures more suitable to reflect reactions to concept drift, allowing calculations in limited processing time, dealing with verification latency, evolving class skew, and changing costs. Methods which test the recovery of the classifier performance from drifts and robustness to noise on a practical, experimental level are also at very early state. In general, distinguishing outliers from incoming sudden changes is still remaining a challenge. Yet another research direction involves rethinking the way how to test data stream mining algorithms and which benchmark data sets should be considered. Note that concept drifts are known in advance mainly when using synthetic data, while in more practical scenarios occurrences and types of concepts are not directly known and new controlled frameworks should be developed. The other theme of this special issue refers to large volumes of processed data and an analysis of such big data. Note that due to different possible points of views shared by people from academia, industry or in particular media the term big data is quite ubiquitous. Basically, the big data does not mean the specific data quantity, but rather refers to the problems of enormous volume, diversity of data types and limited data processing time, usually known as 3Vs (volume, variety, velocity). Besides putting a stress on massive size, computational power, big data also refers to integrating data from different heterogeneous sources including more standard data repositories, as well as new technological sensors or devices (e.g. smartphones), internet resources – in particular social networks. These issues impose research directions that must be taken into account while developing novel data mining methods suitable for large volumes of data and constructing an efficient computing architecture for speeding-up the computational time. Many classification methods for big data use some kind of the problem decomposition and work with a number of simplified tasks. However, the further issue is to rebuild the original problem from a set of local decisions. Note that by decomposing a broad view on the problem and some global relations or dependencies among objects may be lost. Therefore, new learning methods that can process entire large data are remaining as the focus of intense research. Such methods require a vast computational power and have a high complexity. They are often accompanied by novel findings in the field of parallel and distributed computing. Finally, several other more data analysis methodological issues include studying statistical consequences of accessing to all, or nearly all data related to particular phenomenon, shift from traditional representative sampling, considering accuracy of obtained results. Discussing privacy protection and personal impact is also a new ethical and sociological challenge. For instance, using correlations found as a result of analysing big data sets to predict behaviour or even judge/control human individuals is one possible risk. Following the above-mentioned challenges we have decided to prepare a special issue of Neurocomputing journal which should present some recent development in data stream classification and big data analytics. It is a follow-up to a first edition of a special session located under the framework of the 8th Computer Recognition Systems (CORES) 2013 Conference, that was held from 27 to 29 May 2013 in Milków, Poland. Apart from few invited extended submissions, we issued the call for papers to other researchers. In consequence, this issue includes both extended and completely new works in the field of non-stationary and big data analysis.