A Newfangled IoT Big Data Parallel Preprocessing Frameworkto Facilitate Quality IoT Big Dataanalytics

Theadvent of the Internet of Things is the driving force behind the recent technological revolution. Internet of things is the ubiquitous collection of internetconnected devices that collect, analyze and transform the massive amount of big data at an unparalleled rate. This enormous amount of data can be transformed into quality information by developing and deploying appropriate preprocessing techniques. The all-new IoT big data technology necessitates changes to be brought to the existing technologies. In this paper, the importance of preprocessing techniques in the IoT big data environment and also recent researches in IoT big data preprocessing techniques have been discussed. Finally, a newfangled IoT Big data parallel preprocessing framework has been proposed to convert the raw data into treasurable information thereby enabling quality IoT big data analytics to attain the full fruition of this emerging technology. Keywords— Big Data; IoT; preprocessing; analytics; Hadoop map reduce; I.INTRODUCTION The internet of things which is being heralded as the next technological revolution is not a new notion. Kevin Ashton, one of the pioneers had already conceived this concept in the early 2000’s [3]. The Internet of Things (IoT)is an indispensable part of future internet.It refers to the collection of intelligently connected devices and systems possessing unique identities, enabling seamless communication between the real and virtual world entities.The big data is described by four characteristics namely volume, velocity, variety and veracity. It is the massive volume of data,streaming in at an unparalleled rate in a variety of formats.Since it is too big,it cannot be processed using conventional processes or tools [20]. IoT and big data convergence will unleash plenty of opportunities that will improve the quality of life of consumers.The internet of things generates huge amount of heterogeneous data such as machine data, sensor data, weather data, geospatial data, satellite data, RFID data, text, image, audio, video data etc [2]. The vast amount of big data produced by the IoT devices contains hidden treasure of information mixed with undesirable data. Due to the divergent sources, the IoT big data may vary in formats and structures. Data preprocessing will no doubt play a crucial role in dealing with missing data, removing duplicate data, constructing a unified schema by integrating data from multiple sources [1]. Novel preprocessing tools and techniques are needed to prepare this sheer volume of data exploding from IoT devices to garner big insights. The major obstacles deterring the benefits are to be overwhelmed by deploying the suitable preprocessing techniques to realize the full potential of IoT big data.IoT big data preprocessing techniques should function steadfastly to improve data quality and to enable fast and accurate data mining to unravel the hidden knowledge behind the raw data.The IoT big data preprocessing framework built using the big data tools like open source Hadoop, is needed to eliminate invaluable data from the valuable ones. The objective of this paper is to highlight the prominence of pre-processing techniques in the IoT big data environment; to discuss recent researches in IoT big data preprocessing techniques;to propose a newfangled IoT big data parallel preprocessing framework thatwill be built using Hadoop map reduce framework to preprocess the massive volume of big data generated bythe Internet of Things. II. IOT BIG DATA PREPROCESSING IoT big data preprocessing includes data cleaning, data integration, data transformation and data reduction. The big data generated by IoT devices will be in various formats and structures. This data may be extremely affected by undesirable elements such as outliers, noise, missing values, unreliable and International Journal of Engineering Research & Technology (IJERT) ISSN: 2278-0181 Published by, www.ijert. org ICIOT 2016 Conference Proceedings Volume 4, Issue 29 Special Issue 2016