# IoT Analytics

## Summary


IoT analytics involves analyzing data coming from the Internet of Things (IoT) and deriving useful insights from this data.  

IoT analytics is usually discussed within Industrial IoT (IIoT). Data is collected from manufacturing infrastructure, meteorological stations, smart meters, delivery vans, and various sensors on all types of machines. IoT analytics can also be applied to retail and healthcare sectors. Data can be in different formats such as video feeds, geolocation data, social media data, or log files. Given the different types of information sources, data integration can be very difficult. This is exactly where IoT analytics makes a difference.

## Discussion

### Could you showcase IoT analytics with some examples?

By adopting IoT for monitoring, control and automation, Deep Sky Vineyard improved efficiency. By optimizing farming schedules, labour efficiency increased by 30%. Crop efficiency increased by 50% by reducing loss due to rotting, etc. Water flow and soil moisture are monitored to achieve consistent yields and quality. 

Siemens Healthineers applied IoT to detect anomalies in x-ray tube production using multivariate time series. Specifically, they determine the quality of liquid metal bearings. They're also able to identify how much each variable contributes to the anomaly. 

Audi's Neckarsulm factory does 5 million welds per day. Manual inspection of weld quality is expensive. Voltage and current data from welding gun controllers are collected. These are combined with data on weld configuration, metal types, and electrode health. Audi does analytics at the edge on Intel hardware to warn of possible faulty welds. 

British oil and gas company BP fitted many of its wells with 20-30 GE sensors. Temperature, pressure and other parameters are measured. Each well sends 2 million data points per minute to GE's Predix cloud platform. This data helps BP predict well flows and useful life of a well, and see fleetwide performance. 


### How does IoT analytics differ from traditional analytics?

Traditional analytics is done on structured data. IoT devices generate **unstructured data**. Data have different formats and are not standardized. Because sensors are affected by physical process and randomness, data may contain missing points, corrupted messages, and incorrect readings. This is unlike data entered by humans or scanned from forms.

Traditional data systems don't change that often. For example, when filling a form many fields have a predefined range. IoT systems are more **dynamic**. Device IP addresses may change. Devices may be upgraded. New devices may be installed with better capability. Environmental conditions may affect sensor accuracy or precision. 

IoT data often comes in **real-time**. Many applications also require real-time analysis and insights. In fact, it's been said that much of IoT data becomes stale and useless if not analyzed immediately. Deciding whether to give someone a loan based on analysis of past behaviour is non-real-time traditional analysis. But to know available parking spaces requires IoT analytics in real-time. 


### What does it mean to state that IoT data needs context?

Raw data collected by sensors is called **content**. It's value can be enhanced by bringing in **context**, which is about associating a data stream with other data streams and business goals. 

Analysis on any device's data stream can be enhanced by combining it with other related data streams. Relationships across data sources can be modelled digitally. These relationships become useful when we run analytics queries. In the figure, a household's power consumption from the grid is enhanced by including all other households on that street or all households fed by the same power station. These multiple streams are used to more accurately interpolate missing data points or detect anomalies. 

Data collected by sensors on a machine on the factory floor can be enhanced with peak hours of machine use, yields, rejects, operator effectiveness, bottlenecks, material use estimates, and more. Another example is leak detection in oil pipelines. Context comes from integrating different types of data: pipeline inspection data, weather, dig reports, waterways, rain/flood zones, seismic activity, etc. 


### What's a typical process flow in IoT analytics?

IoT analytics commonly adopts the following steps: 

    + **Data Collection**: This step primarily comprises of data collected from various IoT sources including audio, image and light sensors. This heterogeneous nature of data raises the significance of IoT analytics technology.
    + **Data Pre-processing**: This step handles missing data, imports required libraries, encodes categorical data and does feature scaling.
    + **Analysis of Data**: This is mostly about exploratory data analysis that brings out summary statistics. It may suggest possible hypotheses and modelling approaches.
    + **Train and Test of data**: Data scientists build machine learning and deep learning models to suit business requirements. Models are trained from available data. With cross validation and online testing, the efficiency of the model is evaluated.
    + **Deployment and Improvement**: The tested model is deployed to deal with various real-world business problems.

### What are the various types of analytics in IoT?

**Descriptive analytics** is the most basic form that allows users to describe and aggregate incoming IoT data. Descriptive analysis is as simple as calculating mean and standard deviation. It can be used to quickly understand the data being collected. 

**Diagnostic analytics** builds on the initial understanding provided by descriptive analytics. It seeks to explain why something happened in the past. It's really root cause analysis but based on data and powered by algorithms. Data discovery, data mining, data drill down and drill through are some useful processes. 

**Predictive analytics** looks at historical data to predict future events or behaviour. If a sensor starts showing data out of its usual range, this might indicate some fault. Predictive maintenance can be scheduled before things become worse. In an another example, a supplier can proactively analyse client inventory and do early or just-in-time delivery.  

**Prescriptive analytics** is used to optimize decisions for the future to achieve a certain goal. For example, it might suggest best operating conditions to maximize efficiency or uptime. 


### How is edge computing influencing IoT analytics?

With edge computing some of the work is done at the edge of the network. This is where IoT connects the physical world to the cloud. Edge computing is more than just computation and data processing on IoT devices. An integral part of this is the powerful and seamless integration between the IoT and the cloud, between the physical world and the computational world. Edge computing applications use the computing power of IoT devices to filter, pre-process, aggregate, or evaluate IoT data. 

Some of the motivating factors for edge analytics are data privacy, latency and robustness. Some applications might permit processing of sensitive information only on devices. Applications that use image recognition may need low latency. Connectivity issues should not render applications unusable. All these are examples where cloud analytics may not be suitable. 

Healthcare has many examples of edge analytics. Portable MRI machines and image processing software enable real-time analysis at bedside. Insulin pumps obtain current sugar levels from sensors, use algorithms to predict future sugar levels and then deliver correct amounts of insulin. 


### What are some algorithms or techniques used in IoT analytics?

IoT uses many ML/AI algorithms, selected to match the needs of each application. Classification, clustering, estimation, denoising, and feature selection are some of these. Time-series analysis is an important technique. This can be time-series statistical analysis, data mining or search. One technique breaks data into chunks and uses incremental algorithms. This mitigates high memory requirements. 

Edge computing mitigates large capacity requirements by distributing the processing and storage. Only some data that needs complex analytics or long-term storage go to the cloud. In some applications, only aggregates (sum, mean, min, max) are stored. 

When it comes to real-time insights, data is analyzed as it flows into the analytics platform. This is called streaming analytics. Raw data may be stored in a data lake. Processed data is usually stored in databases or data warehouses. 

Time-series data has its own customized databases: InfluxDB, Timescale, Prometheus, Redis, Kdb+, RRDtool, Amazon Timestream, and more. Moreover, IoT leverages many big data technologies including Apache Hadoop, Apache Spark, Apache Storm, SAP HANA, Apache Drill, and more. 


### What are the major industries influenced by IoT analytics?

IoT analytics is playing an important role in manufacturing, retail, oil & gas and healthcare: 

    + **Manufacturing**: This sector is defining the Industrial IoT (IIoT), Industry 4.0 and smart energy management. Machine learning, neural networks, and predictive methodologies are playing major roles. With these technologies, hidden trends, insights and correlations are established that help in strengthening the business structure.
    + **Retail**: IoT analytics in retail assist companies in providing a personalized experience to customers that turn them into repeat buyers. Data provides a contextual view about consumer preference.
    + **Oil & Gas**: Operational efficiency is improved and downtimes are reduced by fixing faults quickly or predicting impending failures. Analytics is also presenting new opportunities such as better pump designs, higher production, reservoir mapping, and more.
    + **Healthcare**: IoT analytics is unlocking valuable insights by analyzing data from various mobile devices and smart technologies such as sensors, heart rate monitors, thermometers and IVs. They help to gain a remote view of patients.

### What are the major barriers to adopting IoT analytics?

**Data privacy** is a concern for end users. Data from their devices may reveal personal information directly or via analytics. Those who collect the data don't always clarify how it will be stored, secured, shared or used.  

Since IoT deals with physical systems (building automation, self-driving cars), **safety** is a major concern. Buggy apps, malware or communication failures can causes systems to fail and put lives in danger. 

**Real-time analytics** itself is a challenge due to large volumes of data that demand fast processing and high amounts of storage.  

IoT as a whole is **diverse and complex**. The architecture spans low-end sensors to high-end servers, edge analytics to cloud storage. Many types of data come in. Various technologies and workflows are used. Interoperability is an issue. Integrating many parts into one effective solution is a challenge. People need to be trained and ecosystems have to mature.  

**Regulations** about data use are not transparent. The legal framework to handle data theft or misuse is also not mature. 

## Milestones

1968

At General Motors, Dick Morley introduces Programmable Logic Controller (PLC) in the automatic transmission manufacturing division. 

1975

Honeywell and Yokogawa introduce the technology of Distributed Computing System (DCS). Control is distributed across the system. Backup redundancies are available. The system eliminates a singular point of failure in a central control room. 

1999

RFID as a technology starts gaining wider adoption. In subsequent years, devices equipped with Near-Field Communication (NFC), barcodes, QR codes and digital watermarking start gaining adoption within the IoT technology. 

2002

With the growing popularity of cloud technology, its integration with IoT technology starts experiencing a linear growth rate. Further, this synergy allows the storage of IoT-based data in the cloud. Analysis of the historical data unlocks hidden patterns, thereby leading to the development of OPC Unified Architecture protocol in 2006.