Edge to core cloud pipeline refers to the data journey through data generation at the edge to AI analytics in the cloud. This end-to-end pipeline is segmented into 4 stages.
Intelligent Edge — The processing of data in motion, captured at the device, cached locally, and with minor analytical models applied.
Core Processing — Referred to as “fast data” this is data passed on from the edge to a core, either locally, or in the cloud. In this stage, more analytics are applied, however, this data is retained for real-time use. Imagine business systems drawing data from retail stores to inform inventory. Data at this point could reside in data warehouses.
Big Data Lakes — A level beyond core processing is data lakes, which provides the grounds for further analysis of data at rest. Data in data lakes are aggregated and prepared for deep learning.
AI/ Machine Learning/ Deep Learning — AI builds and test models based on massive quantities of data, providing predictive insight from that data about the future possibilities. The compute and storage requirements for successful AI can only feasibly be found in the cloud.
From this model pipeline, infrastructure not only represents a bi-directionallity of data, but also a tiered system of analytics, placing the most immediate needs at the edge, and the most intensive needs in cloud cores where compute and storage are more cost effective and real-time is less of a concern.