Tell me different stages in data lifecycle.
1. Data Ingestion: In this stage, data from producers is ingested and collected for later processing.
2. Data Processing: Once the data is ingested and collected, mostly in raw format, it goes through various processing steps. It is one of the core responsibilities of data pipelines. This stage can further divide into sub-stages.
3. Data Storage: All the collected and processing is stored in various formats. Based on specific needs, there can be one or more tools or paradigm employed.
4. Data Analysis or Visualisation: In this layer, processed data is analysed to derive meaningful insights and mostly present in some visual form.