What checks should be done when a data pipeline fails
Generally data pipelines have sources, transformations and targets. First, check for errors on sources. Second, check for errors on transformations built in pipeline. Third, check for errors on targets. Further, check for errors on underlying infrastructure of the pipeline such as network, authentication and memory systems.