Patent ID: 11914496
Assignee: DATA CULPA, INC.
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 14:
15. A non-transitory computer-readable medium storing instructions to maintain data consistency in a data pipeline, wherein the instructions, in response to execution by one or more processors, cause the one or more processors to drive a system to perform operations comprising:
monitoring operations of the data pipeline wherein the data pipeline receives input data sets, processes the input data sets, and responsively generates and transfers output data sets;
receiving the output data sets;
processing the output data sets to identify changes between the output data sets;
utilizing a Sigmoid function to calculate weighted sums of data values, data records, and data schemas for the output data sets;
generating a consistency score for the output data sets based on the changes and the weighted sums, wherein the consistency score indicates a similarity level between the output data sets;
determining when the consistency score exceeds a threshold value;
generating an alert that indicates ones of the output data sets that exceeded the threshold value when the consistency score exceeds the threshold value; and
transferring the alert.