Patent ID: 11934396
Assignee: BANK OF AMERICA CORPORATION
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 17:
18. A data reconciliation system for big data, the system comprising:
a receiver configured to receive:
identification of a first big data set and a second big data set using a lazy evaluation algorithm, the lazy evaluation algorithm delays loading data included in the first big data set and the second big data set until the data is demanded by a loading algorithm; and
a dimension, said dimension including a first set of fields from the first big data set and a second set of fields from the second big data set, wherein the first of fields correspond to the second set of fields;

a processor configured to:
dynamically construct a structured query language (SQL) query to compare the first big data set to the second big data set, the dimension is used as a join set of data fields within the SQL query, the SQL query comprising a where clause, the where clause including each of a set of column names in the first big data set that correspond to column names in the second big data set;
execute the SQL query on the first big data set and the second big data set;
generate a result set from the SQL query, the result set comprising one or more columns that are included in the first big data set that correspond to one or more columns that are included in the second big data set;
identify that the result set includes one or more differences;
order the result set by the one or more differences included in a first column of the one or more columns, the one or more differences are included in a top portion of the first column;
select a first predetermined number of rows in the ordered result set;
transpose the first predetermined number of rows in the ordered result set, wherein the columns become rows and the rows become columns;
identify a first set of differences along a first row that corresponds to the first column;
select the first set of differences from the first row;
dynamically reconstruct the SQL query by removing the first column from the SQL query;
execute the reconstructed SQL query;
generate an updated result set;
identify that the result set includes one or more differences;
order the result set by the one or more differences included in a subsequent column of the one or more columns, where the differences are included in a top portion of the subsequent column;
select a subsequent predetermined number of rows in the ordered result set;
transpose the subsequent predetermined number of rows in the ordered result set, wherein the columns become rows and the rows become columns;
identify a second set of differences along one or more subsequent rows that corresponds to the subsequent column; and
select the second set of differences from the one or more subsequent rows; and
amalgamate the first set of differences and the second set of differences into a group of differences; and

a display configured to display group of differences.