Patent ID: 11934396
Assignee: BANK OF AMERICA CORPORATION
Field: Computer technology (Electrical engineering)
Classification: CPC G | IPC G

Claim 12:
13. A method for data reconciliation in a big data environment, the method comprising:
receiving, using a lazy evaluation algorithm, identification of a first big data set and a second big data set, the lazy evaluation algorithm delays loading data included in the first big data set and the second big data set until the data is demanded by a loading algorithm;
receiving a dimension, said dimension including a first set of fields from the first big data set and a second set of fields from the second big data set, wherein the first set of fields correspond to the second set of fields;
dynamically constructing a structured query language (SQL) query to compare the first big data set to the second big data set, the dimension is used as a join set of data fields within the SQL query, the SQL query comprising a where clause, the where clause including each of a set of column names in the first big data set that correspond to column names in the second big data set;
executing the SQL query on the first big data set and the second big data set;
generating a result set from the SQL query, the result set comprising one or more columns that are included in the first big data set that correspond to one or more columns that are included in the second big data set;
identifying that the result set includes one or more differences;
ordering the result set by the one or more differences included in a first column of the one or more columns, the one or more differences are included in a top portion of the first column;
selecting a first predetermined number of rows in the ordered result set;
transposing the first predetermined number of rows in the ordered result set, wherein the columns become rows and the rows become columns;
identifying a first set of differences along a first row that corresponds to the first column;
selecting the differences from the first row;
dynamically reconstructing the SQL query by removing the first column from the SQL query;
executing the reconstructed SQL query;
generating an updated result set;
identifying that the updated result set includes one or more differences;
ordering the result set by the differences included a subsequent column of the one or more columns, where the differences are included in a top portion of the subsequent column;
selecting a subsequent predetermined number of rows in the ordered result set;
transposing the subsequent predetermined number of rows in the ordered result set, wherein the columns become rows and the rows become columns;
identifying the differences along one or more subsequent rows that correspond to the subsequent column;
selecting a second set of differences from the one or more subsequent rows; and
upon completion of identifying differences between the first big data set and the second big data, displaying the first set of differences and the second set of differences.