Package: dfval
dfvaldfvalcan be used to compare two dataframes and identify differences between the dataframes. It also has methods to validate attributes or update dataframes.Below is example creates two small dataframes and does a comparison usingdfvalas well as more detail on how to usedfval.Example ofcompare()1. Import dfval and pandasimport dfval as dv
import pandas as pd2. Create two dataframes (x and y)dx = [[5000, '2019-12-15', 100, 2], [5000, '2019-12-22', 200, 3], [5000, '2019-12-29', 500, 4], [5000, '2019-12-08', 75, 3]]
dfx = pd.DataFrame(dx, columns = ['loc_id', 'greg_d', 'qty_sum', 'qty_ct'])

dy = [[5000, '2019-12-15', 100.000001, 2], [5000, '2019-12-22', 190, 3], [5000, '2019-12-29', 500, 4], [5000, '2019-12-01', 75, 3]]
dfy = pd.DataFrame(dy, columns = ['loc_id', 'greg_d', 'qty_sum', 'qty_ct'])3. Define the key and quantity fields for comparisonk = ['loc_id', 'greg_d']

qty_f = ['qty_sum', 'qty_ct']4. Callcompare()# call compare passing the x dataframe, y dataframe, key, the quantity field name, and optionally the keep_comparison (True or False), decimal rounding, and threshold for quantity comparison
c = dv.compare(dfx, dfy, k, qty_f, keep_comparison = True, decimal_round = 4, threshold = 0.01)5. Contents ofc.resultsrec_chk_passrec_count_xrec_count_yrec_count_zrec_count_samerec_count_diffrec_count_notinxrec_count_notinyrec_count_exTrue445211136. Contents ofc.exceptionsloc_idgreg_dqty_sum_xqty_ct_xqty_sum_yqty_ct_ydiff_qty_sumdiff_qty_ctexception_type50002019-12-22200.03.0190.03.010.00.0diff_qty50002019-12-0875.03.0NaNNaNNaNNaNnot_in_y50002019-12-01NaNNaN75.03.0NaNNaNnot_in_x7. Contents ofc.comparisonloc_idgreg_dqty_sum_xqty_ct_xqty_sum_yqty_ct_ydiff_qty_sumdiff_qty_ctexception_type50002019-12-15100.02.0100.02.00.00.0match50002019-12-22200.03.0190.03.010.00.0diff_qty50002019-12-29500.04.0500.04.00.00.0match50002019-12-0875.03.0NaNNaNNaNNaNnot_in_y50002019-12-01NaNNaN75.03.0NaNNaNcompare()detailcompare()can be used to compare dimensions in two dataframes or dimensions and quantity values. To compare dimensions only, leave out theqty_n,decimal_round, andthresholdparameters listed below.Inputcompare()has three required parameters and four optional parameters:Required parametersx: one of two dataframes used the comparison (order doesn't matter)y: second of two dataframes used in comparison (order doesn't matter)k: a list of the common key in the dataframes used for comparisonOptional parametersqty_n: a list of the quantity field(s) to compareNote:if theqty_nis not passed,compare()will do a comparison of the dimensions (k) onlykeep_comparison: retains all the records of the comparison in results comparison dataframedecimal_round: number of digits to round the quantity fields in the comparison (fields listed inqty_n)threshold: a threshold for what should be considered an exception in the quantity comparisonOutputcompare()returns an object with two dataframes describing the results of the comparison and the exceptions.exceptions: one record for each differenceexception types (exception_type):not_in_x: key is present in y dataframe but not xnot_in_y: key is present in x dataframe but not ydiff_qty: key is present in both x and y dataframe but difference in one or more of the quantity columns is greater than thresholdresult: one record showing counts for comparisonrec_chk_pass:Does the record count in the outer join of x and y equal the sum of the record counts of: records that match, records with quantity difference, records in x by not y, and records in y but not xAs long as there are not duplicates in the key in the x or y dataset, this should always beTruecomparison: all records from the result of the comparison (only retained ifkeep_comparison = Trueexception types are the same as theexceptionsdataframe, plus this has the records thatmatchNotes:compare()first checks the x and y DataFrames to make sure both have all columns defined in the passed in key and quantity field(s). If one or more columns is missing in either DataFrame, a message prints ("column name check failed") and the comparison is not run.exceptionsandresultswill beNone.NaNvalues in the quantity fields in the x and y datasets are converted to zero. If this happens, a message is printedcompare()checks for duplicate keys and prints a message if duplicates are found. The comparison continues to run and creates results. However, the results may be be what is expected. It is better to handle duplicates in the key prior to usingcompare().The result of the record count check prints if the result isFALSE:"Record counts for same, diff, and in one but not other DO NOT match"