Patent ID: 7414554

Claim:
A method comprising: (a) storing in an integrated circuit a data output value A (DOUTA) corresponding to a maximum integral non-linearity value (MAXINL), wherein for a data output value range of an analog-to-digital converter there is a corresponding integral non-linearity (INL) value function, wherein MAXINL is a maximum INL value of the INL value function over the data output value range, wherein the data output value range is divisible into a first subrange and a second subrange, wherein the second subrange adjoins the first subrange but does not overlap the first subrange, wherein DOUTA is a data output value of the analog-to-digital converter at one end of the first subrange, wherein the analog-to-digital converter is a part of the integrated circuit; (b) storing in the integrated circuit a best fit adjustment value (BFAV 1 ); (c) analog-to-digital converting an input voltage VIN 1 into a data output value E (DOUTE), wherein DOUTE is in the first subrange; (e) scaling DOUTE by a scaling correctional factor SCF 1 to obtain a value DOUTF; and (f) adjusting DOUTF using BFAV 1 to obtain a best fit INL-corrected data output value.