Patent ID: 8791706

Claim:
A calibration process for a signal acquisition system having a compensation digital filter wherein the acquisition system has a signal acquisition probe coupled to a signal processing instrument via a signal cable comprising the steps of: a) acquiring digital values of a broad frequency content signal as a calibration waveform using the signal acquisition probe and the signal processing instrument; b) determining at least a first measured error value between a broad frequency content signal reference calibration waveform stored in the signal processing instrument and the calibration waveform at a common location on the waveforms set by at least one of a time location and a frequency location; c) determining a measured error factor having at least one of a register value and a filter tap value as a function of at least the first measured error value and the common location on the waveforms; d) applying the measured error factor to at least one of a respective register value of an appropriate register in a plurality of registers in feedback loop circuitry of an input amplifier and a filter tap value in a plurality of taps in the compensation digital filter; e) repeating steps b), c), and d) for additional common locations on the waveforms; f) acquiring digital values of a broad frequency content signal as a calibration waveform using the signal acquisition probe and the signal processing instrument after determining the measured error value and the measured error factor at the last common location on the waveform; g) comparing calibration specifications with the calibration waveform acquired in step f) to verify the calibration waveform is within calibration specifications; h) storing register values loaded in the plurality of registers in at least one of the feedback loop circuitry of the input amplifier and filter taps of the compensation digital filter for the calibration waveform within calibration specifications; and i) displaying successful result of the calibration process.