An image processing apparatus, comprising: an input unit configured to receive a plurality of input image frames taken in time series; a gradation distribution calculating unit configured to calculate a first gradation distribution for a first image area in a target frame to be corrected among the image frames, and calculate a second gradation distribution for a second image area, corresponding to the first image area, in a frame for correction used for correcting the target frame; a parameter control unit configured to calculate a set of two indices having different robustness to deformation of an object from the target frame and the frame for correction, and control sizes of the first and the second image area based on relationship between the two indices; a similarity calculating unit configured to calculate a first similarity between the first gradation distribution and the second gradation distribution; an image correcting unit configured to correct the target frame by using the frame for correction; and an output unit configured to output the target frame corrected by the image correcting unit, wherein the image correcting unit corrects the target frame by using the frame for correction after changing a using ratio between the target frame and the frame for correction depending on the first similarity, wherein one index in the set is a first index based on a difference between a plurality of pixels in the first and the second image area or a difference between histograms of a pixel value in the first image area and the second image area, and the other index in the set is a second index based on a difference between individual pixels in the first and the second image area or a sum of absolute differences between pixel values in the first and the second image area; and wherein the parameter control unit sets a first and a second threshold value corresponding to the first and the second index, calculates a plurality of the sets from the first and the second image area provided at a plurality of positions in the image frames, and controls the sizes of the first and the second image area based on the number of sets in which the first index is equal to or higher than the first threshold value and the second index is lower than the second threshold value an input unit configured to receive a plurality of input image frames taken in time series; a gradation distribution calculating unit configured to calculate a first gradation distribution for a first image area in a target frame to be corrected among the image frames, and calculate a second gradation distribution for a second image area, corresponding to the first image area, in a frame for correction used for correcting the target frame; a parameter control unit configured to calculate a set of two indices having different robustness to deformation of an object from the target frame and the frame for correction, and control sizes of the first and the second image area based on relationship between the two indices; a similarity calculating unit configured to calculate a first similarity between the first gradation distribution and the second gradation distribution; an image correcting unit configured to correct the target frame by using the frame for correction; and an output unit configured to output the target frame corrected by the image correcting unit, wherein the image correcting unit corrects the target frame by using the frame for correction after changing a using ratio between the target frame and the frame for correction depending on the first similarity, wherein one index in the set is a first index based on a difference between a plurality of pixels in the first and the second image area or a difference between histograms of a pixel value in the first image area and the second image area, and the other index in the set is a second index based on a difference between individual pixels in the first and the second image area or a sum of absolute differences between pixel values in the first and the second image area; and wherein the parameter control unit sets a first and a second threshold value corresponding to the first and the second index, calculates a plurality of the sets from the first and the second image area provided at a plurality of positions in the image frames, and controls the sizes of the first and the second image area based on the number of sets in which the first index is equal to or higher than the first threshold value and the second index is lower than the second threshold value