An image processing apparatus, comprising: a difference calculation unit to calculate, for each of pixels that constitute a pickup image in each of a plurality of pickup images taken at different focal depths, a difference of a luminance value from an adjacent pixel; and a difference distribution generation unit to generate, on the basis of the difference and the focal depth at which the pickup image including the pixel whose difference is calculated is taken, a distribution of the difference with respect to the focal depth; wherein the difference distribution generation unit includes: a differential image generation unit to generate, for each of the plurality of pickup images, a differential image obtained by replacing a luminance of the pixel that constitute the pickup image by the difference to generate a plurality of differential images corresponding to the plurality of pickup images, respectively, a luminance totalizing unit to totalize, in each of the plurality of differential images, the luminances of the pixels for each of pixel columns that constitute the differential image to generate a luminance total value for each pixel column, a pixel column generation unit to arrange, for each of the plurality of differential images, the pixels having the luminance total value as the luminance in a first direction in order of the pixel columns to generate a plurality of analysis pixel columns corresponding to the plurality of differential images, respectively, and an image generation unit to arrange the plurality of analysis pixel columns in a second direction perpendicular to the first direction in order of the focal depths at which the plurality of pickup images respectively corresponding to the analysis pixel columns are taken, to generate an analysis image. a difference calculation unit to calculate, for each of pixels that constitute a pickup image in each of a plurality of pickup images taken at different focal depths, a difference of a luminance value from an adjacent pixel; and a difference distribution generation unit to generate, on the basis of the difference and the focal depth at which the pickup image including the pixel whose difference is calculated is taken, a distribution of the difference with respect to the focal depth; a differential image generation unit to generate, for each of the plurality of pickup images, a differential image obtained by replacing a luminance of the pixel that constitute the pickup image by the difference to generate a plurality of differential images corresponding to the plurality of pickup images, respectively, a luminance totalizing unit to totalize, in each of the plurality of differential images, the luminances of the pixels for each of pixel columns that constitute the differential image to generate a luminance total value for each pixel column, a pixel column generation unit to arrange, for each of the plurality of differential images, the pixels having the luminance total value as the luminance in a first direction in order of the pixel columns to generate a plurality of analysis pixel columns corresponding to the plurality of differential images, respectively, and an image generation unit to arrange the plurality of analysis pixel columns in a second direction perpendicular to the first direction in order of the focal depths at which the plurality of pickup images respectively corresponding to the analysis pixel columns are taken, to generate an analysis image.