An image processing apparatus, comprising: a parallax information generation unit configured to generate parallax information based on a first phase difference distribution and a second phase difference distribution; wherein the first phase difference distribution is generated on a pixel by pixel basis for a first parallax image and a second parallax image; and wherein the second phase difference distribution is generated on a sub-pixel by sub-pixel basis by: determining detection areas including a corresponding edge portion (EL, ER) within each of the first and second parallax images based on the first phase difference distribution; and performing sub-pixel phase difference detection based on a luminance distribution in both the detection areas by: identifying, for each pixel in the detection area of the first parallax image, a corresponding pixel in the detection area of the second parallax image with a luminance that is the same as the selected pixel; for each detection area, estimating an edge angle (θ) of the edge portion (EL, ER) from the luminance distribution in each of the detection areas; transforming the coordinates (X, Y) of each pixel in the detection area of the first parallax image to produce a coordinate value d according to the formula d=X-Ytanθ; and for each pixel in the detection area of the first parallax image, determining a value (d sub) equal to the difference between the coordinate values of that pixel and the corresponding pixel in the detection area of the second parallax image. a parallax information generation unit configured to generate parallax information based on a first phase difference distribution and a second phase difference distribution; wherein the first phase difference distribution is generated on a pixel by pixel basis for a first parallax image and a second parallax image; and wherein the second phase difference distribution is generated on a sub-pixel by sub-pixel basis by: determining detection areas including a corresponding edge portion (EL, ER) within each of the first and second parallax images based on the first phase difference distribution; and performing sub-pixel phase difference detection based on a luminance distribution in both the detection areas by: identifying, for each pixel in the detection area of the first parallax image, a corresponding pixel in the detection area of the second parallax image with a luminance that is the same as the selected pixel; for each detection area, estimating an edge angle (θ) of the edge portion (EL, ER) from the luminance distribution in each of the detection areas; transforming the coordinates (X, Y) of each pixel in the detection area of the first parallax image to produce a coordinate value d according to the formula d=X-Ytanθ; and for each pixel in the detection area of the first parallax image, determining a value (d sub) equal to the difference between the coordinate values of that pixel and the corresponding pixel in the detection area of the second parallax image. determining detection areas including a corresponding edge portion (EL, ER) within each of the first and second parallax images based on the first phase difference distribution; and performing sub-pixel phase difference detection based on a luminance distribution in both the detection areas by: identifying, for each pixel in the detection area of the first parallax image, a corresponding pixel in the detection area of the second parallax image with a luminance that is the same as the selected pixel; for each detection area, estimating an edge angle (θ) of the edge portion (EL, ER) from the luminance distribution in each of the detection areas; transforming the coordinates (X, Y) of each pixel in the detection area of the first parallax image to produce a coordinate value d according to the formula d=X-Ytanθ; and for each pixel in the detection area of the first parallax image, determining a value (d sub) equal to the difference between the coordinate values of that pixel and the corresponding pixel in the detection area of the second parallax image. identifying, for each pixel in the detection area of the first parallax image, a corresponding pixel in the detection area of the second parallax image with a luminance that is the same as the selected pixel; for each detection area, estimating an edge angle (θ) of the edge portion (EL, ER) from the luminance distribution in each of the detection areas; transforming the coordinates (X, Y) of each pixel in the detection area of the first parallax image to produce a coordinate value d according to the formula d=X-Ytanθ; and for each pixel in the detection area of the first parallax image, determining a value (d sub) equal to the difference between the coordinate values of that pixel and the corresponding pixel in the detection area of the second parallax image.