An image processing apparatus comprising: a polarized image processing unit configured to separate or to extract a reflection component using a polarized image generated by an image pickup element having a configuration in which an identical polarization pixel block made up of a plurality of pixels with an identical polarization direction is provided for each of a plurality of polarization directions and pixels of respective predetermined colors are provided in the identical polarization pixel block; and a normal line information generating unit configured to generate normal line information from the polarized image from which the reflection component has been separated or extracted by the polarized image processing unit, wherein: the polarized image processing unit is configured to separate a diffuse reflection component and a specular reflection component using the polarized image; the normal line information generating unit is configured to integrate the normal line information generated from a polarized image indicating the diffuse reflection component separated by the polarized image processing unit and the normal line information generated from a polarized image indicating the specular reflection component separated by the polarized image processing unit; and the normal line information generating unit is configured to weight the polarized image indicating the diffuse reflection component and the polarized image indicating the specular reflection component and integrate the normal line information. a polarized image processing unit configured to separate or to extract a reflection component using a polarized image generated by an image pickup element having a configuration in which an identical polarization pixel block made up of a plurality of pixels with an identical polarization direction is provided for each of a plurality of polarization directions and pixels of respective predetermined colors are provided in the identical polarization pixel block; and a normal line information generating unit configured to generate normal line information from the polarized image from which the reflection component has been separated or extracted by the polarized image processing unit, wherein: the polarized image processing unit is configured to separate a diffuse reflection component and a specular reflection component using the polarized image; the normal line information generating unit is configured to integrate the normal line information generated from a polarized image indicating the diffuse reflection component separated by the polarized image processing unit and the normal line information generated from a polarized image indicating the specular reflection component separated by the polarized image processing unit; and the normal line information generating unit is configured to weight the polarized image indicating the diffuse reflection component and the polarized image indicating the specular reflection component and integrate the normal line information. the polarized image processing unit is configured to separate a diffuse reflection component and a specular reflection component using the polarized image; the normal line information generating unit is configured to integrate the normal line information generated from a polarized image indicating the diffuse reflection component separated by the polarized image processing unit and the normal line information generated from a polarized image indicating the specular reflection component separated by the polarized image processing unit; and the normal line information generating unit is configured to weight the polarized image indicating the diffuse reflection component and the polarized image indicating the specular reflection component and integrate the normal line information.