An image processing device comprising: a first polarization means which has a first polarization region and a second polarization region that each allows different polarized light to be transmitted therethrough; a second polarization means into which the transmitted light of the first polarization means enters, and which is made up of a third polarization region (15V,H) that allows only the transmitted light of either one of the first polarization region or the second polarization region to be transmitted therethrough, and a total transmission region that allows the transmitted light of the first polarization region and the second polarization region to be transmitted therethrough; an imaging element into which the transmitted light of the second polarization means is input; and an image processing unit which executes signal processing for an output signal of the imaging element, wherein the image processing unit generates parallax information in which subject distance is reflected by applying a first viewpoint image (63, 123) that is based on the transmitted light of the total transmission region of the second polarization means, and a second viewpoint image that is based on the transmitted light of the third polarization region of the second polarization means, and generates a left-eye-purpose image (71, 151) and a right-eye-purpose image (72, 152) for three-dimensional image display by means of image conversion processing of the first viewpoint image for which the parallax information is applied, and characterized in that the image processing unit executes brightness adjustment (105, 126) with which the brightness level of the viewpoint image of either the right-eye-purpose image or the left-eye-purpose image that is the second viewpoint image that is based on the transmitted light of the third polarization region of the second polarization means is made to match the brightness level of the first viewpoint image that is based on the transmitted light of the total transmission region of the second polarization means, and applies the second viewpoint image subsequent to the brightness adjustment to generate the parallax information. a first polarization means which has a first polarization region and a second polarization region that each allows different polarized light to be transmitted therethrough; a second polarization means into which the transmitted light of the first polarization means enters, and which is made up of a third polarization region (15V,H) that allows only the transmitted light of either one of the first polarization region or the second polarization region to be transmitted therethrough, and a total transmission region that allows the transmitted light of the first polarization region and the second polarization region to be transmitted therethrough; an imaging element into which the transmitted light of the second polarization means is input; and an image processing unit which executes signal processing for an output signal of the imaging element, wherein the image processing unit generates parallax information in which subject distance is reflected by applying a first viewpoint image (63, 123) that is based on the transmitted light of the total transmission region of the second polarization means, and a second viewpoint image that is based on the transmitted light of the third polarization region of the second polarization means, and generates a left-eye-purpose image (71, 151) and a right-eye-purpose image (72, 152) for three-dimensional image display by means of image conversion processing of the first viewpoint image for which the parallax information is applied, and characterized in that the image processing unit executes brightness adjustment (105, 126) with which the brightness level of the viewpoint image of either the right-eye-purpose image or the left-eye-purpose image that is the second viewpoint image that is based on the transmitted light of the third polarization region of the second polarization means is made to match the brightness level of the first viewpoint image that is based on the transmitted light of the total transmission region of the second polarization means, and applies the second viewpoint image subsequent to the brightness adjustment to generate the parallax information.