An image processing apparatus comprising: a three-dimensional data acquisition unit that acquires 3D data of a 3D model in which a captured image is mapped on a surface; a correct image acquisition unit that acquires captured image data used for the mapping to the 3D model as a correct image; a two-dimensional image generation unit that generates 2D image data from the 3D data acquired by the 3D data acquisition unit according to a predetermined 2D conversion algorithm; and an evaluation value calculation unit that calculates an evaluation value representing similarity between the 2D image data generated by the 2D image generation unit and the captured image data acquired by the correct image acquisition unit, wherein the 2D image generation unit modifies the 2D conversion algorithm by learning to optimize the evaluation value calculated each time for 2D image data generated when the 3D data is input to the 2D image generation unit and processing is repeatedly performed, the evaluation value calculation unit includes a second evaluation value calculation unit that has a learned neural network for image classification of the 2D image data, the second evaluation value calculation unit: calculates a sum or an average value of differences between color values at respective pixels of the correct image input to the first layer of the neural network and color values at respective pixels of the 2D image data input to the first layer of the neural network, calculates a sum or an average value of differences between feature quantity groups in a feature map generated from each of the correct image and the 2D image data in each of the second and subsequent layers, totals the sum or the average values calculated in the respective layers to get a total difference, and acquires the total difference as the evaluation value, and the 2D conversion algorithm of the 2D image generation unit is modified to minimize the evaluation value calculated by the second evaluation value calculation unit. a three-dimensional data acquisition unit that acquires 3D data of a 3D model in which a captured image is mapped on a surface; a correct image acquisition unit that acquires captured image data used for the mapping to the 3D model as a correct image; a two-dimensional image generation unit that generates 2D image data from the 3D data acquired by the 3D data acquisition unit according to a predetermined 2D conversion algorithm; and an evaluation value calculation unit that calculates an evaluation value representing similarity between the 2D image data generated by the 2D image generation unit and the captured image data acquired by the correct image acquisition unit, wherein the 2D image generation unit modifies the 2D conversion algorithm by learning to optimize the evaluation value calculated each time for 2D image data generated when the 3D data is input to the 2D image generation unit and processing is repeatedly performed, the evaluation value calculation unit includes a second evaluation value calculation unit that has a learned neural network for image classification of the 2D image data, the second evaluation value calculation unit: calculates a sum or an average value of differences between color values at respective pixels of the correct image input to the first layer of the neural network and color values at respective pixels of the 2D image data input to the first layer of the neural network, calculates a sum or an average value of differences between feature quantity groups in a feature map generated from each of the correct image and the 2D image data in each of the second and subsequent layers, totals the sum or the average values calculated in the respective layers to get a total difference, and acquires the total difference as the evaluation value, and calculates a sum or an average value of differences between color values at respective pixels of the correct image input to the first layer of the neural network and color values at respective pixels of the 2D image data input to the first layer of the neural network, calculates a sum or an average value of differences between feature quantity groups in a feature map generated from each of the correct image and the 2D image data in each of the second and subsequent layers, totals the sum or the average values calculated in the respective layers to get a total difference, and acquires the total difference as the evaluation value, and the 2D conversion algorithm of the 2D image generation unit is modified to minimize the evaluation value calculated by the second evaluation value calculation unit.