An image processing apparatus, comprising: a scale-down factor determination unit configured to generate two-dimensional images of an object viewed from a plurality of viewpoints using three-dimensional data representing a three-dimensional shape of the object and determine a scale-down factor for each viewpoint; a template generation unit configured to generate a template for each viewpoint by scaling down the generated two-dimensional image from each viewpoint using the scale-down factor determined for the viewpoint and calculating, from the scaled-down image, a plurality of feature quantities of a plurality of feature points in the scaled-down image, wherein the plurality of feature quantities are to be used for object recognition with template matching, wherein the plurality of feature points are image coordinate positions corresponding to predetermined features, wherein the template represents the plurality of feature quantities of the plurality of feature points in the scaled-down image; and a template information output unit configured to output, for each viewpoint, the template and the scale-down factor used for generating the template, the template and the scale-down factor being associated with each other, wherein the scale-down factor determination unit is configured to determine the scale-down factor to allow the template for each viewpoint to include at least a predetermined number of feature points; or the object to be included in the scaled-down image from each viewpoint to have a fixed area. a scale-down factor determination unit configured to generate two-dimensional images of an object viewed from a plurality of viewpoints using three-dimensional data representing a three-dimensional shape of the object and determine a scale-down factor for each viewpoint; a template generation unit configured to generate a template for each viewpoint by scaling down the generated two-dimensional image from each viewpoint using the scale-down factor determined for the viewpoint and calculating, from the scaled-down image, a plurality of feature quantities of a plurality of feature points in the scaled-down image, wherein the plurality of feature quantities are to be used for object recognition with template matching, wherein the plurality of feature points are image coordinate positions corresponding to predetermined features, wherein the template represents the plurality of feature quantities of the plurality of feature points in the scaled-down image; and a template information output unit configured to output, for each viewpoint, the template and the scale-down factor used for generating the template, the template and the scale-down factor being associated with each other, wherein the scale-down factor determination unit is configured to determine the scale-down factor to allow the template for each viewpoint to include at least a predetermined number of feature points; or the object to be included in the scaled-down image from each viewpoint to have a fixed area. the template for each viewpoint to include at least a predetermined number of feature points; or the object to be included in the scaled-down image from each viewpoint to have a fixed area.