An image processing apparatus comprising: an image input unit for inputting an image of a physical space; a physical object measurement unit for measuring a shape of a physical object and a position and orientation of said physical object in the physical space; a laying unit for configuring an object which approximates the measured shape of said physical object, and laying this approximate object out at a position and orientation indicated by the measured position and orientation of the physical object, in a virtual space on which virtual objects and a viewpoint are laid out; a ray control unit for generating a ray from the viewpoint, and for controlling a subsequent ray route of the ray in the virtual space according to a ray tracing method; a determination unit for executing intersect determination between the ray route and an object laid out on the virtual space; a generation unit for generating a pixel (699, 1606, 1610) based on said ray route to generate an image of the virtual space; and an output unit for outputting a composite image of the image of the physical space and the image of the virtual space; characterized in that said approximate object is configured by combining a plurality of virtual elements so as to approximate said measured shape of said physical object; and said generation unit is configured to generate said pixel only based on said ray route to a first intersection point if the ray route first intersects with an approximate object, if also at least one object successively intersected by the ray route is an approximate object and if a distance between intersection points of the intersects (1003, 1004, 1005) is smaller than the size of a virtual element. an image input unit for inputting an image of a physical space; a physical object measurement unit for measuring a shape of a physical object and a position and orientation of said physical object in the physical space; a laying unit for configuring an object which approximates the measured shape of said physical object, and laying this approximate object out at a position and orientation indicated by the measured position and orientation of the physical object, in a virtual space on which virtual objects and a viewpoint are laid out; a ray control unit for generating a ray from the viewpoint, and for controlling a subsequent ray route of the ray in the virtual space according to a ray tracing method; a determination unit for executing intersect determination between the ray route and an object laid out on the virtual space; a generation unit for generating a pixel (699, 1606, 1610) based on said ray route to generate an image of the virtual space; and an output unit for outputting a composite image of the image of the physical space and the image of the virtual space; characterized in that said approximate object is configured by combining a plurality of virtual elements so as to approximate said measured shape of said physical object; and said generation unit is configured to generate said pixel only based on said ray route to a first intersection point if the ray route first intersects with an approximate object, if also at least one object successively intersected by the ray route is an approximate object and if a distance between intersection points of the intersects (1003, 1004, 1005) is smaller than the size of a virtual element.