A control apparatus comprising: detecting means for detecting a touch operation on a display surface for displaying a virtual-viewpoint image which corresponds to a position of a virtual viewpoint and a direction of view from the virtual viewpoint in a three-dimensional space and which is generated based on image data obtained by a plurality of image capturing apparatuses; determining means for determining an amount of change of at least one of the position of the virtual viewpoint and the direction of view from the virtual viewpoint based on the number of coordinates that are concurrently specified with the touch operation detected by the detecting means and an amount of variation of the coordinates, wherein in a first case where a first number of coordinates are concurrently specified with the touch operation and said touch operation is a slide operation, the position of the virtual viewpoint and the direction of view from the virtual viewpoint are varied in accordance with variation of the coordinates and a first target position to which the virtual viewpoint is directed, wherein the variation of the coordinates is specified with the slide operation and the first target position is held constant; in a second case where a second number of coordinates are concurrently specified with the touch operation and said touch operation is a slide operation, the position of the virtual viewpoint and the direction of view from the virtual viewpoint are varied in accordance with variation of the coordinates and a second target position determined based on the coordinates that are concurrently specified with the touch operation, wherein the variation of the coordinates is specified with the slide operation and the second target position is held constant; in a third case where a third number of coordinates are concurrently specified with the touch operation and said touch operation is a slide operation, the position of the virtual viewpoint is varied in accordance with variation of the coordinates, and the direction of view from the virtual viewpoint is held constant; in a fourth case when a fourth number of the coordinates are concurrently specified with the touch operation and said touch operation is a pinch operation, the position of the virtual viewpoint is moved in a direction corresponding to the direction of view from the virtual viewpoint in accordance with variation of the coordinates, wherein the variation of the coordinates is specified with the pinch operation; wherein said second number and said fourth number are two, and said first and third numbers are different from two and from each other; and control means for controlling at least one of the position of the virtual viewpoint and the direction of view from the virtual viewpoint in accordance with the amount of change determined by the determining means. detecting means for detecting a touch operation on a display surface for displaying a virtual-viewpoint image which corresponds to a position of a virtual viewpoint and a direction of view from the virtual viewpoint in a three-dimensional space and which is generated based on image data obtained by a plurality of image capturing apparatuses; determining means for determining an amount of change of at least one of the position of the virtual viewpoint and the direction of view from the virtual viewpoint based on the number of coordinates that are concurrently specified with the touch operation detected by the detecting means and an amount of variation of the coordinates, wherein in a first case where a first number of coordinates are concurrently specified with the touch operation and said touch operation is a slide operation, the position of the virtual viewpoint and the direction of view from the virtual viewpoint are varied in accordance with variation of the coordinates and a first target position to which the virtual viewpoint is directed, wherein the variation of the coordinates is specified with the slide operation and the first target position is held constant; in a second case where a second number of coordinates are concurrently specified with the touch operation and said touch operation is a slide operation, the position of the virtual viewpoint and the direction of view from the virtual viewpoint are varied in accordance with variation of the coordinates and a second target position determined based on the coordinates that are concurrently specified with the touch operation, wherein the variation of the coordinates is specified with the slide operation and the second target position is held constant; in a third case where a third number of coordinates are concurrently specified with the touch operation and said touch operation is a slide operation, the position of the virtual viewpoint is varied in accordance with variation of the coordinates, and the direction of view from the virtual viewpoint is held constant; in a fourth case when a fourth number of the coordinates are concurrently specified with the touch operation and said touch operation is a pinch operation, the position of the virtual viewpoint is moved in a direction corresponding to the direction of view from the virtual viewpoint in accordance with variation of the coordinates, wherein the variation of the coordinates is specified with the pinch operation; wherein said second number and said fourth number are two, and said first and third numbers are different from two and from each other; and control means for controlling at least one of the position of the virtual viewpoint and the direction of view from the virtual viewpoint in accordance with the amount of change determined by the determining means.