Patent Description:
There has recently been known network cameras that can acquire an image in a wider imaging range using a plurality of cameras (referred to as multi-lens cameras hereinafter) than that using a single camera. The multi-lens cameras include a combination of various mechanisms, such as a combination of a plurality of fixed cameras and a zoom camera that can vary an imaging magnification, and a combination of a camera that have a field of view of <NUM>° such as a fisheye lens (referred to as an omnidirectional camera hereinafter) and a zoom camera. The plurality of fixed cameras include cameras that enables a manual adjustment of an angle of view in a rotational direction around a horizontal axis (pan), a vertical axis (tilt), and an optical axis center (rotation), and sets a desired imaging area.

One known illustrative imaging method of the multi-lens camera is a method that connects images acquired from fixed cameras to each other for wide-range imaging (panoramic imaging), and provides high-resolution imaging with zoom cameras having different mechanisms. <CIT> discloses a system that sets an imaging position of an omnidirectional camera so as to coincide it with an imaging position of a zoom camera, and images a position designated by the omnidirectional camera using the zoom camera.

However, the system disclosed in <CIT> needs to designate the position of the zoom camera for each area in the image when the fixed camera portion is the omnidirectional camera. In this case, the area outside the designated position or a plurality of areas cannot be displayed on the zoom camera. This problem occurs even when, for example, the fixed camera has a rotation mechanism and the zoom camera has no rotation mechanism. When the fixed camera pans and/or tilts, an imaging range of the fixed camera can be covered by panning and/or tilting the zoom camera. However, when the fixed camera provides rotation driving, the same angle of view cannot be imaged, because a functional difference between the fixed camera and the zoom camera cannot compensate a difference of a rotated angle. Document <CIT> discloses a method and a system of generating an image sequence of an object within a scene. The method includes capturing an image of the object with a plurality of camera systems, wherein the camera systems are positioned around the scene. Next, the method includes 2D projective transforming certain of the images such that a point of interest in each of the images is at a same position as a point of interest in a first image from one of the camera systems. The method further includes outputting the transformed images and the first image in a sequence corresponding to a positioning of the corresponding camera systems around the scene. Document <CIT> discloses an information processing apparatus and method wherein an image of a predetermined region can be picked up and a desired moving body in the predetermined region can be tracked readily to pick up an image of the moving body. A sensor image acquisition module acquires a sensor image. A moving body detection module detects moving bodies existing in a predetermined region based on the sensor image. A tracking object designation module designates one of the detected moving bodies. When a moving body is designated by the tracking object designation module, an image pickup mode changeover module sets the image pickup mode to a continuous tracking mode in which the designated moving body is tracked to pick up an image of the moving body.

Document <CIT> discloses a network PTZ camera that can be mounted in different configurations : erect installation, roof-mounted installation or wall-mounted installation. When the camera is not properly fixed in the wall-mounted installation, the captured images may be captured in a slanting state, thus correction is brought by rotating the camera using a rotation mechanism.

The present invention provides a control apparatus, a control method, and a program, each of which can correct and display a positional shift of a designated area from a zoom camera, which is caused by a rotation of a fixed camera.

The present invention in its first aspect provides a control apparatus as specified in claims <NUM>-<NUM>.

The present invention in its second aspect provides a control method as specified in claim <NUM>.

Referring now to the accompanying drawings, a detailed description will be given of embodiments according to the present invention. Corresponding elements in respective figures will be designated by the same reference numerals, and a duplicate description thereof will be omitted.

<FIG> is a configuration diagram of an imaging system (surveillance system) according to this embodiment. The imaging system includes an image pickup apparatus <NUM> installed on a ceiling, a utility pole, or the like, a terminal <NUM> for displaying an image and various information from the image pickup apparatus <NUM> and for remotely controlling the image pickup apparatus <NUM>, and a network (IP network network) <NUM> connecting them with each other. While <FIG> illustrates a single image pickup apparatus <NUM> and a single terminal <NUM>, the number of these devices is not particularly limited.

The image pickup apparatus <NUM> includes a first imaging unit (multi-lens camera) <NUM> including a plurality of cameras capable of changing an imaging direction to an arbitrary angle and of fixing it, and a second imaging unit <NUM> capable of obtaining a high-definition image and of changing the imaging direction and optical zoom magnification in a narrow angle. The pan, tilt, and rotation of each camera in the first imaging unit <NUM> are individually and manually set. Each camera executes a rotation in the set pan direction, a rotation in the set tilt direction, and a rotation in the set rotation direction, is fixed at an arbitrary angle, and captures an image. The rotation direction refers to a rotating direction around the optical axis as a center. In this embodiment, the plurality of cameras are manually set, but may be automatically set by mounting a motor on a driving unit in each camera. The second imaging unit <NUM> is remotely controlled by the terminal <NUM> and can provide at least one of pan, tilt, and zoom that can change an angle of view (at least one of an imaging direction and an angle of view). The second imaging unit <NUM> can rotate the imaging direction over <NUM>°, and capture an area (second imaging range) included in (or part of) the imaging area (first imaging range) of the first imaging unit <NUM>. The imaging speeds of the first and second imaging units <NUM> and <NUM> are set to <NUM> frames/seconds in this embodiment, but the present invention is not limited to this example.

The image pickup apparatus <NUM> serves to transmit an acquired image to the terminal <NUM> on the network <NUM>. The terminal <NUM> is communicably connected to the image pickup apparatus <NUM> via the network <NUM>, and serves to display an image received from the image pickup apparatus <NUM> and to issue various commands to the image pickup apparatus <NUM>.

<FIG> is a block diagram of the image pickup apparatus <NUM>. The image pickup apparatus <NUM> includes the first imaging unit <NUM>, the second imaging unit <NUM>, a control unit <NUM> that controls the first and second imaging units <NUM> and <NUM> and processes an acquired image, a memory <NUM>, and a network (NW) communication unit <NUM> that communicates with the network <NUM>. In this embodiment, the control unit <NUM> is provided in the image pickup apparatus <NUM>, but may be configured as a control apparatus separate from the image pickup apparatus <NUM>.

The first imaging unit <NUM> includes imaging optical systems <NUM> and <NUM> including a wide-angle lens for acquiring a wide-angle image, image sensors <NUM> and <NUM>, driving units <NUM> and <NUM>, position detecting units <NUM> and <NUM>, and an image processing unit <NUM>. In the first imaging unit <NUM>, a structure including the imaging optical system, the image sensor, the driving unit, and the position detecting unit captures a single imaging area. While <FIG> illustrates two structures, the number of structures is not particularly limited. While the imaging optical systems <NUM> and <NUM> include a wide-angle lens in this embodiment, they may include a zoom lens, a focus lens, a diaphragm mechanism, or the like.

The imaging optical systems <NUM> and <NUM> are arranged so that their optical axes are different from each other, and thus individually capture images in different directions. By arranging the imaging optical systems so that the imaging areas partially overlap each other and by increasing the number of structures, a wide range (or all directions) over <NUM> degrees can be covered.

The image sensors <NUM> and <NUM> include two-dimensional image sensors such as a CCD and CMOS, photoelectrically convert object images formed by the imaging optical systems <NUM> and <NUM>, and supply the obtained electric signals to the image processing unit <NUM>.

The driving units <NUM> and <NUM> can change the imaging directions (optical axis directions) of the first imaging unit <NUM>. This embodiment can fix the first imaging unit <NUM> at a predetermined angle by manually and individually setting the pan, tilt, and rotation. However, the setting is not limited to the manual setting, and the automatic setting may be made by mounting a motor on the driving unit.

The position detecting units <NUM> and <NUM> each include an encoder that detects a mechanical position changes of the driving units <NUM> and <NUM>, and outputs an electric signal indicating the position information, and supply the acquired electric signals to the image processing unit <NUM>.

The image processing unit <NUM> converts the electric signals received from the image sensors <NUM> and <NUM> into digital data, and performs demosaic processing, image quality improving processing, and gain processing for improving an acquired light amount as a signal level. The image processing unit <NUM> performs processing of outputting a reset signal that clears accumulated charges in the image sensors <NUM> and <NUM>. The image processing unit <NUM> performs an exposure control for adjusting a light amount incident on the image sensors <NUM> and <NUM> by changing the shutter speed, F-number (aperture value), and the settings of gain processing. The image processing unit <NUM> supplies the processed image data to the control unit <NUM>.

The second imaging unit <NUM> includes an imaging optical system <NUM>, an image sensor <NUM>, a driving unit <NUM>, a position detecting unit <NUM>, and an image processing unit <NUM>. The imaging optical system <NUM> includes a zoom lens, a focus lens, a diaphragm mechanism, and the like. The image sensor <NUM> photoelectrically converts the object image formed by the imaging optical system <NUM>, and supplies the obtained electric signal to the image processing unit <NUM>. The image processing unit <NUM> serves to provide processing similar to that of the image processing unit <NUM>. The driving unit <NUM> drives various motors (not shown) for controlling the pan, tilt, and operation of the zoom lens in accordance with the control signal from the control unit <NUM>. In this embodiment, the driving unit <NUM> is automatically controlled by driving the motor, but the present invention is not limited to this embodiment. The imaging direction (optical axis direction) and the optical zoom magnification of the second imaging unit <NUM> can be varied by the driving unit <NUM>.

The memory <NUM> is, for example, a nonvolatile memory such as a flash memory and a hard disk drive. The memory <NUM> stores information on the set angle of the first imaging unit <NUM>, coordinate information of the area designated by the terminal <NUM>, and the like. The stored information is used for the processing of the control unit <NUM>.

The NW communication unit <NUM> is an interface for communicating with the network <NUM>. The communication between the NW communication unit <NUM> and the network <NUM> may be wired or wireless.

The control unit <NUM> includes a driving control unit (driving unit) <NUM>, an image display generation unit <NUM>, a communication control unit <NUM>, an angle acquisition unit <NUM>, a frame generation unit <NUM>, an angle conversion unit (setting unit) <NUM>, and a coding unit <NUM>, and a command interpretation unit <NUM>. The control unit <NUM> includes, for example, a CPU (Central Processing Unit), a ROM for storing a program executed by the CPU, and a RAM used as a work area for the CPU. Processing units designated by reference numerals <NUM> to <NUM> are realized by the CPU that executes the program. However, some may be implemented with dedicated hardware.

The driving control unit <NUM> generates a driving signal for controlling the pan, tilt, and the zoom magnification of the driving unit <NUM> in the second imaging unit <NUM>. The driving unit <NUM> drives various unillustrated motors for panning, tilting, and moving the zoom lens based on the driving signal generated by the driving control unit <NUM>. The driving control unit <NUM> also controls focusing, diaphragm, and the like by the first and second imaging units <NUM> and <NUM>. The driving control unit <NUM> serves as a control unit configured to control the second imaging unit <NUM> to instruct the second imaging unit <NUM> to image the second imaging range and to acquire a second image by changing at least one of the imaging direction and the angle of view of the second imaging unit <NUM>.

The image display generation unit <NUM> determines how data from the image processing units <NUM> and <NUM> are displayed on the terminal <NUM>. In particular, it designates a display method of the first imaging unit <NUM>, such as an arrangement of a plurality of images acquired from a plurality of image sensors (where the plurality of images are displayed) and the image preparation order specified when the plurality of images are joined together. The image display generation unit <NUM> performs digital zoom processing for cutting an area from the acquired image and scaling up and down into a desired display size.

The communication control unit <NUM> transmits the coded data generated by the coding unit <NUM> to the terminal <NUM> via the NW communication unit <NUM>. The communication control unit <NUM> transmits an image obtained from the second imaging unit <NUM> and a wide-range image obtained from the first imaging unit <NUM> in a proper layout. When the communication control unit <NUM> receives various request commands from the terminal <NUM> via the NW communication unit <NUM>, the communication control unit <NUM> notifies the command interpretation unit <NUM> of the received request commands.

The angle acquisition unit <NUM> determines a direction of each image sensor using the electric signals output from the position detecting units <NUM> and <NUM> in the first imaging unit <NUM>, and transmits the determination result to the memory <NUM>. The angle acquisition unit <NUM> serves as an acquisition unit configured to acquire information on a rotation of the first imaging unit <NUM> in the rotation direction and position information of a designated range designated on a first image imaged by the first imaging unit.

The frame generation unit <NUM> generates (draws) an area frame surrounding a designated area designated by the terminal <NUM> on the images acquired from the first and second imaging units <NUM> and <NUM>. The frame generation unit <NUM> generates the area frame by calculating the coordinate of the designated area on the image by setting the horizontal direction of the image to the X coordinate and the vertical direction to the Y coordinate. When the frame generation unit <NUM> generates the area frame on the image acquired from the first imaging unit <NUM>, the designated area is displayed by the pan, tilt, and zoom operations of the second imaging unit <NUM>. When the frame generation unit <NUM> generates the area frame on the image acquired from the second imaging unit <NUM>, the designated area is displayed by the zoom operation of the second imaging unit <NUM>.

The angle conversion unit <NUM> instructs the frame generation unit <NUM> to reform a frame using the information acquired by the angle acquisition unit <NUM>, the information generated by the frame generation unit <NUM>, and a relationship between the first and second imaging units <NUM> and <NUM>. The angle conversion unit <NUM> serves as a setting unit configured to set the second imaging range so as to include the designated range based on the information on the rotation and the position information.

The coding unit <NUM> encodes the image data acquired from the image processing units <NUM> and <NUM> and generates coded data. The coded data is transmitted to the communication control unit <NUM>.

The command interpretation unit <NUM> analyzes the request command notified from the communication control unit <NUM> and executes processing according to the request command. For example, when a predetermined area in an image acquired from the first imaging unit <NUM> is designated by the terminal <NUM>, the command interpretation unit <NUM> instructs the driving control unit <NUM> so that the second imaging unit <NUM> images the same area as the designated area.

Referring now to <FIG>, a description will be given of an internal configuration of the terminal <NUM>. <FIG> is a block diagram of the terminal <NUM>.

The terminal <NUM> is an information processing apparatus typified by a personal computer. The terminal <NUM> includes a CPU <NUM> that controls the entire terminal, a ROM <NUM> that stores a BIOS and a boot program, and a RAM <NUM> that stores an OS (operating system), a camera application, and the like, and serves as a work area. The terminal <NUM> further includes an HDD <NUM> for storing the OS, a camera application, and the like, and a network I/F <NUM> for communicating with the network <NUM>. The terminal <NUM> further includes an operation unit <NUM> including a keyboard, a mouse, a touch panel, and the like for inputting an instruction from the user. The terminal <NUM> further includes a display control unit <NUM> and a display device <NUM>.

When the terminal <NUM> is powered on, the CPU <NUM> loads the OS from the HDD <NUM> to the RAM <NUM> and executes it according to the boot program stored in the ROM <NUM>. Thereby, the terminal <NUM> serves as an information processing apparatus. As a result, the operation unit <NUM> and the display device <NUM> serves as user interfaces. The user operates the operation unit <NUM> to instruct the execution of the camera application program, so that the camera application is loaded from the HDD <NUM> into the RAM <NUM> and then executed. Thereby, the terminal <NUM> serves as an apparatus for displaying the image acquired from the image pickup apparatus <NUM>.

<FIG> illustrates an illustrative screen that displays the image acquired from the image pickup apparatus <NUM> according to this embodiment, on the display device <NUM> in the terminal <NUM>. The display device <NUM> displays a first image <NUM> acquired from the first imaging unit <NUM> and a second image <NUM> acquired from the second imaging unit <NUM>. An area <NUM> surrounded by a broken line in the first image <NUM> indicates a designated area to be enlarged and displayed by the second imaging unit <NUM>. That is, in the second image <NUM>, the second imaging unit <NUM> provides the pan, tilt, and zoom processing and displays the area <NUM> in the first image <NUM>.

For description convenience, the first image <NUM> is displayed as an image acquired from one of a plurality of image sensors provided in the first imaging unit <NUM>, but actually displayed as images acquired from the plurality of image sensors. The image actually displayed is generated by the image display generation unit <NUM>. For example, when there are four image sensors, the first image <NUM> may be displayed as four images, or may be displayed as one wide-angle image in which the images acquired from the image sensors are joined together.

When the first imaging unit <NUM> pans and tilts, an imaging range that can be captured by the first imaging unit <NUM> can be covered if the second imaging unit <NUM> also pans and tilts. That is, acquiring the coordinate information of the area <NUM> enables the second imaging unit <NUM> to image the area <NUM>.

When the first imaging unit <NUM> rotates in the rotation direction (performs the rotation operation), the area <NUM> cannot be displayed on the second imaging unit <NUM> using the coordinate information of the first image <NUM>. In addition, a shift caused by the rotation generated by the rotation operation cannot be compensated.

<FIG> illustrates an image 401a before the rotation operation of the first image <NUM>, which is acquired by the first imaging unit <NUM> performing the rotation operation at an angle of <NUM>°. When the user designates the area <NUM> using the display device <NUM>, it means that the area 403a in <FIG> is actually designated. Since the second imaging unit <NUM> cannot perform the rotation operation, the area 403a cannot be represented. That is, the second image <NUM> shows that the vehicle tilts by <NUM>°.

A description will now be given of a control method of the second imaging unit <NUM> when the first imaging unit <NUM> according to this embodiment rotates in the rotation direction. <FIG> is a flowchart showing the control method of the second imaging unit <NUM> when the first imaging unit <NUM> according to this embodiment rotates in the rotation direction.

In the step S101, the control unit <NUM> (angle acquisition unit <NUM>) determines whether or not the first imaging unit <NUM> is performing the rotation operation using the electric signals acquired from the position detecting units <NUM> and <NUM>. If it is determined that the first imaging unit <NUM> is performing the rotation operation, the flow proceeds to the step S105, otherwise the flow proceeds to the step S102.

In the step S102, the control unit <NUM> (frame generation unit <NUM>) generates an area frame (corresponding to a frame surrounding the area <NUM> in <FIG>) on the first image <NUM> acquired from the first imaging unit <NUM>.

In the step S103, the control unit <NUM> acquires the coordinate information of the area frame generated in the step S102. The acquired coordinate information is stored in the memory <NUM>.

In the step S104, the control unit <NUM> moves the second imaging unit <NUM> so as to image the area corresponding to the area frame generated in the step S102. In this embodiment, the control unit <NUM> makes the second imaging unit <NUM> pan, tilt, and zoom based on the relationship between the coordinate information of the area frame and the first and second imaging units <NUM> and <NUM> acquired in advance. The image acquired by the second imaging unit <NUM> is displayed as the second image <NUM> on the display device <NUM>. For example, information on the initial angle of view of the first imaging unit <NUM> and information on the setting of the second imaging unit <NUM> in the frame generation may be stored in the memory <NUM>, and it may be moved from the setting of the second imaging unit <NUM> in the frame generation using a shift of an angle of view in the frame generation from the initial angle of view.

In the step S105, the control unit <NUM> (angle acquisition unit <NUM>) acquires the angle of the rotation operation of the first imaging unit <NUM> (information on the rotation in the rotation operation) using the electric signals acquired from the position detecting units <NUM> and <NUM>.

In the step S106, the control unit <NUM> (frame generation unit <NUM>) generates an area frame (corresponding to a frame surrounding the area <NUM> in <FIG>) on the first image <NUM> acquired from the first imaging unit <NUM>. The area <NUM> is an area in which the rotation operation of the first imaging unit <NUM> is reflected.

In the step S107, the control unit <NUM> acquires the coordinate information of the area frame generated in the step S106.

In the step S108, the control unit <NUM> (angle conversion unit <NUM>) converts the coordinate information acquired in the step S107 using the angle of the rotation operation and the relationship between the first and second imaging units <NUM> and <NUM>.

In the step S109, the control unit <NUM> (angle conversion unit <NUM>) generates an area frame (converted area frame hereinafter) on the first image <NUM> acquired from the first imaging unit <NUM> using the converted coordinates.

In the step S110, the control unit <NUM> generates an enlarged area frame in which the second imaging unit <NUM> can move from the converted area frame generated in the step S109.

In the step S111, the control unit <NUM> moves the second imaging unit <NUM> so as to capture an area corresponding to the enlarged area frame generated in the step S110.

In the step S112, the control unit <NUM> (image display generation unit <NUM>) cuts out an area corresponding to the converted area frame from the image acquired from the second imaging unit <NUM> after the movement, and performs enlargement processing so that the cut out area is as large as the second image <NUM>.

In the step S113, the control unit <NUM> (image display generation unit <NUM>) rotates the image processed in the step S112 by the angle of the rotation operation acquired in the step S105 (executes the rotation processing) and displays the result on the display device <NUM>.

Referring now to <FIG>, a description will be given of a method of designating an area. <FIG> explain the setting of the designated area.

<FIG> assume the rotation operation of the first imaging unit <NUM> by <NUM>° around a center point <NUM> as the optical axis center indicated by a first coordinate o(<NUM>, <NUM>), where the X-axis is set to the horizontal direction, the Y-axis is set to the vertical direction, and one square in the figure is set to one pixel.

<FIG> shows, in an superimposition manner, the image 401a and the area 403a before the rotation operation and the first image <NUM> and the area <NUM> after the rotation operation. Coordinates of vertices a to d in the area <NUM> are a(-<NUM>, -<NUM>), b(-<NUM>, -<NUM>), c(-<NUM>, -<NUM>), and d(-<NUM>, -<NUM>), respectively, and correspond to coordinate information acquired in the step S107 in <FIG>. Coordinates of vertices aa to dd in the area 403a are aa(-<NUM>, <NUM>), bb(-<NUM>, <NUM>), cc(-<NUM>, <NUM>), and dd(-<NUM>, -<NUM>), respectively, and are coordinates of the converted area frame generated in the step S109 in <FIG>.

The vertices aa and dd are points where the vertices a and d in the area <NUM> are rotated around the center point <NUM> by <NUM> ° on the coaxial circle indicated by an alternate long and short dash line <NUM>, respectively. The vertex cc is a point where the vertex c in the area <NUM> is rotated around the center point <NUM> on the coaxial circle indicated by a dotted line <NUM> by <NUM>°. The vertex bb is a point where the vertex b in the area <NUM> is rotated around the center point <NUM> on a coaxial circle indicated by a broken line <NUM> by <NUM>°. Each coordinate of the vertices aa to dd is calculated by the three-square theorem.

<FIG> shows an area 403b that is made by connecting points to which the vertices of the area 403a are moved by the maximum amount when the second imaging unit <NUM> performs pan, tilt, and zoom operations, in addition to the display of <FIG>. Coordinates of vertices aaa to ddd in the area 403b are aaa(-<NUM>, <NUM>), bbb(-<NUM>, -<NUM>), ccc(-<NUM>, <NUM>), and ddd(-<NUM>, -<NUM>), respectively. The area 403b includes the area 403a, and the outer circumference of the area 403b is an enlarged area frame generated in the step S110 in <FIG>.

As described above, according to this embodiment, even when the first imaging unit <NUM> rotates, the steps S111 to S113 in <FIG> enable the second imaging unit <NUM> to image an area corresponding to a designated area on the first image <NUM>.

While this embodiment sets the angle of the rotation operation to <NUM>°, the present invention is not limited to this embodiment. Even when a plurality of first imaging units <NUM> are provided, they can be individually controlled. When the angles of the rotation operation are <NUM>°, <NUM>°, and <NUM>°, the compatible image sensor <NUM> may change the read direction.

It is not always necessary to generate an area frame (converted area frame) or an enlarged area frame.

When the area frame (converted area frame) and the enlarged area frame are not generated, the coordinate information of the enlarged area to be imaged by the second imaging unit <NUM> is acquired from the converted coordinates acquired in the step S108. Then, in the step S111, the second imaging unit <NUM> images an area corresponding to the coordinate information of the enlarged area. That is, after the coordinate conversion in the step S108, the steps S109 and S110 are skipped, and instead, the coordinate information of the enlarged area to be imaged by the second imaging unit <NUM> is acquired from the converted coordinates acquired in the step S108. Then, the flow proceeds to the step S110.

An image pickup apparatus according to this embodiment has the same configuration as that of the image pickup apparatus of the first embodiment. This embodiment will discuss differences from the first embodiment, and a description of common parts will be omitted.

This embodiment will discuss a configuration that superimposes the second image <NUM> onto the first image <NUM>. This embodiment intends, when the second image <NUM> is acquired, to acquire a high-resolution image with only optical zooming without the cutout processing or the rotation processing, and to make a shift in the rotation between the first and second imaging units <NUM> and <NUM> more easily visually recognizable.

<FIG> illustrates an example of the first image <NUM> acquired from the first imaging unit <NUM> according to this embodiment. In this embodiment, the angle of the rotation operation of the first imaging unit <NUM> is <NUM>°. <FIG> illustrates on the first image <NUM> that the area is designated by the mouse in the operation unit <NUM> of the terminal <NUM> in forming the area <NUM>. An arrow <NUM> starts at a point where the mouse is clicked, and ends at a point where the mouse that is being clicked is moved to a predetermined position and then the click is released. Determining the direction and size of the arrow <NUM> by this drag-and-drop processing can form the area <NUM>.

This embodiment displays, on the first image <NUM>, an area <NUM> (enlarged area frame) surrounded by an alternate long and two short dashes line, which is an imaging area of the second imaging unit <NUM> in consideration of the rotation of the first imaging unit <NUM>. The area <NUM> is offset by <NUM>° from the area <NUM> and so large that it includes the area <NUM>. The area <NUM> may be displayed according to the action of pulling the arrow <NUM> with the mouse during the drag-and-drop processing, or may be displayed after the drag-and-drop processing is completed.

<FIG> illustrates an example of an image displayed on the display device <NUM> according to this embodiment. In an area adjacent to the first image <NUM>, the second image <NUM> acquired by imaging the area <NUM> is displayed. In the configuration according to this embodiment, a car in the first image <NUM> is displayed and shifted by <NUM>° from a car in a second image <NUM>. The second image <NUM> is a high-resolution image since no digital zoom processing is performed for it.

As described above, the configuration according to this embodiment superimposes the second image <NUM> acquired from the second imaging unit <NUM> onto the first image <NUM>. This configuration provides a high-resolution image and a shift in the rotation between the first and second imaging units <NUM> and <NUM> to be visually expressed.

The user may be able to select a display of the second image with the first image in consideration of the rotation operation or a display of only optical zooming. As a method of converting the angle of the rotation operation, if there is already a calculated value of the same angle, it may be shared with another imaging unit.

Each embodiment provides a control apparatus (<NUM>) for a first imaging unit (<NUM>) rotatable in a rotation direction around an optical axis and configured to image a first imaging range, and a second imaging unit (<NUM>) configured to change at least one of an imaging direction and an angle of view and to image a second imaging range that is part of the first imaging range. The control apparatus includes an acquisition unit (<NUM>) configured to acquire information on a rotation of the first imaging unit in the rotation direction and position information of a designated range (<NUM>) designated on a first image (<NUM>) imaged by the first imaging unit, a setting unit (<NUM>) configured to set the second imaging range so as to include the designated range based on the information on the rotation and the position information, and a control unit (<NUM>) configured to control the second imaging unit to instruct the second imaging unit to image the second imaging range and to acquire a second image (<NUM>) by changing at least one of the imaging direction and the angle of view of the second imaging unit. At least one processor or circuit is configured to perform a function of at least one of the units.

The setting unit sets the second imaging range using position information of an area corresponding to the designated range on an image that can be acquired before the first imaging unit rotates in the rotation direction. The second image is acquired by cutout processing on an image acquired by imaging the second imaging range and rotation processing based on the information on the rotation, according to the designated range. An image of the second imaging range may be superimposed on the first image. A size of an image of the second imaging range may be changed by a user operation. The second imaging unit may not be able to rotate in the rotation direction.

Each embodiment provides a control method for a first imaging unit (<NUM>) rotatable in a rotation direction around an optical axis and configured to image a first imaging range, and a second imaging unit (<NUM>) configured to change at least one of an imaging direction and an angle of view and to image a second imaging range that is part of the first imaging range. The control method includes the steps of acquiring information on a rotation of the first imaging unit in the rotation direction (S105) and position information of a designated range (<NUM>) designated on a first image (<NUM>) imaged by the first imaging unit (S107), setting (S108-S110) the second imaging range so as to include the designated range based on the information on the rotation and the position information, and controlling (S113) the second imaging unit to instruct the second imaging unit to image the second imaging range and to acquire a second image (<NUM>) by changing at least one of the imaging direction and the angle of view of the second imaging unit and by acquiring the second image by cutout processing on an image acquired by imaging the second imaging range and rotation processing based on the information on the rotation, according to the designated range.

Claim 1:
A image pickup apparatus (<NUM>) comprising:
a first imaging unit (<NUM>) rotatable in a rotation direction around an optical axis and configured to image a first imaging range;
a second imaging unit (<NUM>) configured to change at least one of an imaging direction and an angle of view and to image a second imaging range that is part of the first imaging range;
an acquisition unit (<NUM>) configured to acquire information on a rotation of the first imaging unit (<NUM>) in the rotation direction using an electric signal output from a position detection unit (<NUM>, <NUM>) in the first imaging unit (<NUM>) and position information of a designated range (<NUM>) designated by a user using a terminal (<NUM>) on a first image (<NUM>) imaged by the first imaging unit;
a setting unit (<NUM>) configured to set the second imaging range so as to include the designated range based on the information on the rotation and the position information; and
a control unit (<NUM>) configured to control the second imaging unit to instruct the second imaging unit to image the second imaging range and to acquire a second image (<NUM>) by changing at least one of the imaging direction and the angle of view of the second imaging unit,
characterized by further comprising a generation unit (<NUM>) configured to acquire the second image by cutout processing on an image acquired by imaging the second imaging range and rotation processing based on the information on the rotation, according to the designated range.