Imaging device, imaging method, and imaging system

The present invention is an imaging apparatus including: a receiving unit that receives information representing a first focal length from a terminal which produces an image of a first imaging area by capturing an image of a subject at the first focal length; an imaging unit that produces an image of a second imaging area by capturing an image of the subject at a second focal length; an extraction unit that extracts a characteristic area of the subject from the image of the second imaging area based on information representing the first focal length and information representing the second focal length; and a transmitting unit that transmits information regarding the image of the characteristic area to the terminal.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an imaging device, an imaging method, and an imaging system for capturing images of the same object from different positions by combining a plurality of imaging devices in order to use 3-D imaging or the like.

Priority is claimed on Japanese Patent Application No. 2009-243290, filed Oct. 22, 2009, the content of which is incorporated herein by reference.

2. Description of Related Art

There is a function of simultaneously executing a shot instruction to a plurality of cameras through remote controlling. Using this function, a synchronized shot can be performed using a plurality of cameras in various applications such as a panorama image or a 3-D image. In this case, a photographer adjusts framing one by one for a plurality of cameras to make a shot. A method of addressing such cumbersomeness while capturing an image is disclosed in Japanese Unexamined Patent Application Publication No. 2009-94724.

According to Japanese Unexamined Patent Application Publication No. 2009-94724, there is provided an imaging apparatus capable of obtaining the merits of both an integrated type compound eye camera and a separation type compound eye camera and of easily determining a composition when capturing images. Specifically, the main camera measures the counterpart position by transmitting a relative positional detection signal from the main camera to the subsidiary camera and transmitting a receiving time of the relative positional detection signal from the subsidiary camera to the main camera. In addition, in order to match framing between the main camera and the subsidiary camera, a movement direction computation means and a movement direction display means are provided.

SUMMARY OF THE INVENTION

The present invention proposes the following means. A first imaging apparatus according to the present invention includes: a receiving unit that receives information representing a first focal length from a terminal which produces an image of a first imaging area by capturing an image of a subject at the first focal length; an imaging unit that produces an image of a second imaging area by capturing an image of the subject at a second focal length; an extraction unit that extracts a characteristic area of the subject from the image of the second imaging area based on information representing the first focal length and information representing the second focal length; and a transmitting unit that transmits information regarding the image of the characteristic area to the terminal.

In the first imaging apparatus, the extraction unit may further extract characteristics of the subject from an image of the characteristic area, and the information regarding the image of the characteristic area may represent the characteristics of the subject.

In the first imaging apparatus, the transmitting unit may further transmit information representing the second focal length to the terminal.

A second imaging apparatus according to the present invention includes: a receiving unit that receives information representing the second focal length and information regarding an image of a characteristic area of a subject extracted from an image of a second imaging area from a terminal which produces the image of a second imaging area by capturing an image of the subject at a second focal length; an imaging unit that produces an image of a first imaging area by capturing an image of the subject at a first focal length; an area determination unit that determines a detection target area for detecting an image of the first imaging area corresponding to an image of the characteristic area based on information regarding the image of the characteristic area, information representing the first focal length, and information representing the second focal length; a detection unit that detects an image of the first imaging area corresponding to the image of the characteristic area based on information regarding the image of the characteristic area and an image of the detection target area within the first imaging area; and a control unit that adjusts focus of the subject based on a result of the detection.

In the second imaging apparatus, the receiving unit may further receive relative relationship information regarding a relative positional relationship between the terminal and the subject when the terminal images the subject at the second focal length and information representing a second imaging posture when the terminal images the subject at a second focal length. And the control unit may further execute framing computation based on the relative relationship information, information representing the second imaging posture, and information representing a first imaging posture when the imaging apparatus images the subject at the first focal length.

The second imaging apparatus may further include an adjustment unit that adjusts the first imaging posture based on a result of the framing computation.

The second imaging apparatus may further include a display unit that displays a result of the framing computation.

In the second imaging apparatus, the control unit may execute framing computation for capturing an image used together with an image obtained from the terminal by imaging the subject when a 3-D image or a panorama image of the subject is produced.

The second imaging apparatus may further include a transmitting unit that transmits the information representing the first focal length to the terminal.

A first imaging method according to the present invention includes the steps of: receiving information representing a first focal length of a subject from a terminal which produces an image of a first imaging area by capturing an image of the subject at the first focal length; producing an image of a second imaging area by capturing an image of the subject at a second focal length and extracting a characteristic area of the subject from an image of the second imaging area based on information representing the first focal length and information representing the second focal length; and transmitting information regarding the image of the characteristic area to the terminal.

A second imaging method according to the present invention includes the steps of: receiving information representing a second focal length and information regarding an image of a characteristic area of a subject extracted from an image of a second imaging area from a terminal which produces an image of the second imaging area by capturing an image of the subject at the second focal length; producing an image of a imaging area by capturing an image of the subject at a first focal length; determining a detection target area for detecting the image of the first imaging area corresponding to the image of the characteristic area based on the information regarding the image of the characteristic area, information representing the first focal length, and information representing the second focal length; detecting the image of the first imaging area corresponding to the image of the characteristic area based on information regarding the image of the characteristic area and an image of the detection target area within the first imaging area; and adjusting focus of the subject based on a result of the detection.

An imaging system according to the present invention includes a first imaging apparatus that produces an image of a first imaging area by capturing an image of a subject at a first focal length and a second imaging apparatus that produces an image of a second imaging area by capturing an image of the subject at a second focal length. The first imaging apparatus includes: a first receiving unit that receives information regarding an image of a characteristic area of the subject extracted from the image of the second imaging area and information representing the second focal length from the second imaging apparatus; a first imaging unit that produces the image of the first imaging area by capturing an image of the subject at the first focal length; a first transmitting unit that transmits information representing the first focal length to the second imaging apparatus; an area determination unit that determines a detection target area for detecting an image of the first imaging area corresponding to the image of the characteristic area based on information regarding the image of the characteristic area, information representing the first focal length, and information representing the second focal length; a detection unit that detects an image of the first imaging area corresponding to the image of the characteristic area based on information regarding the image of the characteristic area and an image of the detection target area within the first imaging area; and a control unit that adjusts focus of the subject based on a result of the detection. In addition, the second imaging apparatus includes: a second receiving unit that receives information representing the first focal length from the first imaging apparatus; a second imaging unit that produces an image of the second imaging area by capturing an image of the subject at the second focal length; an extraction unit that extracts the characteristic area of the subject from the image of the second imaging area based on information representing the first focal length and information representing the second focal length; and a second transmitting unit that transmits information regarding the image of the characteristic area and information representing the second focal length to the first imaging apparatus.

DETAILED DESCRIPTION OF THE INVENTION

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.FIG. 1illustrates a configuration of an imaging system according to an embodiment of the present invention. As shown inFIG. 1, there are two cameras, camera101and camera102, and these cameras simultaneously image the same subject105. In the present embodiment, while a case where two cameras are used is described, three or more cameras may be used.

The cameras101and102have a communication capability and while communicating posture information or positional information with each other using such a communication capability, compute a relative position with respect to the counterpart camera based on such information. As a result, the cameras101and102obtain relative positional relationship information representing a relative positional relationship with respect to each other. The posture information represents postures of the cameras101and102determined based on the orientation and the rotation amount (angle) around each of the X, Y, and Z axes. The positional information represents positions of the cameras101and102with respect to the reference position. The relative positional relationship information contains information representing a relative distance between the cameras101and102on the X-Y plane and a height difference between the cameras101and102in the Z-axis direction.

The roles of the main camera and the subsidiary camera are determined at the timing when either of the cameras101or102executes the manipulation for imaging. When a user executes the manipulation of imaging using the camera101, the camera101takes a role of the main camera, and the camera102takes a role of the subsidiary camera. A role of the main camera is to notify the subsidiary camera of the instructions regarding imaging, framing, or focusing. On the other hand, a role of the subsidiary camera is to execute framing, focusing, or imaging in response to the aforementioned notification from the main camera. Hereinafter, it is assumed that the camera101takes a role of the main camera, and the camera102takes a role of the subsidiary camera.

In addition, as shown inFIG. 1, in the case where each camera is installed in the camera platforms103and104, each camera notifies the camera platforms103and104of the operational signals. Here, the camera platforms103and104are electronic type camera platforms and are operated in response to the operational signal. The present invention is not related to the operation of the camera platform, and thus, further description of the camera platform will not be made.

The camera102obtains various kinds of information from the camera101and recognizes postures and a relative positional relationship between the two cameras to compute a movement amount (adjustment amount) required to execute framing in the camera102. The adjustment amount is displayed on the display unit of the camera102, and a user adjusts the posture of the camera102by manual manipulation. Alternatively, the camera platform104is notified of the adjustment amount and automatically adjusts the posture of the camera102. As a result, the cameras101and102are capable of executing framing so as to be directed toward the same subject105.

In addition, the camera101extracts characteristics of the image from the characteristic extraction area of the image captured by the camera101and transmits the characteristic data representing the characteristics thereof to the camera102. The camera102receives the characteristic data from the camera101, extracts the area corresponding to the characteristic extraction area of the image captured by the camera101from the image captured by the camera102, moves a focus point to that area, and executes focus adjustment.

The characteristic extraction area includes the focus point of the camera101. In addition, the area corresponding to the characteristic extraction area is an area which includes the characteristic similar to the characteristic indicated by the characteristic data in the image captured by the camera102. Therefore, when the camera102moves the focus point to the area corresponding to the characteristic extraction area and execute focus adjustment, it is possible to focus the cameras101and102on the same subject105.

FIG. 2illustrates the functions and the configuration of the camera101. The cameras101and102have the same functions and configuration.

Each of the camera101and102includes a ranging unit201, an imaging unit202, an image processing unit203, a focus adjustment unit204, a display unit205, a characteristic extraction unit206, a framing computation unit207, a movement amount detection unit208, a characteristic matching unit209, a posture detection unit210, a relative position computation unit211, an area determination unit212, a communication unit213, an input unit214, and a storage unit215.

The ranging unit201measures a distance from the subject. The imaging unit202images the subject and produces image data corresponding to the subject image formed on the imaging area. The image processing unit203processes the image data obtained by the imaging unit202. The focus adjustment unit204drives lenses depending on the distance from the subject obtained from the ranging unit201and focuses on the subject. Alternatively, the focus adjustment unit204drives the lenses to focally concentrate on the area obtained by the characteristic matching unit209and focuses on the subject.

The display unit205has a finder and displays the photograph on the finder such that a user can recognize the photograph resulting from a user capturing an image. In addition, the display unit205displays a framing instruction or a focus position on the finder for a user based on the information obtained using the framing computation unit207to prompt manipulation by a user. The characteristic extraction unit206extracts the characteristics of the image from the image data corresponding to the characteristic extraction area determined by the area determination unit212. The size of the characteristic extraction area changes depending on the focal length of the camera101or102.

The framing computation unit207computes the movement amount necessary for framing (adjustment amount around each of the X-axis, Y-axis, and Z-axis) based on the relative positional relationship information and the posture information obtained from the relative position computation unit211. The movement amount detection unit208measures the movement amount of itself from the reference position using an acceleration sensor or the like. The characteristic matching unit209extracts the area similar to the characteristics represented by the characteristic data from the image data processed by the image processing unit203using the characteristic data received from the other camera (in the following example, the camera101) through the communication unit213.

The posture detection unit210measures a direction (orientation) of itself or inclination information in a tri-axial direction using an azimuth sensor, a tri-axial sensor, or the like. The relative position computation unit211computes a relative positional relationship between itself and the other camera. Such a relative positional relationship is computed based on the posture information of itself and the movement amount from the reference position and the posture information of the other camera notified through the communication unit213. The area determination unit212determines the characteristic extraction area (characteristic area) for producing characteristic data. The camera101determines the characteristic extraction area (characteristic area) based on the focal length information of the camera101and the focal length information notified from the other camera (in the following example, the camera102) obtained through the communication unit213. In addition, the area determination unit212determines the area (detection target area) which is a target of the processing for detecting the area having the characteristic indicated by the characteristic data from the image data processed by the image processing unit203. The camera102determines the area (detection target area) which is a target of the processing for detecting the area having the characteristic indicated by the characteristic data from the image data processed by the image processing unit203based on the focal length information notified from the other camera (in the following example, the camera101) obtained through the communication unit213and the focal length information of the camera102.

The communication unit213transmits and/or receives the movement amount, the posture information, the focal length information, and the characteristic data, for example, through a short-distance wireless communication with other cameras. The input unit214has a release button or the like to receive a user's manipulation such as image capturing. The storage unit215stores the image data, the characteristic data, the movement amount of the camera, the relative positional relationship information, the posture information, and the like.

(Exemplary Method of Obtaining Relative Positional Relationship)

Next, a description will be made for a method of obtaining the relative positional relationship between the cameras101and102. As described above, the relative positional relationship includes a relative distance on the X-Y plane between the cameras101and102and a height difference in the Z-axis direction between the cameras101and102. First, a description will be made for a method of obtaining a relative distance between the cameras.

(First Example of Method of Obtaining Relative Distance)

Hereinafter, a first example of a method of obtaining the relative distance will be described.FIGS. 3A to 3Cillustrate a method of determining a reference position as a pre-process for obtaining the relative distance between the cameras. In addition, the body surfaces of the cameras101and102are provided with a portion capable of connecting with other cameras. When the cameras are overlapped with each other such that these portions are overlapped, the cameras are located in the reference position, and the relative distance is zero.

FIGS. 3A to 3Cillustrate a state that the cameras are overlapped with each other.FIG. 3Aillustrates a state that the lower faces of the cameras101and102are overlapped with each other.FIG. 3Billustrates a state that the side faces of the cameras101and102are overlapped with each other.FIG. 3Cillustrates a state that the rear faces of the cameras101and102are overlapped with each other. As shown inFIGS. 3A to 3C, how the cameras are overlapped is not particularly limited.

FIG. 4illustrates a method of measuring a relative distance when the cameras101and102are moved after the relative distance is obtained as shown inFIGS. 3A to 3C. When the cameras101and102are moved, the movement amount detection unit208of the cameras101or102measures the movement amount in each direction of the X-axis, Y-axis, and Z-axis from the reference position described inFIGS. 3A to 3C. The cameras101and102notify each other of the movement amount in each direction of the X-axis, Y-axis, and Z-axis from the reference position. In addition, in the camera102, the relative position computation unit211computes the relative distance401based on the movement amount thereof.

(Second Example of Obtaining Relative Distance)

Hereinafter, the second example of obtaining the relative distance will be described.FIG. 5illustrates a method of measuring a relative distance when the cameras101and102are separately arranged which is different from the method ofFIGS. 3 and 4.FIG. 5illustrates the state of the cameras101and102as seen in a top plan view.

First, similar toFIG. 5, the camera101is directed to the subject105, and the finder of the camera101recognizes the subject105. At this moment, it is assumed that the orientation502of the camera101is set to north. The camera102measures the relative distance from the camera101by using the position of the camera101as a reference position. For this purpose, a user orients the picture direction of the camera102toward the camera101and performs framing with the camera101as a subject. In this case, the orientation503of the camera101viewed from the camera102is east.

Subsequently, a user performs a ranging manipulation (AF manipulation) on the input unit214of the camera102to instruct measurement of a relative distance from the camera101. In the camera102, the ranging unit201measures the relative distance501from the camera101. As a result, the camera102can obtain the relative distance from the camera101.

(Exemplary Method of Obtaining Height Difference)

Hereinafter, an exemplary method of obtaining a height difference will be described.FIG. 6illustrates a method of measuring a height difference set between the cameras. The relative distance can be obtained as the relative positional relationship in the X-axis and Y-axis directions based on the method described in conjunction withFIGS. 4 and 5. However, in order to perform the framing after obtaining the relative distance, it is necessary to recognize the relative positional relationship in the Z-axis direction between the cameras.

As shown inFIG. 6, a user orients the picture direction of the camera102toward the camera101. In the camera102, the posture detection unit210measures the pitching angle A when the camera102is oriented toward the camera101with respect to the pitching angle at the initial state where the camera is oriented in the horizontal direction. The relative position computation unit211computes the height difference601from the camera101based on the obtained pitching angle A. Specifically, the relative position computation unit211computes the height difference601according to the formula (a relative distance between the cameras101and102)×sin A. As a result, the camera102can obtain a relative positional relationship with the camera101.

(Exemplary Method of Matching Framing)

Next, a description will be made for a method of matching framing between the cameras by computing the movement amount (adjustment amount) for framing based on the relative positional relationship obtained through the aforementioned method. While rotation with respect to three axes, the Z-axis (yawing angle), the Y-axis (pitching angle), and the X-axis (rolling angle), are necessary when the framing is performed by changing the camera direction, a method of computing the movement amount in each direction will be described hereinafter.

FIG. 7illustrates a method of computing the movement amount around the Z-axis for orienting the picture direction of the camera102toward the subject105in a similar way to that of the camera101. The finder of the camera101is directed to the subject105by a user's manipulation or the like and the finder of the camera101recognizes the subject105. The ranging unit201of the camera101measures the distance T from the subject105. In addition, the posture detection unit210of the camera101detects the direction (orientation) of the camera101. Based on this orientation, it is possible to obtain the angle C using a predetermined orientation (west inFIG. 7) as a reference. The camera101transmits such information to the camera102.

The camera102receives such information. In the camera102, the relative position computation unit211computes the angle B as a relative positional relationship based on the relative distance obtained through aforementioned method. If the angle C is set to 90°, it is possible to compute the angle B based on the formula B=tan−1T/L obtained from the formula tan B=T/L. In response to the computed angle B, the camera102displays the movement amount around the Z-axis on the finder of the display unit205to prompt a user to change the direction of the camera. Alternatively, the camera102notifies the camera platform104of the movement amount around the Z-axis and controls the camera platform104to direct the finder of the camera102toward the subject105. As a result, it is possible to orient the picture direction of the camera102toward the subject105.

FIG. 8illustrates a method of computing the movement amount around the Y-axis for directing the camera102toward the subject105. In the camera101, the posture detection unit210detects the angle D in the pitching angle direction by using, as a reference, the pitching angle of the initial state where the camera is oriented in the horizontal direction. The camera101transmits the information regarding the angle D to the camera102. The camera102receives the information regarding the angle D. In the camera102, the framing computation unit207computes the pitching angle by summing the angles A and D described above in conjunction withFIG. 6. The camera102displays an instruction for adjusting the pitching angle of the camera102to the summed pitching angle on the finder of the display unit205. Alternatively, the camera102notifies the camera platform104of the movement amount around the Y-axis.

FIG. 9illustrates a method of adjusting the inclination (angle around the X-axis) of the cameras101and102with respect to the horizontal direction. In the camera101, the posture detection unit210detects the inclination with respect to the horizontal direction and obtains the angle E. The camera101transmits the information regarding the angle E to the camera102. The camera102receives the information regarding the angle E and displays, and on the finder of the display unit205, the instruction for prompting a user to adjust the inclination. Alternatively, the camera102notifies the camera platform104of the movement amount around the X-axis. As a result, the camera102can rapidly perform framing for the subject105located in the picture direction of the camera101.

(Exemplary Method of Displaying Instruction of Framing)

Next, a description will be made for a method of displaying an instruction for framing to a user.FIGS. 10A and 10Billustrate a state that an instruction for the movement direction of the camera is displayed for a user on the finder of the display unit205when the framing is matched as described above.FIG. 10Aillustrates a state that the instruction is displayed on the finder of the display unit205of the camera102.FIG. 10Billustrates a state of the finder of the camera101.

The camera101matches a focus frame1001, which determines the focus position, with the subject. The camera102displays the movement amount (adjustment amount) around each axis, computed using the aforementioned method, on the finder. Information displayed thereon includes a directional movement amount indication index1003for directing the camera102in the direction (orientation) of the subject105described in conjunction withFIG. 7, an pitching angle movement amount indication index1004for indicating the pitching angle described in conjunction withFIG. 8, and a horizontal direction indication index1005for indicating the inclination with respect to the horizontal direction described in conjunction withFIG. 9. As means for displaying each index, how long the camera102should be moved to the target position is displayed, for example, on a scale.

In addition, the camera102displays the focus position as the focus frame1002. When a user manipulates the camera102according to the index displayed on the finder of the camera102, a plurality of cameras can be directed to the same subject.

(Method of Matching Framing Between Two Cameras)

Next, a description will be made for a specific example of a method of matching framing into the same subject.FIG. 11illustrates a state that the subject105is imaged by two cameras101and102from different angles. At this moment, live view images are displayed on the finders of the cameras101and102as shown inFIGS. 12A and 12B. Referring toFIG. 12A, the camera101recognizes the subject105from the front. In addition, referring toFIG. 12B, the camera102recognizes the subject105from the side.

Even when the picture direction of the camera102is adjusted with the camera101as a reference, the shot manipulation of the camera102is not limited. Therefore, as shown inFIGS. 13A and 13B, one of the cameras101and102may take a picture using a zoom-in manipulation, and the other may take a picture using a wide view manipulation. In this manner, for the same subject, framing may be performed freely, or a picture may be taken freely.

(First Example of Framing Instruction Operation)

Next, an operation for instructing framing will be described.FIG. 14illustrates a first example of the operation of the imaging system up until the framing instruction relating to the picture direction of the camera is performed.

First, the cameras101and102initialize the relative positional relationship between the cameras (step S1401). The initialization of the relative positional relationship is performed, after the relative distance between the cameras is set to zero as shown inFIGS. 3A to 3C, by assuming a case where a user moves the camera101or102as shown inFIG. 4(first case) and a case where the cameras101and102are arranged as shown inFIG. 5(second case). Hereinafter, a description will be provided for the first case.

Subsequently, the movement amount detection units208of the cameras101and102measure the movement amount of itself from the reference position (steps S1411and S1421). In addition, the posture detection units210of the camera101or102measure the posture of itself (steps1412and S1422). The posture information resulting from the aforementioned measurements includes orientations and rotation amounts (angles) around each of the X-axis, the Y-axis, and the Z-axis. The movement amounts and the posture information are stored in the storage unit215.

Subsequently, the ranging unit201of the camera101measures the distance from the subject based on the AF manipulation of the input unit214by a user (step S1413). Subsequently, the communication unit213of the camera101transmits information including the distance from the subject, the posture information, and the movement amounts to the camera102. The communication unit213of the camera102receives this information (step S1402).

The relative position computation unit211of the camera102reads the posture information and the movement amount of the camera102from the storage unit215and computes the relative positional relationship with the camera101based on this information and the information received from the camera101(step S1423). As a result, as described above in conjunction withFIGS. 4 and 6, the relative positional relationship including the relative distance from the camera101and the height difference from the camera101is computed.

Subsequently, the framing computation unit207of the camera102computes the movement amount (adjustment amount) necessary for framing (step S1424). As a result, as described above in conjunction withFIGS. 7 to 9, the movement amounts (rotation amounts) around three axes such as the Z-axis (yawing angle), the Y-axis (pitching angle), and the X-axis (rolling angle) are computed.

Subsequently, the finder of the display unit205of the camera102displays, for example, the movement amounts as described above in conjunction withFIGS. 10A and 10Band prompts a user to move the picture direction of the camera (step S1425). Alternatively, the camera102notifies the camera platform104of electronic signals to operate the camera platform104(step S1425).

In the second case described above, the framing is instructed in approximately the same sequence as that described above. However, in the second case, the sequence is initiated after the movement of the cameras101and102is completed. In addition, in step S1401, the relative distance from the camera101is measured using the ranging unit201of the camera102while the finder of the camera102is directed to the camera101.

FIG. 15illustrates operations of the cameras101and102up until the framing instruction relating to the picture direction of the camera is performed.FIG. 15corresponds toFIG. 14. Which one of the cameras101and102takes a role of a main (reference) camera or a subsidiary camera is not distinguished in the initial state.

First, the relative positional relationship between the cameras is initialized (step S1501), and the movement amount (step S1502) and the posture (step S1503) are measured. Then, a user selects roles of each camera at the timing of executing the ranging manipulation (AF manipulation) by pressing a release button or the like on either of the camera101or102. Each camera determines the selected roles (step S1504). In this case, since the roles are determined for the AF manipulation, each camera has a user input standby state. Alternatively, the roles of the cameras101and102may be initially determined.

The camera for which a user executes the ranging manipulation becomes the main camera (camera101) and the distance from the subject is measured (step S1506). Subsequently, the main camera transmits information including the movement amount, the posture information, and the distance from the subject to the subsidiary camera (camera102) (step S1507).

The subsidiary camera repeats role determination (step S1504) and monitoring on whether the information has been received from the main camera (step S1505) if notification of the information by the main camera is not made. When information including the movement amount, the posture information, and the distance from the subject is received from the main camera, the subsidiary camera computes the relative positional relationship with the main camera based on the information received from the main camera and the movement amount and the posture information measured by itself (step S1508).

Subsequently, the subsidiary camera computes the movement amount (adjustment amount) necessary for framing (step S1509). Further, the subsidiary camera displays, for example, the movement amount on the finder of the display unit205to prompt a user to move the picture direction of the camera as described above in conjunction withFIGS. 10A and 10B(step S1510). Alternatively, the subsidiary camera notifies the camera platform of electronic signals to operate the camera platform (step S1510).

The operation of executing a framing instruction by cooperation between the cameras may be terminated by an end manipulation from a user (step S1511). If there is not end manipulation by a user, the processes from the step S1504are repeated. Otherwise, if there is an end manipulation by a user, the aforementioned process is terminated.

Through the aforementioned operation, it is possible to adjust the picture direction of one of the cameras to the same subject using, as a reference, the picture direction of the other camera which receives the shot manipulation such as pressing the release button or the like.

(Second Example of Framing Instruction Operation)

FIG. 16illustrates a second operation example of the imaging system up until the framing instruction relating to the picture direction of the camera is performed. In the first operation example shown inFIGS. 14 and 15, roles of each camera are determined, and the picture direction of one of the cameras is adjusted with the other camera as a reference. On the other hand, in the second operation example, the movement amount or the posture information is exchanged in both directions without clearly determining roles of each camera, and the framing instruction is displayed on the finders of the display units205of both cameras.

First, the cameras101and102initialize the relative positional relationship therebetween (step S1601). Hereinafter, a case will be described where the relative distance between the cameras is set to zero as shown inFIGS. 3A to 3C, and then, a user moves the cameras101and102as shown inFIG. 4.

Subsequently, the movement amount detection units208of the cameras101and102measure the movement amounts of themselves from a reference position (step S1611and S1621). In addition, the posture detection units210of the cameras101and102measure the postures of themselves (step S1612and S1622). The posture information resulting from such measurement includes the rotation amounts (angles) around the X-axis, the Y-axis, and the Z-axis and orientation information. The movement amounts and the posture information are stored in the storage unit215.

Subsequently, the ranging units201of the cameras101and102measure the distance from the subject based on the AF manipulation of the input unit214by a user (step S1613and S1623). Subsequently, the communication unit213of the camera101transmits information including the movement amount, the posture information, and the distance from the subject to the camera102, and the communication unit213of the camera102receives this information (step S1614). On the other hand, the communication unit213of the camera102transmits the information including the movement amount, the posture information, and the distance from the subject to the camera101, and the communication unit213of the camera101receives this information (step S1624). In this manner, the cameras101and102notify each other of information.

Subsequently, the relative position computation units211of the cameras101and102read the movement amounts and the posture information of themselves from the storage unit215and compute the relative positional relationship with the other camera based on such information and the information received from the other camera (step S1615and S1625). As a result, as described above in conjunction withFIGS. 4 and 6, the relative positional relationship including the relative distance from the other camera and the height difference from the other camera is computed.

Subsequently, the framing computation units207of the cameras101and102compute the movement amount (adjustment amount) necessary for framing (step S1615and S1625). As a result, as described above in conjunction withFIGS. 7 to 9, the movement amounts (rotation amounts) around three axes such as the Z-axis (yawing angle), the Y-axis (pitching angle), and the X-axis direction (rolling angle) are computed.

Subsequently, the finders of the display units205of the cameras101and102display, for example, the movement amounts as described above in conjunction withFIGS. 10A and 10Bto prompt a user to move the picture direction of the camera (step S1616and S1626). Through the aforementioned operation, both the cameras101and102compute and display the relative positional relationship with the counterpart camera as a reference and the movement amount of itself for framing. While looking at one of the finders of the cameras101and102, a user can adjust the picture direction of the camera having that finder to the subject already disposed in the picture direction of the other camera.

The relative positional relationship between the cameras may be initialized in the state that the cameras101and102are disposed as shown inFIG. 5. In this case, the sequence is initiated after the movement of the cameras101and102is completed. In step S1601, the cameras101and102measure the relative distance from the counterpart camera using the ranging unit201while the picture direction of itself is directed to the counterpart camera.

FIG. 17illustrates the operations of the cameras101and102up until the framing instruction relating to the picture direction of the camera is performed.FIG. 17corresponds toFIG. 16. First, the relative positional relationship between the cameras is initialized (step S1701), and the movement amount (step S1702) and the posture (step S1703) are measured. Then, a user executes a ranging manipulation (AF manipulation) for each of the cameras101and102through manipulations such as pressing a release button or the like. As a result, the cameras101and102measure the distance from the subject (step S1704).

Subsequently, the cameras101and102transmit, to the counterpart camera, information including the movement amount, the posture information, and the distance from the subject (step S1705) and receive, from the counterpart camera, information including the movement amount, the posture information, and the distance from the subject (steps S1706). After receiving the information from the counterpart camera, the cameras101and102compute the relative positional relationship with the counterpart camera based on the movement amount and the posture information measured by themselves and the information received from the counterpart camera (step S1707).

Subsequently, the cameras101and102compute the movement amount (adjustment amount) necessary for framing (step S1708). The cameras101and102display, for example, the movement amount on the finder of the display unit205as described above in conjunction withFIGS. 10A and 10Bto prompt a user to move the picture direction of the camera (step S1709). As a result, it is possible to adjust the picture direction of one of the cameras to the same subject with the picture direction of the other camera as a reference.

In addition, the operation of executing the framing instruction in cooperation with each camera can be terminated by an end manipulation from a user (step S1710). If there is no end manipulation from a user, the process advances to step S1702, and the processes from the step S1702are repeated. Otherwise, if there is an end manipulation from a user, the aforementioned operation is terminated.

Through the aforementioned operation, it is possible to adjust the picture directions of each camera to the same subject at all times without considering which one of the cameras101and102is used as a reference.

Next, a method of adjusting focus of each camera into the same subject after adjusting the framing will be described.FIG. 18illustrates a method of adjusting focus when the focal lengths of the lenses of each camera are equal.

The camera101executes ranging (AF manipulation). In this case, as shown inFIG. 18, the camera101recognizes a person as a subject. The camera101extracts the characteristic data1801of the subject from the previously determined characteristic extraction area1811. The method of determining the characteristic extraction area will be described below. The characteristic data includes, for example, information that can be obtained from image data such as color, grayscale, or shape of the subject. InFIG. 18, for example, binary data of the image is used as the characteristic data.

The camera101transmits the extracted characteristic data1801and the size information of the characteristic extraction area1811to the camera102. After receiving the data, the camera102extracts the area1812having the characteristics similar to the characteristic data1801based on the image data produced by the camera102. As a result, the camera102moves the focus point to the extracted area1812and focuses on that area.

FIG. 19illustrates a method of adjusting focus when focal lengths of the lenses of each camera are different from each other. This example corresponds to the case where the focal length of the lens of the camera101is short (zoom-in) and the focal length of the lens of the camera102is long (zoom-out). In this case, if the coordinates of the focus points match between the cameras101and102on each finder image, there is a possibility that the focus may be made on the subject which is completely different.

In this case, the camera101determines an appropriate characteristic extraction area1911based on the focal length information of the cameras101and102. The characteristic extraction area1911is enlarged or reduced depending on the focal length of each camera. The camera101extracts the characteristic data1901of the characteristic extraction area1911and transmits the size information of the characteristic extraction area1911and the characteristic data1901to the camera102.

After the camera102received such information, the camera102extracts the area1912having similar characteristics to that of the characteristic data1901based on the image data created from the camera102, and moves the focus point into that area1912to focus thereon. As a result, even when the focus point is not centered on the finder image, such as when the focal length is different or in the case of a panorama shot, each camera can focus on the same subject.

FIG. 20illustrates an operation of determining the focus points and matching the focus points. The cameras101and102are directed to the same subject according to the aforementioned framing adjustment method (step S2001).

Subsequently, the communication units213of each camera101and102transmit, to the counterpart camera, the focal length information representing the focal length of the imaging unit202. In addition, the communication units213receive the focal length information from the counterpart camera. As a result, the cameras101and102notify each other of the current focal length of itself (steps S2002and S2003).

Here, the camera101may take a role of the main camera as described above. The area determination unit212of the camera101determines the characteristic extraction area based on the focal length information received from the camera102and the focal length information of the camera101(step S2004). As this determination method, for example, a method is selected where the size of the characteristic extraction area is adjusted by comparing the focal length of each camera. A difference between the focal lengths of the cameras101and102means the size of the subject located within the finders of each camera. In this regard, the characteristic extraction area is determined based on the ratio between the focal lengths of the cameras101and102, that is, the ratio of the size of the displayed subject.

In the case where the ratio of the subject displayed on the finder is not significantly different, the characteristic extraction area may be determined using the focus frame having a longer focal length as a reference. However, in the case where the ratio of the subject displayed on the finder is significantly different, it is not limited to the entire area indicated by the focus frame of the finder having a longer focal length being displayed on the finder having a shorter focal length. In this case, the area corresponding to the entire finder image having a shorter focal length is determined as the characteristic extraction area. In addition, the position of the characteristic extraction area is determined to be the position corresponding to the position of the focus point of the camera101.

Subsequently, the imaging unit202of the camera101images the subject to create image data, and the image processing unit203processes the image data. Further, the characteristic extraction unit206creates the characteristic data by extracting the characteristics from the data corresponding to the characteristic extraction area determined by the area determination unit212from the image data processed by the image processing unit203(step S2005). Subsequently, the communication unit213of the camera101transmits the characteristic data and the size information of the characteristic extraction area to the camera102, and the communication unit213of the camera102receives the characteristic data and the size information of the characteristic extraction area from the camera101(step S2006).

Subsequently, the area determination unit212of the camera102determines the area (detection target area) which is a target of the processing for detecting the area having the characteristics represented by the characteristic data based on the focal length information and the size information of the characteristic extraction areas of the cameras101and102. In addition, the imaging unit202images the subject to create image data, and the image processing unit203processes the image data. Further, the characteristic matching unit209detects a portion similar to the characteristics represented by the characteristic data received from the camera101from the data of the area determined by the area determination unit212of the image data processed by the image processing unit203using a template matching method or the like (step S2007).

If the focal length is different between the cameras, the size of the subject displayed on the finder of each camera is different. Therefore, in step S2007, the area determination unit212computes the ratio (magnification) of the subject displayed on the finder of each camera by obtaining the ratio of the focal length of each camera. Further, the size of the detection target area is determined by multiplying the size of the characteristic extraction area by the ratio (magnification) of the subject. In addition, in step S2007, the characteristic matching unit209detects a portion similar to the characteristics represented by the characteristic data while moving the position of the detection target area on the image indicated by the image data processed by the image processing unit203.

After detection of the similar portion as described above, the focus adjustment unit204moves the focus frame to the similar portion (step S2008) and executes focusing (matching the focus points) (step S2009). In addition, when the size of the focus frame is different in each camera, the size may be exchanged with each other by mutual communication.

FIG. 21illustrates the operation of the camera101(main or reference camera) during the focus adjustment. First, the framing is instructed as described above in conjunction withFIGS. 14 to 17(step S2101). Subsequently, the camera101transmits the focal length information to the camera102(step S2102) and receives the focal length information from the camera102(step S2103).

Subsequently, the camera101determines the characteristic extraction area based on the focal length information of the cameras101and102as described above in conjunction withFIG. 20(step S2104) and creates the characteristic data by extracting the characteristics from the image data of the characteristic extraction area (step S2105). Further, the camera101transmits the characteristic data to the camera102(step S2106).

FIG. 22illustrates the operation of the camera102(subsidiary camera) during the focus adjustment. First, the framing is instructed as described above in conjunction withFIGS. 14 to 17(step S2201). Here, a user manipulates the camera102to execute framing. Subsequently, the camera102receives the focal length information from the camera101(step S2202) and transmits the focal length information to the camera101(step S2203). Subsequently, the camera102receives the characteristic data and the size information of the characteristic extraction area from the camera101(step S2204).

Subsequently, the camera102determines whether or not the camera102is directed to the subject located in the picture direction of the camera101(step S2205). In the framing instruction sequence of step S2201, the camera102computes the movement amount (adjustment amount) necessary for framing In step S2205, the camera102compares the movement amount produced by executing a user's manipulation relating to the framing with the movement amount necessary for framing. In the case where both movement amounts are equal to each other, it is determined that the picture direction of the camera102is directed to the subject located in the picture direction of the camera101. Otherwise, in the case where both movement amounts are not equal to each other, it is determined that the camera102does not frame the subject framed by the camera101.

In the case where the camera102is not directed to the subject located in the picture direction of the camera101, the camera102displays an instruction for prompting a user to move the picture direction on the finder of the display unit205. A user executes the framing manipulation based on this indication (step S2206).

On the other hand, in the case where the camera102is directed to the subject located in the picture direction of the camera101, the camera102detects the area similar to the characteristics represented by the characteristic data from the image data created by the camera102based on the characteristic data received in step S2204(step S2207). If there is no area similar to the characteristics represented by the characteristic data, the camera102does not focus and terminates the process. On the other hand, if the area similar to the characteristics represented by the characteristic data is detected, the camera102moves the focus frame to that area (step S2208) and performs a focusing (step S2209).

Through the aforementioned process, even when the focus point is not centered on the finder image, it is not necessary for a user to manually determine the focus point. Each camera rapidly focuses on the same subject.

As described above, according to the present embodiment, using the image captured by any one of a plurality of cameras, the image corresponding to the image of the characteristic area captured by the other camera is detected and the focus of the subject can be adjusted based on the detection result. As a result, it is possible to simply and quickly focus a plurality of cameras.

In addition, since one of the cameras receives information representing a relative distance from the subject and posture information from the other camera, and the framing computation is executed based on such information and the posture information of itself, it is possible to compute the movement amount (adjustment amount) necessary for framing. Further, since a user is prompted to move the picture direction by the computed movement amount (adjustment amount) being displayed, or the shot posture is adjusted by controlling the camera platform based on the computed movement amount (adjustment amount), it is possible to simply and quickly adjust the framing between a plurality of cameras such that the picture directions of a plurality of cameras are directed at the same subject.

Hereinbefore, while embodiments of the present invention have been described with reference to the accompanying drawings, a detailed configuration is not particularly limited to the aforementioned embodiment, and various modifications or changes may be made without departing from the spirit of the present invention. For example, when a 3-D image or a panorama image of the subject is created, one of the cameras may execute the framing computation in order to capture an image which is to be used in cooperation with the image of the subject captured by the other camera.

According to the present invention, from the image of a first imaging area captured by one of a plurality of imaging apparatuses, an image corresponding to the image of the characteristic area captured by the other imaging apparatus is detected. Focus of the subject is adjusted based on the detection result thereof. As a result, it is possible to simply and quickly adjust focus between a plurality of imaging apparatuses.