Correlation extract method for generating 3D motion data, and motion capture system and method for easy composition of humanoid character on real background image using the same

Provided are a method for extracting a correlation to generate 3D motion data, and a motion capture system and method using the extracting method to easily compose a humanoid character on a real background image. Cameras for motion capture are installed in a real background capture location (such as an outdoor location or in a studio), and the correlation between motion capture fixed cameras and a moving camera for capturing a real background image is obtained to capture 3D motion data prior to performing motion capture for integrating a humanoid character, so that even when there is interaction between a 3D figure and an actual prop such as a chair or table within the captured space, a humanoid character can easily be composed with the real background.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a motion capture method for easily composing a humanoid character on a real background image, and more particularly, to a method that can accurately match a humanoid character to a real background image by installing cameras for motion capture on-site (outdoors or in a studio) to capture a real background image, and obtaining the correlation between fixed cameras and cameras for capturing the real background image prior to performing motion capturing of the humanoid character for composition, in order to capture 3D motion data.

The present invention is based on a portion of research performed in the IT New Growth Engine Core Technology Development Project, a collaborative effort of the Ministry of Information and Communication of the Republic of Korea and the Institute for Information Technology Advancement, entitled “Development of Realistic Digital Image Contents Production Software”, Filing No. 2005-S-003-02.

2. Description of the Related Art

Methods of generating motion data for humanoid characters in the related art include wearing infrared reflectors or emitters, ultrasound generators, magnetic sensors, fiber optics, etc. on regions of appendage joints, and reproducing 3D positions of sensors.

This method uses markers or sensors attached to joints to detect 3D positions of the joints, so that accurate capture of a character's motion data is possible; however, because the method is not linked to an integrated background image, a substantial amount of editing must be performed on the captured data in order to allow interaction by the character with a real object or another character. Thus, much time must be spent in the composition process.

Also, in a method of extracting background geometric data by tracking a camera against a background image, while there is a way to extract a 3D geometric structure of a prop in a real background, generation of motion capture data with respect to a human form is not possible.

Therefore, because motion capture and capture of a real background image is performed separately in the related art, it is difficult to integrate a character (to which motion capture data is applied) with a real background. Especially when there is interaction with a prop or a third party, much manual work is required, thus consuming much time for composition.

SUMMARY OF THE INVENTION

Accordingly, the present invention is directed to a correlation extract method for generating 3D motion data, and a motion capture system and method for easy composition of a humanoid character on a real background image using the correlation extract method, which substantially obviate one or more problems due to limitations and disadvantages of the related art.

It is an object of the present invention to provide a method of generating 3D humanoid character motion data, capable of automatically composing a real subject within a real captured background image.

To achieve these objects and other advantages and in accordance with the purpose of the invention, as embodied and broadly described herein, there is provided a motion capture system for easy composition of a humanoid character on a real background image, the motion capture system including a calibration frame for calibrating positions of a fixed camera for motion capture and a moving camera for capturing a real background, a fixed camera for capturing the calibration frame and a background marker or a corner point in a real background, and movement of a moving subject, a moving camera for moving and capturing the real background, and a motion capture processor for generating 3D motion data that extracts and automatically integrates a geometric correlation of the fixed camera and the moving camera with the real background.

The motion capture processor may include a correlation extractor for determining 3D geometric relative positions of the fixed camera and the background marker or the corner point, and extracting a geometric correlation of the fixed camera and the moving camera, and a 3D motion data generator for generating 3D motion data that automatically integrates the captured movement of the moving subject with the real background, according to the extracted geometric correlation.

In another aspect of the present invention, there is provided a method of correlation extraction for generating 3D (three-dimensional) motion data, the method including the steps of obtaining a geometric relation between a calibration frame and a background marker or a corner point in a real background, obtaining 3D geometric relative positions of a camera for motion capture and the background marker or the corner point, from the obtained geometric relation through the calibration frame, and extracting a correlation between the camera for motion capture and a moving camera for capturing a real background, through obtaining 3D geometric relative positions of the camera for motion capture and the camera for capturing a real background, from the obtained 3D geometric relative positions of the camera for motion capture and the background marker or the corner point.

In a further aspect of the present invention, a method of motion capture using a fixed camera for motion capture and a moving camera for capturing a real background, the method for easily integrating a humanoid character with a real background image, and including the steps of extracting a geometric correlation of the fixed camera and the moving camera, and using the extracted correlation to capture motion of a real moving subject and generate 3D motion data automatically integrating the captured motion with the real background.

The step of extracting the geometric correlation may include the steps of obtaining a geometric relation between the fixed camera and a calibration frame, obtaining a geometric relation between the calibration frame and a background marker captured by the moving camera or a corner point in the real background, obtaining 3D geometric relative positions of the fixed camera and the background marker or the corner point, using the obtained geometric relation through the calibration frame, and obtaining 3D geometric relative positions of the fixed camera and the moving camera from the obtained 3D geometric relative positions of the fixed camera and the background marker or the corner point, and extracting a correlation of the fixed camera and the moving camera.

DETAILED DESCRIPTION OF THE INVENTION

A detailed description of a motion capture method according to the present invention will be given below with reference toFIGS. 1 and 2.

FIG. 1illustrates a configurative block diagram of a motion capture apparatus that generates 3D motion data for easily composing a humanoid character with a real background image, according to the present invention.

Referring toFIG. 1, a motion capture processor100according to the present invention extracts a geometric correlation between a fixed camera103(for motion capture that captures a calibration frame and a background marker or a corner pointer in a background, and captures motion of a moving subject) and a moving camera101(that captures a real background while moving), and generates 3D motion data that automatically composes the real subject and background.

This motion capture processor100includes a correlation extractor110that extracts geometric correlations between the fixed camera103for motion capture and a moving camera101for capturing a real background after determining mutual 3D geometric positions of a fixed camera103for motion capture and a background marker or a corner point in a background, and a 3D motion data generator120that captures the motion of a real moving subject according to the extracted correlations and generates 3D motion data to compose the subject with the real background.

The calibration frame is for calibrating the positions of the fixed camera103for motion capture and the moving camera101for capturing a real background.

FIG. 2illustrates a flowchart a motion capture method that generates 3D motion data using correlations between a fixed camera103and a moving camera101, according to the present invention.

The present invention can largely be divided into two stages: a preparatory stage and an actual operating stage. First, in the preparatory stage, the correlation between the fixed camera103for motion capture and the moving camera101for capturing a real background is extracted and saved.

Specifically, the system and program are activated in step S210, the moving camera101is set to simultaneously observe the calibration frame102and the background marker104or the background corner point, and a plurality of fixed cameras103are installed on tripods or fixed by being inserted into props in step S211.

Thus, the calibration frame102is used to respectively calibrate (calculate respective locations of) the fixed cameras103for motion capture and the moving camera101for capturing the real background, and determine in step S211the 3D geometric relative positions of the moving camera101for capturing the real background and the fixed cameras103for motion capture.

The two calibrated fixed cameras103are used to calculate the geometric position of a background marker104(or background corner point) through triangulation, in order to determine the 3D geometric relative positions of the moving camera101, the fixed cameras103, and the background marker104or background corner point with respect to the calibration frame, in step S211.

By determining the mutual locations of the fixed cameras103for motion capture and the moving camera101for capturing the real background, with respect to the calibration frame, the mutual positions (correlation) of the fixed cameras103and the moving camera101may be determined in step S212.

As described, after the correlation between the fixed cameras and the moving camera is extracted and saved in the preparatory stage, the actual operating stage is performed in steps S213and S214.

The fixed cameras perform motion capture of an actual moving subject in step S213, and the extracted correlation is applied to the actual background image captured by the moving camera, to generate accurately composed 3D motion data in step S214.

Thereafter, the actual operating stage that is performed after the preparatory stage of the above motion capture method will be described in detail with reference toFIGS. 3 and 4.

FIG. 3illustrates examples of extracting correlations between fixed cameras for motion capturing and moving cameras for capturing real background images, for the motion capture inFIG. 2.

FIG. 3(a) illustrates the extraction of an initial correlation between fixed cameras for motion capture and a moving camera for capturing a real background image, and is configured with a moving camera101for capturing an actual background image, a calibration frame102, fixed cameras103for motion capture, and a background marker104, with the fixed cameras103installed on tripods or fixed by inserting in props.

The fixed cameras103for motion capture are thus installed in a real background (outdoors or in a studio), and correlation of the fixed cameras103and the moving camera101is obtained.

The initial correlation between the fixed cameras103for motion capture and the moving camera101for capturing a real background image is first obtained using the calibration frame102as a medium for a pre-calculation of the respective positions of the fixed cameras.

That is, by obtaining the geometric relation of the calibration frame102to the real background data through the moving camera101, the initial correlation between the fixed cameras103and the moving camera101can be obtained.

Here, the moving camera101,201, and301is positioned to simultaneously observe the calibration frame102for calibrating the fixed cameras103and the marker installed in the background or the corner point in the background.

FIG. 3(b) illustrates extracting a correlation between the fixed cameras103and the moving camera201at another position, andFIG. 3(c) illustrates extracting a correlation between the fixed cameras103and the moving camera301at yet another position.

Referring toFIGS. 3(b) and3(c), through compositions of a background marker104(or a background corner point) captured by the moving camera201and301at other positions, the other positions of the moving camera201and301can be calculated, and simultaneously, the correlation between the moving camera201and301and the fixed cameras103can be extracted through the respective positions of the background marker104and the fixed cameras103.

Accordingly, through extraction of the correlation between the positions of the moving camera101,201, and301and the fixed cameras103inFIGS. 3(a) to3(c), the 3D respective position correlation between the fixed cameras103for motion capture and the background marker104or the corner point in the background is obtained.

As described above, by extracting the correlation of the fixed cameras103and the moving camera101,201, and301beforehand, even when the calibration frame is deleted from the capture area during actual motion capture, the background marker or background corner point can be used to extract a correlation between the fixed cameras for motion capture and the moving camera for capturing the real background image.

According to the present invention, the background marker104or the background corner point used are not limited to markers or corner points, and may be embodied in all forms of single points, lines, surfaces, etc. that are helpful in extracting 3D geometric data of a background.

FIG. 4illustrates examples of generating 3D motion data capable of being automatically composed with a real background image according to the motion capture method inFIG. 2, where (a) illustrates a first position of the moving camera for capturing a real background image, (b) illustrates a second position of the moving camera, and (c) illustrates a third position of the moving camera.

First, the fixed cameras and moving camera are installed as inFIGS. 3(a) to3(c), and the correlation between the fixed cameras and moving camera is extracted for each position of the moving camera.

Then, when motion capture is performed on a moving subject to obtain humanoid character motion data, the 3D motion capture data generated from the real motion capture is automatically generated as 3D data linked to the moving camera capturing the real background image.

That is, motion data, generated as interaction with a third party or actual object such as a chair, table, etc. within the captured space, is generated as 3D motion data that can accurately be composed with data from the moving camera for capturing the real background image, so that the real background image and a humanoid character can easily be composed.

Referring toFIG. 4,FIG. 4(a) illustrates a first position of the moving camera401capturing the real background including the background marker104. Similarly,FIG. 4(b) illustrates a second position of the moving camera501capturing the real background including the background marker104, andFIG. 4(c) illustrates a third position of the moving camera601capturing the real background including the background marker104.

Next, the fixed cameras103capture the moving subject at each point to extract the subject's movement.

Thus, 3D motion data can be generated that can automatically integrate a humanoid character (based on a real moving subject) with a real background for each position captured by a moving camera401to601, extracted from the correlation between the moving camera401to601and fixed cameras103, which is disclosed in Korean Patent Registration No. 0507780, entitled “Apparatus and Method for High-speed Marker-free Motion Capture”, which is hereby incorporated by reference, and is thus omitted herefrom.

As described above, the motion capture apparatus and method according to the present invention use the correlation between motion capture fixed cameras and a moving camera for capturing a real background image to generate 3D motion data that can be precisely composed with the real background image, so that even when there is interaction between a 3D figure and an object such as a chair or table within the captured space, a humanoid character can easily be composed with the real background.