Abstract:
Provided is an apparatus for precisely correcting position and attitude information of a camera by analyzing image information received through a three line scanner in operation of the camera obtaining the position and attitude information from a Global Navigation Satellite System (GNSS) and an Inertial Navigation System (INS), and a method thereof. The method for correcting position and attitude information of a camera includes the steps of: a) calculating the position of the camera by using the GNSS; b) calculating the attitude of the camera by using the INS; c) generating the position and attitude correction information of the camera by analyzing an image received through a three line scanner mounted in the camera; and d) receiving a feedback of the position and attitude correction information in the GNSS and the INS.

Description:
FIELD OF THE INVENTION  
       [0001]     The present invention relates to an apparatus for correcting position and attitude information of a camera, and a method thereof; and, more particularly, to an apparatus for precisely correcting the position and attitude information of a camera by analyzing image information received through a three line scanner, which is mounted on the camera, in operation of the camera obtaining the position and attitude information from a Global Navigation Satellite System (GNSS) and an Inertial Navigation System (INS), and a method thereof.  
       DESCRIPTION OF RELATED ART  
       [0002]     There are three methods to determine a position and an attitude of an observation apparatus, e.g., an artificial satellite, an aircraft and a vehicle, or a camera which is loaded in the observation apparatus. One method is to decide the position and attitude information of the observation apparatus by combining information coming from a Global Navigation Satellite System (GNSS) and an Inertial Navigation System (INS). Another method is to decide the position and the attitude of the observation apparatus by using a Global Positioning System (GPS), which is a kind of the GNSS, in a situation that the INS is not available, and analyzing the signal coming from a plurality of GPS satellites. The other method is to generate three-dimensional (3D) information of an object by analyzing an image received through a stereoscopic camera mounted on the GNSS and the INS.  
         [0003]     Korean Patent Publication No. 1997-41019 entitled “Hybrid Navigation System” discloses a technology for determining a position and an attitude of a vehicle by combining the GNSS and the INS and indicating a present position of the vehicle through map matching of the result.  
         [0004]     Korean Patent Publication No. 2000-85817 entitled “System for Determining Position and Attitude in Mobile Communication System Using GPS and INS” discloses a technology for determining a position and an attitude in a mobile communication by combining information received from a GPS sensor, an INS sensor and a dead-reckoning (DR) sensor.  
         [0005]     Korean Patent Publication No. 2001-41187 entitled “Apparatus for Detecting 3D Position and Method thereof” discloses a technology for acquiring a 3D position and attribute information of an object by using a stereo image obtained through a digital video camera set up in a vehicle employing the GNSS and the INS.  
         [0006]     Korean Patent Publication No. 2002-62417 entitled “Space Image Information System for Supporting Effective Save and Retrieval of Space Image” discloses a technology for generating information by applying internal and external orientation parameters to an acquired image by using the GNSS, the INS and a CCD camera.  
         [0007]     U.S. Publication No. 2002/0004691 entitled “Attitude Determination and Alignment Using Electro-optical Sensors and Global Navigation Satellites” discloses a technology for predicting and inspecting a position and an attitude of a satellite by analyzing a signal coming from a plurality of GPS satellites and information received from a star sensor.  
         [0008]     U.S. Pat. No. 6,580,389 entitled “Attitude Determination Using A Global Positioning System” discloses a technology for determining a position and an attitude of a vehicle by analyzing a signal coming from a plurality of GPS satellites.  
         [0009]     However, the conventional technologies determine a position and an attitude of an observation apparatus by using information received from only the GNSS, or simultaneously using the GNSS and the INS. Therefore, the conventional technologies have a characteristic that precision of position information depends on precision of information acquired from the GNSS, and precision of attitude information depends on precision of information acquired from the INS.  
         [0010]     Therefore, the conventional technologies have a problem that position information cannot be acquired if the time interval for obtaining position information is shorter than a pre-determined time interval, which is limited by the GNSS. For example, since a GPS signal is generally provided at one second interval, the GPS can not calculate precise position information if the time interval is shorter than one second.  
         [0011]     The conventional technologies also have a problem that the attitude information cannot be acquired if the time interval for acquiring the attitude information is shorter than a time interval which is limited by the INS.  
         [0012]     The conventional technologies also have a problem that the precision of the position and the attitude is controlled by the performance of a related system, and additional information in the image information received from the camera, which is an observation apparatus connected to the GNSS and the INS, is not applied.  
       SUMMARY OF THE INVENTION  
       [0013]     It is, therefore, an object of the present invention to provide an apparatus for precisely determining position and attitude information of a camera based on a scan line image information received through a three line scanner of the camera mounting a Global Navigation Satellite System (GNSS) and an Inertial Navigation System (INS), and a method thereof.  
         [0014]     Other objects and advantages of the invention will be understood by the following description and become more apparent from the embodiments of the present invention, which are set forth hereinafter. It will be also apparent that objects and advantages of the invention can be embodied easily by the means defined in claims and combinations thereof.  
         [0015]     In accordance with an aspect of the present invention, there is provided a method for correcting position and attitude information of a camera, including the steps of: a) calculating the position of the camera by using a Global Navigation Satellite System (GNSS); b) calculating the attitude of the camera by using an Inertial Navigation System (INS); c) generating the position and attitude correction information of the camera by analyzing an image received through a three line scanner mounted in the camera; and d) receiving a feedback of the position and attitude correction information in the GNSS and the INS.  
         [0016]     In accordance with another aspect of the present invention, there is provided a position and attitude information correction apparatus of a camera that the GNSS and the INS are connected to generates the position and attitude correction information of the camera by analyzing the image received through a three line scanner equipped in the camera, and improves the precision of the position and attitude information of the camera by receiving a feedback of the position and attitude correction information in the GNSS and the INS.  
         [0017]     The position and attitude information correction apparatus of a camera connected to the GNSS and the INS generates the position and attitude correction information of the camera by analyzing the image received through a three line scanner in the camera, and improves the precision of the position and attitude information of the camera by receiving a feedback of the generated position and attitude correction information in the GNSS and the INS.  
         [0018]     Other objects and advantages of the present invention will become apparent from the following description of the embodiments with reference to the accompanying drawings. Therefore, those skilled in the art that the present invention is included can embody the technological concept and scope of the invention easily. In addition, if it is considered that detailed description on the prior art may blur the point of the present invention, the detailed description will not be provided herein. The preferred embodiments of the present invention will be described in detail hereinafter with reference to the attached drawings. 
     
    
     BRIEF DESCRIPTION OF THE DRAWINGS  
       [0019]     The above and other objects and features of the present invention will become apparent from the following description of the preferred embodiments given in conjunction with the accompanying drawings, in which:  
         [0020]      FIG. 1  is a block diagram showing a system that an image-based position and attitude correction apparatus in accordance with an embodiment of the present invention is applied to;  
         [0021]      FIG. 2  is a block diagram showing a relationship between images acquired through a three line scanner in accordance with an embodiment of the present invention;  
         [0022]      FIG. 3  is a block diagram illustrating an image-based position and attitude correction apparatus in accordance with an embodiment of the present invention; and  
         [0023]      FIG. 4  is a block diagram illustrating a Kalman filter processing block in accordance with an embodiment of the present invention. 
     
    
     DETAILED DESCRIPTION OF THE INVENTION  
       [0024]     Other objects and aspects of the invention will become apparent from the following description of the embodiments with reference to the accompanying drawings, which is set forth hereinafter.  
         [0025]      FIG. 1  is a block diagram showing a system which an image-based position and attitude correction apparatus in accordance with an embodiment of the present invention is applied to.  
         [0026]     A camera  100  equipped with a three line scanner is mounted on an observation apparatus such as a satellite, an aircraft and a vehicle. Therefore, a position and attitude information of the camera  100  is directly connected with the position and attitude information of the observation apparatus.  
         [0027]     The three line scanner includes a nadir scanner  120  toward a right angle direction from a propagation direction of the observation apparatus, a forward scanner  110  toward the direction of propagation, and a backward scanner  130  toward an direction opposite to the propagation direction. Herein, an angle between the nadir scanner  120  and the forward scanner  110 , and the angle between the nadir scanner  120  and the backward scanner  130  are less than 90 degrees.  
         [0028]     A Global Navigation Satellite System (GNSS)  200  connected to the camera  100  determines a position of the camera  100 . A GPS receiver can be used as the GNSS  200 . An Inertial Navigation System (INS)  300  connected to the camera  100  determines the attitude of the camera  100 .  
         [0029]     The GNSS and the INS are set up firmly and physically in the same frame as the camera  100 . Since the common GNSS and INS can be used, detailed description on the structure will not be provided.  
         [0030]     An image-based position and attitude correction apparatus  400  generates position and attitude correction information of the camera  100  based on a scan line image received through the three line scanner of the camera  100 . The generated position and attitude correction information separately receives a feedback in the GNSS and the INS, and the position and attitude information of the camera  100  can be grasped more precisely.  
         [0031]      FIG. 2  is a block diagram showing a relationship between images acquired through a three line scanner in accordance with an embodiment of the present invention.  
         [0032]     The three line scanner includes the forward scanner  110 , the nadir scanner  120 , and the backward scanner  130  according to the propagation direction of the observation apparatus. Therefore, the scan line image includes a forward scanner observation image  510  generated through the forward scanner  110 , a nadir scanner observation image  520  generated through the nadir scanner  120 , and a backward scanner observation image  530  generated through the backward scanner  130 .  
         [0033]     Since the three scanners are arrayed based on the propagation direction of the observation apparatus, a ground surface or a facility, which is an object of observation, is observed by the forward scanner  110 . After a predetermined time Δt 1  passes, the observation object is observed by the nadir scanner  120 , and after a predetermined time Δt 2  passes, the observation object is observed by the backward scanner  130 .  
         [0034]     When the observation apparatus is in a uniform straight-line motion and an inertia motion without changing the attitude, observation time intervals among the forward scanner  110 , the nadir scanner  120  and the backward scanner  130  are the same, i.e., Δt 1 =Δt 2 , and a position deviation between images, which are observed through three scanners  110 ,  120 , and  130 , does not appear. That is, a forward scanner observation image  510 , a nadir scanner observation image  520 , and a backward scanner observation image  530  are identified.  
         [0035]     Meanwhile, when the observation apparatus is not in the uniform straight-line motion and the inertia motion, the position deviation appears among the images  510 ,  520  and  530  observed through the three scanners, just as in  FIG. 2 . The position deviation between the forward scanner observation image  510  and a nadir scanner observation image  520  is (1, 1) in a pixel unit, and the image position deviation between a nadir scanner observation image  520  and the backward scanner observation image  530  is (1, 2) in the pixel unit. An image-based position and attitude correction apparatus  400  of the present invention can precisely correct the position and attitude information of the camera by analyzing the image position deviation.  
         [0036]      FIG. 3  is a block diagram illustrating the image-based position and attitude correction apparatus  400  in accordance with an embodiment of the present invention.  
         [0037]     If the observation apparatus employing the camera  100  equipped with the three line scanner progresses forward, the forward scanner observation image  510 , the nadir scanner observation image  520 , and the backward scanner observation image  530  are generated in order by the three line camera. The generated observation images  510 ,  520  and  530  are inputted to the image-based position and attitude correction apparatus  400 .  
         [0038]     The forward scanner observation image  510  is stored in a first buffer memory  410 . Subsequently, an image matching block  430  performs an image matching on the temporarily stored forward scanner observation image  510  and the nadir scanner observation image  520 , which is inputted by real-time observation through the nadir scanner  120 , thereby calculating image position deviations Δu 1  and Δv 1  between two images  510  and  520  in the pixel unit. The image matching can be performed in well-known diverse methods including an area-based matching method and a feature-based matching method.  
         [0039]     The same procedure can be applied to an image observed through the nadir scanner  120  and the backward scanner  130 . That is, the nadir scanner observation image  520  is temporarily stored in a second buffer memory  420 . Subsequently, the image matching block  430  performs the image matching on the temporarily stored nadir scanner observation image  520  and the backward scanner observation image  530  which is inputted by real-time observation through the backward scanner  130 , thereby calculating image position deviations ≢u 2  and Δv 2  between the images  520  and  530  in a pixel unit. The image matching can be performed in well-known diverse methods such as an area-based matching method and a feature-based matching method.  
         [0040]     First image position deviation information Δu 1 , Δv 1 , t f  and t n  between the forward scanner observation image  510  and the nadir scanner observation image  520 , and second image position deviation information Δu 2 , Δv 2 , t n  and t b  between the nadir scanner observation image  520  and the backward scanner observation image  530  are inputted to a Kalman filter processing block  440 .  
         [0041]     The first image position deviation information Δu 1 , Δv 1 , t f  and t n  include image position deviations Δu 1  and Δv 1  between the forward scanner observation image  510  and the nadir scanner observation image  520 , a time t f  that the forward scanner observation image  510  is generated, and a time t n  that the nadir scanner observation image  520  is generated.  
         [0042]     The second image position deviation information Δu 2 , Δv 2 , t n  and t b  include image position deviations Δu 2  and Δv 2  between the nadir scanner observation image  520  and the backward scanner observation image  530 , a time t n  that the nadir scanner observation image  520  is generated, and a time t b  that the backward scanner observation image  530  is generated.  
         [0043]     The Kalman filter processing block  440  presumes and generates the position and attitude correction information x′, y′, z′, ω′, κ′, φ′ and t m  from the first and second image position deviation information, camera position information x, y, z and t n  of time t n , which is inputted from the GNSS  200 , and the camera position information ω, κ, φ and t n  of time t n , which is inputted from the INS  300 .  
         [0044]     The position and attitude correction information has the camera position and attitude change which is sectionally calculated with respect to each time interval that the image is generated. Herein, time t m  is any time existing between a time t f  that the forward scanner observation scanner the image  510  is generated, and a time t b  that the backward scanner observation image  530  is generated. The time interval that the image is generated can be shorter than the time interval limited by the common GNSS or INS.  
         [0045]      FIG. 4  is a block diagram illustrating a Kalman filter processing block  440  in accordance with an embodiment of the present invention. A projection transforming block  442  performs a projection transform applying  7  parameters x, y, z, ω, κ, φ and t n  defined by the camera position information x, y, z and t n , and the camera attitude information ω, κ, φ and t n  to 7 parameters Δu 1 , Δv 1 , Δu 2 , Δv 2 , t f , t n  and t b  defined by the first and second position information.  
         [0046]     Parameters δx, δy and δz showing a real world position deviation are calculated by applying the performance result of the projection transform to parameters x, y and z showing the camera position information.  
         [0047]     Subsequently, a Kalman filter  444  calculates parameters Δx, Δy, Δz, Δω, Δω and Δφ showing the estimated position and attitude deviation, which is optimized from parameters x, y, z, ω, κ, φ and t n  showing the position and attitude information, and parameters δx, δy and δz showing the real world position deviation.  
         [0048]     A deviation correction block  446  corrects a deviation of parameters x, y, z, ω, κ, φ and t n  showing the position and attitude information by using parameters Δx, Δy, Δz, Δω, Δκ and Δφ showing the estimated position and attitude deviation, thereby outputting the position and attitude correction information x′, y′, z′, ω′, κ′, φ′ and t m , which are optimized with respect to any time t m .  
         [0049]     The estimated position and attitude correction information x′, y′, z′, ω′, κ′, φ′ and t m , which are optimized with respect to any time t m  existing between t f  and t b , are divided into the position correction information x′, y′, z′ and t m  and the attitude correction information ω′, κ′, φ and t m . The GNSS  200  and INS  300  receives feedback separately for the divided position correction information and attitude correction information to precisely correct the position and attitude information of the camera.  
         [0050]     The technology for improving the precision of the position and attitude information by receiving a feedback of the correction information received through the analysis of the three line scanner image can be proceeded by an onboard processing approach method and a post processing approach method.  
         [0051]     The onboard processing approach method observes the image through the three line scanner in a satellite, an aircraft or a vehicle, improves the precision of the position and attitude information by receiving the feedback of the position and attitude correction information in the GNSS and the INS, and directly records the corrected position and attitude information in the image received through the three line scanner.  
         [0052]     The post processing approach method records the original position and the attitude information received from the GNSS and the INS, and the image information received through the three line scanner, and improves the precision of the position and attitude information through a separate process later.  
         [0053]     The present invention can be embodied as a program and stored in a computer-readable recording medium, such as CD-ROM, RAM, ROM, a floppy disk, a hard disk and a magneto-optical disk.  
         [0054]     The present invention can precisely determine external orientation parameters of the camera without using a ground control point with respect to the image observed through the camera, which is mounted in a transporting means such as a satellite, an aircraft and a vehicle.  
         [0055]     The present invention can also precisely determine the position and attitude information of the camera even when a time interval is short.  
         [0056]     The present application contains subject matter related to Korean patent application Nos. 2004-0109575 and 2005-0037882, filed with the Korean Intellectual Property Office on Dec. 21, 2004, and May 6, 2005, respectively the entire contents of which is incorporated herein by reference.  
         [0057]     While the present invention has been described with respect to certain preferred embodiments, it will be apparent to those skilled in the art that various changes and modifications may be made without departing from the scope of the invention as defined in the following claims.