Method for feature recognition in mobile communication terminal

A portable terminal is configured to perform a method for feature recognition. The portable terminal includes a Global Positioning System (GPS), a camera, a sensor, an image geographic information part, an image processor, and a controller. The GPS provides a position of the portable terminal. The camera photographs an image of a feature. The sensor detects a direction of the camera. The image geographic information part stores an image of each feature and geographic information on each feature. The image processor extracts a characteristic of the image of the feature and compares the extracted characteristic with image information stored in the image geographic info illation part. If recognizing a specific feature through the image processor, the controller proportionally compares geographic information of the image geographic information part with size and direction data on a peripheral feature near the specific feature within an image, and recognizes the peripheral feature.

The present application is related to and claims priority under 35 U.S.C. §119(a) to a Korean Patent Application filed in the Korean Intellectual Property Office on Nov. 27, 2008 and assigned Serial No. 10-2008-0119178, the contents of which are herein incorporated by reference.

TECHNICAL FIELD OF THE INVENTION

The present invention relates to a method for providing geographic information using a camera and a Global Positioning System (GPS) in a mobile communication terminal. More particularly, the present invention relates to a method for recognizing a feature using a camera of a mobile communication terminal and, when displaying information on the recognized feature in a real image, providing information on a feature around the recognized feature using a position of the recognized feature and direction information in which a user looks.

BACKGROUND OF THE INVENTION

GPS is the abbreviation of a global positioning system developed by the Pentagon. The GPS includes a plurality of GPS satellites and a GPS receiver. The plurality of GPS satellites are arranged by four in each of six circular orbits of an inclination angle of 55°. The plurality of GPS satellites transmits their own position information signals while moving in a cycle of about 12 hours. The GPS receiver receives position information signals from at least four or more GPS satellites among the plurality of GPS satellites, detects its distances from the GPS satellites and position vectors of the GPS satellites, and calculates its own position vector. A navigation system including the GPS receiver maps its own position vector calculated from the GPS receiver on map data, and displays the mapped position vector. The navigation system provides, as a supplementary service, a road guidance service for informing the shortest path from a current position on a map to a user's destination. Therefore, a user can more easily find a way to her own destination by carrying the navigation system.

FIG. 1illustrates a block diagram of a construction of a portable terminal for recognizing an image using a camera according to the conventional art. As illustrated inFIG. 1, a navigation system is integrated with a camera104. The navigation system can include a command input part112, a GPS receiver100, a map data storage part108, a sensor110, an image processor106, and a controller102. The command input part112receives an input of a user's operation command. The GPS receiver100receives a position information signal transmitted by a GPS satellite. The map data storage part108stores map data. The sensor110detects a direction angle of the camera104and an inclination angle through a geomagnetic sensor and an angle meter. The image processor106processes an image captured by the camera104. The controller102captures an image photographed by the camera104at a predetermined time interval, searches the map data storage part108for buildings of position information consistent with respective buildings within the captured image, and matches them with each other.

However, the conventional art has a disadvantage of consuming much time because having to display geographic information on several features. In this case, the portable terminal has to recognize one feature and display geographic information and, after that, again recognize a next feature and display geographic information. In theory, assuming that number of features required to display geographic information is equal to ‘n’ and a time taken to recognize one feature is equal to ‘t’, a time taken to display geographic information on the total features is equal to ‘n*t’. However, because the ‘t’ can be a long time, a user has to spend much time to obtain interested information.

SUMMARY OF THE INVENTION

To address the above-discussed deficiencies of the prior art, it is a primary object to provide at least the advantages below. Accordingly, one aspect of the present invention is to recognize a user's current position using a Global Positioning System (GPS), recognize a specific feature using a camera, and provide geographic information around the specific feature using position information on the recognized feature and direction information of the camera input from a sensor. In this case, upon recognition of only an initial feature, geographic information on the feature can be easily displayed using previously stored data.

The above aspects are achieved by providing a method for feature recognition in a mobile communication terminal.

According to one aspect of the present invention, a portable terminal is provided. The portable terminal includes a GPS, a camera, a sensor, an image geographic information part, an image processor, and a controller. The GPS provides a position of the portable terminal. The camera photographs an image of a feature. The sensor detects a direction of the camera. The image geographic information part has an image of each feature and geographic information on each feature. The image processor extracts a characteristic of the image of the feature obtained in the camera and compares the extracted characteristic with image information stored in the image geographic information part. If recognizing a specific feature through the image processor, the controller proportionally compares geographic information of the image geographic information part with size and direction data on a peripheral feature near the specific feature within an image, and recognizes the peripheral feature.

The camera may capture an image if there is no motion for a predetermined time.

The sensor can include a geomagnetic sensor.

The image geographic information part can include width and height information on the feature.

According to another aspect of the present invention, a method for feature recognition in a portable terminal is provided. The method includes recognizing a current position of the portable terminal by a GPS, photographing an image of a feature by a camera, detecting a direction of the camera using a sensor, extracting a characteristic of the image of the feature photographed by the camera and comparing the extracted characteristic with image information on a feature positioned in a direction of the camera on a position of the portable terminal and, if recognizing a specific feature, comparing geographic information of an image geographic information part with distance and direction data on a peripheral feature near the specific feature within an image, and recognizing the peripheral feature.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 2illustrates a block diagram of a construction of an image recognition portable terminal according to an exemplary embodiment of the present invention.

Referring toFIG. 2, the portable terminal includes a controller201, a GPS202, a camera203, a sensor204, an image processor205, an image data storage part206, and a Liquid Crystal Display (LCD)207. The GPS202can recognize a user's current position. The camera203can recognize a real image. The sensor204can detect a motion and direction of the terminal, i.e., the camera203. The image processor205can extract a characteristic of a captured image, and can compare the characteristic with image data stored in the image data storage part206. The image data storage part206is operable to store image data on each feature and geographic information on the feature. The controller201can control each part. The LCD207displays the real image and the geographic information.

The GPS202is able to recognize a user's current position. User position information is used to reduce an amount of comparison-target data stored in the image data storage part206at the time of image recognition through the camera203. That is, the position information is used to reduce a time consumed for image recognition by, upon image recognition, not comparing with all data in the image data storage part206but comparing a recognized image with only data on the vicinity of the current position recognized by the GPS202.

The controller201can capture an image using the camera203when determining that there is no motion of the terminal for a predetermined time using motion information on the terminal input from the sensor204. The controller201compares the captured image with image data previously stored in the image data storage part206through the image processor205, and detects geographic information on a captured feature.

The sensor204includes an acceleration sensor and a geomagnetic sensor. The sensor204is attached to the terminal and thus, can detect a direction of the terminal, for example, a direction of the camera203, and can also measure an attitude of the terminal. When determining that there is no change of the attitude of the terminal for a predetermined time using attitude information on the terminal, the controller201determines that a user is concerned about a specific feature photographed by the camera203at this time and captures an image of this time. Also, the sensor204detects a direction corresponding to this time through the geomagnetic sensor and, later on, uses the detected direction for information matching with a peripheral feature.

The image data storage part206can store image data on each feature, and can include position information and name information corresponding to each feature. Thus, the image data storage part206compares an image of a specific feature captured by the camera203with a previously stored image, and detects a name and position of the captured image. Particularly, the image data storage part206includes 3-Dimensional (3D) information (width and height information) on a feature. Thus, if an image is recognized through the camera203, the controller201compares previously stored width information with width information displayed on a screen and calculates how far a real distance per pixel of the screen is. At this time, the image data storage part206detects information on a peripheral feature near the specific feature using the detected position information on the specific feature.

Also, the controller201can calculates a real distance between the detected position of the specific feature, which is a standard, and the peripheral feature. Thereafter, the controller201converts the real distance between the standard feature and the peripheral feature into a distance on the screen. At this time, the controller201can convert the real distance between the two features into the distance on the screen using information on the previously calculated real distance per pixel. The controller201matches geographic information on the peripheral feature to a real image using the calculated distance information on the screen from the standard feature on the screen.

FIG. 3illustrates an example operation screen of the present invention. A GPS recognizes a user's current position. Thereafter, if a camera recognizes a building of “Digital Research Institute” that is a standard feature through an image processor, a controller matches an indication named “Information Communication Research Institute” to a real image using direction information input from a sensor and position information on the two buildings loaded from an image data storage part.

FIG. 4illustrates a flow diagram for a method for matching of geographic information on a feature between an image captured by a camera and image data stored in an image data storage part according to an exemplary embodiment of the present invention.

In step401, if a mobile communication terminal user powers on a GPS receiver included in a mobile communication terminal, the GPS recognizes a user's current position.

Thereafter, while the user photographs a peripheral feature using a camera, a sensor recognizes if the user photographs a specific feature for a predetermined time, i.e., the sensor determines that there is no change of an attitude of the camera through a sensor, and a controller captures an image of the specific feature. At this time, if the sensor does not determine a non-change of an attitude of the camera (i.e., there is no stop determination), the controller determines that the user is not concerned in the specific feature, and does not capture an image. (steps403to407)

Then, if the controller captures the image of the specific feature, in step409, an image processor extracts a characteristic of the specific feature.

The controller compares the extracted characteristic of the specific feature with image information stored in the image data storage part, and loads geographic information on the captured specific feature. Then, the controller uses only data on a peripheral feature that is in the vicinity of the user's current position for image recognition using the user's current position information. The controller acquires name, position, width, and height information on a feature stored in the image data storage part. When this process is completed, the feature becomes a standard feature. (steps411to415)

Then, the controller compares width and height information on the standard feature with width and height information on a screen and calculates a real distance indicated by one pixel on the screen.FIG. 5illustrates the above process. If a “Digital Research Institute” is recognized as a standard feature, the controller extracts (xs, ys), ws, ls, and hs, and loads (xr, yr), wr, lr, and hrfrom the image data storage part. At this time, the controller compares the ws, ls, and hswith the wr, lr, and hrand calculates a real distance indicated by one pixel of the screen. Then, the controller loads real coordinates (xr′, yr′), wr′, lr′, and hr′ of an “Information Communication Research Institute” that is information on a peripheral feature around the (xr, yr). Then, the controller calculates distsusing distrand the previously calculated real distance information per one pixel. Lastly, the controller calculates the (xs′, ys′) using the user direction information and the distson the basis of the (xs, ys) (steps417to425)

FIGS. 6A and 6Billustrate a simple concept of calculating coordinates (xs′, ys′). In an example, it is assumed that, when user direction information is equal to 0°, i.e., in a state where a user directs to due north, an object (B) having screen coordinates (xs′, ys′) is a feature located on the left of an object (A) having screen coordinates (xs, ys) as shown inFIG. 6A. However, when the user direction information is equal to 180°, unlike when the user direction information is equal to 0°, the feature (B) is positioned and output on the right of the object (A) on a screen. This is because a position of a peripheral feature positioned on the screen centering on a standard feature is varied depending on a user's position and direction. By this method, the controller can detect whether the peripheral feature is positioned on the right or left of the standard feature on the screen using the user direction information.

Lastly, the controller outputs geographic information on the peripheral feature in the calculated coordinates (xs′, ys′), thereby being capable of matching the geographic information on the peripheral feature to a real image.

The present invention simply photographs a feature without a user's specific operation command and therefore, captures an image and matches not only geographic information on the captured feature but also geographic information on a peripheral feature to a real image, thus having an advantage that a user can easily recognize a feature.