Patent Publication Number: US-2012029809-A1

Title: Apparatus and method for providing road view

Description:
CROSS-REFERENCE TO RELATED APPLICATION(S) 
     This application claims priority from and the benefit of Korean Patent Application No. 10-2010-0074305, filed on Jul. 30, 2010, the entire disclosure of which is incorporated herein by reference as if fully set forth herein. 
     BACKGROUND 
     1. FIELD 
     The following description relates to an apparatus and method for providing a road view at a specific site to a user. 
     2. Discussion of the Background 
     With the recent expansion of the diversity of services and supplementary features available for mobile communication terminals, various service systems for providing a variety of is information to users have been developed. For example, a system has been recently introduced which provides basic information on the location of a mobile communication terminal when a user requests information about his or her current location through the mobile communication terminal carried by the user. 
     To detect landmark buildings or places in the vicinity of an area where a user is located, current location information identified by a global positioning system (GPS) is transmitted to a server from a GPS-embedded navigation system or mobile terminal, and the server displays a vicinity map and a vicinity image through a screen of the terminal. However, it may be inconvenient for a user who wants to view a vicinity image from different viewpoints since the user only can use key inputs to rotate the screen to see the different viewpoints. 
     SUMMARY 
     Exemplary embodiments of the present invention provide an apparatus and method for providing road view data, which automatically detects road view data corresponding to changes of a location and a photographing direction of a camera, and provides the detected road view data to a user. 
     Additional features of the invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. 
     An exemplary embodiment of the present invention discloses an apparatus for providing road view data, including: a camera to capture an image and output the captured image; a sensor unit to receive the captured image and output sensing data, comprising a photographing direction of the camera; a manipulation unit to received data, the data including a road view data request signal; a control unit to receive the captured image, the sensing data and the road view data request signal, and to determine, at a reference position, road view data corresponding to the photographing direction of the camera and output the determined road view data in response to the received road view data request signal; a display unit to display image data containing the determined road view data. 
     An exemplary embodiment of the present invention discloses a method of providing road view data from an apparatus comprising a camera and a sensor unit, the method including: sensing the photographing direction of the camera; determining a reference position of the camera; detecting the road view data corresponding to the reference position and the photographing direction; and outputting the detected road view data. 
     It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory and are intended to provide further explanation of the invention as claimed. Other features and aspects will be apparent from the following detailed description, the drawings, and the claims. 
    
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
       The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this specification, illustrate embodiments of the invention, and together with the description serve to explain the principles of the invention. 
         FIG. 1  depicts an illustrative apparatus for providing road view data according to an exemplary embodiment. 
         FIG. 2  depicts an illustrative diagram for determining road view data according to an exemplary embodiment. 
         FIG. 3  depicts an exemplary screen shot displayed by a display unit according to an exemplary embodiment. 
         FIG. 4  depicts an illustrative control unit according to an exemplary embodiment. 
         FIG. 5  is an illustrative example of a road map depicting a location change according to an exemplary embodiment. 
         FIG. 6  is an illustrative example of a road map depicting a temporary location change according to an exemplary embodiment. 
         FIG. 7  is an illustrative example of a road map depicting see-through levels according to an exemplary embodiment. 
         FIG. 8A  is a diagram illustrating a road map depicting road view data based on a real-time change of the photographing direction at a see-through level according to an exemplary embodiment. 
         FIG. 8B  is a diagram illustrating a road map depicting road view data based on a real-time change of the reference position at a see-through level according to an exemplary embodiment. 
         FIG. 9  is a flowchart depicting a method of providing road view data according to an exemplary embodiment. 
         FIG. 10  is a flowchart depicting a method of providing road view data by selecting a see-through mode of operation according to an exemplary embodiment. 
         FIG. 11  is a flowchart depicting a method of providing road view data by selecting a location change mode of operation according to an exemplary embodiment. 
         FIG. 12  is a flowchart depicting a method of providing a road view by selecting a temporary location change mode of operation according to an exemplary embodiment. 
     
    
    
     DETAILED DESCRIPTION 
     The invention is described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the invention are shown. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure is thorough, and will fully convey the scope of the invention to those skilled in the art. In the drawings, the size and relative sizes of layers and regions may be exaggerated for clarity Like reference numerals in the drawings denote like elements. 
     It will be understood that when an element or layer is referred to as being “on” or “connected to” another element or layer, it can be directly on or directly connected to the other element or layer, or intervening elements or layers may be present. In contrast, when an element is referred to as being “directly on” or “directly connected to” another element or layer, there are no intervening elements or layers present. 
       FIG. 1  depicts an illustrative apparatus for providing road view data according to an exemplary embodiment. The apparatus includes a control unit  110 , a camera  120 , a sensor unit  130 , a storage unit  140 , a manipulation unit  150 , and a display unit  160 . 
     The camera  120  photographs or captures and outputs an image. The camera  120  may establish a photographing direction in which a user desires to view. In addition, the camera  120  may increase or decrease a size of an image, and may be rotated under the control of the control unit  110  either manually or automatically. 
     The sensor unit  130  detects movement of the apparatus, a current location, a photographing direction, a location change, a speed of the apparatus, and a speed of change of the photographing direction, and outputs the sensing data to the control unit  110 . The sensor unit  130  may include any suitable apparatus for sensing the sensing data, for example, a global positioning system (GPS) receiver that receives a camera location information signal transmitted from a GPS satellite, a gyroscopic sensor that detects and outputs an azimuth and an inclination angle of the camera  120 , and/or an acceleration sensor that measures speed and acceleration a rotation direction and a degree of rotation of the camera  120 , etc. 
     The storage unit  140  stores road view data, which may include data of map services that provides  360  degree panoramic photographs showing detailed images. For example, the storage unit  140  may include photographs for different locations. The exemplary road view data may be obtained by a fisheye lens camera which can capture an image with an angle of view of 180 degrees or more, or may be generated by combining a number of previously captured images into a panoramic picture. The storage unit  140  may be embedded in the apparatus or provided outside of the apparatus to receive data over a network. If the storage unit  140  is provided externally to the apparatus to receive the data over a network, the apparatus may further include a communication interface capable of network communication. 
       FIG. 2  depicts an illustrative diagram for determining road view data according to an exemplary embodiment. The road view data is data determined by variables including a reference position  210  at which the camera  120  is located, and a photographing direction  220  of the camera  120 . The photographing direction  220  may be a unit direction vector that is defined as (X, Y, Z) coordinates on a three-dimensional orthogonal coordinate system having the reference position  210  as an origin. However aspects need not be limited thereto and the photographing direction may be a two-dimensional vector defined as (X,Y). In an exemplary embodiment, see-through levels  235  and  245  may be used as variables to determine road view data. The see-through level  235  is also defined by line L 1 , and see-through level  245  is also defined by the line L 2 . The see-through levels  235 /L 1  and  245 /L 2  are used for detecting a road view behind a building which blocks the view from the photographing direction  220  of the camera. Thus, by using see-through levels, the exemplary apparatus can display the view of a street behind an object blocking the camera&#39;s view of the selected road. By way of example, as shown in  FIG. 2 , if the line L 1  is set as a see-through level, a tangent plane  240  is detected as road view data. Tangent plane  240  is formed at the contact point between a sphere, with reference point  210  as its center point and having a radius of the same length as line L 1 , and a three-dimensional (3D) direction vector with the reference position  210  as a starting point and a length equal of line L 1 . In other words, the tangent plane  240  is tangent to a sphere, with a center at reference point  210 , at a point were the a vector originating from the sphere&#39;s center meets the surface of the sphere. Similarly, tangent plane  230  is formed at the contact point between a sphere having L 2  as its radius and reference point  210  as its center point and a 3D directional vector of length L 2  originating from reference position  210 . The storage unit  140  may store road map data corresponding to the vicinity of the reference position  210 . Examples of the stored road map are illustrated in  FIGS. 5 to 8 , and a reference position and a road view data presence area are displayed in each road map. 
     The manipulation unit  150  is a device that receive user information to provide road view data, and may include, for example, a key input portion that generates key data in response to pressing a key button, a touch screen, a mouse, etc. According to an exemplary embodiment, the information processed by the control unit  110  may be inputted into the apparatus through the manipulation unit  150  in association with the display unit  160 . The inputted information may include at least one of a road view data request signal, a seeing-through request signal, a level selection signal, a location change request signal, and a zoom-in/zoom-out request signal 
     The display unit  160  outputs user interface information received from the control unit  110  and road view data, and may further display a road map. The manipulation unit  150  and the display unit  160  may be integrated with each other to form a user interface, such as a touch screen. An example of a road data provision screen displayed on the display unit  160  is illustrated in  FIG. 3 .  FIG. 3  depicts an exemplary screen shot displayed by a display unit according to an exemplary embodiment. Referring to  FIG. 3 , the road view data provision screen may include a road view  310  as viewed by a camera at a reference position and a road map  320  of the vicinity of the reference position on a portion of the screen. In addition, a see-through level control bar  330  and a temporary location change checkbox  340  may be included as user interface information in the screen. 
     In response to a road view request signal input through the manipulation unit  150 , the control unit  110  detects a road view corresponding to the reference position and the photographing direction of the camera  120  and based on the sensing information about changes in the reference position and the photographing direction received from the sensor unit  130 , and outputs the detected road view to the display unit  160 . Additionally, the control unit  110  may detect the road map corresponding to the reference position and output the detected road map along with the road view to the display unit  160 , or may output to the display unit only the road map. Furthermore, the control unit  110  may provide a perspective road view in response to a see-through request signal from the manipulation unit  150 . 
       FIG. 4  depicts an illustrative control unit according to an exemplary embodiment. 
     The control unit  110  includes a variable determination unit  410 , a road view data detection unit  420 , an output data generation unit  430 , and may further include a road map detection unit  440 . 
     The variable determination unit  410  determines reference position information and see-through level information. The reference position information and see-through level information are variables used for road view data detection. The variable determination unit  410  determines the reference position information and the see-through level information based on information inputted through the manipulation unit  150  and sensing information received from the sensor unit  130 . The variable determination unit  410  outputs the determined information to the road view detection unit  420 . 
     The road view detection unit  420  detects road view data from the storage unit  140  corresponding to the variables received from the variable determination unit  410 . If reference position information is received by the road view detection unit  420 , then the road view detection unit  420  also receives photographing direction information from the sensor unit  130 , or additionally detects road view data corresponding to the see-through level information. 
     The output data generation unit  430  outputs the road view data received from the road view data detection unit  420  to the display unit  160 . The output data generation unit  430  may further adjust a tilt of the road view based on the camera inclination angle received from the sensor unit  130  and outputs the adjusted road view data to the display unit  160 . Moreover, in response to a zoom-in or zoom-out request signal from the manipulation unit  150 , the output data generation unit  430  may adjust an angle of view of the road view data, and output the adjusted road view data. Furthermore, the output data generation unit  430  outputs interface information for receiving inputted information for the manipulation unit  150 . By way of example, to receive see-through level selection information, the output data generation unit  430  may output a see-through level input window or input bar on the display unit  160 . By way of further example, the output data generation unit  430  may generate and output, to the display unit  160 , an input button for receiving information about the occurrence of temporary location change according to a location change request. 
     The road map detection unit  440  detects a road map corresponding to variable information received from the variable determination unit  410  and the sensor unit  130 , and outputs the detected road map to the output data generation unit  430 .  FIG. 7  is an illustrative example of a road map depicting see-through levels according to an exemplary embodiment. In  FIG. 7 , a road view presence area  710 , a reference position  720 , and a real-time photographing direction  730  of a camera are displayed on the road map, and additionally, see-through level contact points  741 ,  742 , and  743  may be displayed on the road map. Accordingly, the output data generation unit  430  can output a road map output from the road map detection unit  440  on a portion of the road view in an overlapping manner. 
     Referring again to  FIG. 4 , the variable determination unit  410  may further include a location determination module  411  and a see-through level determination module  413 . 
     The location determination module  411  determines and outputs a reference position in response to a road view data request from the manipulation unit  150 . The reference position may be inputted or the current location of the apparatus, as detected by the sensor unit  130 , may be used. In exemplary embodiments, the reference position may be inputted by a user or received from a location service. The location determination module  411  may update the location of the reference position in real time by detecting a location change and a location change speed. The location determination module  411  may output real-time reference position is information to the road map detection unit  440  as well as to the road view detection unit  420 , thereby updating the reference position in real time on the road map which is output along with the road view to the display unit  160 . The location determination module  411  may set a location specified by the user as the reference position in accordance with a user&#39;s request to move the reference position. In an exemplary embodiment, the user may request to a see-through level location change or a temporary reference position change. 
     If a see-through level location change is requested, the location determination module  411  sets the location inputted by the user as a reference position and outputs the set reference position to the road view data detection unit  420  or the road map detection unit  440 . In an exemplary embodiment, the road view data detection unit  420  detects the road view data for the set reference position, and outputs the detected road view data. In an exemplary embodiment the road map detection unit  440  detects a road map data for the set reference position, and outputs the detected road map data.  FIG. 5  is an illustrative example of a road map depicting a location change according to an exemplary embodiment.  FIG. 5  depicts the road map detection unit  440  changing a current reference position A to a location B inputted by the user. The photographing direction of the camera and see-through levels displayed after the change may originate from the changed reference position B. 
     If a temporary reference position change is requested, then the location determination module  411  stores the current reference position as a fixed reference position, and sets the location inputted by the user as the reference position. The determination module  411  outputs the set reference position as the reference position until a termination request is received from the user. In an exemplary embodiment, the road view data detection unit  420  detects and outputs the road view data for the set reference position.  FIG. 6  is an illustrative example of a road map depicting a temporary location change according to an exemplary embodiment.  FIG. 6  depicts the results of road map detection unit  440  determining a current reference position A as a fixed reference position, and a location B inputted by the user as a temporary reference position. The location B is a temporary reference position and see-through levels are not provided at the location B. However, the road view detection unit  440  may detect road view data based on the real-time change of the camera location or a photographing direction from the temporary reference position B, as depicted by the dashed arrows in  FIG. 6 . 
     The see-through level determination module  413  determines and outputs a see-through level in response to a see-through request. The determined see-through level may be a previously set see-through level information, or see-through level information which is inputted by a user. If previously set see-through level information is used, in response to receiving a see-through request signal from the manipulation unit  150 , the see-through level determination module  413  may output see-through level information corresponding to a see-through location that is closest to the reference position. 
     If the see-through level information inputted by a user is used, the see-through level may be input through the see-through level input bar  330  as shown in the example illustrated in  FIG. 3 , or by touching the see-through level display contact points  741 ,  742 , and  743  on the screen as shown in the example illustrated in  FIG. 7 . Referring to  FIG. 7 , an initial contact point  741  corresponds to a Level  1 , the second contact point  742  corresponds to a Level  2 , and the third contact point  743  corresponds to a Level  3 . If see-through level information is inputted, the road view data detection unit  420  detects data corresponding to the see-through level in the direction corresponding to the direction that the camera is moving or rotating in.  FIG. 8A  is a diagram illustrating a road map depicting road view data based on a real-time change of the photographing direction at a see-through level according to an exemplary embodiment. In an example illustrated in  FIG. 8A , the area  812  corresponds to the selection of see-through Level  2  in  FIG. 7  and road view data corresponding to Level  2  is displayed. 
     A method of providing a road view will now be described with reference to  FIGS. 9-12 . 
       FIG. 9  is a flowchart depicting a method of providing a road view according to an exemplary embodiment. In operation  910 , the control unit  110  (see  FIG. 1 ) receives a real-time photographing direction from sensor unit  130  (see  FIG. 1 ) and determines a reference position in response to a road view request. Although not illustrated in drawings, if there is no user&#39;s request for a particular location to be established as the reference position, the control unit  110  (see  FIG. 1 ) sets the current location detected by the sensor unit  130  (see  FIG. 1 ) as the reference position. Once the initial reference position is set, the reference position is updated in real time if a location change is detected as the user moves. However, if there is a user&#39;s specific request to establish a reference position, the location requested by the user is set as the reference position by the control unit  110 . There are various methods by which a user may request a specific location be used as a reference position by the apparatus. By way of example, a specific location name may be input by the user and set as the reference position, or a road map of a specific region may be displayed on a touch screen and a specific location on the road map chosen as the reference position by touching the touch screen. In operation  920 , the control unit  110  detects road view data based on the received photographing direction and the reference position determined in real time. In operation  930 , the control unit  110  outputs the detected road view data in the display unit  160  (see  FIG. 1 ). The detected road view data may be displayed as a road map on a portion of the display unit  160  that shows the reference position and a real-time camera photographing direction. 
       FIG. 10  is a flowchart depicting a method of providing road view data by selecting a location change mode of operation according to an exemplary embodiment. 
     In operation  1010 , a see-through mode is selected in response to input of a see-through mode selection signal. The see-through mode selection signal may be inputted after the reference position is determined as shown in the flowchart of  FIG. 9  or while a road view is being provided. 
     In operation  1020 , the control unit  110  receives see-through level selection information. The see-through level selection information may be received by touching the see-through level contact points  741 ,  742 , and  743  as shown in the example illustrated in  FIG. 7 , or by the see-through level selection bar  330  as shown in the example illustrated in  FIG. 3  while the road view data is displayed. 
     Thereafter, in operation  1030 , the control unit  110  detects a real-time photographing direction and determines a real-time reference position. In operation  1040 , the control unit  110  detects road view data according to the real-time photographing direction, the reference position, and the selected see-through level. In operation  1050 , the control unit  110  outputs the detected see-through road view. 
     That is, the control unit  110  detects a real-time photographing direction of the camera, and detects and outputs road view data corresponding to the detected real-time photographing direction. More specifically, the control unit  110  checks for changes in the angle and speed of the received photographing direction of the camera, and outputs road view data corresponding to the changes in the received photographing direction of the camera while displaying a road view moving in the photographing direction. By way of example, when a user at the reference position  810  turns a terminal including the camera left or right to change the photographing direction of the camera as shown in  FIG. 8A , the control unit  110  outputs road view data  812  based on a currently set distance level and the photographing direction of the camera. As described above,  FIG. 8A  depicts the road view data outputted by the apparatus when Level  2  see-through data is selected in  FIG. 7 . If see-through Level  2  is selected and the photographing direction of the camera is changed clockwise around the reference position  810  as represented by the dotted arrow, the road view data corresponding to the see-through Level  2  is road view data of a shaded area  812  in  FIG. 8A  and is output corresponding to the clockwise rotation of the photography direction of the camera. 
     Further, the control unit  110  may detect real-time camera movement and output road view data corresponding to the detected movement. The control unit  110  may receive information about the moving direction and moving speed of the camera from sensor unit and change the reference position in real-time. By way of example,  FIG. 8B  is a diagram illustrating a road map depicting road view data based on a real-time change of the reference position at a see-through level according to an exemplary embodiment. If the location A is set as a reference position and road view data corresponding to a see-through Level  2  of  FIG. 7  is selected, and the camera is facing in a direction of a location A′, the control unit  110  detects a real-time movement of the camera and changes the reference position from the location A to a location B in real time as the camera moves from the location A to the location B. According to the real-time change of the reference position from the location A to the location B, the control unit  110  continuously detects the road view data in a direction from the location A′ to a location B′ and outputs the detected road view data. 
     Although the embodiments describing the reference position change and the photographing direction change are described separately, aspects of the invention are not limited thereto and the control unit  110  may detect and output road view data corresponding to an event of simultaneous change of the reference position and the photographing direction. 
     Further, the control unit  110  may output a road view screen in which the perspective of the see-through road view data is adjusted in response to receiving a zoom-in or zoom-out request signal from the user. 
       FIG. 11  is a flowchart depicting a method of providing road view data by selecting a location change mode of operation according to an exemplary embodiment. 
     In operation  1110 , the control unit  110  (see  FIG. 1 ) selects a location change mode in response to an inputted location change mode selection signal. While the camera is in see-through mode, the location change mode selection signal may be inputted if a user stops moving the camera and selects location change mode. The user may be at a specific location at which the user desires to obtain road view data. However aspects need not be limited thereto such that the camera may remain in movement. If the camera is not in see-through mode, the location change mode selection signal may be inputted by selecting the location change mode. In exemplary embodiments, the location change mode may be inputted by a user or received from a location service. 
     In operation  1120 , location change information is inputted. To receive the location change information, the location change information may be input while the see-through road view is selected location change mode may be selected by touching a road map, or by directly inputting the name of a specific location into the apparatus. 
     In operation  1130 , the input changed location is set as a reference position by the control unit  110 . Accordingly, as depicted and described in reference to  FIG. 5 , the inputted changed location at point B is placed at the center as the new reference position and a real-time photographing direction is displayed. Then, as depicted and described in reference to  FIG. 9  or  10 , detection of the road view data or the see-through road view data may be performed. 
       FIG. 12  is a flowchart depicting a method of providing a road view by selecting a temporary location change mode of operation according to an exemplary embodiment. 
     In operation  1210 , temporary location change mode is selected by the control unit  110  (see  FIG. 1 ) in response to input of a temporary location change mode selection signal. The temporary location change mode selection signal may be inputted when a user stops moving the camera and selects temporary location change mode. The user may be at a specific location at which the user desires to obtain road view data, while moving the camera in see-through mode. However aspects need not be limited thereto such that the camera may remain in movement. If the camera is not in see-through mode, the location change mode selection signal may be inputted by selecting the temporary location change mode. In exemplary embodiments, the location change mode selection signal may be inputted by a user or received from a location service. 
     In operation  1220 , location change information about the changed location is input and received by the control unit  110 . In an exemplary embodiment, the location change information may be inputted while in see-through mode. In an exemplary embodiment, the temporary location change mode may be activated by selecting a particular location on the road map. Further, if a temporary location mode is activated a distance level may be manipulated or changed to 0, information displayed is for the selected location. 
     In operation  1230 , a current reference position is stored as a fixed reference position by the control unit  110 . In operation  1240 , the inputted changed location is set as a is reference position. By way of example as shown in  FIG. 6 , a photographing direction based on the fixed reference position A is displayed, whereas a photographing direction based on the inputted changed location, reference position B, is not displayed and the outputted road view results from moving or rotating the camera at the new reference position B. Thereafter, in operation  1250 , the real-time photographing direction is detected. The road view data corresponding to the reference point and the photographing direction is detected in operation  1260 , and is outputted in operation  1270 . In operation  1280 , the fixed reference position is re-set as a reference position in response to a termination request that is accepted by the user. 
     As described above, the exemplary embodiment detects changes in the location and direction of a camera and automatically adjusts the displayed road view data to correspond to the changes detected. Thus a user can is conveniently provided with the road view data without inputting additional signals regarding changes of in the location and photographing direction. 
     In addition, the road view data is provided automatically according to movement of the camera, and thus the user can experience it as if he or she directly looked around the vicinity of a corresponding location. 
     It will be apparent to those skilled in the art that various modifications and variation can be made in the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention cover the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.