User equipment and control method for the same

An embodiment user equipment includes an image sensor, a user interface configured to display an image photographed by the image sensor and to receive a user input, a location detection sensor configured to detect a location of the user equipment, and a controller configured to identify a point of interest (POI) located in a shooting direction of the image sensor based on an output of the location detection sensor, control the user interface to display an augmented reality (AR) icon corresponding to the POI by superimposing the AR icon on the image photographed by the image sensor, and add information about the POI to a scrap list based on an input to the AR icon.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Korean Patent Application No. 10-2021-0017718, filed on Feb. 8, 2021, which application is hereby incorporated herein by reference.

TECHNICAL FIELD

The disclosure relates to a user equipment and a control method for the same.

BACKGROUND

Recently, services using augmented reality (AR) are increasing. A user may be provided with a more lively service than before through an augmented reality service by using a user equipment (UE).

For example, a map and navigation service using an AR technology may display an AR image of a point of interest (POI) related to various types of businesses such as a restaurant, a café, a grocery store, etc., at the location of the corresponding POI. That is, when a user photographs a POI using a UE, an AR image of the POI which is superimposed on the photographed image may be displayed at the location of the POI.

SUMMARY

The disclosure relates to a user equipment and a control method for the same. Particular embodiments relate to a user equipment providing augmented reality and a control method for the same.

An embodiment of the disclosure provides a user equipment (UE) and a control method for the same that may provide a user interface to enable information on a point of interest (POI) to be more conveniently added to a scrap list, and provide various types of posts about the POI.

According to an embodiment of the disclosure, there is provided a user equipment including an image sensor, a user interface configured to display an image photographed by the image sensor and receive a user input, a location detection sensor configured to detect a location of the user equipment, and a controller configured to identify a point of interest (POI) located in a shooting direction of the image sensor based on an output of the location detection sensor, control the user interface to display an augmented reality (AR) icon corresponding to the POI by superimposing the AR icon on the image photographed by the image sensor, and add information on the POI to a scrap list based on a user's input to the AR icon.

The user's input to the AR icon is double tapping the AR icon.

The controller is configured to control the user interface to display a scrap button and add the information on the POI to the scrap list based on a user's input to the scrap button.

The controller is configured to control the user interface to display the AR icon of the POI by superimposing the AR icon on the image photographed by the image sensor based on a user's selection of a scrap display filter, the POI being included in the scrap list.

The controller is configured to control the user interface to display information on each of a plurality of POIs included in the scrap list based on a user's selection of a scrap list display.

The controller is configured to control the user interface to display information on at least one POI corresponding to a business type filter among the plurality of POIs included in the scrap list based on a user's selection of the business type filter.

The controller is configured to control the user interface to display the information on each of the plurality of POIs in an order of closest to the user equipment.

When adding the information on the POI to the scrap list, the controller is configured to add a preference value for the POI to the information on the POI, and control the user interface to display the information on each of the plurality of POIs in an order of highest preference value.

The user equipment further includes a communicator, wherein the controller is configured to control the communicator to transmit the information on the POI included in the scrap list to an external device.

The user equipment further includes a communicator, wherein the controller is configured to control the user interface to display a list of surrounding POIs located in a vicinity of the user equipment based on a user's input to a create post button, and when a selection of a first POI from the list of surrounding POIs and a post about the first POI is input, control the communicator to transmit, to an external server, post information including an AR image of the post and location information of the first POI.

The controller is configured to control the user interface to display the AR image of the post by superimposing the AR image of the post about a location of the first POI in the image photographed by the image sensor.

The controller is configured to control the user interface to display a list of posts about the POI based on a user's input to the AR icon corresponding to the POI.

The controller is configured to control the communicator to transmit the post information to an external device.

According to an embodiment of the disclosure, there is provided a control method of a user equipment including an image sensor, a user interface and a location detection sensor, the control method including identifying a POI located in a shooting direction of the image sensor based on an output of the location detection sensor, controlling the user interface to display an AR icon corresponding to the POI by superimposing the AR icon on an image photographed by the image sensor, and adding information on the POI to a scrap list based on a user's input to the AR icon.

The user's input to the AR icon is double tapping the AR icon.

The control method further includes controlling the user interface to display a scrap button and adding the information on the POI to the scrap list based on a user's input to the scrap button.

The control method further includes controlling the user interface to display the AR icon of the POI by superimposing the AR icon on the image photographed by the image sensor based on a user's selection of a scrap display filter, the POI being included in the scrap list.

The control method further includes controlling the user interface to display information on each of a plurality of POIs included in the scrap list based on a user's selection of a scrap list display.

The control method further includes controlling the user interface to display information on at least one POI corresponding to a business type filter among the plurality of POIs included in the scrap list based on a user's selection of the business type filter.

The control method further includes controlling the user interface to display the information on each of the plurality of POIs in an order of closest to the user equipment.

The control method further includes, when adding the information on the POI to the scrap list, adding a preference value for the POI to the information on the POI, and controlling the user interface to display the information on each of the plurality of POIs in an order of highest preference value.

The user equipment further includes a communicator, and the control method further includes controlling the communicator to transmit the information on the POI included in the scrap list to an external device.

The user equipment further includes a communicator, and the control method further includes displaying a list of surrounding POIs located in a vicinity of the user equipment based on a user's input to a create post button, and when a selection of a first POI from the list of surrounding POIs and a post about the first POI is input, controlling the communicator to transmit, to an external server, post information including an AR image of the post and location information of the first POI.

The control method further includes controlling the user interface to display the AR image of the post by superimposing the AR image of the post on a location of the first POI in the image photographed by the image sensor.

The control method further includes controlling the user interface to display a list of posts about the POI based on a user's input to the AR icon corresponding to the POI.

The control method further includes controlling the communicator to transmit the post information to an external device.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

Like reference numerals throughout the specification denote like elements. Also, this specification does not describe all the elements according to embodiments of the disclosure, and descriptions well-known in the art to which the disclosure pertains or overlapped portions are omitted.

It will be understood that when an element is referred to as being “connected” to another element, it can be directly or indirectly connected to the other element, wherein the indirect connection includes “connection” via a wireless communication network.

It will be understood that the term “include” when used in this specification, specifies the presence of stated features, integers, steps, operations, elements, and/or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It is to be understood that the singular forms are intended to include the plural forms as well, unless the context clearly dictates otherwise.

Further, the terms such as “˜part”, “˜device”, “˜block”, “˜member”, “˜module”, and the like may refer to a unit for processing at least one function or act. For example, the terms may refer to at least one process processed by at least one hardware, such as field-programmable gate array (FPGA)/application specific integrated circuit (ASIC), software stored in memories or processors.

Hereinafter, an embodiment of a user equipment (UE) and a control method for the same according to embodiments of the disclosure will be described in detail with reference to the accompanying drawings.

FIG.1illustrates a point of interest (POI) display system according to an embodiment.

Referring toFIG.1, the POI display system1includes a UE10, a server20and a network30. The UE10displays an augmented reality (AR) image of a POI, the server20manages identification information for each POI and a post database for each POI, and the network30provides communication between the UE10and the server20.

In this instance, the POI may correspond to stores of various types of businesses such as restaurants, cafes and grocery stores. However, a type of POI is not limited to a store, as long as it corresponds to a place, such as a tourist destination, a public office, and the like.

Also, the identification information is information for identifying a POI, and may include location information of the POI, an AR icon of the POI, detailed information of the POI such as a name of a store, an advertisement (a logo, a signboard, etc.), an address, opening hours, a telephone number, a congestion level, user ratings, and the like.

In addition, the post database may store post information on posts created by users about the POI, and the post information may include location information of the POI and an AR image (photos, videos, text, etc.) of a post.

According to an embodiment, the UE10may display an AR icon of a POI located in a shooting direction of a camera on a display.

That is, the UE10may identify the POI located in the shooting direction of the camera based on location information of the UE10and location information of the POI, and display the AR icon of the POI by superimposing the AR icon on an image photographed by the camera. In this instance, the location information of the POI and the AR icon of the POI are included in identification information of the POI.

According to an embodiment, the UE10may add the identification information of the POI to a scrap list based on a user's input (e.g. double tapping) to the AR icon or a user's input to a scrap button, to enable a user to more conveniently add the identification information of the POI to the scrap list. Adding the identification information of the POI to the scrap list is described in greater detail later.

According to an embodiment, when receiving a user input for posting the POI, the UE10may identify post information including the location information of the POI and an AR image of the post and transmit the post information to the server20, to add the post information to the post database of the POI in the server20. Identifying the post information on the POI is described in greater detail later.

According to an embodiment, the server20may provide the UE10with identification information for each POI and a post database for each POI, receive post information from the UE10and store the post information in a post database of a corresponding POI.

The POI display system1has been described above. Hereinafter, a configuration of the UE10is described in greater detail.

FIG.2is a control block diagram illustrating the UE10according to an embodiment.

Referring toFIG.2, according to an embodiment, the UE10includes a camera110, a location detection sensor120, a controller130, a user interface140and a communicator150. The controller130adds information on a POI to a scrap list based on a user's input and identifies post information on the POI based on a user's input. The user interface140displays an image photographed by the camera110and receives a user input. The communicator150communicates with the server20.

According to an embodiment, the camera110may be provided on a front and/or rear of the UE10to acquire the image. The camera110may be provided as a known type of image sensor, and a type of the camera110is not limited.

According to an embodiment, the location detection sensor120may detect a location of the UE10. For example, the location detection sensor120may identify the location of the UE10by receiving a global positioning system (GPS) signal. Also, the location detection sensor120may detect a shooting direction of the camera110by detecting a direction, an inclination, etc., of the UE10.

According to an embodiment, the controller130may add identification information of the POI to a scrap list based on a user's input (e.g. double tapping) to an AR icon or a user's input to a scrap button, to enable a user to more conveniently add the identification information of the POI to the scrap list. Adding the identification information of the POI to the scrap list is described in greater detail later.

According to an embodiment, when receiving a user input for posting the POI, the controller130may identify the post information including location information of the POI and an AR image of a post and control the communicator150to transmit the post information to the server20. Through the operation above, the UE10may add the post information to a database of the POI in the server20. Identifying the post information on the POI is described in greater detail later.

The controller130may include at least one memory storing a program for performing the aforementioned operations and operations described below, and at least one processor for implementing a stored program. When a plurality of memories and processors are provided, the plurality of memories and processors may be integrated into one chip or provided in physically separated locations.

According to an embodiment, the user interface140may include a display and an input part that receives a user input. The display may be provided on a front of the UE10and display an image photographed by the camera110. Also, the display may be provided as a known type of display module, and be implemented as a touch screen provided integrally with the input part.

According to an embodiment, the communicator150may communicate with the server20, receive identification information for each POI and a post database of the POI from the server20and transmit the post information of the POI. Also, the communicator150may be provided as a known type of wireless communication module.

The configuration of the UE10has been described above. Hereinafter, adding the identification information of the POI to the scrap list is described in greater detail.

FIG.3is a diagram illustrating an example where the UE10photographs a POI according to an embodiment.FIG.4is a diagram illustrating an example where the UE10receives an input for scrapping a POI according to an embodiment.FIG.5is a diagram illustrating an example where the UE10displays an AR icon of a POI included in a scrap list according to an embodiment.FIG.6is a diagram illustrating an example where the UE10displays a scrap list according to an embodiment.

Referring toFIG.3, according to an embodiment, the UE10may display an AR icon310of a POI located in a shooting direction of the camera110on the user interface140.

That is, the UE10may identify the POI located in the shooting direction of the camera110based on location information of the POI and location information of the UE10according to an output of the location detection sensor120, and display the AR icon310of the POI by superimposing the AR icon310on an image photographed by the camera110. In this instance, the location information of the POI and the AR icon310of the POI are included in identification information of the POI.

According to an embodiment, the UE10may display an AR icon310aof the POI located closest to the UE10differently from AR icons310band310cof other POIs. Specifically, the UE10may control the user interface140to include more information in the AR icon310aof the POI located closest to the UE10. For instance, the AR icons310band310cof other POIs display a single piece of information (e.g. a type of business), whereas the AR icon310aof the POI located closest to the UE10displays a plurality of pieces of information (e.g. a street name, a name of a store, a logo, a signboard, a congestion level, a price range, whether to operate a store, and the like).

According to an embodiment, the UE10may display detailed information320of the POI, and display a menu selection tab330for selecting a menu. In this instance, according to an embodiment, the UE10may display the detailed information320of the POI closest to the UE10.

Referring toFIG.4, according to an embodiment, the UE10may display a scrap button325, and add the identification information of the POI to a scrap list based on a user's input to the scrap button325. Through the operation above, a user may add the identification information of the POI to the scrap list by single tapping on the scrap button325, and thereby may store the identification information of the POI more conveniently.

Also, according to an embodiment, the UE10may add the identification information of the POI to the scrap list based on a user's input to the AR icon310. For example, the user's input to the AR icon310may be double tapping the AR icon310. Through the operation above, the user may add the identification information of the POI to the scrap list by simple tapping on the AR icon310, and thereby may store the identification information of the POI more conveniently.

Referring toFIG.5, according to an embodiment, the UE10may display the AR icon310of the POI included in the scrap list by superimposing the AR icon310on the image photographed by the camera110based on a user's selection of a scrap display filter. Through the operation above, the user may check the POI that the user has scrapped on the user interface140.

Specifically, when receiving a user's input to a POI filter menu331and a user's selection of a ‘my scrap’ from the ‘my scrap’ and ‘my posting’ section510, the UE10may display the AR icon310aof the POI included in the scrap list by superimposing the AR icon310aon the image photographed by the camera110.

Also, according to an embodiment, when receiving a user's selection on a type of business in a business type filter section520, the UE10may display the AR icon310only of the POI, corresponding to the selected type of business from POIs included in the scrap list, by superimposing the AR icon310on the image photographed by the camera110.

Referring toFIG.6, according to an embodiment, the UE10may display information on each of a plurality of POIs included in the scrap list based on a user's selection of a scrap list display. Through the operation above, the user may check information on the POI that the user has scrapped.

Specifically, when receiving a user's input to a scrap list display menu332, the UE10may display information610on each of the plurality of POIs included in the scrap list.

In this instance, as shown inFIG.6, the information610on each of the plurality of POIs included in the scrap list may be displayed in an order of closest to the UE10based on a control of the controller130.

According to an embodiment, the information610on each of the plurality of POIs may be displayed in an order of highest preference value. For instance, when adding identification information of a POI to the scrap list, the UE10may receive a preference value for the POI from the user and add the preference value to the identification information.

Also, according to an embodiment, the UE10may display information on at least one POI corresponding to a business type filter among the plurality of POIs included in the scrap list based on a user's selection of the business type filter in a business type filter section620. Through the operation above, the user may check information on a POI corresponding to the selected business type from the POIs that the user has scrapped.

Also, according to an embodiment, the UE10may transmit the identification information to an external device to share the identification information of the POI included in the scrap list with the external device. For example, the UE10may transmit the identification information of the POI to a social networking service (SNS) server or a terminal of an SNS user. In addition, the UE10may check a terminal of a user who subscribed to a service through the server20, and transmit the identification information of the POI to the terminal of the user who subscribed to the service.

Adding the identification information of the POI to the scrap list has been described above. Hereinafter, identifying post information of the POI is described in detail.

FIG.7is a diagram illustrating an example where the UE10receives a user's selection on a POI for inputting a post according to an embodiment.FIG.8is a diagram illustrating an example where the UE10displays an AR image of a post according to an embodiment.FIG.9is a diagram illustrating an example where the UE10displays an AR image of a post by reflecting actual surrounding information according to an embodiment.FIG.10is a diagram illustrating an example where the UE10displays a list of posts about a POI according to an embodiment.

Referring toFIG.7, according to an embodiment, the UE10may display a list710of surrounding POIs located in a vicinity of the UE10based on a user's input to a create post button (a posting menu)333.

In this instance, the UE10may receive a user's selection of a first POI from the list710of the surrounding POIs. Here, the first POI is subject to creating a post.

Also, according to an embodiment, the UE10may receive a user's selection of current location information720of the UE10. In this instance, a current location of the UE10may be identified as the first POI which is subject to creating a post.

When a post about the first POI is input from the user, the UE10may identify post information and transmit the identified post information to the server20. Here, the post information includes an AR image of the post and location information of the first POI. Also, the AR image of the post may be provided as various types of posts such as a photo, a video, text, and the like. In addition, the server20may add the post information received from the UE10to a post database of the first POI, and thereby may enable other users to see the post information.

Referring toFIG.8, according to an embodiment, the UE10may display an AR image810of the post by superimposing the AR image810on a location of the first POI in the image photographed by the camera110. That is, the UE10may identify the location of the first POI based on the post information, display the AR image810of the post by superimposing the AR image810on the location of the first POI, and thereby may enable the user to check the post the user has posted.

In this instance, it is illustrated inFIG.8that the AR image810of the post is a picture of the first POI as an example, but the AR image810is not limited thereto. The AR image810of the post may be a review text or a video of the first POI.

Also, according to an embodiment, the AR image810of the post may include information (e.g. a profile photo) about the user who posted.

Further, according to an embodiment, the UE10may display detailed information820on the first POI, so that the user may check the first POI selected as the object of creating the post.

Referring toFIG.9, according to an embodiment, the UE10may display AR images811,812and813of the post based on actual surrounding information. Specifically, the UE10may identify information about the number of floors in each of a plurality of POIs based on location information of POIs included in the post information, and display the AR images811,812and813of the post to correspond to the information about the number of floors in each of the plurality of POIs.

That is, the UE10may display the AR image811of a post about a POI located on a first floor at a bottom, the AR image812of a post about a POI located on a second floor in a middle, and the AR image813of a post about a POI located on a third floor at a top.

Also, according to an embodiment, the UE10may transmit the post information to an external device to share the post information on the first POI with the external device. For instance, the UE10may transmit the post information to an SNS server or a terminal of an SNS user. In addition, the UE10may check a terminal of a user who subscribed to a service through the server20, and transmit the post information to the terminal of the user who subscribed to the service.

Referring toFIG.10, according to an embodiment, the UE10may display a list1010of posts about the POI based on a user's input to an AR icon corresponding to the POI.

In this instance, each of the posts on the list1010about the POI may be provided as a photo, a video or a text, and may be arranged according to a posted order.

Hereinafter, a control method of the UE10according to an embodiment of the disclosure is described in detail. The UE10according to an embodiment described above may be used in the control method of the UE10. Accordingly, the description with reference toFIGS.1to10may be equally applied to the control method of the UE10.

FIG.11is a flowchart illustrating operations of adding a POI to a scrap list in a control method of the UE10according to an embodiment.

Referring toFIG.11, according to an embodiment, the UE10may identify a POI located in a shooting direction of the camera110(1110), and display an AR icon corresponding to the POI by superimposing the AR icon on an image photographed by the camera110(1120).

When receiving a user's input to the AR icon (Yes in operation1130) or a user's input to a scrap button (Yes in operation1140), the UE10may add information on the POI to a scrap list (1150).

That is, the UE10may display the scrap button325, and add identification information of the POI to the scrap list based on the user's input to the scrap button325. Through the operation above, a user may add the identification information of the POI to the scrap list by single tapping on the scrap button325, and thereby may store the identification information of the POI more conveniently.

Also, the UE10may add the identification information of the POI to the scrap list based on a user's input to the AR icon310. For example, the user's input to the AR icon310may be double tapping the AR icon310. Through the operation above, the user may add the identification information of the POI to the scrap list by simple tapping on the AR icon310, and thereby may store the identification information of the POI more conveniently.

FIG.12is a flowchart illustrating operations of identifying a post about a POI in a control method of the UE10according to an embodiment.

Referring toFIG.12, according to an embodiment, the UE10may identify a POI located in a shooting direction of the camera110(1210), and display an AR icon corresponding to the POI by superimposing the AR icon on an image photographed by the camera110(1220).

According to an embodiment, when receiving a user's input to a create post button (a posting menu)333(Yes in operation1230), the UE10may display the list710of surrounding POIs located in the vicinity of the UE10(1240).

In this instance, when receiving a user's selection of a first POI which is subject to creating a post from the list710of the surrounding POIs (Yes in operation1250), and receiving a post about the first POI from the user (Yes in operation1260), the UE10may transmit post information to the server20(1270). Here, the post information includes an AR image of the post and location information of the first POI.

That is, when receiving the post about the first POI which is subject to creating a post, the UE10may identify the post information including the AR image of the post and the location information of the first POI, and transmit the identified post information to the server20. In this instance, the AR image of the post may be provided as various types of posts such as a photo, a video, text, and the like. In addition, the server20may add the post information received from the UE10to a post database of the first POI, and thereby may enable other users to see the post information.

As is apparent from the above, according to the embodiment of the disclosure, the UE and the control method for the same can provide a user interface to enable information on a POI to be more conveniently added to a scrap list, and provide various types of posts about the POI.

Embodiments can thus be implemented through computer readable code/instructions in/on a medium, e.g., a non-transitory computer readable medium, to control at least one processing element to implement any above described exemplary embodiment. The medium can correspond to any medium/media permitting the storing and/or transmission of the computer readable code.

The computer-readable code can be recorded on a medium or transmitted through the Internet. The medium may include Read Only Memory (ROM), Random Access Memory (RAM), magnetic tapes, magnetic disks, flash memories, and an optical recording medium.