Display apparatus and input method thereof

A display apparatus includes a display, a communication module that communicates with an input device, and a processor. The processor is configured to add pattern information to a first image by changing a pixel value of at least one of a plurality of pixels included in the first image, to display the first image, to which the pattern information is added, on the display, to receive a second image captured by the input device from the input device via the communication module, and to identify input coordinates of the input device or a function to be executed by the display apparatus based on pattern information included in the received second image.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims priority from Korean Patent Application No. 10-2016-0120279, filed on Sep. 20, 2016 in the Korean Intellectual Property Office, the entire disclosure of which is hereby incorporated by reference.

TECHNICAL FIELD

Exemplary embodiments relate to a display apparatus which receives a user input and an input method thereof.

BACKGROUND

With the development of digital technologies, various types of electronic products are being developed and distributed. In recent years, a display apparatus such as a television (TV), a monitor, a smartphone, or the like has been increasingly distributed.

A touch screen that functions as an input device and enables input by using a finger or a pen is being applied to the display apparatus. The touch screen may use various schemes such as a resistive film scheme, an infrared scheme, a capacitive scheme, and the like.

SUMMARY

The touch screen that uses the various schemes needs a separate hardware device to recognize a touch, thereby increasing the thickness and the manufacturing cost of a display apparatus.

Aspects of the present disclosure address at least the above-mentioned problems and/or disadvantages and provide at least the advantages described below. Accordingly, an aspect of the present disclosure provides a display apparatus that enables a touch input to be made via an input device even though the display apparatus does not include a separate hardware device, and an input method thereof.

Another aspect of the present disclosure provides a display apparatus that enables a touch input to be made via an input device by adding pattern information to an image itself without degrading the quality of an image displayed in a display, not generating a separate image frame to which pattern information is added, and displaying pattern information on the display, and an input method thereof.

In accordance with an aspect of one or more exemplary embodiments, a display apparatus includes a display, a communication module that communicates with an input device, and a processor. The processor is configured to add first pattern information to a first image by changing a pixel value of at least one of a plurality of pixels included in the first image, to display the first image, to which the first pattern information has been added, on the display, to receive a second image captured by the input device from the input device via the communication module, and to identify input coordinates of the input device or a function to be executed by the display apparatus based on second pattern information included in the received second image.

In accordance with an aspect of one or more exemplary embodiments, an input method of a display apparatus includes adding first pattern information to a first image by changing a pixel value of at least one of a plurality of pixels included in the first image, displaying the first image, to which the first pattern information has been added, in a display, receiving a second image captured by an input device from the input device via a communication module, and identifying input coordinates of the input device or a function to be executed by the display apparatus based on second pattern information included in the received second image.

In accordance with an aspect of one or more exemplary embodiments, a non-transitory computer-readable recording medium has recorded thereon a program performing a method that includes adding first pattern information to a first image by changing a pixel value of at least one of a plurality of pixels included in the first image, displaying the first image, to which the pattern information has been added, in a display, receiving a second image captured by an input device from the input device via a communication module, and identifying input coordinates of the input device or a function to be executed by the display apparatus based on second pattern information included in the received second image.

Other aspects, advantages, and salient features will become apparent to persons having ordinary in the art from the following detailed description, which, taken in conjunction with the annexed drawings, discloses various exemplary embodiments of the present disclosure.

According to various exemplary embodiments, it may be possible to make a touch input via an input device even though a display apparatus does not include a separate hardware device, thereby decreasing the thickness and the manufacturing cost of the display apparatus.

According to various exemplary embodiments, it may be possible to make a touch input via an input device without degrading the quality of an image displayed in a display.

DETAILED DESCRIPTION

Hereinafter, various exemplary embodiments may be described with reference to accompanying drawings. Accordingly, those of ordinary skill in the art will recognize that modification, equivalent, and/or alternative on the various exemplary embodiments described herein can be variably made without departing from the scope and spirit of the present inventive concept. With regard to description of drawings, similar elements may be marked by similar reference numerals.

The terms, such as “first”, “second”, and the like, as used in the present disclosure, may be used to refer to various elements regardless of the order and/or the priority and to distinguish the relevant elements from other elements, but do not limit the elements. For example, “a first user device” and “a second user device” indicate different user devices regardless of the order or priority. For example, without departing the scope of the present disclosure, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element.

Terms used in the present disclosure are used to describe exemplary embodiments and are not intended to limit the scope of the present inventive concept. The terms of a singular form may include plural forms unless otherwise specified. All the terms used herein, which include technical or scientific terms, may have the same meaning that is generally understood by a person having ordinary skill in the art. It will be further understood that terms, which are defined in a dictionary and commonly used, should also be interpreted as is customary in the relevant related art and not in an idealized or overly formal unless expressly so defined in various exemplary embodiments. In some cases, even if terms are terms which are defined in the present disclosure, they may not be interpreted to exclude exemplary embodiments.

FIG. 1is a block diagram illustrating a configuration of a display system, according to various exemplary embodiments.

Referring toFIG. 1, a display system1000may include a display apparatus100and an input device200.

According to an exemplary embodiment, the display apparatus100and the input device200may be connected with each other by wire or wirelessly. For example, the display apparatus100and the input device200may be connected with each other via a short range wireless communication interface (e.g., Bluetooth, near field communication (NFC), wireless-fidelity (Wi-Fi), or the like).

According to an exemplary embodiment, the display apparatus100may display an image (e.g., a video image) in a display. According to an exemplary embodiment, the display apparatus100may display an image received from an external device or an image stored in an internal memory. For example, the display apparatus100may receive broadcast contents from a broadcast station via a broadcast network or may receive web contents from a web server via an Internet network. For another example, after being connected with the external device via a wired communication interface (e.g., high definition multimedia interface (HDMI) or digital video/visual interactive (DVI), a video graphics array (VGA), or the like) or a short range wireless communication interface (e.g., Bluetooth, NFC, Wi-Fi, or the like), the display apparatus100may receive an image from the external device.

According to an exemplary embodiment, the display apparatus100may add pattern information to the image displayed on the display. According to an exemplary embodiment, the display apparatus100may add pattern information to the image by changing a pixel value of at least one of a plurality of pixels included in the image. For example, the pattern information may include coordinate information for sensing input coordinates of the input device200. For example, the pattern information may include function information that indicates information about a function (e.g., the execution of an application, the movement to a specific website, the outputting of specific sound, or the like) to be executed by the display apparatus100.

According to an exemplary embodiment, the display apparatus100may be implemented with any of various devices, each of which receives contents from the external device, such as a television (TV), a desktop personal computer (PC), a notebook PC, a smartphone, a tablet PC, a monitor, an electronic picture frame, or the like, in order to display the contents.

According to an exemplary embodiment, the input device200may make a touch input on the display of the display apparatus100. According to an exemplary embodiment, the input device200may capture an image of an area, where the touch input is made, by using a camera.

According to an exemplary embodiment, the display apparatus100or the input device200may identify coordinates of the touch input by using the pattern information included in the image captured by the input device200. For example, the input device200may transmit the captured image to the display apparatus100, and the display apparatus100may identify the input coordinates of the input device200by using the pattern information included in the image received from the input device200. As another example, the input device200may detect the pattern information included in the captured image and may transmit the detected pattern information to the display apparatus100. As another example, the input device200may identify input coordinates (or function information) by using the pattern information included in the captured image, and may transmit information (or function information) about the identified input coordinates to the display apparatus100.

The display system1000according to various exemplary embodiments may enable the input device200to make the touch input even though the display apparatus100does not include a separate hardware device, such as a touch panel, by adding the pattern information to an image, which the display apparatus100displays, to identify the input coordinates by using the pattern information. The display system1000according to various exemplary embodiments may add the pattern information to the image itself to be displayed, without generating a separate image frame to which the pattern information is added, in order to display the pattern information on the display.

FIG. 2is a diagram illustrating an input method of a display system, according to an exemplary embodiment.

According to an exemplary embodiment, in operation21, the display apparatus100may add pattern information to an image. According to an exemplary embodiment, the display apparatus100may add pattern information to the image by changing a pixel value of at least one of a plurality of pixels included in the image. According to an exemplary embodiment, the display apparatus100may add the pattern information to at least a part of a plurality of image frames.

According to an exemplary embodiment, in operation22, the display apparatus100may display the image, to which the pattern information has been added, in a display. For example, the display apparatus100may display a plurality of image frames at a predetermined frame rate.

According to an exemplary embodiment, in operation23, the input device200may capture the image displayed on the display. For example, if the input device200contacts the display or the input device200is placed within a specified distance from the display, the input device200may capture a part of the image displayed on the display.

According to an exemplary embodiment, in operation24, the input device200may transmit the captured image to the display apparatus100. For example, the input device200may transmit the captured image to the display apparatus100by wire or wirelessly. According to another exemplary embodiment, the input device200may detect the pattern information from the captured image and may transmit the detected pattern information to the display apparatus100. For example, the input device200may transmit only information (e.g., a position of a pixel including the pattern information) about a pixel, which includes the pattern information, from among pixels included in the captured image to the display apparatus100.

According to an exemplary embodiment, in operation25, the display apparatus100may identify the input coordinates (or function information) of the input device200by using the pattern information included in the image received from the input device200. For example, the display apparatus100may identify the input coordinates of the input device200based on a position of the pattern information included in the received image. As another example, the display apparatus100may identify the input coordinates (or function information) of the input device200by using the pattern information received from the input device200.

According to an exemplary embodiment, in operation26, the display apparatus100may perform an operation that corresponds to the identified input coordinates (or function information). For example, the display apparatus100may display a user interface that corresponds to the input coordinates (or function information), or may perform a function of the user interface that corresponds to the input coordinates (or function information). For another example, the display apparatus100may perform an application that corresponds to the input coordinates (or function information). As another example, the display apparatus100may display a written image that corresponds to the input coordinates (or function information) or may change the image displayed on the display.

FIG. 3is a diagram illustrating an input method of a display system, according to an exemplary embodiment.

According to an exemplary embodiment, in operation31, the display apparatus100may add pattern information to an image. According to an exemplary embodiment, the display apparatus100may add pattern information to the image by changing a pixel value of at least one of a plurality of pixels included in the image. According to an exemplary embodiment, the display apparatus100may add the pattern information to at least a subset of a plurality of image frames.

According to an exemplary embodiment, in operation32, the display apparatus100may display the image, to which the pattern information has been added, in a display. For example, the display apparatus100may display a plurality of image frames at a predetermined frame rate.

According to an exemplary embodiment, in operation33, the input device200may capture the image displayed on the display. For example, if the input device200contacts the display or the input device200is positioned within a predetermined distance from the display, the input device200may capture a part of the image displayed on the display.

According to an exemplary embodiment, in operation34, the input device200may identify input coordinates (or function information) of the input device200by using the pattern information included in the captured image. For example, the display apparatus100may identify the input coordinates (or function information) of the input device200based on a position of the pattern information included in the received image.

According to an exemplary embodiment, in operation35, the input device200may transmit information about the input coordinates (or function information) to the display apparatus100. For example, the input device200may transmit the information about the input coordinates (or function information) to the display apparatus100by wire or wirelessly.

According to an exemplary embodiment, in operation36, the display apparatus100may perform an operation that corresponds to the input coordinates (or function information) received from the input device200. For example, the display apparatus100may display a user interface that corresponds to the input coordinates (or function information), or may perform a function of the user interface that corresponds to the input coordinates (or function information). As another example, the display apparatus100may perform an application that corresponds to the input coordinates (or function information). For another example, the display apparatus100may display a written image that corresponds to the input coordinates (or function information), or may change the image displayed on the display.

FIG. 4is a block diagram illustrating a configuration of a display apparatus, according to various exemplary embodiments.

Referring toFIG. 4, the display apparatus100may include a communication module110, a display120, a memory130, and a processor140.

According to an exemplary embodiment, the communication module110(or a communication circuit) may communicate with an external device. For example, the communication module110may receive an image (e.g., a video image) from the external device. According to an exemplary embodiment, the communication module110may communicate with the input device200. For example, the communication module110may receive an image captured by the input device200, pattern information, and/or information about input coordinates from the input device200. According to an exemplary embodiment, the communication module110may include either or both of a wired communication module and a wireless communication module.

According to an exemplary embodiment, the display120may display the image. According to an exemplary embodiment, the display120may display an image to which the pattern information has been added. For example, the display120may display a plurality of image frames at a predetermined frame rate, and at least a subset of a plurality of image frames displayed on the display120may include the pattern information.

According to an exemplary embodiment, the memory130may store the pattern information. For example, the pattern information stored in the memory130may include the number of pieces of pattern information and information about an added position. According to an exemplary embodiment, the memory130may store the image (e.g., a video image). For example, the memory130may include a nonvolatile memory such as a flash memory or a hard disk.

According to an exemplary embodiment, the processor140may control overall operations of the display apparatus100. For example, the processor140may add the pattern information to the image according to various exemplary embodiments by controlling each of the communication module110, the display120, and the memory130.

According to an exemplary embodiment, the display apparatus100may include at least one processor140. For example, the display apparatus100may include a plurality of the processors140, each of which executes at least one function. According to an exemplary embodiment, the processor140may be implemented with a system on chip (SoC) that includes any of a central processing unit (CPU), a graphic processing unit (GPU), a memory, and the like.

According to an exemplary embodiment, the processor140may display the image on the display120. For example, the processor140may display the image, which is received from the external device via the communication module110, or the image stored in the memory130on the display120.

According to an exemplary embodiment, the processor140may add the pattern information to the image displayed on the display120. For example, the processor140may add the pattern information to the image based on the pattern information stored in the memory130. According to an exemplary embodiment, the processor140may add the pattern information to at least a subset of a plurality of image frames displayed on the display120. For example, the processor140may add the pattern information to all the plurality of image frames. For another example, the processor140may periodically add the pattern information to a subset (e.g., one of three image frames) of the plurality of image frames.

According to an exemplary embodiment, if a pattern adding function is turned on, the processor140may add the pattern information to the image. According to an exemplary embodiment, the processor140may turn on/off the pattern adding function based on a user input received via an input module (not illustrated). According to an exemplary embodiment, in a state where the pattern adding function is turned off, if an image (or pattern information) displayed on the display120or coordinate information is received from the input device200, the processor140may turn on the pattern adding function. For example, in a state where the pattern adding function is turned off, if the image (or pattern information) or the coordinate information is received from the input device200, the processor140may determine that a user is operating the input device200, and thus may turn on the pattern adding function. According to an exemplary embodiment, in a state where the pattern adding function is turned on, if the image (or pattern information) or the coordinate information is not received from the input device200during a predetermined time interval (e.g., three seconds), the processor140may turn off the pattern adding function.

According to an exemplary embodiment, the processor140may add pattern information to the image by changing a pixel value of at least one of a plurality of pixels included in an image. According to an exemplary embodiment, the processor140may divide the image into a plurality of pattern blocks, each of which includes a respective plurality of pixels. According to an exemplary embodiment, the processor140may add unique pattern information to each respective pattern block of the plurality of pattern blocks. For example, the processor140may add the pattern information to at least one of a plurality of pixels included in one pattern block, and the positions and the numbers of pixels to which the pattern information is added may be different, in each of the plurality of pattern blocks.

According to an exemplary embodiment, when changing a pixel value of a pixel included in the image, the processor140may change the pixel value from an original pixel value to a pixel value that falls within a predetermined range. As such, while the change in an original image is minimized, a pattern may be added to the image.

Hereinafter, various exemplary embodiments in which pattern information is added to an image will be described below with reference toFIGS. 5, 6A, 6B, 7, and 8.

FIG. 5is a diagram illustrating a method of adding pattern information to an image, according to an exemplary embodiment.

Image510ofFIG. 5may indicate an image prior to an up-scale operation (i.e., an operation to increase the size of the image by a scale factor). Image520ofFIG. 5may indicate the up-scaled image. Image530ofFIG. 5may indicate an image to which the pattern information is added.

According to an exemplary embodiment, the processor140may scale up an image, i.e., increase the size of the image by a scale factor. For example, the processor140may scale up an image from an image of a full-HD (FHD) resolution (1920×1080) to an image of an ultra-HD (UHD) resolution (3840×2160). Referring to image515obtained by enlarging one pattern block included in image510ofFIG. 5, prior to the up-scaling operation, the one pattern block may include 3×3 pixels. Referring to image525obtained by enlarging one pattern block included in image520ofFIG. 5, after the up-scaling operation, the one pattern block may include 6×6 pixels.

According to an exemplary embodiment, the processor140may scale up an image by generating additional pixels, each of which has a same pixel value as an adjacent one of a plurality of pixels, at a periphery of the plurality of pixels included in the image. For example, referring to image525ofFIG. 5, three pixels having a same respective pixel value may be additionally generated with respect to each of nine pixels included in image515ofFIG. 5.

According to an exemplary embodiment, the processor140may add the pattern information to the image by changing a pixel value of a subset of a plurality of pixels having the same pixel value. For example, the processor140may decrease or increase the pixel value of a subset of the plurality of pixels having the same pixel value by a predetermined value. Referring to image535obtained by enlarging the pattern block included in image530ofFIG. 5, the value of a respective subset of the plurality of pixels having pixel values a, b, e, f, and h may be changed to the values of a′, b′, e′, f′, and h′, respectively.

According to an exemplary embodiment, the pattern information may include reference position information, first axis (e.g., x-axis) coordinate information, and second axis (e.g., y-axis) coordinate information. According to an exemplary embodiment, the processor140may add the reference position information such that the reference position information is distinguished from other pattern information. For example, the processor140may change pixel values of two of the plurality of pixels having the same pixel value in order to add the reference position information, and may change a pixel value of one of the plurality of pixels having the same pixel value in order to add coordinate information (e.g., the first axis coordinate information and the second axis coordinate information). Referring to image535ofFIG. 5, pixel values of two of four pixels having a pixel value of “a” may be changed to a′. The two pixels for which the respective value changed to a′ may include the reference position information. A value of one of four pixels respectively having pixel values of b, e, f and h may be changed to b′, e′, f′ and h′, respectively. Pixels for which the respective value is changed to b′, e′, f′ or h′ may include the coordinate information.

FIG. 6Ais a diagram illustrating a type of a pixel value, according to an exemplary embodiment.

According to an exemplary embodiment, a pixel value of a pixel included in an image may include a normal pixel value that does not include pattern information and a pattern pixel value that includes the pattern information.

Referring toFIG. 6A, in the case where the pixel value is represented by 8 bits, for example, one pixel may have a pixel value of 0 to 255. The pixel value of 0 to 255 may alternately include a normal pixel value group and a pattern pixel value group. For example, pixel values of 0 to 11 may indicate the normal pixel values. Pixel values of 12 to 15 may indicate the pattern pixel values. Pixel values of 16 to 27 may indicate the normal pixel values. Pixel values of 28 to 31 may indicate the pattern pixel values.

According to an exemplary embodiment, the normal pixel value and the pattern pixel value may be determined by a value of at least one bit from among eight bits that indicate the pixel value. For example, if both values of the third bit and the fourth bit are “1”, the pixel value may be set to the pattern pixel value. If a value of at least one of the third bit and the fourth bit is “0”, the pixel value may be set to the normal pixel value. For another example, if the value of the third bit is “1”, the pixel value may be the pattern pixel value. If the value of the third bit is “0”, the pixel value may be the normal pixel value. The normal pixel value and the pattern pixel value may be set in advance by a manufacturer when the display apparatus100is manufactured, or may be set based on a user input.

FIG. 6Bis a diagram illustrating a method of adding pattern information to an image, according to an exemplary embodiment.

Image610ofFIG. 6Bmay indicate an image before the pattern information is added, and image620ofFIG. 6Bmay indicate an image to which the pattern information has been added.

Referring to image615obtained by enlarging a pattern block included in image610ofFIG. 6B, a plurality of pixels included in the pattern block may have a normal pixel value or a pattern pixel value. Information about the number and the positions of pixels having the pattern pixel value of pixels included in image615may be different from the pattern information stored in the memory130.

According to an exemplary embodiment, the processor140may divide a plurality of pixels included in the image into a first subset of pattern pixels that include the pattern information and a second subset of normal pixels that do not include the pattern information. For example, the processor140may divide the plurality of pixels included in the image into the first subset of pattern pixels and the second subset of normal pixels based on the pattern information stored in the memory130. Referring to image615ofFIG. 6B, the processor140may classify p16, p22, p23, p43, p44, p51, and p53of a plurality of pixels included in image615as belonging to the first subset of pattern pixels, and may classify the remaining pixels as belonging to the second subset of normal pixels.

According to an exemplary embodiment, the processor140may change a pixel value of at least one of a plurality of pixels included in the image based on the division result. For example, if the pixel value of the pixel divided into the first subset of pattern pixels is the normal pixel value, the processor140may change the normal pixel value to the pattern pixel value. If the pixel value of the pixel divided into the second subset of normal pixels is the pattern pixel value, the processor140may change the pattern pixel value to the normal pixel value. Referring to image625obtained by enlarging a pattern block included in image620ofFIG. 6B, the processor140may change a pixel value of each of pixels p11, p36, and p61from the normal pixel value to the pattern pixel value. The processor140may change a pixel value of each of pixels p23, p43, and p51from the pattern pixel value to the normal pixel value.

According to an exemplary embodiment, when changing a pixel value of a pixel included in the image, the processor140may change the pixel value from an original pixel value to a pixel value within a predetermined range (e.g., such that each of the original pixel value and the changed pixel value is less than a predetermined value). For example, when changing the pixel value of a pattern pixel from the normal pixel value to the pattern pixel value, the processor140may change the pixel value to a pattern pixel value (or a pattern pixel value included in a period of a pattern pixel value closest to the original pixel value) closest to the original pixel value. When changing the pixel value of a normal pixel from the pattern pixel value to the normal pixel value, the processor140may change the pixel value to a normal pixel value (or a normal pixel value included in a period of a normal pixel value closest to the original pixel value) closest to the original pixel value.

FIG. 7is a diagram illustrating a method of adding pattern information to an image, according to an exemplary embodiment.

According to an exemplary embodiment, each of a plurality of pixels included in an image710may include a respective plurality of sub-pixels. For example, each of the plurality of pixels may include a first sub-pixel for displaying a first color (e.g., red), a second sub-pixel for displaying a second color (e.g., green), and a third sub-pixel for displaying a third color (e.g., blue).

According to an exemplary embodiment, the pattern information may include reference position information, first axis (e.g., x-axis) coordinate information, and second axis (e.g., y-axis) coordinate information. According to an exemplary embodiment, the processor140may add different pattern information for each sub-pixel. For example, the processor140may add the reference position information to the first sub-pixel, may add the first axis coordinate information to the second sub-pixel, and may add the second axis coordinate information to the third sub-pixel. According to an exemplary embodiment, the processor140may change only the pixel value of a sub-pixel, which corresponds to a type of pattern information to be added to a pixel, from among a plurality of sub-pixels included in the pixel. For example, when adding the reference position information, the processor140may change only the pixel value of the first sub-pixel of the plurality of sub-pixels included in the pixel. As another example, when adding the first axis coordinate information, the processor140may change only the pixel value of the second sub-pixel of the plurality of sub-pixels included in the pixel.

FIG. 8is a diagram a method of compensating an image in response to adding pattern information, according to an exemplary embodiment.

Image810ofFIG. 8may indicate an image prior to an addition of pattern information. Image820ofFIG. 8may indicate an image to which the pattern information has been added. Image830ofFIG. 8may indicate an image compensated in response to the adding of the pattern information.

According to an exemplary embodiment, the processor140may identify a pixel group that is recognizable by a user as one pixel. For example, referring to image810, image820, and image830ofFIG. 8, the processor140may determine four pixels as the pixel group that is recognizable by the user as one pixel.

For example, the pixel group that is recognizable by the user as one pixel may be determined based on any or all of the viewing distance of the user, the visual acuity of the user, the size (e.g., width and height) of the display120, and the resolution of the display120. The size and the resolution of the display120may not vary based on the user, because the size and the resolution of the display120are unique characteristics of the display apparatus100, but the viewing distance and the visual acuity of the user may vary based on the user. According to an exemplary embodiment, the processor140may obtain the viewing distance and the visual acuity of the user through any of various methods. For example, the memory130may store information about the pixel group, which is calculated based on a general viewing distance and visual acuity of the user. The processor140may identify the pixel group that is recognizable by the user as one pixel based on the information about the pixel group stored in the memory130. For another example, the display apparatus100may receive information about the viewing distance and the visual acuity from the user via an input module (not illustrated). The processor140may calculate the pixel group that is recognizable by the user as one pixel, based on the information received via the input module. For another example, the display apparatus100may include a distance sensor (not illustrated) that senses the viewing distance of the user. The processor140may calculate the pixel group that is recognizable by the user as one pixel, based on the viewing distance of the user sensed by the distance sensor.

According to an exemplary embodiment, if a pixel value of a subset of a plurality of pixels included in the pixel group is changed, the processor140may compensate a change in an image due to the adding of the pattern information by changing a pixel value of at least one of the remaining pixels such that the pixel value corresponds to the changed pixel value. For example, referring to image820ofFIG. 8, the pixel value of each of two pixels included in the pixel group having a pixel value of “95” may be changed to “80”. The pixel value of one pixel included in the pixel group having a pixel value of “97” may be changed to “82”. The pixel value of one pixel included in the pixel group having a pixel value of “124” may be changed to “109”. The pixel value of one pixel included in the pixel group having a pixel value of “113” may be changed to “98”. The pixel value of one pixel included in the pixel group having a pixel value of “184” may be changed to “169”. Referring to image830ofFIG. 8, the pixel value of each of the remaining two pixels included in the pixel group having a pixel value of “95” may be changed to “110”. The pixel value of each of the remaining three pixels included in the pixel group having a pixel value of “97” may be changed to “102”. The pixel value of each of the remaining three pixels included in the pixel group having a pixel value of “124” may be changed to “129”. The pixel value of each of the remaining three pixels included in the pixel group having a pixel value of “113” may be changed to “118”. The pixel value of each of the remaining three pixels included in the pixel group having a pixel value of “184” may be changed to “189”.

According to an exemplary embodiment, as in an exemplary embodiment described above with respect toFIG. 7, in the case where different pattern information is added for each sub-pixel, a pixel value may be compensated with respect to only a sub-pixel that corresponds to the sub-pixel to which the pattern information is added.

Referring to image810and image830ofFIG. 8, the sum of pixel values of a plurality of pixels included in one pixel group may be the same. According to an exemplary embodiment described with reference toFIG. 8, a pixel value of the pixel group that is recognizable by the user as one pixel may not be changed. As such, even though the pattern information is added to an image, the image of the same quality may be provided to the user.

The case where the pixel value of each of all the remaining pixels is changed is described inFIG. 8to compensate the changed pixel value based on the adding of the pattern information. However, only the pixel value of a part of the remaining pixels may be changed to correspond to the changed pixel value. In addition, the case where pixel values of a plurality of pixels included in the pixel group are the same as each other is exemplified inFIG. 8. However, even though the pixel values of a plurality of pixels included in the pixel group are at least partly different, the above-described compensating method may be applied in the same manner.

According to an exemplary embodiment, the processor140may display an image, to which the pattern information is added, on the display120. For example, the processor140may display a plurality of image frames, to each of which the pattern information has been added, at a predetermined frame rate.

According to an exemplary embodiment, the processor140may receive an image captured by the input device200or pattern information detected from the captured image, from the input device200via the communication module110. For example, the processor140may receive an image, within which the pattern information is included, or the pattern information via the communication module110.

According to an exemplary embodiment, the processor140may identify input coordinates (or a function to be executed by the display apparatus100) of the input device200based on the image received from the input device200or the received pattern information. According to an exemplary embodiment, the processor140may identify a pixel (or a pattern pixel), which includes the pattern information, from among a plurality of pixels included in the received image. For example, the processor140may identify a pixel, the pixel value of which is different from each of pixel values of neighboring pixels, from among the plurality of pixels included in the received image. As another example, the processor140may identify a pixel, which has the pattern pixel value, from among the plurality of pixels included in the received image. According to an exemplary embodiment, the processor140may identify the input coordinates of the input device200based on the number and positions of pixels that include the pattern information. For example, the processor140may compare the pixel that includes the pattern information with the pattern information stored in the memory130in order to identify the input coordinates of the input device200. As another example, the processor140may identify a pattern block by using the reference position information, and may calculate the input coordinates of the input device200based on the number and the positions of pattern pixels included in the pattern block.

According to an exemplary embodiment, if the input coordinates of the input device200are identified based on the image or the pattern information received from the input device200, the processor140may add the pattern information to a partial area, which corresponds to the identified input coordinates, of an image to be displayed on the display120. For example, the processor140may add the pattern information to only an area of a predetermined size (e.g., horizontal 100 pixels×vertical 100 pixels), which includes the identified input coordinates. In the case where a user touches a specific area of the display120by using the input device200such that a first coordinate is recognized, since the possibility that a second coordinate to be recognized afterwards is to be recognized at a periphery of the first coordinate is very high, the pattern information may be added to only a peripheral area of input coordinates, thereby decreasing a throughput of the processor140.

According to an exemplary embodiment, in a state where the pattern adding function is turned on, if the input coordinates are not identifiable based on an image received from the input device200or the pattern information received from the input device200, for example, the processor140may increase a variation of the pixel value. For example, the case where a pixel that includes the pattern information is not normally recognized due to a capture condition (e.g., ambient brightness, camera performance, or the like) of the input device200may occur. As such, if the input coordinates are not identified, the processor140may increase the variation of the pixel value, thereby improving the recognition rate of the pixel that includes the pattern information.

According to an exemplary embodiment, if the input coordinates are not identifiable based on an image received from the input device200or the pattern information received from the input device200, the processor140may increase the size of each of a normal pixel value group and a pattern pixel value group. For example, referring toFIG. 6A, the size of the normal pixel value group may be “12”, and the size of the pattern pixel value group may be “4”. If the input coordinates are not identifiable, the processor140may increase the size of the normal pixel value group to “24” and may increase the size of the pattern pixel value group to “8”.

According to an exemplary embodiment, the processor140may receive information about the input coordinates from the input device200via the communication module110. For example, the input device200may capture an image displayed on the display120and may identify the input coordinates based on the pattern information included in the captured image. According to an exemplary embodiment, the input device200may identify the input coordinates in the same manner as the display apparatus100.

According to an exemplary embodiment, if identifying the input coordinates or if the information about the input coordinates is received from the input device200, the processor140may perform an operation that corresponds to the input coordinates. For example, the processor140may display a user interface that corresponds to the input coordinates on the display120, or may perform a function of the user interface that corresponds to the input coordinates. As another example, the processor140may execute an application that corresponds to the input coordinates. As another example, the processor140may display a written image that corresponds to the input coordinates on the display120or may change an image displayed on the display120.

FIG. 9is a flowchart illustrating an input method of a display apparatus, according to various exemplary embodiments.

The flowchart illustrated inFIG. 9may include operations that the above-described display apparatus100processes. Even though omitted below, details about the display apparatus100described with reference toFIGS. 1 to 8may be applied to the flowchart illustrated inFIG. 9.

According to an exemplary embodiment, in operation910, the display apparatus100may add pattern information to an image. According to an exemplary embodiment, the display apparatus100may add the pattern information to at least a subset of a plurality of image frames. According to an exemplary embodiment, the display apparatus100may change a pixel value of at least one of a plurality of pixels included in the image in order to add the pattern information to the image. According to an exemplary embodiment, the display apparatus100may divide the image into a plurality of pattern blocks, each of which includes a plurality of pixels, and may add unique pattern information to each of the plurality of pattern blocks. According to an exemplary embodiment, when changing a value of a pixel included in the image, the display apparatus100may change the value from an original pixel value to a pixel value that falls within a predetermined range.

According to an exemplary embodiment, the display apparatus100may scale up an image (i.e., increase a size of an image) by generating pixels, each of which has a same pixel value as each of a plurality of pixels, at a periphery of a plurality of pixels included in the image. According to an exemplary embodiment, the display apparatus100may add the pattern information to the image by changing a pixel value of a subset of a plurality of pixels having the same pixel value.

According to an exemplary embodiment, the display apparatus100may divide a plurality of pixels included in the image into a first subset of pattern pixels, in which the pattern information is included, and a second subset of normal pixels, in which the pattern information is not included. According to an exemplary embodiment, the display apparatus100may change a value of at least one pixel of a plurality of pixels included in the image based on the division result. For example, if the pixel value of the pixel divided into the first subset of pattern pixels is the normal pixel value, the display apparatus100may change the normal pixel value to the pattern pixel value. If the pixel value of the pixel divided into the second subset of normal pixels is the pattern pixel value, the display apparatus100may change the pattern pixel value to the normal pixel value.

According to an exemplary embodiment, the display apparatus100may add different pattern information for each sub-pixel. For example, the display apparatus100may add the reference position information to the first sub-pixel, may add the first axis coordinate information to the second sub-pixel, and may add the second axis coordinate information to the third sub-pixel.

According to an exemplary embodiment, the display apparatus100may identify a pixel group that is recognizable by a user as one pixel. According to an exemplary embodiment, if a pixel value of a subset of a plurality of pixels included in the pixel group is changed, the display apparatus100may compensate a change in an image due to the adding of the pattern information by changing a pixel value of at least one of the remaining pixels such that the pixel value corresponds to the changed pixel value.

According to an exemplary embodiment, in operation920, the display apparatus100may display the image, to which the pattern information has been added, in a display. For example, the display apparatus100may display a plurality of image frames, to each of which the pattern information has been added, at a predetermined frame rate.

According to an exemplary embodiment, in operation930, the display apparatus100may receive the image or the pattern information from the input device200. For example, the input device200may capture a part of an image displayed in a display of the display apparatus100and may transmit the captured image to the display apparatus100. According to another exemplary embodiment, the display apparatus100may receive the pattern information from the input device200. For example, the input device200may detect the pattern information from the captured image and may transmit the detected pattern information to the display apparatus100.

According to an exemplary embodiment, in operation940, the display apparatus100may identify the input coordinates (or function information) of the input device200based on the received image. According to an exemplary embodiment, the display apparatus100may identify a pixel (or a pattern pixel), which includes the pattern information, from among a plurality of pixels included in the received image. According to an exemplary embodiment, the display apparatus100may identify the input coordinates (or function information) of the input device200based on the number and positions of pixels that include the pattern information. According to another exemplary embodiment, the display apparatus100may identify the input coordinates (or function information) of the input device200based on the pattern information received from the input device200.

According to an exemplary embodiment, if the input coordinates are not identifiable based on the image or the pattern information received from the input device200, the display apparatus100may increase a variation of the pixel value. According to an exemplary embodiment, if the input coordinates are not identifiable based on the pattern information included in the image received from the input device200, the display apparatus100may increase the size of each of a normal pixel value group and a pattern pixel value group.

According to an exemplary embodiment, in operation950, the display apparatus100may perform an operation that corresponds to the input coordinates (or function information).

FIG. 10is a flowchart illustrating an input method of a display apparatus, according to various exemplary embodiments.

The flowchart illustrated inFIG. 10may include operations that the above-described display apparatus100processes. Even though omitted below, details about the display apparatus100described with reference toFIGS. 1 to 8may be applied to the flowchart illustrated inFIG. 10.

According to an exemplary embodiment, in operation1010, the display apparatus100may add pattern information to an image. According to an exemplary embodiment, the display apparatus100may add the pattern information to at least a subset of a plurality of image frames. According to an exemplary embodiment, the display apparatus100may add pattern information to the image by changing a pixel value of at least one of a plurality of pixels included in the image. According to an exemplary embodiment, the display apparatus100may divide the image into a plurality of pattern blocks, each of which includes a plurality of pixels, and may add unique pattern information to each of the plurality of pattern blocks. According to an exemplary embodiment, when changing a value of a pixel included in the image, the display apparatus100may change the value from an original pixel value to a pixel value that falls within a predetermined range.

According to an exemplary embodiment, the display apparatus100may scale up an image (i.e., increase a size of an image) by generating pixels, each of which has a same pixel value as each of a plurality of pixels, at a periphery of a plurality of pixels included in the image. According to an exemplary embodiment, the display apparatus100may add the pattern information to the image by changing a pixel value of a subset of a plurality of pixels having the same pixel value.

According to an exemplary embodiment, the display apparatus100may divide a plurality of pixels included in the image into a first subset of pattern pixels, in which the pattern information is included, and a second subset of normal pixels, in which the pattern information is not included. According to an exemplary embodiment, the display apparatus100may change a value of at least one pixel of a plurality of pixels included in the image based on the division result. For example, if the pixel value of the pixel classified into the first subset of pattern pixels is the normal pixel value, the display apparatus100may change the normal pixel value to the pattern pixel value. If the pixel value of the pixel classified into the second subset of normal pixels is the pattern pixel value, the display apparatus100may change the pattern pixel value to the normal pixel value.

According to an exemplary embodiment, the display apparatus100may add different pattern information for each sub-pixel. For example, the display apparatus100may add the reference position information to the first sub-pixel, may add the first axis coordinate information to the second sub-pixel, and may add the second axis coordinate information to the third sub-pixel.

According to an exemplary embodiment, the display apparatus100may identify a pixel group that is recognizable by a user as one pixel. According to an exemplary embodiment, if a pixel value of a subset of a plurality of pixels included in the pixel group is changed, the display apparatus100may compensate a change in an image due to the adding of the pattern information by changing a pixel value of at least one of the remaining pixels such that the pixel value corresponds to the changed pixel value.

According to an exemplary embodiment, in operation1020, the display apparatus100may display the image, to which the pattern information has been added, in a display. For example, the display apparatus100may display a plurality of image frames, to each of which the pattern information has been added, at a predetermined frame rate.

According to an exemplary embodiment, in operation1030, the display apparatus100may receive the input coordinates (or function information) from the input device200. For example, the input device200may capture a part of an image displayed in a display of the display apparatus100and may identify the input coordinates (or function information) based on the pattern information included in the captured image.

According to an exemplary embodiment, if the input coordinates are not received from the input device200(or if information indicating that it is impossible to calculate the input coordinates is received), the display apparatus100may increase a variation of the pixel value. According to an exemplary embodiment, if the input coordinates are not received from the input device200(or if the information indicating that it is impossible to calculate the input coordinates is received from the input device200), the display apparatus100may increase the size of each of a normal pixel value group and a pattern pixel value group.

According to an exemplary embodiment, in operation1040, the display apparatus100may perform an operation that corresponds to the input coordinates (or function information).

The term “module” used herein may include a unit, which is implemented with hardware, software, or firmware, and may be interchangeably used with the terms “logic”, “logical block”, “component”, “circuit”, or the like. The “module” may be a minimum unit of an integrated component or a part thereof or may be a minimum unit for performing one or more functions or a part thereof. The “module” may be implemented mechanically or electronically and may include, for example, any of an application-specific IC (ASIC) chip, a field-programmable gate array (FPGA), and a programmable-logic device for performing some operations, which are known or will be developed.

According to various exemplary embodiments, at least a part of an apparatus (e.g., modules or functions thereof) or a method (e.g., operations) may be, for example, implemented by instructions stored in a transitory or non-transitory computer-readable storage medium in the form of a program module. The instruction, when executed by a processor, may cause the processor to perform a function that corresponds to the instruction. The non-transitory computer-readable recording medium may include any of a hard disk, a floppy disk, a magnetic media (e.g., a magnetic tape), an optical media (e.g., a compact disc read only memory (CD-ROM) and a digital versatile disc (DVD), a magneto-optical media (e.g., a floptical disk)), an embedded memory, and the like. The instruction may include codes created by a compiler or codes that are capable of being executed by a computer by using an interpreter. According to various exemplary embodiments, a module or a program module may include at least one of the above elements, or a part of the above elements may be omitted, or other elements may be further included.

According to various exemplary embodiments, operations executed by modules, program modules, or other elements may be executed by a successive method, a parallel method, a repeated method, or a heuristic method, or at least one part of operations may be executed in different sequences or omitted. Alternatively, other operations may be added. While the present disclosure has been shown and described with reference to various exemplary embodiments thereof, it will be understood by persons having ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present disclosure as defined by the appended claims and their equivalents.