Screen position sensing method in multi display system, content configuring method, watermark image generating method for sensing screen position server, and display terminal

A screen position sensing method in a multi display system, a content configuring method, a watermark image generating method for sensing a screen position, a server, and a display terminal are provided. A screen position sensing method in a multi display system performed by a sensing server may include receiving identification information of display terminals from a display server, receiving an image output by each of the display terminals from a camera, and determining a screen position of each of the display terminals based on a watermark included in the image of the display terminals and the received identification information.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application claims the priority benefit of Korean Patent Application No. 10-2015-0123715 filed on Sep. 1, 2015, in the Korean Intellectual Property Office, and Korean Patent Application No. 10-2016-0079943 filed on Jun. 27, 2016, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference for all purposes.

BACKGROUND

One or more example embodiments relate to a screen position sensing method in a multi display system performed by a sensing server, a content configuring method in a multi display system performed by a display server, a watermark generating method for sensing a screen position in a multi display system performed by a display terminal, and the display terminal.

2. Description of Related Art

A multi display system has been utilized for exhibition and artistic representation. A multi display system may be used in, for example, digital signage or other type of electronic board.

A multi display system may be used as an effective alternative when it is difficult to manufacture a large display. Recently, multi display systems have been provided which use a method of two-dimensionally dividing an entire screen of content based on an area of each display, and may use content exclusively for special visual effects.

SUMMARY

An aspect of the present invention is to transmit and display unique information of each display using a digital watermark technique in order to obtain physical information of the display.

Another aspect also provides space information of each display by sensing unique information of each display using a camera based on a digital watermark technique.

According to an aspect, there is provided a screen position sensing method in a multi display system performed by a sensing server, the method including receiving identification information of display terminals from a display server, receiving an image output by each of the display terminals from a camera, and determining a screen position of each of the display terminals based on a watermark included in the image of the display terminals and the received identification information.

The display server may be configured to generate the identification information of each of the display terminals and transmit the generated identification information to the sensing server and the display terminals corresponding to the generated identification information, and each of the display terminals is configured to receive the identification information from the display server and generate the watermark based on the received identification information. The method may further include transmitting the determined screen position to the display server, wherein the display server is configured to transmit content configuration information to the display terminals based on the screen position of each of the display terminals received from the sensing server.

The transmitting may include transmitting identification information of the display terminals and a position of each of the display terminals in an entire display space.

The receiving may include receiving, in real time, an image obtained by photographing the image output by each of the display terminals from the camera.

The determining may include performing reversible watermarking on the image obtained by photographing the image output by each of the display terminals.

The determining may include extracting the watermark based on a performance of the reversible watermarking.

The determining may include determining the screen position of each of the display terminals based on a color of the watermark.

The determining may include adjusting a noise based on the color of the watermark.

The method may further include generating models of the display terminals, wherein the determining may include determining the screen position of each of the display terminals based on the generated models of the display terminals.

According to another aspect, there is provided a content configuring method in a multi display system performed by a display server, the method including generating identification information of display terminals, transmitting the identification information to a sensing server and the display terminals, receiving information on a screen position of each of the display terminals from the sensing server, generating content configuration information on content to be output by the display terminals based on the screen position of each of the display terminals, and transmitting the content configuration information to the display terminals, wherein each of the display terminals is configured to generate a watermark based on the identification information and output an image in which the watermark is embedded, and the sensing server is configured to determine the screen position of each of the display terminals based on the identification information and the watermark embedded in the image output by each of the display terminals.

The method may further include receiving the determined screen position from the sensing server, generating the content configuration information based on the received screen position, and transmitting the generated content configuration information to the display terminals.

The generating may include receiving information on whether each of the display terminals is positioned in an entire display space from each of the display terminals when each of the display terminals is connected to the display server, and generating the identification information of the display terminals based on the information on whether each of the display terminals is positioned in the entire display space.

The receiving may include receiving rotation information and size information of each of the display terminals.

The transmitting of the content configuration information may include transmitting information that controls a screen of each of the display terminals based on the rotation information of each of the display terminals.

According to still another aspect, there is provided a watermark image generating method for sensing a screen position in a multi display system performed by a display terminal, the method including receiving identification information of a display terminal from a display server, and generating a watermark based on the received identification information and outputting an image in which the watermark is embedded, wherein a sensing server is configured to determine a screen position of each of display terminals based on the watermark embedded in the image output by each of the display terminals and the identification information received from the display server.

The method may further include receiving content configuration information generated based on the screen position from the display server, wherein the display server is configured to generate the content configuration information based on the screen position received from the sensing server.

The method may further include outputting content on a screen based on the received content configuration information.

The outputting may include outputting, on a screen, a watermark in a color corresponding to the received identification information.

DETAILED DESCRIPTION

It should be understood, however, that there is no intent to limit this disclosure to the particular exemplary embodiments disclosed. On the contrary, exemplary embodiments are to cover all modifications, equivalents, and alternatives falling within the scope of the exemplary embodiments. Like numbers refer to like elements throughout the description of the figures.

Various exemplary embodiments will now be described more fully with reference to the accompanying drawings in which some exemplary embodiments are shown. In the drawings, the thicknesses of layers and regions are exaggerated for clarity.

FIG. 1is a diagram illustrating a screen position sensing method in a multi display system according to an example embodiment.

Referring toFIG. 1, an entire system includes a sensing server110, a display server120, and display terminals130. As a multi display, the display terminals130may include a first display terminal131to an n-th display terminal132. The sensing server110, the display server120, and the display terminals130may be computing devices including at least one of a memory, a processor, or a data transceiver, but the sensing server110, the display server120, and the display terminals130are not limited thereto. The display terminals130may be display devices, for example, light emitting diode (LED) monitors.

Each of the display terminals130may display identification information using a watermark, the sensing server110may extract watermark configuration information using a camera, and the display server120may combine pieces of space information of each of the display terminals130. Hereinafter, information, for example, space information, identification information, position information, and rotation information, of display terminals also refers to information of each of the display terminals.

The display server120may generate information on the watermark to generate the space information of the display terminals130using the watermark. In this example, the information on the watermark may include a color of the watermark corresponding to the identification information of the display terminals130. The display terminals130may receive the information on the watermark generated by the display server120, generate the watermark, and combine the generated watermark and information on a screen to be output. In this example, the information on the screen may be information on, for example, an image and a video to be output to the screen.

The sensing server110may extract configuration information of the display terminals130by performing reversible watermarking using the camera. The sensing server110may receive, from the camera, an image obtained by photographing the display terminals130or a multi display system by the camera. When the sensing server110is the computing device such as a notebook, the sensing server110may include the camera. The sensing server110may extract reversible watermark information by performing the reversible watermarking on the photographed image. The sensing server110may extract an orientation of each of the display terminals130. The sensing server110may extract the space information of the display terminals130, for example, a position of each of the display terminals130. The sensing server110may transmit the extracted space information.

The display server120may generate a color of a watermark that differently corresponds to a unique identification (ID), as the identification information, of the display terminals130. The display server120may transmit the generated identification information of the different display terminals130or the color of the watermark to each of the display terminals130.

According to the color of the watermark or the received identification information of the display terminals130, the display terminals130may generate the watermark based on the information on the screen of each of the display terminals130. The display terminals130may display the watermark by combining the watermark and an image to be output, and then outputting the watermark with the image to the screen. In this example, the information on the screen of each of the display terminals130may include a size of a screen, a resolution, and a position at which an image is output to the screen.

FIG. 2is a diagram illustrating a screen position sensing method in a multi display system using a camera according to an example embodiment.

Referring toFIG. 2, a sensing server210senses physical information of display terminals230of a multi display system using a camera215. When the display terminals230are positioned in an entire display space, the display terminals230may transmit a message indicating that the display terminals230are currently positioned in the entire display space corresponding to a multi display space through communication with a display server220.

The display server220may generate or store a unique identification (ID) of each of the display terminals230as identification information of the display terminals230. The display server220may transmit, to each of the display terminals230, the unique ID of each of the display terminals230as the identification information of the display terminals230. The display server220may transmit, to the sensing server210, the unique ID of each of the display terminals230as the identification information of the display terminals230.

Each of the display terminals230may generate and configure a watermark based on the unique ID of each of the display terminals230and may embed the watermark in an image that is currently output and displayed. The camera215may transmit, to the sensing server210, information on an image obtained by photographing, in real time, an entire display space corresponding to a multi display space. The information on the photographed image may be a photographed image.

The sensing server210may verify the ID of each of the display terminals230from an image extracted by performing reversible watermarking on the currently photographed image, and determine information on an area of each of the display terminals230, for example, a screen size, a screen position, and rotation information of the display terminals230. The sensing server210may determine a position of each of the display terminals230in the entire display space corresponding to the multi display space, and then transmit the position of each of the display terminals230to the display server220. The display server220may effectively generate or modify content configuration information on content to be output to the display terminals230by utilizing various methods that display the content based on information on the position of each of the display terminals230, and transmit the generated or modified content configuration information to the display terminals230.

FIG. 3is a diagram illustrating a method in which a display terminal generates and embeds a watermark according to an example embodiment.

Referring toFIG. 3, a watermark is generated and embedded in each of display terminals. Each of the display terminals may receive an identification (ID) of each of the display terminals transmitted by a display server and generate a watermark corresponding to the received ID of each of the display terminals. For example, watermarks may be differentiated based on a color, for example, red, yellow, green, or blue.

When the ID of a first display is “1”, a red watermark321corresponding to “1” is generated. When the ID of a second display is “2”, a yellow-green watermark322corresponding to “2” is generated. When the ID of a third display is “3”, a blue-green watermark323corresponding to “3” is generated. When the ID of a fourth display is “7”, a blue watermark324corresponding to “7” is generated.

A predetermined watermark image may be generated. However, a color may be generated based on the ID for detecting forgery. For example, when the ID is “4”, a watermark in a color of which a red-green-blue (RGB) value is (127, 60, 235) may be generated. Here, any one of various colors can be set to correspond to the ID.

The display terminals may embed an image310to be displayed and a watermark image320in a current image based on a watermark embedding method. The display terminals may output and display an image330in which the watermark is embedded. For example, a simple method of embedding a color in a least significant bit (LSB) may be used. Because a viewer is unaware whether the watermark is embedded, the viewer may view current content as usual.

A method of encoding information, for example, steganography, may be used. For example, each of the display terminals may output information corresponding to the ID of each of the display terminals based on the steganography instead of watermarking, such that a user is unaware of the output.

FIG. 4is a diagram illustrating a method in which a sensing server extracts a watermark according to an example embodiment.

Referring toFIG. 4, a watermark is extracted. A sensing server may convert a current image obtained by a camera to a watermark space image410based on a reversible watermarking technique. In this example, a first watermark area411corresponding to a first display may be red, and a second watermark area412corresponding to a second display may be yellow-green. A third watermark area413corresponding to a third display may be blue-green, and a fourth watermark area414corresponding to a fourth display may be blue.

Because an image obtained by the camera is an image from an actual camera, various types of noise may be present in the image. The sensing server may share, with a display server, information on a color generated based on an identification (ID) and a number of watermarks identical to a number of the display terminals. The sensing server may effectively adjust a noise based on information shared with the display server. For example, the sensing server may obtain and generate a sufficient reversible watermark image based on a thresholding method.

Because information on a size of an actual monitor is not included in the sensing server, a refinement process may be performed based on an image obtained based on the thresholding method. In this example, the image may have a color of an area assigned to correspond to an ID of each of the display terminals. The sensing server may extract watermarks420by performing the refinement process. A first watermark421corresponding to the first display is red, and a second watermark422corresponding to the second display may be yellow-green. A third watermark423corresponding to the third display may be blue-green, and a fourth watermark424corresponding to the fourth display may be blue.

FIG. 5is a diagram illustrating a screen position sensing method in a multi display system performed by a sensing server according to an example embodiment.

Referring toFIG. 5, a watermark refinement process is performed. A first watermark area511corresponding to a first display may be red, and a second watermark area512corresponding to a second display may be yellow-green. A third watermark area513corresponding to a third display may be blue-green, and a fourth watermark area514corresponding to a fourth display may be blue. In this example, identifications (IDs) of the first display, the second display, the third display, and the fourth display may be “1”, “2”, “3”, and “7”, respectively.

The sensing server may obtain and generate an image520on which a thresholding process is performed from a watermark space image510in a current image based on color information assigned to the ID of each of the display terminals. Like in the thresholded image520, a noise may exist in an actually obtained or generated image. A first watermark area521corresponding to the first display may be red. Also, a first noise area522, a second noise area523, a third noise area524, and a fourth noise area525may be also red.

The sensing server may set a model of the display terminal or a model of a screen of the display terminal. For example, when the display terminal is a rectangular monitor, the model may be generated as having two parameters based on a size of the display terminal and a degree of rotation of the display terminal. The size of the display terminal may be 16 inches or 24 inches, and the degree of rotation of the display terminal may be 30 degrees or 56 degrees.

The sensing server may extract an optimal model parameter530by matching the set model with the current image. For example, as a model matching method, an active appearance model (AAM) or a scale-invariant feature transform (SIFT) that extracts an invariant feature such as a brightness value may be used. The sensing server may obtain size information, position information, and rotation information on a space of each of the display terminals corresponding to an ID of a current display terminal among the display terminals based on the model matching method. In this example, the model may be a rectangular model. The sensing server may configure a multi display space by performing a refinement process on IDs of the display terminals. For example, a size of the space corresponding to the ID1of the first display may be 16 inches. A degree of rotation may be 0, and a position may be indicated by (x, y) coordinates, for example, (5, 4).

FIG. 6is a diagram illustrating an entire display space according to an example embodiment.

Referring toFIG. 6, a display terminal is moved. The display terminal may be moved without changing currently displayed content. A sensing server may extract and obtain new position information and rotation information of display terminals based on a watermarking technique and may transmit the obtained position information and rotation information to a display server. A viewer may continuously view a display terminal or all display terminals regardless of sensing, that is, the viewer is unaware of the sensing performed by the sensing server. The sensing server may continuously track a change of a space configuration regardless of whether the current content is displayed based on the change of the space configuration. The sensing server may determine and recognize a position of a screen of each of the display terminals in an entire display space610corresponding to a multi display space using an image obtained by a camera. Also, the sensing server may continuously determine and recognize a rotation and the position of the screen of each of the display terminals in an entire display space620changed due to a movement of each of the display terminals continuously using the image obtained from the camera.

When each of the display terminals continuously rotates in a circle, the sensing server may recognize the rotation of each of the display terminals and transmit the recognized rotation information to the display server. The display server may transmit, to the display terminals, an instruction that each of the display terminals outputs a rotating image by allowing a viewer to view the rotating image based on the received rotation information. Depending on circumstances, the display server may perform transcoding on the content to be transmitted to the display terminals.

FIG. 7is a flowchart illustrating a screen position sensing method in a multi display system according to an example embodiment.

A flow of operations of a sensing server710, a display server720, and display terminals730is described with reference toFIG. 7. The display terminals730corresponding to a multi display group may be connected to the display server720, and an identification (ID) may be allocated to each of the display terminals730. The display server720may transmit the generated ID to each of the corresponding display terminals730.

The display terminals730may generate and configure a watermark using an image information generating method determined based on the ID of each of the display terminals730. The display terminals730may output the watermark by embedding the watermark in a currently displayed image. A camera may photograph the display terminals730, such that the image including the watermark may be received in a multi display space.

The sensing server710may receive the image photographed by the camera as an input. The sensing server710may obtain and recognize size information and position information of the display terminals730based on a model matching method by performing reversible watermarking on the photographed image received according to the ID of each of the display terminals730transmitted by the display server720. Subsequently, the sensing server710may transmit the obtained and recognized size information and the position information to the display server720. Here, recognized information may be position information and rotation information of the display terminals730. The display server720may construct a flow of content displayed in the display terminals730and an entire display space by reconstructing the content based on the transmitted physical position information. The display server720may transmit content configuration information to the display terminals730. In this example, the content configuration information may be content of which a position or a rotation angle is changed based on a screen position and a screen rotation angle or a content control instruction on the position or the rotation angle based on the screen position and the screen rotation angle. However, the content configuration information is not limited thereto.

According to embodiments of the present invention, it is possible to verify an ID of a display by sensing, in real-time, physical position information of a multi display using a watermark and a camera and enhance utilization of the multi display by three-dimensionally constructing spatial coordinate information.

According to embodiments of the present invention, it is possible to change an arrangement and a configuration of a display during a service because a wired and wireless connection between displays is not required in a multi display system and a user viewing the multi display is unaware of an ID of each of the displays using a watermark technique.

According to embodiments of the present invention, it is possible to sense, using a camera, position information in order to use the position information in a space, and a level of the position information is higher than that of configuration information of a monitor.

According to embodiments of the present invention, it is possible to be adaptive in real-time even when a multi display system is dynamically changed, and transmit information such that content is automatically extended or reduced based on a position and a rotation, and process, in real-time, a new configuration of a multi display based on a three-dimensional (3D) space configuration or a rotation of the multi display.

According to embodiments of the present invention, it is possible to activate new spatial perceptual content generation to be effectively applied in exhibition, advertisement, and information transmission.