Wireless device receiving a mirroring image from an external device and wireless system including wireless device and external device

A wireless device includes a memory, a display unit configured to display a first image, a wireless communication unit configured to receive a mirroring image from an external device including a mobile display unit configured to display a second image, and a processor configured to control the display unit and the wireless communication unit. The processor is configured to automatically analyze a ratio of the second image based on the received mirroring image, and display a third image having the same ratio as that of the second image on the first image according to the analysis result.

CROSS-REFERENCE TO RELATED APPLICATIONS

Pursuant to 35 U.S.C. § 119(a), this application claims the benefit of earlier filing date and right of priority to Korean Patent Application No. 10-2019-0000476, filed on Jan. 2, 2019, the contents of which are all hereby incorporated by reference herein in its entirety.

BACKGROUND

The present invention relates to a wireless device and a wireless system, and more particularly, to a wireless device and a wireless system which are capable of automatically recognizing a screen ratio of a source device during screen mirroring.

With the recent development of information communication technologies, various wireless communication technologies are under development. Among them, a wireless local area network (WLAN) is a technology for wirelessly accessing Internet in homes, businesses, or specific service providing areas based on a wireless frequency technique by using portable terminals such as Personal Digital Assistants (PDAs), laptop computers, and Portable Multimedia Players (PMPs).

As a direct communication technology for allowing devices to be connected to each other easily without an access point (AP) that is required in a WLAN system, the introduction of Wi-Fi Direct or Wi-Fi peer-to-peer (P2P) is discussed. According to a Wi-Fi Direct standard technology, devices are connected without complex setting processes and in order to provide various services to users, data exchanging operations at a communication speed of a typical WLAN system can be supported.

Various Wi-Fi supporting devices have been used recently, and among them, the number of Wi-Fi Direct supporting devices for communication between Wi-Fi devices without requiring APs has been increasing. The Wi-Fi Alliance (WFA) discusses a technology for introducing a platform that supports various services using Wi-Fi Direct link (for example, send, play, display, and print). This can be referred to as Wi-Fi Direct Service (WFDS).

Display service in WFDS is a service for allowing screen sharing between P2P devices. The screen sharing service is also referred to as a screen mirroring technique in that a wireless device displays the same image through image transmission.

In an existing screen sharing service, when images are transmitted between P2P devices, the aspect ratios of the source devices that transmit the images are various. There is no way to determine the aspect ratio of the source device when the source device is not registered.

Therefore, the screen layout mode of the source device is not correctly recognized, or an image having a proper resolution cannot be provided.

In addition, in the existing screen sharing service, when the source device is in a portrait mode, there is no method that can distinguish between a black image of an active image transmitted from the source device and a letter box inserted for matching resolution.

SUMMARY

According to the present invention, when a screen sharing service is provided, an average peak luminance value of a letter box transmitted from a source device is previously stored, and a screen layout mode and a screen ratio of the source device are correctly recognized by using the stored average peak luminance value.

According to the present invention, an optimized screen sharing service is provided by automatically recognizing a screen ratio of a source device.

According to the present invention, even when content included in an entire image being displayed is at a specific position, a screen layout mode and a screen ratio of a source device can be normally determined.

In one embodiment, a wireless device includes a memory, a display unit configured to display a first image, a wireless communication unit configured to receive a mirroring image from an external device including a mobile display unit configured to display a second image, and a processor configured to control the display unit and the wireless communication unit. The processor is configured to automatically analyze a ratio of the second image based on the received mirroring image, and display a third image having the same ratio as that of the second image on the first image according to the analysis result.

In another embodiment, a wireless system includes a wireless device configured to display a first image, receive a mirroring image from an external device including a mobile display unit configured to display a second image, automatically analyze a ratio of the second image based on the received mirroring image, and display a third image having the same ratio as that of the second image on the first image according to the analysis result, and the external device configured to display the second image and transmit a mirroring image including the displayed second image displayed to the wireless device.

DETAILED DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments relating to the present invention will be described in detail with reference to the accompanying drawings. The suffixes “module” and “unit” for components used in the description below are assigned or mixed in consideration of easiness in writing the specification and do not have distinctive meanings or roles by themselves.

A display device according to an embodiment of the present invention, for example, as an artificial display device that adds a computer supporting function to a broadcast receiving function, can have an easy-to-use interface such as a writing input device, a touch screen, or a spatial remote control device as an Internet function is added while fulfilling the broadcast receiving function. Then, with the support of a wired or wireless Internet function, it is possible to perform an e-mail, web browsing, banking, or game function in access to Internet and computers. In order to perform such various functions, standardized general purpose OS can be used.

Accordingly, since various applications are freely added or deleted on a general purpose OS kernel, a display device described herein, for example, can perform various user-friendly functions. The display device, in more detail, can be a network TV, Hybrid Broadcast Broadband TV (HBBTV), smart TV, light-emitting diode (LED) TV, organic light-emitting diode (OLED) TV, and so on and in some cases, can be applied to a smartphone.

FIG. 1is a block diagram illustrating a configuration of a display device according to an embodiment of the present invention.

Referring toFIG. 1, a display device100can include a broadcast reception unit130, an external device interface unit135, a storage unit140, a user interface unit150, a control unit170, a wireless communication unit173, a display unit180, an audio output unit185, and a power supply unit190.

The broadcast reception unit130can include a tuner131, a demodulation unit132, and a network interface unit133.

The tuner131can select a specific broadcast channel according to a channel selection command. The tuner131can receive broadcast signals for the selected specific broadcast channel.

The demodulation unit132can divide the received broadcast signals into video signals, audio signals, and broadcast program related data signals and restore the divided video signals, audio signals, and data signals to an output available form.

The external device interface unit135can receive an application or an application list in an adjacent external device and deliver it to the control unit170or the storage unit140.

The external device interface unit135can provide a connection path between the display device100and an external device. The external device interface unit135can receive at least one an image or audio output from an external device that is wirelessly or wiredly connected to the display device100and deliver it to the control unit170. The external device interface unit135can include a plurality of external input terminals. The plurality of external input terminals can include an RGB terminal, at least one High Definition Multimedia Interface (HDMI) terminal, and a component terminal.

An image signal of an external device input through the external device interface unit135can be output through the display unit180. A sound signal of an external device input through the external device interface unit135can be output through the audio output unit185.

An external device connectable to the external device interface unit135can be one of a set-top box, a Blu-ray player, a DVD player, a game console, a sound bar, a smartphone, a PC, a USB Memory, and a home theater system, but this is just exemplary.

The network interface unit133can provide an interface for connecting the display device100to a wired/wireless network including the Internet network. The network interface unit133can transmit or receive data to or from another user or another electronic device through an accessed network or another network linked to the accessed network.

Additionally, some content data stored in the display device100can be transmitted to a user or an electronic device, which is selected from other users or other electronic devices pre-registered in the display device100.

The network interface unit133can access a predetermined webpage through an accessed network or another network linked to the accessed network. That is, the network interface unit133can transmit or receive data to or from a corresponding server by accessing a predetermined webpage through the network.

Then, the network interface unit133can receive contents or data provided from a content provider or a network operator. That is, the network interface unit133can receive contents such as movies, advertisements, games, VODs, and broadcast signals, which are provided from a content provider or a network provider, through network and information relating thereto.

Additionally, the network interface unit133can receive firmware update information and update files provided from a network operator and transmit data to an Internet or content provider or a network operator.

The network interface unit133can select and receive a desired application among applications open to the air, through network.

The storage unit140can store signal-processed image, voice, or data signals stored by a program in order for each signal processing and control in the control unit170.

Additionally, the storage unit140can perform a function for temporarily storing image, voice, or data signals output from the external device interface unit135or the network interface unit133and can store information on a predetermined image through a channel memory function.

The storage unit140can store an application or an application list input from the external device interface unit135or the network interface unit133.

The display device100can play content files (for example, video files, still image files, music files, document files, application files, and so on) stored in the storage unit140and provide them to a user.

The user interface unit150can deliver signals input by a user to the control unit170or deliver signals from the control unit170to a user. For example, the user interface unit150can receive or process control signals such as power on/off, channel selection, and screen setting from the remote control device200or transmit control signals from the control unit170to the remote control device200according to various communication methods such as Bluetooth, Ultra Wideband (WB), ZigBee, Radio Frequency (RF), and IR.

Additionally, the user interface unit150can deliver, to the control unit170, control signals input from local keys (not shown) such as a power key, a channel key, a volume key, and a setting key.

Image signals that are image-processed in the control unit170can be input to the display unit180and displayed as an image corresponding to corresponding image signals. Additionally, image signals that are image-processed in the control unit170can be input to an external output device through the external device interface unit135.

Voice signals processed in the control unit170can be output to the audio output unit185. Additionally, voice signals processed in the control unit170can be input to an external output device through the external device interface unit135.

Besides that, the control unit170can control overall operations in the display device100. Additionally, the control unit170can control the display device100by a user command or internal program input through the user interface unit150and download a desired application or application list into the display device100in access to network.

The control unit170can output channel information selected by a user together with processed image or voice signals through the display unit180or the audio output unit185.

Additionally, according to an external device image playback command received through the user interface unit150, the control unit170can output image signals or voice signals of an external device such as a camera or a camcorder, which are input through the external device interface unit135, through the display unit180or the audio output unit185.

Moreover, the control unit170can control the display unit180to display images and control broadcast images input through the tuner131, external input images input through the external device interface unit135, images input through the network interface unit, or images stored in the storage unit140to be displayed on the display unit180. In this case, an image displayed on the display unit180can be a still image or video and also can be a 2D image or a 3D image.

Additionally, the control unit170can play content stored in the display device100, received broadcast content, and external input content input from the outside, and the content can be in various formats such as broadcast images, external input images, audio files, still images, accessed web screens, and document files.

Moreover, the wireless communication unit173can perform a wired or wireless communication with an external electronic device. The wireless communication unit173can perform short-range communication with an external device. For this, the wireless communication unit173can support short-range communication by using at least one of Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, and Wireless Universal Serial Bus (USB) technologies. The wireless communication unit173can support wireless communication between the display device100and a wireless communication system, between the display device100and another display device100, or between networks including the display device100and another display device100(or an external server) through wireless area networks. The wireless area networks can be wireless personal area networks.

Herein, the other display device100can be a mobile terminal such as a wearable device (for example, a smart watch, a smart glass, and a head mounted display (HMD)) or a smartphone, which is capable of exchanging data (or inter-working) with the display device100. The wireless communication unit173can detect (or recognize) a communicable wearable device around the display device100. Furthermore, if the detected wearable device is a device authenticated to communicate with the display device100, the control unit170can transmit at least part of data processed in the display device100to the wearable device through the wireless communication unit173. Accordingly, a user of the wearable device can use the data processed in the display device100through the wearable device.

The display unit180can convert image signals, data signals, or on-screen display (OSD) signals, which are processed in the control unit170, or images signals or data signals, which are received in the external device interface unit135, into R, G, and B signals to generate driving signals.

Furthermore, the display device100shown inFIG. 1is just one embodiment of the present invention and thus, some of the components shown can be integrated, added, or omitted according to the specification of the actually implemented display device100.

That is, if necessary, two or more components can be integrated into one component or one component can be divided into two or more components and configured. Additionally, a function performed by each block is to describe an embodiment of the present invention and its specific operation or device does not limit the scope of the present invention.

According to another embodiment of the present invention, unlikeFIG. 1, the display device100can receive images through the network interface unit133or the external device interface unit135and play them without including the tuner131and the demodulation unit132.

For example, the display device100can be divided into an image processing device such as a set-top box for receiving broadcast signals or contents according to various network services and a content playback device for playing contents input from the image processing device.

In this case, an operating method of a display device according to an embodiment of the present invention described below can be performed by one of the display device described with reference toFIG. 1, an image processing device such as the separated set-top box, and a content playback device including the display unit180and the audio output unit185.

Then, referring toFIGS. 2 and 3, a remote control device is described according to an embodiment of the present invention.

FIG. 2is a block diagram illustrating a remote control device according to an embodiment of the present invention andFIG. 3is a view illustrating an actual configuration of a remote control device according to an embodiment of the present invention.

First, referring toFIG. 2, a remote control device200can include a fingerprint recognition unit210, a wireless communication unit220, a user input unit230, a sensor unit240, an output unit250, a power supply unit260, a storage unit270, a control unit280, and a sound acquisition unit290.

Referring toFIG. 2, the wireless communication unit220transmits/receives signals to/from an arbitrary any one of display devices according to the above-mentioned embodiments of the present invention.

The remote control device200can include a radio frequency (RF) module221for transmitting/receiving signals to/from the display device100according to the RF communication standards and an IR module223for transmitting/receiving signals to/from the display device100according to the IR communication standards. Additionally, the remote control device200can include a Bluetooth module225for transmitting/receiving signals to/from the display device100according to the Bluetooth communication standards. Additionally, the remote control device200can include a Near Field Communication (NFC) module227for transmitting/receiving signals to/from the display device100according to the NFC communication standards and a WLAN module229for transmitting/receiving signals to/from the display device100according to the Wireless LAN (WLAN) communication standards.

Additionally, the remote control device200can transmit signals containing information on a movement of the remote control device200to the display device100through the wireless communication unit220.

Moreover, the remote control device200can receive signals transmitted from the display device100through the RF module221and if necessary, can transmit a command on power on/off, channel change, and volume change to the display device100through the IR module223.

The user input unit230can be configured with a keypad button, a touch pad, or a touch screen. A user can manipulate the user input unit230to input a command relating to the display device100to the remote control device200. If the user input unit230includes a hard key button, a user can input a command relating to the display device100to the remote control device200through the push operation of the hard key button. This will be described with reference toFIG. 3.

Referring toFIG. 3, the remote control device200can include a plurality of buttons. The plurality of buttons can include a fingerprint recognition button212, a power button231, a home button232, a live button233, an external input button234, a voice adjustment button235, a voice recognition button236, a channel change button237, a check button238, and a back button239.

The fingerprint recognition button212can be a button for recognizing a user's fingerprint. According to an embodiment of the present invention, the fingerprint recognition button212can perform a push operation and receive a push operation and a fingerprint recognition operation. The power button231can be button for turning on/off the power of the display device100. The home button232can be a button for moving to the home screen of the display device100. The live button233can be a button for displaying live broadcast programs. The external input button234can be a button for receiving an external input connected to the display device100. The voice adjustment button235can be a button for adjusting the size of a volume output from the display device100. The voice recognition button236can be a button for receiving user's voice and recognizing the received voice. The channel change button237can be a button for receiving broadcast signals of a specific broadcast channel. The check button238can be a button for selecting a specific function and the back button239can be a button for returning to a previous screen.

Again, referring toFIG. 2, if the user input unit230includes a touch screen, a user can touch a soft key of the touch screen to input a command relating to the display device100to the remote control device200. Additionally, the user input unit230can include various kinds of input means manipulated by a user, for example, a scroll key and a jog key, and this embodiment does not limit the scope of the present invention.

The sensor unit240can include a gyro sensor241or an acceleration sensor243and the gyro sensor241can sense information on a movement of the remote control device200.

For example, the gyro sensor241can sense information on an operation of the remote control device200on the basis of x, y, and z axes and the acceleration sensor243can sense information on a movement speed of the remote control device200. Moreover, the remote control device200can further include a distance measurement sensor and sense a distance with respect to the display unit180of the display device100.

The output unit250can output image or voice signals in response to manipulation of the user input unit230or image or voice signals corresponding to signals transmitted from the display device100. A user can recognize whether the user input unit230is manipulated or the display device100is controlled through the output unit250.

For example, the output unit250can include an LED module251for flashing, a vibration module253for generating vibration, a sound output module255for outputting sound, or a display module257for outputting an image, if the user input unit230is manipulated or signals are transmitted/received to/from the display device100through the wireless communication unit220.

Additionally, the power supply unit260supplies power to the remote control device200and if the remote control device200does not move for a predetermined time, stops the power supply, so that power waste can be reduced. The power supply unit260can resume the power supply if a predetermined key provided at the remote control device200is manipulated.

The storage unit270can store various kinds of programs and application data necessary for control or operation of the remote control device200. If the remote control device200transmits/receives signals wirelessly through the display device100and the RF module221, the remote control device200and the display device100transmits/receives signals through a predetermined frequency band.

The control unit280of the remote control device200can store, in the storage unit270, information on a frequency band for transmitting/receiving signals to/from the display device100paired with the remote control device200and refer to it.

The control unit280controls general matters relating to control of the remote control device200. The control unit280can transmit a signal corresponding to a predetermined key manipulation of the user input unit230or a signal corresponding to movement of the remote control device200sensed by the sensor unit240to the display device100through the wireless communication unit220.

Additionally, the sound acquisition unit290of the remote control device200can obtain voice. The sound acquisition unit290can include at least one microphone and obtain voice through the microphone291.

FIG. 4is a view of utilizing a remote control device according to an embodiment of the present invention.FIG. 4(a)illustrates that a pointer205corresponding to the remote control device200is displayed on the display unit180.

A user can move or rotate the remote control device200vertically or horizontally. The pointer205displayed on the display unit180of the display device100corresponds to a movement of the remote control device200. Since the corresponding pointer205is moved and displayed according to a movement on a 3D space as show in the drawing, the remote control device200can be referred to as a spatial remote control device.

FIG. 4(b)illustrates that if a user moves the remote control device200, the pointer205displayed on the display unit180of the display device100is moved to the left according to the movement of the remote control device200.

Information on a movement of the remote control device200detected through a sensor of the remote control device200is transmitted to the display device100. The display device100can calculate the coordinates of the pointer205from the information on the movement of the remote control device200. The display device100can display the pointer205to match the calculated coordinates.

FIG. 4(c)illustrates that while a specific button in the remote control device200is pressed, a user moves the remote control device200away from the display unit180. Thus, a selection area in the display unit180corresponding to the pointer205can be zoomed in and displayed larger.

On the other hand, if a user moves the remote control device200close to the display unit180, a selection area in the display unit180corresponding to the pointer205can be zoomed out and displayed in a reduced size.

On the other hand, if the remote control device200is moved away from the display unit180, a selection area can be zoomed out and if the remote control device200is moved closer to the display unit180, a selection area can be zoomed in.

Additionally, if a specific button in the remote control device200is pressed, recognition of a vertical or horizontal movement can be excluded. That is, if the remote control device200is moved away from or closer to the display unit180, the up, down, left, or right movement cannot be recognized and only the back and forth movement can be recognized. While a specific button in the remote control device200is not pressed, only the pointer205is moved according to the up, down, left or right movement of the remote control device200.

Moreover, the moving speed or moving direction of the pointer205can correspond to the moving speed or moving direction of the remote control device200.

Furthermore, a pointer in this specification means an object displayed on the display unit180in response to an operation of the remote control device200. Accordingly, besides an arrow form displayed as the pointer205in the drawing, various forms of objects are possible. For example, the above concept includes a point, a cursor, a prompt, and a thick outline. Then, the pointer205can be displayed in correspondence to one point of a horizontal axis and a vertical axis on the display unit180and also can be displayed in correspondence to a plurality of points such as a line and a surface.

Then, a configuration of a mobile terminal according to an embodiment of the present invention will be described with reference toFIG. 5.FIG. 5is a block diagram illustrating a mobile terminal according to an embodiment of the present invention.

The mobile terminal500can include a wireless communication unit510, an input unit520, a sensing unit540, an output unit550, an interface unit560, a memory unit570, a controller580, and a power supply unit590. In implementing a mobile terminal, all components shown inFIG. 5may not be necessary, so that a mobile terminal described in this specification can include components less or more than the components listed above.

In more detail, the wireless communication unit510in the components can include at least one module allowing wireless communication between the mobile terminal500and a wireless communication system, between the mobile terminal500and another mobile terminal500, or between the mobile terminal500and an external server. Additionally, the wireless communication unit510can include at least one module connecting the mobile terminal500to at least one network.

The wireless communication unit510can include at least one of a broadcast receiving module511, a mobile communication module512, a wireless Internet module513, a short-range communication module514, and a location information module515.

The input unit520can include a camera521or an image input unit for image signal input, a microphone522or an audio input unit for receiving audio signal input, and a user input unit523(for example, a touch key and a mechanical key) for receiving information from a user. Voice data or image data collected by the input unit520are analyzed and processed as a user's control command.

The sensing unit540can include at least one sensor for sensing at least one of information in a mobile terminal, environmental information around a mobile terminal, and user information. For example, the sensing unit540can include at least one of a proximity sensor541, an illumination sensor542, a touch sensor, an acceleration sensor, a magnetic sensor, a G-sensor, a gyroscope sensor, a motion sensor, an RGB sensor, an infrared (IR) sensor, a finger scan sensor, an ultrasonic sensor, an optical sensor (for example, the camera521), a microphone (for example, the microphone522), a battery gauge, an environmental sensor (for example, a barometer, a hygrometer, a thermometer, a radiation sensor, a thermal sensor, and a gas sensor), or a chemical sensor (for example, an electronic noise, a healthcare sensor, and a biometric sensor). Moreover, a mobile terminal disclosed in this specification can combine information sensed by at least two or more sensors among such sensors and utilize it.

The output unit550is used to generate a visual, auditory, or haptic output and can include at least one of a display unit551, a sound output module552, a haptic module553, and an optical output unit554. The display unit551can be formed with a mutual layer structure with a touch sensor or formed integrally, so that a touch screen can be implemented. Such a touch screen can serve as the user input unit523providing an input interface between the mobile terminal500and a user and an output interface between the mobile terminal500and a user at the same time.

The interface unit560can serve as a path to various kinds of external devices connected to the mobile terminal500. The interface unit560can include at least one of a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, a port connecting a device equipped with an identification module, an audio Input/Output (I/O) port, a video I/O port, or an earphone port. When an external device is connected to the interface unit560, the mobile terminal500can perform an appropriate control relating to the connected external device.

Additionally, the memory unit570can store data supporting various functions of the mobile terminal500. The memory unit570can store a plurality of application programs (for example, application programs or applications) running on the mobile terminal500and also data and commands for operations of the mobile terminal500. At least part of such an application program can be downloaded from an external server through wireless communication. Additionally, at least part of such an application program can be included in the mobile terminal500from the time of shipment in order to perform a basic function (for example, an incoming call, a transmission function, and a message reception) of the mobile terminal500. Moreover, an application program can be stored in the memory unit570and installed on the mobile terminal500, so that it can run to perform an operation (or a function) of the mobile terminal500by the controller580.

The controller580can control overall operations of the mobile terminal500generally besides an operation relating to the application program. The controller580can provide appropriate information or functions to a user or process them by processing signals, data, and information input/output through the above components or executing application programs stored in the memory unit570.

Additionally, in order to execute an application program stored in the memory unit570, the controller580can control at least part of the components shown inFIG. 5. Furthermore, in order to execute the application program, the controller580can combine at least two of the components in the mobile terminal500and operate it.

The power supply unit590can receive external power or internal power under a control of the controller580and supply power to each component in the mobile terminal500. The power supply unit590includes a battery and the battery can be a built-in battery or a replaceable battery.

At least part of each component can operate cooperatively in order to implement operations, controls, or control methods of the mobile terminal500according to various embodiments of the present invention described below. Additionally, the operations, controls, or control methods of the mobile terminal500can be implemented on the mobile terminal500by executing at least one application program stored in the memory unit570.

Hereinafter, prior to examining various embodiments implemented through the mobile terminal500, the above-listed components are described in more detail with reference toFIG. 5.

First, in describing the wireless communication unit510, the broadcast receiving module511of the wireless communication unit510can receive a broadcast signal and/or broadcast related information from an external broadcast management server through a broadcast channel. The broadcast channel can include a satellite channel and a terrestrial channel. At least two broadcast receiving modules for simultaneous broadcast reception for at least two broadcast channels or broadcast channel switching can be provided to the mobile terminal500.

The mobile communication module512can transmit/receive a wireless signal to/from at least one of a base station, an external terminal, and a server on a mobile communication network established according to the technical standards or communication methods for mobile communication (for example, Global System for Mobile communication (GSM), Code Division Multi Access (CDMA), Code Division Multi Access 2000 (CDMA2000), Enhanced Voice-Data Optimized or Enhanced Voice-Data Only (EV-DO), Wideband CDMA (WCDMA), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), and Long Term Evolution-Advanced (LTE-A)).

The wireless signal can include various types of data according to a voice call signal, a video call signal, or text/multimedia message transmission.

The wireless Internet module513refers to a module for wireless Internet access and can be built in or external to the mobile terminal500. The wireless Internet module513can be configured to transmit/receive a wireless signal in a communication network according to wireless internet technologies.

The wireless Internet technology can include Wireless LAN (WLAN), Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, Digital Living Network Alliance (DLNA), Wireless Broadband (WiBro), World Interoperability for Microwave Access (WiMAX), High Speed Downlink Packet Access (HSDPA), High Speed Uplink Packet Access (HSUPA), Long Term Evolution (LTE), and Long Term Evolution-Advanced (LTE-A) and the wireless Internet module513transmits/receives data according to at least one wireless Internet technology including any Internet technologies not listed above.

From the viewpoint that wireless Internet access by WiBro, HSDPA, HSUPA, GSM, CDMA, WCDMA, LTE, and LTE-A is achieved through a mobile communication network, the wireless Internet module513performing wireless Internet access through the mobile communication network can be understood as one type of the mobile communication module512.

The short-range communication module514can support short-range communication by using at least one of Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), Wireless-Fidelity (Wi-Fi), Wi-Fi Direct, and Wireless Universal Serial Bus (USB) technologies. The short-range communication module514can support wireless communication between the mobile terminal500and a wireless communication system, between the mobile terminal500and another mobile terminal500, or between networks including the mobile terminal500and another mobile terminal500(or an external server) through wireless area networks. The wireless area networks can be wireless personal area networks.

Here, the other mobile terminal500can be a wearable device (for example, a smart watch, a smart glass, and an HMD) that is capable of exchanging data (or interworking) with the mobile terminal500. The short-range communication module514can detect (or recognize) a wearable device around the mobile terminal500, which is capable of communicating with the mobile terminal500. Furthermore, if the detected wearable device is a device authenticated to communicate with the mobile terminal500, the controller580can transmit at least part of data processed in the mobile terminal500to the wearable device through the short-range communication module514. Accordingly, a user of the wearable device can use the data processed in the mobile terminal500through the wearable device. For example, according thereto, if a call is received by the mobile terminal500, a user can perform a phone call through the wearable device or if a message is received by the mobile terminal500, a user can check the received message.

The location information module515is a module for obtaining the location (or the current location) of a mobile terminal and its representative examples include a global positioning system (GPS) module or a Wi-Fi module. For example, the mobile terminal500can obtain its position by using a signal transmitted from a GPS satellite through the GPS module. As another example, the mobile terminal500can obtain its position on the basis of information of a wireless access point (AP) transmitting/receiving a wireless signal to/from the Wi-Fi module, through the Wi-Fi module. If necessary, the location information module515can perform a function of another module in the wireless communication unit510in order to obtain data on the location of the mobile terminal substitutionally or additionally. The location information module515is a module for obtaining the position (or the current position) of the mobile terminal and is not limited to a module directly calculating and obtaining the position of the mobile terminal.

Then, the input unit520is used for inputting image information (or signal), audio information (or signal), data, or information inputted from a user and the mobile terminal500can include at least one camera521in order for inputting image information. The camera521processes image frames such as a still image or a video obtained by an image sensor in a video call mode or a capturing mode. The processed image frame can be displayed on the display unit551or stored in the memory unit570. Moreover, a plurality of cameras521equipped in the mobile terminal500can be arranged in a matrix structure and through the camera521having such a matrix structure, a plurality of image information having various angles or focuses can be input to the input terminal500. Additionally, the plurality of cameras521can be arranged in a stereo structure to obtain the left and right images for implementing a three-dimensional image.

The microphone522processes external sound signals as electrical voice data. The processed voice data can be utilized variously according to a function (or an application program being executed) being performed in the mobile terminal500. Moreover, various noise canceling algorithms for removing noise occurring during the reception of external sound signals can be implemented in the microphone522.

The user input unit523is to receive information from a user and if information is input through the user input unit523, the controller580can control an operation of the mobile terminal500according to the input information. The user input unit523can include a mechanical input means (or a mechanical key, for example, a button, a dome switch, a jog wheel, and a jog switch at the front, back or side of the mobile terminal500) and a touch type input means. As one example, a touch type input means can include a virtual key, a soft key, or a visual key, which is displayed on a touch screen through software processing or can include a touch key disposed at a portion other than the touch screen. Moreover, the virtual key or visual key can have various forms and can be disposed on a touch screen and for example, can include graphic, text, icon, video, or a combination thereof.

Moreover, the sensing unit540can sense at least one of information in a mobile terminal, environmental information around a mobile terminal, and user information and can then generate a sensing signal corresponding thereto. On the basis of such a sensing signal, the controller580can control the drive or control of the mobile terminal500or can perform data processing, functions, or operations relating to an application program installed in the mobile terminal500. Representative sensors among various sensors included in the sensing unit540will be described in more detail.

First, the proximity sensor541refers to a sensor detecting whether there is an object approaching a predetermined detection surface or whether there is an object around by using the strength of an electromagnetic field or infrared, without mechanical contact. The proximity sensor541can disposed in an inner area of a mobile terminal surrounded by the touch screen or around the touch screen.

Examples of the proximity sensor541can include a transmission-type photoelectric sensor, a direct reflective-type photoelectric sensor, a mirror reflective-type photoelectric sensor, a high-frequency oscillation-type proximity sensor, a capacitive-type proximity sensors, a magnetic-type proximity sensor, and an infrared proximity sensor. If the touch screen is a capacitive type, the proximity sensor541can be configured to detect the proximity of an object by changes in an electric field according to the proximity of the object having conductivity. In this case, the touch screen (or a touch sensor) itself can be classified as a proximity sensor.

Moreover, for convenience of description, an action for recognizing the position of an object on the touch screen as the object is close to the touch screen without contacting the touch screen is called “proximity touch” and an action that the object actually contacts the touch screen is called “contact touch.” A position that an object is proximity-touched on the touch screen is a position that the object vertically corresponds to the touch screen if the object is proximity-touched. The proximity sensor541can detect a proximity touch and a proximity touch pattern (for example, a proximity touch distance, a proximity touch direction, a proximity touch speed, a proximity touch time, a proximity touch position, and a proximity touch movement state). Moreover, the controller580processes data (for information) corresponding to a proximity touch operation and a proximity touch pattern, detected through the proximity sensor541, and furthermore, can output visual information corresponding to the processed data on the touch screen. Furthermore, according to whether a touch for the same point on the touch screen is a proximity touch or a contact touch, the controller580can control the mobile terminal500to process different operations or data (or information).

The touch sensor detects a touch (or a touch input) applied to the touch screen (or the display unit551) by using at least one of various touch methods, for example, a resistive film method, a capacitive method, an infrared method, an ultrasonic method, and a magnetic field method.

For example, the touch sensor can be configured to convert a pressure applied to a specific portion of the touch screen or changes in capacitance occurring at a specific portion into electrical input signals. The touch sensor can be configured to detect a position and area that a touch target applying a touch on the touch screen touches the touch sensor, a pressured if touched, and a capacitance if touched. Here, the touch target, as an object applying a touch on the touch sensor, can be a finger, a touch pen, a stylus pen, or a pointer, for example.

In such a manner, if there is a touch input on the touch sensor, signal(s) corresponding thereto are sent to a touch controller. The touch controller processes the signal(s) and then transmits corresponding data to the controller580. Therefore, the controller580can recognize which area of the display unit551is touched. Herein, the touch controller can be an additional component separated from the controller580or can be the controller580itself.

Moreover, the controller580can perform different controls or the same control according to types of a touch target touching the touch screen (or a touch key equipped separated from the touch screen). Whether to perform different controls or the same control according to types of a touch target can be determined according to a current operation state of the mobile terminal500or an application program in execution.

Moreover, the above-mentioned touch sensor and proximity sensor are provided separately or combined and can thus sense various types of touches, for example, short (or tap) touch), long touch, multi touch, drag touch, flick touch, pinch-in touch, pinch-out touch, swipe touch, and hovering touch for the touch screen.

The ultrasonic sensor can recognize position information of a detection target by using ultrasonic waves. Moreover, the controller580can calculate the position of a wave source through information detected by an optical sensor and a plurality of ultrasonic sensors. The position of the wave source can be calculated by using the property that light is much faster than ultrasonic wave, that is, a time that light reaches an optical signal is much shorter than a time that ultrasonic wave reaches an ultrasonic sensor. In more detail, the position of the wave source can be calculated by using a time difference with a time that ultrasonic wave reaches by using light as a reference signal.

Moreover, the camera521described as a configuration of the input unit520can include at least one of a camera sensor (for example, CCD and CMOS), a photo sensor (or an image sensor), or a laser sensor.

The camera521and the laser sensor can be combined to detect a touch of a detection target for a three-dimensional image. The photo sensor can be stacked on a display device and is configured to scan a movement of a detection target close to the touch screen. In more detail, the photo sensor mounts a photo diode and a transistor (TR) in a row/column and scans content disposed on the photo sensor by using an electrical signal changing according to an amount of light applied to the photo diode. That is, the photo sensor can calculate the coordinates of a detection target according to the amount of change in light and through this, can obtain the position information of the detection target.

The display unit551can display (output) information processed in the mobile terminal500. For example, the display unit551can display execution screen information of an application program running on the mobile terminal500or user interface (UI) and graphic user interface (GUI) information according to such execution screen information.

Additionally, the display unit551can be configured as a three-dimensional display unit displaying a three-dimensional image.

A three-dimensional display method, for example, a stereoscopic method (a glasses method), an autostereoscopic (no glasses method), a projection method (a holographic method) can be applied to the three-dimensional display unit.

The sound output module552can output audio data received from the wireless communication unit510or stored in the memory unit570in a call signal reception or call mode, a recording mode, a voice recognition mode, or a broadcast reception mode. The sound output module552can output a sound signal relating to a function (for example, a call signal reception sound and a message reception sound) performed by the mobile terminal500. The sound output module552can include a receiver, a speaker, and a buzzer.

The haptic module553generates various haptic effects that a user can feel. A representative example of a haptic effect that the haptic module553generates is vibration. The intensity and pattern of vibration generated by the haptic module553can be controlled by a user's selection or a setting of a control unit. For example, the haptic module553can synthesize and output different vibrations or output different vibrations sequentially.

The haptic module553can generate various haptic effects, for example, effects by a pin arrangement moving vertical to a contact skin surface, injection power or suction power of air through an injection port or a suction port, rubbing a skin surface, electrode contact, stimulus of electrostatic force and effects by the reproduction of cold/warm sense by using a device absorbing or emitting heat.

The haptic module553can be implemented to deliver a haptic effect through a direct contact and also allow a user to feel a haptic effect through a muscle sense such as a finger or an arm. The haptic module553can be more than two according to a configuration aspect of the mobile terminal500.

The optical output unit554outputs a signal for notifying event occurrence by using light of a light source of the mobile terminal500. An example of an event occurring in the mobile terminal500includes message reception, call signal reception, missed calls, alarm, schedule notification, e-mail reception, and information reception through an application.

A signal output from the optical output unit554is implemented as a mobile terminal emits single color of multi-color to the front or the back. The signal output can be terminated if a mobile terminal detects user's event confirmation.

The interface unit560can serve as a path to all external devices connected to the mobile terminal500. The interface unit560can receive data from an external device, receive power and deliver it to each component in the mobile terminal500, or transmit data in the mobile terminal500to an external device. For example, the interface unit560can include a wired/wireless headset port, an external charger port, a wired/wireless data port, a memory card port, a port connecting a device equipped with an identification module, an audio I/O port, a video I/O port, and an earphone port.

Moreover, the identification module, as a chip storing various information for authenticating usage authority of the mobile terminal500, can include a user identity module (UIM), a subscriber identity module (SIM), and a universal subscriber identity module (USIM). A device equipped with an identification module (hereinafter referred to as an identification device) can be manufactured in a smart card form. Accordingly, the identification device can be connected to the terminal500through the interface unit560.

Additionally, if the mobile terminal500is connected to an external cradle, the interface unit560can become a path through which power of the cradle is supplied to the mobile terminal500or a path through which various command signals inputted from the cradle are delivered to the mobile terminal500by a user. The various command signals or the power input from the cradle can operate as a signal for recognizing that the mobile terminal500is accurately mounted on the cradle.

The memory unit570can store a program for an operation of the controller580and can temporarily store input/output data (for example, a phone book, a message, a still image, and a video). The memory unit570can store data on various patterns of vibrations and sounds outputted during a touch input on the touch screen.

The memory unit570can include at least one type of storage medium among flash memory type, hard disk type, Solid State Disk (SSD) type, Silicon Disk Drive (SDD) type, multimedia card micro type, card type memory (for example, SD or XD memory type), random access memory (RAM) type, static random access memory (SRAM) type, read-only memory (ROM) type, electrically erasable programmable read-only memory (EEPROM) type, programmable read-only memory (PROM) type, magnetic memory type, magnetic disk type, and optical disk type. The mobile terminal500can operate in relation to a web storage performing a storage function of the memory unit570on Internet.

Moreover, as mentioned above, the controller580can control operations relating to an application program and overall operations of the mobile terminal500in general. For example, if a state of the mobile terminal500satisfies set conditions, the controller580can execute or release a lock state limiting an output of a control command of a user for applications.

Additionally, the controller580can perform a control or processing relating to a voice call, data communication, and a video call can perform pattern recognition processing for recognizing handwriting input or drawing input on the touch screen as a text and an image, respectively. Furthermore, the controller580can use at least one or a combination of the above components to perform a control in order to implement various embodiments described below on the mobile terminal500.

The power supply unit590can receive external power or internal power under a control of the controller580and can then supply power necessary for an operation of each component. The power supply unit590includes a battery. The battery is a rechargeable built-in battery and can be detachably coupled to a terminal body in order for charging.

Additionally, the power supply unit590can include a connection port and the connection port can be configured as one example of the interface unit560to which an external charger supplying power for charging of the battery is electrically connected.

As another example, the power supply unit590can be configured to charge a battery through a wireless method without using the connection port. In this case, the power supply unit590can receive power from an external wireless power transmission device through at least one of an inductive coupling method based on a magnetic induction phenomenon, and a magnetic resonance coupling method based on an electromagnetic resonance phenomenon.

Moreover, various embodiments below can be implemented in a computer or device similar thereto readable medium by using software, hardware, or a combination thereof.

Then, a communication system using the mobile terminal500is described according to an embodiment of the present invention.

First, the communication system can use different wireless interfaces and/or physical layers. For example, a wireless interface available to the communication system can include Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA), and Universal Mobile Telecommunications Systems (UMTS) (especially, Long Term Evolution (LTE), Long Term Evolution-Advanced (LTE-A), and Global System for Mobile Communications (GSM)).

Hereinafter, for convenience of description, description is made limited to CDMA. However, it is apparent that the present invention is applicable to all communication systems including Orthogonal Frequency Division Multiplexing (OFDM) wireless communication systems in addition to CDMA wireless communication systems.

The CDMA wireless communication system can include at least one terminal500, at least one base station (BS) (it can be referred to as Node B or Evolved Node B), at least one base station controllers (BSCs), and a mobile switching center (MSC). MSC can be configured to be connected to Public Switched Telephone Network (PSTN) and BSCs. BSCs can be connected being paired with a BS through a backhaul line. The backhaul line can be provided according to at least one of E1/T1, ATM, IP, PPP, Frame Relay, HDSL, ADSL, and xDSL. Accordingly, a plurality of BSCs can be included in a CDMA wireless communication system.

Each of a plurality of BSs can include at least one sensor and each sensor can include an omni-directional antenna or an antenna indicating a specific radial direction from a BS. Additionally, each sensor can include at least two antennas in various forms. Each BS can be configured to support a plurality of frequency allocations and each of the plurality of frequency allocations can have a specific spectrum (for example, 1.25 MHz, 5 MHz, and so on).

The intersection of a sector and a frequency allocation can be referred to as a CDMA channel. A BS can be referred to as a Base Station Transceiver Subsystem (BTS). In such a case, one BSC and at least one BS together can be referred to as “BS”. A BS can also represent “cell site”. Additionally, each of a plurality of sectors for a specific BS can be referred to as a plurality of cell sites.

A Broadcasting Transmitter (BT) transmits broadcast signals to the terminals500operating in a system. The broadcast receiving module511shown inFIG. 5is provided in the terminal500for receiving broadcast signals transmitted from the BT.

Additionally, GPS can be linked to a CDMA wireless communication system in order to check the location of the mobile terminal500. Then, a satellite helps obtaining the location of the mobile terminal500. Useful location information can be obtained by at least one satellite. Herein, the location of the mobile terminal500can be traced by using all techniques for tracing the location in addition to GPS tracking technique. Additionally, at least one GPS satellite can be responsible for satellite DMB transmission selectively or additionally.

The location information module515in the mobile terminal500is for detecting and calculating the position of the mobile terminal and its representative example can include a GPS module and a WiFi module. If necessary, the location information module515can perform a function of another module in the wireless communication unit510in order to obtain data on the location of the mobile terminal substitutionally or additionally.

The location information module515such as a GPS module can calculate information on a distance from at least three satellites and accurate time information and then apply triangulation to the calculated information, in order to accurately calculate the 3D current location information according to latitude, longitude, and altitude. A method for calculating location and time information by using three satellites and correcting errors of the calculated location and time information by using another one satellite is being widely used. Additionally, the GPS module515can speed information as continuously calculating the current location in real time. However, it is difficult to accurately measure the location of a mobile terminal by using a GPS module in a shadow area of a satellite signal such as a room. Accordingly, in order to compensate for the measurement of a GPS method, a WiFi Positioning System (WPS) can be utilized.

WPS is a technique for tracking the location of the mobile terminal500by using a WiFi module in the mobile terminal500and a wireless Access Point (AP) for transmitting or receiving wireless signals to or from the WiFi module and can mean a Wireless Local Area Network (WLAN) based location measurement technique using WiFi.

A WiFi location tracking system can include a WiFi location measurement server, a mobile terminal500, a wireless AP connected to the mobile terminal500, and a database for storing arbitrary wireless AP information.

The mobile terminal500in access to a wireless AP can transmit a location information request message to a WiFi location measurement server.

The WiFi location measurement server extracts information of a wireless AP connected to the mobile terminal500on the basis of a location information request message (or signal) of the mobile terminal500. Information of a wireless AP connected to the mobile terminal500can be transmitted to the WiFi location measurement server through the mobile terminal500or can be transmitted from a wireless AP to a WiFi location measurement server.

Based on the location information request message of the mobile terminal500, the extracted information of a wireless AP can be at least one of MAC Address, Service Set Identification (SSID), Received Signal Strength Indicator (RSSI), Reference Signal Received Power (RSRP), Reference Signal Received Quality (RSRQ), channel information, Privacy, Network Type, Signal Strength, and Noise Strength.

As mentioned above, the WiFi position measurement server can extract wireless AP information corresponding to a wireless AP that the mobile terminal500access from a pre-established database by using information of the wireless AP connected to the mobile terminal500. At this point, information of arbitrary wireless APs stored in the database can information such as MAC Address, SSID, channel information, Privacy, Network Type, latitude and longitude coordinates of a wireless AP, a building name where a wireless AP is located, the number of floors, indoor detailed location (GPS coordinates available), the address of the owner of an AP, and phone numbers. At this point, in order to remove a mobile AP or a wireless AP provided using illegal MAC address during a measurement process, a WiFi location measurement server can extract only a predetermined number of wireless AP information in high RSSI order.

Then, the WiFi location measurement server can extract (or analyze) the location information of the mobile terminal500by using at least one wireless AP information extracted from the database. By comparing the included information and the received wireless AP information, location information of the mobile terminal500is extracted (or analyzed).

As a method of extracting (or analyzing) the location information of the mobile terminal500, a Cell-ID method, a finger-print method, a triangulation method, and a landmark method can be used.

The Cell-ID method is a method for determining the location of a wireless AP having the strongest signal intensity in neighbor wireless AP information that a mobile terminal collects as the location of the mobile terminal. Implementation is simple, no additional cost is required, and location information is obtained quickly but if the installation density of wireless APs is low, measurement precision is poor.

The finger-print method is a method for collecting signal intensity information by selecting a reference location from a service area and estimating the location through signal intensity information transmitted from a mobile terminal on the basis of the collected information. In order to use the finger-print method, there is a need to provide a database for storing propagation characteristics in advance.

The triangulation method is a method for calculating the location of a mobile terminal on the basis of a distance between coordinates of at least three wireless APs and a mobile terminal. In order to measure a distance between a mobile terminal and a wireless AP, a signal intensity converted into distance information, Time of Arrival (ToA), Time Difference of Arrival (TDoA), and Angle of Arrival (AoA) can be used.

The landmark method is a method for measuring the location of a mobile terminal by using a landmark transmitter knowing the location.

In addition to the listed methods, a variety of algorithms can be utilized as methods for extracting (or analyzing) the location information of a mobile terminal.

As the extracted location information of the mobile terminal500is transmitted to the mobile terminal500through the WiFi location measurement server, the mobile terminal500can obtain the location information.

As connected to at least one wireless AP, the mobile terminal500can obtain location information. At this point, the number of wireless APs, which are required for obtaining the location information of the mobile terminal500, can vary according to a wireless communication environment where the mobile terminal500is located.

Then, referring toFIG. 6, an example that a WFD session is established between a WFD source and a WFD sink and screen sharing service is provided is described.FIG. 6is a ladder diagram illustrating a process that a WFD session is established between a WFD source and a WFD sink and a screen sharing service is provided according to an embodiment of the present invention.

At least one of operations shown inFIG. 6can be supported by one or more Wi-Fi Direct standard documents.

A display service in WFDS can mean service and application for allowing screen sharing between P2P devices. P2P service using display service can be referred to as a WFD device; a device for supporting streaming of multimedia content through a device P2P link among WFD devices can be referred to as a WFD source; and a device for receiving multimedia content from a WFD source device through a P2P link and rendering it can be referred to as a WFD sink.

The WFD source10can be the mobile terminal500shown inFIG. 5. Accordingly, the WFD source10can include all the components of the mobile terminal500shown inFIG. 5.

The WFD sink20can be the display device100shown inFIG. 1. Accordingly, the WFD sink20can include all the components of the mobile terminal500shown inFIG. 1.

According to an embodiment, an image displayed on a screen can be an image of multimedia content.

According to another embodiment, if the WFD source10is the mobile terminal500shown inFIG. 5, an image displayed on the screen can be an image being displayed by the mobile terminal500, for example, the home screen of the mobile terminal500or the execution screen of an application installed on the mobile terminal500or a video of a media content.

Then, the WFD source10and the WFD sink20can first discover the presence of each other before a WFD connection setup through WFD device discovery (S601). In more detail, WFD devices can recognize the presence of each other through a probe request frame including WFD information element (IE) and a probe reply frame. The WFD IE can include base information for establishing an optimized connection between WFD devices, for example, a device type and a device state. If a WFD device receives a probe request frame including a WFD IE, in response to this, the WFD device can transmit a probe reply frame including its WFD IE. If a WFD device is linked to an AP and operates as a Wi-Fi P2P device, more than two Wi-Fi transceivers operate logically in one physical device. At this point, in order for WFD device discovery, a Wi-Fi Direct transceiver is used. A probe request frame for WFD device discovery can include a P2P IE in addition to a WFD IE and they can be decoded by a Wi-Fi Direct transceiver.

Then, the WFD source10and the WFD sink20can discover the service capability of each other before a WFD connection setup (S603). In more detail, if one WFD device transmits a service discovery request frame where WFD capability is included as information sub element, in response to this, another WFD device can transmit a service discovery reply frame where its WFD capability is included as an information sub element. In relation to a service discovery procedure, a WFD device for supporting a service discovery procedure as a selective procedure can perform a service discovery procedure with a discovered WFD device for supporting a service discovery procedure. In order to perform a service discovery procedure, a probe request frame and reply frame used for a device discovery procedure can include information for indicating whether a WFD device has a capability for supporting a service discovery procedure.

Then, the WFD source10or the WFD sink20can select a peer WFD device for a WFD connection setup. A peer WFD device for processing a WFD connection setup can be selected by a user's input or a peer WFD device for processing a WFD connection setup can be automatically selected according to a policy.

Then, the WFD device can select a method for a WFD connection setup with the selected peer WFD device in operation S605. In more detail, a WFD device can establish a WFD connection through a Connectivity Scheme of one of Wi-Fi P2P and Tunneled Direct Link Setup (TDLS). The WFD devices can determine a Connectivity Scheme based on Preferred Connectivity information and a linked BSSID sub element delivered in addition to a WFD IE.

If a WFD setup is performed successfully by using Wi-Fi P2P or TDLS between WFD devices, a WFD device can process WFD capability negotiation in operation S607. In more detail, the WFD source10and the WFD sink20can exchange a message by using the Real-Time Streaming Protocol (RTSP) to determine a parameter set for defining an audio/video payload during a WFD session. The WFD source10and the WFD sink20can exchange their specification information. The WFD source10can inquire the capability of the WFD sink20by using the RTSP protocol and in response to this, the WFD sink20can transmit its capability information to the WFD source10. For example, the WFD sink20can transmit resolution information including its supportable resolutions to the WFD source10. For example, the WFD sink20can transmit resolution information that its supportable resolution includes a resolution corresponding to a 16:9 or 4:3 aspect ratio to the WFD source10. Herein, a resolution corresponding to 16:9 or 4:3 is just one example, and can be a resolution corresponding to an aspect ratio that a horizontal length is longer than a vertical length.

On the other hand, the WFD source10can also transmit resolution information including its supportable resolutions to the WFD sink20.

FIGS. 7A to 7Dare views illustrating resolution information that a WFD sink transmits to a WFD source according to an embodiment of the present invention.

The WFD sink20can transmit at least one of resolution tables shown inFIGS. 7A to 7Dto the WFD source10. That is, the WFD sink20can transmit a resolution corresponding to a 16:9 or 4:3 aspect ratio and a resolution corresponding to an aspect ratio that a horizontal length is longer than a vertical length to the WFD source10.

The resolution table shown inFIG. 7Acan be a table for representing a set of resolutions supported by a handheld device that a WFD device supports and a refresh rate corresponding to a resolution.

The resolution table shown inFIG. 7Bcan be a table for representing a set of Consumer Electronics Association (CEA) resolutions supported by a WFD device and a refresh rate corresponding to a resolution.

The resolution table shown inFIG. 7Ccan be a table for representing a set of Video Electronics Standards Association (VESA) resolutions supported by a WFD device and a refresh rate corresponding to a resolution.

FIG. 7Dis a view illustrating a message that a WFD source inquires the capability of a WFD sink and a WFD sink includes its capability information in correspondence thereto.

After performing a connection setup with the WFD sink20, the WFD source10can transmit a request message M3for inquiring the capability of the WFD sink20by using the RTSP protocol. Referring toFIG. 7D, the WFD source10can transmit a request message including a wfd-video-formats parameter730to the WFD sink20. The wfd-video-formats parameter730can be a parameter that the WFD sink20inquires a supportable video resolution. In response to the request message received from the WFD source10, the WFD sink20can transmit a request message M3including the resolution information shown inFIGS. 7A to 7Dto the WFD source10. The resolution information can include resolution tables shown inFIGS. 7A to 7C. This can be checked through a value of the video format parameter730in the replay message.

During a WFD capability negotiation process with the WFD sink20, the WFD source10can transmit a necessary request message to the WFD sink20by using the RTSP protocol. For example, the WFD source10can transmit, to the WFD sink20by using the RTSP protocol, a message including content that an image of a video content corresponding to a specific resolution is to be delivered based on resolution information received from the WFD sink20. In more detail, the WFD source10can transmit, to the WFD sink20by using the RTSP protocol, a message including content that an image of a video content corresponding to a specific resolution is to be delivered based on resolution information received from the WFD sink20. That is, the WFD source10can select one of a plurality of resolutions included in resolution information shown inFIGS. 7A to 7D, and transmit a message for representing that an image of the selected resolution is to be transmitted to the WFD sink20. As described later, a resolution that the WFD source10selects cannot correspond to the resolution of an image that the WFD source10displays currently.

The WFD sink20can transmit a reply message through the RTSP protocol on the basis of a request message received from the WFD source10in operation S611.

According to an embodiment of the present invention, a resolution that the WFD source10selects cannot be included in resolutions that the WFD source10supports. That is, a resolution that the WFD source10displays currently cannot be included in resolutions that the WFD sink20supports. Operation S609and operation S611can be included in operation S607for WFD capability negotiation.

If the WFD capability negotiation operation is terminated successfully, a WFD session (or a Mira multicast session) is established between the WFD source10and the WFD sink20, and audio and video contents can be streamed from the WFD source10to the WFD sink20in operation S613.

According to an embodiment, the WFD sink20may display the entire image of the streamed video contents.

In the following description, it is assumed that the WFD source11may be the mobile terminal500described with reference toFIG. 5and the WFD sink20may be the display device100described with reference toFIG. 1.

Accordingly, the WFD source10may include the entire components of the mobile terminal500shown inFIG. 5. Similarly, the WFD sink20may include all the components of the display device100shown inFIG. 1.

FIG. 8is a ladder diagram showing an operating method of a wireless system according to another embodiment of the present invention.

Referring toFIG. 8, the WFD sink20receives a mirroring connection request from the WFD source10through the communication unit2(S801).

In one embodiment, the mirroring connection request may be a request to wirelessly perform a mirroring function.

The processor9of the WFD sink20determines whether the WFD source10is a registered device based on the mirroring connection request (S803).

In one embodiment, the WFD sink20may register the WFD source10that has previously performed the mirroring connection. An average peak luminance (APL) value of a black image (e.g., letter box) transmitted for matching the identifier (e.g., MAC address) of the WFD source10at the time of registration and the resolution at the time of mirroring is stored in the memory3.

When the processor9of the WFD sink20receives the mirroring connection request, the processor9may compare the identifier of the WFD source10that has transmitted the mirroring connection request with the identifier stored in the memory3, and may determine whether the WFD source10is a pre-registered device.

To this end, the mirroring connection request may include the identifier of the WFD source10.

When the identifiers coincide with each other, the processor9of the WFD sink20may determine that the WFD source10that has transmitted the mirroring connection request is a registered device.

When the identifiers do not coincide with each other, the processor9of the WFD sink20may determine that the WFD source10that has transmitted the mirroring connection request is an unregistered device.

When the WFD source10is a registered device, the processor9of the WFD sink20acquires the APL value of the letter box transmitted for matching the resolution at the time of mirroring from the memory3(S805).

In one embodiment, the mirroring image may include an active image and a letter box.

The active image may be an image that is actually displayed through the display of the WFD source10, and the letter box may be a black image that is inserted for matching the resolution.

When the screen ratio of the display of the WFD source10does not match the screen ratio of the display of the WFD sink20, the image transmitted by the WFD source10may be output without matching the resolution in the WFD sink20

Therefore, the WFD sink20needs to know the screen ratio of the WFD source10.

The reason why the WFD sink20obtains the APL value of the letter box is to confirm the screen ratio of the WFD source10.

Thereafter, the processor9of the WFD sink20transmits a WFD mirroring connection permission message807to the WFD source10through the communication unit2(S807).

The processor9of the WFD sink20receives the mirroring image from the WFD source10through the communication unit2(S809).

The processor9of the WFD sink20extracts active region information of the mirroring image by using the obtained APL value (S811).

A region formed by the active image may be referred to as an active region, and a region formed by the letter box may be referred to as an inactive region.

The active region information may be information indicating the position of the active region.

The active region information may include at least one of a horizontal length of the active region, a vertical length of the active region, and coordinates of vertices of the active region.

The processor9of the WFD sink20determines the screen layout mode of the WFD source10based on the extracted active region information (S813).

The processor9of the WFD sink20may determine the screen layout mode of the WFD source10by comparing the horizontal length and the vertical length of the active region.

This will be described below with reference toFIGS. 9 to 10.

FIGS. 9 and 10are views for describing the process for determining the screen layout mode of the WFD sink when the screen is shared between the WFD source and the WFD sink according to an embodiment of the present invention.

Hereinafter, it is assumed that the WFD source10is the mobile terminal500described with reference toFIG. 5and the WFD sink20is the display device100described with reference toFIG. 1.

Also, it is assumed that the mobile terminal500is in a vertical mode (or a portrait mode). The portrait mode may be a mode in which the mobile terminal500is disposed so that a vertical length is longer than a horizontal length. A mode opposite to the portrait mode is a horizontal mode (or a landscape mode).

Referring toFIG. 9, the mobile terminal500may transmit a content image910being displayed on the display unit551to the display device100through the screen mirroring function.

Before the screen mirroring function, the display device100displays another content image901on the display unit180.

The mobile terminal500may transmit, to the display device100, the mirroring image900including letter boxes920and930on both sides of the content image910.

That is, the first letter box920may be displayed on the left side of the content image910, and the second letter box930may be displayed on the right side of the content image910.

Each of the letter boxes may be an image inserted due to a mismatching between the screen ratio of the display device100and the screen ratio of the mobile terminal500.

The display device100may obtain the active region information by using the APL value of the letter box. That is, the display device100may determine, as the letter box, the image having the APL value equal to the APL value of the letter box acquired in step S805in the mirroring image900.

To this end, the display device100may include a measurement circuit capable of measuring the APL value of the image.

The display device100may determine an image other than the letter box as the active image.

The display device100may acquire the coordinate values of the vertices a, b, c, and d of the active region1010forming the active image910, as shown inFIG. 10.

The display device100may calculate the horizontal length x1and the vertical length y1of the active region1010by using the obtained coordinate values of the vertices a, b, c, and d.

When the horizontal length x1of the active region1010is smaller than the vertical length y1of the active region1010, he display device100may determine that the screen layout mode of the display device100is the portrait mode.

When the horizontal length x1of the active region1010is larger than the vertical length y1of the active region1010, he display device100may determine that the screen layout mode of the display device100is the landscape mode.

When the processor9of the WFD sink20determines that the screen layout mode of the WFD source10is the portrait mode (S815), the letter box other than the active image is removed from the mirroring image (S817).

After removing the letter box, the processor9of the WFD sink20determines the screen ratio of the WFD source10based on the active region information (S819).

Similarly, when the processor9of the WFD sink20determines that the screen layout mode of the WFD source10is the landscape mode (S815), the processor9of the WFD sink20determines the screen ratio of the WFD source10based on the active region information (S819).

Even when the screen layout mode of the WFD source10is the landscape mode, the letter box may be inserted into the active image. In this case as well, the WFD sink20may remove the letter box from the mirroring image and determine the screen ratio of the WFD source10based on the active region information.

The processor9of the WFD sink20can determine the screen ratio of the WFD source10based on the horizontal length or the vertical length of the active region included in the active region information.

This will be described below with reference toFIGS. 11A and 11B.

FIG. 11Ais a diagram for describing an example of determining the screen ratio of the WFD source10based on the active region information when the WFD source is in the portrait mode according to an embodiment of the present invention, andFIG. 11Bis a diagram for describing an example of determining the screen ratio of the WFD source10based on the active region information when the WFD source is in the landscape mode according to an embodiment of the present invention.

Referring toFIG. 11A, the memory3of the WFD sink20may store a first screen ratio table1100indicating a correspondence relationship between the screen ratio and the horizontal length range of the active region.

The WFD sink20may determine the screen ratio that matches the horizontal length of the active region by using the first screen ratio table1100.

For example, when the horizontal length of the active region is within the range of 500 to 532, the WFD sink20may determine that the screen ratio of the WFD source10is 18.5:9.

For example, when the horizontal length of the active region is within the range of 533 to 578, the WFD sink20may determine that the screen ratio of the WFD source10is 18:9.

For example, when the horizontal length of the active region is within the range of 579 to 645, the WFD sink20may determine that the screen ratio of the WFD source10is 16:9.

For example, when the horizontal length of the active region is within the range of 646 to 736, the WFD sink20may determine that the screen ratio of the WFD source10is 16:10.

For example, when the horizontal length of the active region is within the range of 737 to 1079, the WFD sink20may determine that the screen ratio of the WFD source10is 4:3.

Referring toFIG. 11B, the memory3of the WFD sink20may store a second screen ratio table1110indicating a correspondence relationship between the screen ratio and the vertical length range of the active region.

FIG. 11Bis an embodiment that can be applied when the screen layout mode of the WFD source10is in the landscape mode.

When the screen layout mode of the WFD source10is the landscape mode, there may be no letter box. Therefore, the region formed by the mirroring image may be the active region itself.

The WFD sink20may determine the screen ratio that matches the vertical length of the active region by using the second screen ratio table1110.

For example, when the vertical length of the active region is within the range of 500 to 532, the WFD sink20may determine that the screen ratio of the WFD source10is 18.5:9.

For example, when the vertical length of the active region is within the range of 533 to 578, the WFD sink20may determine that the screen ratio of the WFD source10is 18:9.

For example, when the vertical length of the active region is within the range of 579 to 645, the WFD sink20may determine that the screen ratio of the WFD source10is 16:9.

For example, when the vertical length of the active region is within the range of 646 to 736, the WFD sink20may determine that the screen ratio of the WFD source10is 16:10.

For example, when the vertical length of the active region is within the range of 737 to 1079, the WFD sink20may determine that the screen ratio of the WFD source10is 4:3.

Thus, even if the WFD sink20does not know the screen ratio of the unregistered WFD source, the active region information is used to recognize the screen ratio of the WFD source, thereby providing an optimal screen sharing service.

The processor9of the WFD sink20displays an active image having the determined screen ratio on the display unit4(S821).

This will be described below with reference toFIG. 12.

FIG. 12is a view for describing an example of performing the screen sharing service between the WFD source and the WFD sink according to the determined screen ratio of the WFD source according to an embodiment of the present invention.

Referring toFIG. 12, the mobile terminal500displays the content image910on the display unit551.

The display device100may receive the mirroring image900including the content image910and the letter boxes920and930from the mobile terminal500.

The display device100may remove the letter boxes920and930from the mirroring image900and acquire the content image910.

The display device100may determine the screen ratio of the display unit551of the mobile terminal500by using the horizontal length of the active region forming the content image910.

The display device100may display the active image910having the resolution according to the determined screen ratio through the display unit180.

The display device100may display the active image910in a superimposed manner on the content image901being displayed through the display unit180.

Similarly, when the processor9of the WFD sink20determines that the screen layout mode of the WFD source10is the landscape mode (S815), the processor9of the WFD sink20determines the screen ratio of the WFD source10based on the active region information, without removing the letter box (S819).

On the other hand, in step S803, when the WFD source10is not a registered device, the processor9of the WFD sink20transmits the mirroring connection permission message to the WFD source10(S823) and receives the mirroring image from the WFD source10(S825).

The processor9of the WFD sink20acquires the APL value corresponding to the specific coordinates of the received mirroring image (S827).

In one embodiment, the specific coordinates of the mirroring image may be (0, 0). However, the present invention is not limited thereto, and the coordinates corresponding to each of the vertices of the region formed by the mirroring image may be a specific coordinates.

This will be described below with reference toFIG. 13.

FIG. 13is a diagram for describing a process of obtaining an APL value when an APL value for a letter box of a WFD source is not stored, according to an embodiment of the present invention.

FIG. 13shows the mirroring image900transmitted by the mobile terminal500to the display device100for the screen sharing service.

When the mobile terminal500is not a pre-registered device and thus does not know the APL values of the pixels constituting the letter boxes920and930, the display device100may measure the APL value of the pixel corresponding to specific coordinates h1of the mirroring image900.

The specific coordinates h1may be (0, 0), but this is merely an example. The specific coordinates h1may be coordinates of any one of the vertices h2, h3, and h4of the mirroring image region forming the mirroring image900.

The reason why the specific coordinates are selected as the vertices of the mirroring image region forming the mirroring image900is that the black image may be included in the active image910.

Specifically, when the active image910itself includes the black image, the APL value at a point that is not related to the active image may be determined as the APL value of the letter box so as to distinguish between the black image and the letter box constituted by separate black images.

The display device100may measure the APL value of the pixel corresponding to the specific coordinates h1and determine the measured APL value as the APL values of the letter boxes920and930.

The determined APL value may be stored in the display device100after matching the identifier of the mobile terminal500at the time of device registration for the mirroring of the mobile terminal500.

Thereafter, the processor9of the WFD sink20returns to step S811to extract active region information of the mirroring image by using the obtained APL value, and sequentially perform steps S813to S821.

On the other hand, according to another embodiment of the present invention, when the screen ratio of the WFD source10is determined, an error may occur in the determination of the screen ratio when the active image is positioned only at a specific position in the mirroring image transmitted by the WFD source10.

That is, when the active image itself includes the black image having the same APL value as that of the letter box and the content is included only at a specific position, the WFD sink20may erroneously determine the screen ratio of the WFD source10.

In this case, the WFD sink20can prevent an error in determining the screen ratio based on the x-coordinate value of the active region.

FIGS. 14 and 15are diagrams for describing a processing method when the active image includes the black image having the same APL value as that of the letter box and the content is included only at a specific position.

FIG. 14shows a mirroring image1400transmitted to the display device100by the mobile terminal500.

The mirroring image1400may include left-shifted content1410and the other black images.

When the x coordinate value x3of the rightmost end point of the content1410is less than a reference lower limit value and the APL value of the black image is equal to the prestored APL value, the display device100may determine that the mobile terminal500is disposed in the landscape mode.

If there is no setting of the reference lower limit value, the display device100will have left only the region of the content1410and removed all the remaining black images. In this case, the display device100may erroneously determine the screen layout mode and the screen ratio of the mobile terminal500, and the content1410may be displayed in a distorted state.

As described above, according to the embodiment of the present invention, the reference lower limit value is set for the x coordinate value of the active area, thereby solving the problem that the screen layout mode of the mobile terminal500is erroneously determined.

FIG. 15shows a mirroring image1500transmitted to the display device100by the mobile terminal500.

The mirroring image1500may include right-shifted content1510and the other black images.

When the sum (x4+d) of the x-coordinate value x4of the leftmost end point of the content1510and the horizontal length d of the active region exceeds a reference upper limit value and the APL value of the black image is equal to the prestored APL value, the display device100may determine that the mobile terminal500is disposed in the landscape mode.

If there is no setting of the reference upper limit value, the display device100will have left only the region of the content1410and removed all the remaining black images. In this case, the display device100may erroneously determine the screen layout mode and the screen ratio of the mobile terminal500, and the content1510may be displayed in a distorted state.

As described above, according to the embodiment of the present invention, the reference upper limit value is set for the x coordinate value of the active area, thereby solving the problem that the screen layout mode of the mobile terminal500is erroneously determined.

On the other hand, according to the embodiment of the present invention, the WFD source10may transmit information on the screen layout mode thereof to the WFD sink20in step S607or S609ofFIG. 6.

The information on the screen layout mode may include a rotational angle of the WFD source10.

For example, when the rotational angle is 90 degrees, it may indicate that the screen layout mode of the WFD source10is the landscape mode.

As the information on the screen layout mode is transmitted to the WFD sink20, it can be used to verify whether the screen layout mode of S813ofFIG. 8has been correctly determined.

According to various embodiments of the present invention, when the screen sharing service is provided, the user can view an optimal image without distortion.