Mobile terminal and controlling method thereof

A mobile terminal including a display; a wireless communication processor configured to transceive data with a drone and an external device through wireless communication; and a controller configured to display a mobile terminal graphic object corresponding to the mobile terminal, an external device graphic object corresponding to the external device, a drone control menu object and a camera control menu object on the display, assign the external device to control the drone in response to the external device graphic object and the drone control menu object being selected, and assign the external device to control the camera on the drone in response to the external device graphic object and the camera control menu object being selected.

Pursuant to 35 U.S.C. § 119(a), this application claims the benefit of earlier filing date and right of priority to Korean Application No. 10-2015-0126341, filed on Sep. 7, 2015, the contents of which are hereby incorporated by reference herein in their entirety.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to a mobile terminal, and more particularly, to a mobile terminal and controlling method thereof. Although the present invention is suitable for a wide scope of applications, it is particularly suitable for setting functions related to a drone control in an external device in order to control a drone connected to a mobile terminal and a camera included on the drone through a plurality of devices.

Discussion of the Related Art

A mobile terminal can perform various functions. Examples of such functions include data and voice communications, capturing images and video via a camera, recording audio, playing music files and outputting music via a speaker system, and displaying images and video on a display. Some terminals include additional functionality which supports game playing, while other terminals are also configured as multimedia players. More recently, mobile terminals have been configured to receive broadcast and multicast signals which permit viewing of contents, such as videos and television programs.

Generally, terminals can be classified into mobile terminals and stationary terminals. Further, the mobile terminals can be classified into handheld terminals and vehicle mounted terminals. There are ongoing efforts to support and increase the functionality of mobile terminals. Such efforts include software and hardware improvements, as well as changes and improvements in the structural components which form the mobile terminal.

Recently, drones operable by individuals or companies are increasingly on the rise. In more detail, a drone is an unmanned flying object in a shape of a plane or helicopter controlled to fly by control signals of radio waves, for example. While a drone is flying, a user can take a video or images through a camera provided on the drone. However, controlling the drone and camera is inconvenient for the user.

SUMMARY OF THE INVENTION

Accordingly, the present invention is directed to a mobile terminal and controlling method thereof that substantially obviate one or more of the problems due to limitations and disadvantages of the related art.

One object of the present invention is to provide a mobile terminal and controlling method thereof, by which a drone can be controlled in accordance with a function assigned to each of a plurality of devices connected to the drone in case of making a video or taking an image for the drone.

Technical tasks obtainable from the present invention are non-limited by the above-mentioned technical tasks. In addition, other unmentioned technical tasks can be clearly understood from the following description by those having ordinary skill in the technical field to which the present invention pertains.

To achieve these and other advantages and in accordance with the purpose of the present invention, as embodied and broadly described, a mobile terminal according to one embodiment the present invention includes a display, a wireless communication processor configured to transceive data with a drone and an external device through wireless communication, and a controller configured to: display a mobile terminal graphic object corresponding to the mobile terminal, an external device graphic object corresponding to the external device, a drone control menu object and a camera control menu object on the display, assign the external device to control the drone in response to the external device graphic object and the drone control menu object being selected, and assign the external device to control the camera on the drone in response to the external device graphic object and the camera control menu object being selected.

To further achieve these and other advantages and in accordance with the purpose of the present invention, as embodied and broadly described, a mobile terminal according to one embodiment the present invention includes a display, a wireless communication unit configured to transceive data with a drone and at least one external device through wireless communication, and a controller configured to: connect to the drone in response to a connecting signal received from the external device, and display a drone control interface for controlling the drone and a camera included on the drone in response to an assignment signal from the external device assigning a control function of the drone to the mobile terminal received.

It is to be understood that both the foregoing general description and the following detailed description of the preferred embodiments of the present invention are exemplary and explanatory and are intended to provide further explanation of the invention as claimed.

Effects obtainable from the present invention may be non-limited by the above mentioned effect. In addition, other unmentioned effects can be clearly understood from the following description by those having ordinary skill in the technical field to which the present invention pertains.

DETAILED DESCRIPTION OF THE INVENTION

Although the terms first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are generally only used to distinguish one element from another. When an element is referred to as being “connected with” another element, the element can be connected with the other element or intervening elements may also be present. In contrast, when an element is referred to as being “directly connected with” another element, there are no intervening elements present.

The mobile terminal100is shown having components such as a wireless communication unit110, an input unit120, a sensing unit140, an output unit150, an interface unit160, a memory170, a controller180, and a power supply unit190. Implementing all of the illustrated components is not a requirement, and that greater or fewer components may alternatively be implemented. Referring now toFIG. 1A, the mobile terminal100is shown having wireless communication unit110configured with several commonly implemented components.

The controller180typically functions to control overall operation of the mobile terminal100, in addition to the operations associated with the application programs. The controller180processes signals, data, and information input or output through the components mentioned in the foregoing description or runs an application program saved in the memory170, thereby providing or processing information or function appropriate for to a user.

At least one portion of the respective components can cooperatively operate to implement operations, controls or controlling methods of a mobile terminal according to various embodiments of the present invention mentioned in the following description. The operations, controls or controlling methods of the mobile terminal can be implemented on the mobile terminal by running at least one application program saved in the memory170.

The broadcast managing entity may be implemented using a server or system which generates and transmits a broadcast signal and/or broadcast associated information, or a server which receives a pre-generated broadcast signal and/or broadcast associated information, and sends such items to the mobile terminal. The broadcast signal may be implemented using any of a TV broadcast signal, a radio broadcast signal, a data broadcast signal, and combinations thereof, among others. The broadcast signal in some cases may further include a data broadcast signal combined with a TV or radio broadcast signal.

The broadcast signal may be encoded according to any of a variety of technical standards or broadcasting methods (for example, International Organization for Standardization (ISO), International Electrotechnical Commission (IEC), Digital Video Broadcast (DVB), Advanced Television Systems Committee (ATSC), and the like) for transmission and reception of digital broadcast signals. The broadcast receiving module111can receive the digital broadcast signals using a method appropriate for the transmission method utilized.

Examples of broadcast associated information may include information associated with a broadcast channel, a broadcast program, a broadcast event, a broadcast service provider, or the like. The broadcast associated information may also be provided via a mobile communication network, and in this instance, received by the mobile communication module112.

The broadcast associated information may be implemented in various formats. For instance, broadcast associated information may include an Electronic Program Guide (EPG) of Digital Multimedia Broadcasting (DMB), an Electronic Service Guide (ESG) of Digital Video Broadcast-Handheld (DVB-H), and the like. Broadcast signals and/or broadcast associated information received via the broadcast receiving module111may be stored in a suitable device, such as a memory170.

Examples of wireless signals transmitted and/or received via the mobile communication module112include audio call signals, video (telephony) call signals, or various formats of data to support communication of text and multimedia messages. The wireless Internet module113is configured to facilitate wireless Internet access. This module may be internally or externally coupled to the mobile terminal100. The wireless Internet module113may transmit and/or receive wireless signals via communication networks according to wireless Internet technologies.

A touch sensor can sense a touch applied to the touch screen, such as display unit151, using any of a variety of touch methods. Examples of such touch methods include a resistive type, a capacitive type, an infrared type, and a magnetic field type, among others. As one example, the touch sensor may be configured to convert changes of pressure applied to a specific part of the display unit151, or convert capacitance occurring at a specific part of the display unit151, into electric input signals. The touch sensor may also be configured to sense not only a touched position and a touched area, but also touch pressure and/or touch capacitance. A touch object is generally used to apply a touch input to the touch sensor. Examples of typical touch objects include a finger, a touch pen, a stylus pen, a pointer, or the like.

In general, a 3D stereoscopic image may include a left image (e.g., a left eye image) and a right image (e.g., a right eye image). According to how left and right images are combined into a 3D stereoscopic image, a 3D stereoscopic imaging method can be divided into a top-down method in which left and right images are located up and down in a frame, an L-to-R (left-to-right or side by side) method in which left and right images are located left and right in a frame, a checker board method in which fragments of left and right images are located in a tile form, an interlaced method in which left and right images are alternately located by columns or rows, and a time sequential (or frame by frame) method in which left and right images are alternately displayed on a time basis.

Also, as for a 3D thumbnail image, a left image thumbnail and a right image thumbnail can be generated from a left image and a right image of an original image frame, respectively, and then combined to generate a single 3D thumbnail image. In general, the term “thumbnail” may be used to refer to a reduced image or a reduced still image. A generated left image thumbnail and right image thumbnail may be displayed with a horizontal distance difference there between by a depth corresponding to the disparity between the left image and the right image on the screen, thereby providing a stereoscopic space sense.

A left image and a right image required for implementing a 3D stereoscopic image may be displayed on the stereoscopic display unit using a stereoscopic processing unit. The stereoscopic processing unit can receive the 3D image and extract the left image and the right image, or can receive the 2D image and change it into a left image and a right image.

An optical output module154can output a signal for indicating an event generation using light of a light source. Examples of events generated in the mobile terminal100may include message reception, call signal reception, a missed call, an alarm, a schedule notice, an email reception, information reception through an application, and the like. A signal output by the optical output module154may be implemented so the mobile terminal emits monochromatic light or light with a plurality of colors. The signal output may be terminated as the mobile terminal senses that a user has checked the generated event, for example.

The power supply unit190may include a connection port. The connection port may be configured as one example of the interface unit160to which an external charger for supplying power to recharge the battery is electrically connected. As another example, the power supply unit190may be configured to recharge the battery in a wireless manner without use of the connection port. In this example, the power supply unit190can receive power, transferred from an external wireless power transmitter, using at least one of an inductive coupling method which is based on magnetic induction or a magnetic resonance coupling method which is based on electromagnetic resonance. Various embodiments described herein may be implemented in a computer-readable medium, a machine-readable medium, or similar medium using, for example, software, hardware, or any combination thereof.

The cases101,102,103may be formed by injection-molding synthetic resin or may be formed of a metal, for example, stainless steel (STS), aluminum (Al), titanium (Ti), or the like. As an alternative to the example in which the plurality of cases form an inner space for accommodating components, the mobile terminal100may be configured such that one case forms the inner space. In this example, a mobile terminal100having a uni-body is formed so synthetic resin or metal extends from a side surface to a rear surface.

The mobile terminal100may be equipped with the display unit151, the first audio output unit152a, the second audio output unit152b, the proximity sensor141, the illumination sensor142, the optical output unit154, the first camera121a, the second camera121b, the first manipulating unit123a, the second manipulating unit123b, the microphone122, the interface unit160, and the like.

The touch sensor may be configured in a form of a film having a touch pattern, disposed between the window151aand a display on a rear surface of the window151a, or a metal wire which is patterned directly on the rear surface of the window151a. Alternatively, the touch sensor may be integrally formed with the display. For example, the touch sensor may be disposed on a substrate of the display or within the display. The display unit151may also form a touch screen together with the touch sensor. Here, the touch screen may serve as the user input unit123(seeFIG. 1A). Therefore, the touch screen may replace at least some of the functions of the first manipulation unit123a.

Embodiments that include the rear input unit may implement some or all of the functionality of the first manipulation unit123ain the rear input unit. As such, in situations where the first manipulation unit123ais omitted from the front side, the display unit151can have a larger screen. As a further alternative, the mobile terminal100may include a finger scan sensor which scans a user's fingerprint. The controller180can then use fingerprint information sensed by the finger scan sensor as part of an authentication procedure. The finger scan sensor may also be installed in the display unit151or implemented in the user input unit123.

As shown inFIG. 1C, a flash124is shown adjacent to the second camera121b. When an image of a subject is captured with the camera121b, the flash124may illuminate the subject. As shown inFIG. 1C, the second audio output module152bcan be located on the terminal body. The second audio output module152bmay implement stereophonic sound functions in conjunction with the first audio output module152a, and may be also used for implementing a speaker phone mode for call communication.

Further, according to an embodiment of the present invention, information processed by the mobile terminal can be displayed using a flexible display. This is described in detail with reference to the accompanying drawings as follows.

FIG. 2is a conceptual view of a deformable mobile terminal according to an alternative embodiment of the present invention. As shown, the mobile terminal200includes a display unit251, which is a type of display that is deformable by an external force. This deformation, which includes display unit251and other components of mobile terminal200, may include any of curving, bending, folding, twisting, rolling, and combinations thereof. The deformable display unit251may also be referred to as a “flexible display unit.” In some implementations, the flexible display unit251may include a general flexible display, electronic paper (also known as e-paper), and combinations thereof. In general, mobile terminal200may be configured to include features that are the same or similar to that of mobile terminal100ofFIGS. 1A-1C.

The flexible display of mobile terminal200is generally formed as a lightweight, non-fragile display, which still exhibits characteristics of a conventional flat panel display, but is instead fabricated on a flexible substrate which can be deformed as noted previously. The term e-paper may be used to refer to a display technology employing the characteristic of a general ink, and is different from the conventional flat panel display in view of using reflected light. E-paper is generally understood as changing displayed information using a twist ball or via electrophoresis using a capsule.

When the flexible display unit251is not deformed (for example, in a state with an infinite radius of curvature and referred to as a first state), a display region of the flexible display unit251includes a generally flat surface. When the flexible display unit251is deformed from the first state by an external force (for example, a state with a finite radius of curvature and referred to as a second state), the display region may become a curved surface or a bent surface. As illustrated, information displayed in the second state may be visual information output on the curved surface. The visual information may be realized so a light emission of each unit pixel (sub-pixel) arranged in a matrix configuration is controlled independently. The unit pixel denotes an elementary unit for representing one color.

According to one alternative embodiment, the first state of the flexible display unit251may be a curved state (for example, a state of being curved from up to down or from right to left), instead of being in flat state. In this embodiment, when an external force is applied to the flexible display unit251, the flexible display unit251may transition to the second state such that the flexible display unit is deformed into the flat state (or a less curved state) or into a more curved state.

If desired, the flexible display unit251may implement a flexible touch screen using a touch sensor in combination with the display. When a touch is received at the flexible touch screen, the controller180can execute certain control corresponding to the touch input. In general, the flexible touch screen is configured to sense touch and other input while in both the first and second states.

One option is to configure the mobile terminal200to include a deformation sensor which senses the deforming of the flexible display unit251. The deformation sensor may be included in the sensing unit140. The deformation sensor may be located in the flexible display unit251or the case201to sense information related to the deforming of the flexible display unit251. Examples of such information related to the deforming of the flexible display unit251may be a deformed direction, a deformed degree, a deformed position, a deformed amount of time, an acceleration that the deformed flexible display unit251is restored, and the like. Other possibilities include most any type of information which can be sensed in response to the curving of the flexible display unit or sensed while the flexible display unit251is transitioning into, or existing in, the first and second states.

In some embodiments, controller180or other component can change information displayed on the flexible display unit251, or generate a control signal for controlling a function of the mobile terminal200, based on the information related to the deforming of the flexible display unit251. Such information is typically sensed by the deformation sensor. The mobile terminal200is shown having a case201for accommodating the flexible display unit251. The case201can be deformable together with the flexible display unit251, taking into account the characteristics of the flexible display unit251.

A battery located in the mobile terminal200may also be deformable in cooperation with the flexible display unit261, taking into account the characteristic of the flexible display unit251. One technique to implement such a battery is to use a stack and folding method of stacking battery cells.

The deformation of the flexible display unit251not limited to perform by an external force. For example, the flexible display unit251can be deformed into the second state from the first state by a user command, application command, or the like. In accordance with still further embodiments, a mobile terminal may be configured as a device which is wearable on a human body. Such devices go beyond the usual technique of a user grasping the mobile terminal using their hand. Examples of the wearable device include a smart watch, a smart glass, a head mounted display (HMD), and the like.

A typical wearable device can exchange data with (or cooperate with) another mobile terminal100. In such a device, the wearable device generally has functionality that is less than the cooperating mobile terminal. For instance, the short-range communication module114of a mobile terminal100may sense or recognize a wearable device that is near-enough to communicate with the mobile terminal. In addition, when the sensed wearable device is a device which is authenticated to communicate with the mobile terminal100, the controller180can transmit data processed in the mobile terminal100to the wearable device via the short-range communication module114, for example. Hence, a user of the wearable device can use the data processed in the mobile terminal100on the wearable device. For example, when a call is received in the mobile terminal100, the user can answer the call using the wearable device. Also, when a message is received in the mobile terminal100, the user can check the received message using the wearable device.

FIG. 3is a perspective view illustrating one example of a watch-type mobile terminal300in accordance with another exemplary embodiment. As illustrated inFIG. 3, the watch-type mobile terminal300includes a main body301with a display unit351and a band302connected to the main body301to be wearable on a wrist. In general, mobile terminal300may be configured to include features that are the same or similar to that of mobile terminal100ofFIGS. 1A-1C.

The main body301may include a case having a certain appearance. As illustrated, the case may include a first case301aand a second case301bcooperatively defining an inner space for accommodating various electronic components. Other configurations are possible. For instance, a single case may alternatively be implemented, with such a case being configured to define the inner space, thereby implementing a mobile terminal300with a uni-body.

The watch-type mobile terminal300can perform wireless communication, and an antenna for the wireless communication can be installed in the main body301. The antenna may extend its function using the case. For example, a case including a conductive material may be electrically connected to the antenna to extend a ground area or a radiation area.

The display unit351is shown located at the front side of the main body301so that displayed information is viewable to a user. In some embodiments, the display unit351includes a touch sensor so that the display unit can function as a touch screen. As illustrated, window351ais positioned on the first case301ato form a front surface of the terminal body together with the first case301a.

The illustrated embodiment includes audio output module352, a camera321, a microphone322, and a user input unit323positioned on the main body301. When the display unit351is implemented as a touch screen, additional function keys may be minimized or eliminated. For example, when the touch screen is implemented, the user input unit323may be omitted.

The band302is commonly worn on the user's wrist and may be made of a flexible material for facilitating wearing of the device. As one example, the band302may be made of fur, rubber, silicon, synthetic resin, or the like. The band302may also be configured to be detachable from the main body301. Accordingly, the band302may be replaceable with various types of bands according to a user's preference.

In one configuration, the band302may be used for extending the performance of the antenna. For example, the band may include therein a ground extending portion electrically connected to the antenna to extend a ground area. The band302may include fastener302a. The fastener302amay be implemented into a buckle type, a snap-fit hook structure, a Velcro® type, or the like, and include a flexible section or material. The drawing illustrates an example that the fastener302ais implemented using a buckle.

FIG. 4is a perspective view illustrating one example of a glass-type mobile terminal400according to another exemplary embodiment. The glass-type mobile terminal400can be wearable on a head of a human body and provided with a frame (case, housing, etc.) therefor. The frame may be made of a flexible material to be easily worn. The frame of mobile terminal400is shown having a first frame401and a second frame402, which can be made of the same or different materials. In general, mobile terminal400may be configured to include features that are the same or similar to that of mobile terminal100ofFIGS. 1A-1C.

The frame may be supported on the head and defines a space for mounting various components. As illustrated, electronic components, such as a control module480, an audio output module452, and the like, may be mounted to the frame part. Also, a lens403for covering either or both of the left and right eyes may be detachably coupled to the frame part.

The control module480controls various electronic components disposed in the mobile terminal400. The control module480may be understood as a component corresponding to the aforementioned controller180.FIG. 4illustrates that the control module480is installed in the frame part on one side of the head, but other locations are possible.

The display unit451may be implemented as a head mounted display (HMD). The HMD refers to display techniques by which a display is mounted to a head to show an image directly in front of a user's eyes. In order to provide an image directly in front of the user's eyes when the user wears the glass-type mobile terminal400, the display unit451may be located to correspond to either or both of the left and right eyes.FIG. 4illustrates that the display unit451is located on a portion corresponding to the right eye to output an image viewable by the user's right eye.

The display unit451may project an image into the user's eye using a prism. Also, the prism may be formed from optically transparent material such that the user can view both the projected image and a general visual field (a range that the user views through the eyes) in front of the user. In such a manner, the image output through the display unit451may be viewed while overlapping with the general visual field. The mobile terminal400may provide an augmented reality (AR) by overlaying a virtual image on a realistic image or background using the display.

The camera421may be located adjacent to either or both of the left and right eyes to capture an image. Since the camera421is located adjacent to the eye, the camera421can acquire a scene that the user is currently viewing. The camera421may be positioned at most any location of the mobile terminal In some embodiments, multiple cameras421may be utilized. Such multiple cameras421may be used to acquire a stereoscopic image.

The glass-type mobile terminal400may include user input units423aand423b, which can each be manipulated by the user to provide an input. The user input units423aand423bmay employ techniques which permit input via a tactile input. Typical tactile inputs include a touch, push, or the like. The user input units423aand423bare shown operable in a pushing manner and a touching manner as they are located on the frame part and the control module480, respectively.

If desired, mobile terminal400may include a microphone which processes input sound into electric audio data, and an audio output module452for outputting audio. The audio output module452may be configured to produce audio in a general audio output manner or an osteoconductive manner. When the audio output module452is implemented in the osteoconductive manner, the audio output module452may be closely adhered to the head when the user wears the mobile terminal400and vibrate the user's skull to transfer sounds.

A communication system which is operable with the variously described mobile terminals will now be described in more detail. Such a communication system may be configured to utilize any of a variety of different air interfaces and/or physical layers. Examples of such air interfaces utilized by the communication system include Frequency Division Multiple Access (FDMA), Time Division Multiple Access (TDMA), Code Division Multiple Access (CDMA), Universal Mobile Telecommunications System (UMTS) (including, Long Term Evolution (LTE), LTE-A (Long Term Evolution-Advanced)), Global System for Mobile Communications (GSM), and the like.

By way of a non-limiting example only, further description will relate to a CDMA communication system, but such teachings apply equally to other system types including a CDMA wireless communication system as well as OFDM (Orthogonal Frequency Division Multiplexing) wireless communication system. A CDMA wireless communication system generally includes one or more mobile terminals (MT or User Equipment, UE)100, one or more base stations (BSs, NodeB, or evolved NodeB), one or more base station controllers (BSCs), and a mobile switching center (MSC). The MSC is configured to interface with a conventional Public Switched Telephone Network (PSTN) and the BSCs. The BSCs are coupled to the base stations via backhaul lines. The backhaul lines may be configured in accordance with any of several known interfaces including, for example, E1/T1, ATM, IP, PPP, Frame Relay, HDSL, ADSL, or xDSL. Hence, the plurality of BSCs can be included in the CDMA wireless communication system.

Each base station may include one or more sectors, each sector having an omni-directional antenna or an antenna pointed in a particular direction radially away from the base station. Alternatively, each sector may include two or more different antennas. Each base station may be configured to support a plurality of frequency assignments, with each frequency assignment having a particular spectrum (e.g., 1.25 MHz, 5 MHz, etc.).

The intersection of sector and frequency assignment may be referred to as a CDMA channel. The base stations may also be referred to as Base Station Transceiver Subsystems (BTSs). In some cases, the term “base station” may be used to refer collectively to a BSC, and one or more base stations. The base stations may also be denoted as “cell sites.” Alternatively, individual sectors of a given base station may be referred to as cell sites.

A broadcasting transmitter (BT) transmits a broadcast signal to the mobile terminals100operating within the system. The broadcast receiving module111ofFIG. 1Ais typically configured inside the mobile terminal100to receive broadcast signals transmitted by the BT.

Global Positioning System (GPS) satellites for locating the position of the mobile terminal100, for example, may cooperate with the CDMA wireless communication system. Useful position information may be obtained with greater or fewer satellites than two satellites. It is to be appreciated that other types of position detection technology, (i.e., location technology that may be used in addition to or instead of GPS location technology) may alternatively be implemented. If desired, at least one of the GPS satellites may alternatively or additionally be configured to provide satellite DMB transmissions.

The location information module115is generally configured to detect, calculate, or otherwise identify a position of the mobile terminal. As an example, the location information module115may include a Global Position System (GPS) module, a Wi-Fi module, or both. If desired, the location information module115may alternatively or additionally function with any of the other modules of the wireless communication unit110to obtain data related to the position of the mobile terminal.

A typical GPS module115can measure an accurate time and distance from three or more satellites, and accurately calculate a current location of the mobile terminal according to trigonometry based on the measured time and distances. A method of acquiring distance and time information from three satellites and performing error correction with a single satellite may be used. In particular, the GPS module may acquire an accurate time together with three-dimensional speed information as well as the location of the latitude, longitude and altitude values from the location information received from the satellites. Furthermore, the GPS module can acquire speed information in real time to calculate a current position. Sometimes, accuracy of a measured position may be compromised when the mobile terminal is located in a blind spot of satellite signals, such as being located in an indoor space. In order to minimize the effect of such blind spots, an alternative or supplemental location technique, such as Wi-Fi Positioning System (WPS), may be utilized.

The Wi-Fi positioning system (WPS) refers to a location determination technology based on a wireless local area network (WLAN) using Wi-Fi as a technology for tracking the location of the mobile terminal100. This technology typically includes the use of a Wi-Fi module in the mobile terminal100and a wireless access point for communicating with the Wi-Fi module.

The Wi-Fi positioning system may include a Wi-Fi location determination server, a mobile terminal, a wireless access point (AP) connected to the mobile terminal, and a database stored with wireless AP information. The mobile terminal connected to the wireless AP may transmit a location information request message to the Wi-Fi location determination server.

The Wi-Fi location determination server extracts the information of the wireless AP connected to the mobile terminal100, based on the location information request message (or signal) of the mobile terminal100. The information of the wireless AP may be transmitted to the Wi-Fi location determination server through the mobile terminal100, or may be transmitted to the Wi-Fi location determination server from the wireless AP.

The information of the wireless AP extracted based on the location information request message of the mobile terminal100may include one or more of media access control (MAC) address, service set identification (SSID), received signal strength indicator (RSSI), reference signal received Power (RSRP), reference signal received quality (RSRQ), channel information, privacy, network type, signal strength, noise strength, and the like.

The Wi-Fi location determination server may receive the information of the wireless AP connected to the mobile terminal100as described above, and may extract wireless AP information corresponding to the wireless AP connected to the mobile terminal from the pre-established database. The information of any wireless APs stored in the database may be information such as MAC address, SSID, RSSI, channel information, privacy, network type, latitude and longitude coordinate, building at which the wireless AP is located, floor number, detailed indoor location information (GPS coordinate available), AP owner's address, phone number, and the like. In order to remove wireless APs provided using a mobile AP or an illegal MAC address during a location determining process, the Wi-Fi location determination server may extract only a predetermined number of wireless AP information in order of high RSSI.

Then, the Wi-Fi location determination server may extract (analyze) location information of the mobile terminal100using at least one wireless AP information extracted from the database. A method for extracting (analyzing) location information of the mobile terminal100may include a Cell-ID method, a fingerprint method, a trigonometry method, a landmark method, and the like.

The Cell-ID method is used to determine a position of a wireless AP having the largest signal strength, among peripheral wireless AP information collected by a mobile terminal, as a position of the mobile terminal. The Cell-ID method is an implementation that is minimally complex, does not require additional costs, and location information can be rapidly acquired. However, in the Cell-ID method, the precision of positioning may fall below a desired threshold when the installation density of wireless APs is low.

The fingerprint method is used to collect signal strength information by selecting a reference position from a service area, and to track a position of a mobile terminal using the signal strength information transmitted from the mobile terminal based on the collected information. In order to use the fingerprint method, it is common for the characteristics of radio signals to be pre-stored in the form of a database.

The trigonometry method is used to calculate a position of a mobile terminal based on a distance between coordinates of at least three wireless APs and the mobile terminal. In order to measure the distance between the mobile terminal and the wireless APs, signal strength may be converted into distance information, Time of Arrival (ToA), Time Difference of Arrival (TDoA), Angle of Arrival (AoA), or the like may be taken for transmitted wireless signals.

The landmark method is used to measure a position of a mobile terminal using a known landmark transmitter. In addition to these position location methods, various algorithms may be used to extract (analyze) location information of a mobile terminal. Such extracted location information may be transmitted to the mobile terminal100through the Wi-Fi location determination server, thereby acquiring location information of the mobile terminal100.

The mobile terminal100can acquire location information by being connected to at least one wireless AP. The number of wireless APs required to acquire location information of the mobile terminal100may be variously changed according to a wireless communication environment within which the mobile terminal100is positioned.

As previously described with regard toFIG. 1A, the mobile terminal may be configured to include short-range communication techniques such as Bluetooth™, Radio Frequency Identification (RFID), Infrared Data Association (IrDA), Ultra Wideband (UWB), ZigBee, Near Field Communication (NFC), Wireless USB (Wireless Universal Serial Bus), and the like.

A typical NFC module provided at the mobile terminal supports short-range wireless communication, which is a non-contactable type of communication between mobile terminals and generally occurs within about 10 cm. The NFC module may operate in one of a card mode, a reader mode, or a P2P mode. The mobile terminal100may further include a security module for storing card information, in order to operate the NFC module in a card mode. The security module may be a physical medium such as Universal Integrated Circuit Card (UICC) (e.g., a Subscriber Identification Module (SIM) or Universal SIM (USIM)), a secure micro SD and a sticker, or a logical medium (e.g., embedded Secure Element (SE)) embedded in the mobile terminal. Single Wire Protocol (SWP)-based data exchange may be performed between the NFC module and the security module.

When the NFC module operates in a card mode, the mobile terminal may transmit card information on a general IC card to the outside. More specifically, if a mobile terminal having card information on a payment card (e. g, a credit card or a bus card) approaches a card reader, a short-range mobile payment may be executed. As another example, if a mobile terminal which stores card information on an entrance card approaches an entrance card reader, an entrance approval procedure may start. A card such as a credit card, a traffic card, or an entrance card may be included in the security module in the form of applet, and the security module may store card information on the card mounted therein. Card information for a payment card may include any of a card number, a remaining amount and usage history, and the like. Card information of an entrance card may include any of a user's name, a user's number (e.g., undergraduate number or staff number), an entrance history, and the like.

When the NFC module operates in a reader mode, the mobile terminal can read data from an external tag. The data received from the external tag by the mobile terminal may be coded into the NFC Data Exchange Format defined by the NFC Forum. The NFC Forum generally defines four record types. More specifically, the NFC Forum defines four Record Type Definitions (RTDs) such as smart poster, text, Uniform Resource Identifier (URI), and general control. If the data received from the external tag is a smart poster type, the controller may execute a browser (e.g., Internet browser). If the data received from the external tag is a text type, the controller may execute a text viewer. If the data received from the external tag is a URI type, the controller may execute a browser or originate a call. If the data received from the external tag is a general control type, the controller may execute a proper operation according to control content.

In some cases in which the NFC module operates in a P2P (Peer-to-Peer) mode, the mobile terminal can execute P2P communication with another mobile terminal. In this instance, Logical Link Control Protocol (LLCP) may be applied to the P2P communication. For P2P communication, connection may be generated between the mobile terminal and another mobile terminal. This connection may be categorized as a connectionless mode which ends after one packet is switched, and a connection-oriented mode in which packets are switched consecutively. For a typical P2P communication, data such as an electronic type name card, address information, a digital photo and a URL, a setup parameter for Bluetooth connection, Wi-Fi connection, etc. may be switched. The P2P mode can be effectively utilized in switching data of a small capacity, because an available distance for NFC communication is relatively short.

Further preferred embodiments will be described in more detail with reference to additional drawing figures. It is understood by those skilled in the art that the present features can be embodied in several forms without departing from the characteristics thereof.

Next,FIG. 5is a diagram illustrating one example of an external appearance of a drone500controlled by a mobile terminal100according to an embodiment of the present invention. Referring toFIG. 5, the drone500includes a propeller guard510, a propeller520, a body530, a camera540and the like. However, the present invention is not limited by the configuration of the drone500shown inFIG. 5. In particular, the present invention is applicable to drones of various types including a helicopter type (e.g., a tricopter with 3 propellers, a quadcopter with 4 propellers shown inFIG. 5, an octacopter with 8 propellers, etc.), an airplane type and the like.

The propeller guard510is configured to prevent a human or animal from being hurt by an operation of the propeller520and may be omitted. The propeller520and the camera540operate in response to a control signal of the body530. In addition, a wireless communication module configured to communicate with a mobile terminal is included in the body530. According to an embodiment, a motion of the camera540can be precisely controlled using a separate control signal different from the control signal of the body530. The configuration of the drone500is described in detail with reference toFIG. 6as follows.

In particular,FIG. 6is a block diagram illustrating one example of configuration modules of a drone according to an embodiment of the present invention. Referring toFIG. 6, the drone500includes a camera540, a control unit550, a communication unit560, a sensing unit570, a battery580, and the like. Such a configuration is not mandatory. According to other embodiments, the drone500may include components more or less than those shown inFIG. 6. A component shown inFIG. 6may also be embodied into at least two components, or at least two components shown inFIG. 6may be integrated into a single component.

In addition, the camera540obtains a still image or a video in accordance with a control signal of the control unit550. The control signal of the control unit550for controlling the camera540can be based on a signal transmitted from the mobile terminal100through the communication unit560. Further, the camera540can be set stationary so as not to have any motion. Alternatively, the camera540can be designed to move at a prescribed angle in a plane formed by an x-axis and y-axis (in addition, planes in parallel with the x-y plane are included) or a plane formed by z-axis (in addition, planes in parallel with the z-axis plane).

According to an embodiment, the camera540can be controlled to face a direction different from a direction faced by the drone500. For example, when the drone500flies in a north direction with reference to a geomagnetic system, the camera540can face a direction other than the north direction. According to an embodiment, the camera540can be controlled by a camera control unit included in the control unit550or a camera control unit configured separately from the control unit550.

In addition, the communication unit560is configured to transmit and receive data through a wireless communication with the wireless communication unit110of the mobile terminal100. In this instance, the wireless communication unit110of the mobile terminal100may include at least one of the mobile communication module112, the wireless internet module113and the short rage communication module114. Further, the communication unit560can receive signals required for the control of the drone500from the mobile terminal100and can also transmit image data obtained through the camera540and sensing data sensed by the sensing unit570to the mobile terminal100.

According to one embodiment, the communication unit560can receive a signal transmitted from a preset external device other than the mobile terminal100. For instance, the external device may include a beacon. In particular, the beacon is a short-range wireless communication device based on Bluetooth 4.0 Protocol, for example. The communication unit560can obtain a location of a beacon using a signal transmitted by an external beacon. Hence, when a user carries the beacon, the drone500can accurately obtain user's location, which may be especially useful for a case that the drone500enables the user to photograph herself/himself in a manner that the drone500follows the corresponding user.

In addition, the sensing unit570may include at least one sensor configured to sense location information of the drone500. In addition, the sensing unit570may include at least one sensor configured to sense state information of the drone500, and more particularly, state information of the camera540included in the drone500. For instance, the state information of the camera540may include a direction information of the camera540. The sensing unit570may also include at least one sensor configured to sense a surrounding environment information around the drone500. For instance, the sensing unit570may include at least one of a GPS sensor, a geomagnetic sensor, an acceleration sensor, a magnetic sensor, a gravity sensor (G-sensor), a gyroscope sensor, a motion sensor, an altitude sensor, an atmospheric sensor, a temperature sensor, and the like.

In addition, the control unit550can transmit the sensing data obtained by the sensing unit570to the mobile terminal100through the communication unit560. The controller500can also combine data sensed by at least two sensors included in the sensing unit570together and can then transmit the combined data to the mobile terminal100through the communication unit560.

In addition, the sensing unit570includes a ultrasonic sensor, an infrared sensor and the like, thereby preventing the drone500from colliding with an object near the drone500. Further, the battery580can supply power required for operations of the respective components of the drone500. In addition, the battery580can be chargeable. Further, the control unit550controls operations of the respective components of the drone500.

Next, a method of controlling a drone and a camera included on the drone in a mobile terminal according to one embodiment of the present invention will be described in detail with reference toFIGS. 7 to 21. In addition, the drone according to one embodiment of the present invention can be controlled simultaneously by a plurality of devices as well as controlled by a single device.

In more detail, one of the plurality of the devices can correspond to a master device, and the rest of the devices can correspond to slave devices. For example, the master device corresponds to a main device and the slave device corresponds to a sub device. Further, a master device can assign or set a drone control function of a slave device. Based on the assignment or settings performed by the master device, the slave device can control a drone and a camera included on the drone. In particular, the slave device can perform only the drone control function assigned by the master device.

Regarding the embodiments shown inFIGS. 7 to 21, operations performed in the mobile terminal can be controlled by the controller180shown inFIG. 1A. For clarity of the following description, such operations are illustrated and described as performed/controlled by the mobile terminal in general. Further, various embodiments of the present invention disclosed in this specification can be implemented individually or by being combined together in part at least.

First,FIGS. 7 to 9show a method of setting a single user mode or a multi user mode for controlling the drone through the mobile terminal according to an embodiment of the present invention. For example,FIG. 7is a diagram illustrating one example of outputting a drone connection interface from a mobile terminal according to an embodiment of the present invention.

Referring toFIG. 7, when intending to control a drone, the user of the mobile terminal can execute a drone control application on a home screen or through a status bar710. In this instance, the status bar710is a graphic user interface (GUI) configured to indicate a current use state of the mobile terminal and can provide the time, Wi-Fi information, switching to vibration/silence, real-time message information, and the like.

Further, if an input signal for a remote control option displayed on the status bar is sensed, the mobile terminal can output a quick menu for the remote control option. Referring toFIG. 7(a), when an input signal for a drone menu option among remote control options for a plurality of devices is sensed, the mobile terminal can output a quick menu720for determining a control mode of the drone. In this instance, the quick menu720can correspond to an interface for connecting to the drone. For example, as shown, the control mode displayed on the drone connection interface may include a single user mode and a multi user mode.

Also, if the drone control application is executed on the home screen, the mobile terminal can display the drone connection interface Referring toFIG. 7(a), the mobile terminal can sense an input signal730for a single user mode menu on the drone connection interface. For example, the input signal730may include a touch input, a gesture input, or the like. In this instance, the mobile terminal can perform a connection (e.g., a pairing) through a wireless communication between the mobile terminal and the drone.

When the connection between the mobile terminal and drone are completed, the mobile terminal can display a drone control interface740. In particular, the drone control interface740can display at least one or more control objects for controlling the drone. For example, the control objects can include a drone motion control, an image photographing control, a video making control, a camera motion control, a function assignment control and the like. According to the embodiment shown inFIG. 7, since the mobile terminal corresponds to the single user mode for controlling the drone solely, the mobile terminal can display all control objects for controlling the drone on the drone control interface740. Moreover, through various control objects displayed on the drone control interface740, a user moves the drone and can perform various jobs such as a photographing job with a camera included on the drone and the like.

Next,FIG. 8is a diagram illustrating one example of outputting a drone connection interface from a mobile terminal according to an embodiment of the present invention. Referring toFIG. 8(a), the mobile terminal can sense a first input signal820for a multi user mode menu on a drone connection interface810. For example, the first input signal820may include a touch input, a gesture input, or the like. In this instance, a multi user mode corresponds to a plurality of devices controlling a drone simultaneously. The mobile terminal can also perform a connection (e.g., a pairing) through a wireless communication between the mobile terminal and the drone.

Further, if the mobile terminal senses a signal for setting the multi user mode from the user, the mobile terminal can correspond to a master device capable of assigning or distributing control functions required for controlling the drone. In particular, when a plurality of devices control the drone simultaneously, a device having set the multi user mode while initially connected to the drone corresponds to a master device or a main device. Hence, the mobile terminal playing a role as the master device connects a slave device to the drone and the mobile terminal and can then assign functions required for the drone control to each slave device.

When the mobile terminal completes the connection to the drone, the mobile terminal can display connectible devices on the drone connection interface810. In this instance, the connectible device corresponds to a device capable of performing a pairing with the drone. According to an embodiment of the present invention, the connectible device can perform a wireless communication with the drone through a pairing by being located in a preset distance range from the drone and to perform such a wireless communication with the mobile terminal using Bluetooth or the like. Referring toFIG. 8(b), the mobile terminal can transmit a connecting signal to a device C from which the user's touch input is sensed. In this instance, the connecting signal can correspond to a signal indicating a connection between the mobile terminal and the device C.

Next,FIG. 9is a diagram illustrating one example of outputting a drone connection interface from a mobile terminal according to an embodiment of the present invention. In particular,FIG. 9shows a method for the mobile terminal, which is the master device, to assign each function to the device set as the connected device. According to an embodiment shown inFIG. 9, each of the connectible devices B, C and D shown inFIG. 8(b)is connected to the drone in response to an input signal.

Referring toFIG. 9, the mobile terminal can display at least one external device connected to the mobile terminal and the drone on a drone connection interface910. In particular, when the mobile terminal transmits a connecting signal to at least one external device inFIG. 8(b), the at least one external device can transmit a successful connection signal to the mobile terminal. Further, the mobile terminal can display a device object corresponding to the connected device on the drone connection interface. According to an embodiment shown inFIG. 9(a), the device A can correspond to a master device or a main device and each of the devices B, C and D can correspond to a slave device or a sub device to which a function is assigned by the device A.

Further, while the device object is displayed, the mobile terminal can display at least one menu object920for controlling the drone and the camera included on the drone. For example, the at least one menu object920can correspond to a drone motion control, a camera motion control, a camera shot, a viewer, and a function assignment, by which at least one menu object920is non-limited. In addition, the at least one menu object920may include various menu objects required for a drone or camera control.

A user of the mobile terminal can distribute functions to the devices B, C and D connected to the corresponding menu objects, respectively. Further, referring toFIG. 9(a), the mobile terminal can sense an input signal930for the at least one menu object920. In particular, the mobile terminal can sense the input signal930for the at least one menu object920and the at least one device object.

In this instance, the input signal930can correspond to a drag touch input. For example, the input signal930can correspond to a drag touch input that starts from a first menu object and then ends on a first device object. Alternatively, the input signal930may include one of various inputs such as a multi touch, a flicking touch and the like. According to the embodiment shown inFIG. 9(a), the first menu object corresponds to a drone control and the first device object corresponds to the mobile terminal that is the master device.

In this instance, the mobile terminal can set a drone control function in the mobile terminal in response to an input signal. Hence, the mobile terminal can control a motion of the drone. In addition, the mobile terminal can use a graphic effect940to indicate that the drone control function is set in the device A corresponding to the mobile terminal. For example, referring toFIG. 9(b), the mobile terminal darkens the menu object corresponding to the drone control and can output a color, which is output to the menu object corresponding to the drone control, to an icon of an object of the device A as a background color.

As mentioned in the foregoing description, the mobile terminal can match a function of the drone to a device easily using a drag touch or a multi touch. In addition, a user can set a camera control function in a device of a user B. In this instance, the mobile terminal can sense a drag touch input which starts from a menu object corresponding to a camera control and then ends on a device object of the user B. Moreover, the mobile terminal can transmit a signal for setting a camera control function to the device B in response to a drag touch input.

Further, the mobile terminal can set a plurality of functions in a single external device. For example, the mobile terminal can transmit a signal for setting a camera control function and a camera shot function to a device of a user C.

Next,FIGS. 10 and 11are diagrams of a drone control interface output to a display unit when completing assignment of each device function set inFIGS. 7 to 9. In particular, the drone control interface is an interface on which a GUI for controlling the drone and the camera includes a drone control program or a drone control application.

In more detail,FIG. 10is a diagram illustrating one example of a drone control interface displayed on a mobile terminal according to an embodiment of the present invention. Further,FIG. 10shows a drone control interface provided to a mobile terminal corresponding to a master device. According to an embodiment shown inFIG. 10, a mobile terminal corresponds to a mobile terminal of a user A and is currently connected to a mobile terminal of a user B (hereinafter named ‘device B’), a mobile terminal of a user C (hereinafter named ‘device C’), and a mobile terminal of a user D (hereinafter named ‘device D’).

Referring toFIG. 10, a drone control interface1010can display various objects for controlling the drone and corresponding camera. For example, the drone control interface1010can display an object corresponding to a control function assigned to control the drone or the camera and a default object unrelated to the control.

In this instance, the object corresponding to the assigned control function may include a drone control object1030, a camera control object1040, a function assignment object1050, a video shot object1060, and an image shot object1070. In addition, the object corresponding to the control function can be simultaneously displayed together with an indicator indicating at least one external device to which the control function is assigned. For example, referring toFIG. 10, the drone control object1030can be simultaneously displayed together with an indicator indicating the device B to which the drone control function is assigned.

The default object may include a user object1020, a connected device object1025, a gallery object1080, and a chat object1090. Further, the mobile terminal can display a device corresponding to each of the objects as well. In this instance, the objects appearing on the drone control interface1010are exemplary only and various other objects can be further included.

Further, the drone control interface1010can display a real-time preview image sensed from a view angle region of the camera included on the drone in the background having various objects displayed thereon. In addition, the user object1020may include a user of the mobile terminal. In particular, the user object1020can display a photo together with a name of ID of the user of the mobile terminal. The connected device object1025can indicate another external device connected to the mobile terminal and the drone. In particular, the connected device object1025can display a photo together with a name or ID of a user of the connected external device.

The drone control object1030can include a direction key for moving the drone by changing a state of the drone from a landing state into a flight state, for example. According to the embodiment shown inFIG. 10, the drone control object1030can be displayed together with an object corresponding to the device B in which the drone control function is set. The camera control object1040can include a direction control key of the camera included on the drone. According to the embodiment shown inFIG. 10, the camera control object1040can be displayed together with an object corresponding to the device C in which the camera control function is set.

The function assignment object1050can include an icon corresponding to a function setting. Although a function assignment can be basically performed by the device A corresponding to the master device, a connected different device can perform the function assignment in accordance with the setting of the master device. According to the embodiment shown inFIG. 10, the setting (function assignment) object1050can be displayed together with an object corresponding to the device B to which a control authority is granted by the mobile terminal.

The video shot object1060can include a control key for starting/ending a video shot through the camera included on the drone. According to the embodiment shown inFIG. 10, the video shot object1060can be displayed together with an object corresponding to the device B in which the video shot function is set.

The image shot object1070can include a control key for capturing a still image through the camera included on the drone. According to the embodiment shown inFIG. 10, the image shot object1070can be displayed together with an object corresponding to the device C having the image shot function set therein and an object corresponding to the device D having the image shot function set therein.

The gallery object1080can include at least one thumbnail image of an image or video shot by the camera included on the drone. If an input signal for the gallery object1080is sensed, the mobile terminal can display a gallery interface. This will be described in detail together withFIGS. 10 and 17later.

Further, while a plurality of users control or use the drone, they can also chat with each other in real time. Since a plurality of the users control a single drone simultaneously, a drone control can be efficiently performed and users' interests can be enhanced. Hence, the mobile terminal displays a chat object1090, whereby real-time chats performed between the devices connected to the mobile terminal can be displayed.

Next,FIG. 11is a diagram illustrating one example of a drone control interface displayed on a mobile terminal according to an embodiment of the present invention. In particular,FIG. 11shows a drone control interface provided not to a master device but to a slave device. Referring toFIG. 11, for a slave device, in accordance with a control function set in a master device, the objects displayed on a drone control interface are different per device. In particular, each of the devices shown inFIGS. 11(a) to 11(d)can display different objects on a corresponding drone control interface.

For example, when a master device grants a function setting authority to a slave device, the slave device can display the same drone control interface of the master device. Referring toFIG. 11(a), a slave device B receives a signal corresponding to a function setting from a master device A. Because the device B is a slave device of the device A, the device B cannot control a function set in the device A. However, the device B can assign a function for a different device other than the device A corresponding to the master device.

In another example, a slave device can display an object set for a function by a master device on a drone control device only. In this instance, the slave device can display a chat object unrelated to a function, a user object, a connected device object, a gallery object and the like. Moreover, in this instance, the slave device may not display an object related to a function set in another device on the drone control interface.

Referring toFIG. 11(b), as a device C receives a camera control signal and an image shot control signal from the device A, the device C can display a camera control object and an image shot object on a drone control interface. Referring toFIG. 11(c), as a device D receives a video shot control signal from the device A, the device C can display a video shot object on a drone control interface.

In another example, if any function is not set in a slave device by a master device, the slave device can display an object unrelated to a function on a drone control interface. Referring toFIG. 11(d), a device E corresponds to a state of being connected to the devices A to D and the drone. Further, the device E can fail to receive a control signal for any function from the device A corresponding to the master device. In this instance, the device E can display a real-time image, which is sensed from a view angle region of a camera included on the drone, and a gallery object on the drone control interface only. In addition, the device E can display a chat object, which is a default object, on the drone control interface.

Next,FIGS. 12 to 14are diagrams illustrating a method for a master device to change or add a control function set in a slave device when a drone and the slave device are initially connected. According to an embodiment of the present invention, since a master device has an authority for setting a function of a slave device, a user of the master device can add or delete a function of the slave device through a simple input to a drone control interface. According to the embodiments shown inFIGS. 12 to 14, assume that a mobile terminal corresponds to a master device.

FIG. 12is a diagram illustrating one example of adding a function for a drone control in a mobile terminal according to an embodiment of the present invention. Referring toFIG. 12, a user A of a master device wants to add a user D of a slave device as a user corresponding to a camera control object.

Referring toFIG. 12(a), the mobile terminal can sense an input signal1240for at least one of connected device objects and at least one of objects corresponding to control functions from a drone control interface1210. In this instance, the input signal1240can correspond to a drag touch input. For example, according to the embodiment shown inFIG. 12, the input signal1240corresponds to a drag touch input that starts from a position corresponding to an object D1230among connected users and then ends to a position corresponding to a camera control object1220. Further, the input signal1240can correspond to one of a multi touch input, a flicking touch input and the like.

In this instance, referring toFIG. 12(b), the mobile terminal can add a device D as a device corresponding to the camera control object1220in the drone control interface1210in addition to a previously set device C. Moreover, the mobile terminal can transmit an assignment signal of a control function corresponding to a camera control function to the device D. In response to the assignment signal, the device D can display the camera control object on the drone control interface. Thus, a user C and a user D can simultaneously control a direction and zoom of the camera included on the drone.

Next,FIG. 13is a diagram illustrating one example of removing a function for a drone control in a mobile terminal according to an embodiment of the present invention. Referring toFIG. 13, a user A of a master device wants to delete a user C of a slave device from users corresponding to an image shot object.

Referring toFIG. 13(a), the mobile terminal can sense an input signal1330for an indicator indicating an external device from a drone control interface1310. In this instance, the input signal1330can correspond to a flicking touch input. For example, according to the embodiment shown inFIG. 13, the input signal1330can correspond to a flicking touch input that starts from a position corresponding to an image shot object1340and then ends to a boundary of one side of the drone control interface1310. Alternatively, the input signal1330can correspond to a drag touch input.

If so, referring toFIG. 13(b), when a user C and a user D are previously set as users corresponding to the image shot object1340, the mobile terminal can remove the user C from the drone control interface1310. In addition, the mobile terminal can transmit a signal for stopping or cancelling a function corresponding to an image shot control to the device C. If so, an image shot object can be deleted from the device C. Thus, a user can solely take an image through the camera included on the drone.

FIG. 14is a diagram illustrating one example of changing a control function of a drone controllable device in a mobile terminal according to an embodiment of the present invention. Referring toFIG. 14, a user of a master device A changes a function set in a user B of a slave device into a video shot function from a drone control function.

Referring toFIG. 14(a), a mobile terminal can sense an input signal1440for selecting an indicator indicating a first external device having a first function assigned thereto and an object corresponding to a second function from a drone control interface1410. In this instance, the input signal1440can correspond to one of a drag touch input and a multi touch input. For example, according to the embodiment shown inFIG. 14, the input signal1440can correspond to a drag touch input that starts from apposition corresponding to a drone control object1320and then ends to a position corresponding to a video shot object1430.

If so, referring toFIG. 14(b), the mobile terminal can change a device of a user B having a drone control function set therein to perform a video shot function in the drone control interface1410. In addition, the mobile terminal can transmit a control signal for stopping a drone control function and adding a video shot function to the device B. In response to the control signal, the device B deletes a drone control object from a drone control interface and can display a video shot object on the drone control interface. Thus, the user B connected to the drone can perform the video shot function on the drone control interface instead of performing the drone control function.

In the following description,FIGS. 15 and 16show a method for a slave device to make a request for controlling a drone. In particular,FIGS. 15 and 16show a method for a slave device to make a request for controlling a function, of which control authority is not granted to the slave device, to a master device or a device having the control authority. The reason for this is that the slave device can perform a limited function required for a drone control or a camera control granted by the master device.

FIG. 15is a diagram illustrating one example of guiding a control of a drone in a mobile terminal according to an embodiment of the present invention. In particular,FIG. 15is a diagram illustrating a method of a user C1510not having a drone control authority and making a request for a control of the drone to a user A having the drone control authority. In the following description, a signal transmission between a device (hereinafter named ‘device C’) of a user C (hereinafter named ‘device C’) and a device (hereinafter named ‘device B’) of a user B is shown.

Referring toFIG. 15(a), a device C can display a camera control object corresponding to a camera control function, of which control authority is granted to a user C, on a display unit. Further, the user C may want to use a different control function, of which control authority is not granted to the device C, other than a camera control. For example, the user C may want to move a location of a drone to the left with reference to a current location.

If so, referring toFIG. 15(a), the device C can sense a first input signal1530afor an object1520corresponding to a user B. For example, the first input signal1530acan correspond to one of a short touch input and a long touch input. Subsequently, the device C can sense a second input signal1530b. For example, the second input signal1530bcan correspond to one of a drag touch input and a flicking touch input in a prescribed direction. According to the embodiment shown inFIG. 15, the second input signal1530bcorresponds to a drag touch input in a left direction. Alternatively, the second input signal1530bcan correspond to a signal in continuation with the first input signal1530a.

In this instance, in response to the first input signal1530aand the second input signal1530b, the device C can transmit a drone control signal to the device B. In particular, the drone control signal can correspond to a signal for requesting to move a location of the drone to the left to the device B having the drone control authority.

Further, the device B can receive a drone control signal from the device C. If so, referring toFIG. 15(b), the device B can output an indicator1550in response to the received drone control signal. For example, the indicator1550may appear as a bar shape on one of top, bottom, right and left sides of the display unit. Moreover, according to the embodiment shown inFIG. 15, since the drone control signal is the signal for requesting to move the location of the drone to the left side, the indicator1550may appear one the left side of the display unit. In this instance, the user B recognizes user C's intention and can then manipulate the drone control object to move the location of the drone to the left side. The embodiment shown inFIG. 15can also be applicable to other functions related to the drone manipulation or the camera control as well as the drone control function.

Next,FIG. 16is a diagram illustrating one example of guiding a control of a drone in a mobile terminal according to an embodiment of the present invention. In particular,FIG. 16is a diagram to describe a method a user C1610failing to have a drone control authority to make a request for a control of a drone to a different user having the drone control authority.

For instance, referring toFIG. 16(a), a device C can display a voice icon1629on a drone control interface. In this instance, the voice icon1620can correspond to an icon for sending a voice message for a different device connected to the device C. In addition, the device C can sense an input signal1630for the voice icon1620. For example, the input signal1630can correspond to one of a short touch input and a long touch input. Thus, a user C can speak a voice message1640and the user C's voice message can be received by a different device connected to the device C.

In another instance, referring toFIG. 16(b), the device C can display a chat object1650on the drone control interface. In this instance, the chat object1650can correspond to an object for enabling the different device connected to the device C to make a real-time chat. In addition, the device C may sense an input signal1660for the chat object1650. For example, the input signal1660can correspond to one of a short touch input and a long touch input. In this instance, the user C can compose a text message. In addition, the text message composed by the user C can be received by a different device.

Next,FIGS. 17 and 18are diagrams illustrating a gallery interface including an image or video taken through a drone control interface. In this instance, the gallery interface can correspond to an interface configured to display an image or video taken using a camera provided to a drone by controlling the camera through the drone control interface.

FIG. 17is a diagram illustrating one example of a gallery interface corresponding to a gallery object in a mobile terminal according to an embodiment of the present invention. According to the embodiment shown inFIG. 17, a mobile terminal is a master device, i.e., a device of a user A.

Further, a mobile terminal can sense a first input signal for a gallery object displayed on a drone control interface. In this instance, for example, the input signal can correspond to one of a short touch input and a long touch input.

Referring toFIG. 17(a), the mobile terminal can display a gallery interface1710in response to the first input signal. For example, the gallery interface1710can correspond to a subordinate item of a gallery application or the drone control interface, which is run separately from the drone control interface.

Referring toFIG. 17(a), the mobile terminal can display a thumbnail image indicating at least one image or video taken through a drone on the gallery interface1710. In addition, the mobile terminal can display a real-time preview1711on a right top end of the gallery interface1710. In this instance, the real-time preview1711can correspond to a screen for displaying an image sensed within a view angle region of a camera included on the drone by real time.

Further, if a second input signal1720for the real-time preview1711is sensed, the mobile terminal ends the gallery interface1710and can display a drone control interface1730. For example, the second input signal1740can correspond to one of a short touch input and a long touch input. Referring toFIG. 17(b), if a third input signal1740for a gallery object1731displayed on the drone control interface1730is sensed, the mobile terminal ends the drone control interface1730and can display the gallery interface1710.

FIG. 18is a diagram illustrating one example of a gallery interface corresponding to a gallery object in a mobile terminal according to an embodiment of the present invention. Substance redundant with the former description with reference toFIG. 17shall be omitted from the following description with reference toFIG. 18.

Referring toFIG. 18(a), a mobile terminal can sense an input signal1820for at least one thumbnail image displayed on a gallery interface1810. For example, the input signal1820can correspond to one of a short touch input and a long touch input. In this instance, referring toFIG. 18(b), in response to the input signal1820, the mobile terminal can display an enlarged image or video1830corresponding to the thumbnail image1811by zoom-in.

Further, the mobile terminal can display an indicator1840indicating a preference on one side of the enlarged image or video. In particular, the indicator1840indicating the preference can display a user having marked the preference for the corresponding image or video. Through this, users of other devices connected to the mobile terminal can check the preference of the image or video and a subject of person who prefers the corresponding image or video.

In addition, in response to an input signal, the mobile terminal can display an indicator indicating a preference on an image corresponding to a thumbnail image (not shown inFIG. 18). In this instance, if the input signal for the thumbnail image further increases, the mobile terminal can change the graphic effects of the indicator indicating the preference more variously.

Next,FIG. 19is a flowchart illustrating a method of controlling a mobile terminal according to an embodiment of the present invention. The respective steps shown inFIG. 19can be controlled by the controller of the mobile terminal shown inFIG. 1.

Referring toFIG. 19, the mobile terminal can sense a first input signal or a second input signal for a drone connection interface (S1910). As mentioned in the foregoing description with reference toFIG. 7andFIG. 8, the mobile terminal can display an icon corresponding to a single user mode and an icon corresponding to a multi user mode on the drone connection interface.

If the first input signal is sensed in the step S1910, the mobile terminal can enter the multi user mode. In particular, if an input signal for the icon corresponding to the multi user mode is sensed, the mobile terminal can enter the multi user mode. If so, the mobile terminal performs a pairing with a drone and can attempt a connection to another device. This is described in detail with reference toFIG. 20later.

Moreover, if the second input signal is sensed in the step S1910, the mobile terminal can enter the single user mode. In particular, if an input signal for the icon corresponding to the single user mode is sensed, the mobile terminal can enter the single user mode. If so, the mobile terminal performs a pairing with a drone and can display objects related to a drone control on the drone control interface.

FIG. 20is a flowchart illustrating a method of controlling a mobile terminal according to an embodiment of the present invention. The respective steps shown inFIG. 20can be controlled by the controller of the mobile terminal shown inFIG. 1. In addition, the respective steps shown inFIG. 20can correspond to an embodiment for the case that the mobile terminal enters the multi user mode inFIG. 19.

Referring toFIG. 20, a mobile terminal can display a device object corresponding to an external device connectible to a drone on a drone connection interface (S2010). As mentioned in the foregoing description with reference toFIG. 8, the connectible device can correspond to a device capable of a wireless communication with the mobile terminal by being connectible to the drone.

In addition, the mobile terminal can sense a first input signal for at least one of the device objects (S2020). As mentioned in the foregoing description with reference toFIG. 8, the first input signal may include one of a touch input, a gesture input and the like. In response to the first input signal, the mobile terminal can transmit a connecting signal to an external device corresponding to the selected device object (S2030). If so, a device corresponding to a first device object receives the connecting signal and can then perform a connection to at least one of the mobile terminal and the drone.

Further, the mobile terminal can display a device object corresponding to at least one device connected to the drone connection interface and a menu object corresponding to a control function for a control of a drone or a control of a camera included on the drone (S2040). As mentioned in the foregoing description with reference toFIG. 8, the at least one menu object may include a drone control, a camera control, a camera shot, a function assignment, and the like.

Further, the mobile terminal can sense a second input signal for at least one of the device objects and at least one of the menu objects (S2050). For example, the second input signal can correspond to one of a drag touch input and a multi touch input. Thereafter, in response to a second input signal, the mobile terminal can transmit an assignment signal of the control function corresponding to the selected menu object to the external device corresponding to the selected device object (S2060). If so, in response to the received control signal, a device corresponding to a first object can display a controllable object on a drone control interface.

FIG. 21is a flowchart illustrating a method of controlling a mobile terminal according to an embodiment of the present invention. The respective steps shown inFIG. 21can be controlled by the controller180of the mobile terminal100shown inFIG. 1. In addition, the respective steps shown inFIG. 21can correspond to an embodiment of a slave device when the mobile terminal enters the multi user mode inFIG. 19.

Referring toFIG. 21, a mobile terminal can receive a connecting signal from an external device (S2110). In this instance, the external device can correspond to a master device. In response to the received connecting signal, the mobile terminal can perform a connection to a drone (S2120). If the connection to the drone is completed, the mobile terminal can display a drone control interface on a display unit.

The mobile terminal can receive an assignment signal of a control function for controlling the drone or camera from the external device (S2130). Thereafter, in response to the received assignment signal of the control function, the mobile terminal can display a control object corresponding to the received control signal on the drone control interface (S2140). For example, as mentioned in the foregoing description with reference toFIG. 11, in response to the received control signal, the mobile terminal can display the drone control interface including all objects displayed on a drone control interface of a master device. In another example, the mobile terminal can display a control object corresponding to the received control signal on the drone control interface only.

Accordingly, the present invention provides the following advantages. For example when a video or image is taken through a camera provided to a drone, a plurality of devices can efficiently control a single drone.

The foregoing embodiments are merely exemplary and are not to be considered as limiting the present disclosure. The present teachings can be readily applied to other types of methods and apparatuses. This description is intended to be illustrative, and not to limit the scope of the claims. Many alternatives, modifications, and variations will be apparent to those skilled in the art. The features, structures, methods, and other characteristics of the exemplary embodiments described herein may be combined in various ways to obtain additional and/or alternative exemplary embodiments. As the present features may be embodied in several forms without departing from the characteristics thereof, it should also be understood that the above-described embodiments are not limited by any of the details of the foregoing description, unless otherwise specified, but rather should be considered broadly within its scope as defined in the appended claims, and therefore all changes and modifications that fall within the metes and bounds of the claims, or equivalents of such metes and bounds, are therefore intended to be embraced by the appended claims.