Head-mounted display device with detachable device

A head-mounted device including a main frame couplable to an electronic device and a support part configured to affix the main frame to a facial side of the user, and an auxiliary input device including a body, an input module disposed on at least one surface of the body and configured to process an input signal for controlling a function of the electronic device, a sensor module disposed in the body and configured to obtain sensing information in accordance with a movement of the auxiliary input device, and a connection part electrically connectable to the electronic device and configured to serve as a path through which the input signal and the sensing information are transmitted to the electronic device is provided.

PRIORITY

This application claims priority under 35 U.S.C. § 119(a) to a Korean Patent Application filed on Nov. 30, 2015 in the Korean Intellectual Property Office and assigned Serial number 10-2015-0168603, the entire disclosure of which is incorporated herein by reference.

BACKGROUND

1. Field of the Disclosure

The present disclosure relates generally to a head-mounted display device (HMD) with a detachable device, and more particularly, to an HMD with a detachable device provided with a sensor module or an input module.

2. Description of the Related Art

In recent years, various wearable electronic devices that may be worn directly on a user's body have been developed. If wearable electronic devices are worn directly on a user's body, mobility and portability of the wearable electronic devices are improved. Wearable electronic devices worn on a part of a user's body or clothing include various devices, such as an HMD, smart glasses, a smart watch or wristband, a contact lens-type device, a ring-type device, a shoe-type device, a clothing-type device, a glove-type device, etc., and a wearable electronic device may be formed to have a shape that is attachable to and detachable from a part of a user's body or clothing. In particular, an HMD may be provided in a goggle-like form or a glasses-like form and mounted on a user's head.

An HMD may be integrated with an electronic device (e.g., a smart phone) or attachable to and detachable from an electronic device to display an image through a screen of the electronic device. In addition, an HMD may include a sensor module (e.g., a six-axis sensor) for sensing movement of a user's head or an input module (e.g., a touch pad) for receiving a user input (e.g., a touch input or a gesture input). An HMD may control functions of an electronic device based on information obtained through a sensor module or an input module.

A sensor module and an input module may be integrated with a main frame (or body) of an HMD. Accordingly, in a case where an HMD may need to be replaced, e.g., in a case where an electronic device integrated with an HMD may malfunction or the electronic device may not fit into a main frame of an HMD due to a change in size or appearance of the electronic device, the sensor module and the input module may need to be replaced with the main frame of the HMD. However, optical members, such as a lens, of an HMD may be required to be replaced in accordance with the size or appearance of an HMD, but a sensor module and an input module are not necessary need to be replaced.

SUMMARY

An aspect of the present disclosure is to provide an HMD and a device (hereinafter, referred to as an auxiliary input device) that is attachable to and detachable from the HMD or an electronic device and provided with a sensor module or an input module.

In accordance with an aspect of the present disclosure, a head-mounted system is provided. The head-mounted device includes a main frame couplable to an electronic device and a support part configured to affix the main frame to a facial side of the user, and an auxiliary input device including a body, an input module disposed on at least one surface of the body and configured to process an input signal for controlling a function of the electronic device, a sensor module disposed in the body and configured to obtain sensing information in accordance with a movement of the auxiliary input device, and a connection part electrically connectable to the electronic device and configured to serve as a path through which the input signal and the sensing information are transmitted to the electronic device.

In accordance with another aspect of the present disclosure, an electronic device detachably attached to a mobile electronic device to provide a signal is provided. The electronic device includes a housing including a first surface, a second surface facing an opposite side of the first surface, and a side surface surrounding at least a portion of a space between the first surface and the second surface; at least one touch sensor disposed on at least one of the first surface, the second surface, and the side surface of the housing; at least one sensor disposed in the housing; a communication circuit disposed in the housing; and a control circuit electrically connected to the at least one touch sensor, the at least one sensor, and the communication circuit, wherein the communication circuit is configured to obtain a gesture or touch input from a user using the at least one touch sensor, obtain information associated with at least one of an orientation and a movement of the electronic device using the at least one sensor, and transmit at least one of the gesture or touch input and the information to the mobile electronic device using the communication circuit.

In accordance with another aspect of the present disclosure, a mobile electronic device detachably attached to a head-mounted display device worn on a head of a user is provided. The mobile electronic device includes a communication circuit; a display; a processor electrically connected to the communication circuit and the display; and a memory electrically connected to the processor, wherein the memory is configured to store instructions, if executed by the processor, which causes the processor to receive at least one input signal from an external sensing device using the communication circuit, display content associated with the head-mounted display device through at least a portion of the display based on at least a portion of the obtained at least one input signal, receive information on at least one of an orientation and a movement of the external sensing device, which are obtained by the external sensing device, from the external sensing device, and change the content displayed on the display based on the received information.

Hereinafter, various embodiments of the present disclosure are described with reference to the accompanying drawings. Accordingly, those of ordinary skill in the art will recognize that modification, equivalent, and/or alternative on the various embodiments described herein may be variously made without departing from the scope and spirit of the present disclosure. With regard to the descriptions of the accompanying drawings, similar components may be marked by similar reference numerals.

In the present disclosure, the terms “have,” “may have,” “include,” “comprise,” “may include,” and “may comprise” used herein indicate the existence of corresponding features (e.g., elements such as numeric values, functions, operations, or components) but do not exclude the presence of additional features.

In the present disclosure, the terms “A or B,” “at least one of A and/or B,” “one or more of A and/or B,” and the like used herein may include any and all combinations of one or more of the associated listed items. For example, the terms “A or B,” “at least one of A and B,” and “at least one of A or B” may refer to all of the case (1) where at least one A is included, the case (2) where at least one B is included, and the case (3) where both of at least one A and at least one B are included.

The terms, such as “first,” “second,” and the like used herein may refer to various elements of various embodiments of the present disclosure, but are not intended to limit the elements. For example, such terms are not intended to limit the order and/or priority of the elements. Furthermore, such terms may be used to distinguish one element from another element. For example, “a first user device” and “a second user device” indicate different user devices. For example, without departing from the scope and spirit of the present disclosure, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element.

The expression “configured to” used herein may be used as, for example, the expression “suitable for,” “having the capacity to,” “designed to,” “adapted to,” “made to,” or “capable of” The term “configured to” does not indicate only “specifically designed to” in hardware. Instead, the term “a device configured to” may indicate that the device is “capable of” operating together with another device or other components. For example, a “processor configured to perform A, B, and C” may indicate a dedicated processor (e.g., an embedded processor) for performing a corresponding operation or a general purpose processor (e.g., a central processing unit (CPU) or an application processor) which may perform corresponding operations by executing one or more software programs which are stored in a memory device.

Terms used in the present disclosure are used to describe certain embodiments of the present disclosure but are not intended to limit the scope of the present disclosure. The terms in singular forms may include plural forms unless otherwise specified. Unless otherwise defined herein, all the terms used herein, may have the same meanings that are generally understood by a person skilled in the art. Terms, which are defined in a dictionary and commonly used, should also be interpreted as is customary in the relevant related art and not in an idealized or overly formal manner unless expressly so defined herein in various embodiments of the present disclosure. In some cases, even terms defined in the present disclosure may not be intended to exclude embodiments of the present disclosure.

An electronic device according to an embodiment of the present disclosure may include at least one of smartphones, tablet personal computers (PCs), mobile phones, video telephones, electronic book readers, desktop PCs, laptop PCs, netbook computers, workstations, servers, personal digital assistants (PDAs), portable multimedia players (PMPs), mobile medical devices, cameras, or wearable devices. According to an embodiment of the present disclosure, the wearable devices may include at least one of accessory type wearable devices (e.g., watches, rings, bracelets, ankle bracelets, necklaces, glasses, contact lenses, or HMDs), textile/clothing type wearable devices (e.g., electronic apparel), body-attached type wearable devices (e.g., skin pads or tattoos), or bio-implantable type wearable devices (e.g., implantable circuits).

According to an embodiment of the present disclosure, the electronic devices may be smart home appliances. The smart home appliances may include at least one of, for example, televisions (TVs), digital versatile disc (DVD) players, audio players, refrigerators, air conditioners, cleaners, ovens, microwave ovens, washing machines, air cleaners, set-top boxes, TV boxes (e.g., Samsung HomeSync®, Apple TV®, or Google TV™), game consoles (e.g., Xbox® and PlayStation®), electronic dictionaries, electronic keys, camcorders, electronic picture frames, and the like.

According to, the electronic devices may include at least one of medical devices (e.g., various portable medical measurement devices (e.g., a blood glucose monitoring device, a heartbeat measuring device, a blood pressure measuring device, a thermometer, and the like)), a magnetic resonance angiography (MRA) device, a magnetic resonance imaging (MRI) device, a computed tomography (CT) device, scanners, and ultrasonic devices), navigation devices, global navigation satellite system (GNSS), event data recorders (EDRs), flight data recorders (FDRs), vehicle infotainment devices, electronic equipment for vessels (e.g., navigation systems and gyrocompasses), avionics, security devices, head units for vehicles, industrial or home robots, automated teller machines (ATMs), points of sales (POSs) devices, or Internet of Things (IoT) devices (e.g., light bulbs, various sensors, electric or gas meters, sprinkler devices, fire alarms, thermostats, street lamps, toasters, exercise equipment, hot water tanks, heaters, boilers, and the like).

According to an embodiment of the present disclosure, the electronic devices may include at least one of furniture or buildings/structures, electronic boards, electronic signature receiving devices, projectors, or various measuring instruments (e.g., water meters, electricity meters, gas meters, or wave meters, and the like). The electronic devices may be one or more combinations of the above-mentioned devices. An electronic device may be a flexible electronic device. However, electronic devices are not limited to the above-mentioned devices, but may include electronic devices to be developed.

The term “user” used herein may refer to a person who uses an electronic device or a device (e.g., an electronic device that includes artificial intelligence) that uses an electronic device.

FIG. 1is an exploded perspective view of an HMD,FIG. 2is a perspective view of an HMD coupled to an auxiliary input device,FIG. 3is a rear perspective view of an HMD coupled to an auxiliary input device, andFIG. 4is an illustration of a user wearing the HMD shown inFIG. 2, according to an embodiment of the present disclosure, respectively.

Referring toFIGS. 1 to 4, the HMD100may include a main frame110(or a frame structure) and a support part150(or a binding member) (e.g., a goggle band). According to an embodiment of the present disclosure, the HMD100may have a shape in which a portion of the support part150corresponding to an upper portion of a head of a user may be removed to leave only a portion of the support part150corresponding to a side portion of the head of the user.

The main frame110may be worn on at least a portion (e.g., a facial side) of a user and secured to a facial side by various components. According to an embodiment of the present disclosure, the main frame110may be constructed from a lightweight material, such as plastic, which provides a user with a comfortable fit and supports an electronic device200(e.g., a mobile electronic device). In addition, the main frame110may further include a material to protect the HMD100.

According to an embodiment of the present disclosure, the material protecting the HMD100may include at least one of various materials, such as glass, plastic (e.g., acrylonitrile butadiene styrene (ABS) or polycarbonate), ceramic, metal (e.g., aluminum), or a metal alloy (e.g., steel, stainless steel, magnesium alloy, or the like).

According to an embodiment of the present disclosure, the support part150may include an adjustable length of a band formed of an elastic material to allow the main frame110to be in close contact with an eye area of a user's face, and cushions may be attached to the band to provide a comfortable fit in consideration of long wearing times. Additionally or generally, the support part150may be configured to include eyeglass temples, helmets, or straps.

Since a rear surface of the main frame110serves as a facial-side contact part130which makes contact with the face of a user, the rear surface of the main frame110may have a shape corresponding to a curve of a face and include an elastomer131as at least a portion thereof. The elastomer131may include at least one cushion material, such as a sponge, to provide a comfortable fit if the facial-side contact part130makes contact with the face of a user. The elastomer131may be configured to include one sponge or may have a shape obtained by laminating one or more sponges having different compression ratios. For example, the elastomer131may be a three-layered sponge (e.g., an upper layer, an intermediate layer, a lower layer), and in this case, the sponge with a low compression ratio may be applied to the upper and lower layers, and the sponge with a high compression ratio may be applied to the intermediate layer.

According to an embodiment of the present disclosure, the elastomer131may be attached to the main frame110to allow the elastomer131to be detachable from the main frame110. The elastomer131may include an adhesive member provided on one surface thereof and may be attached to and detached from the main frame110by the adhesive member. The adhesive member may be Velcro®, a tape, or an adhesive, but it should not be limited thereto or thereby as long as it may be attached to and detached from the main frame110. Accordingly, in a case where multiple users use one HMD100, the users may select the elastomers appropriate to themselves (e.g., elastomers appropriate to adults or children having different facial contours). In a case where a surface of the elastomer131is contaminated (e.g., the surface of the elastomer131is stained with cosmetics or sweat) or the elastomer131is broken by frequent use, the HMD100may be used after the contaminated or broken elastomer131is replaced with another elastomer131.

Considering the ease of wear on the face of a user, the main frame110of the HMD100in the present disclosure may have a shape or structure to cover a user's eye area, and the HMD100may include a nose recess170into which a user's nose is placed. A lens assembly including at least one lens270may be inserted into portions of the facial-side contact part130, which corresponds to two eyes of the user. At least one surface of the lens270may be exposed through the facial-side contact part130to allow a user to see a screen of an electronic device200if a user wears the HMD100.

According to an embodiment of the present disclosure, the main frame110may include a display position controller111, a placing part113on which an auxiliary input device500is placed, and a lens fixing part115. In addition, the main frame110may include a front case117provided with a space or a structure to which the electronic device200is coupled. The main frame110may further include a connector to communicate with the electronic device200coupled thereto.

According to an embodiment of the present disclosure, the connector may include an electrical connection part for the electronic device200, for example, a universal serial bus (USB) connector connected to a USB port, and provide a signal of a graphics user interface (GUI) to the electronic device200through the electrical connection part.

According to an embodiment of the present disclosure, the display position controller111may be implemented by a wheel-like member or a dial-like member. If a user rotates the wheel or dial employed as the display position controller111, the electronic device200may move to control a distance between the display of the electronic device200and the user, and as a result, the user may control the position of the electronic device200to view the image displayed in a suitable manner for his/her eyesight or optimally displayed.

The placing part113may have a concave shape such that the auxiliary input device500is placed on the main frame110. According to an embodiment of the present disclosure, the placing part113may further include a coupling member to prevent the auxiliary input device500placed on the main frame110from inadvertently becoming separated. The coupling member may be implemented by a hook-like member, a magnet, or an electromagnet.

A display or a transparent/semi-transparent lens270may be integrally fixed to the lens fixing member115of the main frame110or detachably fixed to the lens fixing member115. According to an embodiment of the present disclosure, a lens assembly may be inserted inbetween the display and the eyes of a user. The main frame110may include a soft material (e.g., a sponge, a piece of rubber, etc.) provided on a rear surface thereof to prevent the main frame110from excessively closing in on a user's eye area.

The front case117of the main frame110may correspond to the shape of the electronic device200. According to an embodiment of the present disclosure, the front case117may include an elastic material or a flexible material to vary a size thereof, and thus the front case117may accommodate various sizes of the electronic device200.

According to an embodiment of the present disclosure, the front case117may be coupled to an adaptor210used to allow the electronic device200to be mounted on the main frame110. The adaptor210may be connected between the electronic device200and the main frame110to align the display screen of the electronic device200with the lens270of the main frame110. The adaptor210may include at least one opening211to allow a user to view the display screen of the electronic device200through the lens270. In addition, the adaptor210may be changed in accordance with the size or the appearance of the electronic device200. Furthermore, the front case117may further include a fixing member such that the adaptor210with various sizes or appearances may be fixed to a surface of the front case117connected to the adaptor210.

The HMD100according to an embodiment of the present disclosure may further include a cover230to more securely support the electronic device200to the main frame110while the electronic device200is coupled to the main frame110. The cover230may be physically coupled to the main frame110in the manner of a hook, a magnet, or an electromagnet. As described above, the cover230may prevent the electronic device200from inadvertently becoming separated from the main frame110, form the appearance of the main frame110, and improve aesthetics.

According to an embodiment of the present disclosure, the cover230may further include a window250arranged on a front surface thereof, and the window250may be made of various materials and colors to improve aesthetics. The window250may be manufactured using a plastic material, such as acryl, a ceramic material, such as glass, sapphire, a transparent ceramic, etc., or a metal material, such as aluminum, stainless steel, etc. A transparent material may have colors to control transparency.

In addition, the cover230may further include one or more additional openings231. A heat generated by the electronic device200may be easily discharge through the opening231to the outside of the cover230. Accordingly, the temperature of the electronic device200may be lowered to prevent performance degradation of the electronic device200. Further, the weight of the cover230may be reduced due to the one or more openings231, and thus a total weight of the HMD100may be reduced.

According to an embodiment of the present disclosure, the auxiliary input device500(or a sensing device) may be attached to and detached from a surface (e.g., a side surface) of the electronic device200. The auxiliary input device500may include a connection part to be attached to and detached from the electronic device200or the main frame110. The connection part may include, for example, a USB connector connected to a USB port of the electronic device200or a USB port of the main frame110. A signal of a sensor module or an input module included in the auxiliary input device500may be applied to the electronic device200through an electrical connection part of a USB connector.

The auxiliary input device500may include a sensor module, an input module, and control functions of the electronic device200based on information obtained by the sensor module and the input module.

A sensor module included in the auxiliary input device500may measure a physical quantity or sense an operation status of the auxiliary input device500and convert the measured or sensed information to an electrical signal. A sensor module may include, for example, an acceleration sensor, a gyro sensor, or a geomagnetic field sensor. According to an embodiment of the present disclosure, if the auxiliary input device500is connected to the electronic device200coupled to the HMD100or the main frame110, a sensor module may obtain information on an orientation and/or a movement of the auxiliary input device500to sense a movement of the head of a user.

An input module included in the auxiliary input device500may receive a user input (e.g., a touch input or a gesture input). An input module may include, for example, at least one of a touch pad, a trackball, a wheel key, a physical key, a physical button, a touch key, or a joystick. According to an embodiment of the present disclosure, an input module may provide a GUI to control the functions of the electronic device200. For example, a GUI associated with setting a sound level may control the audio volume of the electronic device200, and a GUI associated with image reproduction may control an image displayed in the electronic device200. In addition, an input module may receive a touch input or a hovering input of a user.

The electronic device200may control a function corresponding to a touch input in response to a touch input provided from the auxiliary input device500. For example, the electronic device200may control audio volume or image reproduction in response to a touch input applied thereto.

FIG. 5Ais a diagram of an auxiliary input device detached from and attached to an HMD or an electronic device, respectively, according to an embodiment of the present disclosure,FIG. 5Bis a perspective view of an auxiliary input device according to an embodiment of the present disclosure, andFIG. 5Cis a side view of an auxiliary input device according to an embodiment of the present disclosure.

Referring toFIGS. 5A to 5C, the auxiliary input device500(or sensing device) may include a body510(or housing), an extension part550, and a connection part570. According to an embodiment of the present disclosure, the body510of the auxiliary input device500may have a substantially rectangular parallelepiped shape. For example, at least a certain area of front and rear surfaces of the body500of the auxiliary input device500may have a substantially rectangular shape, and at least a corner of a border surrounding the certain area may be formed in a curved surface having a predetermined curvature. However, the shape of the body510is not limited thereto or thereby. The body510may have a cube shape, a cylinder shape, a hemi-spherical shape, or a spherical shape.

The body510of the auxiliary input device500may include an input module530disposed on a surface thereof. The input module530may include, for example, at least one of a touch pad, a trackball, a wheel key, a physical key, a physical button, a touch key, or a joystick. As shown inFIG. 5C, the input module530may include a first input module531disposed on a first surface (e.g., a front surface) of the body510and a second input module533disposed on a second surface (e.g., a rear surface) of the body510. The input module530may include only one input module disposed on one surface of the body510. The input module530may include a control circuit disposed therein to receive a user input (e.g., a touch input or the gesture input) and process an input signal corresponding to the user input.

According to an embodiment of the present disclosure, the body510may include a sensor module therein. For example, the body510may include at least one of a touch sensor, an acceleration sensor, a gyro sensor, and a geomagnetic field sensor disposed therein. In addition, the body510may further include a sensor (e.g., a pressure sensor) disposed therein to sense pressure and may include a haptic module to provide a haptic feedback. Accordingly, various effects may be provided by changing a haptic profile in accordance with a type of display being displayed. For example, in a case that an external property of an object presently displayed is a smooth object, a haptic module may receive the property of the object and set a haptic profile that is suitable for a smooth property, and thus a smooth surface may be provided to a user.

The extension part550of the auxiliary input device500may extend from one side surface of the body510. The extension part550may have a substantially rectangular parallelepiped shape, but the shape of the extension part550is not limited to the rectangular parallelepiped shape. The extension part550may include a connection circuit line disposed therein to connect an input module or a sensor module of the body510to an electrical connection part of the connection part570.

According to an embodiment of the present disclosure, the extension part550may be connected to the body510by a moving member511. For example, the extension part550may be fixed to the moving member511at one surface (e.g., a side surface) of the body510. The moving member511may move along a moving line formed in the surface of the body510. The moving line may be, but is not limited to, a hole formed in the surface to have a predetermined length and width. Accordingly, the moving member511may move in a longitudinal direction of the hole without being separated from the hole.

According to an embodiment of the present disclosure, a hole may be formed in at least one direction of a vertical direction or a horizontal direction on one surface of the body510. In a case where a hole is formed in a vertical direction of the body510, the moving member511may move in a vertical direction, and thus the extension part550fixed to the moving member511may move in a vertical direction of the body510. In addition, in a case where a hole is formed in a horizontal direction of the body510, the moving member511may move in a horizontal direction, and thus the extension part550fixed to the moving member511may move in a horizontal direction of the body510. In a case where a hole is formed in one surface of the body510along a horizontal direction, a hole may extend to a surface (e.g., a front surface or a rear surface) connected to one surface and may be formed in a portion of a surface connected to the one surface. Accordingly, the moving member511may move to a certain area of a front or rear surface from the side surface of the body510, and in this case, the extension part550may be bent or folded with respect to the body510within a predetermined angle.

According to an embodiment of the present disclosure, the moving member511may further include an elastic material, such as a spring. Thus the moving member511may be deformed or restored due to elastic force in a case where the extension part550is bent or folded with respect to the body510within a predetermined angle.

According to an embodiment of the present disclosure, the moving member511may be implemented by a separate hinge module. A hinge module may have a cam or serration structure, and the extension part550may be rotated or folded with respect to the body510through the above-mentioned structure.

According to an embodiment of the present disclosure, the extension part550may further include a rotation member551therein. The rotation member551may allow the extension part550to rotate about a normal line of one surface (e.g., the side surface) of the body510within a predetermined angle. The rotation member551may be connected to the moving member511and integrally provided with the moving member511. The rotation member551may have a substantially cylindrical shape such that the extension part550is smoothly rotated. However, the shape of the rotation member551is not limited to a cylindrical shape. As shown inFIG. 5B, a hollow may be formed in the extension part550, and the rotation member551may be inserted into the hollow to prevent the rotation member551from being separated from the extension part550. The rotation member551may be integrally formed with the extension part550and provided such that one end of the rotation member551connected to the moving member511is rotatable.

According to an embodiment of the present disclosure, at least one of the moving member511or the rotation member551may be omitted, and the extension part550may be integrally provided with the body510.

The connection part570of the auxiliary input device500may be formed on one side surface of the extension part550to allow the auxiliary input device500to be connected to the electronic device200. The connection part570may include, for example, a USB connector connected to a USB port of the electronic device200. As shown inFIG. 5A, the auxiliary input device500may be connected to the electronic device200through a connection operation501.

According to an embodiment of the present disclosure, the auxiliary input device500may be connected not only to the electronic device200, but also to the main frame110of the HMD100. The auxiliary input device500may be electrically connected to the main frame110since the connection part570is inserted into a USB port formed in the placing part113of the main frame110.

According to an embodiment of the present disclosure, at least one component of the above-mentioned components of the auxiliary input device500may be omitted, or the auxiliary input device500may further include at least one additional component. In this case, the auxiliary input device500may serve as a sensing device that applies sensing information obtained through a sensor module disposed internally to the HMD100or the electronic device200.

FIG. 6Ais a view of an auxiliary input device that is unbent and partially bent, respectively, according to an embodiment of the present disclosure.

Referring toFIG. 6A, a portion of the auxiliary input device500may be bent within a predetermined angle. As shown inFIG. 6A, the body510may bent at a predetermined angle with respect to the extension part550by an operation that applies a force equal to or greater than a predetermined level to the body510or to the extension part550of the auxiliary input device500to bend the body510.

According to an embodiment of the present disclosure, in a case where the auxiliary input device500is placed on the placing part113of the main frame110, a direction to which the auxiliary input device500is bent may be a direction to which the first surface (e.g., the front surface) is exposed to the outside, e.g., a rear surface direction of the main frame110with respect to a normal line direction of the surface on which the placing part113of the main frame110is disposed, or a direction to which the second surface (e.g., the rear surface) is exposed to the outside, e.g., a front surface direction of the main frame110with respect to the normal line direction of the surface on which the placing part113of the main frame110is disposed.

According to an embodiment of the present disclosure, if a portion of the auxiliary input device500is bent and connected to the electronic device200or the main frame110, the auxiliary input device500may be placed on the placing part113of the main frame110. The auxiliary input device500may be placed on the placing part113after being bent while the auxiliary input device500is connected to the electronic device200or the main frame110. For example, the input module530of the auxiliary input device500may be disposed at the side surface of the main frame110(hereinafter, referred to as a side surface touch mode). A user may smoothly operate the input module530during the side surface touch mode using his/her hands.

As shown inFIG. 6A, if the auxiliary input device500is bent and the first surface is exposed to the outside, the user input may be received through the input module530(e.g., the first input module531) formed on the first surface. In addition, the sensor module disposed in the auxiliary input device500may set a state in which a portion of the auxiliary input device500is bent to an initial state of a six-axis motion sensing operation, and thus a sensor module may collect information on the movement of the head of a user.

FIG. 6Ashow an acute angle (or an angle exceeding zero degrees but less than 90 degrees) or a right angle (or an angle of 90 degrees) as the angle at which a portion of the auxiliary input device500is bent, but the present disclosure is not limited thereto or thereby. According to an embodiment of the present disclosure, an angle at which a portion of the auxiliary input device500is bent may be an obtuse angle (or an angle exceeding 90 degrees but less than 180 degrees) or a straight angle (or an angle of 180 degrees). Hereinafter, a state in which a portion of the auxiliary input device500is bent at an obtuse angle with reference to an embodiment of the present disclosure is described below.

FIG. 6Bis a view of an auxiliary input device unfolded and partially folded, respectively, according to an embodiment of the present disclosure.

Referring toFIG. 6B, a direction in which a portion of the auxiliary input device500is bent may be an obtuse angle. For example, the operation601shown inFIG. 6Amay be followed by an operation603that folds a portion of the auxiliary input device500if a force greater than or equal to a predetermined force is applied to the portion of the auxiliary input device500during a predetermined time period. The auxiliary input device500may be folded through the folding operation603to allow the first surface or the second surface of the body510to face one surface of the extension part550.

According to an embodiment of the present disclosure, in a state in which a portion of the auxiliary input device500is folded, the body510of the auxiliary input device500may be placed in the front surface direction of the main frame110instead of being placed on the placing part113of the main frame110. For example, the input module530of the auxiliary input device500may be placed at the front surface of the main frame110(hereinafter, referred to as a front surface touch mode). Since the touch surface of the input module530is the same as the display surface seen through the HMD100, the user may have a more user-intuitive experience.

As shown inFIG. 6B, if the auxiliary input device500is folded and the first surface is exposed to the outside, a user input may be received through the input module530(e.g., the first input module531) disposed on the first surface. In addition, a sensor module disposed in the auxiliary input device500may set a state in which a portion of the auxiliary input device500is folded to an initial state of a six-axis motion sensing operation, and thus the sensor module may collect information on the movement of the head of a user.

FIG. 6Cis a view of an auxiliary input device not rotated and partially rotated, respectively, according to an embodiment of the present disclosure.

Referring toFIG. 6C, a portion of the auxiliary input device500may be rotated. As shown inFIG. 6C, the body510may be rotated with respect to a rotation axis of a normal line direction of a surface (e.g., a side surface) in which the extension part550is connected within a predetermined angle by an operation605that applies a force greater than or equal to a predetermined force to the body510or the extension part550of the auxiliary input device500. InFIG. 6C, the rotation angle may be a right angle, but the present disclosure is not limited thereto or thereby. According to an embodiment of the present disclosure, a portion of the auxiliary input device500may be rotated within an angle (e.g., −360 degrees (or one turn in a counter-clockwise direction) to +360 degrees (or one turn in a clockwise direction)) in which the connection circuit line disposed in the extension part550would not be damaged.

According to an embodiment of the present disclosure, in the state in which a portion of the auxiliary input device500is rotated, the first and second surfaces of the body510of the auxiliary input device500may be exposed to the outside. Accordingly, a user may use the first input module531disposed on the first surface and the second input module533disposed on the second surface.

According to an embodiment of the present disclosure, the input module530of the auxiliary input device500may be provided in various shapes. For example, the input module530of the auxiliary input device500may include at least one of a touch pad, a trackball, a wheel key, a physical key, a physical button, a touch key, or a joystick.

FIGS. 7A to 7Care views of an input module of the auxiliary input device500, which is configured to include a trackball535.FIG. 7Ais a perspective view of the auxiliary input device500including the trackball535,FIG. 7Bis a front view of the auxiliary input device500including a trackball, andFIG. 7Cis a side view of the auxiliary input device500including the trackball535.

Referring toFIGS. 7A to 7C, the input module530disposed on one surface of the body510of the auxiliary input device500may include the trackball535. According to an embodiment of the present disclosure, the trackball535may be disposed in an area (e.g., a center area) of the input module530.FIGS. 7A to 7Cshow only one trackball535disposed in the input module530, but the number of the trackballs535is not limited to one. A plurality of trackballs535may be disposed on one surface of the body510, and one or more trackballs535may be disposed on a first surface (e.g., a front surface) and a second surface (e.g., a rear surface) of the body510.

A user may roll an input part having a spherical shape of the trackball535using his/her fingers or other parts of his/her body to move a cursor to a desired position or to move a screen. In addition, a user may select an object displayed on a screen or execute a predetermined function by pushing an input part having a spherical shape, pushing a button disposed adjacent to an input part having a spherical shape, or a touch pad of the input module530. According to an embodiment of the present disclosure, if an input operation is performed by using the trackball535, a movement may be more precisely controlled, and the movement may be controlled in a wide range as compared to a touch pad in which a movement is structurally restricted.

FIGS. 8A to 8Care views of an input module of the auxiliary input device500, which is configured to include a wheel key537.FIG. 8Ais a perspective view of the auxiliary input device500including the wheel key537,FIG. 8Bis a front view of the auxiliary input device500including the wheel key537, andFIG. 8Cis a side view of the auxiliary input device500including the wheel key537.

Referring toFIGS. 8A to 8C, the input module530disposed on one surface of the body510of the auxiliary input device500may include the wheel key537. According to an embodiment of the present disclosure, the wheel key537may be disposed in an area (e.g., an edge area) of the input module530. In a case where the auxiliary input device500includes the wheel key537, the body510may have approximately the same or similar shape as or to the shape (e.g., a circular shape) of the wheel key537. For example, the body510may have a substantially cylindrical shape. In addition, the input module530configured with the wheel key537may be disposed in an edge area of one surface of the body510having a cylindrical shape.

FIGS. 8A to 8Cshow only one wheel key537disposed on one surface of the body510, but the present disclosure is not limited thereto or thereby. According to an embodiment of the present disclosure, one or more wheel keys537may be disposed on a first surface (e.g., a front surface) and a second surface (e.g., a rear surface) of the body510.

A user may roll an input part having a circular shape of the wheel key537using his/her fingers or other parts of his/her body to move a screen or to execute a scroll operation (800). In addition, a user may select an object displayed on a screen or execute a predetermined function by pushing the wheel key537or pushing a touch pad of the input module530disposed adjacent to the wheel key537(e.g., disposed in the wheel key537). Similar to the trackball535, the wheel key537may allow a movement to be more precisely controlled and allow a movement to be controlled in a wide range as compared to a touch pad in which a movement is structurally restricted.

FIG. 9is a block diagram of an auxiliary input device500according to an embodiment of the present disclosure.

Referring toFIG. 9, the auxiliary input device500may include a control part910(or a control circuit), a first connection part920, a second connection part930, a power management part940, a sensor part950(or a sensor module), and an input part960(or an input module).

The control part910may include one or more of a central processing unit (CPU), an application processor (AP), or a communication processor (CP). The control part910may control a function of controlling at least one component of the auxiliary input device500and/or calculating or processing data of a communication.

According to an embodiment of the present disclosure, the control part910may apply a connection signal to the electronic device200or the HMD100if the auxiliary input device500is connected to the electronic device200or the main frame110of the HMD100. The electronic device200may provide a function selection interface to a screen of the electronic device200in response to a connection signal.

According to an embodiment of the present disclosure, the control part910may apply information obtained based on the sensor part950or the input part960to the electronic device200or the HMD100. For example, the controller910may apply information (e.g., head-tracking) obtained in response to the movement of the head of a user or information (e.g., touch information) obtained in response to a user input.

The first connection part920may provide an interface to connect the auxiliary input device500to the electronic device200or the main frame110of the HMD100. The first connection part920may include, for example, hardware devices, such as a plug or a connector, which allow devices to be connected to each other, and a connection circuit used to communicate between devices, e.g., a data transmission. For example, the first connection part920may include a high definition multimedia interface (HDMI), a USB, a connector in accordance with recommended standard 232 (RS-232), an optical interface, or a D-subminiature (D-sub) connector.

The second connection part930may provide an interface to receive power from an external power source. The second connection part930may have the same or similar configuration as or to the first connection part920. For example, the second connection part930may include an HDMI, a USB, a RS-232 connector, an optical interface, or a D-sub connector.

The power management part940may manage power of the auxiliary input device500. According to an embodiment of the present disclosure, the power management part940may include a power management integrated circuit (PMIC), a charger integrated circuit (charger IC), or a battery gauge. The PMIC may have a wired and/or wireless recharging scheme. The wireless charging scheme may include, for example, a magnetic resonance scheme, a magnetic induction scheme, or an electromagnetic wave based scheme, and an additional circuit, such as a coil loop, a resonance circuit, a rectifier, or the like may be added for wireless charging. The battery gauge may measure an amount of remaining power of a battery, a voltage, a current, or a temperature while a battery is being charged. A battery may include, e.g., a rechargeable battery and/or a solar battery.

According to an embodiment of the present disclosure, if the auxiliary input device500is connected to an external power source through the second connection part930, the power management part940may provide power to the auxiliary input device500and substantially simultaneously charge the electronic device200or the HMD100connected to the auxiliary input device500. If the auxiliary input device500is connected to the external power source, the power management part940may directly charge the electronic device200or the HMD connected to the auxiliary input device500without providing power to the auxiliary input device500.

The sensor part950may measure a physical quantity or sense an operation status of the auxiliary input device500and convert the measured or sensed information to an electrical signal. The sensor part950may include, for example, an acceleration sensor, a gyro sensor, or a geomagnetic field sensor, but the present disclosure is not limited thereto or thereby.

According to an embodiment of the present disclosure, when the auxiliary input device500is connected to the HMD100or the electronic device200connected to the HMD100, the sensor part950may sense the movement of the head of a user wearing the HMD100. For example, the sensor part950may sense a rotation of the head (e.g., a left-and-right rotation or an up-and-down rotation) or a movement in a position of the head to measure a position of the head, a moving distance, a moving speed, a rotation distance, or a rotation speed.

The input part960may receive a user input, e.g., a touch input, a gesture input, or a hovering input. The input part960may include at least one of a touch pad, a trackball, a wheel key, a physical key, a physical button, a touch key, or a joystick. In addition, the input part960may further include a control circuit to control the above-mentioned input device or to process an input signal. A touch pad may recognize a touch input by using at least one of a capacitive method, a resistive method, an infrared method and an ultrasonic detecting method. A capacitive type touch pad may recognize a physical contact or a proximity touch. A touch pad may further include a tactile layer to provide a tactile reaction to a user.

The configuration of the auxiliary input device500is not limited thereto or thereby the configurations described above. According to an embodiment of the present disclosure, at least one component of the above-mentioned components may be omitted, or an additional component may be added to the auxiliary input device500. For example, the auxiliary input device500may further include a communication module (or a wired/wireless communication circuit). A communication module may provide an interface for the wired communication or the wireless communication with the electronic device200, the HMD100, or an external electronic device. A communication module may provide an interface for at least one format of, for example, long-term evolution (LTE), LTE advanced (LTE-A), code division multiple access (CDMA), wideband CDMA (WCDMA), universal mobile telecommunications system (UMTS), wireless broadband (WiBro), global system for mobile communications (GSM), or the like. Furthermore, a communication module may provide an interface for a local area network including at least one of wireless fidelity (WiFi), Bluetooth, near field communication (NFC), magnetic stripe transmission (MST), or global navigation satellite system (GNSS).

As described above, according to an embodiment of the present disclosure, a kit for a head-mounted system may include a head-mounted display device worn on a head of a user and configured to include a main frame coupled to an electronic device and a support part fixing the main frame to a facial side of the user, and an auxiliary input device configured to include a body, an input module disposed on at least one surface of the body and processing an input signal for controlling a function of the electronic device, a sensor module disposed in the body and obtaining sensing information in accordance with a movement of the auxiliary input device, and a connection part electrically connected to the electronic device to serve as a path through which the input signal and the sensing information are transmitted to the electronic device.

According to an embodiment of the present disclosure, a sensor module may include at least one of an acceleration sensor, a gyro sensor, and a geomagnetic field sensor.

According to an embodiment of the present disclosure, an input module may include at least one of a touch pad, a trackball, and a wheel key.

According to an embodiment of the present disclosure, the auxiliary input device may further include a communication module, where the communication module may transmits at least one of an input signal and sensing information to an electronic device in a wireless manner.

According to an embodiment of the present disclosure, the auxiliary input device may further include an extension part disposed between a body and a connection part, and the extension part may be fixed to a moving member formed in one surface of the body to allow the extension part to move.

According to an embodiment of the present disclosure, a moving member may be inserted into a hole formed in one surface of a body, where the hole has a length and a width, and where one end of the moving member may be inserted into the hole such that the moving member is affixed to the hole.

According to an embodiment of the present disclosure, a hole may be formed in at least one of a vertical direction and a horizontal direction on one surface of a body, where the hole has a length.

According to an embodiment of the present disclosure, a hole may extend to another surface connected to one surface of a body and may be formed in a portion of the other surface connected to the one surface, and the moving member may move to the extension portion of the hole such that an extension part is bent or folded with respect to the body within an angle.

According to an embodiment of the present disclosure, a moving member may include an elastic material as at least a portion thereof, and the moving member may be deformed or restored due to an elastic force in a case where an extension part is bent or folded with respect to a body within an angle.

According to an embodiment of the present disclosure, an extension part may further include a rotation member therein, and the rotation member may allow the extension part to rotate about a normal line of one surface of a body within an angle.

According to an embodiment of the present disclosure, a rotation member may be inserted into a hollow formed in an extension part such that the rotation member is affixed to the extension part.

According to an embodiment of the present disclosure, a connection part may be connected to one surface of a main frame and electrically connected to an electronic device through an electrical connection part disposed on the main frame if the electronic device is installed at the main frame.

As described above, according to an embodiment of the present disclosure, a kit for a head-mounted system may include a head-mounted display device worn on a head of a user that is configured to include a frame structure at which a mobile electronic device is installed, a support member for fixing the frame structure to the head of the user, and a sensing device configured to include a housing, a wired and/or wireless communication circuit included in the housing, at least one sensor included in the housing, and a control circuit electrically connected to the communication circuit and the at least one sensor. The control circuit may be configured to obtain information associated with a movement of the sensing device using the at least one sensor, and to transmit the obtained information to the mobile electronic device using the communication circuit.

According to an embodiment of the present disclosure, at least one touch sensor may include at least one of a touch pad, a trackball, and a wheel key.

According to an embodiment of the present disclosure a sensing device may further include a connector (or connection part) disposed on at least a portion of a housing and electrically connected to a control circuit, where the control circuit may be configured to transmit obtained information to a mobile electronic device through the connector.

According to an embodiment of the present disclosure, a control circuit may be configured to transmit obtained information to a mobile electronic device using a communication circuit.

As describe above, according to an embodiment of the present disclosure, an electronic device (or a sensing device) detachably attached to a mobile electronic device to provide a signal may include a housing including a first surface and a second surface facing an opposite side of the first surface, at least one touch sensor disposed on the first surface and/or the second surface of the housing, at least one sensor disposed in the housing, a communication circuit disposed in the housing, and a control circuit electrically connected to the at least one touch sensor, the at least one sensor, and the communication circuit. The communication circuit may be configured to obtain a gesture or touch input from a user using the at least one touch sensor, obtain information associated with an orientation and/or a movement of the electronic device using the at least one sensor, and transmit the obtained input and information to the mobile electronic device using the communication circuit.

As describe above, according to an embodiment of the present disclosure, a mobile electronic device detachably attached to a head-mounted display device worn on a head of a user may include a communication circuit, a display, a processor electrically connected to the communication circuit and the display, and a memory electrically connected to the processor. The memory may store instructions, when executed by the processor, causing the processor to receive at least one input signal from an external sensing device using the communication circuit, display content associated with the head-mounted display device through at least a portion of the display based on at least a portion of the obtained at least one input signal, receive information on an orientation and/or a movement of the external sensing device, which are obtained by the external sensing device, from the external sensing device, and to change the content displayed on the display based on the received information.

FIG. 10is a flowchart of a method of operating an electronic device associated with a coupling of an auxiliary input device according to an embodiment of the present disclosure.

Referring toFIG. 10, the electronic device200may determine whether the auxiliary input device500is coupled to an electronic device in operation1010. According to an embodiment of the present disclosure, the auxiliary input device500may apply a connection signal to the electronic device200when the auxiliary input device500is connected to the electronic device200. The electronic device200may support a plug and play function, and, thus, a system may automatically sense the connection of the auxiliary input device500.

According to an embodiment of the present disclosure, in a case where the auxiliary input device500is not connected to the electronic device200, the electronic device200may not execute the operations (e.g., operation1020to operation1080) described below. In a case where the auxiliary input device500is connected to the electronic device200, the electronic device200may provide a function selection interface of the electronic device200in operation1020. The electronic device200may output a screen through which a function of the electronic device200may be selected.

According to an embodiment of the present disclosure, a function selection interface of the electronic device200may include a display object (e.g., a button object) used to select a first function (e.g., a Gear VR® function) and a second function (e.g., an electronic device function).

In operation1030, the electronic device200may determine whether a first function is selected. According to an embodiment of the present disclosure, in a case where the first function is not selected, the electronic device200may determine whether a second function is selected in operation1040. In a case where the second function is not selected, the electronic device200may return to operation1020and may wait until a user selects the first function or the second function. In a case where a predetermined time lapses while a user does not select any one of the first and second functions, the electronic device200may recognize that one of the first function or the second function is selected. The electronic device200may stop outputting a screen and enter a reduced power (or sleep) mode.

According to an embodiment of the present disclosure, when the second function is selected, the electronic device200may perform the second function in operation1050. The second function may include a general function (e.g., a call function, a message function, a multimedia file execution function, a camera function, or the like) of the electronic device200. If the second function is selected, the electronic device200may prepare the auxiliary input device500to be used as an auxiliary input tool with an input device included in the electronic device200.

According to an embodiment of the present disclosure, if the first function is selected, in operation1060, the electronic device200may convert the state of the electronic device200to a first state to perform the first function. The first function may include a VR function of the electronic device200in association with the use of the HMD100. In addition, the first state (e.g., a plug-in mode) may include a state where the electronic device200may rapidly perform an application for the VR. If the state of the electronic device200is converted to the first state, the electronic device200may prepare an application for the VR to be executed. For example, the electronic device200may output a separate mono image on the screen thereof and drive the application for the VR as its background.

In operation1070, the electronic device200may determine whether the electronic device200is installed on the HMD100. According to an embodiment of the present disclosure, the HMD100may further include a sensor (e.g., a proximity sensor) arranged in an area of the front case117of the main frame110to sense whether the electronic device200is installed at the HMD100or the HMD100is worn on the head of a user. In this case, when the electronic device200is coupled to the HMD100, the HMD100may apply a coupling signal to the electronic device200, and the electronic device200may determine whether the electronic device200is installed at the HMD100in response to the coupling signal. The electronic device200may support a plug and play function, and, thus, the system may automatically sense a connection of the electronic device200when the electronic device200is connected to the connector disposed on the main frame110of the HMD100.

According to an embodiment of the present disclosure, in a case where the electronic device200is not installed at the HMD100or the HMD100is not worn on the head of a user, the electronic device200may return to operation1060and wait until the electronic device200is installed at the HMD100or the HMD100is worn on the head of a user. In a case where a predetermined time elapses while the electronic device200is not installed at the HMD100, the electronic device200may stop outputting a screen and enter into a sleep mode.

According to an embodiment of the present disclosure, in a case where the electronic device200is installed at the HMD100or the HMD100is worn on the head of a user, a state of the electronic device200may be converted to a second state (e.g., a mount mode) in operation1080. If the state of the electronic device200is converted to the second state, the electronic device200may stop outputting a screen that is displayed and may output an execution screen of an application for VR.

According to an embodiment of the present disclosure, a state of the electronic device200may not be converted to the second state even though the electronic device200is installed at the HMD100. For example, if the electronic device200is installed at the HMD100, the state of the electronic device200may not be converted to the second state if a user is not wearing the HMD100on his/her head. If the HMD100is worn on the head of a user, the electronic device200may convert the state of the electronic device200to the second state in operation1080. In this regard, it may be determined whether the HMD100is worn on the head of a user by using a sensor included in the main frame110of the HMD100or the electronic device200. For example, when a proximity sensor is installed in the main frame110, it may be determined whether a user puts on the HMD100by recognizing the approach of the face of the user to the HMD100through the proximity sensor. An iris-recognition sensor may be installed in the main frame110, and, in this case, it may be determined whether the HMD100is worn on the head of a user by identifying an iris of the user through the iris-recognition sensor when the user wears the HMD100.

According to an embodiment of the present disclosure, an order in which the auxiliary input device500is coupled to the electronic device200and the electronic device200is coupled to the HMD100is not limited to a certain order. In addition, the order of the above-mentioned operations may be variously modified or several operations may be omitted. For example, the auxiliary input device500may be coupled to the electronic device200after the HMD100is coupled to the electronic device200. In this case, the electronic device200may perform operation1080with priority to display an execution screen of an application for VR on a screen thereof. However, since the auxiliary input device500is not connected to the electronic device200, some functions of the application for VR may not be performed. For example, the movement of the head of a user may be sensed by the sensor included in the electronic device200, but a touch input may not be received since the input module exposed to the outside may not exist. If the electronic device200is coupled to the HMD100and the auxiliary input electronic device500is coupled to the electronic device200, a user input may be received, and all functions of the application for VR may be performed. In a case where the auxiliary input device500is coupled to the electronic device200after the electronic device200is coupled to the HMD100, operations1010to1070may be omitted.

According to an embodiment of the present disclosure, the auxiliary input device500may be coupled to the main frame110instead of being coupled to the electronic device200. In this case, the order of the above-mentioned operations may be variously modified or several operations may be omitted. For example, in a case where the auxiliary input device500is coupled to the main frame110of the HMD100after the electronic device200is coupled to the HMD100, the same or similar operations as or to the operations when the auxiliary input device500is coupled to the electronic device200after the electronic device200is coupled to the HMD100may be performed. However, in a case where the electronic device200is coupled to the HMD100after the auxiliary input device500is coupled to the main frame110of the HMD100, the above-mentioned operations (e.g., operation1010to operation1080) may be sequentially performed. In this case, a button object displayed on a screen of the electronic device200may be selected by using the input module530of the auxiliary input device500instead of touching the screen of the electronic device200in the function selection interface of the electronic device200provided in operation1020.

According to an embodiment of the present disclosure, in a case where the auxiliary input device500is separated from the main frame110of the HMD100, the auxiliary input device500may automatically form a pairing with the electronic device200coupled to the HMD100and serve as a remote input device of the electronic device200. In a case where the auxiliary input device500is separated from the electronic device200while the electronic device200is coupled to the HMD100and the auxiliary input device200is connected to the electronic device500, the auxiliary input device500may automatically form a pairing with the electronic device200and serve as a remote input device. Then, in a case where the auxiliary input device500is coupled to the main frame110of the HMD100again, a pairing between the auxiliary input device500and the electronic device200may be automatically disabled.

FIG. 11is an illustration of a function selecting interface of the electronic device, which is provided when the auxiliary input device is coupled to the electronic device according to an embodiment of the present disclosure.

Referring toFIG. 11, if the auxiliary input device500is coupled to the electronic device200, the electronic device200may provide an interface on a display220to select a function of the electronic device200. According to an embodiment of the present disclosure, the electronic device200may display a display object (e.g., a first button object221aand a second button object221b) through the display220to select a first function (e.g., a Gear VR® function) and a second function (e.g., an electronic device function) of the electronic device200.

According to an embodiment of the present disclosure, if the first button object221ais selected, the electronic device200may be switched to a first state (e.g., a plug-in mode) to perform a first function. For example, the electronic device200may drive an application for VR as a background to rapidly perform the application for VR. The electronic device200may secure a memory space required to drive the application for VR and load resources of the application for VR into the memory instead of driving the application for VR. If the first button object221ais selected, the electronic device200may stop outputting the function selection interface to return to a screen before an output of a function selection interface or may not display a separate mono image.

According to an embodiment of the present disclosure, if the second button object221bis selected, the electronic device200may perform a second function. The electronic device200may stop outputting the function selection interface and return to a screen before the screen that outputs the function selection interface to general functions of the electronic device200.

FIG. 12Ais an illustration of a first input function performed through an auxiliary input device according to an embodiment of the present disclosure,FIG. 12Bis an illustration of a screen provided in accordance with the first input function ofFIG. 12Aaccording to an embodiment of the present disclosure,FIG. 12Cis an illustration of a second input function performed through an auxiliary input device according to an embodiment of the present disclosure, andFIG. 12Dis an illustration of a screen provided in accordance with the second input function ofFIG. 12Caccording to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, the HMD100may provide a user with various experiences using an input function of the auxiliary input device500. An input function of the auxiliary input device500may include object selection, movement of an object, rotation of an object, movement of a screen, rotation of a screen, screen zoom in/out, or screen scroll.FIGS. 12A to 12Bshow a first input function associated with the movement of an object or the movement of a screen, andFIGS. 12C to 12Dshow a second input function associated with the rotation of an object, the rotation of a screen, or screen zoom in/out.

As shown inFIG. 12A, the auxiliary input device500may receive a first input1210through the input module530arranged in the body510, where the auxiliary input device500may then perform the screen movement1220corresponding to a moving direction and a moving degree indicated by the first input1210on a first screen1201output to the electronic device200to convert the first screen1201to a second screen1203as shown inFIG. 12B. According to an embodiment of the present disclosure, the first input1210may be an input swiping the touch surface of the input module530.

According to an embodiment of the present disclosure, the electronic device200may move not only a screen (i.e., the screen movement1220), but also an object (e.g., a3D object). For example, in a case where an application for VR is a VR shopping mall application, the electronic device200may perform the screen movement1220in response to the first input1210provided through the auxiliary input device500to allow a user to look around a store and to move a certain product.

In addition, as shown inFIG. 12C, if the auxiliary input device500receives a second input through the first and second input modules531and533arranged in the body510, the auxiliary input device500may perform a screen rotation1240corresponding to a rotation direction and a rotation degree indicated by the second input on a third screen1205output to the electronic device200to convert the third screen1205to a fourth screen1207as shown inFIG. 12D. According to an embodiment of the present disclosure, the second input may be an input simultaneously swiping touch surfaces of the first and second input modules531and533or swiping the touch surfaces of the first and second input modules531and533at regular time intervals. InFIG. 12C, the user may swipe the first input module531in a first direction using a thumb (1230a) and swipe the second input module533in a second direction using a finger (1230b).

According to an embodiment of the present disclosure, a first direction may be the same as or different from a second direction. In a case where the first and second directions are the same, the electronic device200may zoom in or zoom out the screen in response to a size of the swipe input. For example, in a case where the first and second directions are of the same direction, i.e., directed toward the electronic device200, the electronic device200may zoom in the screen, and in a case where the first and second directions are directed toward an opposite direction to the electronic device200, the electronic device200may zoom out the screen. In addition, in a case where the first and second directions are different from each other, i.e., directed toward opposite directions, the electronic device200may rotate a screen or an object.FIG. 12Dshows the state where a certain product (e.g., shoes) is rotated due to the first direction swipe1230aand the second direction swipe1230bopposite to the first direction swipe1230a.

InFIGS. 12A to 12D, the input module530of the auxiliary input device500may be implemented by a touch pad, but the auxiliary input device500may provide an input function using various input modules530, such as the trackball535, the wheel key537, or the like.

FIG. 13Ais an illustration of a function sensing a movement of a user's head through an auxiliary input device according to an embodiment of the present disclosure, andFIG. 13Bis an illustration of a function receiving a user input through an auxiliary input device according to an embodiment of the present disclosure.

According to an embodiment of the present disclosure, the electronic device200may control an external electronic device1300(e.g., a drone) paired with the electronic device200. The drone1300may include a camera, and the electronic device200may control a flight of the drone1300and a field of view of the camera included in the drone1300using the input module of the auxiliary input device500.

Referring toFIG. 13A, a user may receive image data photographed by the drone1300while wearing the HMD100and may experience a real-time virtual environment in the form of a real view101. In this case, if the movement1310of the head of the user occurs, the electronic device200may apply head-tracking data1301, which are sensed by a sensor module (e.g., an acceleration sensor, a gyro sensor, a geomagnetic field sensor, or the like) arranged in the auxiliary input device500, the electronic device200(e.g., a mobile phone), or the HMD100, to the drone1300paired with the electronic device200.

The drone1300receiving the head-tracking data1301may analyze the head-tracking data1301and change the field of view of the camera to a second photographing area1313from a first photographing area1311based on the analyzed result. In addition, the drone1300may apply photographing image data1303in accordance with the changed field of view to the electronic device200. Accordingly, a user may confirm the changed screen in the real view101.

Referring toFIG. 13B, the electronic device200may control a flight of the drone1300through the input module530of the auxiliary input device500. For example, in a case where a user swipes a touch surface of the input module530in a predetermined direction or zooms in or zooms out the screen using the first and second input modules531and533, the electronic device200may apply position control data1305corresponding to a user input1330to the drone1300paired with the electronic device200.

According to an embodiment of the present disclosure, the drone1300receiving the position control data1305may analyze the position control data1305and perform a position change1331to a second position from a first position based on the analyzed result. In addition, the drone1300may apply photographing image data1307at the changed position to the electronic device200. Accordingly, a user may confirm the changed screen in the real view101.

As described above, according to an embodiment of the present disclosure, a method of selecting a function of an electronic device coupled to a head-mounted display device may include determining whether an auxiliary input device including at least one of a sensor module or an input module is coupled to the head-mounted display device, providing an interface to select a first function and a second function of the electronic device when the auxiliary input device is coupled to the head-mounted display device, and changing an execution state of the electronic device based on a selected result of the first function or the second function.

According to an embodiment of the present disclosure, providing an interface may include outputting a first display object corresponding to a first function and a second display object corresponding to a second function to a display of an electronic device.

According to an embodiment of the present disclosure, changing an execution state may include switching an execution state to a first state in which an electronic device prepares a virtual reality function associated with a use of a head-mounted display device to be executed when the first function is selected.

According to an embodiment of the present disclosure, switching an execution state to a first state may include driving an application used for a virtual reality function, which is separated from a function displayed on a display of an electronic device.

According to an embodiment of the present disclosure, a function selection method may include switching a first state to a second state in which an execution screen of a virtual reality function is output when an electronic device is coupled to a head-mounted display device.

According to an embodiment of the present disclosure, changing an execution state may include switching an execution state to a third state in which a general function not associated with a use of a head-mounted display device is executed when a second function is selected.

According to an embodiment of the present disclosure, switching an execution state to a third state may include returning to a screen displayed on a display of an electronic device before an interface is provided.

According to an embodiment of the present disclosure, a function selection method may include pairing an auxiliary input device and an electronic device if the auxiliary input device is separated from the electronic device in a state where the electronic device is coupled to a head-mounted display device and the auxiliary input device is then coupled to the electronic device, wireless-transmitting at least one of sensing information obtained through a sensor module of the auxiliary input device or an input signal obtained through an input module of the auxiliary input device to the electronic device, and disabling the pairing when the auxiliary input device is re-coupled to the electronic device.

FIG. 14is a block diagram of an HMD100according to an embodiment of the present disclosure.

Referring toFIG. 14, the HMD100according to an embodiment of the present disclosure may include a micro-controller unit (MCU)1410, a communication module1420, an input module1430, a power management module1440, a battery1441, a sensor module1450, an eye tracker1460, a vibrator1470, adjustable optics (or a lens assembly)1480, and a memory1490.

InFIG. 14, some components, e.g., a display, may not be shown for the convenience of explanation. According to an embodiment of the present disclosure, a portion of the components shown inFIG. 14may be included in the main frame110, and the other portion of the components shownFIG. 14may be included in the electronic device200(e.g., an attachable and detachable smart phone) or the auxiliary input device500.

The micro-controller unit1410may include, for example, a processor which may drive an operating system (OS) or an embedded software program to control a plurality of hardware components connected to the micro-controller unit1410.

The communication module1420may connect the main frame110of the HMD100and the electronic device200, e.g., a smart phone terminal, using wired and/or wireless communications to perform data transmission and reception. According to an embodiment of the present disclosure, the communication module1420may include a USB module1421, a WiFi module1423, a BT module1425, an NFC module1427, or a GPS module1429. At least a portion (e.g., two or more) of the WiFi module1423, the BT module1425, the NFC module1427, or the GPS module1429may be integrated into one integrated circuit (IC) or an IC package.

The input module1430may include a touch pad1431or a button1433. The touch pad1431may recognize a touch input by using at least one of a capacitive method, a resistive method, an infrared method and an ultrasonic detecting method. In addition, the touch pad1431may include a control circuit. A capacitive type touch pad1431may recognize a physical contact or a proximity touch. The touch pad1431may include a tactile layer. In this case, the touch pad1431may provide a tactile reaction to a user. The button1433may include, for example, a physical button, an optical key, or a keypad.

The power management module1440may manage, for example, power of the HMD100. According to an embodiment of the present disclosure, the power management module1440may include a power management integrated circuit (PMIC), a charger integrated circuit (charger IC), or a battery1441gauge. The PMIC may have a wired and/or wireless recharging scheme. The wireless recharging scheme may include, for example, a magnetic resonance scheme, a magnetic induction scheme, or an electromagnetic wave based scheme, and an additional circuit, such as a coil loop, a resonance circuit, a rectifier, or the like may be added for wireless charging. The battery gauge may measure, for example, an amount of remaining power of the battery1441, a voltage, a current, or a temperature while the battery1441is being charged. The battery may include, e.g., a rechargeable battery and/or a solar battery. If the electronic device200is connected to the HMD100, the power management module1440may use a battery included in the electronic device200. The power management module1440may manage power provided from an external power source.

The sensor module1450may measure, for example, a physical quantity or may detect an operation state of the HMD100. The sensor module1450may convert the measured or detected information to an electrical signal. The sensor module1450may include, for example, at least one of an acceleration sensor1451, a gyro sensor1452, a geomagnetic field sensor1453, a magnetic sensor1454, a proximity sensor1455, a gesture sensor1456, a grip sensor1457, or a biometric sensor1458. According to an embodiment of the present disclosure, the movement of the head of a user wearing the HMD100may be sensed by using at least one of the acceleration sensor1451, the gyro sensor1452, or the geomagnetic field sensor1453. It may be determined whether the HMD100is worn by a user by using the proximity sensor1455or the grip sensor1457.

According to an embodiment of the present disclosure, at least a portion of components of the sensor module1450may be included in the electronic device200or the auxiliary input device500, which is attachable or detachable. The sensor module1450may further include an infrared (IR) sensor, a pressure sensor, or a touch sensor and sense at least one of an IR radiation, pressure, a variation in an amount of capacitance (or dielectric constant) to determine whether a user is wearing the HMD100.

The gesture sensor1456may sense a movement of a user's hand or finger and receive an input operation of the HMD100. Additionally or alternatively, the sensor module1450may recognize biometric information of a user using an electronic nose (E-nose) sensor, an electromyography sensor (EMG) sensor, an electroencephalogram (EEG) sensor, an electrocardiogram (ECG) sensor, an iris sensor, and/or a fingerprint sensor. The sensor module1450may include a control circuit for controlling at least one or more sensors included therein. According to an embodiment of the present disclosure, the HMD100may include a processor which is a part of the micro-controller unit1410or independent of the micro-controller unit1410and is configured to control the sensor module1450. The processor may control the sensor module1450while the micro-controller unit1410is in a sleep state.

The eye tracker1460may track the eyes of a user by using at least one of electrical oculography (EOG) sensors, coil systems, dual-Purkinje systems, bright pupil systems, or dark pupil systems. In addition, the eye tracker1460may include a micro camera to track a user's eyes.

The vibrator1470may convert an electrical signal to a mechanical vibration and generate a vibration or haptic effect.

The adjustable optics1480may measure an inter-pupil distance (IPD) of a user and control a distance of a lens and a display position of the electronic device200to allow the user to view an image appropriate to his/her eyesight.

The memory1490may include an internal memory or an external memory. The internal memory may include, for example, at least one of a volatile memory (e.g., a dynamic random access memory (DRAM), a static RAM (SRAM), or a synchronous DRAM (SDRAM)), a nonvolatile memory (e.g., a one-time programmable read only memory (OTPROM), a programmable ROM (PROM), an erasable and programmable ROM (EPROM), an electrically erasable and programmable ROM (EEPROM)), a mask ROM, a flash ROM, a flash memory (e.g., a NAND flash memory or a NOR flash memory), a hard drive, or a solid state drive (SSD). The external memory may include a flash drive, for example, a compact flash (CF) drive, a secure digital (SD) drive, a micro secure digital (Micro-SD) drive, a mini secure digital (Mini-SD) drive, an extreme digital (xD) drive, a multimedia card (MMC), a memory stick, or the like. The external memory may be functionally and/or physically connected to the HMD100through various interfaces.

The memory1490may store instructions or data associated with at least one other component of the HMD100. According to an embodiment of the present disclosure, the memory1490may store software and/or a program. The program may include, for example, a kernel, middleware, an application programming interface (API), and/or an application program (or an application). At least a portion of the kernel, the middleware, or the API may be referred to as an operating system (OS).

A kernel may control or manage system resources (e.g., the micro-controller unit1410or the memory1490) that are used to execute operations or functions of other programs (e.g., middleware, an API, and an application program). Furthermore, a kernel may provide an interface that allows middleware, an API, or an application program to access discrete components of the HMD100so as to control or manage system resources.

Middleware may perform a mediation role such that an API or application program communicates with a kernel to exchange data. Furthermore, middleware may process task requests received from an application program according to a priority. For example, middleware may assign a priority, which makes it possible to use a system resource (e.g., the micro-controller unit1410, the memory1490, or the like) of the HMD100, to at least one of an application program. For example, middleware may process one or more task requests according to a priority assigned to at least one application program, which makes it possible to perform scheduling or load balancing on one or more task requests.

An API may be an interface through which an application program controls a function provided by a kernel or middleware and may include, for example, at least one interface or function (e.g., an instruction) for file control, window control, image processing, character control, or the like.

An application may include a short message service/multimedia messaging service (SMS/MMS) application, an email application, a calendar application, an alarm application, a health care application (e.g., an application measuring an exercise quantity or blood sugar level), or an environmental application (e.g., atmospheric pressure, humidity, or temperature). Additionally or alternatively, an application may include an application associated with information exchange between the HMD100and the electronic device200. The information exchanging application may include, for example, a notification relay application for transmitting certain information to the electronic device200, or a device management application for managing the electronic device200.

A notification relay application may include a function of transmitting notification information, which arises from other applications (e.g., applications for SMS/MMS, e-mail, health care, or environmental information) of the HMD100to the electronic device200. Additionally, a notification relay application may receive, for example, notification information from the electronic device200and provide the notification information to a user.

A device management application may manage (e.g., install, delete, or update), for example, at least one function (e.g., turn-on/turn-off an external electronic device (or a part of components) or adjustment of brightness (or resolution) of a display) of the electronic device200, which communicates with the HMD100, an application running in the electronic device200, or a service (e.g., a call service or a message service) provided from the electronic device200.

FIGS. 15A and 15Bare illustrations of a normal mode, a head-mounted mode (HMM), and a VR mode according to an embodiment of the present disclosure.

The HMM mode or the VR mode may provide at least one of a see-through function for providing an augmented reality (AR) and a see-closed function for providing a virtual reality (VR). For example, in a case where a smart phone is installed at the main frame110of the HMD100as the electronic device200and operated, the operation mode of the smart phone may be switched to the HMM mode or the VR mode from a normal mode. In the HMM mode or the VR mode, one image may be displayed after being divided into two images. According to an embodiment of the present disclosure, an image in the HMM or VR mode may be distorted by the lens270included in the main frame110, and, thus, a plane image may be inversely distorted using characteristics of the lens270to provide a user with an image that is not distorted.

FIGS. 16A and 16Bare illustrations of the HMD100providing a see-through mode using a rear view camera of the smart phone according to an embodiment of the present disclosure.

Referring toFIGS. 16A and 16B, in a case where a smart phone is installed at the main frame110as the electronic device200and operated, the HMD100may provide a see-through mode using a rear view camera of the smart phone. According to an embodiment of the present disclosure, when a see-through mode switching button is pushed during a VR mode, a rear view camera is activated, and, thus, the see-through mode is provided. In this case, a preview screen of the rear view camera may be displayed in a picture-in-picture (PIP) mode through a certain area of a VR screen, or the preview screen of the rear view camera may be displayed through an entire area of a screen after the VR screen is switched to a background image. Thus, a user may explore an external virtual environment and substantially simultaneously confirm a surrounding environment through a camera image.

The term “module” used herein may represent, for example, a unit including one or more combinations of hardware, software and firmware. The term “module” may be interchangeably used with the terms “unit,” “logic,” “logical block,” “component,” and “circuit.” The term “module” may indicate a minimum unit of an integrated component or may be a part thereof. The term “module” may indicate a minimum unit for performing one or more functions or a part thereof. The term “module” may be implemented mechanically or electronically. For example, the term “module” according to an embodiment of the present disclosure may include at least one of an application-specific IC (ASIC), a field-programmable gate array (FPGA), and a programmable-logic device for performing some operations, which are known or will be developed.

According to an embodiment of the present disclosure, at least a portion of an apparatus (e.g., modules or functions thereof) or a method (e.g., operations), for example, may be implemented by instructions stored in a non-transient computer-readable storage medium in the form of a programmable module. An instruction, when executed by one or more processors (e.g., an application processor), may perform a function corresponding to the instruction. A non-transient computer-readable storage medium, for example, may be the memory1490.

A non-transient computer-readable storage medium may include a hard disk, a floppy disk, a magnetic media (e.g., a magnetic tape), an optical media (e.g., compact disc ROM (CD-ROM) and a DVD), a magneto-optical media (e.g., a floptical disk), and a hardware device (e.g., ROM, RAM, a flash memory, or the like). Also, a program instruction may include not only machine code generated by a compiler but also high-level source code executable on a computer using an interpreter. The above hardware unit may be configured to operate via one or more software modules for performing an operation of the present disclosure, and vice versa.

A module or a programming module according to an embodiment of the present disclosure may include at least one of the above elements, or a portion of the above elements may be omitted, or additional other elements may be further included. Operations performed by a module, a programming module, or other elements according to an embodiment of the present disclosure may be executed sequentially, in parallel, repeatedly, or in a heuristic method. Also, a portion of operations may be executed in different sequences, omitted, or other operations may be added.