Patent Description:
An HMD is configured to display a video, in which an augmented reality (AR) image (AR image of an avatar or the like) generated by a computer is superimposed on an image of a real space, on a display screen in the form of spectacles. The AR images are generally stored in external servers. In order for an HMD to accurately extract information about an area where the HMD is currently present from among a large number of AR images stored in the external servers, or in order for the external servers to properly provide each HMD with information, it is necessary to identify the area where the HMD is present.

As a technique for identifying an area where an HMD is present, for example, Patent Literature <NUM> discloses "an experience facility comprises a plurality of section areas. A detection antenna D receives an identification code generated by a head mount display device (HMD) to detect in which section area the HMD exists among the section areas, and in which direction the HMD faces. A storage means stores image information different from each section area and an extract means extracts the image information in response to the detected direction in the image information in the corresponding section area. The HMD displays the extracted virtual image (excerpted from Abstract)".

<CIT> discloses an information providing system that enables a mobile terminal, when it is located within a communicable area of a wireless access point, to easily acquire information content related to the communicable area, a mobile terminal, a wireless access point, a charging server and an information providing method. When entering an area where information can be received, a mobile terminal displays that it has entered an area where information can be acquired on the screen of a display section. When the mobile terminal user operates the mobile terminal to select a predetermined area on the display, a request to acquire information content is sent to an information providing server via a network.

<CIT> discloses an HMD that generates region information indicating a region having, as an axis, a line which passes through the position of the HMD and that of a wireless I/F. The HMD acquires position information of each of wireless I/Fs of other image processing apparatuses. The HMD determines if the position indicated by the acquired position information is included in the region indicated by the region information. The HMD selects one of the wireless I/F and other wireless I/Fs as a wireless communication partner according to the determination result, and receives images via wireless communication with the selected wireless I/F.

In the technique disclosed in Patent Literature <NUM>, a total of sixteen reception antennas receives an identification signal transmitted from the HMD, whereby an area where the HMD is present is identified based on the relative topology of each of the reception antennas. Furthermore, a total of four transmission antennas installed in advance transmits position codes which are different to each other, respectively, and then the HMD identifies an area where it is present based on the relative topology of the received four position codes. Thus, in order to identify an area where an HMD is present, it is necessary to install a new facility such as a plurality of antennas, and this requires a huge cost.

The present invention has been made in view of the circumstances described above, and an object of the present invention is to provide a technique enabling identification of an area where a portable information device having a communication function, such as an HMD, is present with necessary and sufficient accuracy without requiring a special facility (hardware).

In order to solve the problems described above, the present invention provides a portable information device as specified in claim <NUM>. The portable information device may optionally be as specified in any of claims <NUM> to <NUM>.

It is possible to identify an area where an HMD is present with necessary and sufficient accuracy without requiring a special facility (hardware). The objects, configurations, and advantageous effects other than those described above will be clarified by explanation of the embodiments below.

Throughout the drawings, the same features are provided with the same reference signs, and repetitive explanation therefor will be omitted.

Firstly, an outline of the present embodiment will be described. <FIG> illustrates an outline and operating environment of a presence area identification system <NUM> according to the present embodiment.

As illustrated in <FIG>, the presence area identification area <NUM> of the present embodiment includes an HMD <NUM>, an access point <NUM> installed in each room <NUM>, and a server <NUM>. The presence area identification area <NUM> of the present embodiment is a wireless local area network (LAN) system in which the HMD <NUM> and the server <NUM> transmit and receive data wirelessly therebetween through the access point <NUM>. Hereinafter, in the detailed description of the present invention, a space delimited by structures is referred to as a room.

In the present embodiment, each room <NUM> (<NUM>, <NUM>) is provided with an access point (AP) <NUM> (<NUM>, <NUM>) for wireless LAN connection. The server <NUM>, which manages various data necessary for AR display, identifies the room <NUM> based on the identification information on the access point <NUM>.

In the present embodiment, when a user (HMD wearer) wearing the head mounted display (HMD) <NUM> enters the room <NUM>, the HMD <NUM> is made connected to the access point (AP) <NUM>, which is a relay apparatus installed in advance in the room <NUM>, to perform authentication processing. Through the authentication processing, the HMD <NUM> obtains the identification information on the access point <NUM>. Then, the HMD <NUM> makes various kinds of information (display-related information) necessary for AR image display transmitted from the server <NUM> in association with the identification information on the access point <NUM>. This enables the HMD <NUM> to identify to which of the rooms <NUM> the information belongs.

On the other hand, the server <NUM> makes the display-related information on each room <NUM> in association with the identification information on the access point <NUM> installed in each room <NUM> and manages them. Based on the identification information on the access point <NUM> through which a request of data transmission from the HMD <NUM> has passed, the server <NUM> identifies an area where the HMD <NUM> is present (room <NUM>), and provides the HMD <NUM>, which is a transmission source, with the display-related information which has been prepared for the room <NUM>.

Hereinafter, in the present embodiment, spatial configuration information will be described as an example of the display-related information. The spatial configuration information is obtained by scanning the arrangement and shape of walls and installation objects of the room <NUM> and then mapping the scanned data. The spatial configuration information is used to grasp the shape of the inside of the room <NUM> surrounded by the walls and the arrangement and shape of furniture and the like installed in the room <NUM>. Using this spatial configuration information enables AR objects to be arranged in a realistic manner. According to the arrangement in a realistic manner, for example, an AR object is prevented from being arranged behind a wall.

Generally, the spatial configuration information is generated, each time when the wearer of the HMD <NUM> enters a predetermined space such as the room <NUM>, by scanning the space using various sensors provided in the HMD <NUM>. However, this causes a large processing load.

In the present embodiment, since the server <NUM> and the HMD <NUM> themselves can easily identify an area where the HMD <NUM> is present (the room <NUM>), if the spatial configuration information on each room <NUM> was created in the past, the previously created spatial configuration information is used.

Hereinafter, the present embodiment will be described by referring to an example in which the number of rooms is two (first room <NUM> and second room <NUM>). The first room <NUM> is provided with a first access point <NUM>, and the second room <NUM> is provided with a second access point <NUM>. Furthermore, the number of HMDs <NUM> is not necessarily one, but may be two, i.e., a first HMD <NUM> and a second HMD <NUM>. Hereinafter, the exemplary room <NUM> is referred to in the case where it is not necessary to distinguish the rooms. Similarly, the exemplary access point <NUM> and the exemplary HMD <NUM> are referred to, respectively, in the case where it is not necessary to distinguish the access points and the HMDs. Each access point <NUM> can transmit and receive data to and from the server <NUM> through a network <NUM>.

First, the HMD <NUM> according to the present embodiment will be described.

<FIG> illustrates an appearance of the HMD <NUM> of the present embodiment. The HMD <NUM> illustrated in <FIG> is a see-through HMD <NUM>. The HMD <NUM> includes a transmissive or semi-transmissive display <NUM>. The wearer wearing the HMD <NUM> can see a virtual object or video displayed on the display <NUM>, which is superimposed on the outside scenery.

Although not illustrated, the present embodiment is also applicable to an immersive HMD. The wearer wearing the immersive HMD does not directly see the surrounding scene, but sees a virtual world (virtual reality). Furthermore, the immersive HMD employs a video-through method, that is, uses data of an external field image captured by an out-camera <NUM> (see <FIG>) to see a video in which a virtual object is superimposed on the external field image. Although the present embodiment is applicable to any type described above, in the following, the see-through HMD <NUM> will be described as an example.

<FIG> is a hardware configuration diagram illustrating an exemplary internal configuration of the HMD <NUM>.

The HMD <NUM> includes a system bus <NUM>, a main processor <NUM>, a RAM <NUM>, a ROM <NUM>, a flash memory <NUM>, a GPS (Global Positioning System) receiver <NUM>, a geomagnetic sensor <NUM>, a range sensor <NUM>, an acceleration sensor <NUM>, a gyro sensor <NUM>, a timer <NUM>, a wireless communication IF <NUM>, a telephone network communication IF <NUM>, an in-camera <NUM>, the out-camera <NUM>, a display <NUM>, a microphone <NUM>, a speaker <NUM>, an audio decoder <NUM>, a button switch <NUM>, and a touch screen <NUM>. These components are connected to the system bus <NUM>.

The main processor <NUM> controls the entire operations of the HMD <NUM> in accordance with predetermined operation programs. The main processor <NUM> may be configured with any component such as a CPU, an MPU, or a dedicated logic circuitry.

The system bus <NUM> is a data communication channel that interconnects the main processor <NUM> with the respective components in the HMD <NUM>. The main processor <NUM> and the components in the HMD <NUM> transmit and receive various commands, data, and the like through the system bus <NUM>.

The RAM <NUM> defines a rewritable program work area, which is, for example, used by the main processor <NUM> at the time of execution of various programs.

Each of the ROM <NUM> and the flash memory <NUM> retains various programs for implementing the functions of the HMD <NUM>, operation setting values, sensor information including values detected by sensors which will be described later, and various data for display such as virtual objects and content. The ROM <NUM> and flash memory <NUM> are so-called nonvolatile storages that retain stored data even when power is not supplied to the HMD <NUM> from the outside.

The flash memory <NUM> retains operation programs downloaded from a network and various data created by the operation programs. Through download processing from each server device on the network, it is possible to update and enhance each operation program stored in the flash memory <NUM>.

Furthermore, the flash memory <NUM> can retain content such as moving images, still images, and sounds downloaded from a network. In addition, the flash memory <NUM> can retain data such as moving images and still images captured by the in-camera <NUM> or the out-camera <NUM>.

The RAM <NUM>, the ROM <NUM>, and the flash memory <NUM> are examples of storages. Other devices, for example, a semiconductor memory element such as a solid state drive (SSD) or a magnetic disk drive such a hard disc drive (HDD) may be used therefor.

The main processor <NUM> is configured to acquire sensor information from the GPS receiver <NUM>, the geomagnetic sensor <NUM>, the range sensor <NUM>, the acceleration sensor <NUM>, and the gyro sensor <NUM>, respectively, and also acquire a viewing time measured by the timer <NUM>. Then, the main processor <NUM> uses the sensor information and the viewing time to detect the position, tilt, direction, movement, and the like of the HMD <NUM>. Furthermore, the HMD <NUM> may further include other sensors, such as an illuminance sensor, a proximity sensor, an altitude sensor, and the like.

The wireless communication interface (I/F) <NUM> is connected to the network <NUM> such as the Internet through the access point <NUM> to transmit and receive data to and from each server <NUM> on the network <NUM>. In the present embodiment, millimeter waves (<NUM> band) are used for wireless LAN connection to the access point <NUM> and the like. This is because the millimeter waves are highly directional and the radio waves are easily absorbed by the wall of the room. Using millimeter waves enables the HMD <NUM> to be connected to the access point <NUM> that is installed in the room <NUM> where the HMD <NUM> is surely present via the wireless LAN. Note that Wi-Fi (registered trademark) or Bluetooth (registered trademark) may be used for the wireless communication IF <NUM>.

The telephone network communication I/F <NUM> is connected to a communication network through a base station (sometimes referred to as a cell tower) using a mobile communication network to transmit and receive data to and from a server on a communication network according to the third-generation mobile communication system (hereinafter, referred to as "<NUM>") such as Global System for Mobile Communications (GSM, registered trademark), Wideband Code Division Multiple Access (W-CDMA), CDMA <NUM>, or Universal Mobile Telecommunications System (UMTS). Similarly, the telephone network communication I/F <NUM> is connected to the communication network according to the Long-Term Evolution system (LTE), or a communication method such as so-called the fourth-generation system (<NUM>) or fifth-generation system (<NUM>).

Each of the wireless communication I/F <NUM> and telephone network communication I/F <NUM> includes an encoding circuitry, a decoding circuitry, an antenna, and the like.

Furthermore, the HMD <NUM> may include other communication interfaces such as an infrared communication I/F.

Each of the in-camera <NUM> and the out-camera <NUM> converts the light received through a lens into an electric signal using an electronic device such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) sensor to input image data of the surroundings and that of an object. Note that an exemplary camera <NUM> is referred to in the case where it is not particularly necessary to distinguish the in-camera <NUM> and out-camera <NUM>.

The display <NUM> is, for example, a display device such as a liquid crystal panel, and provides the wearer of the HMD <NUM> with image data. The HMD <NUM> includes a video RAM (not illustrated). A virtual object or a video is displayed on the screen of the display <NUM> based on the image data received by the video RAM.

The microphone <NUM> converts the voice of the wearer into sound data and inputs the sound data.

The speaker <NUM> outputs sound information and the like.

The audio decoder <NUM> performs decoding processing of an encoded sound signal as necessary.

Each of the button switch <NUM> and touch screen <NUM> is an operation device used to input an operation instruction to the HMD <NUM>. The operation device is not limited to the button switch <NUM> and the touch screen <NUM>. For example, an operation signal to the HMD <NUM> may be transmitted from a separate portable device (for example, a smartphone or a tablet device) connected thereto by wired communication or wireless communication. Upon receiving the operation signal, the HMD <NUM> is operated in accordance with the received operation signal. Alternatively, based on the sound received by the microphone <NUM>, the main processor <NUM> may execute sound recognition processing to generate an operation signal to control the operation of the HMD <NUM>.

A part of the exemplary configuration of the HMD <NUM> illustrated in <FIG> is not essential to the present embodiment. Accordingly, even if the configuration of the HMD <NUM> does not include such an unessential component, the advantageous effects of the present embodiment are not impaired. In this connection, the configuration of the HMD <NUM> may further include components which are not illustrated, for example, a digital broadcast reception function and an electronic money settlement function.

Next, a functional configuration of the HMD <NUM> according to the present embodiment will be described. <FIG> is a functional block diagram of the HMD <NUM> according to the present embodiment.

The HMD <NUM> of the present embodiment includes the functions of each of a main controller <NUM>, an authentication unit <NUM>, a data management unit <NUM>, a spatial configuration information acquisition unit <NUM>, an AR image processing unit <NUM>, and a communication unit <NUM>. The HMD <NUM> further includes an HMD-side spatial configuration information table (HMD-side library) <NUM> and an authentication information table <NUM>. The main processor <NUM> loads a program stored in the ROM <NUM> or the flash memory <NUM> onto the RAM <NUM> and executes it, whereby the functions mentioned above are implemented. The HMD-side library <NUM> is provided in the ROM <NUM> or the flash memory <NUM>.

The main controller <NUM> is configured to control the operations of each unit of the HMD <NUM>.

The authentication unit <NUM> is configured to access the access point <NUM> installed in each room <NUM> to perform authentication. In the present embodiment, when receiving the network name (name of the wireless LAN network) of the access point <NUM> and the information identifying the access point <NUM> (access point ID) through the wireless communication IF <NUM>, the authentication unit <NUM> transmits a password necessary for network connection to the access point <NUM>. The password is stored in advance in the authentication information table <NUM>. Upon receiving a notification that the authentication is successful from the access point <NUM>, the authentication unit <NUM> provides the other functions such as the main controller <NUM> with a notification to the effect that.

The authentication information table <NUM> retains passwords required in various authentication, the names of the devices, and the like. In the present embodiment, for example, the authentication information table <NUM> retains the passwords for authentication required to establish connection to the access point <NUM>.

The data management unit <NUM> is configured to manage the spatial configuration information and the like of the HMD-side library <NUM>. The spatial configuration information is acquired by the spatial configuration information acquisition unit <NUM>, which will be described later.

In the present embodiment, the data management unit <NUM> is configured to determine, firstly, the presence or absence of the spatial configuration information on the room <NUM> (hereinafter, referred to as the "own room") where the HMD <NUM> is currently present. The spatial configuration information on each room <NUM> is stored in the HMD-side library <NUM> in association with the access point ID provided to the room.

<FIG> illustrates an example of the HMD-side library <NUM> of the present embodiment. As illustrated in <FIG>, in the HMD-side library <NUM>, spatial configuration information 221b is registered in association with an access point ID 221a for each room <NUM>. In addition, other display-related information such as AR image information 221c may be registered in the HMD-side library <NUM>.

In the present embodiment, as the access point ID 221a, information identifying the access point <NUM> acquired in the authentication is used. For example, a MAC address of the access point <NUM> may be used as the access point ID 221a.

In this connection, for example, in the case where the spatial configuration information relating to the room <NUM> is not stored in association with the access point ID 221a, the data management unit <NUM> requests the server <NUM> to transmit the spatial configuration information on the room <NUM>. This request is made through the access point <NUM>.

When receiving the spatial configuration information from the server <NUM> through the access point <NUM>, the data management unit <NUM> stores, in the HMD-side library <NUM>, the received spatial configuration information 221b in association with the access point ID 221a. On the other hand, when receiving a reply that the spatial configuration information is not registered from the server <NUM>, the data management unit <NUM> instructs the spatial configuration information acquisition unit <NUM> to acquire the spatial configuration information.

The data management unit <NUM> obtains the spatial configuration information on the room <NUM> stored in the server <NUM>, and stores it in the HMD-side library <NUM>.

When receiving the instruction to acquire the spatial configuration information from the data management unit <NUM>, the spatial configuration information acquisition unit <NUM> creates the spatial configuration information on an area where the own device (HMD <NUM>) is currently present (room <NUM>). The spatial configuration information acquisition unit <NUM> registers the created spatial configuration information 221b in the HMD-side library <NUM>. At this time, the created spatial configuration information is registered in association with the access point ID 221a acquired by the authentication unit <NUM>.

In the present embodiment, the spatial configuration information acquisition unit <NUM> is configured to scan and capture images of the walls of the room <NUM>, arrangement and shape of the installation objects, and the like with the camera <NUM>. Then, the spatial configuration information acquisition unit <NUM> performs mapping with respect to the captured image data to acquire the spatial configuration information on the room <NUM>.

The AR image processing unit <NUM> is configured to generate an AR image using the AR image information 221c, and display the generated AR image on the display <NUM>. The AR image processing unit <NUM> displays the AR image when the display position of the generated AR image is within the visual field of the wearer of the HMD <NUM>. The HMD <NUM> itself determines whether the display position of the AR image is in within the visual field of the HMD <NUM> by determining the display position of the AR image.

For example, the AR image information 221c may be acquired from the server <NUM> and stored in the HMD-side library <NUM>. Alternatively, the server <NUM> may determine the display position of the AR image based on the visual filed information on the HMD <NUM> which has been transmitted to the server <NUM>, and transmit the necessary AR image information 221c to the HMD <NUM>.

The communication unit <NUM> is configured to transmit and receive data to and from an external device via the wireless communication IF <NUM>. In the present embodiment, the communication unit <NUM> transmits data to the access point <NUM> via the wireless communication IF <NUM> and receives data transmitted from the access point <NUM>.

The HMD <NUM> of the present embodiment may further include, for example, a captured image data storage unit for storing the image data captured by the camera <NUM>. All data other than authentication information may be stored in the outside, for example, a device such as a server connected via a network. In this case, the data is acquired from such a server via the wireless communication IF <NUM>.

The access point <NUM> is configured in the similar manner as a conventional access point. In the present embodiment, the access point <NUM> is a wireless device that includes a router function for connecting a wireless LAN client such as the HMD <NUM> to another network.

In the present embodiment, as described above, since the server <NUM> identifies in which of the rooms <NUM> the HMD <NUM> is present based on the information identifying the access point <NUM>, each room <NUM> is provided with at least one access point <NUM>. Furthermore, in order to prevent the HMD <NUM> from being connected to other access points <NUM> of other rooms <NUM>, in the present embodiment, for example, millimeter waves (<NUM>-GHz band) are used for the wireless LAN connection between the HMD <NUM> and the access point <NUM>.

Note that the wavelength of electromagnetic waves in use is not limited thereto. Electromagnetic waves of any wavelength band may be used as long as they are hardly connected to other access points <NUM> of other rooms and thus are reliably connected to the access point <NUM> of the own room.

Furthermore, preferably, installing each access point <NUM> near the center of each room <NUM> allows the HMD <NUM> to be connected to the access point <NUM> regardless of the position of the wearer of the HMD <NUM> within the room <NUM>.

Since each access point <NUM> is configured with a media access control (MAC) address, which is a unique physical address, the MAC address may be used as an access point ID identifying the access point <NUM>. Instead of a MAC address, an internet protocol (IP) address, which is a logical address, or a unique access point name can be used. Hereinafter, in the present embodiment, the case where a MAC address is used will be described as an example.

Next, the server <NUM> according to the present embodiment will be described. The server <NUM> of the present embodiment retains the information relating to AR display as the display-related information on each room <NUM> to be managed. Then, in response to a request from the HMD <NUM>, the sever <NUM> provides the display-related information on the room <NUM> where the HMD <NUM>, which is the request source, is present. This function is similar to that of the conventional AR server.

<FIG> illustrates an exemplary hardware configuration of the server <NUM> according to the present embodiment. As illustrated in <FIG>, in the similar manner as a general server, the server <NUM> includes a processor (main processor) <NUM>, a storage device (server storage device) including a RAM <NUM>, a ROM <NUM>, a flash memory <NUM>, and the like, and a communication interface (I/F) <NUM>. The sever <NUM> may further include a display <NUM>, an operation device <NUM>, and the like. These components are configured in the same manner as the components of the same names of the HMD <NUM>, and thus the repetitive explanation therefor will be omitted.

<FIG> is a functional block diagram of a part of the server <NUM> according to the present embodiment, which relates to the processing according to the present embodiment. As illustrated in <FIG>, the server <NUM> includes a spatial configuration information management unit <NUM> and server-side spatial configuration information (hereinafter, referred to as a server-side library) <NUM>.

The spatial configuration information management unit <NUM> is configured to manage data stored in the server-side library <NUM>. In the present embodiment, the spatial configuration information management unit <NUM> provides, in response to a request from the HMD <NUM>, the spatial configuration information on the room <NUM> where the access point <NUM> through which the request has passed is present. Before providing the spatial configuration information, the spatial configuration information management unit <NUM> checks whether the spatial configuration information is registered in the server-side library <NUM> in association with the access point ID (MAC address) of the access point <NUM> included in the request data. If it is registered, the spatial configuration information management unit <NUM> provides the registered spatial configuration information. If it is not registered, the spatial configuration information management unit <NUM> provides the HMD <NUM> with data indicating that it is not registered.

In response to the data indicating that it is not registered, in the case where the spatial configuration information is transmitted, the transmitted spatial configuration information is registered in the server-side library <NUM> in association with the access point ID of the relayed access point <NUM>.

<FIG> illustrates an example of the server-side library <NUM>. As illustrated in <FIG>, in the server-side library <NUM>, the display-related information (including spatial configuration information) of each room <NUM> is registered. The server-side library <NUM> includes, for each room <NUM>, a room number <NUM> used to identify a room, an access point ID <NUM>, spatial configuration information <NUM> of the room <NUM>, and information (AR image information) <NUM> on an augmented reality image (AR image) generated by a computer. The server-side library <NUM> may further include sound information relating to the room <NUM> and other information relating to the room <NUM>, and may not include the room number <NUM>.

In the server-side library <NUM> illustrated in <FIG>, the data of which the room number <NUM> is "room <NUM>" includes "Adrs01" as the access point ID <NUM>, "Map01" as the spatial configuration information <NUM>, "AR01" as the AR image information <NUM>, "Snd01" as the sound information, and "Othr01" as other information. Similarly, the data of each of the room <NUM> to room n includes the ones as illustrated in <FIG>.

In the present embodiment, the server <NUM> includes the server-side library <NUM> configured as described above. Thus, if the access point ID of the access point <NUM> is found, the room <NUM> is identified and all information about the room <NUM> is available.

In the server-side library <NUM>, basically, it is preferable to acquire the spatial configuration information <NUM>, AR image information <NUM>, sound information, and other information relating to each room <NUM> in advance and register them in association with the room number <NUM> and the access point ID <NUM>. In particular, it is preferable to generate the spatial configuration information <NUM> of each room <NUM> in advance by the general HMD <NUM> or the like and register it in the server-side library <NUM>.

On the other hand, in the present embodiment, in the case where no data of the access point ID <NUM> of the relayed access point <NUM> is registered, the transmitted spatial configuration information is newly registered. That is, in the present embodiment, the server-side library <NUM> can be replenished later.

The server-side library <NUM> is stored in a storage device of the server <NUM>.

Hereinafter, a flow of initial processing (spatial configuration information acquisition processing) performed when the HMD <NUM> enters the room <NUM> by the presence area identification system <NUM> according to the present embodiment, which is an information processing system capable of identifying an area where an HMD is present, will be described.

As described above, generally, each time a wearer of the HMD <NUM> enters a new room <NUM>, firstly, various sensors provided in the HMD <NUM> are used to collect the spatial configuration information on the room <NUM>. However, in the present embodiment, since an area where an HMD is present can be identified, the spatial configuration information created in the past can be managed in association with (the access point <NUM> provided in) the room <NUM>, and thus can be used later.

Hereinafter, the spatial configuration information acquisition processing according to the present embodiment will be described with reference to <FIG>.

This processing is started when the HMD <NUM> is activated. Alternatively, it is started in accordance with an instruction from the wearer. For example, when entering the predetermined room <NUM>, the wearer provides the HMD <NUM> with the instruction to perform the initial spatial configuration information acquisition processing. Further alternatively, the HMD <NUM> may be configured to detect the network name of the access point <NUM> and automatically initiate this processing.

Firstly, the authentication unit <NUM> performs the authentication processing in order to establish connection to the access point <NUM> of the wireless LAN (step S1101). Details of the authentication processing will be described later.

After the authentication processing is performed by the authentication unit <NUM> and the connection to the wireless LAN through the access point <NUM> is confirmed, the data management unit <NUM> determines whether the spatial configuration information on the room <NUM> is stored in the HMD <NUM> (step S1102).

If it is stored (step S1102; Yes), the data management unit <NUM> ends the processing as it is.

On the other hand, if it is not stored (step S1102; No), the data management unit <NUM> requests the server <NUM> to transmit the spatial configuration information on the room <NUM> through the access point <NUM> (step S1103). Then, the data management unit <NUM> distinguishes the content of a reply provided in response to the request (step S1104).

If the reply indicates that the spatial configuration information is available (step S1104; Yes), the data management unit <NUM> acquires (downloads) the spatial configuration information to be provided from the server <NUM> (step S1105), and registers the acquired spatial configuration information in the HMD-side library <NUM> (step S1108). At this time, the data management unit <NUM> registers the spatial configuration information 221b in association with the access point ID 221a of the access point <NUM> acquired by the authentication unit <NUM>, and ends the processing.

On the other hand, when the reply does not include the spatial configuration information (step S1104; No), the data management unit <NUM> instructs the spatial configuration information acquisition unit <NUM> to acquire the spatial configuration information.

In response to the instruction, the spatial configuration information acquisition unit <NUM> scans the room <NUM> (step S1106) to acquire the mapped spatial configuration information. Then, the spatial configuration information acquisition unit <NUM> transmits the acquired spatial configuration information to the server <NUM> (step S1107), and also registers it in the HMD-side library <NUM> in association with the access point ID of the access point <NUM> acquired by the authentication unit <NUM> (step S1108). At this time, the acquired spatial configuration information is transmitted to the server <NUM> through the access point <NUM>. Then, the spatial configuration information acquisition unit <NUM> ends the processing.

Next, transmission and reception of data between the devices during the authentication processing will be described. <FIG> is a time chart illustrating the transmission and reception of data between the first HMD <NUM> and first access point <NUM>, which are provided in the first room <NUM> of <FIG>, and the server <NUM>.

The time chart includes a total of three time-axes, i.e., a time axis T451 for the processing by the first HMD <NUM>, a time axis T452 for the processing by the first access point <NUM>, and a time axis T453 for the processing by the server <NUM>.

From each access point <NUM>, the network name of the access point <NUM> (network name of the wireless LAN) is constantly transmitted to devices including the HMD <NUM>, which are ready to be connected via the wireless LANs. Thus, the network name of the access point is also transmitted from the first access point <NUM> (step S1301).

The first HMD <NUM> detects the transmitted network names (step S1201), and selects, as a connection target, one of the network names desired by the wearer of the first HMD <NUM> from among the obtained network name group (step S1202). In the present embodiment, only one access point <NUM> is installed in each room <NUM>. Furthermore, millimeter waves are used for the communication between the HMD <NUM> and the access point <NUM> since using the millimeter waves prevents the HMD <NUM> from being connected to the access points other than the access point <NUM> of the room <NUM>. Thus, in the present embodiment, the first HMD <NUM> can detect only the network name transmitted from the first access point <NUM> installed in the first room <NUM>, and select the detected network name as the connection target.

Subsequently, the first HMD <NUM> transmits an authentication request to the first access point <NUM>. Here, the first HMD <NUM> notifies the first access point <NUM> of a password required for the network connection (step S1203). The authentication unit <NUM> acquires the password from the authentication information table <NUM> and transmits it to the first access point <NUM>.

The first access point <NUM> authenticates the received password (step S1302). The first access point <NUM> verifies, for each connectable network, whether the received password matches the password necessary for connection with the network, which is registered in advance. If it matches the registered password, the authentication is successful. If successful, the connection between the first HMD <NUM> and the first access point <NUM> is established via the wireless LAN. The first access point <NUM> transmits a signal for permitting the authentication (authentication OK) to the first HMD <NUM> (step S1303).

Since the authentication processing between the first HMD <NUM> and the first access point <NUM> is performed in the same manner as general wireless LAN connection processing, detailed explanation thereof will be omitted here.

In order to enhance the security of the network, not only password authentication but also identity document (ID) authentication may be performed. Furthermore, an authentication server that authenticates both an ID and password may be additionally provided so that password authentication may be performed by the authentication server.

After the connection is established between the first HMD <NUM> and the AP <NUM>, that is, if the authentication is successful in step S1302, the first access point <NUM> transmits the identification information identifying the first access point <NUM> to the server <NUM> (step S1304). In the present embodiment, the MAC address, which is a physical address unique to the device, is used as the identification information. If the password authentication fails in step S1302, the process in step S1304 is not performed.

In the server <NUM>, the spatial configuration information management unit <NUM> verifies whether the received MAC address is registered in the server-side library <NUM>.

If the MAC address is not registered, the spatial configuration information management unit <NUM> determines that the received MAC address is a new MAC address (new room) and registers it as the access point ID <NUM> in the server-side library <NUM> (step S1401). Thus, the MAC address of the first access point <NUM> installed in the first room <NUM> where the first HMD <NUM> is present is registered in the server <NUM>. The server <NUM> recognizes that the first HMD <NUM> is present in the first room <NUM> in which the first access point <NUM> having the newly registered MAC address is installed.

If it is determined in step S1401 that the MAC address is registered, the server <NUM> recognizes that the first HMD <NUM> is present in the first room <NUM> in which the first access point <NUM> having the corresponding MAC address is installed.

Thereafter, when the first HMD <NUM> requests transmission of the spatial configuration information through the first access point <NUM>, the spatial configuration information management unit <NUM> confirms the presence or absence of the corresponding spatial configuration information. The confirmation is made based on whether the spatial configuration information <NUM> is registered in the server-side library <NUM> in association with the MAC address of the first access point <NUM> which is the transmission source.

If the spatial configuration information <NUM> is registered, the spatial configuration information management unit <NUM> transmits the registered spatial configuration information <NUM> to the transmission source. On the other hand, if it is unregistered, the spatial configuration information management unit <NUM> provides the transmission source with a notification to the effect that.

After transmitting the notification that the spatial configuration information <NUM> is not registered, if receiving the spatial configuration information through the first access point <NUM>, the spatial configuration information management unit <NUM> registers the received spatial configuration information <NUM> in the server-side library <NUM> in association with the MAC address of the first access point <NUM>.

As described above, according to the present embodiment, when the wearer enters the room <NUM>, the HMD <NUM> is authenticated by the access point <NUM>, which is installed in the room, to be connected thereto, and then makes the room associated with the access point. After providing the HMD <NUM> with the connection authentication, the access point <NUM> transmits a MAC address to the server <NUM> as the identification information on the access point <NUM>. Upon receiving the MAC address, the server <NUM> recognizes that the HMD <NUM> is present in the room <NUM> registered in association with the MAC address. The MAC address and the room <NUM> are made associated with each other and stored in advance in the server-side library <NUM> by the server <NUM>.

As described above, in the present embodiment, the server <NUM> uses the identification information on the access point used by the wireless LAN to identify an area where the HMD <NUM> is present, i.e., the room <NUM> where the HMD <NUM> is present. As a result, an area where the HMD <NUM> is present can be easily grasped only with the existing devices. That is, it is possible to identify an area where the HMD <NUM> is present without providing additional hardware.

Furthermore, according to the present embodiment, the HMD <NUM> can provide the server <NUM> with the information necessary for identifying an area where the HMD <NUM> is present simply by performing the general authentication processing with the access point <NUM>. That is, the HMD <NUM> enables the servers <NUM> to grasp the room <NUM> where the own device is present without requesting a new instruction or the like from the wearer.

Still further, according to the present embodiment, information for uniquely identifying the access point <NUM> installed in each room <NUM> is used to identify an area where the HMD <NUM> is present. Accordingly, although, for example, the GPS has a restriction of location identification in the height direction, there is no such a restriction in the present embodiment. That is, in the case where an HMD is present in a building or the like of a plurality of stories, it is difficult for the GPS to distinguish the positions on the same plane even if they are of the different stories. On the other hand, according to the present embodiment, even in such a case, it is possible to accurately identify the area where the HMD <NUM> is present.

As described above, according to the present embodiment, since the server <NUM> can easily identify an area where the HMD <NUM> is present with sufficient accuracy, it is possible to appropriately provide the HMD <NUM> with various kinds of display-related information that was created in the past and is required for the AR display simply by managing the display-related information for each area where the HMD is to be present. Furthermore, the HMD <NUM> can appropriately receive the display-related information managed by the server <NUM>.

Conventionally, the spatial configuration information needs to be generated each time the wearer of the HMD <NUM> moves between the rooms <NUM>, which causes a burden to the processing of the HMD <NUM>. On the other hand, according to the present embodiment, since the server <NUM> can easily grasp the room <NUM> where the HMD <NUM> has entered, in the case where the server <NUM> manages the spatial configuration information created in the past, the spatial configuration information can be provided from the server <NUM>. Therefore, according to the present embodiment, it is possible to greatly reduce the processing burden which may be caused when the HMD <NUM> enters a room.

In the embodiment described above, the access point <NUM> and the wireless LAN are used to identify the room <NUM> where the HMD <NUM> is present. However, a relay apparatus used to identify the room <NUM> is not limited to the access point <NUM>. For example, a base station of mobile phones may be used.

Hereinafter, a method of identifying the room <NUM> where the HMD <NUM> is present by using a base station of mobile phones as a relay apparatus will be described. <FIG> illustrates an outline of the present modification.

A presence area identification system 100a according to the present modification is basically configured in the same manner as the embodiment described above, while the difference therebetween is found in that the presence area identification system 100a uses a base station (hereinafter, referred to as a "BS") <NUM> of mobile phones instead of the access point <NUM> which is used as a relay apparatus in the embodiment described above. <FIG> illustrates an example in which a first BS <NUM> is installed in the first room <NUM>, and a second BS <NUM> is installed in the second room <NUM>.

Thus, the network to which the BS <NUM> is connected is a mobile phone network <NUM> rather than the network <NUM> such as the Internet. In order to establish the connection to the server <NUM> through the mobile phone network <NUM>, a mobile management <NUM> is provided between the BS <NUM> and the server <NUM>.

The mobile management <NUM> is located at the core of the mobile phone network, and configured to manage the mobile phone network together with a home subscriber server <NUM> that retains subscriber information. The BS <NUM> provided in each room <NUM> is connected to the mobile management <NUM> via the mobile phone network <NUM>. Then, each BS <NUM> is connected to the network <NUM> through the mobile management <NUM> to access the server <NUM>.

Note that the mobile management <NUM> is connected to other mobile phone networks <NUM> which are connected to a plurality of mobile phone base stations (not illustrated).

In the present modification, the BS <NUM> is wirelessly connected to the HMD <NUM>, which is as a mobile phone, using millimeter waves (<NUM> band) which are the <NUM> standard of mobile phones. The millimeter waves are highly directional and the radio waves are easily absorbed by the walls of the room <NUM>. Since using the millimeter waves surely enables the HMD <NUM> which is present in the room <NUM> to be wirelessly connected to the BS <NUM> that is installed in the room <NUM>, the HMD <NUM> is prevented from being connected to other BSs <NUM> installed in other rooms <NUM>.

Each BS <NUM> is provided with a unique BS-ID (sometimes referred to as Cell-ID). Accordingly, when the correspondence between the room <NUM> and the BS-ID of the BS <NUM> installed in the room <NUM> is found, the room <NUM> can be identified based on the BS-ID of the BS <NUM>.

Hereinafter, the present modification will be described while focusing on the points different from the first embodiment.

The HMD <NUM> is basically configured in the same manner as that of the first embodiment. On the other hand, as illustrated in <FIG>, the spatial configuration information (HMD-side library <NUM>) of each room <NUM> held by the HMD <NUM> is stored in association with a BS-ID 221d of the BS <NUM> installed in each room <NUM>.

Similarly, the server <NUM> is basically configured in the same manner as that of the embodiment described above, while the difference therebetween is found in that, as illustrated in <FIG>, the server-side library <NUM> is managed in association with the BS-ID 732b instead of the MAC address of the access point <NUM>.

Note that, there may be cases where some rooms <NUM> are provided with the access points <NUM> while other rooms are provided with the BSs <NUM>. In this case, the spatial configuration information is stored for each room <NUM> in association with the identification information on a relay apparatus installed in each room <NUM>.

The spatial configuration information acquisition processing by the HMD <NUM> is also basically the same as that by the embodiment described above, while in the authentication processing, instead of the authentication processing to establish the connection to the access point <NUM>, processing to establish the connection to the BS <NUM> is performed.

Hereinafter, transmission and reception of data between the devices in the authentication processing according to the present modification will be described.

<FIG> is a time chart of the authentication processing according to the present modification. Here, the present embodiment and transmission and reception of data between the first BS <NUM>, the home subscriber server <NUM>, and the server <NUM> will be described. <FIG> includes a total of four time-axes, i.e., a time axis T421 for the processing by the first HMD <NUM>, a time axis T422 for the processing by a BS <NUM>, a time axis T423 for the processing by the home subscriber server <NUM>, and a time axis T424 for the processing by the server <NUM>.

Generally, each BS <NUM> always notifies mobile phone devices (HMDs <NUM>) that are connectable thereto of a mobile network code and a BS-ID. In the present modification, similarly, the first BS <NUM> notifies the first HMD <NUM> of the mobile network code and BS-ID (step S1601).

The first HMD <NUM> detects the notified mobile network code and selects a desired mobile carrier (step S1501). The first HMD <NUM> uses the corresponding mobile phone carrier information stored in a SIM card built in the first HMD <NUM> to select the BS <NUM> provided by the corresponding mobile carrier. The first HMD <NUM> holds the mobile network code and BS-ID of the selected BS <NUM>.

The first HMD <NUM> stores the identification number of the first HMD <NUM>, which is being used as a mobile phone, in the SIM card built in the own device. The first HMD <NUM> transmits the stored identification number to the home subscriber server <NUM> through the BS <NUM> (step S1502).

The home subscriber server <NUM> uses the identification number of the first HMD <NUM> received in step S1502 in the user authentication (step S1701). The home subscriber server <NUM> authenticates a user by comparing the identification number with the subscriber information registered in advance in the home subscriber server <NUM> and determining whether they match. If the received identification number matches the subscriber information, the authentication is successful, and thus the home subscriber server <NUM> transmits a signal indicating that the authentication is successful (authentication OK) to the first HMD <NUM> (step S1702).

Since the authentication processing between the first HMD <NUM> and the first BS <NUM> described above is performed in the same manner as the general mobile phone connection processing, detailed explanation thereof will be omitted here.

Furthermore, when the authentication is successful in step S1701, the first HMD <NUM> transmits, together with the received BS-ID, a request of transmission of the spatial configuration information to the server <NUM> (step S1503).

In response to this request, in order to confirm whether the spatial configuration information may be transmitted to the first HMD <NUM> that is the request source, the server <NUM> requests the first HMD <NUM> to transmit a password (step S1801). In response to this request, the first HMD <NUM> transmits the password to the server <NUM> (step S1504). The password for this authentication is registered in advance in the server <NUM>.

The server <NUM> authenticates the received password (step S1802). If the received password matches the password for authentication, the server <NUM> transmits a signal indicating that the authentication is successful (authentication OK) to the first HMD <NUM> (step S1803).

After the authentication by the server <NUM> is successful, in the same manner as the first embodiment, the first HMD <NUM> acquires the spatial configuration information on the first room <NUM> where the own device is present from the server <NUM> as needed.

As described above, according to the present modification, using a base station of a mobile phone network used by mobile phones enable the server <NUM> to identify an area where an HMD is present (room <NUM>). As a result, like the embodiment described above, an area where the HMD <NUM> is present can be easily grasped only with the existing devices. That is, it is possible to identify an area where the HMD <NUM> is present without providing additional hardware.

Furthermore, according to the present modification, like the embodiment described above, the HMD <NUM> simply performs the general authentication processing for establishing communication, whereby the server <NUM> can grasp an area where the HMD <NUM> is present. That is, the HMD <NUM> can provide the server <NUM> with the information necessary for identifying an area where the own device is present without requesting the wearer to provide a new instruction or the like.

Thus, according to the present modification, it is possible to obtain the same advantageous effects as those of the embodiment above.

In the first modification, using millimeter waves which have a high directivity and attenuation rate is assumed, however, the present invention is not limited thereto.

Recently, a mobile phone network designed with a femtocell, which is a low power mobile phone base station, has been provided. Since a coverage area of the femtocell is much narrower than that of a conventional cell, it is very likely that such a mobile phone base station in the room <NUM> will receive an output from a device in the room <NUM>. Therefore, a mobile phone base station designed with the femtocell may be used.

In the embodiment above, the access point <NUM> is connected to the server <NUM> via the wireless LAN and accesses the server <NUM> through the network <NUM> such as an Internet network, however, the present invention is not limited thereto. For example, as illustrated in <FIG>, instead of the network <NUM>, a wired LAN network <NUM> may be used. In this case, instead of the access point <NUM>, a relay apparatus such as a hub (HUB) <NUM> (<NUM>, <NUM>) is used.

Using the wired LAN will give the advantageous effects of reducing the risk of information leakage through the Internet network, in addition to the advantageous effects which can be obtained by the embodiment and modifications above.

In the present embodiment, a plurality of access points <NUM> is provided in one room <NUM>. Some of the access points <NUM> are different from others in their performance.

Hereinafter, the present embodiment will be described while focusing on the points different from the first embodiment.

<FIG> illustrates an outline of an operating environment of the presence area identification system <NUM> according to the present embodiment. In order to simply explain, an example of the presence area identification system <NUM> including, as the plurality of access points <NUM>, a toral of two systems, i.e., an access point <NUM> and an access point <NUM> will be described. The presence area identification system <NUM> may be configured to be connectable to the server <NUM> via three or more systems of wireless LANs of different transmission standards.

<FIG> illustrates a table <NUM> showing the difference in performance between the access point <NUM> and the access point <NUM>. The access point <NUM> and the access point <NUM> are different from each other in performance, such as connection speed and the presence or absence of limit of data volume. This information may be held in, for example, the HMD <NUM> or the server <NUM>. Note that they may have a difference in the amount of money to be charged.

As illustrated in <FIG>, in the case of the access point <NUM>, the connection speed is medium, data volume is limited, and advertisements are included. On the other hand, in the case of the access point <NUM>, the connection speed is high, data volume is not limited, and no advertisements are included.

Thus, for example, in the case of transmitting and receiving the spatial configuration information or AR image information, using the access point <NUM> as a relay apparatus enables transmission and reception of more detailed data in a short time.

Furthermore, in the present embodiment, the server <NUM> determines that the HMD <NUM> is present in the room <NUM> when the HMD <NUM> is connected thereto via any of the access points <NUM> installed in the room <NUM>. On the other hand, the server <NUM> may hold, as the server-side library <NUM>, the spatial configuration information and AR image information having different resolutions for each access point <NUM> having different performance.

<FIG> illustrates an example of the server-side library <NUM> in this case. As illustrated in <FIG>, information is held for each access point <NUM> installed in each room <NUM>. Specifically, the server-side library <NUM> includes the access point ID of each access point <NUM>, the spatial configuration information <NUM>, the information indicating the resolution of the spatial configuration information <NUM> (hereinafter, simply referred to as resolution) <NUM>, the AR image information <NUM>, and a resolution <NUM> of the AR image information <NUM>. In the following, an example in which the quality (resolution) of the AR image information to be provided is changed in accordance with the performance of each access point <NUM> will be described, however, the quality of the AR image information may be one type.

The spatial configuration information management unit <NUM> of the server <NUM> transmits the spatial configuration information stored in the server-side library <NUM> in accordance with the relayed access point <NUM>.

Thus, if requesting the spatial configuration information through the access point <NUM> with higher performance, the HMD <NUM> can acquire the spatial configuration information with higher resolution and higher quality.

Note that which of the access points <NUM> to be used to request the server <NUM> to transmit the spatial configuration information may be selected by the wearer of the HMD <NUM>, or may be determined in advance in the HMD <NUM>.

In the case where it is to be selected by the wearer and if a listing of the access points <NUM> is detected, the HMD <NUM> displays the list on the display <NUM> and accepts the selection by the wearer.

In the case where it is determined in advance, a selection rule is set in advance and registered in the ROM <NUM>, the flash memory <NUM>, or the like. The selection rule may be, for example, selecting the one with the highest performance among from the charge-free access points <NUM>, selecting the one with the highest performance regardless of whether it is charge-free or with charge, or the like. In accordance with the selection rule, the authentication unit <NUM> selects the access point <NUM> to be used as a relay apparatus.

According to the present embodiment, as in the first embodiment, the server <NUM> uses the access point ID of the access point <NUM> installed in each room <NUM> to recognize an area where the HMD <NUM> is present. Therefore, the same advantageous effects as those of the first embodiment can be obtained.

Furthermore, according to the present embodiment, in the case where the access points <NUM> having different performance are installed in one room <NUM>, the server <NUM> manages them separately. Then, the server <NUM> provides the display-related information in accordance with the performance of the access point <NUM> with the connection authentication with the HMD <NUM> has been established.

As described above, according to the present embodiment, the server <NUM> can easily identify an area where the HMD <NUM> is present with enough accuracy, and also can grasp the data quality desired by the HMD <NUM>. This enables the server <NUM> to appropriately provide the HMD <NUM> with various types of display-related information required for AR display, which were created in the past, only by managing the display-related information for each presence area and for each quality. Furthermore, the HMD <NUM> can appropriately receive the display-related information managed by the server <NUM>.

Next, a third embodiment of the present invention will be described. In the present embodiment, the spatial configuration information held by the server <NUM> includes information on the resolution thereof. Then, in accordance with the performance of the HMD <NUM>, the spatial configuration information of the server <NUM> is overwritten.

The HMD <NUM> according to the present embodiment has basically the same configuration as that of the first embodiment, while the difference is found in that, when acquiring the spatial configuration information, the spatial configuration information acquisition unit <NUM> according to the present embodiment also identifies the resolution thereof.

As illustrated in <FIG>, in the HMD-side library <NUM> of the present embodiment, a resolution 221e of the spatial configuration information 221b is registered in association with the access point ID 221a.

The data management unit <NUM> of the present embodiment is basically configured to perform the same processing as that of the first embodiment. Furthermore, the data management unit <NUM> of the present embodiment grasps in advance the best resolution of the spatial configuration information that can be acquired by the spatial configuration information acquisition unit <NUM>. The performance of the HMD <NUM> is determined, for example, based on the spatial resolution of the sensors. Then, the data management unit <NUM> compares the grasped resolution with the resolution of the spatial configuration information on the own room held in the server-side library, and determines from which of the devices, i.e., the server <NUM> or the own device, the spatial configuration information is to be acquired.

That is, when acquiring the spatial configuration information from the server <NUM>, the data management unit <NUM> of the present embodiment compares the resolution of the acquired spatial configuration information with the resolution (best resolution) of the spatial configuration information that can be generated by the own device. If the best resolution is higher than that of the acquired spatial configuration information, the data management unit <NUM> causes the spatial configuration information acquisition unit <NUM> to generate the spatial configuration information by the own device. Then, the data management unit <NUM> transmits the generated spatial configuration information to the server <NUM> together with the information indicating the resolution.

The server <NUM> of the present embodiment is also basically configured in the same manner as that of the first embodiment, while the difference therebetween is found in that the server-side library <NUM> of the present embodiment includes the information on the resolution of spatial configuration information.

<FIG> illustrates an example of the server-side library <NUM> of the present embodiment. As illustrated in <FIG>, the server-side library <NUM> includes the resolution <NUM> of the spatial configuration information and the resolution <NUM> of the AR image information in addition to the information held by the server-side library <NUM> of the first embodiment.

The information on the best resolution of the spatial configuration information that can be acquired by the spatial configuration information acquisition unit <NUM> of the own device is stored in advance.

Upon receiving the spatial configuration information, the spatial configuration information management unit <NUM> of the server <NUM> determines whether to register the received spatial configuration information in the server-side library <NUM> in association with the access point ID of the access point <NUM> through which the transmission source has passed.

In the present embodiment, the spatial configuration information management unit <NUM> registers the received spatial configuration information in the case where it has not been registered. On the other hand, in the case where the spatial configuration information has already registered in association with the access point ID, the spatial configuration information management unit <NUM> compares the resolution of the registered spatial configuration information with the resolution of the received spatial configuration information. If the resolution of the received spatial configuration information is higher, the spatial configuration information management unit <NUM> replaces the existing spatial configuration information with the received spatial configuration information. The information on the existing resolution is also replaced with the information on the resolution of the received spatial configuration information.

Thus, the server-side library <NUM> of the present embodiment retains the spatial configuration information having the best resolution at that time.

Hereinafter, a processing flow of the spatial configuration information acquisition processing according to the present embodiment will be described. <FIG> illustrates a processing flow of the spatial configuration information acquisition processing by the HMD <NUM> according to the present embodiment. This processing is started at the same timing as that of the first embodiment. After the processing is started, firstly, the authentication unit <NUM> performs the authentication processing (step S1101). This processing is the same as that of the first embodiment.

As in the first embodiment, the data management unit <NUM> determines whether the spatial configuration information on the room <NUM> is held in the HMD <NUM> (step S1102).

If it is held (step S1102; Yes), the data management unit <NUM> ends the processing as it is.

On the other hand, if it is not held (step S1102; No), the data management unit <NUM> requests the server <NUM> to transmit the spatial configuration information through the access point <NUM> (step S1103). Then, the data management unit <NUM> distinguishes the content of the reply provided in response to the request (step S1104).

If the reply indicates the spatial configuration information is available (step S1104; Yes), the data management unit <NUM> downloads the spatial configuration information to be provided from the server <NUM> (step S1105).

Then, the data management unit <NUM> of the present embodiment compares the resolution of the downloaded spatial configuration information with the best resolution of the spatial configuration information that can be acquired by the spatial configuration information acquisition unit <NUM> of the own device (step S4101). If the resolution of the spatial configuration information of the own device is equal to or less than the resolution of the spatial configuration information held by the server <NUM> (step S4101; No), as in the first embodiment, the data management unit <NUM> registers the received spatial configuration information in the HMD-side library <NUM> (step S1108), and ends the processing. At this time, the data management unit <NUM> registers the spatial configuration information in association with the access point ID of the access point <NUM> which was acquired by the authentication unit <NUM>.

On the other hand, when the reply does not include the spatial configuration information (step S1104; No) and the best resolution of the spatial configuration information of the own device is higher than that of the spatial configuration information held by the server <NUM> (step <NUM>; Yes), the data management unit <NUM> instructs the spatial configuration information acquisition unit <NUM> to acquire the spatial configuration information.

In response to the instruction, the spatial configuration information acquisition unit <NUM> scans the room <NUM> (step S1106) to acquire the spatial configuration information. Then, the spatial configuration information acquisition unit <NUM> transmits the acquired spatial configuration information to the server <NUM> (step S1107), and also registers it in the HMD-side library <NUM> in association with the access point ID of the access point <NUM> acquired by the authentication unit <NUM> (step S1108). At this time, the acquired spatial configuration information is transmitted to the server <NUM> through the access point <NUM>. Then, the spatial configuration information acquisition unit <NUM> ends the processing.

Next, the spatial configuration information management processing on the server side according to the present embodiment will be described. <FIG> illustrates a processing flow of the spatial configuration information management processing by the spatial configuration information management unit <NUM> of the server <NUM> according to the present embodiment. The spatial configuration information management unit <NUM> starts the processing upon receiving the spatial configuration information from the HMD <NUM>.

The spatial configuration information management unit <NUM> determines whether the spatial configuration information on the room <NUM>, which was identified based on the received spatial configuration information, has already been registered in the server-side library <NUM> (step S4201). Specifically, the spatial configuration information management unit <NUM> determines whether the spatial configuration information has already been registered in association with the access point ID of the relayed access point <NUM>.

If it is not registered (step S4201; No), the spatial configuration information management unit <NUM> registers the received spatial configuration information in association with the access point ID of the relayed access point <NUM> (step S4203). At this time, the information on the resolution is also registered.

On the other hand, if the spatial configuration information has already been registered (step S4201; Yes), the spatial configuration information management unit <NUM> compares the resolution of the corresponding spatial configuration information in the library with the resolution of the received spatial configuration information (step S4202).

If the resolution of the received spatial configuration information is higher than the resolution of the spatial configuration information in the library (step S4202; Yes), the spatial configuration information management unit <NUM> replaces the spatial configuration information in the library with the received spatial configuration information (step S4203), and ends the processing. At this time, the information on the resolution is also replaced with the information on the resolution of the received spatial configuration information.

Otherwise (step S4202; No), the spatial configuration information management unit <NUM> discards the received spatial configuration and ends the processing.

As described above, in the present embodiment, the display-related information (spatial configuration information) is held together with the information on the resolution thereof in the server-side library <NUM>. Then, in the case where the HMD <NUM> can acquire the information having higher resolution, the held information is replaced with the display-related information (space configuration information) acquired by the HMD <NUM>.

Thus, according to the present embodiment, the server <NUM> retains the display-related information (spatial configuration information) having high resolution provided by the HMD <NUM> having the highest performance among the HMDs <NUM> that may be used. This allows the HMD <NUM> to use the spatial configuration information with high resolution.

Therefore, according to the present embodiment, the HMD <NUM> can use the spatial configuration information with high resolution regardless of the spatial resolution of the sensors of the own device.

In the above embodiment, in the case where the resolution of the received spatial configuration information is high, the server <NUM> replaces the registered spatial configuration information therewith. However, the present invention is not limited thereto. For example, the server may hold a plurality of pieces of spatial configuration information having different resolutions.

In such a case, when receiving the spatial configuration information, the HMD <NUM> can designate the resolution thereof.

Next, a fourth embodiment of the present invention will be described. In the embodiments and modifications described above, the HMD <NUM> is always connected to a relay apparatus installed in each room <NUM>, and the server <NUM> uses the identification information on the relay apparatus to determine an area where the HMD <NUM> is present. However, in practical communication environments, the HMD <NUM> is not always connected to a relay apparatus installed in the room <NUM> where the wearer is present.

In the present embodiment, a method which enables identification of an area where the HMD is present even in such environments is provided. In the present embodiment, the server-side library <NUM> in which the room <NUM>, the access point ID of the access point installed in the room <NUM>, and the spatial configuration information on the room <NUM> are associated with each other is created in advance and held by the server <NUM>.

<FIG> illustrates an outline of the present embodiment.

Generally, the electric field intensity of the wireless LAN generated from the first access point <NUM> in the first room <NUM> where the HMD <NUM> is present is more than the electric field intensity of the wireless LAN generated from the second access point <NUM> in the other second room <NUM>. Thus, the first access point <NUM> in the first room <NUM> is automatically selected as an access point to which the HMD <NUM> is to be connected.

However, the HMD <NUM> is not always connected to the AP <NUM> of the room <NUM> where the HMD <NUM> is present. For example, depending on the relation between the installation position of the access point <NUM> and the location of the HMD <NUM>, even when the wearer of the HMD <NUM> is in the first room <NUM>, the HMD <NUM> may be connected to the AP <NUM> installed in the adjacent second room <NUM>. In the present embodiment, even in such a case, the server <NUM> can appropriately identify the room <NUM> where the HMD <NUM> is present.

The hardware configuration of the HMD <NUM> according to the present embodiment is the same as that of the first embodiment. On the other hand, the functional aspect of the HMD <NUM> according to the present embodiment is different from that of the first embodiment.

<FIG> is a functional block diagram of the HMD <NUM> according to the present embodiment. The HMD <NUM> further includes an image acquisition unit <NUM> in addition to the units of the first embodiment. The image acquisition unit <NUM> is configured to acquire (capture) an image as visual information using the camera <NUM> in accordance with an instruction from the wearer. The image to be acquired may be a still image or a moving image.

The spatial configuration information acquisition unit <NUM> of the present embodiment requests the server <NUM> to transmit the spatial configuration information when no spatial configuration information on the own room is available in the HMD-side library <NUM>. At this time, in the present embodiment, the spatial configuration information acquisition unit <NUM> further transmits an image in which the own room is captured as the visual field information.

Next, the server <NUM> will be described. The server <NUM> according to the present embodiment basically has the same configuration as the first embodiment, while the difference therebetween is found in the processing by the spatial configuration information management unit <NUM> and the data held by the server-side library <NUM>.

Firstly, as illustrated in <FIG>, the server-side library <NUM> of the present embodiment further retains the location information on each room <NUM> or each access point <NUM>. Here, for example, an example in which the information on a predetermined position (for example, the southwest corner) of the room <NUM> is held will be described. The location information may be, for example, latitude and longitude information or information based on a coordinate system defined in the system. In the case of a polygonal shaped room, latitude and longitude information on each corner of the room may be held.

The spatial configuration information management unit <NUM> receives the image as the visual field information together with the spatial configuration information request. Then, the spatial configuration information management unit <NUM> determines whether the spatial configuration information on the room <NUM>, which could be identified based on the simultaneously received access point ID, includes an area that matches the visual field information. If the spatial configuration information on the room <NUM> does not include such an area, the spatial configuration information management unit <NUM> sequentially determines, in order of proximity to the room <NUM>, whether each spatial configuration information on each of the other rooms <NUM> includes the area that matches the visual field information. Then, the spatial configuration information management unit <NUM> transmits the spatial configuration information on the room including the area that matches the visual field information to the transmission source. In the case where no spatial configuration information on each room <NUM> in a predetermined range includes the area that matches the visual field information, the spatial configuration information management unit <NUM> replies that any spatial configuration information that matches the visual field information is not registered.

The spatial configuration information management unit <NUM> registers, in the server-side library <NUM>, the spatial configuration information, which was transmitted from the HMD <NUM> in response to the reply, in association with the access point ID of the relayed access point <NUM>.

The flow of the spatial configuration information acquisition processing and spatial configuration information management processing according to the present embodiment will be described. The spatial configuration information acquisition processing and authentication processing on the HMD <NUM> side is the same as that of the first embodiment. Here, the spatial configuration information management processing by the server <NUM> will be described. <FIG> is a flowchart of the processing flow on the server <NUM> side according to the present embodiment. The server <NUM> starts the processing upon receiving the visual field information together with the request to transmit the spatial configuration information from the HMD <NUM>.

The spatial configuration information management unit <NUM> determines whether the spatial configuration information <NUM> (spatial configuration information on the room) stored in the server-side library <NUM> in association with the access point ID of the transmission source includes an area that matches the visual field information (step S2101).

If the spatial configuration information <NUM> includes such an area (step S2101; Yes), the spatial configuration information management unit <NUM> transmits the spatial configuration information to the transmission source (step S2102), and ends the processing.

If the spatial configuration information <NUM> does not include the area that matches the visual field information (step S2101; No), the spatial configuration information management unit <NUM> searches for the room <NUM> including the area that matches the visual field information (step S2103). Here, firstly, the spatial configuration information management unit <NUM> uses the location information on the room <NUM> with which the access point ID of the transmission source is associated and the location information on another room <NUM> to calculate the distance therebetween, and performs the same determination processing as that in step S2101 as described above with respect to each room <NUM> in order from the nearest one. Note that an upper limit M (M is an integer of <NUM> or more) of the number of rooms to be determined is set in advance.

Based on the search result obtained in step S2103, the spatial configuration information management unit <NUM> determines whether the spatial configuration information including the area that matches the visual field information is found (step S2104). If it is found (step S2104; Yes), the spatial configuration information management unit <NUM> transmits the spatial configuration information to the HMD <NUM> which is the transmission source (step S2102), and ends the processing. If it is not found (step S2104; No), the spatial configuration information management unit <NUM> transmits a reply to that effect (step S2105), and ends the processing.

In the present embodiment, when receiving the reply from the server <NUM> that the space configuration information on the room <NUM> is not stored on the server side, as in the first embodiment, the HMD <NUM> executes mapping of the room <NUM> by the own device to generate the space configuration information. Then, the generated spatial configuration information is transmitted to the server <NUM> and registered therein. At this time, in the present embodiment, the access point <NUM> installed in the room <NUM> is intentionally selected and transmitted.

Upon receiving the spatial configuration information from the HMD <NUM>, the spatial configuration information management unit <NUM> registers the spatial configuration information in the server-side library <NUM> in association with the access point ID of the access point of the transmission source.

According to the present embodiment, not only an access point ID of an access point but also an image acquired by the HMD <NUM> are used to identify an area where the HMD <NUM> is present.

According to the present embodiment, even in the case where the HMD <NUM> may be connected to the access point <NUM> which is installed outside of the room <NUM> where the wearer of the HMD <NUM> is present, it can appropriately acquire and use the spatial configuration information stored in the server <NUM>. Furthermore, at this time, no special authentication processing is required.

In the case where the spatial configuration information is stored in the HMD <NUM>, the stored spatial configuration information is transmitted to the server <NUM>, whereby the server can compare the spatial configuration information with each other to confirm whether it matches the room.

As described above, a Global Positioning System (GPS) is not used in the embodiments and modifications described above. However, GPS information acquired by the GPS receiver <NUM> of the HMD <NUM> may be used.

For example, in the case of a room in a one-story building, using the GPS information enables identification of the room <NUM> where the HMD <NUM> is present. For example, in the case where the room has a rectangular shape, the latitudes and longitudes at the four corners of the room are registered in the server-side library <NUM> of the server <NUM> as location information on each room.

By comparing the GPS information (latitude and longitude) transmitted from the HMD <NUM> with the location information on each room registered in the server <NUM>, the room where the HMD <NUM> is present is identified.

The registration date and time of the spatial configuration information may be registered in the server-side library <NUM>. Whether the spatial configuration information has been registered may be determined based on not only the presence or absence thereof, but also the registration date and time. That is, even if the spatial configuration information is registered, in the case where the difference between the registration date and time and the current date and time is equal to or more than a predetermined value, the spatial configuration information is determined to be unregistered.

For example, since a condition within the room <NUM> may change, managing the spatial configuration information based on the date and time improves responsibility to the change in the arrangement of the interior or the like.

In each of the embodiments described above, as the identification information on each room, basically, a relay apparatus which is installed in each room <NUM> to relay the HMD <NUM> to a communication network is used. However, the identification information is not limited thereto. For example, in the case where each room is provided with a unique device capable of transmitting information identifying the room, such a device may be used. The device capable of transmitting information identifying the room is, for example, a room access management device having a communication function.

Furthermore, in the embodiments described above, as a mobile information device, the HMD <NUM> is used. However, a mobile information device is not limited thereto. For example, it may be a notebook PC, a tablet PC, or any other portable information device having similar functions.

The present invention is not limited to the embodiments and modifications described above, and various modifications are possible. For example, a part of an exemplary embodiment may be replaced with a part of other embodiments, and a part of other embodiments may be added to the exemplary embodiment. These modifications belong to the scope of the present invention. Furthermore, the numerical values, messages, and the like appearing in the description and drawings are merely examples, and the effect of the present invention is not impaired even if different ones are used.

A part of or all of the functions and the like of the present invention described above may be implemented by hardware, for example, by designing it by an integrated circuit. Alternatively, a microprocessor unit or the like may interpret and execute a program for implementing each function or the like, thereby implementing it by software. Hardware and software may be used together. The software may be stored in advance in the ROM <NUM> and the like of the HMD <NUM> at the time of product shipment. The software may be acquired from various servers or the like on the Internet after the product is shipped. Furthermore, the software which is provided in a memory card, an optical disk, or the like may be acquired.

Claim 1:
A portable information device (<NUM>), comprising:
a display (<NUM>);
a communication interface (<NUM>, <NUM>) provided for connection to a network;
a storage device (<NUM>, <NUM>);
a camera (<NUM>); and
a processor (<NUM>),
wherein:
the portable information device further comprises a range sensor (<NUM>); and
the processor is configured to:
acquire, from a relay apparatus (<NUM>, <NUM>) that is installed in a space where the portable information device is present and connects the portable information device to the network, relay apparatus identification information which is information identifying the relay apparatus;
determine whether spatial configuration information obtained by mapping the space, which is necessary for displaying on the display, is registered in the storage device;
in a case where the spatial configuration information is not registered, send a request to transmit the spatial configuration information, from the communication interface to a server connected to the network through the relay apparatus;
receive the spatial configuration information transmitted from the server in response to the request;
store the spatial configuration information that has been received in the storage device in association with the relay apparatus identification information;
in a case of not receiving the spatial configuration information from the server in response to the request, use the camera and the range sensor to generate the spatial configuration information on the space; and
store the spatial configuration information that has been generated in the storage device in association with the relay apparatus identification information.