Patent Description:
The invention further relates to a method of displaying data associated with at least one device, said data being displayed next to and/or overlaid on a view of a scene, said scene comprising said at least one device.

Now that more and more devices in buildings are connected to a network, more and more devices are able to provide data to other devices. For example, settings of devices may be obtained remotely and more and more sensor data are becoming available. Not only is the number of standalone sensor devices increasing but also the number of other devices that incorporates a sensor. For instance, advanced lighting systems comprise multiple detection means to detect user presence, activities, room usage, environmental conditions (e.g. ambient light levels and/or temperature). As a result, such systems can gather huge amounts of data, which can be made available to users such as facility managers, store owners, cleaning staff, and hospitality managers.

Since more and more systems, such as these advanced lighting systems, gather huge amounts of data, it gets hard for users to select the data they need. Solutions are needed that make it easy for a user to access the data that the user needs. Augmented reality (AR) is a user-friendly technology for allowing a user to get information about real-world objects.

<CIT> discloses an augmented reality system which provides an augmented reality experience by augmenting the user's real-world view with contextual information. An AR application may determine that augmentation should be provided for one, more than one, or none of the mapped objects/nodes in the user's current field of view. In one embodiment, the AR application may determine that augmentation is not to be provided for mapped objects that are in the field of view but more than a threshold distance away from the user. The user can select a particular node by focusing on a node marker or by issuing a voice command.

However, <CIT> does not address the need for a user to get data that is not related to one individual device.

<CIT> discloses a system for displaying data, the system comprising a first and a second image capturing device, a transparent display and a processor. The system is configured to capture an image within the line of sight of the user viewing the transparent display, to identify multiple objects in the image and to either individually display object information of the objects or to group display object information of the objects.

It is a first object of the invention to provide a system, which can use augmented reality to display data relating to devices in a camera's field of view on a level other than device level.

It is a second object of the invention to provide a method, which can use augmented reality to display data relating to devices in a camera's field of view on a level other than device level.

In a first aspect of the invention, a system for displaying data associated with at least one device, said data being displayed next to and/or overlaid on a view of a scene, said scene comprising said at least one device, comprises at least one input interface, at least one output interface, and at least one processor configured to obtain an image captured by a camera via said at least one input interface, said image capturing said scene and said at least one device, determine a distance from said camera to the closest device of said at least one device, select a device level or a group level based on said distance, and display said data by displaying, via said at least one output interface, data associated with only said closest device if said device level is selected or an aggregation of data associated with a group of devices if said group level is selected, said group of devices comprising said closest device.

This augmented reality system can both display data on a device level and on a group level. On the group level, an aggregation of data related to multiple devices is displayed. The level is selected based on the distance from the camera to the closest device in the field of view. By changing their positions and camera orientations, users can choose whether they want to get data relating an individual device or aggregated data related to a group of devices. Said displayed data may comprise sensor data associated with said closest device or associated with said group of devices, for example.

Said closest device may be a lighting device, for example. Said at least one processor may be configured to identify said closest device by obtaining an identifier communicated by said closest device via modulated light communication and obtain said data based on said identifier. Visible and/or non-visible light sources may be used to communicate identifiers. For example, identifiers may be transmitted using infra-red LiFi. Alternatively or additionally, said data may be obtained via modulated light communication.

Said at least one processor may be configured to identify said closest device by performing image recognition and obtaining an identifier of said closest device based on a result of said image recognition and obtain said data based on said identifier. For example, features may be extracted from a captured image and compared with features of known devices and/or features of current light effects rendered by known lighting devices. When a device or light effect of a lighting device is recognized in the image, the associated identifier may be obtained. For instance, a HueGo (a Philips Hue Go lamp) rendering a pink light setting may be recognized in the image.

The result of the image recognition may be combined with position information. For example, if a HueGo is recognized in the image, but one HueGo has been installed in the kitchen and another HueGo has been installed in the living room, then detecting that the system is present in the kitchen makes it possible to identify the HueGo in the kitchen. In case of many similar devices (e.g. luminaires in an office ceiling), the system needs more accurate position (and orientation) information and the locations of the devices need to be known more precisely (e.g. obtained from a Building Information Model).

Said at least one processor may be configured to select said device level if said distance is determined not to exceed a first distance threshold. Said at least one processor may be configured to select said device level or said group level further based on a quantity of devices captured in said image. For example, said at least one processor may be configured to determine a quantity of devices captured in said image and select said group level by default if said distance is determined to exceed said first distance threshold and said quantity is determined to be higher than one.

Said at least one processor may be configured to determine a second distance from said camera to the second closest device of said at least one device if at least one further device is captured in said image and select said device level or said group level further based on said second distance. In this case, the device level may be selected even if the distance to the closest device exceeds the first distance threshold.

Said at least one processor may be configured to calculate a difference between said distance and said second distance and select said device level if said difference is determined to exceed a difference threshold. In this case, if the distance to the closest device exceeds the first distance threshold but the second closest device is significantly farther away than the closest device, the device level and not the group level is selected. If the distance to the closest device exceeds the first distance threshold and the distances to the closest device and the second closest device do not differ too much, then a group level is selected. The group comprises at least the closest device and the second closest device, and typically all devices captured in the image. Thus, this (first) group level is selected when the user is believed to have intended to capture a group of devices in the image.

Said at least one processor may be configured to select said group level if said distance and second distance are determined to exceed said first distance threshold and not exceed a second distance threshold, said group further comprising said second closest device. Thus, this (first) group level is selected when the user is believed to have intended to capture a group of devices in the image, but without the need to determine a difference between distances. If the distance to the closest device exceeds the first distance threshold and the second distance to the second closest device exceeds the second distance threshold, then the device level is selected.

Said at least one processor may be configured to determine whether said closest device forms a group with at least one other device not captured in said image and select said group level if said distance is determined to exceed said first distance threshold, said second distance is determined to exceed said second distance threshold or no further device than said closest device is captured in said image, and said closest device is determined to form said group with said at least one other device, said group further comprising said at least one other device. Said at least one other device may have a same type as said closest device and/or be located in the same space as said closest device, for example.

It may not be necessary to select the device level if it was not possible to determine the second distance (because no further device is captured in the image) or to select the above-described first group level if the second distance exceeds the second distance threshold. Instead, if the distance to the closest device exceeds the first distance threshold and the closest device forms a group with at least one other device not captured in the image, then a second group level may be selected in these cases. For example, aggregated data related to all devices in the room may be displayed. In this case, the second group level is also referred to as room level.

The at least one processor may be configured not just to choose from one device level and one group level but to choose from a device level and multiple group levels. Whether the first group level or the second group level is chosen may depend, for example, on the quantity of devices captured in the image and/or the distances from the camera to at least some of these devices. The type of data displayed for the second group level may be different than for the first group level.

As described above, devices in the same group may have a same type and/or be located in the same space, but other grouping criteria may also be used. User-defined and system-defined grouping criteria may be distinguished. As an example of the former, a user may be able to group lights in rooms and zones where zones consist of a subset of lights from the room. For instance, a "TV zone" group could be a part of a "Living room" group. In this case, if a user points the camera at a few lighting devices that are part of both the Living room and the TV zone, the system could decide to show either zone or room level information. The room-based group might also include other connected devices that are assigned to it, e.g., a presence sensor and/or physical light controls.

System-defined groups may be static or dynamic. A static group may be determined, for example, based on a type of device (e.g., if two spotlights are captured in the view, the system might provide group information for only spotlights present in the area and not for other types of fixtures) or based on location (somewhat similar to user defined room-based grouping but using different heuristics of how lights are grouped; this is especially beneficial for smart buildings with multifunctional and open areas). A dynamic group may be determined in real-time based on the state of at least one of the devices captured in the image. For example, if the image captures two lighting devices that are switched on, then the system might provide information about all devices in the neighborhood that are also switched on.

Said at least one processor may be configured to determine whether said closest device forms a group with at least one other device not captured in said image and select said group level if said distance is determined to exceed a third distance threshold and said closest device is determined to form said group with said at least one other device, said group further comprising said at least one other device. Thus, the second group level may be selected even if the at least one processor is not configured to determine a second distance. The third distance threshold may be the same as or different from the first distance threshold.

For example, the first group level may be selected when the distance exceeds the first distance threshold but does not exceed the third distance threshold (on the condition that a further device is captured in the image) and the second group level may be selected when the distance exceeds both the first distance threshold and the third distance threshold (on the condition that the closest device forms a group with at least one other device not captured in the image).

Alternatively or additionally, a choice between the first group level and the second group level may be made in dependence on the quantity of devices captured in the image. For example, if the distance to the closest device exceeds the first distance threshold and more than a certain number of devices are captured in the image, the second group level (e.g. room level) may be selected. If the distance to the closest device exceeds the first distance threshold and not more than the certain number of devices are captured in the image, the first group level may be selected.

Said at least one processor may be configured to select said device level or said group level further based on a user preference if said distance is determined to exceed said third distance threshold, no further device than said closest device is determined to be captured in said image, and said closest device is determined to form said group with said at least one other device. If no further device than the closest device is captured in the image and the closest device is relatively far away, then some users may prefer it if the device level would be selected while other users may prefer it if the second group level, e.g. room level, would be selected (on the condition that the closest device forms a group with at least one other device not captured in the image). It may therefore be beneficial to let the user provide a user preference.

Said at least one processor may be configured to render an indication of said selected level. For example, clear feedback about the current level (device or group) may be provided. This feedback may be visual (e.g. displayed next to and/or overlaid on the view of the scene) or may be auditory (e.g. if during a data visualization, the user changes the AR device distance/perspective). The auditory feedback may indicate a change in selected level.

In a second aspect of the invention, a method of displaying data associated with at least one device, said data being displayed next to and/or overlaid on a view of a scene, said scene comprising said at least one device, comprises obtaining an image captured by a camera, said image capturing said scene and said at least one device, determining a distance from said camera to the closest device of said at least one device, selecting a device level or a group level based on said distance, and displaying said data by displaying data associated with only said closest device if said device level is selected or an aggregation of data associated with a group of devices if said group level is selected, said group of devices comprising said closest device. Said method may be performed by software running on a programmable device. This software may be provided as a computer program product.

A non-transitory computer-readable storage medium stores at least one software code portion, the software code portion, when executed or processed by a computer, being configured to perform executable operations for displaying data associated with at least one device, said data being displayed next to and/or overlaid on a view of a scene, said scene comprising said at least one device.

The executable operations comprise obtaining an image captured by a camera, said image capturing said scene and said at least one device, determining a distance from said camera to the closest device of said at least one device, selecting a device level or a group level based on said distance, and displaying said data by displaying data associated with only said closest device if said device level is selected or an aggregation of data associated with a group of devices if said group level is selected, said group of devices comprising said closest device.

<FIG> shows a first embodiment of the system for displaying data associated with at least one device. The data are displayed next to and/or overlaid on a view of a scene which comprises the at least one device. In this first embodiment, the system is a mobile device <NUM>. The mobile device <NUM> may be a mobile phone, a tablet, or augmented reality glasses, for example.

In the example of <FIG>, a lighting system comprises a light controller <NUM> and lighting devices <NUM>-<NUM>. Lighting devices <NUM>-<NUM> can be controlled via light controller <NUM>, e.g. using Zigbee technology. Light controller <NUM> is connected to a wireless LAN access point <NUM>, e.g. via Wi-Fi or Ethernet. The wireless LAN access point <NUM> is connected to the Internet <NUM>. In an alternative embodiment, one or more of lighting devices <NUM>-<NUM> can be controlled directly, e.g. via Bluetooth, or via an Internet server <NUM> and the wireless LAN access point <NUM>. The lighting devices <NUM>-<NUM> may be capable of receiving and transmitting Wi-Fi signals, for example. The Internet server <NUM> is also connected to the Internet <NUM>.

The mobile device <NUM> comprises a transceiver <NUM>, a transmitter <NUM>, a processor <NUM>, memory <NUM>, a camera <NUM>, and a (e.g. touchscreen) display <NUM>. The processor <NUM> is configured to obtain an image captured by the camera <NUM> via an interface to the camera <NUM>, determine a distance from the camera to the closest device of the one or more devices captured in the image, and select a device level or a group level based on the distance. For example, many of today's smartphones have depth-sensing capabilities. A distance towards a surface may be determined by combining motion-sensing data of the mobile device with image processing data from one or more cameras (e.g. of the mobile device).

In the example of <FIG>, one or more of lighting devices <NUM>-<NUM> are captured in the image. The closest device is one of lighting devices <NUM>-<NUM>. In another example, additionally or alternatively, one or more other types of devices are captured in the image and the closest device might be something other than a lighting device. In the embodiment of <FIG>, the processor <NUM> is configured to identify the closest device by obtaining an identifier communicated by the closest device via visible light communication and obtain the data based on the identifier, e.g. from the device itself.

The processor <NUM> is configured to display, via the display <NUM>, data associated with only the closest device if the device level is selected or an aggregation of data associated with a group of devices if the group level is selected. The group of devices comprises the closest device. Data associated with the closest device or associated with the group of devices may be obtained from the device(s) via the wireless LAN access point <NUM> or alternatively, via visible light communication. For example, the lighting devices <NUM>-<NUM> may be able to dynamically modulate the light output signal such that a large amount of data can be emitted.

In the embodiment of <FIG> or in an alternatively embodiment, input about a user profile input (e.g. user role, user ID) may be used to determine a data presentation which matches the user's authorization and interests.

In the embodiment of the mobile device <NUM> shown in <FIG>, the mobile device <NUM> comprises one processor <NUM>. In an alternative embodiment, the mobile device <NUM> comprises multiple processors. The processor <NUM> of the mobile device <NUM> may be a general-purpose processor, e.g. from ARM or Qualcomm or an application-specific processor. The processor <NUM> of the mobile device <NUM> may run an Android or iOS operating system for example. The camera <NUM> may comprise a CMOS or CCD sensor, for example. The display <NUM> may comprise an LCD or OLED display panel, for example. The processor <NUM> may use (touch screen) display <NUM> to provide a user interface, for example. The memory <NUM> may comprise one or more memory units. The memory <NUM> may comprise solid state memory, for example.

The receiver <NUM> and the transmitter <NUM> may use one or more wireless communication technologies, e.g. Wi-Fi (IEEE <NUM>) for communicating with the wireless LAN access point <NUM>, for example. In an alternative embodiment, multiple receivers and/or multiple transmitters are used instead of a single receiver and a single transmitter. In the embodiment shown in <FIG>, a separate receiver and a separate transmitter are used. In an alternative embodiment, the receiver <NUM> and the transmitter <NUM> are combined into a transceiver. The mobile device <NUM> may comprise other components typical for a mobile device such as a battery and a power connector. The invention may be implemented using a computer program running on one or more processors.

<FIG> shows a second embodiment of the system for displaying data associated with at least one device. The data are displayed next to and/or overlaid on a view of a scene which comprises the at least one device. In this second embodiment, the system is a computer <NUM>. The computer <NUM> is connected to the Internet <NUM> and acts as a server. The computer <NUM> may be operated by a lighting company, for example. In the embodiment of <FIG>, the computer <NUM> is able to control the lighting devices <NUM>-<NUM> via the wireless LAN access point <NUM> and the light controller <NUM>.

The computer <NUM> comprises a receiver <NUM>, a transmitter <NUM>, a processor <NUM>, and storage means <NUM>. The processor <NUM> is configured to obtain an image captured by a camera of a mobile device <NUM> via receiver <NUM>, determine a distance from the camera to the closest device of the one or more devices captured in the image, and select a device level or a group level based on the distance. The processor <NUM> may be able to determine the distance based solely on the image, based on the image and on other data received from the mobile device <NUM> or from another system, or based solely on other data.

The processor <NUM> is configured to display data associated with only the closest device if the device level is selected or an aggregation of data associated with a group of devices if the group level is selected. The group of devices comprises the closest device. The processor <NUM> may be configured to display the data by transmitting the data to the mobile device <NUM> via the transmitter <NUM> and causing the mobile device <NUM> to display the data, e.g. via an app running on the mobile device <NUM>.

In the embodiment of the computer <NUM> shown in <FIG>, the computer <NUM> comprises one processor <NUM>. In an alternative embodiment, the computer <NUM> comprises multiple processors. The processor <NUM> of the computer <NUM> may be a general-purpose processor, e.g. from Intel or AMD, or an application-specific processor. The processor <NUM> of the computer <NUM> may run a Windows or Unix-based operating system for example. The storage means <NUM> may comprise one or more memory units. The storage means <NUM> may comprise one or more hard disks and/or solid-state memory, for example. The storage means <NUM> may be used to store an operating system, applications and application data, for example.

The receiver <NUM> and the transmitter <NUM> may use one or more wired and/or wireless communication technologies such as Ethernet and/or Wi-Fi (IEEE <NUM>) to communicate with devices on the Internet <NUM>, for example. In an alternative embodiment, multiple receivers and/or multiple transmitters are used instead of a single receiver and a single transmitter. In the embodiment shown in <FIG>, a separate receiver and a separate transmitter are used. In an alternative embodiment, the receiver <NUM> and the transmitter <NUM> are combined into a transceiver. The computer <NUM> may comprise other components typical for a computer such as a power connector. The invention may be implemented using a computer program running on one or more processors.

In the embodiment of <FIG>, the computer <NUM> transmits data to the lighting devices <NUM>-<NUM> via light controller <NUM>. In an alternative embodiment, the lighting devices <NUM>-<NUM> are able to receive data from the computer <NUM> which have not passed through a light controller.

<FIG> shows an example of a building in which the system of <FIG> may be used. In the example of <FIG>, a user <NUM> uses mobile device <NUM> of <FIG>. The building <NUM> comprises a hallway <NUM>, a kitchen <NUM>, and a living room <NUM>. Wireless LAN access point <NUM> has been installed in the hallway <NUM>. Lighting devices <NUM>-<NUM> and light controller <NUM> have been installed in the living room <NUM>, and lighting device <NUM> has been installed in the kitchen <NUM>. In the example of <FIG>, the camera of the mobile device <NUM> captures an image of the lighting devices <NUM> and <NUM>.

A first embodiment of the method of displaying data associated with at least one device is shown in <FIG>. The data is displayed next to and/or overlaid on a view of a scene which comprises the at least one device. The method may be performed by the mobile device <NUM> of <FIG> or the cloud computer <NUM> of <FIG>, for example.

A step <NUM> comprises obtaining an image captured by a camera. The image captures the scene and the at least one device. The camera is typically embedded in a mobile device. The camera may be embedded in a mobile phone or in augmented reality glasses, for example. A step <NUM> comprises determining a distance from the camera to the closest device captured in the image. This determination may be performed by using a dedicated time-of-flight distance sensor integrated into the mobile device which comprises the camera, or by combining motion-sensing data from the inertial sensor of the mobile device with data obtained by analyzing image(s) from the camera and/or from another camera, for example.

Step <NUM> may comprise analyzing the image obtained in step <NUM>. If the closest device is a lighting device and this lighting device is generating a high lumen output, the distance may be determined towards surfaces very close to the light source. Another approach is to determine the distance to the closest device based on the relative signal strength of the closest device's RF signal (e.g. WiFi or Bluetooth) as detected by the mobile device which comprises the camera.

A step <NUM> comprises selecting a device level or a group level based on the distance determined in step <NUM>. A step <NUM> comprise determining whether a device level or a group level was selected in step <NUM> and performing a step <NUM> if a device level was selected and performing step <NUM> if a group level was selected.

Step <NUM> comprises identifying the closest device. Step <NUM> may comprise identifying the closest device by obtaining an identifier communicated by the closest device via modulated light communication, e.g. visible light communication (VLC). For example, the mobile device may use its camera to detect the identifier as emitted by the VLC (lighting) device or use a (directional) light sensor which is able to detect the VLC signal. Step <NUM> comprises obtaining data associated with only the closest device. Step <NUM> comprises obtaining the data based on the identifier obtained in step <NUM>. The data may be sensor data, for example. If the closest device is a lighting device, the data may be lighting usage data or sensor data from a sensor embedded in, connected to, or associated with the lighting device, for example.

Step <NUM> comprises identifying devices in a group of devices. The group of devices comprises the closest device. Step <NUM> may comprise identifying the closest device by obtaining an identifier communicated by the closest device via visible light communication, as described in relation to step <NUM>, and identifying the other devices in the group based on the identifier of the closest device. Alternatively, step <NUM> may comprise identifying the devices in the group by obtaining the identifiers communicated by these devices via visible light communication.

Step <NUM> comprises obtaining data associated with the group of devices, e.g. based on the identifiers obtained in step <NUM>. Step <NUM> comprises aggregating the data obtained in step <NUM>. The data may be sensor data or lighting usage data, for example. In step <NUM> and/or step <NUM>, a subset of data available for a certain device or group of device may be obtained. A user may be able to select which subset of data should be obtained for a certain device, for a certain group of devices, or for a level in general.

A step <NUM> is performed after step <NUM> or step <NUM> has been performed. Step <NUM> comprises displaying the data obtained in step <NUM> or determined in step <NUM> next to and/or overlaid on a view of the scene. Step <NUM> comprises displaying data associated with only the closest device if the device level is selected in step <NUM> or an aggregation of data associated with a group of devices if the group level is selected in step <NUM>. For instance, when distant to a lamp, the data could show most frequently used light scenes in the room, whereas when close to a lamp, the data shown could indicate most frequently used lamp settings.

As mentioned above, the data is displayed next to and/or overlaid on a view of a scene which comprises at least the closest device. The data may be displayed next and/or overlaid on the image obtained in step <NUM> or may be displayed using augmented reality glasses, for example. In the latter case, the image obtained in <NUM> does not need to be displayed.

In step <NUM>, a subset of the data obtained in step <NUM> and/or step <NUM> may be displayed. A user may be able to select which subset of the obtained data should be displayed for a certain device, for a certain group of devices, or for a level in general. A user may be able to choose from pre-defined data dashboards and/or from personalized data dashboards. A user may be able to switch between dashboard by changing the orientation of his mobile device. The system performing the method may record which dashboards users select and use the same dashboard that the user selected previously or use a dashboard that was selected most often by users in general for a certain device, for a certain group of devices, or for a level in general.

In an advanced implementation, user information (e.g. user role, user ID) is used to select a dashboard which matches the user's authorization and/or interests. As a result, different types of users will each see different types of data presentations for the same device or group of devices, showing relevant data tailored to the individual users (e.g. based on role, authorization, preferences, interests). The personalized data dashboards may also be selected by a learning system which has monitored individual data presentation interests in various situations over time.

In addition to the (aggregated) data, the distance determined in step <NUM> or an indication thereof may be displayed as well. This may be done in absolute metrics, e.g. by showing a distance slider, or by indicating the determined level of proximity, e.g. whether the mobile device is remote, near or close to the closest device.

A second embodiment of the method of displaying data associated with at least one device is shown in <FIG>. This second embodiment is a variation on the embodiment of <FIG>. In this second embodiment, steps <NUM> and <NUM> are performed instead of steps <NUM> and <NUM> of <FIG>. Step <NUM> is an implementation of step <NUM> and step <NUM> is performed between steps <NUM> and <NUM>.

Step <NUM> comprises identifying the devices in the image. Step <NUM> may comprise identifying the devices in the image by obtaining identifiers communicated by the devices via visible light communication, e.g. if the devices are lighting devices. Step <NUM> comprises determining distances from the camera to each of the devices in the image.

Step <NUM> comprises obtaining data associated with only the closest device based on the identifier obtained in step <NUM>. Step <NUM> comprises obtaining data associated with the group of devices, e.g. based on one or more of the identifiers obtained in step <NUM>.

A third embodiment of the method of displaying data associated with at least one device is shown in <FIG>. This third embodiment is a variation on the embodiment of <FIG>. In this third embodiment, step <NUM> is implemented by a step <NUM> and a step <NUM> is performed between steps <NUM> and <NUM>. Step <NUM> comprises determining a quantity of devices captured in the image obtained in step <NUM>.

Step <NUM> comprises selecting the device level if the distance determined in step <NUM> does not exceed a first distance threshold T and selecting the group level if the distance determined in step <NUM> exceeds the first distance threshold T and the quantity determined in step <NUM> is higher than one. If the distance determined in step <NUM> exceeds the first distance threshold T and the quantity determined in step <NUM> is one, then the device level may be selected in step <NUM>, or alternatively, either the device level or a second group level may then be selected in step <NUM>. The latter may be user-configurable or defined by the implementor, for example.

<FIG> show examples in which the distance from the camera to the closest device captured in the image does not exceed the first distance threshold T and the device level is therefore selected. In the example of <FIG>, a mobile device <NUM> displays an image <NUM> which captures two lighting devices <NUM> and <NUM>. In the example of <FIG>, the mobile device <NUM> displays an image <NUM> which captures lighting device <NUM>. In both examples, lighting device <NUM> is the closest device. As the device level has been selected, data <NUM> associated with the (closest) lighting device <NUM> are displayed. Data <NUM> comprise recent light settings associated with the lighting device <NUM>.

A fourth embodiment of the method of displaying data associated with at least one device is shown in <FIG>. This fourth embodiment is a variation on the embodiment of <FIG>. In this fourth embodiment, step <NUM> is implemented by a step <NUM> and steps <NUM>, <NUM>, and <NUM> have been added between steps <NUM> and <NUM>.

Step <NUM> comprises determining whether at least one further device is captured in the image and if so, performing steps <NUM> and <NUM>. Otherwise, steps <NUM> and <NUM> are skipped. Step <NUM> comprises determining a second distance from the camera to the second closest device of devices captured in the obtained image. Step <NUM> comprises calculating a difference between the distance determined in step <NUM> and the second distance determined in step <NUM>.

Step <NUM> comprises selecting the device level if the distance determined in step <NUM> does not exceed a first distance threshold or if the difference determined in step <NUM> exceeds a difference threshold. Step <NUM> comprises selecting the group level if the distance determined in step <NUM> exceeds the first distance threshold and the difference determined in step <NUM> does not exceed the difference threshold.

A fifth embodiment of the method of displaying data associated with at least one device is shown in <FIG>. This fifth embodiment is a variation on the embodiment of <FIG>. In this fifth embodiment, a step <NUM> is performed after step <NUM>, step <NUM> is implemented by a step <NUM>, steps <NUM> and <NUM> have been added between steps <NUM> and <NUM>, step <NUM> is implemented by a step <NUM>, and steps <NUM>, <NUM>, and <NUM> have been added between steps <NUM> and <NUM>.

Step <NUM> comprises determining whether the closest device forms a group with at least one other device not captured in the image, e.g. with one or more devices which have a same type as the closest device or with one or more devices located in the same space as the closest device.

Step <NUM> comprises determining whether at least one further device, i. e, other than the closest device, is captured in the image and if so, performing step <NUM>. Otherwise, step <NUM> is skipped. Step <NUM> comprises determining a second distance from the camera to the second closest device captured in the image.

Step <NUM> comprises selecting the device level if the distance determined in step <NUM> does not exceed a first distance threshold. Step <NUM> comprises selecting the device level if the distance determined in step <NUM> exceeds the first distance threshold, it was determined in step <NUM> that the closest device does not form a group with at least one other device, and step <NUM> was skipped.

Step <NUM> comprises selecting a first group level if the distance determined in step <NUM> and the second distance determined in step <NUM> exceed the first distance threshold and if either a) these two distances do not exceed a second distance threshold or b) it was determined in step <NUM> that the closest device does not form a group with at least one other device.

Step <NUM> comprises selecting a second group level if the distance determined in step <NUM> exceeds the first distance threshold, it was determined in step <NUM> that the closest device forms a group with at least one other device, and if either a) the second distance determined in step <NUM> exceeds the second distance threshold or b) step <NUM> was skipped.

Step <NUM> comprises determining whether the device level, the first group level, or the second group level was selected in step <NUM> and performing step <NUM> if the device level was selected, performing step <NUM> if the first group level was selected, and performing step <NUM> if the second group level was selected. Steps <NUM> and <NUM> are performed after step <NUM>. Steps <NUM>, <NUM>, and <NUM> are similar to steps <NUM>, <NUM>, and <NUM>, respectively. However, in step <NUM> data is obtained associated with a first group of devices which comprises at least the closest device and the second closest device, typically all devices captured in the image. In step <NUM>, data is obtained associated with a second group of devices which comprises the closest device and the at least one other device not captured in the image.

<FIG> show examples in which the distance from the camera to the closest device captured in the image exceeds the first distance threshold. In these examples, the mobile device <NUM> displays images <NUM> and <NUM> which capture two lighting devices <NUM> and <NUM>. In the example of <FIG>, lighting device <NUM> is the closest device and lighting device <NUM> is the second closest device.

In the example of <FIG>, the distance from the camera to the second closest lighting device does not exceed the second distance threshold and as a result, the first group level is selected. As the first group level has been selected, data <NUM> associated with the group of lighting devices captured in the image, i.e. lighting devices <NUM> and <NUM>, are displayed. Data <NUM> comprise the top three of light settings used on any of these two lighting devices.

In the example of <FIG>, the distances from the camera to lighting devices <NUM> and <NUM> are the same. The closest lighting device may be chosen arbitrarily from lighting devices <NUM> and <NUM>. The second closest lighting device is the lighting device not chosen as closest lighting device. In the example of <FIG>, the distance from the camera to the closest device captured in the image and the second distance from the camera to the second closest device captured in the image both exceed the first distance threshold and the second distance threshold.

Since the lighting devices <NUM> and <NUM> form a group with a lighting device <NUM>, which is located in the same room but not captured in the image <NUM>, the second group level is selected. As the first group level has been selected, data <NUM> associated with the group of lighting devices in the same room, i.e. lighting devices <NUM>-<NUM>, are displayed. Data <NUM> comprise the top three of light scenes involving one or more of these three lighting devices.

A sixth embodiment of the method of displaying data associated with at least one device is shown in <FIG>. This sixth embodiment is a variation on the embodiment of <FIG>. In this sixth embodiment, step <NUM> of <FIG> and a step <NUM> are performed after step <NUM>, step <NUM> is implemented by a step <NUM>, step <NUM> is implemented by step <NUM> of <FIG>, and steps <NUM>, <NUM>, and <NUM> of <FIG> have been added between steps <NUM> and <NUM>.

Step <NUM> comprises determining whether the closest device forms a group with at least one other device not captured in the image, e.g. with one or more devices which have a same type as the closest device or with one or more devices located in the same space as the closest device. Step <NUM> comprises determining whether at least one further device, i. e, other than the closest device, is captured in the image. This step is somewhat similar to step <NUM> of <FIG>.

Step <NUM> comprises selecting the device level if the distance determined in step <NUM> does not exceed a first distance threshold. Step <NUM> comprises selecting the device level if the distance determined in step <NUM> exceeds the first distance threshold, it was determined in step <NUM> that the closest device does not form a group with the at least one other device, and it was determined in step <NUM> that no further device is captured in the image.

Step <NUM> comprises selecting a first group level if the distance determined in step <NUM> exceeds the first distance threshold and it was determined in step <NUM> that a further device other than the closest device is captured in the image. Step <NUM> comprises selecting the device level or a second group level based on a user preference if the distance determined in step <NUM> exceeds the further distance threshold, it was determined in step <NUM> that the closest device forms a group with the at least one other device, and it was determined in step <NUM> that no further device than the closest device is captured in the image. After step <NUM>, steps <NUM>-<NUM>, and <NUM>-<NUM> of <FIG> are performed.

A seventh embodiment of the method of displaying data associated with at least one device is shown in <FIG>. This seventh embodiment is a variation on the embodiment of <FIG>. In this seventh embodiment, step <NUM> of <FIG> is performed after step <NUM>, step <NUM> is implemented by a step <NUM>, step <NUM> is implemented by step <NUM> of <FIG>, and steps <NUM>, <NUM>, and <NUM> of <FIG> have been added between steps <NUM> and <NUM>.

Step <NUM> comprises selecting the device level if the distance determined in step <NUM> does not exceed a first distance threshold. Step <NUM> comprises selecting a first group level if the distance determined in step <NUM> exceeds the first distance threshold and it was determined in step <NUM> that the closest device does not form a group with at least one other device. Step <NUM> comprises selecting a second group level if the distance determined in step <NUM> exceeds the first distance threshold and it was determined in step <NUM> that the closest device forms a group with at least one other device. After step <NUM>, steps <NUM>-<NUM>, and <NUM>-<NUM> of <FIG> are performed.

The embodiments of <FIG>, <FIG>, and <FIG> differ from each other in multiple aspects, i.e. multiple steps have been added or replaced. In variations on these embodiments, only a subset of these steps is added or replaced and/or one or more steps is omitted. For example, steps <NUM> and <NUM> of <FIG> may be added to, and steps <NUM> and <NUM> may be omitted from, the embodiments of <FIG>, <FIG>, and <FIG>. In the embodiments of <FIG> and <FIG>, step <NUM> may be omitted for the same reason as steps <NUM> and <NUM>.

<FIG> depicts a block diagram illustrating an exemplary data processing system that may perform the method as described with reference to <FIG>, <FIG>, and <FIG>.

The data processing system may be an Internet/cloud server, for example.

Claim 1:
A system (<NUM>,<NUM>) for displaying data associated with at least one device (<NUM>,<NUM>), said data being displayed next to and/or overlaid on a view of a scene, said scene comprising said at least one device (<NUM>,<NUM>), said system (<NUM>,<NUM>) comprising:
at least one input interface (<NUM>,<NUM>);
at least one output interface (<NUM>,<NUM>); and
at least one processor (<NUM>,<NUM>) configured to:
- obtain an image (<NUM>-<NUM>) captured by a camera of a mobile device (<NUM>, <NUM>) via said at least one input interface (<NUM>,<NUM>), said image (<NUM>-<NUM>) capturing said scene comprising said at least one device (<NUM>,<NUM>),
- determine a distance from said camera to the closest device (<NUM>) of said at least one device,
- select a device level or a group level based on said distance, and
- display said data by displaying, via said at least one output interface (<NUM>,<NUM>), data (<NUM>) associated with only said closest device (<NUM>) if said device level is selected or an aggregation (<NUM>,<NUM>) of data associated with a group of devices (<NUM>-<NUM>,<NUM>-<NUM>) if said group level is selected, said group of devices (<NUM>-<NUM>,<NUM>-<NUM>) comprising said closest device (<NUM>), wherein said at least one processor (<NUM>,<NUM>) is configured to select said device level if said distance is determined not to exceed a first distance threshold.