Patent Description:
Brainwave-based device control is a rising new technology. A brain-computer-interface (BCI) is used to detect brain signals of a user, whereupon information from these brain signals is derived. This information may, for example, be indicative of a thought or an action of the user. The thought may, for example, be indicative of a control command for a controllable device, such as a lighting device. An example of such as system is disclosed in <CIT>. There are two main types of BCIs: non-invasive and invasive BCIs. The non-invasive versions are the most common, and comprise sensors (electrodes) placed on the human head. These measure brain activity and translate that data to a computer. Most BCIs utilize electroencephalography (EEG) systems, which typically feature electrodes are attached to the scalp, which measure the electrical current sent by the neurons inside the brain. Changes in this electrical current reflect brain activity, because when an individual performs an action or thinks about something, hundreds of thousands of neurons are fired. This generates the electrical current, which is large enough to be measured on the scalp. A computer system then tries to make sense of this data to derive the user's action or thought. Alternatives to EEG systems are electrooculography (EOG), electromyography (EMG), electrodermal activity (EDA) and photoplethysmography (PPG) systems. As alternative to utilizing electrodes on the surface of the scalp, implantable brain-computer interfaces may be used. Here, probes are inserted into the brain through an automated process performed by a surgical robot. Each probe comprises an area of wires that contains electrodes capable of locating electrical signals in the brain, and a sensory area where the wire interacts with an electronic system that allows amplification and acquisition of brain signals.

A research study (<NPL>. ) conducted several experiments of sustained attention on subjects under different illumination conditions. EEG was recorded from the parietal region of the brain. The study found that brain pulses were significantly influenced by the illuminance factor. Their mean values indicate that high illuminance resulted in significantly longer latencies than low illuminance. The study concluded that the illumination condition substantially influences the attentional processing as reflected in the significant modulations of EEG activity.

A related study (<NPL>) shows that both short-wavelength and long-wavelength light increase alertness at night, as shown in EEG power change. Additionally, <NUM> lx of red light is also found to significantly affect the EEG measures compared to preceding dark conditions. In another study (<NPL>), two levels (<NUM> lx and <NUM> lx) of blue and red lights were both found to increase EEG beta power.

In a related study (<NPL>) it was investigated how exposures to long-wavelength lights of two different levels (<NUM> lx and <NUM> lx) affect objective alertness (as measured by EEG). A significant effect of light levels on EEG beta (<NUM>-<NUM>) power was observed. Exposure to both <NUM> lx and <NUM> lx long-wavelength lights significantly increased beta power compared to the Dim condition.

In a related study (<NPL>), three participants who were visually blind but had intact non-visual responses, were subjected to an on-off patterns of blue light. The study concluded that the blue light impacts the occipital region of the brain and decreases the power of the alpha EEG rhythm in this specific part of the brain.

Light that impacts brain signals may originate from artificial lighting and/or natural daylight. The natural daylight present in the room may depend on the time of the day and/or the current position of window blinds. <CIT> relates to a lighting control device comprising an EEG receiver for receiving an EEG of a user, an EEG analyzer for classifying and extracting the type of EEG according to the frequency from the received EEG, using the changes of the extracted EEG and a state determination unit that determines a user's sleep state, and an illumination control unit that controls lighting according to a user's sleep or weather state. <CIT> relates to a user-oriented customized lighting control system and method in consideration of the surrounding environment in which indoor lighting is adjusted to converge with a color that generates a specific EEG after measuring an EEG response signal for a current lighting color. <CIT> relates to brain interfacing apparatus and methods for using such apparatus, by employing artificial intelligence (adaptive learning) implemented using computing arrangements that modify a manner of operation of the brain interfacing apparatus when processing signals therethrough, when in operation.

The inventors have realized that light effects, for instance effects that include substantial amounts of blue light, bright light or dynamics, or light of specific wavelengths may compromise brainwave-based device control when utilizing the occipital brain region. As a result, a BCI may look at brainwaves in different regions of the brain, but these brainwaves may not reflect the correct cues for control of a device, because the resulting brainwaves may be affected by illumination (e.g. brainwaves are attenuated or amplified). This may result in false or incorrect triggers. It is therefore an object of the present invention to provide a brain control interface system that reduces the chance of false/incorrect triggers.

According to a first aspect of the present invention, the object is achieved by a brain control interface system, comprising:.

By adjusting the light scene of the one or more lighting devices (and therewith the light output of the one or more lighting devices), the effect of the light output on the level of noise in the brain signals is reduced. Since the light effects provided by the one or more lighting devices affect the brain signals, it is beneficial to adjust the light scene to reduce the effect of the light effects. By adjusting the light scene, the brain control interface system reduces the chance of false/incorrect triggers. The adjusting of the of light scene may be incrementally adjusting the lighting scene.

The light scene may be a dynamic light scene that changes over time, wherein the dynamic light scene has a dynamics level. The one or more processors may be configured to adjust the dynamics level until the target level of noise in the brain signals has been established. The level of dynamics may be defined by the amount of change of properties of the light output of the one or more lighting devices within a time period. The properties of the light output may include but are not limited to: hue, saturation, brightness, flicker, beam direction, etc. Dynamic effects, and more specifically dynamic effects with higher dynamics levels, may affect the brain signals and result in a higher level of noise. Adjusting (reducing) the dynamics level is beneficial because it reduces the chance of false/incorrect triggers.

The one or more processors may be configured to iteratively adjust the light scene while monitoring the level of noise until the target level of noise in the brain signals has been established. The one or more processors may repeat the adjustment by further adjusting the light scene while monitoring the level of noise until the target level has been established.

The one or more processors may be configured to adjust the light scene towards a target light scene until the target level of noise in the brain signals has been established. The target light scene may be a predefined light scene. The adjusting may be incremental adjusting. The predefined light scene may comprise light output characteristics which reduce the level of noise in detected brain signals. (Iteratively/continuously/incrementally) adjusting the light scene to the target light scene - until target level of noise in the brain signals has been established - is beneficial because the (original) light scene is maintained as much as possible while reducing the level of noise. The target light scene may comprise at least one of the following characteristics: a higher intensity compared to the light scene, a lower level of dynamics compared to the light scene, and a spectrum comprising more blue light compared to the light scene. These characteristics are examples of characteristics that affect (reduce) the level of noise.

The one or more processors may be configured to analyze the brain signals to derive a control command from the brain signals for a controllable device. The one or more processors may be further configured to switch to a brain control mode when the target level of noise in the brain signals has been established, wherein, in the brain control mode, the one or more processors are configured to control the controllable device based on the derived control command. The one or more processors may only control (either directly or indirectly) the controllable device based on the derived control command only when the brain control mode is active. This is beneficial, because the controllable device is not controlled when the level of noise (still) exceeds the threshold. The one or more processors may be further configured to select the target level of noise based on an expected control command that will be provided by the user. Similarly, the one or more processors may be further configured to select the target level of noise based on an expected emotion state of the user, for instance the user may be expected to transition from neutral emotional state to a relaxed emotional state, for instance when the user has just started meditating.

The controllable device may be a lighting device of the one or more lighting devices. Alternatively, the controllable device may, for example, be a connected (home) appliance or connected (office) equipment.

The input may be configured to obtain the data indicative of the current light scene by obtaining sensor data from a light sensor (located in the environment). Alternatively, the input may be a receiver configured to receive the data indicative of the current light scene from a lighting system controller. The lighting system controller may, for example, be a central (home) lighting controller, a bridge, a smartphone, etc..

The current light scene may be provided by a plurality of lighting devices. The one or more processors may be configured to obtain position and/or orientation information indicative of a position and/or an orientation of the user relative to the plurality of lighting devices. The one or more processors may be configured to select one or more of the plurality of lighting devices based on the position and/or the orientation of the user relative to the plurality of lighting devices, and adjust the light scene by adjusting the light output of the one or more selected lighting devices. The one or more processors may, for example, be configured to select one or more lighting devices which are located (or of which their light effect is located) in the field of view of the user. Next to artificial lighting, also the natural daylight present in the room may affect the brain signals. One or more window blinds located (or of which their light effect is located) in the field of view of the user may be selected and controlled accordingly. Selecting the one or more lighting devices based on the position and/or orientation of the user is beneficial because it optimizes reduction of noise in the detected brain signals. Additionally, when multiple users would be present in the environment, the light scene for the specific user may be adjusted while minimizing the effect of the change of the light scene on other users.

The one or more processors may be configured to initiate identifying the noise levels and adjusting the light scene upon receiving a control input indicative of that the user will provide a control command. The control input may be a direct command, or for example based on a change of the user's emotional state (which may be indicative of or preceded by the control input). This is beneficial, because the one or more processors are not required to continuously monitor the noise level, but only when required when the user intends to adjust to provide the control command. The control input may be provided by a user, by a sensor or by the brain control interface. The control input may, for example, be received from the user, be received from a user interface, a sensor, a memory, etc..

According to a second aspect of the present invention, the object is achieved by a method of adjusting a light scene, the method comprising:.

According to a third aspect of the present invention, the object is achieved by a computer program product for a computing device, the computer program product comprising computer program code to perform the method when the computer program product is run on the brain control interface system according to the first aspect.

It should be understood that the method and the computer program product may have similar and/or identical embodiments and advantages as the above-mentioned brain control interface systems.

In the context of the present invention, the term "light scene" relates to lighting control instructions/light settings for one or more lighting devices. The lighting control instructions may be the same for each lighting device, or be different for different lighting devices. The lighting control instructions may relate to one or more light settings, which may for instance be defined as RGB/HSL/HSB color values, CIE color values, brightness values, etc..

<FIG> shows schematically an overview of a brain control interface system <NUM>. The brain control interface system <NUM> comprises a brain control interface <NUM> (e.g. a head-worn device). The brain control interface <NUM> (BCI) is configured to detect brain signals indicative of brain activity of a user <NUM> in an environment <NUM>. The system <NUM> further comprises or more processors <NUM> configured to analyze the brain signals. The BCI <NUM> may comprise one or more electrodes <NUM> in contact with the user's scalp, which electrodes <NUM> are used for detecting EEG signals of the user. It should be understood that such a BCI <NUM> is an example, and that other types of brain signal detection may be used.

The brain control interface system <NUM> further comprises an input <NUM> configured to obtain data indicative of a current light scene of one or more lighting devices <NUM>, <NUM> in the environment <NUM>. The input <NUM> may be an input to the one or more processors <NUM> configured to obtain the data indicative of the current light scene of one or more lighting devices <NUM>, <NUM> from a memory, for instance memory <NUM>. Alternatively, the input <NUM> may be configured to obtain the data indicative of the current light scene by obtaining sensor data from a light sensor located in the environment <NUM>. Alternatively, the input <NUM> may be a receiver configured to (wirelessly) receive the data indicative of the current light scene, for instance from a lighting controller <NUM> such as a central home/office control system, from a remote lighting controller connected to the one or more lighting devices <NUM>, <NUM> via the cloud, etc. The lighting controller <NUM> may be configured to control the one or more lighting devices <NUM>, <NUM> by communicating lighting control signals to the one or more lighting devices <NUM>, <NUM> (e.g. via Zigbee, BLE, Ethernet, etc.) to generate the light scene. The control signals comprise light settings indicative of light output properties (for example hue, saturation, brightness, beam direction, etc.). The one or more lighting devices <NUM>, <NUM> are configured to receive the control signals and a driver is configured to adjust the light output of one or more (LED) light sources accordingly.

The brain control interface system <NUM> further comprises one or more processors <NUM> (e.g. circuitry, one or more microcontrollers, etc.). The one or more processors <NUM> are configured to obtain data indicative of the brain signals as detected by the BCI <NUM>. The one or more processors <NUM> may be comprised in a single device or distributed across multiple devices, which may depend on the system architecture of the BCI system <NUM>. For instance, in the example of <FIG>, the one or more processors <NUM> and the input <NUM> are comprised in a single device <NUM>, which device <NUM> is communicatively coupled with the lighting controller <NUM>, the BCI <NUM> and the controllable device <NUM>. It should be understood that this system architecture is merely an example, and that the skilled person is able to design alternative system architectures without departing from the scope of the appended claims. For instance, a first processor of the one or more processors <NUM> may be comprised in the BCI <NUM>, and a second processor on a remote server or in the lighting controller <NUM>. In another example, the one or more processors <NUM> and the input <NUM> may be comprised in the lighting controller <NUM>. In another example, a first processor of the one or more processors <NUM> may be comprised in a remote server and a second processor in the lighting controller <NUM>. In yet another example, one or more of the system components <NUM>, <NUM> may be comprised in the BCI <NUM>, or in the controllable device <NUM>.

The one or more processors <NUM> are configured to analyze the brain signals to identify a level of noise in the brain signals when the current light scene is active. Light impacting brain signals may originate from artificial lighting and/or natural daylight. The natural daylight present in the room may depend on the time of the day and/or the current position of window blinds. The one or more processors <NUM> may, for example, compare the detected brain signals to reference brain signals to determine the level of noise based on the differences between the detected brain signals and the reference brain signals. Additionally or alternatively, the one or more processors <NUM> may compare the detected brain signals with one or more thresholds and/or baselines to determine level of noise. In another example, the one or more processors <NUM> may be configured to obtain the data indicative of a current light scene of one or more lighting devices in the environment, and to determine which brain signals may be affected by the current light scene. For instance, dynamic effects, and more specifically dynamic effects with higher dynamics levels, may affect brain signals originating from certain regions of the brain, and the one or more processors <NUM> may be configured to analyze these regions for noise to determine the level of noise. Certain colors of light may also affect the brain signals. For instance, short wavelength light, such as blue light, impacts the occipital region of the brain and decreases the power of the alpha EEG rhythm in this specific part of the brain. Similarly, long wavelength light, such as red light, affects the beta EEG rhythm. The one or more processors <NUM> may be configured to analyze a specific region associated with that light scene for noise to determine the level of noise.

The one or more processors <NUM> are further configured to determine if the level of noise exceeds a threshold, and if so, adjust the light scene (by controlling the one or more lighting devices <NUM>, <NUM> via the lighting controller <NUM>) while monitoring the level of noise until a target level of noise in the brain signals has been established. The one or more processors <NUM> may, for example, change the color, the brightness or a level of dynamics of the light. Certain light effects affect the brain signals less compared to others, and the one or more processors <NUM> may be configured to adjust the light scene such that the adjusted light scene reduces the effect on the brain signals. The one or more processors <NUM> may, for example, increase the intensity of the light scene, adjust the color point of the light scene (e.g. towards more blueish light), reduce the level of dynamics of a dynamic light scene, etc. The one or more processors <NUM> may, for example, control the one or more lighting devices <NUM>, <NUM> and gradually adjust the light scene until the target level of noise in the brain signals has been established. The one or more processors <NUM> may for example gradually change the light scene from the current light scene to a target light scene. Alternatively, the one or more processors may be configured to iteratively adjust the light scene while monitoring the level of noise until the target level of noise in the brain signals has been established. In other words, the one or more processors <NUM> may control the one or more lighting devices <NUM>, <NUM> sequentially according to different light scenes until a light scene is active for which the level of noise in the brain signals is below the threshold.

<FIG> shows a graph schematically illustrating brain signals s captured over time t, and a threshold value th. The one or more processors <NUM> may determine that the intensity i of the brain signals s exceed threshold th, which may be indicative of that the level of noise exceeds a threshold. Based thereon, the one or more processors <NUM> may adjust the active light scene to reduce the level of noise, resulting in brain signals s that do not exceed the threshold th as illustrated in <FIG>. Alternatively, not depicted, the threshold may comprise both an upper threshold and a lower threshold, wherein the brain signals no longer exceed the threshold if the brain signals no longer exceed the upper threshold and the lower threshold.

<FIG> shows another example of a graph, wherein a set of brain signals are detected. Letters A-E indicate different brain regions, and the length of the bars indicates the level of (change in) brain activity for the different brain regions. Each brain signal A-E may correspond to an electrode positioned on the user's scalp. The one or more processors <NUM> may determine that the intensity of the brain signals C exceed threshold th1, which may be indicative of that the level of noise for brain region C exceeds a threshold. Based thereon, the one or more processors <NUM> may adjust the active light scene to reduce the level of noise, resulting in brain signals A-E of <FIG>, which do not exceed the threshold th1.

The one or more processors <NUM> may be configured to adjust the light scene towards a target light scene until the target level of noise in the brain signals has been established. The target light scene may be a predefined light scene. The predefined light scene may comprise light output characteristics which reduce the level of noise in detected brain signals. By gradually adjusting the light scene towards the target light scene, the original (current) light scene is maintained as much as possible while reducing the level of noise. The target light scene may comprise at least one of the following characteristics: a higher intensity compared to the light scene, a lower level of dynamics compared to the light scene, and a spectrum comprising more blue light compared to the light scene. These characteristics are examples of characteristics that affect (reduce) the level of noise. If, for example, the current light scene is a reddish light scene, the one or more processors <NUM> may adjust the light scene towards a blueish target light scene until the target level of noise in the brain signals has been established. If during the adjustment (transition) towards the blueish light scene the target level of noise has been established, the one or more processors <NUM> may cease the adjustment towards the target blueish light scene. In another example, the current light scene may be a light scene with low brightness (e.g. <NUM>%), and the one or more processors <NUM> may adjust the light scene towards a light scene with a higher brightness (e.g. <NUM>%) until the target level of noise in the brain signals has been established. If at a certain brightness (e.g. <NUM>%), the target level of noise has been established, the one or more processors <NUM> may cease the adjustment towards the target light scene with the higher brightness (e.g. <NUM>%). In another example, the current light scene may be a dynamic light scene with a (high) level of dynamics, and the one or more processors <NUM> may adjust the light scene towards a target dynamic light scene with a lower level of dynamics until the target level of noise in the brain signals has been established. If at a certain level of dynamics the target level of noise has been established, the one or more processors <NUM> may cease the adjustment towards the target light scene with the lower level of dynamics.

The one or more processors <NUM> may be configured to analyze the brain signals to derive a control command from the brain signals for a controllable device <NUM>. The controllable device may be a lighting device of the one or more lighting devices. Alternatively, the controllable device <NUM> may, for example, be a connected (home) appliance or connected (office) equipment. The one or more processors <NUM> may be further configured to switch to a brain control mode when the target level of noise in the brain signals has been established, wherein, in the brain control mode, the one or more processors <NUM> are configured to control the controllable device <NUM> based on the derived control command. The one or more processors <NUM> may only control (either directly or indirectly) the controllable device <NUM> based on the derived control command only when the brain control mode is active.

The one or more processors <NUM> may be further configured to select the target level of noise based on an expected control command that will be provided by the user. For certain control commands a lower level of noise may be required, which may be necessary to distinguish between different control commands, which different control commands may be control commands for the same device/service. The one or more processors <NUM> may thus be configured to obtain an expected control command and select the target level of noise based on the expected control command. The expected control command may be obtained from a memory, or for example from a machine learning system that has learned which control commands have been provided over time.

Additionally or alternatively, the one or more processors <NUM> may be configured to analyze the brain signals to derive information about the user, for instance about the mental/emotional state of the user. The current light scene may affect the brain signals such that the level of noise is too high for the one or more processors <NUM> to derive this information. If the one or more processors <NUM> determine that the level of noise exceeds the threshold, the one or more processors may control the one or more lighting devices <NUM>, <NUM> to adjust the light scene to reduce the noise, and to properly derive the information about the user.

The current light scene may be provided by a plurality of lighting devices <NUM>, <NUM>. The one or more processors <NUM> may be configured to obtain position and/or orientation information indicative of a position and/or an orientation of the user <NUM> relative to the plurality of lighting devices <NUM>, <NUM>. The one or more processors <NUM> may be configured to select one or more of the plurality of lighting devices based on the position and/or the orientation of the user <NUM> relative to the plurality of lighting devices <NUM>, <NUM>, and adjust the light scene by (only) adjusting the light output of the one or more selected lighting devices <NUM>, <NUM>. The positions of the plurality of lighting devices <NUM>, <NUM>, <NUM>, <NUM> relative to the user may be obtained (e.g. via the input) from an (indoor) positioning system, for instance an RF-based positioning system, a coded light positioning system, a camera-based positioning system, from an internal memory, etc. Alternatively, the positions of the plurality of lighting devices <NUM>, <NUM>, <NUM>, <NUM> may be defined by a user via a user interface, wherein the user may provide information about the positions of the plurality of lighting devices <NUM>, <NUM>, for instance by positioning virtual counterparts of the lighting devices on a map of an environment wherein the lighting devices are located. The user may further indicate a typical user position (and orientation) on the map. Such techniques for determining locations of lighting devices in an environment relative to a user are known in the art and will therefore not be discussed in detail. The one or more processors <NUM> may be configured to select one or more lighting devices which are located (or of which their light effect is located) in the field of view of the user based on the position and/or the orientation of the user relative to the plurality of lighting devices <NUM>, <NUM>. When multiple users would be present in the environment, the light scene for the specific user may be adjusted while minimizing the effect of the change of the light scene on other users.

<FIG> shows schematically an example of a system wherein lighting devices <NUM>, <NUM>, <NUM>, <NUM> are selected based on the position and/or the orientation of the user <NUM> relative to the plurality of lighting devices <NUM>, <NUM>, <NUM>, <NUM>. The one or more processors <NUM> may for example determine that lighting devices <NUM> and <NUM> are located closer to the user <NUM> (and within the field of view of the user) compared to lighting devices <NUM> and <NUM>. Based thereon, the one or more processors <NUM> may select lighting devices <NUM> and <NUM> and adjust the light scene by adjusting the light output of the one or more selected lighting devices <NUM> and <NUM>. The light output of non-selected lighting devices <NUM> and <NUM> may be maintained or adjusted to a lesser extent compared to the adjustment of the light output of the selected lighting devices <NUM> and <NUM>.

The one or more processors <NUM> may be configured to initiate identifying the noise levels and adjusting the light scene upon receiving a control input indicative of that the user will provide a control command. The one or more processors <NUM> may thus monitor the noise level only when required when the user intends to adjust to provide the control command. The control input may be provided by a user, by a sensor or by the brain control interface. The control input may, for example, be received from the user, be received from a user interface (e.g. a user interface of an augmented reality device, a voice assistant, a smart device, etc.). Alternatively, a (remote or local) sensor may detect that the user is about to provide a control command. Alternatively, the BCI may determine, based on detected brain signals, that a user is in the process of or planning to generate the control command. Alternatively, the one or more processors may access a memory storing moments in time when the user typically provides a control command, and initiate identifying the noise levels and adjusting the light scene at these moments in time.

<FIG> shows schematically an example of a method of adjusting a light scene. The method <NUM> comprises detecting <NUM>, by a brain control interface, brain signals indicative of brain activity of a user in an environment, obtaining <NUM> data indicative of a current light scene of one or more lighting devices in the environment, analyzing <NUM> the brain signals to identify a level of noise in the brain signals when the current light scene is active, and, if the level of noise exceeds a threshold, adjusting <NUM> the light scene while monitoring the level of noise until a target level of noise in the brain signals has been established.

The method <NUM> may be executed by computer program code of a computer program product when the computer program product is run on a processing unit of a computing device, such as the one or more of the one or more processors <NUM> of the system <NUM>.

Claim 1:
A brain control interface system (<NUM>), comprising:
- a brain control interface (<NUM>) configured to detect brain signals indicative of brain activity of a user (<NUM>) in an environment (<NUM>),
- an input (<NUM>) configured to obtain data indicative of a current light scene of one or more lighting devices (<NUM>, <NUM>) in the environment (<NUM>),
- a lighting controller (<NUM>) configured to control the one or more lighting devices (<NUM>, <NUM>), and
- one or more processors (<NUM>) configured to analyze the brain signals to identify a level of noise in the brain signals when the current light scene is active, and, if the level of noise exceeds a threshold, adjust the light scene while monitoring the level of noise until a target level of noise in the brain signals has been established.