Patent ID: 12243280

Corresponding elements in the drawings are denoted by the same reference numeral.

DETAILED DESCRIPTION OF THE EMBODIMENTS

FIG.1shows a first embodiment of the system for controlling a light source based on an analysis of image content in an analysis region of the image content while a display displays the image content: an HDMI module1. The HDMI module1may be a Hue Play HDMI Sync Box, for example. In the example ofFIG.1, the image content is rendered on a display23. Alternatively, the image content may be rendered on multiple displays, e.g. a video wall.

In the example ofFIG.1, The HDMI module1can control the lighting devices13-15via a bridge19. The bridge19may be a Hue bridge, for example. The bridge19communicates with the lighting devices13-15, e.g. using Zigbee technology. The HDMI module1is connected to a wireless LAN access point21, e.g. via Wi-Fi. The bridge19is also connected to the wireless LAN access point21, e.g. via Wi-Fi or Ethernet.

Alternatively or additionally, the HDMI module1may be able to communicate directly with the bridge19, e.g. using Zigbee technology, and/or may be able to communicate with the bridge19via the Internet/cloud. Alternatively or additionally, the HDMI module1may be able to control the lighting devices13-15without a bridge, e.g. directly via Wi-Fi, Bluetooth or Zigbee or via the Internet/cloud.

The wireless LAN access point21is connected to the Internet25. A media server27is also connected to the Internet25. Media server27may be a server of a video-on-demand service such as Netflix, Amazon Prime Video, Hulu, Disney+ or Apple TV+, for example. The HDMI module1is connected to the display23and local media receivers31and32via HDMI. The local media receivers31and32may comprise one or more streaming or content generation devices, e.g. an Apple TV, Microsoft Xbox One and/or Sony PlayStation 4, and/or one or more cable or satellite TV receivers.

In the example ofFIG.1, the lighting devices13and14are vertically arranged arrays of light sources, like e.g. the Philips Hue Signe, and the lighting device15is a horizontally arranged array of light sources, e.g. a horizontally placed light strip. The lighting device13comprises four light sources (pixels)41-44and one distance sensor67, the lighting device14comprises four light sources (pixels)46-49and no distance sensor, and the lighting device15comprises five light sources (pixels)61-65and two distance sensors68and69. The distance sensors67-69may comprise one or more infrared distance sensors and/or one or more ultrasonic distance sensors, for example.

The lighting devices13-15are also referred to as pixelated lighting devices. In practice, a pixelated lighting device comprises more than four or five pixels. In the example ofFIG.1, each light source (pixel) comprises two light elements, e.g. LEDs, which are not able to render light effects that are not the same. Light sources41-44,46-49, and61-65are also referred to as individually addressable segments of light elements.

The HDMI module1comprises a receiver3, a transmitter4, a processor5, and memory7. The processor5is configured to obtain video content via the receiver3, e.g. from media receiver31or32, obtain a distance between the light sources of each lighting device and a surface, e.g. from one or more of sensors67-69or from a user device29, and determine, for each light source, a size and/or location for an analysis region associated with the light source based on the distance. User device29may be a mobile phone or a tablet, for example.

The processor5is further configured to determine a characteristic of the video content by analyzing the video content in the analysis region, determine a color and/or intensity for a light effect based on the characteristic, and control, via the transmitter4, the light sources41-44,46-49, and61-65to render the light effects. Said characteristic may e.g. comprise or be a color, e.g. a pixel value, and/or e.g. an intensity.

In the example ofFIG.1, a distance between the light sources and a surface is known for each for each of lighting devices13-15and the analysis regions can therefore be determined based on these distances. Analysis regions with distance-independent sizes and locations may be used for light sources of which no distance between the light sources and a surface is known.

In the example ofFIG.1, two device distances are obtained from the lighting device15and one device distance is obtained from the lighting device13. A device distance is a distance between the lighting device and a surface on which the light is projected, e.g. a wall. A second device distance may be obtained for lighting device13from user device29. Alternatively, the single device distance obtained from the lighting device13may be considered to represent the distance between all light sources of the lighting device13and the surface, or, if the top of the lighting device13is intended to lean against the surface, a second device distance may be determined to be zero. One or more device distances for lighting device14are obtained from the user device29.

If only a single device distance is obtained for a lighting device, the distance between the surface and each light source of the lighting device is assumed to be this single device distance. If multiple device distances are obtained for a lighting device, the distance between the surface and each light source of the lighting device may be an average of the device distances or the distance between a light source and the surface may be determined based on the device distances and the position of the light source on the lighting device.

As an example of the former, if the distance sensor68measures a distance of 30 cm and the distance sensor69measures a distance of 50 cm, the distance between the surface and each light source of lighting device15may be considered to be 40 cm. As an example of the latter, if the distance sensor68measures a distance of 30 cm and the distance sensor69measures a distance of 50 cm, the distance between light sources61-65and the surface may be considered to be 30, 35, 40, 45, and 50 cm, respectively.

If the distance between each light source of a lighting device and the surface is assumed to be the same, then it is not necessary to determine a distance per light source after determining the (average) device distance. The processor5may then determine the sizes and/or locations for the analysis regions directly based on the (average) device distance.

In the embodiment of the HDMI module1shown inFIG.1, the HDMI module1comprises one processor5. In an alternative embodiment, the HDMI module1comprises multiple processors. The processor5of the HDMI module1may be a general-purpose processor, e.g. ARM-based, or an application-specific processor. The processor5of the HDMI module1may run a Unix-based operating system for example. The memory7may comprise one or more memory units. The memory7may comprise solid-state memory, for example.

The receiver3and the transmitter4may use one or more wired or wireless communication technologies such as Zigbee to communicate with the bridge19and HDMI to communicate with the display23and with local media receivers31and32, for example. In an alternative embodiment, multiple receivers and/or multiple transmitters are used instead of a single receiver and a single transmitter. In the embodiment shown inFIG.1, a separate receiver and a separate transmitter are used. In an alternative embodiment, the receiver3and the transmitter4are combined into a transceiver. The HDMI module1may comprise other components typical for a network device such as a power connector. The invention may be implemented using a computer program running on one or more processors.

In the embodiment ofFIG.1, the system of the invention is an HDMI module. In an alternative embodiment, the system may be another device, e.g. a mobile device, laptop, personal computer, a bridge, a media rendering device, a streaming device, or an Internet server. In the embodiment ofFIG.1, the system of the invention comprises a single device. In an alternative embodiment, the system comprises multiple devices.

The analysis of the video content may be performed in real-time, i.e. just before the light sources are controlled and the video content is displayed. Alternatively, the analysis of the video content may be performed earlier, e.g. by using automatic light scripting. Automatic light scripting may be performed by the above-mentioned Internet server, for example. In automatic light scripting, analysis of the video content is done prior to the user watching/streaming it (it could also be done near real-time with e.g. a buffer of 5 minutes), typically in the cloud. This may be used to ensure a perfect synchronization between content and light effects.

A processing system running in the cloud could use a user profile, indicating the distance to the surface amongst others, to generate a personalized script. Alternatively, the system might pre-generate a set of scripts for a few common distances (e.g. 0-5 cm, 5-10 cm and more than 10 cm), and when the user starts streaming a movie, the system could then choose the script that is closest to the user's setup. The latter will save resources in the cloud when popular movies are streamed.

FIG.2shows a second embodiment of the system for controlling a light source based on an analysis of image content in an analysis region of the image content while a display displays the image content: a mobile device51. The mobile device51may be a smart phone or a tablet, for example. The lighting devices13-15can be controlled by the mobile device51via the bridge19. The mobile device51is connected to the wireless LAN access point21, e.g. via Wi-Fi.

The mobile device51comprises a receiver53a transmitter54, a processor55, a memory57, and a display59. The image content is preferably displayed on the external display23but could also be displayed on display59of the mobile device51. The processor55is configured to obtain video content via the receiver53, e.g. from the media server27, and obtain a distance between the light sources of each lighting device and a surface, e.g. from one or more of sensors67-69, seeFIG.1, or from an input interface (e.g. a touchscreen display or a microphone) of the mobile device51itself,

The processor55is further configured to determine, for each light source, a size and/or location for an analysis region associated with the light source based on the distance, determine a characteristic of the video content by analyzing the video content in the analysis region, determine a color and/or intensity for a light effect based on the characteristic, and control, via the transmitter54, the light sources41-44,46-49, and61-65to render the light effects. Alternatively, said “determine a color and/or intensity for a light effect” may be phrased as “determine a light effect”, wherein said light effect may comprise a color and/or intensity.

In the embodiment of the mobile device51shown inFIG.2, the mobile device51comprises one processor55. In an alternative embodiment, the mobile device51comprises multiple processors. The processor55of the mobile device51may be a general-purpose processor, e.g. from ARM or Qualcomm or an application-specific processor. The processor55of the mobile device51may run an Android or iOS operating system for example. The display59may be a touchscreen display, for example. The display59may comprise an LCD or OLED display panel, for example. The memory57may comprise one or more memory units. The memory57may comprise solid state memory, for example.

The receiver53and the transmitter54may use one or more wireless communication technologies such as Wi-Fi (IEEE 802.11) to communicate with the wireless LAN access point21, for example. In an alternative embodiment, multiple receivers and/or multiple transmitters are used instead of a single receiver and a single transmitter. In the embodiment shown inFIG.2, a separate receiver and a separate transmitter are used. In an alternative embodiment, the receiver53and the transmitter54are combined into a transceiver. The mobile device51may further comprise a camera (not shown). This camera may comprise a CMOS or CCD sensor, for example. The mobile device51may comprise other components typical for a mobile device such as a battery and a power connector. The invention may be implemented using a computer program running on one or more processors.

In the embodiment ofFIG.2, the lighting devices13-15are controlled via the bridge19. In an alternative embodiment, one or more of the lighting devices13-15is controlled without a bridge, e.g. directly via Bluetooth. In the embodiment ofFIGS.1and2, the system of the invention comprises only local devices. In an alternative embodiment, the system of the invention comprises one or more Internet/cloud servers.

FIG.3shows an example of the lighting device15ofFIGS.1and2having a first distance71to a wall81. The lighting device15has been attached to the back of display23ofFIG.1.FIG.4shows an example of the lighting device15having a second distance72to the wall81.FIG.5shows an example of the lighting device15having a third distance73to the wall81. The first distance71is shorter than the second distance72. The second distance72is shorter than the third distance73.

FIG.6depicts an example of a space in which the system ofFIG.1is used. A floor91of a home comprises a hallway93, a kitchen94and a living room95. Lighting devices13-15have been installed in the living room65. Vertically arranged lighting devices13and14have been placed on respectively the left and right side of the display23, which may be a TV, for example. Horizontally arranged lighting device15has been attached to the back of display23.

The wireless LAN access point21has been installed in the hallway93. The HDMI module1has been installed next to the display23in the living room65. The bridge19has been installed in the living room65near the wireless LAN access point21. A person99is watching TV. The lighting device15has a second distance72to the wall, as shown inFIG.4. The lighting device13has distance76to the display23. The lighting device14has a distance77to the display23.

A first embodiment of the method of controlling a light source based on an analysis of image content in an analysis region of the image content while a display displays the image content is shown inFIG.7. A step201comprises obtaining video content.

A step202comprises obtaining lighting device information relating to the lighting devices. The lighting device information may comprise, per lighting device, the width of a horizontally arranged lighting device or the height of a vertically arranged lighting device and the size of the light sources/pixels, for example.

The lighting device information relating to a certain lighting device may be obtained, for example, when a user adds this lighting device to his lighting system. For example, when a new lighting device is added, an HDMI module, mobile device or bridge may download the lighting device information from the lighting device and store it. When this information is stored on a bridge, an HDMI module or mobile device may be able to later obtain this information from the bridge.

In a simpler embodiment, step202may be omitted. In a more advanced embodiment, the lighting device information may comprise further information about the lighting device, for example the pixel density (e.g. number of light pixels per meter), the pixel spacing (e.g. the distances between the centers of two individual pixels) and/or indicate optical properties of the pixels (e.g. the beam width and beam angle of the pixelated light sources).

A step203comprises obtaining a distance between each light source, i.e. each pixel, of the lighting device and a surface, e.g. a wall. Typically, one or more device distances are obtained from a sensor or from a user device. The distance between each light source of the lighting device and the surface may be the same. Alternatively, a different distance per light source may be determined based on the obtained one or more device distances.

The distance between the pixelated light sources and the to be illuminated area, i.e. the surface, can be determined in various ways. In a simple embodiment, the user provides input on the distance for example via a smart phone UI. This can be an input of a numeric value or an approximation e.g. by selecting an icon on how the TV is installed (wall-mount vs standing). In a more advanced embodiment, the distance between the light source and the to be illuminated area, i.e. the surface, is determined automatically via an external sensor, e.g. by using one or more sensors (e.g. time-of-flight sensors) embedded in or attached to the lighting device or by analyzing a (depth) image from the TV setup as captured by a smartphone camera.

A step205comprises determining, for each light source, a size and/or location for an analysis region associated with the light source based on the distance. A first analysis region with a first size and a first location is determined in step205for a first light source when the distance has a first value and a second analysis region with a second size and a second location is determined in step205for this first light source when the distance has a second value.

The first analysis region has a larger overlap with adjacent analysis regions than the second analysis region and the first value is smaller than the second value. The second size is different than the first size and/or the second location is different than the first location. Although it may be possible to create a larger overlap by only changing the location of at least one of the adjacent analysis regions, it may be easier to achieve this by (also) using a larger analysis region.

In the embodiment ofFIG.7, the formula C=p+b/(0.1*d+1) is used to calculate the size of the analysis region for a pixel in centimeters when d is smaller than a certain threshold T, C being the size of the analysis region in centimeters, b being a blending factor, d being an (estimated or measured) numeric distance value in centimeters and p being the size of the pixel in centimeters. A minimum analysis region size is used when d is greater than or equal to threshold T.

A pixel may comprise multiple light elements, e.g. multiple LEDs. A pixel of 6 centimeters may comprise, for example, 6 LED packages, one placed each centimeter of the lighting device (e.g. light strip). All 6 LEDs are controlled in one color. The 6 LEDs may comprise three RGB LEDs and three white LEDs, for example.

For example, for horizontally arranged pixels with a blending factor of 6, a pixel width of 6 cm and a distance of 50 cm to the wall, the width of a color analysis region could be calculated as follows: 6+6/(0.1*50+1)=7 cm. If the TV would be placed against the wall (0 cm), the width of the color analysis region would be 12 cm. This would mean that it would take 50% of both adjacent pixels. When put 10 cm from the wall, the color analysis region would be 9 cm wide (taking 25% of both adjacent pixels). In this example, the threshold T is larger than 50 cm. If the threshold T is lower than 50 cm and the distance is 50 cm to the wall, the width of the color analysis region would be determined to be 6 cm.

For vertically arranged pixels, the height of the color analysis region may be determined in the manner described above in relation to the width of color analysis region for horizontally arranged pixels.

The color analysis region size in pixels may be determined by multiplying the width of the color analysis region size in centimeters with the amount of horizontal pixels of the video content divided by the width of the horizontally arranged lighting device in centimeters or multiplying the height of the color analysis region size in centimeters with the amount of vertical pixels of the video content divided by the height of the vertically arranged lighting device in centimeters.

For horizontally arranged pixels, the height of the color analysis region may be determined independent of the distance to the wall, e.g. may be a fixed value, or may be determined based on the width of the color analysis region, for example. For vertically arranged pixels, the width of the color analysis region may be determined independent of the distance to the wall, e.g. may be a fixed value, or may be determined based on the height of the color analysis region, for example

The horizontally and vertically arranged pixels may be part of light strips attached the display device or may be part of a lighting device located a bit farther from the display device, e.g. a floor standing luminaire like Hue Signe that is put near a wall and joins the entertainment experience.

In a variant on this embodiment, the ratio between pixel (light source) size and display size is further used as parameter. For instance, the above-mentioned formula may be modified such that C is a function of p (size of a pixel), d (distance to the surface) and r (ratio between display size and pixel size). In a first implementation, the larger the size of the pixel relative to the size of the display, the smaller the overlap percentage. For example, with a distance of 5 cm to a wall, for a TV with a 55-inch display (55 inch being the diagonal dimension), the overlap between adjacent analysis regions might be 40% for a 6.25 cm pixel and 20% for a 12.5 cm pixel and for a TV with a 75-inch display, the overlap between adjacent analysis regions might be 50% for a 6.25 cm pixel and 25% for a 12.5 cm pixel. Information indicating the display size may be obtained in step202or in a separate step performed before, after, or (partly) in parallel with step202, for example.

In a simpler embodiment, the determination of the color analysis region size may be a simple binary decision. If the TV (with attached or embedded light sources) is mounted on a stand, the color analysis region is one-to-one mapped to the pixel. So, if a pixel is 6 cm wide, the color analysis region has a corresponding width. If the TV is mounted against the wall, the color analysis region is enlarged and takes 50% of the color analysis region of both adjacent pixels.

In a slightly less simple embodiment, a mapping table may be used, where the percentage of overlap is given for multiple ranges of distances to the surface. For example, a distance between 0 and 5 cm may be mapped to an overlap percentage of 50%, a distance between 5 and 10 cm may be mapped to an overlap percentage of 25%, etc. The size of the analysis region may then be determined based on the size of the pixel and the determined overlap percentage.

In a more advanced embodiment, a more complex function is used. Furthermore, the blend factor b may be variable and selected by a user or by the system, for example based on type of content.

A step206comprises checking whether there is a further lighting device to be controlled based on an analysis of video content for which the analysis region(s) have not been determined yet. If so, steps203and205are repeated for this further lighting device. If not, a step207is performed. Step201may be performed in parallel with at least part of one or more of steps202-206, before step202is performed or between steps206and207, for example.

Step207comprises determining a characteristic of the video content by analyzing the video content, typically a video frame of the video content, in an analysis region associated with one of the light sources. In the embodiment ofFIG.7, a color is extracted from the video content in the analysis region. Various color extraction methods can be used, such as taking the average or trimean color in this region. The color extraction method could also vary depending on the amount of overall and absolute size of the analysis regions. For example, for smaller regions with no overlap, taking the average might work best (a stable but desaturating method), while for larger overlapping regions, a trimean method might work best (resulting in less stable but more saturated colors).

If desired, the size and/or locations of the color analysis region determined in step203may be adjusted in step207, e.g. based on color and/or brightness contrast and/or the number of edges in the content. For example, if a video frame contains high contrasting elements that are aligned with light pixels, the analysis regions could be reduced even in the case of a short distance to the wall. Similarly, if content is already very smooth, overlapping regions will not be beneficial and will only result in desaturation of the light effects. This analysis can be done per pixel and thereby allows an overlap to be reduced for some pixels and to be increased for other pixels. However, this will normally only be possible if the system can analyze the content fast enough, as it will need to be done every video frame.

A step209comprises determining a light effect to be rendered on this light source based on the characteristic. If a color is extracted in step207, this color may be used as color for the light effect to be rendered on the light source.

A step210comprises checking whether there is a further light source that has been associated with an analysis region and for which no light effect has been determined yet. If so, steps207and209are repeated for this further light source. If not, a step211is performed. Step211comprises controlling the light sources to render the light effects determined in step209by transmitting a light control command specifying one of the light effects to a corresponding light source, either directly or to the lighting device that comprises the light source.

A step212comprises checking whether the end of the video content has been reached. If not, steps207-211are repeated for the next part, e.g. next frame, of the video content.

FIG.8shows an example of a video frame101of the video content being rendered, e.g. on display23ofFIG.1.FIG.9shows examples of analysis regions111-121that may be used to extract characteristics from video frame101. In this example, multiple regions of the screen are mapped to different lighting devices and each analysis region is analyzed separately, e.g. an average color is extracted from each analysis regions. For example, the analysis regions111to114may be mapped to pixels41-44of lighting device13ofFIG.1, the analysis regions114to118may be mapped to pixels61-65of lighting device ofFIG.1, and the analysis regions118to121may be mapped to pixels46-49of lighting device14ofFIG.1.

FIG.10shows further examples of analysis regions that may be used to extract characteristics from video frame101. In this example, the analysis regions131to134may be mapped to pixels41-44of lighting device13ofFIG.1, the analysis regions134to138may be mapped to pixels61-65of lighting device15ofFIG.1, and the analysis regions138to141may be mapped to pixels46-49of lighting device14ofFIG.1.

The analysis regions131-141ofFIG.10are larger than the analysis regions111-121ofFIG.9and as a result, there is overlap between adjacent analysis regions inFIG.10, while there is no such overlap between adjacent analysis regions inFIG.9. Such an overlap is beneficial if the distance between the pixels of the pixelated lighting device and the surface, e.g. wall, is relatively small.

In the example ofFIGS.9and10, each analysis region has the same size. However, it is also possible to use differently sized analysis regions, as shown inFIG.11. Of the analysis regions151-154at the left side of the video frame, analysis region151is largest and analysis region154is smallest.

FIG.11also demonstrates that it is possible to increase the size of an analysis regions without increasing the overlap between adjacent analysis regions. For example, if a pixelated lighting device is far enough from the surface so that overlap between adjacent analysis regions is not needed, the size of an analysis region may be still increased to focus the light effect more towards ambiance rather than focusing solely on the colors that are close to the side of screen where the light source is located. So, without changing the overlap, the size of the analysis region is changed such that it takes up a bigger part of the video frame without impacting adjacent analysis regions.

In the example ofFIG.11, the pixels/light sources of the lighting device have different distances to the surface. For example, a pixelated light device may lean against a wall, i.e. has an angle with respect to the wall, may be attached to the back of a curved display, or may be placed behind a display in a corner of two walls. In the example ofFIG.11, the light source associated with analysis region151is farthest from the surface and the light source associated with analysis region154is closest to the surface. The farther away the light source is from the wall, the larger effect it creates (on the wall). So, it may sometimes be beneficial to analyze a larger part of the video frame even if the distance to the wall is large such that the large effect on the wall, instead of reflecting only small part of the video frame on the side, reflects a larger part of the video frame.

In the example ofFIG.10, each pair of adjacent analysis regions has the same overlap. However, it is also possible to use different overlaps for different pairs of adjacent analysis regions, as is shown inFIG.12. Of the analysis regions161-164at the left side of the video frame, adjacent analysis regions163and164have the largest overlap and adjacent analysis regions161and162have the smallest overlap. This may be beneficial, for example, if the pixels/light sources of the lighting device have different distances to the surface.

By determining the distance to the surface for each light source individually, it becomes possible to use different overlaps for different pairs of adjacent analysis regions. In the example ofFIG.12, the light source associated with analysis region161is farthest from the surface and the light source associated with analysis region164is closest to the surface. In the example ofFIG.12, all analysis regions are of the same size but they stack on top of each other differently; the overlap area between the adjacent pixels that are farther from the wall is relatively small and the overlap area between the adjacent pixels that are closer to the wall is relatively large. Thus, only the position, and not the size, of the analysis region depends on the distance to the surface.

FIG.12also demonstrates that it is possible to change the overlap between two adjacent analysis areas without changing the total overlap between all analysis areas. When the angle of the pixelated lighting device with respect to the wall is increased and the distance between the light source associated with analysis region164is therefore increased, the overlap between adjacent analysis areas163and164may be decreased and the overlap between adjacent analysis areas161and162may be increased, thereby keeping the total overlap the same.

In the example ofFIG.12, there is still a small overlap between adjacent analysis areas161and162. However, it is also possible for analysis area161, associated with the light source farthest from the surface, not to overlap with the adjacent analysis area162at all. Moreover, the overlap between adjacent areas163and164may even be larger than shown inFIG.12.

In the examples ofFIGS.9to12, all analysis regions have a rectangle shape. It is also possible to use a different shape or different shapes, as is shown inFIG.13with respect to analysis regions171-174.

A part of a second embodiment of the method of controlling a light source based on an analysis of image content in an analysis region of the image content while a display displays the image content is shown inFIG.14. In this second embodiment, step203ofFIG.7is implemented by sub steps231-241. After step203, steps205-212are performed, as shown inFIG.7.

Step231comprises obtaining one or more device distances between the lighting device and the surface. Typically, these one or more device distances are obtained from a sensor or from a user device, e.g. a user input device. If one end of a vertically arranged lighting device needs to lean against the wall and a device distance is obtained for the other end, then a second device distance of zero may be obtained automatically.

Next, a step232comprises checking whether one device distance was obtained in step231or whether multiple device distances were obtained in step231. If is determined in step232that a single device distance was obtained in step231, a step233is performed next. Step233comprises determining the distances between the further light sources and the surface based on the single device distance determined in step232. Typically, the distances between the further light sources and the surface are equal to the single device distance. Step205is performed after step233.

If is determined in step232that multiple device distances were obtained in step231, a step235is performed next. Step235comprises calculating an average of the multiple device distances determined in step231. If two device distances, e.g. of the two ends of the lighting device, were determined in step231, then a single average is calculated in step235. If more than two device distances were determined in step231, then multiple averages may be calculated in step235. Next, a step237comprises determining the distances between the further light sources and the surface based on the average(s) calculated in step235. If a single average was calculated in step235, the distances between the further light sources and the surface are typically equal to this single average. Step205is performed after step237.

A part of a third embodiment of the method of controlling a light source based on an analysis of image content in an analysis region of the image content while a display displays the image content is shown inFIG.15. Compared to the second embodiment ofFIG.14, steps235and237have been replaced with steps251and253, steps255and257have been added before step205, and step205is implemented by a step259.

Step251comprises determining the positions of the light sources on the lighting device. Step253comprises determining the distance between each light source and the surface based on at least two of the device distances determined in step231and the position of the light source determined in step251. The device distances are indicated with respect to a reference point on the lighting device, e.g. the edges of the lighting device. If the position of a light source is between two reference points, the distance between this light source and the surface is determined based on the two device distances corresponding to these reference points and the distances between the position of the light source and these reference points.

Step255comprises estimating an amount of light overlap between light projected on the surface by adjacent light sources based on the distances determined in step203. Obtained lighting device information, see step202ofFIG.7, may be used to estimate the amount of overlap. For example, beam width, beam angle and distance to the wall may be used to more accurately calculate the amount of light overlap between the illumination regions of different pixels as projected on the wall. The beam width and beam angle normally have an effect on the optical mixing. For instance, more optical mixing occurs when using a wider beam and/or when the beam angle is sharper (due to the longer distance to the surface).

Step257comprises determining an amount of desired region overlap between the adjacent analysis regions of the adjacent light sources based on the estimated amount of light overlap. When more optical mixing occurs, the amount of desired region overlap may be lower. Step259comprises determining the sizes and/or locations for the analysis regions based on the amount of desired region overlap. Step206is performed after step259.

A fourth embodiment of the method of controlling a light source based on an analysis of image content in an analysis region of the image content while a display displays the image content is shown inFIG.16. In this fourth embodiment, additional steps271,273and/or275are optionally performed before step205and step205is implemented by a step277.

Step271comprises determining the size of the light source(s) of the lighting device. Step273comprises determining the size of the lighting device. Step275comprises determining a distance between the light source(s) and the display. Step277comprises determining the sizes and/or locations for the analysis regions based on the distances determined in step203and optionally based on the size of the light source(s) of the lighting device determined in step271, the size of the lighting device determined in step273, and/or the distance between the light source(s) and the display. Step206is performed after step277.

The embodiments ofFIGS.7and14to16differ from each other in multiple aspects, i.e. multiple steps have been added or replaced. In variations on these embodiments, only a subset of these steps is added or replaced and/or one or more steps is omitted. For example, one or more of steps271,273and275may be added to the embodiments ofFIG.14and/orFIG.15and steps255,257,259ofFIG.15may be added to the embodiment ofFIG.14and/or omitted from the embodiment ofFIG.15.

FIG.17depicts a block diagram illustrating an exemplary data processing system that may perform the method as described with reference toFIGS.7and14to16.

As shown inFIG.17, the data processing system300may include at least one processor302coupled to memory elements304through a system bus306. As such, the data processing system may store program code within memory elements304. Further, the processor302may execute the program code accessed from the memory elements304via a system bus306. In one aspect, the data processing system may be implemented as a computer that is suitable for storing and/or executing program code. It should be appreciated, however, that the data processing system300may be implemented in the form of any system including a processor and a memory that is capable of performing the functions described within this specification.

The memory elements304may include one or more physical memory devices such as, for example, local memory308and one or more bulk storage devices310. The local memory may refer to random access memory or other non-persistent memory device(s) generally used during actual execution of the program code. A bulk storage device may be implemented as a hard drive or other persistent data storage device. The processing system300may also include one or more cache memories (not shown) that provide temporary storage of at least some program code in order to reduce the quantity of times program code must be retrieved from the bulk storage device310during execution. The processing system300may also be able to use memory elements of another processing system, e.g. if the processing system300is part of a cloud-computing platform.

Input/output (I/O) devices depicted as an input device312and an output device314optionally can be coupled to the data processing system. Examples of input devices may include, but are not limited to, a keyboard, a pointing device such as a mouse, a microphone (e.g. for voice and/or speech recognition), or the like. Examples of output devices may include, but are not limited to, a monitor or a display, speakers, or the like. Input and/or output devices may be coupled to the data processing system either directly or through intervening I/O controllers.

In an embodiment, the input and the output devices may be implemented as a combined input/output device (illustrated inFIG.17with a dashed line surrounding the input device312and the output device314). An example of such a combined device is a touch sensitive display, also sometimes referred to as a “touch screen display” or simply “touch screen”. In such an embodiment, input to the device may be provided by a movement of a physical object, such as e.g. a stylus or a finger of a user, on or near the touch screen display.

A network adapter316may also be coupled to the data processing system to enable it to become coupled to other systems, computer systems, remote network devices, and/or remote storage devices through intervening private or public networks. The network adapter may comprise a data receiver for receiving data that is transmitted by said systems, devices and/or networks to the data processing system300, and a data transmitter for transmitting data from the data processing system300to said systems, devices and/or networks. Modems, cable modems, and Ethernet cards are examples of different types of network adapter that may be used with the data processing system300.

As pictured inFIG.17, the memory elements304may store an application318. In various embodiments, the application318may be stored in the local memory308, the one or more bulk storage devices310, or separate from the local memory and the bulk storage devices. It should be appreciated that the data processing system300may further execute an operating system (not shown inFIG.17) that can facilitate execution of the application318. The application318, being implemented in the form of executable program code, can be executed by the data processing system300, e.g., by the processor302. Responsive to executing the application, the data processing system300may be configured to perform one or more operations or method steps described herein.

Various embodiments of the invention may be implemented as a program product for use with a computer system, where the program(s) of the program product define functions of the embodiments (including the methods described herein). In one embodiment, the program(s) can be contained on a variety of non-transitory computer-readable storage media, where, as used herein, the expression “non-transitory computer readable storage media” comprises all computer-readable media, with the sole exception being a transitory, propagating signal. In another embodiment, the program(s) can be contained on a variety of transitory computer-readable storage media. Illustrative computer-readable storage media include, but are not limited to: (i) non-writable storage media (e.g., read-only memory devices within a computer such as CD-ROM disks readable by a CD-ROM drive, ROM chips or any type of solid-state non-volatile semiconductor memory) on which information is permanently stored; and (ii) writable storage media (e.g., flash memory, floppy disks within a diskette drive or hard-disk drive or any type of solid-state random-access semiconductor memory) on which alterable information is stored. The computer program may be run on the processor302described herein.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of all means or step plus function elements in the claims below are intended to include any structure, material, or act for performing the function in combination with other claimed elements as specifically claimed. The description of embodiments of the present invention has been presented for purposes of illustration, but is not intended to be exhaustive or limited to the implementations in the form disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the present invention. The embodiments were chosen and described in order to best explain the principles and some practical applications of the present invention, and to enable others of ordinary skill in the art to understand the present invention for various embodiments with various modifications as are suited to the particular use contemplated.