Patent Description:
Modern lighting fixtures often include additional features above and beyond their ability to provide light. For example, many lighting fixtures now include communications circuitry for sending and receiving commands to and from other devices, control circuitry for setting the light output thereof, and sensor circuitry for measuring one or more environmental parameters. Recently, lighting fixtures have begun to incorporate image sensors. Image sensors in lighting fixtures are generally expected to detect occupancy (i.e., the presence of a person) in the area within the field of view of the image sensor. While there are several well-known methods for determining occupancy using an image sensor, these methods are complex and computationally expensive. As a result, lighting fixtures utilizing an image sensor to detect occupancy must include relatively powerful processing circuitry, which consumes additional power and drives up the cost of the lighting fixture. Accordingly, there is a need for systems and methods for detecting occupancy using an image sensor with reduced complexity and computational expense.

<CIT> discloses an electronic apparatus that includes: an imaging section that has an imaging device and is capable of imaging a first region and a second region around the first region; and a control section to set a higher frame rate for imaging the second region than a frame rate for imaging the first region.

<CIT> discloses methods and systems for controlling lighting fixtures independently with image sensors. In an embodiment, a luminaire includes a controller, an image sensor and wide-angle lens component, and a light source. The controller receives detection signal data from the image sensor and wide-angle lens component when a user is within a detection area associated with a view angle of the wide angle lens, and then determines the position of the user. The controller then controls the illuminance of the light source based on the position of the user. In particular, illuminance is increased as the user approaches a center portion of the detection area, and decreased as the user moves away from the center portion.

<CIT> discloses an electronic apparatus including: a photographing part having a photographing element capable of coping with light fluxes imaged from a plural kinds of interchangeable photographic optical systems; and a control part that sets an area to be in an active state in a photographing area of the photographing element according to lens information concerning characteristics of an interchangeable lens part or the photographic optical systems.

<CIT> discloses systems and techniques to realize pointing and tracking applications with CMOS imaging devices. The technique includes: sampling multiple rows and multiple columns of an active pixel sensor array into a memory array (e.g., an on-chip memory array), and reading out the multiple rows and multiple columns sampled in the memory array to provide image data with reduced motion artifact.

According to the present invention, there is provided an intelligent lighting fixture comprising an image sensor and a solid state light source as set out in claim <NUM>.

According to the present invention, there is further provided a method for detecting occupancy by an intelligent lighting fixture as set out in claim <NUM>.

Futher aspects of the invention are defined by the dependent claims.

<FIG> shows an image sensor <NUM> according to one embodiment of the present disclosure. The image sensor <NUM> includes an active pixel array <NUM>, control circuitry <NUM>, a pixel selection circuitry <NUM>, sampling circuitry <NUM>, analog-to-digital converter circuitry <NUM>, an output register <NUM>, and an output <NUM>. The control circuitry <NUM> is coupled to each one of the pixel selection circuitry <NUM>, the sampling circuitry <NUM>, the analog-to-digital converter circuitry <NUM>, and the output register <NUM>. The pixel selection circuitry <NUM> is coupled to the active pixel array <NUM>. The sampling circuitry <NUM> is coupled between the active pixel array <NUM> and the analog-to-digital converter circuitry <NUM>. The analog-to-digital converter circuitry <NUM> is coupled to the output register <NUM>, which is in turn coupled to the output <NUM>.

In operation, the control circuitry <NUM> provides control signals to each one of the pixel selection circuitry <NUM>, the sampling circuitry <NUM>, the analog-to-digital circuitry <NUM>, and the output register <NUM> to facilitate capturing an image frame and providing a digitized version thereof at the output <NUM> of the image sensor <NUM>. The pixel selection circuitry <NUM> selects one or more pixels in the active pixel array <NUM> to be reset and/or read out. In a conventional rolling shutter read process, the pixel selection circuitry <NUM> serially selects rows of pixels in the active pixel array <NUM> to be reset and subsequently read out one after the other. Selected pixels provide analog signals proportional to an amount of light detected thereby to the sampling circuitry <NUM>. The analog-to-digital converter circuitry <NUM> digitizes the analog signals from the sampling circuitry <NUM> into pixel data and provides the pixel data to the output register <NUM>, where it can be retrieved via the output <NUM>.

<FIG> shows details of a pixel <NUM> in the active pixel array <NUM> according to one embodiment of the present disclosure. The pixel <NUM> includes a light detecting element <NUM> and support circuitry <NUM>. The light detecting element <NUM> may be a photodiode, photogate, or the like. The support circuitry <NUM> generally includes one or more switching devices such as transistors that reset and facilitate read out of the pixel <NUM>. One or more select signals provided to a select signal input <NUM> (from the pixel selection circuitry <NUM>) initiate reset and read out the pixel <NUM>. During a read operation, analog signals indicative of the amount of light detected by the pixel <NUM> are provided to a column bus output <NUM>, which is coupled to the sampling circuitry <NUM>.

For purposes of discussion herein, the pixel <NUM> generally operates in one of three states: idle, reset, and read out. In an idle state, photons that collide with the light detecting element <NUM> dislodge electrons that accumulate in a potential well of the light detecting element <NUM>. The number of electrons that accumulate in the potential well of the light detecting element <NUM> is proportional to the number of photons that contact the light detecting element <NUM>. In the idle state, the components of the support circuitry <NUM> remain off. Accordingly, the pixel <NUM> consumes minimal if any power and dissipates little if any heat in the idle state. The idle state is also referred to as an inactive state herein. During a reset operation, one or more reset switching components (e.g., transistors) in the support circuitry <NUM> flush out the electrons accumulated in the potential well of the light detecting element <NUM>. Some power is consumed by the one or more reset switching components and thus some heat is generated by the pixel <NUM> during the reset operation. During a read operation, one or more read out switching elements in the support circuitry are turned on to process and transfer the charge stored in the potential well of the light detecting element <NUM> (e.g., as a voltage or current) to the column bus output <NUM>. Some power is consumed by the one or more read out switching components and thus some heat is generated by the pixel <NUM> during the read operation.

Conventionally, all of the pixels in the active pixel array <NUM> are read out to provide a single image frame from the image sensor <NUM>. Generally, this is done as part of a rolling shutter readout, wherein every pixel in a row of pixels is reset, allowed to remain in an idle state for some amount of time (i.e., the integration time), then read out. This process repeats for each row of pixels until all of the pixels in the active pixel array <NUM> have been reset and subsequently read out. As the number of rows in an active pixel array <NUM> increases, the time to capture and read out a single image frame also increases. This may limit the number of image frames that can be captured in a given period of time, known as the frame rate of the image sensor. A limited frame rate may be problematic in some applications. Additionally, the resulting digitized image frame including pixel data for all of the pixels in the active pixel array <NUM> may be quite large. This may result in increased transfer time of the digitized image frame between the image sensor <NUM> and external processing circuitry (not shown), as such a transfer is often performed serially. Further, this may result in increased analysis time of the digitized image frame by said external processing circuitry, for example, to detect occupancy in the image frame or a set of image frames. Finally, as discussed above, every reset and read out of a pixel in the active pixel array <NUM> consumes power and dissipates heat. Over time, continually resetting and reading out every pixel in the active pixel array <NUM> may raise the temperature of the active pixel array <NUM>. As the temperature of the active pixel array <NUM> increases, the signal to noise ratio of each one of the pixels therein decreases due to thermal noise. This may make it difficult to analyze the resulting image frame or set of image frames, for example, to detect occupancy.

The inventors of the present disclosure discovered that it is highly inefficient and unnecessary to analyze the entirety of an image frame or set of image frames to detect a transition from an unoccupied state to an occupied state. This is because persons entering the field of view of an image sensor necessarily must first pass through one or more areas within the field of view before being present in other parts of the field of view. For example, for an image sensor in the middle of a room, a person must necessarily pass through an outer edge of the field of view before being present in the center of the field of view. As another example, for an image sensor located in a hallway where the field of view includes the entirety of the area between the two enclosing walls of the hallway, a person must necessarily pass through either the top or the bottom of the field of view before being present in the center of the field of view. As yet another example, for an image sensor with a field of view including the only door to a room and it is known that the room is empty (e.g., due to the absence of occupancy for a given period of time), a person must necessarily pass through the area of the field of view near the door before being present in any other part of the field of view.

Accordingly, <FIG> is a flow diagram illustrating a method for detecting occupancy using an image sensor according to one embodiment of the present disclosure. The method starts by obtaining pixel data for a subset of pixels in an active pixel array of an image sensor such that the pixels not in the subset remain inactive (step <NUM>). As discussed herein, when a pixel is inactive or idle, the supporting circuitry therein is off and thus the pixel is consuming minimal if any power and producing minimal if any heat. Accordingly, obtaining the pixel data from the subset of pixels in the active pixel array involves reading out only the subset of pixels while allowing the remaining pixels to remain inactive. Next, the pixel data is analyzed to determine if a person has entered the field of view of the image sensor (step <NUM>). Details regarding analyzing the pixel data to determine occupancy therefrom can be found in <CIT>, <CIT>, and <CIT>. It is then determined if a person has entered the field of view (step <NUM>). If a person has entered the field of view, additional pixel data are obtained (step <NUM>), where the additional pixel data contains pixel data for a larger portion of pixels in the active pixel array than the subset of pixels. For example, the additional pixel data may contain pixel data for all of the pixels in the active pixel array. Finally, the additional pixel data may be analyzed to determine if an area within the field of view of the image sensor is occupied (step <NUM>). Step <NUM> may be used as a verification of step <NUM>, or may be used to verify the continuing occupancy of the area within the field of view of the image sensor. Once again, details regarding analyzing the pixel data to determine occupancy therefrom can be found in the above-mentioned patent applications.

By obtaining pixel data for only the subset of pixels in the active pixel array such that the pixels not in the subset remain inactive, the temperature of the active pixel array can be kept much lower than if all of the pixels in the active pixel array were read out. This results in significant improvements in the signal to noise ratio of the pixels within the subset due to a reduction in thermal noise. Such improvements may be significantly evident in environments that are hot and dark, since signal to noise ratios in these environments tend to be highly unfavorable. Further, analyzing the pixel data to determine if a person has entered the field of view of the image sensor is far less computationally expensive due to the reduction in the total amount of pixel data to analyze. Obtaining, and analyzing the additional pixel data may improve the reliability of detecting occupancy in this manner.

By choosing the subset of pixels in the active pixel array wisely, the efficacy of detection of a person entering the field of view of the image sensor can be very high. Notably, the subset of pixels may be chosen such that all of the pixels within the subset reside in a single contiguous area or such that the pixels within the subset are located in separate, discrete areas. In embodiments in which the pixels within the subset reside in a single contiguous area, such a contiguous area may be defined by a polygon containing any number of sides, and at least five sides (i.e., a non-rectangular shape) in some embodiments.

In one embodiment, the subset of pixels is chosen such that they reside along an outer border of the active pixel array as illustrated in <FIG>. Specifically, <FIG> illustrates an exemplary readout pattern <NUM> for an active pixel array in which only the pixels along the outer edges of the active pixel array (illustrated by a group of shaded pixels) are read out, while the remaining pixels along the interior of the active pixel array remain inactive or idle. A person entering the field of view of an image sensor will necessarily first pass through an outer edge of the field of view before arriving in any other portion thereof. Accordingly, by analyzing pixel data from only the pixels in an area along the outer edges of the active pixel array, one can easily detect persons entering the field of view of the camera using only a subset of the pixels therein.

In some embodiments, it may be desirable to choose the subset of pixels such that it resides around or near one or more ingress and/or egress points within the field of view of the image sensor. Accordingly, <FIG> is a flow diagram illustrating a method for detecting occupancy using an image sensor according to an additional embodiment of the present disclosure. The method starts by determining a portion of a field of view of an image sensor that is near an ingress and/or egress point (step <NUM>). The ingress and/or egress point may be a door, a hallway, or the like. In general, the ingress and/or egress point is one that a person must travel through in order to gain access to the remaining portion of the field of view. Next, a subset of pixels in an active pixel array that detect light in the area near the ingress and/or egress point is determined (step <NUM>). This may involve a simple mapping of an area of the field of view to corresponding pixels in the active pixel array that detect light within this area. Next, pixel data is obtained from the subset of pixels in the active pixel array such that the pixels not in the subset remain inactive (step <NUM>). As discussed herein, when a pixel is inactive, the supporting circuitry therein is off and thus the pixel is consuming minimal if any power and producing minimal if any heat. Accordingly, obtaining the pixel data from the subset of pixels in the active pixel array involves reading out only the subset of pixels while allowing the remaining pixels to remain inactive. The pixel data is then analyzed to determine if a person has entered the field of view of the image sensor (step <NUM>). Once again, details regarding analyzing the pixel data to determine occupancy therefrom can be found in the above-mentioned patent applications. If a person has entered the field of view, additional pixel data are obtained (step <NUM>), where the additional pixel data contains pixel data for a larger portion of pixels in the active pixel array than in the subset of pixels. For example, the additional pixel data may contain pixel data for all of the pixels in the active pixel array. Finally, the additional pixel data may be analyzed to determine if an area within the field of view of the image sensor is occupied (step <NUM>). Step <NUM> may be used as a verification of step <NUM>, or may be used to verify the continuing occupancy of the area within the field of view of the image sensor. Once again, details regarding analyzing the pixel data to determine occupancy therefrom can be found in the above-mentioned patent applications.

<FIG> and <FIG> illustrate exemplary readout patterns <NUM> for an active pixel array according to various embodiments of the present disclosure. With respect to <FIG>, only those pixels in the lower left corner of the active pixel array (illustrated by a group of shaded pixels) are read out, while the remaining pixels are inactive or idle. Such a pattern may be effective, for example, when there is only one ingress and/or egress point to the field of view of the image sensor (e.g., a door leading to a room in which the image sensor is located) and it is in the lower left corner thereof. Notably, the subset of pixels forms a polygon including six sides. As discussed above, the subset of pixels may be chosen to occupy an arbitrary area defined by a polygon having any number of sides.

With respect to <FIG>, only those pixels along a lower left edge, lower bottom edge, and lower right edge of the active pixel array (forming a "U" shape) are read out, while the remaining pixels are inactive or idle. Such a pattern may be effective, for example, when it is known that person must pass through the lower outside edges of the image frame before being present in any other portion of the field of view.

With respect to <FIG>, the subset of pixels includes a first region of interest 38A in the lower left corner of the active pixel array and a second region of interest 38B on the right side of the active pixel array. Notably, the first region of interest 38A and the second region of interest 38B are not contiguous. While not shown, the first region of interest 38A and the second region of interest 38B may also be polygons having any number of sides. Further, while only the first region of interest 38A and the second region of interest 38B are shown, the subset of pixels may include any number of separate regions of interest that are either discrete or semi-contiguous without departing from the principles of the present disclosure. The pattern shown in <FIG> may be effective, for example, when a person must pass through the lower left side of the field of view or the right side of the field of view before being present in any other portion of the field of view.

Conventional image sensors are not able to read out pixels in an active pixel array in an arbitrary pattern. The image sensor <NUM> discussed herein may include modifications thereto such that the pixel selection circuitry <NUM> is capable of selecting pixels in an arbitrary fashion in order to read out only a subset of the pixels in the active pixel array <NUM> such that the subset includes polygons having any number of sides and/or noncontiguous regions of interest.

The image sensor <NUM> may further be configured to read out the pixels in the subset of pixels in a continuous fashion such that there are no pauses for pixels that are not being read out. This may require the control circuitry <NUM> to compute and implement specialized timing for the various parts of the image sensor <NUM> specific to the subset of pixels such that the pixel data can be properly sampled. Accordingly, the control circuitry <NUM> may be configured to alter the timing of pixel selection by the pixel selection circuitry <NUM> in order to provide a proper read out of the subset of pixels. Further, the control circuitry <NUM> may be configured to change operating parameters of the sampling circuitry <NUM> and the analog-to-digital converter circuitry in order to properly digitize the pixel data from the subset of pixels. Continuously reading out the subset of pixels without pausing for those pixels that are not being read out may significantly lower read times and thus allow for increases in frame rate above and beyond that which is achievable by a conventional image sensor.

Finally, the control circuitry <NUM> may be configured to change operating parameters of the output register <NUM> such that the pixel data for the subset of pixels is properly arranged and thus communicated to external circuitry for analysis. In particular, the image sensor <NUM> may be configured to capture, store, and facilitate transfer of the pixel data for the subset of pixels as a sparse data structure that does not include reserved spots (i.e., blank spaces) for pixels that are not in the subset of pixels. This may allow for a much smaller data structure to be provided via the output <NUM>, improving transmit times when provided to another device.

The image sensor <NUM> discussed herein is incorporated into an intelligent lighting fixture <NUM> as shown in <FIG>. The intelligent lighting fixture <NUM> includes the image sensor <NUM>, driver circuitry <NUM>, communications circuitry <NUM>, and a solid-state light source <NUM>. The driver circuitry <NUM> is coupled to the image sensor <NUM>, the communications circuitry <NUM>, and the solid-state light source <NUM>. The driver circuitry <NUM> may use the communications circuitry <NUM> to communicate with other devices such as other lighting fixtures within a distributed lighting network. Further, the driver circuitry <NUM> may control one or more light output parameters (e.g., brightness, color temperature, color rendering index, and the like) of the solid-state light source by providing one or more driver signals thereto. Finally, the driver circuitry <NUM> obtains and analyzes pixel data from the image sensor according to the methods discussed above with respect to <FIG> and <FIG> to determine and react to occupancy. By using pixel data for only a subset of pixels in the active pixel array <NUM> of the image sensor <NUM>, the processing resources of the driver circuitry <NUM> may be significantly conserved. This may in turn lead to reduced power consumption of the lighting fixture <NUM>, reduced cost due to the reduced processing requirements of the driver circuitry <NUM>, and improved longevity of the driver circuitry <NUM>.

The intelligent lighting fixture <NUM> may be one of many intelligent lighting fixtures <NUM> in an intelligent lighting network <NUM>, as shown in <FIG>. The intelligent lighting fixtures <NUM> may communicate with one another in order to provide certain functionality such as responding to occupancy events. Each one of the intelligent lighting fixtures <NUM> may be configured to detect occupancy using the image sensor <NUM> as discussed above. However, each one of the intelligent lighting fixtures <NUM> may be configured with a different read out pattern for the active pixel array <NUM> of the image sensor <NUM>, such that the subset of pixels used to determine occupancy is different for different ones of the intelligent lighting fixtures <NUM>.

In one embodiment, the intelligent lighting fixtures <NUM> may be configured to save processing resources by only requiring certain ones of the lighting fixtures <NUM> to detect occupancy when a space is unoccupied. In particular, those intelligent lighting fixtures <NUM> where a field of view of the image sensor <NUM> thereof includes an ingress and/or egress point to a space in which the intelligent lighting fixtures <NUM> are located may be tasked with detecting occupancy when the space is currently unoccupied, while the remaining intelligent lighting fixtures <NUM> are not required to do so. Those lighting fixtures <NUM> where a field of view of the image sensor <NUM> thereof does not include an ingress and/or egress point to the space do not need to detect occupancy when the space is currently unoccupied, because a person cannot enter the space without first passing through an ingress and/or egress point thereto. When occupancy is detected by one of the intelligent lighting fixtures <NUM> tasked with detecting occupancy, the remaining intelligent lighting fixtures <NUM> may begin to detect occupancy as well in order to verify occupancy or more accurately or precisely determine occupancy.

Claim 1:
An intelligent lighting fixture (<NUM>), comprising:
a solid-state light source (<NUM>);
driver circuitry (<NUM>);
an image sensor (<NUM>) coupled to the driver circuitry (<NUM>), wherein the image sensor (<NUM>) comprises:
an active pixel array (<NUM>) comprising a plurality of pixels; and
image sensor control circuitry (<NUM>) configured to:
perform read operations only on a subset of the plurality of pixels such that the plurality of pixels that are not in the subset of the plurality of pixels remain inactive; and
wherein the driver circuitry (<NUM>) is configured to obtain pixel data for the subset of the plurality of pixels from the image sensor (<NUM>) and analyze the pixel data to determine if a person has entered a field of view of the image sensor and react by obtaining additional pixel data from the image sensor, where the additional pixel data contains pixel data for a larger portion of pixels in the active pixel array (<NUM>) than the subset of the plurality of pixels.