Source: https://patents.google.com/patent/US9717118B2/en
Timestamp: 2020-01-22 16:32:50
Document Index: 282299479

Matched Legal Cases: ['Application No. 61', 'Application No. 61', 'Application No. 61', 'Application No. 61', 'Application No. 61', 'Application No. 61', 'arts 346']

US9717118B2 - Light control systems and methods - Google Patents
US9717118B2
US9717118B2 US14/048,505 US201314048505A US9717118B2 US 9717118 B2 US9717118 B2 US 9717118B2 US 201314048505 A US201314048505 A US 201314048505A US 9717118 B2 US9717118 B2 US 9717118B2
US14/048,505
US20150023019A1 (en
2013-07-16 Priority to US201361846738P priority Critical
2013-10-08 Application filed by Chia Ming Chen filed Critical Chia Ming Chen
2013-10-08 Priority to US14/048,505 priority patent/US9717118B2/en
2014-07-16 Priority claimed from PCT/US2014/046807 external-priority patent/WO2015009795A1/en
2015-01-22 Publication of US20150023019A1 publication Critical patent/US20150023019A1/en
2017-07-25 Publication of US9717118B2 publication Critical patent/US9717118B2/en
Provided is a light-emitting device control system, comprising: a light source module that outputs a beam of light at a first surface location; a beam steering mechanism, the light source module coupled to the beam steering mechanism for directing the beam of light at the first surface location, wherein an illumination region is formed at the first surface location in response to the directed beam of light; a control module that detects a signal corresponding to a hand gesture, the control module positioned at a separate location than the light source module and the beam steering mechanism, and communicating with the light source module via a network; and a control spot generator that generates a control spot at a second surface location, the control module detecting a presence of a hand forming the hand gesture at the control spot, and wherein the beam steering mechanism moves the illumination region in response to the hand gesture at the control spot.
This application claims the benefit of U.S. Provisional Patent Application No. 61/846,738 filed Jul. 16, 2013, the content of each of which is incorporated herein by reference in its entirety. This application is related to U.S. patent application Ser. No. 13/826,177 filed on Mar. 14, 2013, U.S. Provisional Patent Application No. 61/643,535 filed on May 7, 2012, U.S. Provisional Patent Application No. 61/684,336 filed on Aug. 17, 2012, U.S. Provisional Patent Application No. 61/760,966 filed on Feb. 5, 2013, U.S. Provisional Patent Application No. 61/840,791 filed on Jun. 28, 2013, and U.S. Provisional Patent Application No. 61/696,518 filed on Sep. 4, 2012, the content of each of which is incorporated herein by reference in its entirety.
In some embodiments, the control module processes distance information related to the hand gesture and a background for separating the hand gesture image data from image data of the background.
FIG. 6 is a diagram of a beam steering mechanism (BSM), in accordance with an embodiment;
FIG. 39 is a diagram illustrating an operation of the multispectral flashlight camera of FIGS. 37 and 38;
FIG. 40 is a top view of elements of a flashlight camera, in accordance with another embodiment;
FIG. 41 is a view of a light emitting device control system, in accordance with an embodiment;
FIG. 42 is a view of a light emitting device control system, in accordance with an embodiment;
FIG. 43 is a view of a light emitting device control system including a two (2)-dimension position sensitive detector (PSD), in accordance with an embodiment;
FIG. 44 is an illustrative view of a hand gesture motion tracking system using a camera, in accordance with an embodiment;
FIG. 45 is a view of a control spot generator, in accordance with an embodiment;
FIG. 46 is a view of a light emitting device control system, in accordance with an embodiment;
FIG. 47 is a view of a mobile device controlling a light source, in accordance with an embodiment;
FIG. 47A is diagram of elements of the mobile device of FIG. 47, in accordance with an embodiment;
FIG. 47B is flow diagram of a method performed by the mobile device of FIG. 47, in accordance with an embodiment;
FIG. 48 is a view of a three dimensional (3D) stereo camera, in accordance with an embodiment; and
FIG. 49 is a view of a light emitting device control system, in accordance with an embodiment;
The terminology used herein is for the purpose of describing particular embodiments and is not intended to be limiting of the inventive concepts. As used herein, the singular forms “a,” “an” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms “comprises.” “comprising,” “includes” and/or “including,” when used herein, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
In some embodiments, the light emitting device control system 100 comprises a light emitting diode (LED) light source. In another embodiment, the light emitting device control system 100 comprises an organic light emitting diode (OLED)). Although not shown, other sources that emit light can equally apply. The light managed by the control system 100 can be in the visible light spectrum, or other light spectrum. In some embodiments, the emitted light is concentrated from the light emitting device control system 100 onto an illumination region 102 as a light spot, for example, a white light spot or the like. The illumination region 102 can include a reduced light spot, for example, having an illumination cone angle of 50° or less. In some embodiments, the location of the illumination region 102 is controlled by a user. For example, a user can determine by a hand gesture a movement of the illumination region 102 from a first location shown at FIGS. 2A and 2B to a second location shown in FIG. 2C. In some embodiments, the user can activate and interrupt the illumination region 102 without restriction. In some embodiments, the control system 100 includes a controller (described below) that can adjust the brightness, intensity, distribution, size, color, and/or other characteristic of the emitted light in response to a hand gesture or other signal-generating motion. The control system 100 can perform control operations without aid of a physical device, such as a hand-held device. In some embodiments, a single control spot 104 is employed. In other embodiments, multiple control spots are employed at an illumination region 102. The control spot 104 can be a different color, shade, or other characteristic than the illumination region 102 so that a human eye can distinguish the control spot 104 in the illumination region 102 from the illumination region 102 itself. In some embodiments, the control spot 104 is a color in the visible spectrum and positioned in a light spot of the illumination region 102 so that a user of the light emitting device control system 100 can distinguish the control spot 104 from the illumination region 102. In some embodiments, the control spot 104 is dark and the illumination region 102 includes a white light spot visible to a user of the light emitting device control system 100. In some embodiments, the control spot 104 is a small portion, or minority, of the size of the illumination region 102. The control spot 104 can be produced by a filter, a light pipe, a control spot LED, or a combination thereof, for example, described herein.
As previously described, the partial filter 326 can be constructed and arranged to generate a control spot 104 for positioning in an illumination region 102. The filter 326 can comprise glass, plastics, or related material. The filter 326 can include a small region that is coated with a dark or colored coating, or related coating that forms a filtered region 323 and permits a high contrast control spot 104 against the white illumination region 102. In some embodiments, a remainder of the region, or unfiltered region 325 of the partial filter 326 is uncoated or coated with an anti-reflection (AR) material. The AR coating can minimize a Fresnel reflection loss. In some embodiments, as shown in FIG. 5, the LEI) light source 302 when constructed and arranged as a narrow beam LED 322 can comprise a compound LED chip 338 or the like, which can include one or more LEDs. The narrow beam LED 302 can further comprise narrow-beam optics 334 for generating a narrow light beam at the lens so that its diameter is equal or smaller than the aperture diameter of the lens. This ensures maximum throughput and minimum light absorption by internal housing wall. The narrow beam LED 302 can further include a heat sink 332 for dissipating heat generated by the LEDs. The heat sink 332 can be coupled to a one side of an LED board 336, and the LED chip 338 can be coupled to an opposite side of the LED board 336.
FIG. 6 is a diagram of a beam steering mechanism 304A, in accordance with an embodiment, which can be implemented at the light emitting device control system 100 shown in FIG. 3. The beam steering mechanism 304A can comprise a dual-axis gimbal 340 and a mirror 346. The gimbal 340 can include an outer ring 342 and an inner ring 344 that can be connected to each other by shafts 337A, B that extend along a first axis. Motors 345 and 343 are mounted on the inner ring 344 and outer ring 342, respectively. Counterweights 347, 349 can be mounted on the outer ring 342 and inner ring 344, respectively, for balancing and stabilizing the gimbal 340. Counterweight 347 is coupled to the shaft 339A at one side of the inner ring 344 and the second motor 345 is coupled to the shaft 339B at an opposite side of the inner ring 344 as the counterweight 347. Counterweight 349 is coupled to the shaft 337B at one side of the outer ring 342 and the first motor 343 is coupled to the shaft 337A at an opposite side of the outer ring 342 as the first counterweight 347. The rotation of the mirror 346 about the inner axis can be referred to as pitch. The rotation of the inner gimbal including parts 346, 344, 345,339A, and 339B can be referred to as yaw.
Returning to FIG. 3, in some embodiments, the tracking and control sensor 306 comprises a sensor 309 and a visible camera 310. The sensor 309 can be a thermal sensor that is radiometricly calibrated and constructed and arranged to measure the temperature of the target. In doing so, the sensor 309 can detect an object such as a human hand. The camera 310 captures an image and, as shown in FIG. 10, can output the image data to a DSP chip or other processor at the controller 308 for processing if the target temperature is close to or at that of a human body temperature, e.g., ˜36° C. In some embodiments, an image is generated in response a hand gesture, whereby the processor, e.g., a digital signal processor (DSP), compares a set of hand gesture images stored at a repository such as a database and known commands corresponding to the stored hand gesture images, and determines from the comparison a command based on the hand gesture.
A hand gesture 402 is made over a control spot, for example, the control spot 104 as shown in FIG. 2B. Thermal data, for example, a temperature, related to the hand gesture 402 is determined by the thermal sensor 406. The thermal sensor 406 can be the same as or similar to the thermal sensors described with respect to FIG. 3, 9, 9A, or 9B, respectively, or other embodiments such as those described herein. If the detected temperature of the hand making the gesture is determined to be within a predetermined range of temperatures known to be that of the human body, for example, ˜36° C., then the thermal sensor 406, or a processor in communication with the sensor 406, generates a request that the camera 404 provide an image of the hand gesture 402 to a processor 410. The processor 410 can include a digital signal processing (DSP) chip or single board computer in the LED lamp control electronics, for example, at the controller 308 of the control system 100 of FIGS. 2-9. The image data provided by the camera 404 can be compared to a library of known hand gestures to determine a command intended by the hand gesture, for example, a command to move the illumination region to a different location. At the processor 410, the acquired data corresponding to the hand gesture 402 is converted into a command signal in response to the comparison result. If the command signal identifies the hand gesture 402 as indicating a command to move the illumination region 102 generated by an LED lamp or the like at the light source module 302, then the beam steering mechanism 412 can direct the illumination region 102 to a different surface location, or otherwise modify the illumination region 102 in accordance with the command. Alternatively, the hand gesture 402 can correspond to a command to increase the brightness. The beam steering mechanism 412 can rely on the camera 404 to track the hand motion relative to the control spot 104. Here, the DSP or single board computer can generate a control signal that is output to the LED power supply 414, which adjusts the LED lamp or the like accordingly. An LED mode processor can place the LED lamp or the like in different modes according to a control signal, for example, turning the LED lamp on or off increasing or decreasing brightness, and so on.
In an embodiment, the distance between the lens and the focus is equal to the focal length regardless the shape of the beam path, the fold distance from the lens 324 to the FPA 506 via the beamsplitter 504 is equal to the focal length (f) of the lens 324. The distance from the lens 324 to the partial filter 326, or (f), is the same.
FIG. 14 is a block diagram of a skin detection sensor 600, in accordance with an embodiment. As described herein, some embodiments include the sensing of human hand being performed by a thermal sensor. In other embodiments equally applicable to the principles of the present inventive concepts, the skin detection sensor 600 can be used instead of a thermal sensor for sensing the presence of a human hand, for example, to determine hand gesture commands. The skin detection sensor 600 can comprise a camera or a radiometer 605 and a 3-color LED source 623 mounted on a blank filter 626. The radiometer 605 can comprise a lens and a silicon detector. It measures radiance. In some embodiments, the camera and the 3-colored LED source 623 do not share the lens. In some embodiments, the camera and the 3-colored LED share 623 the same lens as skin sensor 600 illustrated in FIG. 14. The camera of skin detection sensor 600 can include a lens 624, an FPA 606, and a beamsplitter 604. The beamsplitter 604 folds the focusing beam to the FPA 606 and allows the transmission of the 3-colored LED light. The transmission and reflection ratio is but limited to a ratio of 90:10. In some embodiments, the blank filter 626 and the 3-colored LED 623 can be positioned at or near the focal plane of the lens 624 having a focal length (f). This will ensure the 3-colored LED only illuminate the control spot.
To perform skin detection, target reflectance at 800 nm and 970 nm must be measured in target images. The two NIR LEDs from the 3-colored LED) at 800 nm and 970 nm can be used to illuminate the target in the control spot. The camera, the radiometer, and the 3-colored LED are synchronized. In some embodiments, the LEDs of the light source can turn on and off sequentially so the camera 606 can capture the images at the wavelengths corresponding to the three colors separately, and the radiometer radiance of the three illuminations separately. In an embodiment, each color has duration of 33 milliseconds. For example, the second LED 802B, e.g., the red LED, is activated first, followed by the first LED 802A, e.g., the 800 nm LED, then the third LED 802C, e.g., 970 nm LED. The output of the captured images are in unit of digital number (DN) not reflectance. Radiometric calibration of the camera is needed to obtain reflectance for each pixel. In some embodiments, skin detection relies on a radiometricly calibrated camera 606 to measure target reflectance. In some embodiments, performing a skin detection operation includes the use of a radiometer 605 to measure target reflectance. Radiometric calibration is typically performed to determine a set of radiometric coefficients to convert the camera images into input light at the camera aperture in unit of radiance. FIG. 15C and Eq. (1) to (4) illustrate the radiometric calibration process and its application.
To keep the camera response in the linear region, the signal levels are selected such that the detector DN is 30% and 70% of the full well, respectively. In Eq. (1) (Lk30, Lk70) are radiance of source k at two different signal levels, k=1, 2, 3 is the color index. Index 1 is for NIR LED at 800 nm, index 2 for red LED, and index 3 for NIR LED at 970 nm. (DNk30, DNk70) are the pixel value of the camera detector due to color k at two radiance levels (Lk30, Lk70). Radiometric calibration coefficients (ck1,ck0) can be solved from Eq (1) for each color k.
Once the camera radiometric coefficients (ck1,ck0) are found for all camera detectors, the coefficients can be stored in the processor 410. The camera 606 can be integrated into the system. Images of the control spot illuminated by the 3-color LEDs 800 can be converted into radiance by using radiometric coefficients as shown in Eq. (2). DN′k is the detector response to target radiance L′k. The prime is used here to indicate the images are scene images.
L′ k =c k1 *DN′ k +c k0 (2)
FIG. 17 is a flowchart illustrating a method 950 for determining a hand gesture, in accordance with an embodiment. In describing the method 950, reference can be made to elements of FIGS. 15 and 16, or elements of other figures described herein. At the processor 410, the NIR (800 nm and 970 nm) images 956, related to the first and third LEDs 802A, 802C, respectively, are convened into an aperture radiance by using the radiometric calibration coefficients for human hand identification. If a determination is made, for example, at the processor 410, that the ratio of these two images 956 in the control spot region 104 matches the band ratio at 800 nm and 970 nm of the human hand, for example, shown in FIG. 13. Pixels in regions with human hand identified are set to 1. The rest of the regions belonging to background are set to zero. The background regions in all bands are the same. The processor 410 can then process the red image 954, i.e., related to the second LED 802B, for hand gesture determination. Once the hand gesture is recognized 958, the processor 410 can generate an appropriate command 962, as described herein, for example, after performing a comparison with known hand gesture images stored at a database or the like.
FIG. 21 is a diagram of the light-emitting device control system 1200 of FIG. 20, in accordance with another embodiment. Here, a camera 1310 placed externally from a beam steering portion 1308 of the light-emitting device control system 1300. Referring again to FIGS. 18A and 18B, a camera can be placed outside the gimbal. In some embodiments, a lens with a 2-dimensional position sensitive detector can be implemented instead of a camera. Other elements of the light-emitting device control system 1300, such as a light source module 1302, controller 1308, and sensor 1309 are similar to or the same as those described in FIG. 3, and/or corresponding elements described in FIG. 20; thus, details thereof are omitted for brevity.
In FIG. 20A, both a camera and thermal sensor are outside of the beam steering mechanism. The camera requirements and functionalities are the same as in FIG. 20 and will not be repeated for reasons related to brevity. The thermal sensor is however, mounted on a mini-gimbal similar to that in FIGS. 18A and 18B. The gimbal uses the gimbal angles of the beam steering mechanism of 304 to track the control spot. FIG. 21A is similar to FIG. 21 except that the thermal sensor is positioned outside of the beam steering mechanism.
As described above with respect to FIG. 21A, a camera 1310 and/or a sensor 1309 can be positioned outside a beam steering mechanism 1308. The sensor 1309 can be a skin detection sensor and/or a heat detection sensor, for example, described in FIGS. 9A, 9B, 9C, and 14, respectively. A control spot generator (not shown) can be mounted inside a light source module 1302 and/or a beam steering mechanism, for example, described herein. In other embodiments, the control spot generator is positioned externally from the light source module and the beam steering mechanism. In some embodiments, a control module and a control spot generator, for example, described herein, are co-located under a single housing constructed and arranged as one unit. In some embodiments, the control module and the control spot generator are separate from each other, for example, constructed and arranged as separate units. The control module, which can include a tracking and control sensor and a camera described herein, can be positioned on a ground surface, a wall, or other relevant location. In some embodiments, an assembled unit is placed near the light source module and the beam steering mechanism. In some embodiments, the assembled unit is placed a significant distance from the light source module and the beam steering mechanism. In some embodiments, Wi-Fi transceivers can be mounted to the light source module and the control module, respectively. Hand gesture commands from the control module can be transmitted to the beam steering mechanism and the light source module through a Wi-Fi signal in some embodiments. In other embodiments, the commands can be exchanged through Ethernet or by other means to the light source module and beam steering mechanism. The benefit of separating a light source module 1302 and a tracking and control sensor 1309 and a control spot generator is to simplify a device design so that the manufacturing costs can be less expensive than other devices. For example, a beam steering mechanism constructed and arranged to include counter-rotating wedge prisms is less expensive than a gimbal-based construction.
In an embodiment, the tracking and control sensors 1400, 1410 are each assembled in a common unit. Here, a plurality of LEDs or the like, a visible FPA, and a thermal detector or FPA can share the same lens but split the aperture, and illuminate at the same target. As shown in FIG. 22 and FIG. 24, the tracking and control sensor 1400 comprises a refracted optics element 1402, such as a refracted Cleartran™ lens, a multi-face pyramid prism mirror 1404, a thermal detector or array 1428 comprising thermopile, pyroelectric, or related materials, and a visible FPA 1406 preferably including a filter. As shown in FIG. 23, the sensor 1410 can include a reflected focusing optics 1412, such as a Cassagrain type, a three-face pyramid prism mirror 1404, a thermal detector 1428 (an array can equally apply), and a visible FPA 1416 (1416, 1406, and 1426 are the same). The thermal detectors in both 1410 and 1400 have fillers.—The tracking and control sensors 1400, 1410 can each include a color LED source or three-color LED-lightpipe assembly, for example, as shown in FIG. 15A and FIG. 15B.
An operation will now be described with reference to FIG. 24. FIG. 24 is the top view of a sensor 1420, which can be the same or similar to sensor 1400 or sensor 1410. The dotted circle denotes a lens or Cassagrain optics entrance aperture projected onto the 3-faced pyramid mirror. Light is received from a scene target, for example, an illuminated region 102 having the control spot 104 over which is positioned a human hand or other object as shown in FIG. 2B. The received light cone propagates through a lens, for example, lens 1402 of FIG. 22 or the Cassagrain optics 1412 of FIG. 23 to the pyramid prism mirror. The light cone is divided into a plurality of equal portions. The received light contains visible light from the control spot 104 and the thermal light from a hand gesture. A first portion L1 of the received light can pass through a visible filter (not shown) and focus onto a visible FPA 1406. In particular, the visible filter rejects the thermal light and permits the visible light pass through onto the visible FPA 1406. A second portion L2 of the received light can pass through an infrared filter (not shown) and focus onto a thermal FPA or thermal detector 1428. The thermal detector 1428 can be part of the thermal sensor 309 described herein. In particular, the thermal filter can reject the received visible light and permit the thermal light pass through to the thermal detector 1428. A third portion L3 of the received light is focused onto a color LED 1422 or the like, for example part of the light source module 302 described herein, and is not used. The light cone in the region L3 has both incoming light from the target and outgoing light from the color LED 1422. The incoming light is much weaker than the outgoing light from the color LED 1422. The outgoing light follows the path of the incoming light to illuminate the control spot 104. The control spot 104 is generated by the filter region of a partial filter, for example, as described by FIG. 4. The color LED in the region L3 can be used for modulation of the control spot so the user can see it more easily. If the 3-color LED-lightpipe assembly described in FIGS. 15A and 15B is placed at L3, one can use the two NIR LEDs to remove background of the hand gesture by using the radiometric band ratio method as described by Eq. (1), (2), (3), and (4) and FIGS. 14, 15C herein.
In the transmitter 1602, incoming data is modulated at the modulator 1612, then converted to an input current at the D/A converter 1614, then output to the LED lamp 1616. The LED lamp 1616 can include some or all of the elements of a light-emitting device control system in accordance with an embodiment, for example, the control system 100 described herein. The LED lamp 1616 can output modulated light to the receiver 1604. In the receiver 1604, the photo detector 1622 converts the light signal from the transmitter into an electrical signal. The signal is demodulated and converted to data by a demodulator 1624 and the A/D converter 1626, respectively. Aspects of visible light wireless communication including transceiver designs and modulation methods can be found in H. Eglala, R. Mesleh, H. Haas, entitled “Indoor Optical Wireless Communication: Potential and state-of-the-art,” IEEE Communication Magazine, September 2011 at pp. 56-62, N. Kumar, N. Laurenco, M. Spiez, R. Aguiar, Visible Light Communication Systems: Conception and VIDAS, IETE Technical Review, September 2008 at pp. 359-367, R. Mesleh, H. Haas, B. Pricope, entitled “OFDM Visible Light Wireless Communication Based on White LEDs,” VTC Spring 2007 IEEE at pp. 2185-2189, and Y. Wu, A. Yang, L. Feng, L. Zuo, Y. Sun, entitled “Modulation base cells distribution for visible light communication,” 2012 Optical Society of America, published Oct. 12, 2012, each incorporated by reference in its entirety.
An optical Wi-Fi user may desire to reduce an illumination when using an electronic device such as a laptop computer or smart phone. However, if the user reduces the illumination too much, the signal at the receiver may become too weak to obtain quality data. A high data rate infrared transmitter can be mounted to the LED lamp system. The white light LED) has very little spectrum in the infrared. Therefore, the illumination light will not increase the background of the data transmission light. Operations of the LED lamp system will not affect the data transmission. The user can dim down or turn off the LED lamp system 1710, the transmitter at the LED lamp system 1710 will continue to transmit data. In this embodiment, illumination and data transmission are separated. In some embodiments, near infrared (NIR) light can be used as the data carrier. In other embodiments, short wave infrared (SWIR) light can be used as the data carrier. In some embodiments, the transmitter light source can be LED. In other embodiments, the transmitter light source can be laser. The LOS of the transmitter can be parallel to that of the LED lamp system 1710. The transmitter beam spot 1705 and the illumination spot 1702 are almost concentric, and similar in shape, size, and related configuration parameters. Because the transmitter and the LED lamp system share the same gimbal platform, the illumination spot and the transmitter beam spot move together. In some embodiments where the signal strength is strong, the transmitter beam spot 1705 can be larger than or equal to the illumination spot 1702. In some embodiments where the signal strength is weak, the transmitter beam spot 1705 can be smaller than the illumination spot 1702. Because the transmitter light source is in the infrared not visible to human eye, color illumination of the transmitter beam spot 1705 is needed when it is smaller than the illumination spot 1702. A color illumination spot can be created using a technique described in the control spot in embodiments herein, for example, described with reference to FIGS. 4, 12, and 12A, respectively.
The integration of camera technology and functionality into a flashlight is desirable because it can enable a user to take a picture of a target he/she shines a light on. In current flashlight camera technology, LEDs are arranged around the perimeter of the camera. Light from the LEDs shine on the target and the camera captures its image. However, in such designs, the field of view of the camera may not coincide with the LED) light cone produced by the LEDs arranged around the perimeter of the camera. Accordingly, dark regions, or regions of diminished light intensity, may be present in the resulting camera image due to the mismatch. Furthermore, an image in only one spectral band is available.
The embodiment in FIG. 40 comprises a visible channel 2526, an uncooled thermal channel 2528, and a LEI) light source 2502. This flashlight camera 2550 allows a user access both visible and thermal images of the same illuminated target. Because visible LEDs don't have emission beyond 700 nm, the thermal channel 2526, for example, above 8000 nm, will only see emitted infrared light from the target.
FIG. 41 is a light emitting device control system 2600, in accordance with an embodiment. As shown in FIG. 41, a beam steering mechanism is not employed for collocation with the control module 2606 having a tracking and control sensor and/or a camera, and the control spot generator 2614 has no beam steering mechanism as illustrated by FIG. 41. However, a light source module 2602 can be coupled to a beam steering mechanism 2604, which is separated by a predetermined distance from the control module 2606. The light source module 2602 and/or the beam steering mechanism 2604 can communicate with the control module 2606 and/or the control spot generator 2614 via a communication mechanism such as Wi-Fi transceivers 2670 a, 2670 b, respectively. In other embodiments, the light source module 2602 and/or the beam steering mechanism 2604 can communicate with the control module 2606 and/or the control spot generator 2614 via a wire, cable or the like that transfers electrical signals there between, or via a network known to those of ordinary skill in the art, such as a wide area network, local area network, mobile communication network, and so on. In an embodiment, the control spot 104 does not move during an operation. In some embodiments, the light emitting device control system 2600 does not include a control spot generator 2614. Here, a camera can operate using ambient light instead of an illuminated control spot. In an embodiment, a sensor such as the thermal sensor 309 described herein can detect heat signature from a hand. A hand gesture 2610 is placed directly below, or otherwise along a line of sight of, the control module 2606 or, more specifically, the tracking and control sensor.
FIG. 42 is a light emitting device control system 2700, in accordance with an embodiment. In some embodiments, a beam steering mechanism 2712 is added to the control module 2706 having a tracking and control sensor and/or camera, and the control spot generator 2714. In this embodiment, the control spot 104 can be moved. To move the control spot, a hand gesture for tracking is placed in the control spot 104. A tracking command is sent to the beam steering mechanism 2712. The tracking and control sensor is now in the tracking mode. The control spot will follow the user's hand until a stop hand gesture is given. At the desired location, the user can control a light source module 2702, or other related apparatus, by performing various hand gestures in the control spot. This permits access of a tracking and control sensor at different places. Here, one can control a light source from different places. For example, one can move the control spot to a desired position without moving the illumination spot. Once the user is at a predetermined location, the user can control the light source. This is different than embodiments where a control spot is positioned within an illumination spot, where the control spot and the tracking and control sensor follow the illumination spot.
The light emitting device control system 2700 includes Wi-Fi transceivers 2770 a, b or other communication device, which can be similar or the same as the Wi-Fi transceivers 2670 a, 2670 b of FIG. 41, and a light source module 2702 and beam steering mechanism 2704, which are similar to or the same as the light source module 2602 and beam steering mechanism 2604 of FIG. 41.
In some embodiments, the motion of a hand gesture 2610, 2710 can be tracked by the camera in the control module 2606, 2706 of FIGS. 41 and 42, respectively. As illustrated in FIG. 44, a hand gesture motion tracking system 2900 can include a camera FPA 2902, a camera lens 2906, and a color LED 2912, for example, described in other embodiments herein. A hand gesture position 2910 is at a first location of an image 2920 a, and moves to the different location of the image 2920 b. From the camera frame rate and the position delta of the hand gesture between the two images 2920 a, 2920 b, a hand gesture speed from its location at time t1 to its location at time t2 can be calculated. In some embodiments, for example, shown in FIG. 43, a two-dimension position sensitive detector (PSD) 2802 can be added to the control module 2606, 2706 in FIGS. 41 and 42, respectively. The function of the PSD 2802 is tracking while the function of the camera is hand gesture recognition.
As shown in FIG. 43, a light emitting device control system 2800 can include the 2-dimension PSD 2802 added to a tracking and control sensor. The 2-dimension PSD 2802 can comprise a single active element in which a photodiode surface resistance is used to determine a position. The light beam I0 output from the LED light source 2812 is split into two portions, I1 and I2 at the beamsplitter 2814. The transmitted I1 is reflected by a mirror 2808. Portion of I1, I1r, is reflected into PSD 2802 where it is focused to a beam spot by the lens 2806. This beam spot serves as a reference position (0,0). The reflected beam I2 illuminates the hand gesture 2810. The light source 2812 has an illumination angle which is minimized in the figure in order to avoid confusion about the light splitting at the beamsplitter 2814. Because of this angle, the illumination spot at the hand gesture is bigger than the hand gesture. Depending on the location of the hand gesture in the illumination spot, the reflected beam from the hand gesture I2r with respect to the PSD at an angle different from that of I1r. The beam spot of I2r is at a different location at the PSD than the reference position. As the hand gesture 2810 moves, the beam spot of I2r also moves as illustrated in 2820. The relative distance between the reference spot and the hand gesture beam spot allows one to quantify the amount of hand gesture motion. This is done internally because the PSD 2802 finds the centroid position by averaging the light intensity on the PSD 2802. A color filter can be 2804 positioned in front of the PSD 2802, which allows LED light in and blocks ambient light. Because the PSD 2802 tracks the position of the beam spot of the hand gesture 2810, the motion of the hand gesture 2810 can be monitored. The PSD 2802 has very high resolution and can detect very small hand gesture motion.
In some embodiments, a control spot generator 3000 comprises multiple LED modules, for example, configured to generate three different colors. As illustrated in FIG. 45, in some embodiments, the LEDs include a red LED module 3012 b, a NIR LED module 3012 a, which outputs light at or near 800 nm, and a NIR LED module 3012 c, which outputs light at or near 970 nm. In some embodiments, each LED module 3012 a-c (generally, 3012) comprises a lens 3006 and a LED chip 3004. In some embodiments, the red color LED 3012 b can be replaced by other colors. In some embodiments, the control spot generator comprises only one visible color LED. As shown at FIG. 15B herein, NIR LEDs in the 3-color LED are placed inside a light source module. In other embodiments, they can be placed outside the light source module. In some embodiments, light at different wavelengths other than those referred to herein can be output, the control spot generator comprises a plurality of light emitting diodes (LEDs), each LED constructed and arranged to emit light at a different wavelength. In an embodiment, the 3 LEDs are placed next to each other. Their illumination spots may overlap but do not coincide with one another. The overlap region or control spot is large enough to accommodate the range of a hand gesture motion.
A control module in accordance with some embodiments relies on a hand gesture recognition method to control a light emitting control system. In some embodiments, speech recognition devices and software such as Dragon™ products by Nuance Communications can be added to a control system, for example, a control system referred to herein. In some embodiments, speech recognition can increase the number of commands available in the hand gesture library for the light emitting control system described in this patent. The number of hand gestures that are easy to perform and distinguishable by machines is not very large. In some embodiment, speech recognition can help to alleviate hand gestures with complex backgrounds for the light emitting control system described herein. In some situations, the background blends in with hand gestures. Hand gesture recognition can be difficult. If hand gesture recognition fails, speech recognition can be used in some embodiments.
FIG. 46 is a view of a light emitting device control system 3100, in accordance with an embodiment. As shown in FIG. 46, a microphone 3118 used to detect sounds, in particular, voice signals, is attached to the control module in a light emitting control system 3100. In some embodiments, a collected voice signal can be output to a processor (not shown), such as a digital signal processor (DSP) and processed by a speech recognition processor or the like. A command signal can be generated in response to the processing of the voice signal.
The light emitting device control system 3100 can also include a control module 3106 having a tracking and control sensor and at least one camera, a control spot generator 3114, a light source module 3102, at least one beam steering mechanism 3104, Wi-Fi transceivers 3170 a. 3170 b, which are similar to comparable elements described herein, for example, at FIGS. 41 and 42. Descriptions of these elements are therefore not repeated for brevity. A hand gesture 3110 can placed at a control spot 104 in a similar manner as that described herein.
As described herein, some embodiments include a control module having a single camera. In other embodiments, for example, illustrated in FIG. 48, a control module 3150 can comprise two cameras 3152 a, b separated by a distance (d). The orientation of the two cameras 3152 a, b is preferably the same with parallel line-of-sights. In an embodiment, the cameras 3152 a, b are thermal cameras. In an embodiment, the cameras 3152 a, b are visible cameras. The control module 3150 allows 3-D imaging of hand gestures and the background to be performed.
In order to perform 3-D imaging, the two cameras 3152 a,b must be aligned and identical in focal length, field of view, detector size, and detector array size. The coordinate difference or disparity of the target point between two camera images along with the focal length and camera separation allows one to extract the distance information as illustrated by the equation below.
z = f ⁡ ( 1 + d Δ ⁢ ⁢ x )
Where z is the distance, f is the focal length of the camera, d is the camera separation, and Δx is the disparity of a target point between the two images.
As shown in FIG. 48, the hand gesture 3160 is generally at a distance from a surface 3162, for example, a floor, wall, and so on. To find the disparity of a target point, one needs to identify the same target point from the two images. Correlation methods such as methods described in the article entitled “Basics of 3D Digital Image Correlation,” Application Note—T-Q-400-Basics-3DCORR-002a-EN authored by Christian Herbst, Technical University Braunschweig Institute for Production Metrology (IPROM), Braunschweig, Germany, and Karsten Splitthofof Dantec Dynamics GmbH Ulm, Germany and the article entitled “Fast Stereo Matching Using Rectangular Subregioning and 3D Maximum-Surface Techniques” International Journal of Computer Vision. vol. 47, no. 1/2/3, pp. 99-117, May 2002, authored by Changming Sun, each incorporated by reference in its entirety, can be employed. In addition, radiometric calibration of the two cameras described in U.S. patent application Ser. No. 13/826,177 incorporated by reference in its entirety, can reduce camera non-uniformity and bring the two cameras to the same measurement unit, radiance. The two image points of the target point in the two cameras must have the same radiance. Such techniques permit distance information of the hand gesture and background to be used to separate hand gesture image data from background image data. Related techniques can be applied, such as those described in U.S. Pat. No. 8,139,935 incorporated herein by reference in its entirety.
As described herein, a control module can receive data related to hand gestures and/or voice commands to control a light source module. In some embodiments, as shown in FIG. 47, a mobile device 20 can replace a tracking and control sensor of the control module. The mobile device 20 can control a light source module or the like and/or the beam steering device from a software application that is stored in a memory of, and executes at one or more processors of, the mobile device 20.
FIG. 47A is diagram of elements of the mobile device of FIG. 47, in accordance with an embodiment. FIG. 47B is flow diagram of a method 2760 performed by the mobile device 20 of FIG. 47, in accordance with an embodiment;
As illustrated in FIG. 47A, a lamp application icon 2754 or the like appears in the mobile device screen 2752. By touching the icon 2754, the mobile device 20 initiates a hand shake with the Wi-Fi device 2770 b at the light source module and the beam steering mechanism. A control panel 2756 is displayed on the touch screen of the mobile device 20. In some embodiment, the control panel 2756 comprises an illumination spot position panel 2757, an on/off switch 2758, and a light level adjustment bar 2759. In the position panel 2756, a white spot representing the position of the illumination spot appears in a square box. The box represents the region the illumination spot can move. The user can adjust the position of the illumination spot by adjusting the position of the white spot in the illumination spot position panel 2756. The light adjustment bar 2759 allows the user to change the light level by sliding on the bar. The user can turn on or off the light source module by touching the switch 2758. Adjustment commands are sent to the light source module and the beam steering mechanism through the Wi-Fi network of the mobile device as illustrated by the flow chart of FIG. 47B.
Referring to the method 2760, at block 2762, the lamp application is activated. For example, a user can click the application icon 2754.
At block 2764, a handshaking occurs between the mobile device 20 and the light source module 2702 and/or beam steering mechanism 2704 of the control system 2700.
At block 2766, a control panel 2756 at the mobile device display lamp is displayed.
At block 2768, a light level command is generated, for example, by the light adjustment bar 2759, and/or at block 2770, an illumination spot location command is generated, for example, at the illumination spot position panel 2756, and/or at block 2772, an/off switch command is generated, for example, at the displayed on/off switch 2758.
As described herein, elements of a light emitting device control system can be mounted at different locations. For example, as shown in FIG. 49, a light emitting device control system 3300 can include a tracking and control sensor of a control module 2606 and a control spot generator 2614 (shown also at FIG. 41) that are mounted on a wall surface or other surface. In some embodiments, a tracking and control sensor 3306 of a control module and the control spot generator 3314 can be mounted at a different designated location, for example, mounted at a wall surface 3350. A hand gesture 3310 can be placed in front of the tracking and control sensor 3306 to control the light source module 3302 and BSM 3304. The light emitting device control system 3300 can also include a light source module 3302, BSM 3304, WiFi transmitter/receivers 3370 a, b, in addition to the tracking and control sensor 3306 and the control spot generator 3314, each of which can be similar or the same as those counterpart elements of other light emitting device control systems described herein. Therefore, a detailed description of the various elements of the light emitting device control system 3300 are not repeated for reasons related to brevity.
a light source that outputs a beam of light at a first surface location;
a beam steering mechanism that includes a movable element and that receives and directs the beam of light in response to the movement of the element, the beam steering mechanism movable about at least two axes for steering the beam of light, the light source coupled to the beam steering mechanism for directing the beam of light at the first surface location, wherein an illumination region is formed at the first surface location in response to the directed beam of light;
a control module that includes at least one sensor that includes a radiometrically calibrated thermal sensor, the control module positioned at a separate location than the light source and the beam steering mechanism, and communicating with the light source via a network, wherein the control module comprises at least one of a tracking and control sensor that detects a presence of a hand forming a hand gesture at a control spot based on a temperature of the hand; and a camera that tracks a motion of the hand gesture and recognizes the hand gesture, wherein the tracking and control sensor includes a lens that transmits both at least one of visible or thermal light, a thermal detector or array, and a visible FPA coupled to the thermal detector or array, the visible FPA positioned between the thermal detector or array and the lens, the thermal sensor FPA sensing emitted thermal light from the hand and the visible FPA sensing reflected light from hand; and
a control spot generator that generates a color control spot for control area identification at a second surface location, the control module detecting a presence of a hand forming a hand gesture at the color control spot, and wherein the beam steering mechanism moves the illumination region in response to the hand gesture at the color control spot, the radiometrically calibrated thermal sensor detecting a temperature of the hand making the gesture in the color control spot, and the control module recognizing the hand gesture using a combination of the detected hand temperature and a library of known hand gestures.
2. The system of claim 1, further comprising a first Wi-Fi transmitter/receiver in communication with the control module and a second Wi-Fi transmitter/receiver in communication with at least one of the light source and the beam steering mechanism, the first and second Wi-Fi transmitter/receivers communicating with each other via the network.
3. The system of claim 1, further comprising a beam steering mechanism coupled to the control spot generator and the control module for moving the control spot and the field of view of the camera and the thermal sensor in the control module.
4. The system of claim 1, further comprising a two-dimension position sensitive detector (PSD) module in communication with the control module for determining a position of the hand gesture.
5. The control system of claim 1, wherein the tracking and control sensor comprises a thermal imager having a linear or area focal plane array (FPA), and wherein the tracking and control system further comprises a scan mirror for the linear array.
6. The control system of claim 1, wherein the tracking and control sensor comprises a thermal sensor and a visible camera for capturing an image of a hand making the hand gesture and recognizing the hand gesture based on the temperature of the hand, and wherein the thermal sensor comprises a lens and a thermal detector or a detector focal plane array.
7. The control system of claim 1, wherein the tracking and control sensor comprises two identical thermal cameras separated by a distance.
8. The control system of claim 1, wherein the tracking and control sensor comprises two identical visible cameras separated by a distance.
9. The control system of claim 1, wherein the control module processes distance information related to the hand gesture and a background for separating the hand gesture image data from image data of the background.
10. The control system of claim 1, wherein the control module comprises a microphone that detects a voice signal, and wherein the beam steering mechanism moves the illumination region in response to the voice signal.
11. The system of claim 1, wherein the control spot generator comprises a plurality of light emitting diodes (LEDs), each LED constructed and arranged to emit light at a different wavelength.
12. The system of claim 1, wherein the control module is positioned on a wall surface.
US14/048,505 2013-07-16 2013-10-08 Light control systems and methods Active 2035-02-10 US9717118B2 (en)
US201361846738P true 2013-07-16 2013-07-16
US14/048,505 US9717118B2 (en) 2013-07-16 2013-10-08 Light control systems and methods
PCT/US2014/046807 WO2015009795A1 (en) 2013-07-16 2014-07-16 Light control systems and methods
US20150023019A1 US20150023019A1 (en) 2015-01-22
US9717118B2 true US9717118B2 (en) 2017-07-25
ID=52343430
US14/048,505 Active 2035-02-10 US9717118B2 (en) 2013-07-16 2013-10-08 Light control systems and methods
US (1) US9717118B2 (en)
US9810392B2 (en) * 2012-10-11 2017-11-07 Mitsubishi Electric Corporation Vehicle headlight device
WO2018054831A1 (en) * 2016-09-22 2018-03-29 Philips Lighting Holding B.V. Thermal imaging for space usage analysis
2013-10-08 US US14/048,505 patent/US9717118B2/en active Active
Application and Prosecution history of U.S. Appl. No. 13/826,177, filed Mar. 14, 2013, by Chia Ming Chen, Entitled "Light Control Systems and Methods", which is stored in the United States Patent and Trademark Office (USPTO) System.
Borah et al., "A review of communication-oriented optical wireless systems," EURASIP Journal on Wireless Communications and Networking, 91:1-28 (2012).
Elgala et al., "Indoor Optical Wireless Communication: Potential and State-of-the-Art," IEEE Communication Magazine, Sep. 2011 at pp. 56-62.
Elgala et al., "OFDM Visible Light Wireless Communication Based on White LEDs," VTC Spring 2007 IEEE at pp. 2185-2189.
Final Office Action dated Dec. 24, 2015, issued in related U.S. Appl. No. 13/826,177.
Final Office Action issued on Jan. 4, 2016 in related U.S. Appl. No. 14/318,019.
Herbst, "Basics of 3D Digital Image Correlation," Application Note-T-Q-400-Basics-3DCORR-002a-EN, Technical University Braunschweig Institute for Production Metrology (IPROM), Braunschweig, Germany, and Karsten Splitthof of Dantec Dynamics GmbH Ulm, Germany.
Herbst, "Basics of 3D Digital Image Correlation," Application Note—T-Q-400-Basics-3DCORR-002a-EN, Technical University Braunschweig Institute for Production Metrology (IPROM), Braunschweig, Germany, and Karsten Splitthof of Dantec Dynamics GmbH Ulm, Germany.
International Search Report and the Written Opinion dated Oct. 8, 2015, issued in corresionding International Patent Application No. PCT/US15/28163.
Kumar et al. "Visible Light Communication Systems Conception and VIDAS," IETE Technical Review, 25(6):359-367 (2008).
Office Action dated Aug. 7, 2015; issued in related U.S. Appl. No. 13/826,177.
Office Action dated Mar. 28, 2016, issued in related U.S. Appl. No. 13/826,177.
Office Action issued Jul. 6, 2015 issued in related U.S. Appl. No. 14/318,019.
PCT Application No. PCT/US2013/039666, filed May 6, 2013, by Chia Ming Chen, Entitled "Light Control Systems and Methods", which is stored in the United States Patent and Trademark Office (USPTO) System.
Sun, "Fast Stereo Matching Using Rectangular Subregioning and 3D Maximum-Surface Techniques", International Journal of Computer Vision, 47:99-117 (2002).
U.S. Appl. No. 61/643,535, filed May 7, 2012, by Chia Ming Chen, Entitled "Smart Lamps, Systems Employing Same, Methods for Controlling and Operating Same and Methods for Forming Same", which is stored in the United States Patent and Trademark Office (USPTO) System.
U.S. Appl. No. 61/684,336, filed Aug. 17, 2012, by Chia Ming Chen, Entitled "Smart Lamps, Systems Employing Same, Methods for Controlling and Operating Same and Methods for Forming Same", which is stored in the United States Patent and Trademark Office (USPTO) System.
U.S. Appl. No. 61/696,518, filed Sep. 4, 2012, by Chia Ming Chen, Entitled "Multispectral Flashlight Camera", which is stored in the United States Patent and Trademark Office (USPTO) System.
U.S. Appl. No. 61/760,966, filed Feb. 5, 2013, by Chia Ming Chen, Entitled "Alternative Design for Tracking and Control Sensor", which is stored in the United States Patent and Trademark Office (USPTO) System.
U.S. Appl. No. 61/840,791, filed Jun. 28, 2013, by Chia Ming Chen, Entitled "Hand Gesture Control Module for Machines, Instrument, Robots, Vehicles, and Devices", which is stored in the United States Patent and Trademark Office (USPTO) System.
U.S. Appl. No. 61/846,738, filed Jul. 16, 2013, by Chia Ming Chen, Entitled "Light Control Systems and Methods", which is stored in the United States Patent and Trademark Office (USPTO) System.
Wang et al., "12.5 Gbps Indoor Optical Wireless Communication System with Single Channel Imaging Receiver," ECOC Technical Digest; 37th European Conference & Exhibition on Optical Communication (2011).
Wu et al., "Modulation based cells distribution for visible light communication," Optics Express 20(22):2012; Optical Society of America, published Oct. 12, 2012.
US20150023019A1 (en) 2015-01-22
AU2014310703B2 (en) 2018-09-27 Optical detector