Source: https://patents.google.com/patent/US10412280B2/en
Timestamp: 2019-12-14 03:55:58
Document Index: 694756407

Matched Legal Cases: ['Application No. 10821000', 'Application No. 10821000', 'Application No. 10821000', 'Application No. 2012', 'Application No. 2012', 'Application No. 201080043779', 'Application No. 201080043779', 'Application No. 201080043779']

US10412280B2 - Camera with light valve over sensor array - Google Patents
Camera with light valve over sensor array Download PDF
US10412280B2
US10412280B2 US15/040,862 US201615040862A US10412280B2 US 10412280 B2 US10412280 B2 US 10412280B2 US 201615040862 A US201615040862 A US 201615040862A US 10412280 B2 US10412280 B2 US 10412280B2
US15/040,862
US20170230551A1 (en
Jayachandra Gullapalli
2016-02-10 Priority to US15/040,862 priority Critical patent/US10412280B2/en
2016-02-12 Assigned to MICROSOFT TECHNOLOGY LICENSING, LLC reassignment MICROSOFT TECHNOLOGY LICENSING, LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: AKKAYA, ONUR CAN, BAMJI, CYRUS, BENEDETTI, ARRIGO, FENTON, MICHAEL S., GULLAPALLI, JAYACHANDRA
2017-08-10 Publication of US20170230551A1 publication Critical patent/US20170230551A1/en
2019-09-10 Publication of US10412280B2 publication Critical patent/US10412280B2/en
arranged in parallel over the plurality of sensor elements of the sensor array, two or more light valves having two or more polarizable liquid-crystal layers arranged between fixed polarizers, wherein each light valve is switchable electronically between a closed state and an open state and configured to, in the closed state, block light of a visible stopband and transmit light outside the visible stopband, and, in the open state, transmit the light of the visible stopband, wherein the visible stopband differs for each of the two or more light valves; and
coupled operatively to the illumination sensor and to each of the two or more light valves, an electronic controller configured to individually and sequentially switch the light valve from the closed state to the open state and synchronously address the sensor elements of the sensor array, such that each sensor element responds, in sequence, to the light of the visible stopband of each of the two or more light valves,
wherein the controller is configured to maintain each of the two or more light valves in the closed state for a repeatable closed duration and in the open state for a repeatable open duration, and to vary the closed and open durations responsive to output of the illumination sensor.
2. The camera of claim 1 wherein the stopband of each of the two or more light valves is a visible band.
3. The camera of claim 1 wherein, for each of the two or more light valves, the light outside the stopband includes infrared light.
5. The camera of claim 4 wherein addressing the sensor elements includes, in the closed state of the two or more light valves, resolving a phase offset from each of the plurality of sensor elements relative to infrared-emitter modulation.
6. The camera of claim 1 further comprising one or more passive filters configured to limit response of the sensor array.
7. The camera of claim 1 wherein at least one of the liquid-crystal layer and at least one of the fixed polarizers of each of the two or more light valves exhibit wavelength-dependent transmissivity.
8. The camera of claim 7 wherein at least one of the fixed polarizers provide a higher contrast ratio in the stopband than outside the stopband.
9. The camera of claim 7 wherein the fixed polarizers of each of the two or more light valves include upstream and downstream polarizers with mutually perpendicular transmission planes.
10. The camera of claim 1 wherein the two or more light valves include an optically downstream light valve coupled directly to the sensor array or to a microlens array portion of the sensor array.
11. A combination depth and flat-image camera comprising:
arranged in parallel over the plurality of sensor elements of the sensor array, two or more light valves having two or more polarizable liquid-crystal layers arranged between fixed polarizers, wherein each light valve is switchable electronically between a closed state and an open state and configured to, in the closed state, block light of a visible stopband and transmit light outside the visible stopband, and, in the open state, transmit the light of the visible stopband, wherein the visible stopband differs for each of the two or more light valves;
coupled operatively to the infrared emitter and to each of the two or more light valves, an electronic controller configured to individually and sequentially switch each light valve from the closed state to the open state and synchronously address the sensor elements of the sensor array, such that each sensor element responds, in sequence, to the light of the visible stopband of each of the two or more light valves,
wherein the controller is further configured to maintain each of the two or more light valves concurrently in the closed state, synchronously energize the infrared emitter and address the sensor elements of the sensor array to resolve subject depth at each sensor element based on a time-of-flight measurement.
12. The camera of claim of claim 11 wherein resolving depth to an image locus includes resolving a phase offset from each sensor element relative to infrared-emitter modulation.
13. The camera of claim 11 wherein wherein addressing the sensor elements of the sensor array includes integrating light intensity in three or more color channels.
14. The camera of claim 11 wherein the controller is further configured to:
switch the two or more light valves to the closed state;
integrate light intensity received at each element of the sensor array to assemble a reference image while the two or more light valves are in the closed state; and
subtractively correct, based on the reference image, subsequent light intensities integrated when one of the two or more light valves is in the open state.
coupled operatively to each of the two or more light valves, an electronic controller configured to individually and sequentially switch each light valve from the closed state to the open state and synchronously address the sensor elements of the sensor array, such that each sensor element responds, in sequence, to the light of the visible stopband of each of the two or more light valves.
16. The camera of claim 11 wherein the time-of-flight measurement is a phase-discriminating time-of-flight measurement.
17. The camera of claim 15 wherein the two or more light valves include an upstream light valve and a downstream light valve downstream of the upstream light valve, wherein the fixed polarizers of each of the upstream and downstream light valves include an entry polarizer and an exit polarizer, and wherein the exit polarizer of the upstream light valve and the entry polarizer of the downstream light valve are oriented in parallel.
18. The camera of claim 15 wherein the two or more light valves include a first light valve having a stopband in a red spectral region, a second light valve having a stopband in a green spectral region, and a third light valve having a stopband in a blue spectral region.
US15/040,862 2016-02-10 2016-02-10 Camera with light valve over sensor array Active 2036-07-19 US10412280B2 (en)
US15/040,862 US10412280B2 (en) 2016-02-10 2016-02-10 Camera with light valve over sensor array
EP17705239.6A EP3414897A1 (en) 2016-02-10 2017-02-01 Camera with an electronically switchable light valve over the sensor array for selectively blocking wavelengths in a stopband
CN201780010735.4A CN108605117A (en) 2016-02-10 2017-02-01 The camera of light valve on belt sensor array
PCT/US2017/015926 WO2017139143A1 (en) 2016-02-10 2017-02-01 Camera with an electronically switchable light valve over the sensor array for selectively blocking wavelengths in a stopband
US20170230551A1 US20170230551A1 (en) 2017-08-10
US10412280B2 true US10412280B2 (en) 2019-09-10
ID=58044181
US15/040,862 Active 2036-07-19 US10412280B2 (en) 2016-02-10 2016-02-10 Camera with light valve over sensor array
US (1) US10412280B2 (en)
EP (1) EP3414897A1 (en)
CN (1) CN108605117A (en)
WO (1) WO2017139143A1 (en)
US3848129A (en) 1973-08-24 1974-11-12 Sanders Associates Inc Spectral discriminating radiation detection apparatus
JPH1051668A (en) 1996-08-06 1998-02-20 Iseki & Co Ltd Image pickup device for agricultural robot
US20020030755A1 (en) 2000-09-11 2002-03-14 Fumiko Uchino Digital image sensing apparatus, image processing system, and digital image sensing method
JP2003198898A (en) 2001-12-28 2003-07-11 Matsushita Electric Works Ltd Solid-state imaging device camera and intercom with camera
US20040252230A1 (en) 2003-06-13 2004-12-16 Microsoft Corporation Increasing motion smoothness using frame interpolation with motion analysis
US7155363B1 (en) 1997-12-01 2006-12-26 Mks Instruments, Inc. Thermal imaging for semiconductor process monitoring
US20070087564A1 (en) 1998-10-14 2007-04-19 Stuart Speakman Method of forming an electronic device
US20140078459A1 (en) 2010-10-29 2014-03-20 Seong Jin Kim Filter for selective transmission of visible rays and infrared rays using an electrical signal
US20140232912A1 (en) 2011-10-06 2014-08-21 Sony Corporation Image pickup unit and electronic apparatus
US20140263991A1 (en) 2011-05-24 2014-09-18 Board Of Regents, The University Of Texas System Methods and apparatus for optically encoded position multiple-point scintillation detector using a single collecting light guide
US20150092059A1 (en) 2007-10-04 2015-04-02 Magna Electronics Inc. Imaging system for vehicle
US20150200220A1 (en) 2014-01-14 2015-07-16 Microsoft Corporation Image sensing system
US20150009205A1 (en) * 2013-07-03 2015-01-08 ArcSoft ( Hangzhou) Multimedia Technology Co., Ltd. Method of Improving Ericsson Texture Compression 1 Image Quality
2016-02-10 US US15/040,862 patent/US10412280B2/en active Active
2017-02-01 EP EP17705239.6A patent/EP3414897A1/en active Pending
2017-02-01 WO PCT/US2017/015926 patent/WO2017139143A1/en active Application Filing
2017-02-01 CN CN201780010735.4A patent/CN108605117A/en active Search and Examination
"3DV Systems-Zmini-Discontinued," ThingLab Website, Available Online at www.thinglab.co.uk/scanning_product.php?URL_=product_digiscan_3dvsystems_zmini&SubCatID_=53, Retrieved Jul. 13, 2009, 2 pages.
"3DV Systems—Zmini—Discontinued," ThingLab Website, Available Online at www.thinglab.co.uk/scanning_product.php?URL_=product_digiscan_3dvsystems_zmini&SubCatID_=53, Retrieved Jul. 13, 2009, 2 pages.
Chen, et al., "Single Camera Imaging System for Color and Near-Infrared Fluorescence Image Guided Surgery", In Journal of Biomedical Optics Express, vol. 5, Issue 8, Jul. 25, 2014, 7 pages.
Cho, J. et al., "Depth Image Processing Technique for Representing Human Actors in 3DTV Using Single Depth Camera," Proceedings of 3DTV Conference, May 2007, 4 pages.
Eijk, R., "Beyond the flat screen: Minimal and optimal camera-base distances for viewing 3-D images," Master's Thesis, Eindhoven University of Technology, Aug. 2003, 87 pages.
European Patent Office, Office Action Issued in European Patent Application No. 10821000.6, dated Apr. 20, 2015, Germany, 4 pages.
European Patent Office, Office Action Issued in European Patent Application No. 10821000.6, dated Oct. 21, 2014, Germany, 6 pages.
European Patent Office, Search Report Issued in European Patent Application No. 10821000.6, dated Sep. 29, 2014, Germany, 3 pages.
ISA European Patent Office, International Search Report and Written Opinion Issued in PCT Application No. PCT/US2017/015926, dated Apr. 20, 2017, WIPO, 16 Pages.
ISA Korean Intellectual Property Office, International Search Report and Written Opinion Issued in Application No. PCT/US2010/047564, dated Apr. 27, 2011, WIPO, 8 pages.
Japanese Patent Office, Office Action Issued in Japanese Patent Application No. 2012-532098, dated Apr. 22, 2014, 5 pages.
Japanese Patent Office, Office Action Issued in Japanese Patent Application No. 2012-532098, dated Dec. 25, 2014, 4 pages.
Kim, et al., "A 1.5Mpixel RGBZ CMOS Image Sensor for Simultaneous Color and Range Image Capture", In Proceedings of IEEE International Solid-State Circuits Conference Digest of Technical Papers, Feb. 19, 2012, pp. 392-393.
Lu, et al., "Designing Color Filter Arrays for the Joint Capture of Visible and Near-Infrared Images", In Proceedings of 16th IEEE International Conference on Image Processing, Nov. 7, 2009, pp. 3797-3800.
Schechner, Y. et al., "Generalized Mosaicing," Proceedings of the Eighth IEEE International Conference on Computer Vision, ICCV 2001, vol. 1, Vancover, BC, Jul. 7, 2001, 8 pages.
Sinha, S., "Calibration of a Heterogeneous Network of Color and Depth Cameras," Idea for Canesta Vision Contest, Dec. 2004, 3 pages.
State Intellectual Property Office of the People's Republic of China, First Office Action Issued in Chinese Patent Application No. 201080043779.5, dated Apr. 3, 2013, 11 pages.
State Intellectual Property Office of the People's Republic of China, Second Office Action Issued in Chinese Patent Application No. 201080043779.5, dated Sep. 22, 2013, 6 pages.
State Intellectual Property Office of the People's Republic of China, Third Office Action Issued in Chinese Patent Application No. 201080043779.5, dated Feb. 7, 2014, 6 pages.
Wang, O. et al., "Automatic Natural Video Matting with Depth," 15th Pacific Conference on Computer Graphics and Applications, Oct. 2007, 4 pages.
EP3414897A1 (en) 2018-12-19
WO2017139143A1 (en) 2017-08-17
US20170230551A1 (en) 2017-08-10
CN108605117A (en) 2018-09-28
JP2010011415A (en) 2010-01-14 Solid-state imaging device and camera
US20030052989A1 (en) 2003-03-20 Non-polarizing shutter/CCD module
JP2006507514A (en) 2006-03-02 Three-dimensional light engine architecture
CN103680367B (en) 2018-03-23 Quantum dot/remote fluorescence display system is improved
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:AKKAYA, ONUR CAN;BAMJI, CYRUS;BENEDETTI, ARRIGO;AND OTHERS;REEL/FRAME:037725/0360