Patent application EP19178782A (EP), filed 2019-06-06
Publication number: EP3748395A1
Title: METHOD AND APPARATUS FOR COMPENSATING STRAY LIGHT CAUSED BY AN OBJECT IN A SCENE THAT IS SENSED BY A TIME-OF-FLIGHT CAMERA
Abstract: [0001]    A method 100 for compensating stray light caused by an object in a scene that is sensed by a time-of-flight camera is provided. The method includes receiving 102 an image of the scene from the time-of-flight camera. Further, the method includes controlling 104 the time-of-flight camera to capture a reference image of the scene using a code modulated signal for illumination such that a measurement range of the time-of-flight camera is limited to a distance range around the object. The method additionally includes modifying 106 the image of the scene or an image derived therefrom using the reference image to obtain a compensated image of the scene. The method includes outputting 108 the compensated image. Method 100 may allow to compensate for stray light caused by an object (e.g. a highly reflective object or an object close to the ToF camera) in a scene that is sensed by a Time-of-Flight (ToF) camera. The stray light emitted the object in the scene may be characterized b y capturing the scene using a code modulated signal for illumination such that the measurement range of the ToF camera is limited to a distance range around the object causing the stray light. T he image of the scene or the image derived therefrom may be modified pixelwise using the reference image. Hence, erroneous measurements by the ToF camera as represented by the pixels of the image of the scene may be at least mitigated in the compensated image of the scene. In some examples, the compensated image of the scene may be free from effects of the light reflections (stray light) from the object. Tthe ToF camera uses a continuous wave modulated signal for illuminating the scene and generates measurement signals based on reflected light from the scene for capturing the imaging of the scene. Method 100 may comprise adjusting the modulation code such that the code modulated signal correlates like a continuous wave function in the range of the reflective surface.
IPC codes: G01S   7/   487            A I
G01S   7/   493            A I
G01S  17/    32            A I
G01S  17/   894            A I
Claims: number: 1
text: A method (100) for compensating stray light caused by an object in a scene that is sensed by a time-of-flight camera, the method comprising:
number: 2
text: The method of claim 1, wherein the image of the scene is one of a raw image or an image derived from the raw image.
number: 3
text: The method of claim 1 or claim 2, further comprising:
determining that the object causes stray light based on the image of the scene.
number: 4
text: The method of claim 3, wherein, for capturing the image of the scene, the time-of-flight camera uses a continuous wave modulated signal for illuminating the scene and generates measurement signals based on reflected light from the scene, wherein the image of the scene is based on correlations of the continuous wave modulated signal and the measurement signals according to a correlation function, and wherein determining that the object causes stray light comprises:
number: 5
text: The method of any of claims 1 to 4, further comprising:
number: 6
text: The method of claim 5, wherein adjusting the code modulated signal comprises selecting one of a plurality of predefined code modulated signals based on the distance of the object to the time-of-flight camera.
number: 7
text: The method of claim 5 or claim 6, wherein adjusting the code modulated signal is further based on information about a movement of the time-of-flight camera after capturing the image of the scene.
number: 8
text: The method of any of claims 1 to 7, wherein modifying (106) the image of the scene or the image derived therefrom using the reference image comprises:
subtracting reference values indicated by the pixels of the reference image from values indicated by the pixels of the image of the scene or the image derived therefrom.
number: 9
text: The method of claim 8, wherein the reference values indicate a phase shift between the code modulated signal and measurement signals generated by the time-of-flight camera based on reflected light from the scene.
number: 10
text: The method of claim 8, wherein the reference values indicate phasors.
number: 11
text: The method of any of claims 1 to 10, wherein modifying (106) the image of the scene or an image derived therefrom using the reference image comprises:
number: 12
text: The method of any of claims 1 to 11, wherein the image of the scene is a raw image of a first series of raw images captured by the time-of-flight camera or is derived from the first series of raw image, wherein the reference image is a raw image of a second series of raw images captured by the time-of-flight camera or is derived from the second series of raw image, wherein the first series of raw images comprises more raw images than the second series of raw images.
number: 13
text: The method of any of claims 1 to 12, wherein at least one pixel of the compensated image indicates a distance to the time-of-flight camera based on the modification of the image of the scene or the image derived therefrom using the reference image and additionally indicates a distance of the object to the time-of-flight camera.
number: 14
text: The method of any of claims 1 to 13, further comprising:
number: 15
text: An apparatus (800) for compensating stray light caused by an object in a scene that is sensed by a time-of-flight camera, the apparatus comprising:
Office action: search_report, dated 2020-12-09
Rejection reasons: novelty_epc_54
inventive_step_epc_56
Prior art cited: ref_id: US2011149071A1
ref_type: patent
severity: background
ref_id: WO2015075413A1
ref_type: patent
severity: background
ref_id: EP3015881A2
ref_type: patent
severity: background
ref_id: US6714286B1
ref_type: patent
severity: background
ref_id: US6714286
ref_type: patent
severity: obviousness
ref_id: US2011149071
ref_type: patent
severity: novelty_destroying
ref_id: WO2015075413
ref_type: patent
severity: obviousness
ref_id: EP3015881
ref_type: patent
severity: obviousness
Final disposition: pending (2020-12-09)