Patent application EP19182040A (EP), filed 2019-06-24
Publication number: EP3757945A1
Title: DEVICE FOR GENERATING AN AUGMENTED REALITY IMAGE
Abstract: [0001]    The present invention relates to the technical field of augmented reality (AR). In particular, the invention presents a device and a method, respectively, for generating an AR image (or AR video). The device is configured to obtain a first image of a scene, obtain a second image of an object, obtain a depth range, and capture a third image of only the parts of the scene inside the depth range. Further, the device is configured to generate occlusion information based on the third image, and overlay the first image and the second image based on the occlusion information, in order to generate the AR image.
IPC codes: G06T  15/    50            A I
G06T  19/    00            A I
Claims: number: 1
text: A device (10) for generating an Augmented Reality, AR, image (11), the device (10) being configured to:
number: 2
text: The device (10) according to claim 1, configured to, in order to generate the AR image (11):
number: 3
text: The device (10) according to claim 1 or 2, wherein:
the occlusion information (16) is a binary occlusion map.
number: 4
text: The device (10) according to claim 3, wherein:
the occlusion map has the same pixel resolution as the third image (15).
number: 5
text: The device (10) according to one of the claims 1 to 4, wherein:
the occlusion information (16) identifies the parts of the first image (12) that show the scene (12a) behind the depth range, and the parts of the first image (12) that show the scene (12a) in front of the depth range.
number: 6
text: The device (10) according to one of the claims 1 to 5, wherein:
the occlusion information (16) identifies the parts of the first image (12) that are to be replaced by the second image (13) when overlaying the first image (12) and the second image (13), in order to generate the AR image (11).
number: 7
text: The device (10) according to one of the claims 1 to 6, wherein:
the third image (15) and/or the occlusion map (16) has a field of view that is substantially the same as the field of view of the first image (12).
number: 8
text: The device (10) according to one of the claims 1 to 7, comprising:
an imaging device (20) configured to capture the first image (12) and the third image (15).
number: 9
text: The device (10) according to one of the claims 1 to 8, comprising:
a range gated imager (20) configured to capture the third image (15).
number: 10
text: The device (10) according to one of the claims 1 to 9, further configured to:
display the generated AR image (11).
number: 11
text: The device (10) according to one of the claims 1 to 10, wherein:
number: 12
text: Method (30) of generating an Augmented Reality, AR, image (11), the method (30) comprising:
number: 13
text: Computer program product comprising a program code for carrying out, when implemented on an imaging device, the method (12) according to claim 12.
Office action: search_report, dated 2020-12-30
Rejection reasons: novelty_epc_54
inventive_step_epc_56
Prior art cited: ref_id: US2019058859A1
ref_type: patent
severity: background
ref_id: US2014282275A1
ref_type: patent
severity: background
ref_id: - JAN FISCHER ET AL: "Using time-of-flight range data for occlusion handling in augmented reality", JVRC12: JOINT VIRTUAL REALITY CONFERENCE OF ICAT - EGVE - EUROVR, 15 July 2007 (2007-07-15), pages 109 - 116, XP055239237, ISSN: 1727-530X, ISBN: 978-3-905674-40-8, DOI: 10.2312/EGVE/IPT_EGVE2007/109-116
ref_type: npl
severity: background
ref_id: - KOCH REINHARD ET AL: "MixIn3D: 3D Mixed Reality with ToF-Camera", 9 September 2009, INTERNATIONAL CONFERENCE ON COMPUTER ANALYSIS OF IMAGES AND PATTERNS. CAIP 2017: COMPUTER ANALYSIS OF IMAGES AND PATTERNS; [LECTURE NOTES IN COMPUTER SCIENCE; LECT.NOTES COMPUTER], SPRINGER, BERLIN, HEIDELBERG, PAGE(S) 126 - 141, ISBN: 978-3-642-17318-9, XP047390416
ref_type: npl
severity: background
ref_id: WO2017106053A1
ref_type: patent
severity: background
ref_id: - SUYANG DONG ET AL: "Resolving Incorrect Visual Occlusion in Outdoor Augmented Reality Using TOF Camera and OpenGL Frame Buffer", 4 November 2010 (2010-11-04), XP055245057, Retrieved from the Internet <URL:http://pathfinder.engin.umich.edu/documents/Dong&Kamat.CONVR.2010.pdf> [retrieved on 20160127]
ref_type: npl
severity: background
ref_id: WO2017106053
ref_type: patent
severity: obviousness
ref_id: - Suyang Dong ET AL, "Resolving Incorrect Visual Occlusion in Outdoor Augmented Reality Using TOF Camera and OpenGL Frame Buffer", (20101104), URL: http://pathfinder.engin.umich.edu/documents/Dong&Kamat.CONVR.2010.pdf, (20160127), XP055245057 [X] 1-8,10-13 * section 2; figure 2 * [Y] 9
ref_type: npl
severity: novelty_destroying
Final disposition: pending (2020-12-30)