Source: http://www.google.com.ar/patents/US9762793
Timestamp: 2018-01-19 11:24:39
Document Index: 264801294

Matched Legal Cases: ['Application No. 14157971', 'Application No. 15176943', 'Application No. 15188440', 'Application No. 16152477', 'Application No. 16168216', 'Application No. 15190306', 'Application No. 16172995', 'Application No. 16173429', 'Application No. 14157971', 'Application No. 14157971', 'Application No. 15188440', 'Application No. 15189214', 'Application No. 16152477', 'Application No. 16168216', 'Application No. 15190315', 'Application No. 16175410', 'art 11', 'Application No. 13186043', 'art 11', 'Application No. 1517112', 'Application No. 2015220810562', 'Application No. 1607394']

Patent US9762793 - System and method for dimensioning - Google Patents
A system and method for structured-light dimensioning is disclosed. The method includes combining multiple images using different camera settings to provide all of the information necessary dimensioning. What results is an improved ability to sense a light pattern reflected from an object's surfaces,...http://www.google.com.ar/patents/US9762793?utm_source=gb-gplus-sharePatent US9762793 - System and method for dimensioning
Publication number US9762793 B2
Application number US 14/519,211
Also published as CN205621075U, EP3012579A1, US20160112631
Publication number 14519211, 519211, US 9762793 B2, US 9762793B2, US-B2-9762793, US9762793 B2, US9762793B2
Patent Citations (423), Non-Patent Citations (105), Classifications (10), Legal Events (1)
US 9762793 B2
1. A structured-light dimensioning system, comprising:
a projector subsystem for projecting a light pattern onto an object having dimensioning surfaces to cause pattern distortions of the light pattern;
the software program when executed by the processor configures the control subsystem to (i) enable the camera subsystem to capture multiple pattern images, each pattern image captured using different camera-subsystem settings, (ii) incorporate the multiple pattern images into an image composite with a resolvable light pattern on the dimensioning surfaces, and (iii) use the pattern distortions in the image composite to compute a range.
2. The structured-light dimensioning system according to claim 1, wherein the structured-light dimensioning system is capable of being handheld.
3. The structured-light dimensioning system according to claim 2, wherein the image composite comprises multiple pattern images aligned and combined to remove differences caused by hand motion.
4. The structured-light dimensioning system according to claim 1, wherein the projected light pattern is an infrared (IR) pattern of dots.
5. The structured-light dimensioning system according to claim 1, wherein the resolvable light pattern comprises at least one visible pattern element.
6. The structured-light dimensioning system according to claim 1, wherein the composite image comprises segmented pattern images.
7. The structured-light dimensioning system according to claim 1, wherein the composite image comprises a single image that is a mathematical combination of pattern images.
8. The structured-light dimensioning system according to claim 1, wherein the camera-subsystem settings comprise an image-sensor shutter speed.
9. The structured-light dimensioning system according to claim 1, wherein the camera-subsystem settings comprise an imaging lens aperture size.
10. The structured-light dimensioning system according to claim 1, wherein the software program configures the control subsystem to identify dimensioning surfaces in a pattern image.
11. The structured-light dimensioning system according to claim 1, wherein the software program configures the control subsystem to compute an object volume using the image composite.
12. A method to compute an object's dimensions using a structured-light dimensioning system, the method comprising:
projecting a light pattern onto an object to cause pattern distortions of the light pattern;
capturing a pattern image of the light pattern projected onto the object;
processing the image composite to compute the object's dimensions, comprising using pattern distortions in the image composite to compute a range.
13. The method according to claim 12, wherein at least three dimensioning surfaces are selected and the object's dimension is the object's volume.
14. The method according to claim 12, wherein the image composite comprises at least two pattern images.
15. The method according to claim 12, wherein the image composite comprises a combination of pattern images into a single image.
16. The method according to claim 12, wherein the requisite pattern comprises at least one discernable pattern element.
17. The method according to claim 12, wherein, the processing of the image composite comprises computing at least one range between the structured light dimensioning system and at least one dimensioning surface.
18. The method according to claim 12, wherein the camera subsystem settings comprises an image sensor's exposure time.
19. The method according to claim 12, wherein the camera subsystem settings comprises an image-lens aperture size.
20. A structured-light dimensioning system, comprising:
the software program when executed by the processor configures the control subsystem to (i) enable the camera subsystem to capture multiple pattern images, each pattern image captured using different camera-subsystem settings, (ii) incorporate the multiple pattern images into an image composite with a resolvable light pattern on the dimensioning surfaces, and (iii) use the pattern distortions in the image composite to compute a volume of the object.
FIG. 1 schematically depicts a block diagram representing an exemplary method to compute an object's dimensions using a structured-light dimensioning system.
FIG. 2 schematically depicts an exemplary embodiment of a structured-light dimensioning system block diagram.
A block diagram representing an exemplary method to compute an object's dimensions using a structured-light dimensioning system (e.g., handheld structured-light dimensioning system) is shown in FIG. 1. The method 1 starts with an object 10 for dimensioning. This object 10 is typically a package and the goal of the method 1 is typically to compute the volume of this package. The object size is typically constrained within a prescribe range, and the object typically must be placed roughly at a prescribed distance from the dimensioner during measurement. The exact prescribed values are related to a projector subsystem that projects the light pattern and a camera subsystem that captures a pattern image of the light pattern reflected from the object.
The dimensioner captures a pattern image 11 by projecting a light pattern onto the object and then capturing an image of the object with the overlaid light pattern. The light projected may be infrared (IR), visible, or ultraviolet (UV) but is typically IR. The light may be pulsed or continuous during the dimensioning process. The pattern may be dynamic or static and may consist of projected patterns including geometrical shapes (e.g., hexagons or line grids), dot arrays, and/or Du Bruijn diagrams. In an exemplary dot pattern (i.e., the pattern elements are dots) the size, shape, and distribution of the dot pattern are known and may be stored in memory as a reference pattern.
An exemplary embodiment of a structured-light dimensioning system 100 (i.e., dimensioner) block diagram is shown in FIG. 2. A dimensioner 100 may be positioned with an object 10 in its field of view 24. The object may have its dimensions (e.g., volume) measured remotely. To accomplish this measurement, the dimensioner 100 utilizes a variety of subsystems.
A camera subsystem 30 captures pattern images of the object 10 and the projected light pattern 23. To accomplish this, the camera subsystem 30 may use an imaging lens 31 to render a real image of the imaging lens's field of view 24 onto an image sensor 32. This imaging lens field of view 24 overlaps at least partially with the projected light pattern 23. The image sensor 32 may be a charge coupled device (i.e., CCD) or a sensor using complementary metal oxide semiconductor (i.e., CMOS) technology. The image sensor 32 includes a plurality of pixels that sample the real image and convert the real-image intensity into an electronic signal. An imager digital signal processor (i.e., DSP) 33 is typically included to convert the electronic signals from the image sensor 32 into a digital signal.
A control subsystem 40 is communicatively coupled to the projector subsystem 20 and the camera subsystem 30 via an interconnection system (e.g., bus) 50, which interconnects all of the dimensioners subsystems. The control subsystem 40 includes one or more processors 42 (e.g., one or more controllers, digital signal processor (DSP), application specific integrated circuit (ASIC), programmable gate array (PGA), and/or programmable logic controller (PLC)) to configure subsystems for the generation and capturing processes and then perform the processing necessary on pattern images and the image composite necessary for dimensioning. The processor 42 is typically configured by a software program stored in memory 41 (e.g., read only memory (ROM), flash memory, random access memory (RAM), and/or a hard-drive). The software program, when executed by the processor 42 configures the control subsystem to enable the camera subsystem 20 to: (i) capture multiple pattern images, each pattern image captured using different camera-subsystem settings and (ii) incorporate the multiple pattern images into an image composite with a resolvable light pattern on all dimensioning surfaces.
US2010701 10 Jan 1935 6 Aug 1935 Gen Electric Time delay undervoltage protective system
US8773507 * 9 Aug 2010 8 Jul 2014 California Institute Of Technology Defocusing feature matching system to measure camera pose with interchangeable lens cameras
US9247235 * 15 Jul 2013 26 Jan 2016 California Institute Of Technology Method and device for high-resolution imaging which obtains camera pose using defocusing
US20020067855 24 Jul 2001 6 Jun 2002 Ming-Yee Chiu Method and arrangement for camera calibration
US20080273210 29 Oct 2007 6 Nov 2008 U.S. Of A. As Represented By Secretary Of The Navy Three dimensional shape correlator
US20107018294 Title not available
US20140097238 19 Dec 2012 10 Apr 2014 Mansoor Ghazizadeh Measurement using a calibraton pattern
US20140104416 * 16 Oct 2013 17 Apr 2014 Hand Held Products, Inc. Dimensioning system
US20140192187 * 8 Jan 2014 10 Jul 2014 Faro Technologies, Inc. Non-contact measurement device
US20140347553 24 May 2013 27 Nov 2014 Samsung Electronics Co., Ltd. Imaging devices with light sources for reduced shadow, controllers and methods
WO2000077726A1 16 Jun 2000 21 Dec 2000 Psc Inc. Method and apparatus for calibration of an image based verification device
3 Chinese Notice of Reexamination in related Chinese Application 201520810313.3, dated Mar. 14, 2017, English computer Translation provided, 7 pages [No new art cited].
4 Collings et al., "The Applications and Technology of Phase-Only Liquid Crystal on Silicon Devices", Journal of Display Technology, IEEE Service Center, New, York, NY, US, vol. 7, No. 3, Mar. 1, 2011 (Mar. 1, 2011), pp. 112-119.
5 Decision to Grant in counterpart European Application No. 14157971.4 dated Aug. 6, 2015, pp. 1-2.
6 Dimensional Weight-Wikipedia, the Free Encyclopedia, URL=http://en.wikipedia.org/wiki/Dimensional-weight, download date Aug. 1, 2008, 2 pages.
7 Dimensional Weight—Wikipedia, the Free Encyclopedia, URL=http://en.wikipedia.org/wiki/Dimensional—weight, download date Aug. 1, 2008, 2 pages.
8 Dimensioning-Wikipedia, the Free Encyclopedia, URL=http://en.wikipedia.org/wiki/Dimensioning, download date Aug. 1, 2008, 1 page.
9 Dimensioning—Wikipedia, the Free Encyclopedia, URL=http://en.wikipedia.org/wiki/Dimensioning, download date Aug. 1, 2008, 1 page.
13 El-Hakim et al., "Multicamera vision-based approach to flexible feature measurement for inspection and reverse engineering", published in Optical Engineering, Society of Photo-Optical Instrumentation Engineers, vol. 32, No. 9, Sep. 1, 1993, 15 pages.
16 European Exam Report in related EP Applciation 16172995.9, dated Jul. 6, 2017, 9 pages [No art to be cited].
17 European Exam Report in related EP Application No. 15176943.7, dated Apr. 12, 2017, 6 pages [Art previously cited in this matter].
18 European Exam Report in related EP Application No. 15188440.0, dated Apr. 21, 2017, 4 pages [No new art to cite].
19 European Exam Report in related EP Application No. 16152477.2, dated Jun. 20, 2017, 4 pages [No art to be cited].
20 European Exam Report in related, EP Application No. 16168216.6, dated Feb. 27, 2017, 5 pages, [References lave been previously cited; WO2011/017241 and U.S. 2014/0104413].
23 European extended search report in related EP Application 16190833.0, dated Mar. 9, 2017, 8 pages [only new art has been cited; U.S. Publication 2014/0034731 was previously cited].
24 European Extended search report in related EP Application No. 15190306.9, dated Sep. 9, 2016, 15 pages [only new references are cited; remaining references were cited with partial search report in same application dated May 6, 2016].
25 European Extended Search Report in Related EP Application No. 16172995.9, dated Aug. 22, 2016, 11 pages (Only new references have been cited; U.S. Pat. No. 8,463,079 (formerly U.S. Publication 201010220894) and U.S. Publication 2001/0027955 have been previously cited.).
26 European Extended Search Report in related EP Application No. 16173429.8, dated Dec. 1, 2016, 8 pages [Only new references cited: U.S. 2013/0038881 was previously cited].
30 European Patent Office Action for Application No. 14157971.4-1906, dated Jul. 16, 2014, 5 pages.
31 European Patent Search Report for Application No. 14157971.4-1906, dated Jun. 30, 2014, 6 pages.
32 European Search Report for application No. EP13186043 (now EP2722656 (Apr. 23, 2014)): Total pp. 7.
33 European Search Report for related Application EP 15190249.1, dated Mar. 22, 2016, 7 pages.
34 European Search Report for related EP Application No. 15188440.0, dated Mar. 8, 2016, 8 pages.
35 European Search Report for Related EP Application No. 15189214.8, dated Mar. 3, 2016, 9 pages.
36 European Search Report for related EP Application No. 16152477.2, dated May 24, 2016, 8 pages [New Reference cited herein; Reference DE102007037282 A1 and its U.S. Counterparts have been previously cited.].
37 European Search Report from related EP Application No. 16168216.6, dated Oct. 20, 2016, 8 pages. [New reference cited above; U.S. Publication 2014/0104413 has been previously cited].
38 European Search Report in related EP Application No. 15190315.0, dated Apr. 1, 2016, 7 pages [Commonly owned Reference 2014/0104416 has been previously cited].
39 Extended European search report in related EP Application 16199707.7, dated Apr. 10, 2017, 15 pages.
40 Extended European Search Report in related EP Application No. 16175410.0, dated Dec. 13, 2016, 5 pages.
41 Fukaya et al., "Characteristics of Speckle Random Pattern and Its Applications", pp. 317-327, Nouv. Rev. Optique, t. 6, n. 6. (1975) {Cited by Examiner in Feb. 9, 2017 Final Office Action in related matter: downloaded Mar. 2, 2017 from http://iopscience.iop.org}.
42 Grabowski, Ralph; "New Commands in AutoCADS 2010: Part 11 Smoothing 3D Mesh Objects" Dated 2011 (per examiner who cited reference), 6 pages, [Examiner Cited Art in Office Action dated Jan. 20, 2017 in related Application.].
43 Great Britain Search Report for related Application On. GB1517843.7, dated Feb. 23, 2016; 8 pages.
44 Gupta, Alok; Range Image Segmentation for 3-D Objects Recognition, May 1988, Technical Reports (CIS), Paper 736, University of Pennsylvania Department of Computer and Information Science, retrieved from Http://repository.upenn. edu/cis-reports/736, Accessed May 31, 2015, 157 pages.
45 Gupta, Alok; Range Image Segmentation for 3-D Objects Recognition, May 1988, Technical Reports (CIS), Paper 736, University of Pennsylvania Department of Computer and Information Science, retrieved from Http://repository.upenn. edu/cis—reports/736, Accessed May 31, 2015, 157 pages.
46 HETZEL G., LEIBE B., LEVI P., SCHIELE B.: "3D object recognition from range images using local feature histograms", PROCEEDINGS 2001 IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION. CVPR 2001. KAUAI, HAWAII, DEC. 8 - 14, 2001., IEEE COMPUTER SOCIETY, LOS ALAMITOS, CALIF. [U.A.], vol. 2, 8 December 2001 (2001-12-08) - 14 December 2001 (2001-12-14), Los Alamitos, Calif. [u.a.], pages 394 - 399, XP010584149, ISBN: 978-0-7695-1272-3
47 Hetzel, Gunter et al.; "3D Object Recognition from Range Images using Local Feature Histograms,", Proceedings 2001 IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2001. Kauai, Hawaii, Dec. 8- 14, 2001; pp. 394-399, XP010584149, ISBN: 978-0-7695-1272-3.
48 Hood, Frederick W.; William A. Hoff, Robert King, Evaluation of an Interactive Technique for Creating Site Models from Range Data, Apr. 27-May1, 1997 Proceedings of the ANS 7th Topical Meeting on Robotics & Remote Systems, Augusta GA, 9 pages.
51 James Chamberlin, "System and Method for Picking Validation", U.S. Appl. No. 14/865,797, filed Sep. 25, 2015, 44 pages, not yet published.
53 Kazantsev, Aleksei et al. "Robust Pseudo-Random Coded Colored STructured Light Techniques for 3D Object Model Recovery"; ROSE 2008 IEEE International Workshop on Robotic and Sensors Environments (Oct. 17-18, 2008), 6 pages.
54 Leotta, Matthew J.; Joseph L. Mundy; Predicting High Resolution Image Edges with a Generic, Adaptive, 3-D Vehicle Model; IEEE Conference on Computer Vision and Pattern Recognition, 2009; 8 pages.
58 M. Zahid Gurbuz, Selim Akyokus, Ibrahim Emiroglu, Aysun Guran, An Efficient Algorithm for 3D Rectangular Box Packing, 2009, Applied Automatic Systems: Proceedings of Selected AAS 2009 Papers, pp. 131-134 [Examiner cited art in related U.S. matter with Notice of Allowance dated Aug. 11, 2016].
62 Mouaddib E. et al. "Recent Progress in Structured Light in order to Solve the Correspondence Problem in Stereo Vision" Proceedings of the 1997 IEEE International Conference on Robotics and Automation, Apr. 1997; 7 pages.
63 Office Action in counterpart European Application No. 13186043.9 dated Sep. 30, 2015, pp. 1-7.
64 Padzensky, Ron; "Augmera; Gesture Control", Dated Apr. 18, 2015, 15 pages [Examiner Cited Art in Office Action dated Jan. 20, 2017 in related Application.].
65 Peter Clarke, Actuator Claims Anti-Shake Breakthrough for Smartphone Cams, Electronic Engineering Times, p. 24, May 16, 2011.
66 Proesmans, Marc et al. "Active Acquisition of 3D Shape for Moving Objects" 0-7803-3258-X/96 1996 IEEE; 4 pages.
67 Ralph Grabowski, "Smothing 3D Mesh Objects," New Commands in AutoCAD 2010: Part 11, Examiner Cited art in related matter Non Final Office Action dated May 19, 2017; 6 pages.
68 Reisner-Kollmann, lrene; Anton L. Fuhrmann, Werner Purgathofer, Interactive Reconstruction of Industrial Sites Using Parametric Models, May 2010, Proceedings of the 26th Spring Conference of Computer Graphics SCCG ″10, 8 pages.
70 Search Report and Opinion in Related EP Application 15176943.7, dated Jan. 8, 2016, 8 pages, (U.S. Application 2014/0049635 has been previously cited).
71 Search Report and Opinion in related GB Application No. 1517112.7, dated Feb. 19, 2016, 6 Pages (GB2503978 is commonly owned now abandoned application and not cited above).
75 Second Chinese Office Action in related CN Application No. 2015220810562.2, dated Mar. 22, 2016, 5 pages. English Translation provided [No references].
77 Still Optics, Examiner Cited NPL in Advisory Action dated Apr. 12, 2017 in related commonly owned application, http://www.silloptics.de/1/products/sill-encyclopedia/laser-optics/f-theta-lenses/, 4 pages.
79 Thorlabs, Examiner Cited NPL in Advisory Action dated Apr. 12, 2017 in related commonly owned application, downloaded from https://www.thorlabs.com/newgrouppage9.cfm?objectgroup-id=6430, 4 pages.
80 Thorlabs, Examiner Cited NPL in Advisory Action dated Apr. 12, 2017 in related commonly owned application, downloaded from https://www.thorlabs.com/newgrouppage9.cfm?objectgroup—id=6430, 4 pages.
82 U.S. Appl. No. 13/912,262, not yet published, filed Jun. 7, 2013, Hand Held Products Inc., Method of Error Correction Application for 3D Imaging Device: 33 pages.
83 U.S. Appl. No. 14/055,234, not yet published, Hand Held Products, Inc. filed Oct. 16, 2013; 26 pages.
84 U.S. Appl. No. 14/453,019, not yet published, filed Aug. 6, 2014, Hand Held Products Inc., Dimensioning System With Guided Alignment: 31 pages.
85 U.S. Appl. No. 14/461,524, not yet published, filed Aug. 18, 2014, Hand Held Products Inc., System and Method for Package Dimensioning: 21 pages.
86 U.S. Appl. No. 14/490,989, not yet published, filed Sep. 19, 2014, Intermec IP Corporation, Volume Dimensioning System Calibration Systems and Methods.
87 U.S. Appl. No. 14/715,916, H. Sprague Ackley, filed May 9, 2015, not published yet, Evaluating Image Values; 54 pages.
88 U.S. Appl. No. 14/740,373, H. Sprague Ackley et al., filed Jun. 16, 2015, not published yet, Calibrating A Volume Dimensioner; 63 pages.
89 U.S. Appl. No. 14/747,197, Serge Thuries et al., filed Jun. 23, 2015, not published yet, Optical Pattern PROJECTOR; 33 pages.
90 U.S. Appl. No. 14/747,490, Brian L. Jovanovski et al., filed Jun. 23, 2015, not published yet, Dual-PROJECTOR Three-Dimensional Scanner; 40 pages.
91 U.S. Appl. No. 14/793,149, H. Sprague Ackley, filed Jul. 7, 2015, not published yet, Mobile Dimensioner Apparatus for Use in Commerce; 57 pages.
92 U.S. Appl. No. 14/800,757, Eric Todeschini, filed Jul. 16, 2015, not published yet, Dimensioning and Imaging Items, 80 pages.
93 U.S. Appl. No. 14/801,023, Tyler Doomenbal et al., filed Jul. 16, 2015, not published yet, Adjusting Dimensioning Results Using Augmented Reality, 39 pages.
94 Ulusoy et al., One-Shot Scanning using De Bruijn Spaced Grids, 2009 IEEE 12th International Conference on computer Vision Workshops, ICCV Workshops, 7 pages [Cited in EP Extended search report dated Apr. 10, 2017].
95 United Kingdom Combined Search and Examination Report in related Application No. GB1620676.5, dated Mar. 8, 2017, 6 pages [References have been previously cited; WO2014/151746, WO2012/175731, U.S. 2014/0313527, GB2503978].
96 United Kingdom combined Search and Examination Report in related GB Application No. 1607394.2, dated Oct. 19, 2016, 7 pages.
97 United Kingdom Search Report in related application GB1517842.9, dated Apr. 8, 2016, 8 pages.
98 United Kingdom Search Report in related Application No. GB 1700338.5, dated Jun. 20, 2017, 5 pages.
99 Ward, Benjamin, Interactive 3D Reconstruction from Video, Aug. 2012, Doctoral Thesis, Univesity of Adelaide, Adelaide, South Australia, 157 pages.
100 Wikipedia, "3D projection" Downloaded on Nov. 25, 2015 from www.wikipedia.com, 4 pages.
101 Wikipedia, "Microlens", Downloaded from https://en.wikipedia.org/wiki/Microlens, pp. 3. {Cited by Examiner in Feb. 9, 2017 Final Office Action in related matter}.
102 Wikipedia, YUV description and definition, downloaded from http://www.wikipeida.org/wiki/YUV on Jun. 29, 2012, 10 pages.
103 YUV Pixel Format, downloaded from http://www.fource.org/yuv.php on Jun. 29, 2012; 13 pages.
104 YUV to RGB Conversion, downloaded from http://www.fource.org/fccyvrgb.php on Jun. 29, 2012; 5 pages.
105 Zhang, Zhaoxiang; Tieniu Tan, Kaiqi Huang, Yunhong Wang; Three-Dimensional Deformable-Model-based Localization and Recognition of Road Vehicles; IEEE Transactions on Image Processing, vol. 21, No. 1, Jan. 2012, 13 pages.
International Classification G01B11/30, H04N5/232, H04N5/33, G01B11/25, H04N5/225
Cooperative Classification G01B11/2518, H04N5/33, H04N5/2256, G01B11/25, H04N5/23222
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:ACKLEY, H. SPRAGUE;LAFFARGUE, FRANCK;REEL/FRAME:033988/0637