Patent Description:
Vision systems that perform measurement, inspection, alignment of objects and/or decoding of symbology in the form of machine-readable symbols (also termed "IDs", such as a 2D matrix symbol) are used in a wide range of applications and industries. These systems are based around the use of an image sensor, which acquires images (typically grayscale or color, and in one, two or three dimensions) of the subject or object, and processes these acquired images using an on-board or interconnected vision system processor. The processor generally includes both processing hardware and non-transitory computer-readable program instructions that perform one or more vision system processes to generate a desired output based upon the image's processed information. This image information is typically provided within an array of image pixels each having various colors and/or intensities. In the example of an ID reader (also termed herein, a "camera"), the user or automated process acquires an image of an object that is believed to contain one or more barcodes. The image is processed to identify barcode features, which are then decoded by a decoding process and/or processor obtain the inherent alphanumeric data represented by the code.

In operation, an ID reader typically functions to illuminate the scene containing one or more IDs. The illuminated scene is then acquired by an image sensor within the imaging system through optics. The array sensor pixels is exposed, and the electronic value(s) generated for each pixel by the exposure is/are stored in an array of memory cells that can be termed the "image" of the scene. In the context of an ID-reading application, the scene includes an object of interest that has one or more IDs of appropriate dimensions and type. The ID(s) are part of the stored image.

A common use for ID readers is to track and sort objects moving along a line (e.g. a conveyor) in manufacturing and logistics operations. The ID reader, or more typically, a plurality (constellation) of readers, can be positioned over the line at an appropriate viewing angle(s) to acquire any expected IDs on the face(s) of respective objects as they each move through the field of view. Generally, the focal distance of the ID reader with respect to the object can vary, depending on the placement of the reader with respect to the line and the size of the object.

Typical ID readers operate to acquire 2D images of the object, which can contain multiple sides, with ID codes printed at differing heights and/or orientations (box sides) on the object. It can be challenging for the image sensor to focus in on the complete working range of the (typically moving) object in which an ID code may be located-particularly where the optical axis of the camera assembly is oriented at a tilted (non-perpendicular) orientation with respect to one or more object sides. To ensure that the working range is appropriately focused, the depth of field (DOF) of the camera optics should be maximized. In a specific logistics example, it is desirable to be able to accurately image sides of objects (boxes) that define a height of approximately <NUM> millimeters and the minimum distance box-to-box <NUM> millimeters on a conveyor travelling at (e.g.) up to approximately <NUM>/s. Relevant prior art can be found in <CIT>.

A system for maximizing depth of field according to the invention is defined in claim <NUM>. A method for maximizing depth of field according to the invention is defined in claim <NUM>.

The invention description below refers to the accompanying drawings, of which:.

<FIG> shows a vision system arrangement <NUM> for use in providing an enhanced depth of field (DOF) useful in imaging detailed features, such as ID codes located on imaged object surfaces over a relatively large distance. The arrangement <NUM> includes a imaging system (also herein termed a "camera assembly") <NUM> with a corresponding optics package <NUM> that directs received light from the surface of the depicted, exemplary object a box) <NUM> onto the surface of a sensor S (described below in <FIG>). As shown, the camera assembly is directed to acquire an image of the side <NUM> of the box <NUM>, where it is desired to acquire and decode various ID codes <NUM>, <NUM> and <NUM> along the box height HB. The variation in height, in combination with the small and precise details present in the ID codes, can make the task of decoding somewhat challenging. That is, a camera with a conventional DOF may only be able to focus adequately upon one or two ID codes, but not all three codes. Changing focus and acquiring more than one image in a single scene can allow ID codes at differing distances to be acquired. However, this approach can also be disadvantageous where objects are moving down a conveyor line (e.g. in a logistics situation), and rapid imaging at an inspection station is desired.

The illustrative camera assembly <NUM>, as described further below, therefore incorporates a tiling optics package <NUM> that can vary its angular orientation during imaging to allow for increased DOF. The camera assembly <NUM> image sensor (S in <FIG>) is typically 2D grayscale or color pixel arrays but can be a 1D array in various embodiments, and is interconnected with a vision system process(or) <NUM>. Image data <NUM> is transmitted to the process(or) <NUM> from the camera assembly <NUM>. The process(or) <NUM> can be contained completely or partially within the housing of the vision system camera arrangement <NUM>. The processor <NUM> carries out various vision system processes using image data transmitted from the image sensor. The process(or) <NUM> can include, but is not limited to, vison tools <NUM>, such as edge detectors, segmenting tools, blob analyzers, caliper tools, pattern recognition tools, and other useful modules. The vision system process(or) <NUM> can also include an ID finder and decoder <NUM>, according to a conventional or custom arrangement, which data from vision system tools and determines whether ID candidates are present in the analyzed image(s). The ID decoder function can then employ conventional functional modules, as well as custom processors/processes, to attempt to decode found ID candidates within the image.

Other processes and/or modules can provide various control functions-for example, auto-focus, illumination, image acquisition triggering, etc. Such functions should be clear to those of skill. Notably, an angle or tilt control process(or) <NUM> is provided. As described below, this function serves to vary the tilt of the glass portion of the optics <NUM> (or another optical component, such as a mirror, prism, etc.) so as to allow for increased DOF according to the exemplary embodiment. Appropriate control data/signals <NUM> can be transmitted from the process(or) module <NUM> to the drive mechanism (described below) for the optics <NUM>. Additionally, a focus control process(or) <NUM> can provide focus information to a variable (e.g. liquid) lens assembly within the optics <NUM> of the camera assembly as described further below as part of the control signal information <NUM>.

Alternatively, some or all of the vision system process(or) <NUM> can be contained within a general purpose computing device <NUM>, such as a PC, server, laptop, tablet or handheld device (e.g. smartphone), which can include a display and/or touchscreen <NUM> and/or other forms of conventional or custom user interface, such as a keyboard <NUM>, mouse <NUM>, etc. It should be clear that a variety of processor arrangements and implementations can be employed to provide vision system functionality to the arrangement <NUM> in alternate embodiments. Similarly, where the camera arrangement is used for tasks other that ID decoding, appropriate vision system process modules can be employed-for example, where the vision system is used for inspection, a training process module and trained pattern data can be provided.

The computing device <NUM> and/or process(or) <NUM> is shown linked to one or more data utilization processes and/or devices <NUM>. Results <NUM> from ID-decoding and/or other vision system tasks, are delivered to such downstream components, and used to perform (e.g.) logistics operations-for example package sorting, routing, rejection, etc..

By way of non-limiting example, note that a photo detector, or other presence sensing device <NUM>, can be located at an appropriate position along the flow of objects (e.g. conveyor line) to issue a trigger signal <NUM>, that is used by the vision system process(or) <NUM> to begin image acquisition of the object <NUM>. The detector can also signal when the object has left the inspection area, and awaits arrival of a new object to begin a new round of image acquisition.

With further reference to <FIG>, an exemplary implementation of the camera assembly <NUM> is shown in further detail. The depicted camera assembly, defines a housing <NUM> having a plurality of circuit components (boards, chips, etc.) <NUM>, <NUM>, <NUM>, including a main board <NUM> carrying an image sensor S as described above. The sensor S defines an image plane <NUM>, onto which a focused image of the object <NUM> is projected by the optics <NUM>, and converted into pixel data for processing by the vision system. The sensor S defines a system optical axis OAS, which extends perpendicularly from the sensor image plane <NUM>. The optics defines and image plane <NUM> that is oriented at a non-perpendicular angle AOP with respect to the system optical axis OAS, and the associated optics optical axis OAO is arranged at an acute (non-parallel) angle AOA with respect to the system axis OAS. This angle AOA can vary, in part, based upon an adjustment mechanism <NUM> that changes the tilt of at least a portion of the optics assembly <NUM>. In a non-limiting example, the angle AOA is between approximately <NUM> and <NUM> degrees. The varying-tilt portion of the optics <NUM>, in this exemplary embodiment, is the so-called glass lens <NUM> at the front-most part of the overall optics assembly <NUM>. As shown, the glass lens <NUM>, which can be removable, consists of a plurality of (e.g.) convex and concave lenses <NUM>, <NUM> and <NUM> arranged according to known optical principles to achieve a given focal distance, etc. Behind the variably tilting (double-curved arrow <NUM>) glass lens <NUM> is a variable-focus lens assembly <NUM>. This variable-focus lens assembly <NUM> can be electronically controlled based upon control signals <NUM> from the process(or) <NUM>. The varying component can be a mechanical or liquid lens, such as those available from Optotune of Switzerland and Varioptic of France. In such arrangements, the liquid lens <NUM> can be controlled using any appropriate focus data (for example, detecting sharpness with vision tools, use of range finders, including LIDAR, time-of-flight sensors, etc.) to provide inputs <NUM> that direct the proper focal distance. Such information is handled via the focus control processor <NUM> (<FIG>) in a manner clear to those of skill. More particularly, lens focus can be controlled by the vision system processor <NUM>, described above, or by a separate focus processor-or a combination of operatively connected process components. The variable lens assembly <NUM> can be oriented a fixed-angle tilt and includes a set of rear concave and convex lenses <NUM>, <NUM> and <NUM>. The overall lens arrangement is appropriate to provide a generally desirable focus distance and DOF for the task. Reference is made to commonly assigned <CIT>, which describes various optics arrangements (e.g. lens stacks) for providing extended DOF in the presence of a variable lens. The optics <NUM> can use the depicted variable focus lens <NUM> to provide an auto-focus functionality that allows operation within a predetermined focal range.

The tilted optics <NUM> relative to the sensor image plane <NUM> generally operates according to the Scheimpflug principle (refer also to aforementioned <CIT> for description) with respect to points on the object surface <NUM> where the optics axis OAO and system, optical axis OAS converge. The resulting vision system arrangement <NUM> affords a DOF when imaging features with smaller details (e.g. barcodes <NUM>, <NUM>, <NUM>) across a range of distances as shown along the side of the <NUM> of the exemplary box <NUM>. That is, object (box <NUM>) is entirely focused on the FOV of the camera assembly <NUM>. An illustration of the use of the Scheimpflug principle is depicted in the diagram <NUM>. In this example, a (e.g.) 3MP image sensor S of a type used herein, with (e.g.) <NUM> millimeters of full diagonal size, works together with a lens L of (e.g.) <NUM> millimeters, and an aperture setting of F8. The three planes described by the sensor S, the lens L and the vertical front side <NUM> of the object/box <NUM> (defining vertical distance V1-V2) meet on the Scheimpflug point Sc. This particular configuration allows the system to image a box of (e.g.) <NUM> millimeters of height (V1-V2) distance having a box gap of <NUM> millimeters (V2-V3) with respect to an adjacent box <NUM>. Note that the Scheimpflug configuration, as described herein, is one of a variety of arrangements that can be achieved by the (variable) geometry of the lend arrangement herein. It is contemplated that other appropriate configurations that enhance DOF can be employed, including generally those that vary the optical path between an on-axis and another (typically non-on-axis) configuration in which the optical plane of the lens assembly is non-parallel with the image plane relative to the image sensor. The use of such alternate non-on-axis configurations should be clear to those of skill in the art of optical arrangements.

The DOF is defined by the intersection between the lines defined by the front focal lens plane FF and the front side of the box defined by (distance V1-V2). The DOF presents a wedge shape with the vertex placed at the point D. The minimum DOF of interest for this case is determined by the points (H1-H2); and in that case, the DOF for the maximum height of the box is <NUM>. Assuming a frame rate of <NUM> of reading-out time for the camera sensor S, and <NUM>/ms box travelling speed (arrow <NUM>) through the FOV (e.g. using a conveyor), the number of frames that the camera sensor Scan acquire with the entire side of the box in acceptable focus is calculated as follows: <MAT> Thus, if the entire range of the box side can be imaged using <NUM> frames, then an accurate and reliable read of all potential candidate features on the box can be acquired and (where applicable) decoded to obtain useful results. Acquisition of the entire surface can be achieved using a can be achieved using a variable tilting (steerable) optics system in combination with a variable focus (e.g. liquid) lens assembly.

Reference is made to <FIG>, which shows a variable-tilt (steerable) lens assembly <NUM>, which can be integrated with the overall optics (<NUM>) described above. The lens assembly <NUM> includes a housing <NUM> that can be attached to a camera assembly lens mount (for example a C-mount) via mating threads or another attachment technique (e.g. set screws, etc.). Alternatively, the housing <NUM> can be attached to the front face of the camera housing via screws or other fasteners using a flange (<NUM>). The housing includes a geared motor <NUM> that receives power and control signals from the angle control process(or) (<NUM> in <FIG>) through any appropriate motor control circuit and/or interface. The motor drives a yoke <NUM> that rotates about the depicted axis of rotation SC to generate a non-perpendicular tilt in the axis OAO of a lens assembly <NUM> with respect to the image plane of the sensor S (non-parallel with the sensor system axis). The motion of the tilt is defined by the radius RL between the rotation axis SC and the contact point P between the motor <NUM> with the yoke <NUM>. The rotation axis SC can define orthogonal exes-for example perpendicular to the page as depicted and vertically (line <NUM>). Hence the yoke <NUM> can be arranged to move about each of two orthogonal axes under the drive of the motor and/or a second motor <NUM>. The yoke <NUM> and confronting surface of the lens housing <NUM> can define hemispheres in a two-axis arrangement that allow for free movement in the manner of a ball and socket within a range of tilt angles relative to each side of the system axis. In this manner, the camera can rotate about a given axis regardless the camera housing's spatial orientation with respect to the object.

The yoke <NUM> and motor(s) <NUM> (and <NUM>) can interact in a variety of ways to achieve an adjustable tilt angle-for example, the yoke <NUM> can contain a gear rack driven by a pinion gear of the motor. The motor can also include a worm drive gear interacting with a yoke-mounted rack. Alternatively, the motor can drive an elastomeric tire that bears against a smooth or textured contact surface on the yoke. In another arrangement, one motor can drive the yoke about a tilt axis, and that entire assembly can be rotated about an orthogonal axis, in the manner of a gimbal, to provide a second degree of freedom. Appropriate feedback can be returned to the angle control process(or) <NUM> to track angular position of the lens barrel <NUM> and its associated axis OAO. Such feedback can be generated by tracking motor steps (i.e. where the motor <NUM> (and <NUM>) is arranged as a stepper motor), or via a discrete encoder that is operatively connected to the motor and yoke drive train. In other embodiments, an accelerometer arrangement can be employed to track relative position. Other techniques for tracking spatial position and orientation of the lens assembly <NUM>. A data/power connection (not shown) between the lens assembly <NUM> and the camera assembly housing can be provided as appropriate. By way of non-limiting example, this connection can be interengaging contact pads that come into contact when the lens housing <NUM> is mounted in the camera housing, or a removable cable <NUM> that extends from the lens housing <NUM> to a socket (not shown) on the camera housing.

In addition to power and control of the motor <NUM> the above-described cable <NUM> can connect the variable (e.g. liquid) lens assembly <NUM> to the focus control (<NUM> in <FIG>) on the vision system processor <NUM>. The liquid lens <NUM> in this example is integrated, and tilts with, the barrel <NUM> of the overall lens assembly <NUM>. The barrel <NUM> collectively houses a front glass lens arrangement (stack) <NUM>, the variable/liquid lens <NUM>, and a rear lens arrangement (stack) <NUM>. In alternate arrangements, the glass front lens arrangement can be tilted exclusively, while the liquid lens and/or rear glass lens arrangement remain stationary with respect to the sensor S. In general, the number of glass lenses and their placement in the barrel <NUM> are highly variable based upon the desired characteristics of the optical system.

In operation, the motor <NUM> operates to tilt the lens assembly <NUM> to different orientations about the axis SC while the focus of the liquid lens <NUM> is adjusted to the proper focal distance to image each portion of the overall FOV to fully encompass the needed DOF for an entire side of the object. Notably, the focal position of the lens <NUM> can be adapted with respect to the sensor S for the different uses cases, i.e., focus for a <NUM>-millimeter lens differs from that of a <NUM>-millimeter lens, and the system allows for flexibility to accommodate these physical differences and provide the user options in choosing lens assemblies that are most suited to a given task. More particularly, as the motor <NUM> tilts the lens, the focus feedback causes the process(or) (<NUM>) to adjust to proper focal plane for that tilt setting.

Note that part of the information provided to the processor can include the current angle of the lens with respect to the camera axis. This is used in combination with other information to set the focus of the variable (liquid) lens so as to place the plane of interest on the object in proper focus. The system also determines the distance between the object and the camera image plane. This data is combined with other system/application constraints, including the speed of motion of the object through the FOV, the maximum size of the object, and the minimum distance object-to-object (described further below).

In an alternate arrangement <NUM> for providing a variable angle for the lens optical axis (observing the Scheimpflug principle) is shown in <FIG>. A lens assembly <NUM> is oriented with respect to the sensor S. The lens assembly <NUM> can include an appropriate lens stack, including a variable (e.g. liquid) lens, as described above. The variable lens is controlled by the above-described focus control process(or) <NUM>. A steerable mirror <NUM>, sized to reflect the FOV onto the sensor S is positioned in front of the lens assembly <NUM> along the optical path <NUM>. The overall arrangement defines the above-described Scheimpflug principle. The steerable mirror <NUM> can be pivoted in one, or two orthogonal, axes (double-curved arrows T1 and T2). The optical path <NUM> extends onto an overlying, folding mirror <NUM>, which can be mounted at a convenient location with respect to the imaged scene (e.g. the conveyor surface <NUM>).

In operation, the folding mirror <NUM> is oriented into an appropriate configuration to image the scene, and the steerable mirror <NUM> tilts (in response to the angle control process(or) <NUM>) along one or both axes to allow the system to scan the different zones of the conveyor, whilst sweeping rapidly across the perpendicular direction of travel. The steerable mirror <NUM> can be actuated using a variety of techniques (e.g. servos, steppers, voice coils, etc.), which should be clear to those of skill. The steerable mirror <NUM> can be integrated with the optics of the camera assembly so that it is fully contained and appropriately protected (e.g. using a transparent viewing window) from the environment.

As shown in <FIG> and <FIG>, the optics assembly with steerable mirror <NUM> (combination of lens <NUM> and mirror <NUM> in <FIG>) is shown in operation to read ID codes (or other features) along the side <NUM> of on exemplary box <NUM> located adjacent to a second box <NUM> in the gap space G between boxes. The imaged scene is defined by the positioning of the folding mirror <NUM>, which is located and angled so as to direct the FOV onto the height of the box side <NUM>. The sweeping motion of the steerable mirror can place various regions of this height (and width) into focus as the box moves (arrow <NUM>) at a predetermined speed down a conveyor line <NUM>. Sufficient frames are acquired of the box during the scan motion of the optical path <NUM> to ensure all features along the side <NUM> are sufficiently imaged for decoding. In an example (described above), the boxes have a maximum dimension of approximately <NUM> X <NUM> X <NUM> millimeters, but this maximum size (in some or all dimensions) is highly variable in alternate implementations. More particularly, this arrangement is well suited to applications in logistics. In such applications, the maximum height of the boxes is approximately <NUM> millimeters, and the minimum gap distance, box-to-box is <NUM> millimeters. The boxes can travel up to <NUM>/s through the system's FOV. The system desirably allows for the maximum expected size of the box to be placed in focus, whereby the number of cameras are minimized and the minimizing both, the number of cameras, and complexity, of the system can be minimized.

As shown further in <FIG>, the steerable mirror <NUM> is angled variously (e.g. a left, center and right position) so that the optical path <NUM> is redirected laterally so that the associated FOV can cover the entire longitudinal (width) dimension of an exemplary object (box <NUM>) side <NUM>. This is accomplished in three discrete positions (and image frames) in the depicted example, but more or fewer lateral positions (for example a left-centered and right-centered position) can be employed in alternate implementations. As shown, the optical path <NUM> is directed approximately <NUM> degrees laterally leftward by the steerable mirror <NUM> in <FIG> and <FIG>, thereby covering the left portion of the box side <NUM> along its approximate height, as an image frame is acquired. In <FIG> and <FIG>, the central portion of the box side <NUM> is imaged, with the optical path <NUM> at approximately <NUM> degrees (neutral) in position. Then, in <FIG> and <FIG>, the optical path <NUM> is directed laterally rightward so as to image the right portion of the box side <NUM>. With appropriate overlap the three images generated by this scan process ensure that the entire side is fully imaged with sufficient detail to find and decode minimally size ID code ID features. Note that the use of a +/- <NUM>-degree tilt angle is exemplary, and a wide range of possible angles can be employed based upon the prevailing FOV and DOF of the system, size of features to be imaged and size of the overall object under inspection.

Advantageously, the exemplary implementation maximizes the image quality and focus, while extending the DOF. The glass lens component can advantageously operate in both a regular mode, in which its optical axis perpendicular to the sensor plane and in a Scheimpflug configuration, allowing for substantial flexibility that can benefit a variety of vision system tasks, including (e.g.) logistics applications that require top/side scanning of an object. More particularly, the arrangement allows for a maximized DOF in the case of moving objects viewed from an extreme perspective. These performance characteristics also allow the number of cameras used to image objects arrangement to be minimized. That is, other systems based on multiple cameras generally require a relatively greater number of readers to cover the same DOF as the illustrative arrangement herein, while such other systems struggle with perspective distortion, a reduced usable FOV and a requirement of differing focal distance for each camera in the system. Typically such systems must incorporate three or four cameras (with a significant increase in system complexity and data volume to be processed) to perform the same operation a single (or possibly two) camera assembly can perform herein. The illustrative arrangement also inherently integrates low drift characteristics and allows for conventional autofocus functionality. Notably, the illustrative arrangement maximizes DOF with no (free of) changes to sensor or camera housing hardware aside from the use of a modified opto-mechanical in the lens configuration and forward of the lens (e.g. a folding mirror).

Reference is now made to <FIG>, which shows results of an exemplary image of a plurality of objects (a stack of boxes) <NUM> acquired according to an illustrative embodiment herein. In this example, a camera assembly <NUM> having (e.g.) a <NUM>-millimeter F6 lens has been implemented on a tilted position working together with a <NUM>-Mp image sensor. The position of the lens is arranged to focus upon the plane of interest <NUM> of the object(s). In this configuration, the FOV is increased so that the plane of interest <NUM> defines an observable height of approximately <NUM> millimeters. In general, distortion can be observed inside the plane of interest <NUM>. An upper image segment <NUM> and lower image segment contain identifiable ID codes <NUM>, <NUM> that are relatively small (e.g., <NUM> mil-which is typically a minimum size in various logistics applications).

Similarly, <FIG> shows an image <NUM> of the FOV with a sharply focused region (arrow <NUM>) that spans substantially the entire height thereof by employing the principles of the system and method herein, which provide enhanced DOF. Conversely, a much smaller focused region (arrow <NUM>) occurs using conventional optics in the image <NUM> of the same FOV in <FIG>.

It should be clear that the system and method described above effectively expands DOF and the ability to precisely image a taller object by advantageously combining a vision system, a glass lens designed for both regular (on-axis) and Scheimpflug configurations, a variable (e.g. liquid) lens and a mechanical system to adapt the lens to the different configurations without (free of) screwing-out (or otherwise detaching) the optics and associated mechanics. This system and method effectively addresses certain challenges encountered in , for example, logistics applications, and provides ID code readers with improved features that increase performance, reduce the complexity of the installation, and provide more flexibility for a single lens, that can now be employed in different applications. Notably, the above-described system and method does not require (in various embodiments) the acquisition of multiple images, sweeping different optical powers over all, or part, of available dynamic range and/or exhaustive calibration prior to runtime operation.

Claim 1:
A system (<NUM>) for maximizing depth of field, DOF, with respect to an object (<NUM>) imaged by a vision system camera (<NUM>) and providing low drift, the system comprising:
a vision system processor (<NUM>) and said vision system camera (<NUM>), wherein the vision system camera comprises:
an image sensor that transmits image data to the vision system processor, the image sensor defining a system optical axis (OAS), which extends perpendicularly from the sensor image plane (<NUM>);
a lens assembly (<NUM>, <NUM>, <NUM>) arranged to provide a high DOF, the lens assembly defining a lens optical axis (OAO);
a variable lens (<NUM>, <NUM>), arranged between the lens assembly and the image sensor, that changes focus in response to the processor; and
a tilting mechanism (<NUM>, <NUM>) that is configured to automatically vary the lens optical axis relative to the system optical axis while the focus of the variable lens is adjusted to the proper focal distance to image each portion of the overall field of view, FOV, so as to maximize DOF.