Patent Description:
In many contexts, it may be useful to use imaging systems to evaluate symbols. For example, in direct part marking ("DPM") and other contexts, acquired images of barcodes can be analyzed in order to verify that the barcodes have been appropriately printed or marked, or for other reasons. In some cases, the symbols to be evaluated may be moving. For example, it may be useful to verify DPM barcodes as objects on which the barcodes are marked are moving along a production line or other conveyance.

<CIT> relates to a system for local tone mapping for symbol reading. A local pixel neighborhood metric is determined for a raw pixel in a region-of-interest. A local mapping function maps the value of the raw pixel to a smaller bit depth based on a value of at least one other raw pixel in a local pixel neighborhood.

In "<NPL>et al. , a two-stage quality measure for mobile-phone captured 2D barcodes is disclosed. Using global bimodal distribution features and a local finder pattern detection, a trained algorithm can assign a probability score for readability to a barcode image. It is an object to provide a system and a computer-implemented method for evaluating a symbol on an object, the system and method being optimized, respectively, with respect known systems and methods.

The invention is defined by means of its independent claims.

To the accomplishment of the foregoing and related ends, embodiments of the invention can include one or more of the features hereinafter fully described. The foregoing and following description and the annexed drawings set forth in detail certain example aspects of the invention. However, these aspects are indicative of but a few of the various ways in which the principles of the invention can be employed. Other aspects, advantages and novel features of the invention will become apparent from the detailed description herein as considered along with the drawings.

The features and advantages of the invention are apparent from the following description taken in conjunction with the accompanying drawings in which:.

In the drawings, similar symbols typically identify similar components, unless context dictates otherwise. The example embodiments described in the detailed description, drawings, and claims are not meant to be limiting.

The term "article of manufacture" as used herein is intended to encompass a computer program accessible from any computer-readable device, carrier (e.g., non-transitory signals), or media (e.g., non-transitory media). For example, computer-readable media can include but are not limited to magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips, and so on), optical disks (e.g., compact disk (CD), digital versatile disk (DVD), and so on), smart cards, and flash memory devices (e.g., card, stick). Additionally it should be appreciated that a carrier wave can be employed to carry computer-readable electronic data such as those used in transmitting and receiving electronic mail or in accessing a network such as the Internet or a local area network (LAN).

Also as used herein, unless otherwise specified or limited, the terms "component," "system," "module," and the like are intended to refer to a computer-related system that includes hardware, software, a combination of hardware and software, or software in execution. For example, a component may be, but is not limited to being, a processor device, a process running on a processor device, an object, an executable, a thread of execution, a program, and/or a computer. By way of illustration, both an application running on a computer and the computer can be a component. One or more components (or system, module, and so on) may reside within a process and/or thread of execution, may be localized on one computer, distributed between two or more computers or other processor devices, and/or included within another component (or system, module, and so on).

Also as used herein, unless otherwise specified or limited, the term "symbol" indicates an information carrying indicia, including indicia that can be imaged by an imaging device. In some implementations, a symbol may be formed as by printing, deposition, laser etching, and so on, including in the form of a DPM symbol, such as a DPM 2D (or other) barcode. In some implementations, symbols can be formed in other ways, such as by printing with ink.

Some embodiments of the invention can be implemented as systems and/or methods, including computer-implemented methods. Some embodiments of the invention can include (or utilize) a device such as an automation device, a special purpose or general purpose computer including various computer hardware, software, firmware, and so on, consistent with the discussion below.

In some embodiments, aspects of the invention, including computerized implementations of methods according to the invention, can be implemented as a system, method, apparatus, or article of manufacture using standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof to control a computer or other processor device to implement aspects detailed herein. Accordingly, for example, embodiments of the invention can be implemented as a set of instructions, tangibly embodied on a non-transitory computer-readable media, such that a processor device can implement the instructions based upon reading the instructions from the computer-readable media.

Certain operations of methods according to the invention, or of systems executing those methods, are represented schematically in the FIGS. Unless otherwise specified or limited, representation in the FIGS. of particular operations in particular spatial order is not intended to require those operations to be executed in a particular order. Certain operations represented in the FIGS. , or otherwise disclosed herein, can be executed in different orders, as appropriate for particular embodiments of the invention. Further, in some embodiments, certain operations can be executed in parallel, including by dedicated parallel processing devices, or separate computing devices configured to interoperate as part of a large system.

As also discussed above, it may be useful to evaluate symbols, including 2D barcodes and others, using imaging systems. For example, in DPM contexts, it may be useful to use acquired images of a DPM barcode (e.g., a 2D barcode) to verify the quality of the barcode. In this way, for example, users can ensure that DPM (or other) symbols exhibit appropriately high print quality and overall fidelity.

Likewise, in some cases, it may be useful to evaluate symbols on objects that are moving or are stationary only for a short time. For example, it may be useful to conduct verification of DPM symbols as marked objects are moving along a production line. This may significantly complicate evaluation operations.

It may also be useful to evaluate symbols in contexts in which lighting conditions may be variable or otherwise non-ideal, or for objects that may produce relatively specular reflection. This may also significantly complicate evaluation operations.

Some conventional approaches aim to improve symbol evaluation by executing an iterative process that includes the acquisition and evaluation of multiple images, including through the use of machine vision systems. For example, a machine vision system can be configured to evaluate symbols based on aspects of Technical Report <NUM>, issued by the International Organization for Standardization ("ISO") and the International Electrotechnical Commission ("IEC") - also referred to as the "AIM DPM" guidelines.

The AIM DPM guideline describes the evaluation of the quality of DPM symbols, based on acquisition of multiple images and analysis thereof. In particular, AIM DPM requires a particular system response (e.g., gain, f-stop, or shutter-time settings) for evaluation of barcodes in images and provides for adjustment of the system response of an imaging device if evaluation of an initial image indicates incorrect exposure. For example, under AIM DPM, if an initial image is excessively saturated or exhibits insufficient resolution, appropriate adjustments should be made to the system response and a subsequent image captured. The subsequent image can then be evaluated to determine whether the adjusted system response has achieved the correct system response (i.e., such that the ratio of mean bright-pixel brightness to saturation matches a desired value). In this way, through a process of multiple successive image acquisitions, an operational system response can be obtained that appropriately balances saturation and resolution and, in particular, that maintains a certain relationship between overall image brightness and the brightness level at which saturation occurs. The operational system response can then be used for run-time image capture and evaluation of symbols. In this way, for example, evaluation of symbols can be uniformly applied for a given application or context.

Evaluation of symbols under the principles of AIM DPM can be useful, but acquisition of multiple images for evaluation may be impractical or undesirable in some contexts. For example, on active production lines it may not be feasible to acquire and process multiple images of a particular object with a DPM symbol without stopping operation of the line. Accordingly, it may be useful to provide a system that can support accurate evaluation of DPM (or other) symbols without necessarily requiring capture of multiple images.

Embodiments of the invention can address the issues noted above, or otherwise provide improved systems or methods for evaluation of symbols. For example, in some embodiments, in order to evaluate a symbol, a first image of the symbol can be acquired using an imaging device. A second image can then be derived from the acquired first image, without necessarily requiring any further acquisition of images of the symbol (at least for the current evaluation cycle).

In particular, in some embodiments, it may be useful to derive the second image from the first image based upon determining a saturation threshold for the second image that will preserve a target relationship between saturation threshold and image brightness. For example, a target ratio between saturation threshold and image brightness for the second image can be predetermined (e.g., based upon AIM DPM or other relevant factors), and the saturation threshold of the second image can then be determined so that mapping the pixels of the first image to the second image, including setting pixels beyond the second-image saturation threshold to a maximum, results in an image brightness for the second image that will preserve the target ratio.

In some embodiments, a saturation threshold for a derived second image can be determined iteratively. For example, an initial saturation threshold can be selected for the second image based on any of a variety of considerations. Pixels of the first image can then be mapped accordingly to an "initial" second image and an image brightness of the resulting initial second image can be determined. If this initial image brightness exhibits an appropriate relationship with the initial saturation threshold, evaluation of the symbol in the initial second image can then proceed. In contrast, if an appropriate relationship between the initial image brightness and the initial saturation threshold is not achieved, a subsequent saturation threshold can be selected for the second image, and pixels of the first image can be re-mapped accordingly to a "subsequent" second image. An image brightness of the subsequent second image can then be determined, as well as a relationship between this subsequent image brightness and the subsequent saturation threshold.

As with the initial derived image, if the subsequent image brightness exhibits an appropriate relationship with the subsequent saturation threshold, evaluation of the symbol in the subsequent second image can then proceed. If an appropriate relationship is still not obtained, however, further selection of saturation thresholds, mapping of first-image pixels, and evaluation of image brightness can continue. In this regard, for example, a subsequent second image can effectively become a new initial second image, which can be succeeded by a new subsequent second image with a new subsequent saturation threshold.

In some embodiments, a saturation threshold for a derived second image can be determined without iteration, such as the example iteration described above. For example, an image brightness of a first acquired (or other) image can be determined and then a saturation threshold selected that exhibits an appropriate relationship with the determined image brightness. The selected saturation threshold can then be set as a saturation threshold for the second image and the pixels of the first image can be remapped accordingly to the second image. Notably, in some implementations, such iteration can be implemented using only a single acquired image, i.e., without requiring capture of subsequent images, such as may not be possible for moving parts.

In some embodiments, the availability of higher bit depth imaging systems in particular can make it possible to achieve a target relationship between image brightness and saturation level (e.g., such as specified in AIM DPM), based on acquisition of only a single image. For example, evaluating a symbol on an object can include acquiring a first image of the object, including the symbol, that exhibits a first bit depth (e.g., <NUM> or <NUM> bits). The first image can be analyzed to determine a mapping of the pixels of the first image to a second bit depth that is smaller than the first bit depth (e.g., <NUM> bits). A second image with the second bit depth can then be generated, based on the first image and the determined mapping, and can be evaluated to determine at least one quality attribute of the symbol. Accordingly, in these and other embodiments, it may be possible to evaluate symbols without acquisition of multiple images or without an iterative prior adjustment of system response, such as specified by AIM DPM.

In some embodiment, a first image, after being acquired with the first, larger bit depth, can be analyzed in a variety of ways to determine a mapping of pixels to the second bit depth. In some implementations, for example, one or more of the following operations can be executed. A dynamic range can be determined for the first image, and sets of light and dark pixels can be identified, such as specified, for example, in AIM DPM. An average intensity value of the light pixels can be determined, and then a saturation threshold can be determined based on the average intensity value. Pixels with intensity values that exceed the saturation threshold can be set to a particular value (e.g., a maximum brightness value), and the mapping of pixels of the first to the second bit depth can be determined based upon the determined saturation threshold or the average light-pixel intensity value. Or, in some embodiments, a saturation threshold can be determined such that an average intensity value of light pixels, after mapping of the pixels in view of the determined saturation threshold, exhibits an appropriate relationship with the determined saturation threshold.

In some implementations, in this regard, analysis of the first image can proceed similarly to the iterative analysis of AIM DPM. But, in some embodiments, in contrast to AIM DPM, analysis according to the present invention may require only the acquisition of a single image. i.e., may not necessarily require the acquisition of a second image of a symbol to be evaluated.

<FIG> illustrates an example system <NUM>, for evaluating a symbol <NUM> on an object <NUM>, according to one embodiment of the invention. In the illustrated embodiment, the symbol <NUM> is a flat DPM 2D barcode and the object <NUM> is a parallelepiped box. In other embodiments, other configurations are possible. For example, any variety of geometries are possible for an object to be imaged, and any variety of symbols can be imaged and evaluated, including dot-peen or other DPM symbols, and non-DPM symbols.

In the illustrated context of <FIG>, the object <NUM> is disposed on a conveyer <NUM> that is configured to move the object <NUM> past the system <NUM> at a predictable and continuous rate. In other embodiments, objects may be moved into or past a system according to the invention in a variety of ways. In some embodiments, objects may be stationary relative an evaluation system. In some embodiments, evaluation systems may be configured to be moved relative to the objects, and symbols, that are to be evaluated.

Generally, systems according to the invention can include an imaging device and a processor device. In the embodiment illustrated in <FIG>, the system <NUM> includes an imaging system <NUM> that includes a processor device <NUM> and an imaging device <NUM> with a field-of-view ("FOV") <NUM> that includes part of the conveyer <NUM>. Accordingly, the imaging system <NUM> can be configured to capture one or more images of the object <NUM> as the object <NUM> is moved by the conveyer <NUM>.

The imaging device <NUM> and the processor device <NUM> can be configured in a variety of ways, including, respectively, as an arrangement of one or more electronic imaging sensors and one or more lens assemblies, and a programmable general purpose computer device or machine-vision computer. In some embodiments, an imaging device and a processor device can be part of physically separated systems. For example, an imaging device can be configured to communicate with a remotely disposed processor device (e.g., a cloud-based computing system) in order to execute certain operations in an implementation of the invention.

In some embodiments, an imaging device can be configured to acquire an image with a first bit depth, and an associated processor device can be configured to analyze an image with a second bit depth that is smaller than the first bit depth, including to evaluate a symbol within the image. In different embodiments, different relative scales of the first and second bit depths are possible. In the embodiment illustrated in <FIG>, for example, the imaging device <NUM> is configured to acquire <NUM>-bit images of the FOV <NUM> and the processor device <NUM> is configured to analyze <NUM>-bit images. In some implementations, a processor device may be capable of analyzing images with the same (or greater) bit depth than may be acquired by an associated imaging device, even if particular implementations include analyzing images with smaller bit depths than those acquired by the imaging device. In some embodiments, different acquisition and processing bit-depths are possible, including in arrangements in which a processor device is configured to process the full bit depth of an associated imaging device (e.g., such that a derived second image may exhibit the same bit depth as an acquired first image).

Usefully, in some embodiments, the bit depth for image acquisition and the bit depth for image analysis can be selected so that images can be readily processed over a wide range of imaging parameters (e.g., dynamic range, average brightness, and so on). For example, as also discussed below, the bit depth for image acquisition can be selected, along with an appropriate system response, so that the brightest expected images can be captured without excessive over-saturation and so that the darkest expected images can be captured with sufficient resolution to allow appropriate analysis thereof. Similarly, the bit depth for image analysis can be selected so that images with a range of brightness profiles can be usefully analyzed, including after downward scaling of the images to the relevant smaller bit depth.

In this regard, for example, the imaging device <NUM> can be configured to acquire a <NUM>-bit image of the object <NUM>, including the symbol <NUM>. For example, as the object <NUM> passes into the FOV <NUM>, the imaging device <NUM> can acquire the image with a predetermined system response (e.g., predetermined f-stop and shutter speed settings). The processor device <NUM> can then generate a second (non-acquired) image from the first (acquired) image, such as a second image with <NUM>-bit or otherwise reduced bit depth. For example, the processor device <NUM> can be configured to map relevant pixels from a first acquired image to a derived second image, including as further detailed below, in order to provide a target relationship (e.g., a ratio as specified in AIM DPM) between the saturation level and image brightness of the second image.

Once generated, the second (e.g., lower bit-depth) image can then be analyzed in order to evaluate the symbol. For example, the generated second image can be analyzed in order to evaluate one or more quality attributes of a symbol therein, including those specified in AIM DPM or other relevant guides.

In general, a generated image under embodiments of the invention can include the same subject matter and similar information as the acquired image. But the generated image may exhibit somewhat lower resolution relative to the acquired image, such as may result from a reduced bit depth. With appropriate training, however, sufficient resolution can generally be maintained to allow appropriate analysis of the acquired image (via the generated image), including for evaluation of a symbol therein, subject to limits inherent to the captured bit depth, and the minimum and maximum system response needed for the relevant application.

As also noted above, in some embodiments, a processor device can be configured to determine a mapping from a first, acquired (e.g., larger bit depth) image to a second, derived (e.g., smaller bit depth) image based upon analysis of the acquired image. For example, the processor device <NUM> can be configured to determine a mapping from an acquired image to a generated image based upon determining an appropriate saturation threshold for the derived image, setting pixels of the acquired image that exceed the saturation threshold to a maximum value, and then mapping pixels of the acquired image that are below the saturation threshold to corresponding brightness levels in the derived image. In this way, for example, the processor device <NUM> can help to ensure that relevant information from the acquired image is mapped to the generated image, while potentially superfluous pixels are given less emphasis.

As also noted above, in some embodiments, an appropriate saturation threshold for a derived image can be determined based upon analysis of image brightness that accounts for the mapping, to a maximum value for the derived image, of pixels in an acquired image that exceed the determined saturation threshold. For example, as illustrated in <FIG>, the imaging device <NUM> (see <FIG>) has acquired a first image as represented in histogram <NUM>. Using the processor device <NUM> (see <FIG>), an initial saturation threshold <NUM> can be determined for a derived second image. The pixels of the first image can then be mapped to an initial derived image, as represented in histogram <NUM>, with pixels <NUM> of the first image that exceed the saturation threshold <NUM> (as also shown in the histogram <NUM>) having been mapped to a maximum (as shown) for the initial derived image. The image brightness of the initial derived image, such as a mean brightness <NUM> of light pixels <NUM>, including the mapped pixels <NUM>, can then be determined and compared with the saturation threshold <NUM> to evaluate a relationship between these factors.

If an appropriate relationship between the mean brightness <NUM> and the saturation threshold <NUM> has been achieved, such as a target ratio (e.g., approximately <NUM>/<NUM>) as specified by AIM DPM, the initial derived image (as represented in the histogram <NUM>) can then be used to evaluate the symbol <NUM>. If not, then a new saturation threshold can be determined, such as a subsequent saturation threshold <NUM> (see, e.g., the histogram <NUM>), in an attempt to better establish the desired relationship.

In this regard, as also discussed below, an iterative process can be employed. For example, because remapping of pixels based on a selected saturation threshold can affect the resulting mean brightness of light pixels (or other brightness measurements), a succession of operations may sometimes be iteratively executed, including: to determine successive saturation thresholds; to remap pixels and calculate the resulting respective brightness measurements; and then to evaluate whether the current saturation threshold exhibits an appropriate relationship with the associated brightness measurement of the remapped pixels, or whether another saturation threshold (and corresponding pixel mapping) should be selected.

Once the saturation threshold <NUM> has been determined, the pixels of the first image, including pixels <NUM> that exceed the saturation threshold <NUM>, can then be re-mapped to a subsequent derived image, as represented in histogram <NUM>, and an evaluation can be made of image brightness (e.g., determined as a mean light-pixel brightness <NUM>) and of the relationship of the image brightness to the saturation threshold <NUM>. Again, if an appropriate relationship between the brightness <NUM> and the saturation threshold <NUM> is achieved, the corresponding (subsequent) derived image can then be used to evaluate the symbol <NUM>. If not, a similar iterative process can continue, with a newly selected saturation threshold (not shown), corresponding mapping of pixels, and so on.

Initial and subsequent saturation thresholds for derived images can be determined in a variety of ways. In some implementations, for example, a system may step through a predetermined set of saturation thresholds between a lower and an upper range (e.g., within a relevant dynamic range) such as by, for each saturation threshold in the predetermined set, sequentially mapping pixels, evaluating image brightness, and evaluating a relationship of the image brightness to the saturation threshold. In some implementations, such sequential evaluation may be terminated if, for example, the target relationship is established for any particular saturation threshold, or based upon other relevant criteria such as a subsequent saturation threshold resulting in worse correspondence with a target relationship than a preceding saturation threshold.

In some implementations, in contrast, a system may execute a search, such as a binary search, as may be appropriate to streamline determination of an appropriate saturation threshold and pixel mapping. For example, an initial saturation threshold may be determined, such as by identifying a saturation threshold at a mid-point of a relevant brightness range, and corresponding analysis can then be completed (e.g., as described above). As appropriate, a new saturation threshold can then be identified based upon on whether the initial analysis indicates that a higher saturation threshold or a lower saturation threshold may be needed in order to obtain the target relationship. For example, if a higher (or lower) saturation threshold is needed, a new saturation threshold can be identified that is halfway between the previous saturation threshold and the upper (or lower) end of the relevant range. In other implementations, other types of searches are also possible.

In some embodiments, an appropriate saturation threshold can be determined based upon analysis of the intensity of pixels in an acquired image, without necessarily accounting for the effects on derived-image image brightness of the mapping of those pixels to a derived image. <FIG> illustrates an example of this type of analysis, as executed by the system <NUM> (see <FIG>). In the illustrated implementation, for example, the imaging device <NUM> has acquired a <NUM>-bit image. Correspondingly, the processor device <NUM> has determined a dynamic range <NUM> of the image, and has generated a histogram of the image, as represented graphically in <FIG>. (With this and other examples, those of skill in the art will recognize that the processor device <NUM> may not necessarily generate a graphical representation, but may perform certain tasks entirely numerically, such as by storing and performing calculations on lists of pixel counts.

For the particular <NUM>-bit image represented in <FIG>, the dynamic range <NUM> is notably smaller than a maximum intensity <NUM> that may be possible. Accordingly, certain information in the <NUM>-bit image may not be particularly useful or relevant to a desired image analysis (e.g., evaluation of a symbol). Embodiments of the invention can generally eliminate some of that information from a reduced bit-depth (or other derived) image for analysis, including as discussed above and below.

In some implementations, having identified the dynamic range <NUM> and generated the histogram counts (as shown in <FIG>), the processor device <NUM> can interrogate the resulting bimodal distribution to identify a set of light pixels <NUM> and a set of dark pixels <NUM>. An average light-pixel value <NUM> (e.g., a mean intensity) can be determined from analysis of the light pixels <NUM> and then a saturation threshold <NUM> can be determined based upon the average light-pixel value <NUM>.

In some implementations, the saturation threshold <NUM> can be determined as a predetermined multiple (e.g., approximately <NUM>/<NUM>) of the average light-pixel value <NUM>. In this way, for example, the average light-pixel value <NUM> can be approximately <NUM> percent of the saturation threshold <NUM>. In other implementations, other approaches are possible.

With the saturation threshold <NUM> having been determined, pixels <NUM> that exceed the threshold <NUM> can be set to a predetermined value. For example, in some embodiments, the pixels <NUM> can be set to the maximum intensity value within the saturation threshold <NUM>, which may result in an increase in pixel count at the greatest intensity range under the threshold <NUM>, as represented by pixels <NUM> in <FIG>. Pixels <NUM> of the acquired image that are below the saturation threshold can then be mapped to the lower bit depth of the determined image, which can be analyzed to evaluate the symbol <NUM> (see <FIG>).

As also noted above, the selection of the particular value of the saturation threshold <NUM> and the corresponding remapping of the pixels <NUM> can result in changes to average brightness (or other measurements of image brightness) relative to the initially acquired image. In some cases, accordingly, selection of an initial saturation threshold, such as the threshold <NUM>, may result in average image brightness that does not appropriately exhibit a desired relationship (e.g., ratio) with the initial saturation threshold. In that case, for example, as also discussed above, an iterative selection of one or more subsequent saturation thresholds can then proceed, along with corresponding remapping of pixels and calculation of updated brightness measurements, until an appropriate relationship has been achieved.

In some implementations, measures of image brightness other than an average (e.g., mean) intensity of light pixels can be used. For example, in some implementations, the processor device <NUM> can be configured to determine a variance or other statistical measure of the light pixels and can then determine the saturation threshold <NUM> (or other mapping parameter) accordingly. Likewise, in some implementations for which higher dynamic range is preferred, such as for reading applications (e.g., in contrast to evaluation applications), other measures of image brightness may be beneficially used.

In some implementations, operations discussed herein can be executed in different orders than those in which they are expressly presented. For example, in some implementations, it may be possible to set the pixels <NUM> to a predetermined value(s) after mapping the pixels <NUM> to a lower bit-depth (or other derived) image.

<FIG> illustrates another example of an analysis process that can be implemented by the system <NUM> of <FIG>. In the illustrated implementation, the imaging device <NUM> has acquired another <NUM>-bit image. Correspondingly, the processor device <NUM> has determined a corresponding dynamic range <NUM> of the image and has generated a histogram of the image, as represented graphically in <FIG>. (Again, those of skill in the art will recognize that the processor device <NUM> may not necessarily generate a graphical representation, but may perform certain tasks entirely numerically such as by storing and performing calculations on lists of pixel counts.

For the particular <NUM>-bit image represented in <FIG>, the dynamic range <NUM> is almost equal to a maximum intensity <NUM> that may be possible for the image. Accordingly, relatively little information in the <NUM>-bit image may be irrelevant or inutile for desired image analyses (e.g., symbol evaluation). In this regard, for example, it can be seen that the acquisition of a larger bit depth image, which can accommodate the intensities in the image represented in <FIG>, can allow for high-resolution acquisition of images with a wide variety of dynamic ranges. Further, as also discussed above, reduction of the bit depth for analysis can still allow for reliable and high quality analysis of those same images, including in keeping with aspects of AIM DPM. Also, in contrast to conventional approaches, analysis under some embodiments of the invention can proceed based on acquisition of only a single image.

In some implementations, for example, having identified the dynamic range <NUM> and generated the histogram counts (as shown in <FIG>), the processor device <NUM> can interrogate the resulting bimodal distribution to identify a set of light pixels <NUM> and a set of dark pixels <NUM>. A statistical measure of the light pixels, such as a mean light-pixel intensity <NUM> can be determined from analysis of the light pixels <NUM> and then a saturation threshold <NUM> can be determined based upon the mean <NUM>.

In some implementations, the saturation threshold <NUM> can be determined as a predetermined multiple (e.g., approximately <NUM>/<NUM>) of the mean light-pixel value <NUM>. In this way, for example, the mean light-pixel value <NUM> can be approximately <NUM> percent of the saturation threshold <NUM>. In other implementations, other approaches are possible. In other implementations, other multiples (or ratios) may be appropriate. In some implementations, such as discussed with regard to <FIG>, a saturation threshold for a derived image can be similarly determined, but based also upon calculations of image brightness after the pixels of an acquired image have been mapped to a derived image, taking into account any brightness cut-off imposed by the determined saturation threshold.

With the saturation threshold <NUM> having been determined, pixels that exceed the threshold <NUM> can be set to a predetermined value that may be encompassed by the threshold <NUM>. In the illustrated implementation, however, the light pixels are sufficiently clustered so that few or no pixels of the acquired image exceed the saturation threshold <NUM>. Accordingly, for example, it may not be necessary to set the intensity values of any pixels to a new value, other than may be necessary to map pixels <NUM> of the acquired image to a lower bit-depth (e.g., in a scaling operation). The mapped, lower bit-depth image can then be analyzed to evaluate the symbol <NUM> (see <FIG>).

In some implementations, it may be useful to train a system for evaluating a symbol before acquiring images for evaluation or at other times, including dynamically during run-time operations. This may be useful, for example, to prevent over-saturation of acquired images of the brightest targets.

In different implementations, different training approaches may be possible, including those that appropriately optimize a balance between resolution and dynamic range. In this regard, <FIG> illustrates aspects of example training operations that can be executed by the system <NUM> (see <FIG>). In other implementations, other operations are possible.

In the implementation illustrated in <FIG>, the imaging device <NUM> (see <FIG>) can be configured to acquire at least two training images: a bright-image training image and a dark-image training image. In particular, the bright-image training image, as represented in histogram <NUM>, can be configured to appropriately acquire an image that represents a maximum expected image brightness for the relevant context. For example, a training object or scene can be arranged so that the imaging device <NUM> will be exposed to the brightest specular reflections or overall lighting as may be reasonably expected during run-time operation of the system <NUM>. In contrast, the dark-image training image, as represented in histogram <NUM> can be configured to represent a minimum expected image brightness for the relevant context. For example, a training object or scene can be arranged so that the imaging device <NUM> will be exposed to the dimmest specular reflections or overall lighting as may be reasonably expected during run-time operation of the system <NUM>.

Once acquired, the bright-image or dark-image training images can be analyzed in order to determine an appropriate operational system response <NUM> for the imaging device <NUM> (i.e., the system response for the imaging device <NUM> during run-time operations). This may be useful, for example, to ensure optimally (e.g., substantially) full use of the possible dynamic range of the imaging device <NUM> for the bright-image training image, and appropriate resolution of relevant features in the dark-image training image.

If an appropriate balance between these (or other) factors is obtained, the system response that was used to acquire the training images can then be set as the operational system response. In contrast, if an appropriate balance is not obtained, the operational system response can be determined after adjustment of the system response for training, including by computational adjustment (e.g., based on look-up tables or previous training or run-time efforts), or by acquisition and analysis of one or more additional dark-image or light-image training images using updated system responses.

In some implementations, a bright-image training image can be acquired with a different system response than a dark-image training image. An operational system response determined based on comparative analysis of the training images (or system responses) or based on iterative acquisition of training images with converging system responses.

In some implementations, iterative training operations may be possible. For example, as discussed above, some implementations can include iterative acquisition of training images with different system responses, in order to identify an appropriately optimized operational system response.

In some implementations, an operational system response can be determined so as to provide an appropriate relationship between a statistical measure of the light pixels of a bright-image training image and a maximum intensity provided by a bit depth for operational image acquisition. For example, the operational system response <NUM> can sometimes be determined as a system response that ensures that a mean intensity value of light pixels in the bright-image training image (e.g., as represented in the histogram <NUM>) is equal to a predetermined fraction (e.g. <NUM> percent) of a maximum brightness possible for the first training image (and the imaging system generally). In this way, for example, the operational system response <NUM> can be selected to ensure that the maximum expected brightness during run-time operations can be recorded with substantially full use of the first bit depth and little to no over-saturation. In this regard, operations such as those detailed above can then also ensure that darker acquired images are also appropriately analyzed (e.g., via appropriately scaled mapping of acquired images to lower bit depths).

In some implementations, an operational system response can be determined so as to provide appropriate resolution for features represented in a relevant dark-image training image (e.g., as represented by the histogram <NUM>). For example, if a system response that optimizes use of an imaging bit depth for a bright-image training image (e.g., as represented by the histogram <NUM>)) results in unacceptably low resolution for a dark-image training image, the operational system response <NUM> may be adjusted accordingly. In this regard, for example, in order to obtain appropriate resolution for the darkest expected images, an optimized operational system response may sometimes be determined so as to result in some oversaturation of the brightest expected images.

In some implementations, training can be executed prior to run-time operations. In some implementations, training can be executed as part of run-time operations. For example, in some implementations, the imaging system <NUM> (see <FIG>) can record candidates for bright-image and dark-image training images during a period of run-time operations. As appropriate, the imaging system <NUM> can then update the operational system response for the imaging device <NUM>. For example, the imaging system <NUM> can be configured to analyze multiple bright-image or dark-image training images similarly to the approach discussed above, in order to appropriately balance image saturation, image resolution, or other relevant factors.

In some embodiments, two imaging devices can be employed, with a first imaging device configured to acquire an initial image, and the initial image used to guide selection of system response for a second imaging device for acquisition of images for symbol evaluation. For example, a relatively fast first imaging device, such as an imaging device <NUM> in <FIG>, can be configured to capture a test image of the object <NUM> before the imaging device <NUM> captures an image for evaluation of the symbol <NUM>. The brightness (or other aspects) of the test image can then be analyzed in order to determine an appropriate system response for image acquisition by the imaging device <NUM>, including as based on the operational bit depth of the imaging device <NUM> or other relevant consideration. For example, the brightness of the test image can be analyzed in the context of the system response and other known characteristics of the imaging device <NUM> as well as known characteristics of the imaging device <NUM> in order to determine an appropriate system response for the imaging device <NUM> to capture an appropriate image. In this way, for example, acquisition of a test image by the imaging device <NUM> can help to support using the imaging device <NUM> to acquire a first image that exhibits appropriate characteristics. In some implementations, such an image acquired by the imaging device <NUM> can then be mapped to a derived image, such as described above, based upon an appropriately determined saturation threshold.

Consistent with the discussion above, some embodiments of the invention can include computer-implemented methods, including methods executed by software or hardware modules of symbol-evaluation or other (e.g., general machine-vision) systems. In this regard, for example, methods that can include one or more of the operations discussed above or below can be implemented by modules of the system <NUM> (see <FIG>), or by modules of other systems.

As one example, as illustrated in <FIG>, an embodiment of the invention can include a method <NUM> for evaluating a symbol on an object. In some implementations, the method <NUM> can be at least partly implemented using an imaging device and a processor device, such as the imaging device <NUM> and processor device of the system <NUM> (see <FIG>).

Among other operations, the method <NUM> includes acquiring <NUM>, with the imaging device, a first image of the object, including the symbol. In some cases, the acquired <NUM> first image can exhibit a particular bit depth <NUM>, as may sometimes support generation of a lower bit-depth derived image (as also discussed above).

After the first image has been acquired <NUM>, the method further includes generating <NUM>, with the processor device, a second image that is derived from the first image. In the illustrated implementation, for example, generating <NUM> the derived second image includes determining <NUM> a saturation threshold for the second image and mapping <NUM> pixels of the acquired image to the derived image, including mapping <NUM> to a maximum value, pixels of the acquired image that exceed the determined <NUM> saturation threshold. In some implementations, as also noted above, the generated <NUM> second image can exhibit a bit depth that is smaller than the bit depth <NUM>.

In some implementations and as also discussed above, it can be useful to determine <NUM> a saturation threshold based upon a target relationship <NUM> between the saturation threshold and image brightness. For example, in some implementations, the method <NUM> includes determining <NUM> a saturation threshold such that a predetermined ratio (e.g., approximately <NUM>/<NUM>) is achieved between a brightness of the generated <NUM> second image and the saturation threshold, including after mapping <NUM>, <NUM> of the pixels as discussed above.

In the illustrated implementation, once an appropriate derived image has been generated <NUM>, the method <NUM> further includes evaluating <NUM> a symbol in the second image. For example, a relevant processor device can evaluate <NUM> the symbol in order to verify that the symbol has been appropriately printed or marked, or for other reasons.

In some implementations, for the method <NUM> and others, it may be useful to determine a saturation threshold and map pixels of a particular region of interest of an image. In some cases, such a region of interest may include only a part of the relevant image. For example, for image evaluation directed mainly at evaluation <NUM> of a symbol, it may be useful to identify a region of interest for an image that encompasses the relevant symbol but excludes certain other parts of the image. A saturation threshold can then be determined <NUM>, and pixels of the image mapped <NUM> accordingly, only (or mainly) for the region of interest rather than the entire image. In some cases, an image derived based on a determined saturation threshold and corresponding pixel mapping may include only (or mainly) pixels from a region of interest in an acquired image.

In some implementations, a derived image can be presented to a user for various additional uses. For example, a second image that has been generated <NUM>, based on an appropriately determined <NUM> saturation threshold and the corresponding mapping <NUM> of pixels, can be more visually useful to a user than an original image, which may be substantially over-saturated or may have meaningful pixels clustered over a relatively small (and, e.g., dark) range. In this regard, a generated <NUM> second image can provide a useful visual display for a user, such as to allow the user to better understand the content and value of a relatively dark (or other) initial image, as well as being useful for evaluation <NUM> of a symbol or other purposes.

In some implementations, determining <NUM> a saturation threshold can include a potentially iterative process. In the implementation illustrated in <FIG>, for example, determining <NUM> a saturation threshold includes first identifying <NUM> an initial saturation threshold. As also noted above, the initial saturation threshold can be identified <NUM> in a variety of ways, including based on identifying a lower or upper end of a preset range of saturation thresholds, a midpoint or other intermediary point in such a range, or otherwise.

After the initial saturation threshold has been identified <NUM>, pixels of the acquired <NUM> first image can be mapped <NUM> to an initial (derived) second image (see also mapping <NUM> of <FIG>). A brightness measurement of the initial second image can then be determined <NUM>. For example, a mean light-pixel brightness for the initial second image can be determined <NUM> using one or more of a variety of known approaches, such as the approach taught in AIM DPM.

To determine whether the initial second image exhibits appropriate characteristics, a relationship between the brightness measurement and the initial saturation threshold can then be determined <NUM>. For example, the processor device <NUM> can determine <NUM> a ratio of the determined <NUM> brightness measurement to the identified <NUM> saturation threshold.

If the determined <NUM> relationship corresponds to a target predetermined relationship, such as a predetermined ratio (e,g, <NUM>/<NUM>), the initial second image can be set <NUM> as a final second image and a symbol in the derived image can be evaluated <NUM> (see <FIG>). If, in contrast, the determined <NUM> relationship does not correspond to the predetermined relationship, the process illustrated in <FIG> can be repeated, with identification <NUM> of a subsequent saturation threshold, mapping <NUM> of pixels to a subsequent (derived) second image based on the subsequent saturation threshold, evaluation of a relationship between the resulting image brightness and the saturation threshold, and so on.

In this regard, for example, as also discussed above, an iterative process can be employed. For example, because remapping of pixels based on an identified <NUM> saturation threshold can affect the resulting mean brightness of light pixels (and other brightness measurements), a succession of operations may be executed to identify <NUM> successive saturation thresholds, map <NUM> the pixels accordingly, determine <NUM> the resulting respective brightness measurements, and then evaluate whether the determined <NUM> relationship is appropriate or whether another saturation threshold should be identified <NUM>.

As another example, as illustrated in <FIG>, an embodiment of the invention can include a method <NUM> for evaluating a symbol on an object. In some implementations, the method <NUM> can be at least partly implemented using an imaging device and a processor device, such as the imaging device <NUM> and processor device of the system <NUM> (see <FIG>).

Among other operations, the method <NUM> includes acquiring <NUM>, with an imaging device, a first image of an object, including a symbol. In particular, the image can be acquired <NUM> with a first bit depth <NUM> (e.g., a bit depth of <NUM> bits or more). In order to facilitate further analysis, the method <NUM> also includes generating <NUM>, with a processor device, a second image from the first image, with the second image including a second bit depth <NUM> that is smaller than the first bit depth <NUM> (e.g., a bit depth of <NUM> bits or less). In some regards, this can include determining <NUM> a mapping of pixels of the first image to pixels of the second image, and implementing the determined <NUM> mapping. As appropriate, the second image can then be evaluated <NUM>, with the processor device, including to determine at least one quality attribute of the symbol.

As noted above, in some embodiments, generating <NUM> the second image can be based upon determining <NUM> a mapping of the first image to the second bit depth. In some embodiments, the mapping can be determined <NUM> based upon determining <NUM> a light-pixel mean (or other brightness measure) or a saturation threshold for the first image. For example, as also discussed above, pixels exceeding a determined saturation threshold can be set a predetermined (e.g., maximum) value and the pixels below the saturation threshold can be mapped to a lower bit depth.

In some embodiments, determining <NUM> a saturation threshold can be based upon statistical analysis of the first image. For example, a processing device can be configured to determine <NUM> a dynamic range of the first image, segment <NUM> the first image into sets of light pixels and sets of dark pixels, and then determine <NUM> an average (e.g., mean) value of the set of light pixels. The saturation threshold can then be determined <NUM> based upon the determined <NUM> average, such as by multiplying the determined <NUM> average by a predetermined multiple (e.g., approximately <NUM>/<NUM>).

In some implementations, one or both of the methods <NUM>, <NUM> can include, or can be implemented along with a training method, such as the method <NUM> illustrated in <FIG>. In some implementations, operations of the method <NUM> can be executed before operations of either of the methods <NUM>, <NUM>. In some implementations, operations of the method <NUM> can be executed simultaneously with or after operations of either of the methods <NUM>, <NUM>. In some implementations, similarly to the methods <NUM>, <NUM>, the method <NUM> can be at least partly implemented using an imaging device and a processor device, such as the imaging device <NUM> and processor device of the system <NUM> (see <FIG>).

In the embodiment illustrated in <FIG>, the method <NUM> includes determining <NUM> a bright-image system response for a relevant imaging device. In particular, for example, the bright-image system response can be determined in order to appropriately configure the imaging device to capture, with the first bit depth <NUM> (see <FIG>), a bright-image training image that represents a maximum expected brightness during run-time operations.

In some implementations, determining <NUM> the bright-image system response can be based upon a target statistical (e.g., mean intensity) value <NUM> of light pixels of a bright-image training image and a target saturation threshold <NUM>. For example, the target saturation threshold <NUM> can be set to be substantially equal to a maximum intensity value for the bit depth of the bright-image training image (or the acquired <NUM> first image of <FIG>). Similarly, the target statistical (e.g., mean intensity) value <NUM> of the light pixels can be set to exhibit an appropriate relationship to the maximum intensity value (e.g., to be <NUM> percent thereof). The bright-image system response can then be determined <NUM> as a system response that can generally ensure that the actual statistical (e.g. mean intensity) value of the light pixels in the bright-image training image substantially equals the target statistical (e.g., mean intensity) value <NUM>.

Also in the embodiment illustrated in <FIG>, the method <NUM> includes determining <NUM> a dark-image system response for the imaging device. In particular, for example, the dark-image system response can be determined in order for the imaging device to capture, with an appropriate resolution, a dark-image training image that represents a minimum expected brightness during run-time operations. For example, a target resolution <NUM> for appropriate analysis of symbols during run-time can be determined, and the dark-image system response can be determined <NUM> in order to ensure that the target resolution <NUM> is generally obtained.

Continuing, the method <NUM> includes determining <NUM>, based on one or more of the bright-image or dark-image system responses, an operational system response for the imaging device for acquisition of a first (larger bit-depth) run-time image (e.g., the acquired <NUM> image of <FIG>). For example, the processor device <NUM> (see <FIG>) can be configured to determine <NUM> an operational system response that corresponds to the determined <NUM>, <NUM> bright-image or dark-image system responses, depending on an appropriate balancing of the target resolution <NUM> for darker images and the target saturation threshold <NUM> or statistical value <NUM> of light pixels for brighter images.

As another example, consistent with the discussion above, a system for evaluating a symbol on an object can include an imaging device and a processor device in communication with the imaging device. The imaging device can be configured to acquire a first image of the object, including the symbol, with a first bit depth. The processor device can be configured to generate a second image that is derived from the first image based upon: determining a saturation threshold for the second image and mapping pixels of the first image to the second image based upon the saturation threshold for the second image. The saturation threshold for the second image can be determined based upon a predetermined target relationship between the saturation threshold for the second image and a brightness measurement of at least one of the first image or the second image.

In some embodiments, the symbol can be a DPM symbol and the object is a moving object. In some embodiments, the brightness measurement of the at least one of the first image or the second image can be determined, respectively, based upon: segmenting the at least one of the first image of the second image into light pixels and dark pixels and determining a respective mean value of the light pixels. In some embodiments, the predetermined target relationship can specify a predetermined ratio between the saturation threshold of the second image and the brightness measurement of the second image.

In some embodiments, the processor device and the imaging device can be further configured to execute training operations, prior to acquiring the first image. For example, some training operations can include: determining a bright-image system response for the imaging device to capture, with a target bit depth, a first training image that represents a maximum expected brightness; determining a dark-image system response for the imaging device to capture, with a target resolution, a second training image that represents a minimum expected brightness; and determining, based on one or more of the bright-image or dark-image system responses, an operational system response for the imaging device for acquisition of the first image.

Claim 1:
A system (<NUM>) for evaluating a symbol (<NUM>) on an object (<NUM>), the system comprising:
an imaging system (<NUM>) that includes:
an imaging device (<NUM>); and
a processor device (<NUM>) in communication with the imaging device (<NUM>);
the imaging device (<NUM>) being configured to acquire a first image of the object (<NUM>), including the symbol (<NUM>); and
the processor device (<NUM>) being configured to generate a second image that is derived from the first image, based upon:
determining a saturation threshold (<NUM>) for the second image; and
mapping pixels of the first image to the second image, including mapping pixels (<NUM>) of the first image that have values outside of the saturation threshold (<NUM>) to a maximum value for the second image, so that a brightness measurement (<NUM>) of the second image exhibits a predetermined target relationship with the saturation threshold (<NUM>) for the second image,
wherein the predetermined target relationship specifies a brightness measurement of the mapped pixels having a predetermined percentage of the saturation threshold (<NUM>) of the second image,
wherein the brightness measurement of the mapped pixels is determined based upon:
segmenting the mapped pixels into light pixels (<NUM>) and dark pixels; and
determining a mean value (<NUM>) of light pixels (<NUM>),
wherein determining the saturation threshold for the second image and mapping the pixels of the first image includes:
identifying an initial saturation threshold (<NUM>);
initially mapping pixels of the first image to an initial second image, including mapping pixels (<NUM>) of the first image that have values outside of the initial saturation threshold (<NUM>) to an initial maximum value;
determining an initial brightness measurement (<NUM>) of the initial second image and an initial relationship between the initial brightness measurement and the initial saturation threshold; and
if the initial relationship deviates from the predetermined target relationship:
identifying a subsequent saturation threshold (<NUM>);
subsequently mapping pixels of the first image to a subsequent second image, including mapping pixels (<NUM>) of the first image that have values outside of the subsequent saturation threshold (<NUM>) to a subsequent maximum value; and
determining a subsequent brightness measurement (<NUM>) of the subsequent second
image and a subsequent relationship between the subsequent brightness measurement and the subsequent saturation threshold.