Range differentiators for auto-focusing in optical imaging systems

A range differentiator useful for auto-focusing, the range differentiator including an image generator providing an image of a scene at various physical depths, a depth differentiator distinguishing portions of the image at depths below a predetermined threshold, irrespective of a shape of the portions, and providing a depth differentiated image and a focus distance ascertainer ascertaining a focus distance based on the depth differentiated image.

FIELD OF THE INVENTION

The present invention relates generally to optical imaging systems and more particularly to systems and methods useful for auto-focusing in optical imaging systems.

BACKGROUND OF THE INVENTION

Various types of auto-focusing systems for use in optical imaging systems are known in the art.

SUMMARY OF THE INVENTION

The present invention seeks to provide systems and methods relating to depth range differentiation for use in auto-focusing in optical imaging systems.

There is thus provided in accordance with a preferred embodiment of the present invention a range differentiator useful for auto-focusing, the range differentiator including an image generator providing an image of a scene at various physical depths, a depth differentiator distinguishing portions of the image at depths below a predetermined threshold, irrespective of a shape of the portions, and providing a depth differentiated image and a focus distance ascertainer ascertaining a focus distance based on the depth differentiated image.

In accordance with a preferred embodiment of the present invention the image generator includes a feature specific illuminator for illuminating the scene during acquisition of the image. Additionally, the depth differentiator is operative to distinguish between the portions of the image at depths below the predetermined threshold and portions of the image at depths at or above the predetermined threshold based on differences in optical properties therebetween, under illumination by the feature specific illuminator.

In accordance with a preferred embodiment of the present invention the feature specific illuminator includes a UV illumination source and the depth differentiator is operative to distinguish between the portions of the image based on differences in fluorescence therebetween. Alternatively, the feature specific illuminator includes dark field and bright field illumination sources and the depth differentiator is operative to distinguish between the portions of the image based on differences in reflectance therebetween.

Preferably, the focus distance ascertainer is operative to ascertain the focal distance based on one of the portions of the image at depths below the predetermined threshold and the portions of the image at a depth at or above the predetermined threshold.

In accordance with a preferred embodiment of the present invention the range differentiator also includes an image focus analyzer operative to provide a focus score based on portions of the image at a depth at or above the predetermined threshold and the focus distance ascertainer is operative to ascertain the focus distance based on the focus score. Additionally, the image focus analyzer includes an illuminator for illuminating the scene with illumination for enhancing an imaged texture of the portions of the image at a depth at or above the predetermined threshold. Additionally, the illuminator includes a dark field illuminator. Alternatively or additionally, the focus score is assigned irrespective of a shape of the portions. In accordance with a preferred embodiment of the present invention the focus score is individually assigned for each pixel corresponding to the portions of the image at a depth at or above the predetermined threshold.

Preferably, the portions of the image at a depth at or above the predetermined threshold are machine identifiable.

In accordance with a preferred embodiment of the present invention the image generator includes a camera and the depth differentiated image includes a two-dimensional image of the scene. Additionally or alternatively, the image generator includes a plenoptic camera and the depth differentiated image includes a three-dimensional image of the scene. In accordance with a preferred embodiment of the present invention the feature specific illuminator includes a dark field illuminator.

In accordance with a preferred embodiment of the present invention the image generator includes a projector projecting a repeating pattern onto the scene and the depth differentiator includes a phase analyzer operative to analyze shifts in phase of the repeating pattern and derive a map of the physical depths based on the shifts in phase, the map forming the depth differentiated image. Additionally, the focus distance ascertainer is operative to ascertain the focus distance based on at least one of the physical depths.

In accordance with a preferred embodiment of the present invention the repeating pattern includes at least one of a sinusoidal repeating pattern and a binary repeating pattern. Additionally, the repeating pattern has a sufficiently low spatial frequency such that the phase analyzer is operative to uniquely correlate the shifts in phase to the physical depths. Additionally or alternatively, the map of the physical depths is one of a two dimensional map and a three dimensional map.

There is also provided in accordance with another preferred embodiment of the present invention a range differentiator useful for auto-focusing, the range differentiator including an image generator providing an image of a scene at various physical depths, a depth differentiator distinguishing portions of the image at depths below a predetermined threshold, an image focus analyzer operative to provide a focus score based on portions of the image at a depth at or above the predetermined threshold and a focus distance ascertainer ascertaining a focus distance based on the focus score.

In accordance with a preferred embodiment of the present invention the image generator includes a feature specific illuminator for illuminating the scene during acquisition of the image. Additionally, the feature specific illuminator includes a UV illumination source and the depth differentiator distinguishes portions of the image based on differences in fluorescence therebetween. Alternatively, the feature specific illuminator includes a combined dark field and bright field illuminator and the depth differentiator distinguishes portions of the image based on differences in reflectance therebetween.

In accordance with a preferred embodiment of the present invention the image focus analyzer includes an illuminator for illuminating the scene with illumination for enhancing an imaged texture of the portions of the image at a depth at or above the predetermined threshold. Additionally, the illuminator includes a dark field illuminator. Additionally or alternatively, the illuminator and the feature specific illuminator share at least one common illumination component.

In accordance with a preferred embodiment of the present invention the focus score is assigned irrespective of a shape of the portions. Additionally or alternatively, the focus score is individually assigned for each pixel corresponding to the portions of the image at a depth at or above the predetermined threshold.

Preferably, the portions of the image at a depth at or above the predetermined threshold are machine identifiable.

There is further provided in accordance with yet another preferred embodiment of the present invention a range differentiator useful for auto-focusing, the range differentiator including a target identifier including a user interface enabling a user to identify a machine identifiable feature of an object in an image, a feature detector operative to identify at least one occurrence of the machine identifiable feature in an image irrespective of a shape of the feature and a focus distance ascertainer ascertaining a focal distance to the machine identifiable feature.

Preferably, the range differentiator also includes a feature specific illuminator for illuminating the object during acquisition of the image.

In accordance with a preferred embodiment of the present invention the feature specific illuminator includes a UV illumination source and the feature identifier identifies the machine identifiable feature based on fluorescence thereof. Alternatively, the feature specific illuminator includes a combined dark field and bright field illuminator and the feature identifier identifies the machine identifiable feature based on reflectance thereof.

In accordance with a preferred embodiment of the present invention range ascertainer includes an illuminator for illuminating the object with illumination for enhancing an imaged texture of the feature of the object in the image. Additionally, the illuminator includes a dark field illuminator.

Preferably, the illuminator and the feature specific illuminator share at least one common illumination component.

In accordance with a preferred embodiment of the present invention the feature of the object includes a conductive feature. Additionally, the feature of the object includes an indent in the conductive feature.

There is yet further provided in accordance with still another preferred embodiment of the present invention a range differentiator useful for auto-focusing, the range differentiator including a first image generator including a first imaging modality and providing a first image of a scene at various physical depths, a depth differentiator distinguishing portions of the first image at depths below a predetermined threshold and providing a depth differentiated image, a focus distance ascertainer ascertaining a focal distance based on the depth differentiated image and a second image generator including a second imaging modality and providing a second image of the scene automatically focused at the focal distance.

In accordance with a preferred embodiment of the present invention the first imaging modality includes combined bright and dark field illumination and the second imaging modality includes dark field illumination. Additionally, the second image generator includes a plenoptic camera.

In accordance with a preferred embodiment of the present invention the first imaging modality includes dark field illumination and the second imaging modality includes combined bright and dark field illumination. Additionally, the first image generator includes a plenoptic camera.

There is still further provided in accordance with still another preferred embodiment of the present invention a range differentiator useful for auto-focusing, the range differentiator including a projector projecting a repeating pattern onto an object including features of various physical depths, a sensor acquiring an image of the object having the repeating pattern projected thereon, a phase analyzer analyzing shifts in phase of the repeating pattern and deriving a map of the physical depths of the features based on the shifts in phase and a focus analyzer ascertaining a focus distance to at least one of the features.

In accordance with a preferred embodiment of the present invention the repeating pattern includes at least one of a sinusoidal repeating pattern and a binary repeating pattern. Additionally or alternatively, the repeating pattern has a sufficiently low spatial frequency such that the phase analyzer is operative to uniquely correlate the shifts in phase to the physical depths.

Preferably, the map of the physical depths is one of a two dimensional map or a three dimensional map.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Reference is now made toFIG. 1, which is a simplified illustration of an optical imaging system including auto-focusing functionality, constructed and operative in accordance with a preferred embodiment of the present invention, and toFIG. 2, which is a simplified block-diagram representation of elements of a system of the type illustrated inFIG. 1.

As seen inFIGS. 1 and 2, there is provided an optical imaging system100, preferably including an optical imaging head102mounted on a chassis104. Chassis104preferably includes a table106adapted for placement thereon of an object108to be imaged. Optical imaging system100is preferably operative to provide an image of object108, for example for the purposes of inspection or processing of object108.

Object108is preferably a non-planar object comprising physical features at more than one physical depth. Here, by way of example, object108is shown to be embodied as a PCB including a non-conductive substrate109having metallic traces110formed thereon, which metallic traces110may be embedded or may protrude with respect to a surface of substrate109. It is appreciated, however, that optical imaging head102may be used to acquire images of any suitable target or scene having physical features at more than one physical height or depth including, but not limited to, PCBs, wafer dies, assembled PCBs, flat panel displays and solar energy wafers.

In some cases, it may be desirable to generate a focused image of a feature of interest included in object108, which feature of interest is at a different physical height or depth with respect to other features of object108. For example, in the case of object108, it may be desirable to generate an image in which metallic traces110are in focus for the purposes of inspection thereof. It is a particular feature of a preferred embodiment of the present invention that optical imaging system100includes a range differentiator120providing depth differentiated images and thereby enabling auto-focusing on a feature of interest, such as metallic traces110, notwithstanding the difference in physical depth between the feature of interest and other features, such as substrate109. Furthermore, such auto-focusing may be achieved by range differentiator120irrespective of a shape of the feature of interest.

As seen most clearly inFIG. 2, range differentiator120preferably includes an image generator operative to provide an image of a scene at various physical depths, here embodied, by way of example, as including illumination module122for illuminating object108. Illumination provided by illumination module122is preferably directed towards object108by way of a movable lens portion124, which moveable lens portion124is preferably mounted on a translation stage126controlled by a controller128. Light emanating from object108is preferably directed by way of moveable lens portion124towards a camera sensor130, which camera sensor130is preferably coupled to a processor132.

Range differentiator120preferably operates in two modes. In a first mode of operation of range differentiator120, object108is preferably imaged by camera sensor130under illumination conditions in which the feature of interest is clearly distinguishable from the other features of object108having a different physical depth than the feature of interest. Such imaging is preferably carried out following an initial coarse focusing of camera sensor130on object108, such that the image acquired thereby is in sufficiently good focus for subsequent processing.

Illumination under which the feature of interest is clearly distinguishable from the other features of object108having a different physical depth than the feature of interest may be termed feature specific illumination and may be provided by a feature specific illuminator140included in illumination module122. Here, by way of example only, feature specific illuminator140is shown to be embodied as a UV light source, preferably providing very short wavelength illumination having a wavelength of less than or equal to approximately 420 nm.

Under UV illumination provided by feature specific illuminator140, non-conductive substrate109fluoresces whereas metallic traces110do not. An exemplary image of substrate109and metallic traces110thereon under UV feature specific illumination conditions is shown inFIG. 3A. As seen inFIG. 3A, non-conductive substrate109has a bright appearance due to the fluorescence thereof whereas metallic traces110have a dark appearance. Non-conductive substrate109is thus clearly distinguishable from metallic traces110in the image ofFIG. 3A. Furthermore, as a result of the fluorescence of substrate109, additional features of object108that may lie beneath the surface of substrate109are masked and thereby do not appear in the image ofFIG. 3A, thus simplifying subsequent image processing.

Following the generation of an initial feature specific image, such as that shown inFIG. 3A, a tagged or segmented image is preferably generated, which segmented image is based on the initial feature specific image. An exemplary segmented image based on the feature specific image ofFIG. 3Ais illustrated inFIG. 3B. In the segmented image ofFIG. 3B, pixels corresponding to dark metallic traces110are marked in gray, identifying these pixels as corresponding to regions of interest, and pixels corresponding to bright substrate regions109are marked in white, identifying these pixels as corresponding to regions of non-interest, which regions of non-interest are to be ignored in subsequent image processing steps. Pixels corresponding to regions of unclear identity are marked in black, such as in a region112, identifying these pixels as corresponding to regions of questionable interest, which regions are also to be ignored in subsequent image processing steps. Preferably, a predetermined threshold for level of pixel brightness inFIG. 3Amay be applied in order to distinguish between dark pixels corresponding to metallic traces110and bright pixels corresponding to background substrate109.

It is understood that the segmented image ofFIG. 3Bthus effectively forms a depth differentiated mask image, in which portions of the feature specific image ofFIG. 3Aat or below a given depth, here, by way of example comprising substrate109, are distinguished from portions of the feature specific image ofFIG. 3Aabove the given depth, here, by way of example, comprising metallic traces110. It is appreciated that the differentiation between portions of the feature specific image ofFIG. 3Aat different physical depths is based on the difference in optical properties therebetween, and more specifically the difference in fluorescence under UV illumination therebetween, and is independent and irrespective of the physical shapes of the features.

The generation of the segmented mask image ofFIG. 3Bmay be automatically carried out by computing functionality included in system100, here embodied, by way of example only, as processor132, which processor132may be included in a computer144. It is appreciated that processor132thus preferably operates as a depth differentiator, operative to distinguish portions of an initial feature specific image, such as the image ofFIG. 3A, at depths below a predetermined threshold, irrespective of a shape of the portions, and to provide a depth differentiated image, such as the depth differentiated image ofFIG. 3B.

It is further appreciated that feature specific UV illuminator140in combination with sensor130and processor132constitute a particularly preferred embodiment of an image generator, providing an image of object108including substrate109and metallic traces110. It is understood, however, that the image generation functionality of range differentiator120is not limited to the particular camera and illumination components described herein and rather may comprise any suitable components functional to generate an image of a scene at various physical depths, in which features having different physical depths are differentiable based on the optical properties thereof and irrespective of the shape thereof.

Computer144may include a user interface, enabling a user to identify the feature of interest in the feature specific image, such as metallic traces110inFIG. 3A. It is appreciated that the feature of interest may be identifiable by a user as well as preferably being a machine identifiable feature, the presence of which machine identifiable feature in the feature specific image may be detected based on the appearance thereof, irrespective of the shape of the feature, by computer144. It is appreciated that computer144therefore may operate both as a target identifier, enabling a user to identify a machine identifiable feature, and as a feature detector, for preferably automatically identifying the machine identifiable feature.

In a second mode of operation of range differentiator120, following the generation of a segmented image, such as that shown inFIG. 3B, object108is preferably imaged by camera sensor130under illumination conditions best suited for enhancing the imaged texture of the feature of interest, which feature of interest is here embodied as metallic traces110. Such illumination may be termed feature focusing illumination and is preferably provided by a feature focusing illuminator150included in illumination module122. Here, by way of example only, feature focusing illuminator150is shown to be embodied as a bright field illuminator.

It is appreciated that although feature specific illuminator140and feature focusing illuminator150are shown herein to be embodied as two separate illuminators included in illumination module122, feature specific illuminator140and feature focusing illuminator150may alternatively be provided by at least partially common illumination elements having at least partially overlapping functionality, for providing both feature specific and feature focusing illumination, as is exemplified hereinbelow with reference toFIG. 6.

During the imaging of object108under lighting provided by feature focusing illuminator150, the vertical position of lens124with respect to object108is preferably incrementally shifted, such that a focal height of lens124with respect to object108is correspondingly adjusted. Adjustment of lens124may be controlled by controller128, which controller128is preferably operative to incrementally move stage126, and thereby lens124, with respect to object108. Additionally or alternatively, the focal height of lens124with respect to object108may be adjusted by way of adjustment to the height of table106and/or of optical head102in its entirety.

For each position of lens124, an image of object108is preferably acquired by sensor130. A series of images at a range of focal heights of lens124above object108is thus preferably generated. An image focus analyzer, preferably embodied as processor132, is preferably operative to perform image focus analysis on the series of images, in order to provide a focus score based on portions of each image, at a depth at or above a predetermined depth and to ascertain a focus distance based on the focus score. It is appreciated that processor132thus additionally preferably operates as a focus distance ascertainer, ascertaining a focus distance based on a depth differentiated image, such as the image ofFIG. 3B.

The focus score is preferably calculated for each image acquired under lighting conditions provided by feature focusing illuminator150, the focus score being based only on those pixels identified in the segmented depth differentiated image, such as the image ofFIG. 3B, as corresponding to regions of interest. In the case of metallic traces110on substrate109, by way of example, each pixel identified in the depth differentiated image, such as the image ofFIG. 3B, as corresponding to regions of interest, such as metallic traces110, is assigned a focus measure based on the local texture. Such a focus measure may be, by way of example, the gradient magnitude at the pixel neighborhood, or may be any other focus measure known in the art.

Pixels identified in the depth differentiated image, such as the image ofFIG. 3B, as corresponding to regions of non-interest, such as substrate109, are preferably assigned a focus measure of zero. The overall focus score of each image acquired under lighting conditions provided by feature focusing illuminator150is preferably given by the sum of the focus measures of all of the individual pixels in the image corresponding to the region of interest, such as the metallic traces110. Since the focus measures of pixels corresponding to regions of non-interest, such as substrate109, is set to zero, pixels corresponding to regions of non-interest do not contribute to the overall focus score of the image and are effectively ignored in the focus score calculation.

It is appreciated that in the above described embodiment the focus score for each image is thus preferably based only on those portions of the image at a depth equal to or above the predetermined depth, in this case corresponding to the depth of metallic traces110, and does not take into account those portions of the image below the predetermined depth, in this case corresponding to substrate109. Alternatively, the focus score may be calculated based only on those portions of the depth differentiated image at a depth below a predetermined depth, for example in the case of a feature of interest being embedded within a substrate.

The focus score obtained for each image may be plotted as a function of the focal height of lens124, as illustrated inFIG. 4. The lens position at which the feature of interest is in optimum focus may be identified as that lens position corresponding to the image having the highest focal score. In the case of data presented inFIG. 4, the highest focal score of 80 is seen to correspond to a focal height of approximately 6487 μm. A representative image having the highest focus score, in which metallic traces110are in best focus, is seen inFIG. 3C. As appreciated from consideration of the focused image ofFIG. 3C, the texture of metallic traces110is highly visible whereas substrate109appears smooth, since the image ofFIG. 3Chas been acquired at a focal height optimum for focus on metallic traces110without taking into account substrate109, which substrate109is at a different physical height than metallic traces110.

It is appreciated that the optimum focal height corresponding to the focal height of the image having the highest focus score, is preferably found to an accuracy greater than the height step between consecutive images. This may be achieved by any method suitable for finding the maximum of a function, such as, by way of example only, fitting the data in the region close to the maximum to a parabolic function.

It is further appreciated that the feature specific illumination, preferably provided by feature specific illuminator140, is not limited to UV illumination and may be any type of illumination under which target features of different physical depths exhibit a correspondingly different optical response and hence may be distinguished between in an image thereof. By way of example, UV feature specific illuminator140may be replaced by an alternative illuminator, as seen in the embodiment ofFIGS. 5 and 6.

Turning now toFIGS. 5 and 6, an optical imaging system500may be provided generally resembling optical imaging system100in relevant aspects thereof, with the exception of UV feature specific illuminator140of illuminator122of range differentiator120being replaced by a combined bright and dark field illuminator or broad angle illuminator540, as seen inFIG. 6. Feature specific illuminator540may be of a type generally described in Chinese patent application 201510423283.5, filed Jul. 17, 2015, or other illuminators known in the art.

Here, by way of example only, object108is shown to be embodied as a PCB508including a laminate region509having copper traces510formed thereon and protruding with respect thereto. For example, in the case of PCB508, it may be desirable to generate an image in which copper traces510are in focus for the purposes of inspection thereof.

Under combined bright and dark field illumination or broad angle illumination provided by feature specific illuminator540, laminate region509is significantly less reflective than copper traces510. An exemplary image of laminate region509and copper traces510under feature specific reflective illumination conditions provided by feature specific illuminator540is shown inFIG. 7A. As seen inFIG. 7A, laminate region509has a dark appearance due to the lower reflectivity thereof whereas copper traces510have a bright appearance. Laminate region509is thus clearly distinguishable from copper traces510in the image ofFIG. 7A. Furthermore, as a result of the opaque appearance of laminate509, additional features of object508that may lie beneath laminate509are masked and thereby do not appear in the image ofFIG. 7A, thus simplifying subsequent image processing.

A depth differentiated or segmented image based on the initial feature specific image ofFIG. 7Ais shown inFIG. 7B. In the segmented image ofFIG. 7B, pixels corresponding to bright copper traces510are marked in white, identifying these pixels as corresponding to regions of interest, and pixels corresponding to dark laminate region509are marked in black, identifying these pixels as corresponding to regions of non-interest, which regions of non-interest are to be ignored in subsequent image processing steps. Preferably, a predetermined threshold for level of pixel brightness may be applied in order to distinguish between white pixels corresponding to copper traces510and black pixels corresponding to laminate509.

It is understood that the segmented image ofFIG. 7Bthus effectively forms a depth differentiated image, in which portions of the feature specific image ofFIG. 7Aat depths below a given threshold, here, by way of example comprising laminate509, are distinguished from portions of the feature specific image ofFIG. 7Aat depths at or above the given threshold, here, by way of example, comprising copper traces510. It is appreciated that the differentiation between portions of the feature specific image ofFIG. 7Aat different physical depths is based on the difference in optical properties therebetween, and more specifically the difference in reflectance under combined bright and dark field or broad angle illumination therebetween, and is independent of the physical shapes of the features.

The generation of the segmented mask image ofFIG. 7Bmay be automatically carried out by computing functionality included in system500, here embodied, by way of example only, as processor132, which processor132may be included in computer144. It is appreciated that processor132thus preferably operates as a depth differentiator within system500, operative to distinguish portions of an initial feature specific image, such as the image ofFIG. 7A, at depths below a predetermined threshold, irrespective of a shape of the portions, and to provide a depth differentiated image based thereon, such as the depth differentiated image ofFIG. 7B.

The acquisition of a series of images under illumination conditions provided by feature focusing illumination150and the subsequent preferably automated selection of an image in which copper traces510are best in focus at an optimal focus distance, based on a comparison of focal scores assigned only to pixels corresponding to copper traces510identified in the segmented, depth differentiated image, such as the image ofFIG. 7B, is generally as described above with reference toFIGS. 3B-4. Generally in the manner as described hereinabove with reference toFIG. 4, processor132within system500preferably additionally operates as a focus distance ascertainer, ascertaining a focus distance based on a depth differentiated image, such as the image ofFIG. 7B.

An image of object508assigned the highest focus score, in which metallic traces510are thus in optimum focus, is seen inFIG. 7C. It is appreciated that the focus score is here preferably calculated based only on those portions of the depth differentiated image, such as the image ofFIG. 7B, at a depth at or above a predetermined depth threshold, here corresponding to protruding copper traces510. Alternatively, the focus score may be calculated based only on those portions of the depth differentiated image at a depth below a predetermined depth threshold, for example in the case of a feature of interest being embedded within a substrate.

It is appreciated that the automatically focused images generated by the systems ofFIGS. 1-2 and 5-6, such as images shown inFIGS. 3C and 7C, correspond to images obtained at a focal distance such that a particular feature of interest of the object being imaged is in best focus, notwithstanding the difference in physical height or depth between the particular feature of interest and other features that may form a part of the object being imaged.

However, systems of the present invention may alternatively be operative to automatically generate a range image of an object or scene, in order to obtain a depth profile of a particular feature of interest of the object or scene to be imaged, which feature of interest preferably has a physical depth or height differing from the depth or height of other features forming a part of the object or scene to be imaged.

The operation of a system of the type shown inFIGS. 5 and 6is now described in relation to the generation of a range image of an object1108. Object1108may include a non-conductive substrate1109having a copper region1110formed thereon, images of which object1108are shown inFIGS. 8A-8C. The system ofFIGS. 5 and 6is preferably operative to automatically generate a range image of copper region1110, which copper region1110may protrude or be recessed with respect to substrate1109. Such a range image may be useful, for example, in detecting the presence and measuring the depth of indents within copper region1110. It is appreciated that although the generation of a range image is described hereinbelow with reference to the system ofFIGS. 5 and 6, any of the systems described hereinabove may alternatively be configured to provide a range image of a feature of interest, with appropriate modifications as will be evident to one skilled in the art.

In the first mode of operation of range differentiator120in system500, object1108is preferably imaged by camera sensor130under illumination conditions in which the feature of interest is clearly distinguishable from the other features of object1108having a different physical depth than the feature of interest. An exemplary image of substrate1109and copper region1110thereon under feature specific illumination conditions is shown inFIG. 8A. As seen inFIG. 8A, non-conductive substrate1109has a dark appearance due to the low reflectance thereof whereas copper region1110has a brighter appearance. Non-conductive substrate1109is thus clearly distinguishable from copper region1110in the image ofFIG. 8A. Furthermore, as a result of the opaque appearance of substrate1109, additional features of object1108that may lie beneath substrate1109are masked and thereby do not appear in the image ofFIG. 8A, thus simplifying subsequent image processing.

Following the generation of an initial feature specific image, such as that shown inFIG. 8A, a depth differentiated or segmented image is preferably generated, which segmented image is based on the initial feature specific image. An exemplary segmented image based on the feature specific image ofFIG. 8Ais illustrated inFIG. 8B. In the segmented image ofFIG. 8B, pixels corresponding to bright copper region1110are marked in white, identifying these pixels as corresponding to a region of interest, and pixels corresponding to dark substrate regions1109are marked in black, identifying these pixels as corresponding to regions of non-interest, which regions of non-interest are to be ignored in subsequent image processing steps. Preferably, a predetermined threshold for level of pixel brightness may be applied in order to distinguish between bright pixels corresponding to copper region1110and dark pixels corresponding to background substrate1109.

It is understood that the segmented image ofFIG. 8Bthus effectively forms a depth differentiated image, in which portions of the feature specific image ofFIG. 8Aat depths below a given threshold, here, by way of example comprising substrate1109, are distinguishable from portions of the feature specific image ofFIG. 8Aat depths at or above a given threshold, here, by way of example, comprising copper region1110.

It is appreciated that the differentiation between portions of the feature specific image ofFIG. 8Aat different physical depths is based on the difference in optical properties therebetween, and more specifically the difference in reflectance under appropriate illumination therebetween, and is independent of the physical shapes of the features.

The generation of the segmented mask image ofFIG. 8Bmay be automatically carried out by processor132, which processor132may be included in computer144. It is appreciated that processor132thus preferably operates as a depth differentiator, operative to distinguish portions of an initial feature specific image, such as the image ofFIG. 8A, at depths below a predetermined threshold, irrespective of a shape of the portions, and to provide a depth differentiated image, such as the depth differentiated image seen inFIG. 8B, based thereon.

It is further appreciated that feature specific illuminator540in combination with sensor130and processor132thus constitutes a preferred embodiment of an image generator, providing an image of object1108including substrate1109and copper region1110.

Computer144may include a user interface, enabling a user to identify the feature of interest in the feature specific image, such as copper region1110inFIG. 8A. It is appreciated that the feature of interest may be identifiable by a user as well as preferably being a machine identifiable feature, the presence of which machine identifiable feature in the feature specific image may be detected based on the appearance thereof, irrespective of the shape of the feature. It is appreciated that computer144therefore may operate both as a target identifier, enabling a user to identify a machine identifiable feature, and as a feature detector, for preferably automatically identifying the machine identifiable feature.

In the second mode of operation of range differentiator120, following the generation of a segmented depth differentiated image, such as that shown inFIG. 8B, object1108is preferably imaged by camera sensor130under illumination conditions best suited for generating a depth profile of the feature of interest, which feature of interest is here embodied as copper region1110.

During the imaging of object1108under lighting provided by feature focusing illuminator150, the vertical position of lens124with respect to object1108is preferably incrementally shifted, such that focal height of lens124with respect to object1108is correspondingly adjusted. Adjustment of lens124may be controlled by controller128, which controller128is preferably operative to incrementally move stage126, and thereby lens124, with respect to object1108. Additionally or alternatively, the focal height of lens124with respect to object1108may be adjusted by way of adjustment to the height of table106and/or of optical head102in its entirety.

For each position of lens124, an image of object1108is preferably acquired by sensor130. A series of images at a range of focal heights of lens124above object108is thus preferably generated. An image focus analyzer, preferably embodied as processor132, is preferably operative to perform image focus analysis on the series of images, in order to provide a focus score based on portions of each image and to ascertain a focus distance based on the focus score. It is appreciated that processor132thus preferably operates as a focus distance ascertainer, ascertaining a focus distance based on a differentiated image, such as the image ofFIG. 8B.

It is appreciated that the focus score may be calculated based only on those portions of the depth differentiated image, such as the image ofFIG. 8B, at a depth at or above a predetermined depth threshold, in the case of protruding copper traces1110. Alternatively, the focus score may be calculated based only on those portions of the depth differentiated image at depths below a predetermined depth threshold, for example in the case of copper traces1110being embedded within substrate1109.

In this case, a focus score is preferably calculated on a pixel by pixel basis in each of the images acquired under lighting conditions provided by feature focusing illuminator150, the focus score being calculated only for those pixels identified in the segmented depth differentiated image, such as the image ofFIG. 8B, as corresponding to regions of interest. It is appreciated that in order to generate a range image, the focus score is preferably calculated for each pixel, in order to ascertain the optimum focal height corresponding to maximum measured feature texture in that pixel. It is noted that in contrast to the focal score calculation described hereinabove with reference to system100, an overall focus score, based on the sum of the focus scores of all the pixels in the region of interest in each image, is preferably not calculated in this embodiment.

In the case of copper region1110on substrate1109, by way of example, each pixel identified in the depth differentiated image, such as the image ofFIG. 8B, as corresponding to copper region1110is assigned a focus score based on an appropriate local texture measure such as the gradient magnitude or any other suitable focus measure known in the art. Pixels in the depth differentiated image, such as the image ofFIG. 8B, identified as regions of non-interest, corresponding to substrate1109in the illustrated embodiment, are assigned a focus score of zero. It is appreciated that the focus score is not calculated for those portions of each image below the predetermined brightness threshold, which portions in this case correspond to substrate1109.

The focus score obtained for each pixel may be plotted as a function of the focal height of lens124, as illustrated inFIG. 9. As seen inFIG. 9, a first trace1202represents variation of focal score with focal height in the case of a pixel corresponding to a first indent1204seen inFIG. 8A, wherein a highest focal score of 100 is seen to correspond to an absolute focal height of approximately 6486 μm. As further seen inFIG. 9, a second trace1206represents variation of focal score with focal height in the case of another pixel corresponding to a second indent1208. In this case, second indent1208is not as deep as first indent1204represented by first trace1202. As is appreciated from a comparison of first and second traces1202and1206, the height at which the maximal focal score in the case of the second indent1208occurs is shifted with respect to that of the first indent1204due to the differences in depth therebetween.

Based on functions such as those illustrated inFIG. 9, a height image may be created wherein each pixel is assigned a value equal to the focal height at which that pixel was found to have its highest focus score. Such a height image is shown inFIG. 8Cwhere the gray color scale corresponds to the pixel height in microns. As seen inFIG. 8C, gray pixels in region1110represent higher regions and white pixels in regions1204and1208represent lower regions. Black pixels in region1109correspond to pixels for which no focus score was calculated, since these pixels were identified as belonging to regions of non-interest, based on the segmented depth differentiated image, such as the image ofFIG. 8B.

It is appreciated that the height or range image ofFIG. 8Cmay be further analyzed in order to find the depth of indents1204and1208relative to the bulk of copper region1110.

It is understood that in the above-described approaches, the focal metric based on which autofocusing is achieved is applied to the features of interest only and is preferably confined within the boundaries of the features of interest. This is in contrast to conventional autofocusing methods wherein a focal metric is typically derived over the entire field of view of a camera and is thus heavily influenced by the shape and size of various features, rather than by depth alone, as is the case in the present invention.

Reference is now made toFIG. 10, which is a simplified schematic illustration of an optical processing system including depth differentiating functionality, constructed and operative in accordance with a further preferred embodiment of the present invention.

As seen inFIG. 10, there is provided an optical imaging system1300, preferably including an optical imaging head1302mounted on a chassis1304. Chassis1304preferably includes a table1306adapted for placement thereon of an object1308to be imaged. Optical imaging system1300is preferably operative to provide a depth profile image of object1308, for example for the purposes of inspection or processing of object1308.

Object1308is preferably a non-planar object comprising physical features at more than one physical depth. Here, by way of example, object1308is shown to be embodied as a PCB including a non-conductive substrate1309having metallic traces1310formed thereon, which metallic traces1310may be embedded or may protrude with respect to a surface of substrate1309. It is appreciated, however, that optical imaging head1302may be used to acquire images of any suitable target or scene having physical features at more than one physical height or depth including, but not limited to, PCBs, wafer dies, assembled PCBs, flat panel displays and solar energy wafers.

For inspection purposes, it is often desirable to generate a two-dimensional image of object1308, wherein the metallic traces1310are clearly distinguished from substrate1309based on differences in optical properties therebetween.

In some cases, it may also be desirable to generate a three-dimensional depth profile of a feature of interest included in object1308, which feature of interest is at a different physical height or depth with respect to other features of object1308. For example, in the case of substrate1309, it may be desirable to generate a depth profile image of metallic traces1310for the purposes of inspection thereof.

It is a particular feature of a preferred embodiment of the present invention that optical imaging system1300includes a combined 2D spatial and 3D range differentiator1320providing both spatially segmented and depth differentiated images of a feature of interest, such as metallic traces1310, notwithstanding the difference in physical depth between the feature of interest and other features, such as substrate1309. Particularly preferably, range differentiator1320includes a 3D plenoptic camera1321for generating a depth profile image of the feature of interest.

Range differentiator1320preferably includes an image generator operative to provide an image of a scene at various physical depths, here embodied, by way of example, as including an illumination module1322for illuminating object1308. Illumination provided by illumination module1322is preferably directed towards object1308by way of a lens portion1324. Light emanating from object1308is preferably directed towards a two-dimensional imaging camera1330, as well as towards plenoptic camera1321, via a beam splitter1332.

Illuminator module1322preferably operates in two modes, a 2D mode and a 3D mode. In a 2D mode of operation, object1308is preferably imaged by two-dimensional imaging camera1330under illumination conditions in which the feature of interest is clearly distinguishable from the other features of object1308having a different physical depth range than the feature of interest. Such illumination may be termed feature specific illumination and may be provided, by way of example only, by a bright field illuminator1340and a dark field illuminator1342included in illumination module1322. Bright field illuminator1340of illumination module1322in combination with dark field illuminator1342of illumination module1322may be considered to comprise a first portion of an image generator, delivering combined bright field and dark field illumination modalities.

Under a combination of bright and dark field illumination provided by bright field illuminator1342and dark field illuminator1342, non-conductive substrate1309exhibits reduced reflectance in comparison with the reflectance exhibited by metallic traces1310. An exemplary image of substrate1309and metallic traces1310thereon under feature specific dark and bright field illumination conditions is shown inFIG. 11A. As seen inFIG. 11A, non-conductive substrate1309has a dark appearance relative to the metallic traces1310due to the lower reflectance thereof whereas metallic traces1310have a lighter appearance relative to substrate1309. Non-conductive substrate1309is thus clearly distinguishable from metallic traces1310in the image ofFIG. 11A. Furthermore, as a result of the opacity of substrate1309, additional layers of PCB1308that may lie beneath substrate1309are obscured and thereby do not appear in the image ofFIG. 11A, thus simplifying subsequent image processing.

Following the generation of an initial feature specific image, such as that shown inFIG. 11A, a depth differentiated or segmented image is preferably generated, which segmented image is based on the initial feature specific image. An exemplary segmented image based on the feature specific image ofFIG. 11Ais illustrated inFIG. 11B. In the segmented image ofFIG. 11B, pixels corresponding to bright metallic traces1310are marked in white, distinguishing these pixels from pixels corresponding to darker substrate regions1309which are marked in black. Preferably, a predetermined threshold for level of pixel brightness may be applied in order to distinguish between bright pixels corresponding to metallic traces1310and darker pixels corresponding to background substrate1309.

It is understood that the segmented image ofFIG. 11Bthus effectively forms a depth differentiated image, in which those portions of the feature specific image ofFIG. 11Aat depths below a predetermined threshold, here, by way of example corresponding to substrate1309, are distinguished from those portions of the feature specific image ofFIG. 11Aat depths above a predetermined threshold, here, by way of example, corresponding to metallic traces1310. It is appreciated that the differentiation between portions of the feature specific image ofFIG. 11Aat different physical depths is based on the difference in optical properties therebetween, and more specifically the difference in reflectance under dark and bright field illumination therebetween, and is independent of the physical shapes of the features.

The generation of the segmented mask image ofFIG. 11Bmay be automatically carried out by computing functionality included in a processor (not shown) forming part of system1300. It is appreciated that the processor thus preferably operates as a depth differentiator, operative to distinguish portions of an initial feature specific image obtained under illumination by a first imaging modality, such as the image ofFIG. 11A, at depths below a predetermined threshold, irrespective of a shape of the portions, and to provide a depth differentiated image, such as the depth differentiated image ofFIG. 11B.

It is appreciated that the feature of interest may be identifiable by a user in the feature specific images ofFIGS. 11A and 11Bas well as preferably being a machine identifiable feature, the presence of which machine identifiable feature in the feature specific images may be detected based on the appearance thereof, irrespective of the shape of the feature.

In the 3D mode of operation of system1300, following the generation of a segmented image such as that shown inFIG. 11B, object1308is preferably imaged by plenoptic camera1321under illumination conditions best suited for enhancing the imaged texture of the feature of interest, here embodied as metallic traces1310. Such illumination may be termed feature focusing illumination and is preferably provided here by dark field illuminator1342. Dark field illuminator1342may be considered to comprise a second portion of an image generator, delivering a dark field illumination modality to object1308.

An exemplary image illustrating the appearance of metallic traces1310under dark field illumination only, in which a heightened texture of metallic traces1310is visible, in shown inFIG. 11C.

It is appreciated that although dark field illuminator1342is described herein as contributing both to the feature specific illumination and feature focusing illumination, the feature specific illumination and feature focusing illumination may alternatively be provided by disparate illumination elements not having overlapping functionality.

Furthermore, it is appreciated that the image generation functionality of range differentiator1320is not limited to the particular camera and illumination components described herein and rather may comprise any suitable components functional to generate an image of a scene at various physical depths, in which features having different physical depths are differentiable based on the optical properties thereof and irrespective of the shape thereof.

In an exemplary embodiment, plenoptic camera1321preferably provides a depth profile image of those portions identified as being suspected defects based on the 2D segmented image, such as the image ofFIG. 11B. It is appreciated that the nature of certain suspected defects identifiable in a 2D segmented image of the type shown inFIG. 11Bmay be better ascertained by way of a depth profile image, as the true nature and criticality of the suspected defect is often only revealed upon identifying its 3D profile. Efficient 2D segmentation typically requires, in addition to generating brightness differences between the substrate1309and metallic traces1310, suppressing the texture of the metal traces. This is achieved by a proper combination and careful balancing of both bright and darkfield illuminations. In contrast, 3D profiling by the plenoptic camera1321is strongly dependent on surface texture, for example in deriving stereo disparity between adjacent micro images. Using darkfield illumination alone maximizes the contrast of the surface texture of both the metallic traces1310and the substrate1309, leading to accurate depth rendering by plenoptic camera1321.

An exemplary image illustrating a depth profile of metallic traces1310as acquired by plenoptic camera1321under dark field illumination provided by dark field illuminator1342is shown inFIG. 11D. It is appreciated that although the field of view over which the depth profile ofFIG. 11Dis acquired is greater than that of the initial and segmented images ofFIGS. 11A, 11B and 11C, the depth profiling of metallic traces1310may alternatively be confined to a smaller portion of the metallic traces1310, such as in the region of a suspected defect, in order to ascertain the nature of the defect and classify the defect accordingly. In this case, the processor may operate as a focus distance ascertainer, ascertaining the focus distance at each point for depth profiling of a region in which a suspected defect lies, based on the depth differentiated image, such as the image ofFIG. 11B.

In another preferred mode of operation of the combined 2D spatial and 3D depth range differentiator1320, plenoptic camera1321may be employed to automatically focus 2D camera1330prior to acquiring of the 2D image thereby.

In this autofocusing mode, the inspected object1308is preferably initially brought to a coarse focus of plenoptic camera1321under feature-focusing illumination conditions, such as dark field illumination conditions preferably provided by dark field illuminator1342. Such a preliminary coarse focus may be based on system optimization and engineering parameters and may involve pre-calibration of system1300, as is well known by those skilled in the art.FIG. 12Ashows an exemplary coarsely focused image of a substrate1410, as acquired by plenoptic camera1321. In the illustrated embodiment, substrate1410is a silicon wafer, containing an abrupt height step1420, with laser inscribed pits1430thereon. A corresponding out-of-focus 2D image as received by 2D camera1330is shown inFIG. 12B.

The coarsely focused image acquired by plenoptic camera1321may then be processed by computing functionality included in the processor of system1300, in order to derive a depth profile of the instant field of view of substrate1410. An exemplary depth differentiated profile image based on the coarsely focused image ofFIG. 12Ais shown inFIG. 12C. It is appreciated that, in contrast to the example illustrated inFIGS. 11A-11D, in this mode of operation of range differentiator1320, the bright field illumination modality provided by bright field illuminator1340preferably constitutes a first imaging illumination modality, under which illumination a depth differentiable image is preferably acquired.

Based on the depth profile image ofFIG. 12C, the feature depth at which 2D camera1330should optimally be focused may be selected. By way of example, in the case of substrate1410, the optimal focal depth of 2D camera1330may be that depth corresponding to the height of the upper side1440of the step in the silicon wafer in the image ofFIG. 12C. As is appreciated by one skilled in the art, the depth of focus of plenoptic camera1321typically straddles the depth of field of 2D camera1330and may be in the range of 2-4 times greater, such that the accuracy of the depth profile analysis based on the plenoptic image ofFIG. 12Ais at least as good as the accuracy achievable based on the depth of focus on lens1324.

2D camera1330may then be automatically focused on the upper side1440of the silicon step at the optimal focus depth identified based on the depth profile image ofFIG. 12Cand a focused 2D image of substrate1410correspondingly acquired under feature specific bright field illumination conditions. It is noted that in this case the focus specific illumination is the same as the feature specific illumination. This is a consequence of the optical reflection properties of both the silicon wafer and the laser formed pits on its surface. An exemplary automatically focused 2D image acquired under feature specific bright field illumination conditions is shown inFIG. 12D.

It is appreciated that following the automatically focused 2D imaging, additional 3D plenoptic imaging of object1308may be performed if necessary, for example for the purpose of better classifying the nature of suspected defects present in the 2D autofocused image, as described hereinabove with reference toFIGS. 11C and 11D.

Reference is now made toFIG. 13, which is a simplified illustration of an optical processing system including auto-focusing functionality, constructed and operative in accordance with a further preferred embodiment of the present invention, and toFIGS. 14A-14C, which are simplified examples of images produced by a system of the type illustrated inFIG. 13.

As seen inFIG. 13, there is provided an optical imaging system1500including a projector module1502operative to project a pattern onto an object1508. Imaging system1500further preferably includes a camera sensor module1510operative to acquire an image of object1508when a pattern is projected thereon by projector module1502. Preferably, projector module1502and camera module1510are angled with respect to a longitudinal axis1512defined with respect to object1508. Projector module1502in combination with camera module1510may be considered to form an image generator, operative to generate an image of object1508.

Object1508is preferably a non-planar object comprising physical features at more than one physical depth including, but not limited to, PCBs, wafer dies, assembled PCBs, flat panel displays and solar energy wafers. Alternatively, object1508may be embodied as any object or scene containing features at a range of physical depths.

In some cases, it may be desirable to generate a focused image of a feature of interest included in object1508, which feature of interest is at a different physical height or depth with respect to other features of object1508. This may be automatically achieved in system1500by way of projecting a regularly repeating pattern, such as a sinusoidal or binary moiré fringe pattern, onto a surface of object1508and analyzing the shift in phase of the projected fringes, as is detailed herein below.

The operation of system1500may be best understood with reference to the images generated thereby, examples of which images are presented inFIGS. 14A-14C.

Turning now toFIG. 14A, an image of a fringe pattern1600, preferably projected by projector module1502onto a surface of object1508, is illustrated. As seen inFIG. 14A, fringe pattern1600undergoes variable phase shifts depending on the surface topology of the features on object1508upon which the fringe pattern falls. Computing functionality included in a processor1516forming part of system1500may be operative to compute, preferably in real time, the phase shift in fringe pattern1600in order to derive at least the height of the physical feature upon which the fringe pattern is projected. Processor1516may be operative as a depth differentiator, for differentiating portions of the images acquired by camera module1510at various physical heights, irrespective of the shape thereof. Fringe phase shift analysis carried out by computing functionality included in processor1516may include, by way of example, a windowed Fourier transform. Additionally, processor1516may also control the generation of fringe patterns projected by projector module1502.

The height of the physical feature is preferably computed relative to the height of a reference target incorporated in system1500. The height of the reference target may be calibrated with respect to an additional imaging functionality (not shown) of system1500maintained in focus relative to object1508or may be calibrated with respect to camera sensor1510.

A two-dimensional height map and a three-dimensional height map of object1508based on the projected fringe map ofFIG. 14Aare respectively illustrated inFIGS. 14B and 14C. As seen inFIGS. 14B and 14C, the shifts in phase of the projected fringe pattern may be used as a basis for segmenting object1508according to the relative heights of the physical features responsible for producing the corresponding shifts in the phase pattern. A feature of given height may thus be selected for optimum focusing thereupon, whilst features at heights other than the selected height are effectively ignored in subsequent image focusing. It is appreciated that the height maps ofFIGS. 14B and 14Cthus constitute segmented or depth differentiated images, based on which a depth of features selected for optimum focus thereon may be ascertained. Based on height selection alone, autofocusing of camera1510may thus be performed on features at a given height level, irrespective of a shape of those features. The optimum focus distance may be ascertained by way of processor1516based on the depth differentiated images ofFIGS. 14B and 14C.

It is appreciated that the optimum spatial frequency of the fringe pattern projected by projector module1502is preferably set by taking into account and balancing several opposing requirements. The spatial frequency of the fringe pattern is preferably selected so as to be low enough to allow projection and imaging thereof with good contrast. In addition, the spatial frequency of the fringe pattern is preferably selected so as to be high enough to allow sufficiently high resolution height differentiation. Furthermore, the inter-fringe spacing within the fringe pattern is preferably selected so as to be large enough to encompass the full expected depth of object1508without phase ambiguity. Preferably, the fringe pattern has a sufficiently low spatial frequency such that shifts in phase thereof may be uniquely correlated to the physical depths giving rise to such shifts, without phase ambiguity.

At least these various factors are preferably balanced in order to derive the optimum spatial frequency of the fringe pattern for a particular imaging application.

System1500may be particularly well-suited for use in a closed-loop tracking autofocus mode, wherein object1508is preferably scanned continuously. In a continuous scanning mode, projector module1502is preferably strobed so as to operate in a pulsed mode, preferably in synchronization with the operation of camera module1510. Alternatively, projector module1502may operate continuously, preferably in conjunction with a globally shuttered camera module1510.

In use of system1500for continuous closed loop autofocusing operation, various operational parameters of system1500are preferably optimized. The temporal rate at which the height of object1508is sampled, by way of the projection of fringe pattern1600thereon and subsequent analysis of phase shifts thereof, is preferably selected so as to be sufficiently high to be suited to the scanning speed of object1508and the rate of height variations thereof. The operational frame rate of camera module1510is preferably set in accordance with the height sampling rate.

Additionally, the elapsed time between fringe image acquisition by camera module1510and the obtaining of an analyzed height map, which time delay may be termed the system latency, is preferably optimized. The system latency may be primarily dependent on the computing performance of a system controller of system1500. The system latency is preferably set so as to be sufficiently short in order to avoid an excessive lag in the operation of the autofocusing functionality following the fringe image acquisition, which excessive lag would otherwise lead to focusing errors of the imaging functionality.

In certain embodiments of the present invention, the pixel resolution of camera module1510may be set so as to optimize the performance of system1500. The fewer the imaging pixels of camera1510, the higher the camera frame rate operation and the shorter the processing time. Additionally or alternatively, rather than computing the phase shift over the entirety of the images acquired by camera module1510, the phase shift may only be computed within sparsely selected regions inside the image frames outputted by camera module1510, whereby processing time may be accelerated. The number, size, aspect ratio and spacing of those regions within which the phase shift is computed may be selected by taking into account physical or other characteristics of object1508.

It will be appreciated by persons skilled in the art that the present invention is not limited by what has been particularly claimed hereinbelow. Rather, the scope of the invention includes various combinations and subcombinations of the features described hereinabove as well as modifications and variations thereof as would occur to persons skilled in the art upon reading the forgoing description with reference to the drawings and which are not in the prior art.