Device and method for generating photometric stereo images and a color image

A device for generating a photometric stereo image and a color image of an object which is moved at a predetermined relative speed relative to the device. The device includes a lighting area and a lighting arrangement which is oriented to the lighting area. The lighting arrangement cyclically emits a lighting sequence in the lighting area. The lighting sequence has at least four lighting pulses each of which have a wavelength range and a lighting direction. At least three of the at least four lighting pulses have a different wavelength range, and at least two of the at least four lighting pulses have a different lighting direction and a same wavelength range.

CROSS REFERENCE TO PRIOR APPLICATIONS

Priority is claimed to German Patent Application No. DE 10 2021 004 071.6, filed Aug. 6, 2021. The entire disclosure of said application is incorporated by reference herein.

FIELD

The present invention relates to a device and to a method for generating photometric stereo images, and to a color image of an object moved relative to an image sensor arrangement.

BACKGROUND

Numerous industrial and scientific applications need to examine objects via an imaging technique. In the manufacturing industry, the objects may, for example, be individual or several products. In order to better detect and sort out rejects, image recordings of the products are regularly taken and evaluated to determine whether the respective product complies with specified product specifications.

Product specifications can, for example, establish requirements regarding a products' surface quality. It is frequently necessary, for example, to exclude the presence of unevenness or scratches. Other applications may in turn require that a stamping pattern attached to the products be completely and properly stamped. In photometric stereo images such as, for example, tilt images, texture images, curvature images, gradient images or height images, such three-dimensional or spatially distinct characteristics of objects can be visibly well rendered and thus be easier to examine.

Product specifications will also frequently provide defaults on two-dimensional and/or on distinctly flat characteristics, such as, for example, colorings. Unintentional discolorations can, for example, be detected or the color matching of an imprint on the objects with a printing template can be ensured. Color images of the objects must be made therefor. Color sensors or color cameras are therefore mostly used which are, however, disadvantageous especially due to their comparatively high costs of acquisition.

SUMMARY

An aspect of the present invention is to provide possibilities which allow the generation of images of the initially indicated type with low-cost expenditures, in particular without the use of color sensors or color cameras.

In an embodiment, the present invention provides a device for generating a photometric stereo image and a color image of an object which is moved at a predetermined relative speed relative to the device. The device includes a lighting area and a lighting arrangement which is oriented to the lighting area. The lighting arrangement is configured to cyclically emit a lighting sequence in the lighting area. The lighting sequence comprises at least four lighting pulses each of which have a wavelength range and a lighting direction. At least three of the at least four lighting pulses have a different wavelength range, and at least two of the at least four lighting pulses have a different lighting direction and a same wavelength range.

DETAILED DESCRIPTION

The present invention provides a device for generating photometric stereo images and the color image, wherein the device comprises a lighting area and a lighting arrangement oriented towards the lighting area, wherein the lighting arrangement is designed to cyclically radiate a lighting sequence into the lighting area, wherein the lighting sequence comprises at least four lighting pulses, wherein each lighting pulse comprises a wavelength range and a lighting direction, wherein at least three of the lighting pulses differ by their respective wavelength ranges, and wherein at least two of the lighting pulses differ by their respective lighting directions.

The device according to the present invention is advantageously able to generate (due to the different lighting directions of the lighting pulses) photometric stereo lighting which, due to the different wavelength ranges of the lighting pulses, will simultaneously render color differentiation possible.

The device according to the present invention can, for example, comprise an image data processor for the calculation of photometric stereo images and a color image of an object moved with a predetermined relative velocity relative to an image sensor arrangement with a predetermined sensor geometry, wherein the image data processor is designed to calculate the photometric stereo images and the color image from at least four image data records, wherein at least three of the image data records represent brightness values from different parts of an area of the object lit in respectively different wavelength ranges, and wherein at least two of the image data records represent brightness values of these areas of the object lit from respectively different lighting directions, wherein the image data processor is designed (based on the sensor data representing the sensor geometry and the speed values from the image data records with different wavelength ranges representing the relative velocity), to combine, as a multicolored pixel tuple, and wherein the image data processor is designed to generate brightness values of individual pixels of the same place of the object area from the image data records with different lighting directions as a multidirectional pixel tuple, and wherein the image data processor is designed to generate (based on every multicolored pixel tuple and the wavelength ranges) a pixel of the color image, and to generate (on the basis of each multidirectional pixel tuple and the lighting directions) a pixel of a photometric stereo image.

The brightness values of the image data records are in particular brightness values or intensity values measured by pixels of the image sensor arrangement. In other words, every image data record includes brightness values or intensity values measured by pixels of the image sensor arrangement under lighting in a predetermined wavelength range from a predetermined lighting direction, wherein at least three of the image data records differ due to the wavelength ranges represented therein, and wherein at least two of the image data records differ by the lighting directions represented therein. Concrete information about the different wavelength ranges and lighting directions need not, however, come from the image sensor arrangement since it is sufficient if such information is merely assignable to the image data records and in particular the brightness values included therein.

Because all brightness values are uniquely assignable to one wavelength range each, they may advantageously be available as monochromatic gray values and the pixels of the color image are nonetheless reconstructible as follows. In combination with the pertinent wavelength range, the brightness value of an individual pixel of any multicolored pixel tuple inevitably corresponds with the brightness value of a corresponding pixel of a color channel of the color image if the color channel reflects that wavelength range. Superposition of the accordingly reconstructed pixels of the color channels of all different wavelength ranges represented in the image data records from one and the same multicolored pixel tuple consequently results in the complete pixel of the color image. The entire color image results from the entirety of the generated pixels.

A similar procedure is possible in generating the pixels of a photometric stereo image. It is in this case proceeded according to the so-called photometric stereo process which is known to the expert from the state of the art and will therefore only generally be explained below. The brightness value of an individual pixel of each multidirectional pixel tuple and the associated lighting direction are entered (in accordance with the photometric stereo method) into an equation of a system of equations for the calculation of a surface normal in the place of the object area imaging the respective pixel tuple.

The system of equations here describes the qualitative connection that an object surface facing a light source will reflect more light and thus appears to be brighter for an observer (here: image sensor arrangement) than an object surface which faces away from that light source. Object surfaces standing laterally to the light source accordingly show a mean brightness. With sufficient individual pixels in the multidirectional pixel tuple, a sufficiently high number of linearly independent equations of the system of equations can be established whose set of solutions determines the surface normal. In a simple case, a differential image can be produced from only two lighting directions which can render visible many topological defects in a specific orientation. In the end, all defects in all orientations are detectable due to the calculation of additional differential images with other directional combinations.

Via an arbitrarily determinable color coding, the orientation of the surface normal calculated for the respective multidirectional pixel tuple may be allocated a color value or a gray value. This color value or gray value corresponds to the color value or gray value of the pixel in the photometric stereo image. A complete photometric stereo image results from all the pixels generated in this manner.

The present invention accordingly has the advantage that not only photometric stereo images, but also the color image, can be calculated on the basis of recorded monochromatic brightness values. The result is that no color sensor or color camera is necessary. Expenditures are correspondingly low when the image data processor is used to generate photometric stereo images and the color image.

The present invention provides a method for generating the photometric stereo image and the color image, wherein the method comprises the step of cyclical illumination of a lighting area in which the object is located with a lighting sequence, wherein the lighting sequence comprises at least four lighting pulses with each lighting pulse having a wavelength range and a lighting direction, wherein at least three of the lighting pulses differ due to their respective wavelength ranges, and wherein at least two of the lighting pulses differ by their respective lighting directions. This method provides the same advantages as the device according to the present invention described above.

The wavelength ranges may be continuous, interrupted and/or formed of individual spectral lines. Lighting directions here describe the spatial directions in which the pertinent lighting pulses spread toward the lighting area.

The present invention can be further improved via different embodiments, each of which is in itself advantageous and can be combined with each other in any manner required. These embodiments and the advantages connected therewith will be addressed below. The advantages described in relation to the image data processor and the device apply equally for the methods according to the present invention and vice versa.

According to a first possible embodiment of the present invention, the image data processor can, for example, be designed to directly read out the image data records from the image sensor arrangement after they had been recorded there. Additionally or alternatively, the image data processor can be designed to read out the image data records from the storage medium. The image data records may accordingly be saved or cached on the storage medium. If necessary, a temporal separation can thus be advantageously realized between the recording of the image data records and the calculation of the images.

In an embodiment of the device according to the present invention and/or of the method according to the present invention, the lighting sequence may also, for example, have more than four lighting pulses. In particular, at least two lighting pulses of different lighting directions may have the same wavelength range. The applied photometric stereo method is thus less vulnerable to any falsification due to the absorption of the lighting pulses. In other words, the use of the same wavelength range provides that measured brightness values purely rely on the orientation of the object surface and are not influenced by the color of the object surface.

Additionally or alternatively, at least two lighting pulses of different wavelength ranges may have the same lighting direction. Falsifications due to shadows can thus be prevented since the shadows cast can be reproduced by the same lighting direction and consequently may be identified as such.

According to another possible embodiment of the present invention, the wavelength ranges of the lighting pulses may, for example, cover an RGB color space. In other words, any of the wavelength ranges may present or form a part of the RGB color space. The wavelength ranges may thus concern the wavelength ranges of red light, green light, and blue light. This embodiment is advantageously suitable for generating RGB color images.

Additionally or alternatively, the wavelength ranges may also correspond to UV-radiation and/or IR-radiation. The color image to be calculated may then be a multispectral image in false color presentation, in particular a multispectral image or a hyperspectral image.

At least one lighting pulse, for example, each lighting pulse, can optionally form a light sheet or a light disk, respectively. In other words, the lighting pulses may be designed as light sheets or light disks directed to the lighting area. The lighting pulses may thus be distributed as homogeneously as possible over the lighting area.

As already mentioned above and according to another possible embodiment of the present invention, the device according to the present invention can, for example, have a monochromatic image sensor arrangement with a recording area, wherein the recording area of the image sensor arrangement and the lighting area overlap, and wherein the image sensor arrangement is designed to measure brightness values in the recording area during each lighting pulse. Due to the use of a monochromatic image sensor arrangement, this embodiment is advantageously characterized by particularly low costs of procurement.

According to another embodiment of the method according to the present invention, the process step of illumination can, for example, comprise the steps of emitting a lighting pulse in a first wavelength range from a first lighting direction, from a second lighting direction, from a third lighting direction, and from a fourth lighting direction; emitting a lighting pulse in a second wavelength range from each of the first, second, third and fourth lighting direction; and emitting a lighting pulse in a third wavelength range from each of the first, second, third and fourth lighting directions; wherein the first, second and third wavelength range are, among each other, not congruent, and wherein the first, second, third and fourth lighting direction are not parallel among each other. These steps can, for example, be repeated periodically.

The use of four different lighting directions adequately improves the precision of the photometric stereo method for most applications without having the computing expenditure increase disproportionately. The lighting pulses may of course be emitted from more than four different lighting directions for applications requiring yet more precise photometric stereo images.

According to another embodiment of the present invention, the lighting pulses can, for example, be individually emitted successively and each generate a unidirectional lighting constellation. Optionally, due to the simultaneous emittance of several lighting pulses with the same wavelength range and different lighting directions, a bidirectional lighting constellation or a multilaterally diffuse lighting constellation may result. This will be explained below.

For example, at least two lighting pulses in the same wavelength range may thus be emitted at the same time and generate the bidirectional lighting constellation. Vectors presenting the lighting directions of these two lighting pulses can here, for example, provide two rectified room components and one counter-directional room component. At least two additional lighting pulses in this same wavelength range can also be emitted simultaneously and generate still another bidirectional lighting constellation.

The multilateral diffuse lighting constellation mentioned previously will come about, for example, when four lighting pulses are emitted simultaneously in the same wavelength range. Vectors presenting the lighting directions of these four lighting pulses show with two of the four lighting pulses only one rectified room component and two counter-directional room components. Optionally, four additional lighting pulses in the same wavelength range can also be emitted simultaneously and the remaining lighting pulses one after the other. The multilaterally diffuse lighting constellation for the last mentioned can, in retrospect, be mathematically reconstructed so that the brightness values measured during the radiated lighting pulses one after the other are averaged.

The multilaterally diffuse lighting constellation provides uniform lighting conditions for recording the brightness values for the color image. The unilateral or bidirectional lighting constellations are primarily used for recording brightness values for a photometric stereo image. Depending on how many different unilateral or bidirectional lighting constellations there are, a tilt image, a texture image, a curvature image, a gradient image and/or a height image can be calculated as a photometric stereo image. Tilt images, texture images, curvature images, gradient images and height images can also be converted with each other via numerical integration or differentiation.

The device according to the present invention may have the previously mentioned image data processor and can thus advantageously fulfill its function without other additional devices. The device may additionally or alternatively also be designed to be connected to the computer mentioned above.

The image data processor can be designed to read out, after every lighting pulse, the brightness values from the image sensor arrangement as image data records and, in so doing, assign to each brightness value the wavelength range and the lighting direction of the lighting pulse.

According to another possible embodiment of the present invention, the image sensor arrangement can, for example, show at least four pixel fields distanced from each other at a predetermined spatial distance, wherein the time interval between two consecutive lighting pulses of the lighting sequence is equal to the time during which an image of the object projected onto the image sensor arrangement advances, due to relative speed, by a distance which is equivalent to the spatial distance of two neighboring pixel fields.

Pixel synchronicity can thereby be generated between the brightness values recorded by the pixel fields. The object can, for example, therefor move in a straight line and the pixel fields are arranged precisely behind each other, with regard to the direction of movement of the object. That means that any section of the object which is photographed by the first pixel field will also be photographed by all pixel fields arranged lying behind it.

The temporal distance corresponds with a time span and/or a period of time which is calculated from the pixel distance on the object and the relative speed of the object moved relative to the image sensor arrangement.

In other words, the lighting arrangement is designed so that the temporal distance between two consecutive lighting pulses of the lighting sequence is selected so that a specific object area is sequentially depicted on the at least four pixel fields. Pixel synchronicity is thus advantageously provided, with the generated measuring values becoming comparable.

The pixel fields of the image sensor arrangement may, for example, be pixel lines. The spatial distance accordingly presents a line distance between the pixel lines. The image sensor arrangement can optionally provide at least one pixel line for each different lighting constellation resulting from the lighting sequence. It is thereby provided that, for every multicolored pixel tuple, a multidirectional pixel tuple can also be recorded which presents the same section of the object. The image sensor arrangement can, for example, be designed as a multi-line line camera or a matrix camera readable by lines therefor. A combination of several such cameras may optionally form the image sensor arrangement.

According to an alternative embodiment of the present invention, the number of pixel lines of the image sensor arrangement can, for example, also be smaller than the number of the different lighting constellations resulting from the lighting sequence. Pixel lines are thus in fact missing to photograph every object area of the object moved in every different lighting constellation; however, the pixel synchronicity already defined above can be realized sufficiently precisely. For every “missing” pixel line, brightness values for one lighting constellation each must be recorded “between” the existing pixel lines. In other words, the temporal distance to the lighting pulse of the previous lighting constellation must be smaller than the time during which the picture and/or image of the object projected onto the image sensor arrangement is pushed forward due to the relative speed by the distance which is equivalent to the line distance of two neighboring pixel lines.

For example, in one application, the image sensor arrangement may have only two pixel lines while the lighting arrangement emits a lighting sequence with four different lighting constellations. In another application case, the image sensor arrangement may have four pixel lines while the lighting arrangement emits a lighting sequence with six different lighting constellations. The number of pixel lines and lighting sequences is in principal variable, but, the number of pixel lines is in every case smaller than the number of the lighting sequences.

Basically, however, there are not enough pixel lines in these cases to photograph every object area of the object moved in each of the different lighting constellations. In order to nonetheless achieve at least in parts the pixel synchronicity previously mentioned, for at least two lighting constellations, the time interval between two consecutive lighting pulses must be smaller than the time during which the picture of the object projected on the image sensor arrangement pushes forward due to the relative speed by the distance which is equivalent to the spatial distance of two neighboring pixel lines. Brightness values for these lighting constellations must accordingly be recorded “between” the pixel lines. The offset between the two pixel lines recorded shortened is here, for example, 50%, for example, less than 25%, for example, less than 10% of the pixel width of an individual pixel of the image sensor arrangement. Such precision is sufficient for most applications and will still be regarded as pixel synchronicity within the scope of the present invention. If this somewhat imprecise pixel synchronicity is not sufficient, the brightness values can be additionally approximated by interpolation or averaging two pixel lines.

Even with not directly neighboring pixel lines, the above-mentioned pixel synchronicity can be sufficiently realized if the lighting constellations “inserted in between” are generated in the shortest possible time space to the respective prior lighting constellation and if the measurement of the brightness values is performed immediately. That means in this case that many lighting constellations “inserted in between” are being generated while the picture and/or image of the object projected on the image sensor arrangement pushes forward due to the relative speed by the distance which is equivalent to the line distance of two neighboring pixel lines.

The straight movement of the object mentioned further above can be generated, for example, by conveying equipment. The device according to the present invention may accordingly feature the conveying equipment, and the method according to the present invention can comprise the steps of driving the conveying equipment, moving the object through the lighting area and through the recording area of the monochromatic image sensor arrangement, and recording the brightness values in the recording area during each lighting pulse. Endless recordings are possible with this embodiment so that it is in particular suitable for applications with elongated or continuous objects.

Relative speed and sensor geometry can, for example, be constant. Depending on the application, speed values and/or sensor data may change in time. The image data processor can in both cases be designed to read out or receive the speed values and the sensor data, for example, via one interface or one interface each.

It is also provided for further training and education within the scope of the method according to the present invention that a monochromatic image sensor arrangement is provided with at least four pixel fields spaced from each other at a predetermined spatial distance, wherein the time interval between two consecutive lighting pulses of the lighting sequence is selected so as to be smaller/equal to the time during which an image of the object projected onto the image sensor arrangement advances, due to relative speed, by a distance which is equivalent to the spatial distance of two neighboring pixel fields.

The present invention will be explained in greater detail and exemplarily under reference to the drawings. The combination of characteristics exemplarily presented with the forms of embodiment shown may be supplemented by additional characteristics, in accordance with the explanations above, corresponding, for a specific case of application, to the necessary properties of the image data processor and/or the device according to the present invention. In accordance with the explanations above, individual characteristics may also be left out with the embodiment described if the effect of this characteristic is not important in a concrete case of application. In the drawings, the same references are always used for elements of the same function and/or the same structure.

In the following, an image data processor1is described with reference toFIG.1. Exemplary embodiments of a device2according to the present invention are also described on the basis ofFIGS.2and3. Although some aspects of the present invention are merely described within the scope of the device2, it is of course possible that these aspects also present a description of the corresponding method wherein, for example, a block, a module, a unit or a device corresponds with a method step or a function of a method step. Analogously, aspects described within the scope of a method step also accordingly present a description of a block, a module, a unit or a property of the device2.

FIG.1shows a highly simplified, schematic presentation of an exemplary embodiment of the image data processor1. The image data processor1may have an independent processor board4and/or may be integrated on a processor board (not shown) of device2(seeFIGS.2and3). The below described blocks, modules, and units of the image data processor1which may be implemented, in each case, in hardware, software or in a combination thereof.

The image data processor1is intended for the calculation of photometric stereo images6as well as of a color image8of an object14moved with a predetermined relative speed10relative to a monochromatic image sensor arrangement12(seeFIGS.2and3) with a predetermined sensor geometry. The image data processor1is designed to calculate (in an image calculation unit16) the photometric stereo images6and the color image8from at least four image data records18. This will be explained in greater detail below.

FIG.1on the left suggests that the image data records18can be directly read out, for example, from individual monochromatic pixels20of the image sensor arrangement12after they were recorded there consecutively, as will be explained further below. The image data records18can alternatively also be stored on a storage medium22and read from there. For reading out (see thought bubble201inFIG.1), the image data processor1can provide a corresponding readout unit24and a readout interface26.

According to the present invention, the image data records18include monochromatic gray values constituting brightness values28of different sections of an area of an object14. At least three of the image data records18here represent brightness values28of the object14lit in respectively different wavelength ranges30. Such image data records18are designated inFIG.1exemplarily as image data record group32a. At least two of the image data records18represent brightness values28of the object14lit from respectively different lighting directions34. Such image data records18are designated inFIG.1exemplarily as image data record group32b.

In other words, every image data record18includes (under lighting in a predetermined wavelength range30from a predetermined lighting direction34) brightness values28measured by the image sensor arrangement12at different points of the object14, wherein at least three of the image data records18differ due to the wavelength ranges30represented therein (i.e., image data record group32a) and wherein at least two of the image data records18differ by the lighting directions30presented therein (i.e., image data records group32b). As can be seen, the same image data records18may simultaneously belong to the image data records group32aand32b.

As will still be explained further below, the different wavelength ranges30may concern, for example, the wavelength ranges of red light36, green light38, and blue light40. The wavelength ranges30may accordingly cover an RGB color space and the color image8to be calculated may be an RGB color image42. Additionally or alternatively, UV-radiation and/or IR-radiation may also be used in lighting. The color image8to be calculated may then be a multispectral image in false color presentation, in particular a multispectral image or a hyperspectral image.

As will also be still explained further below, the different lighting directions34may result from a unidirectional lighting constellation44, a bidirectional lighting constellation46, or a multilaterally diffuse lighting constellation48. The unidirectional or bidirectional lighting constellations44,46are primarily used for recording brightness values28for the photometric stereo image6. Depending on how many unidirectional or bidirectional lighting constellations44,46are available, a tilt image may be calculated, or a texture image, a curvature image, a gradient image and/or a height image(s) as a photometric stereo image6. The multilaterally diffuse lighting constellation48provides uniform lighting conditions for recording the brightness values28for the color image8.

Further below, in connection with the device2according to the present invention, the precise coming about of the lighting constellations44,46,48in the different wavelength ranges30and from the different lighting directions34will be explained in greater detail. The image data processor1, in particular the readout unit24, can be designed to allocate to every brightness value28the associated wavelength range30and the associated lighting direction34.

For improved understanding, in the presentation of the image data records18inFIG.1, the brightness values28are allocated to the individual pixels20of the image sensor arrangement12from which they had been recorded and shown together with the entire object14, the different wavelength ranges30, and the different lighting directions34. Concerned are here respectively top views50on object14from the perspective of the image sensor arrangement12. The respective lighting constellation44,46,48at hand is suggested by arrows in the top views50, wherein different arrow directions present different lighting directions34while different arrow lines (for example, dot/dash, solid line, thick line) render the different wavelength ranges30.

Brightness values28of the image data records18can be measured by the image sensor arrangement12using an electronic lock. It is accordingly identifiable that the image sensor arrangement (via which the image data records18are recorded) has four pixel lines52a,52b,52c,52d. The already above-mentioned sensor geometry is in that case primarily defined by the mutual spatial distance54of the pixel lines52a,52b,52c,52d(i.e., line space124).

The image data processor1is furthermore designed to identify (on the basis of the spatial distance54and on the basis of speed values56representing the predetermined relative speed10) from the image data records18with different wavelength ranges30, in particular from the respective image data record groups32a, brightness values28of individual pixels20of the same place of the object area and to summarize them as a multicolored pixel tuple58(see thought bubble202inFIG.1). The image data processor1is moreover designed to identify (on the basis of the spatial distance54and the speed values56from the image data records18with different wavelength ranges34, in particular from the respective image data record groups32b) brightness values of individual pixels20at the same place of the object area and to summarize them as a multidirectional pixel tuple60(see thought bubble203inFIG.1). The image calculation unit16can show a pixel tuple identification module62with a block64for identifying and summarizing multicolored pixel tuples58and a block66for identifying and summarizing multidirectional pixel tuples60therefor.

Every multicolored pixel tuple58is a tuple of length n and includes, as elements, the brightness values28from the image data records18with different wavelengths ranges30which were measured by pixels of the image sensor arrangement12at one and the same place of the object area, with length n being equivalent to the number of different wavelengths ranges30. Analogously, every multidirectional pixel tuple60is a tuple of length m and includes, as elements, brightness values28from the image data records18with different lighting directions34which were measured by pixels of the image sensor arrangement12at one and the same place of the object area, with length m being equivalent to the number of different lighting directions34.

Image data records18can, for example, be recorded so that, for every multicolored pixel tuple58a multidirectional pixel tuple60is also available which images the same place of the object area and vice-versa. This will be explained still further below in connection with the device2according to the present invention and enables the generation of pixel synchronicity between the color components of the color image8and the photometric stereo image6.

Brightness values28of every multicolored pixel tuple58can, for example, be uniquely assignable to a wavelength range30due to their arrangement within the multicolored pixel tuple58. The brightness values28of every multidirectional pixel tuple60are accordingly uniquely assignable to a lighting direction34, due to their arrangement within the pixel tuple60. InFIG.1, this is suggested, in a purely exemplary fashion, for pixel tuple58,60with three/two elements70(see thought bubbles202,203). Multicolored pixel tuples58and the multidirectional pixel tuples60can, of course, differ by their number of elements70.

The image data processor1is further designed to generate a pixel72of the color image8on the basis of every multicolored pixel tuple58and the wavelength ranges30, in particular the wavelength ranges30of the multilaterally diffuse lighting constellations48(see thought bubble204inFIG.1). Since all brightness values28, as already described above, are merely available as monochromatic gray values; assignable, however, to a wavelength range30; pixel72of the color image8can be reconstructed as follows. The brightness value28of an element70of the multicolored pixel tuple58in combination with the associated wavelength range30is in accordance with the brightness value28of a pixel74of a color channel of the color image8, wherein the color channel reflects this wavelength range30. Superposition of the accordingly reconstructed pixels74of different wavelength ranges30from one and the same multicolored pixel tuple58consequently results in the complete pixel72of the color image8. The entire color image8results from the entirety of the generated pixels72(see thought bubble206inFIG.1).

The image data processor1is moreover designed, based on each multidirectional pixel tuple60and the lighting directions34, in particular the lighting directions34of the directional lighting constellations44,46, to generate a pixel76of the photometric stereo image6(see thought bubble205inFIG.1). In generating pixel76, it is proceeded according to the so-called photometric stereo process which is known to the expert from the state of the art and will therefore only be generally explained below. Brightness value28of every element70of the multidirectional pixel tuple60and the associated lighting direction34are entered according to the photometric stereo method into an equation of a systems of equations for the calculation of a surface normal in the place of the object area imaging the respective multidirectional pixel tuple60.

The system of equations here describes the qualitative connection that an object surface facing a directional light source will reflect more light, and thus appears to be brighter for an observer (here: image sensor arrangement12), than an object surface which faces away from that light source. Object surfaces standing laterally to the light source accordingly show medium brightness. With sufficient elements70in the multidirectional pixel tuple60, a sufficiently high number of linearly independent equations of the system of equations can be established whose set of solutions determines the surface normal.

Via an arbitrarily determinable color coding, the orientation of the surface normal calculated for the multidirectional pixel tuple60may be allocated a color value or a gray value78. This color value or gray value78then corresponds with the color value or gray value80of the pixel76. The entire photometric color image6results from the entirety of the pixels76generated in this manner (see thought bubble207inFIG.1).

For the purpose of generating pixels72,76, the image calculation unit16may present a pixel generating module82with a block84for generating the pixels72of the color image8and a block86for generating pixels76of the photometric stereo image6. Based on the already mentioned pixel synchronicity, the color image8and the photometric stereo image6can be optionally combined (in the image calculation unit16) to a photometric stereo color image.

The image data processor1may optionally be provided with an internal data storage device88which is designed to save or cache intermediate results of the process steps running in the image data processor1. Thought bubbles201,202,203,204,205,206and207ofFIG.1here merely present a symbolization of the process steps running in the image data processor1. Their placement in the area of the internal data storage device88is due to reasons of space and should not mean that such graphic reproduction must take place in the image data processor1or in the internal data storage device88.

FIGS.2and3show schematic presentations of exemplary forms of embodiments of the device2according to the present invention which shows the image sensor arrangement12and a lighting arrangement90which has previously been mentioned above. The structure and function of the lighting arrangement90will be explained below with reference toFIG.2in which the image sensor arrangement12is presented in a highly simplified manner. The image sensor arrangement12is, in turn, described with reference toFIG.3in which the lighting arrangement90is presented merely in a highly simplified manner for reasons of a better overview.

Device2serves to generate the photometric stereo image6and the color image8. The lighting arrangement90of device2in particular serves to generate the lighting constellations44,46,48and the image sensor arrangement12of device2to record brightness values28for image data records18. In order to generate the relative velocity10, device2can furthermore be provided by conveying equipment92, for example, with a conveyor belt94, on which object14is movable relative to device2, in particular relative to the image sensor arrangement12. Additionally or alternatively, device2, in particular image sensor arrangement12, can also be movably designed relative to object14.

The lighting arrangement90is oriented towards a lighting area96and is designed to cyclically illuminate a lighting sequence98in the lighting area96. The lighting sequence98is provided with at least four lighting pulses100which can, for example, be periodically repeated, wherein each lighting pulse100has a wavelength range30and a lighting direction34. The wavelength range30can here be formed continuously, interrupted or formed from individual spectral lines. The lighting direction34here is the spatial direction in which the pertinent lighting pulse100spreads towards the lighting area96.

At least three of the lighting pulses100are different due to their respective wavelength ranges30, and at least two of the lighting pulses100are different due to their respective lighting directions34which is reflected in the image data record groups32a,32bpreviously mentioned.

Lighting pulses100can be individually emitted from the lighting arrangement90and respectively generate a unidirectional lighting constellation44. Optionally, due to the simultaneous emission of several lighting pulses100with the same wavelength range30and different lighting directions34, a bidirectional lighting constellation46(seeFIG.3) or a multilaterally diffuse lighting constellation48may result. This will be explained below.

The lighting arrangement90can be designed, for example, to emit a first lighting pulse100ain a first wavelength range30afrom a first lighting direction34a, a second lighting pulse100bin a second wavelength range30bfrom a second lighting direction34b, and a third lighting pulse100cin a third wavelength range30cfrom a third lighting direction34c, wherein the wavelength ranges30a,30b,30care, among each other, not congruent, and wherein the lighting directions34a,34b,34care not parallel among each other. All lighting directions may, however, intersect in the lighting area96.

As presented top left in the top views50ofFIG.1, each lighting pulse100a,100b,100cis individually emitted and each generates precisely one unidirectional lighting constellation44. Lighting pulses100a,100b,100cmay be periodically repeated.

The lighting arrangement90may furthermore be designed to emit a fourth lighting pulse100din the third wavelength range30cfrom the lighting direction34d, the lighting directions34c,34dnot being parallel among each other.

In other words, the lighting arrangement90may be designed to emit precisely one lighting pulse in the first wavelength range from each of the first, second, third and fourth lighting direction; precisely one lighting pulse in the second wavelength range from each of the first, second, third and fourth lighting direction; and precisely one lighting pulse in the third wavelength range from each of the first, second, third and fourth lighting direction. This is not separately shown. In that case, correspondingly more pixel lines would also need to be made available. Actually, exactly the number of wavelengths multiplied by the number of directions.

Two lighting pulses100can also be emitted at the same time in the same wavelength range30in order to generate a bidirectional lighting constellation46. The lighting directions of the two simultaneously emitted lighting pulses can, for example, be provided with a rectified horizontal component102and one counter-directional horizontal component104. The remaining lighting pulses in the same wavelength range30will thereafter, for example, also be emitted at the same time. This is presented in the top views50ofFIG.3.

As shown inFIG.3, lighting arrangement90may be provided with at least three electromagnetic beamers106oriented towards the lighting area96, each being designed to emit precisely one of the lighting pulses100in precisely one wavelength range30and precisely one lighting direction34so that the lighting sequence98develops which is presented top left in the top views50ofFIG.1. In particular with the lighting arrangement90presented inFIG.3, the electromagnetic beamers106are provided with a red light36, a green light38, and a blue light40, where the red light36, the green light38and the blue light40are alternately available and per electromagnetic beamer106, radiate from different spatial directions onto the lighting area96.

As shown inFIG.2, the electromagnetic beamers106are designed so that, to the extent possible, they distribute the lighting pulses100homogeneous over the lighting area. Every electromagnetic beamer106has an optical system (not shown) oriented towards the lighting area therefor.

The lighting arrangement90can, for example, provide at least one combined electromagnetic beamer106for every different wavelength range30of the lighting pulses100and/or for each different lighting direction34of the lighting pulses100. The lighting arrangement90(as exemplarily shown inFIG.3(can thus provide four electromagnetic beamers106which are designed as light strips108. In every light strip108, LEDs110of each of the different wavelength ranges30can be arranged alternatingly (for example, in the following sequence: R-G-B- . . . -R-G-B). In this respect, LEDs110of the same wavelength range30are here independently controllable from the LEDs110of the other wavelength ranges. Light strips108can also be designed so that they radiate in four different spatial orientations onto the lighting area96. It is thereby possible to generate the lighting sequences98presented in the top views50.

At least one lighting pulse100, for example, each lighting pulse100, can optionally form a light sheet112or, respectively, a light disk. Every electromagnetic beamer106may comprise an optical system (not shown) oriented towards the lighting area96therefor.

With regard to recording the brightness values28, as previously indicated above, the image sensor arrangement12has a recording area116which is superimposed with the lighting area96. The image sensor arrangement12is thereby designed so that, during each lighting pulse100, the brightness values28in the recording area116are measured.

As previously mentioned, the image sensor arrangement12is provided with at least four pixel lines52a,52b,52c,52darranged at a spatial distance54from each other. The image sensor arrangement12can in particular provide at least one pixel line for each different lighting constellation44,46,48resulting from the lighting sequence98. It is thereby provided that, for every multicolored pixel tuple58, a multidirectional pixel tuple60can also be recorded which presents the same place of the object area. The image sensor arrangement12can be designed therefor, for example, as a multi-line line camera118(seeFIG.3) or as a matrix camera120readable line by line (seeFIG.2). A combination of several such cameras may optionally also form the image sensor arrangement12.

In order to achieve the previously mentioned pixel synchronicity in a simple manner, the time interval122between two consecutive lighting pulses of the lighting sequence98can, for example, be equal to the time during which the illustration of the object14projected onto the image sensor arrangement12is pushed forward via the predetermined relative speed10by a distance which is equivalent to the spatial distance54of two neighboring pixel lines. In order to realize this, the spatial distance54and the predetermined relative speed10can be specified and the time interval122between two consecutive lighting pulses100can be adjusted thereto. This adjustment may be effected, for example, by controlling the lighting arrangement90.

The spatial distance54and the time interval122may alternatively be specified and the predetermined relative speed10adjusted thereto. The conveying equipment92may be accordingly controlled in this respect.

The time interval122and the predetermined relative speed10may furthermore be alternatively specified and the spatial distance54adjusted thereto. The spatial distance54can be structurally adjusted in advance at the image sensor arrangement12. For the matrix camera120, it is also conceivable that the line space124is influenced via which lines are read by the matrix camera120. In order to expand the spatial distance54, instead of every pixel line being read, for example, only every second pixel line will be read.

The number of pixel lines of the image sensor arrangement12can optionally also be smaller than the number of the different lighting constellations44,46,48resulting from the lighting sequence98. This is shown inFIG.3. The image sensor arrangement12there has four pixel lines52a,52b,52c,52d, while the lighting arrangement90is designed to emit a lighting sequence with six different lighting constellations46,48(see top views50inFIG.3).

There are basically thus not enough pixel lines available to record every object area of the object14moved in each of the six different lighting constellations46,48. In order to nonetheless achieve at least in parts the pixel synchronicity previously mentioned, the time interval122between two consecutive lighting pulses of the lighting sequence98must be smaller with at least two lighting constellations46than the time during which the picture of the object14projected on the image sensor arrangement12pushes forward due to the relative speed10by the distance which is equivalent to the spatial distance54of two neighboring pixel lines. Brightness values28for these lighting constellations46must accordingly be taken “between” the pixel lines. This is suggested inFIG.3by the top views50being arranged in a staggered manner. If directly neighboring pixel lines are concerned, brightness values28can be approximated by interpolation or averaging. Even with not directly neighboring pixel lines, the above-mentioned pixel synchronicity can be sufficiently realized if the lighting constellations46“inserted in between” are generated in the shortest possible time interval to the respectively prior lighting constellation and if the measurement of the brightness values is performed immediately.

FIG.3presents an imaginary grid projection126which suggests the object areas recorded by the individual pixel lines52a,52b,52c,52dof the image sensor arrangement. The spatial distance54and the pixel width68are suggested at the grid projection126.

As an alternative to pixel lines, the image sensor arrangement12may also provide different pixel fields. The image data processor1is in this case arranged to identify the multicolored pixel tuples58and the multidirectional pixel tuples60based on other sensor data57representing the sensor geometry.

Speed values56and sensor data57can, for example, be constant. Depending on the application, speed values56and/or sensor data57may change in time. For entering or reading the speed values56, the image data processor1is provided with an interface128. Via interface128, the image data processor1can be connected to an encoder (not shown) of the conveying equipment92(seeFIG.2) or receive from it the speed values56in real time. Another interface130for entering or reading out the sensor data57can optionally also be provided with the image data processor1.

Device2can present the image data processor1. Device2can alternatively also be connected to a computer132on which a computer program is executable which comprises commands causing the computer132to execute, for the image data processor1, the process steps previously described.

LIST OF REFERENCE NUMERALS