Methods and apparatus for produce identification using time resolved reflectance spectroscopy

Systems and techniques for produce identification for transaction processing. A produce item to be entered into a transaction is identified using time resolved reflectance spectroscopy. A produce item is injected with laser light over a selected range of wavelengths, and light emitted from the produce item is detected and measured over time. The measurement is processed to generate absorption and scattering spectra for the produce item. The absorption and scattering spectra are compared against those of known produce items, and upon determination that an acceptable match has been achieved, a transaction record is updated with information relating to the produce item.

FIELD OF THE INVENTION

The present invention relates generally to improved systems and techniques for product identification in retail transactions. More particularly, the invention relates to systems and techniques for detecting time resolved reflectance spectroscopy properties of produce and comparing the detected properties against stored information relating to known time domain resolved reflectance spectroscopy properties of the items.

BACKGROUND OF THE INVENTION

The increased use of automation in transaction processing has led to substantially increased efficiency and reduced costs in many applications, including supermarket transaction applications. Many products are packaged or otherwise presented so that standardized identifiers, such as bar codes, can be read by automated equipment. However, produce has been and continues to be particularly resistant to automated identification by prior art techniques. Produce identification frequently relies on books or charts of photographs available to a cashier, matching by a customer or cashier of the appearance of a produce item to a photograph on a checkout terminal display, knowledge by a cashier of the identities of various produce items, labels on each piece of produce, or any of a number of other mechanisms involving manual selections or entries by a cashier or customer. Such procedures involve labor costs to a retailer and in the case of self service entries, increase the time spent by the customer in completing a transaction. In addition, the use of labels directly affixed to food items has created its own particular difficulties since its inception. The affixing of labels to food items adds costs due to the labor or machinery needed to affix the labels, and the presence of labels on food items frequently decreases customer satisfaction. In addition, not every produce item is conducive to the use of labels.

SUMMARY OF THE INVENTION

According to one aspect, the present invention addresses such problems, as well as others, by providing systems and techniques that recognize produce items based on their chemical compositions. One particularly promising technique for determining characteristics of produce items is time resolved reflectance spectroscopy. Time resolved reflectance spectroscopy involves recognition that light injected into a turbid medium involves scattering and therefore photon migration. A light pulse injected into a medium is rcemitted over time, and the intensity of the light reemitted typically diminishes as absorption of scattered light within the medium reduces the reemission of light from the medium. The detected light intensity over time may be referred to as a temporal profile, and the temporal profile can be analyzed to generate scattering and absorption spectra.

The use of time resolved reflectance spectroscopy to analyze properties of fruits and vegetables is described by Cubbedu, Pifferi, Taroni, and Toricelli, “Measuring Fresh Fruit and Vegetable Quality: Advanced Optical Methods,” appearing inFruit and Vegetable Processing: Improving QualityEd. Jongen, Wim, Woodhead publishing (2001) pp. 150-169. (Cubbedu et al.), which is incorporated herein by reference in its entirety.

The present invention takes advantage of the fact that absorption and scattering properties of produce items can exhibit distinctive characteristics due to the chemical makeup of the produce items, and absorption and scattering spectra resulting from the presence and amount of particular components will be consistent in examples of produce items having the same components in the same quantity. Therefore, a collection of scattering and absorption signatures can be produced and stored, and used for comparison against a produce item to be identified.

A transaction terminal according to an aspect of the present invention therefore includes a produce identifier employing time domain reflectance spectroscopy. One or more produce items submitted for purchase is appropriately illuminated and light emitted back from the items as a result of the illumination are detected. The detected light is analyzed to compute a time reflectance spectroscopy signature for the submitted item, including absorption and scattering signatures. Special note is taken of the presence and level of peaks and the signature is compared against a collection of stored signatures. Upon determination that the computed signature matches a stored signature within predetermined limits, the terminal determines that identification has been successful. The name and image of the item may be presented to the user, such as a customer or operator, as a further check on the identification, with the user being allowed to accept or reject recognition of the item.

DETAILED DESCRIPTION

FIG. 1illustrates a transaction processing system100according to an aspect of the present invention. The system100includes a transaction terminal102, operating under the control of a processor104. The terminal102further includes memory106, long term storage108, a user interface112, which may be a touch screen display, and a network interface113, communicating over a bus114. The terminal102communicates with a central server116over a network, such as a local area network118.

The terminal102also includes a produce analyzer120, which employs time resolved reflectance spectroscopy. The produce analyzer120injects light into produce samples to be identified in order to cause the sample to re-emit the light. The re-emitted light is detected and analyzed in order to generate absorption and scattering spectra for the sample. The absorption and scattering spectra are compared against stored absorption and scattering spectra for known produce items. The absorption and scattering spectra are stored in association with identifications of the known produce items that they characterize.

The produce analyzer120employs its own processor122, memory124, and storage126, communicating over a bus128. The produce recognition module130further employs a light source132and a light detector134, with the light source132and light detector134operating as directed by the processor122. The produce analyzer120may suitably operate under the direction of a control module136, implemented as software residing in storage126and transferred to memory124as needed for execution by the processor122.

The produce analyzer120injects light into produce items to be identified by counting the number of photons received by the light detector134over time and processing the count information to generate absorption and scattering spectra for the item under examination. The light source132is capable of injecting light into a sample under examination, resulting in the migration of photons within the medium and the increasing absorption of photons over time.

An exemplary arrangement that may be employed is described in. Cubbedu et al., 155-157. The light source132may suitably comprise a set of pulsed laser sources138A and138B. The pulsed laser sources138A and138B are chosen to provide a desired range of wavelengths. Different produce items may produce spectra exhibiting similar features at some wavelengths, and differing features and other wavelengths, and the laser sources138A and138B are preferably chosen so that a sufficient range of wavelengths will be employed that spectra will be produced exhibiting sufficient distinctive features that different produce items can be distinguished. Two exemplary laser sources are illustrated here, with the source138A employing a wavelength of 672 nanometers (nm) and the source138B employing a wavelength of 800 nm. The pulse duration of the sources138A and138B is 100 picoseconds (ps), and the repetition rate is up to 80 megaHertz (MHz).

Light generated by the pulsed laser sources138A-138C is preferably coupled into an optical fiber140. The optical fiber140feeds into a fiber optic splitter142, which directs approximately 5% of the signal to the detector134and approximately 95% of the signal to an injection fiber146, which conveys this 95% of the signal to the produce item. The 5% of the signal received at the detector134serves to account for time drifts and to provide a time reference. The detector134is fed by a collector fiber148, and may suitably comprise a photomultiplier tube150detecting emissions from the produce item, with the tube150feeding a single-photon counter152.

The injection fiber146and the collector fiber148are maintained in position by a holder154, which maintains the fibers146and148in parallel. A produce item may be placed in a receptacle156, in contact with the fibers146and148, allowing for direct injection of light and collection of emitted light.

The produce analyzer120analyzes the temporal profile of the light emission from the produce item using the radiative transport equation under the diffusion approximation for a semi-infinite homogenous medium. As explained by Cubbedu et al., this equation is as follows:

R(ρ,t) is the number of photons per unit time (t) and area re-emitted from the tissue at a distance ρ from the injection point. ρ is the distance between the injection fiber146and the collecting fiber148. v=c/n is the speed of light in the medium. n is the refraction index. D=(3μS′)−1is the diffusion coefficient, z0=(μS′)−1is the isotropisation length, and z0is the extrapolated distance that takes into account the refraction index mismatch at the surface. The processor122operates under the control of an analysis module160to perform curve fitting according to the theoretical function, using the photon count over time and the known parameters, and the fitted curves are then analyzed to determine absorption and scattering properties of the item. Absorption properties include the absorption coefficient versus wavelength of injected light, and scattering properties include the scattering coefficient versus wavelength of injected light. The absorption and scattering properties of different items vary according to the chemical composition of the items, and such variations can be employed to identify produce items.

The server116therefore stores a produce identification database162comprising a record for each produce item that can be identified by the produce analyzer120. The database162is hosted in storage164, and the server116further employs a processor166, memory168, network interface170, and bus172.

Each record in the database162includes information such as item name, price, identification code, and other relevant information. Each record also includes information representing an optical signature of the item. Such information may include scattering and absorption spectra, comprising scattering and absorption coefficients for light injected over a range of wavelengths. The wavelengths are chosen to detect differing chemical composition of differing items, and thus to distinguish the items. Absorption spectra of chemical components often exhibit greater or lesser absorption coefficients at particular wavelengths, and differing items of produce will typically contain similar quantities of many components. Thus, many items will exhibit absorption spectra that exhibit peaks at similar wavelengths characteristic of chemical components they share in similar quantities. However, other chemical components will exist in differing quantities between items, resulting in absorption and scattering coefficients that differ between items at wavelengths characteristic of chemical components whose quantities differ between items.

The server116therefore stores absorption and scattering spectra for each produce item to be identified. Each record stored in the database162includes information relating to an absorption spectrum and a scattering spectrum representative of the produce item with which the record is associated.

The terminal100employs a transaction processing module174, suitably implemented as software residing in storage108and transferred to memory106as needed for execution by the processor104. As items are presented for entry into the transaction, relevant information is retrieved from a repository, such as a price lookup table176, and a transaction record is updated. When a produce item is presented that is to be identified, the transaction processing module174communicates with the control module136to activate the produce analyzer120.

When the produce analyzer120is activated, the control module136activates the light source132and the light detector134. The light source132injects light into the item, with appropriate ones of the lasers138A and138B being activated to provide illumination at desired wavelengths. For each of the lasers that is activated, the light detector134counts the photons conveyed to the detector134by the collector fiber148, and supplies a count per time interval to the processor122. The processor122, under the control of an emission analysis module156, analyzes the photon count over time for each wavelength and generates absorption and scattering spectra for the item. The control module136communicates the absorption and scattering spectra to the transaction processing module174, which invokes a produce identification module178to use the absorption and scattering spectra to match the produce item to a record corresponding to the produce item. The produce identification module178examines the absorption and scattering spectra to identify distinctive characteristics that can be used for matching. Such characteristics may include overall curve shape, as well as peaks or troughs representing higher or lower absorption or scattering values at different wavelengths. Characteristics that may be chosen may include each distinctive absorption value, and the wavelength at which it is exhibited, as well as each distinctive scattering value, and the wavelength at which it is exhibited. Other characteristics may include ratios between characteristics appearing at different wavelengths, to take into account the fact that the presence and thickness of skin may alter the absorption values at each wavelength while preserving the overall shape of the spectrum.

The illumination of an item and detection of the light emitted from that item, will produce characteristics typical of chemical components making up an item, and will provide distinctive features for matching against known produce items.

Once the absorption and scattering spectra analyzed and distinctive features have been identified, the produce identification module178consults the produce identification database162, and matches distinctive features of absorption and scattering spectra for the item being examined against known stored features associated with produce items whose records are stored in the database162. If matching is accomplished against one item with a sufficient degree of certainty, the item is identified and the item identification is provided to the transaction module174. If a match against a single item cannot be accomplished, but isolation to a specified number of items can be accomplished, the possible matches may be presented to a user or operator, who may then conduct further investigation to determine the identity of the item. If identification cannot be accomplished, an appropriate notification is provided to a user or operator, who may then use other mechanisms to identify the item. Once identification is provided to the transaction processing module174, the transaction processing module174enters updates a transaction record with appropriate information, such as item identification, item price, item weight or number of units, and other desired information, into a transaction record. The transaction processing module174then proceeds with the transaction. The results of the resolution of a failure of identification or the resolution of an ambiguity may be compiled and used to provide a learning capability for the produce identification module178. For example, absorption and scattering spectra associated with a failed or ambiguous identification may be stored along with the actual identification of the produce item produced by alternative mechanism for identification, and characteristics associated with such stored absorption and scattering spectra may be used to enhance the stored absorption and scattering information associated with the produce item identified using the alternative mechanism.

A system such as the system100will typically implement a number of mechanisms for entering information relating to products, and information made available through the use of these mechanisms may be used by the produce identification module178to supplement information generated by the produce analyzer120. For example, the system100may employ a scanner/scale combination180for reading bar codes and weighing items such as produce items. Once absorption and scattering spectrum information has been delivered by the produce analyzer178, this information may be evaluated in light of additional information provided by the scanner/scale combination180. A produce item may be placed on the scanner/scale combination180for weighing, and if the weight of the produce item does not conform to an expected weight for an item corresponding to the information provided by the produce analyzer120, the transaction processing module174can report the anomaly and request clarification. For example, if the produce analyzer provides information associated with a grapefruit, but the weight of the item is less than a grapefruit is expected to weigh, the transaction processing module174may prepare a message reporting the identity if the item as detected by the produce analyzer120and report that the detected weight does not conform to expectations. Weight information may also be employed to help resolve ambiguities. For example, if the produce analyzer120provides information that may be associated with either of two possible items, one weighing substantially more and one weighing substantially less, the produce identification module178may use weight information to identify the item.

The scanner scale combination180may provide imaging capabilities, and such capabilities may be employed to supply image information to the produce identification module178, which may employ such information to improve produce identification. For example, the scanner/scale combination180may provide image scanning capabilities such as are typically used for capturing and processing bar codes, and image capture may be performed using an imaging device as known in the art. The scanner/scale combination180may be configured such that an operator is allowed to choose to capture a complete image of an object in a field of view of a scan window. The produce identification database162may include image information for each item, as well as the information relating to the absorption and scattering spectra for the item. Comparisons of image information for an item against image information stored in the database162can be correlated against comparisons of the absorption and scattering spectra for an item against stored information. Correlation can be performed between the results of the comparisons, helping to identify discrepancies and resolve ambiguities. Alternative mechanisms for the use of image information to enhance produce identification may employ detection of reflected light produced as a result of optical scanning. Proper analysis of such reflected light may provide information relating to the size and shape of an object, and size and shape information may be stored in the database162for use in combination with information provided by the produce analyzer120. Exemplary systems and techniques relating to the use of image information provided by optical scanning are discussed in Mergenthaler et al., U.S. Pat. No. 7,059,527, assigned to the assignee of the present invention and incorporated herein in its entirety.

FIG. 2illustrates the steps of a process200according to the present invention. The process200may suitably be carried out using a system such as the system100ofFIG. 1. At step202, a set of records relating to produce items is compiled. Each record includes information such as item identifier, description, and price, and also includes optical characteristics used to identify the item. At step204, as a transaction at a point of sale terminal is carried out, product information is entered into a transaction record as products are presented at the terminal. When an indication is received that no further transaction entries are desired, the process proceeds to step250.

At step206, when a produce item that is to be identified is presented, a produce analyzer comprising a laser light source and a light detector is activated. The light source injects laser light over a selected range of wavelengths into the produce item, and the light detector receives light emitted from the produce item. Because a produce item is a turbid medium, light is subjected to absorption and scattering, and the absorption and scattering can be evaluated by measuring the light emitted over time, using time resolved reflectance spectroscopy. Therefore, light from the produce item s captured and fed to a measuring device, such as a detector in combination with a photon counter. At step208, the measurements of the light emitted from the produce item are processed to generate absorption and scattering spectra, and at step210, the absorption and scattering spectra are compared against stored absorption and scattering spectra associated with known produce items. Comparison may include examining absorption and scattering spectra for distinctive features, such as peaks occurring at particular wavelengths, or overall average values, and performing an initial search for spectra sharing those features, and then doing more detailed examination of spectra selected as a result of the initial search. At optional step212, identification information produced by the examination and comparison of absorption and scattering spectra is compared against additional sources of information, such as weight and image information.

If absorption and scattering spectra associated with a single item are found matching the absorption and scattering spectra for the item under examination within predetermined limits, and anomalies produced by comparison with additional information are not detected, the process proceeds to step214. If an unambiguous match cannot be found, the process skips to step220.

At step214, identification information and other relevant information for the identified produce item is retrieved and entered into a transaction record. The process then returns to step204.

At step220, a notice is presented that an unambiguous match to a known produce item cannot be achieved and requesting use of an alternative identification mechanism, such as manual entry of a code or other identification information. At step222, upon entry of appropriate identification information, identification and other relevant information is entered into the transaction record. At step224, the information used to perform identification using analysis of absorption and scattering spectra is refined using the results of the alternative identification mechanism.

The process then returns to step204.

At step250, reached after an indication that no further transaction entries are desired, a transaction total is presented, payment is processed, and the transaction is concluded.

While the present invention is disclosed in the context of a presently preferred embodiment, it will be recognized that a wide variety of implementations may be employed by persons of ordinary skill in the art consistent with the above discussion and the claims which follow below.