Patent Description:
One challenge for dairy farmers is to handle the occurrence of microbial infections in the animals, and in particular in the milk they produce. A conventional way of determining presence of microbes in milk is to collect a milk sample from an animal and to send this sample for testing.

Another challenge for livestock farmers in general, is the occurrence of certain pathogens in livestock, their feed or their environment. Pathogens, such as salmonella and EHEC, which may be found in the animals' feed, inner organs and in the manure, may be particular causes for concern. A conventional way of determining presence of the pathogens is to collect a sample of manure from an animal or a herd and send this sample for testing. Another way is to collect a sample from an autopsy of an animal and send this sample for testing.

For the purpose of this document, the term "livestock" includes, but is not limited to, cattle, pigs, sheep and poultry, regardless of whether such livestock is kept for production of milk, meat, hide or other purposes.

The test procedure generally involves applying the sample on an agar plate, storing the agar plate for a time sufficient to allow bacterial growth to form and then to have a trained expert determine what microbes were present in the sample.

Based on such determination, actions to be taken can be determined, such as to administer antibiotics.

Unfortunately, the procedures outlined above are slow, not only due to the time it takes for the bacterial growth to form, but also due to the time it takes to ship the sample, in the sample waiting to be assessed, e.g. due to backlogs, and in the work to be performed for entering sample results for reporting.

The references <NPL> and "<NPL> disclose the use of a separate image sensor for capturing images of a growth medium test plate (ePetri dish).

<NPL>, discloses a method of processing a milk sample obtained from a livestock animal, comprising a microscopic and Gram stain evaluation and classifying the micro-organism based thereon.

<NPL> discloses a multicolour digital image analysis system and method for the identification of bacteria in a milk sample obtained from a livestock animal based on a visual spectrum image depicting at least part of the test surface.

<NPL>; <NPL> and <NPL> all disclose a method of training an image classifier algorithm for determining a micro-organism type based on a micro-organism growth pattern depicted in a visible spectrum image.

<NPL> discloses PCR based analysis of the Structure of a bacterial community in livestock manure.

There is a need for an improved method of detecting microorganisms in farm animals.

The waiting time may be determined as a predetermined time period, such as <NUM>-<NUM> hours, <NUM>-<NUM> hours, and preferably about <NUM> hours. Alternatively, the waiting time may be determined based on a growth amount.

The term "visual spectrum" implies that the image contains a spectrum that is visible to the human eye, which normally comprises wavelengths of about <NUM> to <NUM> nanometers.

A "growth pattern" is combination of shape(s) and colors that is provided by the microorganism(s) as they grow and form colonies on the growth medium.

Applicant's tests reveal that it is possible to train an image classifier algorithm to a level where the ability to correctly determine a microbe type based on a visual spectrum image of the test plate is comparable to that of a trained expert.

Hence, the method provides a user friendly way of determining presence and type of microbes in milk samples. Moreover, the method can be implemented at a substantially reduced cost compared and the availability of testing capacity can be greatly increased, with a reduction in test lead times being reduced.

Moreover, the method is implemented with hardware that is readily available to most people, such as a smartphone or a tablet, or which can be provided at low cost.

The test plate may comprise at least two juxtaposed growth medium regions, said regions differing in at least one of type, color, concentration and composition of the respective growth medium.

In particular, the test plate may present <NUM>-<NUM> such different growth medium regions, preferably <NUM>-<NUM> different growth medium regions.

The method may further comprise arranging the test plate with a predetermined orientation relative to the image capture device prior to acquiring said image, such that the growth medium regions present a predetermined orientation in said image; and/or reorienting the acquired image, such that the growth medium regions present a predetermined orientation in said image.

The waiting step may comprise maintaining the sample in a temperature controlled environment, preferably at a constant temperature of <NUM>-<NUM> degrees C.

The image capture device forms part of a smartphone or a tablet.

The method comprises positioning the test plate on a first part of an image capture support and positioning the image capture device on a second part of the image capture support, said second part being spaced from the first part, wherein the acquisition of the image is performed while the image capture device and the test plate are positioned on the image capture support.

The method may further comprise enclosing the test plate so as to shield it from ambient light and supplying light from a light source, optionally via a reflector. The light source may be provided inside an enclosure.

The image capture support may be an image capture support as described in the present document.

The method may further comprise an image limitation step, comprising cropping or masking unwanted portions of the image.

For example, the image which is sent may be the originally captured image or a processed version of the image, such as a partially cropped or masked image.

The pre-trained image classifier algorithm may comprise at least one supervised learning algorithm configured and trained to identify at least two microorganism types or microorganism classes.

Examples of supervised learning algorithms include, but are not limited to, convolutional neural networks, decision trees (such as random forest), support vector machine and fully connected neural network.

Alternatively, or as a supplement, the pre-trained image classifier algorithm may comprise a plurality of supervised learning algorithms, each of which being configured and trained to identify one microorganism type or microorganism class.

Said applying an image classifier algorithm may comprise sending the image from the image capture device via a data communication network to a remotely located processing device, feeding said image to the pre-trained image classifier to obtain a processing result based on the image, and sending the processing result via the data communication network to the image capture device or to another processing device.

The other processing device may be a further user device, such as a smartphone or tablet; a user computer, a web page or a web portal, a veterinarian's computer, etc..

The processing result may comprise an indication of a microorganism type deemed to be present on the test plate depicted on the image, and optionally a value indicating a confidence level of the processing result.

The method may further comprise waiting for a second time sufficient to allow further microbial growth to form on said test surface, acquiring a second visual spectrum image of the test surface using an image capture device, and applying the image classifier algorithm to said second image in order to determine a microorganism type based on a microorganism growth pattern visible on the image.

The second waiting time may be determined as a predetermined time period, such as <NUM>-<NUM> hours, <NUM>-<NUM> hours, and preferably about <NUM> hours. Alternatively, the waiting time may be determined based on a growth amount.

It is possible to apply further cycles of waiting and acquiring further images that are processed by being applied to the image classifier algorithm, in the manner described above.

The sample may be a milk sample from a lactating animal. In this case, the milk may be applied directly to the test surface.

Alternatively, the sample may be a manure sample from an animal. In this case, the sample may be applied directly to the test plate. Alternatively, the manure sample may be diluted, dissolved or suspended in a liquid, such as water, whereby such dilution, solution or suspension is applied to the test surface.

It is also possible to sample the animals' feed, in the same manner as manure, and in particular by first diluting, dissolving or suspending the feed in a liquid and then applying that liquid to the test surface.

According to a second aspect, there is provided a system as defined in claim <NUM> for processing a sample obtained from a livestock animal, comprising a growth medium test plate, an image capture support as described above, a user device comprising an image capture device as defined in claim <NUM> and a communication device, and a central processing device, wherein the user device is configured acquire a visual spectrum image depicting at least part of a test surface of the growth medium test plate, using the image capture device, and to send the acquired image to the central processing device, and wherein the central processing device is configured to receive the image, provide a computer-implemented pre-trained image classifier algorithm, said image classifier algorithm being pre-trained to determine a microorganism type based on a visible spectrum image depicting a growth pattern of a known microorganism, and apply the image to the pre-trained image classifier algorithm to determine a microorganism type based on a microorganism growth pattern visible on the image.

<FIG> schematically illustrates a non-limiting diagram of a system in which the present concept can be implemented.

The system may comprise a central processing unit <NUM>, a user device <NUM>; a veterinarian work station <NUM>, which is connected to a journal storage <NUM>; an image classifier subsystem <NUM> and a data storage unit <NUM>. The system may comprise further user devices and one or more user work stations <NUM>.

The central processing <NUM> unit may be implemented as a server, such as a web server, with storage and processing capability. The central processing unit may comprise the image classifier subsystem <NUM> and the storage unit <NUM>.

The storage unit <NUM> may store image data and data relating to such images. The identifiers may include one or more of position coordinates, farm id, user id, animal id, teat id, date and time.

The central processing unit <NUM> may thus run software for receiving data for communicating with the user device(s) <NUM>, <NUM>, the veterinarian work station <NUM>, and for implementing the image classifier subsystem <NUM> and the storage unit <NUM>.

Alternatively, the central processing unit may be implemented as a cloud device.

Further, the image classifier subsystem <NUM> and/or the storage unit <NUM> may be implemented as cloud devices.

The veterinarian work station <NUM> may comprise a journal storage <NUM> for storing general veterinarian records relating individual animals. Such records may be supplemented by image data, corresponding to what is stored at the storage unit <NUM>. Alternatively, or additionally, the records may merely be supplemented by processing results from the central processing unit <NUM>, as will be described in the following.

The image classifier subsystem <NUM> can be provided in the form of a supervised learning algorithm that may be implemented in the form of a neural network, such as CNN (Convolutional Neural Network) or RNN (Recurring Neural Network).

The image classifier subsystem <NUM> needs to be trained, which can be achieved by inputting a number of images of test plates with bacterial growth, which images each is associated with one or more microbe types, as identified by expert users and/or by DNA analysis.

Such image classifier subsystems <NUM> are known and available as open source software. Alternatively, the image classifier may be implemented in the central processing unit <NUM>.

Referring to <FIG>, the user device(s) <NUM> may take the form of a smart phone or tablet, which comprises an image capture device <NUM>, a processing device <NUM>, a memory <NUM>, a communication device <NUM> and a user interface <NUM>. The user device <NUM> may run software for implementing related parts of the method disclosed herein and for communicating with the central processing unit <NUM>.

<FIG> schematically illustrates a flowchart of a method in which the present concept can be implemented.

The method is described with reference to a milk sample. However, a manure sample may be processed in the same way, with the possible modification that a manure sample may, depending on its texture or viscosity, be diluted, dissolved, or suspended in a liquid, such as water, prior to its application to the test surface.

In step <NUM>, a new sample operation is initiated. This step may be preceded by a detection of an anomaly relating to the animal, e.g. in accordance with the disclosure of <CIT>. Such anomaly detection may then trigger the acquisition of a milk sample.

In step <NUM>, an animal id is entered, e.g. scanned from a tag on the animal or manually input.

In step <NUM>, a teat id is entered, e.g. manually entered by selection from a schematic image on the user interface.

In step <NUM>, a test tube id is entered, e.g. scanned or manually input.

In connection with step <NUM>, the udder is prepared, such as cleaned and a sample is collected in the test tube. These steps are typically performed in the direct vicinity of the animal.

The steps above are typically performed using a user device <NUM> in the form of a smartphone or a tablet.

The following steps may be performed in a designated area, such as a local lab or in a local control central.

In particular, milk from the test tube is transferred onto a test plate and the test tube id is also transferred to the test plate, e.g. by peeling off an id carrying sticker from the test tube and attaching the sticker to the test plate, or by associating a pre-provided test plate id with the test tube id.

After step <NUM>, the test plate may be stored for a predetermined time, such as <NUM>-<NUM> hours, preferably <NUM> hours, and preferably in a controlled environment, e.g. in a controlled temperature.

In step <NUM>, the sample is removed from storage, any lid provided on the test plate may be removed and the test plate is positioned on an image capture support, after which a first image is captured by means of the user device <NUM> (the same user device as before, or another user device having the same functionality) and sent to the central processing device <NUM>.

Before step <NUM>, the user device that is to be used for image capture may need to be initialized, e.g. by entering or scanning test plate id. Alternatively, if the test plate id is visible when the test plate is positioned in the image capture support, the identification and thus initialization may be performed in the same step as the image capture.

In step <NUM>, the image, or a limited version of the image, such as a cropped or masked version of the image, is sent to the central processing device <NUM>. The image may be sent together with data identifying the farm, the animal and the individual teat and a time and date stamp.

Analysis is then carried out in the central processing device <NUM>, wherein the pre-trained image classifier algorithm determines the type(s) of microorganisms preset on the test plate based on the visible spectrum image. The image classifier algorithm may also determine a confidence level, i.e. a value indicating to what extent the analysis can be expected to be reliable.

In step <NUM>, results are received from the central processing device <NUM>. The results may comprise an indication of one or more microbe types found to be present on the test plate when the image was captured, along with a measure of the confidence level of said result.

If the result has a sufficient level of confidence, the result may be presented in step <NUM> to the user, for example via the user device <NUM>. Such presentation may include an indication of microbe type and optionally an indication of action to be taken, such as what antibiotic to administer.

Optionally, statistical data based on the test and other tests may be presented to the user in step <NUM>.

If the result does not have a sufficient level of confidence (step Conf?), and it is determined in that another growth cycle should be performed (step Rpt?), the user may be prompted to return the test plate to the storage and wait for another predetermined amount of time, such as <NUM>-<NUM> hours, preferably <NUM> hours, and preferably in a controlled environment, e.g. in a controlled temperature.

After waiting, a second image may be captured in step <NUM> by means of the user device <NUM> and sent in step <NUM> to the central processing device <NUM>.

In step <NUM>, results are again received from the central processing device <NUM>. The results may comprise an indication of one or more microbe types found to be present on the test plate when the image was captured, along with a measure of the confidence level of said result.

If the result is determined (step Conf?) to have a level of confidence, which is not sufficiently high, and it is determined (step Rpt?) that no more growth cycles should be performed, the first and/or the second image may be sent to an evaluator, such as a veterinarian or other expert for a manual assessment in step <NUM>. Such manual classification may be based on visual inspection of the test plate by an expert user and/or by chemical or DNA analysis of microbes present on the plate.

If the result now has a sufficient level of confidence, the result may be presented in step <NUM> to the user, for example via the user device <NUM>. Such presentation may include an indication of microbe type and optionally an indication of action to be taken, such as what antibiotic to administer.

The outcome of the manual assessment made in step <NUM> may be forwarded in step <NUM> to the image classifier <NUM> for further training of the image classifier.

Referring to <FIG> and <FIG>, the image capture support <NUM> will now be described.

The image capture support comprises a pair of vertical members <NUM>, a first horizontal member <NUM> and a second horizontal member <NUM>.

The first horizontal member <NUM> is used as a test plate support and the second horizontal member <NUM> is used as an image capture device support. In the illustrated example, the first horizontal member <NUM> is positioned at a lower vertical level than the second horizontal member <NUM>.

In the illustrated example, the first horizontal member <NUM> is provided with a test plate holder <NUM>, which is a holder device that is adapted specifically to receive a test plate <NUM>. Preferably, the test plate holder <NUM> presents a vertical support surface and edges that ensure correct positioning of the test plate <NUM>. Preferably, the edges should ensure correct position in at least two mutually orthogonal directions. In the illustrated example, three edges are provided, thus ensuring correct positioning of the test plate <NUM> in three directions.

The edges may be designed such that a standardized test plate <NUM> fits snugly within the edges, with no, or very little play.

Alternatively, the edges may be designed such that the standardized test plate <NUM> is press fit between at least one pair of opposing edges. To this end, the test plate holder <NUM> may be at least partially formed of an elastic material.

As illustrated, the vertical position of the first horizontal member may be adjustably attached to the vertical member <NUM>.

The second horizontal member <NUM> is positioned above the first horizontal member <NUM>. In the illustrated example, the second horizontal member <NUM> is provided with a holder device <NUM> that is adapted to receive a user device in the form of a smartphone. To this end, the holder device <NUM> may present edges <NUM> designed to ensure that the user device is positioned in the correct position every time it is placed in the holder <NUM>.

The holder device, and thus also the second horizontal member <NUM> may further comprise a window <NUM>, which is positioned and adapted such that the user device can be positioned with its user interface facing upwardly and its camera facing downwardly, towards the first horizontal member <NUM>.

The holder device <NUM> may be designed such that its edges are horizontally movable to enable the holder device to snugly accommodate user devices of different sizes and with different camera positions.

Hence, in the illustrated example, the test plate <NUM> is to be positioned on the first horizontal member <NUM> with its test surface facing upwardly and the user device is to be positioned on the second horizontal member <NUM> with its camera facing downwardly towards the test plate <NUM>.

One or more light sources (not shown) can be provided on the image capture support <NUM>.

As a first example, a downwardly illuminating light source may be provided on the underside of the second horizontal member <NUM> and directed towards the test plate holder <NUM>.

As a second example, an upwardly illuminating light source may be provided on the underside of the test plate holder <NUM>, so as to provide back lighting of the test plate when it is positioned in the test plate holder <NUM>.

One or both light sources may be a white light source having a fixed or tunable color temperature. Alternatively, the light source may be an adjustable light source, that is capable of providing a range of colors by color mixing, such as an RGB type light source. A combination of an RGB and a tunable white light source may be provided.

The light source(s) may be configured for being controlled by the user device <NUM>. For example, the light sources may be activated in a predetermined sequence for providing front lit and back lit versions of an image. As another example, a light source may be activated in a specific sequence in order to provide an image sequence with different light colors or color temperatures.

Communication between the user device and the light source(s) may be through short range radio frequency, such as wifi or Bluetooth, or through cable.

Power supply for the light sources may be from the user device or from a separate power supply.

The image capture support <NUM> may comprise one or both of such light sources.

The light sources may be designed to provide light in the visible spectrum, and in particular white light. Optionally, the light may be tunable white light.

Referring to <FIG>, there is illustrated the image capture support <NUM> with a user device in the form of a smartphone <NUM> received in the image capture device holder <NUM> and a test plate <NUM> received in the test plate holder <NUM>.

The image capture support <NUM> may be adapted to space the image capture device <NUM> and the test plate <NUM> on the order of <NUM>-<NUM> from each other, preferably <NUM>-<NUM>.

<FIG> schematically illustrate another embodiment of an image capture support <NUM>, which differs from the image capture support <NUM> in that the sample holder is enclosed, so as to reduce the impact of ambient light conditions on the image capture process.

Just like the image capture support <NUM>, the image capture support <NUM> presets vertical members 311a, 311b, 311c, 311d, a first horizontal member <NUM> and a second horizontal member <NUM>. The second horizontal member <NUM> supports an image capture device holder <NUM>, which may be designed as described above.

The image capture support <NUM> presents vertical walls 312a, 312b, 312c, surrounding the sample holder so as to shield it from laterally incoming light. The vertical walls may include a pair of side walls 312a, 312c, a front wall 312b and a rear wall (not shown). One of the walls may comprise an opening <NUM> through which a sample holder is insertable. In the illustrated example, the opening <NUM> is provided in the front wall 312b.

The vertical walls may be formed as separate walls that are assembled and attached to the vertical members 311a, 311b, 311c, 311d, as illustrated. Alternatively, the vertical walls may be formed in one piece. As yet another alternative, the vertical walls may provide a self-supporting body, to which the horizontal members <NUM>, <NUM> are attached.

The sample holder <NUM> may be provided on a slidable member <NUM>, which is received in a sliding mechanism <NUM>. The slidable member <NUM> may comprise a front cover plate <NUM>, a handle <NUM> and a sample holder support <NUM>. The sliding mechanism may comprise horizontal grooves <NUM>, in which edges of the slidable member <NUM> are slidably received.

A light source <NUM> may be provided inside a space enclosed by the walls 312a-312c. A reflector <NUM>, <NUM>, <NUM> may be provided for reflecting the light from the light source towards the sample holder <NUM>. The reflector may comprise two or more portions <NUM>, <NUM>, <NUM>, which extend at an angle relative to each other. The portions <NUM>, <NUM>, <NUM> may be separate parts or integrated with each other, such as formed in one piece. In the illustrated example, the reflector is formed as a plate comprising a planar central portion <NUM> having an opening <NUM> for the optical path of the image capture device <NUM> and two planar side portions <NUM>, <NUM>, extending at an angle relative to the central portion <NUM>.

The reflector has a reflective surface. The reflective surface may have a mirror finish or a matte finish of a reflective color, such as white or silver, such that the reflected light is diffused for a more even distribution.

The sample holder <NUM> has the same basic function as the sample holder <NUM> disclosed above.

In addition, the insertion of the test plate <NUM> is achieved as follows.

The empty slidable member <NUM> is slid out of the opening <NUM>, after which a test plate <NUM> is positioned in the sample holder <NUM>, and the slidable member <NUM> is slid back through the opening <NUM>. The slidable member <NUM> may be designed such that, when it is fully inserted, the sample holder <NUM> is in a predetermined position relative to the image capture device.

The light source <NUM> is activated, such that light is reflected off the reflector <NUM>, <NUM>, <NUM> to provide illumination of the test plate <NUM>.

The image capture process is then carried out as described above.

The enclosure, together with the lighting arrangement, comprising the light source <NUM> and optionally the reflector <NUM>, <NUM>, <NUM> ensures consistent light conditions for all image captures.

Claim 1:
A method of processing a sample obtained from a livestock animal, comprising:
applying at least some of said sample to a test surface of a growth medium test plate (<NUM>),
waiting for a time sufficient to allow microbial growth to form on said test surface,
characterized by
acquiring a visual spectrum image depicting at least part of the test surface, using an image capture device, and
providing a computer-implemented pre-trained image classifier algorithm, said image classifier algorithm being pre-trained to determine a microorganism type based on a visible spectrum image depicting a growth pattern of a known microorganism, and
applying said image to the pre-trained image classifier algorithm to determine a microorganism type based on a microorganism growth pattern visible on the image,
wherein the image capture device (<NUM>) forms part of a smartphone (<NUM>) or a tablet,
wherein the method further comprises positioning the test plate (<NUM>) on a first part (<NUM>) of an image capture support (<NUM>) and positioning the image capture device (<NUM>) on a second part (<NUM>) of the image capture support (<NUM>), said second part being spaced from the first part, and
wherein the acquisition of the image is performed while the image capture device (<NUM>) and the test plate (<NUM>) are positioned on the image capture support (<NUM>).