Source: https://patents.google.com/patent/US10127446B2/en
Timestamp: 2019-04-25 03:22:30+00:00

Document:
A system comprises a memory operable to store first light intensity information for a first pixel of an image that includes a dairy livestock, and second light intensity information for a second pixel of the image, wherein the second pixel is adjacent to the first pixel. The system further comprises a processor communicatively coupled to the memory and operable to determine that a difference between the first light intensity information and the second light intensity information exceeds a threshold, and discard one of the first pixel or the second pixel from the image.
This application is a continuation of U.S. Ser. No. 14/548,399, filed Nov. 20, 2014 entitled System and Method for Filtering Data Captured by a 2D Camera, which is a continuation of U.S. Ser. No. 13/448,799 filed Apr. 17, 2012 entitled System and Method for Filtering Data Captured by a 2D Camera, now U.S. Pat. No. 8,903,129 issuing Dec. 2, 2014, which is a continuation-in-part application of U.S. patent application Ser. No. 13/095,994 entitled “Vision System for Robotic Attacher”, filed Apr. 28, 2011, which is now U.S. Pat. No. 8,671,885 issued Mar. 18, 2014, the disclosure of each is hereby incorporated by reference herein.
In particular, enclosure 100 may be divided into a number of regions 110 (e.g., regions 110 a and 110 b), and each region 110 may include resting stalls, feeding troughs, walking paths, and/or other structure suitable for housing dairy livestock. Although the present disclosure contemplates enclosure 100 as housing any suitable dairy livestock (e.g., dairy cows, goats, sheep, water buffalo, etc.), the remainder of this description is detailed with respect to dairy cows.
Each milking box 120 may include a stall portion 122 configured to house a dairy cow being milked. The stall portion 122 of each milking box 120 may be defined by a number of walls 124, each of which may each be constructed from any suitable materials arranged in any suitable configuration operable to maintain a dairy cow within stall portion 122 during milking. In certain embodiments, stall portion 122 of milking box 120 may include walls 124 a, 124 b, 124 c, and 124 d. For purposes of illustration, wall 124 a may be designated as the front of milking box 120 such that the head of a dairy cow being milked would be facing wall 124 a. Wall 124 c may be positioned opposite wall 124 a and may be designated as the rear of milking box 120. Walls 124 b and 124 d may each form a side extending between the front and rear of milking box 120. Walls 124 a, 124 b, 124 c, and 124 d may be spaced apart a suitable distance to ensure the comfort of the dairy cow within stall portion 122.
Walls 124 b and/or 124 d may comprise one or more gates 126. In certain embodiments, wall 124 b and/or wall 124 d may comprise an entry gate 126 a and an exit gate 126 b. A dairy cow may enter milking box 120 through an opened entry gate 126 a and exit milking box 120 through an opened exit gate 126 b. Closing gates 126 may maintain the dairy cow within milking box 120 during milking, while opening one or more gates 126 may allow the dairy cow to exit milking box 120. In certain embodiments, gates 126 may each be coupled to a corresponding actuator such that the gates 126 may be automatically opened and/or closed. For example, the actuators corresponding to gates 126 may each be configured to communicate (e.g., via wireless or wireline communication) with a controller 200, depicted in detail in FIG. 2.
Each milking box 120 may additionally include an equipment portion 128 located to the rear of stall portion 122 (i.e., adjacent to rear wall 124 c of stall portion 122). Equipment portion 128 may comprise any structure suitable for housing and/or storing a robotic attacher (e.g., robotic attacher 150, described below with regard to FIG. 3), one or more preparation cups, teat cups, receiver jars, separation containers, and/or any other suitable milking equipment. Rear wall 124 c (which may include a backplane 138, as described below with regard to FIG. 3) may separate stall portion 122 from equipment portion 128 such that equipment portion 128 is substantially inaccessible to a dairy cow located in stall portion 122. Accordingly a dairy cow located in stall portion 122 may be prevented from accidentally damaging the milking equipment by kicking, biting, trampling, or exposing the milking equipment to dirt, fluids, etc.
In certain embodiments, the equipment portion 128 being located to the rear of stall portion 122 may allow milking boxes 120 to be aligned in a single row such that walls 124 b and 124 d of each milking box 120 may comprise an entry gate 126 a and an exit gate 126 b (as illustrated in FIG. 1A). As a result, milking boxes 120 may be used to sort dairy cows into particular regions 110 by controlling the opening/closing of each gate 126 (e.g., in response to signals from a controller 200, as described above). For example, a dairy cow needing a health check or medical attention may be sorted into an appropriate region 110 (e.g., a veterinary pen). As another example, a dairy cow determined to be finished milking for the year and needing to be dried off and bread may be sorted out of the milking heard. As yet another example, a dairy cow may be sorted into one of a number of regions 110 based on the stage of lactation of the dairy cow (as dairy cows in different stages may require different feeds).
In certain other embodiments, the equipment portion 128 being located to the rear of stall portion 122 may allow pairs of milking boxes 120 to be located side by side such that the milking boxes share a wall 124 (e.g., wall 124 b may be shared between milking box 120 c and milking box 120 d, as depicted in FIG. 1B). As a result, a single robotic attacher (e.g., robotic attacher 150, described below with regard to FIG. 3) may be shared by the pair of milking boxes 120, which may reduce to cost of installing multiple milking boxes 120 in the enclosure 100.
In certain embodiments, feed bowl 130 may dispense feed in order to attract a dairy cow so that the dairy cow will enter milking box 120 voluntarily. Accordingly, at least one of the entry gates 126 a may remain open when there is no dairy cow present to allow a dairy cow to enter. Once the dairy cow has entered milking box 120, presence sensor 132 may detect the presence of the dairy cow. For example, presence sensor 132 may detect when the dairy cow has passed through the entrance gate 126 a and/or when the dairy cow is generally centered in the stall portion 122. Upon detecting the presence of the dairy cow, presence sensor 132 may send a signal to controller 200. In response to the signal, controller 200 may cause one or more actuators to close gates 126. Gate sensor 134 may determine when gates 126 have closed. Gate sensor 134 may communicate a signal to controller 200 upon determining that gates 126 have closed. Controller 200 may initiate a milking procedure in response to the signal.
Controller 200 may use the information retrieved according to the identity of the dairy cow to determine how the particular dairy cow should be handled. If the information indicates the dairy cow should not be milked, controller 200 may cause an actuator to open one or more of the exit gates 126 b. For example, if controller 200 determines that the dairy cow should be sorted into a particular region 110 of enclosure 100, such as a veterinary pen, it may cause the exit gate 126 b that accesses the selected region 110 to open. Alternatively, controller 200 may cause multiple exit gates 126 b to open if the dairy cow is to be given the option of which region 110 to occupy upon exiting milking box 120. In certain embodiments, a prod may be used to encourage the dairy cow to exit. Examples of prods include a noise, a mechanical device, or a mild electric shock.
In certain embodiments, rear wall 124 c of stall portion 122 includes a backplane 138. Backplane 138 may comprise any suitable configuration of materials suitable for locating the rear of the dairy cow in order to facilitate the efficient attachment of the milking equipment. For example, backplane 138 may comprise a tracker operable to track a displacement of the dairy livestock in a certain direction. Backplane 138 may also comprise an encoder communicatively coupled to the tracker and operable to determine the distance traveled by the tracker. In certain embodiments, the dairy cow may be backed toward backplane 138 by moving feed bowl 130 as described above. In certain other embodiments, backplane 138 may be moved forward toward the dairy cow. In certain other embodiments, a combination of backing the dairy cow toward backplane 138 and moving backplane 138 forward toward the dairy cow may be used. It may be determined that the rear of the dairy cow has been located when a portion of backplane 138, such as a pipe or bracket, touches the rear of the dairy cow at any suitable location, such as approximately mid-flank (i.e., between the udder and the tail). Backplane 138 may additionally include a manure gutter for directing manure toward a side of stall portion 122 (e.g., away from the dairy cow's udder and the milking equipment).
In certain embodiments, equipment portion 128 of milking box 120 may include a robotic attacher 150, one or more preparation cups 166, teat cups 168, pumps 170, receiver jars 172, milk separation containers 174, and/or any other suitable milking equipment. In certain embodiments, robotic attacher 150 may be suspended into equipment portion 128 from a rail 160. Rail 160 may be generally located above the level of the udder of a dairy cow located in stall portion 122 such that the teats of the dairy cow may be accessible to robotic attacher 150 when suspended from rail 160. For example, rail 160 may extend across the top of equipment portion 128 of milking box 120 and may be oriented substantially parallel to rear wall 124 c.
In general, the teats of the dairy cow may be relatively less visible when looking at the dairy cow from the rear and relatively more visible when looking at the dairy cow from the side. Vision system 158 may facilitate locating the teats from a position to the rear of the dairy cow. Vision system 158 may include multiple cameras, such as a first camera 158 a and a second camera 158 b. In certain embodiments, cameras 158 a, 158 b may be coupled to robotic attacher 150 and may be positioned at any suitable location along main arm 152 or supplemental arm 154. As an example, second camera 158 b may be coupled to gripping portion 156 of supplemental arm 154 at a location proximate to the part of gripping portion 156 adapted to hold a teat cup, and first camera 158 a may be coupled to supplemental arm 154 at a location between second camera 158 b and main arm 152.
Generally, vision system 158 may perform at least two operations: locating reference point 178 of the udder of the dairy cow and determining the positions of the teats of the dairy cow. First camera 158 a may be used to determine the reference point of the udder of the dairy cow. Reference point 178 may be a point near the udder of the dairy cow where robotic attacher 150 may move to, or near, in order to perform a particular function. In certain embodiments, first camera 158 a may comprise a three-dimensional camera adapted to generate a first image 176 depicting the rear of the dairy cow, including the hind legs and the udder. Using a three-dimensional camera may facilitate generating a relatively complete image of the rear of the dairy cow within approximately a couple of seconds (e.g., one second), which may be faster than the amount of time it would take for a two-dimensional camera to generate a similar image.
To facilitate the determination of reference point 178, controller 200 may detect the location of the hips, hind legs, and the udder by analyzing first image 176. To do this, controller 200 may find the edges of the dairy livestock. Controller 200 may find the edges of the diary livestock by comparing the depth information of pixels in an image. Once the edges of the dairy livestock are found, using this information, controller 200 may determine reference point 178 near the udder. At any point, controller 200 may determine that erroneous visual data (e.g., a fly in front of first camera 158 a) has been captured in first image 176. In such instances, controller 200 may filter out such erroneous data.
After determining reference point 178, vision system 158 may be used to determine the locations of the teats of the diary cow. For example, controller 200 may instruct robotic attacher 150 to maneuver near reference point 178 to start determining the location of teats of the dairy cow. Controller 200 may determine the location of the teats of the dairy cow by utilizing second camera 158 b. In certain embodiments, second camera 158 b may comprise lens 264 and transmitter 260 (e.g., a laser-emitting device) adapted to generate a second image 180 depicting at least a portion of the udder to facilitate locating the teats. Second camera 158 b may facilitate locating the end of each teat with a relatively high degree of accuracy, such as within a few millimeters. The location of the teat may be used to instruct robotic attacher 150 where to attach the milking equipment. In determining the location of a teat, controller 200 may encounter erroneous visual data captured by second camera 158 b. In such instances, controller 200 may filter out the erroneous data.
In certain embodiments, robotic attacher 150 may further comprise a nozzle 182. Nozzle 182 may be coupled to gripping portion 156. Nozzle 182 may spray disinfectant on the teats of the dairy cow at the end of a milking cycle, that is, after the dairy cow has been milked and the teat cups have been removed. The disinfectant may be sprayed to prevent mastitis or other inflammation or infection. In certain embodiments, gripping portion may be operable to rotate 180° around the x-axis. During milking, second camera 158 b may be generally oriented on top of gripping portion 156, and nozzle 182 may be generally oriented underneath gripping portion 156 (i.e., opposite second camera 158 b). Orienting nozzle 182 underneath gripping portion 156 during milking may prevent milk or other contaminants from accessing nozzle 182. Once the milking has been completed, gripping portion 156 may rotate such that nozzle 182 may be generally oriented on top of gripping portion 156, and second camera 158 b may be generally oriented underneath gripping portion 156. Orienting nozzle 182 on top of gripping portion 156 after milking may facilitate spraying the teats with disinfectant from nozzle 182.
The operation of vision system 158 will now be discussed in more detail. In operation, generally, controller 200 may access a first image 176 generated by first camera 158 a (e.g., from memory module 240) and use first image 176 to determine, using any suitable logic 220, a reference point 178 proximate to the udder, which may then be stored (e.g., in memory module 240). Reference point 178 may be defined relative to certain features of the dairy cow, such as the hind legs and/or the udder. In certain embodiments, reference point 178 point may be center location 712 of FIG. 7, discussed below.
To determine reference point 178, first camera 158 a may begin by generating the first image 176 in response to a signal from controller 200 indicating that the dairy cow is positioned proximate to the milking equipment. As an example, the signal may indicate that the rear of the dairy cow has been detected by the backplane 138 of the milking box 120. In certain embodiments, controller 200 may communicate the signal to first camera 158 a after determining the dairy livestock has settled down. For example, controller 200 may communicate the signal after feed is dropped into feed bowl 130. As another example, controller 200 may communicate the signal to first camera 158 a after identification sensor 136 communicates the identity of the dairy cow to controller 200 and controller 200 determines that the dairy cow may be milked. As a further example, there may be a time buffer after a particular event before controller 200 communicates the signal to first camera 158 a. The time buffer may be after the dairy cow enters milking box 120, after the feed is dropped into feed bowl 130, after the rear of the dairy cow has been detected by backplane 138, after the identification sensor 136 communicates the identity of the dairy cow, or any other suitable event.
First camera 158 a may begin generating the first image 176 from a starting point and may update the first image 176 in real-time as robotic attacher 150 approaches the dairy cow. The starting point may be determined according to a default position of robotic attacher 150 (e.g., a position determined relative to milking stall 122). Thus, the starting point may be determined without the use of historical data 184 associated with the particular dairy cow being milked. First camera 158 a may then generate first image 176, capturing visual data generally depicting the rear of the dairy cow. First camera 158 a may communicate the first image 176 to controller 200, and controller 200 may use the image to locate main features of the dairy cow, such as the right hind leg, the left hind leg, the udder, and/or the tail.
More specifically, controller 200 may use first image 176 to determine reference point 178 based on the location of the main features of the dairy cow. Reference point 178 may be defined relative to certain features of the dairy cow, such as the hind legs and/or the udder. As an example, reference point 178 may be defined between the hind legs and/or below the udder. In certain embodiments, the reference point 178 may be located proximate to a mid-point of the udder. The mid-point of the udder may refer to a point generally located between the front teats and the rear teats in the x-direction and/or between the left teats and the right teats in the z-direction. In certain embodiments, the mid-point of the udder may be estimated prior to determining the precise location of the teats, for example, according to the general size and location of the udder. Reference point 178 may be spaced apart from the dairy cow in the y-direction to minimize the likelihood that second camera 158 b touches the dairy cow. For example, reference point 178 may be located a few inches below the mid-point of the udder. In certain embodiments, reference point 178 may be center location 712, discussed further below.
In particular, controller 200 may begin to determine reference point 178 by locating hip location 704 a of FIG. 7. Controller 200 may do this by comparing the depth locations of pixels of an upper outer area of first image 176, or any other area of first image 176 likely to include the hip of the dairy cow. For example, controller 200 may access first image 176 generated by first camera 158 a. Controller 200 may compare the pixels of first image 176 by determining the depth of the pixels. The depth of the pixels may be a distance in the x-dimension (as illustrated in FIGS. 3, 4A, and 4B), between first camera 158 a and a particular object. In certain embodiments, the depth may be determined by measuring the time of flight of a light signal between first camera 158 a and a particular object captured in first image 176 in the x-dimension.
By comparing the depth locations of various pixels to each other, controller 200 may attempt to locate particular edges of the dairy livestock. For example, controller 200 may compare the depth information of a group of pixels to determine if a portion of the pixels are closer than other portions of pixels. A cluster of pixels closer to first camera 158 a may signify that an edge of a dairy livestock has been found. The cluster of pixels with depth information further away from camera 158 a may signify that the image data is of an object other than an edge of the dairy livestock. Controller 200 may associate this location of the cluster of pixels that are closer to first camera 158 a with an edge of the dairy livestock. For example, controller 200 may have determined that the cluster of pixels represents a first edge corresponding to the hip of the dairy livestock. In certain embodiments, this location may correspond with hip location 704 a of FIG. 7. Controller 200 may store the association between the determined location and hip location 704 a in memory 240 or in any other suitable component of controller 200.
After finding the hip of the dairy livestock, controller 200 may attempt to locate the hind leg of the dairy livestock. Generally, controller 200 may begin to locate the hind leg of the dairy livestock by analyzing visual data in a downward direction from hip location 704 a in an attempt to determine outer hind location 706 a of FIG. 7. To do this, controller 200 may compare the depth information of pixels in a lower outer area of first image 176, or any other area of first image 176 likely to include visual data of the hind leg of the dairy livestock.
For example, controller 200 may traverse pixels of first image 176 in a downward direction in order to locate the outer edge of a hind leg of the dairy livestock. In certain embodiments, controller 200 may traverse pixels of first image 176 in a downward direction from hip location 704 a to determine outer hind location 706 a of FIG. 7. At any point, controller 200 may filter data as discussed further below. Controller 200 may determine whether some pixels are closer, to first camera 158 a, than other pixels signifying an edge of a hind leg has been found. Controller 200 may associate the location of the cluster of pixels that are closer to first camera 158 a with an edge of the dairy livestock. For example, controller 200 may have determined that the cluster of pixels represents an edge corresponding to an outer edge of a hind leg of the dairy livestock. In certain embodiments, this location may correspond with outer edge location 706 a of FIG. 7. Controller 200 may store the association between the determined location and outer edge location 706 a in memory 240 or in any other suitable component of controller 200.
Controller 200 may then search for an inner edge of the hind leg of the dairy livestock. For example, controller 200 may attempt to determine inner hind leg location 708 a of FIG. 7. To do this, controller 200 may begin to scan the depth information of pixels along a lower inner area of first image 176, or any other portion of first image 176 likely to include visual data of the inner hind leg of the dairy livestock.
For example, controller 200 may traverse pixels along the z-dimension (as illustrated in FIGS. 3, 4A, and 4B) from outer edge location 706 a to the center of first image 176 trying to locate an inner edge of the hind leg of the dairy livestock. According to some embodiments, controller 200 may filter image data as described further below. Controller 200 may determine whether some pixels are closer than other pixels signifying an inner edge of the hind leg has been found. Controller 200 may associate the location of the cluster of pixels that are closer to first camera 158 a with an edge of the dairy livestock. For example, controller 200 may have determined that the cluster of pixels represents an edge corresponding to an inner edge of a hind leg of the dairy livestock. In certain embodiments, this location may correspond with inner edge location 708 a of FIG. 7. Controller 200 may store the association between the determined location and inner edge location 708 a in memory 240 or in any other suitable component of controller 200.
After locating the inner edge of the hind leg, controller 200 may search for the location of the udder of the dairy livestock. Controller 200 may begin to scan the depth information of pixels along an upper area of first image 176, or any other portion of first image 176 likely to include the udder of the dairy livestock. For example, controller 200 may scan pixels along a vertical dimension above the location of the inner edge (e.g., inner edge location 708 a of FIG. 7), trying to locate an edge of the udder of the dairy livestock. In certain embodiments, this edge may be where the udder of the livestock meets an inner edge of a hind leg of the dairy livestock. According to some embodiments, controller 200 may filter visual data as discussed further below.
Controller 200 may determine whether some pixels are closer than other pixels signifying an edge of the dairy livestock has been found. For example, controller 200 may compare the depth information of a group of pixels to determine if a portion of the pixels are closer than other portions of pixels. A cluster of pixels closer to first camera 158 a than other clusters may signify an edge has been found. If the edge is substantially vertical (e.g., edge 702 b of FIG. 7), then controller 200 may be analyzing an inner edge of the hind leg. Controller 200 may continue traversing first image 178 until the location of the udder is found. This location may be determined where the edges in depth transition from being substantially vertical, indicating the inside of the hind legs, to substantially horizontal, indicating the udder. Once the edges in depth detected by controller 200 transition to being substantially horizontal, controller 200 may then associate the location with an edge of the dairy livestock. For example, controller 200 may have determined that the cluster of pixels represents an edge in depth corresponding to an udder edge of the dairy livestock where the udder meets the hind leg. In certain embodiments, this location may correspond with udder edge location 710 a of FIG. 7. Controller 200 may store the association between the determined location and udder edge location 710 a in memory 240 or in any other suitable component of controller 200.
After finding the edges corresponding to a side of the dairy livestock, controller 200 may determine if data points from both sides of the dairy livestock have been collected. In certain embodiments, this determination may be based on whether controller 200 has enough data points to calculate a center location of the udder of the dairy livestock. For example, controller 200 may use at least two locations of the udder to calculate the center of the udder (e.g., center location 712 of FIG. 7), where each location identifies where the udder intersects with each hind leg (e.g., udder edges 710). If controller 200 determines that only a single udder edge 710 has been found, controller 200 may proceed to determine the locations of the other hind leg and the other udder edge 710 of the dairy livestock. For example, controller 200 may determine hip location 704 b, outer hind location 706 b, inner hind location 708 b, and udder edge 710 b of FIG. 7.
Once controller 200 has found a number of locations of edges of the dairy livestock, controller 200 may calculate a center location of the udder. For example, controller 200 may calculate center location 712 of FIG. 7 based on the acquired locations discussed above. According to some embodiments, center location 712 may correspond to reference point 178. In certain embodiments, the center location may be determined by calculating a coordinate that is approximately equidistant from each determined udder edge. For example, location 712 of FIG. 7 may be calculated by finding the center point between udder edge locations 710 a and 710 b of FIG. 7. Controller 200 may also determine the depth location of the center of the udder. In certain embodiments, controller 200 may determine the depth location by analyzing visual data captured by first camera 158 a. In other embodiments, the depth location of the center of the udder may be calculated by using historical data 184 of the udder's location in relation to another portion of the dairy livestock (e.g., the rear of the dairy livestock) as well as a displacement measurement of the dairy livestock within a particular stall. The displacement measurement may be obtained using backplane 138.
At any point in determining reference point 178, controller 200 may filter particular visual data deemed undesirable. Generally, depth information analyzed from first image 176 should stay fairly constant. This signifies that the same object is being analyzed. However, controller 200 may determine that undesirable visual data has been captured by first camera 158 a in first image 176. Examples of undesired data captured by first camera 158 a may be a fly, a livestock's tail, dirt, fog, moisture, a reflection off of a metal post in enclosure 100, or any other object that may interfere with controller 200 analyzing first image 176. Controller 200 may make this determination by determining whether some pixels exceed a distance threshold. For example, controller 200 may determine that one or more pixels are too close to first camera 158 a. Pixels that are too close to first camera 158 a may suggest undesired data has been captured by first camera 158 a. As another example, controller 200 may determine that the measured depths of adjacent pixels are fluctuating, exceeding a certain threshold. As a further example, controller 200 may determine that measured depths of adjacent pixels are changing excessively, exceeding a certain threshold. Any of these examples may signify undesirable visual data.
If controller 200 has determined that some pixels exceed a distance threshold and/or have depth information signifying certain pixels represent undesirable visual data captured by first camera 158 a, then controller 200 may filter that particular visual data. Thus, controller 200 may determine that a certain set of pixels are too close to or too far from camera 158 a and may eliminate those pixels from consideration when analyzing first image 176. Or controller 200 may have determined that certain adjacent pixels contained depth information that fluctuated beyond a threshold. As another example, controller 200 may have determined that certain adjacent pixels contained depth information that changed excessively from pixel to pixel. All of these examples may be examples of data potentially filtered by controller 200 when analyzing first image 176.
Once controller 200 has determined reference point 178 (e.g., center location 712 of FIG. 7), controller 200 may facilitate the scanning of teats of the dairy livestock. Controller 200 may begin by facilitating the positioning of robotic attacher 150 such that the teats may be scanned by second camera 158 b. For example, controller 200 may communicate reference point 178 and/or information describing the main features of the dairy cow to robotic attacher 150. The reference point 178 may be used to position second camera 158 b. The information describing the main features of the dairy cow may be used to prevent robotic attacher 150 from colliding with the dairy cow when navigating second camera 158 b toward reference point 178. Information describing the main features of the dairy cow may include the position of the hind legs, the space between the hind legs, the position of the udder, the height of the udder, the position of the tail, and/or other information. Once robotic attacher 150 has positioned second camera 158 b relative to the reference point 178, second camera 158 b may begin scanning the udder.
Controller 200 may send a signal to robotic attacher 150 causing robotic attacher 150 to position second camera 158 b relative to the reference point 178. Accordingly, second camera 158 b may have a consistent point of reference from one milking cycle to the next, which may allow the teats to be located efficiently. Controller 200 may access a second image 180 generated by second camera 158 b (e.g., from memory module 240) in order to determine, using any suitable logic 220, a location of a teat.
In certain embodiments, second camera 158 b may determine where to look for one or more of the teats according to historical data 184. Historical data 184 may be received from controller 200 and may describe a previously-determined location of the teats relative to the reference point 178. The previously-determined location may be based on the location of the teats during one or more previous milking cycles. As an example, the previously-determined location may comprise the location of the teats during the most recent milking cycle. As another example, the previously-determined location may comprise an average of the locations of the teats during a number of previous milking cycles. As another example, the previously-determined location may comprise the location of the teats during a previous milking cycle in which the udder was likely to be as full of milk as the current milking cycle. For example, if eight hours have elapsed since the dairy cow was last milked, the previously-determined location may be determined from a previous milking cycle in which the dairy cow had not been milked for approximately eight hours. Referring to historical data 184 may minimize the area that second camera 158 b may scan in order to locate the teat and may reduce the amount of time required to locate the teat.
Second camera 158 b may communicate the second image 180 to controller 200, and controller 200 may access the second image 180 to locate the teats of the dairy cow. As described below in FIG. 4B, in certain embodiments, second camera 158 b may comprise lens 264 and transmitter 260, such as a horizontal laser-emitting device. If the horizontal laser scans a portion of the udder other than the teats (e.g., a relatively even surface of the udder), the scan communicated to controller 200 may generally resemble a substantially solid line. If the horizontal laser scans a portion of the udder that includes the teats, the scan communicated to controller 200 may generally resemble a broken line depicting the teats and the spaces between the teats. As an example, controller 200 may determine that a teat has been located if the scan comprises a broken line in which a solid portion of the line generally corresponds to the width of a teat and the broken portions of the line generally correspond to the proportions of the space between teats.
Using this information, controller 200 may calculate reference coordinates for particular teats of the dairy livestock. Controller 200 may use reference coordinates to position robotic attacher 150 in the vicinity of a particular teat in order to subsequently determine a more accurate location of the particular teat using second camera 158 b.
Using the reference coordinates, second camera 158 b may be positioned near the teats of the dairy livestock. Robotic attacher 150 may move into position to scan the udder for teats. Robotic attacher 150 may move to the calculated reference coordinates. In certain embodiments, the reference coordinates may be slightly offset to avoid collision with one or more of the teats of the dairy livestock. According to some embodiments, robotic attacher 150 may move into position to allow second camera 158 b to determine current coordinates of a particular teat of the dairy livestock. For example, the coordinates of the particular teat may correspond to coordinates in the x-, y-, and z-dimensions.
Controller 200 may begin to scan for the tip of a particular teat by utilizing second camera 158 b. In certain embodiments, second camera 158 b may generate second image 180 using lens 264 and transmitter 260 described in FIG. 4B below. Second image 180 may comprise data signifying the light intensity measurements of particular portions of the visual data captured by second image 180. Controller 200 may then scan second image 180 generated by second camera 158 b to locate a first teat. In certain embodiments, analyzing second image 180 may include analyzing light intensity measurements captured by second camera 158 b.
Controller 200 may calculate a first coordinate of the tip of a particular teat by analyzing second image 180. In certain embodiments, the first coordinate may be a coordinate in the z-dimension (as depicted in FIGS. 3, 4A, and 4B) of the dairy livestock. Controller 200 may begin to calculate the first coordinate of the teat of the dairy livestock using the data captured by second camera 158 b. Controller 200 may begin to analyze second image 180 generated by second camera 158 b in a vertical dimension relative to the dairy livestock. The light intensity measurements of a particular teat should appear in clusters of similar measurements. As the scan proceeds in a downward vertical direction and the light intensity measurements have been determined to deviate from the measurements of the teat, controller 200 may determine that the tip of the teat has been found and the coordinates of the particular teat may be calculated. In certain embodiments, controller 200 may determine the first coordinate based on one or more measurements of a collection of horizontal lines included in second image 180.
Controller 200 may also calculate a third coordinate of the particular teat. For example, the third coordinate may signify the distance between second camera 158 b and the tip of the particular teat in an x-dimension (as depicted in FIGS. 3, 4A, and 4B). In certain embodiments, controller 200 may calculate the third coordinate of the tip of the particular teat based at least in part on the calculated second coordinate and the known angle θ1 between signal 262 of transmitter 260 and supplemental arm 154 relative to the x-dimension as depicted in FIG. 4B. Using the angle information (e.g., θ1), the second coordinate (or any other distance calculation), and a standard geometry equation based on the properties of triangles, controller 200 may calculate the third coordinate of the tip of the particular teat of the dairy livestock.
Controller 200 may also calculate the distance between the center of teat cup 168 and the tip of the teat based on the calculation of the third coordinate and the known distance between second camera 158 b and teat cup 168. Finally, controller 200 may determine if there are any other teats for which the coordinates must be calculated. If there are other teats that remain for which coordinates need to be calculated, the process may repeat. The vision-based determination process described above facilitates the movement of robotic attacher 150 allowing for the proper attachment of teat cups 168 to teats of a dairy livestock, disinfection of teats by nozzle 182, or any other suitable action by robotic attacher 150. Furthermore, controller 200 is operable to detect a movement of the dairy livestock. In response to detecting the movement, controller 200 may re-calculate any coordinate previously calculated using first camera 158 a and/or second camera 158 b.
At any point in determining the location of teats, controller 200 may filter undesirable visual data. Controller 200 may detect undesirable visual data by determining whether any light intensity measurements exceed a particular threshold. For example, controller 200 may scan second image 180 searching for light intensity measurements that vary greatly in intensity from neighboring pixels. Controller 200 may also determine that the distance between particular pixels with similar light intensity measurements may be spaced too far apart. In these examples, light intensity measurements exceeding certain thresholds may signify objects other than the teats of a dairy livestock such as hair, dirt, fog, or a fly. In certain embodiments, controller 200 may instruct second camera 158 b to generate two images. One image may be generated using the laser turned on and the other image may be generated while the laser is turned off. Using the light intensity measurements from both of these generated images, controller 200 may determine an ambient light measurement which will be taken into account when calculating the light intensity measurements of second image 180. If any light intensity measurements exceed a certain threshold, then controller 200 may filter such data. Such data may be determined to have captured an object that may lead to an erroneous calculation for the coordinates of a particular teat of the dairy livestock. For example, when calculating the coordinates of a particular teat, controller 200 may ignore filtered data in its calculations.
FIG. 4B illustrate an example of a side plan view of second camera 158 b according to certain embodiments of the present disclosure. In certain embodiments, second camera 158 b includes transmitter 260 that transmits signal 262 and lens 264 that receives a reflection of signal 262. Lens 264 may provide the reflection of signal 262 to image processing components operable to generate second image 180. In some embodiments, signal 262 comprises a two-dimensional laser signal. According to some embodiments, transmitter 264 may be a laser-emitting device. Transmitter 264 may transmit signal 262 as a horizontal plane oriented at a fixed angle θ1 relative to the x-axis of supplemental arm 154. For example, when second camera 158 b is positioned in an upright orientation, angle θ1 may be configured at an upward angle between 5 and 35 degrees relative to the x-axis.
Coordinate data 600 may also contain teat coordinates 604. Teat coordinates 604 may be historical coordinates for particular teats of a dairy livestock. For example, teat coordinates 604 a-d each represent example coordinates for a particular one teat of a dairy livestock. In certain embodiments, each coordinate of teat coordinates 604 may represent the distance from the center of the udder of the dairy livestock in a particular dimension. Teat coordinates 604 may be in any suitable format and in any suitable measurement unit usable by controller 200 to calculate coordinates in real-time or for any other particular purpose. In the illustrated example, each record 606 contains a set of three coordinates for each teat in teat coordinates 604. Teat coordinates 604 may be coordinates in any suitable dimension. For example, the coordinates may represent the location of a particular teat in the x-, y-, and z-dimensions. In certain embodiments, teat coordinates 604 may correspond to coordinates in the left-right dimension, head-to-tail dimension, and the up-down dimension. In the illustrated example, record 606 a may contain teat coordinates 604 a of (10, 12, 5), teat coordinates 604 b of (−11, 10, 4), teat coordinates 604 c of (−8, −13, 6), and teat coordinates 604 d of (−12, 11, 5). Record 606 b may contain teat coordinates 604 a of (9, 10, 6), teat coordinates 604 b of (−13, 8, 5), teat coordinates 604 c of (−7, −12, 5), and teat coordinates 604 d of (−10, 10, 6). Record 606 c may contain teat coordinates 604 a of (10, 8, 7), teat coordinates 604 b of (−12, 9, 5), teat coordinates 604 c of (−9, −10, 6), and teat coordinates 604 d of (−9, 12, 6).
FIG. 7 illustrates an example snapshot 700 of first image 176 identifying various portions of a dairy livestock. Example snapshot 700 may include located edges 702 corresponding to the edges of the hind legs of a dairy livestock. Example snapshot 700 may also include hip locations 704, outer hind locations 706, inner hind locations 708, udder edges 710, and center udder location 712. Controller 200 may be operable to determine located edges 702 from snapshot 700 as described above. For example, located edge 702 a may correspond to an outer edge of a first hind leg of a dairy livestock. Located edge 702 b may correspond to an inner edge of the first hind leg of the dairy livestock. Located edge 702 c may correspond to an outer edge of a second hind leg of the dairy livestock. Located edge 702 d may correspond to an inner edge of the second hind leg.
Controller 200 may be operable to determine various locations in the vicinity of the hind legs as discussed previously. For example, controller 200 may be operable to determine hip locations 704 of the dairy livestock. Hip location 704 a may correspond to a located first hip of the diary livestock and hip location 704 b may correspond to a located second hip of the dairy livestock. After determining hip location 704, controller 200 may be further operable to determine outer hind locations 706. For example, 706 a may correspond to a located outer hind edge of a first hind leg of the dairy livestock and 706 b may correspond to a located outer hind edge of a second hind leg of the dairy livestock. Controller 200 may also determine inner hind leg locations 708. For example, inner hind leg location 708 a may correspond to a located inner hind edge of the first hind leg and 708 b may correspond to a located inner hind edge of the second hind leg.
Controller 200 may be further operable to determine a position of the udder of the dairy livestock. In certain embodiments, controller 200 may determine the position of the udder of the dairy livestock based on the accessed first image 176 and/or the determined positions of the hind legs of the dairy livestock. For example, controller 200 may process first image 176 (which may change as vision system 158 moves toward the dairy livestock, as described above) in order to trace the located edges in depth corresponding to the inside of the hind legs of the dairy livestock (e.g., inner hind locations 708) upwardly until they intersect with the udder of the dairy livestock at udder edges 710. In certain embodiments, controller 200 may process first image 176 to determine where the edges in depth transition from being substantially vertical, indicating the inside of the hind legs, to substantially horizontal, indicating the udder. This location may correspond to udder edge 710. For example, udder edge 710 a may correspond to the edge of the udder near one hind leg, while udder 710 b may correspond to the edge of the udder near the other hind leg. Additionally, controller 200 may use udder edges 710 a and 710 b to calculate center udder location 712. In certain embodiments, center udder location 712 may be a location on the udder in the middle of udder edges 710 a and 710 b.
Controller 200, having determined the positions of each of the hind legs of the dairy livestock and the udder, may then communicate signals to one or more of actuators that may facilitate movement of robotic attacher 150 such that at least a portion of robotic attacher 150 (e.g., supplemental arm 154) extends toward the space between the hind legs of the dairy livestock (e.g., at a predetermined height relative to the milking stall in which the dairy livestock is located). Because first image 176 may comprise a three-dimensional video image, first image 176 may change in real time as first camera 158 a moves toward the dairy livestock. Accordingly, the present disclosure contemplates that controller 200 may update, either continuously or at predetermined intervals, the determined leg positions as first image 176 changes.
FIG. 8 illustrates an example dairy livestock that may be milked by the system of the present disclosure. Dairy livestock 800 includes udder center 802 and teat tips 804. Udder center 802 may be any location that generally may be considered the center of the udder of dairy livestock 800. In certain embodiments, udder center 802 may be determined by controller 200 using first camera 158 a. According to some embodiments, udder center 802 may be reference point 178 or center udder location 712. Dairy livestock 800 also includes teat tips 804. In the illustrated example, dairy livestock includes teat tips 804 a-d. In certain embodiments, the coordinates of teat tips 804 a-d may be determined by controller 200 using second camera 158 b. In some embodiments, the coordinates of teat tips 804 a-d may be stored as historical data 184 in memory 240 as described in FIG. 4A above. According to some embodiments, teat tips 804 a-d may be drawn into teat cup 168 to facilitate milking of dairy livestock 800.
Second axis 904 may be any unit of measurement that may specify a distance in a particular dimension. For example, second axis 904 may represent the distance from first camera 158 a to an object depicted in a particular portion, such as a pixel, of first image 176. Data points 906 may represent the distance of a particular portion of first image 176 in a particular dimension. For example, a data point 906 may signify the distance of a particular pixel from first camera 158 a. Threshold band 908 may be any threshold that can be used by controller 200 to filter particular data. For example, controller 200 may filter data that is outside of threshold band 908, i.e., is too far or too close to first camera 158 a. Controller 200 may determine that a cluster of pixels within threshold band 908 are part of the same object and pixels adjacent to that cluster that may fall outside of threshold band 908 may be part of a different object. This may signify that an edge of an object has been found by controller 200.
The example method may begin at step 1100. At step 1100, controller 200 may begin to compare pixels of an upper outer area of an image. For example, controller 200 may access first image 176 generated by first camera 158 a. Controller 200 may compare the pixels of first image 176 by determining the depth of the pixels. In certain embodiments, the depth may be determined by measuring the time of flight of a light signal between first camera 158 a and a particular object captured in first image 176. After collecting the depth information of a particular portion of pixels, the method may proceed to step 1101. At step 1101, controller 200 may determine whether some pixels exceed a distance threshold. Generally, depth information analyzed from first image 176 should stay fairly constant signifying that a particular object is being analyzed. However, controller 200 may determine that one or more pixels are too close to first camera 158 a. Pixels that are too close to first camera 158 a may suggest undesirable data has been captured by first camera 158 a. Examples of undesirable data captured by first camera 158 a may be a fly, a livestock's tail, dirt, fog, moisture, a reflection off a metal post in enclosure 100, or any other object that may interfere with controller 200 analyzing first image 176. As another example, controller 200 may determine that the measured depths of adjacent pixels are fluctuating, exceeding a certain threshold. As a further example, controller 200 may determine that measured depths of adjacent pixels are changing excessively, exceeding a certain threshold. If controller 200 has determined that some pixels do exceed a distance threshold and have depth information signifying certain pixels represent undesirable visual data captured by first camera 158 a, then the example method may proceed to step 1102. Otherwise, the example method may proceed to step 1104.
Once it is determined that certain visual data exceeds a distance threshold, that data may be filtered. At step 1102, controller 200 may filter pixels containing depth information that exceeds a certain distance threshold. For example, controller 200 may determine that a certain set of pixels are too close to or too far from camera 158 a and will eliminate those pixels from consideration when analyzing first image 176. Or controller 200 may have determined that certain adjacent pixels contained depth information that fluctuated. As another example, controller 200 may have determined that certain adjacent pixels contained depth information that changed excessively from pixel to pixel. All of these examples may be examples of data potentially filtered by controller 200.
Controller 200 may next attempt to locate particular edges of the dairy livestock by comparing the depth locations of various pixels to each other at step 1104. Controller 200 may determine whether some pixels are closer than other pixels. For example, controller 200 may compare the depth information of a group of pixels to determine if a portion of the pixels are closer than other portions of pixels. A cluster of pixels closer to first camera 158 a may signify that an edge of a dairy livestock has been found. The cluster of pixels with depth information further away from camera 158 a may signify that the image data is of an object other than an edge of the dairy livestock. If controller 200 has determined that some pixels are not closer than other pixels, then the example method may return to step 1100 and continue analyzing information captured by first camera 158 a. Otherwise, the example method may proceed to step 1108.
At step 1108, controller 200 may associate the location of the cluster of pixels that are closer to first camera 158 a with an edge of the dairy livestock. For example, controller 200 may have determined that the cluster of pixels represents a first edge corresponding to the hip of the dairy livestock. In certain embodiments, this location may correspond with hip location 704 a of FIG. 7. Controller 200 may store this association in memory 240 or in any other suitable component of controller 200.
Controller 200 may then proceed with determining the location of an outer edge of a hind leg at step 1116. Controller 200 may do this by determining whether some pixels are closer than other pixels. For example, controller 200 may compare the depth information of a group of pixels to determine if a portion of the pixels are closer than other portions of pixels. A cluster of pixels closer to first camera 158 a may signify that an edge of a dairy livestock has been found. The cluster of pixels with depth information further away from camera 158 a may signify that the image data is of an object other than an edge of the dairy livestock. If controller 200 has determined that some pixels are not closer than other pixels, then the example method may return to step 1112 and continue analyzing information captured by first camera 158 a. Otherwise, the example method may proceed to step 1120.
At step 1120, controller 200 may associate the location of the cluster of pixels that are closer to first camera 158 a with an edge of the dairy livestock. For example, controller 200 may have determined that the cluster of pixels represents an edge corresponding to an outer edge of a hind leg of the dairy livestock. In certain embodiments, this location may correspond with outer edge location 706 a of FIG. 7. Controller 200 may store this association in memory 240 or in any other suitable component of controller 200.
Controller 200 may then attempt to determine an inner edge location of a hind leg. At step 1124, controller 200 may begin to scan the depth information of pixels along a lower inner area of first image 176. For example, controller 200 may traverse pixels along the z-dimension (as illustrated in FIGS. 3, 4A, and 4B) from outer edge location 706 a to the center of first image 176 trying to locate an inner edge of the hind leg of the dairy livestock. At step 1125, controller 200 may determine whether some pixels exceed a distance threshold. Controller 200 may make this determination similar to the determination in step 1101. If controller 200 has determined that some pixels exceed a distance threshold, then the example method may proceed to step 1126. Otherwise, the example method may proceed to step 1128. At step 1126, controller 200 may filter pixels containing depth information that exceed a certain distance threshold. Controller 200 may filter pixels as discussed in step 1102.
Controller 200 may then proceed with determining the location of an inner edge of a hind leg at step 1128. Controller 200 may determine whether some pixels are closer than other pixels. For example, controller 200 may compare the depth information of a group of pixels to determine if a portion of the pixels are closer than other portions of pixels. A cluster of pixels closer to first camera 158 a may signify that an edge of the dairy livestock has been found. The cluster of pixels with depth information further away from camera 158 a may signify that the image data is of an object other than an edge of the dairy livestock. If controller 200 has determined that some pixels are not closer than other pixels, then the example method may return to step 1124 and continue analyzing information captured by first camera 158 a. Otherwise, the example method may proceed to step 1132.
At step 1132, controller 200 may associate the location of the cluster of pixels that are closer to first camera 158 a with an edge of the dairy livestock. For example, controller 200 may have determined that the cluster of pixels represents an edge corresponding to an inner edge of a hind leg of the dairy livestock. In certain embodiments, this location may correspond with inner edge location 708 a of FIG. 7. Controller 200 may store this association in memory 240 or in any other suitable component of controller 200.
Continuing to determine the location of the udder edge, at step 1140, controller 200 may determine whether the edges in depth of first image 178 have transitioned from being substantially vertical to substantially horizontal. For example, controller 200 may compare the depth information of a group of pixels to determine if a portion of the pixels are closer than other portions of pixels. A cluster of pixels closer to first camera 158 a than other clusters may signify that an edge has been found. If the located edge is substantially vertical, the edge of the udder has not been found and the example method may return to step 1136 and controller 200 may continue to scan information captured by first camera 158 a. If controller 200 has determined that the located edge has is substantially horizontal, an edge of the udder may have been found. This location may signify where the edges in depth transition from being substantially vertical, indicating the inside of the hind legs, to substantially horizontal, indicating the udder. The example method may proceed to step 1144.
At step 1144, controller 200 may associate the location of the cluster of pixels where pixels are no longer substantially closer to first camera 158 a than other pixels with an edge of the dairy livestock. For example, controller 200 may have determined that the cluster of pixels represents an edge corresponding to an udder edge of the dairy livestock where the udder meets the hind leg. In certain embodiments, this location may correspond with udder edge location 710 a of FIG. 7. Controller 200 may store this association in memory 240 or in any other suitable component of controller 200.
After determining edge locations for both sides of the dairy livestock, at step 1152, controller 200 may calculate a center location of the udder. For example, controller 200 may calculate center location 712 of FIG. 7 based on the acquired locations in the prior steps. In certain embodiments, the center location may be determined by calculating a coordinate that is approximately equidistant from each determined udder edge. For example, location 712 of FIG. 7 may be calculated by finding the center point between udder edge locations 710 a and 710 b of FIG. 7. Finally, at step 1156, controller 200 may determine the depth location of the center of the udder. In certain embodiments, controller 200 may determine the depth location by analyzing visual data captured by first camera 158 a. In other embodiments, the depth location of the center of the udder may be calculated by using historical data 184 of the udder's location in relation to another portion of the dairy livestock, as well as a displacement measurement of the dairy livestock within a particular stall.
The example method may begin at step 1198. At step 1198, gripping portion 156 may grip teat cup 168 and be positioned near the rear of the dairy livestock. At step 1200, stored coordinates signifying the location of teats may be received. For example, controller 200 of FIG. 3 may access a set of historical coordinates (e.g., historical data 184) signifying the location of teats of a dairy livestock in relation to some location on the dairy livestock, such as the center of the udder, the rear, and/or reference point 178. In certain embodiments, the center of the udder may be reference point 178. At step 1204, controller 200 may receive coordinates of a center of the udder of the dairy livestock. In certain embodiments, the coordinates for the center of the udder of the dairy livestock may be received after analyzing first image 176 generated by first camera 158 a. The example method of FIG. 11 may be one method for determining the center of the udder of a dairy livestock in real-time.
Once reference coordinates for a particular teat are determined, steps may be taken to prepare robotic attacher 150 for attaching teat cup 168 to the particular teat. At step 1224, using the reference coordinates calculated, second camera 158 b may be positioned near the teats of the dairy livestock. Robotic attacher 150 may move into position to scan the udder for teats by moving to the calculated reference coordinates. In certain embodiments, the reference coordinates may be slightly offset to avoid collision with one or more of the teats of the dairy livestock. According to some embodiments, robotic attacher 150 may move into position to allow second camera 158 b to determine current coordinates of a particular teat of the dairy livestock. For example, the coordinates of the particular teat may correspond to coordinates in the x-, y-, and z-dimensions.
Once in position, controller 200 may start to scan the udder for a particular teat. At step 1228, controller 200 may begin by scanning for the tip of a particular teat using second camera 158 b. In certain embodiments, second camera 158 b may generate second image 180 using lens 264 and transmitter 260. Second image 180 may comprise data signifying the light intensity measurements of particular portions of the visual data captured by second image 180. Controller 200 may then analyze second image 180 generated by second camera 158 b to locate a first teat. In certain embodiments, analyzing second image 180 may include analyzing light intensity measurements captured by second camera 158 b.
In certain embodiments, controller 200 may instruct second camera 158 b to generate two images. One image will be generated using the laser turned on and the other image will be generated while the laser is turned off. Using the light intensity measurements from both of these generated images, controller 200 may determine an ambient light measurement which will be taken into account when calculating the light intensity measurements of second image 180. If any light intensity measurements exceed a certain threshold, then the example method may proceed to step 1236. Otherwise, the example method may proceed to step 1240. At step 1236, controller 200 may filter data that is determined to exceed a certain threshold. Such data may be determined to have captured an object that may lead to an erroneous calculation for the coordinates of a particular teat of the dairy livestock. For example, when calculating the coordinates of a particular teat, controller 200 may ignore filtered data in its calculations.
After scanning the udder for a teat has been initiated, controller 200 may begin to calculate the actual coordinates of a particular teat location. At step 1240, controller 200 may calculate a first coordinate of the tip of a particular teat. In certain embodiments, the first coordinate may be a coordinate in the z-dimension (as depicted in FIGS. 3, 4A, and 4B) of the dairy livestock. Controller 200 may begin to calculate the first coordinate of the teat of the dairy livestock using the data captured by second camera 158 b. Controller 200 may begin to analyze second image 180 generated by second camera 158 b in a vertical dimension relative to the dairy livestock. The light intensity measurements of a particular teat should appear in clusters of similar measurements. As the scan proceeds in a downward vertical direction and the light intensity measurements have been determined to deviate from the measurements of the teat, controller 200 may determine that the tip of the teat has been found and the coordinates of the particular teat may be calculated. In certain embodiments, controller 200 may determine the first coordinate based on one or more measurements of a collection of horizontal lines included in second image 180.
At step 1248, controller 200 may calculate a third coordinate of the particular teat. For example, the third coordinate may signify the distance between second camera 158 b and the tip of the particular teat in an x-dimension (as depicted in FIGS. 3, 4A, and 4B). In certain embodiments, controller 200 may calculate the third coordinate of the tip of the particular teat based at least in part on the calculated second coordinate and the known angle θ1 between signal 262 of transmitter 260 and supplemental arm 154 relative to the x-dimension as depicted in FIG. 4B. Using the angle information (e.g., θ1), the second coordinate (or any other distance calculation), and a standard geometry equation based on the properties of triangles, controller 200 may calculate the third coordinate of the tip of the particular teat of the dairy livestock. Controller 200 may also calculate the distance between the center of teat cup 168 and the tip of the teat based on the calculation of the third coordinate and the known distance between second camera 158 b and teat cup 168.
a robotic attacher configured to position milking equipment relative to the dairy livestock based at least in part upon the light intensity image, excluding the discarded pixel.
the processor discards the first pixel if the difference between the first light intensity measurement and the light intensity measurements for the plurality of pixels exceeds the threshold.
3. The system of claim 2, wherein the difference between the second light intensity measurement and the light intensity measurements for the plurality of pixels does not exceed the threshold.
the processor discards the second pixel if the difference between the second light intensity measurement and the light intensity measurements for the plurality of pixels exceeds the threshold.
5. The system of claim 4, wherein the difference between the first light intensity measurement and the light intensity measurements for the plurality of pixels does not exceed the threshold.
6. The system of claim 1, further comprising a camera that captures the light intensity image.
7. The system of claim 6, wherein the camera comprises a laser device.
positioning milking equipment relative to the dairy livestock based at least in part upon the light intensity image, excluding the discarded pixel.
discarding the first pixel if the difference between the first light intensity measurement and the light intensity measurements for the plurality of pixels exceeds the threshold.
10. The method of claim 9, wherein the difference between the second light intensity measurement and the light intensity measurements for the plurality of pixels does not exceed the threshold.
discarding the second pixel if the difference between the second light intensity measurement and the light intensity measurements for the plurality of pixels exceeds the threshold.
12. The method of claim 11, wherein the difference between the first light intensity measurement and the light intensity measurements for the plurality of pixels does not exceed the threshold.
14. The system of claim 13, further comprising a camera that captures the light intensity image.
15. The system of claim 14, wherein the camera comprises a laser device.
GB2258382A (en) 1991-06-20 1993-02-10 British Tech Group Applying milking apparatus to a milk animal.
Canadian Intellectual Property Office; Office Action for Application No. 2,775,130; 2 pages, dated Aug. 20, 2012.
Canadian Intellectual Property Office; Office Action for Application No. 2,775,130; 4 pages, dated Nov. 27, 2012.
Canadian Intellectual Property Office; Office Action for Application No. 2,775,132; 3 pages, dated Aug. 20, 2012.
Canadian Intellectual Property Office; Office Action for Application No. 2,775,169; 3 pages, dated Aug. 20, 2012.
Canadian Intellectual Property Office; Office Action for Application No. 2,775,177; 2 pages, dated Aug. 20, 2012.
Canadian Intellectual Property Office; Office Action for Application No. 2,775,177; 3 pages, dated Nov. 27, 2012.
Canadian Intellectual Property Office; Office Action for Application No. 2,775,252; 3 pages, dated Aug. 21, 2012.
Canadian Intellectual Property Office; Office Action for Application No. 2,783,887; 2 pages, dated Oct. 1, 2012.
Canadian Intellectual Property Office; Office Action for Application No. 2,784,070; 3 pages, dated Oct. 1, 2012.
Canadian Office Action for Application No. 2,775,395, 2 pages, dated Jan. 14, 2014.
Partial European Search Report (R. 64 EPC), Application No. 18161863.8, dated Jul. 24, 2018, 12 pages.
PCT Invitation to Pay Additional Fees and, Wher Applicable, Protest Fee and Partial National Search Report for Application No. PCT/US2012/035077; 7 pages, dated Jul. 25, 2012.
PCT Invitation to Pay Additional Fees and, Where Applicable, Protest Fee and Partial National Search Report for Application No. PCT/US2012/035074; 7 pages, dated Jul. 16, 2012.
PCT Invitation to Pay Additional Fees and, Where Applicable, Protest Fee and Partial National Search Report for Application No. PCT/US2012/035079; 8 pages, dated Jul. 31, 2012.
PCT Invitation to Pay Additional Fees and, Where Applicable, Protest Fee and Partial National Search Report for Application No. PCT/US2012/035107; 7 pages, dated Jul. 31, 2012.
PCT Notification of Transmittal of the International Search Report and the Written Opinion of the International Searching Authority for .Application No. PCT/US2012/035356; 14 pages, dated Jul. 31, 2012.
PCT Notification of Transmittal of the International Search Report and the Written Opinion of the International Searching Authority for Application No. PCT/US2011/047510; 9 pages, dated Jan. 2, 2012.
PCT Notification of Transmittal of the International Search Report and the Written Opinion of the International Searching Authority for Application No. PCT/US2011/047511; 9 pages, dated Jan. 2, 2012.
PCT Notification of Transmittal of the International Search Report and the Written Opinion of the International Searching Authority for Application No. PCT/US2012/033892; 13 pages, dated Jul. 31, 2012.
PCT Notification of Transmittal of the International Search Report and the Written Opinion of the International Searching Authority for Application No. PCT/US2012/033894; 11 pages, dated Jul. 23, 2012.
PCT Notification of Transmittal of the International Search Report and the Written Opinion of the International Searching Authority for Application No. PCT/US2012/035077; 25 pages, dated Dec. 12, 2012.
PCT Notification of Transmittal of the International Search Report and the Written Opinion of the International Searching Authority for International Application No. PCT/US2012/035107; 28 pages, dated Oct. 16, 2012.

References: Application No. 2
 Application No. 2
 Application No. 2
 Application No. 2
 Application No. 2
 Application No. 2
 Application No. 2
 Application No. 2
 Application No. 2
 Application No. 2
 Application No. 18161863