Abstract:
A fruits sorting apparatus includes: a conveyance device configured to convey a plurality of charged fruits from a detection area located on an upstream side to a sorting area disposed on a downstream side; an imaging device configured to photograph the fruits in the detection area of the conveyance device; a processor configured to detect a rotten fruit and a nonstandard fruit having an irregular shape or size based on images of the fruits and vegetables imaged by the imaging device; and a robot configured to pick up the rotten fruit and the nonstandard fruit detected by the processor in the sorting area.

Description:
BACKGROUND 
       [0001]    Technical Field 
         [0002]    The present invention relates to an apparatus and a method for sorting and removing fruits in a case where the collected fruits such as mandarin oranges are rotten or nonstandard. 
         [0003]    Related Art 
         [0004]    When collected fruits such as mandarin oranges arrive at a fruit sorting place, the fruits flow to a fruit sorting apparatus configured by a conveyance device. In an upstream process, the fruit sorting apparatus sorts and removes rotten fruits and nonstandard fruits having nonstandard sizes or shapes so as not to allow them to flow to a downstream process. Usually, such a sorting operation is manually performed by about four to eight workers through visual observation. 
         [0005]    However, such a sorting process is performed by individual workers determining the degree of a rotten state or the degree of a nonstandard shape or size of each of fruits. Accordingly, non-uniformity of the accuracy can easily occur. In addition, it takes too long for the sorting process, and there are also cases where rotten fruits and nonstandard fruits that have been overlooked from the sorting process are transferred to the downstream process. 
         [0006]    In addition, the population engaging in the agricultural industry decreases year by year, and most of the workers are aging. Currently, while there is a problem of labor shortage, a further challenging situation is expected in the years to come. 
         [0007]    The present invention has been made in view of the circumstances described above, and an object thereof is to automate the process of sorting rotten fruits and nonstandard fruits and to improve the sorting accuracy, the sorting efficiency, and the stability of the sorting operation. 
       SUMMARY 
       [0008]    In order to achieve the above object, according to a first aspect of the present invention, there is provided a fruits sorting apparatus including: a conveyance device configured to convey a plurality of charged fruits from a detection area located on an upstream side to a sorting area located on a downstream side; an imaging device configured to photograph the fruits in the detection area of the conveyance device; a processor configured to detect a rotten fruit and a nonstandard fruit having an irregular shape or size based on images of the fruits photographed by the imaging device; and a robot controller configured to control a robot that picks up the rotten fruit and the nonstandard fruit detected by the processor in the sorting area. 
         [0009]    According to the first aspect of the present invention, the conveyance device conveys the fruits while rotating the fruits in the detection area and conveys the fruits in a stopped state in the sorting area. 
         [0010]    According to the first aspect of the present invention, the processor acquires images of a plurality of faces photographed by the imaging device in the detection area for each of the fruits and detects the each of the fruits as the rotten fruit when the size of a rotten portion of the each of the fruits exceeds a decay threshold based on the plurality of the images. 
         [0011]    According to the first aspect of the present invention, the processor acquires images of a plurality of faces photographed by the imaging device in the detection area for each of the fruits and detects the each of the fruits as the nonstandard fruit when the shape and the size of the each of the fruits deviate from ranges of standard thresholds based on the plurality of the images. 
         [0012]    According to the first aspect of the present invention, the processor assigns an identification symbol to each of the fruits conveyed in the detection area and tracks movement of the each of the fruits, and, when the rotten fruit or the nonstandard fruit is conveyed from the detection area to the sorting area, sends out the identification symbol of the rotten fruit or the nonstandard fruit, data of timing of charging into the sorting area, and data of a position in a horizontal direction perpendicular to a conveyance direction of the conveyance device to the robot controller. 
         [0013]    According to the first aspect of the present invention, the robot controller issues a pickup instruction for picking up the rotten fruit or the nonstandard fruit to the robot based on the identification symbol, the data of the timing, and the data of the position sent out from the processor. 
         [0014]    According to the first aspect of the present invention, the fruit sorting apparatus further includes: a fruits disposal box for rotten fruits, into which the rotten fruit picked up by the robot is discarded; and a fruits disposal box for nonstandard fruits, into which the nonstandard fruit picked up by the robot is discarded. 
         [0015]    According to the first aspect of the present invention, each of the fruits disposal box for rotten fruits and the fruits disposal box for nonstandard fruits is arranged on a right outer side and a left outer side of the sorting area of the conveyance device in the conveyance direction, and the robot controller controls the robot such that the rotten fruit is discarded into the fruits disposal box for rotten fruits arranged on the right outer side or the left outer side in accordance with the data of the position of the rotten fruit when the robot picks up the rotten fruit, and the nonstandard fruit is discarded into the fruits disposal box for nonstandard fruits arranged on the right outer side or the left outer side in accordance with the data of the position of the nonstandard fruit when the robot picks up the nonstandard fruit. 
         [0016]    According to a second aspect of the present invention, there is provided a fruit sorting method including: photographing a plurality of charged fruits in a detection area located in the middle of conveyance executed by a conveyance device by using an imaging device; detecting a rotten fruit and a nonstandard fruit having an irregular shape or size based on images acquired by the imaging device; and picking up the rotten fruit and the nonstandard fruit detected in a sorting area located on a downstream side of the detection area by using a robot. 
         [0017]    According to the second aspect of the present invention, the fruits are conveyed while being rotated in the detection area, and the fruits are conveyed in a stopped state in the sorting area. 
         [0018]    According to the second aspect of the present invention, each of the fruits is detected as the rotten fruit when the size of a rotten portion of the each of the fruits exceeds a decay threshold. 
         [0019]    According to the second aspect of the present invention, each of the fruits is detected as the nonstandard fruit when the shape and the size of the each of the fruits deviate from ranges of standard thresholds. 
         [0020]    According to the second aspect of the present invention, the fruits sorting method further includes assigning an identification symbol to each of the fruits conveyed in the detection area, tracking movement of the each of the fruits, and, when the rotten fruit or the nonstandard fruit is conveyed from the detection area to the sorting area, acquiring the identification symbol of the rotten fruit or the nonstandard fruit, data of timing of charging into the sorting area, and data of a position in a horizontal direction perpendicular to a conveyance direction of the conveyance device. 
         [0021]    According to the second aspect of the present invention, the rotten fruit or the nonstandard fruit is picked up in the sorting area by the robot based on the identification symbol, the data of the timing, and the data of the position that are acquired. 
         [0022]    According to the second aspect of the present invention, the fruits sorting method further includes discarding the rotten fruit picked up by the robot into a fruits disposal box for rotten fruits and throwing the nonstandard fruit picked up by the robot into a fruit disposal box for nonstandard fruits. 
         [0023]    According to the second aspect of the present invention, the rotten fruit picked up by the robot is discarded into the fruits disposal box for rotten fruits arranged on the right outer side or the left outer side in accordance with the data of the position, and the nonstandard fruit picked up by the robot is discarded into the fruit disposal box for nonstandard fruits arranged on the right outer side or the left outer side in accordance with the data of the position. 
         [0024]    According to the present invention, by only charging collected fruits, rotten fruits and nonstandard fruits among the fruits can be automatically sorted and removed. For this reason, the shortage of labor engaging in the agricultural industry can be solved, and the instability of the sorting operation according to the labor shortage, the capability, and the physical strength can be solved. In addition, a sorting process of high accuracy can be performed through automation. Furthermore, the sorting operation can be performed at night that is a time slot in which the operation is not currently performed, and the efficiency of the sorting operation can be increased. 
     
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         [0025]      FIG. 1  is an entire configuration diagram of a fruits sorting apparatus according to an embodiment of the present invention; 
           [0026]      FIG. 2  is a side view of a main portion of a fruits sorting apparatus according to an embodiment of the present invention; 
           [0027]      FIG. 3  is a plan view of a main portion of a fruits sorting apparatus according to an embodiment of the present invention; 
           [0028]      FIG. 4  is a flowchart illustrating the entire sorting process executed by a fruits sorting apparatus according to an embodiment of the present invention; 
           [0029]      FIG. 5  is a flowchart illustrating a rotten fruit detection process executed by a fruits sorting apparatus according to an embodiment of the present invention; 
           [0030]      FIG. 6  is a flowchart illustrating a nonstandard fruit detection process executed by a fruit sorting apparatus according to an embodiment of the present invention; and 
           [0031]      FIG. 7  is an explanatory diagram illustrating a sorting operation executed by a fruit sorting apparatus according to an embodiment of the present invention. 
       
    
    
     DETAILED DESCRIPTION 
       [0032]    Hereinafter, a fruit sorting apparatus according to an embodiment of the present invention will be described.  FIG. 1  illustrates the entire configuration of the fruits sorting apparatus according to this embodiment. In  FIG. 1 , a conveyance device  10  is a conveyer which conveys fruits A in a direction of arrow X, a lighting device  20  includes an LED or a halogen lamp which lights the fruits A conveyed by the conveyance device  10 , and an imaging device  30  uses a CCD camera which continuously photographs the fruits A lighted by the lighting device  20  in the middle of the conveyance process. 
         [0033]    In addition, a processor  40  includes a computer including a CPU and takes in an image captured by the imaging device  30  and performs image processing thereon, detects each of the fruits A and assigns an identification symbol thereto, tracks the movement thereof, and detects rotten fruits and nonstandard fruits present among the fruits A. Then, the processor  40  outputs identification symbols, Y coordinates in a direction perpendicular to the conveyance direction denoted by arrow X, send-out timings, and the like for rotten fruits and nonstandard fruits when the rotten fruits and the nonstandard fruits are sent out from a detection area to be described later to a sorting area. 
         [0034]    In addition, an operation unit  50  used for operating the entire fruits sorting apparatus and a monitor  60  used for displaying a manipulation content or an operation content are illustrated. 
         [0035]    A robot controller  70  takes in the identification symbols, the Y coordinates, the send-out timings, and the like for the rotten fruits and the nonstandard fruits that have been assigned by the processor  40 , and controls a robot  80  having a catcher  81 . 
         [0036]    The catcher  81  of the robot  80  moves in the conveyance direction of the conveyance device  10  that is denoted by arrow X, in the Y direction that is perpendicular to arrow X, and in the vertical direction denoted by arrow Z to pick up rotten fruits and nonstandard fruits. 
         [0037]    Rotten fruits and nonstandard fruits picked up by the catcher  81  are discarded into a fruit disposal box  90 . 
         [0038]      FIG. 2  is a side elevation of the conveyance device  10 , and  FIG. 3  is a partial plan view thereof. The conveyance device  10  is configured to be endless by aligning a plurality of rotary rollers  10   a  in the conveyance direction denoted by arrow X, the entire circumference of said rotary rollers being painted in a predetermined color. This conveyance device  10  sequentially conveys fruits A charged into a charging area  11  along a direction of an arrow X to a detection area  12  located on a downstream thereof, a sorting area  13  located on a further downstream thereof, and a send-out area  14  located on a still further downstream thereof. On the left outer side of the sorting area  13  in the conveyance direction denoted by arrow X, fruit disposal boxes  91 L and  92 L are arranged, and, on the right outer side thereof, fruits disposal boxes  91 R and  92 R are arranged. The fruits disposal boxes  91 L and  91 R are used for rotten fruits, and the fruits disposal boxes  92 L and  92 R are used for nonstandard fruits. 
         [0039]    While the rotary rollers  10   a  configuring the conveyance device  10  only move (revolution) in a direction of an arrow X without rotating (autorotation) in the charging area  11 , the sorting area  13 , and the send-out area  14 , the rotary rollers move (revolution) in a direction of an arrow X and rotates (autorotation) in the detection area  12 . 
         [0040]      FIG. 4  is a flowchart of a sorting operation for fruits and vegetables. Fruits and vegetables A charged into the charging area  11  are conveyed in the X direction according to the movement (revolution) of the rotary roller  10   a  and rotate according to the rotation (autorotation) thereof in the detection area  12  of the conveyance device  10 . 
         [0041]    At this time, the external shape of each of the fruits A is not a perfect sphere and thus, moves in the X direction according to the movement and rotation of the rotary rollers  10   a  and moves in a Y direction (horizontal direction) perpendicular to the X direction or forward diagonally, rearward diagonally, or due rearward. For this reason, a plurality of faces of each of the fruits A are photographed in the detection area  12 . 
         [0042]    As described above, the rotary rollers  10   a  of the conveyance device  10  are painted in a predetermined color, and, by setting said color to a color different from the color of the fruits, for example, setting the color of the rotary rollers to an opposite color of fruits A or setting the density of the color to a different density, the fruits disposed on the conveyance device  10  can be easily identified. Accordingly, the processor  40  identifies and detects each of the fruits A in an image captured by the imaging device  30  and assigns an identification symbol to each of the fruits A. 
         [0043]    Each of the fruits and vegetables A that have been detected is automatically tracked, and the images of each of the fruits A are updated at a period of ¼ of the rotation speed of the fruit A or higher, while the period also depends on the processing speed of the processor  40 . Then, for each of the fruits A that have been detected, the processor  40  detects whether the fruits A is a rotten fruit A 1  or a nonstandard fruit A 2  based on the updated images thereof (S 2 ). 
         [0044]    Then, when the rotten fruits A 1 , the nonstandard fruits A 2 , and the other fruits A move from the detection area  12  to the sorting area  13  over a boundary B, the automatic tracking ends. 
         [0045]    In the sorting area  13  where the rotary rollers  10   a  stop to rotate, the rotten fruits A 1 , the nonstandard fruits A 2 , and the other fruits A are conveyed in a direction of an arrow X with the coordinate of a direction of an arrow Y at a time point after the boundary B being fixed. 
         [0046]    Immediately after (after no influence of the rotation of the rotary rollers  10   a  is received) crossing the boundary B, information of the identification symbol of the rotten fruit A 1  or the nonstandard fruit A 2 , the timing at the time of crossing the boundary B, the Y coordinate, and the like for each of the rotten fruits A 1  and the nonstandard fruits A 2  is transmitted from the processor  40  to the robot controller  70  (S 3 ). 
         [0047]    In the sorting area  13 , the current coordinates of each of rotten fruits A 1  and nonstandard fruits A 2  in the middle of the conveyance process executed by the conveyance device  10  are acquired by the robot controller  70 . 
         [0048]    In other words, for rotten fruits A 1  and nonstandard fruits A 2 , the X coordinate of each of the rotten fruits A 1  and the nonstandard fruits A 2  at the current time point is acquired based on an elapsed time from the timing at the time of crossing the boundary B to the current time point and the conveyance speed of the conveyance device  10  by the robot controller  70 . In addition, the Y coordinate (not changing) is acquired as well. 
         [0049]    Thus, according to an instruction of the current coordinates output from the robot controller  70 , the robot  80  picks up such rotten fruits A 1  or nonstandard fruits A 2  by using the catcher  81  and discards them into the fruit disposal boxes  91 L,  91 R,  92 L, and  92 R (S 4 ). 
         [0050]    Here, rotten fruits A 1  and nonstandard fruits A 2  are caught by using the catcher  81  preferably in the order of earliest to latest timing of the time point of crossing the boundary B among the rotten fruits A 1  and the nonstandard fruits A 2 . However, in a case where the processing capability of the catcher  81  is high, the catching process may be executed in an arbitrary order. 
         [0051]    At this time, in a case where the Y coordinate of a caught rotten fruit A 1  is closer to the fruit disposal box  91 R than to the fruit disposal box  91 L, the rotten fruit A 1  is discarded into the fruit disposal box  91 R. Otherwise, the rotten fruit A 1  is caught into the fruit disposal box  91 L. 
         [0052]    On the other hand, in a case where the Y coordinate of a caught nonstandard fruit A 2  is closer to the fruit disposals box  92 R than to the fruits disposal box  92 L, the nonstandard fruit A 2  is discarded into the fruits disposal box  92 R. Otherwise, the nonstandard fruit A 2  is discarded into the fruits disposal box  92 L. 
         [0053]    In this way, by discarding a rotten fruit A 1  or a nonstandard fruit A 2  into a closer fruits disposal box in accordance with the Y coordinate assigned to the rotten fruit A 1  or the nonstandard fruit A 2 , the efficiency of the sorting process can be increased. 
         [0054]      FIG. 5  is a flowchart illustrating step S 2  represented in  FIG. 4  in detail and illustrates the process of determining whether or not a fruit A is a rotten fruit A 1 . In this process, each updated image of the fruit/vegetable A is binarized such that a rotten portion occurring therein is set to “1”, and a normal portion is set to “0” (S 21 ). Since the color of a rotten portion of a citrus fruit such as a mandarin orange is different from that of a normal portion in the image processing, the rotten portion can be identified. Then, the area of the rotten portion is calculated based on the number of pixels of the area of “1” (S 22 ), and the area is compared with a decay threshold set in advance (S 23 ). When the rotten portion is larger than the decay threshold, the fruit A is detected as a rotten fruit A 1  (S 24 ). 
         [0055]      FIG. 6  is a flowchart illustrating step S 2  represented in  FIG. 4  in detail and illustrates the process of determining whether or not a fruit A is a nonstandard fruit A 2 . In this process, each updated image of the fruit A is binarized such that the outer shape is set to “1”, and the periphery thereof is set to “0” (S 25 ). Then, based on the pixels of an image acquired by cutting out the outer shape, a circle-equivalent diameter (a diameter converted into a circle), a maximum diameter, and an area are calculated (S 26 ), and the calculated values are compared with standard thresholds set in advance (S 27 ). When the calculated values deviate from the ranges of the standard thresholds, the fruit A is detected as a nonstandard fruit A 2  (S 28 ). 
         [0056]    The process of detecting a rotten fruit A 1  illustrated in  FIG. 5  and the process of detecting a nonstandard fruit A 2  illustrated in  FIG. 6  as above are sequentially executed for the individual fruit A. 
         [0057]    Here, an operation of detecting a rotten fruit A 1  and a nonstandard fruit A 2  among a plurality of fruits A conveyed in the middle of the detection area  12  of the conveyance device  10  will be described with reference to  FIG. 7 . 
         [0058]    At time t 1 , during the conveyance process executed by the conveyance device  10  in the detection area  12 , rotten fruits A 1  (rotten fruits A 11  and A 12 ) having the identification symbols A 11  and A 12  among a plurality of fruits A are detected and automatically tracked by the processor  40 . In addition, nonstandard fruits A 2  (nonstandard fruits A 21  and A 22 ) of which the identification symbols are A 21  and A 22  are detected and automatically tracked. Furthermore, although the other fruits A have identification symbols assigned thereto and are automatically tracked, the identification symbols are not illustrated for simplicity. 
         [0059]    Thereafter, as the conveyance process executed by the conveyance device  10  is continued, at time t 2 , the nonstandard fruit A 21  crosses the boundary B to enter the sorting area  13  from the detection area  12 . When the nonstandard fruit A 21  crosses the boundary B, data of the identification symbol A 21  of the nonstandard fruit A 21 , the timing to of a time point of the passage, and the Y-direction coordinate ya at the Lime point of the passage is transmitted from the processor  40  to the robot controller  70 . At this time, the rotten fruit A 12  is displaced to the lower side, and the rotten fruit A 11  is displaced to the upper side. 
         [0060]    In addition, at this time t 2 , in the detection area  12 , rotten fruits A 1  (rotten fruits A 13  and A 14 ) having the identification symbols A 13  and A 14  are newly detected. Furthermore, a nonstandard fruit A 2  (a nonstandard fruit A 23 ) having the identification symbol A 23  is detected. 
         [0061]    Furthermore, as the conveyance process executed by the conveyance device  10  is continued, at time t 3 , the rotten fruit A 11  crosses the boundary B and enters the sorting area  13 . When the rotten fruit A 11  crosses the boundary B, data of the identification symbol A 11  of the rotten fruit A 11 , timing tb of a time point of the passage, and the Y-direction coordinate yb at the time point of the passage is transmitted from the processor  40  to the robot controller  70 . At this time, the rotten fruit A 14  is displaced to the upper side, and the nonstandard fruit A 23  is displaced to the lower side. 
         [0062]    In addition, at this time t 3 , in the detection area  12 , rotten fruits A 1  (rotten fruits A 15  and A 16 ) having the identification symbols A 15  and A 16  are newly detected, and a nonstandard fruit (a nonstandard fruit A 24 ) having the identification symbol A 24  is detected. 
         [0063]    In the sorting area  13 , the nonstandard fruit A 21  is managed by the robot controller  70  based on the identification symbol A 21 , the timing ta, and the coordinate Ya, is picked up by the catcher  81  of the robot  80  based on the XY coordinates after the elapse of a predetermined time, and is discarded into the fruit disposal box  92 L or  92 R based on the coordinate Ya. 
         [0064]    In addition, the rotten fruit A 11  is managed by the robot controller  70  based on the identification symbol A 11 , the timing tb, and the coordinate Yb, is picked up by the catcher  81  of the robot  80  based on the XY coordinates after the elapse of a predetermined time, and is discarded into the fruit disposal box  91 L or  91 R based on the coordinate Yb.