Patent Publication Number: US-2023154012-A1

Title: Determination apparatus

Description:
TECHNICAL FIELD 
     The present invention relates to a determination apparatus, a determination method, a recording medium, and a determination system. 
     BACKGROUND ART 
     A technique for detecting foreign matter in a liquid in a container is known. 
     One of the techniques used in detecting foreign matter is described in Patent Document 1, for example. Patent Document 1 describes a foreign matter detection apparatus that has a first irradiation source irradiating a container with first irradiation light, a second irradiation source irradiating the container with second irradiation light, an imaging means, and an inspecting means. According to Patent Document 1, the imaging means captures an image of transmitted light on the transmitted light path of the first irradiation light from the container, and captures an image of diffuse reflected light of the second irradiation light. Then, the inspecting means detects foreign matter in a filled liquid based on the images acquired by the imaging means. Moreover, Patent Document 1 discloses irradiation with the first irradiation light and the second irradiation light in different time periods.
     Patent Document 1: Japanese Unexamined Patent Application Publication JP-A 2003-107010   

     In the case of irradiating with the first irradiation light and the second irradiation light simply in different time periods as described in Patent Document 1, an inspection may take time, and it may be impossible to capture an image at an appropriate timing for the properties of foreign matter, such as the weight of foreign matter. As a result, there is a possibility that foreign matter cannot be efficiently detected with accuracy. 
     SUMMARY 
     Accordingly, an object of the present invention is to provide a determination apparatus, a determination method, a recording medium, and a determination system which solve the problem of a possibility that foreign matter cannot be efficiently detected with accuracy. 
     In order to achieve the object, a determination apparatus as an aspect of the present disclosure includes: a dividing unit configured to divide chronological image data acquired by imaging a liquid filled in a container while switching between a plurality of illumination conditions, into chronological image data corresponding to the illumination conditions; and a determining unit configured to determine foreign matter contained in the container based on each of the chronological image data obtained by division by the dividing unit. 
     Further, a determination method as another aspect of the present disclosure is a determination method executed by a determination device. The determination method includes: dividing chronological image data acquired by imaging a liquid filled in a container while switching between a plurality of illumination conditions, into chronological image data corresponding to the illumination conditions; and determining foreign matter contained in the container based on each of the chronological image data obtained by division. 
     Further, a recording medium as another aspect of the present disclosure is a non-transitory computer-readable recording medium having a program recorded thereon. The program includes instructions for causing a determination apparatus to implement: a dividing unit configured to divide chronological image data acquired by imaging a liquid filled in a container while switching between a plurality of illumination conditions, into chronological image data corresponding to the illumination conditions; and a determining unit configured to determine foreign matter contained in the container based on each of the chronological image data obtained by division by the dividing unit. 
     Further, a determination system as another aspect of the present disclosure includes: a first lighting unit configured to emit light so that light transmitted through a container enters an imaging device acquiring image data; a second lighting unit configured to emit light so that light transmitted through the container does not enter the imaging device acquiring image data; the imaging device configured to image a liquid filled in the container in an environment that an illumination condition is switched between a plurality of illumination conditions including a first state in which the first lighting unit emits light and a second state in which the second lighting unit emits light; and a determination apparatus including a dividing unit configured to divide chronological image data acquired by the imaging device into chronological image data corresponding to the illumination conditions, and a determining unit configured to determine foreign matter contained in the container based on each of the chronological image data obtained by division by the dividing unit. 
     With the respective configurations as described above, it is possible to provide a determination apparatus, a determination method, a recording medium, and a determination method which enables accurate and efficient detection of foreign matter. 
    
    
     
       BRIEF DESCRIPTION OF DRAWINGS 
         FIG.  1    is a view showing an example of a configuration of a determination system in a first example embodiment of the present disclosure; 
         FIG.  2    is a view showing an example of a positional relation between a container and a light; 
         FIG.  3    is a view for describing an example of turning on the light; 
         FIG.  4    is a view for describing an example of turning on the light; 
         FIG.  5    is a block diagram showing an example of a configuration of an illumination control device; 
         FIG.  6    is a view for showing an example of lighting control by the illumination control device; 
         FIG.  7    is a view for showing an example of lighting control by the illumination control device; 
         FIG.  8    is a block diagram showing an example of a configuration of a determination apparatus; 
         FIG.  9    is a view showing an example of image information; 
         FIG.  10    is a view showing an example of processing by a dividing unit; 
         FIG.  11    is a flowchart showing an example of an operation of the determination apparatus in the first example embodiment of the present disclosure; 
         FIG.  12    is a flowchart showing an example of processing by a determining unit; 
         FIG.  13    is a view showing an example of another positional relation between the container and the light; 
         FIG.  14    is a view showing an example of a hardware configuration diagram of a determination apparatus in a second example embodiment of the present disclosure; and 
         FIG.  15    is a block diagram showing an example of a configuration of the determination apparatus in the second example embodiment of the present disclosure. 
     
    
    
     EXAMPLE EMBODIMENTS 
     First Example Embodiment 
     A first example embodiment of the present disclosure will be described with reference to  FIGS.  1  to  13   .  FIG.  1    is a view showing an example of a configuration of a determination system  100 .  FIG.  2    is a view showing an example of a positional relation between a container  700  and a light.  FIGS.  3  and  4    are views for describing examples of turning on the light.  FIG.  5    is a block diagram showing an example of a configuration of an illumination control device  400 .  FIGS.  6  and  7    are views showing examples of lighting control by the illumination control device  400 .  FIG.  8    is a block diagram showing an example of a configuration of a determination apparatus  600 .  FIG.  9    is a view showing an example of image information  631 .  FIG.  10    is a view showing an example of processing by a dividing unit  642 .  FIG.  11    is a flowchart showing an example of an operation of the determination apparatus  600 .  FIG.  12    is a flowchart showing an example of processing by a determining unit  645 .  FIG.  13    is a view showing an example of another positional relation between the container  700  and the light. 
     In the first example embodiment of the present disclosure, a determination system  100  will be described that detects objects mixed in the container  700  filled with a liquid such as water or medicine and determines foreign matter among the detected objects. As will be described later, the determination system  100  acquires chronological image data by imaging with a camera  500  while switching between a plurality of illumination conditions. For example, the determination system  100  acquires chronological image data while switching between a state of irradiating the container  700  with light from a first illumination unit  310  and a state of irradiating the container  700  with light from a second illumination unit  320  at predetermined intervals. Then, the determination system  100  divides the acquired chronological image data into chronological image data corresponding to the respective states, and thereafter determines foreign matter based on each of the division chronological image data. 
     In this example embodiment, foreign matters to be determined include foreign matter that transmits light, such as a glass chip, and foreign matter that does not transmit light, such as a rubber chip and hair. In general, foreign matter that transmits light, such as a glass chip, has a greater specific gravity than foreign matter that does not transmit light, such as a rubber chip and hair, and more quickly sinks to the bottom from a swirling state in the liquid filled inside the container  700 . 
     Further, in this example embodiment, in a state shown in  FIG.  1   , a side from which a liquid is filled in the container  700  is defined as an upper side, and the opposite side is defined as a lower side. In the case of  FIG.  1   , the liquid filled inside the container  700  is pooled on the lower side of the container  700 . 
       FIG.  1    is a side view showing an example of an overall configuration of the determination system  100 . Referring to  FIG.  1   , the determination system  100  includes, for example, a grasping device  200 , the first illumination unit  310 , the second illumination unit  320 , the illumination control device  400 , the camera  500 , and the determination apparatus  600 . As shown in  FIG.  1   , the first illumination unit  310  and the illumination control device  400  are connected so as to be able to communicate with each other. Moreover, the second illumination unit  320  and the illumination control device  400  are connected so as to be able to communicate with each other. Moreover, the camera  500  and the determination apparatus  600  are connected so as to be able to communicate with each other. Moreover, the illumination control device  400  and the camera  500  can be connected so as to be able to communicate with each other. 
     The grasping device  200  is a device which grasps the container  700 . For example, the grasping device  200  includes an upper grasping part projecting downward and contacting the upper side of the container  700 , and a lower grasping part projecting upward and contacting the lower side of the container  700 . As shown in  FIG.  1   , the upper grasping part of the grasping device  200  contacts the container  700  from the upper side of the container  700 , the lower grasping part contacts the container  700  from the lower side of the container  700 , and consequently, the grasping device  200  grasps the container  700  while sandwiching the container  700  from above and below. Meanwhile, the grasping device  200  may grasp the container  700  by a method other than that illustrated above. 
     Further, the grasping device  200  can tilt or rotate the container  700  while sandwiching and grasping the container  700 . By tilting or rotating the grasping device  700  in a state where the grasping device  200  sandwiches the container  700 , foreign matter subsiding in the container  700  can be stirred up. In this example embodiment, a configuration for tilting or rotating the container  700  of the grasping device  200  is not particularly limited. The grasping device  200  may be configured to tilt or rotate the container  700  by a known method. 
     The first illumination unit  310  emits light to a liquid filled in the container  700  in accordance with control by the illumination control device  400 .  FIG.  1    shows an example of a positional relation of the first illumination unit  310 , the second illumination unit  320 , and the container  700  when viewed along the lateral direction, and  FIG.  2    shows an example of a positional relation of the first illumination unit  310 , the second illumination unit  320 , and the container  700  when viewed along the imaging direction of the camera  500 . As shown by  FIGS.  1  and  2   , the first illumination unit  310  is installed on the opposite side to a side where the camera  500  is installed when viewed from the container  700  and the grasping device  200 . With such a configuration, the first illumination unit  310  emits light to the camera  500  through the container  700 . As a result, as shown in  FIG.  3   , while the first illumination unit  310  is emitting light, transmitted light that is the light transmitted through the container  700  enters the camera  500 . At the time, in a case where foreign matter that does not transmit light is present inside the container  700 , the transmitted light applied to a site where the foreign matter that does not transmit light is present is blocked by the foreign matter. For example, in  FIG.  3   , the foreign matter that does not transmit light is shown with a solid black circle. 
     Further, the first illumination unit  310  has a shutter curtain using liquid crystal or the like. The shutter curtain of the first illumination unit  310  is closed when the first illumination unit  310  is not emitting light. Therefore, it can be said that the first illumination unit  310  can be brought in two states, a lighting state in which light is emitted and a lights-out state in which the shutter curtain is closed. 
     As with the first illumination unit  310 , the second illumination unit  320  emits light to a liquid filled in the container  700  in accordance with control by the illumination control device  400 . As shown in  FIGS.  1  and  2   , the second illumination unit  320  is installed, for example, on the side surface of the container  700 . With such a configuration, the second illumination unit  320  emits light along the lateral direction of the container  700 . As a result, as shown in  FIG.  4   , while the second illumination unit  320  is emitting light, transmitted light that is the light transmitted through the container  700  does not directly enter the camera  500 . In a case where foreign matter that transmits light is present inside the container  700 , the foreign matter that transmits light existing in the container  700  diffusely reflects the light. Therefore, as shown in  FIG.  4   , in a case where foreign matter that transmits light is present inside the container  700 , the second illumination unit  320  emits light, and consequently, diffuse reflected light that is the light diffusely reflected by the foreign matter enters the camera  500 . For example, in  FIG.  4   , a white circle represents the foreign matter that transmits light. 
     Further, the second illumination unit  320  can have a shutter curtain using liquid crystal or the like. The shutter curtain of the second illumination unit  320  is closed when the second illumination unit  320  is not emitting light, as with that of the first illumination unit  310 . That is to say, it can be said that the second illumination unit  320  can be brought in two states, a lighting state in which light is emitted and a lights-out state in which the shutter curtain is closed. Meanwhile, the second illumination unit  320  does not necessarily need to have a shutter curtain. 
     The installation position of the second illumination unit  320  is not limited to the case illustrated in  FIGS.  1  and  2   . The second illumination unit  320  may be installed at a position other than that illustrated in  FIGS.  1  and  2   , such as diagonally behind, as long as light emitted by the second illumination unit  320  and transmitted through the container  700  does not directly enter the camera  500 . 
     The illumination control device  400  is a control device that switches illumination conditions by controlling the lighting state and the lights-out state of the first illumination unit  310  and the second illumination unit  320 .  FIG.  5    shows an example of a configuration of the illumination control device  400 . Referring to  FIG.  5   , the illumination control device  400  has, for example, a control unit  410 . 
     The illumination control device  400  has, for example, an operation device such as a CPU and a storage device. For example, the illumination control device  400  implements the above processing units by execution of a program stored in the storage device by the operation unit. The illumination control device  400  may implement the above processing units by hardware. 
     The control unit  410  controls the states of the first illumination unit  310  and the second illumination unit  320  by transmitting lighting instructions to the first illumination unit  310  and the second illumination unit  320 . For example, the control unit  410  controls the stats of the first illumination unit  310  and the second illumination unit  320  to be either a first state in which the first illumination unit  310  is in the lighting state and the second illumination unit  320  is in the lights-out state at the same time or a second state in which the first illumination unit  310  is in the lights-out state and the second illumination unit  320  is in the lighting state at the same time. As described before, in a case where the illumination condition is the first state, transmitted light that is the light transmitted through the container  700  enters the camera  500 . On the other hand, in a case where the illumination condition is the second state, transmitted light that is the light transmitted through the container  700  does not enter the camera  500 . 
     Further, the control unit  410  can control the states of the first illumination unit  310  and the second illumination unit  320  so that the lighting intervals of the first illumination unit  310  and the second illumination unit  320  are synchronized with the imaging intervals of the camera  500 . In other words, the control unit  410  can control the first illumination unit  310  and the second illumination unit  320  so as to switch between the first state and the second state at intervals corresponding to the imaging intervals of the camera  500 . For example, as will be described later, the camera  500  acquires image data at a high frame rate of approximately 150 to 200 fps. Therefore, the control unit  410  can control the first illumination unit  310  and the second illumination unit  320  so as to switch between the first state and the second state at an interval of 0.005 to 0.0067 seconds or a multiple thereof. The control unit  410  may be configured to transmit and receive information necessary for synchronizing the interval with the camera  500 . 
       FIG.  6    shows an example of control by the control unit  410 . For example, the control unit  410  can control the states of the first illumination unit  310  and the second illumination unit  320  so as to switch between the first state and the second state every time the camera  500  performs imaging as shown in  FIG.  6   . In other words, the control unit  410  can control the states of the first illumination unit  310  and the second illumination unit  320  so as to switch between the first state and the second state at regular intervals such as every 0.005 to 0.0067 seconds. 
     Further, the control unit  410  can be configured to change the interval to switch between the first state and second state with the passage of time.  FIG.  7    shows another example of control by the control unit  410 . For example, referring to  FIG.  7   , the control unit  410  keeps the second state until the camera  500  performs imaging two times and then switches to the first state. For example, the control unit  410  keeps the second state for 0.01 to 0.013 seconds, and then keeps the first state for 0.005 to 0.0067 seconds. The control unit  410  continues such a switching process for a predetermined period of time. Subsequently, the control unit  410  continues a process to switch between the first state and the second state at regular intervals for a predetermined period of time. After that, the control unit  410  performs, for a predetermined period of time, a process to keep the first state for 0.01 to 0.013 seconds and then keep the second state for 0.005 to 0.0067 seconds. Thus, the control unit  410  can change the interval to switch so that the second state is longer than the first state immediately after the start of imaging and, with the passage of time, the first state is longer than the second state. Meanwhile, the interval at which the control unit  410  switches between the first state and the second state may be changed in any manner, such as keeping the second state for 0.015 to 0.02 seconds and then keeping the first state for 0.005 to 0.0067 seconds. Moreover, the period of time to switch so that the second state is kept longer than the first state, the period of time to switch at regular intervals, and the period of time to switch so that the first state is kept longer than the second state may be set to any period of time, respectively. As described above, in general, a glass chip or the like, which is foreign matter that transmits light, has a greater specific gravity than foreign matter that does not transmit light, such as a rubber chip and hair, and more quickly sinks to the bottom from a swirling state in a liquid filled inside the container  700 . Therefore, by making the ratio of the second state suitable for detecting foreign matter that transmits light larger at an early stage after the start of imaging, and making the first state suitable for detecting foreign matter that does not transmit light larger with the passage of time, detection in accordance with the properties of foreign matter is enabled. 
     For example, as described above, the control unit  410  can control the states of the first illumination unit  310  and the second illumination unit  320  so as to switch between the first state and the second state at predetermined intervals. 
     The camera  500  is an imaging device that images the container  700  to acquire image data. For example, the camera  500  tilts the container  700  and then starts capturing image data. As shown in  FIG.  1   , the camera  500  is previously installed at a predetermined position on the opposite side to a side where the first illumination unit  310  is located when viewed from the grasping device  200 . The camera  500  may be configured to transmit and receive information necessary for synchronizing the intervals to and from the illumination control device  400 . 
     For example, the camera  500  consecutively acquires image data at a high frame rate of approximately 150 to 200 fps. Then, the camera  500  transmits the acquired image data to the determination apparatus  600  together with information indicating the imaging time and the like. The camera  500  may acquire image data at a frame rate other than that illustrated above. 
     As described above, the states of the first illumination unit  310  and the second illumination unit  320  are controlled by the illumination control device  400 . Therefore, chronological image data acquired by the camera  500  is mixture of the image data captured in the first state and the image data captured in the second state in a manner such that the image data captured in the first state and the image data captured in the second state are present alternately. 
     The determination apparatus  600  is an information processing apparatus that detects objects mixed inside the container  700  filled with a liquid and determines foreign matter based on image data captured/acquired by the camera  500 .  FIG.  8    shows an example of a configuration of the determination apparatus  600 . Referring to  FIG.  8   , the determination apparatus  600  has, as main components, a screen display unit  610 , a communication I/F unit  620 , a storage unit  630 , and an operation processing unit  640 , for example. 
     The screen display unit  610  is formed by a screen display device such as a LCD (Liquid Crystal Display). The screen display unit  610  can display on a screen various kinds of information stored in the storage unit  630  such as image information  631 , tracking information  632  and determination result information  633  in accordance with an instruction from the operation processing unit  640 . 
     The communication I/F unit  620  is formed by a data communication circuit. The communication I/F unit  620  performs data communication with the camera  500 , an external device and the like connected via a communication line. 
     The storage unit  630  is a storage device such as a hard disk and a memory. The storage unit  630  stores therein processing information necessary for various kinds of processing by the operation processing unit  640  and a program  634 . The program  634  is loaded to and executed by the operation processing unit  640  to implement various kinds of processing units. The program  634  is previously loaded from an external device or a recording medium via a data input/output function such as the communication I/F unit  620 , and is stored in the storage unit  630 . Main information stored in the storage unit  630  are, for example, the image information  631 , the tracking information  632 , and the determination result information  633 . 
     The image information  631  includes chronological image data acquired by the camera  500 . In the image information  631 , for example, image data is associated with information indicating time and date of acquisition of the image data by the camera  500  (information indicating the imaging time). 
       FIG.  9    shows an example of the chronological image data included by the image information  631 . Referring to  FIG.  9   , in the image information  631 , image data captured in the first state and image data captured in the second state are mixed. As described before, when the camera  500  acquires image data, the illumination control device  400  switches between the first state and the second state. Therefore, in the image data stored as the image information  631 , the image data captured in the first state and the image data captured in the second state are mixed in a manner such that the image data captured in the first state and the image data captured in the second state are present alternately. 
     The tracking information  632  includes information corresponding to the result of tracking by a tracking unit  644  to be described later. For example, in the tracking information  632 , identification information assigned to each object is associated with chronological information indicating the position of the object. Moreover, the chronological information indicating the position of the object includes, for example, time information and position information such as coordinates indicating the position of the object at each time. 
     As will be described later, after the dividing unit  642  divides the chronological image data included by the image information  631  into the chronological image data captured in the first state and the chronological image data captured in the second state, the tracking unit  644  performs tracking based on the respective chronological image data after division. Therefore, the tracking information  632  includes information corresponding to the result of tracking with the chronological data corresponding to the first state and information corresponding to the result of tracking with the chronological data corresponding to the second state. 
     Meanwhile, the tracking information  632  may include information other than the information of the position of the object as illustrated above. For example, the tracking information  632  may include information indicating the area of a detection region. 
     The determination result information  633  includes information indicating the result of determination by the determining unit  645 . For example, in the determination result information  633 , identification information assigned to each object is associated with information indicating the result of determination based on the tracking information  632  by the determining unit  645 . That is to say, the determination result information  633  includes information indicating whether the detected object is foreign matter, an air bubble, or a scratch, a stain or the like on the container. 
     The operation processing unit  640  has a microprocessor such as a MPU and a peripheral circuit thereof. The operation processing unit  640  loads the program  634  from the storage unit  630  and executes the program  634  to make the abovementioned hardware cooperate with the program  634  and implement various kinds of processing units. Main processing units implemented by the operation processing unit  640  are, for example, an image acquisition unit  641 , the dividing unit  642 , a detecting unit  643 , the tracking unit  644 , the determining unit  645 , and an output unit  646 . 
     The image acquisition unit  641  acquires image data acquired by the camera  500  from the camera  500  via the communication I/F unit  620 . Then, the image acquisition unit  641  associates the acquired data with, for example, time and date of acquisition of the image data (information indicating the imaging time), and stores as the image information  631  into the storage unit  630 . 
     The dividing unit  642  divides chronological image data included by the image information  631  into chronological image data captured in the first state and chronological image data captured in the second state. For example, the determination apparatus  600  has information indicating under what condition (at what interval) the illumination control device  400  switches between the first state and the second state. Then, by using the above information, the dividing unit  642  divides the chronological image data included by the image information  631  into the chronological image data captured in the first state and the chronological image data captured in the second state based on time or the like corresponding to image data. 
       FIG.  10    is a view for describing an example of a division process by the dividing unit  642 . As shown in  FIG.  10   , the dividing unit  642  divides the image information  631  in which the image data captured in the first state and the image data captured in the second state are mixed, into the chronological image data captured in the first state and the chronological image data captured in the second state. 
     Meanwhile, the dividing unit  642  may divide the chronological image data based on information other than the time. For example, the dividing unit  642  may divide the chronological image data into the chronological image data captured in the first state and the chronological image data captured in the second state based on the ratio of lightness and darkness when image data is binarized. The dividing unit  642  may perform the division process by combination of the methods illustrated above. 
     The detecting unit  643  detects objects existing within a region corresponding to the container  500  and a liquid filled in the container  500 . For example, the detecting unit  643  performs a binarization process on image data and detects objects based on the result of the binarization process. Meanwhile, the detecting unit  643  may detect objects by another known technique. 
     In this example embodiment, the detecting unit  643  detects an object from each of the chronological image data captured in the first state obtained by division by the dividing unit  642 , and also detects an object from each of the chronological image data captured in the second state. As shown in  FIG.  3   , in the first state, the transmitted light enters the camera  500 , whereas the transmitted light applied to a site where foreign matter that does not transmit light is present is blocked by the foreign matter. Therefore, the detecting unit  643  can detect mainly an object including foreign matter that does not transmit light based on the image data captured in the first state. On the other hand, as shown in  FIG.  4   , in the second state, the transmitted light does not enter the camera  500 , whereas diffuse reflected light by foreign matter that transmits light enters the camera  500 . Therefore, the detecting unit  643  can detect mainly an object including foreign matter that transmits light based on the image data captured in the second state. 
     The tracking unit  644  tracks an object detected by the detecting unit  643 . As described before, the camera  500  acquires image data at a high frame rate such as 200 fps. Therefore, it is assumed that the positions of the same object in two pieces of image data with consecutive imaging times (the imaging times are closer than a predetermined value) are extremely close to each other. Therefore, the tracking unit  644  compares the position of an object detected by the detecting unit  643  with the position of an object detected by the detecting unit  643  in image data with the previous imaging time (or within the predetermined value). In a case where an object with a distance equal to or less than a predetermined threshold value from the object detected by the detecting unit  643  is present in the image data with the previous imaging time, the tracking unit  644  determines that the object detected by the detecting unit  643  and the object detected by the detecting unit  643  in the image data with the previous imaging time that are separated by the distance equal to or less than the threshold value are the same objects. In this case, the tracking unit  644  provides the object detected by the detecting unit  643  with identification information such as an ID provided to the object determined to be the same object. On the other hand, in a case where an object with a distance equal to or less than the predetermined threshold value from the object detected by the detecting unit  643  is not present in the image data with the previous imaging time (or within the predetermined value), the tracking unit  644  determines that a new object is detected. In this case, the tracking unit  644  assigns identification information such as a new ID to the detected object. 
     For example, in the above manner, the tracking unit  644  performs tracking based on a distance between objects in different image data and thereby assigns identification information to an object detected by the detecting unit  643 . Moreover, the tracking unit  644  acquires coordinates indicating the position of the object detected by the detecting unit  643 . Then, the tracking unit  644  associates the identification information, time information indicating the imaging time of the image data, and the coordinates, and stores as the tracking information  632  into the storage unit  630 . 
     In this example embodiment, the tracking unit  644  performs tracking based on chronological image data captured in the first state obtained by division by the dividing unit  642 , and also performs tracking based on chronological image data captured in the second state. That is to say, the tracking unit  644  performs comparison of positions based on the chronological image data captured in the first state, and performs comparison of positions based on the chronological image data captured in the second state. As described before, the detecting unit  643  detects mainly an object including foreign matter that does not transmit light based on the image data captured in the first state. Therefore, the tracking unit  644  can track mainly an object including foreign matter that does not transmit light based on the chronological image data captured in the first state. On the other hand, the detecting unit  643  detects mainly an object including foreign matter that transmits light based on the image data captured in the second state. Therefore, the tracking unit  644  can track mainly an object including foreign matter that transmits light based on the chronological image data captured in the second state. 
     The determining unit  645  determines whether an object detected by the detecting unit  643  is an air bubble or foreign matter based on the tracking information  632 . Then, the determining unit  645  stores the result of determination of each object detected by the detecting unit  643  as the determination result information  633  into the storage unit  630 . 
     For example, in the case of determining that the object is moving upward based on the tracking information  632 , the determining unit  645  determines that the object is an air bubble. On the other hand, in the case of determining that the object is moving downward based on the tracking information  632 , the determining unit  645  determines that the object is foreign matter. For example, in the above manner, by determining the moving direction of the object based on the tracking information  632 , the determining unit  645  can determine whether the object is an air bubble or foreign matter. Meanwhile, the determining unit  645  may determine whether the object is an air bubble or foreign matter by a method other than the method illustrated above, such as performing the above determination using a previously trained model or the like. 
     As described above, the tracking information  632  includes information corresponding to the result of tracking using chronological data corresponding to the first state and information corresponding to the result of tracking using chronological data corresponding to the second state. Therefore, the determining unit  645  may determine whether the object is an air bubble or foreign matter that does not transmit light based on the information corresponding to the result of tracking using the chronological data corresponding to the first state among the tracking information  632 . Likewise, the determining unit  645  may determine whether the object is an air bubble or foreign matter that transmits light based on the information corresponding to the result of tracking using the chronological data corresponding to the second state among the tracking information  632 . Thus, the determining unit  645  can determine whether the object is an air bubble or foreign matter, and can also determine whether or not the foreign matter transmits light. 
     The determining unit  645  may use information other than the tracking information  632  when determining the object. For example, the determining unit  645  can determine the object also using information indicating the image characteristics, size, average luminance value and the like of the object. By determining also using information other than movement, such as the size and average brightness value of the object, it is possible to comprehensively determine the characteristics of an air bubble and foreign matter, so that higher determination accuracy can be obtained. 
     The output unit  646  outputs the image information  631 , the tracking information  632 , the determination result information  633  and the like. For example, the output unit  646  can display the image information  631 , the tracking information  632 , the determination result information  633  and the like on the screen display unit  610 , and transmit them to an external device via the communication I/F unit  620 . 
     The above is an example of the configuration of the determination apparatus  600 . 
     The determination apparatus  600  may have a configuration other than that described above. For example, the determination apparatus  600  can be configured to instruct the grasping device  200  to tilt/rotate or to stop tilting/rotating. 
     The container  700  is a translucent container such as a glass bottle or a plastic bottle. The inside of the container  700  is filled with a liquid such as water or medicine. Foreign matter may be contained inside the container  700 . Possible foreign matter are, for example, foreign matter that does not transmit light such as a rubber chip, hair, a fiber piece and soot, and foreign matter that transmits light such as glass or a plastic chip. 
     The above is an example of the configurations of the grasping device  200 , the first illumination unit  310 , the second illumination unit  320 , the illumination control device  400 , the camera  500 , and the determination apparatus  600  included by the determination system  100 . Next, an example of an operation of the determination apparatus  600  will be described with reference to  FIGS.  11  and  12   . 
     Referring to  FIG.  11   , the image acquisition unit  641  acquires image data acquired by the camera  500  from the camera  500  via the communication I/F unit  620  (step S 101 ). The image acquisition unit  641  acquires chronological image data in which image data captured in the first state and image data captured in the second state are mixed. 
     The dividing unit  642  divides the chronological image data into chronological image data captured in the first state and chronological image data captured in the second state (step S 102 ). For example, the determination apparatus  600  has information indicating under what condition (at what interval) the illumination control device  400  switches between the first state and the second state. Using the above information, the dividing unit  642  divides the chronological image data included in the image information  631  into the chronological image data captured in the first state and the chronological image data captured in the second state. The dividing unit  642  may perform the division process by a method other than the method illustrated above. 
     The detecting unit  643  detects an object existing in a region corresponding to the container  700  and the liquid filled in the container  700  based on the chronological image data captured in the first state (step S 111 ). In processing at step S 111 , the detecting unit  643  detects mainly an object including foreign matter that does not transmit light. The detecting unit  643  may detect the object by a known technique. 
     The tracking unit  644  tracks the object detected by the detecting unit  643  (step S 112 ). That is to say, the tracking unit  644  tracks mainly the object including foreign matter that does not transmit light. For example, the tracking unit  644  tracks the object based on a distance between objects in image data captured at close (or consecutive) imaging times. 
     In a case where the tracking unit  644  has not tracked all the objects detected by the detecting unit  643  in the image data (step S 113 , No), the tracking unit  644  tracks the object having not been tracked. On the other hand, in a case where the tracking unit  644  has tracked all the objects detected by the detecting unit  643  in the image data (step S 113 , Yes), the tracking unit  644  finishes the tracking based on the image data. 
     In a case where the tracking on all the chronological image data captured in the first state obtained by division by the dividing unit  642  is finished (step S 114 , Yes), the determination apparatus  600  ends the processing based on the chronological image data captured in the first state. On the other hand, in a case where image data having not been detected or tracked is left (step S 114 , No), the determination apparatus  600  continues the detection by the detecting unit  643  and the tracking by the tracking unit  644 . 
     Further, the detecting unit  643  detects an object existing in the region corresponding to the container  700  and the liquid filled in the container  700  based on the chronological image data captured in the second state (step S 121 ). In processing at step S 121 , the detecting unit  643  detects mainly an object including foreign matter that transmits light. The detecting unit  643  may detect the object by a known technique. 
     The tracking unit  644  tracks the object detected by the detecting unit  643  (step S 122 ). That is to say, the tracking unit  644  tracks mainly the object including foreign matter that transmits light. For example, the tracking unit  644  tracks the object based on a distance between objects in image data captured at close (or consecutive) imaging times. 
     In a case where the tracking unit  644  has not tracked all the objects detected by the detecting unit  643  in the image data (step S 123 , No), the tracking unit  644  tracks the object having not been tracked. On the other hand, in a case where the tracking unit  644  has tracked all the objects detected by the detecting unit  643  in the image data (step S 123 , Yes), the tracking unit  644  finishes the tracking based on the image data. 
     In a case where the tracking on all the chronological image data captured in the second state obtained by division by the dividing unit  642  is finished (step S 124 , Yes), the determination apparatus  600  ends the processing based on the chronological image data captured in the second state. On the other hand, in a case where image data having not been detected or tracked is left (step S 124 , No), the determination apparatus  600  continues the detection by the detecting unit  643  and the tracking by the tracking unit  644 . 
     The above is an example of the operation of the determination apparatus  600 . Meanwhile, the processing at steps S 111  to S 114  and the processing at steps S 121  to S 124  may be performed in parallel. Subsequently, an example of processing by the determining unit  645  will be described with reference to  FIG.  12   . 
     Referring to  FIG.  12   , the determining unit  645  acquires the tracking information  632  (step S 201 ). 
     Based on the tracking information  632 , the determining unit  645  determines whether the object detected by the detecting unit  643  is an air bubble or foreign matter (step S 202 ). For example, the determining unit  645  can determine whether the object is an air bubble or foreign matter based on the moving direction of the object. The determining unit  645  may perform the determination by a method other than the method illustrated above. Meanwhile, the determining unit  645  may determine whether the object is an air bubble or foreign matter and also determine whether the foreign matter is foreign matter that transmits light or foreign matter that does not transmit light. 
     The above is an example of the processing by the determining unit  645 . 
     As described above, the determination apparatus  600  has the dividing unit  642 , the detecting unit  643 , the tracking unit  644 , and the determining unit  645 . With such a configuration, the tracking unit  644  can perform tracking of an object detected by the detecting unit  643  based on chronological image data captured in the first state obtained by division by the dividing unit  642 , and tracking of an object detected by the detecting unit  643  based on chronological image data captured in the second state. Moreover, the determining unit  645  can perform determination based on the result of the tracking by the tracking unit  644 . Consequently, it becomes possible to, based on chronological data obtained by consecutively imaging, perform detection, tracking, and determination by a method favorable for foreign matter that transmits light, and also perform detection, tracking, and determination by a method favorable for foreign matter that does not transmit light. As a result, it becomes possible to efficiently detect foreign matter with high accuracy. 
     The configuration of the determination system  100  is not limited to the case illustrated in this example embodiment. For example, as shown in  FIG.  13   , the determination system  100  can have a third illumination unit  330 . The third illumination unit  330  emits light to the liquid filled in the container  700  at the same intervals as those of the second illumination unit  320 . As shown in  FIG.  13   , the third illumination unit  330  can be installed, for example, on a side along the lateral direction of the container  700  and opposite to the side where the second illumination unit  320  is installed when viewed from the container  700 . Thus, the determination system  100  may have a plurality of lights. 
     Further, in this example embodiment, a case where the function of the determination apparatus  600  is realized by one information processing apparatus has been described. However, the function of the determination apparatus  600  may be realized by a plurality of information processing apparatuses connected via a network. 
     Further, in this example embodiment, the determination system  100  has the illumination control device  400  and the determination apparatus  600 . However, the determination apparatus  600  may have the function of the illumination control device  400 . That is to say, the illumination control device  400  and the determination apparatus  600  can be configured integrally. In a case where the determination apparatus  600  has the function of the illumination control device  400 , the determination system  100  does not need to have the separate illumination control device  400 . 
     Further, in this example embodiment, as an example of the illumination condition, the first state in which the first illumination unit  310  is in the lighting state and the second illumination unit  320  is in the lights-out state at the same time, and the second state in which the first illumination unit  310  is in the lights-out state and the second illumination unit  320  is in the lighting state at the same time are shown. However, the illumination condition is not necessarily limited to that illustrated in this example embodiment. For example, the illumination condition may include a state other than that illustrated above, such as a state in which the first illumination unit  310  is in the lighting state and the second illumination unit  320  is in the lighting state at the same time, and a state in which the first illumination unit  310  and the second illumination unit  320  are in the lights-out state and the third illumination unit  330  is in the lighting state. Moreover, for example, switching other than that illustrated in this example embodiment may be performed, such as switching between a state in which the first illumination unit  310  is in the lighting state and the second illumination unit  320  is in the lighting state and a state in which the first illumination unit  310  is in the lights-out state and the second illumination unit  320  is in the lighting state. 
     Second Example Embodiment 
     Next, a second example embodiment of the present invention will be described with reference to  FIGS.  14  and  15   . In the second example embodiment, the outline of a configuration of a determination apparatus  800  will be described. 
       FIG.  14    shows an example of a hardware configuration of the determination apparatus  800 . Referring to  FIG.  14   , the determination apparatus  800  has, as an example, a hardware configuration as shown below including; 
     a CPU (Central Processing Unit)  801  (operation device), 
     a ROM (Read Only Memory)  802  (storage device), 
     a RAM (Random Access Memory)  803  (storage device), 
     programs  804  loaded to the RAM  803 , 
     a storage device  805  for storing the programs  804 , 
     a drive device  806  reading from and writing into a recording medium  810  outside the information processing apparatus, 
     a communication interface  807  connected to a communication network  811  outside the information processing apparatus, 
     an input/output interface  808  performing input and output of data, and 
     a bus  809  connecting the respective components. 
     Further, by acquisition and execution of the programs  804  by the CPU  801 , the determination apparatus  800  can implement the functions of a dividing unit  821  and a determining unit  822  shown in  FIG.  15   . The programs  804  are, for example, stored in the storage device  805  or the ROM  802  in advance, and are loaded to the RAM  803  or the like and executed by the CPU  801  as necessary. Moreover, the programs  804  may be supplied to the CPU  801  via the communication network  811 , or may be stored in the recording medium  810  in advance and retrieved and supplied to the CPU  801  by the drive device  806 . 
       FIG.  14    shows an example of the hardware configuration of the determination apparatus  800 . The hardware configuration of the determination apparatus  800  is not limited to that described above. For example, the determination apparatus  800  may be configured by part of the above configuration, for example, excluding the drive device  806 . 
     The dividing unit  821  divides chronological image data acquired by imaging a liquid filled in a container while switching between a plurality of illumination conditions into chronological image data corresponding to the illumination conditions. For example, the illumination conditions include a first state in which transmitted light that is light having transmitted the container enters an imaging device that acquires image data, and a second state in which the transmitted light that is the light having transmitted the container does not enter the imaging device that acquires image data. In a case where the illumination conditions include the first state and the second state, the dividing unit  821  can divide the acquired chronological image data into chronological image data captured in the first state and chronological image data captured in the second state. 
     The determining unit  822  determines foreign matter contained inside the container based on the respective chronological image data obtained by division by the dividing unit  821 . 
     Thus, the determination apparatus  800  has the dividing unit  821  and the determining unit  822 . With such a configuration, the determining unit  822  can determine foreign matter contained inside the container based on the respective chronological image data obtained by division by the dividing unit  821 . Consequently, it becomes possible to perform determination after dividing chronological data obtained by consecutively imaging based on the imaging conditions. As a result, it becomes possible to efficiently detect foreign matter with accuracy. 
     The determination apparatus  800  described above can be implemented by installation of a predetermined program in the determination apparatus  800 . Specifically, a program as another aspect of the present invention is a program for causing the determination apparatus  800  to implement: a dividing unit that divides chronological image data acquired by imaging a liquid filled in a container while switching between a plurality of illumination conditions into chronological image data corresponding to the illumination conditions; and a determining unit that determines foreign matter contained inside the container based on the respective chronological image data obtained by division by the dividing unit. 
     Further, a determination method executed by the determination apparatus  800  described above is a method executed by the determination apparatus  800  including: dividing chronological image data acquired by imaging a liquid filled in a container while switching between a plurality of illumination conditions into chronological image data corresponding to the illumination conditions; and determining foreign matter contained inside the container based on the respective chronological image data obtained by division by the dividing unit. 
     The invention of the program or the determination method having the above configuration also has the same actions and effects as the determination apparatus  800 , and therefore, can achieve the abovementioned object of the present invention. 
     &lt;Supplementary Notes&gt; 
     The whole or part of the example embodiments disclosed above can be described as the following supplementary notes. Below, the overview of a determination apparatus and others according to the present invention will be described. However, the present invention is not limited to the following configurations. 
     (Supplementary Note 1) 
     A determination apparatus comprising: 
     a dividing unit configured to divide chronological image data acquired by imaging a liquid filled in a container while switching between a plurality of illumination conditions, into chronological image data corresponding to the illumination conditions; and 
     a determining unit configured to determine foreign matter contained in the container based on each of the chronological image data obtained by division by the dividing unit. 
     (Supplementary Note 2) 
     The determination apparatus according to Supplementary Note 1, wherein 
     the illumination conditions include a first state and a second state, the first state being a state in which light transmitted through the container enters an imaging device acquiring image data, the second state being a state in which light transmitted through the container does not enter the imaging device acquiring image data. 
     (Supplementary Note 3) 
     The determination apparatus according to Supplementary Note 2, wherein in the second state in which the light transmitted through the container does not enter the imaging device acquiring image data, a shutter curtain included by a lighting unit configured to emit light in the first state is in a closed state. 
     (Supplementary Note 4) 
     The determination apparatus according to Supplementary Note 2 or 3, wherein an interval to switch between the first state and the second state changes with passage of time. 
     (Supplementary Note 5) 
     The determination apparatus according to any one of Supplementary Notes 1 to 4, wherein 
     the dividing unit is configured to divide chronological image data acquired by imaging while switching between the illumination conditions in accordance with an imaging interval of the imaging device acquiring image data. 
     (Supplementary Note 6) 
     The determination apparatus according to Supplementary Note 5, wherein 
     the dividing unit is configured to divide the chronological image data based on information indicating time at which image data has been acquired. 
     (Supplementary Note 7) 
     The determination apparatus according to any one of Supplementary Notes 1 to 6, comprising 
     a detecting unit configured to detect an object from each image data included by each of the chronological image data obtained by division by the dividing unit; and 
     a tracking unit configured to track the object detected by the detecting unit for each of the chronological image data obtained by division by the dividing unit, 
     wherein the determining unit is configured to determine whether or not the object detected by the detecting unit is the foreign matter based on a result of tracking by the tracking unit. 
     (Supplementary Note 8) 
     The determination apparatus according to Supplementary Note 7, wherein 
     the determining unit is configured to determine whether the object is an air bubble or the foreign matter. 
     (Supplementary Note 9) 
     The determination apparatus according to Supplementary Note 7 or 8, wherein 
     the determining unit is configured to determine whether or not the object is the foreign matter, and also determine whether or not the foreign matter transmits light. 
     (Supplementary Note 10) 
     A determination method executed by a determination apparatus, the determination method comprising: 
     dividing chronological image data acquired by imaging a liquid filled in a container while switching between a plurality of illumination conditions, into chronological image data corresponding to the illumination conditions; and 
     determining foreign matter contained in the container based on each of the chronological image data obtained by division. 
     (Supplementary Note 11) 
     A non-transitory computer-readable recording medium having a program recorded thereon, the program comprising instructions for causing a determination apparatus to implement: 
     a dividing unit configured to divide chronological image data acquired by imaging a liquid filled in a container while switching between a plurality of illumination conditions, into chronological image data corresponding to the illumination conditions; and 
     a determining unit configured to determine foreign matter contained in the container based on each of the chronological image data obtained by division by the dividing unit. 
     (Supplementary Note 12) 
     A determination system comprising: 
     a first lighting unit configured to emit light so that light transmitted through a container enters an imaging device acquiring image data; 
     a second lighting unit configured to emit light so that light transmitted through the container does not enter the imaging device acquiring image data; 
     the imaging device configured to image a liquid filled in the container in an environment that an illumination condition is switched between a plurality of illumination conditions including a first state in which the first lighting unit emits light and a second state in which the second lighting unit emits light; and 
     a determination apparatus including a dividing unit and a determining unit, the dividing unit being configured to divide chronological image data acquired by the imaging device into chronological image data corresponding to the illumination conditions, the determining unit being configured to determine foreign matter contained in the container based on each of the chronological image data obtained by division by the dividing unit. 
     (Supplementary Note 13) 
     The determination system according to Supplementary Note 12, wherein 
     the first lighting unit has a shutter curtain closed when light is not emitted. 
     (Supplementary Note 14) 
     The determination system according to Supplementary Note 12 or 13, comprising 
     an illumination control device configured to switch between the first state in which the first lighting unit emits light and the second state in which the second lighting unit emits light. 
     (Supplementary Note 15) 
     The determination system according to Supplementary Note 14, wherein 
     the illumination control device is configured to switch between the first state and the second state at regular intervals. 
     (Supplementary Note 16) 
     The determination system according to Supplementary Note 14 or 15, wherein 
     the illumination control device is configured to change an interval to switch between the first state and the second state with passage of time. 
     (Supplementary Note 17) 
     The determination system according to any one of Supplementary Notes 14 to 16, wherein 
     the illumination control device is configured to change an interval to switch between the first state and the second state from a state in which the second state is longer than the first state to a state in which the first state is longer than the second state with passage of time. 
     (Supplementary Note 18) 
     The determination system according to any one of Supplementary Notes 14 to 17, wherein 
     the illumination control device is configured to switch between the first state and the second state in accordance with an imaging interval of the imaging device acquiring image data. 
     The program described in the example embodiments and supplementary notes is stored in a storage device, or recorded on a computer-readable recording medium. For example, the recording medium is a portable medium such as a flexible disk, an optical disk, a magnetooptical disk, and a semiconductor memory. 
     Although the present invention has been described above with reference to the example embodiments, the present invention is not limited to the above example embodiments. The configurations and details of the present invention can be changed in various manners that can be understood by one skilled in the art within the scope of the present invention. 
     DESCRIPTION OF REFERENCE NUMERALS 
     
         
           100  determination system 
           200  grasping device 
           310  first illumination unit 
           320  second illumination unit 
           330  third illumination unit 
           400  illumination control device 
           410  control unit 
           500  camera 
           600  determination apparatus 
           610  screen display unit 
           620  communication I/F unit 
           630  storage unit 
           631  image information 
           632  tracking information 
           633  determination result information 
           634  program 
           640  operation processing unit 
           641  image acquisition unit 
           642  dividing unit 
           643  detecting unit 
           644  tracking unit 
           645  determining unit 
           646  output unit 
           700  container 
           800  determination apparatus 
           801  CPU 
           802  ROM 
           803  RAM 
           804  programs 
           805  storage device 
           806  drive device 
           807  communication interface 
           808  input/output interface 
           809  bus 
           810  recording medium 
           811  communication network 
           821  dividing unit 
           822  determining unit