Abstract:
In one example embodiment, an information processing system includes an image pickup apparatus connected to a microscope having a stage configured to support an observation target object. In this example embodiment, the stage has a first position. The information processing system has a buffer having an amount of data. In response to the amount of data including a predetermined amount, the information processing system causes the stage to move from the first position to a different position. In response to the amount of data including the predetermined amount, the information processing system causes the buffer to, while the stage is moving, store images associated with the observation target object. In one example embodiment, the images are continuously captured by the image pickup apparatus. Thereafter, the information processing system, in response to the amount of data not including the predetermined amount, causes the image pickup apparatus to terminate capturing the images.

Description:
CROSS REFERENCES TO RELATED APPLICATIONS 
       [0001]    The present application claims priority to Japanese Patent Application No. JP 2009-285154, filed in the Japanese Patent Office on Dec. 16, 2009, the entire contents of which is being incorporated herein by reference. 
       BACKGROUND 
       [0002]    In a field of medicine, pathology, or the like, there has been proposed a system that digitizes an image of an object to be observed such as a cell, a tissue, an organ, or the like of a living body, that is obtained by an optical microscope, to examine the tissue or the like or diagnose a patient by a doctor or a pathologist based on the digitized image. The above-mentioned system is generally called a virtual microscope system. 
         [0003]    For example, Japanese Patent Application Laid-open No. 2009-37250 (hereinafter, referred to as Patent Document 1) discloses a method in which an image optically obtained from a slide specimen placed on a stage of a microscope is digitized by a video camera with a CCD (charge coupled device), a digital signal thereof is input to a PC (personal computer), and the image is visualized on a monitor. A pathologist performs examination or the like while viewing the image displayed on the monitor (see, for example, paragraphs [0027] and [0028] and FIG. 5 of Patent Document 1). 
         [0004]    In the virtual microscope system, raw data of an image obtained by an image pickup apparatus is transferred to a PC (image processing apparatus) via a bus. Here, to directly store the raw date in an HDD (hard disk drive) of the PC is difficult in terms of a writing speed of the HDD. In view of this, it is conceivable that the PC stores the raw data in the HDD after subjecting the raw data to various image processing such as a developing process and a compression process to a predetermined format including a JPEG format or the like. 
         [0005]    However, in the case where the image pickup apparatus continuously takes images of different regions of an observation target object while the observation target is being moved on a stage and the PC processes the images in real time, if the PC is overloaded, there may arise a fear that a defect is caused in part of the image data during the transfer of the raw data from the image pickup apparatus side to the computer side. 
         [0006]    In particular, in the field of medicine, an image treated with the virtual microscope is directly linked to a pathological diagnosis. Therefore, not only the defect in data but also a partial complement to the image is not permitted. In addition, if the defect in the data is caused, the image data can be transmitted again from the image pickup apparatus side to the PC side. However, in this case, a wasted band and a latency time are caused by an amount corresponding to the image in which the defect is caused. 
       SUMMARY 
       [0007]    The present disclosure relates to an image processing system, an image processing apparatus, an image processing method, and a program for processing image information obtained by a microscope in a field of medicine, pathology, biology, materials science, or the like. 
         [0008]    In one example embodiment, an information processing system includes a processor, an image pickup apparatus operatively connected to a microscope which has a stage configured to support an observation target object (i.e., a pathological specimen), the stage being in a first position, and a memory device operatively coupled to the processor, the memory device having a buffer which has an amount of data, the memory device storing instructions that cause the processor, in response to the amount of data including a predetermined amount, to: (a) cause the stage to move from the first position to a second, different position; (b) cause the buffer to, while the stage is moving, store images which are associated with the observation target object, the images being continuously captured by the image pickup apparatus; and (c) thereafter, in response to the amount of data not including the predetermined amount, cause the image pickup apparatus to terminate the capture of the images. 
         [0009]    In one example embodiment, the image pickup apparatus includes the processor. In one example embodiment, the information processing system includes an information processing apparatus which includes the processor. 
         [0010]    In one example embodiment, the instructions cause the processor, in cooperation with another processor, to, for a designated captured image perform an image processing. 
         [0011]    In one example embodiment, the information processing apparatus is connected to the image pickup apparatus via a bidirectional communication channel. 
         [0012]    In one example embodiment, the information processing apparatus includes the memory device. 
         [0013]    In one example embodiment, the instructions cause the processor to: (a) determine whether the stage has a first state; and (b) in response to the stage having the first state, cause the stage to move from the first position to the second, different position. 
         [0014]    In one example embodiment, the stage has the first state if the stage is not moving. 
         [0015]    In one example embodiment, the microscope includes a sensor. In this example embodiment, the instructions cause the processor to: (a) determine whether the sensor has a second state; and (b) in response to the sensor having the second state, cause the stage to move from the first position to the second, different position. 
         [0016]    In one example embodiment, the sensor has the second state if the sensor is in a standby state. 
         [0017]    In one example embodiment, the instructions cause the processor to determine whether the amount of data includes the predetermined amount. 
         [0018]    In one example embodiment, the instructions cause the processor to determine whether the amount of data includes the predetermined amount by determining whether the amount of data is less than a maximum capacity amount. 
         [0019]    In one example embodiment, the microscope includes a sensor. In this example embodiment, the instructions, cause the processor to: (a) determine whether the stage has a first state; (b) determine whether the sensor has a second state; and (c) in response to: (i) the stage having the first state; and (ii) the sensor having the second state, cause the stage to move from the first position to the second, different position. 
         [0020]    In one example embodiment, the instructions cause the processor, in cooperation with another processor, to, for a designated captured image: (a) perform an input process; (b) thereafter, perform a developing process; (c) thereafter, perform a stitching process; (d) thereafter, perform an encoding process; and (e) thereafter, perform a storing process. 
         [0021]    In one example embodiment, the instructions cause the processor to issue a shutter-on command to the image pickup apparatus. 
         [0022]    In one example embodiment, an information processing system includes: (a) an image pickup apparatus operatively connected to a microscope which has a stage configured to support an observation target object, the stage being in a first position; and (b) a memory device having a buffer which has an amount of data, the method comprising: In this example embodiment, a method of operating the information processing system includes, in response to the amount of data including a predetermined amount, causing a processor to execute instructions to: (a) cause the stage to move from the first position to a second, different position; (b) cause the buffer to, while the stage is moving, store images which are associated with the observation target object, the images being continuously captured by the image pickup apparatus; and (c) thereafter, in response to the amount of data not including the predetermined amount, cause the image pickup apparatus to terminate the capture of the images. 
         [0023]    In one example embodiment, an information processing apparatus includes a processor operatively connected to an image pickup apparatus which is operatively connected to a microscope which has a stage configured to support an observation target object (i.e., a pathological specimen), the stage being in a first position, and a memory device operatively coupled to the processor, the memory device having a buffer which has an amount of data, the memory device storing instructions that cause the processor, in response to the amount of data including a predetermined amount, to: (a) cause the stage to move from the first position to a second, different position; (b) cause the buffer to, while the stage is moving, store images which are associated with the observation target object, the images being continuously captured by the image pickup apparatus; and (c) thereafter, in response to the amount of data not including the predetermined amount, cause the image pickup apparatus to terminate the capture of the images. 
         [0024]    In one example embodiment, the instructions cause the processor to: (a) determine whether the stage has a first state; and (b) in response to the stage having the first state, cause the stage to move from the first position to the second, different position. 
         [0025]    In one example embodiment, the microscope includes a sensor. In this example embodiment, the instructions cause the processor to: (a) determine whether the sensor has a second state; and (b) in response to the sensor having the second state, cause the stage to move from the first position to the second, different position. 
         [0026]    In one example embodiment, the instructions cause the processor to determine whether the amount of data includes the predetermined amount. In one example embodiment, the instructions cause the processor to determine whether the amount of data includes the predetermined amount by determining whether the amount of data is less than a maximum capacity amount. 
         [0027]    In one example embodiment, the microscope includes a sensor. In this example embodiment, the instructions cause the processor to: (a) determine whether the stage has a first state; (b) determine whether the sensor has a second state; and (c) in response to: (i) the stage having the first state; and (ii) the sensor having the second state, cause the stage to move from the first position to the second, different position. 
         [0028]    In one example embodiment, the instructions cause the processor, in cooperation with another processor, to, for a designated captured image: (a) perform an input process; (b) thereafter, perform a developing process; (c) thereafter, perform a stitching process; (d) thereafter, perform an encoding process; and (e) thereafter, perform a storing process. 
         [0029]    In one example embodiment, the instructions cause the processor to issue a shutter-on command to the image pickup apparatus. 
         [0030]    In one example embodiment, the instructions cause the processor, in cooperation with another processor, to, for a designated captured image, perform image processing. 
         [0031]    In one example embodiment, an information processing apparatus includes a processor operatively connected to an image pickup apparatus which is operatively connected to a microscope which has a stage configured to support an observation target object, the stage being in a first position, and a memory device operatively coupled to the processor, the memory device having a buffer which has an amount of data, the method comprising: In one example embodiment, a method of operating the information processing apparatus includes, in response to the amount of data including a predetermined amount, causing the processor to execute instructions to: (a) cause the stage to move from the first position to a second, different position; (b) cause the buffer to, while the stage is moving, store images which are associated with the observation target object, the images being continuously captured by the image pickup apparatus; and (c) thereafter, in response to the amount of data not including the predetermined amount, cause the image pickup apparatus to terminate the capture of the images. 
         [0032]    In one example embodiment, a computer-readable medium stores instructions structured to cause the information processing apparatus to, in response to the amount of data including a predetermined amount: (a) cause the stage to move from the first position to a second, different position; (b) cause the buffer to, while the stage is moving, store images which are associated with the observation target object, the images being continuously captured by the image pickup apparatus; and (c) thereafter, in response to the amount of data not including the predetermined amount, cause the image pickup apparatus to terminate the capture of the images. 
         [0033]    In one example embodiment, an image pickup apparatus includes a processor operatively connected to: (a) a microscope having a stage configured to support an observation target object (i.e., a pathological specimen), the stage being in a first position; and (b) an information processing apparatus having a first memory device having a buffer which has an amount of data. In this embodiment, the image pickup apparatus also includes a second memory device operatively coupled to the processor, the second memory device storing instructions that cause the processor, in cooperation with the second memory device, to, in response to the amount of data including a predetermined amount: (a) cause the stage to move from the first position to a second, different position; (b) cause the buffer to, while the stage is being moved, store images which are associated with the observation target object, the images being continuously captured by the image pickup apparatus; and (c) thereafter, in response to the amount of data not including the predetermined amount, cause the image pickup apparatus to terminate the capture of the images. 
         [0034]    In one example embodiment, the instructions cause the processor to determine whether the stage has a first state. In one example embodiment, the stage has the first state if the stage is not moving. 
         [0035]    In one example embodiment, the microscope includes a sensor. In this example embodiment, the instructions cause the processor to: (a) determine whether the sensor has a second state; and (b) in response to the sensor having the second state, cause the stage to move from the first position to the second, different position. 
         [0036]    In one example embodiment, the sensor has the second state if the sensor is in a standby state. 
         [0037]    In one example embodiment, the instructions cause the processor to determine whether the amount of data includes the predetermined amount. In one example embodiment, the instructions cause the processor to determine whether the amount of data includes the predetermined amount by determining whether the amount of data is less than a maximum capacity amount. 
         [0038]    In one example embodiment, the microscope includes a sensor. In one example embodiment, the instructions cause the processor to: (a) determine whether the stage has a first state; (b) determine whether the sensor has a second state; and (c) in response to: (i) the stage having the first state; and (ii) the sensor having the second state, cause the stage to move from the first position to the second, different position. 
         [0039]    In one example embodiment, the instructions cause the processor, in cooperation with another processor, to, for a designated captured image: (a) perform an input process; (b) thereafter, perform a developing process; (c) thereafter, perform a stitching process; (d) thereafter, perform an encoding process; and (e) thereafter, perform a storing process. 
         [0040]    In one example embodiment, an image pickup apparatus includes: (a) a processor operatively connected to: (i) a microscope having a stage configured to support an observation target object, the stage being in a first position; and (ii) an information processing apparatus having a first memory device having a buffer which has an amount of data; and (b) a second memory device operatively coupled to the processor. In one example embodiment, a method of operating the image pickup apparatus includes, in response to the amount of data including a predetermined amount, causing the processor to execute instructions to: (a) cause the stage to move from the first position to a second, different position; (b) cause the buffer to, while the stage is being moved, store images which are associated with the observation target object, the images being continuously captured by the image pickup apparatus; and (c) thereafter, in response to the amount of data not including the predetermined amount, cause the image pickup apparatus to terminate the capture of the images. 
         [0041]    As described above, according to the embodiments of the present disclosure, it is possible to process the image data in real time as much as possible while preventing the defect in the image data that is continuously transferred from the image pickup apparatus side to the image processing apparatus side. 
         [0042]    These and other objects, features and advantages of the present disclosure will become more apparent in light of the following detailed description of best mode embodiments thereof, as illustrated in the accompanying drawings. 
     
    
     
       BRIEF DESCRIPTION OF THE FIGURES 
         [0043]      FIG. 1  is a diagram showing the outline of an example image processing system according to an example embodiment of the present disclosure. 
           [0044]      FIG. 2  is a block diagram showing a hardware structure of an example image pickup apparatus and an interface therearound in the example image processing system according to the example embodiment of the present disclosure. 
           [0045]      FIG. 3  is a block diagram showing the hardware structure of an example PC according to the example embodiment of the present disclosure. 
           [0046]      FIG. 4  is a diagram showing the flow of image processing in the example PC according to the example embodiment of the present disclosure. 
           [0047]      FIG. 5  is a flowchart showing an image pickup process by the image pickup apparatus and the PC according to the example embodiment of the present disclosure. 
           [0048]      FIG. 6  is a block diagram showing an example control system in the image pickup process by the image pickup apparatus and the PC according to the example embodiment of the present disclosure. 
           [0049]      FIG. 7  is a timing chart showing an example image pickup timing and the like in the image pickup process of a system according to the example embodiment of the present disclosure. 
           [0050]      FIG. 8  is an example timing chart showing the case where a back pressure process is generated in the image pickup process shown in  FIG. 7 . 
           [0051]      FIG. 9  is an example timing chart showing an image pickup timing and the like in the image pickup process of a system according to another example embodiment of the present disclosure. 
           [0052]      FIG. 10  is an example timing chart showing the case where a back pressure process is generated in the image pickup process shown in  FIG. 9 . 
       
    
    
       [0053]    Additional features and advantages are described herein, and will be apparent from the following Detailed Description and the figures. 
       DETAILED DESCRIPTION 
       [0054]    Hereinafter, exemplary embodiments of the present disclosure will be described in detail with reference to the drawings. 
         [0055]      FIG. 1  is a diagram showing the outline of an example image processing system according to an example embodiment of the present disclosure. As shown in  FIG. 1 , the image processing system includes an image pickup apparatus  100  and a PC  200 . The image pickup apparatus  100  is connected with a microscope  300  to control an operation of the microscope  300  and read an image taken. 
         [0056]    The microscope  300  includes an XYZ stage  31 . On the XYZ stage  31 , a specimen  30  as an observation target object is placed movably in X, Y, and Z directions. The movement of the XYZ stage  31  is controlled by a stage control unit provided to the image pickup apparatus  100 . The specimen  30  is, for example, a pathological specimen  30 . A sliced tissue or organ of a human body is bonded on a slide glass, stained, and formed into a preparation form. 
         [0057]    On a lower-side area of the XYZ stage  31 , a flash light source  32  and an illumination optical system are provided. The flash light source  32  irradiates the specimen  30  with light for generating an optical image as an image pickup target. The illumination optical system collects light from the flash light source  32  to the specimen  30 . The image pickup apparatus  100  also controls the operation of the flash light source  32 . 
         [0058]    Above the XYZ stage  31 , an objective lens  33 , an image forming lens  34 , and a CMOS (complementary metal oxide semiconductor) substrate  35  are disposed in the stated order from the specimen  30  side. The objective lens  33  collects light that has passed through the specimen  30  to form an optical image of the specimen  30 . The optical image is formed on a CMOS sensor mounted on the CMOS substrate  35  by the image forming lens  34  through a light guiding optical system (not shown). 
         [0059]    The CMOS substrate  35  includes the CMOS sensor and an electronic shutter or a mechanical shutter that is interlocked with the flash light source  32 . The CMOS sensor continuously obtains, as image data, the formed optical images of the specimen  30  by performing photoelectric conversion each time the XYZ stage  31  is moved. Instead of the CMOS sensor, another image sensor such as a CCD (charge coupled device) image sensor may be used. 
         [0060]    The images as raw data sequentially obtained by the CMOS sensor are read by the image pickup apparatus  100  and sequentially transferred to the PC  200 . The image pickup apparatus  100  and the PC  200  are connected to each other via a PCIe bus  10  serving as a high-speed communication channel having bidirectionality and reliability. The PC  200  performs various image processing on the images to store the images, and displays the images after the image processing on a display  400  that is externally connected to the PC  200 . A manager of the PC  200  views the images that have been subjected to the image processing and are displayed on the display, thereby checking the quality thereof, for example. 
         [0061]    The PC  200  is connected with another PC (not shown) as a viewer via a network. An observer can view the image stored in the PC  200  on the display connected to the different PC, perform various editing processing or the like, and make a final pathological diagnosis or the like. 
         [0062]    [Hardware Structure of an Example Image Pickup Apparatus] 
         [0063]      FIG. 2  is a block diagram showing a hardware structure of the image pickup apparatus  100  and an interface therearound. 
         [0064]    As shown in  FIG. 2 , the image pickup apparatus  100  includes a camera control substrate  11  and a CMOS I/F (interface) substrate  14 . To the camera control substrate  11 , a cameral control unit  12  and a memory  13  are provided. The camera control unit  12  is structured as an FPGA (field programmable gate array), for example, and has a logic circuit therein. The memory (hereinafter, referred to as camera memory)  13  is a DRAM (dynamic random access memory) or the like, and functions as a buffer that stores an image read from a CMOS sensor  36  that is mounted on a CMOS control substrate  38 . 
         [0065]    On the camera control substrate  11  and the CMOS substrate  35 , Imager I/Fs  16  and  37  in conformity with a predetermined communication standard are provided, respectively. The camera control substrate  11  and the CMOS substrate  35  are connected through the CMOS I/F substrate  14  and cables  15 . The camera control unit  12  reads, through the cables  15 , an image taken by the CMOS sensor  36  and stores the image in the camera memory  13 . Further, under the control of the PC, the camera control unit  12  controls the operations of the CMOS sensor  36 , the XYZ stage  31 , and the flash light source  32 . A control box dedicated to the XYZ stage may be provided separately from the camera control substrate  11 . 
         [0066]    On the other hand, to the PC  200 , a memory (hereinafter, referred to as PC memory)  23  and a camera I/F substrate  29  for connection with the camera control substrate  11  are provided. As described above, the camera I/F substrate  29  of the PC  200  and the camera control substrate  11  of the image pickup apparatus  100  are connected with each other through the PCIe bus  10 . The camera control unit transfers the image stored in the camera memory  13  to the PC memory  23  of the PC  200  via the PCIe bus  10 . The PC memory  23  functions as a buffer for the image received. 
         [0067]    The transfer of the image between the camera memory  13  and the PC memory  23  is performed by a DMA (direct memory access). In addition, the camera memory  13 , the PCIe bus  10 , and the PC memory  23  are structured as an FIFO (first in, first out). That is, the camera memory  13  functions as a transmission FIFO buffer and the PC memory  23  functions as a reception FIFO buffer. 
         [0068]    Here, a bandwidth (band B in  FIG. 2 ) of the transmission using the PCIe bus  10  is larger than a bandwidth (band A in  FIG. 2 ) of the transmission using the cables  15  between the CMOS substrate  35  and the camera control substrate  11 . With this structure, bulk raw data of a pathological image or the like obtained by the CMOS sensor  36  is smoothly transferred to the PC  200  side. 
         [0069]    [Hardware Structure of an Example PC] 
         [0070]      FIG. 3  is a block diagram showing the hardware structure of the PC  200 . As shown in  FIG. 3 , the PC  200  includes CPUs  21  and  22  serving as a dual-core processor and the PC memory  23  and a PC memory  24  serving as the buffers corresponding to the CPUs  21  and  22 , respectively. In each of the PC memories  23  and  24 , a buffer for storing an image received from the image pickup apparatus  100  is provided. The buffer is formed as a redundant buffer such as a double buffer and a triple buffer. With this structure, even if the bulk raw data of the pathological image or the like is continuously transmitted from the image pickup apparatus  100  side via the PCIe bus  10 , the bulk raw data can be stored by the buffer, and hence it is possible to prevent an error in transmission to the extent possible. 
         [0071]    To the CPU  22 , two GPUs  25  and  26  are connected through a PCIe bridge  28 . For example, the GPU  25  performs a developing process on an image received from the image pickup apparatus  100 , and the GPU  26  performs an encoding process on the image to a JPEG format or the like (to be described later in detail). 
         [0072]    To the CPU  21 , the camera I/F substrate  29  and an HDD  20  are connected through a PCIe bridge  27 , for example. The image that has been received from the image pickup apparatus  100  and subjected to the developing process, the encoding process, and the like is stored in the HDD  20 . In addition, various applications necessary for the developing process, the encoding process, and the like are also stored in the HDD  20 . 
         [0073]    [Operation of an Example Image Processing System] 
         [0074]    Next, the operation of the image processing system structured as described above will be described. 
         [0075]    [Example Pipeline Process] 
         [0076]    First, various image processing in the PC  200  will be described.  FIG. 4  is a diagram showing a flow of the image processing. 
         [0077]    In this example embodiment, the PC  200  can pipeline, in addition to the developing process and the encoding process, an input process, a stitching process, a storing process, and the like and perform those processes with respect to the image received from the image pickup apparatus  100  side and stored in the PC memories  23  and  24 . The stitching process refers to a process of connecting a plurality of images to be one image. 
         [0078]    That is, as shown in  FIG. 4 , the CPU  21  performs the input process on an image (frame) from the PC memories  23  and  24 , for example. Subsequently, the GPU  25  performs the developing process on the input image, for example. Then, the CPU  22  performs the stitching process on the image after the developing process. Further, the GPU  26  performs the encoding (compressing) process on the image after the stitching process. In addition, the CPU  21  further performs the storing process of the image after the encoding process in the HDD  20 , after the CPU  21  is released from the input process. The combinations of those various processes with the processors that perform the processes are not of course limited to the example shown in  FIG. 4 . 
         [0079]    As described above, the PC  200  pipelines the five different processes with those parallel processors, for example, thereby making it possible to perform the image processing on the bulk raw data received via the PCIe bus  10  in real time as much as possible. As shown in  FIG. 4 , in the case where an input frame rate is set to Ts per frame, a latency from the input to the storage of one image (frame) becomes significantly small, specifically, 4*Ts, thanks to the pipeline process. 
         [0080]    [Example Image Pickup Control Process] 
         [0081]    Next, a description will be given on an image pickup process by the image pickup apparatus  100  and a control process by the PC  200  with respect to the image pickup process. Those processes are performed in parallel with the pipeline process.  FIG. 5  is a flowchart showing the image pickup process by the image pickup apparatus  100  and the PC  200 . Further,  FIG. 6  is a block diagram showing a control system in the image pickup process. 
         [0082]    As shown in  FIG. 5 , first, the CPU  21  or  22  (hereinafter, collectively referred to as CPU  21  for convenience) of the PC  200  judges whether the XYZ stage  31  is in a stop state, that is, the movement thereof is completed or not (Step  51 ). 
         [0083]    In the case where it is judged that the XYZ stage  31  is in the stop state (Yes), the CPU  21  judges whether the CMOS sensor  36  is in a standby state, that is, the CMOS sensor  36  is in an image-reading operation state to the camera memory  13  or not (Step  52 ). 
         [0084]    In the case where it is judged that the CMOS sensor  36  is in the standby state (Yes), the CPU  21  judges whether the buffer of the PC memory  23  or  24  (hereinafter, collectively referred to as PC memory  23  for convenience) has a free space, that is, the volume of data in the buffer is less than a predetermined capacity or not (Step  53 ). 
         [0085]    In the case where it is judged that the buffer of the PC memory  23  has the free space (Yes), the CPU  21  issues a shutter-on command to the camera control unit  12  and causes a shutter and the flash light source  32  to operate (Step  54 ). 
         [0086]    Subsequently, when the shutter is released in response to the shutter-on command, the camera control unit  12  starts the movement of the XYZ stage  31  for the next image pickup (Step  55 ). Further, the camera control unit  12  reads, from the CMOS sensor  36 , the image taken in response to the shutter-on command and stores the image in the camera memory  13  (Step  56 , ( 1 ) and ( 2 ) of  FIG. 6 ). 
         [0087]    The camera control unit  12  reads the image stored in the camera memory  13  and performs a DMA transfer of the image to the PC memory  23  of the PC  200  (Step  57 , ( 3 ) and ( 4 ) of  FIG. 6 ). 
         [0088]    In the PC  200 , the image stored in the PC memory  23  is read, and the pipeline process described above is performed with the CPU  21 , the GPU  25 , and the like, and various applications (( 5 ) of  FIG. 6 ). 
         [0089]    The image pickup apparatus  100  and the PC  200  repeatedly perform the above-mentioned processes until a specified number of images are taken and stored in the PC  200 . 
         [0090]    By the above-mentioned processes, in the image processing system according to this example embodiment, the movement of the XYZ stage  31  is completed during a time period when the image is read from the CMOS sensor  36  to the camera memory  13 . In addition, a shutter timing is synchronous with the start of the movement of the XYZ stage  31 . Further, the completion of the image reading process from the CMOS sensor  36  to the camera memory  13  is also synchronous with the shutter timing. As a result, a latency time is reduced by a time period necessary for the movement of the XYZ stage  31 , and therefore the image is taken and processed at a higher speed. 
         [0091]    In addition, in the image processing system according to this example embodiment, the shutter is released only when the three conditions that the PC memory  23  has the free space, the CMOS sensor  36  is in the standby state, and the XYZ stage  31  is in the stop state are met. Further, by this control, the PC  200  can perform a back pressure process of temporarily stopping the shutter operation by the image pickup apparatus  100  in the case where the PC memory  23  does not have the free space. 
         [0092]      FIG. 7  is a timing chart showing an image pickup timing and the like in the image pickup process.  FIG. 8  is a timing chart showing the case where the back pressure process is caused in the image pickup process. 
         [0093]    As shown in  FIGS. 6 and 7 , the camera control unit  12  monitors the state of the XYZ stage  31  and the CMOS sensor  36  and transmits, to the CPU  21  of the PC  200 , notification signals (stage movement state notification command and CMOS sensor operation completion notification command) for making notifications of the stop of the XYZ stage  31  and the completion of the reading operation from the CMOS sensor  36  (standby state) as interrupt signals. The CPU  21  processes the notification signals as interrupt signals and monitors the capacity of the buffer of the PC memory  23 . When the CPU  21  that a free space is generated (released) in the buffer, the CPU combines the conditions, to generate the shutter-on command. In synchronization with the issue of the shutter-on command, the processes of ( 1 ) to ( 5 ) of  FIG. 6  are performed. The PC  200  is provided with an interrupt controller (not shown) for performing the interrupt process. By processing, as the interrupt signals, the conditions for the generation of the shutter-on command, the PC  200  can more flexibly arbitrate the conditions. 
         [0094]    On the other hand, as shown in  FIG. 8 , in the case where a load is applied on the image processing by the CPU  21 , the GPU  25 , and the like, exceeding the predetermined capacity of the buffer of the PC memory  23 , the third condition for the generation of the shutter-on command is not met. Accordingly, even when the stage movement state notification command and the CMOS sensor operation completion notification command are notified of, the issue of the shutter-on command is suspended, with the result that the CPU  21  is brought into the back pressure state. 
         [0095]    In a normal state, which is not the back pressure state, a reading completion cycle from the CMOS sensor  36  is a critical path. In the back pressure state, a release of the buffer of the PC memory  23  is a critical path. 
         [0096]    As a result, even in the case where it may be impossible to store the images in the PC memory  23  due to the increase in the load of the pipeline process, the PC  200  can prevent the defect in the image data in a continuous transfer of the images through the PCIe bus  10 . 
         [0097]    As described above, the camera memory  13  of the image pickup apparatus  100  and the PC memories  23  and  24  of the PC  200  are structured as the FIFO. Therefore, handshaking therebetween is unnecessary in performing the back pressure process. The CPUs  21  and  22  of the PC  200  just manages a queue of the FIFO, thereby allowing the back pressure process to be smoothly performed irrespective of the kind or state of a parallelism. 
         [0098]    [Modified Example] 
         [0099]    The present disclosure is not limited to the above example embodiment and can be variously modified without departing from the gist of the present disclosure. 
         [0100]    In the above example embodiment, the PC  200  processes and arbitrates the conditions for the generation of the shutter-on command as the interrupt signals. However, the arbitrating process may be performed on the image pickup apparatus  100  side.  FIG. 9  is a timing chart showing an image pickup timing and the like in this case, and  FIG. 10  is a timing chart showing the case where the back pressure process is generated in the example of  FIG. 9 . 
         [0101]    As shown in  FIGS. 9 and 10 , in this example, an arbitration unit is provided to the image pickup apparatus  100 . The arbitration unit may be implemented as hardware or software. The arbitration unit monitors the operation state of the XYZ stage  31  and the CMOS sensor  36  to generate a condition relating thereto, and receives a notification signal as to a condition of a free space state of the PC memory  23  from the PC  200 . Further, the arbitration unit combines the conditions to generate the shutter-on command, thereby causing the camera control unit  12  to release the shutter. 
         [0102]    In addition, as shown in  FIG. 10 , even if the operation of the XYZ stage  31  and the CMOS sensor  36  is completed, the arbitration unit suspends the issue of the shutter-on command until the arbitration unit receives the notification signal relating to the conditions of the free space state of the PC memory  23  from the PC  200 , thereby performing the back pressure process. 
         [0103]    As described above, since the arbitration unit on the image pickup apparatus  100  side arbitrates the conditions for the generation of the shutter-on command, the handshaking between the image pickup apparatus  100  and the PC  200  is reduced, and the overhead on the PC  200  side is reduced. Further, since the shutter-on command is issued on the image pickup apparatus  100  side without the PC  200 , the latency until the shutter is released is reduced. 
         [0104]    Further, the generation processes of the conditions on the image pickup apparatus  100  side and on the PC  200  side may be combined. In other words, the condition that is desirable to be generated on the PC  200  side may be generated by the PC  200 , and the arbitration unit that combines the condition on the PC  200  side and the condition on the image pickup apparatus  100  side may be provided on the image pickup apparatus  100  side. With this structure, the flexible arbitration by the PC  200  and the reduction in the latency are compatible with each other. The condition to be generated on the PC  200  side may be arbitrarily changed as appropriate. 
         [0105]    The image pickup apparatus  100  and the PC  200  are connected by the PCIe in the above example embodiment, but may be connected by another general-purpose high-speed interface such as USB 3.0 instead. That is, any communication channel may be used for the transfer of the image from the image pickup apparatus to the PC, as long as the communication channel has a larger bandwidth than that used for the reading of the image from the CMOS sensor and has the bidirectional reliability.