An imaging and processing device, comprising: a separation section for separating visible light into at least a first color component and a second color component; a first imaging section for taking a moving image of the first color component, the first imaging section taking images which form the moving image with a first spatial resolution corresponding to a first number of pixels and a first temporal resolution by exposure for a first charge accumulation time period; a second imaging section for taking a moving image of the second color component, the second imaging section taking images which form the moving image with a second spatial resolution corresponding to a second number of pixels higher than the first spatial resolution and a second temporal resolution lower than the first temporal resolution by exposure for a second charge accumulation time period longer than the first charge accumulation time period; a control section for controlling imaging conditions of the first imaging section and the second imaging section; and a processing section for generating a moving image of the second color component having the temporal resolution thereof increased, based on information on the moving image of the first color component and information on the moving image of the second color component, characterized in that the processing section holds a relational expression of the moving image of the second color component taken by the second imaging section and the moving image of the second color component having the temporal resolution thereof increased, the relational expression comprising, as constraining conditions, temporal and spatial correspondence between the moving image of the first color component and the moving image of the second color component, and a condition for smoothness between pixels close to each other in the image; and the processing section is adapted for obtaining a moving image which minimizes the relational expression as the moving image of the second color component having the temporal resolution thereof increased. a separation section for separating visible light into at least a first color component and a second color component; a first imaging section for taking a moving image of the first color component, the first imaging section taking images which form the moving image with a first spatial resolution corresponding to a first number of pixels and a first temporal resolution by exposure for a first charge accumulation time period; a second imaging section for taking a moving image of the second color component, the second imaging section taking images which form the moving image with a second spatial resolution corresponding to a second number of pixels higher than the first spatial resolution and a second temporal resolution lower than the first temporal resolution by exposure for a second charge accumulation time period longer than the first charge accumulation time period; a control section for controlling imaging conditions of the first imaging section and the second imaging section; and a processing section for generating a moving image of the second color component having the temporal resolution thereof increased, based on information on the moving image of the first color component and information on the moving image of the second color component, characterized in that the processing section holds a relational expression of the moving image of the second color component taken by the second imaging section and the moving image of the second color component having the temporal resolution thereof increased, the relational expression comprising, as constraining conditions, temporal and spatial correspondence between the moving image of the first color component and the moving image of the second color component, and a condition for smoothness between pixels close to each other in the image; and the processing section is adapted for obtaining a moving image which minimizes the relational expression as the moving image of the second color component having the temporal resolution thereof increased.