A fluorescence observation device comprising: excitation light supplying means supplying excitation light for fluorescence observation of an object to be observed and being capable of switching between ON/OFF of the excitation light supply; an interlaced imaging device taking an optical image from the object to be observed, and alternately outputting first field images and second field images in time series as obtained image data of the object to be observed; image storage means storing the first field image or the second field image output from the imaging device; and difference image generating means generating a difference image obtained by a difference between one of the first field image and the second field image output from the imaging device, and the other of the first field image and the second field image stored in the image storage means, wherein the excitation light supplying means supplies the excitation light such that one of the first field image acquiring period and the second field image acquiring period by the imaging device is to be a period in which the excitation light supply is turned ON to acquire a fluorescence image, and the other is to be a period in which the excitation light supply is turned OFF to acquire a background image, and the difference image generating means, as its generation mode, switches between a first mode of, in a case where the fluorescence image is output from the imaging device, generating the difference image by subtracting the background image, which is acquired in advance of the fluorescence image to be stored in the image storage means, from the fluorescence image, and a second mode of, in a case where the background image is output from the imaging device, generating the difference image by subtracting the background image from the fluorescence image, which is acquired in advance of the background image to be stored in the image storage means. excitation light supplying means supplying excitation light for fluorescence observation of an object to be observed and being capable of switching between ON/OFF of the excitation light supply; an interlaced imaging device taking an optical image from the object to be observed, and alternately outputting first field images and second field images in time series as obtained image data of the object to be observed; image storage means storing the first field image or the second field image output from the imaging device; and difference image generating means generating a difference image obtained by a difference between one of the first field image and the second field image output from the imaging device, and the other of the first field image and the second field image stored in the image storage means, wherein the excitation light supplying means supplies the excitation light such that one of the first field image acquiring period and the second field image acquiring period by the imaging device is to be a period in which the excitation light supply is turned ON to acquire a fluorescence image, and the other is to be a period in which the excitation light supply is turned OFF to acquire a background image, and the difference image generating means, as its generation mode, switches between a first mode of, in a case where the fluorescence image is output from the imaging device, generating the difference image by subtracting the background image, which is acquired in advance of the fluorescence image to be stored in the image storage means, from the fluorescence image, and a second mode of, in a case where the background image is output from the imaging device, generating the difference image by subtracting the background image from the fluorescence image, which is acquired in advance of the background image to be stored in the image storage means.