An image following information detecting device (10, 20) comprising: an image display system including an image light projecting unit that projects image light, and a transmissive reflection member that reflects a part of the image light projected from the image light projecting unit toward an eyeball and transmits other parts; and a line-of-sight detection system including at least one invisible light source (211, 211a, 211b), a diffractive optical element provided integrally with the transmissive reflection member and including a reflective diffraction portion (212c1) that reflects and diffracts invisible light from the invisible light source toward the eyeball, and a light receiving element that receives the invisible light reflected by the eyeball and reflected and diffracted by the reflective diffraction portion, wherein a portion irradiated with a central portion of the image light of the transmissive reflection member and the reflective diffraction portion overlap each other when viewed from a thickness direction of the transmissive reflection member; characterized in that the diffractive optical element includes a plurality of the reflective diffraction portions stacked in a thickness direction and having different diffraction wavelengths and sizes, and the line-of-sight detection system includes: a pupil diameter acquisition unit that acquires a pupil diameter of the eyeball; a plurality of the invisible light sources (211, 211a, 211b) having different emission wavelengths; and a lighting control unit that can selectively light the plurality of invisible light sources on a basis of an acquisition result in the pupil diameter acquisition unit. an image display system including an image light projecting unit that projects image light, and a transmissive reflection member that reflects a part of the image light projected from the image light projecting unit toward an eyeball and transmits other parts; and a line-of-sight detection system including at least one invisible light source (211, 211a, 211b), a diffractive optical element provided integrally with the transmissive reflection member and including a reflective diffraction portion (212c1) that reflects and diffracts invisible light from the invisible light source toward the eyeball, and a light receiving element that receives the invisible light reflected by the eyeball and reflected and diffracted by the reflective diffraction portion, wherein a portion irradiated with a central portion of the image light of the transmissive reflection member and the reflective diffraction portion overlap each other when viewed from a thickness direction of the transmissive reflection member; characterized in that the diffractive optical element includes a plurality of the reflective diffraction portions stacked in a thickness direction and having different diffraction wavelengths and sizes, and the line-of-sight detection system includes: a pupil diameter acquisition unit that acquires a pupil diameter of the eyeball; a plurality of the invisible light sources (211, 211a, 211b) having different emission wavelengths; and a lighting control unit that can selectively light the plurality of invisible light sources on a basis of an acquisition result in the pupil diameter acquisition unit.