Patent ID: 12207875

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments of an eye movement analysis system, an eye movement analysis method, and a program of the present invention will be described with reference to the drawings.

First Embodiment

FIG.1is a diagram showing an example of the configuration of an eye movement analysis system1of a first embodiment.FIGS.2A and2Bare diagrams showing a first application example of the eye movement analysis system1of the first embodiment. In detail,FIG.2Ashows an example of the relationship between the eye movement analysis system1and a subject SB in the first application example, andFIG.2Bshows an example of the relationship between the right eye R and the left eye L of the subject SB and a comparison target M.

In the example shown inFIG.1, the eye movement analysis system1of the first embodiment includes an acquisition unit11, a display unit12, an image-capturing unit13, an analysis unit14, and an output unit15.

The acquisition unit11acquires information of the comparison target M attached to the surface of the face of the subject SB (seeFIG.2A). The comparison target M is a sticker attached between eyebrows of the subject SB.

In another example (for example, an example in which the image-capturing unit13captures the images of only one eye of the subject SB and the comparison target M), the comparison target M may be attached to a place other than the place between the eyebrows of the subject SB (for example, a place directly below the eye to be captured).

In the examples shown inFIGS.1,2A and2B, the comparison target M is circular. The information of the comparison target M acquired by the acquisition unit11includes the actual size of the diameter or radius of the comparison target M.

In another example, the comparison target M may have a shape other than circular.

In the example shown inFIG.1, the display unit12displays an optotype.

In another example, the eye movement analysis system1may not include the display unit12. In this example (for example, the example shown inFIGS.2A and2B), the image-capturing unit13functions as the optotype presented to the subject SB.

In the example shown inFIG.1, the image-capturing unit13captures a moving image including the right eye R and the left eye L of the subject SB (seeFIG.2A) and the comparison target M.

The analysis unit14analyzes the movements of the eyeballs of the right eye R and left eye L of the subject SB, based on the information of the comparison target M acquired by the acquisition unit11and the relationship between the right eye R and left eye L of the subject SB and the comparison target M on the moving image captured by the image-capturing unit13.

Specifically, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the actual size (the unit is, for example, millimeters) of the comparison target M acquired by the acquisition unit11, the dimensions (the unit is, for example, pixel) of the comparison target M on the moving image captured by the image-capturing unit13, and the distance (the unit is, for example, pixel) between the right eye R of the subject SB and the comparison target M on the moving image captured by the image-capturing unit13. Further, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the actual size (the unit is, for example, millimeters) of the comparison target M acquired by the acquisition unit11, the dimensions (the unit is, for example, pixel) of the comparison target M on the moving image captured by the image-capturing unit13, and the distance (the unit is, for example, pixel) between the left eye L of the subject SB and the comparison target M on the moving image captured by the image-capturing unit13.

The output unit15outputs the analysis result by the analysis unit14. Specifically, the output unit15outputs the analysis result of the movement of the eyeball of the right eye R and the analysis result of the movement of the eyeball of the left eye L of the subject SB.

In the example shown inFIG.2A, the output unit15displays the analysis result by the analysis unit14, but in another example, the output unit15may print the analysis result by the analysis unit14.

FIG.3is a flowchart for explaining an example of a process executed in the eye movement analysis system1of the first embodiment.

In the example shown inFIG.3, in step S1, the acquisition unit11acquires the information of the comparison target M attached to the surface of the face of the subject SB.

Further, in step S2, the display unit12displays an optotype.

Next, in step S3, the image-capturing unit13captures a moving image including the right eye R and the left eye L of the subject SB and the comparison target M.

Next, in step S4, based on the information of the comparison target M acquired in step S1and the relationship between the right eye R and left eye L of the subject SB and the comparison target M on the moving image captured in step S3, the movements of the eyeballs of the right eye R and left eye L of the subject SB are analyzed.

Next, in step S5, the output unit15outputs the result of the analysis executed in step S4.

In the example shown inFIG.3, step S2is executed in the eye movement analysis system1, but in other examples, step S2may not be executed.

FIGS.4A and4Bare diagrams for explaining a first example of analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment. In detail,FIG.4Ashows the relationship between the right eye R of the subject SB and the comparison target M on the moving image at time t11, among the moving images captured by the image-capturing unit13.FIG.4Bshows the relationship between the right eye R of the subject SB and the comparison target M on the moving image at time t12after the time t11, among the moving images captured by the image-capturing unit13.

In the examples shown inFIGS.4A and4B, the acquisition unit11acquires the actual size D1[mm] of the diameter of the comparison target M. The actual size D1[mm] is, for example, 5 [mm] to 10 [mm].

Further, the diameter of the comparison target M on the moving image at the time t11captured by the image-capturing unit13is D11[pixel]. Further, the distance between the center (area centroid) RP11of the pupil RP of the right eye R of the subject SB and the center (area centroid) M11of the comparison target M on the moving image at the time t11captured by the image-capturing unit13is L11[pixel].

Further, the diameter of the comparison target M on the moving image at the time t12captured by the image-capturing unit13is D12[pixel]. Further, the distance between the center (area centroid) RP12of the pupil RP of the right eye R of the subject SB and the center (area centroid) M12of the comparison target M on the moving image at the time t12captured by the image-capturing unit13is L12[pixel].

In the examples shown inFIGS.4A and4B, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the actual size D1[mm] of the diameter of the comparison target M acquired by the acquisition unit11, the diameters D11[pixel] and D12[pixel] of the comparison target M on the moving image captured by the image-capturing unit13, and the distances L11[pixel] and L12[pixel] between the centers (area centroids) RP11and RP12of the pupil RP of the right eye R of the subject SB and the centers (area centroid) M11and M12of the comparison target M on the moving image captured by the image-capturing unit13.

Specifically, the analysis unit14calculates the actual distance (L11[pixel]×D1[mm]/D11[pixel]) between the center (area centroid) RP11of the pupil RP of the right eye R of the subject SB and the center (area centroid) M11of the comparison target M at the time t11.

Further, the analysis unit14calculates the actual distance (L12[pixel]×D1[mm]/D12[pixel]) between the center (area centroid) RP12of the pupil RP of the right eye R of the subject SB and the center (area centroid) M12of the comparison target M at the time t12.

Further, the analysis unit14further calculates, for example, the actual movement amount (L12[pixel]×D1[mm]/D12[pixel]−L11[pixel]×D1[mm]/D11[pixel]) of the center (area centroid) of the pupil RP of the right eye R of the subject SB during the period from time t11to time t12.

Further, the analysis unit14further calculates, for example, the actual movement speed (L12[pixel]×D1[mm]/D12[pixel]−L11[pixel]×D1[mm]/D11[pixel])/(t12−t11)) of the center (area centroid) of the pupil RP of the right eye R of the subject SB during the period from time t11to time t12.

By performing such an operation, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB (for example, the movement amount (amplitude) and the movement speed of the eyeball of the right eye R due to nystagmus).

That is, in the examples shown inFIGS.4A and4B, the eye movement analysis system1of the first embodiment does not need to use the Purkinje image, does not need to perform calibration in advance, and can analyze the movement of the eyeball of the right eye R of the subject SB noninvasively and highly accurately, even when the distance between the image-capturing unit13and the subject SB changes (for example, when the face of the subject SB moves).

In the examples shown inFIGS.4A and4B, the center (area centroid) of the pupil RP is used as a point on the right eye R of the subject SB on the moving image captured by the image-capturing unit13. However, in another example, as a point on the right eye R of the subject SB on the moving image captured by the image-capturing unit13, a point other than the center (area centroid) of the pupil RP (for example, a characteristic point on the iris RR (not shown), characteristic points on conjunctival blood vessels (not shown), or the like) may be used.

In the examples shown inFIGS.4A and4B, the diameter of the comparison target M is used as the dimension of the comparison target M, but in other examples, the radius of the comparison target M may be used as the dimension of the comparison target M.

In the first example of the analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, in the same manner as the examples shown inFIGS.4A and4B.

Specifically, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the actual size of the diameter or radius of the comparison target M acquired by the acquisition unit11, the diameter or radius of the comparison target M on the moving image captured by the image-capturing unit13, and the distance between the point on the left eye L of the subject SB (for example, the center (area centroid) of the pupil of the left eye L) and the center (area centroid) of the comparison target M on the moving image captured by the image-capturing unit13.

In the first example of the analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment, a high-speed infrared video camera (300 fps) is used as the image-capturing unit13.

In another example, a camera other than the high-speed infrared video camera may be used as the image-capturing unit13.

Further, in the first example of the analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment, the moving image captured by the high-speed infrared video camera as the image-capturing unit13is binarized by the analysis unit14, for example, as shown inFIG.2A.

In another example, the analysis unit14may perform another any image processing on the moving image captured by the image-capturing unit13.

FIGS.5A and5Bare diagrams for explaining a second example of analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment. In detail,FIG.5Ashows the relationship between the right eye R of the subject SB and the comparison target M on the moving image at time t21, among the moving images captured by the image-capturing unit13.FIG.5Bshows the relationship between the right eye R of the subject SB and the comparison target M on the moving image at time t22after the time t21, among the moving images captured by the image-capturing unit13.

In the examples shown inFIGS.5A and5B, the acquisition unit11acquires the actual size D2[mm] of the diameter of the comparison target M. The actual size D2[mm] is, for example, 5 [mm] to 10 [mm].

Further, the diameter of the comparison target M on the moving image at the time t21captured by the image-capturing unit13is D21[pixel]. Further, the distance between the center (area centroid) of the pupil RP of the right eye R of the subject SB and the center (area centroid) M21of the comparison target M on the moving image at the time t21captured by the image-capturing unit13is L21[pixel].

Further, the diameter of the comparison target M on the moving image at the time t22captured by the image-capturing unit13is D21[pixel] as at the time t21.

That is, in the examples shown inFIGS.5A and5B, the distance between the image-capturing unit13and the subject SB does not change between the time t21and the time t22.

Further, the distance between the center (area centroid) of the pupil RP of the right eye R of the subject SB and the center (area centroid) M22of the comparison target M on the moving image at the time t22captured by the image-capturing unit13is L21[pixel], as at the time t21.

That is, in the examples shown inFIGS.5Aand B, the distance between the center (area centroid) of the pupil RP of the right eye R of the subject SB and the centers (area centroids) M21and M22of the comparison target M does not change between time t21and time t22.

In the examples shown inFIGS.5A and5B, in order to analyze the movement of the eyeball of the right eye R of the subject SB, the distance between the points E21and E31on the iris RR of the right eye R of the subject SB and the centers (area centroids) M21and M22of the comparison target M on the moving image captured by the image-capturing unit13is used.

That is, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the actual size D2[mm] of the diameter of the comparison target M acquired by the acquisition unit11, the diameter D21[pixel] of the comparison target M on the moving image captured by the image-capturing unit13, and the distance between the points E21and E31on the iris RR of the right eye R of the subject SB and the centers (area centroids) M21and M22of the comparison target M on the moving image captured by the image-capturing unit13.

Specifically, in the examples shown inFIGS.5A and5B, the two-dimensional coordinates of the center (area centroid) M21of the comparison target M at the time t21on the moving image captured by the image-capturing unit13is set to the origin (0,0). Further, the two-dimensional coordinates of the point E21on the iris RR of the right eye R of the subject SB at the time t21on the moving image captured by the image-capturing unit13are (X21, Y21), and the two-dimensional coordinates of the point E31on which are (X31, Y31).

Next, the two-dimensional coordinates of the center (area centroid) M22of the comparison target M at the time t22on the moving image captured by the image-capturing unit13is set to the origin (0,0). Further, the two-dimensional coordinates of the point E21on the iris RR of the right eye R of the subject SB at the time t22on the moving image captured by the image-capturing unit13are (X22, Y22)(≠(X21, Y21)), and the two-dimensional coordinates of the point E31on which are (X32, Y32)(≠(X31, Y31)).

That is, in the examples shown inFIGS.5A and5B, based on the diameter D21[pixel] of the comparison target M at the times t21and t22on the moving image captured by the image-capturing unit13, the two-dimensional coordinates (X21, Y21) of the point E21and the two-dimensional coordinates (X31, Y31) of the point E31on the iris RR of the right eye R of the subject SB at the time t21on the moving image captured by the image-capturing unit13, the two-dimensional coordinates (X22, Y22) of the point E21and the two-dimensional coordinates (X32, Y32) of the point E31on the iris RR of the right eye R of the subject SB at the time t22on the moving image captured by the image-capturing unit13, the analysis unit14can analyze that the eyeball of the right eye R of the subject SB has rotated counterclockwise as shown by the arrow inFIG.5B, although the distance between the center (area centroid) of the pupil RP of the right eye R of the subject SB and the centers (area centroids) M21and M22of the comparison target M does not change between time t21and time t22.

In the examples shown inFIGS.5A and5B, the distance between the image-capturing unit13and the subject SB does not change between the time t21and the time t22, but even when the distance between the image-capturing unit13and the subject SB changes between the time t21and the time t22, by using the information of the diameter (unit is [pixel]) of the comparison target M on the moving image at time t21and the information of the diameter (unit is [pixel]) of the comparison target M on the moving image at time t22, it can be analyzed that the eyeball of the right eye R of the subject SB has rotated, as in the examples shown inFIGS.5A and5B.

In the examples shown inFIGS.5A and5B, the diameter of the comparison target M is used as the dimension of the comparison target M, but in other examples, the radius of the comparison target M may be used as the dimension of the comparison target M.

In the second example of the analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, in the same manner as the examples shown inFIGS.5A and5B.

Specifically, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the actual size of the diameter or radius of the comparison target M acquired by the acquisition unit11, the diameter or radius of the comparison target M on the moving image captured by the image-capturing unit13, and the distance between the point on the iris of the left eye L of the subject SB and the center (area centroid) of the comparison target M on the moving image captured by the image-capturing unit13.

That is, in the second example of the analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment, the analysis unit14can analyze that the eyeball of the left eye L of the subject SB has rotated, based on the diameter (unit is [pixel]) of the comparison target M at different time points on the moving image captured by the image-capturing unit13, and the two-dimensional coordinates of two points on the iris of the left eye L of the subject SB at the different time points on the moving image captured by the image-capturing unit13.

FIGS.6A and6Bare diagrams for explaining a third example of analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment. In detail,FIG.6Ashows the relationship between the right eye R of the subject SB and the comparison target M on the moving image at time t31, among the moving images captured by the image-capturing unit13.FIG.6Bshows the relationship between the right eye R of the subject SB and the comparison target M on the moving image at time t32after the time t31, among the moving images captured by the image-capturing unit13.

In the examples shown inFIGS.6A and6B, the shape of the comparison target M is polygonal (for example, rectangular). The acquisition unit11acquires the actual size S1[mm] of the distance between the vertex M11and the vertex M12of the comparison target M. The actual size S1[mm] is, for example, 5 [mm] to 10 [mm].

Further, the two-dimensional coordinates of the vertex M11of the comparison target M at the time t31on the moving image captured by the image-capturing unit13are set to the origin (0,0). Further, the two-dimensional coordinates of the vertex M12of the comparison target M at the time t31on the moving image captured by the image-capturing unit13are (X2, Y2). Further, the two-dimensional coordinates of the point E41on the iris RR of the right eye R of the subject SB at the time t31on the moving image captured by the image-capturing unit13are (X41, Y41).

Next, the two-dimensional coordinates of the vertex M11of the comparison target M at the time t32on the moving image captured by the image-capturing unit13are set to the origin (0, 0). Further, the two-dimensional coordinates of the vertex M12of the comparison target M at the time t32on the moving image captured by the image-capturing unit13are (X2, Y2) as at the time t31.

That is, in the examples shown inFIGS.6A and6B, the distance between the image-capturing unit13and the subject SB does not change between the time t31and the time t32.

Further, the two-dimensional coordinates of the point E41on the iris RR of the right eye R of the subject SB at the time t32on the moving image captured by the image-capturing unit13are (X42, Y42).

In the examples shown inFIGS.6A and6B, in order to analyze the movement of the eyeball of the right eye R of the subject SB, the actual size S1[mm] of the distance between the vertex M11and the vertex M12of the comparison target M acquired by the acquisition unit11, the dimension of the comparison target M on the moving image captured by the image-capturing unit13(specifically, the dimension (distance) between the vertex M11and the vertex M12obtained from the two-dimensional coordinates (0,0) of the vertex M11and the two-dimensional coordinates (X2, Y2) of the vertex M12), and the positional relationship between the point E41on the iris RR of the right eye R of the subject SB and the comparison target M on the moving image captured by the image-capturing unit13are used.

That is, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the actual size S1[mm] of the distance between the vertex M11and the vertex M12of the comparison target M acquired by the acquisition unit11, the dimension of the comparison target M on the moving image captured by the image-capturing unit13(specifically, the dimension (distance) between the vertex M11and the vertex M12obtained from the two-dimensional coordinates (0,0) of the vertex M11and the two-dimensional coordinates (X2, Y2) of the vertex M12), and the positional relationship between the point E41on the iris RR of the right eye R of the subject SB and the comparison target M on the moving image captured by the image-capturing unit13.

Specifically, in the examples shown inFIGS.6A and6B, the two-dimensional coordinates of the point E41on the iris RR of the right eye R of the subject SB at the time t31on the moving image captured by the image-capturing unit13are (X41, Y41).

Further, the two-dimensional coordinates of the point E41on the iris RR of the right eye R of the subject SB at the time t32on the moving image captured by the image-capturing unit13are (X42, Y42)(≠(X41, Y41)).

That is, in the examples shown inFIGS.6A and6B, based on the relative coordinates (X2, Y2) of the vertex M12relative to the vertex M11of the comparison target M at the times t31and t32on the moving image captured by the image-capturing unit13, the two-dimensional coordinates (X41, Y41) of the point E41on the iris RR of the right eye R of the subject SB at the time t31on the moving image captured by the image-capturing unit13, and the two-dimensional coordinates (X42, Y42) of the point E41on the iris RR of the right eye R of the subject SB at the time t32on the moving image captured by the image-capturing unit13, the analysis unit14can analyze, between the time t31and time t32, not only that the distance between the eyeball of the right eye R of the subject SB and the comparison target M has changed (increased), but also that as shown by the arrow inFIG.6B, the eyeball of the right eye R of the subject SB has moved toward the upper left ofFIG.6B(that is, the direction of movement of the eyeball).

That is, in the examples shown inFIGS.6A and6B, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the actual size S1[mm] of the distance between the vertex M11and the vertex M12of the polygon of the comparison target M acquired by the acquisition unit11, the distance ((X22+Y22)1/2[pixel]) between the vertex M11and the vertex M12of the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point E41on the iris RR of the right eye R of the subject SB and the vertices M11and M12of the polygon of the comparison target M at the time t31and time t32on the moving image captured by the image-capturing unit13.

In the examples shown inFIGS.6A and6B, the distance between the image-capturing unit13and the subject SB does not change between the time t31and the time t32, but when the distance between the image-capturing unit13and the subject SB changes between the time t31and the time t32, the direction of the movement of the eyeball of the right eye R of the subject SB can be analyzed by using the information of the two-dimensional coordinates of the vertex M12of the comparison target M on the moving image at time t31and the information of the two-dimensional coordinates of the vertex M12of the comparison target M on the moving image at time t32, in the same manner as the example shown inFIGS.6A and6B.

In the third example of the analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, in the same manner as the examples shown inFIGS.6A and6B.

Specifically, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the actual size S1[mm] of the distance between the vertex M11and the vertex M12of the polygon of the comparison target M acquired by the acquisition unit11, the distance (unit is [pixel]) between the vertex M11and the vertex M12of the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point (not shown) on the iris of the left eye L of the subject SB at the time t31and time t32and the vertices M11and M12of the polygon of the comparison target M on the moving image captured by the image-capturing unit13.

That is, in the third example of the analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment, the analysis unit14can analyze the direction of movement of the eyeball of the left eye L of the subject SB, based on the distance (unit is [pixel]) between the vertex M11and the vertex M12of the comparison target M at different time points on the moving image captured by the image-capturing unit13, and the two-dimensional coordinates of the point on the iris of the left eye L of the subject SB at different time points on the moving image captured by the image-capturing unit13.

FIGS.7A and7Bare diagrams for explaining a fourth example of analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment. In detail,FIG.7Ashows the relationship between the right eye R of the subject SB and the comparison target M on the moving image at time t41, among the moving images captured by the image-capturing unit13.FIG.7Bshows the relationship between the right eye R of the subject SB and the comparison target M on the moving image at time t42after the time t41, among the moving images captured by the image-capturing unit13.

In the examples shown inFIGS.7A and7B, the shape of the comparison target M is polygonal (for example, rectangular). The acquisition unit11acquires the actual size S1[mm] of the distance between the vertex M21and the vertex M22, connecting the vertex M21and the vertex M22of the comparison target M. The actual size S1[mm] is, for example, 5 [mm] to 10 [mm].

Further, the two-dimensional coordinates of the vertex M21of the comparison target M at the time t41on the moving image captured by the image-capturing unit13are set to the origin (0,0). Further, the two-dimensional coordinates of the vertex M22of the comparison target M at the time t41on the moving image captured by the image-capturing unit13are (X3, Y3). Further, the two-dimensional coordinates of the point E51on the iris RR of the right eye R of the subject SB at the time t41on the moving image captured by the image-capturing unit13are (X51, Y51), and the two-dimensional coordinates of the point E61on which are (X61, Y61).

Next, the two-dimensional coordinates of the vertex M21of the comparison target M at the time t42on the moving image captured by the image-capturing unit13are set to the origin (0, 0). Further, the two-dimensional coordinates of the vertex M22of the comparison target M at the time t42on the moving image captured by the image-capturing unit13are (X3, Y3) as at the time t41.

That is, in the examples shown inFIGS.7A and7B, the distance between the image-capturing unit13and the subject SB does not change between the time t41and the time t42.

Further, the two-dimensional coordinates of the point E51on the iris RR of the right eye R of the subject SB at the time t42on the moving image captured by the image-capturing unit13are (X52, Y52), and the two-dimensional coordinates of the point E61on which are (X62, Y62).

In the examples shown inFIGS.7A and7B, in order to analyze the movement of the eyeball of the right eye R of the subject SB, the actual size S1[mm] of the distance between the vertex M21and the vertex M22of the comparison target M acquired by the acquisition unit11, the dimension of the comparison target M on the moving image captured by the image-capturing unit13(specifically, the dimension (distance) between the vertex M21and the vertex M22obtained from the two-dimensional coordinates (0,0) of the vertex M21and the two-dimensional coordinates (X3, Y3) of the vertex M22), and the positional relationship between the points E51and E61on the iris RR of the right eye R of the subject SB and the comparison target M on the moving image captured by the image-capturing unit13are used.

That is, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the actual size S1[mm] of the side length of the comparison target M acquired by the acquisition unit11, the dimension of the comparison target M on the moving image captured by the image-capturing unit13(specifically, the dimension (distance) between the vertex M21and the vertex M22obtained from the two-dimensional coordinates (0,0) of the vertex M21and the two-dimensional coordinates (X3, Y3) of the vertex M22), and the positional relationship between the points E51, E61on the iris RR of the right eye R of the subject SB and the comparison target M on the moving image captured by the image-capturing unit13.

Specifically, in the examples shown inFIGS.7A and7B, the two-dimensional coordinates of the point E51on the iris RR of the right eye R of the subject SB at the time t41on the moving image captured by the image-capturing unit13are (X51, Y51).

Further, the two-dimensional coordinates of the point E51on the iris RR of the right eye R of the subject SB at the time t42on the moving image captured by the image-capturing unit13are (X52, Y52) (≠(X51, Y51)).

Further, the two-dimensional coordinates of the point E61on the iris RR of the right eye R of the subject SB at the time t41on the moving image captured by the image-capturing unit13are (X61, Y61).

Further, the two-dimensional coordinates of the point E61on the iris RR of the right eye R of the subject SB at the time t42on the moving image captured by the image-capturing unit13are (X62, Y62) (≠(X61, Y61)).

That is, in the examples shown inFIGS.7A and7B, based on the relative coordinates (X3, Y3) of the vertex M22relative to the vertex M21of the comparison target M at the times t41and t42on the moving image captured by the image-capturing unit13, the two-dimensional coordinates (X51, Y51) of the point E51on the iris RR of the right eye R of the subject SB at the time t41on the moving image captured by the image-capturing unit13, the two-dimensional coordinates (X52, Y52) of the point E51on the iris RR of the right eye R of the subject SB at the time t42on the moving image captured by the image-capturing unit13, the two-dimensional coordinates (X61, Y61) of the point E61on the iris RR of the right eye R of the subject SB at the time t41on the moving image captured by the image-capturing unit13, and the two-dimensional coordinates (X62, Y62) of the point E61on the iris RR of the right eye R of the subject SB at the time t42on the moving image captured by the image-capturing unit13, the analysis unit14can analyze that, between the time41and time42, the distance between the eyeball of the right eye R of the subject SB and the comparison target M has not changed, but as shown by the arrow A1inFIG.7B, the eyeball of the right eye R of the subject SB has swirled clockwise around the comparison target M (that is, the direction of movement of the eyeball), and as shown by the arrow A2inFIG.7B, the eyeball of the right eye R of the subject SB has rotated counterclockwise.

That is, in the examples shown inFIGS.7Aand B, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the actual size S1[mm] of the distance between the vertex M21and the vertex M22of the polygon of the comparison target M acquired by the acquisition unit11, the distance ((X32+Y32)1/2[pixel]) between the vertex M21and the vertex M22of the comparison target M on the moving image captured by the image-capturing unit13, the positional relationship between the point E51on the iris RR of the right eye R of the subject SB and the vertices M21and M22of the polygon of the comparison target M at the time t41and time t42on the moving image captured by the image-capturing unit13, and the positional relationship between the point E61on the iris RR of the right eye R of the subject SB and the vertices M21and M22of the polygon of the comparison target M at the time t41and time t42on the moving image captured by the image-capturing unit13.

In the examples shown inFIGS.7A and7B, the distance between the image-capturing unit13and the subject SB does not change between the time t41and the time t42, but even when the distance between the image-capturing unit13and the subject SB changes between the time t41and the time t42, the direction of the movement of the eyeball of the right eye R of the subject SB and the rotation of the eyeball can be analyzed by using the information of the two-dimensional coordinates of the vertex M22of the comparison target M on the moving image at time t41and the information of the two-dimensional coordinates of the vertex M22of the comparison target M on the moving image at time t42, as in the example shown inFIGS.7A and7B.

In the fourth example of the analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, in the same manner as the examples shown inFIGS.7A and7B.

Specifically, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the actual size S1[mm] of the distance between the vertex M21and the vertex M22of the polygon of the comparison target M acquired by the acquisition unit11, the distance (unit is [pixel]) between the vertex M21and the vertex M22of the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between two points (not shown) on the iris of the left eye L of the subject SB and the vertices M21and M22of the polygon of the comparison target M at the time t41and time t42on the moving image captured by the image-capturing unit13.

That is, in the fourth example of the analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment, the analysis unit14can analyze the direction of movement of the eyeball and the rotation of the eyeball of the left eye L of the subject SB, based on the distance (unit is [pixel]) between two vertices M21and M22of the comparison target M at different time points on the moving image captured by the image-capturing unit13, and the two-dimensional coordinates of the two points on the iris of the left eye L of the subject SB at different time points on the moving image captured by the image-capturing unit13.

As described above, in the first to fourth examples of the analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment, the acquisition unit11acquires information of the actual size D1[mm] of the diameter of the comparison target M, but in the fifth example of the analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment, the acquisition unit11may not acquire information of the actual size D1[mm] of the diameter of the comparison target M.

In the fifth example of the analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment, the movement of the eyeball of the right eye R of the subject SB (for example, whether or not the movement has been done, the direction of movement, or the like) is analyzed, by comparing the distance between a plurality of points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the comparison target M and the right eye R of the subject SB on the moving image captured by the image-capturing unit13at a plurality of time points.

Further, in the fifth example of the analysis executed by the analysis unit14of the eye movement analysis system1of the first embodiment, the movement of the eyeball of the left eye L of the subject SB (for example, whether or not the movement has been done, the direction of movement, or the like) is analyzed, by comparing the distance between a plurality of points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the comparison target M and the left eye L of the subject SB on the moving image captured by the image-capturing unit13at a plurality of time points.

That is, in the fifth example of the analysis performed by the analysis unit14of the eye movement analysis system1of the first embodiment, it is possible to obtain the time waveforms of the eye movements of the right eye R and the left eye L of the subject SB, and it can be used for simple screening.

In a fifth example of the analysis performed by the analysis unit14of the eye movement analysis system1of the first embodiment, the morphological features on the surface of the face of the subject SB (for example, areas where the position changes little due to the change in the facial expression of the subject SB, such as moles, ears, nose, outer corners of eyes, and inner corners of eyes) are used as the comparison target M.

Specifically, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the relationship between the right eye R of the subject SB and the two points on the morphological feature (comparison target M) on the moving image captured by the image-capturing unit13. Specifically, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the distance between the two points on the comparison target M (morphological feature) on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the comparison target M (morphological feature) on the moving image captured by the image-capturing unit13.

Further, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the relationship between the left eye L of the subject SB and the two points on the morphological feature (comparison target M) on the moving image captured by the image-capturing unit13. Specifically, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the distance between the two points on the comparison target M (morphological feature) on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the left eye L of the subject SB and the two points on the comparison target M (morphological feature) on the moving image captured by the image-capturing unit13.

When the morphological feature on the surface of the face of the subject SB is used as the comparison target M, the acquisition unit11may acquire the distance (actual size) between the two points on the morphological feature (comparison target M). In this example, the distance (actual size) between two points on the morphological feature (comparison target M) is obtained by measurement with a caliper, for example. In yet another example, by using the autofocus function of the image-capturing unit13, the distance (actual size) between the two points on the morphological feature (comparison target M) may be estimated, based on the distance between the image-capturing unit13and the subject SB.

FIGS.8A-8Hare diagrams for explaining another example of the comparison target M.

In the example shown inFIG.8A, the comparison target M is an elliptical sticker. The information of the comparison target M (elliptical sticker) acquired by the acquisition unit11includes the actual size of the distance between two points on the comparison target M (for example, one of the actual size of the major axis AC, the actual size of the semimajor axis (½ of the major axis AC), the actual size of the minor axis BD, and the actual size of semiminor axis (½ of minor axis BD).

In the example shown inFIG.8A, in order to analyze the movement of the eyeball of the right eye R of the subject SB, the actual size of the distance between the two points on the comparison target M acquired by the acquisition unit11, the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13are used.

That is, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the actual size of the distance between the two points on the comparison target M acquired by the acquisition unit11, the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

Similarly, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the actual size of the distance between the two points on the comparison target M acquired by the acquisition unit11, the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the left eye L of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

In the example in which the acquisition unit11does not acquire the actual size of the distance between the two points on the comparison target M, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

Further, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the left eye L of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

For example, in the example in which the comparison target M is attached to the subject SB such that the major axis AC shown inFIG.8Ais included in the vertical line (that is, the minor axis BD is included in the horizontal line), the analysis unit14can determine whether or not the inclination of the face of the subject SB has changed, based on the orientation of the comparison target M (oval sticker) on the moving image captured by the image-capturing unit13. Specifically, the analysis unit14can determine the inclination of the face of the subject SB, based on the positional relationship between points A and C (or positional relationship between points B and D) of the comparison target M (elliptical sticker) on the moving image captured by the image-capturing unit13.

That is, in this example, an elliptical sticker as the comparison target M is attached to the subject SB such that the inclination of the face of the subject SB can be determined from the moving image captured by the image-capturing unit13.

Further, in this example, the analysis unit14determines the inclination of the face of the subject SB from the moving image captured by the image-capturing unit13by using two points on the elliptical sticker.

In the example shown inFIG.8B, the comparison target M is a sticker having an arrow symbol indicating the top and bottom. The information of the comparison target M (sticker having the arrow symbol) acquired by the acquisition unit11includes the actual size of the distance between the two points on the arrow symbol of the comparison target M.

In the example shown inFIG.8B, in order to analyze the movement of the eyeball of the right eye R of the subject SB, the actual size of the distance between the two points on the arrow symbol of the comparison target M acquired by the acquisition unit11, the distance between the two points on the arrow symbol of the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the arrow symbol of the comparison target M on the moving image captured by the image-capturing unit13are used.

That is, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the actual size of the distance between the two points on the arrow symbol of the comparison target M acquired by the acquisition unit11, the distance between the two points on the arrow symbol of the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the arrow symbol of the comparison target M on the moving image captured by the image-capturing unit13.

Similarly, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the actual size of the distance between the two points on the arrow symbol of the comparison target M acquired by the acquisition unit11, the distance between the two points on the arrow symbol of the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the left eye L of the subject SB and the two points on the arrow symbol of the comparison target M on the moving image captured by the image-capturing unit13.

In the example in which the acquisition unit11does not acquire the actual size of the distance between the two points on the arrow symbol of the comparison target M, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the distance between the two points on the arrow symbol of the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the arrow symbol of the comparison target M on the moving image captured by the image-capturing unit13.

Further, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the distance between the two points on the arrow symbol of the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the left eye L of the subject SB and the two points on the arrow symbol of the comparison target M on the moving image captured by the image-capturing unit13.

For example, in the example in which the comparison target M is attached to the subject SB such that the arrow symbol shown inFIG.8Bis located on the vertical line (that is, the comparison target M is arranged as shown inFIG.8B), the analysis unit14can determine whether or not the inclination of the face of the subject SB has changed, based on the orientation of the comparison target M (the sticker having the arrow symbol) on the moving image captured by the image-capturing unit13. Specifically, the analysis unit14can determine the inclination of the face of the subject SB, based on the orientation of the arrow symbol of the comparison target M on the moving image captured by the image-capturing unit13.

That is, in this example, a sticker having an arrow symbol as the comparison target M is attached to the subject SB such that the inclination of the face of the subject SB can be determined from the moving image captured by the image-capturing unit13.

Further, in this example, the analysis unit14determines the inclination of the face of the subject SB from the moving image captured by the image-capturing unit13, by using two points on the arrow symbol of the sticker.

In the example shown inFIG.8C, the comparison target M is composed of two stickers. The information of the comparison target M acquired by the acquisition unit11includes the actual size (for example, measured by a caliper) of the interval between the two stickers constituting the comparison target M.

In the example shown inFIG.8C, in order to analyze the movement of the eyeball of the right eye R of the subject SB, the actual size of the interval between the two stickers constituting the comparison target M acquired by the acquisition unit11, the interval between the two stickers constituting the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two stickers constituting the comparison target M on the moving image captured by the image-capturing unit13are used.

That is, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the actual size of the interval between the two stickers constituting the comparison target M acquired by the acquisition unit11, the interval between the two stickers constituting the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two stickers constituting the comparison target M on the moving image captured by the image-capturing unit13.

Similarly, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the actual size of the interval between the two stickers constituting the comparison target M acquired by the acquisition unit11, the interval between the two stickers constituting the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the left eye L of the subject SB and the two stickers constituting the comparison target M on the moving image captured by the image-capturing unit13.

In the example in which the acquisition unit11does not acquire the actual size of the interval between the two stickers constituting the comparison target M, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the interval between the two stickers constituting the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two stickers constituting the comparison target M on the moving image captured by the image-capturing unit13.

Further, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the interval between the two stickers constituting the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the left eye L of the subject SB and the two stickers constituting the comparison target M on the moving image captured by the image-capturing unit13.

For example, in the example in which the two stickers constituting the comparison target M are attached to the subject SB such that the two stickers shown inFIG.8Care located on the vertical line (that is, the two stickers are arranged as shown inFIG.8C), the analysis unit14can determine whether or not the inclination of the face of the subject SB has changed, based on the arrangement (orientation) of the two stickers on the moving image captured by the image-capturing unit13. Specifically, the analysis unit14can determine the inclination of the face of the subject SB, based on the arrangement (orientation) of the two stickers on the moving image captured by the image-capturing unit13.

That is, in this example, the two stickers constituting the comparison target M are attached to the subject SB such that the inclination of the face of the subject SB can be determined from the moving image captured by the image-capturing unit13.

Further, in this example, the analysis unit14determines the inclination of the face of the subject SB from the moving image captured by the image-capturing unit13by using, for example, the center point of each of the two stickers.

In the example shown inFIG.8D, the comparison target M is a rectangular sticker. The information of the comparison target M (rectangular sticker) acquired by the acquisition unit11includes the actual size of the distance between two points (two vertices of a rectangle) on the comparison target M (for example, the actual size of the side AB, the actual size of the side DC, the actual size of the side AD, the actual size of the side BC, and the like).

In the example shown inFIG.8D, in order to analyze the movement of the eyeball of the right eye R of the subject SB, the actual size of the distance between the two points on the comparison target M acquired by the acquisition unit11, the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13are used.

That is, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the actual size of the distance between the two points on the comparison target M acquired by the acquisition unit11, the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

Similarly, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the actual size of the distance between the two points on the comparison target M acquired by the acquisition unit11, the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the left eye L of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

In the example in which the acquisition unit11does not acquire the actual size of the distance between the two points on the comparison target M, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

Further, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the left eye L of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

For example, in the example in which the comparison target M is attached to the subject SB such that the side AB shown inFIG.8Dis included in the vertical line (that is, the side AD is included in the horizontal line), the analysis unit14can determine whether or not the inclination of the face of the subject SB has changed, based on the orientation of the comparison target M (rectangular sticker) on the moving image captured by the image-capturing unit13. Specifically, the analysis unit14can determine the inclination of the face of the subject SB, based on the positional relationship between vertices A and B (or positional relationship between vertices A and D) of the comparison target M (rectangular sticker) on the moving image captured by the image-capturing unit13.

That is, in this example, a rectangular sticker as the comparison target M is attached to the subject SB such that the inclination of the face of the subject SB can be determined from the moving image captured by the image-capturing unit13.

Further, in this example, the analysis unit14determines the inclination of the face of the subject SB from the moving image captured by the image-capturing unit13, by using two rectangular vertices A and B (or vertices A and D).

In the example shown inFIG.8E, the comparison target M is a diamond-shaped sticker. The information of the comparison target M (diamond-shaped sticker) acquired by the acquisition unit11includes the actual size of the distance between two points (two vertices of the diamond shape) on the comparison target M (for example, the actual size of the diagonal line AC, the actual size of the diagonal line BD, and the like).

In the example shown inFIG.8E, in order to analyze the movement of the eyeball of the right eye R of the subject SB, the actual size of the distance between the two points on the comparison target M acquired by the acquisition unit11, the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13are used.

That is, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the actual size of the distance between the two points on the comparison target M acquired by the acquisition unit11, the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

Similarly, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the actual size of the distance between the two points on the comparison target M acquired by the acquisition unit11, the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the left eye L of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

In the example in which the acquisition unit11does not acquire the actual size of the distance between the two points on the comparison target M, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

Further, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the left eye L of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

For example, in the example in which the comparison target M is attached to the subject SB such that the diagonal line AC shown inFIG.8Eis included in the vertical line (that is, the diagonal line BD is included in the horizontal line), the analysis unit14can determine whether or not the inclination of the face of the subject SB has changed, based on the orientation of the comparison target M (the diamond-shaped sticker) on the moving image captured by the image-capturing unit13. Specifically, the analysis unit14can determine the inclination of the face of the subject SB, based on the positional relationship between vertices A and C (or positional relationship between vertices B and D) of the comparison target M (diamond-shaped sticker) on the moving image captured by the image-capturing unit13.

That is, in this example, a diamond-shaped sticker as the comparison target M is attached to the subject SB such that the inclination of the face of the subject SB can be determined from the moving image captured by the image-capturing unit13.

Further, in this example, the analysis unit14determines the inclination of the face of the subject SB from the moving image captured by the image-capturing unit13, by using two diamond-shaped vertices A and C (or vertices B and D).

In the examples shown inFIGS.8F and8G, the comparison target M is a triangular sticker. The information of the comparison target M (triangular sticker) acquired by the acquisition unit11includes the actual size of the distance between two points (two vertices of the triangle) on the comparison target M (for example, the actual size of the side BC).

In the examples shown inFIGS.8F and8G, in order to analyze the movement of the eyeball of the right eye R of the subject SB, the actual size of the distance between the two points on the comparison target M acquired by the acquisition unit11, the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

That is, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the actual size of the distance between the two points on the comparison target M acquired by the acquisition unit11, the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

Similarly, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the actual size of the distance between the two points on the comparison target M acquired by the acquisition unit11, the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the left eye L of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

In the example in which the acquisition unit11does not acquire the actual size of the distance between the two points on the comparison target M, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

Further, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the left eye L of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

For example, in an example in which the comparison target M is attached to the subject SB such that the side BC shown inFIG.8Fis included in the horizontal line and the vertex A is located above the side BC, or, for example, in an example in which the comparison target M is attached to the subject SB such that the side BC shown inFIG.8Gis included in the horizontal line and the vertex A is located below the side BC, the analysis unit14can determine whether the inclination of the face of the subject SB has changed, based on the orientation of the comparison target M (triangular sticker) on the moving image captured by the image-capturing unit13. Specifically, the analysis unit14can determine the inclination of the face of the subject SB, based on the positional relationship between the vertex A and the side BC of the comparison target M (triangular sticker) on the moving image captured by the image-capturing unit13.

That is, in this example, a triangular sticker as the comparison target M is attached to the subject SB such that the inclination of the face of the subject SB can be determined from the moving image captured by the image-capturing unit13.

Further, in this example, the analysis unit14determines the inclination of the face of the subject SB from the moving image captured by the image-capturing unit13, by using the vertex A and the side BC of the triangle.

In the example shown inFIG.8H, the comparison target M is a sticker of a figure (for example, a character figure) in which a plurality of straight lines, curves, and the like are combined. The information of the comparison target M (figure sticker) acquired by the acquisition unit11includes the actual size of the distance between two points on the comparison target M (for example, the right eye and the left eye of the character shown inFIG.8H).

In the example shown inFIG.8H, in order to analyze the movement of the eyeball of the right eye R of the subject SB, the actual size of the distance between the two points on the comparison target M acquired by the acquisition unit11, the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13are used.

That is, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the actual size of the distance between the two points on the comparison target M acquired by the acquisition unit11, the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

Similarly, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the actual size of the distance between the two points on the comparison target M acquired by the acquisition unit11, the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the left eye L of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

In the example in which the acquisition unit11does not acquire the actual size of the distance between the two points on the comparison target M, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the right eye R of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

Further, the analysis unit14analyzes the movement of the eyeball of the left eye L of the subject SB, based on the distance between the two points on the comparison target M on the moving image captured by the image-capturing unit13, and the positional relationship between the point on the left eye L of the subject SB and the two points on the comparison target M on the moving image captured by the image-capturing unit13.

For example, in the example in which the comparison target M is attached to the subject SB such that the right eye and the left eye of the character shown inFIG.8Hare located on the horizontal line (that is, the comparison target M is arranged as shown inFIG.8H), the analysis unit14can determine whether or not the inclination of the face of the subject SB has changed, based on the orientation of the comparison target M on the moving image captured by the image-capturing unit13. Specifically, the analysis unit14can determine the inclination of the face of the subject SB, based on the orientation of the face of the character of the comparison target M on the moving image captured by the image-capturing unit13.

That is, in this example, the sticker of the figure as the comparison target M is attached to the subject SB such that the inclination of the face of the subject SB can be determined from the moving image captured by the image-capturing unit13.

Further, in this example, the analysis unit14determines the inclination of the face of the subject SB from the moving image captured by the image-capturing unit13by using two points on the comparison target M.

FIGS.9Aand B are diagrams showing a second application example of the eye movement analysis system1of the first embodiment. In detail,FIG.9Ashows an example of the relationship between the eye movement analysis system1and the subject SB in the horizontal plane, andFIG.9Bshows an example of the relationship between the eye movement analysis system1and the subject SB in the vertical plane.

In the example shown inFIGS.9Aand B, the eye movement analysis system1of the first embodiment includes a moving unit16in addition to the acquisition unit11, the display unit12, the image-capturing unit13, the analysis unit14, and the output unit15.

The moving unit16integrally moves the display unit12and the image-capturing unit13. Specifically, the moving unit16has a function of integrally moving the display unit12and the image-capturing unit13such that the distance between the display unit12and the image-capturing unit13and the subject SB does not change.

That is, as shown inFIG.9A, the moving unit16can integrally move the display unit12and the image-capturing unit13along an arc centered on the subject SB (specifically, an arc on a horizontal plane). Further, as shown inFIG.9B, the moving unit16can integrally move the display unit12and the image-capturing unit13along an arc centered on the subject SB (specifically, an arc on the vertical plane).

Further, the moving unit16can integrally move the display unit12and the image-capturing unit13along the surface of a sphere centered on the subject SB (specifically, a sphere having the same radius as the arc shown inFIGS.9A and9B).

As described above, in the first application example and the second application example of the eye movement analysis system1of the first embodiment, the acquisition unit11, the analysis unit14, and the output unit15of the eye movement analysis system1are configured by, for example, a personal computer as shown inFIG.2A, the display unit12of the eye movement analysis system1is configured by a monitor (not shown), and the image-capturing unit13of the eye movement analysis system1is configured by, for example, a camera as shown inFIG.2A.

In the third application example of the eye movement analysis system1of the first embodiment, the eye movement analysis system1is configured by, for example, a mobile terminal device (not shown) such as a smartphone or the like. Specifically, the acquisition unit11and the analysis unit14of the eye movement analysis system1are configured by a built-in computer (not shown) of a mobile terminal device. Further, the display unit12and the output unit15of the eye movement analysis system1are configured by a display (not shown) of a mobile terminal device. Further, the image-capturing unit13of the eye movement analysis system1is composed of a built-in camera (not shown) of the mobile terminal device.

That is, in the third application example of the eye movement analysis system1of the first embodiment, it can be used as an application in conjunction with a built-in camera such as a smartphone.

Second Embodiment

Hereinafter, a second embodiment of the eye movement analysis system, the eye movement analysis method, and the program of the present invention will be described.

The eye movement analysis system1of the second embodiment is configured in the same manner as the eye movement analysis system1of the first embodiment described above, except for the points described later. Therefore, according to the eye movement analysis system1of the second embodiment, the same effect as the effect of the eye movement analysis system1of the first embodiment described above can be obtained except for the points described later.

As described above, in the eye movement analysis system1of the first embodiment, the image-capturing unit13captures a moving image including the right eye R and the left eye L of the subject SB (seeFIG.2A) and the comparison target M.

On the other hand, in the eye movement analysis system1of the second embodiment, the image-capturing unit13captures a moving image including the right eye R of the subject SB and the comparison target M, or a moving image including the left eye L of the subject SB and the comparison target M.

As described above, in the eye movement analysis system1of the first embodiment, the analysis unit14analyzes the eyeball movements of the right eye R and left eye L of the subject SB, based on the information of the comparison target M and the moving image including the right eye R and left eye L of the subject SB and the comparison target M.

On the other hand, in the eye movement analysis system1of the second embodiment, the analysis unit14analyzes the movement of the eyeball of the right eye R of the subject SB, based on the information of the comparison target M and the moving image including the right eye R of the subject SB and the comparison target M, or analyzes the movement of the eyeball of the left eye L of the subject SB, based on the information of the comparison target M and the moving image including the left eye L of the subject SB and the comparison target M.

In the process executed in the eye movement analysis system1of the second embodiment, in step S1ofFIG.3, the acquisition unit11acquires the information of the comparison target M attached to the surface of the face of the subject SB.

Further, in step S2ofFIG.3, the display unit12displays an optotype.

Next, in step S3ofFIG.3, the image-capturing unit13captures a moving image including, for example, the right eye R of the subject SB and the comparison target M.

Next, in step S4ofFIG.3, based on the information of the comparison target M acquired in step S1and the moving image including the right eye R of the subject SB and the comparison target M captured in step S3, the movement of the eyeball of the right eye R of the subject SB is analyzed.

Next, in step S5ofFIG.3, the output unit15outputs the result of the analysis of the movement of the eyeball of the right eye R of the subject SB executed in step S4.

It is considered that the nystagmus analysis method using the eye movement analysis system, the eye movement analysis method and the program of the present invention becomes the standard of the nystagmus analysis method and can greatly contribute to the elucidation of the pathological condition of the nystagmus.

Further, according to the eye movement analysis system, the eye movement analysis method, and the program of the present invention, detailed nystagmus analysis in children becomes possible. By using the eye movement analysis system, eye movement analysis method, and program of the present invention, even an adult subject can undergo a nystagmus test that is simpler and much less burdensome than a nystagmus measuring device in the related art.

That is, according to the eye movement analysis system, the eye movement analysis method, and the program of the present invention, it is possible to significantly change the method of measuring nystagmus and the way of medical treatment of nystagmus.

Although the embodiments of the present invention have been described in detail with reference to the drawings, the specific configuration is not limited to these embodiments and can be appropriately modified without departing from the spirit of the present invention. The configurations described in each of the above-described embodiments and examples may be combined.

It should be noted that all or a part of the functions of each unit included in the eye movement analysis system1in the above-described embodiment may be realized by recording a program for realizing these functions on a computer-readable recording medium, loading the program recorded on the computer-readable recording medium into a computer system, and executing the program. The term “computer system” as used herein includes hardware such as an OS and peripheral devices.

Further, the “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, or a CD-ROM, or a storage unit such as a hard disk built in a computer system. Further, a “computer-readable recording medium” may include those which dynamically hold programs for a short period of time, such as a communication line when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, or those which hold programs for a certain period of time, such as a volatile memory inside a computer system that serves as a server or client in that case. Further, the above-described program may be a program for realizing a part of the above-described functions, and may be a program for realizing the above-described functions in combination with a program already recorded in the computer system.

REFERENCE SIGNS LIST

1: Eye movement analysis system11: Acquisition unit12: Display unit13: Image-capturing unit14: Analysis unit15: Output unit16: Moving unitSB: SubjectR: Right eyeRP: PupilRR: IrisL: Left eyeM: Comparison target