Patent ID: 12245864

DETAILED DESCRIPTION OF THE EMBODIMENTS

Embodiments of the present disclosure will be described in detail with reference to the drawings. In the drawings, identical or equivalent components are identically denoted and will not be described redundantly.

First Embodiment

An ocular movement data processing system and an imaging device according to a first embodiment will be described with reference to the drawings.FIG.1is a schematic diagram showing a configuration of an ocular movement data processing system10according to the first embodiment.FIGS.2A and2Bare schematic diagrams for illustrating a configuration of an imaging device400according to the embodiment.FIG.3is a block diagram generally showing a configuration of ocular movement data processing system10according to the first embodiment.

An operator1can diagnose vertigo of a subject2with ocular movement data processing system10. Note that an “operator” may be any person who uses ocular movement data processing system10, such as doctors belonging to clinics, general hospitals, university hospitals and the like; teachers, student and the like of medical colleges; and the like. It should be noted that the operator may belong not only to medical departments of ophthalmology, otorhinolaryngology or the like specialized for treatment of vertigo, but also other medical departments such as internal medicine and dentistry. A “subject” may be any person to be diagnosed through ocular movement data processing system10, such as a patient of a clinic, a general hospital or a university hospital, or a subject in a medical college. “Vertigo” includes a state of subject2who suffers some abnormality in his/her vision, such as rotary vertigo causing a vision to spin around, floating dizziness causing a floating sensation, and syncopic dizziness causing a vision to black out.

As shown inFIG.1, ocular movement data processing system10according to the first embodiment comprises a data processing device100. A display300, an imaging device400, a keyboard501, and a mouse502are connected to data processing device100. Display300is an example of a display device. Keyboard501and mouse502are an example of an input device.

In general, vertigo is diagnosed through observation of nystagmus (an involuntary movement of a rhythmically moving eyeball). Nystagmus includes spontaneous nystagmus that occurs spontaneously with no stimulation applied, and evoked nystagmus caused when stimulation is applied. Further, evoked nystagmus includes positional nystagmus evoked when a head position is displaced, and a positioning nystagmus evoked when a body position is displaced. For evoked nystagmus, it is known that when a physiological rotational stimulus or the like is applied to the head, in particular, the eyeballs move opposite to the head in order to stabilize the field of view, and such a phenomenon is also referred to as vestibulo ocular reflex (VOR).

Specifically, in ocular movement data processing system10, in order to observe subject2for nystagmus, imaging device400images the eyeballs of subject2, and data processing device100processes, stores, and displays the image data. Accordingly, data processing device100is connected to imaging device400. Imaging device400is a goggle-shaped device mounted on the head of subject2, and captures an image of the eyeballs of subject2and obtains image data of an ocular movement for use in diagnosis of vertigo. As shown inFIG.1, while subject2has imaging device400mounted on his/her head, operator1performs a nystagmus examination and thus obtains image data of ocular movement of subject2, and inputs the obtained image data to data processing device100. Data processing device100processes the image data obtained by imaging device400and provides operator1with information necessary for diagnosis of vertigo.

Imaging device400shown inFIG.2Ais in a state such that it has a front side with a shading cover402attached thereto. A wiring403is provided on an upper surface of housing401for connection to data processing device100. Note that imaging device400may be connected to data processing device100not only via a wire but also wirelessly insofar as a sufficient transmission rate is ensured for transmission of image data.

Imaging device400shown inFIG.2Bis shown disassembled into an upper housing401a, a lower housing401b, and an eyepiece401cbrought into contact with subject2. Lower housing401bis provided with a first imaging unit411that is an infrared imaging device that captures an image of the right eye of subject2, and a second imaging unit412that is an infrared imaging device that captures an image of the left eye of subject2. Although not shown, upper housing401ais provided with an operation processing unit420shown inFIG.3. While in the present disclosure a configuration in which first imaging unit411and second imaging unit412are provided for imaging device400will be described, the imaging device may be provided with a single imaging unit and the single imaging unit may be used to capture an image of one or both eyes.

Eyepiece401chas an opening401dsuch that first and second imaging units411and412can image the eyeballs of subject2while subject2is covered in front of his/her eyes. Eyepiece401cis formed of synthetic resin or soft rubber having appropriate flexibility and elasticity so as to be in close contact with the face of subject2when the imaging device is mounted on the head of subject2.

Shading cover402is provided with a magnet, for example, and easily detachably attachable to imaging device400. When shading cover402is detached from imaging device400, subject2can see ahead through a hot mirror410and thus see an index or the like emitted from a visual stimulation signal processing device600. Hot mirror410is an optical component that is a glass or resin plate coated with a material which transmits visible light and reflects infrared light to obtain an infrared image of an eyeball of the subject while ensuring a field of view for the subject. First and second imaging units411and412capture an image of the eyeballs of subject2reflected by hot mirror410.

In imaging device400, as shown inFIG.3, image data A from first imaging unit411and image data B from second imaging unit412are processed by operation processing unit420and transmitted to data processing device100as image data C. First imaging unit411includes an infrared imaging device, and a processing circuit (not shown) that attaches information of a frame number and a time stamp (first information) to an image that is captured by the infrared imaging device for each frame to provide image data A and output image data A to operation processing unit420. The information included in the first information is not limited to a frame number and a time stamp, and may be information of at least one of the frame number and the time stamp, and may include information such as a frame rate, an amount of exposure, and a contrast. Similarly, second imaging unit412includes an infrared imaging device, and a processing circuit (not shown) that attaches information of a frame number and a time stamp (second information) to an image that is captured by the infrared imaging device for each frame to provide image data B and output image data B to operation processing unit420. The information included in the second information is not limited to a frame number and a time stamp, and may be information of at least one of the frame number and the time stamp, and may include information such as a frame rate, an amount of exposure, and a contrast. The first information and the second information are also referred to as information provided for a captured image.

First and second imaging units411and412can capture an image at 60 frames/sec or 240 frames/sec. The infrared imaging device used for first and second imaging units411and412is, for example, a CMOS (Complementary Metal Oxide Semiconductor) sensor, a CCD (Charge Coupled Device), or the like capable of capturing an infrared ray.

Operation processing unit420performs operation-processing, that is, generates image data C by synchronizing together an image captured by first imaging unit411(a first image), an image captured by second imaging unit412(a second image) and information of visual stimulation signal processing device600and additional device700. In the present disclosure, visual stimulation signal processing device600and additional device700may collectively be referred to as an external device. Accordingly, operation processing unit420is a computing entity that performs processing of image data, and it is an example of a computer and for example is composed of a central processing unit (CPU), an field-programmable gate array (FPGA), or the like. Furthermore, operation processing unit420includes memories such as a random access memory (RAM) for storing images and the like and a read only memory (ROM) for storing programs and the like. In addition to the configuration in which operation processing unit420is executed as a control unit (a controller) that synchronizes together the image captured by first imaging unit411, the image captured by second imaging unit412, and an external signal received from the external device, operation processing unit420also has a configuration in which operation processing unit420is executed as a communication unit (a communication circuit) that externally transmits the synchronized images. The external signal from the external device includes a vital signal from a vital sign detection device, a visual stimulation signal from the visual stimulation signal processing device, and the like.

Operation processing unit420may use a synchronization signal as a method for synchronizing together the image captured by first imaging unit411, the image captured by second imaging unit412, and the external signal received from the external device. Specifically, operation processing unit420transmits a synchronization signal α to each of first and second imaging units411and412. First and second imaging units411and412can use synchronization signal α as a start signal to start an operation (exposure→obtaining a signal→transmission) to obtain their respectively captured first and second images in synchronization. When first and second imaging units411and412are configured to have no memory for storing two or more pieces of image data, and one imaging unit fails to obtain a signal, operation processing unit420can only obtain one piece of image data, and will thus never obtain unsynchronized image data. Operation processing unit420transmits a synchronization signal β to visual stimulation signal processing device600. Synchronization signal α and synchronization signal β are set to be of the same timing, and operation processing unit420synchronizes in accordance with synchronization signal β the external signal from the external device with the first and second images synchronized by synchronization signal α. Additional device700may be directly connected to operation processing unit420and operation processing unit420may transmit synchronization signal β to additional device700.

When first and second imaging units411and412and visual stimulation signal processing device600have a configuration to provide images and information, respectively, with a time stamp, then, there is a method for synchronization based on the time stamp added to the images and information. The time stamp is generated based on a time counted by a counter of each of first imaging unit411, second imaging unit412, and visual stimulation signal processing device600, and in order to use the time stamp to synchronize images and information, it is necessary to synchronize the time counted by each counter. Accordingly, operation processing unit420transmits synchronization signal α to each of first and second imaging units411and412and transmits synchronization signal β to visual stimulation signal processing device600, and, based on synchronization signals α and β, first and second imaging units411and412and visual stimulation signal processing device600synchronize and thus adjust times counted by the counters and add a time stamp to their images and information. By synchronizing the image captured by first imaging unit411, the image captured by second imaging unit412, and the external signal received from the external device together based on time stamps adjusted by synchronization signals α and β, operation processing unit420can reliably obtain a right eye image and a left eye image of the same timing and can also refer to information of visual stimulation signal processing device600and that of additional device700of the same timing together. In particular, when first and second imaging units411and412capture images at 240 frames/sec, simply correlating the images output from the respective imaging units with each other does not provide a right eye image and a left eye image of the same timing and appropriate diagnosis cannot be performed.

Visual stimulation signal processing device600is a laser device as shown inFIG.3, and generates a laser point on a screen and thus displays the laser point as a visual target. Visual stimulation signal processing device600can transmit and receive data to and from data processing device100, and also receives a synchronization signal β from imaging device400. Visual stimulation signal processing device600includes a synchronization unit601and a processing unit602. Based on synchronization signal β received from imaging device400, synchronization unit601synchronizes and adjusts time counted by a counter and provides a time stamp to processing unit602.

Furthermore, visual stimulation signal processing device600is connectable to an additional device700such as a vital sign detection device. Visual stimulation signal processing device600receives an analog signal of a vital sign (a heart rate, a respiration rate, a blood pressure value, a pulse rate, etc.) from additional device700, and converts the analog signal into a vital signal of a digital signal by processing unit602. Visual stimulation signal processing device600includes the vital signal in an external signal of visual stimulation signal processing device600, adds a time stamp of synchronization unit601to the external signal, and outputs the external signal with the time stamp to imaging device400. The external signal output from the external device includes information of the external device, such as a vital signal of additional device700and the visual stimulation signal of visual stimulation signal processing device600. Operation processing unit420can use time stamps to synchronize the image captured by first imaging unit411, the image captured by second imaging unit412, and the external signal received from the external device together. Note that the visual stimulation signal specifically includes a signal indicating that a visual target is projected on the screen and visual stimulation is thus generated, a positional signal (a signal of XY coordinates) of an index on the screen serving as visual stimulation, and the like.

Synchronization signals α and β transmitted by operation processing unit420to first and second imaging units411and412and visual stimulation signal processing device600, respectively, are clock signals repeated periodically as prescribed (for example at 60 Hz). This is not exclusive, however, and operation processing unit420may transmit a single-shot pulse signal as synchronization signals α and β to first and second imaging units411and412and visual stimulation signal processing device600, respectively, as timed when started, as prescribed, or the like.

Further, operation processing unit420may not transmit synchronization signals α and β to first and second imaging units411and412and visual stimulation signal processing device600, respectively, and may instead synchronize the time counted by each counter, for example as timed when first and second imaging units411and412and visual stimulation signal processing device600are powered on. Further, operation processing unit420may not provide synchronization based on a time stamp added to each image and information, and when first and second imaging units411and412have a configuration to each provide an image with a frame number and visual stimulation signal processing device600has a configuration to provide information with a number, operation processing unit420may provide synchronization based on the frame number added to the image and the number added to the information.

FIG.4is a schematic diagram representing image data of each of first imaging unit411and second imaging unit412, data of visual stimulation signal processing device600, and image data output by operation processing unit420. As shown inFIG.4, first imaging unit411adds time information (a frame number, a time stamp, etc.) to an image of one frame and outputs the image with the time information as image data A. Similarly, as shown inFIG.4, second imaging unit412adds time information (a frame number, a time stamp, etc.) to an image of one frame and outputs the image with the time information as image data B. Visual stimulation signal processing device600adds time information (a number, a time stamp, etc.) to the visual stimulation signal and outputs the visual stimulation signal with the time information as data X. As shown inFIG.4, operation processing unit420processes the image of one frame of first imaging unit411and the image of one frame of second imaging unit412including the same time information as one image, and outputs the processed image, data X of visual stimulation signal processing device600and the time information as one image data C. Image data C provides both eyes' real-time images as one image data, and is thus suitable for simultaneously displaying both eyes' images in data processing device100without delay. Image data C shown inFIG.4has the time information of data X, the time information of image data A, and the time information of image data B combined into one piece of time information, followed by the visual stimulation signal of data X, and the image of one frame of image data A and the image of one frame of image data B that are processed into one image. This is not exclusive, however, and image data C may not have the time information of data X combined with those of image data A and B, and may instead have data X simply aligned with the processed one image.

Transmission and reception of information including additional device700will be described.FIG.5is a sequence diagram for illustrating transmission and reception of data among the imaging device, the visual stimulation signal processing device, and the external device. Initially, operation processing unit420outputs a synchronization signal α and a start imaging signal to the imaging unit (first imaging unit411and second imaging unit412) (S1). Operation processing unit420outputs synchronization signal β to synchronization unit601of visual stimulation signal processing device600. When the imaging unit (first imaging unit411and second imaging unit412) receives the start imaging signal, the imaging unit starts capturing an image of an eye, based on a predetermined setting, to obtain an image of an eyeball (S2). The imaging unit (first imaging unit411and second imaging unit412) adds to the obtained image a time stamp synchronized based on synchronization signal α and outputs the image with the time stamp as image data A and B to operation processing unit420(S3).

Operation processing unit420outputs synchronization signal β to synchronization unit601of visual stimulation signal processing device600(S4). Synchronization unit601provides adjustment to synchronize a counter based on synchronization signal β received. Additional device700outputs an analog signal such as a vital sign to visual stimulation signal processing device600(S5). Processing unit602of visual stimulation signal processing device600converts the received analog signal into a digital signal to provide an external signal and adds a time stamp of synchronization unit601to the external signal (S6). Processing unit602transmits to operation processing unit420the external signal received from the external device (S7). The external signal from the external device may include information such as the visual stimulation signal of visual stimulation signal processing device600in addition to the vital signal of additional device700.

Operation processing unit420synchronizes the images of first and second imaging units411and412and the external signal from the external device together based on the time stamps added to the images and the external signal. Operation processing unit420combines the image of first imaging unit411and the image of second imaging unit412together, and further combines the external signal from the external device thereto to generate image data C (S8).

Subsequently operation processing unit420transmits the generated image data C to data processing device100(S9). Data processing device100processes the received image data C, and processes, stores, and displays captured image data.

The above process is performed for one frame, and the same process is repeated for each frame.FIG.5illustrates processing frames 1, 2, . . . , and N.

Hereinafter reference will be made to a flowchart to describe a control method in which imaging device400synchronizes an image captured by first imaging unit411, an image captured by second imaging unit412, and the external signal from the external device together to output an image, information and the like.FIG.6is a flowchart of a method for controlling imaging device400according to the first embodiment. Initially, operation processing unit420transmits synchronization signals α and β to first and second imaging units411and412and visual stimulation signal processing device600(step S11). First and second imaging units411and412and visual stimulation signal processing device600perform adjustment based on synchronization signals α and β to synchronize times counted by their respective counters.

Operation processing unit420determines whether image data A (first image data) is received from first imaging unit411and image data B (second image data) is received from second imaging unit412(step S12). When image data A and B are not received (NO in step S12), operation processing unit420returns to step S12. When image data A and B are received (YES in step S12), operation processing unit420proceeds to step S14. Operation processing unit420determines whether an external signal (information such as a visual stimulation signal and a vital sign) from an external device has been received from visual stimulation signal processing device600(step S13). When there is no external signal received from an external device (NO in step S13), operation processing unit420returns to step S13. While in the flowchart shown inFIG.6, image data A, image data B, and the external signal from the external device are input to operation processing unit420at the same time, image data A from first imaging unit411, image data B from second imaging unit412, and the external signal from the external device from visual stimulation signal processing device600may be input to operation processing unit420sequentially.

When there is an external signal received from an external device (YES in step S13), operation processing unit420synchronizes the image captured by first imaging unit411(a first image), the image captured by second imaging unit412(a second image) and the external signal from the external device together based on time stamps adjusted by synchronization signals α and β (step S14). When information of any one of image data A and B from the first and second imaging units in step S12and the external signal from the external device in step S13is not received and another information is received, the received one(s) of information of image data A and B and the external signal may alone be synchronized with the received other information. When one of information is not received, then the synchronization step of step14may not be performed and the control may return to step S12and step S13. Operation processing unit420outputs the image captured by first imaging unit411(the first image) and the image captured by second imaging unit412(the second image) as well as the external signal from the external device that are synchronized together (step S15).

Subsequently, the image captured by first imaging unit411(the first image) and the image captured by second imaging unit412(the second image) as well as the external signal from the external device that are synchronized together are transmitted to data processing device100, and processing of ocular movement data required for diagnosis of vertigo is performed.FIG.7is a schematic diagram showing an example of an image of the eyeballs of subject2and data of an ocular movement thereof displayed on display300of data processing device100. The example of displaying by display300shown inFIG.7displays on a screen at an upper side a first image41captured by first imaging unit411and a second image42captured by second imaging unit412. Data processing device100can sample first image41and second image42for each frame, binarize the sampled images to perform elliptical approximation, or apply a template image to the sampled images to perform pattern-matching to detect each eyeball's pupil contour and center. An image indicating each eyeball's pupil contour and center, as detected by data processing device100, is superimposed and displayed on first image41and second image42displayed on display300.

As shown inFIG.3, data processing device100includes an image processing unit110and a control unit120. Image processing unit110is composed of a graphics processing unit (GPU) or the like, and can cause display300to display first image41and second image42, and superimpose an image indicating each eyeball's pupil contour and center on first image41and second image42and thus display the superimposed images on display300. Control unit120is a computing entity which performs elliptical approximation or uses a template image to perform pattern-matching or the like to detect an eyeball's pupil contour and center. It is an example of a computer composed for example of a CPU, an FPGA, or the like. Further, control unit120includes memory such as a RAM used to store an image or the like and a ROM having a program or the like stored therein. In addition to the configuration in which control unit120is executed as a processing unit (a processor) that subjects the synchronized first and second images41and42to prescribed data processing, control unit120also has a configuration in which control unit120is executed as a receiving unit (a receiving circuit) that receives the synchronized first and second images41and42from imaging device400.

Control unit120further determines a horizontal ocular angle (right), a vertical ocular angle (right) and an ocular cycloduction angle (right) from first image41, and a horizontal ocular angle (left), a vertical ocular angle (left) and an ocular cycloduction angle (left) from second image42, each through an operation. Specifically, control unit120determines each eyeball's pupil contour and center position of first image41and second image42as detected for each frame, and from the position calculates horizontal ocular angles (right and left), vertical ocular angles (right and left), and ocular cycloduction angles (right and left). Data processing device100records how the horizontal ocular angles (right and left), vertical ocular angles (right and left), and ocular cycloduction angles (right and left) calculated by control unit120change in value with time, and causes display300to display it on a screen at a lower side.

Although not shown, imaging device400is provided with a head sensor including an acceleration sensor and an angular velocity sensor, and the head sensor outputs a measurement signal corresponding to the movement of the head of subject2. The head sensor may be mounted on the head of subject2separately from imaging device400. Operation processing unit420or control unit120determines a head angle and a head angular velocity through an operation based on the measurement signal received from the head sensor. Data processing device100records how the head angle and head angular velocity calculated by operation processing unit420or control unit120change in value with time, and causes display300to display it on the screen at a lower side. Head angle and head angular velocity are represented in a graph, which represent them in value along each of three axes (X-, Y- and Z-axes).

Further, an example of a nystagmus examination used in diagnosis of vertigo will be described.FIG.8is a schematic diagram for illustrating an example of a nystagmus examination. The nystagmus examination is performed under a prescribed condition. For example, in an examination for spontaneous nystagmus, operator1diagnoses vertigo based on an ocular movement of subject2while the subject has his/her head fixed and thus gazes frontward. In an examination for positional nystagmus, as shown inFIG.8A, operator1diagnoses vertigo based on an ocular movement of subject2induced as the subject displaces his/her head's position to various positions. In an examination for positioning nystagmus, as shown inFIGS.8(B)and8(c), operator1diagnoses vertigo based on an ocular movement of subject2induced as the operator1displaces the subject's bodily and head positions.

FIG.9is a schematic diagram showing an example of an image of the eyeballs of subject2and data of an ocular movement thereof displayed on a display in a nystagmus examination. Data processing device100in the nystagmus examination processes ocular movement data from an image of the pupil, iris pattern and the like of an eyeball of subject2. When subject2has only a single eye imaged with an imaging device during a nystagmus examination, and the subject assumes some head position, the subject may lower one eyelid or the like, and the imaging device may be unable to constantly obtain an image of the eyeball. In contrast, imaging device400obtains images of both eyes of subject2through first and second imaging units411and412, respectively, and synchronizes the images and thus processes data, and even if the subject lowers one eyelid or the like and one eyeball's image cannot be obtained, the other eyeball's image of the same timing has been obtained without fail. Thus, even when subject2changes his/her head position during a nystagmus examination, data processing device100ensures that an image of an eyeball allowing data to be processed is obtained, and data processing device100can thus steadily process ocular movement data of subject2. In the example shown inFIG.9, the nystagmus examination is performed by processing data E1of how ocular cycloduction angles (right and left) and a head angle change with time.

Hereinafter, an example of a visual stimulation test employed in diagnosis of vertigo will be described.FIG.10is a schematic diagram for illustrating an example of a visual stimulation test.FIG.10shows a screen5for projecting a visual target S generated by visual stimulation signal processing device600, and shows how visual target S is positionally moved rightward/leftward as it is positionally changed by visual stimulation signal processing device600. Visual target S may positionally not be moved rightward/leftward and instead be moved upward/downward or upward/downward/rightward/leftward. When the visual stimulation test is performed, subject2wears imaging device400having detached therefrom shading cover402shown inFIG.2A, and subject2can thus visually recognize visual target S projected on screen5through hot mirror410.

Visual stimulation signal processing device600includes a laser device to generate a laser point on screen5shown inFIG.10and thus display the laser point as visual target S. As shown inFIG.3, visual stimulation signal processing device600can transmit and receive data to and from data processing device100, and also receives a synchronization signal β from imaging device400. Visual stimulation signal processing device600adds a time stamp of synchronization unit601to the visual stimulation signal and an external signal received from an external device such as a vital sign detection device, and outputs the signal with the time stamp to the imaging device400.

FIG.11is a schematic diagram showing an example of an image of the eyeballs of subject2and data of an ocular movement thereof displayed on a display in a visual stimulation test. Data processing device100in theFIG.10visual stimulation test processes ocular movement data from an image of the pupil, iris pattern and the like of an eyeball of subject2. A pursuit eye movement test, which is an example of the visual stimulation test, examines how an eyeball of subject2follows visual target S. Accordingly, in the example shown inFIG.11, data processing device100performs the pursuit eye movement test by processing data E2of how horizontal ocular angles (right and left) and a visual stimulation signal change with time. Note that, inFIG.11, display300displays on a screen at a lower side a graph corresponding to how a visual stimulation signal of those received from visual stimulation signal processing device600that indicates that visual stimulation is generated, changes with time. Further, display300displays on the screen at a lower left side that visual target S projected on screen5positionally moves from left to right, based on a signal of XY coordinates of visual target S included in the visual stimulation signal. When visual target S positionally moves upward/downward rather than rightward/leftward, data E2needs to be how vertical ocular angles (right and left) and a vertical visual stimulation angle change with time, and when visual target S positionally moves upward/downward/rightward/leftward, data E2needs to be how horizontal ocular angles (right and left), vertical ocular angles (right and left), a horizontal visual stimulation angle and a vertical visual stimulation angle change with time.

Another example of the visual stimulation test is a saccadic eye movement test. In the saccadic eye movement test, visual target S flashed on/off right and left alternately or the like is shown to subject2to test how the eyeballs of subject2move. Processing ocular movement data of subject2in the saccadic eye movement test requires sampling an image at a rate of 6 ms=166 fps or more. Accordingly, imaging device400is required to capture an image of the eyeballs of subject2at a high sampling rate of 240 fps rather than doing so at a normal sampling rate of 60 fps.

In addition, ocular cycloduction movement data may be processed in diagnosis of vertigo.FIG.12is a schematic diagram showing an example of an image of the eyeballs of subject2and data of an ocular movement thereof displayed on a display when ocular cycloduction movement data is processed. Data processing device100processes ocular cycloduction movement data from recognition (or pattern-matching) of an image of the pupil, iris pattern and the like of an eyeball of subject2. When processing ocular cycloduction movement data, data processing device100detects ocular cycloduction movement, as indicated inFIG.12by an arrow44, records how an ocular cycloduction angle changes with time, and causes display300to display it on a screen at a lower side. Data processing device100processes ocular cycloduction movement data of subject2by examining data E3of how ocular cycloduction angles (right and left) change with time.

Further, in diagnosis of vertigo, the orientation of the head is also important information along with ocular movement, and accordingly, operator1desires to be able to recognize an image of an eyeball and the orientation of the head simultaneously. Accordingly, data processing device100superimposes the image of the eyeball and an image representing the orientation of the head, one on the other, and causes display300to display the superimposed images.FIG.13is a schematic diagram of superimposing an image representing an orientation of the head of subject2on an image of the eyeballs of the subject, and causing a display to display the thus superimposed images. On the screen shown inFIG.13at an upper side are displayed first image41captured by first imaging unit411and second image42captured by second imaging unit412, with an image45therebetween indicating the orientation of the head of subject2. Image45indicating the orientation of the head of subject2shows in a 3D model of the head the orientation of the head based on information of a head angle and a head angular acceleration as measured by the head sensor. Image45indicating the orientation of the head of subject2is not limited to the 3D model of the head as shown inFIG.13, and may for example be displayed in the form of a 2D model, textual information, etc.

Further, in diagnosis of vertigo, information of a vital sign (such as a heart rate) is also important information along with ocular movement, and accordingly, operator1desires to be able to recognize an image of an eyeball and the information of the vital sign simultaneously. Accordingly, data processing device100superimposes the image of the eyeball and an image representing the information of the vital sign, one on the other, and causes display300to display the superimposed images.FIG.14is a schematic diagram of superimposing an image of the eyeballs of a subject and information of a vital signal, one on the other, and causing a display to display the same. On the screen shown inFIG.14at an upper side are displayed first image41captured by first imaging unit411and second image42captured by second imaging unit412, and at a lower side is displayed a graph corresponding to how a heart rate included in the vital signal received from additional device700changes with time. Further, display300digitally displays on the screen at a lower left side a heart rate of subject2based on the vital signal received from additional device700.

As described above, imaging device400according to the first embodiment is a device that captures an image of an eyeball in an equilibrium examination. Imaging device400comprises housing401mounted on the head of subject2, an imaging unit (first imaging unit411and second imaging unit412) that is held by housing401and captures an image of an eyeball of subject2, operation processing unit420(a communication unit) that communicates information with an external device (visual stimulation signal processing device600and additional device700), and operation processing unit420(a control unit) that synchronizes together the image captured by the imaging unit (first imaging unit411and second imaging unit412) and an external signal received from the external device.

Thus, imaging device400according to the first embodiment synchronizes together the image captured by the imaging unit (first imaging unit411and second image unit412) and the external signal received from the external device, and can thus obtain image data of ocular movement, and together therewith, relevant data of the external device.

Operation processing unit420transmits the image and the external signal from the external device that are synchronized together to data processing device100that is externally provided and processes data. Data processing device100can thus process data of the image and the external signal from the external device that are synchronized together, and an appropriate diagnosis of vertigo can thus be made.

The imaging unit (first imaging unit411and second imaging unit412) adds information provided for each captured image (the first information and the second information) to the captured image to provide image data A and B and outputs image data A and B to operation processing unit420, operation processing unit420synchronizes together image data A and B output by the imaging unit (first imaging unit411and second imaging unit412) and the external signal received from the external device, and transmits the same to data processing device100as image data A and B including the information provided for the captured image (the first information and the second information) and the external signal from the external device. Thus, operation processing unit420can reliably synchronize first and second images41and42and the external signal from the external device together based on the information provided for captured images (the first information and the second information) included in image data A and B.

Operation processing unit420transmits synchronization signals α and β to the imaging unit (first imaging unit411and second imaging unit412) and visual stimulation signal processing device600, and the imaging unit (first imaging unit411and second imaging unit412) and visual stimulation signal processing device600synchronize the information provided for captured images (the first information and the second information) and added thereto and information of visual stimulation signal processing device600together based on synchronization signals α and β. Operation processing unit420can thus reliably perform adjustment of synchronizing the first information of first imaging unit411, the second information of second imaging unit412and the information of visual stimulation signal processing device600together. Synchronization signals α and β transmitted to the imaging unit (first and second imaging units411and412) and visual stimulation signal processing device600are signals repeated periodically as prescribed.

The information provided for captured images (the first information and the second information) and the external signal from the external device at least include information of a time stamp. Operation processing unit420can reliably synchronize first image41, second image42, and the external signal from the external device together based on a time stamp included in image data A and B and a time stamp included in the external signal from the external device.

The imaging unit includes first imaging unit411that captures an image of one eyeball of subject2and second imaging unit412that is held in housing401and captures an image of the other eyeball of subject2, and first imaging unit411outputs each captured first image41with first information added thereto as image data A to operation processing unit420and second imaging unit412outputs each captured second image42with second information added thereto as image data B to operation processing unit420, and operation processing unit420synchronizes first image41, second image42, and the external signal from the external device together based on the first information included in image data A, the second information included in image data B, and the external signal from the external device and externally transmits image data A and B and the external signal from the external device that are synchronized together. Thus, operation processing unit420can reliably synchronize first image41, second image42, and the external signal from the external device together based on the first information included in image data A, the second information included in image data B, and the external signal from the external device.

Operation processing unit420processes the synchronized first and second images41and42as one image, and operation processing unit420externally transmits image data C including the processed image, the external signal (a visual stimulation signal) from the external device, the corresponding first information and second information, and time information of the external device. Data processing device100receiving from operation processing unit420first image41and second image42processed as one image may simply cause display300to display the processed one image.

The external device is either one of a device relating to visual stimulation (visual stimulation signal processing device600) and a device which detects a vital sign of subject2. Operation processing unit420can thus obtain relevant data of the external device necessary for appropriate diagnosis of vertigo together.

Ocular movement data processing system10according to the first embodiment is a system that processes ocular movement data in an equilibrium examination. Ocular movement data processing system10comprises imaging device400that captures an image of an eyeball of subject2, data processing device100that receives data from imaging device400and processes the data, and an external device (visual stimulation signal processing device600and additional device700) that obtains at least one of information about subject2and information about an equilibrium examination. Imaging device400includes housing401mounted on the head of subject2, an imaging unit (first imaging unit411and second imaging unit412) that is held by housing401and captures an image of the eyeball of subject2, operation processing unit420(a communication unit) that communicates information with the external device, and operation processing unit420(a control unit) that synchronizes together the image captured by the imaging unit (first imaging unit411and second imaging unit412) and an external signal received from the external device. Data processing device100includes control unit120(a receiving unit) that receives the synchronized first and second images from imaging device400, and control unit120(a processing unit) that subjects the received, synchronized first and second images to prescribed data processing.

A method for control by imaging device400according to the first embodiment comprises the steps of: causing the imaging unit (first imaging unit411and second imaging unit412) to capture an image of an eyeball of subject2; receiving an external signal from an external device; and synchronizing together the image captured by the imaging unit (first imaging unit411and second imaging unit412) and the external signal received from the external device.

Second Embodiment

For ocular movement data processing system10according to the first embodiment is described a configuration in which first image41captured by first imaging unit411, second image42captured by second imaging unit412, and information of an external device (visual stimulation signal processing device600and additional device700) are synchronized together in imaging device400. For an ocular movement data processing system according to a second embodiment will be described a configuration to further synchronize information of a sensor provided to imaging device400.

FIG.15is a block diagram generally showing a configuration of an ocular movement data processing system10aaccording to the second embodiment. Ocular movement data processing system10ashown inFIG.15is the same in configuration as ocular movement data processing system10shown inFIG.3except that how an imaging device400ais configured. Accordingly, any configuration in theFIG.15ocular movement data processing system10athat is identical to that of theFIG.3ocular movement data processing system10is identically denoted and will not be described specifically.

In imaging device400a, as shown inFIG.15, image data A from first imaging unit411and image data B from second imaging unit412are processed in operation processing unit420and transmitted to data processing device100as image data C. First imaging unit411includes an infrared imaging device, and a processing circuit (not shown) that attaches information of a frame number and a time stamp (first information) to an image that is captured by the infrared imaging device for each frame to provide image data A and output image data A to operation processing unit420. The information included in the first information is not limited to a frame number and a time stamp, and may be information of at least one of the frame number and the time stamp, and may include information such as a frame rate, an amount of exposure, and a contrast. Similarly, second imaging unit412includes an infrared imaging device, and a processing circuit (not shown) that attaches information of a frame number and a time stamp (second information) to an image that is captured by the infrared imaging device for each frame to provide image data B and output image data B to operation processing unit420. The information included in the second information is not limited to a frame number and a time stamp, and may be information of at least one of the frame number and the time stamp, and may include information such as a frame rate, an amount of exposure, and a contrast. The first information and the second information are also referred to as information provided for a captured image.

Imaging device400aincludes, in addition to first and second imaging units411and412, an attachment sensor430that senses an attached state of housing401to subject2, a shade sensor440that senses a shaded state of first and second imaging units411and412, and a head sensor450that senses how subject2moves his/her head, how the subject orients his/her head, in particular, in housing401. Attachment sensor430is, for example, a contact sensor, and when the contact sensor issues an OFF signal, operation processing unit420can determine that housing401of imaging device400ais detached from the head of subject2or displaced from a prescribed position. Shade sensor440is, for example, an optical sensor, and when shading cover402is attached and imaging device400ais internally dark, the optical sensor issues an OFF signal, and operation processing unit420can determine that first and second imaging units411and412are shaded. Head sensor450is composed of an acceleration sensor, an angular velocity sensor, and a geomagnetic sensor each provided for directions along three axes for a total of nine sensors. The acceleration sensor can sense the posture of the head of subject2by sensing gravitational acceleration. The angular velocity sensor can sense the angular velocity of the head of subject2. The geomagnetic sensor can sense the orientation (or azimuth) of the head of subject2. Operation processing unit420calculates head angle, head angular velocity and the like through an operation based on a measurement signal received from head sensor450. While for imaging device400awill be described a configuration in which attachment sensor430, shade sensor440, and head sensor450are all provided, at least one of attachment sensor430, shade sensor440, and head sensor450may be provided. As a matter of course, imaging device400amay be provided with sensors other than attachment sensor430, shade sensor440, and head sensor450.

For imaging device400a, the signals from attachment sensor430, shade sensor440, and head sensor450are also synchronized with the images captured by first and second imaging units411and412. Specifically, as a method in which imaging device400asynchronizes an image captured by first imaging unit411, an image captured by second imaging unit412, and the signals received from attachment sensor430, shade sensor440and head sensor450together, there is a method, for example, of synchronization based on time stamps added to the images and the signals received from the sensors. The time stamps are generated based on the times counted by the counters of first imaging unit411, second imaging unit412, attachment sensor430, shade sensor440, and head sensor450, and in order to use the images and the signals received from the sensors for synchronization, it is necessary to synchronize the times counted by the counters. In order to synchronize the times counted by the counters of the imaging units and sensors, operation processing unit420transmits synchronization signal α to each of first imaging unit411, second imaging unit412, attachment sensor430, shade sensor440, and head sensor450.

Based on synchronization signal α, first imaging unit411, second imaging unit412, attachment sensor430, shade sensor440, and head sensor450synchronize and thus adjust the time counted by each counter, and add a time stamp to each image and each sensor's signal. Based on each time stamp adjusted by synchronization signal α, operation processing unit420can reliably synchronize the image captured by first imaging unit411, the image captured by second imaging unit412, and the signals from attachment sensor430, shade sensor440, and head sensor450together to obtain a right eye image, a left eye image, and each sensor's signal of the same timing.

Operation processing unit420may not provide synchronization based on the time stamp added to each image and each sensor's signal, and may instead provide synchronization based on other information (e.g., a frame number, a number, etc.) added to each image and each sensor's signal. Further, operation processing unit420may not transmit synchronization signal α to each of first imaging unit411, second imaging unit412, attachment sensor430, shade sensor440, and head sensor450, and, for example, may instead synchronize the time counted by each counter, as timed when each imaging unit and each sensor are powered on. Further, attachment sensor430, shade sensor440, and head sensor450may output a result of sensing to operation processing unit420, as timed by synchronization signal α, without adding information such as a time stamp.

Operation processing unit420outputs each image and each sensor's signal that are synchronized together to data processing device100.FIG.16is a schematic diagram representing image data of each of first imaging unit411and second imaging unit412, signals from sensors, and image data output by operation processing unit420. As shown inFIG.16, first imaging unit411adds time information (a frame number, a time stamp, etc.) to an image of one frame and outputs the image with the time information as image data A. Similarly, as shown inFIG.16, second imaging unit412adds time information (a frame number, a time stamp, etc.) to an image of one frame and outputs the image with the time information as image data B. Visual stimulation signal processing device600adds time information (a number, a time stamp, etc.) to the visual stimulation signal and outputs the visual stimulation signal with the time information as data X.

As shown inFIG.16, attachment sensor430adds time information (such as a time stamp) to attachment information (for example, an ON signal or an OFF signal) including a result of sensing, and outputs the attachment information with the time information as data F. As shown inFIG.16, shade sensor440adds time information (such as a time stamp) to shade information (for example, an ON signal or an OFF signal) including a result of sensing, and outputs the shade information with the time information as data G. As shown inFIG.16, head sensor450adds time information (such as a time stamp) to information in orientation of the head (e.g., a measurement signal) including a result of sensing, and outputs the information with the time information as data H. As shown inFIG.16, operation processing unit420processes the image of one frame of first imaging unit411and the image of one frame of second imaging unit412including the same time information as one image, and outputs the processed image, data X of visual stimulation signal processing device600, the time information, data F of attachment sensor430, data G of shade sensor440, and data H of head sensor450as one image data C.

As described above, imaging device400aaccording to the second embodiment further comprises head sensor450(a first detection unit) that is held by housing401and senses movement of the head of subject2, and operation processing unit420synchronizes together a first image captured by first imaging unit411, a second image captured by second imaging unit412, an external signal (a visual stimulation signal) from an external device, and a result of sensing (a measurement signal) by head sensor450. Thus, imaging device400acan accurately grasp an external signal received from the external device and a movement of the head of subject2, that are obtained at a time when first and second imaging units411and412capture images, and thus allows appropriate diagnosis.

Further, the attachment sensor430that is held by housing401and senses an attached state of housing401is further comprised and operation processing unit420synchronizes together the first image captured by first imaging unit411, the second image captured by second imaging unit412, the external signal (the visual stimulation signal) from the external device, and a result of sensing (an ON signal or an OFF signal) by attachment sensor430. Thus, imaging device400acan accurately grasp an external signal received from the external device and an attached state of imaging device400a, that are obtained at a time when first and second imaging units411and412capture images, and thus allows appropriate diagnosis.

Further, the shade sensor440that is held by housing401and senses a shaded state of a portion imaged by first imaging unit411and a shaded state of a portion imaged by second imaging unit412is further comprised, and operation processing unit420synchronizes together the first image captured by first imaging unit411, the second image captured by second imaging unit412, the external signal (the visual stimulation signal) from the external device, and a result of sensing (an ON signal or an OFF signal) by shade sensor440. Thus, imaging device400acan accurately grasp an external signal received from the external device and a shaded state of the imaging unit, that are obtained at a time when first and second imaging units411and412capture images, and thus allows appropriate diagnosis.

Modified Example

In the first embodiment, first and second images41and42are included in image data C as one image and thus output. However, image data C output by imaging device400is not limited as shown inFIG.4, that is, to a configuration in which an image of one frame of first imaging unit411and an image of one frame of second imaging unit412including the same time information are processed as one image, and the processed image and the time information of first and second imaging units411and412are output as one image data C. For example, imaging device400may alternately dispose image data A of first imaging unit411, image data B of second imaging unit412and an external signal received from an external device that include the same time information, and imaging device400may output the same as image data C.

FIG.17is a schematic diagram for illustrating an image of first imaging unit411and that of second imaging unit412alternately disposed to output image data. As shown inFIG.17, operation processing unit420alternately disposes data X of visual stimulation signal processing device600, and image data A of first imaging unit411and image data B of second imaging unit412that include the same time information to package them into image data C and thus outputs image data C. An image of one frame of first imaging unit411and an image of one frame of second imaging unit412including the same time information are not processed as one image; rather, image data A and B and the external signal from the external device are simply packaged into one image data C and output. While in the example shown inFIG.17the time information of first imaging unit411, the time information of second imaging unit412and the time information of the external device are separately output, the time information of first imaging unit411, the time information of second imaging unit412and the time information of the external device may be combined into one and thus output.

The modified example shown inFIG.17is also similarly applicable to image data C output by imaging device400aaccording to the second embodiment.

As described above, operation processing unit420of the modified example alternately transmits image data A and image data B with a first image and a second image synchronized together. This ensures that data processing device100receives the synchronized image data A and B, and providing the synchronized images to operator1allows an appropriate diagnosis of vertigo to be made.

Although the present disclosure has been described and illustrated in detail, it is clearly understood that the same is by way of illustration and example only and is not to be taken by way of limitation. The scope of the present disclosure is interpreted by the terms of the appended claims, and any modification within the meaning and scope equivalent to the terms of the claims is intended to be encompassed.