IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND ENDOSCOPE SYSTEM

An image processing apparatus includes a processor, in which the processor is configured to receive a picked-up image of a swallowing motion in a swallowing test in which a swallow object is provided to a subject to observe the swallowing motion, detect a color of the swallow object in the picked-up image, and perform processing for improving an image quality of the picked-up image according to a detection result of the color of the swallow object.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus which effectively assists a swallowing test, an image processing method, and an endoscope system.

2. Description of the Related Art

Dysphagia is a disorder occurring in a series of processes of sending food from a mouth to an esophagus, and becomes a cause of aspiration pneumonia or the like in which food enters a trachea to cause inflammation. Heretofore, Japanese Patent Application Laid-Open Publication No. 2018-15263 has disclosed a technique of testing for dysphagia by using an X-ray image. However, a scale of an image diagnosis apparatus using X-rays is large. On the other hand, there is a swallowing test method of photographing, by a nasal endoscope, a series of swallowing situations after test food is put into the mouth and determining whether there is dysphagia by photographed images. A wireless endoscope may be adopted as such a nasal endoscope. Note that in the swallowing test, a test using colored test food is conducted.

Note that a general endoscope obtains observation images at a frame rate of 30 frames/second (fps) or 60 fps, for example.

SUMMARY OF THE INVENTION

An image processing apparatus according to an aspect of the present invention includes a processor, and the processor is configured to receive a picked-up image of a swallowing motion in a swallowing test in which a swallow object is provided to a subject to observe the swallowing motion, detect a color of the swallow object in the picked-up image, and perform processing for improving an image quality of the picked-up image according to a detection result of the color of the swallow object.

An image processing method according to an aspect of the present invention includes receiving a picked-up image of a swallowing motion in a swallowing test in which a swallow object is provided to a subject to observe the swallowing motion, detecting a color of the swallow object in the picked-up image, and performing processing for improving an image quality of the picked-up image according to a detection result of the color of the swallow object.

An endoscope system according to an aspect of the present invention includes an endoscope configured to obtain a picked-up image of a swallowing motion in a swallowing test in which a swallow object is provided to a subject to observe the swallowing motion, and an image processing apparatus including a communication circuit configured to receive the picked-up image, an image processing circuit configured to detect a color of the swallow object in the picked-up image, and a control circuit configured to perform processing for improving an image quality of the picked-up image according to a detection result of the image processing circuit.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

FIG.1is a block diagram illustrating an image processing apparatus according to an embodiment of the present invention. According to the present embodiment, an image pickup frame rate during a period (hereinafter, which is referred to as an effective observation period) for obtaining an observation image considered to be particularly effective in a swallowing diagnosis at a time of a swallowing test (hereinafter, which is referred to as an effective observation image) is increased to set an image quality of the effective observation image to an image quality appropriate for an observation, and an image pickup frame rate during a period other than the effective observation period (hereinafter, which is referred to as a non-effective observation period) is decreased. Thus, according to the present embodiment, an excess increase of battery consumption in an endoscope is suppressed, and the swallowing diagnosis is effectively assisted by the effective observation image.

FIG.2andFIG.3are explanatory diagrams for describing a normal swallowing motion.FIG.2andFIG.3illustrate a mechanism of swallowing, in whichFIG.2illustrates an oral stage, andFIG.3illustrates a pharyngeal stage.

FIG.2illustrates a state where a food bolus9(shaded portion) is held in an oral cavity in the oral stage. A tongue1rises upwards and a soft palate3falls downwards to partition the oral cavity and a pharynx2. The food bolus9is held inside the mouth, and a state is established where the food bolus9may be chewed and swallowed when necessary. Note that in the above described state, the food bolus9does not flow into an esophagus6, and the epiglottis4rises to open an air passage5.

When chewing is ended, due to movement of the tongue1and contraction of the pharynx2, the food bolus9is sent from the oral cavity into the esophagus6.FIG.3illustrates a state in the pharyngeal stage as described above. A broken line inFIG.3represents a state of the tongue1, the soft palate3, and the epiglottis4in the oral stage. Since the tongue1, the soft palate3, and the epiglottis4in the pharyngeal stage change as indicated by arrows, the food bolus9is sent into the esophagus6. In the above described case, the epiglottis4falls (closes) to block the air passage5, such that the food bolus9does not enter the air passage5.

Note that when the food bolus9flows into the esophagus6, the epiglottis4rises (opens) to return to an original position. The air passage5, the oral cavity, and a nasal cavity are connected, such that breathing can be performed.

Test food or test liquid such as colored water, jelly, or yogurt having a color which is different from a color of a living body and which does not normally exist in the living body is used as the food bolus9, so that checking of the swallowing motion is facilitated. Note that in the following description, an object which includes the test food, the test liquid, or the like and which is set as a target of the swallowing is referred to as a swallow object. In a test using the swallow object in the series of swallowing tests, a diagnosis on triggering of swallowing reflex and a diagnosis on pharyngeal clearance by swallowing of the swallow object are conducted.

In the diagnosis on the triggering of the swallowing reflex, a test is conducted with regard to (a1) that a slight laryngeal inflow of the swallow object is only observed, (a2) that a state can be observed where the swallow object reaches an epiglottic vallecula, (a3) that a state can be observed where the swallow object reaches a pyriform sinus, and (a4) that even after the swallow object reaches the pyriform sinus, the swallowing reflex does not occur fora while.

In the diagnosis on the pharyngeal clearance by the swallowing of the swallow object, a test is conducted with regard to (b1) that there is no residue of the swallow object after the swallowing, (b2) that there is a low level of residue of the swallow object, but the residue is washed out by performing dry swallowing twice or three times, (b3) that residue of the swallow object is present, and is not washed out by performing swallowing a plurality of times, and (b4) that there is a high level of residue of the swallow object, and the residue flows into a laryngeal cavity.

FIG.4is an explanatory diagram for describing a situation of a room in which a swallowing test is conducted.FIG.5is an explanatory diagram illustrating a state in which an insertion portion21of an endoscope20is inserted into the nasal cavity for the swallowing test and an observation image V of the endoscope.

The swallowing test is conducted in a hospital room11or the like instead of an operation room in many cases. Consequently, a test using the wireless endoscope20which is easy to move as the endoscope is preferable. A subject12and an examiner13who operates the wireless endoscope20are present in the hospital room11. The examiner13grasps the wireless endoscope20. An image processing apparatus (hereinafter, referred to as an image processing display apparatus30) which has a reception function and an image processing function and which includes a display apparatus is arranged on a table14in the hospital room11. The wireless endoscope20and the image processing display apparatus30are configured to be able to wirelessly communicate with each other, and observation images (movie and still images) obtained by the wireless endoscope20are provided to the image processing display apparatus30to be displayed on a display screen30a.

The endoscope20is provided with an image pickup device22(not illustrated inFIG.5) at a distal end of the insertion portion21that is elongated and has flexibility. At the time of the test, as illustrated inFIG.5, a distal end portion of the insertion portion21of the endoscope20is arranged to be stopped at an upper end of the pharynx2, such that an optical axis of the image pickup device22at the distal end faces the esophagus6. With observation images V obtained by the endoscope20, it is possible to observe movements or the like of the pharynx2, the esophagus6, the swallow object (not illustrated inFIG.5) flowing from the pharynx2to the esophagus6, the air passage5(filled portion), and the epiglottis4which blocks the air passage5.

FIG.6is an explanatory diagram illustrating examples of the observation images at the time of the swallowing test. Observation images V1to V3inFIG.6each have a circular effective image region15, and a mask region (no image region)16disposed in a surrounding of the effective image region15.

The image V1in an upper section ofFIG.6has an image part (filled portion) of an opening of the air passage5at a center of the image, has an image part of the epiglottis4in an opened state below the image part of the air passage5, and has image parts of the esophagus6in a blocked state above the image of the air passage5and also on the left and right of the image of the air passage5. In other words,FIG.5illustrates an image in a state before the food bolus9flows into the pharynx2.

The image V2in a middle section ofFIG.6illustrates a state in which a swallow object9aflows into the pharynx2in an image part of a shaded portion. The image V3in a bottom section ofFIG.6illustrates a state in which the swallow object9aflows into the esophagus6(filled portion), and the swallow object9ais not observed out on the observation image, that is, a state immediately after the swallow object9ais normally swallowed.

The images of the oral stage, the pharyngeal stage, the state immediately after the swallowing reflex, and the like are obtained by the endoscope20, and the situation of swallowing is checked from the observation images, so that an operator can perform a diagnosis of dysphagia. In such a swallowing test, the swallow object9ahas a color of green, for example, which is different from the color of the living body. Therefore, it is facilitated to observe the situation of the inflow of the swallow object9afrom the observation images.

However, since the swallowing reflex occurs in a very short time period, it may not be easy to carry out the observation even when the swallow object9ais colored to have a color different from the color of the living body. In view of the above, according to the present embodiment, in order that the situation of the inflow of the swallow object9acan be reliably assessed, an image pickup frame rate of the image pickup device22of the wireless endoscope20is increased, and at a time of an observation in the image processing display apparatus30, an image quality of the observation image to be displayed on the display screen30ais optimized, so that an accurate diagnosis in the swallowing test is effectively assisted.

In other words, according to the present embodiment, the effective observation period is considered as a period during which observation images can be obtained which are effective for the diagnosis on the triggering of the swallowing reflex and the diagnosis on the pharyngeal clearance by the swallowing of the swallow object, and the effective observation period is determined depending on whether or not the swallow object9ais observed in a picked-up image (observation image). According to the present embodiment, by switching the image pickup frame rate and an image quality adjustment method in the effective observation period and the non-effective observation period, the battery consumption is suppressed, convenience is ensured, and the observation images with a high medical value are provided upon the swallowing diagnosis to improve effectiveness of the diagnosis.

InFIG.1, the wireless endoscope20has the image pickup device22configuring a CCD sensor, a CMOS sensor, or the like placed in the insertion portion21which is not illustrated in the drawing. The wireless endoscope20has a light source apparatus23. The light source apparatus23generates illumination light, and emits the generated illumination light towards a subject from a distal end of the insertion portion21. The illumination light is reflected by the subject, and an optical image of the subject based on the reflected light is formed on an image pickup plane of the image pickup device22. The image pickup device22performs photoelectric conversion of the optical image of the subject to obtain an image pickup output.

A control unit24may be configured by a processor using a central processing unit (CPU), a field programmable gate array (FPGA), or the like. The control unit24may control each unit through an operation according to a program stored in a memory which is not illustrated in the drawing, or may realize part or all of functions by an electronic circuit of hardware. The control unit24controls each unit of the wireless endoscope20. In other words, the control unit24has functions as an image pickup control unit configured to control the image pickup device22, a light source control unit configured to control the light source apparatus23, an image processing unit configured to perform image processing, and a communication control unit configured to control a wireless communication unit25.

The control unit24controls the light source apparatus23to adjust a quantity of the illumination light. The control unit24drives and controls the image pickup device22to control image pickup by the image pickup device22. For example, the control unit24provides frame setting information to the image pickup device22, so that the image pickup frame rate at the time of the image pickup can be adjusted. The image pickup output from the image pickup device22is provided to the control unit24. The control unit24applies predetermined image processing such as distortion correction to the image pickup output, and then outputs the processed image pickup output to the wireless communication unit25. Note that the image pickup output supplied from the control unit24to the wireless communication unit25is a so-called RAW image.

The wireless communication unit25that serves as a wireless communication circuit is controlled by the control unit24. The wireless communication unit25performs modulation processing according to a predetermined wireless communication standard and converts the image pickup output into a transmission signal to be provided to an antenna26. The antenna26wirelessly transmits the transmission signal from the wireless communication unit25. The wireless transmission signal transmitted from the wireless communication unit25is received in the image processing display apparatus30. The antenna26receives the wireless transmission signal from the image processing display apparatus30, and provides a reception signal to the wireless communication unit25. The wireless communication unit25demodulates the reception signal transmitted in the predetermined wireless communication standard to be outputted to the control unit24.

A battery27is disposed inside the wireless endoscope20. The battery27can supply power to each unit in the wireless endoscope20.

The image processing display apparatus30has a control unit31configured to control each unit in the image processing display apparatus30. The control unit31that serves as a control circuit may be configured by a processor using a CPU, an FPGA, or the like. The control unit31may operate according to a program stored in a memory which is not illustrated in the drawing to control each unit, or may realize part or all of the functions by the electronic circuit of the hardware.

An antenna33is disposed in the image processing display apparatus30, and a wireless transmission signal from the wireless endoscope20is received by the antenna33. A wireless communication unit32that serves as a wireless communication circuit is controlled by the control unit31. The wireless communication unit32demodulates a reception signal received by the antenna33according to the predetermined wireless communication standard, and outputs a demodulation output to the control unit31. When a transmission signal to the wireless endoscope20is provided from the control unit31, the wireless communication unit32performs the modulation processing according to the predetermined wireless communication standard on the transmission signal, and transmits the modulated transmission signal via the antenna33.

The control unit31provides, to an image processing unit34, an image pickup output (RAW image) by the image pickup device22among demodulation outputs from the wireless communication unit32. The image processing unit34is controlled by the control unit31. The image processing unit34develops the image pickup output that is inputted and obtains a picked-up image, and applies predetermined image signal processing to the picked-up image. For example, the image processing unit34performs various image signal processing including various image quality setting processing such as color signal generation processing, white balance processing, y conversion processing, matrix conversion processing, and noise reduction processing. Note that the image processing unit34and a drawing control unit35which will be described below may also be configured by a processor using a CPU, an FPGA, or the like. The image processing unit34and the drawing control unit35may operate according to a program stored in a memory which is not illustrated in the drawing to control each unit, or may realize part or all of the functions by the electronic circuit of the hardware.

According to the present embodiment, the image processing unit34determines, by image processing, whether or not the swallow object exists in a photographing range of the image pickup device22, that is, whether or not the image part of the swallow object exists in the picked-up image obtained by the image pickup by the image pickup device22. For example, the image processing unit34may determine a color of each pixel in the picked-up image. When pixels with a same color as a color of the swallow object exist as a block in a predetermined region, the image processing unit34may regard the predetermined region as an image region corresponding to the swallow object (hereinafter, referred to as a swallow object image region) and obtain a judgement result indicating that the image part of the swallow object exists in the picked-up image. The image processing unit34outputs to the control unit31the judgement result on whether or not the image part of the swallow object exists in the picked-up image. For example, the image processing unit34may output the judgement result for each frame.

In other words, a judgement result indicating that the image part of the swallow object exists in the picked-up image (hereinafter, referred to as an inflow detection judgement result) or a judgement result indicating that the image part of the swallow object does not exist in the picked-up image (hereinafter, referred to as a non-inflow detection judgement result) is provided to the control unit31from the image processing unit34. In other words, according to the present embodiment, a period during which the inflow detection judgement result is obtained is the effective observation period, and a period during which the non-inflow detection judgement result is obtained is the non-effective observation period.

In an initial setting immediately after start of the swallowing test, the control unit31generates frame setting information in which an image pickup frame rate for the image pickup by the image pickup device22is set as a reference frame rate. When the inflow detection judgement result is provided from the image processing unit34, the control unit31generates frame setting information to set the image pickup frame rate for the image pickup by the image pickup device22to a frame rate higher than the reference frame rate (hereinafter, referred to as a high frame rate), and outputs the frame setting information to the wireless communication unit32.

When the output of the image processing unit34is changed from the inflow detection judgement result to the non-inflow detection judgement result, in a case where the non-inflow detection judgement result continues for a predetermined time period (for example, for several seconds) without interruption, the control unit31may generate frame setting information for returning the image pickup frame rate for the image pickup by the image pickup device22to the reference frame rate, and output the frame setting information to the wireless communication unit32.

The frame setting information generated by the control unit31is transmitted from the antenna33by the wireless communication unit32. When the frame setting information transmitted from the antenna33is received, the antenna26of the wireless endoscope20outputs the frame setting information to the control unit24. When the frame setting information from the image processing display apparatus30is received, the control unit24adjusts the image pickup frame rate of the image pickup device22according to the frame setting information. Note that power consumption at the time of image pickup at the high frame rate in the wireless endoscope20is larger than power consumption at the time of image pickup at a standard frame rate.

After the inflow detection judgement result is outputted, the image pickup output obtained by the image pickup at a frame rate higher than the reference frame rate (hereinafter, referred to as a high frame rate) is provided to the image processing unit34of the image processing display apparatus30. When the inflow detection judgement result is obtained, the image processing unit34determines a color of the swallow object image region as a swallow object color. The image processing unit34performs image setting to emphasize a component of a complementary color of the swallow object color for a region other than the swallow object image region in the picked-up image, and applies to the picked-up image the image processing according to the image setting. Note that the image processing unit34continues the emphasis processing until the non-inflow detection judgement result is obtained. The image processing unit34outputs the picked-up image after the image processing to the drawing control unit35and a recording processing unit37.

The drawing control unit35that serves as a drawing control circuit applies drawing processing for drawing the input picked-up image to the picked-up image, and then outputs the processed picked-up image to a display36. The display36displays the picked-up image from the drawing control unit35on the display screen30a. In other words, the display36performs display at the standard frame rate for the picked-up images obtained by the image pickup at the standard frame rate by the wireless endoscope20, and performs display at the high frame rate for the picked-up images obtained by the image pickup at the high frame rate by the wireless endoscope20.

The frame setting information is also provided to the recording processing unit37configuring a recording apparatus from the control unit31. The recording processing unit37performs recording processing of the picked-up images from the image processing unit34at a frame rate designated by the frame setting information to record the picked-up images in a memory38. Note that the memory38may be a predetermined recording medium which can record information and may also be configured, for example, by a memory medium such as a flash memory or a magnetic medium such as a hard disc. The recording processing unit37can perform recording in a format corresponding to a format of the memory38.

A battery39is disposed inside the image processing display apparatus30. The battery39can supply power to each unit in the image processing display apparatus30. Note that the image processing display apparatus30is used in a state of being installed on a table or the like, but the wireless endoscope20is used by being grasped by the examiner13, so that the battery27has a smaller capacity than a capacity of the battery39.

Next, an operation of an embodiment configured as described above will be described with reference toFIG.7toFIG.11.FIG.7is a flowchart for describing the operation of the embodiment, andFIG.8is an explanatory diagram for describing the operation of the embodiment.FIG.7illustrates processing of each unit of the image processing display apparatus30. A horizontal axis inFIG.8represents time, and a series of picked-up images obtained in a swallowing test are arranged along a time axis. Note that by taking into account a size of a sheet inFIG.8, each of the picked-up images is illustrated in a state of being compressed in a horizontal direction. FIG.9is a chart illustrating an example of image quality processing in accordance with a swallow object color.FIG.10andFIG.11are sequence diagrams illustrating a processing sequence in the wireless endoscope20and the image processing display apparatus30. Note that inFIG.10andFIG.11, encircled letters indicate that processing is connected between identical letters. Note that inFIG.10andFIG.11, a suffix n indicates processing at a standard frame rate, a suffix h indicates processing at a high frame rate, a suffix nh indicates processing at a time of transition from the standard frame rate to the high frame rate, and a suffix hn indicates processing at a time of transition from the high frame rate to the standard frame rate.

Upon the swallowing test, initial setting is performed in the wireless endoscope20and the image processing display apparatus30. As illustrated inFIG.10, pairing is implemented between the wireless communication unit25of the wireless endoscope20and the wireless communication unit32of the image processing display apparatus30. When the pairing is established (A1), the control unit31of the image processing display apparatus30causes the wireless communication unit32to transmit frame setting information for deciding an image pickup frame rate of the image pickup device22(A2n). For example, the frame setting information may be a notification for deciding vertical synchronization timing at the time of image pickup of the image pickup device22(image pickup V synchronization notification), and the notification is transmitted for each frame, for example. The control unit24of the wireless endoscope20drives the image pickup device22according to the notification, and sets the image pickup frame rate of the image pickup device22to the standard frame rate such as 30 fps or 60 fps.

The image pickup device22performs image pickup at the standard frame rate (A3n), and outputs the image pickup output obtained by the image pickup to the control unit24. Note that inFIG.10andFIG.11, the image pickup (A3n) is discretely illustrated, but is continuously performed at the designated image pickup frame rate. InFIG.10andFIG.11, due to a space of the sheet, only a small number of frames are illustrated for the image pickup, but in actuality, the image pickup is performed for a time period of the swallowing test.

In the swallowing test, the wireless endoscope20is inserted into the nasal cavity, and the distal end portion of the insertion portion21of the wireless endoscope20is caused to stop at a position (test position) at the upper end of the pharynx2where the pharynx2can be observed. The swallowing test is started in the above described state. In other words, first, the examiner13instructs a subject to put a swallow object such as test food and test water into the mouth and hold the swallow object in the oral cavity. In the above described state, images P1ninFIG.8are obtained. The plurality of images P1ninFIG.8are images before the swallow object flows into the pharynx2immediately after the test start, and are obtained by the image pickup at the standard frame rate. Note that before the swallowing test using the swallow object, dry swallowing without using the swallow object may also be implemented.

After image processing is applied to the image pickup output of the image pickup device22, the control unit24causes the wireless communication unit25to transmit the image pickup output to the image processing display apparatus30via the antenna26(A4n). The wireless communication unit32of the image processing display apparatus30supplies the image pickup output received via the antenna33to the image processing unit34. The image processing unit34performs image processing on the image pickup output (A5n). In other words, in step S1ofFIG.7, the image processing unit34develops and applies predetermined image processing to the image pickup output to be then outputted to the drawing control unit35. The drawing control unit35executes drawing processing for causing the picked-up image to be displayed on the display36(A7n). Accordingly, the picked-up image obtained by the image pickup device22of the wireless endoscope20is displayed by the drawing control unit35on the display screen30aof the display36.

Note that in the image processing (A5n), image processing for obtaining the inflow detection judgement result or the non-inflow detection judgement result is also performed. In step S2ofFIG.7, the image processing unit34of the image processing display apparatus30determines by the image processing whether or not the image part of the swallow object exists in the picked-up image. The image processing unit34outputs to the control unit31the inflow detection judgement result or the non-inflow detection judgement result which is the judgement result in the above described case.

According to the present embodiment, after the image processing (A5n), the control unit31implements control (A6n) on the image pickup frame rate based on the inflow detection judgement result or the non-inflow detection judgement result. When the inflow detection judgement result is inputted, the control unit31shifts the processing from step S2to step S3to determine whether or not the image pickup at the high frame rate is performed. During a period (standard frame rate image pickup period inFIG.8) up to a time point before an image of the swallow object is picked up, the image pickup at the standard frame rate is performed, and the image processing unit34returns the processing from step S3to step S1.

From the start of the swallowing test until an image of the swallow object9ais picked up, the non-inflow detection judgement result is outputted from the image processing unit34, and A2nto A7ninFIG.10are repeated at the standard frame rate. Note that in the control (A6n), the control unit31also provides information on a frame rate of the picked-up image (frame normal notification indicating the standard frame rate) to the drawing control unit35.

Furthermore, for example, when a recording instruction is generated by the examiner13or the like, in the control (A6n) inFIG.10, the control unit31provides the picked-up images from the image processing unit34to the recording processing unit37too to cause the recording processing unit37to start recording of the picked-up images (A8n). The recording processing unit37hereinafter performs movie compression recording of the picked-up images in the memory38(A9n). Note that in the control (A6n), the control unit31also provides information on the frame rate of the picked-up images (frame normal notification indicating the standard frame rate) to the recording processing unit37.

Here, it is assumed that the examiner13issues a swallowing instruction. The subject12swallows the swallow object according to the swallowing instruction by the examiner13. In other words, the tongue of the subject12rises, and the throat moves in conjunction with the above described movement, such that the swallow object9ais sent into the pharynx2. The image pickup device22picks up an image of the pharynx2, and the image part of the swallow object9ais included in the picked-up image.

An image P2ninFIG.8represents a picked-up image in the above described case, and the image part of the swallow object9ais illustrated by the shaded portion. The image is displayed on the display screen30aof the display36by the drawing control unit35(A7nh). In step S2inFIG.7, when it is determined by the image processing that the image part of the swallow object9aexists in the picked-up image, the image processing unit34of the image processing display apparatus30outputs the inflow detection judgement result to the control unit31. For example, it is assumed that the swallow object9ais colored green. In the above described case, when a color of each pixel in the picked-up image is determined and pixels with a same color of green exist as a block in a predetermined region, the predetermined region is assumed to be a swallow object image region9bcorresponding to the swallow object9awhich is colored green, and the image processing unit34outputs the inflow detection judgement result indicating that the swallow object image region9bthat is the image part of the swallow object9aexists in the picked-up image (A5nhinFIG.11). When the inflow detection judgement result is provided, the control unit31shifts the processing from step S2to step S6.

In step S6(A6nhinFIG.11), the control unit31generates frame setting information for changing the image pickup frame rate by the image pickup device22from the standard frame rate to the high frame rate. The frame setting information is transmitted by the wireless communication unit32to the wireless endoscope20via the antenna33(A2h). Note that the frame setting information in the above described case may also be, for example, a notification for shortening a vertical synchronization period (image pickup V synchronization increase notification) at the time of the image pickup of the image pickup device22.

When the frame setting information is received, the wireless communication unit25of the wireless endoscope20provides the frame setting information to the control unit24. Thus, the control unit24performs control to change the image pickup frame rate of the image pickup device22according to the frame setting information. Accordingly, the image pickup device22performs the image pickup at the high frame rate (A3h). As described above, when the swallow object9ais detected in the pharynx2, the image pickup frame rate of the image pickup device22is changed to the high frame rate, and hereinafter the image pickup is performed at the high frame rate of 120 fps, for example. Note that inFIG.10, the image pickup (A3h) is discretely illustrated, but the image pickup is continuously performed at the designated high frame rate.

The high frame rate image pickup period inFIG.8indicates a period during which the high frame rate image pickup is performed. The image pickup output picked up and obtained at the high frame rate is provided to the image processing display apparatus30(A4h). During the high frame rate image pickup period, the image processing unit34of the image processing display apparatus30performs the image processing on the image pickup output that is picked up and obtained at the high frame rate (A5h). Note that during the above described period, the power consumption of the wireless endoscope20increases.

In A6hinFIG.11, when the inflow detection judgement result is provided from the image processing unit34, the control unit31instructs image quality setting for the image processing unit34in accordance with a color of the swallow object9a. In A6h, the image processing unit34analyzes the color of the swallow object. In step S7, the image processing unit34determines a swallow object image region6bas the swallow object color. The image processing unit34sets an image quality with which the swallow object image region6bis conspicuously identifiable in the picked-up image according to the judgement result of the swallow object color such that a change in an image part of a swallow object6ais clarified. For example, the image processing unit34performs an image setting to emphasize a component of a complementary color of the swallow object color for a region other than the swallow object image region9bin the picked-up image, and applies the image processing according to the image setting to the picked-up image. Note that when the image quality setting to emphasize the component of the complementary color is performed, a color which is a color other than the color of the living body and which is not an achromatic color is adopted as the color of the swallow object9a.

Accordingly, picked-up images P3hofFIG.8in which the component of the complementary color of the swallow object color is emphasized for the region other than the swallow object image region9bare obtained. As illustrated inFIG.9, for example, when the swallow object is green, a complementary color is red purple. In the above described case, in step S9, the image processing unit34performs image processing to emphasize a component of red purple which is the complementary color for the region other than the swallow object image region9b. When the swallow object is yellow, for example, a complementary color is blue purple. In the above described case, in step S10, the image processing unit34performs image processing to emphasize a component of blue purple for the region other than the swallow object image region9b. When the swallow object is blue, for example, a complementary color is orange. In the above described case, in step S11, the image processing unit34performs image processing to emphasize a component of orange for the region other than the swallow object image region9b.

The image processing unit34outputs the picked-up image after the image processing to the drawing control unit35and the recording processing unit37. During the high frame rate image pickup period, the adjustment is made to such an image quality that it is easy to clearly distinguish the color of the swallow object9aand the color of the region other than the swallow object image region9bfrom each other, and the smooth picked-up images are displayed on the display screen30aof the display36due to the high frame rate (A7h), and movie compression recording is performed as a movie file by the recording processing unit37(A9h). Since the picked-up images of the swallowing motion are recorded at the high frame rate, it is possible to check the high speed swallowing motion in detail even when the recorded picked-up images are reproduced frame by frame.

When swallowing is normally performed, by the swallowing reflex of the subject12, the epiglottis4blocks the air passage5, and the pharynx2contracts to send the swallow object9ainto the esophagus6. The epiglottis4returns to the original position, and air flows through the air passage5. Then, the swallow object9ais out of an image pickup range, and the image part of the swallow object9ais no longer included in the picked-up image. A picked-up image P4hinFIG.8represents an image obtained in the above described case.

Thus, the image processing unit34determines that the image part of the swallow object9ais no longer included in the picked-up image, and outputs the non-inflow detection judgement result to the control unit31(A5hninFIG.11). In step S3, when it is determined that a current period is the high frame rate image pickup period, the control unit31determines whether or not a predetermined period (for example, several seconds) elapses in the following step S4. Until the predetermined period elapses, S1to S4are repeated. When the predetermined period has elapsed after the image part of the swallow object is no longer included in the picked-up image, the control unit31determines that the swallow object9adoes not remain in the pharynx2or trachea, and generates frame setting information to return the image pickup frame rate to the standard frame rate (A6hn). The frame setting information is supplied to the control unit24via the wireless communication unit32, the antenna33, the antenna26, and the wireless communication unit25(A2n). The control unit24returns the image pickup frame rate of the image pickup device22to the standard frame rate (A3n). Thus, hereinafter, the power consumption of the wireless endoscope20is reduced. Accordingly, the image pickup at the standard frame rate is hereinafter performed, and picked-up images P4ninFIG.8are obtained.

As described above, in the swallowing test, the wireless endoscope20picks up the image of the state before the swallow object is to be swallowed, the image of the state at a moment when the swallow object is swallowed, and the image of the state after the swallow object is swallowed. In the above described case, during the high frame rate image pickup period when and after the swallow object reaches the pharynx2, the picked-up images are obtained at the high frame rate. As a result, the checking of the flow of the swallow object or the like is facilitated also with regard to the swallowing reflex which is performed during a relatively short time period. During the high frame rate image pickup period, the image quality setting at which the distinguishment between the swallow object and the living body is facilitated is applied to the picked-up image, so that the checking of the swallow object or the like is further facilitated. The high frame rate image pickup period during which the power consumption increases is set to have a relatively short time period from a time point when the swallow object is included in the image pickup range of the image pickup device22until the swallowing reflex ends and the swallow object is no longer included in the image pickup range. According to the present embodiment, the increase of the power consumption is suppressed, and the picked-up image effective for the swallowing diagnosis is obtained.

Note that even after the swallow object moves to the esophagus6after the swallowing reflex ends, dysphagia in which the swallow object remains in the pharynx2may be confirmed. In view of the above, in step S2, after the swallow object is detected, in a case where the swallow object is still detected even after a predetermined time period has elapsed, it is assumed that dysphagia in which the swallow object remains is occurring. The control unit31may generate frame setting information for forcedly stopping the image pickup at the high frame rate to return to the image pickup at the standard frame rate.

As described above, according to the present embodiment, by detecting the swallow object, the high frame rate image pickup period is set, and the image quality setting is applied to facilitate identification of the swallow object. The increase of the power consumption is suppressed, and the convenience is ensured. The observation image with a high medical value is provided upon the swallowing diagnosis to be able to improve the effectiveness of the diagnosis.

Note that in the above explanation, the wireless endoscope is described as a swallowing endoscope as an example, but the configuration may also be applied to an endoscope configured to transmit an image pickup output by using a cable.

The present invention is not directly limited to the respective embodiments described above, but can be embodied in an implementation stage by alternating the components in a scope without departing from a gist of the present invention. Various inventions can be formed by appropriate combinations of the plurality of components disclosed in the respective embodiments described above. For example, some components of all components illustrated in the embodiments may also be deleted. Furthermore, components across different embodiments may also be appropriately combined.