Patent ID: 12185916

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, embodiments of the present invention will be explained with reference to the drawings. However, the present invention is not limited by the embodiments which will be explained below.

Note that in the illustration of the drawings, the same or corresponding elements have the same reference characters allotted as appropriate. It should be noted that the drawings are schematic, and a dimensional relationship between respective elements, ratios of the respective elements, and the like in one of the drawings may differ from actual ones. Further, the plurality of drawings may include portions having dimensional relationships and ratios different from one another.

First Embodiment

FIG.1toFIG.6show a first embodiment of the present invention, andFIG.1is a perspective view showing a configuration example of an endoscope apparatus1.

As shown inFIG.1, the endoscope apparatus1of the present embodiment includes an endoscope2, a light source device3, a processor4, a monitor5, and a keyboard6.

Note that although the following will be explained using a case in which the endoscope2is a flexible endoscope as an example, the endoscope2may be a rigid endoscope. Although the following will be explained using a case in which the endoscope2is for medical use as an example, the endoscope2may be for industrial use.

The endoscope2includes an insertion section11, an operation section15, and a universal cable16.

The insertion section11is, for example, an elongated member to be inserted into a subject such as the body of a living organism. The insertion section11includes a flexible tube portion (corrugated tube)12, a bending portion13, and a distal end portion14from a proximal end side to a distal end side.

The flexible tube portion12is a tube portion that is provided to extend from the distal end side of the operation section15in an elongated manner, and has flexibility.

The bending portion13is provided on the distal end side of the flexible tube portion12, and is bendable by an operation of the operation section15. By bending the bending portion13, a direction in which the distal end portion14is directed, that is, a direction of observation with the endoscope2can be changed.

The distal end portion14is provided on the distal end side of the bending portion13, and includes an illumination lens (not shown) and components related to image pickup (components including an image pickup device25) as will be explained later with reference toFIG.2and the like.

The operation section15is a part to be grasped and operated by a user, and is provided with an input device such as a bending operation knob for a bending operation of the bending portion13and various scope switches including a switch for performing an image pickup operation.

The universal cable16extends out of the operation section15, and is configured to be connected to the light source device3with a connector17.

A light guide (not shown) for transmitting illumination light generated by the light source device3and a signal line (not shown) connected to the image pickup device25are disposed in the endoscope2including the insertion section11, the operation section15, and the universal cable16.

The light source device3includes a light source. Illumination light emitted from the light source enters an incident end of the light guide, and is transmitted to an emission end of the light guide positioned in the distal end portion14of the insertion section11. The subject is irradiated with the illumination light emitted from the emission end of the light guide via an illumination lens.

Signal cables18, in which signal lines to be connected to the image pickup device25are disposed, branch and extend out of the connector17. Electric connectors19are provided for distal end portions of the signal cables18, and the electric connectors19are connected to the processor4. The signal lines connected to the image pickup device25are electrically connected to the processor4accordingly. Note that a signal line through which the processor4and the light source device3communicate with each other is also disposed in the signal cables18. Note that a form may be adopted in which the function of the electric connectors19is integrally included in the connector17rather than separately providing the connector17and the electric connectors19. In this case, by electrically connecting the light source device3and the processor4individually, the signal lines from the image pickup device25are electrically connected to the processor4via the light source device3.

The processor4is assumed to be configured such that an ASIC (application specific integrated circuit) including a CPU (central processing unit) or the like, an FPGA (field programmable gate array), or the like reads and executes a processing program stored in a storage device such as a memory (or a recording medium) to carry out a function of each part. However, the processor4is not limited to this, and may be implemented as a dedicated electronic circuit for carrying out the function of each part, for example.

The processor4supplies power to the image pickup device25, and transmits a control signal to the image pickup device25to cause the image pickup device25to perform image pickup. An image pickup signal generated by image pickup performed by the image pickup device25is transmitted to the processor4via the signal line, and is subjected to image processing, so that an image is generated. The processor4generates a video signal for display from the generated image, and outputs the video signal to the monitor5. The processor4also controls the amount of illumination light emitted from the light source device3based on the generated image (or the image pickup signal acquired from the endoscope2) such that brightness of the subject becomes proper.

The monitor5is a display device that receives the video signal from the processor4, and displays an endoscope image.

The keyboard6is an input device connected to the processor4for inputting a command and data in accordance with an operation of a user to the processor4. Note that various devices such as a mouse, a track ball, and a foot switch may further be provided as the input device.

Next,FIG.2is a diagram showing a principal part of components related to image pickup at the distal end portion14of the insertion section11of the endoscope2.

The distal end portion14of the insertion section11is provided with a first optical lens21, a second optical lens22, an optical path length changing filter24, and the image pickup device25.

The first optical lens21forms a first optical image of the subject.

The second optical lens22is provided separately from the first optical lens21, and forms a second optical image of the subject with parallax relative to the first optical image.

Herein, the first optical lens21and the second optical lens22are configured by combining one or more lenses and an optical aperture, for example. Optical lenses of the same design are preferably used for the first optical lens21and the second optical lens22such that the first optical image and the second optical image conform with each other in a case in which the optical path length changing filter24is not present on an optical path.

The first optical lens21and the second optical lens22are arranged in parallel at an angle of convergence such that horizontal binocular parallax when an endoscope image is displayed occurs, for example. A second optical axis O2of the second optical lens22is an optical axis different from a first optical axis O1of the first optical lens21. Such a horizontal arrangement is suitable for a stereoscopic observation mode which will be explained later in a third embodiment.

Note that for obtaining an extended depth-of-focus image, the first optical lens21and the second optical lens22preferably have smaller parallax. For this reason, particularly for an endoscope of a type for observing tissues of a digestive organ at a very proximate position, for example, the first optical axis O1of the first optical lens21and the second optical axis O2of the second optical lens22may be designed to be located as proximate as possible in order to minimize the influence of the angle of convergence.

The optical path length changing filter24is an optical path length changing member that makes an optical path length of the first optical image and an optical path length of the second optical image different. The optical path length changing filter24is provided in an optical path between the second optical lens22and the image pickup device25, for example.

Note that although the example of providing the optical path length changing filter24in an optical path between the second optical lens22and the image pickup device25is shown herein, it goes without saying that the optical path length changing filter24may alternatively be provided in an optical path between the first optical lens21and the image pickup device25.

Specific examples of the optical path length changing filter24include an optical filter formed of a transparent optical material (such as glass or optical plastic) having a refractive index larger than refractive indexes of the first optical lens21and the second optical lens22so as to have a thickness in accordance with an optical path length to be changed.

With such a configuration, the optical path length changing filter24transmits the second optical image so as to make the optical path length between the second optical lens22and the image pickup device25different from an actual distance between the second optical lens22and the image pickup device25.

By providing the optical path length changing filter24in this manner, the first optical image and the second optical image become images having different focuses (being in focus at different positions).

The image pickup device25performs image pickup of the first optical image formed by the first optical lens21to generate a first image pickup signal, and performs image pickup of the second optical image formed by the second optical lens22to generate a second image pickup signal.

The image pickup device25performs image pickup of the first optical image at one portion25aof a single image pickup plane, and performs image pickup of the second optical image at another portion25bof the image pickup plane, for example.

However, the image pickup device25is not limited to the configuration, but may be composed of two image pickup devices divided at a portion indicated by a dashed-and-double-dotted line. In other words, the image pickup device25may have a configuration in which a first image pickup device that performs image pickup of the first optical image and the second image pickup device that performs image pickup of a second optical image are arranged in parallel.

In the case of configuring the image pickup device25by the first image pickup device and the second image pickup device which are separate devices, a member other than the optical path length changing filter24may be employed as the optical path length changing member. For example, a stepped fixing member (an image pickup device holding member) may be used as the optical path length changing member to fix the first image pickup device on a first step of the fixing member and the second image pickup device on a second step of the fixing member, thereby making a distance from the first optical lens21to the first image pickup device along the first optical axis O1and a distance from the second optical lens22to the second image pickup device along the second optical axis O2different, and furthermore, making optical path lengths different.

Further as shown inFIG.2, a focus switching mechanism23that switches a focusing position of the first optical lens21and a focusing position of the second optical lens22to at least two of a near point and a far point may be provided. Employment of such a configuration enables an extended depth-of-focus image at the near point (in a case of observing the subject in a closely proximate state) and an extended depth-of-focus image at the far point (in a case of observing subject tissues at a distance) to be switched and observed.

FIG.3is a flowchart showing depth-of-focus extension and display processing in the endoscope apparatus1.

In main processing (not shown) in the endoscope apparatus1, when, for example, an operation of selecting a depth-of-focus extension and display mode is performed with the keyboard6, the input device of the operation section15, or the like, the depth-of-focus extension and display processing is started.

Then, the first optical image of the subject is formed by the first optical lens21(step S1a), and the second optical image of the subject having a different optical path length from the optical path length of the first optical image is formed by the second optical lens22(step S1b).

Then, image pickup of the first optical image is performed at the one portion25aof the image pickup plane of the image pickup device25to generate the first image pickup signal (step S2a), and image pickup of the second optical image is performed at the other portion25bof the image pickup plane to generate the second image pickup signal (step S2b).

Herein, for example, the image pickup device25is assumed to concurrently perform image pickup of the first optical image and image pickup of the second optical image to generate the first image pickup signal and the second image pickup signal. However, the image pickup device25is not limited thereto, but may alternately perform image pickup of the first optical image and image pickup of the second optical image (hence, make an image pickup start time and an image pickup termination time of the first optical image different from an image pickup start time and an image pickup termination time of the second optical image) to alternately generate the first image pickup signal and the second image pickup signal.

The image pickup device25transmits, via the signal line, the first image pickup signal to the processor4(step S3a), and transmits the second image pickup signal to the processor4(step S3b).

The processor4generates the first image and the second image based on the first image pickup signal and the second image pickup signal generated by concurrent image pickup (or consecutive image pickup in alternate image pickup) (steps S4aand S4b).

Herein,FIG.4is a chart for explaining a procedure of generating a combined image from a first image and a second image, andFIG.5is a chart for explaining a way of generating a combined image from a common portion of a first image and a second image having parallax.

The first image generated from the first image pickup signal is assumed to be an image shown on the left side of an acquired image column inFIG.4, and the second image generated from the second image pickup signal is assumed to be an image shown on the right side of the acquired image column inFIG.4.

One of the first image and the second image generated from the first optical image and the second optical image having binocular parallax is a right-eye image obtained by looking at the subject from the right side, and the other is a left-eye image obtained by looking at the subject from the left side. In this case, coordinates on the first image and coordinates on the second image of a specific position (for example, a position at the back of a lumen) of the subject are typically different. Further, since a distance from the first optical lens21to any position of the subject and a distance from the second optical lens22to the same position are different, the first image and the second image are different in perspective. The first image and the second image do not conform with each other accordingly even if the first image and the second image are overlapped.

Therefore, the processor4subjects at least one of the first image or the second image to image processing of correcting the parallax.

First, the processor4performs processing of correcting a distortion of at least one of the first image or the second image as the image processing of correcting the parallax. In the present embodiment, processing of correcting a distortion caused by the angle of convergence is performed on the first image (step S5a) and on the second image (step S5b).

More specifically, it is assumed that the first image shown on the left side of the acquired image column inFIG.4is the right-eye image, and the second image shown on the right side is the left-eye image. It is further assumed that the first optical axis O1and the second optical axis O2are parallel, and the back of the lumen of the subject is intermediately located between the first optical axis O1and the second optical axis O2.

In this case, the back of the lumen is positioned on the right side of the whole lumen in the right-eye image, and positioned on the left side of the whole lumen in the left-eye image. Further, when considering a square object plane vertical to the first optical axis O1and the second optical axis O2, the object plane appears in a trapezoidal shape with the left side reduced by a perspective in the right-eye image, and the object plane appears in a trapezoidal shape with the right side reduced by the perspective in the left-eye image.

Consequently, the processor4performs shift correction such that a central partial area of the first image is shifted leftward, and a central partial area of the second image is shifted rightward in order that a portion (for example, the back of the lumen of the subject) intermediately located between the first optical axis O1and the second optical axis O2is positioned at the center of the image. In this shift correction, other partial areas are also shifted in accordance with coordinates.

The processor4also performs distortion correction such as trapezoid correction on the first image and the second image such that a distortion caused by the perspective is corrected (see a distortion-corrected image column inFIG.4).

Further, a subject OBJ1appearing at the lower left in the first image shown on the left side of the acquired image column inFIG.4does not appear in the second image shown on the right side of the acquired image column inFIG.4, and a subject OBJ2appearing at the upper right in the second image shown on the right side of the acquired image column inFIG.4does not appear in the first image shown on the left side of the acquired image column inFIG.4.

Thus, the processor4performs processing (trimming processing) of removing an area including a figure present only in one of the first image and the second image. Note that the trimming processing may be performed as processing of extracting areas including a common figure from the first image and the second image, respectively. Hatched areas in a trimmed image column inFIG.4indicate the areas removed by the trimming processing.

In a case of alternately performing image pickup of the first optical image and image pickup of the second optical image as described above, the processor4may further perform processing of correcting a positional displacement between the first image and the second image resulting from a difference in image pickup time point.

In addition, the processor4may perform other types of processing for matching the first image and the second image, such as correction of aberrations (including various aberrations such as a barrel or spool distortion aberration, a spherical aberration, and a chromatic aberration) of the optical lenses and correction of a difference in magnification.

Then, the processor4extracts a first focusing area P1from the first image (step S6a), and extracts a second focusing area P2from the second image (step S6b) (see an extracted image column inFIG.4).

Herein,FIG.6is a chart showing the first focusing area P1extracted from the first image and the second focusing area P2extracted from the second image, as well as a combined focusing area PS of a combined image.

More specifically, with respect to the area including the figure common to the first image and the second image, the processor4causes an area in the first image having contrast higher than contrast of an area in the second image to be included in the first focusing area P1, causes an area in the second image having contrast higher than contrast of an area in the first image to be included in the second focusing area P2, cuts out the first focusing area P1from the first image, and cuts out the second focusing area P2from the second image.

Then, the processor4combines the first focusing area P1and the second focusing area P2to generate an endoscope image which is a combined image (step S7).FIG.5shows a way in which the combined image is generated from a common portion (for example, a portion in which the shape of the subject is the same within a predetermined allowable range in the first image and the second image) of the first image and the second image having parallax.

When generating the combined image, boundary processing of aligning and positioning portions roughly matched in contour and de-emphasizing the boundary (processing of filling pixels little by little from adjacent portions) may be performed. On this occasion, adjacent areas may be matched in hue and brightness.

The endoscope image thus generated is converted by the processor4into a video signal and outputted to be displayed on the monitor5(step S8).

The endoscope image displayed on the monitor5is an extended depth-of-focus image (for example, an image in focus both on the far side (back side) and the proximal side of the lumen) having the combined focusing area PS including the first focusing area P1in the first image and the second focusing area P2in the second image as shown inFIG.6.

Thereafter, the processor4determines whether an operation of terminating the depth-of-focus extension and display mode has been performed (step S9). In a case of determining that the operation has not been performed, the processor4returns to steps S1aand S1bdescribed above to perform the processing as described above. In a case of determining that the operation has been performed, the processor4leaves the processing to return to the main processing (not shown).

According to the first embodiment, optical images are formed by the first and second optical lenses21and22separately configured, respectively, and the parallax between the first image and the second image obtained by performing image pickup of the respective optical images by the image pickup device25is corrected to combine an endoscope image. This enables an extended depth-of-focus image to be observed. At this time, a beam splitter in which a plurality of prisms are combined is not required, which provides advantages that the distal end portion14of the insertion section11of the endoscope2can be prevented from increasing in weight and insertion performance of the endoscope2is less impaired.

The use of the optical path length changing filter24as the optical path length changing member enables the optical path length of the first optical image and the optical path length of the second optical image to be different in a desired manner merely by adjusting the refractive index and the thickness of the optical path length changing filter24. This enables the first optical lens21and the second optical lens22to have the same design, and further enables the first optical image and the second optical image to be formed on the same image pickup plane of the image pickup device25.

Since the first image and the second image are combined after removing the area including the figure present only in one of the first image and the second image, an unnatural feeling when images having parallax are combined can be reduced.

Further, since with respect to the area including the figure common to the first image and the second image, areas having higher contrast are combined to generate an endoscope image, an endoscope image having an appropriately extended depth of focus can be obtained.

In addition, since a distortion of at least one of the first image or the second image is corrected, mismatch when combining the first image and the second image can be reduced to obtain a more natural endoscope image.

In particular, by performing the processing of correcting a distortion caused by the angle of convergence on the first image and the second image, an endoscope image that gives little unnatural feeling, close to an extended depth-of-focus image acquired using a single optical lens and a beam splitter can be obtained.

Since the first image pickup signal and the second image pickup signal generated by concurrent image pickup are processed to generate an endoscope image, a high-integrity endoscope image can be generated based on the first image and the second image with no blur even if relative positions of the distal end portion14and the subject are changed with the lapse of time.

On the other hand, in the case of alternately performing pickup of the first optical image and pickup of the second optical image, a relatively high-integrity endoscope image can be generated based on the first image and the second image having reduced blurs by processing the first image pickup signal and the second image pickup signal generated by consecutive image pickup (that is, image pickup with a small temporal difference) to generate an endoscope image.

In a case of performing image pickup of the first optical image at a portion of the single image pickup plane provided for the image pickup device25and performing image pickup of the second optical image at another portion, an extended depth-of-focus image can be obtained merely by providing the single image pickup device25, which can achieve cost reduction.

In the case of providing the first image pickup device that performs image pickup of the first optical image and the second image pickup device that performs image pickup of the second optical image, the frame rate can be improved by driving the two image pickup devices in parallel.

Further, by providing the focus switching mechanism23that switches the focusing position of the first optical lens21and the focusing position of the second optical lens22to at least two of the near point and the far point, an extended depth-of-focus image at the near point and an extended depth-of-focus image at the far point can be switched and observed.

Thus, even in a case in which the distal end portion14of the endoscope2approaches a subject having a depth, the subject can be observed as a whole in a focusing state while eliminating the need for switching focuses and eliminating focus fine adjustment. This enables a lesion having a depth, for example, to be clearly observed as a whole, and a diagnosis to be performed immediately without requiring the labor such as moving the endoscope2close to the subject or away from the subject.

Second Embodiment

FIG.7shows a second embodiment of the present invention, and is a diagram showing a principal part of components related to image pickup at the distal end portion14of the insertion section11of the endoscope2.

In the second embodiment, portions similar to portions of the above-described first embodiment are denoted the same reference characters and explanation is omitted as appropriate, and different points will mainly be explained.

In the present embodiment, a prism27as a reflecting optical system is further provided as a component related to image pickup at the distal end portion14.

The image pickup device25is provided such that the image pickup plane is vertical to the first optical axis O1and the second optical axis O2in the above-described first embodiment. In contrast, the image pickup device25of the present embodiment is arranged such that the image pickup plane is parallel to the first optical axis O1and the second optical axis O2, for example.

The prism27reflects the first optical image from the first optical lens21and the second optical image from the second optical lens22to the image pickup device25.

Note that the image pickup device25performs image pickup of the first optical image at the one portion25aof the image pickup plane, and performs image pickup of the second optical image at the other portion25bof the image pickup plane, similarly to the above-described first embodiment.

In addition, the image pickup device25may be composed of two image pickup devices, and the focus switching mechanism23that switches the focusing positions may further be provided, similarly to the above-described first embodiment.

Further, the image pickup device25may perform image pickup of the first optical image and image pickup of the second optical image concurrently or alternately, similarly to the above description in the first embodiment.

In the case of alternately performing image pickup, for example, a shutter26that controls passage/light shielding of the first optical image and passage/light shielding of the second optical image may further be provided.

The shutter26in the configuration example shown inFIG.7includes a light shielding member configured to rotate about a rotation shaft26a, for example, and is configured such that the first optical image is shielded and the second optical image is passed when the light shielding member is located on the first optical axis O1, and the second optical image is shielded and the first optical image is passed when the light shielding member is located on the second optical axis O2.

Alternate image pickup of the first optical image and the second optical image as explained in the above-described first embodiment may be performed using the shutter26of such a configuration.

However, the use of the shutter26is not a limitation, and electronic shutters may be independently applied to the one portion25aof the image pickup plane and the other portion25bof the image pickup plane to close the electronic shutter for the other portion25bof the image pickup plane (for example, reset a photodiode) when the electronic shutter for the one portion25aof the image pickup plane is opened (more specifically, charges are being accumulated in the photodiode), and to open the electronic shutter for the other portion25bof the image pickup plane when the electronic shutter for the one portion25aof the image pickup plane is closed, thereby alternately performing image pickup of the first optical image and the second optical image.

Further, the one portion25aof the image pickup plane and the other portion25bof the image pickup plane may be configured as different pixel groups, and reading from the pixel group of the other portion25bmay be performed when image pickup is performed with the pixel group of the one portion25a, and reading from the pixel group of the one portion25amay be performed when image pickup is performed with the pixel group of the other portion25b.

The second embodiment exerts effects substantially similar to the effects of the above-described first embodiment, and even if the single prism27is provided, an extended depth-of-focus image can be obtained while achieving weight reduction as compared with the case of combining a plurality of prisms.

Since the optical path can be bent by the prism27, the flexibility in layout of the image pickup device25can be increased, and a design suitable for the model of the endoscope2(to cite an example, a side-viewing endoscope) can be performed.

Third Embodiment

FIG.8shows a third embodiment of the present invention, and is a diagram showing a principal part of components related to image pickup at the distal end portion14of the insertion section11of the endoscope2.

In the third embodiment, portions similar to portions of the above-described first and second embodiments are denoted the same reference characters and explanation is omitted as appropriate, and different points will mainly be explained.

In the configuration of the first embodiment shown inFIG.2, the optical path length changing filter24is provided fixedly in the optical path between the second optical lens22(or the first optical lens21) and the image pickup device25. However, the present embodiment is configured such that the optical path length changing filter24can be inserted/withdrawn from above the optical path.

In other words, the present embodiment further includes, as a component related to image pickup at the distal end portion14, an actuator29as a driving mechanism that inserts/withdraws the optical path length changing filter24into/from the optical path between the second optical lens22and the image pickup device25(which may be the optical path between the first optical lens21and the image pickup device25; the same applies hereinafter). The actuator29inserts the optical path length changing filter24into the optical path or withdraws the optical path length changing filter24from the optical path based on control of the processor4.

In such a configuration, in the case in which the depth-of-focus extension and display mode which is a first mode is selected, the processor4controls the actuator29to insert the optical path length changing filter24into the optical path between the second optical lens22and the image pickup device25.

Then, similarly to the above-described first and second embodiments, the processor4processes the first image pickup signal generated by performing image pickup of the first optical image and the second image pickup signal generated by performing image pickup of the second optical image formed with the optical path length different from the optical path length of the first optical image, thereby generating an extended depth-of-focus endoscope image.

In a case in which a stereoscopic observation mode (3D mode) which is a second mode is selected, the processor4controls the actuator29to withdraw the optical path length changing filter24from the optical path between the second optical lens22and the image pickup device25.

Then, the processor4processes the first image pickup signal generated by performing image pickup of the first optical image and the second image pickup signal generated by performing image pickup of the second optical image formed with the optical path length identical to the optical path length of the first optical image, thereby generating the first image and the second image.

At this time, the processor4generates a 3D endoscope image for stereoscopic vision utilizing the parallax without performing the processing of correcting the parallax between the first image and the second image. The 3D endoscope image thus generated is displayed on the monitor5corresponding to stereoscopic display.

Note that also in the configuration of the present embodiment, the image pickup device25may concurrently or alternately perform image pickup of the first optical image and image pickup of the second optical image. Further, it goes without saying that the prism27as the reflecting optical system as explained in the above-described second embodiment may be provided for the configuration of the present embodiment.

Since the third embodiment exerts effects substantially similar to the effects of the above-described first and second embodiments, and the actuator29as the driving mechanism that inserts/withdraws the optical path length changing filter24into/from the optical path is further provided, a 3D endoscope image utilizing the parallax can be generated when the optical path length changing filter24is withdrawn from the optical path.

In this manner, the endoscope apparatus1having multiple functions that enables an extended depth-of-focus endoscope image and a 3D endoscope image to be switched and observed.

Note that although the case in which the present invention is an endoscope apparatus has been mainly explained above, the present invention is not limited thereto, but may be a method for generating an endoscope image by a method similar to a method of the endoscope apparatus, a computer program for causing a computer to perform processing similar to processing of the endoscope apparatus, a computer-readable non-transitory recording medium having the computer program recorded on the recording medium, or the like.

The present invention is not limited to the embodiments as described above, but can be embodied with constitutional elements modified within a range not deviating from the gist in an implementation phase. In addition, various aspects of the invention can be formed by appropriate combinations of a plurality of constitutional elements disclosed in the above-described embodiments. For example, some constitutional elements may be removed from all the constitutional elements shown in the embodiments. Further, constitutional elements in different embodiments may be combined as appropriate. In this manner, it is obviously possible to make various modifications and applications within the range not deviating from the gist of the invention.