Line of sight detecting device and method

A line of sight detecting method includes estimating a face direction of an object person based on a shot face image of the object person, detecting a part of an eye outline in the face image of the object person, detecting a pupil in the face image of the object person, and estimating the direction of a line of sight of the object person based on the correlation of the pupil position in the eye outline and the face direction with respect to the direction of the line of sight, and the pupil position and the face direction of the object person.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a line of sight detecting device and method, and more particularly, relates to a line of sight detecting device and method capable of detecting more accurate direction of the line of sight.

2. Description of Related Art

Conventionally, it is proposed to decide a direction of a line of sight (hereinafter referred to as a LOS) based on the center position of a pupil in a reference image when a user is looking at five positions, i.e., the center position and the four corners of a display and the center position of a pupil in a detected image for detecting the LOS. (see Japanese patent publication No. 2006-141862)

However, the center position of a pupil in the detected image is varied depending on a direction and a position of a user's face even if the user's looking directions are same. Therefore it is difficult to decide an accurate direction of LOS only by using the center position of a pupil. The Japanese patent publication suggests that the direction and position of the user's face are detected and combined with the direction of pupil to decide the direction of the user's LOS. However, the specific processes are not described.

The present invention has been created to make it possible to detect an accurate direction of a LOS.

SUMMARY

A LOS detecting device according to an aspect of the present invention includes a face-direction estimation means for estimating a face-direction of an object person based on a shot face image of the object person to detect the LOS direction, eye detection means for detecting a part of an eye-outline of in the face image of the object person, pupil detection means for detecting a pupil in the face image of the object person and LOS direction estimation means for estimating the LOS direction of the object person based on the correlation of the pupil position in the eye-outline and the face direction with respect to the LOS direction, and the pupil position and the face direction of the object person

The LOS detecting device according to an aspect of the present invention estimates the face direction of the object person based on the shot face image of the object person to detect the LOS direction, detects a part of the eye-outline in the face image of the object person, detects the pupil in the face image of the object person and estimates the LOS direction of the object person based on the correlation of the pupil position in the eye-outline and the face direction with respect to the LOS direction, and the pupil position and the face direction of the object person. Thus, an accurate LOS direction can be detected.

The face direction estimation means, eye detection means, pupil detection means and LOS direction estimation means are configured with a CPU (Central Processing Unit).

The LOS direction estimation means can estimate the LOS direction of the object person based on the correlation of the position of the pupil, the face direction and the shape of the eye-outline with respect to the LOS direction, and the pupil position, face direction and the shape of the eye-outline of the object person. Thus, an accurate LOS direction can be detected.

The LOS direction detection device further includes correlation analysis means for analyzing the correlation based on the pupil position data in multiple combinations of the face directions and the LOS directions, wherein the LOS direction estimation means can estimate the LOS direction of the object person based on the correlation analyzed by the correlation analysis means. Thus, an accurate LOS direction can be detected flexibly responding to a use condition and a use environment.

The correlation analysis means is configured with a CPU (Central Processing Unit).

The LOS detecting method according to an aspect of the present invention includes steps of estimating the face direction of the object person based on a shot face image of the object person to detect the LOS direction, detecting a part of the eye-outline in the face image of the object person, detecting the pupil in the face image of the object person and estimating the LOS direction of the object person based on the correlation of the pupil position in the eye-outline and the face direction with respect to the LOS direction, and the pupil position and the face direction of the object person

The LOS detecting method according to an aspect of the present invention estimates the face direction of an object person based on a shot face image of the object person to detect the LOS direction, detects a part of the eye-outline in the face image of the object person, detects the pupil in the face image of the object person and estimates the LOS direction of the object person based on the correlation of the pupil position in the eye-outline and the face direction with respect to the LOS direction, and the pupil position and the face direction of the object person Thus, an accurate LOS direction can be detected.

The steps of estimating the face direction, detecting the eye-outline, detecting the pupil and estimating the LOS direction are executed by a CPU (Central Processing Unit).

DETAILED DESCRIPTION

In the following detailed description, reference is made to the accompanyingFIGS. 1 through 11, which illustrate the described embodiments. The embodiments are described in sufficient detail to enable those skilled in the art to make and use them.

FIG. 1is a block diagram showing an embodiment of a LOS detection system according to the present invention. The LOS detection system shown inFIG. 1is configured to include a camera111and a LOS detection device112.

The camera111takes a shot of the face of a person whose LOS is detected (hereinafter referred to as an object person) and send the shot image (hereinafter referred to as a face image) to the LOS detection device112.

The LOS detection device112detects the pupil direction and LOS direction of the object person appearing in the face image and outputs the data including the detection results to the latter part of the LOS detection device112.

Next, the face direction, the pupil direction and the LOS direction are defined in reference toFIG. 2. The arrow211shows the direction of the pupil of the object person201, the axis212shows the direction of the face of the object person201(the front face direction of the face of the object person201) and the optical axis213shows the optical axis of the camera111.

First, the face direction of the object person201is defined in reference to the optical axis213of the camera111, in other words, as the face direction viewed from the camera111. For example,FIG. 2shows that the face direction of the object person201is in the direction of an axis212in reference to the optical axis213, i.e. in the direction at an angle of α+β on the right. The face direction on the right side or the upper side in reference to the optical axis213is a positive direction, while the face direction on the left side or the lower side in reference to the optical axis213is a negative direction.

Secondly, the pupil direction of the object person201is defined in reference to the face direction of the object person201, in other words, as the LOS direction in reference to the face direction of the object person201. For example,FIG. 2shows that the pupil direction of the object person201is in the direction of an arrow211in reference to the axis212, i.e. in the direction at an angle of α on the left. The pupil direction on the right side or the upper side in reference to the axis212is a positive direction, while the pupil direction on the left side or the lower side in reference to the axis212is a negative direction.

Further, the LOS direction of the object person201is defined as the direction of the pupil in reference to the optical axis213of the camera111, in other words, as the LOS direction of the object person201viewed from the camera111. For example,FIG. 2shows that the LOS direction of the object person201is in the direction of an arrow211in reference to the optical axis213, i.e. in the direction at an angle of β on the right. The LOS direction on the right side or the upper side in reference to the optical axis213is a positive direction, while the LOS direction on the left side or the lower side in reference to the optical axis213is a negative direction.

Going back toFIG. 1, the LOS detection device112is configured to include a learning part121and a detection part122. The learning part121, as described later in reference toFIG. 3, etc., learns the correlation of the pupil position in the eye-outline and the face direction with respect to the pupil direction or learns the correlation of the pupil position in the eye-outline, the face direction and the shape of the eye-outline with respect to the LOS direction, based on the face image and the face data including the face direction and pupil direction of the object person when the face image is shot. The learning part121is configured with an eye detection part131, a pupil detection part132, a feature parameter calculation part133, a leaning data generation part134, a learning data accumulation part135and a correlation analysis part136.

The eye detection part131, as described later in reference toFIG. 3, etc., detects a part of the outline of the eye in the face image of the object person using a given method. The eye detection part131sends the face image and the data showing the detection results to the pupil detection part132. Further, the eye detection part131sends the data showing the detection results to the feature parameter calculation part133.

The pupil detection part132, as described later in reference toFIG. 3, etc., detects the pupil in the face image of the object person using a given method. The pupil detection part132sends the data showing the detection results to the feature parameter calculation part133.

The feature parameter calculation part133, as described later in reference toFIG. 3, etc., calculates a feature parameter showing the pupil position in the eye-outline or calculates a feature parameter showing the pupil position in the eye-outline and the shape of the eye-outline, based on the eye-outline and the detection results of the pupil of the object person. The feature parameter calculation part133sends the data showing the calculated feature parameter to the learning data generation part134.

The learning data generation part134generates learning data correlating the pupil direction of the object person with the feature parameter calculated based on the face direction when the face image is shot and the face image. The learning data generation part134accumulates the generated learning data in the learning data accumulation part135. The learning data generation part134notifies to the correlation analysis part136that a certain amount of the learning data has been accumulated in the learning data accumulation part135.

The correlation analysis part136analyzes the correlation of the face direction and the feature parameter with respect to the pupil direction based on the learning data accumulated in the learning data accumulation part135and sends the data showing the analysis results to the pupil direction estimation part145in the detection part122.

The detection part122, as described later in reference toFIG. 7, etc., detects the pupil direction and the LOS direction of the object person based on the face image of the object person shot by the camera111and the correlation learned by the learning part121. The detection part122is configured to include a face direction estimation part141, an eye detection part142, a pupil detection part143, a feature parameter calculation part144, a pupil direction estimation part145and a LOS direction calculation part146.

The face direction estimation part141estimates the face direction of the object person based on the face image of the object person shot by the camera111using a given method. The face direction estimation part141sends the face image and the data showing the estimation results to the eye detection part142. The face direction estimation part141sends the data showing the estimation results to the pupil direction estimation part145and the LOS direction calculation part146.

The eye detection part142, in the same process as that of the eye detection part131, detects a part of the eye-outline in the face image of the object person using a given method. The eye detection part142sends the face image and the data showing the detection results to the pupil detection part132. Further, the eye detection part142sends the data showing the detection results to the feature parameter calculation part144.

The pupil detection part143, in the same process as that of the pupil detection part132in the learning part121, detects the pupil in the face image of the object person using a given method. The pupil detection part143sends the data showing the detection results to the feature parameter calculation part144.

The feature parameter calculation part144, in the same process as that of the feature parameter calculation part133in the learning part121, calculates the feature parameter based on the eye-outline of the object person and the detection results of the pupil. The feature parameter calculation part144sends the data showing the calculated feature parameter to the pupil direction estimation part145.

The pupil direction estimation part145estimates the pupil direction of the object person based on the correlation of the face direction and the feature parameter with respect to the pupil direction, the face direction and the feature parameter of the object person. The pupil direction estimation part145sends the data showing the estimated pupil direction to the LOS direction calculation part146and the latter part of the LOS detection device112.

The LOS direction calculation part146calculates the LOS direction of the object person based on the face direction and the pupil direction of the object person. The LOS direction calculation part146sends the data showing the calculated LOS direction to the latter part of the LOS detection device112.

Next, the operation of the LOS detection system101is described in reference toFIG. 3throughFIG. 7. First, the learning process executed by the LOS detection device112is described in reference to the flowchart shown inFIG. 3.

In step S1, the learning part121obtains the face data. Using the camera111, the user takes a shot of the face of the object person with a face direction and a pupil direction. The user inputs to the learning part121the face image and the data showing the face direction and the pupil direction when the face image is shot. The eye detection part131in the learning part121obtains the face image from the inputted face data, and the learning data generation part134obtains the data showing the face direction and the pupil direction of the object person from the inputted face data.

In step S2, the eye detection part131detects the eye-outline in the face image of the object person using a given method. The eye detection part131sends the face image and the data showing the detection results to the pupil detection part132. Further, the eye detection part131sends the data showing the detection results to the feature parameter calculation part133.

In step S3, the pupil detection part132detects the pupil in the face image of the object person using a given method. The pupil detection part132uses the detection results of the eye-outline of the object person as necessary. The pupil detection part132sends the data showing the detection results to the feature parameter calculation part133.

In step S4, the feature parameter calculation part133calculates the feature parameter based on the eye-outline and the pupil detection results of the object person. The feature parameter calculation part133sends the data showing the calculated feature parameter to the learning data generation part134.

In reference toFIGS. 4 and 5, the specific processes of the steps S2through S4are described. Using a given method, the eye detection part131detects the left and the right ends (Pe1and Pe2) of an eye which are parts of the outline of the eye251as shown inFIG. 4. The method of detecting the both ends of the eye is not limited to a particular method. The methods capable of detecting them more accurately, quickly and simply are preferable. For example, the method of defining Pe1and Pe2after detecting the eye-outline using edge-detection or detecting Pe1and Pe2using corner-detection can preferably be used.

Using a given method, the pupil detection part132detects the left end Pp1and the right end Pp2of the pupil252in the face image as shown inFIG. 4. The method of detecting the both ends of the pupil252is not limited to a particular method. The methods capable of detecting them more accurately, quickly and simply are preferable. For example, the method of detecting the left end Pp1and the right end Pp2after detecting the outline of the pupil252using edge-detection can preferably be used.

The feature parameter calculation part144calculates the feature parameter showing a correlation of the both eye ends Pe1and Pe2with respect to the both pupil ends Pp1and Pp2. For example, the feature parameter can be the relative coordinate of the eye end Pe1with respect to the pupil end Pp1, in other words, the vector in between the pupil end Pp1and the eye end Pe1, and the relative coordinate of the eye end Pe2with respect to the pupil end Pp2, in other words, the vector in between the pupil end Pp2and the eye end Pe2. Alternatively, the ratio of the length of the both vectors can be the feature parameter. Any feature parameters represent the position of the pupil with respect to the eye-outline, more specifically with respect to parts of the eye-outline, the left end Pe1and the right end Pe2.

Furthermore, additional parameters can be the relative coordinate of the right end Pp2with respect to the left end Pp1of the pupil, or the relative coordinate of the left end Pp1with respect to the right end Pp2and the length in between the left end Pp1and the right end Pp2of the pupil.

Alternatively, using a given method such as an elliptical degree in separation filter, the eye detection part131detects a given number of feature points Pe11through Pe22, e.g. 12 feature points on the outline of the eye251as shown inFIG. 5. The method of detecting the feature points on the outline of the eye251is not limited to a particular method. The methods capable of detecting them more accurately, quickly and simply are preferable.

The pupil detection part132, using a given method, detects the center Pp11of the pupil252in the face image as shown inFIG. 5. The method of detecting the center Pp11of the pupil252is not limited to a particular method. The methods capable of detecting it more accurately, quickly and simply are preferable.

The feature parameters described above are merely examples. Other feature parameters representing the position of the pupil252with respect to the outline of the eye251and the shape of the eye-outline can be used. For example, inFIG. 4, the number and the positions of the feature points on the outline of the detected pupil252can be changed or the center of the pupil252can be used to create a feature parameter instead of using the feature points on the outline of the pupil252. Further, the width and height of the outline of the eye251and a parameter showing the shape of the pupil can be added to the feature parameters.

According to the above description, the learning part121detects the feature points of the outline of the eye251and the pupil252. Alternatively, a user can detect the feature points by reviewing the face image and include the coordinate of the detected feature points or the feature parameters calculated from the detected feature points into the face data.

In the above example, description is made to the right eye. It goes without saying that the same process can be applied to create the feature parameters for the left eye.

In step S5, the learning data generation part134generates learning data. Specifically, the learning data generation part134generates data correlating the pupil direction of the object person with respect to the feature parameters calculated based on the face direction and the face image when the face image is shot. The learning data generation part134accumulates the generated learning data into the learning data accumulation part135.

In step S6, the learning data generation part134judges whether a given amount of the learning data is accumulated. If judged not so, the process goes back to step S1, and the processes of steps S1through S6are repeatedly executed until it is judged that the given amount of the learning data is accumulated.

The user takes a shot of the face image of the object person with the face direction and pupil direction changing and inputs to the learning part121the shot face image and the data showing the face direction and the pupil direction of the object person when the face image is shot, thus the processes of steps S1through S6are repeatedly executed. In this way, the feature parameters are calculated based on multiple different combinations of the face directions and the pupil directions of the object person and the learning data with respect to the different respective combinations is accumulated.

It is preferable to collect the learning data based on a large number of combinations of the face directions and the pupil directions and to collect the learning data based on a large number of the object persons for accurately analyzing the correlation of the face direction and the feature parameter with respect to the pupil direction in the later described processes. Moreover, it is preferable to select the object persons diversified in age, gender, shape of face and positions of respective face parts.

In step S6, the learning data generation part134, when judging that a given amount of the learning data is accumulated, notify it to the correlation analysis part136and the process goes to step S7.

In step S7, the correlation analysis part136, using a given method, analyzes the correlation of the face direction and the feature parameter with respect to the pupil direction based on the learning data accumulated in the learning data accumulation part135. The method of analyzing the correlation is not limited to a particular method. The methods capable of analyzing it more accurately, quickly and simply are preferable.

For example, as shown inFIG. 6, the distribution of the learning data is approximated by a hyperplane301in a coordinate space with axes of the feature parameters, the face direction and the pupil direction The correlation analysis part136calculates the approximation representing the hyperplane301as the correlation of the face direction and the feature parameter with respect to the pupil direction, for example by Canonical Correlation Analysis (CCA). In the coordinate space shown inFIG. 6, the feature parameter is represented by a single axis for ease of explanation, but actually the same number of axes are provided for the respective parameters included in the feature parameter.

In step S8, the correlation analysis part136outputs the analysis results. The correlation analysis part136outputs to the pupil direction estimation part145the data showing the analysis results of the correlation of the face direction and the feature parameter with respect to the pupil direction. After that, the learning process terminates.

Next, the detection process executed by the LOS detection device112is described in reference to the flowchart shown inFIG. 7. The process starts when the face image of the object person shot by the camera111enters to the face direction estimation part141.

In step S51, the face direction estimation part141estimates the face direction. Specifically, the face direction estimation part141, using a given method, estimates the face direction of the object person appearing in the face image. The method of estimating the face direction is not limited to a particular method. The methods capable of estimating it more accurately, quickly and simply are preferable. The face direction estimation part141sends the face image and the data showing the estimation results to the eye detection part142. Further, the face direction estimation part141sends the data showing the estimation results to the pupil direction estimation part145and the LOS direction calculation part146.

In step S52, the eye detection part142detects the eye-outline in the face image of the object person in the same process as that of the eye detection part131in step S2ofFIG. 3. The eye detection part142sends the face image and the data showing the detection results to the pupil detection part143. Further, the eye detection part142sends the data showing the detection results to the feature parameter calculation part144.

In step S53, the pupil detection part143detects the pupil in the face image of the object person in the same process as that of the pupil detection part132shown in step S3ofFIG. 3. The pupil detection part143sends the data showing the detection results to the feature parameter calculation part144.

In step S54, the feature parameter calculation part144calculates the feature parameter based on the eye-outline and detection results of the pupil of the object person in the same process as that of the feature parameter calculation part133shown in step S4ofFIG. 3. The feature parameter calculation part144sends the data showing the calculated feature parameter to the pupil direction estimation part145.

In step S55, the pupil direction estimation part145estimates the pupil direction. Specifically, the pupil direction estimation part145estimates the pupil direction by applying the face direction estimated by the face direction estimation part141and the feature parameter calculated by the feature parameter calculation part144to the correlation of the face direction and the feature parameter with respect to the pupil direction, the correlation being analyzed by the correlation analysis part136. As described above, the feature parameter represents the position of the pupil in the eye-outline or the position of the pupil in the eye-outline and the shape of the eye-outline. In other words, the pupil direction estimation part145estimates the pupil direction of the object person based on the correlation of the pupil position in the eye-outline and the face direction with respect to the LOS direction, and the pupil direction in the eye-outline and the face direction of the object person. Alternatively, the pupil direction estimation part145estimates the pupil direction of the object person based on the pupil position in the eye-outline and the correlation of the face direction and the shape of the eye-outline with respect to the LOS direction, and the pupil position in the eye-outline, the face direction and the shape of the eye-outline of the object person. The pupil direction estimation part145sends the data showing the estimated pupil direction to the LOS direction calculation part146.

In step S56, the LOS direction calculation part146calculates the LOS direction. Specifically, the LOS direction calculation part146calculates the LOS direction of the object person by adding the face direction estimated by the face direction estimation part141to the pupil direction estimated by the pupil direction estimation part145. For example, if the face direction is estimated horizontally Θfx°, vertically Θfy°and the pupil direction is estimated horizontally Θpx°, vertically Θpy°, then LOS horizontal and vertical directions Θsx°and Θsy°are calculated by the following formulas (1) and (2) respectively.
Θsx°=Θfx°+Θpx°(1)
Θsy°=Θfy°+Θpy°(2)

In step S57, the detection part122detects the pupil direction and the LOS direction. Specifically, the pupil direction estimation part145outputs the data showing the estimated pupil direction to the latter stage of the LOS detection device112and the LOS direction calculation part146outputs the data showing the calculated LOS direction to the latter stage of the LOS detection device112. After that, the detection process terminates.

In this way, the pupil direction and the LOS direction of the object person can be accurately detected regardless of face directions of object persons or differences among individuals.FIGS. 8 through 10illustrate examples of detecting pupil directions with different face directions in reference to the center position of the pupil with respect to the both eye ends when the face and the LOS of the object person are directed to front face.

FIG. 8shows an example of positions of left-end eye Pe51, right-end eye Pe52and pupil center Pb51when the face and the pupil of an object person351are facing toward the camera111. The pupil direction is detected as described below based on the positional relation of the pupil center Pb51to the left-end eye Pe51and the right-end eye Pe52.

As shown inFIG. 9, when the pupil is directed leftwards at Θ1°with the face of the object person351facing toward the camera111, the right-end eye Pe61and the left-end eye Pe62are detected at substantially the same positions as those of the right-end eye Pe51and the left-end Pe52shown inFIG. 8. In this case, the reference position Px61for determining the pupil direction of the object person351is set so that the length in between the right-end eye Pe61and the reference position Px61: the length in between the reference position Px61and the left-end eye Pe62≈the length in between the right-end eye Pe51and the pupil center Pb51: the length in between the pupil center Pb51and the left-end eye Pe52, thus the reference position Px61is set nearly at the same position as that of the pupil center Pb51shown inFIG. 8. Accordingly, the pupil direction of the object person351can be accurately detected as it is directed leftwards at Θ1°based on the actual difference between the pupil center Pb61and the reference position Px61.

On the other hand, as shown inFIG. 10, when the pupil is directed in front face with the face of the object person351facing leftwards at Θ1°to the camera111, the reference position Px71is set so that that the length in between the right-end eye Pe71and the reference position Px71: the length in between the reference position Px71and the left-end eye Pe72≈the length in between the right-end eye Pe51and the pupil center Pb51: the length in between the pupil center Pb51and the left-end eye Pe52. The pupil direction is falsely detected as it is directed leftwards despite it is directed in front face (0°) because the actual pupil center Pb71is displaced leftwards from the reference position Px71.

The LOS direction detection device112can accurately estimate the pupil direction of the object person351even if the object person351does not face toward the camera111as shown inFIG. 10, because, as described above, the pupil directions are estimated based on the data showing the pupil positions with respect to the eye-outline in the multiple combinations of the face directions and the pupil directions.

It becomes possible to accurately detect the pupil direction and the LOS direction by learning the above described correlation, thus flexibly responding to various conditions and environments where the LOS detection system101is used, for example, the performances of the camera111or the positions where it is installed.

In the above description, the pupil direction is used for the face data and the learning data. Instead of the pupil direction, the LOS direction, i.e. the direction of the LOS viewed from the camera111can be used. Thus, following that the LOS direction of the object person is estimated in step S55ofFIG. 7as described above, the pupil direction can be calculated in step S56based on the face direction and the LOS direction of the object person.

If the object persons are specific individuals, the more accurate detection of the pupil direction and the LOS direction becomes possible by measuring the learning data for the each individual and analyzing and utilizing the correlation for the each individual.

One or more embodiments of the present invention can be applied to devices using the LOS and the view points of users, for example, a camera for controlling shooting by detecting the LOS of an object person, a driving support device for supporting a driver detecting the driver's LOS, a display for controlling displaying based on a user's view point and a suspicious individual detecting device for detecting suspicious individuals by detecting changes in suspicious persons' LOS.

The processes described above can be executed by a hardware or by a software. If it is executed by a software, the program configuring the software can be built in a device or installed through a program recording media in a general-purpose personal computer for executing a various functions with a various programs installed.

FIG. 11is a block diagram showing an example of a configuration of a computer for executing programs of above described processes. CPU (Central Processing Unit)501, ROM (Read Only Memory)502and RAM (Random Access Memory)503are mutually connected by bus504.

The bus504is connected to IO interface505. The IO interface505is connected to input part506including a keyboard, mouse and microphone, output part507including a display and a speaker, memory508including a hard disc and nonvolatile memory, communication part509including a network interface and a drive510for driving removable media511including a magnetic disc, a photo disc, a photomagnetic disc or a semiconductor memory.

In a computer configured above, the CPU501loads the programs stored in the memory508into the RAM503through the IO interface505and the bus504and executes them, thus the above described processes are executed.

The programs executed by the computer are provided by a packaged removable media such as a magnetic disc including a flexible disc, CD-ROM, DVD, a photomagnetic disc or a semiconductor memory or provided through a wired or wireless transmission media such as LAN, internet and digital satellite broadcasting.

The programs can be installed in the memory508through IO interface505by loading the removable media511to the drive510. Alternatively, the programs can be received by the communication part509through a wired or wireless transmission media and installed in the memory508. Furthermore, the programs can be previously installed in the ROM502or memory508.

The programs executed in the computer can be executed in time series according to the order described in the present specification, or can be executed in parallel or upon a calling when it is requested.

The term “system” used in the present specification means an overall device configured with multiple devices and means.

In the above description and drawings are only to be illustrative of exemplary embodiments which achieve the aspects and advantages described above. Modifications of, and substitutions to, specific operating conditions and structures can be made without departing from the spirit and scope of the invention.

Accordingly, the claimed invention is not to be considered as being limited by the foregoing description and drawings, but is only limited by the scope of the appended claims.