Information processing apparatus and image processing apparatus with a line of sight detector

An information processing apparatus includes a controller that controls the information processing apparatus by making a transition of a state of the information processing apparatus in relation to power consumption of the information processing apparatus from a first state to a second state in which the power consumption is higher than that in the first state, and a line-of-sight detector that detects a line of sight toward the information processing apparatus. The controller makes a transition from the first state to the second state when the line-of-sight detector detects a line of sight toward the information processing apparatus.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under USC 119 from Japanese Patent Application No. 2016-061694 filed on Mar. 25, 2016.

BACKGROUND

Technical Field

The present invention relates to an information processing apparatus and an image processing apparatus.

SUMMARY

According to an aspect of the invention, there is provided an information processing apparatus, including: a controller that controls the information processing apparatus by making a transition of a state of the information processing apparatus in relation to power consumption of the information processing apparatus from a first state to a second state in which the power consumption is higher than that in the first state; and a line-of-sight detector that detects a line of sight toward the information processing apparatus, wherein the controller makes a transition from the first state to the second state when the line-of-sight detector detects a line of sight toward the information processing apparatus.

DETAILED DESCRIPTION

FIG. 1illustrates a hardware configuration of an image processing apparatus10according to an example. The image processing apparatus10provides image processing functions such as scan, print, copy, and facsimile. The image processing apparatus10is a computer which includes a CPU (Central Processing Unit)11, a RAM (Random Access Memory)12, a ROM (Read Only Memory)13, an HDD (Hard Disk Drive)14, an operation panel15, an image scanner16, a printer17, a transport mechanism18, and a digital camera19.

The CPU11executes a program stored in the ROM13or the HDD14using the RAM12as a work area to control operations of the respective units. The HDD14is a memory which stores data and the program used in the control of the CPU11. The memory is not limited to the HDD, and may be another recording medium such as an SSD (Solid State Drive).

The operation panel15is provided with a touch screen and a button. The operation panel15displays information indicating a state of the own apparatus (the image processing apparatus) and a state of the process, and displays an image such as an operator to receive a user's operation, and receives an operation when the button is pressed. In this way, the operation panel15is an instance of an operation receiving unit which receives an operation of the own apparatus. The image scanner16is provided with an image sensor, and optically reads out an image displayed on the surface of a sheet. The image scanner16supplies image data showing the read image to the CPU11.

The printer17forms, for instance, an image on a sheet by an electrophotographic process. The image scanner16and the printer17both are an instance of a processing unit which performs a process on the sheet. The transport mechanism18is a mechanism which transports the sheet, and transports a sheet from which the image is read out by the image scanner16and a sheet on which the image is formed by the printer17. The digital camera19is provided with a lens and an image capturing device, and captures a person or a scene of surroundings which are expressed by the incident light passing through the lens. The digital camera19in this example is a visible light camera which captures an image with the visible light. The digital camera19may be an infrared camera which captures an image with the infrared light, or may be an integration type camera which captures an image using both the infrared light and the visible light.

FIG. 2illustrates an outer appearance of the image processing apparatus10. The image processing apparatus10is assumed as a front face where the operation panel15is provided, and is used by the user standing in front of the front face. A sheet feeding tray181is provided in the front of the image processing apparatus10. In addition, an original document feeding unit182and an original document receiving unit183are provided on the upper portion of the image processing apparatus10. A paper discharging unit184is provided in the side surface of the image processing apparatus10.

A sheet to form an image is stored in the sheet feeding tray181. The original document to be scanned is set in the original document feeding unit182. The sheet feeding tray181and the original document feeding unit182both are an instance of a sheet feeding unit which supplies the sheet to a processing unit such as the image scanner16and the printer17described above. The scanned original document is discharged to the original document receiving unit183. The image-formed sheet is discharged to the paper discharging unit184. The original document receiving unit183and the paper discharging unit184both are an instance of a discharging unit to which the sheet processed by the processing unit is discharged.

A lens191of the digital camera19is provided in the operation panel15. The user who stands in front of the image processing apparatus10is captured by the digital camera19through the lens191.

FIG. 3illustrates an instance of an angle of view of the digital camera19. InFIG. 3, a region interposed by the two-dot chain lines in front of the image processing apparatus10shows a capturing region A1of the digital camera19, and the outside regions show a capturing exception region A2.

For instance, when a user U1in the capturing exception region A2comes to use the image processing apparatus10, the user U1enters the capturing region A1in order to operate the operation panel15and stands in front of the image processing apparatus10. The digital camera19is provided toward a capturing direction of the user at a position in a direction to capture the user who uses the image processing apparatus10.

FIG. 4illustrates an instance of a captured image. InFIG. 4, a face F1and eyes E1of the user U1are shown in a captured image B1. In the image processing apparatus10, the operation panel15is provided at a position lower than a height of an adult for the easy operation. The lens191of the digital camera19provided in the operation panel15is provided in a looking-up direction to capture the user's face, for instance. Therefore, a ceiling and a fluorescent light9attached to the ceiling are shown in the image B1.

The CPU11of the image processing apparatus10executes a program to control the respective units, so that functions are realized as will be described below.

FIG. 5illustrates a functional configuration which is realized by the image processing apparatus10. The image processing apparatus10is provided with an image capturing unit101, a line-of-sight detecting unit102, a detection state determining unit103, and a mode switching unit104. The image capturing unit101has a function of capturing the surroundings of the own apparatus, and is realized by the CPU11and the digital camera19illustrated inFIG. 1, for instance.

The image capturing unit101captures the capturing region A1illustrated inFIG. 3and captures a face of the user who uses the own apparatus as illustrated inFIG. 4. The image capturing unit101repeatedly captures an image in a specific period. The fluorescent light9shown in the image captured by the image capturing unit101flickers at a double frequency compared with that of a domestic power source (for instance, 100 Hz when the domestic power source is 50 Hz). A capture timing of the image capturing unit101may be set in an unflickering period of the fluorescent light9when the effect of backlight is less.

FIG. 6illustrates an instance of the capture timing. InFIG. 6, the fluorescent light9is repeatedly turned on and off at every period T1. With this regard, the image capturing unit101repeatedly captures the image in a sampling period T2. The period T2is a period of T1×N (N is a natural number). In addition, the image capturing unit101captures an image at a sampling time C1such that the capturing time is within the period when the fluorescent light9is turned off. When the turning-on period and the turning-off period share the period T1half and half, the sampling time C1corresponds to a speed shorter than the period T1÷2(when the period T1is 0.01 second at a frequency of 100 Hz, the sampling time becomes 1/250 seconds or 1/500 seconds shorter than 0.005 seconds). The image capturing unit101supplies the captured image to the line-of-sight detecting unit102.

The line-of-sight detecting unit102has a function of detecting the light of sight with respect to the own apparatus, and performs an operation (line-of-sight detecting operation) for detecting the line of sight. The line-of-sight detecting unit102is an instance of “line-of-sight detector”. The line-of-sight detecting unit102analyzes the image supplied from the image capturing unit101. In this example, since the image is captured with the visible light, the line of sight is detected based on a positional relation between a reference point (for instance, the head) and a moving point (for instance, the iris). In other words, the line-of-sight detecting unit102detects the line of sight based on the captured image of the surroundings of the own apparatus.

In a case where the image is captured with the infrared light, the line-of-sight detecting unit102may detect the line of sight by setting a cornea reflection point as the reference point and a pupil as the moving point. The line-of-sight detecting unit102may detect the line of sight using a well-known technology other than the above ones.

In addition, the line-of-sight detecting unit102in this example detects the line of sight toward a predetermined place. The predetermined place is, for instance, an interest place of which the user is interested in the own apparatus (specifically, the operation panel15, the sheet feeding tray181, the original document feeding unit182, the original document receiving unit183, and the paper discharging unit184illustrated inFIG. 2). The line-of-sight detecting unit102stores the coordinates of the respective interest places. When the detected line of sight is roughly directed toward one of the interest places or the own apparatus, the line-of-sight detecting unit102supplies information (detection result information) that the line of sight is directed toward the own apparatus to the detection state determining unit103. When the line of sight toward the own apparatus is detected for a predetermined time, the line-of-sight detecting unit102determines that the line of sight toward the own apparatus is detected.

In a case where the detected line of sight is not directed toward the interest place or the own apparatus, or in a case where the line of sight itself is not detected, the line-of-sight detecting unit102supplies information (detection result information) indicating the fact that the line of sight toward the interest place is not detected to the detection state determining unit103. When the image is supplied from the image capturing unit101, the line-of-sight detecting unit102repeatedly performs the line-of-sight detecting operation and supplies the detection result information at that time to the detection state determining unit103.

The detection state determining unit103determines whether a switching condition for switching the mode of the own apparatus from a power saving mode to a normal mode is satisfied based on a detection state of the line of sight of the line-of-sight detecting unit102. The normal mode is a mode in which the power is supplied to the respective units of the own apparatus for making a normal operation possible. The power saving mode is a mode in which the power supplied units are limited to save the power consumption as less as possible, or the power supply is lowered compared to the case of the normal mode. In the power saving mode, the power consumption is less than that in the normal mode.

However, even in the power saving mode, the power is supplied to the respective units illustrated inFIG. 5, that is, the image capturing unit101, the line-of-sight detecting unit102, the detection state determining unit103, and the mode switching unit104). In other words, the power saving mode is a mode for operating at least the respective units illustrated inFIG. 5. The state of the image processing apparatus10operated in the power saving mode is an instance of a “first state”. The state of the image processing apparatus10operated in the normal mode is an instance of a “second state”. The first state and the second state both are the states of the image processing apparatus10related to the power consumption. The second state is a state having higher power consumption with respect to the first state.

For instance, when the line of sight toward the interest place is detected by the line-of-sight detecting unit102, the detection state determining unit103determines whether the switching condition to be switched from the power saving mode to the normal mode is satisfied based on the detection state.

For instance, in a case where a predetermined detection period is 2.0 seconds, the ratio is 70%, and the sampling period T2is 0.1 second, the detection state determining unit103determines that the switching condition is satisfied when fourteen or more pieces of the detection result information among twenty pieces of the detection result information supplied during 2.0 seconds indicate that the line of sight toward the interest place or the own apparatus is detected after the detection result information indicating that the line of sight toward the interest place is detected is supplied from the line-of-sight detecting unit102. When there are detected fourteen or less pieces of the detection result information indicating that the line of sight toward the interest place, the detection state determining unit103determines that the switching condition is not satisfied. In a case where it is determined that the switching condition is satisfied, the detection state determining unit103notifies the fact to the mode switching unit104.

The mode switching unit104has a function of switching the mode of the own apparatus. For instance, in a case where a period having no user's operation in the own apparatus exceeds a predetermined length, the mode switching unit104switches the mode of the own apparatus from the normal mode to the power saving mode. The mode switching unit104performs the switching from the power saving mode to the normal mode according to the detection state of the line-of-sight detecting unit102.

Specifically, in a case where the fact that the switching condition is satisfied is notified from the detection state determining unit103(that is, in a case where a predetermined ratio of the line of sight toward the interest place or the own apparatus is detected among the line-of-sight detections performed during the detection period), the mode switching unit104performs the switching from the power saving mode to the normal mode. When the line-of-sight detecting unit102detects the line of sight, the mode switching unit104performs the switching to control the own apparatus to transition from the first state to the second state. The mode switching unit104is an instance of a “controller”.

The image processing apparatus10performs a mode switching process in which the mode of the own apparatus is switched based on the above configuration.

FIG. 7illustrates an instance of an operational sequence of the image processing apparatus10in the mode switching process. The operational sequence starts when the image processing apparatus10is switched from the normal mode to the power saving mode.

First, the image processing apparatus10(the image capturing unit101) captures an image of the surroundings of the own apparatus (Step S11). Next, the image processing apparatus10(the line-of-sight detecting unit102) performs the line-of-sight detecting operation for detecting the line of sight toward the interest place (Step S12). Subsequently, the image processing apparatus10(the detection state determining unit103) determines whether the detection state of the line of sight satisfies the switching condition (Step S13).

In a case where it is determined in Step S13that the switching condition is not satisfied (NO), the image processing apparatus10returns to Step S11and continues to operate. In a case where it is determined in Step S13that the switching condition is satisfied (YES), the image processing apparatus10(the mode switching unit104) switches the mode of the own apparatus from the power saving mode to the normal mode (Step S14).

As described above, when the own apparatus is used by the user, the image processing apparatus10is used to switch the mode related to the power supply of the own apparatus from the power saving mode to the normal mode. In related arts, there is a method of switching the mode when a motion sensor detects the approach of a person for instance. In this case, the mode may be switched only when a person passes by. In this example, since the mode is switched in a case where the line of sight is detected, the mode is hardly switched only just when a person passes by compared to a case where the mode is switched when the approach of a person is detected.

In this example, the line of sight toward the interest place is detected. For instance, when the line of sight toward the operation panel15is detected, the approach of the user who tries to operate the own apparatus is detected. In addition, when the line of sight toward the sheet feeding tray181and the original document feeding unit182is detected, the approach of the user who tries to set a sheet in the own apparatus is detected.

In this example, in a case where the line of sight is detected in a predetermined ratio of the operation among the line-of-sight detecting operations performed in the detection period even when the approaching user who tries to use the image processing apparatus10instantaneously averts the line of sight from the interest place or the own apparatus, the mode related to the power supply is switched from the power saving mode to the normal mode.

In this example, the line of sight is detected in a period when the emission of the fluorescent light is less affected. Therefore, the user's face is not backlit with the fluorescent light, so that the detection accuracy of the line of sight is improved compared to a case where the emission period of the fluorescent light is not taken into consideration.

[2] Modification Examples

The above-described example is given as a mere instance, and may be modified as follows. In addition, the example and the respective modification examples may be implemented in combination with each other as needed.

[2-1] Detection Period

In the example, the detection period used in determination on the switching condition may be changed. For instance, the detection period may be changed according to the position of the eye of which the line of sight is detected by the line-of-sight detecting unit102in the image used for detecting the line of sight toward the interest place. In this case, the image processing apparatus10dissects the image into plural regions and stores region information indicating the dissected regions.

FIG. 8illustrates an instance of the dissected regions. In the instance ofFIG. 8, the image B1captured by the image capturing unit101is illustrated. In the image B1, a corridor G1extending in the front direction of the image processing apparatus10and a corridor G2extending in the lateral direction of the image processing apparatus10are illustrated. In this instance, the image is dissected into a region D1showing the corridor G1and a region D2in the region showing the corridor G2except the region D1.

FIG. 9illustrates an instance of a relation between a user's path and the dissected regions. InFIG. 9, the corridors G1and G2, a capturing region A1-1, a capturing region A1-2, and the capturing exception region A2when viewed from the ceiling are illustrated. The capturing region A1-1corresponds to the region D1illustrated inFIG. 8in the actual space captured by the image capturing unit101. The capturing region A1-2corresponds to the region D2illustrated inFIG. 8in the actual space captured by the image capturing unit101.

In a case where the user U1passes through the corridor G1to use the image processing apparatus10, and when the user U1positioned in a region J1in the drawing turns the line of sight toward the interest place of the image processing apparatus10, the line of sight is detected. On the other hand, in a case where the user U1passes through the corridor G2, and when the user U1positioned in a region J2in the drawing turns the line of sight toward the interest place of the image processing apparatus10, the line of sight is detected. Since a distance passing through the corridor G1is farther than a distance passing through the corridor G2, the face of the user U1is captured. Therefore, the region J1is larger than the region J2.

The detection state determining unit103of this modification uses a detection period table in which the regions in the image are associated with the detection periods.

FIG. 10illustrates an instance of the detection period table. In the instance ofFIG. 10, the detection period “H1” is associated with the region “D1” ofFIG. 8, and the detection period “H2(H2<H1)” is associated with the region “D2” ofFIG. 8. For instance, when the position of the eye when the line of sight toward the interest place is detected is in the region D1for the first time, the detection state determining unit103performs the determination on the switching condition based on the detection period H1. When the position of the eye at that time is in the region D2, the detection state determining unit103performs the determination on the switching condition based on the detection period H2.

The detection state determining unit103may perform the determination on the switching condition using not only the position of the eye at the first detection time but also the position of the eye at the second detection time or an intermediate position of the first two detected positions. In either case, after the line of sight is detected, the mode switching unit104switches the mode when the line of sight is detected in the line-of-sight detecting operation matched to the condition among the line-of-sight detecting operations performed in the detection period according to the position of the eye of which the line of sight is detected in the captured image.

When the detection period H1matched to the length of the region J1illustrated inFIG. 9is uniformly used, a time taken until the user coming close to the image processing apparatus10through the corridor G2waits for the mode change becomes long compared to a case where the detection period H2is used. On the other hand, when the short detection period H2matched to the length of the region J2is used, the mode switching due to the detection of the line of sight of the user who comes through the corridor G1but passes before the image processing apparatus10and then proceeds to the corridor G2easily occurs compared to the case of using the detection period H1.

In this modification example, the period according to the position of the eye of which the line of sight in the captured image is detected is used as the detection period. Therefore, the time taken until the user waits for the mode switching becomes short. It is suppressed that the mode is switched due to the detection of the line of sight of the user who will not use the image processing apparatus10. The period according to the position of the eye may also be used for the period determined with respect to the switching condition in the past described in the modification example (the detection period in the modification example). Even in this case, the time waiting for the mode switching becomes short, and an erroneous mode switching is suppressed.

[2-2] Detection of Approach

The image processing apparatus may be provided with a motion sensor to detect an approaching user.

FIG. 11illustrates a hardware configuration of an image processing apparatus10aaccording to this modification example. The image processing apparatus10ais provided with a motion sensor20in addition to the hardware configuration illustrated inFIG. 1. The motion sensor20is a sensor for detecting whether a person approaches by measuring a reflecting quantity of infrared light, ultrasonic waves, or visible light. When measuring the reflecting quantity indicating that a person approaches, the motion sensor20notifies the fact to the CPU11.

FIG. 12illustrates a functional configuration which is realized by the image processing apparatus10a. The image processing apparatus10ais provided with an approach detecting unit105and an operation start instructing unit106in addition to the respective units illustrated inFIG. 5. The approach detecting unit105detects whether a person approaches the own apparatus. The approach detecting unit105is an instance of an “approach detector”. When an approach of a person is detected by the motion sensor20, the approach detecting unit105notifies the fact to the operation start instructing unit106.

In a case where the approach of a person is detected by the approach detecting unit105, the operation start instructing unit106instructs the image capturing unit101, the line-of-sight detecting unit102, the detection state determining unit103, and the mode switching unit104to start their operations. For instance, in a case where the approach detecting unit105is configured with a camera and monitors a tiptoe of an approaching person for instance while the tiptoe is lain in a direction approaching the own apparatus, the determination on whether the person will use the own apparatus can be more accurately performed by detecting the line of sight of the person.

Alternatively, in a case where a direction of the face of an approaching person is monitored in place of the tiptoe of the approaching person and the face is directed to the own apparatus, the determination on whether the person will use the own apparatus can be more accurately performed by detecting the line of sight of the person. In addition, in the image processing apparatus mounted with an authentication unit, the face used for determining the direction of the face can be used for determining the authentication, or the iris of the eye used in the line-of-sight detecting unit can be used for determining the authentication, so that a trouble in authentication can be alleviated.

When the instructions are received, the image capturing unit101starts to capture the surroundings. When the approach of a person toward the own apparatus is detected by the approach detecting unit105, the line-of-sight detecting unit102starts the line-of-sight detecting operation. Therefore, the line-of-sight detecting unit102detects the line of sight toward the own apparatus with respect to the person detected by the approach detecting unit105. The detection state determining unit103starts to determine a state of the line-of-sight detection, and the mode switching unit104starts to switch the mode. When the instructions are issued by the operation start instructing unit106, the approach detecting unit105and the operation start instructing unit106end their operations.

Therefore, the capturing of the image capturing unit101and the line-of-sight detecting operation of the line-of-sight detecting unit102are not operated until the person approaches the own apparatus. In addition, for instance, when the power consumption of the respective units illustrated inFIG. 5is larger than the power consumed by the operations of the approach detecting unit105and the operation start instructing unit106, and the mode is switched to the power saving mode, the power consumption in the power saving mode is reduced compared to a case where the operations of the respective units illustrated inFIG. 5start.

The above description in the example has been made about that the line-of-sight detecting unit102detects the line of sight toward the interest place. However, the line-of-sight detecting unit102may detect not only a local interest place but also the line of sight even when the line of sight is large enough to cover the size of the own apparatus.

[2-4] Image Capturing

The image processing apparatus may not capture an image.

FIG. 13illustrates a hardware configuration of an image processing system1baccording to this modification. The image processing system1bis provided with an image processing apparatus10band a digital camera30. The image processing apparatus10bis provided with the configurations from the CPU11to the transport mechanism18illustrated inFIG. 1, and an interface21.

The interface21relays the exchange of data with the digital camera30. The digital camera30is provided around the image processing apparatus10b, and captures the surroundings of the image processing apparatus10bcontaining a place where the user stands to use the image processing apparatus10b.

FIG. 14illustrates a functional configuration to be realized by the image processing apparatus10b. The image processing apparatus10bis provided with the line-of-sight detecting unit102, the detection state determining unit103, the mode switching unit104, and an image acquisition unit107. The image acquisition unit107acquires an image obtained by capturing the surroundings of the own apparatus from an external capturing device (the digital camera30in this instance). The line-of-sight detecting unit102performs the line-of-sight detecting operation based on the image captured by the image acquisition unit107, that is, the image containing the surroundings of the own apparatus.

The modes switched by the mode switching unit104are not limited to those described in the example. For instance, the normal mode has been described as a mode in which the power to the respective units of the own apparatus is not limited. However, the normal mode may be a mode in which the power is limited for a specific function. In addition, the other functions may be supplied with power in the power saving mode in addition to the respective units illustrated inFIG. 5. Even in any case, the mode switching unit104may switch the mode from a first mode to a second mode (in this case, the second mode is a mode in which the power consumption is larger than that in the first mode) according to the detection state of the line-of-sight detecting unit102.

[2-6] Information Processing Apparatus

The above respective instances have been described about the image processing apparatus which performs image processing, but the invention may be applied to another information processing apparatus without being limited thereto. For instance, when the invention is applied to an information processing apparatus such as a kiosk terminal, an ATM (Automatic Teller Machine), and an automatic ticket machine which is installed at a place where an unspecified large number of users visit to use, the mode is not changed only just by the user who passes by as described in the example, so that it is convenient.

[2-7] Category of Invention

The invention is implemented as an information processing method which realizes a process of the information processing apparatus besides the information processing apparatus for realizing the respective units illustrated inFIG. 5similarly to the above-described image processing system, and is implemented as a program which, when executed by a computer, causes the computer to perform a function of controlling the information processing apparatus. The program may be provided in a form of a recording medium such as an optical disk storing the program therein, or may be installed and usable by being downloaded to the computer through a network such as the Internet.