Image display apparatus, image processing apparatus, and non-transitory computer-readable medium

An image display apparatus and an image processing apparatus include an imaging unit, a display unit, a detection unit, a measurement unit, and a notification unit. The imaging unit images an operator. The imaging unit performs imaging on an operator. The display unit displays operation information for performing image processing. The detection unit detects viewing of the display unit by the operator on a basis of a result of the imaging performed by the imaging unit. The measurement unit accumulatively measures a time in which the detection unit detects the viewing. The notification unit provides a notification of guidance according to the image processing when the accumulatively measured time exceeds a predetermined threshold.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 from Japanese Patent Application No. 2014-063719 filed Mar. 26, 2014.

BACKGROUND

Technical Field

The present invention relates to an image display apparatus, an image processing apparatus, and a non-transitory computer-readable medium.

SUMMARY

According to an aspect of the present invention, there is provided an image display apparatus including an imaging unit, a display unit, a detection unit, a measurement unit, and a notification unit. The imaging unit performs imaging on an operator. The display unit displays operation information for performing image processing. The detection unit detects viewing of the display unit by the operator on a basis of a result of the imaging performed by the imaging unit. The measurement unit accumulatively measures a time in which the detection unit detects the viewing. The notification unit provides a notification of guidance according to the image processing when the accumulatively measured time exceeds a predetermined threshold.

According to another aspect of the present invention, there is provided an image processing apparatus including an imaging unit, a display unit, a detection unit, a measurement unit, and a notification unit. The imaging unit performs imaging on an operator. The display unit displays operation information for performing image processing. The detection unit detects viewing of the display unit by the operator on a basis of a result of the imaging performed by the imaging unit. The measurement unit accumulatively measures a time in which the detection unit detects the viewing. The notification unit provides a notification of guidance according to the image processing when the accumulatively measured time exceeds a predetermined threshold.

DETAILED DESCRIPTION

FIG. 1is a perspective view of the appearance of an image forming apparatus10according to an exemplary embodiment of the present invention. The image forming apparatus10is an exemplary image processing apparatus or an image display apparatus according to the exemplary embodiment of the present invention. The image forming apparatus10is, for example, an electrophotographic image forming apparatus. The image forming apparatus10has an image forming function of forming an image on a recoding medium such as a sheet of paper, a copying function of copying an image, a scan function of reading out an image formed on a recording medium, and the like. The functions of the image forming apparatus10are not limited to these. For example, the image forming apparatus10may have a facsimile function. In addition, the image forming apparatus10is not limited to one having all of the above-described functions. For example, the image forming apparatus10may have an image forming function and a copying function, but not the other functions.

The image forming apparatus10includes a pyroelectric sensor105and a camera106. The sensor105is a sensor which consumes power lower than that of the camera106and which detects a motion of a user in a detection area broader than that of the camera106. The camera106photographs a user who is present in front of the image forming apparatus10, and outputs photographic data presented as a movie or a still image obtained through the photographing. The photographic data is used to perform user authentication. For example, an operator presses a power saving button, whereby the image forming apparatus10enters the sleep mode. In this state, when the pyroelectric sensor105detects a user, the image forming apparatus10releases the sleep mode, activates the camera106, and performs authentication of the user. In this example, user authentication is performed by analyzing photographic data obtained through photographing using the camera106.

FIG. 2is a diagram illustrating the functional configuration of the image forming apparatus10. The image forming apparatus10includes an imaging unit22, a display unit20, a detection unit12, a measurement unit14, a notification unit16, a first receiving unit11, a second receiving unit13, an acquiring unit15, a storage controller17, a second notification unit18, and a specifying unit19. The imaging unit22performs imaging on an operator. The display unit20displays operation information for supplying the image forming apparatus10with an instruction to perform one of image processing operations. The detection unit12detects viewing of the display unit20which is performed by an operator, on the basis of the result of the imaging using the imaging unit22. The measurement unit14accumulatively measures a time in which the detection unit12detects the viewing, for the image processing operation. The notification unit16provides a notification of guidance for the multiple image processing operations when the accumulatively measured time exceeds a predetermined threshold.

The first receiving unit11receives input of identification information for identifying an operator. In this example, the first receiving unit11analyzes photographic data which is output from the imaging unit22, and obtains data for authentication which is extracted from the photographic data, as identification information. The second receiving unit13receives input of an instruction to perform an image processing operation, from an operator. In this example, examples of the “image processing operation” performed by the image forming apparatus10include various types of processing using image data, such as “copying” and “scanning”.

The acquiring unit15acquires information corresponding to the identification information received by the first receiving unit11, from a storage unit21which stores identification information for identifying an operator and information describing an image processing operation in such a manner that the identification information is associated with the information. When the accumulatively measured time obtained by the measurement unit14exceeds the predetermined threshold, the storage controller17stores information describing an image processing operation corresponding to an instruction received by the second receiving unit13, in the storage unit21in such a manner that the information is associated with the identification information received by the first receiving unit11.

When the acquiring unit15acquires the information from the storage unit21, the second notification unit18provides a notification of guidance according to the information obtained by the acquiring unit15, in at least part of the period from a predetermined start timing to a time when the second receiving unit13receives input of an instruction to perform an image processing operation. The specifying unit19specifies whether or not an operator is present in a predetermined area with respect to the image forming apparatus10. In this example, the specifying unit19specifies whether or not an operator is present in a predetermined area with respect to the image forming apparatus10, on the basis of a signal which is output from the pyroelectric sensor105.

FIG. 3is a diagram illustrating the hardware configuration of the image forming apparatus10. The image forming apparatus10includes a central processing unit (CPU)101, a read only memory (ROM)102, a random access memory (RAM)103, a storage104, the pyroelectric sensor105, the camera106, a UI unit107, an image forming unit108, and an image reading unit109. The CPU101is a controller (processor) which controls units of the image forming apparatus10. The ROM102is a nonvolatile storage which stores programs and data. The RAM103is the main storage which is volatile and which functions as a work area when the CPU101executes programs. The storage104is an auxiliary storage which is nonvolatile and which stores programs and data. The pyroelectric sensor105is a sensor which detects a motion of a user (operator). The camera106photographs a user who is present in front of the image forming apparatus10, and outputs photographic data presented as a movie or a still image obtained through the photographing.

The UI unit107has operators, such as various types of keys and a touch sensor, and supplies operation information according to an operation performed by a user, to the CPU101. The CPU101performs processing according to the operation information. The UI unit107is provided with a liquid-crystal panel and a liquid-crystal drive circuit, and displays an image under control of the CPU101. The image forming unit108forms an image on a recording medium such as a sheet of paper, by using an electrophotographic system. The image reading unit109optically reads out an image on a document.

In this example, the CPU101executes a guidance display program stored in the storage104(or the ROM102), achieving the functions illustrated inFIG. 2. The CPU101executing the guidance display program serves as the first receiving unit11, the detection unit12, the second receiving unit13, the measurement unit14, the acquiring unit15, the storage controller17, and the specifying unit19. The CPU101executing the guidance display program, or the CPU101and the UI unit107serve as the notification unit16and the second notification unit18. The UI unit107serves as the display unit20. The storage104serves as the storage unit21. The camera106serves as the imaging unit22.

Operations performed by the image forming apparatus10will be described. To operate the UI unit107of the image forming apparatus10, a user approaches the image forming apparatus10. The user is detected by the pyroelectric sensor105. The detection of a user causes the camera106to be activated, and photographing is started. The camera106photographs the user and outputs photographic data.

FIG. 4is a flowchart of a process performed by the CPU101of the image forming apparatus10. In step S101, the CPU101analyzes the photographic data which is output from the camera106, and checks the analysis result against predetermined authentication data, thereby performing user authentication. If the authentication succeeds, the CPU101proceeds to step S102. In step S102, the CPU101starts measurement of a time (hereinafter, referred to as an “operation time”) from a time when the authentication succeeds to a time when the user presses a start button. If the user authentication fails in step S101, the CPU101performs the authentication process again until the user authentication succeeds. When the user authentication succeeds, the CPU101causes the UI unit107to display a screen for receiving input of an instruction to perform an image processing operation.

FIG. 5is a diagram illustrating an exemplary screen displayed on the UI unit107(hereinafter, referred to as a “UI screen”). In this example, a copy button B1, a scan button B2, a facsimile button B3, and an OK button B4are displayed on the UI unit107. The user uses the UI unit107to cause the image forming apparatus10to perform an image processing operation. For example, when copying is to be performed, the user presses the copy button B1displayed on the UI unit107. When the copy button B1is pressed, a screen for performing various types of setting for the copy process is displayed on the UI unit107. For example, a screen for setting “how to read (single-sided to single-sided, single-sided to double-sided, double-sided to double-sided, and the like)”, and “magnification” is displayed on the UI unit107. For example, the user selects one from process types, such as “single-sided to single-sided”, “single-sided to double-sided”, and “double-sided to double-sided”. In addition, for example, when an enlarged copy or a reduced copy is to be obtained, the user sets a magnification. These setting processes may require a large amount of time if the user is not familiar with operations of the image forming apparatus10. When the user completes various types of setting for the image processing operation, the user uses the UI unit107to provide an instruction to perform the image processing operation for which setting has been performed. In this example, the user presses the start button provided on the UI unit107. When the start button is pressed, the UI unit107outputs information according to the operations performed by the user, and the CPU101starts execution of the image processing operation for which setting has been performed, on the basis of the information which is output from the UI unit107.

In step S103and its subsequent steps, the CPU101accumulatively measures a time (hereinafter, referred to as an “accumulated time”) in which the user's line of sight is directed to the UI unit107in the period from a time when the user authentication succeeds to a time when the start button is pressed. In step S103, the CPU101detects the line of sight from a face image of the user by analyzing the photographic data, and determines whether or not the detected line of sight is directed to the UI unit107, that is, whether or not the user looks at the UI unit107(views the UI unit107). A known technique of detecting a line of sight is used to determine whether or not the user views the UI unit107. If it is determined that the line of sight is not directed to the UI unit107(NO in step S103), the CPU101stops the accumulative measurement of time which is performed through analysis of photographic data from the camera106(face camera) in step S104, and the process returns back to the determination process in step S103again. If it is determined that the line of sight is directed to the UI unit107(YES in step S103), the CPU101accumulatively measures a time (in step S105).

In step S106, the CPU101determines whether or not the start button has been pressed. If it is determined that the start button has been pressed (YES in step S106), the CPU101proceeds to step S107. If it is determined that the start button has not been pressed (NO in step S106), the CPU101returns back to step S103. In step S107, the CPU101ends measurement of the operation time, and registers the measurement result (the operation time and the accumulated time) as a history in the history table42. In addition, the CPU101uses the measurement result to update the average accumulated time which is calculated for a corresponding one of the process types and which is stored in the average-accumulated-time table41.

FIG. 6is a diagram illustrating exemplary information in the average-accumulated-time table41. As illustrated inFIG. 6, items “process type” and “average accumulated time” are stored in this table in such a manner that the items are associated with each other. Among these items, the item “process type” stores information representing an image processing type, such as “copy” or “scan”. The item “average accumulated time” stores information representing the average accumulated time for each of the process types.

In step S108, the CPU101determines whether or not the accumulated time obtained through measurement is larger than a predetermined threshold. In this example, the average accumulated time for a corresponding one of the process types which is stored in the average-accumulated-time table41is used as the threshold. In this example, in step S108, the CPU101specifies the process type of the image processing operation on the basis of information which is output from the UI unit107on the basis of operations on the UI unit107which have been performed by the user, and specifies the average accumulated time corresponding to the specified process type as a threshold. If the accumulated time is equal to or less than the threshold (NO in step S108), the CPU101skips step S109, and ends the process. If the accumulated time is larger than the threshold, the CPU101proceeds to step S109, and displays guidance information. In step S109, the CPU101displays guidance information on the UI unit107on the basis of information stored in the history table42in the storage104. The threshold used in the determination in step S108is not limited to the average accumulated time corresponding to the process type. For example, a threshold may be set in advance for each of users, or a common threshold may be used for all of the process types.

FIG. 7is a diagram illustrating exemplary information in the history table42. In this table, information about a process which corresponds to an operation among operations that were performed by a user in the past and for which the accumulated time is larger than the predetermined threshold is accumulated as a history. As illustrated inFIG. 7, in the table, items “user ID”, “process type”, and “accumulated time” are stored in such a manner as to be associated with each other. Among these items, the item “user ID” stores identification information for identifying a user. The item “process type” stores information describing an image processing operation, such as “copy” or “scan”. This information includes information representing the type of an image processing operation, such as “copy” or “scan”, and various types of setting information used in the image processing operation (for example, a setting value of “how to read”, and a setting value of “magnification”). The item “accumulated time” stores an accumulated time required when the user performed the operation of giving an instruction to perform the image processing operation.

In step S109, the CPU101displays guidance information on the UI unit107on the basis of information stored in the history table42. In this example, the CPU101displays shortcut buttons for performing image processing operations for the process types registered as a history in the history table42, on the UI unit107as guidance information. In a normal operation, a user presses one of the buttons displayed on the UI screen illustrated inFIG. 5so as to select an image processing operation of which an instruction is to be provided, and determine detailed settings for the selected image processing operation. For example, a user presses the copy button B1on the UI screen illustrated inFIG. 5. Then, the user determines settings, such as “how to read” and “magnification”, one by one for the copy process. In contrast, a shortcut button is a button for providing an instruction to perform an image processing operation by using setting values which were set by the user in a past operation. That is, the user may just press a shortcut button to give an instruction to perform a desired image processing operation. It is not necessary to perform another operation for determining detailed settings (operation of setting “how to read”, “magnification”, and the like).

FIG. 8is a diagram illustrating an exemplary UI screen on which shortcut buttons are displayed. As illustrated inFIG. 8, in addition to the operation buttons displayed on the UI screen illustrated inFIG. 5, shortcut buttons B11and B12for performing processes for the process types registered as a history in the history table42are displayed on the UI unit107.

In step S109, if multiple process types are stored in the history table42, the CPU101may display all of the shortcut buttons corresponding to the stored process types, on the UI unit107. Alternatively, the CPU101may select one from the process types and display it on the UI unit107. When one process type is to be selected, for example, the CPU101may select the process type for which the accumulated time is the largest, from the process types, or, for example, may select a process type for which the accumulated time is equal to or larger than the predetermined threshold.

In the present exemplary embodiment, when the accumulated time for a user exceeds the predetermined threshold, the shortcut buttons B11and B12according to the operation history of the user are displayed. The user may just press the shortcut button B11or B12to instruct the image forming apparatus10to perform a desired image processing operation, and does not need to perform other troublesome operations.

The above-described exemplary embodiment is an example of the present invention. The exemplary embodiment may be modified as described below. In addition, modified exemplary embodiments described below may be combined with each other.

3-1. First Modified Exemplary Embodiment

In the above-described exemplary embodiment, when an accumulated time in which the line of sight of a user is directed to the UI unit107is equal to or larger than the predetermined threshold, the CPU101displays guidance information. The timing at which guidance information is displayed may be, for example, a timing at which the image forming apparatus10authenticates the user again. In this case, the CPU101determines whether or not the history table42stores information corresponding to the user at a timing at which the user authentication succeeds. When corresponding information is stored, the CPU101displays guidance information according to the stored information. That is, when the history table42stores information corresponding to the user, the CPU101may display guidance information prior to start of measurement of an operation time. Another example is that, when an accumulated time has exceeded the predetermined threshold until a user presses the start button, the CPU101may display guidance information at a timing at which the accumulated time exceeds the threshold.

3-2. Second Modified Exemplary Embodiment

In the above-described exemplary embodiment, the CPU101employs a timing at which user authentication through photographing succeeds, as a timing of start of measurement of an accumulated time. The timing of start of measurement is not limited to this. For example, the CPU101may employ a timing at which the pyroelectric sensor105detects a user, as a timing of start of measurement. Another example is that, instead of the user authentication through photographing of a user, the user inputs the user ID and the password on the UI unit107, whereby user authentication is performed. In this case, the CPU101may employ a timing at which user authentication using a user ID and a password succeeds, as a timing of start of measurement of an operation time.

Another example of a timing of start of measurement of an operation time is at least either one of a timing at which the CPU101successfully performs user authentication and a timing at which execution of an image processing operation caused by the immediately preceding instruction is completed. For example, when one user successively instructs the image forming apparatus10to perform multiple image processing operations, the CPU101may start measurement of an operation time of the next image processing operation at a timing at which execution of the image processing operation caused by the immediately preceding instruction is completed.

3-3. Third Modified Exemplary Embodiment

In the above-described exemplary embodiment, the CPU101causes the UI unit107to display shortcut buttons, thereby notifying a user of a guide (guidance information) of operations. The way of notification is not limited to this. For example, the CPU101may cause the UI unit107to display operational guidance information corresponding to information stored in the history table42. The operational guidance information which is displayed may be, for example, a message and an image which specifically describe the procedure of the operation, and may include a message such as “Is the operation to be registered as a history?” In this case, when the user selects registration of the operation as a history, the CPU101may register the operation which has been performed, in the history table42on the basis of the selection. In addition, for example, the image forming apparatus10may have a speaker, and the CPU101may provide a notification by outputting a voice message of the operational guidance information from the speaker.

Another example is that the CPU101may transmit data representing operational guidance, to a predetermined mail address. In this case, for example, a message such as “A shortcut button for an operation which is frequently performed will be generated the next time.” may be transmitted to a personal computer or a portable terminal carried by the user. Thus, various types of notification transmitted to a user may be used. In short, any method may be employed as long as the CPU101notifies a user.

The CPU101may transmit data indicating that the accumulated time for an image processing operation exceeds the threshold, to a predetermined management apparatus. In this case, the management apparatus may summarize such information, thereby performing statistical processing for obtaining information describing which operation is one which requires time.

Another example is that the CPU101specifies whether or not a user is present in a predetermined area with respect to the image forming apparatus10on the basis of a signal which is output from the pyroelectric sensor105. When a user is not present in the predetermined area with respect to the image forming apparatus10, the CPU101may transmit data representing operational guidance, to a predetermined terminal. In addition, for example, when the pyroelectric sensor105fails to detect a user in a state in which, for example, a document used for scanning is left on the image forming apparatus10, the CPU101may transmit a notification of this. Specifically, for example, a voice message such as “Something is left” or a beep may be output from the image forming apparatus10. In addition, the CPU101may transmit a message saying that something is left, to a predetermined terminal (terminal carried by the user).

3-4. Fourth Modified Exemplary Embodiment

In the above-described exemplary embodiment, the average-accumulated-time table41and the history table42are stored in the storage104of the image forming apparatus10. The average-accumulated-time table41and the history table42may be stored in a storage unit provided outside of the image forming apparatus10. In this case, for example, the CPU101of the image forming apparatus10may refer to or update the tables via a communication network.

3-5. Fifth Modified Exemplary Embodiment

In the above-described exemplary embodiment, a user presses the start button or a shortcut button displayed on the UI unit107, thereby inputting an instruction to cause the image forming apparatus10to perform an image processing operation. The way in which a user gives an instruction to perform an image processing operation is not limited to this. For example, a so-called line-of-sight input or a so-called voice input may be used to provide an instruction to perform an image processing operation. The display unit according to the exemplary embodiment of the present invention is not limited to one which is manually operated by a user. Examples of the display unit include one on which an image for providing guidance or support for a user operation is displayed when a user operates the image forming apparatus10(including when the user operates the image forming apparatus10through a line-of-sight input or a voice input).

3-6. Sixth Modified Exemplary Embodiment

In the above-described exemplary embodiment, the programs executed by the CPU101of the image forming apparatus10may be downloaded via a communication line such as the Internet. In addition, the programs may be provided by recording them in a computer-readable recording medium, such as a magnetic recording medium (e.g., a magnetic tape or a magnetic disk), an optical recording medium (e.g., an optical disk), a magneto-optical medium, or a semiconductor memory.