Patent Description:
An image forming apparatus such as a multi-function peripheral (MFP) is known that can operate multiple functions such as copying, scanning, printing, and faxing by a voice instruction input by a user (see <CIT>).

In addition, many types of smart speakers are currently in practical use. The smart speaker can acquire a voice of a user, output, as voice, a response corresponding to the acquired voice, and instruct an external device or the like via a cloud server to perform an operation corresponding to the acquired voice.

When the above-described image forming apparatus and smart speaker are connected, it is possible to transmit the voice instruction of the user to the image forming apparatus via the cloud server and operate the image forming apparatus by voice.

Incidentally, it may be specified that the cloud server times out if there is no response from the image forming apparatus to the cloud server for a certain period of time (for example, five to seven seconds) after the voice instruction of the user is transmitted to the image forming apparatus for a response operation of the image forming apparatus.

In the case of such specifications, when the voice instruction of the user is transmitted to the image forming apparatus, the cloud server times out if it takes time for the image forming apparatus to process the voice instruction and the image forming apparatus cannot respond to the cloud server within the certain period of time.

For example, when a setting for image formation (for example, sheet selection) is input to the image forming apparatus, input processing is performed on an operation screen whether the user performs an input operation directly from the operation screen or the user performs the input operation by voice from the smart speaker. Therefore, when the input is performed by voice from the smart speaker, the response from the image forming apparatus to the cloud server is delayed until the input processing of the setting on the operation screen is completed. In particular, if an error (such as selecting a sheet that cannot be selected) occurs or prohibition processing (confirmation of whether an input value is a value that cannot be selected) is required due to the input setting, the processing time is longer, and thus the response from the image forming apparatus to the cloud server is delayed.

As described above, when the image forming apparatus is operated by voice from the smart speaker, the cloud server may time out depending on the contents of the setting input by the voice operation. In such a case, for example, due to the timeout of the cloud server, the voice operation may be interrupted and a subsequent voice operation may not be continued. Furthermore, for example, due to the timeout of the cloud server, the operation may have to be restarted from the beginning, and operability of the voice operation may deteriorate.

<CIT> discloses an image processing system which includes an image processing apparatus and a cloud server, the image processing system being capable of operating the image processing apparatus through a cooperation process among the image processing apparatus, the cloud server, and an external device, wherein the cloud server includes: an acquirer that acquires from the external device a voice instruction for operating the image processing apparatus, the voice instruction being given by the user in a conversation between the user and the external device; and an instruction transmitter that transmits an instruction command based on the voice instruction to the image processing apparatus, the image processing apparatus includes: an operation inputter; a setter; and a reset controller, and, when the image processing apparatus starts to be operated by the voice instruction, the reset controller of the image processing apparatus prevents execution of the auto reset.

<CIT> discloses an information processing apparatus which remotely controls an image forming apparatus having an operating state changeable. The information processing apparatus includes a remote operation accepting portion to accept a user operation, and a remote control portion to remotely control the image forming apparatus on the basis of a remote operation accepted by the remote operation accepting portion. The remote control portion includes a motion detecting portion to detect a motion of the user, and a state maintaining portion to control the image forming apparatus not to change the operating state while a remote-operation motion for inputting a remote operation is detected by the motion detecting portion after a communication with the image forming apparatus has been established.

<CIT> discloses an example method for time-out handling which comprises establishing a session from a network-based application at a client to a web server. The method includes identifying a time-out parameter for the web server. A timestamp of the established session is saved at the client. The method also includes determining a first timeout by adding the identified time-out parameter to the timestamp and triggering a system-based communication from the network-based application to keep the session alive. The method may be implemented by a client of a learning management system.

<CIT> discloses a time-out avoidance program which avoids a time-out under prescribed conditions and prevents conflicts between a dummy request for avoiding a time-out and an authorized request in a Web server which automatically logs-out, when prescribed time elapses after log-in, to manage a time-out. A client terminal, which logs in a Web server, transmits a dummy request for avoiding a time-out to the Web server periodically or by a prescribed operation in a prescribed input screen which requires time for input. At that time, the reception of data inputted to the inputting screen is temporally stopped in addition to transmission of the dummy request, and the stop is released after the transmission.

<CIT> discloses a time-out control system capable of automatically extending time-out according to the operating situation of the terminal side in a Web server managing time-out by automatically performing log-out when a predetermined time has elapsed from log-in. The Web server counts the elapsed time from a Web page request or the like from a client terminal to manage time-out of log-in. A dummy telegraphic message for resetting the elapsed time is transmitted triggered by predetermined operation such as the start of a new line or character input in the client terminal, and the Web server receiving the telegraphic message updates the elapsed time to zero to attain automatic extension of time-out corresponding to the operating situation of the terminal side.

An object of the present invention is to provide an image forming apparatus, a response notification method, and a response notification program that can avoid a timeout of an external device when the image forming apparatus is operated by voice from the external device.

To achieve the abovementioned object, the present invention provides an image forming apparatus, in accordance with claim <NUM>.

The present invention also provides a response notification method performed by an image forming apparatus, in accordance with claim <NUM>.

The present invention also provides a response notification program, in accordance with claim <NUM>.

The advantages and features provided by one or more embodiments of the invention will become more fully understood from the detailed description given hereinbelow and the appended drawings which are given by way of illustration only, and thus are not intended as a definition of the limits of the present invention:.

Hereinafter, one or more embodiments of the present invention will be described with reference to the drawings. However, the scope of the invention is not limited to the disclosed embodiments.

<FIG> is a block diagram for describing a schematic configuration of an image forming system <NUM> of the present embodiment. The image forming system <NUM> includes an image forming apparatus <NUM>, a network <NUM>, a voice input/output device <NUM>, and a server <NUM>. The image forming apparatus <NUM> is connected to the voice input/output device <NUM> and the server <NUM> via the network <NUM>.

The image forming apparatus <NUM> is an MFP having multiple processing functions such as copying, scanning, printing, and faxing, and includes a network unit <NUM>, a scanner unit <NUM>, a printer unit <NUM>, and a fax unit <NUM>.

The network unit <NUM> performs communication processing for transmitting and receiving various types of information to and from the voice input/output device <NUM> and the server <NUM> via the network <NUM> such as a local area network (LAN) or a telephone line. The network <NUM> is not limited to the LAN or the telephone line, but may be a wide area network (WAN) or the like, and may be wireless or wired.

The scanner unit <NUM> optically reads a document and creates image data to perform scan processing on an image. The scanner unit <NUM> may include an automatic document feeding device called an auto document feeder (ADF). In this case, when a user sets a large number of documents in a document tray of the ADF, the documents are automatically transported one by one, and the scan processing can be continuously performed on images of the documents at once.

The printer unit <NUM> prints an image on a sheet material such as printing paper based on the image data read by the scanner unit <NUM> or image data for printing from an external device, to perform printing processing on the image. For example, in the case of copying, the scanner unit <NUM> scans the document, and the printer unit <NUM> prints the image of the document on the sheet material. As a printing method, any known printing method such as a laser method or an inkjet method can be applied.

The fax unit <NUM> transmits the image data read by the scanner unit <NUM> and receives the image data transmitted from the outside via the network <NUM> such as a telephone line, to perform fax processing on the image.

Furthermore, the image forming apparatus <NUM> includes a controller <NUM> (computer), an operation panel <NUM> (display), and the like.

The controller <NUM> includes a processor such as a central processing unit (CPU), a memory such as a random access memory (RAM) or a read only memory (ROM), and the like although detailed illustration thereof is omitted. The controller <NUM> controls the network unit <NUM>, the scanner unit <NUM>, the printer unit <NUM>, and the fax unit <NUM> described above based on input from the operation panel <NUM> or input from the voice input/output device <NUM> and the server <NUM>. The controller <NUM> also controls the operation panel <NUM>.

The operation panel <NUM> includes, for example, a display device such as a liquid crystal display (LCD) and an input device having a transparent touch pad, an input button, or the like attached to the front of the display device, although detailed illustration thereof is also omitted. Operation instruction information (command) input to the operation panel <NUM> by use of the touch pad or the input button is input to the controller <NUM>.

The voice input/output device <NUM> is a device that inputs and outputs a voice. The voice input/output device <NUM> includes a microphone, a speaker, a network unit, a controller, and the like, although detailed illustration thereof is also omitted.

When the voice is input from the outside, the microphone outputs the voice as an electric signal to the controller. The speaker outputs an electric signal input from the controller to the outside as voice. The network unit performs communication processing for transmitting and receiving various types of information to and from the image forming apparatus <NUM> and the server <NUM> via the network <NUM>. The controller encodes the electric signal of the voice input from the microphone into voice data, and transmits the voice data to the server <NUM> via the network unit. Furthermore, the controller receives voice data transmitted from the server <NUM> via the network unit, converts the voice data into an electric signal, and outputs the electric signal to the speaker.

The server <NUM> includes a processor such as a CPU, a memory such as a RAM or a ROM, and the like although detailed illustration thereof is omitted. The server <NUM> includes a network unit <NUM>, a controller <NUM>, and the like.

The network unit <NUM> performs communication processing for transmitting and receiving various types of information to and from the image forming apparatus <NUM> and the voice input/output device <NUM> via the network <NUM>.

The controller <NUM> performs voice recognition processing for converting the voice data transmitted from the voice input/output device <NUM> into operation instruction information (command). Any known method can be applied to the voice recognition processing in the controller <NUM>. Furthermore, the controller <NUM> performs processing corresponding to the command obtained by the voice recognition processing, for example, response processing to the voice input/output device <NUM> or transmission of the command to the image forming apparatus <NUM>. For example, when a voice instructing the image forming apparatus <NUM> to perform processing is input to the voice input/output device <NUM>, voice data of the voice is converted into a command, and the command is transmitted to the image forming apparatus <NUM>. Such processing in the controller <NUM> is provided as, for example, an application on the server <NUM>.

The above-described voice input/output device <NUM> functions as a so-called smart speaker, and the server <NUM> functions as a cloud server that responds to the voice input from the voice input/output device <NUM> by the user and gives an instruction to the image forming apparatus <NUM>.

Therefore, when the user inputs an operation instruction by voice (voice instruction) from the voice input/output device <NUM>, the server <NUM> converts voice data of the voice instruction into a command and transmits the command to the image forming apparatus <NUM>. The image forming apparatus <NUM> then performs processing corresponding to the command transmitted from the server <NUM>. That is, in the image forming system <NUM>, the image forming apparatus <NUM> can be operated by a voice operation.

Incidentally, it may be specified that the server <NUM> times out if there is no response from the image forming apparatus <NUM> to the server <NUM> for a certain period of time after transmitting the voice instruction of the user to the image forming apparatus <NUM> for a response operation of the image forming apparatus <NUM>.

In the case of such specifications, when the voice instruction of the user is transmitted to the image forming apparatus <NUM>, the server <NUM> times out if it takes time for the image forming apparatus <NUM> to process the voice instruction and the image forming apparatus <NUM> cannot respond to the server <NUM> within the certain period of time.

Therefore, in the present embodiment, the image forming apparatus <NUM> includes a processing executor that executes processing according to an instruction (voice instruction) input via an external device, and a first notifier that sends, to the external device, a first notification according to a completion of the execution of the processing. Furthermore, the image forming apparatus <NUM> includes a second notifier that sends, to the external device, a second notification different from the first notification when it is estimated that the execution of the processing is not completed before a timeout time set in the external device for the first notification.

In the present embodiment, the external device is the voice input/output device <NUM> and the server <NUM>. Note that the external device may be the voice input/output device <NUM> as long as the voice input/output device <NUM> has the function of the server <NUM>.

The processing executor, the first notifier, and the second notifier are each provided as one function of the controller <NUM>. For example, the processing executor, the first notifier, and the second notifier are provided as programs executed by the controller <NUM>.

The processing executor executes the processing in the image forming apparatus <NUM> in response to the voice instruction input via the voice input/output device <NUM> and the server <NUM>. For example, when a setting for image formation (for example, sheet selection) is input to the image forming apparatus <NUM> by the voice instruction, the processing executor executes input processing of the setting for image formation on an operation screen of the operation panel <NUM>.

In addition, the first notifier sends, to the voice input/output device <NUM> and the server <NUM>, the first notification according to the completion of the execution of the processing in the image forming apparatus <NUM> (first notification processing). For example, as described above, when the processing executor executes the input processing of the setting for image formation and the input processing is completed without an error, the first notification to the effect that "setting is completed" is sent to the voice input/output device <NUM> and the server <NUM> in response to the completion of the execution.

In addition, the second notifier estimates whether the processing in the image forming apparatus <NUM> is completed before the timeout time set in the voice input/output device <NUM> and the server <NUM> for the first notification. Furthermore, when it is estimated that the processing in the image forming apparatus <NUM> is not completed before the timeout time, the second notifier sends the second notification different from the first notification to the voice input/output device <NUM> and the server <NUM> (second notification processing). For example, as described above, when the processing executor executes the input processing of the setting for image formation and it is estimated that the input processing is not completed before the timeout time, the second notification to the effect that "setting is impossible" is sent to the voice input/output device <NUM> and the server <NUM>.

Here, first, with reference to <FIG>, a procedure for a response in a case where the user gives an instruction directly from the operation panel <NUM> will be described. <FIG> is a sequence diagram for describing the procedure in the case of giving an instruction directly from the operation panel <NUM> in the image forming system <NUM>. After that, a procedure for responding to the voice instruction (response notification method and response notification program) in the image forming apparatus <NUM> having the above-described configuration will be described with reference to <FIG>.

The user selects a "color mode" as a setting for image formation by directly performing an input operation from the operation screen on the operation panel <NUM>.

Since, on the operation panel <NUM>, the "color mode" is selected in step S1, a command of the "color mode" is input to the controller <NUM>.

The controller <NUM> performs processing of setting the "color mode" in the image forming apparatus <NUM>.

If the image forming apparatus <NUM> cannot set the "color mode", the controller <NUM> outputs an error to the operation panel <NUM>. For example, if the image forming apparatus <NUM> is a device that does not correspond to the "color mode" but corresponds to a "monochrome mode", the controller <NUM> determines that the image forming apparatus <NUM> cannot set the "color mode", and output an error to the operation panel <NUM>.

The operation panel <NUM> displays the error input from the controller <NUM> on the operation screen. For example, the message "setting is impossible" is displayed on the operation screen.

The operation panel <NUM> displays "setting is impossible" on the operation screen to notify the user of this message.

As described above, when the user directly inputs the setting for image formation from the operation screen of the operation panel <NUM>, the processing is performed by the image forming apparatus <NUM>, and thus the operation is not interrupted even if the setting processing takes time.

Next, a procedure for responding to the voice instruction in a case where there is no delay in the response but there is an error will be described with reference to <FIG> is a sequence diagram for describing the procedure in the case where there is no delay in the response to the voice instruction but there is an error in the image forming system <NUM>.

The user gives the voice instruction to the voice input/output device <NUM> by saying "color mode". As described above, the voice instruction made by voice is, for example, a setting instruction for a condition for performing the image formation.

The voice input/output device <NUM> encodes the input voice into the voice data and transmits the voice data to the server <NUM>.

The server <NUM> (controller <NUM>) performs the voice recognition processing on the voice data transmitted from the voice input/output device <NUM>, and transmits the command of the "color mode" to the controller <NUM> of the image forming apparatus <NUM> since the instruction to set the "color mode" is made by voice in step S11. In this step and subsequent steps, the server <NUM> controls the response between the image forming apparatus <NUM> and the voice input/output device <NUM>.

The controller <NUM> performs the processing of setting the "color mode" in the image forming apparatus <NUM>. At this time, the controller <NUM> starts measuring a processing time by a timer T1 as described with reference to <FIG> (see step S64) to be described later. Furthermore, before starting this measurement, the controller <NUM> sets a response waiting time T0 according to the timeout time of the server <NUM> as described with reference to <FIG> (see step S62). The response waiting time TO is appropriately changed according to the timeout time of the server <NUM>, and a time shorter than the timeout time is set.

If the processing is completed and the image forming apparatus <NUM> cannot set the "color mode", the controller <NUM> notifies the server <NUM> of an error (first notification). The processing time measured by the timer T1 does not exceed the response waiting time T0 at this time. That is, the controller <NUM> notifies the server <NUM> of the error before the response waiting time T0. Therefore, in this case, the server <NUM> does not time out. Furthermore, here, the server <NUM> is notified of the error before step S16 to be described below, that is, immediately after completing the processing, and a time from the start of the processing to the error notification to the server <NUM> is shortened to prevent the timeout.

If the image forming apparatus <NUM> cannot set the "color mode", the controller <NUM> outputs the error to the operation panel <NUM>.

Upon receiving the error notification from the controller <NUM>, the server <NUM> transmits voice data of the message "setting is impossible" to the voice input/output device <NUM>.

The voice input/output device <NUM> converts the voice data of the message "setting is impossible" transmitted from the server <NUM> into an electric signal, outputs the electric signal to the speaker of the voice input/output device <NUM>, and causes the speaker to output a voice saying "setting is impossible".

As illustrated in <FIG>, if the timeout time of the server <NUM> is longer than the processing time of the image forming apparatus <NUM>, the timeout problem does not occur. However, the timeout time may vary depending on the server <NUM>, and thus if the processing in the image forming apparatus <NUM> is not completed within the timeout time, a dummy response is made to extend the timeout as illustrated in <FIG> and <FIG> to be described later.

Next, a procedure for responding to the voice instruction in a case where there is a delay in the response and there is an error will be described with reference to <FIG> is a sequence diagram for describing the procedure in the case where there is a delay in the response to the voice instruction and there is an error in the image forming system <NUM>.

Since steps S21 to S24 are the same as steps S11 to S14 illustrated in <FIG>, redundant description will be omitted here.

If it takes time for the image forming apparatus <NUM> to perform the processing of setting the "color mode", the controller <NUM> notifies the server <NUM> of a dummy (second notification). For example, the server <NUM> is notified that "processing is in progress", which is the dummy. Furthermore, the controller <NUM> may estimate the processing time, or may estimate that the processing takes time if the processing time being measured by the timer T1 exceeds the response waiting time T0 as described with reference to <FIG> (see step S67). That is, the controller <NUM> notifies the server <NUM> of the dummy before the timeout time of the server <NUM> to avoid the timeout of the server <NUM> and extend the timeout.

At this time, the controller <NUM> resets the timer T1 for the processing time to set the processing time measured by the timer T1 to zero as described with reference to <FIG> (see step S69). The timer T1 then starts remeasurement.

Note that processing according to the setting instruction in the controller <NUM> includes processing of displaying contents of the setting instruction on the operation panel <NUM> of the image forming apparatus <NUM>, prohibition processing of determining whether the contents of the setting instruction include prohibited contents, and the like. For example, when the image data is transmitted, the prohibition processing is required to specify a destination, and when such processing is performed, the setting processing may take time.

Furthermore, as the dummy, a fixed phrase such as "work is in progress" or "please wait for a while" may be used instead of "processing is in progress", or remaining time of the processing may be used.

If it takes time for the image forming apparatus <NUM> to perform the processing of setting the "color mode", the controller <NUM> outputs the dummy to the operation panel <NUM>. For example, the message "processing is in progress" is output as the dummy.

The operation panel <NUM> displays the dummy input from the controller <NUM> on the operation screen. For example, the message "processing is in progress" is displayed on the operation screen.

Upon receiving the dummy notification from the controller <NUM>, the server <NUM> transmits voice data of the message "processing is in progress" to the voice input/output device <NUM>.

The voice input/output device <NUM> converts the voice data of the message "processing is in progress" transmitted from the server <NUM> into an electric signal, outputs the electric signal to the speaker of the voice input/output device <NUM>, and causes the speaker to output a voice saying "processing is in progress".

If the image forming apparatus <NUM> cannot set the "color mode", the controller <NUM> notifies the server <NUM> of an error (first notification). In step S25, the timer T1 is reset so that the processing time measured by the timer T1 is set to zero, and the timer T1 remeasures the processing time. The processing time remeasured by the timer T1 does not exceed the response waiting time T0 at this time. That is, the controller <NUM> notifies the server <NUM> of the error before the response waiting time T0 after the remeasurement. Therefore, in this case, the server <NUM> does not time out. Furthermore, here as well, the server <NUM> is notified of the error before step S31 to be described below, that is, immediately after completing the processing, and the time from the start of the remeasurement to the error notification to the server <NUM> is shortened to prevent the timeout.

Since steps S31 to S34 are the same as steps S16 to S19 illustrated in <FIG>, redundant description will be omitted here.

Next, a procedure for responding to the voice instruction in a case where there is a delay in the response and there is no error will be described with reference to <FIG> is a sequence diagram for describing the procedure in the case where there is a delay in the response to the voice instruction and there is no error in the image forming system <NUM>.

Since steps S41 to S49 are the same as steps S21 to S29 illustrated in <FIG>, redundant description will be omitted here.

After setting the "color mode" in the image forming apparatus <NUM>, the controller <NUM> notifies the server <NUM> of the setting completion (first notification). In step S45, the timer T1 is reset so that the processing time measured by the timer T1 is set to zero, and the timer T1 remeasures the processing time. The processing time remeasured by the timer T1 does not exceed the response waiting time T0 at this time. That is, the controller <NUM> notifies the server <NUM> of the setting completion before the response waiting time T0 after the remeasurement. Therefore, in this case, the server <NUM> does not time out. Furthermore, here as well, the server <NUM> is notified of the setting completion before step S51 to be described below, that is, immediately after completing the processing, and the time from the start of the remeasurement to the notification of the setting completion to the server <NUM> is shortened to prevent the timeout.

After setting the "color mode" in the image forming apparatus <NUM>, the controller <NUM> outputs the setting completion to the operation panel <NUM>.

The operation panel <NUM> displays the setting completion input from the controller <NUM> on the operation screen. For example, the message "setting is completed" is displayed on the operation screen.

Upon receiving the notification of the setting completion from the controller <NUM>, the server <NUM> transmits voice data of the message "setting is completed" to the voice input/output device <NUM>.

The voice input/output device <NUM> converts the voice data of the message "setting is completed" transmitted from the server <NUM> into an electric signal, outputs the electric signal to the speaker of the voice input/output device <NUM>, and causes the speaker to output a voice saying "setting is completed".

As described above, if the processing in the image forming apparatus <NUM> is not completed within the timeout time of the server <NUM>, the dummy response is made to extend the timeout. As a result, it is possible to prevent interruption and re-operation of the voice operation, and it is possible to prevent operability of the voice operation from deteriorating.

Next, a control procedure in the controller <NUM> of the image forming apparatus <NUM> in the above-described sequence of <FIG> will be described with reference to <FIG> is a flowchart for describing the control procedure in the image forming apparatus <NUM>.

The controller <NUM> determines whether a voice operation service is selected. If the voice operation service is selected (YES), the flow proceeds to step S62, and if the voice operation service is not selected (NO), the flow proceeds to step S70. If the voice operation service is not selected, for example, the sequence illustrated in <FIG> described above is implemented. Furthermore, the selection of the voice operation service is preset by the user using, for example, the operation panel <NUM>.

The controller <NUM> sets the response waiting time T0 according to the timeout time of the server <NUM>. The response waiting time T0 is set shorter than the timeout time by a predetermined time. For example, when the timeout time is seven seconds, the response waiting time T0 is six seconds.

The controller <NUM> confirms whether there is the voice instruction from the server <NUM>. If there is the voice instruction (YES), the flow proceeds to step S64, and if there is no voice instruction (NO), step S63 is repeated.

If there is the voice instruction from the server <NUM>, the controller <NUM> starts measuring the processing time by the timer T1. The timer T1 measures the time taken for the processing executed by the image forming apparatus <NUM> in response to the voice instruction, for example, the setting processing described above.

The controller <NUM> starts executing the processing executed by the image forming apparatus <NUM> in response to the voice instruction, for example, the setting processing described above.

The controller <NUM> determines whether the processing executed by the image forming apparatus <NUM> is completed. If the processing is completed (YES), the flow proceeds to step S70, and if the processing is not completed (NO), the flow proceeds to step S67. That is, the controller <NUM> repeats steps S67 to S69 to be described later until the processing is completed. Note that, in order to prevent the processing from being prolonged, an upper limit of the number of times of extending the timeout may be set, and steps S67 to S69 to be described later may be repeated up to the upper limit.

The controller <NUM> compares the processing time measured by the timer T1 with the response waiting time T0, the flow proceeds to step S68 if the processing time measured by the timer T1 exceeds the response waiting time T0 (YES), and the flow returns to step S66 if the processing time measured by the timer T1 does not exceed the response waiting time T0 (NO).

In the case of T1 > T0, that is, in a case where the processing time measured by the timer T1 exceeds the response waiting time T0, the controller <NUM> notifies the server <NUM> of the dummy and displays the dummy on the screen of the operation panel <NUM>. For example, the server <NUM> is notified that "processing is in progress", which is the dummy, and the message "processing is in progress" is displayed on the screen of the operation panel <NUM>. As a result, the server <NUM> does not time out.

The controller <NUM> resets the timer T1 for the processing time to set the processing measured by the timer T1 to zero. The timer T1 then starts the remeasurement, and the flow returns to step S66.

In the case of repeating steps S67 to S69, the controller <NUM> notifies, in the case of T1 > T0 in step S67, the server <NUM> of the dummy again and displays the dummy on the operation panel <NUM> again.

When the processing is completed, the controller <NUM> notifies the server <NUM> of a processing result and displays the processing result on the screen of the operation panel <NUM>. For example, in the case of <FIG>, the server <NUM> is notified that "setting is completed", and the message "setting is completed" is displayed on the screen of the operation panel <NUM>.

As described above, in the image forming apparatus <NUM>, the controller <NUM> measures, by the timer T1, the processing time of the processing according to the voice instruction, and notifies the server <NUM> of the dummy if the processing time measured by the timer T1 exceeds the preset response waiting time T0. Since the response waiting time T0 is set shorter than the timeout time of the server <NUM>, notifying the server <NUM> of the dummy in the case of T1 > T0 makes it possible to reliably avoid the timeout of the server <NUM>.

As described in detail above, in the image forming system <NUM> of the present embodiment, the image forming apparatus <NUM> includes a processing executor that executes processing according to an instruction (voice instruction) input via an external device, and a first notifier that sends, to the external device, a first notification according to a completion of the execution of the processing. Furthermore, the image forming apparatus <NUM> includes a second notifier that sends, to the external device, a second notification different from the first notification when it is estimated that the execution of the processing is not completed before a timeout time set in the external device for the first notification.

According to the present embodiment configured as described above, it is possible to avoid the timeout of the server <NUM> when the voice operation is performed from the voice input/output device <NUM> and the server <NUM>, which are the external devices.

Therefore, since the voice operation is not interrupted by the timeout of the server <NUM>, a subsequent voice operation can be continued and the operation does not have to be restarted from the beginning. Thus, it is possible to prevent the voice operation from being interrupted and the operability of the voice operation from deteriorating.

Claim 1:
An image forming apparatus (<NUM>) operated by voice from an external device, said image forming apparatus comprising:
a processing executor (<NUM>) that executes processing according to an instruction received from the external device (<NUM>, <NUM>);
a first notifier (<NUM>) that sends, to the external device (<NUM>, <NUM>), a first notification according to a completion of the execution of the processing; and
a second notifier (<NUM>) that sends, to the external device (<NUM>, <NUM>), a second notification different from the first notification when it is estimated that the execution of the processing is not completed before a timeout time of the external device (<NUM>, <NUM>) for the first notification.