Information processing system, information processing apparatus, control method executed by the same, and storage medium

An information processing apparatus that communicates with a terminal includes a reception unit configured to receive from the terminal an execution instruction to execute a job using a function of the information processing apparatus, wherein the execution instruction at least includes setting information regarding the job, and a transmission unit configured to, in a case where the reception unit receives the execution instruction to execute the job and a predetermined screen is displayed on a display unit of the information processing apparatus, transmit information for the terminal to display the screen to the terminal having transmitted the execution instruction.

BACKGROUND

Field of the Disclosure

The present disclosure relates to a data processing system where a data processing apparatus and an image forming apparatus communicate with each other to execute a job.

Description of the Related Art

In recent years, with the proliferation of high-performance mobile terminals called “smartphones”, an application for using an image forming apparatus from the mobile terminal is provided. For example, in the execution of the function of transmitting a scanned document to a specified destination, the terminal searches for an image forming apparatus on a local area network (LAN) and adds the email address of a transmission destination set by a user and various pieces of scan setting information (e.g., color and two-sided), thereby generating job information. The terminal transmits the generated job information to an image forming apparatus found by the search, whereby a series of operations on the image forming apparatus necessary to execute a job can be executed from the terminal. At this time, network communication between the mobile terminal and the image forming apparatus is established in accordance with a communication protocol such as Transmission Control Protocol/Internet Protocol (TCP/IP) and can be implemented by a known technique. Japanese Patent Application Laid-Open No. 2013-232085 discusses a technique for setting a job in a client terminal such as a mobile terminal and transmitting the settings of the job from the client terminal to an image forming apparatus to cause the image forming apparatus to execute the job. In this manner, a user can cause the image forming apparatus to execute the job without operating the image forming apparatus.

However, in a case where the user causes the image forming apparatus to execute the job by operating the terminal, the user is operating the terminal and therefore may not view a notification to the user displayed on an operation panel of the image forming apparatus. For example, in a case where the image forming apparatus having received an execution instruction to execute a job automatically executes a login process, the user operating the terminal may not view a notice screen displayed to the user on the operation panel of the image forming apparatus after the login process.

SUMMARY

Embodiments of the present disclosure are directed to providing a method in which even in a case where a user gives an execution instruction to execute a job to an apparatus from a terminal, the user operating the terminal can confirm a screen displayed on the apparatus.

According to embodiments of the present disclosure, an information processing apparatus that communicates with a terminal includes a reception unit configured to receive from the terminal an execution instruction to execute a job using a function of the information processing apparatus, wherein the execution instruction at least includes setting information regarding the job, and a transmission unit configured to, in a case where the reception unit receives the execution instruction to execute the job and a predetermined screen is displayed on a display unit of the information processing apparatus, transmit information for the terminal to display the screen to the terminal having transmitted the execution instruction.

DESCRIPTION OF THE EMBODIMENTS

With reference to the drawings, exemplary embodiments of the present disclosure will be described below. Components described in the exemplary embodiments, however, are merely illustrative, and are not intended to limit the scope of the present disclosure to them only. Although the following description is given using an image forming apparatus as an example of an information processing apparatus, the present disclosure is not limited to this.

FIG.1is a diagram illustrating the configuration of an information processing system according to a first exemplary embodiment. A data processing system100includes a data processing apparatus101, a wireless local area network (LAN) terminal102, and an image forming apparatus104. These apparatuses are connected to each other via a network103such as a LAN. To the network103, an apparatus other than those illustrated inFIG.1may be connected.

The data processing apparatus101is a mobile terminal such as a smartphone, and may be an information processing apparatus in which an operating system for a small-sized terminal and programs for controlling a voice call, position detection, and data communication operate. Alternatively, the data processing apparatus101may be a personal computer (PC) that does not have the functions of voice call control and position detection control. The data processing apparatus101is connected to the network103via the wireless LAN terminal102.

The wireless LAN terminal102is a wireless LAN base station having a general network/router function and provides a wireless LAN in a home or an office.

The image forming apparatus104is a multifunction peripheral having various functions such as a printer function, a copy function, a scanner function, and a fax transmission function. The image forming apparatus104, however, is not limited to this, and may be an apparatus having only a single function. The image forming apparatus104may include a wireless access point inside and may be configured to directly communicate with the data processing apparatus101. Using the data processing apparatus101communicating with the image forming apparatus104, a user can instruct the image forming apparatus104to execute a job and can execute various functions of the image forming apparatus104. In the present exemplary embodiment, a form is employed in which the image forming apparatus104is connected to the network103in a wired manner. Alternatively, similarly to the data processing apparatus101, the image forming apparatus104may be wirelessly connected to the network103using the wireless LAN terminal102.

Further, the data processing apparatus101and the image forming apparatus104can perform short-range wireless communication using near-field communication (NFC) or Bluetooth®. The image forming apparatus104transmits connection information (an Internet Protocol (IP) address, a media access control (MAC) address, or a Service Set Identifier (SSID)) for establishing a wireless LAN connection with the image forming apparatus104via NFC communication units or Bluetooth Low Energy (BLE) communication units (described below) to the data processing apparatus101. Then, the data processing apparatus101starts wireless LAN communication with the image forming apparatus104using the acquired connection information.

FIG.2is a hardware configuration diagram of the data processing apparatus101according to the present exemplary embodiment. The data processing apparatus101includes a controller unit201. The controller unit201controls various communication units such as an NFC communication unit210, a BLE communication unit211, and a wireless LAN communication unit212, and various user interface (UI) units such as a microphone/loudspeaker213, a display214, a touch panel215, and a camera216.

The controller unit201includes a central processing unit (CPU)202, a read-only memory (ROM)203, a random-access memory (RAM)204, a network interface (I/F)205, a sound control unit206, a display control unit207, an input control unit208, and a storage device209. These components are connected to each other by a system bus217.

The CPU202controls the entirety of the system of the data processing apparatus101. The ROM203stores an operating system of the data processing apparatus101and programs for controlling a telephone call and data communication. The CPU202executes various programs. The RAM204is used as a temporary storage area such as a main memory or a work area for the CPU202.

The storage device209is a non-volatile storage device and records various operation mode settings that need to be held even after the restart of the data processing apparatus101, and an operation log.

The network I/F205is connected to the NFC communication unit210, the BLE communication unit211, and the wireless LAN communication unit212and exchanges data with the image forming apparatus104through various types of wireless communication.

The sound control unit206controls the input and output of sound data via the microphone/loudspeaker213. The display control unit207controls the output of a screen to be displayed on a display unit such as the display214. The input control unit208controls the input of information designated by the user using a hardware key or the touch panel215. Various applications executed by the data processing apparatus101use the sound control unit206, the display control unit207, and the input control unit208.

FIG.3is a hardware configuration diagram of the image forming apparatus104according to the present exemplary embodiment. The image forming apparatus104includes a controller unit301. The controller unit301controls various communication units such as an NFC communication unit310, a BLE communication unit311, and a wireless LAN communication unit312, an operation unit307, a scanner313that executes a scan function, and a printer314that executes a print function. In a case where the user uses a copy function of the image forming apparatus104, the controller unit301controls the scanner313to acquire image data on a document and controls the printer314to print the image on a sheet and output the sheet. In a case where the user uses a “scan and transmit” function, the controller unit301controls the scanner313to acquire image data on a document and convert the image data into code data. Then, the controller unit301transmits the code data to the data processing apparatus101via a network I/F308. In a case where the user uses the print function, the controller unit301receives image data (code data) from the data processing apparatus101via the network I/F308. Then, the controller unit301transmits the received image data to the printer314. Based on the received image data, the printer314prints the image on a sheet and outputs the sheet. The image forming apparatus104also has a fax reception function for receiving data from an Integrated Services Digital Network (ISDN) line and printing the data, and a fax transmission function for transmitting scanned data to the ISDN line. A task as a processing target for which settings are made by the user to execute each function as illustrated above is referred to as a “job”. The image forming apparatus104executes a job in accordance with job information. For example, in the case of a “scan and transmit” job, the image forming apparatus104performs scanning based on scan settings included in job information and transmits image data generated by the scanning to a destination included in the job information.

The controller unit301includes a CPU302, a RAM303, a ROM304, a hard disk drive (HDD)305, an operation unit I/F306, the network I/F308, and a device I/F309. These components are connected to each other by a system bus312.

The CPU302controls the entirety of the system of the image forming apparatus104. The RAM303is a system work memory for the operation of the CPU302and temporarily stores image data.

The RAM303also stores scan image data read by the scanner313and stores print image data received from the data processing apparatus101via the network103.

The ROM304stores a boot program for the system and an application. The HDD305stores an operating system, system software, application software, image data, and setting data.

The operation unit307displays information to the user and receives an operation from the user.

The operation unit307includes an operation panel such as a touch panel display, and a hardware key.

The operation unit I/F306outputs information to be displayed on the operation unit307to the operation unit307.

The operation unit I/F306also receives information input by the user from the operation unit307. The network I/F308is connected to the NFC communication unit310, the BLE communication unit311, and the wireless LAN communication unit312and exchanges information with the data processing apparatus101through various types of communication. The wireless LAN communication unit312performs wireless LAN communication with the data processing apparatus101via the network103. The NFC communication unit310and the BLE communication unit311perform short-range wireless communication with the data processing apparatus101. The image forming apparatus104receives job setting information, image data, and a job execution command from the data processing apparatus101via the network I/F308and executes a job. The device I/F309connects the scanner313and the printer314that read and print image data to the controller unit301, and inputs and outputs image data.

FIG.4is a block diagram illustrating the software configuration of the data processing apparatus101according to the present exemplary embodiment.FIG.4is a functional block diagram of software implemented by the CPU202reading a control program stored in the ROM203or the storage device209.

An application401is an application installed on the data processing apparatus101and used to cause the image forming apparatus104to operate. The application401communicates with the image forming apparatus104and transmits various requests to the image forming apparatus104or receives various responses from the image forming apparatus104, thereby cooperating with the image forming apparatus104. For example, print job information or scan job information can be generated by making settings on the application401, and the generated job information can be transmitted to the image forming apparatus104. Various other applications are installed on the data processing apparatus101in addition to the application401, but are not described. The configuration of the application401is described below.

A UI control unit402receives job setting information input by the user using the touch panel215and connection information for connecting to the image forming apparatus104and provides the job setting information and the connection information to a multifunction peripheral (MFP) control unit403. The UI control unit402acquires a response from the MFP control unit403and outputs the response to the display214via the display control unit207. Further, the UI control unit402controls an auto clearing function as a part of user interface control. “Auto clearing” is the process of, in a case where the data processing apparatus101does not receive an operation from the user for a predetermined time, clearing all setting values set on a display screen or displaying an initial screen set in advance. The “initial screen” can be set in advance by the user. If the user is logged into the data processing apparatus101when a function screen is displayed or auto clearing occurs, the UI control unit402automatically performs a logout process. Further, the UI control unit402also has a function as a web browser that displays a web content screen. That is, the UI control unit402can also acquire web content using a Uniform Resource Locator (URL) and display the web content. Further, the UI control unit402can also operate as a Virtual Network Computing (VNC) client.

The MFP control unit403establishes communication with the image forming apparatus104and sends a request to the image forming apparatus104or receives a response from the image forming apparatus104. The MFP control unit403generates job information from job settings and authentication information received by the UI control unit402and transmits the job information to the image forming apparatus104via the network103. The MFP control unit403also acquires the execution status of a job from the image forming apparatus104via the network103and transmits the execution status of the job to the UI control unit402.

FIG.5is a block diagram illustrating the software configuration of the image forming apparatus104according to the present exemplary embodiment.

System software501is software executed by the CPU302of the image forming apparatus104and is stored in the RAM303.

A UI control unit502receives information input via the operation unit307by the user and controls a screen. The UI control unit502also has a function as a web browser that displays web content. For example, the UI control unit502can acquire web content as a notice screen through which the user is given notice from an external server or the inside of the image forming apparatus104, and display the web content.

An authentication control unit503manages authentication information about the user, an authentication setting, and an authentication state. For example, the authentication control unit503performs a user authentication process based on authentication information received from the data processing apparatus101and returns the result of the user authentication process to the data processing apparatus101.

An automatic control processing unit504performs automatic control of the image forming apparatus104based on a request received from the data processing apparatus101. The “automatic control” refers to the process of successively executing a series of processes that is defined in advance by combining a plurality of processes.

An automatic control management table505is a table that manages information necessary for the automatic control processing unit504to perform the automatic control of the image forming apparatus104.

A job control unit506controls processing units such as a scan processing unit511, a copy processing unit512, a print processing unit513, and a fax processing unit514in accordance with job settings to execute various jobs. Although not illustrated, the image forming apparatus104can also function as a VNC server.

FIGS.6A and6Bcollectively illustrate an example of the automatic control management table505according to the present exemplary embodiment. The automatic control management table505defines processes of the automatic control performed by the image forming apparatus104in each condition that is determined based on a combination of the authentication setting and the authentication state of the image forming apparatus104. The automatic control includes a plurality of steps, and a single row of the table defines a process performed in a single step. If the image forming apparatus104receives job information as a job execution request from the data processing apparatus101, the processes of steps are started.

An “authentication setting” column601is a column indicating the authentication setting of the image forming apparatus104. The authentication setting is set by the administrator of the image forming apparatus104. The authentication setting includes an “authentication unnecessary” setting that does not require user authentication to operate the image forming apparatus104, and an “authentication necessary” setting that requires authentication to operate the image forming apparatus104. The “authentication necessary” setting includes “authentication necessary/display login screen when operation on device is started”, which is a setting for requesting authentication in a case where the user starts operating the image forming apparatus104, and “authentication necessary/display login screen when function that requires authentication is selected”, which is a setting for requesting authentication only in a case where the user uses a predetermined function.

An “authentication state” column602is a column indicating the authentication state of the image forming apparatus104. The authentication state includes an authenticated state and an unauthenticated state. The automatic control to be performed is determined based on the combination of the “authentication setting” column601and the “authentication state” column602.

An “execution step” column603defines the processing order of the automatic control. The processes are performed in ascending order of numbers. Execution step0is a special step and defines the process of checking to determine whether the automatic control can be started. If it is determined in execution step0that the automatic control can be started, the subsequent execution steps are performed. If not, the subsequent execution steps are not performed.

A “process” column604defines a process to be performed in each execution step. A “parameter” column605defines a parameter to be assigned when the process defined in the “process” column604is performed.

An “execution condition” column606defines a condition for performing each execution step. A step in which the execution condition is “-” is always performed. In an execution step in which a specific condition is defined as the execution condition, only if the condition is satisfied, the execution step is performed. If the condition is not satisfied, the execution step is not performed, and the automatic control proceeds to the next execution step.

An “automatic transition” column607defines whether to automatically proceed to the next execution step without an operation of the user. In an execution step in which the automatic transition is “YES”, if the defined process is performed, the process of the next execution step is automatically performed. In an execution step in which the automatic transition is “NO”, after the defined process is performed, an operation of the user is waited for. A “user operation” column608defines a user operation that can be received after an execution step in which the automatic transition is “NO” is performed.

After an execution step in which the automatic transition is “NO” is performed, the next execution step is performed upon receipt of the operation defined in the “user operation” column608.

A “notification method” column609is a column describing a method for causing the data processing apparatus101to display a screen displayed on the image forming apparatus104in a case where an execution step is performed. This column is used in a determination in step S910(described below). If the “notification method” column609indicates “URL”, a URL for accessing the web content currently displayed on the image forming apparatus104is transmitted to the data processing apparatus101.

If the “notification method” column609indicates “VNC”, the data processing apparatus101displays the same screen as a screen currently displayed on the image forming apparatus104, using VNC.

An example of the automatic control is described. For example, in a case where the authentication setting is “authentication necessary/display login screen when operation on device is started” and the authentication state is “unauthenticated”, upon receipt of an execution instruction to execute a job from the data processing apparatus101, the processes of rows617to625are executed. If the automatic control can be started, then in execution step1, a login screen for receiving the input of authentication information is displayed. Next, in execution step2, a login process is performed using authentication information included in job information. In execution step3, a display language is switched. If a setting is made such that the image forming apparatus104displays the notice screen after login, then in execution step4, the notice screen is displayed. If the notice screen is displayed, the automatic control does not automatically transition to the next step. If the image forming apparatus104or the data processing apparatus101receives a user operation, the automatic control proceeds to the next execution step. In execution step5, an initial screen set by the logged-in user is displayed. Then, in execution step6, a setting screen for the job for which the execution instruction is given, i.e., a screen of an application, is displayed. Then, the image forming apparatus104executes the job based on job settings included in the execution instruction to execute the job. At this time, if a sheet absence error is detected, a sheet setting screen is displayed. Then, if a setting is received, the job is executed. As described above, if the image forming apparatus104receives job information from the data processing apparatus101, the image forming apparatus104causes screens to sequentially automatically transition and executes a job.

FIG.7is a sequence diagram illustrating the exchange of requests and responses between the data processing apparatus101and the image forming apparatus104according to the present exemplary embodiment.

In step S701, the data processing apparatus101transmits job information including the setting values of a job input by the user, as a job execution request to the image forming apparatus104.

In step S702, the image forming apparatus104checks whether the job corresponding to the job information received by the data processing apparatus101can be executed. In step S703, the image forming apparatus104transmits the result of the check as a response to the job information to the data processing apparatus101.

In step S704, the image forming apparatus104references the automatic control management table505illustrated inFIGS.6A and6Band executes the processes of the execution steps in the automatic control. In step S705, the image forming apparatus104determines whether the execution of the automatic control are is completed up to the last step. If the execution of the automatic control is completed up to the last step (YES in step S705), the processing ends. If not (NO in step S705), the processing returns to step S704. In step S704, the image forming apparatus104executes the process of the next execution step.

Next, the processes of steps S710to S713are described. The processes of step S710and the subsequent steps are processes for displaying a screen indicating the status of the image forming apparatus104or a notice screen on the data processing apparatus101, and the data processing apparatus101inquires of the image forming apparatus104at regular intervals. The process of displaying the execution status on the data processing apparatus101(the processes of steps S710to S713) and the automatic control process in the image forming apparatus104(the processes of steps S704and S705) are performed asynchronously.

In step S710, the data processing apparatus101transmits an execution status acquisition request to inquire of the image forming apparatus104about the execution status. In step S711, the image forming apparatus104acquires the execution status and transmits the execution status to the data processing apparatus101. In step S712, based on the response received from the image forming apparatus104, the data processing apparatus101displays a screen on the display214. In step S713, if the execution status received from the image forming apparatus104indicates that the execution is completed (YES in step S713), the processing ends. If not (NO in step S713), the processing returns to step S710.

FIGS.13A to13Eare examples of screens displayed on the operation unit307of the image forming apparatus104in a case where the user causes the image forming apparatus104to execute a job.

FIG.13Ais an example of an initial screen that is a home screen and is displayed if the user succeeds in authentication and logs into the image forming apparatus104. On the home screen, buttons corresponding to applications that can be executed by the image forming apparatus104are arranged. The home screen has the function of, if it is detected that the user selects any of the buttons, calling the selected application.

An area1301indicates the name of the currently displayed screen. A button1302is used to call an application. As an example, the button1302indicates a button for calling a “scan and transmit” application. An icon1303is used to advance the page. The initial screen to be displayed if the user logs into the image forming apparatus104is not limited to the home screen, and may be a screen of each application.

FIG.13Bis an example of a screen of an application for executing a job. As an example,FIG.13Billustrates a screen of the “scan and transmit” application which is displayed after the button1302is pressed.

An area1311is used to set a destination to which an electronic mail is to be transmitted. A button1312is used to set the setting value of scanning. A button1313is used to call setting items that cannot be included in the screen. A button1314is used to receive the start of the job from the user.

FIG.13Cis an example of a screen during the execution of the job. This screen is displayed if the button1314is pressed on the screen inFIG.13B.FIG.13Cillustrates a pop-up1321that is displayed on the screen illustrated inFIG.13B, and the pop-up1321indicates the execution status of the job. An area1322displays the setting values of the currently executed job. An area1323indicates the progress status of the job. A button1324is used to receive the cancellation of the job from the user.

FIG.13Dis an example of a notice screen. The “notice screen” is a screen for the administrator to display content of which the user is be notified when using the image forming apparatus104. Although the description is given on the assumption that the notice screen is web content included in the image forming apparatus104or an external server, the notice screen may not be web content. The timing when the notice screen is displayed is determined based on the setting. For example, the notice screen can be set to be displayed immediately after the user logs into the image forming apparatus104(before the initial screen is displayed) or immediately after the user logs out of the image forming apparatus104. A configuration may be employed in which every time the notice screen is displayed, the web content is acquired from the external server, or a configuration may be employed in which the web content acquired from the server is saved in advance in the image forming apparatus104and displayed. A button1331is used to close the notice screen. If the button1331is pressed, the automatic control proceeds to the next process (e.g., the display of the initial screen). An area1332displays an image and a message. A cancel button1333is a button for cancelling the next process. For example, in a case where the notice screen is displayed after login, upon pressing of the cancel button1333, the initial screen is not displayed, and the notice screen returns to the screen before login.

FIG.13Eis a display screen of operation assistance information. The “operation assistance information” is information for notifying the user of an operation to be performed by the user so that the image forming apparatus104continues the execution of the job. For example,FIG.13Eillustrates a setting screen1341for, in a case where set sheets run out during the execution of a copy job, newly selecting sheets to be used. A pop-up dialog may be displayed on the screen currently displayed on the operation unit307to present operating instructions, or operation information may be displayed in a blank portion of the screen so that an operation of the user is not hindered. If the execution of the job is to be continued, the user selects the setting of sheets to be used and presses an OK button1342. If the processing is not to be continued, the user presses a cancel button1343. A user operation for continuing the execution of the job is defined in the “user operation” column608in the automatic control management table505. Based on this definition, the UI control unit502determines content to be displayed as the operation assistance information.

FIGS.12A to12Gare examples of screens displayed by the application401of the data processing apparatus101.

FIG.12Gillustrates a screen for inputting the settings of job information to be transmitted to the image forming apparatus104. An area1261indicates an application to be executed. As an example, a “scan and transmit” application is displayed. An area1262is an area to which a transmission destination is input.

An area1263is an area where the setting values of scanning are set. An area1264is an area to which authentication information for logging into the image forming apparatus104is input. If the user presses an “execute job” button1265, the job information is transmitted from the data processing apparatus101to the image forming apparatus104. The job information includes the job setting information input to the areas1262and1263and the authentication information input to the area1264. The image forming apparatus104having received the job information executes a login process using the authentication information included in the job information and then executes a job.

FIG.12Ais an example of a notice screen. The notice screen is a screen displayed by acquiring web content using a URL acquired from the image forming apparatus104. The same screen as the notice screen displayed on the image forming apparatus104(FIG.13D) is displayed. The timing when the notice screen is displayed may be the same as or different from the timing when the notice screen is displayed on the image forming apparatus104. For example, if the notice screen is set to be displayed after the user logs into the image forming apparatus104, the notice screen is displayed by the application401after the job information is transmitted from the data processing apparatus101and the image forming apparatus104executes a login process. A button1201is used to close the notice screen and proceed to the next process. An area1202displays an image and a message. A cancel button1203is used to stop the processing.

FIG.12Bis another example of a screen displayed by the application401in a case where the notice screen is currently displayed on the image forming apparatus104. This screen displays a message1211indicating that a user operation on the image forming apparatus104is required, a “Yes” button1212for continuing the automatic control, and a “No” button1213for suspending the automatic control. The application401may perform display for prompting the user to confirm the screen displayed on the image forming apparatus104. In this manner, if a screen that does not automatically transition, such as the notice screen, is currently displayed on the image forming apparatus104, the application401notifies the user that a user operation is required.

FIG.12Cis an example of a screen indicating the execution status of the job. For example, this screen is displayed after the “execute job” button1265is pressed. An area1221displays the current execution status of the job using a message and an image. A button1222is used to send a cancellation request to suspend the automatic control to the image forming apparatus104by a user operation.

FIG.12Dis an example of a screen including a message indicating the status of the image forming apparatus104in addition to the execution status of the job. This screen displays a message1231received from the image forming apparatus104in step S818(described below).

FIG.12Eis an example where the same screen as a screen displayed on the image forming apparatus104is displayed using VNC in the process of step S810(described below). The same screen as the screen of the operation assistance information inFIG.13Eis displayed in this example. That is, the user can operate the image forming apparatus104on the application401. An OK button1242is a button for finalizing the settings and continuing the processing of the job. A cancel button1243is a button for cancelling the processing.

FIG.12Fis an example of a screen displayed in step S821(described below), and this screen is displayed if web content cannot be accessed. An area1251indicates that content to be displayed cannot be accessed. A button1252is used to close this screen.

FIGS.8A and8Bare a flowchart illustrating a job execution instruction process executed by the data processing apparatus101according to the present exemplary embodiment. The flowchart inFIGS.8A and8Bis implemented by the CPU202of the data processing apparatus101loading programs for achieving the control modules stored in the ROM203or the storage device209into the RAM204and executing the programs. The flowchart inFIGS.8A and8Bcorresponds to the processing performed by the data processing apparatus101in the sequence diagram inFIG.7.

In step S801, based on an operation of the user on a screen of the application401, the UI control unit402receives the input of connection information (e.g., an IP address) for connecting to the image forming apparatus104to which an execution instruction to execute a job is to be given. The information for connecting to the image forming apparatus104needs not be input by the user. Alternatively, the UI control unit402may search for the image forming apparatus104connected to the network103using the wireless LAN communication unit212, or may acquire the information from the image forming apparatus104via the NFC communication unit210or the BLE communication unit211. In the subsequent steps, the UI control unit402communicates with the image forming apparatus104using the information received in step S801.

In step S802, based on an operation of the user on the screen of the application401, the UI control unit402receives settings for generating job information. Specifically, the UI control unit402receives various settings from the user on the screen inFIG.12G. The job information includes an application to be executed by the image forming apparatus104, the setting values of the job, and information necessary for the image forming apparatus104to execute the job, such as authentication information.

In step S803, based on the reception of a predetermined operation from the user, the MFP control unit403transmits the job information including the settings received in step S802as a job execution request to the image forming apparatus104. The “predetermined operation” is, for example, the pressing of the “execute job” button1265.

In step S804, the MFP control unit403receives a job execution response as a response to the job execution request in step S803from the image forming apparatus104. In step S805, the MFP control unit403determines the content of the job execution response. If the content of the job execution response indicates success (YES in step S805), the processing proceeds to step S806. If not (NO in step S805), the processing proceeds to step S830.

In step S806, the MFP control unit403transmits an execution status acquisition request to acquire the execution status of the job to the image forming apparatus104. In step S807, the MFP control unit403receives an execution status acquisition response as a response to the execution status acquisition request in step S806from the image forming apparatus104.

In step S808, based on the execution status acquisition response received in step S807, the MFP control unit403determines whether waiting for a user operation occurs in the image forming apparatus104. Specifically, the MFP control unit403determines whether the image forming apparatus104executes a step in which the column607is “NO” inFIGS.6A and6B, and whether the automatic control is stopped. If waiting for a user operation occurs (YES in step S808), the processing proceeds to step S809. If not (NO in step S808), the processing proceeds to step S817.

In step S809, based on the execution status acquisition response received in step S807, the MFP control unit403determines the type of the response. Specifically, the MFP control unit403determines whether the type of the response is “VNC” or “URL” defined in the column609inFIGS.6A and6B. If a URL is included in the execution status acquisition response, it is determined that the type of the response is “URL” (URL in step S809), and the processing proceeds to step S811. If a VNC connection instruction and information necessary for a VNC connection are included in the execution status acquisition response, it is determined that the type of the response is “VNC” (VNC in step S809), and the processing proceeds to step S810.

In step S810, the data processing apparatus101connects to the image forming apparatus104with VNC using a remote framebuffer (RFB) protocol, and the UI control unit402displays the same screen as a screen displayed on the operation unit307of the image forming apparatus104. The user can remotely operate the image forming apparatus104. The VNC connection may be automatically executed, or an inquiry is made to the user, and if the VNC connection is permitted, the data processing apparatus101may connect to the image forming apparatus104. If the operation with VNC is completed, the processing returns to step S806to acquire the next execution status.

In step S811, the UI control unit402attempts to access a server using the URL included in the response acquired in step S807. If the attempt is successful (YES in step S811), the processing proceeds to step S812. If the attempt fails (NO in step S811), the processing proceeds to step S821.

In step S812, the UI control unit402accesses the URL acquired in step S807and displays web content on the display214of the data processing apparatus101(FIG.12A). In this case, a WebView function built into the application401is used. Thus, the web content may be any content that can be displayed by WebView. Specifically, the content may not only be an image in the HyperText Markup Language (HTML) format, but also be an image in the Joint Photographic Experts Group (JPEG) or Portable Network Graphics (PNG) format.

In step S813, the UI control unit402waits for a user operation on the touch panel215of the data processing apparatus101.

In step S814, if a user operation is not performed for a certain time (TIMEOUT in step S814), the processing returns to step S806. If a received user operation is an operation for continuing the processing (CONTINUE PROCESSING in step S814), the processing proceeds to step S815. If a received user operation is an operation for cancelling the processing (CANCEL PROCESSING in step S814), the processing proceeds to step S822. Specifically, if the button1201is selected on the screen inFIG.12A, the processing proceeds to step S815. If the button1203is selected, the processing proceeds to step S822.

In step S815, the MFP control unit403transmits to the image forming apparatus104an operation request including a message that the processing of the job in the image forming apparatus104is to be continued. On the other hand, in step S822, the MFP control unit403transmits to the image forming apparatus104an operation request including that the processing of the job in the image forming apparatus104is to be cancelled (hereinafter, a “cancellation request”).

In step S816, the MFP control unit403receives an operation response as a response to the operation request in step S815or S822from the image forming apparatus104. The operation response includes the success or failure of the operation request and the current execution status of the automatic control in the image forming apparatus104.

In step S817, the MFP control unit403determines whether message information is included in the execution status acquisition response received in step S807or the operation response received in step S816. The image forming apparatus104can include in a response a message of which the user is be notified. If message information is included (YES in step S817), the processing proceeds to step S818. If message information is not included (NO in step S817), the processing proceeds to step S819.

In step S818, the UI control unit402displays the message information included in the response on the display214. Specifically, the UI control unit402displays the message1231inFIG.12D. For example, the UI control unit402may display a message displayed on the operation panel of the image forming apparatus104, such as a message regarding the currently executed job or a message notifying the user that a document is forgotten to be removed.

In step S819, based on the execution status acquisition response received in step S807, the UI control unit402displays information for the user to know the execution status of the job on the display214. Specifically, the UI control unit402displays the screen inFIG.12C. The screen is displayed based on screen information included in advance in the application401.

In step S820, if the execution status acquisition response received in step S807indicates that the execution of the job is completed (YES in step S820), the flowchart ends. If not (NO in step S820), the processing returns to step S806. In step S830, the UI control unit402displays on the display214a message indicating that the execution instruction to execute the job is an error, and the flowchart ends.

By the processing of the above flowchart, even in a case where an image forming apparatus is instructed to execute a job from a data processing apparatus, a screen that is displayed on the image forming apparatus and is to be presented to a user is also displayed on the data processing apparatus. Although a configuration is employed in which a cancellation request to cancel the execution of the job in the image forming apparatus104is received only at a predetermined timing in the present exemplary embodiment, a configuration may be employed in which a cancellation request is received at any timing. For example, when the execution status is displayed in step S819, a cancel button may be displayed together with the execution status. Then, if the UI control unit402detects the pressing of the cancel button, the MFP control unit403may transmit a cancellation request to the image forming apparatus104.

If it is determined in step S808that waiting for a user operation occurs, the UI control unit402may display the screen inFIG.12Bfor prompting the user to view the screen displayed on the operation unit307of the image forming apparatus104.

Although an example has been illustrated where the content is displayed in step S812, the timing of the display is not limited to this.

For example, the content may be displayed at the timing when it is determined in step S805that the content of the job execution response indicates success. Alternatively, the content may be displayed at the timing when the execution of the job is completed in step S820. Further, although an example has been illustrated where the web content is acquired using the URL and displayed in step S812, the web content itself may be acquired from the image forming apparatus104and displayed.

FIG.9is a flowchart illustrating an automatic control process performed by the image forming apparatus104according to the present exemplary embodiment. The flowchart inFIG.9is implemented by the CPU302of the image forming apparatus104loading programs for achieving the control modules stored in the ROM304or the HDD305into the RAM303and executing the programs. The flowchart inFIG.9corresponds to the processing performed by the image forming apparatus104in steps S701to S705in the sequence diagram inFIG.7.

In step S901, the automatic control processing unit504receives a job execution request transmitted from the data processing apparatus101via the network103. The job execution request is, for example, the information transmitted in step S803. In step S902, the authentication control unit503recognizes the authentication setting and the authentication state of the image forming apparatus104. In step S903, the UI control unit502acquires information corresponding to the authentication setting and the authentication state recognized in step S902as automatic control information from the automatic control management table505. For example, if the authentication setting is “authentication necessary/display login screen when operation on device is started” and the authentication state is “unauthenticated”, the UI control unit502reads the rows617to624. Hereinafter, steps will be described using as an example a case where the UI control unit502reads the rows617to624as the automatic control information.

In step S904, the automatic control processing unit504determines whether the automatic control can be started. Specifically, the automatic control processing unit504executes a check process defined in the first execution step (a row in which the “execution step” column603indicates “0”) in the automatic control read in step S903. If the condition is satisfied (YES in step S904), the processing proceeds to step S905. If not (NO in step S904), the processing proceeds to step S920. A specific example of the check process is described using the row617as an example. In the row617, the “process” column604defines “screen check”, and the “parameter” column605defines “login screen, job status screen”. Thus, in step S904, the automatic control processing unit504checks whether a screen currently displayed on the operation unit307is a login screen or a job status screen. If the screen currently displayed on the operation unit307is the login screen or the job status screen, the automatic control processing unit504determines that the condition is satisfied.

In step S905, as a response to the job execution request received in step S901, the automatic control processing unit504transmits to the data processing apparatus101a job execution response including information indicating that the job execution request is successful. In step S906, the automatic control proceeds to the next execution step. In step S907, the automatic control processing unit504determines whether the execution step as the processing target satisfies the execution condition in the “execution condition” column606. If the execution condition is satisfied (YES in step S907), the processing proceeds to step S908. If not (NO in step S907), the processing proceeds to step S915. In step S908, the automatic control processing unit504performs the process defined in the “process” column604in the execution step as the processing target, using the information in the “parameter” column605. Specific examples of the processes of steps S907and S908are described using the row618as an example. In the row618, the “execution condition” column606defines “currently displayed screen!=login screen”. Thus, if the screen currently displayed on the operation unit307of the image forming apparatus104is other than the login screen, the automatic control processing unit504determines in step S907that the execution condition is satisfied. Then, in step S908, based on “screen display” in the “process” column604and “login screen” in the “parameter” column605, the UI control unit502performs the process of displaying the login screen for inputting authentication information.

In the case of the row621, if the display timing of the notice screen in the image forming apparatus104is set such that the notice screen is displayed after login, it is determined in step S907that the execution condition is satisfied. Then, in step S908, the automatic control processing unit504requests the UI control unit502to display the notice screen (e.g.,FIG.13D), and the UI control unit502displays the notice screen on the operation unit307of the image forming apparatus104. Further, if the execution step as the processing target is the row625, the “process” column604defines “job execution”, and the “parameter” column605defines “job execution request: job settings”. In this case, the automatic control processing unit504requests the job control unit506to execute a job using job settings included in the job execution request received in step S901.

The description returns to the flowchart. In step S909, using the value of the “automatic transition” column607, the automatic control processing unit504determines whether the execution step as the processing target automatically transitions. If the “automatic transition” column607indicates “YES” (YES in step S909), the processing proceeds to step S915. If the “automatic transition” column607indicates “NO” (NO in step S909), the processing proceeds to step S910.

In step S910, the automatic control processing unit504confirms the notification method defined in the “notification method” column609in the execution step as the processing target. If the notification method is “VNC” (VNC in step S910), the processing proceeds to step S911. If the notification method is “URL” (URL in step S910), the processing proceeds to step S930.

In step S911, the automatic control processing unit504performs a preparation operation and a setting process for a VNC connection. For example, the automatic control processing unit504performs a preparation process for starting the image forming apparatus104as a VNC server, and a preparation process for including information necessary for a VNC connection in an execution status acquisition response to be transmitted to the data processing apparatus101.

In step S930, to include URL information in an execution status acquisition response to be transmitted to the data processing apparatus101, the automatic control processing unit504acquires a URL for accessing the currently displayed content.

In step S912, the automatic control processing unit504waits for a user operation on the operation unit307of the image forming apparatus104and an operation signal from the data processing apparatus101executing the processing asynchronously. If the automatic control processing unit504detects either of a user operation and an operation signal, the processing proceeds to step S913.

In step S913, the automatic control processing unit504determines the type of the event detected in step S912. If the event is a user operation on the operation unit307(USE OPERATION in step S913), the processing proceeds to step S914. If the event is a processing continuation signal (PROCESSING CONTINUATION SIGNAL in step S913), the processing proceeds to step S915. If the event is a cancellation signal (CANCELLATION SIGNAL in step S913), the flowchart ends. The processing continuation signal and the cancellation signal will be described below with reference toFIG.11.

In step S914, the automatic control processing unit504determines whether the user operation detected in step S912is the operation defined in the “user operation” column608in the execution step. If the user operation is the operation defined in the “user operation” column608in the execution step (YES in step S914), the processing proceeds to step S915. If not (NO in step S914), the flowchart ends. Then, the automatic control processing unit504suspends the automatic control.

In step S915, the automatic control processing unit504determines whether there is a next execution step. If there is a next execution step (YES in step S915), the processing proceeds to step S906. If not (NO in step S915), the flowchart ends.

In step S920, as a response to the job execution request received in step S901, the automatic control processing unit504transmits to the data processing apparatus101a job execution response including information indicating that the job execution request fails. Then, the flowchart ends.

Alternatively, a configuration may be employed in which the automatic control can be suspended at a timing other than steps S914and S913. For example, if screen transition or logout occurs in response to a user operation, the flowchart may end. A trigger for suspending the automatic control may be other than a user operation. For example, if it is detected that the image forming apparatus104becomes unable to perform network communication with the data processing apparatus101, the flowchart may end.

By the above processing, an image forming apparatus having received job information from a data processing apparatus can automatically execute steps, and in a predetermined step, the image forming apparatus can stop automatic control and wait for a user operation or an instruction from the data processing apparatus.

If the execution step does not automatically transition in step S909, then in step S912, the automatic control processing unit504waits for a user operation and a signal. If, however, the notification method is set, and the data processing apparatus101can display a corresponding screen, the process of step S912may be skipped, and the processing may automatically proceed to the next step. Consequently, if the data processing apparatus101can display the screen displayed on the image forming apparatus104, the image forming apparatus104can automatically perform screen transition. Thus, it is not necessary to stop the processing.

FIG.10is a flowchart illustrating a process executed in response to an execution status acquisition request by the image forming apparatus104according to the present exemplary embodiment. The flowchart inFIG.10is implemented by the CPU302of the image forming apparatus104loading programs for achieving the control modules stored in the ROM304or the HDD305into the RAM303and executing the programs. The flowchart inFIG.10corresponds to the processing performed by the image forming apparatus104in steps S710and S711in the sequence diagram inFIG.7. The processing of the flowchart inFIG.9and the processing of the flowchart inFIG.10are executed asynchronously.

In step S1001, the automatic control processing unit504receives an execution status acquisition request transmitted from the data processing apparatus101via the network103. The execution status acquisition request is, for example, the request transmitted from the data processing apparatus101in step S806.

In step S1002, the automatic control processing unit504acquires the execution status of a job. The “execution status” is, for example, the progress status of the execution steps in the automatic control in the automatic control management table505, information regarding the currently executed process, information indicating whether a user operation is waited for, and information indicating whether the automatic control is completed. Additionally, the execution status also includes information for the data processing apparatus101to display information corresponding to the execution status, specifically, the information in the “notification method” column609inFIGS.6A and6Bcorresponding to the current execution status. The information is, for example, information such as a URL for the data processing apparatus101to display the notice screen, information for the data processing apparatus101to display the operation assistance information displayed on the image forming apparatus104, using a VNC function, or message information. That is, the execution status of the job also includes the information necessary for a VNC connection set in step S911and the URL information acquired in step S930.

In step S1003, as a response to the execution status acquisition request received in step S1001, the automatic control processing unit504transmits the execution status of the automatic control acquired in step S1002as an execution status acquisition response to the data processing apparatus101.

By the above processing, information for the data processing apparatus101to display a screen or information displayed on the image forming apparatus104can be transmitted to the data processing apparatus101having given an execution instruction to execute a job. Only in a case where a predetermined screen for which the “automatic transition” column607inFIGS.6A and6Bindicates “NO” (the notice screen) is displayed, the image forming apparatus104transmits information for the data processing apparatus101to display the screen to the data processing apparatus101. That is, only in a case where a screen including content of which the user is be notified is displayed on the image forming apparatus104, the data processing apparatus101can display the screen, and the data processing apparatus101does not display an unnecessary screen. Thus, for example, if the authentication setting of the image forming apparatus104is “authentication unnecessary”, there is not a screen for which the “automatic transition” column607indicates “NO”. Thus, the process of displaying on the data processing apparatus101the screen currently displayed on the image forming apparatus104is not executed.

FIG.11is a flowchart illustrating a process executed in response to an operation request by the image forming apparatus104according to the present exemplary embodiment. The flowchart inFIG.11is implemented by the CPU302of the image forming apparatus104loading programs for achieving the control modules stored in the ROM304or the HDD305into the RAM303and executing the programs.

In step S1101, the automatic control processing unit504receives an operation request transmitted from the data processing apparatus101via the network103. The operation request is, for example, the request transmitted from the data processing apparatus101in step S815or S822. The processing of the flowchart inFIG.11and the processing of the flowchart inFIG.9are executed asynchronously.

In step S1102, the automatic control processing unit504determines the content of the operation request transmitted from the data processing apparatus101. If the content of the operation request is the continuation of the processing (CONTINUE PROCESSING in step S1102), the processing proceeds to step S1103. If the content of the operation request is the cancellation of the processing (CANCEL PROCESSING in step S1102), the processing proceeds to step S1104. If the content of the request is other than the continuation and the cancellation of the processing (OTHERWISE in step S1102), the processing proceeds to step S1110.

In step S1103, the automatic control processing unit504notifies an automatic control process that is being executed asynchronously (the flowchart inFIG.9) of a continuation signal as an operation signal. A “signal” is a method for notifying a process that is being executed asynchronously of information.

In step S1104, the automatic control processing unit504notifies the automatic control process that is being executed asynchronously (the flowchart inFIG.9) of a cancellation signal as an operation signal.

In step S1105, as a response to the operation request received in step S1101, the automatic control processing unit504transmits the current execution status of the automatic control process and information indicating that the operation request is successful, as an execution status acquisition response to the data processing apparatus101.

In step S1110, as a response to the operation request received in step S1101, the automatic control processing unit504transmits the current execution status of the automatic control process and information indicating that the operation request fails, as an execution status acquisition response to the data processing apparatus101.

In the above exemplary embodiment, an example has been illustrated where the notice screen inFIG.13Ddisplayed on the image forming apparatus104or the display screen of the operation assistance information inFIG.13Eis displayed on the data processing apparatus101. A screen to be displayed is not limited to this, and may be the home screen inFIG.13Aor the screen of the job in execution inFIG.13C. In this case, the image forming apparatus104transmits screen information to the data processing apparatus101.

By the processing of the above exemplary embodiment, even in a case where a user executes a job in an image forming apparatus by operating a data processing apparatus, a message to the user displayed on the image forming apparatus is displayed on the data processing apparatus. Thus, the user does not miss the message, and does not need to view both the screen of the data processing apparatus and the screen of the image forming apparatus.

According to the above exemplary embodiment, even in a case where an execution instruction to execute a job is given to an apparatus from a terminal, a user operating the terminal can confirm a screen displayed on the apparatus.

Other Embodiments

This application claims the benefit of Japanese Patent Application No. 2021-196730, filed Dec. 3, 2021, which is hereby incorporated by reference herein in its entirety.