Image forming system including image forming apparatus remotely operated by portable information device, and cooperation method

A portable information device includes a shooting portion to shoot a subject, and a notice portion to notify an image forming apparatus that an image is shot, in response to an operation screen of the image forming apparatus being shot. The image forming apparatus includes a display control portion to display an operation screen, a data decision portion to decide related data related to an operation screen displayed at a point of time when the notice is received, from among data held by the image forming apparatus at that point of time, and a related data transmission portion to transmit the related data to the portable information device. The portable information device further includes a related data reception portion to receive the related data from the image forming apparatus and a related data storage portion to store the related data.

This application is based on Japanese Patent Application No. 2012-176876 filed with Japan Patent Office on Aug. 9, 2012, the entire content of which is hereby incorporated by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image forming system and a cooperation method, and more particularly to an image forming system including an image forming apparatus remotely operated by a portable information device and a cooperation method performed in the image forming system.

2. Description of the Related Art

In recent years, the functions of Multi-Functional Peripherals (MFPs) have become increasingly complex and involve wide-ranging conditions for executing processes, which often makes the operation of setting the conditions complicated. Meanwhile, portable information devices in recent years have been sophisticated and increased in variety to meet personal needs. Portable information devices carried by individuals are used relatively more frequently than MFPs, so that users can learn the operation procedures thereof even though they are complicated. For this reason, techniques for remotely operating an MFP with a portable information devices have been developed.

Japanese Patent Laid-Open No. 2008-228036 discloses an image processing system including a mobile phone and an image processing apparatus, in which the mobile phone executes a function setting information input program to display an input screen for the image processing apparatus to perform function settings. When the user inputs function setting information, a code image generated by converting the input function setting information is displayed on a display unit. When the user holds the code image displayed on the mobile phone over a scan unit of the image processing apparatus, the image processing apparatus scans the code image, performs user authentication, and then extracts the function setting information from the code image for image processing in accordance with the function setting information.

In the conventional image processing system, however, a dedicated function setting information input program corresponding to the image processing apparatus has to be installed in the mobile phone. For example, when the functions of the image processing apparatus are expanded, or when an image processing apparatus having different functions is used, a function setting information input program adapted thereto has to be installed. Accordingly, the mobile phone is dependent on the functions of the image processing apparatus that it cooperates with.

SUMMARY OF THE INVENTION

In accordance with an aspect of the present invention, an image forming system is configured with an image forming apparatus and a portable information device capable of communicating with the image forming apparatus. The portable information device includes a shooting portion to shoot an image of a subject and a notice portion to give a notice that an image is shot to the image forming apparatus, in response to an image of an operation screen of the image forming apparatus being shot. The image forming apparatus includes a display control portion to display an operation screen, a data decision portion to, in response to the notice being received from the portable information device, decide related data that is related to the operation screen displayed at a point of time when the notice is received, from among data held by the image forming apparatus at that point of time, and a related data transmission portion to transmit the decided related data to the portable information device. The portable information device further includes a related data reception portion to receive the related data from the image forming apparatus in response to the notice and a related data storage portion to store the received related data.

In accordance with another aspect of the present invention, a cooperation method is executed in an image forming system configured with an image forming apparatus and a portable information device capable of communicating with the image forming apparatus. The portable information device includes a shooting portion to shoot an image of a subject. The method allows the portable information device to execute a notice step of giving a notice that an image is shot to the image forming apparatus, in response to an image of an operation screen of the image forming apparatus being shot by the shooting portion. The method allows the image forming apparatus to execute: a display control step of displaying an operation screen; a data decision step of, in response to the notice being received from the portable information device, deciding related data that is related to the operation screen displayed at a point of time when the notice is received, from among data held by the image forming apparatus at that point of time; and a related data transmission step of transmitting the decided related data to the portable information device. The method further allows the portable information device to execute: a related data reception step of receiving the related data from the image forming apparatus in response to the notice; and a related data storage step of storing the received related data.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Embodiments of the present invention will be described below in conjunction with the drawings. In the following description, the same or corresponding parts are denoted by the same reference characters. Their names and functions are also the same. Thus, a detailed description thereof will not be repeated.

FIG. 1is a diagram showing an overview of an image forming system in an embodiment of the present invention. Referring toFIG. 1, an image forming system1includes MFPs (Multi-Functional Peripherals)100,101,102as image forming apparatuses, a personal computer (hereinafter referred to as “PC”)105, a portable information device200, and a radio station300, each connected to a network2.

PC105is a general computer and is installed with a printer driver program for controlling MFPs100,101,102. In a case where MFPs100,101,102are of different kinds, three kinds of printer driver programs for controlling MFPs100,101,102are installed in PC105.

MFPs100,101,102each have a document scanning function for scanning a document, an image forming function of forming an image on a recording medium such as paper based on image data, and a facsimile transmitting/receiving function of transmitting/receiving facsimile data. Although MFPs100,101,102are illustrated in the present embodiment, MFPs100,101,102may be replaced by any other devices having the function of forming an image, such as printers and facsimile machines. MFPs100,101,102have the same functions and, therefore, MFP100will be described as an example below unless otherwise specified.

Portable information device200is a general mobile phone and can make a call by wirelessly communicating with a mobile phone base station to connect to a mobile phone network. Portable information device200has a wireless LAN function.

Network2is a Local Area Network (LAN), either wired or wireless. Network2is not limited to a LAN and may be a network such as a Public Switched Telephone Network. Network2is further connected to a Wide Area Network (WAN) such as the Internet.

Radio station300is a relay apparatus of network2that communicates with portable information device200having a wireless LAN communication function to connect portable information device200to network2. MFPs100,101,102are each able to transmit/receive data to/from PC105through network2. MFPs100,101,102each can transmit/receive data with portable information device200through network2and radio station300.

FIG. 2is an external perspective view of an MFP.FIG. 3is a block diagram showing an overall hardware configuration of the MFP. Referring toFIGS. 2 and 3, MFP100includes a main circuit110, a document scanning unit130for scanning a document, an automatic document feeder120for conveying a document to document scanning unit130, an image forming unit140for forming an image on a sheet of paper based on image data output by document scanning unit130scanning a document, a paper feed unit150for supplying paper to image forming unit140, and an operation panel160serving as a user interface.

Main circuit110includes a CPU111, a communication interface (I/F) unit112, a ROM113, a RAM114, an HDD (Hard Disk Drive)115serving as a mass storage device, a facsimile unit116, and an external storage device117to which a CD-ROM118is attached. CPU111is connected to automatic document feeder120, document scanning unit130, image forming unit140, paper feed unit150, and operation panel160to control the entire MFP100.

ROM113stores a program executed by CPU111or data necessary to execute the program. RAM114is used as a working area for CPU111to execute a program. RAM114temporarily stores scan data (image data) successively sent from document scanning unit130.

Operation panel160is provided on the upper surface of MFP100and includes a display unit161and an operation unit163. Display unit161is a display such as an LCD (Liquid Crystal Display) or an organic ELD (Electroluminescence Display) for displaying instruction menus to users, information about the acquired image data, and the like. Operation unit163includes a plurality of keys for accepting input of a variety of instructions and data such as characters and numerals through user's operation corresponding to the keys. Operation unit163further includes a touch panel provided on display unit161.

Communication I/F unit112is an interface for connecting MFP100to network2. CPU111communicates with MFP101,102, PC105, or portable information device200through communication I/F112to transmit/receive data. Communication I/F unit112can also communicate with a computer connected to the Internet via network2.

Facsimile unit116is connected to a PSTN (Public Switched Telephone Network) to transmit facsimile data to the PSTN or receive facsimile data from the PSTN. Facsimile unit116stores the received facsimile data into HDD115or outputs the received facsimile data to image forming unit140. Image forming unit140prints the facsimile data received by facsimile unit116on a sheet of paper. Facsimile unit116also converts data stored in HDD115into facsimile data and transmits the facsimile data to a facsimile machine connected to the PSTN.

CD-ROM118is attached to external storage device117. CPU111can access CD-ROM118through external storage device117. CPU111loads the program stored in CD-ROM118attached to external storage device17into RAM114for execution. A medium encoded with a program executed by CPU111is not limited to CD-ROM118but may be an optical disk (MO (Magnetic Optical Disc)/MD (Mini Disc)/DVD (Digital Versatile Disc)), an IC card, an optical card, or a semiconductor memory such as a mask ROM, an EPROM (Erasable Programmable ROM), or an EEPROM (Electrically EPROM).

The program executed by CPU111is not limited to a program stored in CD-ROM118, and a program stored in HDD115may be loaded into RAM114for execution. In this case, another computer connected to network3may overwrite the program stored in HDD115of MFP100or additionally write a new program. MFP100may download a program from another computer connected to network2and store the program into HDD115. The program referred to here includes not only a program directly executable by CPU111but also a source program, a compressed program, an encrypted program, and the like.

In image forming system1in the present embodiment, data is transmitted/received between MFPs100,101,102and portable information device200. Any protocol for transmitting/receiving data can be used as long as the receiver device can specify the sender. The protocol for transmitting/receiving data is, for example, HTTP (Hyper Text Transfer Protocol), FTP (File Transfer Protocol), SMPT (Simple Mail Transfer Protocol), or POP (Post Office Protocol).

FIG. 4is a functional block diagram showing an overall hardware configuration of the portable information device. Referring toFIG. 4, portable information device200includes a CPU201for controlling the entire portable information device200, a camera202, a flash memory203storing data in a nonvolatile manner, a radio communication unit204connected to a call unit205, a display unit206displaying information, an operation unit207accepting user's operation input, and a wireless LAN I/F208.

Wireless communication unit204wirelessly communicates with a mobile phone base station connected to a telephone communication network. Wireless communication unit204connects portable information device200to the telephone communication network to establish calls using call unit205. Wireless communication unit204decodes a voice signal obtained by demodulating a radio signal received from the mobile phone base station and outputs the decoded signal to call unit205. Wireless communication unit204encodes voice input from call unit205and transmits the encoded voice to the mobile phone base station. Call unit205includes a microphone and a speaker to output voice input from wireless communication unit204from the speaker and output voice input from the microphone to wireless communication unit204. Wireless communication unit204is controlled by CPU201, and connects portable information device200to an email server to transmit/receive emails.

Camera202includes a lens and an optoelectronic transducer. Light collected by the lens is imaged at the optoelectronic transducer, which transduces the received light into image data for output to CPU201. The optoelectronic transducer is, for example, a CMOS (Complementary Metal Oxide Semiconductor) sensor or a CCD (Charge Coupled Device) sensor.

Display unit206is a display such as a Liquid Crystal Display (LCD) or an organic ELD and displays instruction menus to the user and information related to the acquired image data. Operation unit207includes a plurality of keys and accepts input of various instructions and data such as characters and numerals through user's operation corresponding to the keys.

Wireless LAN I/F208is an interface that communicates with radio station300for connecting portable information device200to network2. The respective IP addresses of MFPs100,101,102are registered in advance in portable information device200so that portable information device200can communicate with MFPs100,101,102and transmit/receive data. Although portable information device200communicates with MFPs100,101,102using wireless LAN I/F208in the present embodiment, by way of example, any other communication method can be used for communication. Specifically, in a case where portable information device200and MFPs100,101,102are each equipped with a short distance radio device such as Bluetooth®, portable information device200may communicate one-to-one with any one of MFPs100,101,102. Portable information device200may be connected by wire such as a USB (Universal Serial Bus) cable to any one of MFPs100,101,102so that portable information device200communicates one-to-one with any one of MFPs100,101,102.

Flash memory203stores a program executed by CPU201or data necessary to execute the program. CPU201loads the program stored in flash memory203into the RAM of CPU201for execution. The program executed by CPU201may be a program stored in flash memory203, or another computer connected to network2may overwrite the program stored in flash memory203or additionally write a new program. Portable information device200may download a program from another computer connected to network2. The program referred to here includes not only a program directly executable by CPU201but also a source program, a compressed program, an encrypted program, and the like.

A medium encoded with a program executed by CPU201is not limited to flash memory203but may be an optical disk (MO (Magnetic Optical Disc)/MD (Mini Disc)/DVD (Digital Versatile Disc)), an IC card, an optical card, or a semiconductor memory such as a mask ROM, an EPROM (Erasable Programmable ROM, or an EEPROM (Electrically EEPROM).

FIG. 5is a block diagram showing an overview of functions of the CPU of the MFP in the present embodiment. The functions shown inFIG. 5are formed in CPU111by CPU111of MFP100executing a remote control program stored in ROM113, HDD115or CD-ROM118. Referring toFIG. 5, CPU111includes a display control portion51for controlling display unit161, an authentication portion55authenticating a user, a notice reception portion57receiving a shooting notice from portable information device200, a data decision portion53deciding related data, a related data transmission portion59transmitting related data to portable information device200, a remote control portion61executing a process in accordance with a remote operation command received from the portable information device, and a process control portion63.

Process control portion63executes a process in accordance with a setting value. The process executed by process control portion63includes an image forming process of controlling image forming unit140for forming an image, a document scanning process of controlling image scanning unit130for scanning a document image, a data transmitting/receiving process of controlling communication I/F unit112for transmitting/receiving data, a facsimile transmitting/receiving process of controlling facsimile unit116for transmitting/receiving facsimile data, and a data management process of controlling HDD115or external storage device117for writing or reading data.

In the case where an image forming process is executed, process control portion63sets data to be subjected to image forming as a process target. In the case where a document scanning process is executed, process control portion63sets data output by document scanning unit130scanning a document as a process target. In the case where a data transmitting/receiving process is executed, process control portion63sets data transmitted/received by communication I/F unit112as a process target. In the case where a facsimile transmitting/receiving process is executed, process control portion63sets facsimile data transmitted/received by facsimile unit116as a process target. In the case where a data management process is executed, process control portion63sets data written into HDD115or external storage device117, or data read from HDD115or external storage device117, as a process target.

Display control portion51displays one of a plurality of operation screens stored in HDD115on display unit161. A plurality of operation screens are associated with each other, and one operation screen has a transition button for switching to another operation screen. When operation unit163detects operation of designating the transition button, display control portion51reads out the operation screen associated with the transition button from EEPROM115and displays the read operation screen on display unit161.

Display control portion51includes a setting screen display portion71, a job history display portion73, a preview screen display portion75, and a BOX screen display portion77. The plurality of operation screens include different kinds of operation screens. Any one of setting screen display portion71, job history display portion73, preview screen display portion75, and BOX screen display portion77displays an operation screen on display unit161.

Setting screen display portion71displays a setting-type operation screen among different kinds of operation screens. The setting-type operation screen is an operation screen for setting a setting value for process control portion63to execute a process and includes an area for displaying a setting value that is set. Of the operation screens, the setting-type screen is called a setting screen.

When setting screen display portion71displays a setting screen on display unit161, the user can enter a setting value into operation unit163. With the setting screen appearing on display unit161, process control portion63accepts a setting value input to operation unit163and sets the accepted setting value. For example, a setting value can be set by storing a setting value into RAM114. When a start key of operation unit163is designated, process control portion63executes a process in accordance with the set setting value.

Job history display portion73displays a history-type operation screen among different kinds of operation screens. The history-type operation screen is an operation screen for displaying history information and includes an area for displaying history information stored by process control portion63. Of the operation screens, the history-type operation screen is called a job history screen. Process control portion63executes a process and stores history information showing the result of executing the process into HDD115, so that job history display portion73reads out the history information stored in HDD115and displays a job history screen on display control portion161.

History information includes job identification information for identifying a job which is a unit of a process executed by process control portion63, a date and time when the job is executed, process identification information for identifying a process, and a setting value set when the process is executed. For example, if the job is a facsimile transmitting process of transmitting facsimile, the history information includes job identification information assigned to the facsimile transmitting process, the date and time when the facsimile transmitting process is executed, process identification information “Facsimile Transmission” for identifying the facsimile transmitting process, the facsimile number of the destination to which facsimile data is transmitted, as a setting value set for the facsimile transmitting process, and data identification information for identifying facsimile data.

Preview screen display portion75displays a preview-type operation screen among different kinds of operation screens. The preview-type operation screen is an operation screen for displaying an image of data set as a process target by process control portion63and includes an area for displaying an image of data set as a process target by process control portion63. Of the operation screens, the preview-type operation screen is called a preview screen. Data corresponding to an image included in the preview screen includes data output by document scanning unit130scanning a document and data read out from HDD115or external storage device117, among data set as process targets by process control portion63.

BOX screen display portion77displays a BOX-type operation screen among different kinds of operation screens. The BOX-type operation screen is an operation screen for accepting designation of data set as a process target by process control portion63and includes an area for displaying a list of data stored in HDD115. Of the operation screens, the BOX-type operation screen is called a BOX screen. The area for displaying a list of data includes data identification information for identifying data.

Authentication portion55transmits a login screen to portable information device200when communication I/F unit112establishes connection with portable information device200and receives a login request from portable information device200. The login screen includes an area for inputting user identification information for identifying a user and an area for inputting authentication information. Authentication portion55receives the user identification information and authentication information from portable information device200that the login screen is transmitted to, and authenticates the user who operates portable information device200. MFP100has user data including user identification information for identifying users and authentication information stored in HDD115. Authentication portion55compares the user identification information and authentication information received from portable information device200with the user data stored in HDD115. If they agree, authentication portion55determines that authentication is successful, and permits login. If they do not agree, authentication portion55determines that authentication is failed, and does not permit login. Authentication portion55returns a permission signal if login is permitted, and returns a non-permission signal if login is not permitted.

Authentication portion55acquires device identification information of portable information device200when connection is established with portable information device200. The identification information of portable information device200is, for example, location information on network2allocated to portable information device200. The location information on network2is, for example, an IP (Internet Protocol) address or a MAC (Media Access Control) address. If authentication is successful, authentication portion55outputs a set of the user identification information for identifying the user who operates the authenticated portable information device200and the device identification information of portable information device200to notice reception portion57, related data transmission portion59, and remote control portion61.

In the present embodiment, a password is used as authentication information, by way of example. Biometric information such as fingerprints, vein patterns, and irises of users may be used for authentication. In this case, portable information device200includes a reader for reading biometric information.

Notice reception portion57receives a set of the user identification information of the user and the device identification information of portable information device200from authentication portion55and thereafter receives a shooting notice transmitted from portable information device200specified by the device identification information. When communication I/F unit112receives a shooting notice, notice reception portion57acquires the device identification information of the device that has transmitted the shooting notice and compares the acquired device identification information with the device identification information input from authentication portion55. If the device identification information of the device that has transmitted the shooting notice is the same as the device identification information input from authentication portion55, notice reception portion57acquires the shooting notice received by communication I/F unit112. If they are different, notice reception portion57discards the shooting notice received by communication I/F unit112. Accordingly, a shooting notice can be received only when the user authenticated by authentication portion55operates portable information device200.

The shooting notice is a signal transmitted by portable information terminal200described later to MFP100when camera202shoots an image of an operation screen appearing on display unit161of MFP100. Portable information device200is operated by a user registered beforehand in MFP100so that only the shooting notice transmitted from portable information device200can be received. Notice reception portion57receiving the shooting notice then outputs a set of a notice reception signal indicating that the shooting notice is received and the device identification information of portable information device200to data decision portion53and related data transmission portion59.

At the point of time when the notice reception signal is input from notice reception portion57, data decision portion53decides related data that is related to the operation screen appearing on display unit161among data held at that point of time. Data decision portion53outputs the decided related data to related data transmission portion59. Data decision portion53includes a setting value decision portion81, a history specifying portion83, a preview data acquisition portion85, a page number specifying portion87, and a selected data acquisition portion89. Setting value decision portion81, history specifying portion83, preview data acquisition portion85, page number specifying portion87, and selected data acquisition portion89each decides related data.

Setting value decision portion81decides a setting value set in the setting screen as related data if a setting screen is displayed on display unit161by setting screen display portion71at the point of time when the notice reception signal is input from notice reception portion57. If a setting screen is displayed on display unit161by setting screen display portion71, a setting value is accepted by process control portion63and stored into RAM114. Setting value decision portion81acquires the setting value by reading out the setting value from RAM114. The setting value set in the setting screen is not limited to a setting value displayed on the setting screen but includes all the setting values for executing the process corresponding to the setting screen. For example, in a case where the setting screen includes an area for displaying the number of sheets for image formation as a setting value for executing an image forming process, all the setting values set as setting values for executing the image forming process are included. The setting values for executing an image forming process include, for example, the size of sheet, the enlargement/reduction ratio, whether to sort, and the like.

History specifying portion83specifies history information included in a job history screen and decides the specified history information as related data if a job history screen is displayed on display unit161by job history display portion73at the point of time when the notice reception signal is input from notice reception portion57. If the job history screen includes a plurality of history information, one history information selected from among a plurality of history information is specified through user's operation input to operation unit163.

Preview data acquisition portion85acquires data corresponding to an image included in a preview screen and decides the acquired data as related data if a preview screen is displayed on display unit161by preview screen display portion75at the point of time when the notice reception signal is input from notice reception portion57. The data corresponding to the image included in the preview screen includes data output by document scanning unit130reading a document and data read out from HDD115or external storage device117, of data set as process targets by process control portion63. The acquired data thus may be comprised of a plurality of pages.

Page number specifying portion87specifies a page corresponding to an image included in a preview screen and decides page identification information for identifying the specified page as related data if the data specified by preview data acquisition portion85is comprised of a plurality of pages.

Selected data acquisition portion89decides data selected as a process target by the user on a BOX screen as related data if a BOX screen is displayed on display unit161by BOX screen display portion77at the point of time when the notice reception signal is input from notice reception portion57. With the BOX screen displayed on display unit161by BOX screen display portion77, the user can operate operation unit163to select one of data identification information included in the area for displaying a list of data in the BOX screen. Selected data acquisition portion89accepts the user's operation input to operation unit163, specifies the one selected by the user from among data identification information included in the area for displaying a list of data in the BOX screen, and acquires the specified data from HDD115or external storage device117.

Related data transmission portion59receives a set of the user identification information and the device identification information from authentication portion55, receives a set of the notice reception signal and the device identification information from notice reception portion57, and receives the related data from data decision portion53. If a set of the notice reception signal and the same device identification information as the device identification information input from authentication portion55is input from notice reception portion57, related data transmission portion59transmits the related data input from data decision portion53to portable information device200identified by the device identification information through communication I/F unit112.

After a set of the user identification information and the device identification information is input from authentication portion55, remote control portion61receives a remote operation command transmitted from portable information device200specified by the device identification information. When communication I/F unit112receives a remote operation command, remote control portion61acquires the device identification information of the device that has transmitted the remote operation command, and compares the acquired device identification information with the device identification information input from authentication portion55. If the device identification information of the device that has transmitted the remote operation command is the same as the device identification information input from authentication portion55, remote control portion61acquires the remote operation command received by communication I/F unit112. If they are different, remote control portion61discards the remote operation command received by communication I/F unit112. Accordingly, the remote operation command can be received only when the user authenticated by authentication portion55operates portable information device200.

When a remote operation command is received, remote control portion61allows process control portion63to execute a process in accordance with the remote operation command. The remote operation command includes related data as described later. The remote operation command includes a remote operation command to set a setting value and a remote operation command to set a process target. The related data included in the remote operation command to set a setting value is a setting value, and the related data included in the remote operation command to set a process target is data different from a setting value.

When a remote operation command to set a setting value is received, remote control portion61sets a setting value that is related data included in the remote operation command. Specifically, a setting value is stored into RAM114. When a remote operation command to set a process target is received, remote control portion61sets data included in related data included in the remote operation command as a process target. Specifically, data included in the related data is stored into RAM114.

FIG. 6is a block diagram showing an overview of functions of the CPU of the portable information device in the present embodiment. The functions shown inFIG. 6are formed in CPU201by CPU201of portable information device200executing a remote operation program stored in flash memory203. Referring toFIG. 6, CPU201of portable information device200includes a device selection portion251selecting a device serving as a target for remote operation, a login portion253logging in to the device serving as a target for remote operation, a shooting control portion263controlling camera202, a notice portion255for giving a notice that an image is shot by camera202to the device serving as a target for remote operation, an image storage portion265storing an image shot by camera202, a related data reception portion257receiving related data from the device serving as a target for remote operation, a page image display control portion259displaying the related data, an association portion267associating the related data with the shot image, an image display control portion269displaying the shot image stored by image storage portion265, an image selection portion271selecting the shot image, and a remote operation portion273remotely operating the device serving as a target for remote operation.

Device selection portion251selects a device selected by the user from among MFPs100,101,102as a device serving as a target for remote operation. MFPs100,101,102are registered beforehand in device selection portion251as devices that can be a target for remote operation. Device selection portion251displays a select screen including the respective device identification information of MFPs100,101,102on display unit206. The user inputs operation of selecting the device identification information displayed on the select screen into operation unit207, so that device selection portion251selects the device having the device identification information selected from among MFPs100,101,102as a device serving as a target for remote operation. Device selection portion251outputs the device identification information of the device selected as a device serving as a target for remote operation to login portion253.

Login portion253logs in to the device serving as a target for remote operation in response to input of the device identification information of the device serving as a target for remote operation from device selection portion251. Here, it is assumed that MFP100is selected by the user as a device serving as a target for remote operation, by way of example. Specifically, a login request is transmitted to MFP100through wireless LAN I/F208. MFP100receiving the login request returns a login screen, so that login portion253displays the login screen received from MFP100on display unit206and accepts user identification information and authentication information input to operation unit207by the user in accordance with the login screen. Login portion253transmits a set of the accepted user identification information and authentication information to MFP100through wireless LAN I/F208. MFP100then returns a permission signal if the login is permitted or returns a non-permission signal if the login is not permitted, as described above. If wireless LAN I/F208receives a permission signal from MFP100, login portion253outputs the device identification information of MFP100permitted to log in to notice portion255and remote operation portion273.

Shooting control portion263controls camera202and allows camera202to shoot an image when the user presses a shutter button included in operation unit207. Shooting control portion263then accepts the shot image output by camera202shooting an image of a subject. The subject of camera202is the operation screen displayed on display unit161of MFP100serving as a target for remote operation. Shooting control portion263outputs the accepted shot image to image storage portion265and outputs a notice instruction to allow notice portion255to give a notice that an image is shot.

Image storage portion265stores the shot image into flash memory203and outputs image identification information for identifying the shot image to association portion267.

After the device identification information of the device serving as a target for remote operation is input from login portion253, notice portion255transmits a shooting notice indicating that an image is shot to the device serving as a target for remote operation that is specified by the device identification information, in response to input of the notice instruction from shooting control portion263, and outputs the device identification information of the device serving as a target for remote operation that the shooting notice is transmitted to, to related data reception portion257. In other words, notice portion255notifies MFP100that the user operating portable information device200shoots an image of an operation screen, as a subject, that is displayed on display unit161of MFP100, that is, the device serving as a target for remote operation.

Related data reception portion257receives the device identification information from notice portion255. As described above, when MFP100is a device serving as a target for remote operation, MFP100returns related data in response to the shooting notice being received from portable information device200, so that related data reception portion257acquires the related data received by wireless LAN I/F208from MFP100. When the device identification information of the device that has transmitted the related data received by wireless LAN I/F208is the same as the device identification information input from notice portion255, related data reception portion257receives the related data. Accordingly, related data can be received only from the device that the shooting notice is transmitted to. Related data reception portion257outputs a set of the received related data and the device identification information to related data storage portion261and to page image display control portion259.

Page image display control portion259receives related data from related data reception portion257and determines whether the related data includes page identification information. If the related data includes page identification information, page image display control portion259controls display unit206to display an image of the page specified by the page identification information, of data of a plurality of pages included in the related data.

An example in which the device serving as a target for remote operation is MFP100will be specifically described. In the case where page image display control portion259displays an image of a page of related data on display unit206, the user operates portable information device200to log in to MFP100with a preview screen appearing on display unit161of MFP100and thereafter presses the shutter button of operation unit207to shoot the preview screen appearing on display unit161as a subject. In this case, MFP100transmits data corresponding to the image included in the preview screen and page identification information of the page of the image as related data to portable information device200. Page image display control portion259displays the image of the page specified by the page identification information, of data of a plurality of pages included in the related data received from MFP100, so that the image displayed on display unit206is the same as the image included in the preview screen displayed on display unit206of MFP100. The user therefore can confirm that the data corresponding to the image included in the preview screen is transferred from MFP100to portable information device200, based on that the same image is displayed on the source and the destination of data transmitted.

Related data storage portion261stores a set of related data and device identification information into flash memory203and outputs the data identification information for identifying the related data to association portion267.

Association portion267associates related data with a shot image. Association portion267receives the data identification information of related data from related data storage portion261and receives the image identification information of a shot image from image storage portion265. Association portion267generates an association record including the data identification information and the image identification information and stores the generated association record into flash memory203.

Image display control portion269displays the shot image stored in flash memory203by image storage portion265on display unit206. If a plurality of shot images are stored in flash memory203by image storage portion265, a plurality of shot images are displayed. A plurality of shot images may be displayed in one screen, or screens each including one or more shot images may be successively displayed. The shot images may be reduced in size for display.

Image selection portion271selects a shot image displayed on display unit206. One is selected from among one or more shot images displayed on display unit206in accordance with user's operation input to operation unit207. Image selection portion271outputs the image identification information of the selected shot image to remote operation portion273.

Remote operation portion273receives the device identification information of the device serving as a target for remote operation from login portion253and receives the image identification information from image selection portion271. In response to input of the image identification information from image selection portion271, remote operation portion273acquires related data associated with the image identification information by association portion267. Since association portion267stores the association records in flash memory203, the association record that includes the image identification information input from image selection portion271is extracted from the association records stored in flash memory203, and the data identification information included in the extracted association record is acquired. The related data specified by the acquired data identification information is then acquired from the related data stored in flash memory203. Remote operation portion273generates a remote operation command including the related data and transmits the generated remote operation command to the device specified by the device identification information input from login portion253through wireless LAN I/F208.

Remote operation portion273generates a remote operation command to allow the device serving as a target for remote operation to set a setting value included in the related data if the related data includes a setting value for MFPs100,101,102to execute a process. Remote operation portion273generates a remote operation command to allow the device serving as a target for remote operation to set the related data as a process target if the related data does not include a setting value. The operation of MFPs100,101,102receiving the remote operation command has been described above and therefore is not repeated here.

The device serving as a target for remote operation at the stage when notice portion255transmits a shooting notice may be the same as or different from the device serving as a target for remote operation at the stage when remote operation portion273transmits a remote operation command.

FIG. 7is a flowchart showing an exemplary flow of a remote control process. The remote control process is a process executed by CPU111of MFP100executing a remote control program stored in ROM113, HDD115, or CD-ROM118.

Referring toFIG. 7, CPU111determines whether a login request is received (step S21). If communication I/F unit112establishes connection with portable information device200and receives a login request from portable information device200, the process proceeds to step S22. If a login request is not received, the process proceeds to step S24.

In step S22, it is determined whether authentication is successful. If authentication is successful, the process proceeds to step S23. If authentication is failed, the process proceeds to step S24. Specifically, a login screen is transmitted to portable information device200through communication I/F unit112, and the user identification information and authentication information transmitted by portable information device200is received. The received user identification information and authentication information is then compared with the user data stored in HDD115. If they agree, it is determined that authentication is successful. If they do not agree, it is determined that authentication is failed. In step S23, the login by the user of portable information device200is permitted. The process then proceeds to step S24.

In step S24, it is determined whether operation is accepted. Operation is accepted when the user inputs operation to operation unit163. If authentication of the user of portable information device200is successful and login is permitted in step S22, when communication I/F unit112receives a remote operation command from portable information device200, the operation determined by the received remote operation command is accepted. If operation is accepted, the process proceeds to step S25. If not, the process proceeds to step S28.

In step S25, it is determined whether the accepted operation is an execution instruction to give an instruction to execute a process. If it is an execution instruction, the process proceeds to step S26. If not, the process proceeds to step S27. In step S26, the process is executed in accordance with a setting value. The process then proceeds to step S28. The process executed in step S26includes, for example, an image forming process of controlling image forming unit140for forming an image, a document scanning process of controlling document scanning unit130for scanning an image of a document, a data transmitting/receiving process of controlling commination I/F unit112for transmitting/receiving data, a facsimile transmitting/receiving process of controlling facsimile unit116for transmitting/receiving facsimile data, and a data management process of controlling HDD115or external storage device117for writing or reading data.

On the other hand, the process proceeds to step S27when the accepted operation is operation of setting a setting value. In step S27, a setting value is set, and the process proceeds to step S28. For example, input of a setting value specified by the operation is accepted, and the setting value set so far is updated with the input setting value. When the operation accepted in step S24is operation of switching the operation screen displayed on display unit161, the operation screen specified by the operation is displayed on display unit161.

In step S28, it is determined whether a shooting notice is received. If a shooting notice is received, the process proceeds to step S29. If not, the process proceeds to step S32. In step S29, it is determined whether the user who operates the device that has transmitted the shooting notice is logging in. If the user is logging in, the process proceeds to step S30. If not, the process proceeds to step S32. It is determined whether the device that has transmitted the shooting notice is the same as portable information device200operated by the user permitted to log in in step S23. More specifically, if the device identification information of the device that has transmitted the login request received in step S21is the same as the device identification information of the device that has transmitted the shooting notice received in step S28, it is determined that the user is logging in.

In step S30, a data decision process is executed, and the process proceeds to step S31. The data decision process, which will be detailed later, is a process of selecting and deciding related data that is related to the operation screen displayed on display unit161from among data held at that point of time. In step S31, the related data is transmitted to portable information device200that has transmitted the shooting notice. The process then proceeds to step S39.

In step S32, it is determined whether the related data is received. If the related data is received, the process proceeds to step S33. If not, the process proceeds to step S39. In step S33, it is determined whether the user who operates the device that has transmitted the related data is logging in. If the user is logging in, the process proceeds to step S34. If not, the process proceeds to step S39. It is determined whether the device that has transmitted the related data is the same as portable information device200operated by the user permitted to log in in step S23. More specifically, if the device identification information of the device that has transmitted the login request received in step S21is the same as the device identification information of the device that has transmitted the related data received in step S32, it is determined that the user is logging in.

In step S34, it is determined whether the related data includes a setting value. If the related data includes a setting value, the process proceeds to step S35. If not, the process proceeds to step S37. In step S35, the setting value included in the related data is set. The process then proceeds to step S36. The setting value set so far is updated with the setting value included in the related data. In the next step S36, a setting screen is displayed on display unit161. The process then proceeds to step S39. Here, the setting screen displayed on display unit161is a setting screen that includes the item for which the setting value is set in step S35.

On the other hand, the process proceeds to step S37when the related data does not include a setting value but includes data. In step S37, the data included in the related data is set as a process target. The process then proceeds to step S38. Here, the data included in the related data is stored into HDD115, and the stored data is set as a process target. In step S38, a BOX management screen is displayed. The process then proceeds to step S39. The BOX management screen displayed here is in a state in which the data stored in HDD115in step S37is selected.

In step S39, it is determined whether a logout request is received. If a logout request is received, the process proceeds to step S40. If not, the process returns to step S21. In step S40, it is determined whether the user who operates the device that has transmitted the logout request is logging in. If the user is logging in, the process proceeds to step S41. If not, the process returns to step S21. It is determined whether the device that has transmitted the logout request is the same as portable information device200operated by the user permitted to log in in step S23. More specifically, if the device identification information of the device that has transmitted the login request received in step S21is the same as the device identification information of the device that has transmitted the logout request received in step S39, it is determined that the user is logging in.

In step S41, the user permitted to log in in step S23is logged out. The process then returns to step S21.

FIG. 8is a flowchart showing an exemplary flow of the data decision process. The data decision process is a process executed in step S30inFIG. 7. Referring toFIG. 8, the kind of the operation screen displayed on display unit161is determined, and the process branches depending on the kind of the operation screen (step S51). If the operation screen is a setting screen, the process proceeds to step S52. If the operation screen is a preview screen, the process proceeds to step S53. If the operation screen is a job history screen, the process proceeds to step S56. If the operation screen is a BOX screen, the process proceeds to step S57.

In step S52, the setting value set in the setting screen displayed on display unit161is set as the related data. The process then returns to the remote control process. The setting value set in the setting screen includes a setting value for use in an image forming process, a setting value for use in a document scanning process, a setting value for use in a data transmitting/receiving process, a setting value for use in a facsimile transmitting/receiving process, and a setting value for use in a data management process. The setting value set in the setting screen is not limited to the setting value set in the setting screen but includes all the setting values for executing a process corresponding to the setting screen. For example, in a case where the setting screen includes an area for displaying a number of sheets for image formation as a setting value for executing an image forming process, all the setting values set as setting values for executing the image forming process are included. The setting values for executing an image forming process include, for example, the size of paper, the enlargement/reduction ratio, whether to sort, and the like.

In step S53, data as a preview target is set as the related data. Specifically, data corresponding to the image included in the preview screen displayed on display unit161is acquired, and the acquired data is set as the related data. The data corresponding to the image included in the preview screen includes, for example, data output by document scanning unit130scanning a document and data read out from HDD115or external storage device117. The acquired data therefore may be comprised of a plurality of pages.

In the next step S54, the page of the image displayed in the preview screen is specified. If the data specified in step S53is comprised of a plurality of pages, the page corresponding to the image included in the preview screen is specified. The page identification information for identifying the specified page is then set as the related data (step S55). The process then returns to the remote control process.

In step S56, the history information selected in the job history screen is set as the related data. The process then returns to the remote control process. Specifically, the history information included in the job history screen displayed on display unit161is specified, and the specified history information is decided as the related data. If the job history screen includes a plurality of history information, one history information selected from among a plurality of history information is specified through user's operation input to operation unit163.

In step S57, the data selected in the BOX screen is set as the related data. The process then returns to the remote control process. Specifically, with the BOX screen appearing on display unit161, the user can operate operation unit163to select one of data identification information included in the area for displaying a list of data in the BOX screen. The data selected as a process target by the user on the BOX screen displayed on display unit161is set as related data.

FIG. 9is a flowchart showing an exemplary flow of a remote operation process. The remote operation process is a process executed by CPU201of portable information device200executing a remote operation program stored in flash memory203. Referring toFIG. 9, CPU201selects one of MFPs100,101,102(step S01). A select screen including the respective device identification information of MFPs100,101,102registered beforehand as devices that can serve as a target for remote operation is displayed on display unit206, so that the user inputs operation of selecting device identification information displayed on the select screen into operation unit207. The device having the device identification information selected from among MFPs100,101,102is thus selected. In the following description, the user selects MFP100as a device that can serve as a target for remote operation, by way of example.

In the next step S02, the selected device is logged in. A login request is transmitted to MFP100, and a login screen returned by MFP100is displayed on display unit206. The user identification information and authentication information input by the user to operation unit207in accordance with the login screen is transmitted to MFP100. It is then determined whether login is successful (step S03). If login is successful, the process proceeds to step S04. If not, the process returns to step S01.

In step S04, it is determined whether the shutter button is pressed. If the shutter button of operation unit207is pressed by the user, the process proceeds to step S05. If not, the process proceeds to step S11.

In step S05, camera202is controlled to shoot an image. In the next step S06, it is determined whether the subject of camera202is operation panel160of MFP100. If the subject of camera202is operation panel160, the process proceeds to step S07. If not, the process proceeds to step S11. For example, when MFP100is registered in the portable information device as a device that can serve as a target for remote operation, an image of operation panel160of MFP100as a subject is shot by camera202, and the shot image output by camera202is stored as a registered image. The shot image obtained by shooting an image in step S05is then compared with the registered image, whereby it is determined whether the subject is operation panel160.

In step S07, the shot image output by camera202shooting an image in step S05is stored into a predetermined area of flash memory203. In the next step S08, a shooting notice is transmitted. A shooting notice indicating that an image is shot is transmitted to MFP100that is logged in in step S02through wireless LAN I/F208. MFP100receiving the shooting notice returns related data. In the next step S09, the process waits until related data is received from MFP100(NO in step S09). If related data is received, the process proceeds to step S10. If the device identification information of the device that has transmitted the related data received by wireless LAN I/F208is the same as the device identification information of the device that is logged in in step S02, here, MFP100, related data is received. In this manner, related data can be received only from the device that the shooting notice is transmitted to in step S08.

In step S10, the related data received in step S09is associated with the shot image stored in step S07for storage into flash memory203. The process then proceeds to step S11.

In step S11, it is determined whether the related data received in step S09includes page identification information. If the related data includes page identification information, the process proceeds to step S12. If not, the process proceeds to step S18. In step S12, an image of a page is displayed on display unit206. The process then proceeds to step S13. Specifically, if the related data includes page identification information, the data included in the related data is comprised of a plurality of pages. Therefore, of a plurality of pages of the data included in the related data, the image of the page specified by the page identification information included in the related data is displayed on display unit206. Accordingly, the same image as the image displayed at MFP100is displayed on display unit206, so that the user can confirm that the data set as a display target in MFP100is transmitted to and stored in portable information device200.

In step S13, it is determined whether a shot image view instruction is accepted. If a shot image view instruction is accepted, the process proceeds to step S14. If not, the process proceeds to step S18. A shot image view instruction is operation of giving an instruction to view the shot image stored in flash memory203in step S07. In step S14, the shot image stored in flash memory203is displayed on display unit206. If a plurality of shot images are stored in flash memory203, a list of a plurality of shot images is displayed. A plurality of shot images may be displayed in one screen, or screens each including one or more shot images may be successively displayed.

In the next step S15, it is determined whether the shot image displayed on display unit206is selected by the user. If one is selected from one or more shot images displayed on display unit206in accordance with user's operation input to operation unit207, the process proceeds to step S16. If not, the process proceeds to step S18.

In step S16, the related data that is related to the selected shot image is read out from flash memory203. The related data stored in flash memory203is associated with the shot image and stored into flash memory203in step S10. Since the shot image selected by the user is associated with any one of the related data stored in flash memory203, the related data associated with the shot image selected by the user is read out.

In the next step S17, the related data is transmitted to the device that is logged in in step S03, here, to MFP100. The process then proceeds to step S18. In step S18, it is determined whether a logout instruction is accepted. If a logout instruction is accepted, the process ends. If not, the process returns to step S04.

A flow of forming an image using portable information device200and MFPs100,101,102will now be specifically described. In the description here, the user first operates MFP100to form an image and thereafter operates MFP101to form an image, by way of example.

FIG. 10is a first diagram showing an exemplary flow of forming an image between the portable information device and the MFPs. Referring toFIG. 10, the flow of time is shown from above to below, and the respective time axes of portable information device200, MFP100, and MFP101are shown from the left in this order. First, the user operates portable information device200to select MFP100as a target for remote operation. Portable information device200then transmits a login request to MFP100. MFP100receives the login request and, if authentication is successful, returns the authentication result to portable information device200.

Afterwards, when the user of portable information device200operates operation unit163of MFP100to input operation, MFP100accepts the operation input by the user and displays a setting screen. If the user inputs operation of setting a setting value, the input setting value is set.

Next, the user operates portable information device200and presses the shutter button with the angle of view of camera202set to operation panel160of MFP100as a subject. Portable information device200then transmits a shooting notice to MFP100. MFP100receiving the shooting notice returns related data that is related to the setting screen displayed on display unit161, here, the related data including the setting value included in the setting screen.

Portable information device200then receives the related data and stores the related data in association with the shot image. Afterwards, the user may operate operation unit163of MFP100to execute a process of forming an image, for example.

Next, the user operates portable information device200to select MFP101as a target for remote operation. Portable information device200then transmits a login request to MFP101. MFP101receives the login request and, if authentication is successful, returns the authentication result to portable information device200.

Afterwards, the user of portable information device200selects the shot image stored in portable information device200. Portable information device200then acquires the related data stored in association with the shot image selected by the user and transmits a remote operation command including the related data. Here, if the shot image obtained by shooting an image of operation panel160was selected in a state in which a setting screen is displayed on display unit161of MFP100, a remote operation command including the related data received from MFP100earlier is transmitted to MFP101. The related data received from MFP100includes a setting value.

MFP101receiving the related data from portable information device200then sets the setting value included in the related data and displays a setting screen including the setting value. The same setting value as the setting value set in MFP100is therefore set in MFP101. The user thus does not have to input operation of setting a setting value in MFP101. Afterwards, when the user operates operation unit163of MFP101, MFP101accepts the input operation and executes a process.

In the example shown inFIG. 10, the user presses the shutter button of portable information device200in a state in which a setting screen is displayed on display unit161of MFP100. By contrast, in a case where the user presses the shutter button of portable information device200in a state in which a job history screen is displayed on display unit161of MFP100, the related data transmitted from MFP100includes history information. In addition, the related data included in the remote operation command transmitted to MFP101includes history information.

More specifically, the user operates portable information device200to log in to MFP100and presses the shutter button of portable information device200to shoot the job history screen displayed on display unit161of MFP100as a subject. MFP100then returns the related data including the history information selected in the job history screen to portable information device200. Portable information device200stores the shot image obtained by shooting an image of the job history screen and the related data. The user then operates portable information device200to log in to MFP101and selects the shot image obtained by shooting an image of the job history screen. Portable information device200in turn returns the related data associated with the shot image to MFP101. This related data includes history information indicating history of the process executed in MFP100. MFP101receiving the related data sets the setting value included in the history information included in the related data, so that the same setting value as the setting value used in the process executed in MFP100is set in MFP101. For example, in a case where history information of execution of a facsimile transmitting process has been selected in the job history screen, the history information includes the facsimile number of a transmission destination of facsimile. In this case, MFP101sets the facsimile number included in the history information as a setting of transmission destination. The user therefore does not have to input operation of setting a facsimile number in MFP101.

The history information may include, for example, history information in a case where a process of transmitting/receiving an email is executed. In the case where the history information includes history information of execution of a process of transmitting/receiving an email, the email address of the sender/recipient of the email is the setting value. The history information may include history information in a case where a process of transmitting/receiving a file is executed. If the history information includes history information of execution of a process of transmitting/receiving a file, the IP (Internet Protocol) address of the device sending/receiving the file is the setting value.

FIG. 11is a second diagram showing an exemplary flow of forming an image between the portable information device and the MFPs. Referring toFIG. 11, the flow of time is shown from above to below, and the respective time axes of portable information device200, MFP100, and MFP101are shown from the left in this order. First, the user operates portable information device200to select MFP100as a target for remote operation. Portable information device200then transmits a login request to MFP100. MFP100receives the login request and, if authentication is successful, returns the authentication result to portable information device200.

Afterwards, when the user of portable information device200operates operation unit163of MFP100and inputs operation, MFP100accepts the operation input by the user and displays a BOX screen. If the user inputs operation of selecting data identification information included in the BOX screen, MFP100sets the data specified by the selected data identification information as a process target.

Next, the user operates portable information device200and presses the shutter button with the angle of view of camera202set to operation panel160of MFP100as a subject. Portable information device200then transmits a shooting notice to MFP100. MFP100receiving the shooting notice returns related data that is related to the BOX screen displayed on display unit161, here, the related data including the data specified by the data identification information selected in the BOX screen. Afterwards, the user may operate operation unit163of MFP100to execute a process of forming an image, for example.

Portable information device200receives the related data and stores the related data in association with the shot image. Afterwards, the user may operate operation unit163of MFP100and execute a process of forming an image, for example.

Next, the user operates portable information device200to select MFP101as a target for remote operation. Portable information device200then transmits a login request to MFP101. MFP101receives the login request and, if authentication is successful, returns the authentication result to portable information device200.

Afterwards, the user of portable information device200selects the shot image stored in portable information device200. Portable information device200then acquires the related data stored in association with the shot image selected by the user and transmits a remote operation command including the related data. Here, assuming that the shot image obtained by shooting an image of operation panel160was selected in a state in which a BOX screen is displayed on display unit161of MFP100, a remote operation command including the related data received from MFP100earlier is transmitted to MFP101. The related data received from MFP100includes data.

MFP101receiving the related data from portable information device200sets the data included in the related data as a process target and displays a BOX screen in a state in which the data as a process target is selected. Accordingly, the same data as the data selected as a process target in MFP100can be set as a process target in MFP101. Afterward, when the user operates operation unit163of MFP101, MFP101accepts the input operation and executes a process on the data set as a process target.

FIG. 12is a third diagram showing an exemplary flow of forming an image between the portable information device and the MFPs. Referring toFIG. 12, the flow of time is shown from above to below, and the respective time axes of portable information device200, MFP100, and MFP101are shown from the left in this order. First, the user operates portable information device200to select MFP100as a target for remote operation. Portable information device200then transmits a login request to MFP100. MFP100receives the login request and, if authentication is successful, returns the authentication result to portable information device200.

Afterwards, when the user of portable information device200operates operation unit163of MFP100and inputs operation, MFP100accepts the operation input by the user and displays a preview screen.

Next, the user operates portable information device200and presses the shutter button with the angle of view of camera202set to operation panel160of MFP100as a subject. Portable information device200then transmits a shooting notice to MFP100. MFP100receiving the shooting notice returns related data that is related to the preview screen displayed on display unit161, here, the related data including the data set as a preview target and page identification information of the page corresponding to the image included in the preview screen.

Portable information device200receiving the related data displays the image of the page specified by the page identification information, of a plurality of pages of the data included in the related data, on display unit161. Accordingly, the same image as the image included in the preview screen displayed on display unit161of MFP100is displayed on display unit206of portable information device200, so that the user can confirm that the data set as a preview target in MFP100is stored into portable information device200, based on the image of the page displayed on display unit206.

In addition, portable information device200stores the related data in association with the shot image. Afterwards, the user may operate operation unit163of MFP100to execute a process of forming an image, for example.

Next, the user operates portable information device200and selects MFP101as a target for remote operation. Portable information device200then transmits a login request to MFP101. MFP101receives the login request and, if authentication is successful, returns the authentication result to portable information device200.

Afterwards, the user of portable information device200selects the shot image stored in portable information device200. Portable information device200then acquires the related data stored in association with the shot image selected by the user and transmits a remote operation command including the related data. Here, assuming that the shot image obtained by shooting an image of operation panel160was selected in a state in which a preview screen is displayed on display unit161of MFP100, a remote operation command including the related data received from MFP100earlier is transmitted to MFP101. The related data received from MFP100includes data and page identification information.

MFP101receiving the related data from portable information device200sets the data included in the related data as a process target and displays a BOX screen in a state in which the data as a process target is selected. Accordingly, the same data as the data selected as a preview target in MFP100can be set as a process target in MFP101. Afterward, when the user operates operation unit163of MFP101, MFP101accepts the input operation and executes a process on the data set as a process target.

For example, operation of scanning a document is input and an image of a preview screen is shot at MFP100, so that the data obtained by scanning a document in MFP100can be set as a process target in MFP101when MFP101is logged in next time. If operation is subsequently input to MFP101, the data obtained by scanning a document in MFP100can be subjected to image formation, facsimile transmission, file transfer, email transmission, etc. in MFP101.

As described above, in image forming system1in the present embodiment, when portable information device200shoots an image of an operation screen displayed on display unit161of MFP100, a shooting notice is transmitted to MFP100. MFP100then transmits, to portable information device200, related data that is related to the operation screen displayed at the point of time when the shooting notice is received from portable information device200. Portable information device200then stores the related data received from MFP100. Accordingly, the related data that is related to the operation screen displayed at MFP100can be stored into portable information device200with such easy operation of shooting the operation screen.

The image obtained by shooting the operation screen of MFP100is stored in portable information device200. In response to the stored image being designated, a remote operation command including the related data associated with the designated image is transmitted to MFP100. MFP100in turn executes a process based on the related data included in the remote operation command. Since the image obtained by shooting the operation screen and the related data are associated with each other, the user looking at the image can easily specify the related data that is related to the operation screen included in the image.

Since MFP100executes a process based on the related data received from portable information device200, the user can bring MFP100at present into the state in which MFP100was operated in the past, through the operation of selecting the image of the operation screen at the time when MFP100was operated in the past.

In a case where the related data is a setting value for MFP100to execute a process, a remote operation command to allow MFP100to set a setting value is transmitted to MFP100. Accordingly, a setting value in MFP100can be set to the same value as the one at the time when MFP100was operated in the past.

In a case where the operation screen displayed at the point of time when a shooting notice is received from portable information device200is a setting screen, MFP100decides the setting value set in the setting screen as related data. The user therefore can select a setting value to be set in MFP100by selecting the image of the setting screen.

In a case where the operation screen displayed at the point of time when a shooting notice is received from portable information device200is a history display screen, MFP100decides the setting value set for the process specified on the history display screen as related data. The user therefore can set the setting value to be set in MFP100to the same value as the setting value at the time when MFP100executed the process in the past, by selecting the image of the history display screen.

In a case where one data is selected as a process target from among data stored in HDD115at the point of time when a shooting notice is received from portable information device200, MFP100decides the data selected as a process target as related data. If the related data is data that can be processed by MFP100, portable information device200transmits a remote operation command to allow MFP100to set the data as a process target. Accordingly, the data to be processed by MFP100can be set to the same data as the data that was set as a process target when MFP100was operated in the past.

The user can remotely operate MFP101in the same state as the state of MFP100operated in the past, through the operation of selecting the image of the operation screen at the time when MFP100was operated in the past.

In a case where MFP100displays a preview screen at the point of time when a shooting notice is received, data comprised of a plurality of pages and page identification information of the page corresponding to the image displayed on the preview screen are decided as the related data. Portable information device200then displays the image of the page specified by the page identification information of the data included in the related data. Accordingly, the same image as the image included in the preview screen displayed at MFP100is displayed at portable information device200. As a result, the user is notified that the data including the page displayed in the preview screen is stored into the portable information device.

In a case where MFP100displays a BOX screen at the point of time when a shooting notice is received, the data selected in the BOX screen in MFP100is decided as the related data, and portable information device200stores the data included in the related data into portable information device200. As a result, the data selected in the BOX screen can be stored into portable information device200.

Although image forming system1has been described in the foregoing embodiment, it is needless to say that the present invention can be understood as a remote operation method to allow MFPs100,101,102to execute the remote control process shown inFIG. 7andFIG. 8and to allow the portable information device to execute the remote operation process shown inFIG. 9, or as a remote control program to allow CPU111of each MFP100,101,102to execute the remote control process and a remote operation program to allow CPU201of portable information device200to perform the remote operation method.