Electronic equipment provided with an AI processor

An image forming apparatus as electronic equipment includes an AI processor provided with a memory, a storage unit capable of storing a binary, which is a program for the AI processor, and an AI processor operation unit that operates the AI processor. The AI processor operation unit deploys, in the memory, the binary stored in the storage unit. The AI processor executes a function of the binary by loading the binary deployed in the memory. The binary is not included in firmware of the image forming apparatus.

INCORPORATION BY REFERENCE

This application is based upon, and claims the benefit of priority from, corresponding Japanese Patent Application No. 2020-114646 filed in the Japan Patent Office on Jul. 2, 2020, the entire contents of which are incorporated herein by reference.

BACKGROUND

Field of the Invention

The present disclosure relates to electronic equipment provided with an AI processor.

Description of Related Art

As typical electronic equipment, there is known electronic equipment in which, after a program is written in a non-volatile memory, the program is executed at a startup time of the electronic equipment.

SUMMARY

Electronic equipment according to the present disclosure is electronic equipment including an AI processor provided with a memory, a storage unit capable of storing a program for the AI processor, and an AI processor operation unit that operates the AI processor. The AI processor operation unit deploys, in the memory, the program stored in the storage unit. The AI processor executes a function of the program by loading the program deployed in the memory. The program is not included in firmware of the electronic equipment.

DETAILED DESCRIPTION

In the following, an embodiment according to the present disclosure is described with reference to the drawings.

First, a configuration of an image forming apparatus as electronic equipment according to one embodiment of the present disclosure is described.

FIG.1is a block diagram of an image forming apparatus10according to the present embodiment.

As shown inFIG.1, the image forming apparatus10is a multi function peripheral (MFP) device including: an operation unit11, which is an operation device through which various operations are input such as, for example, a button; a display unit12, which is a display device for displaying various pieces of information such as, for example, a liquid crystal display (LCD); a printer13, which is a printing device for printing an image on a recording medium such as paper; a scanner14, which is a reading device for reading an image from a document; a facsimile communication unit15, which is a facsimile device for performing facsimile communication with an unillustrated external facsimile device via a communication line such as a public telephone line; a communication unit16, which is a communication device for communicating with an external device via a network such as a local area network (LAN) or the Internet, or directly by wired or wireless communication without a network; an AI processor17, as a processor for enabling inference processing of machine learning; a storage unit18, which is a non-volatile storage device for storing various types of information such as, for example, a semiconductor memory or a hard disk drive (HDD); and a control unit19for controlling the entirety of the image forming apparatus10.

The AI processor17includes a memory17ain which a program for commanding processing to be executed by the AI processor17is deployed. Once the program is deployed in the memory17a, the AI processor17executes processing commanded by the program.

The storage unit18is capable of storing firmware18a. The firmware18amay, for example, be installed in the image forming apparatus10at a manufacturing stage of the image forming apparatus10, or may be additionally installed in the image forming apparatus10from an external storage medium such as a universal serial bus (USB) memory, an SD card, or the like, or may be additionally installed in the image forming apparatus10from a network.

The storage unit18is capable of storing a binary18b, which is a program for the AI processor17. The storage unit18is capable of storing at least one binary, which is a program for the AI processor17, in addition to the binary18b. The storage unit18is capable of storing a binary in a binary storage area (hereinafter, referred to as a “binary area”). A binary may, for example, be installed in the image forming apparatus10at a manufacturing stage of the image forming apparatus10, or may be additionally installed in the image forming apparatus10from an external storage medium such as a USB memory, an SD card, or the like, or may be additionally installed in the image forming apparatus10from a network.

FIG.2is a diagram showing one example of information included in the binary18bshown inFIG.1.

As shown inFIG.2, the binary18bincludes function information indicating a function that can be achieved by causing the AI processor17to load the binary18b, a function version indicating a version of a function of the binary18b, an input format of data to the binary18b, and an output format of data from the binary18b.

The function indicated by the function information includes, for example, a function of converting monochrome image data into color image data, and a function of making image quality clearer.

As the input format and the output format, for example, there is an image in a JPEG format of 125×125 dpi.

The control unit19shown inFIG.1includes, for example, a central processing unit CPU), a read only memory (RAM) storing a program and various types of data, and a random access memory (RAM) as a memory for use as a work area of the CPU of the control unit19. The CPU of the control unit19executes a program stored in the storage unit18or in the ROM of the control unit19.

FIG.3is a block diagram showing a control configuration of the image forming apparatus10.

The control unit19achieves the control configuration shown inFIG.3by executing a program stored in the storage unit18or in the ROM of the control unit19.

As shown inFIG.3, the control configuration of the image forming apparatus10includes an equipment control platform30that manages hardware of each unit of the image forming apparatus10such as, for example, the operation unit11, the display unit12, the printer13, the scanner14, the facsimile communication unit15, and the communication unit16, and controls the hardware in a function unit; a standard application execution unit40that executes a standard application41, which is an application program for executing various types of standard processing of the image forming apparatus10; and an extended application platform50that executes an extended application60, which is an application program for executing various types of processing that cannot be executed only by the standard application41. The standard application execution unit40and the extended application platform50are respectively operated on the equipment control platform30.

The equipment control platform30includes a general-purpose operating system (OS)31such as, for example, Linux (registered trademark); a unit control service32that controls hardware of each unit of the image forming apparatus10associated with a function of the image forming apparatus10in an appropriate manner; and a platform service IF unit33that provides an interface for using hardware from outside the equipment control platform30. The control unit19achieves the equipment control platform30by executing the firmware18a.

The standard application execution unit40includes the standard application41, and a standard application service IF unit42that provides an interface for using a function of the standard application41from outside the standard application execution unit40.

For example, as the standard application41, there are a copy application41A that prints an image read from a document by the scanner14by means of the printer13in response to an instruction via the operation unit11; a printer application41B that performs printing by the printer13, based on data input via the communication unit16or data stored in the storage unit18; a facsimile application41C that facsimile-transmits, by the facsimile communication unit15, an image read from a document by the scanner14in response to an instruction via the operation unit11, or an image based on data input via the communication unit16, or performs printing of an image that is facsimile-received by the facsimile communication unit15by means of the printer13or performs storage into the storage unit18; and a transmission application41D that transmits data via the communication unit16.

The standard application service IF unit42may provide a different interface for each standard application41. The standard application service IF unit42can perform execution, installation, setting change, deletion and the like of the standard application41.

The extended application platform50includes a standard function service51that causes the standard application41to execute a function via the standard application service IF unit42, an extended function service52that causes hardware of each unit of the image forming apparatus10to execute a function via the platform service IF unit33, a general-purpose OS installation service53for adding a function to the general-purpose OS31, and an extended application management unit54that manages installation, activation, execution, execution termination, uninstallation, and the like of the extended application60by means of an API of the extended application platform50.

The extended application60can use a function of the standard application41by calling the standard function service51, use a function of hardware of each unit of the image forming apparatus10by calling the extended function service52, and add a function to the general-purpose OS31by calling the general-purpose OS installation service53. For example, as the extended application60, there are practical applications such as optical character recognition (OCR), a surveillance camera application, watermark creation, invoice creation, employee list management, slip management, contract management, business card management, PDF creation, ledger management, and group management; and management applications such as an installer application that performs addition and/or change of a device driver, firmware, and the like on the general-purpose OS31.

FIG.4is a block diagram showing a detailed configuration of the unit control service32and the platform service IF unit33.

As shown inFIG.4, the unit control service32includes an image processing service70that provides a service related to creation, conversion, or output of image processing data, such as drawing image data to be printed by the printer13; a device operation service80that provides a service for controlling hardware i.e., a device of each unit of the image forming apparatus10; and a network communication service90that provides a service related to network transmission and reception.

The platform service IF unit33includes an image processing service IF33aas an interface associated with a function of the equipment control platform30related to creation, conversion, or output of image processing data; a device operation service IF33bas an interface associated with a function of the equipment control platform30for controlling a device of the image forming apparatus10; and a network communication service IF33cas an interface associated with a function of the equipment control platform30related to network transmission and reception. The device operation service IF33bconstitutes a display content control unit that controls display content on the display unit12.

Note that, the image processing service IF33a, the device operation service IF33b, and the network communication service IF33cof the platform service IF unit33are respectively configured to call the image processing service70, the device operation service80, and the network communication service90of the unit control service32independently of one another without cooperation. For example, for a job of the copy application41A, the image processing service IF33acalls the image processing service70and the device operation service80. Therefore, when a change occurs in any of the functions of the standard application41, it is possible to cope with the change only by changing a service related to this function among the image processing service IF33a, the device operation service IF33b, and the network communication service IF33c.

FIG.5is a block diagram showing a detailed configuration of the image processing service70.

As shown inFIG.5, the image processing service70includes an AI processor operation unit71, which is a driver for operating the AI processor17, an AI processor control service72that controls the AI processor17via the AI processor operation unit71, and an AI processor control service providing unit73as an interface for calling the AI processor control service72from outside the image processing service70. The AI processor control service72can control the AI processor17only through an interface provided from the AI processor operation unit71.

The image processing service70can execute AI processing with respect to an image read by the scanner14, for example, by using the AI processor17. For example, the image processing service70can clarify a line that become blurred when an image is enlarged by means of AI processing, and extract handwritten characters included in an image by means of AI processing.

Next, an operation of the image forming apparatus10is described.

First, an operation of the AI processor operation unit71when a binary is written in a binary area of the storage unit18is described.

FIG.6is a flowchart of an operation of the AI processor operation unit71when a binary is written in a binary area of the storage unit18.

As shown inFIG.6, the AI processor operation unit71determines whether function information of a binary (hereinafter, referred to as a “new binary”) written in a binary area of the storage unit18is the same as any of function information of a binary (hereinafter, referred to as an “existing binary”) already stored in the binary area of the storage unit18(S101).

When the AI processor operation unit71determines in S101that the function information of the new binary is not the same as any of the function information of the existing binary, the AI processor operation unit71notifies the device operation service IF33bof the function information of the new binary (S102). Therefore, the device operation service IF33bcauses the display unit12to display an instruction accept portion such as, for example, a button that accepts an instruction to execute the function indicated by the function information notified in S102via the device operation service80and the general-purpose OS31in this order.

When the AI processor operation unit71determines in S101that the function information of the new binary is the same as any of the function information of the existing binary, or when processing of S102is terminated, the AI processor operation unit71determines whether an output format of the new binary is the same as any of input formats of the existing binary (S103).

When the AI processor operation unit71determines in S103that the output format of the new binary is the same as any of the input formats of the existing binary, the AI processor operation unit71notifies the device operation service IF33bof a combination of the new binary, and the existing binary whose input format is the same as the output format of the new binary (S104). Specifically, after a new binary is executed, the AI processor operation unit71notifies the device operation service IF33bthat an existing binary whose input format is the same as the output format of the new binary can be executed with respect to data output from the new binary. Therefore, the device operation service IF33bcauses the display unit12to display an instruction accept portion such as, for example, a button that accepts an instruction to execute in such a way that functions of the combination notified in S104are used together via the device operation service80and the general-purpose OS31in this order.

When the AI processor operation unit71determines in S103that the output format of the new binary is not the same as any of the input formats of the existing binary, or after processing of S104is terminated, the AI processor operation unit71determines whether the input format of the new binary is the same as any of the output formats of the existing binary (S105).

When the AI processor operation unit71determines in S105that the input format of the new binary is the same as any of the output formats of the existing binary, the AI processor operation unit71notifies the device operation service IF33bof a combination of the existing binary whose output format is the same as the input format of the new binary, and the new binary (S106). Specifically, the AI processor operation unit71notifies the device operation service IF33bthat, after execution of an existing binary whose output format is the same as an input format of a new binary, the new binary can be executed with respect to data output from the existing binary. Therefore, the device operation service IF33bcauses the display unit12to display an instruction accept portion such as, for example, a button that accepts an instruction to execute in such a way that functions of the combination notified in S106are used together via the device operation service80and the general-purpose OS31in this order.

When the AI processor operation unit71determines in S105that the input format of the new binary is not the same as any of the output formats of the existing binary, or when processing of S106is terminated, the operation shown inFIG.6is terminated.

In the foregoing, an operation of the AI processor operation unit71when a binary is written in a binary area of the storage unit18is described. However, when the image forming apparatus10is started, the AI processor operation unit71is operated as shown inFIG.7.

FIG.7is a flowchart of an operation of the AI processor operation unit71when the image forming apparatus10is started.

As shown inFIG.7, the AI processor operation unit71notifies the device operation service IF33bof kinds of function information of all binaries stored in a binary area of the storage unit18(S121). Specifically, when there are a plurality of binaries whose function information is the same, the AI processor operation unit71notifies the device operation service IF33bof only one piece of function information, as function information of these binaries. Therefore, the device operation service IF33bcauses the display unit12to display an instruction accept portion such as, for example, a button that accepts an instruction to execute a function indicated by function information notified in S121via the device operation service80and the general-purpose OS31in this order.

After processing of S121is terminated, the AI processor operation unit71determines whether there is a combination, among binaries stored in the binary area of the storage unit18, in which an output format of one of the binaries is the same as an input format of the other binary (S122).

When the AI processor operation unit71determines in S122that there is a combination in which an output format of one of the binaries is the same as an input format of the other binary, the AI processor operation unit71notifies the device operation service IF33bof the combination in which the output format of one of the binaries is the same as the input format of the other binary (S123). Specifically, after one of the binaries is executed, the AI processor operation unit71notifies the device operation service IF33bthat the other binary can be executed with respect to data output from the one binary. Therefore, the device operation service IF33bcauses the display unit12to display an instruction accept portion such as, for example, a button that accepts an instruction to execute in such a way that functions of the combination notified in S123are used together via the device operation service80and the general-purpose OS31in this order.

When the AI processor operation unit71determines in S122that there is no combination in which an output format of one of the binaries is the same as an input format of the other binary, or when processing of S123is terminated, the operation shown inFIG.7is terminated.

Note that, when there are a plurality of function versions having a same function in a binary stored in the binary area of the storage unit18, the device operation service IF33bcauses the display unit12to display an instruction accept portion that can accept designation of a function version, as an instruction accept portion that accepts an instruction to execute the function.

Next, an operation of the AI processor operation unit71when execution of a function is requested from the AI processor control service72is described.

FIG.8is a flowchart of an operation of the AI processor operation unit71when execution of a function is requested from the AI processor control service72.

When a user operates an instruction accept portion displayed on the display unit12via the operation unit11, the AI processor control service72requests the AI processor operation unit71to execute a function associated with the operated instruction accept portion.

As shown inFIG.8, when the AI processor operation unit71is requested to execute a function from the AI processor control service72, the AI processor operation unit71determines whether a function version of the function is designated (S141).

When the AI processor operation unit71determines in S141that a function version of a function whose execution is requested from the AI processor control service72is designated, the AI processor operation unit71specifies a binary, from among the binaries stored in the binary area of the storage unit18, which includes function information indicating the function whose execution is requested from the AI processor control service72, and the function version determined to be designated in S141(S142).

When the AI processor operation unit71determines in S141that a function version of a function whose execution is requested from the AI processor control service72is not designated, the AI processor operation unit71specifies a binary, from among the binaries stored in the binary area of the storage unit18, which includes a latest function version among those including function information indicating a function whose execution is requested from the AI processor control service72(S143).

After processing of S142or S143, the AI processor operation unit71deploys, in the memory17aof the AI processor17, the binary specified in S142or S143(S144), and terminates the operation shown inFIG.8. Therefore, the AI processor17executes the function of the binary by loading the binary deployed in the memory17a.

As described above, since a binary, which is a program for the AI processor17, is not included in the firmware18aof the image forming apparatus10, the image forming apparatus10can update a function to be executed by the AI processor17, even if the firmware18ais not updated. Herein, a power supply of the image forming apparatus10is needed to be turned off and then turned on again when the firmware18ais updated. However, since the image forming apparatus10can update a function to be executed by the AI processor17, even if the firmware18ais not updated, the image forming apparatus10can update the function to be executed by the AI processor17even without turning off the power supply of the image forming apparatus10and then turning on again. Therefore, the image forming apparatus10can reduce a period of time during which a user cannot use the image forming apparatus10.

Since the image forming apparatus10can update a function to be executed by the AI processor17even without turning off the power supply of the image forming apparatus10and then turning on again, the image forming apparatus10can update the function to be executed by the AI processor17without stopping the function to be executed by the AI processor17.

Since the image forming apparatus10can update the function to be executed by the AI processor17without updating the firmware18a, the image forming apparatus10does not have to create the firmware18afor updating the function to be executed by the AI processor17. Specifically, the image forming apparatus10can facilitate updating of a function to be executed by the AI processor17.

Since the image forming apparatus10causes the display unit12to display an instruction accept portion that accepts an instruction to execute a function indicated by function information of a binary, which is a program for the AI processor17, even if the firmware18aof the image forming apparatus10is not updated, it is possible to display, on the display unit12, the instruction accept portion for causing the AI processor17to execute the function.

Since the image forming apparatus10causes the display unit12to display an instruction accept portion that can accept designation of a function version, the image forming apparatus10allows the AI processor17to execute a function of a function version desired by a user regarding a specific function.

Since the image forming apparatus10causes the display unit12to display that functions of a combination in which an output format of one of binaries is the same as an input format of the other binary can be used together, convenience can be improved. Note that, the image forming apparatus10may cause the display unit12to display that functions of a combination can be used together according to a method other than a method according to which the display unit12is caused to display an instruction accept portion that accepts an instruction to execute in such a way that functions of a combination in which an output format of one of binaries is the same as an input format of the other binary are used together. For example, the image forming apparatus10may simply cause the display unit12to display that functions of a combination in which an output format of one of binaries is the same as an input format of the other binary can be used together.

In the present embodiment, the AI processor17is used in the image processing service70. Alternatively, the AI processor17may be used in a service other than the image processing service70. For example, the AI processor17may be used in the device operation service80or the network communication service90.

Although the electronic equipment according to the present disclosure is an MFP device in the present embodiment, the electronic equipment may be an image forming apparatus other than an MFP device, such as a printer-dedicated machine, a scanner-dedicated machine, a facsimile-dedicated machine, or a copy machine, or may be electronic equipment other than an image forming apparatus, such as, for example, a personal computer (PC).