Information processing apparatus, method of controlling the same and non-transitory computer-readable storage medium

An information processing apparatus that has a display unit and a plurality of graphic controllers, the apparatus comprises: a detection unit that detects an event that is generated by a user operation on the display unit; a determination unit that determines, when the event is detected by the detection unit, which of the plurality of graphic controllers an application displaying a screen on the display unit uses; and a transmission unit that, based on a resolution of the display unit and information of the application for which the determination is made by the determination unit, converts coordinate information of the user operation that the event indicates to a value of a coordinate system corresponding to the graphic controller that the application uses, and transmits a result of the conversion to the application.

BACKGROUND OF THE INVENTION

Field of the Invention

The present invention relates to an information processing apparatus, a method of controlling the same, and a non-transitory computer-readable storage medium.

Description of the Related Art

Touch panels by which it is possible to select a target object that is displayed on an operation unit of an information processing apparatus by touching the target object with a finger are spreading. Each such touch panel has a respective determined resolution, and it is possible to connect and use touch panels of different resolutions to a single device in accordance with the situation.

Depending on the resolution of the touch panel, the appearance differs even when the same target object is displayed. For example, a target object that was displayed on the full area of the touch panel in a touch panel whose resolution is low is displayed in only a partial region of the touch panel in a touch panel whose resolution is high. For this reason, there is a problem in that regions in which nothing is displayed are created when a touch panel whose resolution is high is connected. To solve this problem, a method of uniformly magnifying/reducing coordinates in which to display or detect in accordance with a detection region or a resolution of a touch panel (for example, Japanese Patent No. 5713180) is known. As another method, there is a method in which a target object that is displayed on a low resolution touch panel is displayed as is on a high resolution touch panel, and another target object is displayed in a free region.

When a user touches a target object on a touch panel, as internal processing of an information processing apparatus, the target object is selected by generating event information that represents the fact that it was touched, and a graphic controller performing processing based on this event information. There are various types of graphic controllers, and the necessary event information for selecting the target object differs depending on the type.

In an information processing apparatus that has a touch panel, a plurality of applications (hereinafter referred to as apps) operate, and different graphic controllers exist for each app implementation language. Here, there are cases in which a difference arises in the contents of the event information that is necessary depending on the graphic controller. For example, position information for operation by a user on the touch panel is an example of event information. There are graphic controllers that can use, as is, position information of a user operation that the touch panel detected, but there are others for which it is necessary to convert to position information (a coordinate system) that differs depending on a relation with other objects that are being displayed. There is a problem in that if the event information that a graphic controller requires is not passed appropriately, the app will cease to operate normally.

SUMMARY OF THE INVENTION

According to one aspect of the present invention, there is provided an information processing apparatus that has a display unit and a plurality of graphic controllers, the apparatus comprising: a detection unit configured to detect an event that is generated by a user operation on the display unit; a determination unit configured to determine, when the event is detected by the detection unit, which of the plurality of graphic controllers an application displaying a screen on the display unit uses; and a transmission unit configured to, based on a resolution of the display unit and information of the application for which the determination is made by the determination unit, convert coordinate information of the user operation that the event indicates to a value of a coordinate system corresponding to the graphic controller that the application uses, and transmit a result of the conversion to the application.

According to another aspect of the present invention, there is provided a method of controlling an information processing apparatus that has a display unit and a plurality of graphic controllers, the method comprising: detecting an event that is generated by a user operation on the display unit; determining, when the event is detected, which of the plurality of graphic controllers an application displaying a screen on the display unit uses; and based on a resolution of the display unit and information of the application for which the determination is made, converting coordinate information of the user operation that the event indicates to a value of a coordinate system corresponding to the graphic controller that the application uses, and transmitting a result of the conversion to the application.

According to another aspect of the present invention, there is provided a non-transitory computer-readable storage medium storing a program for causing a computer that has a display unit and a plurality of graphic controllers to function as: a detection unit configured to detect an event that is generated by a user operation on the display unit; a determination unit configured to determine, when the event is detected by the detection unit, which of the plurality of graphic controllers an application displaying a screen on the display unit uses; and a transmission unit configured to, based on a resolution of the display unit and information of the application for which the determination is made by the determination unit, convert coordinate information of the user operation that the event indicates to a value of a coordinate system corresponding to the graphic controller that the application uses, and to transmit a result of the conversion to the application.

By the present invention, in a case where a plurality of graphic controllers that operate using different event information are operating, it is possible to cause an app to operate correctly whatever the graphic controller that the app uses.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, a description of embodiments for implementing the present invention is given using drawings.

First Embodiment

A description of the present embodiment is given using an image forming apparatus which is one embodiment of an information processing apparatus.

(Controller Unit of Image Forming Apparatus)

FIG. 1illustrates an example of an internal structure of a controller unit of an image forming apparatus100according to the embodiment. A controller unit120has a CPU101for executing various control programs. The CPU101causes a system to boot based on a boot program stored in a ROM103. Then, the CPU101reads out a control program stored in an HDD (hard disk apparatus)104and executes predetermined processing using a RAM102as a work area. By this control program, the CPU101is able to execute a predetermined control of a Java program, or the like. The HDD104is a non-volatile storage region, stores various control programs, and stores image data and information relating to all communication means that a network unit107has.

The RAM102, the ROM103, and the HDD104are connected to the CPU101via a system bus111. Furthermore, an operation unit I/F105, the network unit107, an image bus I/F109, and a power management unit110are connected to the CPU101through the system bus111.

The operation unit I/F105is an interface unit for an operation unit (display)106and performs processing for obtaining and transferring image data to be displayed on the operation unit106from the RAM102and processing for transferring a signal generated by the operation unit106to the CPU101.

The operation unit106performs display processing for displaying a target object such as a button that a user can operate, and input processing for detecting a signal (input signal) of a user operation with respect to information displayed by the display processing. That is, the operation unit106functions as a user interface and includes a display unit which has an output function.

The power management unit110performs management of power OFF/ON of the image forming apparatus100. Note, if power ON is detected, the CPU101causes the system to boot based on the boot program of the ROM103, and executes a control program stored in the HDD104.

The network unit107is connected to a LAN108and performs input and output of information via the LAN108. If a Web server (not shown) or an external apparatus400is connected to the LAN108, the image forming apparatus100can obtain information via the LAN108from the external apparatus. Also, the image forming apparatus100is capable of connecting to the Internet (not shown) via a proxy server (not shown) in the LAN108and obtaining Web content from a Web server on the Internet.

The image bus I/F109is a bus bridge connected between the system bus111and an image bus112, and converts data structures. The image bus112is configured by a bus in compliance with an IEEE 1394 specification or a PCI bus capable of transferring image data at a high rate. An RIP (Raster Image Processor)113, a device I/F114, and an image processing unit119are connected to the image bus112.

The RIP113expands a PDL code into a bitmap image. The device I/F114connects a scanner116and a printer118, which are image input and output devices, with the controller unit120, and performs a conversion between synchronous/asynchronous systems of image data. Here, the device I/F114and the scanner116are connected through a cable115, and the device I/F114and the printer118are connected through a cable117.

The image processing unit119performs compression/decompression processing (JPEG for multi-value image data, and JBIG, MMR, and MH for binary image data). Also, it performs correction, modification, and editing, such as a printer correction or a resolution conversion, on input image data or output screens.

In this way, the CPU101of the controller unit120, based on each control program, comprehensively controls access to and from various devices connected to the system bus111and reads image information from the scanner116via the device I/F114. After the CPU101performs predetermined processing on read-in image information, control in which image information is outputted to the printer118via the device I/F114is performed. Furthermore, the CPU101performs analysis/conversion processing of character data and image data obtained via the network unit107, and saves to the HDD104or performs a display control in the operation unit106.

(Control of Operation Unit by CPU)

FIG. 2illustrates a configuration example of the operation unit106and its periphery inFIG. 1. A touch panel203of the operation unit106and a key group202of various hard keys are connected to the CPU101through an input port201of the operation unit I/F105. Also, position information indicating a press position on the touch panel203and key information corresponding to a press of the key group202are inputted to the CPU101through the input port201.

An LED205and an LCD206of the operation unit106are connected to the CPU101through an output port204of the operation unit I/F105. Lighting of the LED205and display of the LCD206are controlled by the CPU101. The touch panel203, on which a very small amount of electric current flows, is an electrostatic type apparatus that detects a touch position based on a change of electric charge that occurs upon a touch by a user, and the touch panel203is laid out to overlap with the LCD206. By a user directly touching an operation target object such as a virtual button (hereinafter referred to as a button) displayed on the LCD206, the touch panel203detects the position information (coordinates).

When the CPU101detects the position information, the operation target selected by the user is specified based on a target object currently displayed on the LCD206and position information thereof. Then, the CPU101detects a command to be executed in a case of selection of the specified operation target object from out of commands stored in the ROM103or the HDD104in advance, and executes the command. With this, screen data to be displayed on the LCD206is created. By the CPU101sending the created screen data to the LCD206via the output port204, the display on the operation unit106can be changed. Note that the above described internal processing of the LCD206upon the user operation is common throughout all of the following description, and so description thereof is omitted hereinafter.

Also, the operation unit106(touch panel203) has various resolutions, for example SVGA or WSVGA are examples of useable resolutions. Also, as necessary the user can change the resolution of the operation unit106that is used by physically replacing the device configuration of the image forming apparatus100.

Next, a configuration of software modules operating on each piece of hardware such as the CPU101or the HDD104of the image forming apparatus100is described usingFIG. 3. Note, processing of each module is realized by reading a program stored in the ROM103or the HDD104and executing predetermined processing thereof by commands of the CPU101using the RAM102as a work area. Also, various information generated by executing predetermined processing is stored in the RAM102or the HDD104. This kind of processing for each module is similar unless a clear recitation is particularly made, so this kind of hardware description is omitted hereinafter. Also, the detailed processing of each module illustrated inFIG. 3is described later usingFIG. 7. Also, the image forming apparatus100includes both a C graphic controller and a Java graphic controller, and the detailed processing of these controllers is described later.

A driver unit301is a touch panel driver and is a module for detecting event information such as coordinates or a touch event that is generated when the user performs an operation such as a touch on the operation unit106(strictly speaking the LCD206of the operation unit106). This event information includes a type of the event (an operation type such as a press or a release) and the coordinates of the event. The driver unit301transmits the detected event information to an event management unit302.

The event management unit302is a module for receiving event information transmitted from the driver unit301, performing predetermined processing using each module for the event information, and then transmitting the event information to a window management unit305. Specifically, the event management unit302uses the window management unit305in a case of performing a determination of the app being displayed, and uses an event information changing unit304in a case of performing a change of the event information. An operation unit management unit303is a module for obtaining information to be displayed on the operation unit106of the image forming apparatus100. The operation unit management unit303is capable of obtaining information of the resolution of the operation unit106. The event information changing unit304is a module for receiving an event information change request from the event management unit302, performing a change of the event information, and transmitting the changed event information to the event management unit302.

The window management unit305is a module for receiving event information transmitted from the event management unit302, and performing a transmission of the event information to the graphics controller unit corresponding to an app being displayed on the operation unit106. In the present embodiment, it is assumed that a C graphics controller unit306and a Java graphics controller unit308are included as graphics controller units. Also, the window management unit305is capable of performing management of a window displayed on the operation unit106, determining which window of which app is displayed on the operation unit106, and displaying any window on the operation unit106.

The C graphics controller unit306is a module comprised of rendering controllers such as a button, a list, or a window, used by C applications of a C app unit307(hereinafter called C apps) implemented in the C language. Note, these rendering controllers of the C graphics controller unit306are all laid out on windows, and it is possible to lay out a plurality of rendering controllers on one window. Therefore, the screen of the operation unit106is switched by switching the window. Note, it is assumed that information such as lay out coordinates of a rendering controller or execution processing corresponding to the rendering controller is included in the app program. Also, the C graphics controller unit306transmits an execution processing request to an app of a window being displayed on the operation unit106when event information is received from the window management unit305.

The C app unit307is a group of C apps. The C apps of the C app unit307execute processing corresponding to the execution processing request when the execution processing request is received from the C graphics controller unit306. For example, if a process A of a C app is desired to be executed, a button A for executing the process A is laid out on a specific window, and this window is displayed to the operation unit106. Then, when the user presses the button A, the C graphics controller unit306detects a selection of the button A and transmits an event to request the execution of the process A which is associated with the button A to the C app.

The Java graphics controller unit308is a module comprised of rendering controllers such as a button or a list, used by Java applications of a Java app unit309(hereinafter called Java apps) implemented in the Java language. Also, the Java graphics controller unit308transmits an execution processing request to an app in a window being displayed on the operation unit106when the event information is received from the window management unit305, similarly to the C graphics controller unit306.

The Java app unit309is a group of Java apps. The Java apps of the Java app unit309execute processing corresponding to the event when an event is received from the Java graphics controller unit308.

[Resolution of Operation Unit and App Display]

Here, a process in a case where the resolution of the operation unit106differs is described.

It is assumed that in the image forming apparatus100, an operation unit106with SVGA (800*600) resolution and an operation unit106with WSVGA (1024*600) resolution can be used. It is possible to determine which operation unit106is connected to the image forming apparatus100by the CPU101performing a query to the operation unit106upon an activation of the image forming apparatus100. Also, at this time, the information obtained from the operation unit106is stored on the RAM102or the HDD104. Note, this kind of processing is the same as device detection processing of a general information processing apparatus and so details are omitted.

Firstly, it is described how the same app is displayed in cases when two types of operation units with different resolutions are used. Note, a region where the app (the C app or the Java app) displays a target object on the operation unit106is called an “app display region”.

FIG. 4Ais a screen displayed if the operation unit106with the SVGA resolution is used, and is an example of a display of a copy app which is one of the Java apps. The entire region surrounded by a dashed line inFIG. 4Ais an app display region401and only this app is displayed on the operation unit106. That is, only the screen of the copy app is displayed on the entirety of the operation unit106.

On the other hand, if the operation unit106with the WSVGA resolution is used, this copy app is displayed as shown inFIG. 4B. The app display region401inFIG. 4Bis the same as the operation unit106with the SVGA resolution but a left menu region402is displayed on the left side of the app display region401in the case of the WSVGA operation unit106. That is, in addition to the display region where an app screen is displayed, another display region (a region where other display content is displayed) is included in the case of the WSVGA operation unit106. A change of the display content of the app (a screen switch) is executed only on the app display region401, and a change of the display content of the app is not executed on the left menu region402.

Also, the same thing is always displayed on the left menu region402, regardless of the display content of the app display region401. That is, in a case of the example inFIG. 4B, a TOP button404and a SHORT button405are always displayed. The TOP button404is a button for displaying a screen (hereinafter, an initial screen) displayed in the app display region401on the operation unit106firstly after the power of the image forming apparatus100is turned on. The SHORT button405is a button (a shortcut button) for displaying any screen set by the user on the app display region401. Note, the TOP button404and the SHORT button405are displayed here, but other buttons which are common across apps may be displayed.

In this way, by displaying the left menu region402only in a case where the operation unit106with the WSVGA resolution is used, the same display content and operability can be provided to the user using the same program for the app for both of the operation units106of the two resolutions. Note, in the case of WSVGA resolution, because the left menu region402is displayed, a difference arises in the coordinates included in the event information detected by the driver unit301when the user touches the app display region401. The processing of the event management unit302for handling this difference of the coordinate is described later usingFIG. 7. Note, the description is given using examples of SVGA and WSVGA, but it is similar even with configurations of the operation unit106having other resolutions.

Next, a description of processing of the C graphics controller unit306and the Java graphics controller unit308is given. These graphic controllers have different configurations, and event information that is required in each graphic controller is also different. Note, description is given assuming that the SVGA resolution is a standard resolution (hereinafter referred to as a standard resolution) and all apps operating on the graphic controller are created based on this the standard resolution in the present embodiment. Note, it is assumed that the resolution that is made to be the standard resolution is not limited to SVGA, and it is possible to define which resolution is made to be the standard in accordance with the relationship between resolutions.

Firstly, a description of the C graphics controller unit306is given. The information of a rendering controller used by an app of the C app unit307is stored for each app on the HDD104in a table format in advance. Hereinafter, the expression of information of rendering controllers is called a “rendering controller management table”. Here the rendering controller management table is described using a concrete example.

As illustrated inFIGS. 5A and 5B, a controller type, an ID, a region, a parent ID, and a corresponding operation, are associated and managed in the rendering controller management table. Controller type expresses the type of the rendering controller and includes window, button, list, or the like. ID is a number functioning as identification information for identifying each controller uniquely. Region is an arrangement position (display position) of the respective controller, and top-left coordinates and bottom-right coordinates are recited. Note, here X indicates a horizontal direction and Y indicates a vertical direction in coordinates (X, Y). Parent ID is the ID of a window that is a base (base window) for the laying out of the respective controller. The C graphics controller unit306according to the embodiment is described assuming that there is only one base window and that the size of the base window is always the same size as the app display region401. Corresponding operation is an operation performed when each controller is selected.

A concrete example of a corresponding operation is described. It is assumed that the information of the rendering controller used by an app (app X) is as inFIG. 5A. The rendering controller management table illustrated inFIG. 5Ais representing that a window of the ID “1000” and a window of the ID “2000” is displayed on a window (base window) of the ID “1” for a app X. Also, a button of the ID “1001” and a button of the ID “1002” are laid out on the window of the ID “1000” inFIG. 5A. If the user selects the button of the ID “1001”, a screen of the window of the ID “2000” is displayed. Also, if the user selects the button of the ID “1002”, a screen of the window of the ID “3000” is displayed. Note, a result of displaying on the operation unit106the window of the ID “1000”, the button of the ID “1001”, and the button of the ID “1002” inFIG. 5Ais displayed inFIG. 6A. Note, reference numerals shown inFIG. 6Acorrespond to the IDs inFIG. 5A.

Next, a description of coordinates of event information required by the C graphics controller unit306is given. For selection of a button displayed by the app of the C app unit307, it is necessary for the coordinates included in the event information transmitted to the C graphics controller unit306to be coordinates in a region of the selected button. All apps of the C graphics controller unit306performs a determination as to whether or not the coordinates are in the region of the button using relative coordinates. Then, the coordinates are in the region of the button, the C graphics controller unit306recognizes that the button is selected and an execution request of an operation corresponding to the button is transmitted to the C app unit307. Note, the relative coordinates are similarly used in a case of a controller other than a button.

Here, a description is given of the relative coordinates and the absolute coordinate as coordinate systems used by the respective graphics controller units. The relative coordinates are coordinates representing a position touched by the user from the origin point, when, for example, a top-left of the app display region401is made to be the origin point. On the other hand, the absolute coordinates are coordinates representing a position touched by the user from the origin point, when, for example, a top-left of the operation unit106is made to be the origin point. Therefore, if the resolution of the operation unit106is SVGA, values of the relative coordinates and the absolute coordinates are the same in the present embodiment. On the other hand, if the resolution is WSVGA, the absolute coordinates and the relative coordinates have different coordinate values. Here, if the same position of the operation unit106is indicated, coordinates in at least one of the vertical direction or the horizontal direction of the relative coordinates becomes smaller. The relative coordinate values are calculated by the event management unit302obtaining resolution information of the operation unit106from the operation unit management unit303. The detail of this obtainment processing is described later.

Next, the description of the Java graphics controller unit308will be given. The Java graphics controller unit308is a group of rendering controllers that apps of the Java app unit309can use. The rendering controller is for example a button or a window. However, the Java graphics controller unit308is a window-based controller and capable of laying out only one rendering controller on one window. Therefore, in a case where a screen for displaying2buttons asFIG. 6Bis displayed on the operation unit106, the rendering controller management table is configured as inFIG. 5B. Note, reference numerals shown inFIG. 6Bcorrespond to the IDs inFIG. 5B.

InFIG. 5B, a window of the ID “5000” is laid out on a window (base window) of the ID “1”. Also, a window of the ID “5100” and a window of the ID “5200” are laid out on the window of the ID “5000”. Additionally, a button of the ID “5101” is laid out on a window of the ID “5100” and a button of the ID “5201” is laid out on a window of the ID “5200”. A result of displaying the rendering controller management table ofFIG. 5Bon the operation unit106is shown inFIG. 6B. Note, hereinafter a window for laying out a controller such as a button is called a “child window”.

Here, a description of coordinates of event information required by the Java graphics controller unit308is given. The Java graphics controller unit308is a window-based controller and which window the event information is transmitted to is determined using both relative coordinates and absolute coordinates. However, the absolute coordinates necessary for the Java graphics controller unit308are coordinates whose origin point is made to be the top-left of the app display region and not coordinates whose origin point is made to be the top-left of the operation unit106. Therefore, unlike the C graphics controller unit306, it is necessary to create coordinates whose origin point is made to be the top-left of base window. Note, the relative coordinates are coordinates whose origin point is made to be the top-left of the child window that transmits the event, and the thinking behind the relative coordinates is the same in the C graphics controller unit306and the Java graphics controller unit308. The values of the absolute coordinates and the relative coordinates are set by the event management unit302using the operation unit management unit303and the event information changing unit304. The detail of the setting processing here is described later.

Next, processing from when event information is detected in the driver unit301up until when the event information is transmitted to the C graphics controller unit306or the Java graphics controller unit308is described usingFIG. 7. Note, a calculation of various coordinates performed during processing is explained in detail later using concrete coordinate values.

When the user performs an operation such as pressing a target object in the operation unit106, the driver unit301detects the event information in step S701. This event information includes the type of the event (a type of an operation such as a press or a release) and the coordinate information, and the coordinate information is the absolute coordinates whose origin point is the top-left of the operation unit106. Note, the approach to detecting the event information may be to use a detection approach of a general resistive type (pressure-sensitive type) or an electrostatic type operation unit, for example, and details are omitted. The driver unit301transmits the detected event information to the event management unit302. When the event management unit302receives the event information, a request to obtain information of the window being displayed on the operation unit106is transmitted to the window management unit305. Here, the window information is information indicating whether the displayed window is a window of a C app or a window of a Java app. When the window management unit305receives a request to obtain information of the window, it obtains information of the window being displayed on the operation unit106, and transmits that information to the event management unit302.

In step S702, when the window information is received, the event management unit302determines whether or not a window of a C app is being displayed. If it is determined that a window of a C app is being displayed (YES in step S702), the processing proceeds to step S703. On the other hand, if it is determined that the C app window is not being displayed (NO in step S702), that is if a Java app window is being displayed, the processing proceeds to step S705.

In step S703, the event management unit302performs processing for determining the relative coordinates. The relative coordinates are included in the event information transmitted to the window management unit305and the C graphics controller unit306. Firstly, the event management unit302transmits a request for obtaining the resolution of the operation unit106to the operation unit management unit303. When the operation unit management unit303receives the resolution obtainment request, it obtains the resolution of the operation unit106, and transmits the resolution to the event management unit302. When the event management unit302receives the resolution information, it performs a determination of the relative coordinates. Specifically, in a case where the received resolution information indicates the standard resolution (SVGA), the event management unit302determines coordinates that are the same as the absolute coordinates as the relative coordinates. On the other hand, in the case of a non-standard resolution, the event management unit302determines coordinates that take into consideration the difference between the resolution indicated by the obtained resolution information and the standard resolution as the relative coordinates. Here, the coordinates that take into consideration the difference between the resolution indicated by the obtained resolution information and the standard resolution correspond to coordinates for which the size of the left menu region402is subtracted from the absolute coordinates. Then, the processing proceeds to step S704.

In step S704, the event management unit302creates event information including the determined relative coordinates in the event information received from the driver unit301and transmits the information to the window management unit305. The window management unit305which has received the event information specifies a window of an app being displayed on the operation unit106and specifies a graphic controller including the window. Here, the window of the C app is displayed on the operation unit106, so the window management unit305transmits the event information transmitted from the event management unit302to the C graphics controller unit306. This processing flow is then terminated.

In step S705, the event management unit302performs a determination as to whether or not the display region of the operation unit106is a specific display region. In the present embodiment, the event management unit302obtains the resolution information by transmitting a resolution obtainment request to the operation unit management unit303, and determines that it is a specific display region if the resolution indicated by the received resolution information is the standard resolution. If it is determined that the resolution of the operation unit106is the standard resolution (YES in step S705) the processing proceeds to step S707, and if it is determined not to be the standard resolution (NO in step S705) the processing proceeds to step S706.

In step S707, the event management unit302determines the relative coordinates based on the coordinates (absolute coordinates) of the event information received from the driver unit301. A description of an approach to determining the relative coordinates will be given. The event management unit302transmits the absolute coordinates received from the driver unit301to the window management unit305. Then, the window management unit305transmits the coordinates to the Java graphics controller unit308. The Java graphics controller unit308searches for a window including the received absolute coordinates from out of windows being displayed on the operation unit106, and transmits origin point coordinates of the window (top-left coordinates) to the window management unit305. After reception of the origin point coordinates of the window, the window management unit305transmits them to the event management unit302. After reception of the origin point coordinates of the window, the event management unit302calculates relative coordinates using the received origin point coordinates and the absolute coordinates received from the driver unit301. Specifically, the event management unit302determines values resulting from subtracting the origin point coordinates of the window from the absolute coordinates to be the relative coordinate. In this way the event management unit302determines the relative coordinates.

On the other hand, if it is determined that the resolution of the operation unit106is not the standard resolution (NO in step S705), the event management unit302changes the absolute coordinates of the event information received from the driver unit301using the operation unit management unit303and the event information changing unit304in step S706. A description of processing for changing the absolute coordinate will be given. Firstly, the event management unit302transmits a request for obtaining the resolution of the operation unit106to the operation unit management unit303. When the operation unit management unit303receives the resolution obtainment request, it obtains the resolution of the operation unit106, and transmits obtained resolution information to the event management unit302. After reception of the resolution information, the event management unit302transmits the received resolution information and the absolute coordinates received from the driver unit301to the event information changing unit304. The event information changing unit304calculates the difference between the resolution indicated by the received resolution information and the standard resolution, then transmits the coordinates obtained using the calculated difference to the event management unit302. The event management unit302, having received this, determines the received coordinates to be the absolute coordinates. Then, the processing proceeds to step S707and the event management unit302determines the relative coordinates based on the absolute coordinates determined in step S706. That is, the event management unit302determines as the relative coordinates values obtained by subtracting the origin point coordinates of the window from the absolute coordinates determined in step S706.

After determining the relative coordinate in step S707, the event management unit302creates event information including the absolute coordinates and the relative coordinates, and transmits it to the window management unit305in step S704. After receiving the event information, the window management unit305transmits the event information transmitted from the event management unit302to the Java graphics controller unit308. The Java graphics controller unit308transmits a request to execute an operation corresponding to an app that is being displayed to the Java app unit309based on the received event information. The processing is the same as that of a general windows-based graphic system (for example an X11-based windows system), so details are omitted. This processing flow is then terminated.

Processing Example

A concrete example of the processing indicated inFIG. 7is described. Firstly, description is given of a concrete example of a C app and a Java app in a case when the operation unit106having the standard resolution (SVGA) is connected to the image forming apparatus100. After that, description is given of a concrete example of a C app and a Java app in a case when the operation unit106having a resolution (WSVGA) that is not the standard resolution is connected to the image forming apparatus100.

(Standard Resolution and C App)

Firstly, a description of a concrete example in a case when a browser app (a C app) is displayed in the operation unit106of the standard resolution will be given.FIG. 8Ais an example of displaying of the browser app displayed in the operation unit106. A URL is displayed in the upper part of the screen ofFIG. 8A, a keyword input button for inputting a search keyword is displayed in the middle part of the screen, and buttons to perform a page transition and various kinds of settings are displayed in the lower part of the screen. In this screen, a concrete example in a case when a MENU button802which is one of various kinds of setting buttons is pressed is described. Note, it is assumed that a region of the MENU button802is (220, 470) to (370, 550), similarly to the button of the ID “1001” inFIG. 5A.

If the user presses the MENU button802in the operation unit106, the driver unit301detects event information (step S701). It is assumed that this event information is that the event type is touch and the coordinates are absolute coordinates (250, 500). The driver unit301transmits this event information to the event management unit302.

The event management unit302, having received this event information, transmits a request to obtain window information to the window management unit305, and based on a result transmitted from the window management unit305, determines whether or not the window of the C app is displayed in the operation unit106(step S702). Here because it is the window of the C app (YES in step S702), the event management unit302performs processing to determine the relative coordinates (step S703). Firstly, the event management unit302transmits a resolution obtainment request to the operation unit management unit303, and obtains information of the resolution. Because it is the standard resolution here, the relative coordinates are the coordinates (250, 500) which are the same coordinates as the absolute coordinates.

Then, the event management unit302adds the relative coordinates to the event information received from the driver unit301, and transmits the information to the window management unit305(step S704). Here, the window management unit305, having received this event information, transmits the event information to the C graphics controller unit306, and furthermore the event information from the C graphics controller unit306is transmitted to the browser app. Then, the browser app performs press processing for the MENU button802, as the processing corresponding to a touch event of a relative coordinate (250, 500).

(Standard Resolution and Java app)

Next, a description of a concrete example in a case when a copy app (a Java app) is displayed in the operation unit106of the standard resolution will be given.FIG. 4Ais an example of displaying of the copy app displayed in the operation unit106. A screen title is displayed in the upper part of the screen ofFIG. 4A, setting content is displayed in the middle part of the screen, and buttons performing various kinds of setting is displayed in the lower part of the screen. Description is given of a concrete example in a case when a paper size button403which is one of various kinds of setting buttons is pressed in this screen. Note, it is assumed that a region of the paper size button403is (220, 470) to (370, 550), similarly to the button of the ID “5101” inFIG. 5B.

If the user presses the paper size button403in the operation unit106, the driver unit301detects event information (step S701). It is assumed that this event information is that the event type is touch and the coordinates are absolute coordinates (250, 500). The driver unit301transmits this event information to the event management unit302.

The event management unit302, having received this event information, transmits a request to obtain window information to the window management unit305, and based on a result transmitted from the window management unit305, determines whether or not the window of the C app is displayed in the operation unit106(step S702). Because it is the window of the Java app here (NO in step S702), the event management unit302transmits a resolution obtainment request to the operation unit management unit303, and obtains information of the resolution. Then, the event management unit302performs a determination of whether or not the obtained resolution is the standard resolution (step S705). Here because it is the standard resolution (YES in step S705), the event management unit302performs processing to determine the relative coordinates (step S707).

Specifically, the event management unit302transmits the absolute coordinates (250, 500) to the window management unit305, and the window management unit305transmits this information to the Java graphics controller unit308. Then, the Java graphics controller unit308detects the origin point coordinates of the window including the received coordinates from out of one or a plurality of windows being displayed on the operation unit106, and transmits them to the window management unit305. Here, the window of ID “5100” ofFIG. 5Bis searched, and the coordinates (220, 470) are transmitted to the window management unit305as the origin point coordinates. The window management unit305, having received these coordinates, transmits this information to the event management unit302. The event management unit302subtracts the origin point coordinates (220, 470) from the absolute coordinates (250, 500), and determines the relative coordinates to be the coordinates (30, 30). Then, the event management unit302transmits the event information including the absolute coordinates and the relative coordinates to the window management unit305(step S704). This event information, is further transmitted to the Java graphics controller unit308and the copy app, and the copy app performs press processing for the paper size button403, as the processing corresponding to the touch event of the absolute coordinates (250, 500) and the relative coordinates (20, 30).

(Non-Standard Resolution and C app)

Next, a description of a concrete example in a case when the browser app (a C app) is displayed in the operation unit106of the standard resolution will be given.FIG. 8Bis an example of displaying of the browser app displayed in the operation unit106. The display content of the app display region401in this screen is the same of the display content of the app display region401inFIG. 8A. In this screen, a concrete example in a case when a MENU button802which is one of various kinds of setting buttons is pressed is described. Note, it is assumed that a region of the MENU button802is (220, 470) to (370, 550), similarly to the button of the ID “1001” inFIG. 5A.

If the user presses the MENU button802in the operation unit106, the driver unit301detects event information (step S701). It is assumed that this event information is that the event type is touch and the coordinates are absolute coordinates (474, 500). The driver unit301transmits this event information to the event management unit302.

The event management unit302, having received this event information, determines whether or not the window of the C app is displayed in the operation unit106(step S702). The detail of this processing is same as in the case of the previously described standard resolution, so the description is omitted. Here because it is the window of the C app (YES in step S702), the event management unit302performs processing to determine the relative coordinates (step S703). Firstly, the event management unit302transmits a resolution obtainment request to the operation unit management unit303, and obtains information of the resolution. Here because it is not the standard resolution, values for which a difference of the obtained resolution and the standard resolution are considered are made to be the relative coordinates.

Specifically, if the obtained resolution is WSVGA (1024*600) and the standard resolution is SVGA (800*600), the x-coordinate of the obtained resolution is 224 larger. Thus, 224 is subtracted from the x-coordinate of the absolute coordinates (474, 500), and the relative coordinates are determined to be the coordinates (250, 500). After the relative coordinate determination, the event management unit302adds the relative coordinates to the event information received from the driver unit301, and transmits the information to the window management unit305(step S704). The processing until pressing of the MENU button802after this is the same as in the case of the previously described standard resolution, so description thereof is omitted.

(Non-Standard Resolution and Java app)

Finally, a description of a concrete example in a case when a copy app (a Java app) is displayed in the operation unit106of the standard resolution will be given.FIG. 4Bis an example of displaying of the copy app displayed in the operation unit106. The display content of the app display region401in this screen is the same of the display content of the app display region401inFIG. 4A. Description is given of a concrete example in a case when the paper size button403which is one of various kinds of setting buttons is pressed in this screen. Note, it is assumed that a region of the paper size button403is (220, 470) to (370, 550), similarly to the button of the ID “5101” inFIG. 5B.

If the user presses the paper size button403in the operation unit106, the driver unit301detects event information (step S701). It is assumed that this event information is that the event type is touch and the coordinates are absolute coordinates (474, 500). The driver unit301transmits this event information to the event management unit302.

The event management unit302, having received this event information, determines whether or not the window of the C app is displayed in the operation unit106(step S702). The detail of this processing is same as in the case of previously described the standard resolution, so the description is omitted. Here because it is the window of the Java app (NO in step S702), the event management unit302obtains information of the resolution, and performs a determination as to whether or not it is the standard resolution (step S705). The detail of this processing is same as in the case of previously described the standard resolution, so the description is omitted. Here because it is not the standard resolution (NO in step S705), the event management unit302performs a change of the absolute coordinates of the event information (step S706).

Specifically, the event management unit302transmits the resolution (1024*600) which is the information of the resolution from the operation unit management unit303and the absolute coordinate (474, 500) which is received from the driver unit301to the event information changing unit304. The event information changing unit304received this calculates that the difference between the received resolution (1024*600) and the standard resolution (800*600) is 224 for the x-coordinate. Then, the event information changing unit304changes the coordinates (250, 500) to absolute coordinates by subtracting the calculated224from the x-coordinate of the absolute coordinates. Furthermore, the event information changing unit304transmits these changed absolute coordinates to the event management unit302. The event management unit302makes these absolute coordinates be the absolute coordinates of the event information. Then, the event management unit302transmits the event information to the window management unit305(step S704). With this, the press processing for the paper size button403of the copy app is performed, but these processes are the same as in the case of the previously described the standard resolution, so the description is omitted.

As described above, the coordinates of a detected event are changed in accordance with the window of the app which is being displayed and the resolution of operation unit. With this, it is possible to cause apps to operate normally without changing already existing apps even in an image forming apparatus comprising a plurality of the graphics controller units which operates using different event information.

In the present embodiment, a description of an example using operation units whose resolution is SVGA or WSVGA is given as a concrete example of determination of the display region. However, it is possible to apply the present embodiment to operation units of other resolutions. Also, depending on the relationship between the resolutions, configuration may be such that not only the x-coordinate but also the y-coordinate is changed in the change (step S706) of the absolute coordinates included in the event information. With this, whatever the resolution of the operation unit, the app is able to perform the operation correctly without remaking the existing app.

In the present embodiment, description is given using examples of the C graphic controller and the Java graphic controller, but limitation is not made to this. If there are a plurality of graphic controllers for which conversion of event information (absolute coordinates/relative coordinates) is required, a graphic controller other than those described above may also be used. Additionally, in a case when the information processing apparatus has three or more graphic controllers, configuration may be taken so as to change the coordinates in accordance with the characteristics of each graphic controller.

In the present embodiment, the top-left of the window is made to be the origin point, but a reference point other than the top-left may be generated as the origin point, and coordinates from this reference point may be used. With this, even in a system in which the origin point of position information of the system is not the top-left but the bottom-right, the app is able to operate correctly without remaking the conventional app.

Second Embodiment

In the first embodiment, in the determination of the display region (step S705inFIG. 7), the determination is performed using the resolution of the operation unit106. However, the determination may be performed using information of the operation unit106other than the resolution. In the present embodiment, description will be given of an embodiment in which a window size of an app being displayed is used in the determination of the display region. Note, basic processing and control is the same as in the first embodiment, so description will be given only for points that differ from the first embodiment.

UsingFIGS. 9A and 9B, description will be given of an example in which the window size (the app region) of the app is used in the determination of the display region (step S705).FIG. 9Ais a screen in which an app is displayed in an operation unit106whose resolution is SVGA similarly toFIG. 4A. On the other hand,FIG. 9B, similarly toFIG. 4B, displays the left menu region402, and the display region of the app is made to be a screen that is an app display region901by aligning to the resolution of SVGA.

In the determination of the display region (step S705), the event management unit302makes a request to obtain a window size of an app being displayed in the operation unit106to the window management unit305. In a case of the display on the operation unit106illustrated inFIG. 9A, the window size of the app display region401is the same as the display region of the operation unit106. In such a case, the display region is treated as a specific display region (YES in step S705). Thus, the event management unit302, performs (step S707) processing to determine the relative coordinates without changing the absolute coordinates included in the event information. On the other hand, in a case of the display on the operation unit106illustrated inFIG. 9B, the window size of the app display region401is different to the display region of the operation unit106. That is, the app display region401is a display region that is smaller than the whole screen. In that case, the display region is treated as not being a specific display region (YES in step S705). Thus, the event management unit302performs (step S706) processing to change the absolute coordinates included in the event information and then performs (step S707) processing to determine the relative coordinates. Subsequent processing is the same as in the first embodiment, so the description is omitted.

As described above, by using the window size of the app in the determination of the display region, it becomes possible for an existing app to operate correctly without remaking the app in a case when the resolution of the operation unit106is same but the window size of the app is different.

Other Embodiments

This application claims the benefit of Japanese Patent Application No. 2016-031461, filed Feb. 22, 2016, which is hereby incorporated by reference herein in its entirety.