Patent Description:
Manually opening and moving faceplates and/or views and/or popups to a desired position by an operator is slow and/or can waste time of an operator working at an industrial workplace, particular when fast reaction is required in critical situations.

<CIT> discloses management of overlaps of windows to prevent covering important information on the screen.

<CIT> discloses a drawing data generating unit, a variation pattern that varies at regular intervals to be displayed, a display unit that displays drawing data, and a comparator that compares whether input signals are coincident.

<CIT> discloses monitoring of objects in the Internet of things in real time and potentially outputting an alarm, potentially with a parameter state of an alarm device.

<CIT> discloses a device for controlling home automation equipment of a building.

<CIT> discloses a graphical trend monitoring system with zoom function for use in a process control system.

Accordingly, the present invention is related to a method for generating a series of content areas for presentation at a display screen, a method to assign a series of content areas, a first processing unit, a use of the first processing unit for generating a series of content areas, a computer program and a computer readable medium as described in the independent claims.

Advantageous modifications of the invention are stated in the dependent claims.

In this entire description of the invention, the sequence of procedural steps is presented in such a way that the process is easily comprehensible. However, the skilled person will recognize that many of the process steps can also be executed in a different order and lead to the same or a corresponding result. In this sense, the sequence of the process steps can be changed accordingly. Some features are provided with counting words to improve readability or to make the assignment more clear, but this does not imply the presence of certain features. The term "configured to" and equivalent can mean that something is implemented and/or realised and/or modified and/or built and/or adapted to provide a specific functionality.

To achieve these and other advantages and in accordance with the purpose of the invention, as embodied and broadly described herein, there is provided a method for generating a series of content areas for presentation at a display screen, wherein the series of content areas is related to an overall machine-related-status of a machine system, as defined in claim <NUM>.

Using other words, the method for generating a series of content areas can detect circumstances about an overall machine-related-status and generate meaningful popup windows and/or views, as part of a series of content areas, at specific positions at the workplace. The method enables to make a recommendation of a position where a faceplate and/or view and/or a pop-up, as part of a series of content areas, is opened at a specific display and/or device, which is typically used by a user respectively an operator, within a recognized context describing the machine-related-status.

In one sense, the method can recommend additional relevant faceplates and/or views and/or popups as examples of content areas that can be generated to be presented at a display screen. That means, if a computer program is comprising instructions that, when the computer program is executed by the HMI workspace executes the method for generating the series of content areas, the display screen of the HMI workplace will present a series of content areas at a required position every time the overall machine-related-status is similar to a stored overall machine-related-status in the first database-system.

Advantageously, the method for generating a series of content areas can increase the effectiveness of an operator when working on an industrial workplace. This can reduce a time to take action and can increase safety based on a faster reaction, particularly in critical situations. Additionally, the method for generating a series of content areas improves the usability of the first processing unit and can be adapted to an operator's behavior and expectations, to increase productivity.

According to an aspect, the machine-system is one of a plurality of machine-systems, and wherein preferably the plurality of machine-systems can be coupled for communication by a distributed process control system.

According to an aspect, the respective position of each content area of the series of content areas is specific for the actual overall machine-related status.

According to an aspect, at least one content area of the series of content areas is generated, and preferably presented at a display screen, to operate a machine-system, particularly by a user.

According to an aspect, the content area includes an operating area to provide at least one control element and/or a process data visualisation area and/or a selection menu area and/or an alarm band visualisation area and/or a trend view and/or a process graphic and/or a faceplate for control of the machine-system.

According to an aspect, the comparing of the actual provided overall machine-related-status with the stored plurality of overall machine-related-status is based on a calculation, whether a similarity value of the actual provided overall machine-related-status and one of the stored plurality of overall machine-related-status is within boundary values for comparing.

Using other words, all interactions of the user with the HMI workplace are collected together with contextual information about the machine-system and machine-operation related information. This includes the current condition of the machine-system and the HMI workplace, but can also incorporate the situation of the active user, e.g. which display is inside the operators' field of view.

According to an aspect, the overall machine-related-status includes a machine-system status and a machine-operation related information.

According to an aspect, the machine-operation related information includes operational context information of the machine-system and/or device information of a device driving the display screen and/or login information of the device driving the display screen and/or login user information of a user, which is logged-in at the device driving the display screen.

Particularly, the series of content areas can be presented at a multitude of display screens and/or a display of different types of devices.

According to an aspect, the login information includes an identifier of the user of the device driving the display screen and/or a type of the device operated by the user.

According to an aspect, the operational context information of the machine system includes device information driving the display screen, particularly which is operated by a user and/or operational user information and/or individual user information.

According to an aspect, the operational user information includes a time of a day and/or a day of the week and/or name of the shift the user of the device driving the display screen is participating and/or a number of operators actual present in a control room for controlling the plurality of machine-systems.

According to an aspect, the individual user information includes an identifier of the user and/or a professional role of the user and/or professional duties of the user and/or a video of the user, particularly for determining a mood of the user and/or for determining a stress level of the user and/or a detected stress level of the user, preferably including a heart rate of the user and/or currently active views and/or a currently actively operated device, which is preferably determined by a last interaction with the device driving the display screen and/or an actively used display screen, which preferably is determined by tracking eyes of the user and/or determined by motion detection of the user.

According to an aspect, the machine-system is part of a plurality of machine-systems and the machine-system status includes a distributed-process-control-system state of the machine-system and/or process values of the machine-system and/or current alarms of the machine-system and/or process values deviations of the machine-system.

According to an aspect, the plurality of machine-systems are communicatively coupled to a distributed-process-control-system and particularly an actual status of the at least one machine system is provided by the distributed process control system.

According to an aspect, the method can further include steps to assign a series of content areas, which are generated for presentation at a display screen, to an overall machine-related-status of a machine-system, is proposed including:.

Using other words the method to assign a series of content areas to an overall machine-related-status collects information on where and when a user opens specific series of content areas, as e.g. views or popup windows, at his workplace.

The method to assign a series of content area to an overall machine-related-status collects also additional information that represents the situation, when the GUI interaction, as presented at the display screen, happened.

The method captures interactions with series of content areas as e.g. faceplates and/or views and/or popups and correlates them with contextual process information as included in the overall machine-related-status.

In case there is a maximum number of content areas, as e.g. faceplates and/or views and/or popups, that can be opened on a display screen, e.g., because of small screen size on tablets, the algorithm can, based on the analysis of past behavior, "decide" what part of a series of content areas should be replaced, i.e., closed in favor of a new instance of content area.

This results in, that based on the method as described above, the method is able to "understand" the information needed by a human being to analyze a certain situation in respect to the overall machine-related-status. The method is set up in such a way to be able to learn how to "reason", in respect to a series of content areas generated, in such situations.

Such a detailed logging of user inputs and interactions including generated series of content areas for presentation at a display screen, respectively at a GUI, enables to capture training data reflecting user's behavior while optionally considering the system context.

Such system context, respectively machine-system status, e.g. can include active alarms and/or process values and/or currently active views and/or time of the day and/or operator's mood and/or a facial expression of the user and/or a stress level of the user and/or a heart rate of the user and/or a currently active device, as determined by a last interaction with a processing unit including a display screen and/or an active monitor, which can be determined by tracking eyes to determine a direction where the user is looking to and/or a motion detection, if the user is moving.

The correlation between the logged plurality of series of content areas generated and the linked overall machine-related-status can be determined using learning algorithms, as e.g. statistical analysis and/or pattern recognition and/or Artificial Neural Networks and/or machine learning, etc..

According to an aspect, the method can, based on a user input and the overall machine-related-status either directly generate, and preferably display, a series of content areas assigned to the overall machine-related-status on a most suitable device, and most suitable viewport on the device or make a proposal to the user to generate the series of content areas respectively.

According to an aspect, the method for generating a series of content areas suggests a series of content areas to be generated to be displayed on a display screen, particularly e.g. by using a dialog and/or by proposing the target layout of views using transparency.

According to an aspect, the method for generating a series of content areas for presentation at a display screen and the method to assign a series of content areas operate concurrently.

Using other words, a learning and training mode to assign a series of content areas to an overall machine-related-status of a machine-system can run concurrently with the method for generating a series of content areas for presentation at a display screen at runtime. In the training mode, the method can log a state of the controlled process and assign a sequence of GUI actions, as an example for a series of content areas, and/or include operational context and/or user context and/or the used device and/or an interaction with/at a workstation, as an example for machine-operation related information.

According to an aspect, the first database-system is equal to the second database-system.

According to an aspect, the second database system is configured to capture interactions of a user operating a user interface of the device driving the display screen, particularly to collect information of the overall machine-system-status.

According to an aspect, the series of content areas includes content areas selected by a user of the device driving the display screen and an individual rearrangement of a position of each of the content areas selected by the user.

According to an aspect, the correlation of the logged plurality of series of content areas generated for presentation at the display screen and the linked overall machine-related-status is determined based on a statistical analysis and/or a pattern recognition method and/or a principal component analysis method to filter most significant correlations and/or comprises utilization of an unsupervised learning algorithm to train a neural network to identify appropriate correlating relationships.

Using other words, determining correlations of the logged plurality of series of content areas generated and the linked overall machine-related-status is to find a probability that a specific first content area, i.e. a trigger action, is executed at a specific overall machine-related-status of the machine-system. Additionally or alternatively a probability is determined that a user, which started a series of content areas with a specific first content area, will complete the series of content areas as before, particularly if a similar overall machine-related-status is currently present.

Preferably, to determine the correlation, it can be counted how many times the same series of content areas is generated at a similar overall machine-related-status versus a sum of other overall machine-related-status of the machine-system with the same trigger action. That means it can be determined for a plurality of similar overall machine-related-status how often comparable series of content areas are generated to be presented at the display screen.

A comparable series of content areas can be determined by comparing the number of content areas within a series of content areas and/or the specific content areas included in the series of content areas and/or the quadrants of the screen where the individual content areas of the series of content areas are generated.

The first content area of the series of content areas, i.e. the trigger action, can be more important for determining a comparable series of content areas than the order of the content areas within the series of content areas. That means that the number of content areas and the type of the content areas can be used to determine a comparable series of content areas.

According to an aspect, the determination of correlations and assigning of the logged plurality of series of content areas generated for presentation at the display screen with the linked overall machine-related-status of a machine-system comprises:.

According to an aspect, the similarity value is determined using a probability metric. Such a probability metric can measure, e.g., how frequently a user generated a series of content areas in a respective overall machine-related-status in the past, based on probability calculations.

According to an aspect, each content area of the action series of content areas is removed from the series, if the respective content area is redundant.

Using other words, for assigning a series of content areas to an overall machine-related-status of a machine-system raw data of the logged plurality of series of content areas generated for presentation at the display screen and the respective overall machine-related-status are stored respectively in the first database-system and the second database-system are post-processed, where redundant content areas within the series of content areas are removed, as e.g. obviously erroneous actions.

Primarily based on timely correlating series of content areas are formed including a common specific first content area, that means a trigger action, as e.g. "a user opened a view with a process graphic A" and related follow-up actions, e.g. "a user moved the view to screen <NUM>" and "a user opened a trend view".

Using other words, the series of content areas or the grouped content areas are assigned to a sub-set of related overall machine-related-status, particularly to relevant machine-system status and stored in the second database-system, which can be named state-actions database.

Additionally or alternatively, the machine-system status can be filtered in respect to relevance for assignment to the series of content areas.

A relevance of a subset of machine-system status can be determined based on the time of the actions taken place, as e.g. alarms and/or process values, and relationships amongst machine-systems and/or machine-subsystems involved. The machine-system status can include status values logged prior to a point in time where the user generates a first content area for presentation at the display screen.

According to an aspect, the similarity value of a selected relation of overall machine-related-status with an action series of content areas is overwritten by the user for compensation of a misleading determined correlation.

Using other words, the method to assign a series of content areas includes a feedback functionality to be used by a user to directly improve the assigned series of content areas with the overall machine-related-status by modifying and/or overwriting the similarity value manually, to provide direct feedback on the appropriateness of follow-up actions as defined by an assigned series of content areas. If a user was unhappy with the system response in form of the series of content areas assigned to the overall machine -related-status, e.g., because there was a substantial change in the controlled process, the similarity value can be directly affected by the user.

Such a direct feedback method can be used to directly improve the assignment of the overall machine-related-status to a series of content areas.

A first processing unit is proposed according to claim <NUM>.

A use of a first processing unit for generating a series of content areas to be presented at a display screen is proposed according to claim <NUM>.

A computer program is proposed, comprising instructions that, when the computer program is executed by a computer, cause the computer program to execute a method as described above.

Advantageously, such a computer program enables to transfer the method easily to different computer systems.

A computer readable medium is proposed, having stored the computer program described above.

Advantageously, this enables to transfer the method easily for different applications.

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiments of the invention and together with the description serve to explain the principle of the invention. The drawings display:.

<FIG> sketches schematically a machine-system <NUM> signally coupled to a processing unit <NUM>, which can be included in a user workspace and coupled to a display screen <NUM> with different kinds of content areas: including an alarm band <NUM>, a first trend area <NUM>, a second trend area <NUM>, a faceplate <NUM> and a manual area <NUM>.

<FIG> sketches schematically a flow diagram of data exchanged within a distributed process control system <NUM>.

A second database-system <NUM> is configured to store data of a plurality of machine-systems <NUM> and related machine-operation related information, which can include related data of a user <NUM>, and related data provided by a HMI workplace <NUM>. The stored data of the second database-system <NUM> can be analyzed, using a second processing unit <NUM> to generate a plurality of series of content areas, which are assigned to an overall machine-related-status, and stored in a first database-system <NUM>. A user at the HMI workspace <NUM> can initiate a generation of a series of content areas, which are assigned to the actual overall machine-related-status of the machine-system <NUM>. The user <NUM>, who can operate at the HMI workspace <NUM>, can change a specific assignment of an overall machine-related-status with a series of content areas stored in the first database-system <NUM>.

<FIG> sketches schematically a layout of a display screen <NUM> with presented content areas <NUM>, <NUM>, <NUM>, <NUM> and <NUM>, including an alarm band <NUM>, a first trend area <NUM>, a second trend area <NUM>, a faceplate <NUM> and a manual area <NUM>.

The upper part of <FIG> sketches schematically a flow diagram <NUM> of a method to assign a series of content areas, which are generated for presentation at a display screen of a human machine interface (HMI) of a workplace <NUM>, to an overall machine-related-status of a machine-system <NUM>.

A plurality of series of content areas generated for presentation at the display screen of the HMI <NUM>, including a respective position for presentation at the display screen of the HMI <NUM> of each of the content areas can be logged using a third database-system <NUM>.

Optionally, the plurality of series of content areas can be formed, by grouping consecutive generated content areas for the display screen, based on a common specific first content area, that means a trigger action, as e.g. "a user opened a view with a process graphic A" and related follow-up actions, e.g. "a user moved the view to screen <NUM>" and "a user opened a trend view".

Optionally, the plurality of series of content areas can be processed in step <NUM> to remove redundant content areas within each of the plurality of series of content areas, as e.g. obviously erroneous actions to generate a content area, as e.g. a content area which is opened and shortly afterwards closed.

A plurality of machine-system status of the overall machine-related-status of the machine system <NUM> can be logged using a second database-system <NUM>. Machine-operation related information <NUM>, including device information and/or operational user information and/or individual user information, as provided by the HMI <NUM>, is logged and stored respectively with a related machine-system status in the second database-system <NUM>, which is coupled to the machine-system <NUM>, resulting in a plurality of overall machine-related-status, which are stored in the second database-system <NUM>.

Optionally, at step <NUM> the machine-system status can be filtered in respect to relevance for assignment to the series of content areas.

At step <NUM>, the plurality of series of content areas and the, particularly timely, related plurality of overall machine-related status, which are respectively current, when the series of content areas are generated, are linked to each other and stored in a fourth database-system <NUM>.

At step, <NUM> correlations of the logged plurality of series of content areas generated and the linked overall machine-related-status are determined, particularly using a second processing unit.

Overall machine-system-status with linked respective series of content areas are determined, including the respective position of each of the content areas generated, which are correlating, are assigned to each other. These correlating series of content areas generated, including the respective position, and respectively assigned overall machine-related-status are stored using a first database-system <NUM>.

The correlation of the logged plurality of series of content areas generated for presentation at the display screen and the linked overall machine-related-status can be determined based on a statistical analysis and/or a pattern recognition method and/or a principal component analysis method to filter most significant correlations and/or comprises utilization of an unsupervised learning algorithm to train a neural network to identify appropriate correlating relationships.

The correlation can be measured by a probability metric, which is determined for each trigger action, which is linked to a comparable overall machine-related-status.

The lower part of <FIG> sketches schematically a flow diagram <NUM> of a method for generating a series of content areas for presentation at a display screen.

If a user of the HMI workplace <NUM> initiates a generation of a series of content areas for presentation at a display screen, an actual overall machine-related-status is provided at, an input of a first processing unit is provided by the machine-system and particularly by the HMI workplace, which is configured to provide machine-operation related information. The first processing unit, which can be a part of the HMI workplace <NUM>, is accessing the first database-system <NUM>, wherein the first database-system <NUM> is storing a plurality of series of content areas, including a respective position of each of the content areas, for presentation at a display screen of the HMI workplace <NUM> assigned to the respective overall machine-related-status. The first processing unit is configured to determine the series of content areas related to the actual overall machine-related-status of the machine-system <NUM>, by comparing the actual provided overall machine-related-status with the stored plurality of overall machine-related-status, which are assigned to the respective series of content areas, for selecting the series of content areas assigned to the actual overall machine-related-status. Further, the first processing unit can be configured to generate the selected series of content areas, including a respective position of each of the content areas, for presentation the series of content areas related to the actual provided machine-system status at the respective position at the display screen, preferably of the HMI workplace <NUM>.

Claim 1:
A method (<NUM>) for generating a series of content areas for presentation at a display screen (<NUM>), wherein the series of content areas is related to an overall machine-related-status of a machine-system (<NUM>), comprising:
providing an actual overall machine-related-status at an input of a first processing unit;
accessing a first database-system (<NUM>) by means of the first processing unit, the first database-system (<NUM>) storing a plurality of series of content areas including a respective position of each of the content areas for presentation at a display screen (<NUM>) assigned to the respective overall machine-related-status;
determining the series of content areas related to the actual overall machine-related-status, by comparing the actual provided overall machine-related-status with the stored plurality of overall machine-related-status, which are assigned to the respective series of content areas, for selecting the series of content areas assigned to the actual overall machine-related-status, by means of the first processing unit; and
generating the selected series of content areas, including a respective position of each of the content areas, by means of the first processing unit (<NUM>), for presentation of the series of content areas related to the actual provided machine-system status at the respective position at the display screen (<NUM>).