EXPERTISE AND EVIDENCE BASED DECISION MAKING

A method, system, and computer program product are disclosed for implementing enhanced expertise and evidence based decision making in knowledge-based applications. Expertise and evidence based decision making operations include differentiating between an average user and an expert user, and using real time feedback from expert users to update and embed expert knowledge into a predefined baseline command sequence model for a given task or problem.

BACKGROUND

The present invention relates to the data processing field, and more specifically, to a method, system and computer program product for expertise and evidence based decision making in knowledge-based applications.

Computer knowledge-based applications can provide automated decision support for users for various application fields and contexts. Some knowledge-based applications use time of error data for diagnosing user or customer problems. However, a tremendous amount of knowledge and expertise are required to navigate and find relevant information in a system time of error data dump. Users of knowledge-based applications for various application fields struggle in diagnosing problems that are new to the user or not explored frequently.

SUMMARY

Embodiments of the present disclosure are directed to enhanced expertise and evidence based decision making in knowledge-based applications. A non-limiting example computer-implemented method for expertise and evidence based decision-making includes differentiating between an average user and an expert user and using expert knowledge in baseline command sequences for given tasks.

One non-limiting example computer implemented method includes generating baseline command sequence models based upon commands entered and frequencies of the commands for given tasks. Command sequences entered by the expert user are monitored. When an expert entered command that diverges from the baseline command sequence model for the given task or problem is identified and the expert user is prompted to input a confidence level for the command that diverges from the baseline command sequence model and evidence supporting the diverging command. A new decision point is embedded into the baseline command sequence model using the expert user entered confidence level and supporting evidence. Recalibrating the baseline command sequence model for the given task is based upon the expert user entered confidence level and supporting evidence. Recalibrating the baseline command sequence model further comprises identifying and factoring an expertise level of the expert user, and identifying and factoring baseline command sequence model frequencies. A recommended new baseline command sequence model is generated based upon the expert user entered confidence level and supporting evidence.

Other disclosed embodiments include a computer system and computer program product for performing expertise and evidence based decision making implementing features of the above-disclosed method.

DETAILED DESCRIPTION

In accordance with features of embodiments of the disclosure, improved computing system operations for expertise and evidence based decision making comprises differentiating between average users versus an expert user and using expert knowledge to update and embed expert knowledge into baseline command sequence models. Expert knowledge includes an expert user entered confidence level, and expert user entered supporting evidence. A non-limiting method for expertise and evidence based decision making comprise monitoring command sequences entered by the expert users for a given task and identifying a command entered by the expert user that deviates from a baseline command sequence model for the given task. A new decision point for command entered by the expert user that deviates from a baseline command sequence model is embedded into the baseline command sequence model for the given task. Computing system operations to perform recalibration of the baseline command sequence model for a given task are based upon the expert entered confidence level, and expert supporting evidence. Recalibration of the baseline command sequence model further comprise factoring command sequence frequencies, and an expertise level of the expert user. Computing system operations for expertise and evidence based decision making logic provide updated baseline command sequence models for given tasks that minimize use of computer processing resources and minimize processing time for the given tasks

With reference now toFIG.1, there is shown an example computing environment100. Computing environment100contains an example of an environment for the execution of at least some of the computer code involved in performing inventive expertise and evidence based decision-making methods at block180, such as Expertise And Evidence Based Decision-Making (EEDM) logic182, baseline command sequences and frequencies184, expert user profile, expert entered confidence level and supporting evidence186, and scripts/models, decision points and recommendation values188used to embed expert knowledge and update a baseline command sequence. In addition to block180, computing environment100includes, for example, computer101, wide area network (WAN)102, end user device (EUD)103, remote server104, public cloud105, and private cloud106. In this embodiment, computer101includes processor set110(including processing circuitry120and cache121), communication fabric111, volatile memory112, persistent storage113(including operating system122and block180, as identified above), peripheral device set114(including user interface (UI) device set123, storage124, and Internet of Things (IoT) sensor set125), and network module115. Remote server104includes remote database130. Public cloud105includes gateway140, cloud orchestration module141, host physical machine set142, virtual machine set143, and container set144. Computer101may take the form of a desktop computer, laptop computer, tablet computer, smart phone, smart watch or other wearable computer, mainframe computer, quantum computer or any other form of computer or mobile device now known or to be developed in the future that is capable of running a program, accessing a network or querying a database, such as remote database130. As is well understood in the art of computer technology, and depending upon the technology, performance of a computer-implemented method may be distributed among multiple computers and/or between multiple locations. On the other hand, in this presentation of computing environment100, detailed discussion is focused on a single computer, specifically computer101, to keep the presentation as simple as possible. Computer101may be located in a cloud, even though it is not shown in a cloud inFIG.1. On the other hand, computer101is not required to be in a cloud except to any extent as may be affirmatively indicated.

Referring toFIG.2, there is shown a flow chart illustrating example operations of a computer-implemented method200of one or more embodiments of expertise and evidence based decision-making. Method200may be implemented with computer101for example, with operations of method200performed by EEDM logic182including generating and storing baseline command sequences and frequencies184, expert user profiles, expert user entered confidence level and supporting evidence186, and scripts/models, decision point and recommendation values188. EEDM logic182enables updating and embedding expert user knowledge into a recommended baseline command sequence model using real time feedback from expert users.

Referring also toFIG.3, EEDM logic182can be implemented using an illustrated command tree300.FIG.3illustrates a tree300of commands together with example expert user application data302. The example application expert user data302illustrates CPU usage breakdown of an example server data dump for a job ASID including an expert user JOB A. Command Tree300includes blocks or command-line tools: STATUS304, DUMPINFO306, PERFDATA,308, SUMM FO,310, and SYSTRACE, JOB A.312. As shown inFIG.2, at a block202, EEDM logic182generates baseline command sequences models for given tasks based upon command sequence that are most frequently used for a given problem or task. EEDM logic182generates the baseline command sequences from frequencies of commands issued by expert users and average users, based upon problem type and area of expertise for a given task.

At block204, EEDM logic182identifies the expert user's skill level, and EEDM logic182monitors command sequences entered by an expert user. EEDM logic182identifies the user's skill lever, for example from user profiles including a variety of factors, such as number of cases solved, years of experience, and recommendations of others. The user's skill level can be manually entered and validated. The user's skill level varies based upon various application fields and contexts of a given task.

At block204, EEDM logic182monitors command entered by expert users for a given task to identify a command entered by an expert user that diverges from the baseline command sequence model for a given task, i.e., a command that deviates from expected command sequence model. For example, blocks STATUS,304, DUMPINFO306, and PERFDATA,308of tree300can be used in identifying the deviating command at block204, for example with DUMPINFO306providing data dump information for the deviating command, and PERFDATA,308providing performance data for the deviating command, such as shown for the example application302.

At block206EEDM logic182prompts the expert user for input of a confidence value and supporting evidence for the command that diverges from the baseline command sequence model, i.e., if the new command/sequence will be helpful, for example on a numerical scale between 0 and 100%. At block206EEDM logic182prompts the expert user to input supporting evidence for their command that diverges from the baseline command sequence model, such as evidence captured by the expert user that indicates why the expert user entered the diverging command. For example, the supporting evidence may be identified from data dump information for the command that diverges from the baseline command sequence model at block DUMPINFO306inFIG.3.FIG.5further illustrates and describes operations performed at blocks204and206of method200.

At block208, the confidence level and supporting evidence entered by the expert is used to build a script/model that is added at this deviating command step into the baseline command sequence model to be detected by users in future instances of the baseline sequence model. At block210, a new decision point is added for the deviating command into the baseline command sequence model for the given task, i.e., the new decision point is embedded into the baseline command sequence model at the point of the diverging command.

At block210, EEDM logic182recalibrates of the baseline command sequence model to determine if the baseline command sequence model for the given task could be improved. SUMM FO,310for example provides frequencies of commands issued by expert users and average users that are used at block212to recalibrate and possibly update the baseline command sequence models generated at block202and SYSTRACE, JOB A,312records and reports selected expert user values for the expert user JOB A over a defined time period. EEDM logic182recalibrates the baseline command sequence model using weighted frequencies, expertise level expert confidence, and expert supporting evidence, such as shown inFIG.4.

Referring also toFIG.4, weighted components400are shown that are measured and used to identify a record or recommendation value. Identified recommendation values are used to selectively update a baseline command sequence model. Weighted_frequency402is measured by commands entered over time and bucketized in a backend database stored in baseline command sequences and frequencies146in system memory106. Weighted_expertlevel404can be automatically assigned by a user profile, for example, including expert information of how many problems solved, years of experience, component area versus problem area. In addition, the weighted_expertlevel404can be manually entered and the manual entry can be validated. A weighted_expertconfidence406is a user-inputted value when user entered command deviating from a sequence entered by most users. The weighted user entered expert confidence406is a numerical value, for example in a range between 0 and 100, in response to a computer generated prompt, and factored to provide the weighted expert confidence406. The weighted_expertevidence408is a user-inputted response providing supporting documentation or evidence for the command deviating from a baseline sequence. A repository of supporting evidence captured can be compared where available (e.g. indicated by certain bit that is off or on) to provide the weighted_expertevidence408. Adding the values: weighted_frequency, weighted_expertlevel, weighted_expertconfidence, weighted_expertevidence values together can provide a numerical value representing the recommendation value.

Shown below command process tree300inFIG.3are example values of frequencies, expert level, expert confidence level and expert evidence values scaled with selected factors to provide weighted frequencies, weighted expert level, weighted expert confidence and weighted expert evidence. The weighted values are used to calculate an illustrated record or recommendation value. Example recommendation value are shown that are calculated using the example weighted frequencies, weighted expert level, weighted expert confidence and expert evidence.

At block212, EEDM logic182recalibrates the baseline command sequence model using weighted frequencies402, weighted expertise level404, weighted expert confidence406and weighted expert evidence408. In generating a new recommended baseline command sequence model with improved decision making capabilities, the weighted expert confidence and expert evidence are factored into a mathematical value used to create an updated baseline command sequence for future instances of the given task. The weighted frequencies and weighted expertise level also are factored into the mathematical value used to create the updated command sequence.

Referring now toFIG.5, method500further illustrates and describes operations performed at blocks204and206of method200ofFIG.2. EEDM logic182at block502monitors issued commands of command sequences entered by an expert user for given tasks, such as using command block STATUS304inFIG.3. . . EEDM logic182identifies a decision point for a command entered by the expert user that diverges from a baseline command sequence model for a given task at block504. EEDM logic182, at block506collects an expert user entered confidence level value, and expert user entered evidence supporting the expert user's decision for the command that diverges from the baseline command sequence model. The supporting evidence may be found in data dump information that points to data corruption, such as identified using command blocks DUMPINFO306and PERFDATA308inFIG.3. EEDM logic182, at block508identifies an expert user's skill level or expertise level, such as from a user profile for the expert user identified at block180and stored with expert user profile, expert entered confidence level and supporting evidence186. An expert user's expertise level may be identified from an expertise level entered by the expert user that can be validated. EEDM logic182, at block510calculates weighted frequencies, weighted expertise level, weighted expert confidence and expert evidence, as illustrated described with respect toFIG.4. EEDM logic182, at block512calculates a recommendation value factoring the weighted expert confidence level and expert evidence. EEDM logic182, at block512calculating the recommendation value also includes factoring weighted frequencies and weighted expertise level with the weighted expert confidence and expert evidence. The identified recommendation value is stored at block188and used to determine whether to update the baseline command sequence model for the given task. Depending on the identified recommendation value, for example less than a threshold value, the update baseline command sequence model for the given task may only include script with the added decision point without changing the commands in the baseline command sequence model,