FEEDBACK SYSTEM AND METHOD

A computer-implemented method, computer program product and computing system for receiving a result set for content processed by an automated analysis process; receiving human feedback concerning the result set; and providing feedback information to the developer of the automated analysis process based, at least in part, upon the result set and the human feedback.

TECHNICAL FIELD

This disclosure relates to feedback systems and methods and, more particularly, to feedback systems and methods concerning artificial intelligence and machine learning functionality.

BACKGROUND

Recent advances in the fields of artificial intelligence and machine learning are showing promising outcomes in the analysis of clinical content, examples of which may include medical imagery. Accordingly, processes and algorithms are constantly being developed that may aid in the processing and analysis of such medical imagery. Unfortunately, such processes and algorithms often need to be revised/finetuned to address inaccuracies and unanticipated results. Traditionally, when an unanticipated result occurs, the data that caused the unanticipated result is sent to the developer of the process/algorithm for trouble shooting.

SUMMARY OF DISCLOSURE

In one implementation, a computer-implemented method is executed on a computing device and includes: receiving a result set for content processed by an automated analysis process; receiving human feedback concerning the result set; and providing feedback information to the developer of the automated analysis process based, at least in part, upon the result set and the human feedback.

One or more of the following features ay be included. The result set may be an auto-populated report. The human feedback may include amendments to the auto-populated report. The human feedback may concern the accuracy of the result set. The human feedback may include amendments to the result set. The content may be medical imagery. Providing feedback information to the developer of the automated analysis process may include: providing at least a portion of the result set and/or the human feedback to the developer of the automated analysis process. Providing feedback information to the developer of the automated analysis process may include: providing differential information that defines differences between the result set and the human feedback to the developer of the automated analysis process. The feedback information may be processed to remove confidential data. The feedback information may be processed to remove confidential data in accordance with one or more medical data privacy rules.

In another implementation, a computer program product resides on a computer readable medium and has a plurality of instructions stored on it. When executed by a processor, the instructions cause the processor to perform operations including: receiving a result set for content processed by an automated analysis process; receiving human feedback concerning the result set; and providing feedback information to the developer of the automated analysis process based, at least in part, upon the result set and the human feedback.

One or more of the following features ay be included. The result set may be an auto-populated report. The human feedback may include amendments to the auto-populated report. The human feedback may concern the accuracy of the result set. The human feedback may include amendments to the result set. The content may be medical imagery. Providing feedback information to the developer of the automated analysis process may include: providing at least a portion of the result set and/or the human feedback to the developer of the automated analysis process. Providing feedback information to the developer of the automated analysis process may include: providing differential information that defines differences between the result set and the human feedback to the developer of the automated analysis process. The feedback information may be processed to remove confidential data. The feedback information may be processed to remove confidential data in accordance with one or more medical data privacy rules.

In another implementation, a computing system includes a processor and a memory system configured to perform operations including: receiving a result set for content processed by an automated analysis process; receiving human feedback concerning the result set; and providing feedback information to the developer of the automated analysis process based, at least in part, upon the result set and the human feedback.

One or more of the following features ay be included. The result set may be an auto-populated report. The human feedback may include amendments to the auto-populated report. The human feedback may concern the accuracy of the result set. The human feedback may include amendments to the result set. The content may be medical imagery. Providing feedback information to the developer of the automated analysis process may include: providing at least a portion of the result set and/or the human feedback to the developer of the automated analysis process. Providing feedback information to the developer of the automated analysis process may include: providing differential information that defines differences between the result set and the human feedback to the developer of the automated analysis process. The feedback information may be processed to remove confidential data. The feedback information may be processed to remove confidential data in accordance with one or more medical data privacy rules.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

System Overview

Referring toFIG. 1, there is shown online platform process10. Online platform process10may be implemented as a server-side process, a client-side process, or a hybrid server-side/client-side process. For example, online platform process10may be implemented as a purely server-side process via online platform process10s. Alternatively, online platform process10may be implemented as a purely client-side process via one or more of online platform process10c1, online platform process10c2, online platform process10c3, and online platform process10c4. Alternatively still, online platform process10may be implemented as a hybrid server-side/client-side process via online platform process10sin combination with one or more of online platform process10c1, online platform process10c2, online platform process10c3, and online platform process10c4. Accordingly, online platform process10as used in this disclosure may include any combination of online platform process10s, online platform process10c1, online platform process10c2, online platform process10c3, and online platform process10c4. Examples of online platform process10may include but are not limited to all or a portion of the PowerShare™ platform and/or the PowerScribe™ platform available from Nuance Communications™ of Burlington, Mass.

Online platform process10smay be a server application and may reside on and may be executed by computing device12, which may be connected to network14(e.g., the Internet or a local area network). Examples of computing device12may include, but are not limited to: a personal computer, a server computer, a series of server computers, a mini computer, a mainframe computer, or a cloud-based computing platform.

The instruction sets and subroutines of online platform process10s, which may be stored on storage device16coupled to computing device12, may be executed by one or more processors (not shown) and one or more memory architectures (not shown) included within computing device12. Examples of storage device16may include but are not limited to: a hard disk drive; a RAID device; a random access memory (RAM); a read-only memory (ROM); and all forms of flash memory storage devices.

Examples of online platform processes10c1,10c2,10c3,10c4may include but are not limited to a web browser, a game console user interface, a mobile device user interface, or a specialized application (e.g., an application running on e.g., the Android™ platform, the iOS™ platform, the Windows™ platform, the Linux™ platform or the UNIX™ platform). The instruction sets and subroutines of online platform processes10c1,10c2,10c3,10c4, which may be stored on storage devices20,22,24,26(respectively) coupled to client electronic devices28,30,32,34(respectively), may be executed by one or more processors (not shown) and one or more memory architectures (not shown) incorporated into client electronic devices28,30,32,34(respectively). Examples of storage devices20,22,24,26may include but are not limited to: hard disk drives; RAID devices; random access memories (RAM); read-only memories (ROM), and all forms of flash memory storage devices.

Examples of client electronic devices28,30,32,34may include, but are not limited to, a smartphone (not shown), a personal digital assistant (not shown), a tablet computer (not shown), laptop computers28,30,32, personal computer34, a notebook computer (not shown), a server computer (not shown), a gaming console (not shown), and a dedicated network device (not shown). Client electronic devices28,30,32,34may each execute an operating system, examples of which may include but are not limited to Microsoft Windows™, Android™, iOS™, Linux™, or a custom operating system.

Users36,38,40,42may access online platform process10directly through network14or through secondary network18. Further, online platform process10may be connected to network14through secondary network18, as illustrated with link line43.

The various client electronic devices (e.g., client electronic devices28,30,32,34) may be directly or indirectly coupled to network14(or network18). For example, laptop computer28and laptop computer30are shown wirelessly coupled to network14via wireless communication channels44,46(respectively) established between laptop computers28,30(respectively) and cellular network/bridge48, which is shown directly coupled to network14. Further, laptop computer32is shown wirelessly coupled to network14via wireless communication channel50established between laptop computer32and wireless access point (i.e., WAP)52, which is shown directly coupled to network14. Additionally, personal computer34is shown directly coupled to network18via a hardwired network connection.

WAP52may be, for example, an IEEE 802.11a, 802.11b, 802.11g, 802.11n, Wi-Fi, and/or Bluetooth device that is capable of establishing wireless communication channel50between laptop computer32and WAP52. As is known in the art, IEEE 802.11x specifications may use Ethernet protocol and carrier sense multiple access with collision avoidance (i.e., CSMA/CA) for path sharing. As is known in the art, Bluetooth is a telecommunications industry specification that allows e.g., mobile phones, computers, and personal digital assistants to be interconnected using a short-range wireless connection.

While the following discussion concerns medical imagery, this is for illustrative purposes only and is not intended to be a limitation of this disclosure, as other configurations are possible and are considered to be within the scope of this disclosure. For example, the following discussion may concern any type of clinical content (e.g., DNA sequences, EKG results, EEG results, blood panel results, lab results, etc.) and/or non-medical content.

Assume for the following example that users36,38are medical service providers (e.g., radiologists) in two different medical facilities (e.g., hospitals, labs, diagnostic imaging centers, etc.). Accordingly and during the normal operation of these medical facilities, medical imagery may be generated by e.g., x-ray systems (not shown), MRI systems (not shown), CAT systems (not shown), PET systems (not shown) and ultrasound systems (not shown). For example, assume that user36generates medical imagery54and user38generates medical imagery56; wherein medical imagery54may be stored locally on storage device20coupled to laptop computer28and medical imagery56may be stored locally on storage device22coupled to laptop computer30. When locally storing medical imagery54,56, this medical imagery may be stored within e.g., a PACS (i.e., Picture Archiving and Communication System). Additionally/alternatively, the medical imagery (e.g., medical imagery54,56) may be stored on a cloud-based storage system (e.g., a cloud-based storage system (not shown) included within online platform58).

Online platform process10may enable online platform58that may be configured to allow for the offering of various medical diagnostic services to users (e.g., users36,38) of online platform58. For the following example, assume that user40is a medical research facility (e.g., the ABC Center) that performs cancer research. Assume that user40produced a process (e.g., automated analysis process60) that analyzes medical imagery to identify anomalies that may be cancer. Examples of automated analysis process60may include but are not limited to an application or an algorithm that may process medical imagery (e.g., medical imagery54and medical imagery56), wherein this application/algorithm may utilize artificial intelligence, machine learning and/or probabilistic modeling when analyzing the medical imagery (e.g., medical imagery54and medical imagery56). Examples of such probabilistic modeling may include but are not limited to discriminative modeling (e.g., a probabilistic model for only the content of interest), generative modeling (e.g., a full probabilistic model of all content), or combinations thereof.

Further assume that user42is a medical research corporation (e.g., the XYZ Corporation) that produces applications/algorithms (e.g., automated analysis process62) that analyze medical imagery to identify anomalies that may be cancer. Examples of automated analysis process62may include but are not limited to an application or an algorithm that may process medical imagery (e.g., medical imagery54and medical imagery56), wherein this application/algorithm may utilize artificial intelligence, machine learning algorithms and/or probabilistic modeling when analyzing the medical imagery (e.g., medical imagery54and medical imagery56). Examples of such probabilistic modeling may include but are not limited to discriminative modeling (e.g., a probabilistic model for only the content of interest), generative modeling (e.g., a full probabilistic model of all content), or combinations thereof.

Assume for the following example that user40(i.e., the ABC Center) wishes to offer automated analysis process60to others (e.g., users36,38) so that users36,38may use automated analysis process60to process their medical imagery (e.g., medical imagery54and medical imagery56, respectively). Further assume that user42(i.e., the XYZ Corporation) wishes to offer automated analysis process62to others (e.g., users36,38) so that users36,38may use automated analysis process62to process their medical imagery (e.g., medical imagery54and medical imagery56, respectively).

Accordingly, online platform process10and online platform58may allow user40(i.e., the ABC Center) and/or user42(i.e., the XYZ Corporation) to offer automated analysis process60and/or automated analysis process62(respectively) for use by e.g., user36and/or user38. Therefore, online platform process10and online platform58may be configured to allow user40(i.e., the ABC Center) and/or user42(i.e., the XYZ Corporation) to upload a remote copy of automated analysis process60and/or automated analysis process62to online platform58, resulting in automated analysis process60and/or automated analysis process62(respectively) being available for use via online platform58. Accordingly, online platform process10may offer a plurality of computer-based medical diagnostic services (e.g., automated analysis process60,62) within the online platform (e.g., online platform58), wherein online platform process10may identify the computer-based medical diagnostic services (e.g., automated analysis process60,62) that are available via online platform58and users (e.g., user36,38) may utilize these computer-based medical diagnostic services (e.g., automated analysis process60,62) to process the medical imagery (e.g., medical imagery54and medical imagery56).

As could be expected, when users (e.g., user36,38) utilize these computer-based medical diagnostic services (e.g., automated analysis process60,62) to process the medical imagery (e.g., medical imagery54and medical imagery56), it is foreseeable that unexpected results may occur. As discussed above, automated analysis processes60,62may be utilized to identify anomalies within medical imagery (e.g., medical imagery54and medical imagery56, respectively) that may be cancer. Unfortunately, misidentifications may occur. For example and once the medical imagery (e.g., medical imagery54and medical imagery56) is processed by automated analysis processes60,62, the results of automated analysis processes60,62may be reviewed by e.g., a radiologist. At this point, the radiologist(s) can determine if any misidentifications occurred. Examples of such misidentifications may include but are not limited to false negatives (e.g., when anomalies are present within medical imagery54,56but automated analysis processes60,62indicates that none exist) and false positives (e.g., when anomalies are not present within medical imagery54,56but automated analysis processes60,62indicates that some exist)

While the following discussion concerns the processing of medical imagery, this is for illustrative purposes only and is not intended to be a limitation of this disclosure, as other configurations are possible and are considered to be within the scope of this disclosure. For example, other types of medical information may be processed, such as DNA sequences, EKG results, EEG results, blood panel results, lab results, etc. Additionally, other types of information may be processed that need not be medical in nature. Accordingly and with respect to this disclosure, the content processed may be any type of content for which automated processing may be applicable, such as medical data, financial records, personal records, and identification information.

Referring also toFIG. 2and for the following discussion, assume that user38(e.g., a radiologist) has a chest x-ray (e.g., chest x-ray100) of a patient that is being processed by automated analysis process60to determine if there are any anomalies within chest x-ray100. Assume for this example that automated analysis process60generates result set102that identifies one anomaly (e.g., anomaly104). In this example, result set102may include annotated x-ray106and auto-populated report108. For example, annotated x-ray106may visually locate anomaly104; while auto-populated report108may be a radiologist report that is automatically generated by automated analysis process60and populated with the findings made by automated analysis process60(e.g., the identification, description and location of anomaly104).

While result set102is shown to include annotated x-ray106and auto-populated report108, this is for illustrative purposes only and is not intended to be a limitation of this disclosure, as other configurations are possible. As discussed above, while this example concerns medical imagery (e.g., chest x-ray100), other types of data are possible and are considered to be within the scope of this disclosure (e.g., DNA sequences, EKG results, EEG results, blood panel results, lab results, non-medical information, etc. Accordingly and in such situations, result set102may be related to those other types of data that do not concern medical imagery and/or are not medical in nature.

Referring also toFIG. 3, online platform process10may receive200a result set (e.g., result set102) for content (e.g., chest x-ray100) processed by an automated analysis process (e.g., automated analysis process60), which may be reviewed by user38(e.g., a radiologist). Assume that upon user38(e.g., a radiologist) who is reviewing result set102determines that result set102is inaccurate, as e.g., chest x-ray100is clean (i.e., it does not show any anomalies) and the identified anomaly (i.e., anomaly104) is shown to be located outside of the body.

Accordingly, user38(e.g., a radiologist) may revise result set102to generate human feedback110concerning result set102. Human feedback110may generally concern the accuracy of result set102. As discussed above and in this example, result102is inaccurate, as it contains a false positive (i.e., falsely identifies anomaly104). Accordingly, human feedback110may identify/document such inaccuracies within result set102and/or may include amendments to result set102.

For example, human feedback110may include amendments to annotated x-ray106(resulting in amended x-ray106′) and/or amendments to auto-populated report108(resulting in amended report108′). For example, amended x-ray106′ may be a revised/annotated version of annotated x-ray106that (in this particular example) removes any indication of anomaly104). Additionally, amended report108′ may be a revised/annotated version of auto-populated report108that (in this particular example) removes any reference of anomaly104).

Online platform process10may receive202human feedback110concerning result set102and may provide204feedback information (e.g., feedback information112) to the developer (e.g., user40of the ABC Center) of automated analysis process60based, at least in part, upon result set102and human feedback110.

When providing204feedback information (e.g., feedback information112) to the developer (e.g., user40of the ABC Center) of automated analysis process60, online platform process10may provide206at least a portion of result set102and/or human feedback110to the developer (e.g., user40of the ABC Center) of automated analysis process60. For example, feedback information112may include all or a portion of annotated x-ray106(which shows anomaly104) and all or a portion of amended x-ray106′ (which deletes anomaly104) to illustrate any inaccuracies associated with automated analysis process60. Further, feedback information112may include all or a portion of auto-populated report108(which discusses anomaly104) and all or a portion of amended report108′ (which deletes reference to anomaly104) to illustrate any inaccuracies associated with automated analysis process60.

Accordingly and when providing204feedback information (e.g., feedback information112) to the developer (e.g., user40of the ABC Center) of automated analysis process60, online platform process10may provide208differential information that defines differences between result set102and human feedback110to the developer (e.g., user40of the ABC Center) of automated analysis process60. Specifically and in this example, feedback information112may identify that automated analysis process60defined anomaly104within result set102, while human feedback110did not define such an anomaly, thus indicating that automated analysis process60generated a false positive.

While the above discussion concerns automated analysis process60producing inaccurate results and there being a differential between result set102and human feedback110, it is understood that there will be little (if any) differential between result set102and human feedback110if automated analysis process60produced accurate results.

The developer (e.g., user40of the ABC Center) of automated analysis process60may utilize feedback information112to gauge the quality/accuracy of automated analysis process60and troubleshoot any problems identified therein. For example and through the use of feedback information112, the source of any misidentifications (e.g., false negatives or false positives) may be identified, as feedback information112may include e.g., a description of the problem (e.g., anomaly104being shown to be located outside of the body), the problematic result set (e.g., result set102), and the input image (e.g., chest x-ray100).

Unfortunately, the above-described procedures may get complicated when dealing with confidential data (such as medical imagery), as various laws, rules and regulations (e.g., HIPAA Privacy Rules) strictly control the dissemination of confidential medical data. For example, the HIPAA Privacy Rules establishes national standards to protect individuals' medical records and other personal health information and applies to health plans, health care clearinghouses, and those health care providers that conduct certain health care transactions electronically. Additionally, it is good practice not to share such confidential data even if permitted by law, rule and regulation. Accordingly, online platform process10may be configured to allow for the submission of such feedback information112without the submission of such confidential data. Therefore, feedback information112may be processed to remove confidential data (generally) and in accordance with one or more medical data privacy rules (specifically), resulting in the generation of non-confidential data that is related to the confidential data (e.g., chest x-ray100).

Referring also toFIG. 4, online platform process10may process chest x-ray100to generate (in this example) one or more instantiations of non-confidential data (e.g., non-confidential data300, non-confidential data302, non-confidential data304, and non-confidential data306), wherein each of these instantiations is related to the confidential data (e.g., chest x-ray100). When processing the confidential data (e.g., chest x-ray100) to generate these instantiations of non-confidential data (e.g., non-confidential data300, non-confidential data302, non-confidential data304, and non-confidential data306) that is related to the confidential data (e.g., chest x-ray100), online platform process10may apply one or more medical data privacy rules (e.g., HIPAA Rules) to the confidential data (e.g., chest x-ray100) to generate the non-confidential data (e.g., non-confidential data300, non-confidential data302, non-confidential data304, and non-confidential data306) that is related to the confidential data (e.g., chest x-ray100).

For example and after applying these medical data privacy rules (e.g., HIPAA Rules) to the confidential data (e.g., chest x-ray100), the non-confidential data (e.g., non-confidential data300, non-confidential data302, non-confidential data304, and non-confidential data306) may include one or more of:instantiations of obscured data, wherein online platform process10may obscure one or more portions of the confidential data (e.g., chest x-ray100) to generate non-confidential data.instantiations of pixelated data, wherein online platform process10may pixelate one or more portions of the confidential data (e.g., chest x-ray100) to generate non-confidential data.instantiations of ambigutized data, wherein online platform process10may ambigutize one or more portions of the confidential data (e.g., chest x-ray100) to generate non-confidential data.instantiations of redacted data, wherein online platform process10may redact one or more portions of the confidential data (e.g., chest x-ray100) to generate non-confidential data.

Accordingly and by obscuring/pixelating/ambigutizing/redacting some or all of the confidential data (e.g., chest x-ray100), the newly-generated non-confidential data may adhere to and meet the requires of the medial data privacy rules (e.g., the HIPAA rules).

GENERAL