Method and system for intelligent qualitative and quantitative analysis for medical diagnosis

An Intelligent Qualitative and Quantitative Analysis (IQQA) system and method is disclosed that allows a user to specify certain identifying information used to search and obtain medically-related information from different sources. The medically-related information may be of different types and may be dynamically fused, when needed, to generate matrices of diagnostic information. Medical decisions such as a diagnostic decision may be made based on such matrices of diagnostic information.

BACKGROUND

1. Field of Invention

The present invention relates to systems and methods for medical information processing and analysis. Specifically, the present invention relates to intelligent qualitative and quantitative analysis of medical information for medical decision making.

2. Description of Related Art

In disease diagnosis using modem imaging techniques, physicians are often overwhelmed by the amount of information made available through different acquisition devices. Such devices may include, but are not limited to, ultrasound (US), Computerized Tomography (CT), and Magnet Resonance Imaging (MRI). Diagnostic information (DI), may differ from patient data and it may include key diagnostic evidence identified from diagnostic data that directly or indirectly supports and/or disaffirms a physician/specialist's hypothesis about a diagnosis. Often, important diagnostic information relevant to a specific disease is buried in the huge volume of data. In addition, although patient records and laboratory test results, such as blood tests, may provide important clues to suspected diseases/abnormalities, the interpretation of such information is not conventionally integrated with various image-based diagnosis processes in a coherent fashion. Consequently, physicians have to look manually for all disease-relevant information embedded in both non-visual and visual data from different sources. This task is labor intensive and requires a high level of skill. In addition, a manual process is also subject to mistakes, which may lead to misdiagnosis due to either negligence or lack of skill, qualitative and quantitative measurements, as well as intuitive visualization means that allow a physician to seamlessly integrate information across multi-modalities into a clinic workflow.

Another problem of the prior art is that conventional Computer Aided Detection/Diagnosis (CAD) systems usually allow a physician to access only either a computer's output in the form of binary decisions or raw data. That is, a wide variety of rich DI that can be derived between the raw data and the binary decision output (i.e., yes or no decisions) is ignored or discarded. Since such a loss is irreversible, the physician does not have the opportunity to interactively and quantitatively examine or identify suspicious regions, with the assistance of a computer system, in order to make their independent decision.

A third problem with conventional CAD systems is that multi-modality information, including both non-visual and visual information, is not utilized simultaneously. Current CAD systems usually operate on data of a single modality.

FIG. 1illustrates a flowchart of a conventional computer-aided detection system, in which a physician starts, at104, by selecting a patient image of a particular modality with respect to a pre-defined disease. At106, a computer detects suspicious regions in the image that may correspond to where a specific disease manifests itself. The result of such computer detection is a binary yes/no decision for each location in the image. When the computer indicates the existence of an abnormality in an image location, a mark may be displayed, at108, near the corresponding location in the image. At110, the physician makes diagnostic decisions based on the computer derived marks.

DETAILED DESCRIPTION

An Intelligent Qualitative and Quantitative Analysis (IQQA) system is disclosed, which is aimed at systematically extracting and integrating diagnostic information from various sources. The system may be deployed on a general or a special purpose computer and adapted to any specific computing environment. The IQQA is a diagnostic-information-oriented computerized system that dynamically embeds different information processing and disease-specific applications into optimal clinical workflow. Such information processing and applications may include, but are not limited to, diagnostic data searching, diagnostic information (DI) extraction, DI quantification, DI visualization, DI fusion, as well as diagnosis of abnormalities and/or diseases based on data across multi-modality/multi-information type. Functionalities may be dynamically embedded through, for example, a) selectively activating functionalities based on a type of disease under investigation; b) dynamically forming a disease-specific workflow driven by specific diagnostic information available; and c) adaptively integrating physician's case-specific knowledge and experience of physicians gained during computer-human real-time interaction with previously built-in domain-specific knowledge to expand the knowledge base of the system in performing a diagnosis.

The IQQA system as described herein provides a wealth of information between raw diagnostic data and a binary diagnostic decision, thereby to assist physicians to make diagnostic decisions more effectively with potentially a higher precision. A Dynamic Matrix of Diagnostic Decision (dynamic MDD) platform, is designed to facilitate various functionalities such as extraction, presentation, navigation, visualization, fusion, and modification of, and interaction with diagnostic information.

The IQQA as described herein provides an interactive, intelligent qualitative and quantitative analysis of diagnostic data across all modalities from different sources to assist physicians to make a diagnostic decision.FIG. 2depicts an exemplary high level block diagram200of the IQQA, according to a preferred embodiment of the present invention. The system200comprises a patient info storage225and a disease info storage225, a plurality of databases210-a, . . . ,210-b, a Diagnostic Data Search Engine and DI Extractor240, a Dynamic Matrix of Diagnostic Decision (dynamic MDD)260, an original data space250, a diagnostic report generation mechanism255, and a diagnostic report storage265. The Dynamic MDD260comprises of a plurality of dynamic DI matrices230-1,230-2, and230-1, and an Interactive Diagnostic Decision Making Mechanism (IDDM)235.

In operation, a physician may first select a disease type, through the IDDM mechanism235, from the disease information storage220and a patient ID or patient name from the patient information storage225. Such information is used to start a patient-specific and disease-specific diagnostic data searching process. To do so, the patient information and disease information may be fed to the Diagnostic Data (DD) Search Engine and Diagnostic Information (DI) Extractor240. The DD search engine and DI extractor240then searches for diagnostic data related to the given patient and disease from different databases, e.g., database1210-a, database N210-band extracts Disease Specific (DS) and Patient Specific (PS) DI, including visual and non-visual, from the searched diagnostic data. The databases searched may include local, remote, or distributed databases. Such databases may be accessible through any communication channels, including, but not limited to, a Local Area Network (LAN), a Wide Area Network (WAN), an Intranet, the Internet, a proprietary network, or a wireless network. The diagnosis decision produced by the IDDM mechanism235may be selectively fedback into the disease type storage220so that new disease types may be saved and utilized in the future.

Through the Dynamic Matrix of Diagnostic Decision (dynamic MDD) platform260, the physician may make an interactive diagnostic decision. First, the extracted DI may be constructed in the form of DI matrices230, which may be represented as a set of hierarchically organized dynamic matrices such as the dynamic DI matrix1230-1, the dynamic DI matrix2230-2, . . . , and dynamic DI matrix i230-3. The interactive diagnostic decision making mechanism235may facilitate the physician to explore the DI space, to perform DI extraction or re-extraction, DI modification, and/or DI fusion. The physician may rely on the dynamic diagnostic information represented by such matrices presented using interactive tools embedded in these dynamic MDDs260to reach a diagnostic decision. A diagnostic report265may also be generated through the diagnostic report generation mechanism255.

FIG. 3depicts an exemplary construct of the Dynamic Matrix of Diagnostic Decision (dynamic MDD)260, according to an embodiment of the present invention. The Dynamic MDD260is a platform for physicians to perform real-time interactive diagnostic decision making. The Dynamic MDD260comprises a plurality of DI matrices230, a DI storage mechanism302, a DI grouping mechanism304, a DI presentation mechanism306, a DI navigation mechanism308, a DI retrieving mechanism310, a DI re-extraction mechanism312, a DI modification mechanism314, and a DI fusion mechanism316.

The DI storage mechanism302is for storing diagnostic information. For example, for each suspicious region/volume of interest (ROI or VOI) associated with a specific disease, one or more dynamic DI matrices may be generated. These dynamic DI matrices may store key DI/evidence about the disease extracted from both non-visual and visual data around the ROI/VOI.

The DI grouping mechanism304is for organizing the extracted DI. Dynamic DI matrices230are a set of dynamically and hierarchically organized tables, constructed based on, e.g., anatomical, physiological, and biomedical knowledge and categories. It may also be grouped based on body parts, e.g., the entire body, a specific organ, some parts of the organ, biomarks, tissues, and molecules. Cells in the matrices may contain dynamic links to different sources of DI and/or the patient's raw diagnostic data. A matrix of a certain level may be embedded in its parent matrix at a higher level, except for the root matrix. Information contained in a matrix at each level may be systematically fused into its parent matrix one level above. The contained matrix hierarchy may be designed so that the most important diagnostic information related to making a diagnostic decision may be presented in the upper-most level, and more detailed DI information appears at lower levels. That is, an upper level matrix may provide an overview of the diagnostic information of the levels below, whereas the lower level matrices provide supporting information which further explains the items in the upper level table.

The DI presentation mechanism306is for presenting information in the DI matrices to the user in different forms. The DI in the Dynamic MDD may be presented either qualitatively or quantitatively. The presentation may be in a variety of formats, including, but are not limited to, numerals, strings, graphs, charts, colors, curves, and pictures. Representative images and results from computer analysis may be organized according to their relevance to the disease and may be directly visualized in one or more cells of a Dynamic DI matrix. The format and appearance of a Dynamic DI Matrix may be dynamically configured according to different criteria, such as the disease type selected, the image modality under examination, or the acquisition protocol. DI measurements that are considered abnormal may be highlighted using different colors in the dynamic DI matrices.

The DI navigation mechanism308provides means for a physician to explore the DI space. The DI may be organized in such a manner so that users may navigate in the DI space freely in real-time as needed. Through the DI navigation mechanism308, a physician can explore information in DI matrices at any level by activating a desired matrix, bringing information into the physician's view via a simple mouse click on the corresponding items in the DI hierarchy. A user may also go back to the original DI data for re-examination or inspection, either qualitatively or quantitatively. In addition, a global view of the navigation trail may also be visualized while a user is exploring the DI space.

The DI retrieving mechanism310is for retrieving DI stored previously. The retrieved DI may be loaded into the Dynamic DI Matrices230. A comparison between the current DI and previously stored DI may also be made.

The DI re-extraction mechanism312may be activated by a user to re-extract DI from previously saved ROI or VOI images to facilitate understanding and comparison with the current DI.

The DI modification mechanism314may be activated by a physician to make changes to extracted DI. For example, when a physician feels that a piece of DI extracted by the DD Search Engine and DI Extractor240is inaccurate or wrong, the physician may modify the DI by clicking on the corresponding cell in the dynamic DI matrix and then enter revisions. A set of interactive analytical tools and visualization tools encapsulated within the extracted visual DI may be activated and presented to the physician. The physician may then incorporate personal knowledge in the analysis procedure interactively. For example, a physician may interactively control the analysis/visualization tools through manual adjustment of their parameters. The physician may also directly edit certain items in the dynamic DI matrices and/or add new items and diagnostic comments into the matrices.

The DI fusion mechanism316for fusing domain-specific knowledge in the built-in IQQA system and a physician's case-specific knowledge. This may be achieved through a hierarchical decision making, man-machine real-time interaction and information exchanging process. In other words, dynamic MDD260in the IQQA makes all qualitative and quantitative diagnostic information transparent to physicians in their diagnostic decision making process. The information fusion methods provided by the DI fusion mechanism316are embedded in each hierarchical level of the DI presentation. Such fusion may also be performed across different levels. Disease-specific knowledge may be incorporated in the dynamic MDD260through information fusion, visualization and display. The DI fusion mechanism may employ patient and disease specific knowledge as well as non-patient specific disease specific knowledge to compute alert levels for each DI or abnormalities, lesions, cancers, or other diseases detected either by the system or by physicians. The dynamic MDD260may take into account a physician's preference and case-specific knowledge in a diagnostic decision making process by fusing the physician's knowledge with the built-in domain-specific knowledge in the IQQA system during the physician's real-time interaction with the dynamic MDD260.

FIG. 4shows an exemplary construct of the DD Search Engine and DI Info Extractor240in relation to other parts, according to one embodiment of the present invention. A non-visual DD Search Engine and DI Exactor404may search different databases406-408for non-visual diagnostic data and extract non-visual diagnostic information410. Such databases may include local, remote, and distributed databases, and may be accessed through a communication channel. A visual DD Search Engine and DI Extractor405may search the databases406-408for visual diagnostic data and extract visual diagnostic information412.

In addition to patient specific data, the non-visual DD Search Engine and DI extractor404may also search for non-patient-specific, but disease specific information across the different databases to, for example, bring the most up-to-date statistics about the disease into the IQQA decision support process. Such information may be utilized in different ways. For example, such information may be integrated in the IQQA's qualitative and quantitative analysis process. Knowledge about the disease may also be used to build and update any internal diagnostic model for a corresponding disease within IQQA. Furthermore, when physicians have direct access to such up-to-date statistical information about specific diseases, they can interpret the information, based on his/her own experience and knowledge, and rely on such interpretation in the decision making process.

The search results returned by the DD Search Engine and DI extractor240may be presented to a user in the form of, for example, file trees. The user may choose one of the files for further review. Alternatively, the user may exercise discretion to view all the studies in a sequential fashion. Those disease and patient specific, disease and non-patient specific diagnostic data may include both non-visual data or visual data. Non-visual data may comprise patient record, blood test results, and genotype data. Visual data may comprise diagnostic images of different modalities (e.g., CT, MRI, X-ray), and pathological images.

FIG. 5shows an exemplary construct of the non-visual DD search engine and DI extractor404in relation to other parts, according to one embodiment of the invention. A non-visual data search engine504is deployed to search for non-visual diagnostic data. Such a search engine may be any commercial or off-the-shelf search engine, implemented in any language such as SQL. Based on the searched non-visual data, a disease-specific non-visual diagnostic evidence extractor506may extract one or more pieces of disease-specific key evidence. Such extraction may be guided by a disease-specific evidence template508that pre-defines what constitutes key evidence given a disease type. After the key evidence is extracted, an automatic key evidence analyzer510may be used to analyze the key evidence. Statistical measurements of corresponding evidence may be retrieved from a database512and used to evaluate the extracted key evidence. Such statistical measurements may be derived based on studies over a statistically significant population so that they can be used to evaluate the significance of the key evidence extracted. For example, each piece of patient specific key evidence may be compared with the statistical range of the same evidence present in the database512. Evidences with measures that are beyond the given normal range,514, may then be stored in a non-visual DI storage410.

In addition to the key evidence analysis, a risk factor analyzer516may further analyze the risk factors associated with the patient with respect to the given a disease type based on information related to the patient, such as age, sex, medication record, blood test or family history. The output518of such analysis may also provide estimated degrees of impact of these risk factors on the hypothesized disease or lesion.

FIG. 6shows an exemplary construct of the visual DD search engine and DI extractor405according to an embodiment of the present invention. A visual DD search engine604performs a search for all visual data related to specific patient and disease, such as, CT data, MRI data, and X-ray data. Such data forms a multi-modality, multi-dimensional data set606. Here multi-dimensionality may include both spatial and temporal dimensions. Various visual DI extraction functions, according to the present invention, may be performed while a physician navigates through an imagery subspace, e.g., in selected 2D, 3D, 4D, or higher dimensional sub-space created by each of the modalities in real-time or near real-time interactively.

In the depicted exemplary embodiment, there comprises a DI-based enhancement mechanism608, a cross-modality DI extraction mechanism614, a computer-aided DI quantification mechanism616, and a cross-modality fusion-based abnormality detection mechanism618. These visual DI extractors may extract both qualitative and quantitative diagnostic information. For example, to obtain qualitative visual DI, the DI-based enhancement mechanism608enhances diagnostic information contained in the imageries. Based on global or local anatomic and disease information as well as information extracted in corresponding locations of other imageries, the DI-based enhancement mechanism608may automatically determine what is the most focused DI for the given specific disease in the given image and compute feature statistics about suspicious regions detected either by the system or by the user. Such statistics may be used to automatically set the enhancement parameters to bring the best visible diagnostic information to the eyes of a physician for diagnosis purposes. A user can choose to apply a global disease-specific enhancement via a global enhancement mechanism610to enhance an entire image set, based on information gathered from both non-visual DI and other available visual DI, including, but not limited to, statistics about shape, size, and brightness of all suspicious regions. A user can also choose to use a local enhancement mechanism612to enhance each local suspicious region of interest. Parameters used in enhancement may be computed based on DI information about individual suspicious region of interest and the disease.

Visual DI may also be extracted via the cross-modality DI extraction process614. Multi-modality and multi-dimensional information may be exploited in such a way that DI extracted from one modality or dimension may be used to confirm or disaffirm DI in another modality or dimension. Patient and disease specific, as well as non-patient specific and disease specific non-visual information may also be used to aid such visual DI extraction process. For example, age and sex may be used to determine focus regions of suspicion for a specific disease. Visual DI extraction and/or re-extraction may be performed automatically or interactively in real-time by physicians. In an interactive extraction process, a physician may apply his/her own knowledge to the extraction process. For example, a physician may make some manual adjustment to already extracted DI by adjusting parameters that are used by the DI extraction process.

Visual DI may also be quantified via the computer-aided DI quantification (CAQ) mechanism616. The CAQ mechanism616is capable of automatically quantifying DI across multiple modalities. Such quantification may be applied to measurements such as size, shape, and intensity variations of suspicious regions within and across multimodalities.

Visual DI may also be extracted via the cross-modality fusion-based abnormality detection mechanism618. A physician may pose different hypotheses or sub-hypotheses about diseases, including both disease type and disease location, to the system based upon, e.g., spatial reasoning or temporal reasoning about the extracted DI in the hierarchically organized DI space. Cross-modality fusion, according to the present invention, may be performed in different modes. For example, fusion can be performed based on positional or DI information. In the positional fusion mode, visual data cross different modalities and dimensions are registered prior to the fusion. In a DI fusion mode, fusion is based on the nature of the information contained in the DI space.

FIG. 7illustrates the overall operational scheme of the IQQA system, according one embodiment of the invention. First, a user700selects a patient ID from an existing patient ID list, or enter a patient ID703. The user may also select a disease type704. With the patient ID703and disease type704, a Diagnostic Data Search Engine706automatically identifies all available diagnostic data, both patient-specific and non-patient specific, that is relevant to the specified disease from one or more relevant databases702, either local, remote, or distributed. Such identified diagnostic data may be non-visual or visual, including genotype data740, lab test results742, phenotype data744, and pathological data746. The retrieved patient and non-patient specific and disease-specific data may then be organized according to some pre-defined criteria. For example, data type and date of examination may be used to organize the retrieved data.

The search for diagnostic data may be conducted via an on-line search750or via an off-line search748. An on-line search may be performed while a physician interacts with the IQQA system during a diagnostic decision making process. An off-line search may be performed before the physician starts the diagnosis process via the IQQA system. The search results may be presented to the user in the form of, for example, file trees707so that a user may choose one or more of the files for further review (710). A DI extraction and quantification mechanism708may extract diagnostic information from the diagnostic data with respect to the specified disease. The patient and disease-specific, as well as non-patient specific and disease specific data and the extracted diagnostic information may be made accessible to a user during a diagnosis session. The user may manually examine, if needed simultaneously, all the accessible diagnostic data and diagnostic information associated with the selected patient at the same time.

The extracted diagnostic information may then be organized hierarchically in one or more dynamic DI matrix/matrices in a diagnostic information space716according to some pre-determined criteria such as degree of details of the lesion, parts of the body where disease occurs.

The user700may further interact with the dynamic DI matrices in the diagnostic information space716via an interactive decision making mechanism724. The user700may also choose to load dynamic DI matrices generated from previous exams711of the same patient and merged with the current DI matrices. Decisions may be made also in the context of the previous DI matrices.

The dynamic DI matrices in the DI space716offers an information space where a physician may navigate to identify rich information from different modalities and along different dimensions, all at different levels of detail. A physician may navigate in the diagnostic information space716in a Real-Time Interactive (RTI) mode in either an imagery sub-space or a DI sub-space. The imagery sub-space may further include 2D image sub-space, 3D image sub-space, or image sub-space in higher dimensions. A physician may select to operate in any sub-space according to needs. While navigating in the diagnostic information space716, a user may also navigate at different levels of detail, exploring and searching for diagnostic information that may be relevant to diagnostic decision making.

Diagnostic information may be presented to the user700after appropriate processing such as information fusion, visualization, and real-time user interaction of the dynamic MDD. The user700may perform RTI image space navigation712, RTI visual DI extraction and quantification718, or RTI DI-space navigation720. During real time user interaction, the user700may also re-extract existing DI or make modifications to existing DI (722). Through RTI decision making726, the user can make diagnostic decisions728.

Information contained in different dynamic DI matrices may be compiled to generate a report734by a diagnostic report generation mechanism732. Such a generated report may be particularly structured based on some specified requirements, which may be automatically set up as default or may be dynamically updated by the user700. The generated report734may contain hyperlinks associated with a medical decision included in the report734connecting to the original images or other diagnostic information that are used in reaching the underlying medical decision. The hyperlinks included in the report734may also embed some integrated mechanisms that may be launched to provide the capability of displaying, navigation, or reviewing of the supporting diagnostic information. When the user700completes the examination, the dynamic DI matrices730derived during the current examination may be stored for future use.

FIG. 8is a flowchart of an exemplary process, in which the IQQA system facilitates a user to reach a medical decision using diagnostic information from different sources, according to one embodiment of the invention. After the system is started, a user may select a patient name or patient ID at804, and a disease type at806. At808, patient-specific and disease-specific, and non-patient-specific and disease-specific diagnostic data is retrieved from one or more databases. From the diagnostic data, non-visual DI may be extracted at810, and visual DI may be extracted at812. The extracted DI is stored in dynamic DI matrices at814. Based on the extracted DI from different sources, modalities, and dimensions, dynamic fusion is performed based on diagnostic needs at816. Such a dynamic fusion may be carried out via either an automatic or an interactive process. At818, diagnostic decisions may then be made based on the dynamically fused DI. At820, a diagnostic report is generated. Such a report may include information automatically generated based on DI, such as summary of key evidence or statistics. The generated report may also include information entered by a physician. After a physician makes relevant diagnostic decisions, associated DI may be archived and stored, at822, for future reference.