Abstract:
Improved pulmonary embolism (PE) detection may be obtained through computer aided-diagnosis. In particular, PE detection may be accomplished through patient-level diagnosis, embolus-level detection, or a combination of the two. Patient-level diagnosis operates to quickly exclude non-PE patients and dispatch PE-patients to treatment. Embolus-level detection operates to localize individual emboli to support personalized medicine via risk stratification. Multiple instance-based learning (MIBL) classification at the patient level explores the key observation that once any TP candidate of a patient is classified as positive, the patient is identified as PE positive. That is, MIBL focuses on correct classification of patients rather than individual candidates, to effectively and rapidly distinguish between PE patients and non-PE patients.

Description:
CROSS-REFERENCE TO RELATED APPLICATIONS 
       [0001]    This applications claims the benefit of priority to U.S. Provisional Patent No. 61/931,933 filed on Jan. 27, 2014 and entitled “Diagnosing Pulmonary Embolism By Integrating Patient-Level Diagnosis And Embolus-Level Detection,” which is hereby incorporated by reference in its entirety. 
     
    
     FIELD OF THE DISCLOSURE 
       [0002]    This disclosure relates to methods and apparatuses for patient diagnosis, and more particular detection and diagnosis of pulmonary embolism in patients. 
       BACKGROUND 
       [0003]    Pulmonary embolism (PE) is a sudden blockage in a lung artery due to an embolus that is formed in one part of the body and travels to the lungs in the bloodstream through the heart. It is a common cardiovascular emergency with about 600,000 cases occurring annually and causing approximately 200,000 deaths in the United States. Most patients who succumb to PE do so within the first few hours following the event. 
         [0004]    A major clinical challenge, particularly in an emergency department (ED), is to quickly and correctly diagnose patients with PE and dispatch them to the proper treatment, so that hazardous yet life-saving therapy can be prescribed appropriately. 
         [0005]    Unfortunately, PE is among the most difficult conditions to diagnose. The primary symptoms are protean and may be manifested by a number of other conditions that require different therapeutic interventions. The correct PE diagnosis may be overlooked in as many as 84% of cases, resulting in many preventable deaths. 
         [0006]    CT pulmonary angiography (CTPA) is the current diagnostic standard for suspected PE. It reveals embolus as a dark region residing in bright vessel lumen. Each CTPA scan consists of hundreds of axial images. The interpretation of these images is complex and time consuming because of the intricate branching structure of the pulmonary arteries, the demand for specialized knowledge to distinguish PE from the various causes of cardiopulmonary pathology that may resemble PE, and a myriad of artifacts that may obscure or mimic emboli, such as flow-related artifacts, streak artifacts, lymph nodes, etc. The accuracy and efficiency of interpreting such large 3-D image datasets is further limited by human factors, such as attention span and eye fatigue. Incorrect CTPA interpretations are not infrequent in clinical practice. The number of CTPA examinations has increased by an order of magnitude over the past decade, while the rate of true positive examinations has fallen to just 5-10%. There is thus a critical need to mitigate rapidly mounting radiologist workloads and improve the efficiency and accuracy of PE diagnosis in CTPA. 
       BRIEF SUMMARY 
       [0007]    Improved pulmonary embolism (PE) detection may be obtained through computer aided-diagnosis. In particular, PE detection may be accomplished through patient-level diagnosis, embolus-level detection, or a combination of the two. Patient-level diagnosis operates to quickly exclude non-PE patients and dispatch PE-patients to treatment. Embolus-level detection operates to localize individual emboli to support personalized medicine via risk stratification. 
         [0008]    Patient-level diagnosis determines a patient is “PE negative” by a computer-aided detection (CAD) system when the system reports no CAD findings. Thus, if any CAD findings are reported to a patient, the patient is determined “PE positive” by the system. To correctly classify a patient as “PE positive,” it is sufficient to classify one of its true positive (TP) candidates as positive, while it must correctly classify all false positives (FP) candidates negative. Patient-level diagnosis may be used to quickly exclude non-PE patients from PE patients by correctly classifying at least one TP candidate for each PE patient, while eliminating FP candidates for all patients, including PE and non-PE patients. 
         [0009]    Multiple instance-based learning (MIBL) classification at the patient level explores the key observation that once any TP candidate of a patient is classified as positive, the patient is identified as PE positive. That is, patient-level MIBL focuses on correct classification of patients rather than individual candidates, to effectively and rapidly distinguish between PE patients and non-PE patients. According to one embodiment, this may be performed mathematically by adjusting a classification boundary such that a minimum of hinge errors occurred on all the TP candidates associated with each patient be zero. 
         [0010]    According to one embodiment, a method may include receiving candidates for classification as true positives and false positives, wherein the candidates are associated with one of a plurality of groups; determining a classification boundary for the true positives and the false positives; calculating hinge errors for the received candidates based, at least in part, on the determined classification boundary; setting a portion of the calculated hinge errors to zero, wherein the portion comprises hinge errors corresponding to candidates on a false positive side of the determined classification boundary belong to a group of the plurality of groups also having candidates on a true positive side of the determined classification boundary; determining a shifted classification boundary based, at least in part, on the hinge errors after setting the portion of the calculated hinge errors to zero; and identifying as true positives the candidates on a true positive side of the shifted classification boundary. 
         [0011]    According to another embodiment, a computer program product may include a non-transitory computer readable medium having code to perform the steps of receiving candidates for classification as true positives and false positives, wherein the candidates are associated with one of a plurality of groups; determining a classification boundary for the true positives and the false positives; calculating hinge errors for the received candidates based, at least in part, on the determined classification boundary; setting a portion of the calculated hinge errors to zero, wherein the portion comprises hinge errors corresponding to candidates on a false positive side of the determined classification boundary belong to a group of the plurality of groups also having candidates on a true positive side of the determined classification boundary; determining a shifted classification boundary based, at least in part, on the hinge errors after setting the portion of the calculated hinge errors to zero; and identifying as true positives the candidates on a true positive side of the shifted classification boundary. 
         [0012]    According to a further embodiment, an apparatus may include memory and a processor coupled to the memory. The processor may be configured to perform the steps of receiving candidates for classification as true positives and false positives, wherein the candidates are associated with one of a plurality of groups; determining a classification boundary for the true positives and the false positives; calculating hinge errors for the received candidates based, at least in part, on the determined classification boundary; setting a portion of the calculated hinge errors to zero, wherein the portion comprises hinge errors corresponding to candidates on a false positive side of the determined classification boundary belong to a group of the plurality of groups also having candidates on a true positive side of the determined classification boundary; determining a shifted classification boundary based, at least in part, on the hinge errors after setting the portion of the calculated hinge errors to zero; and identifying as true positives the candidates on a true positive side of the shifted classification boundary. 
         [0013]    The foregoing has outlined rather broadly the features and technical advantages of the present invention in order that the detailed description of the invention that follows may be better understood. Additional features and advantages of the invention will be described hereinafter that form the subject of the claims of the invention. It should be appreciated by those skilled in the art that the conception and specific embodiment disclosed may be readily utilized as a basis for modifying or designing other structures for carrying out the same purposes of the present invention. It should also be realized by those skilled in the art that such equivalent constructions do not depart from the spirit and scope of the invention as set forth in the appended claims. The novel features that are believed to be characteristic of the invention, both as to its organization and method of operation, together with further objects and advantages will be better understood from the following description when considered in connection with the accompanying figures. It is to be expressly understood, however, that each of the figures is provided for the purpose of illustration and description only and is not intended as a definition of the limits of the present invention. 
     
    
     
       BRIEF DESCRIPTION OF THE DRAWINGS 
         [0014]    The following drawings form part of the present specification and are included to further demonstrate certain aspects of the present disclosure. The disclosure may be better understood by reference to one or more of these drawings in combination with the detailed description of specific embodiments. 
           [0015]      FIG. 1  is a flow chart illustrating a method of multiple instance-based learning (MIBL) for patient-level or emboli-level diagnosis according to one embodiment of the disclosure. 
           [0016]      FIG. 2A  is a graph illustrating an operation of method of multiple instance-based learning (MIBL) of  FIG. 1  on a sample data set according to one embodiment of the disclosure. 
           [0017]      FIG. 2B  is a graph illustrating an operation of method of multiple instance-based learning (MIBL) of  FIG. 1  on a sample data set according to one embodiment of the disclosure. 
           [0018]      FIG. 2C  is a graph illustrating an operation of method of multiple instance-based learning (MIBL) of  FIG. 1  on a sample data set according to one embodiment of the disclosure. 
           [0019]      FIG. 3  is a flow chart illustrating a method of pulmonary embolism (PE) diagnosis through a combination of patient-level and embolism-level application of multiple instance-based learning (MIBL) according to one embodiment of the disclosure. 
       
    
    
     DETAILED DESCRIPTION 
       [0020]      FIG. 1  is a flow chart illustrating a method of multiple instance-based learning (MIBL) for patient-level or emboli-level diagnosis according to one embodiment of the disclosure. The method  100  begins at block  102  with receiving candidates for classification as true positives and false positives, wherein the candidates are associated with one of a plurality of groups. The plurality of groups may represent emboli or patients, depending on whether patient-level of emboli-level diagnosis is desired. The candidates may be plotted in a two-dimensional space as illustrated in  FIG. 2A . A first patient may correspond to candidates  202 , a second patient may correspond to candidates  204 , a third patient may correspond to candidates  206 , and a fourth patient may correspond to candidates  208 . True positive candidates are indicated by filled in markers, and false positive candidates are indicated by hollow markers. 
         [0021]    At block  104 , a classification boundary is determined for the true positives and the false positives. The classification boundary may be created by a support vector machine (SVM). At block  106 , hinge errors are calculated for the received candidates based, at least in part, on the determined classification boundary. The classification boundary and the calculated hinge errors may be shown in the two-dimensional space as illustrated in  FIG. 2B . However, a conventional classification boundary  212  may misclassify, for example, the third patient by placing a false positive candidate  214  on the true positive side of the classification boundary  212 . 
         [0022]    Recognizing that it is sufficient to correctly classify one TP candidate of each PE patient, some of the hinge errors may be effectively set to zero to shift the classification boundary  212 . At block  108 , a portion of the calculated hinge errors may be set to zero, wherein the portion comprises hinge errors corresponding to candidates on a false positive side of the determined classification boundary belong to a group of the plurality of groups also having candidates on a true positive side of the determined classification boundary. For example, hinge errors for candidates  216 ,  218 , and  220  may be set to zero. 
         [0023]    At block  110 , a shifted classification boundary may be determined based, at least in part, on the hinge errors after setting the portion of the calculated hinge errors to zero. The shifted classification boundary may be shown in the two-dimensional space as illustrated in  FIG. 2C . A shifted classification boundary  230  marks only candidates  232 ,  234 , and  236  as TP candidates. 
         [0024]    Then, at block  112 , true positives of the candidates may be identified based on their location on a true positive side of the shifted classification boundary. For example, candidates  232 ,  234 , and  236  may be identified as true positives for pulmonary embolism. If the groups at block  102  are patients, then the patients corresponding to the candidates  232 ,  234 , and  236  in the received set of candidates may be diagnosed with pulmonary embolism. If the groups at block  102  are emboli, then the emboli corresponding to the candidates  232 ,  234 , and  236  in the received set of candidates may be identified as emboli. 
         [0025]    In one embodiment, multiple classifiers may be aggregated to reduce variance in the learned classifier over different sample sets of patient images. For example, aggregation such as bagging, boosting, Bayesian averaging, and random subspace aggregation may be used to enhance classification accuracy. In an embodiment utilizing bagging for aggregation, N trials may be completed, and in each trial two-thirds of the training cases may be randomly sampled and used for training a multiple instance-based classifier by solving Equation (4) below. The remaining one-third of cases may be used for cross-validation. A linear function f t (x)=w t   T x+b i  may be construed for a trial t. The final classifier may be based on the average model: 
         [0000]    
       
         
           
             
               f 
                
               
                 ( 
                 x 
                 ) 
               
             
             = 
             
               
                 1 
                 N 
               
                
               
                 
                   ∑ 
                   
                     t 
                     = 
                     1 
                   
                   N 
                 
                  
                 
                   
                     
                       f 
                       t 
                     
                      
                     
                       ( 
                       x 
                       ) 
                     
                   
                   . 
                 
               
             
           
         
       
     
         [0000]    In the average model, some features may have very small weights. In some embodiments, these features&#39; weights may be set to zero in the final aggregated classifier. The aggregated model f may then be applied to the entire training set to draw a ROC curve. A threshold a may be determined according to a desired specificity rate on the ROC curve. The patient-level classifier may then output any candidate x&#39;s that satisfy f(x)&gt;a. The learned model and threshold may be tested on the hold-out set of test cases for evaluation. 
         [0026]    A combination of patient-level and emboli-level diagnosis may be performed to identify emboli in patients with pulmonary embolism.  FIG. 3  is a flow chart illustrating a method of pulmonary embolism (PE) diagnosis through a combination of patient-level and embolism-level application of multiple instance-based learning (MIBL) according to one embodiment of the disclosure. A method  300  begins at block  302  with identifying pulmonary embolism (PE) patients based on classification of a first set of candidates. For example, block  302  may include executing the method  100  of  FIG. 1  on a set of candidates corresponding to patients. At block  304 , a second set of candidates corresponding to a plurality of emboli may be built. At block  306 , emboli may be identified from the second set of candidates as particular emboli in the PE positive patients. Execution of block  306  may include, for example, repeating the method  100  of  FIG. 1  with the set of candidates built in block  304  where the groups of block  102  correspond to emboli. 
         [0027]    One mathematical formulation for patient-level diagnosis for use in certain embodiments of the invention described above is now presented. Assume that there are n patients, and l candidates are generated by a candidate generator, each represented by a feature vector x i  associated with a label y i . The label y i =1 if the candidate is a true positive, otherwise y i =1. For patient p, let Ip +  be a set containing indices of true positive candidates and Ip −  be an index set for false positive candidates. Ip +  or Ip −  could be empty if there are no true positive candidates or no false positive candidates. For convenience, the following definition may be used: 
         [0000]        I   + =Σ p=1   n   I   p   + , and  I   − =Σ p=1   n   I   p   − .
 
         [0028]    Support vector machines (SVMs) may be used for classification and regression, aiming to construct linear classification functions of the form w T x+b by minimizing the hinge error defined as ξ=max{0, 1−y(w T x−b)} for all candidates as written in the following optimization problem: 
         [0000]      min w,ξ    γ∥w∥   1 +Σ i=1   l ξ i  
 
         [0000]      s.t.  y   i ( w   T   x   i   +b )≧1−ξ i   , i= 1, . . ., l,  
 
         [0000]      ξ i ≧0, i=1, . . . ,l.   (1)
 
         [0029]    If a candidate is classified to the correct class with a margin, it receives a hinge loss ξ=0; otherwise, it receives a positive hinge loss. The training process involves learning a linear boundary, which minimizes the total hinge errors over all the candidates under the regularization terms. To help formulate multiple instance-based classification, Equation 1 may be rewritten by separating candidates into true positives and false positives associated each patient: 
         [0000]    
       
         
           
             
               
                 
                   
                     
                       
                         
                           min 
                           
                             w 
                             , 
                             ξ 
                           
                         
                          
                         
                             
                         
                          
                         
                           γ 
                            
                           
                             
                                
                               w 
                                
                             
                             1 
                           
                         
                       
                       + 
                       
                         
                           ∑ 
                           
                             p 
                             = 
                             1 
                           
                           n 
                         
                          
                         
                           ( 
                           
                             
                               
                                 ∑ 
                                 
                                   i 
                                   ∈ 
                                   
                                     I 
                                     P 
                                     + 
                                   
                                 
                               
                                
                               
                                 ξ 
                                 i 
                               
                             
                             + 
                             
                               
                                 ∑ 
                                 
                                   i 
                                   ∈ 
                                   
                                     I 
                                     P 
                                     - 
                                   
                                 
                               
                                
                               
                                 - 
                                 
                                   ξ 
                                   i 
                                 
                               
                             
                           
                           ) 
                         
                       
                     
                     = 
                     
                       
                         
                           
                             γ 
                              
                             
                               
                                  
                                 w 
                                  
                               
                               1 
                             
                           
                           + 
                           
                             
                               ∑ 
                               
                                 p 
                                 = 
                                 1 
                               
                               n 
                             
                              
                             
                               
                                 ∑ 
                                 
                                   i 
                                   ∈ 
                                   
                                     I 
                                     P 
                                     + 
                                   
                                 
                               
                                
                               
                                 ξ 
                                 i 
                               
                             
                           
                           + 
                           
                             
                               ∑ 
                               
                                 p 
                                 = 
                                 1 
                               
                               n 
                             
                              
                             
                               
                                 ∑ 
                                 
                                   i 
                                   ∈ 
                                   
                                     I 
                                     P 
                                     - 
                                   
                                 
                               
                                
                               
                                 - 
                                 
                                   ξ 
                                   i 
                                 
                               
                             
                           
                         
                          
                       
                       = 
                       
                         
                           γ 
                            
                           
                             
                                
                               w 
                                
                             
                             1 
                           
                         
                         + 
                         
                           
                             ∑ 
                             
                               p 
                               = 
                               1 
                             
                             n 
                           
                            
                           
                             
                               ∑ 
                               
                                 i 
                                 ∈ 
                                 
                                   I 
                                   P 
                                   + 
                                 
                               
                             
                              
                             
                               ξ 
                               i 
                             
                           
                         
                         + 
                         
                           
                             ∑ 
                             
                               i 
                               ∈ 
                               I 
                             
                           
                            
                           
                             - 
                             
                               ξ 
                               i 
                             
                           
                         
                       
                     
                   
                    
                   
                     
 
                   
                    
                   
                     
                       
                         
                           
                             s 
                             . 
                             t 
                             . 
                             
                                 
                             
                              
                             
                               w 
                               T 
                             
                           
                            
                           
                             x 
                             i 
                           
                         
                         + 
                         b 
                       
                       ≥ 
                       
                         1 
                         - 
                         
                           ξ 
                           i 
                         
                       
                     
                     , 
                     
                       i 
                       ∈ 
                       
                         I 
                         + 
                       
                     
                     , 
                     
                       
                         
                           
                             w 
                             T 
                           
                            
                           
                             x 
                             i 
                           
                         
                         + 
                         b 
                       
                       ≤ 
                       
                         
                           - 
                           1 
                         
                         + 
                         
                           ξ 
                           i 
                         
                       
                     
                     , 
                     
                       i 
                       ∈ 
                       
                         I 
                         - 
                       
                     
                     , 
                     
                       
                         ξ 
                         i 
                       
                       ≥ 
                       0 
                     
                     , 
                     
                         
                     
                      
                     
                       i 
                       = 
                       1 
                     
                     , 
                     … 
                      
                     
                         
                     
                     , 
                     
                        
                       . 
                     
                   
                 
               
               
                 
                   ( 
                   2 
                   ) 
                 
               
             
           
         
       
     
         [0030]    Multiple instance-based classification at the patient level may explore the key observation that once any TP candidate of patient p is classified as positive, patient p is identified as “PE positive.” At the patient-level the focus may be on correct classification of patients rather than individual candidates, and thus facilitates the effective distinction of PE patients from non-PE patients. A geometric interpretation in a 2-D feature space is illustrated in  FIG. 3 . 
         [0031]    Mathematically, distinguishing at least one TP candidate for each patient from the negative class (all FP candidates) may be equivalent to requiring that the minimum of the hinge errors (ξ) occurred on all the TP candidates associated with each patient be 0. In other words, in multiple-instance based learning (MIBL), what really matters for the TP candidates in each PE patient is the minimum of their hinge errors. For example, if there are 3 TP candidates in a patient and they are associated with ξ1=125, ξ 2 =70, and ξ 3 =0, then the patient is correctly classified, because there is a candidate with no hinge error, regardless of hinge errors with other candidates. This means that the summation of the hinge errors may be replaced for all true positive candidates in Equation 2 with the summation of the minimum of hinge errors occurred on all the TP candidate associated with each patient, yielding the following optimization problem: 
         [0000]      min w,ξ    γ∥w∥   1 +Σ p=1   n min{ξ i   ,i∈I   p   + }+Σ i∈I −ξ i  
 
         [0000]      s.t.  w   T   x     i   +β≧1−ξ i   , i∈I   + ,
 
         [0000]        w   T   x     i     +b≦− 1+ξ i   , i∈I   − ,
 
         [0000]      ξ i ≧0, i=1, . . . ,l.  (3)
 
         [0032]    To improve computation of this optimization problem, an equivalent optimization problem may be generated by introducing additional parameters λ i : 
         [0000]      min w,ξ,λ    γ∥w∥   1 +Σ p=1   n (Σ i∈I     p   +λ i ξ i )+Σ i∈I −ξ i  
 
         [0000]      s.t.  w   T   x     i     +b≧ 1−ξ i   , i∈I   + ,
 
         [0000]        w   T   x     i     +b≦− 1+ξ i   , i∈I   − ,
 
         [0000]      ξ i ≧0, i=1, . . . ,l,
 
         [0000]      Σ i∈I     p   +λ i =1, λ i ≧0,  i∈I   p   +   , p= 1, . . . , n.   (4)
 
         [0033]    One mathematical formulation for embolus-level diagnosis for use in certain embodiments of the invention described above is now presented. Assume that totally l candidates are generated by a candidate generator, each represented by a feature vector x i  associated with a label y i . The label y i =1 if the candidate is a true positive, otherwise y i =−1. Let I +  be a set containing indices of true positive candidates and I −  be an index set for false positive candidates. Let m be the total number of embolus marked by expert radiologists in the n image datasets. Denote I j  as the index set of the candidates that detect the j-th embolus, j=1, . . . , m. As a result, ∪ j=1 , . . . ,  m I j =I + . Notice that I j ∩I k =φ for j≠k, because there is no candidate that detects more than one embolus. 
         [0034]    Refer to Equation 1, to help formulate multiple instance-based classification at the embolus level, the problem may be rewritten by separating candidates into true positives and false positives: 
         [0000]      min w,ξ    γ∥w∥   1 +Σ i∈I +ξ i +Σ i∈I −ξ i  
 
         [0000]      s.t.  w   T   x     i     +b≧ 1−ξ i   , i∈I   + ,
 
         [0000]        w   T   x     i     +b≦− 1+ξ i   , i∈I   − ,
 
         [0000]      ξ i ≧0, i=1, . . . ,l.  (5)
 
         [0035]    Embolus-level detection may explore the observation that once any candidate in is classified as positive, the j-th embolus is identified. It focuses on correct classification of emboli rather than individual candidates, and thus facilitates the reduction of false positives by ignoring redundant “outlier” true positives when there are other true positives that can be easily classified correctly. A geometric interpretation in a 2-D feature space may be used in certain embodiments to illustrate this data. 
         [0036]    Now, distinguishing at least one candidate for each embolus from the negative class is equivalent to requiring that the minimum of the hinge errors (ξ) occurred on all the candidates associated with each embolus be 0. In other words, in multiple instance-based classification, what really matters for each embolus is the minimum of hinge errors of its candidates. For example, if there are 3 candidates that detect an embolus and are associated with ξ 1 =0, ξ 2 =5, and ξ 3 =100, then the embolus is correctly detected, because there is a candidate with no hinge error, regardless of hinge errors with other candidates. This means that the summation of the hinge errors for all true positive candidates in Equation 5 may be replaced with the summation of the minimum of hinge errors occurred on all the candidate associated with each embolus, yielding the following optimization problem: 
         [0000]      min w,ξ    γ∥w∥   1 +Σ j=1   m min{ξ i   ,i∈I   j }+Σ i∈I −ξ i  
 
         [0000]      s.t.  w   T   x     i     +b≧ 1−ξ i   , i∈I   + ,
 
         [0000]        w   T   x     i     +b≦− 1+ξ i   , i∈I   − ,
 
         [0000]      ξ i ≧0, i=1, . . . ,l.  (6)
 
         [0037]    Given the great similarity of Equation 6 to Equation 3this optimization problem may be solved by introducing additional parameters λ i : 
         [0000]      min w,ξ,λ    γ∥w∥   1 +Σ j=1   m (Σ i∈I     j   +λ i ξ i )+Σ i∈I −ξ i  
 
         [0000]      s.t.  w   T   x     i     +b≧ 1−ξ i   , i∈I   + ,
 
         [0000]        w   T   x     i     +b≦− 1+ξ i   , i∈I   − ,
 
         [0000]      ξ i ≧0, i=1, . . . ,l,
 
         [0000]      Σ i∈I     j   λ i =1, λ i ≧0,  i∈I   j   , j= 1, . . . , m.   (7)
 
         [0038]    Algorithmically, the difference between patient-level diagnosis and embolus-level detection is that the former finds the minimum of hinge errors among all the TP candidates of each patient (see, e.g., Equation 3 above), while embolus-level detection does so on each embolus (see, e.g., Equation 6 above). Embodiments of the MIBL described above offer a unified framework for both patient-level diagnosis and embolus-level detection. PE detection imposes a unique classification problem with several distinct aspects from the traditional multiple instance learning (MIL) problems investigated in the machine learning community. The objective of traditional MIL may be to distinguish bags, each of which may have multiple instances. This implies that labels (either positive or negative) are associated with the bags rather than their instances, and each instance exclusively belongs to a bag. The bag information may be present throughout both training and testing phases, and the bag information may be utilized by the trained classifiers during run-time. However, PE detection generally attempts to classify at least one TP candidate of a patient (or an embolus) correctly. Therefore, in the training phase, based on the ground truth provided by radiologists, which candidate belongs to which embolus and thus which patient is known. As a result, each embolus may be regarded as a bag and its candidates as instances of the bag. In some embodiments, there may be no negative bags but only negative instances (false positive candidates), even though virtual negative bags may be created by regarding each individual negative instance as a negative bag with a single instance. For PE detection, such bag information may not be not available in run-time. Therefore, a trained classifier may be independent from the bag information in testing phase and run-time. False positives may be eliminated by selectively sacrificing the redundant true PE candidates (instances in positive bags) without hurting overall detection sensitivity, even though the objective remains to classify instances rather bags. 
         [0039]    Embodiments described above may implement multiple instance-based learning (MIBL), whereas the traditional MIL may be referred to as bag-based multiple instance learning. The MIBL embodiments described above are also applicable to the traditional MIL problems, when correctly classifying one instance in a bag implies correct classification of the bag. The proposed multiple instance-based learning described in embodiments above may also have many applications in domains other than pulmonary embolism detection. 
         [0040]    If implemented in firmware and/or software, the functions described above may be stored as one or more instructions or code on a computer-readable medium. Examples include non-transitory computer-readable media encoded with a data structure and computer-readable media encoded with a computer program. Computer-readable media includes physical computer storage media. A storage medium may be any available medium that can be accessed by a computer. By way of example, and not limitation, such computer-readable media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer. Disk and disc includes compact discs (CD), laser discs, optical discs, digital versatile discs (DVD), floppy disks and blu-ray discs. Generally, disks reproduce data magnetically, and discs reproduce data optically. Combinations of the above should also be included within the scope of computer-readable media. 
         [0041]    In addition to storage on computer readable medium, instructions and/or data may be provided as signals on transmission media included in a communication apparatus. For example, a communication apparatus may include a transceiver having signals indicative of instructions and data. The instructions and data are configured to cause one or more processors to implement the functions outlined in the claims. 
         [0042]    Although the present disclosure and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the disclosure as defined by the appended claims. Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, composition of matter, means, methods and steps described in the specification. As one of ordinary skill in the art will readily appreciate from the present invention, disclosure, machines, manufacture, compositions of matter, means, methods, or steps, presently existing or later to be developed that perform substantially the same function or achieve substantially the same result as the corresponding embodiments described herein may be utilized according to the present disclosure. Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, compositions of matter, means, methods, or steps.