A computer implemented image retrieval method comprising: receiving an image descriptor of a query image and non-image patient data associated with the query image, the query image indicating a medical abnormality, the image descriptor of the query image being a vector representing the query image; receiving, for each of a plurality of candidate images stored in a database, an image descriptor of the candidate image and medical abnormality data (D I) indicating a medical abnormality known to be indicated by the candidate image, the image descriptor of the candidate image being a vector representing the candidate image; determining, for each of the plurality of candidate images, a similarity metric representing a similarity between the image descriptor of the query image and the image descriptor of the candidate image, the similarity metric being a distance in a vector space between the image descriptor of the query image and the image descriptor of the candidate image; determining, for each of the plurality of candidate images, a first probability P(D=D I | N) of the medical abnormality indicated by the query image being the medical abnormality known to be indicated by the candidate image given the non-image patient data associated with the query image, wherein the first probability P(D=D I | N) is determined using an equation: P D = D I N = P N D = D I P D = D I P N where P(N | D=D I) is a first likelihood of observing the non-image patient data given the medical abnormality (D I) known to be indicated by the candidate image, P(D=D I) is a second probability of an occurrence of the medical abnormality (D I) known to be indicated by the candidate image among medical abnormalities, and P is the marginal likelihood of observing the non-image patient data among other non-image patient data; determining, for each of the plurality of candidate images, based on the determined similarity metric and the determined first probability P(D=D I | N), a score S; and retrieving one or more of the candidate images from the database in accordance with the determined scores S. receiving an image descriptor of a query image and non-image patient data associated with the query image, the query image indicating a medical abnormality, the image descriptor of the query image being a vector representing the query image; receiving, for each of a plurality of candidate images stored in a database, an image descriptor of the candidate image and medical abnormality data (D I) indicating a medical abnormality known to be indicated by the candidate image, the image descriptor of the candidate image being a vector representing the candidate image; determining, for each of the plurality of candidate images, a similarity metric representing a similarity between the image descriptor of the query image and the image descriptor of the candidate image, the similarity metric being a distance in a vector space between the image descriptor of the query image and the image descriptor of the candidate image; determining, for each of the plurality of candidate images, a first probability P(D=D I | N) of the medical abnormality indicated by the query image being the medical abnormality known to be indicated by the candidate image given the non-image patient data associated with the query image, wherein the first probability P(D=D I | N) is determined using an equation: P D = D I N = P N D = D I P D = D I P N where P(N | D=D I) is a first likelihood of observing the non-image patient data given the medical abnormality (D I) known to be indicated by the candidate image, P(D=D I) is a second probability of an occurrence of the medical abnormality (D I) known to be indicated by the candidate image among medical abnormalities, and P is the marginal likelihood of observing the non-image patient data among other non-image patient data; determining, for each of the plurality of candidate images, based on the determined similarity metric and the determined first probability P(D=D I | N), a score S; and retrieving one or more of the candidate images from the database in accordance with the determined scores S.