Patent ID: 6494720
Filing Date: 2002-12-17
Classification: G06K

Abstract:
A method for procuring information in an objective way about classifiers' consistency and agreement in iteratively classifying in a subjective way (m) pre-determined constructed classification sets of (n) pre-determined and marked classifications objects, comprising:A) setting up a classification taxonomy including: i) for different defined types of visual classifications objects, a classification criterion is defined for each characteristic type of a classification object; ii) each classification criteria is named by a unique and selectable classification label, which is defined literary by a text and visually by a reference object; iii) the classification labels are organized into a classification taxonomy of related labels; iv) the classification taxonomy is electronically stored and displayed on a computer screen; B) setting up (n) number of classification objects, including: i) a person or a computerized system is selecting a set of digitized images for marking a plurality set of classification objects; ii) a classification object is marked by a person or computerized system by creating marking objects pointing at or are encircling parts or all of one or more images in an image set; iii) using marking objects as variable pointers to access information about the dynamic object classifications, the marking objects are electronically stored as a data structure consisting of two fields: an address field, defining the location of a marked classification object as parts or all of one or more images; an identifier field, containing an assigned label selected by a classifier when classifying a marked object; C) setting up (m) number of classification sets, including: i) (m) number of identical classification sets are setup manually or automatically by selecting for each classification set (n) number of the same marked classification objects as being defined in steps (B)(i)-(B)(iii); and D) classifying (n) number of marked objects in (m) number of classification sets, including: i) displaying simultaneously the classification taxonomy consisting of organized classification labels, and for each classification label a reference object and a textual description of the reference object; ii) displaying for each of the (m) number of classification sets, (n) number of marked classification objects and a marking object for each of the (n) number of marked objects; iii) inspecting and interpreting by a classifying system according to a subjective understanding of the classification taxonomy, the marking objects in an image set, in attempt to carry out visual classification in a consistent way and in agreement with other classifying systems, who also are attempting to practising in a consistent way the same classification rules embodied in the classification taxonomy; iv) based on a subjective comparison carried out by a classifying system between interpreted inspections of marked classification objects and interpretation of the textual and visual definitions of classification types of objects embodied in the taxonomy, said classifying system is instructed to select the most appropriate label from the accessible classification taxonomy for each marked classification object, wherein the classifier believes that the selected classification label is identifying a classification type most alike the marked classification object; v) the classifying system is then assigning the selected classification label to the marking object, which identify and represent the interpreted classification object; vi) the assignment of the selected classification label to a marking object, is stored electronically; vii) for classifying all images sets in a classification set, steps (D)(i)-(D)(vi) are repeated for (n) number of classification objects; viii) for carrying out (m) number of classification sets, step (D)(vii) is repeated (m) times.