Ranking : A compilation of recent studies and methodologies

Due to the exponentially increasing number of amateur photographers and their photo collections, there is a growing need for automatic aesthetic assessment within amateur photography, both to rank within the collections and to help amateur photographers take better pictures. The problem is inherently difficult not only in sterilizing the data set from preference for content, but due to the soft and personal preferences inherent in photography appreciation. While many labs are beginning to make indents into this problem with algorithms based on historically taught features, each has taken their own path. This paper looks at three recent studies ([1], [2], and [3] ) and compares their data sets, algorithms, and user testing. Finally this paper does a new analysis on the Panorama data and individual users to yield promising results in line with the other papers, getting an average of 89% with all features and 91% with spacial features. It also followed [3] ’s lead and attempted to maintain the maximum efficiency with as few interpretable features as possible. Its results lend credence to the concept that different features are more significant in distinguishing the good and bad in amateur photography than professional photography. In the world of digital cameras and cheap data space, the field of photography and art production is no longer limited to classically trained or monetarily invested individuals. Consequently, photography has been inundated with a deluge of amateur photographs. Not only does this distill the quality of the photographs produced, but it perhaps changes the conception of what aesthetic qualities are needed to call a photo good or powerful. Furthermore, with the significantly lowered cost of time and money per pixel, collections of photos has grown within the private space of personal hard drives. Theres a need to be able to distill personally preferred aesthetics in order to help both professional and amateur photographers sort the good from the bad. Within this world, the need for aesthetic assessment arises, whether to glean the bad photos from the good, to improve all photos, or as suggestions to photographers for better shots. However this problem is inherently troublesome in finding a good dataset, free from content bias and classically trained phoCopyright c © 2012, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. tographers. Not only is the current free online public content mostly unlabeled but a ranking of a photos quality is greatly related to the photos content. A blurry photo of a sleeping kitten can rate much higher than a well composed photo of a table. Historical efforts (such as [4], [5], and [6]) have used the DPChallenge.com photo contest database of 12,000 photos due to its robust labeling. Table 1 shows the cited aesthetic assessment studies, their databases, and results. However that particular database is not only based on users preferences skewed towards significant content but a photography sample potentially skewed towards classically taught photographers (aka those who want to make money off their content). Others have looked into other databases [7], and [8] but have not yet come up with one free from the content bias. Three more recent studies, [1], [2], and [3] have made valiant efforts at creating an aesthetic assessment tools and tested them on data suitably less noisy with content bias. They not only work on eliminating content bias within their databases, they look into novel ways on making accurate algorithms based on classic photography rules, and follow up with user studies to assess the accuracy of their algorithms on new data. [2]uses the DBChallenge.com photographs to evaluate while [3] uses a 2 class Fickr.com 650 photo database and [1] uses AB game generated 100 photo database with 2,470 unique comparisons with ten user inputs apiece. While each looks at particular classic photography strengths,[3] relies on simply on rule of thirds and visual weight, and the horizon line, [2]s algorithm relies heavily on what other papers call simplicity, hue, and contrast but from a more bottom up perspective, and [1] relies heavy on rule of thirds, occlusion, subject size ratio, and spacial from a bottom up perspective. Each then rates how well their algorithm does in the real world: [3] uses their algorithm to first edit the photos and then compare them to the original photo; [2] takes new photos and edits them for best and worst optimization and then ask users their preferences; [1] similarly predicts on users preference between two photos. The following sections go into further details on each studies database, algorithms, and user studies. This paper looks into the flaws in each and creates new results with the Panorama database using the strengths of each. It also suggests where we take the Panorama data from here to make it more robust and widely used. Paper Data Set source Algorithm Features Accuracy on Test Set [4] DPchallenge.com Naive Bayes Classifier Blur, Simplicity, Hue Count, etc 76% [6] DPchallenge.com Adaboost Clarity, Composition,Simplicity, Lighting, Color combination 93% [5] DPchallenge.com ListNet Rule of Thirds, Simplicity, Contrast 93% [2] DPchallenge.com Adaboost (BOaP) Simplicity, Saliency, Texture, Color hue, etc 92% [3] Flickr.com soft-mapped SVR Rule of thirds, golden weight ratio, 86% [8] Photo.net Naive Bayes Classifier 56 features [1] Panorama SVR Rule of Thirds, Spacing, Occlusion, Number of objects, etc 65% This Paper Panorama SVR Horizon Tilt, Cropped Images, Sight Line, Occlusions 85% This Paper Panorama SVR Spacial BOaP 91% Figure 1: Past Studies, their features and their reported accuracies