  Image Super-Resolution (SR) techniques improve visual quality by enhancing
the spatial resolution of images. Quality evaluation metrics play a critical
role in comparing and optimizing SR algorithms, but current metrics achieve
only limited success, largely due to the lack of large-scale quality databases,
which are essential for learning accurate and robust SR quality metrics. In
this work, we first build a large-scale SR image database using a novel
semi-automatic labeling approach, which allows us to label a large number of
images with manageable human workload. The resulting SR Image quality database
with Semi-Automatic Ratings (SISAR), so far the largest of SR-IQA database,
contains 8,400 images of 100 natural scenes. We train an end-to-end Deep Image
SR Quality (DISQ) model by employing two-stream Deep Neural Networks (DNNs) for
feature extraction, followed by a feature fusion network for quality
prediction. Experimental results demonstrate that the proposed method
outperforms state-of-the-art metrics and achieves promising generalization
performance in cross-database tests. The SISAR database and DISQ model will be
made publicly available to facilitate reproducible research.
