  In order to be able to deliver today's voluminous amount of video contents
through limited bandwidth channels in a perceptually optimal way, it is
important to consider perceptual trade-offs of compression and space-time
downsampling protocols. In this direction, we have studied and developed new
models of natural video statistics (NVS), which are useful because high-quality
videos contain statistical regularities that are disturbed by distortions.
Specifically, we model the statistics of divisively normalized difference
between neighboring frames that are relatively displaced. In an extensive
empirical study, we found that those paths of space-time displaced frame
differences that provide maximal regularity against our NVS model generally
align best with motion trajectories. Motivated by this, we build a new video
quality prediction engine that extracts NVS features from displaced frame
differences, and combines them in a learned regressor that can accurately
predict perceptual quality. As a stringent test of the new model, we apply it
to the difficult problem of predicting the quality of videos subjected not only
to compression, but also to downsampling in space and/or time. We show that the
new quality model achieves state-of-the-art (SOTA) prediction performance
compared on the new ETRI-LIVE Space-Time Subsampled Video Quality (STSVQ)
database, which is dedicated to this problem. Downsampling protocols are of
high interest to the streaming video industry, given rapid increases in frame
resolutions and frame rates.
