Image dataset for comparison of subjective methods for image quality assessment

Author: Bangor University

Partner: No

Contact: Rafał K. Mantiuk (




Subjective scores: true

SRC: 10

HRC: 6

Ratings: 17

Method: SS ACR, DS ACR, PC


The major methods for data analysis are reviewed, such as establishing confidence intervals, statistical testing and retrospective power analysis. Two methods of visualizing ranking results together with the meaningful information about the statistical and practical significance are explored. Four most prominent subjective quality assessment methods are compared: single-stimulus, double-stimulus, forced-choice pairwise comparison, and similarity judgements. Selected 10 images from the Kodak Photo CD photo sampler collection are used in the experiment.


Images used in the experiment are available at: Links: The ZIP file contains all images used to measure quality in the study, both distorted and test images: Link: Experiemntal data are available in the CSV format: Link:

References and Citation

We do not restrict any rights to use the data sets as long as the original publication and the source of the data is referred [MTM12].


  • MTM12: Rafał K. Mantiuk, Anna Tomaszewska, and Radosław Mantiuk, Comparison of four subjective methods for image quality assessment, Computer Graphics Forum, 31(8), pp. 2478–2491, 2012.