Thanks for your interest in the Picture Quality Test web application, formerly pqtest.cascadestream.com
This application is not currently online. Please contact us if you would like to see or discuss this application.

How do you measure video quality?

We often see poor quality video in our daily lives. Streaming video from on-demand content providers, satellite and cable channels, and broadcast television all use video processing techniques that are susceptible to visual distortion. The first step to improve video quality is to measure it, but how?

Video quality evaluation is based on human perception. Subjective evaluation relies on human observers looking at video samples and scoring them according to their opinion. The purpose of the Picture Quality Test web application is to host picture quality evaluation test sessions, where users score the quality of  pictures they see.

There are various impairment or quality scales that are commonly used. The one used in the evaluation hosted here comes from the ITU recommendations ITU-R BT.500 and ITU-T P.913

• 5 – Excellent
• 4 – Good
• 3 – Fair
• 2 – Poor
• 1 – Bad or Unacceptable

How scores are used

In this evaluation, opinion scores are collected from each user for a set of about 100 pictures. These scores are averaged with those of other observers to form a Mean Opinion Score, or MOS. The MOS is the quantitative quality value for each picture, according to the methods of the ITU recommendations.

Some of the pictures in the test set are not distorted or impaired by video processing; they are in the best form obtainable from the video post-production facility. These are reference pictures, and the distorted or impaired versions of the same scene are test pictures. Another score that is commonly computed is derived from the difference in opinion scores between test and reference pictures. Averaging difference scores over many observers results in a Differential Mean Opinion Score, or DMOS. The standard deviation of DMOS can be smaller than MOS because the difference operation removes the general bias of each observer, for example when an observer consistently rates picture quality high or low across the whole set of pictures.

The video pictures on this site along with their MOS and DMOS scores form a useful library for research on modeling picture quality and making it better.

Standards

The following recommendations from the International Telecommunication Union form the basis for the evaluations hosted on this site:

• • ITU-R BT.500-13
  • ITU-T P.910 (04/2008)
  • ITU-T P.913 (01/2014)