Features & Benefits
- Fast, Accurate, Repeatable,
and Objective Picture Quality Measurement
- Predicts DMOS (Differential
Mean Opinion Score) Measurement based on Human Vision System Model.
- Picture Quality Measurements can be made on a Variety of HD
Video Formats (1080i, 720p) and SD Video Formats (525 or 625)
- Makes Picture Quality Comparison across Different Resolutions
from HD to SD, or HD/SD to CIF
- User-configurable Viewing
Condition and Display Models for Reference and Comparison
- Attention/Artifact Weighted Measurement
- Automatic Temporal
and Spatial Alignment
- Easy Regression Testing and Automation
using XML Scripting with "Export/Import" File
- Multiple Results
View Options
- Optional SD/HD SDI Interface with Simultaneous
Generation/Capture, 2-channel Capture and 2-channel Generation with
Swap-channel Capability
- Preinstalled Sample Reference and
Test Sequences
Applications
- CODEC Design, Optimization,
and Verification
- Conformance Testing, Transmission Equipment,
and System Evaluation
- Digital Video Mastering
- Video
Compression Services
- Digital Consumer Product Development
and Manufacturing
Picture
Quality Analysis System
The PQA500 is the latest-generation
Picture Quality Analyzer built on Tektronix’ Emmy Award winning PQA200/300.
Based on the concepts of the human vision system, the PQA500 provides
a suite of repeatable, objective quality measurements that closely
correspond with subjective human visual assessment. These measurements
provide valuable information to engineers working to optimize video
compression and recovery, and maintaining a level of common carrier
and distribution transmission service to clients and viewers.
Compressed Video Requires New Test Methods
The true measure
of any television system is viewer satisfaction. While the quality
of analog and full-bandwidth digital video can be characterized indirectly
by measuring the distortions of static test signals, compressed television
systems pose a far more difficult challenge. Picture Quality in a
compressed system can change dynamically based on a combination of
data rate, picture complexity, and the encoding algorithm employed.
The static nature of test signals does not provide true characterization
of picture quality. A test scene with natural content and motion can
be used, with human viewers reporting the results, but this method
of evaluating the capabilities of a compressed video system is inefficient
and not very objective. Subjective testing with human viewers is impractical
for CODEC design and operational quality evaluation. The PQA500 provides
a fast, practical, repeatable, and objective measurement alternative
to subjective evaluation of picture quality.
Human viewer testing
has been traditionally conducted as described in ITU-R Rec. BT.500-11.
A test scene with natural content and motion is displayed in a tightly
controlled environment, with human viewers expressing their opinion
of picture quality to create a Differential Mean Opinion Score, or
DMOS. Extensive testing using this method can be refined to yield
a consistent subjective rating. However, this method of evaluating
the capabilities of a compressed video system can be inefficient,
taking several weeks to months to perform the experiments. This test
methodology can be extremely expensive to complete, and often the
results are not repeatable. Thus, subjective DMOS testing with human
viewers is impractical for the CODEC design phase, and inefficient
for ongoing operational quality evaluation. The PQA500 provides a
fast, practical, repeatable, and objective measurement alternative
to subjective DMOS evaluation of picture quality.
User Interface of PQA500. Showing reference, test sequences,
with difference map and statistical graph.
System Evaluation
The PQA500 can be used for installation,
verification, and troubleshooting of each block of the video system
because it is video technology agnostic: any visible differences between
video input and output from processing components in the system chain
can be quantified and assessed for video quality degradation. Not
only can CODEC technologies be assessed in a system, but any process
that has potential for visible differences can also be assessed. For
example, digital transmission errors, format conversion (i.e. 1080i
to 480p in set-top-box conversions), 3-2 pull-down, analog transmission
degradation, data errors, slow display response times, frame rate
reduction (for mobile transmission and videophone teleconferencing),
and more can all be evaluated, separately or in any combination.
How It Works
The PQA500 takes two video files as inputs:
a reference video sequence and a compressed, impaired, or processed
version of the reference. First, the PQA500 performs a spatial and
temporal alignment between the two sequences, without the need for
a calibration stripe embedded within the video sequence. Then the
PQA500 analyzes the quality of the test video, using measurements
based on the human vision system and attention models, and then outputs
quality measurements that are highly correlated with subjective assessments.
The results include overall quality summary metrics, frame-by-frame
measurement metrics, and an impairment map for each frame. The PQA500
also provides traditional picture quality measures such as PSNR (peak
signal-to-noise ratio) as an industry benchmark impairment diagnosis
tool for measuring typical video impairments and detecting artifacts.
Each reference video sequence and test clip can have different
resolutions and frame rates. The PQA500 can provide picture quality
measurement between HD vs SD, SD vs CIF, or any combination. This
capability supports a variety of repurposing applications such as
format conversion, DVD authoring, IP broadcasting, and semiconductor
design. The PQA500 can also support measurement clips with long sequence
duration, allowing a video clip to be quantified for picture quality
through various conversion processes.
Prediction of Human
Vision Perception
PQA500 measurements are developed from the
human vision system model and additional algorithms have been added
to improve upon the model used in the PQA200/300. This new extended
technology allows legacy PQR measurements for SD while enabling predictions
of subjective quality rating of video for a variety of video formats
(HD, SD, CIF, etc.). It takes into consideration different display
types used to view the video (for example, interlaced or progressive
and CRT or LCD) and different viewing conditions (for example, room
lighting and viewing distance).
Picture Quality Analysis
System
A model of the human vision system has
been developed to predict the response to light stimulus with respect
to the following parameters:
- Contrast including supra-threshold
- Mean Luminance
- Spatial Frequency
- Temporal
Frequency
- Angular Extent
- Temporal Extent
- Surround
- Eccentricity
- Orientation
- Adaptation
effects
A: Modulation
Sensitivity vs. Temporal Frequency
B: Modulation Sensitivity vs. Spatial Frequency
This model has been calibrated, over the appropriate
combinations of ranges for these parameters, with reference stimulus-response
data from vision science research. As a result of this calibration,
the model provides a highly accurate prediction.
The graphs
above are examples of scientific data regarding human vision characteristics
used to calibrate the human vision system model in the PQA500. Graph
(A) shows modulation sensitivity vs. temporal frequency, and
graph (B) shows modulation sensitivity vs. spatial frequency.
The use of over 1400 calibration points supports high-accuracy measurement
results.
C: Reference Picture
D: Perceptual Contrast Map
Picture (C) is a single frame from the reference sequence
of a moving sequence, and picture (D) is the perceptual contrast
map calculated by the PQA500. The perceptual contrast map shows how
the viewer perceives the reference sequence. The blurring on the background
is caused by temporal masking due to camera panning and the black
area around the jogger shows the masking effect due to the high contrast
between the background and the jogger. The PQA500 creates the perceptual
map for both reference and test sequences, then creates a perceptual
difference map for use in making perceptual-base, full-reference picture
quality measurements.
E: Reference
F: Test
G: PSNR Map
H: Perceptual Difference Map for DMOS
Comparison of Predicted DMOS with PSNR
In the
example above, Reference (E) is a scene from one of the VClips
library files. The image Test (F), has been passed through
a compression system which has degraded the resultant image. In this
case the background of the jogger in Test (F) is blurred compared
to the Reference image (E). A PSNR measurement is made on the
PQA500 of the difference between the Reference and Test clip and the
highlighted white areas of PSNR Map (G) shows the areas of
greatest difference between the original and degraded image. Another
measurement is then made by the PQA500, this time using the Predicted
DMOS algorithm and the resultant Perceptual Difference Map for DMOS
(H) image is shown. Whiter regions in this Perceptual Contrast
Difference map indicate greater perceptual contrast differences between
the reference and test images. In creating the Perceptual Contrast
Difference map, the PQA500 uses a human vision system model to determine
the differences a viewer would perceive when watching the video.
The Predicted DMOS measurement uses the Perceptual Contrast Difference
Map (H) to measure picture quality. This DMOS measurement would
correctly recognize the viewers perceive the jogger as less degraded
than the trees in the background. The PSNR measurement uses the difference
map (G) and would incorrectly include differences that viewers do
not see.
Attention Map Example: The jogger is highlighted
Attention Model
The PQA500 also incorporates
an Attention Model that predicts focus of attention. This model considers:
- Motion of objects
- Skin Coloration (to identify
people)
- Location
- Contrast
- Shape
- Size
- Viewer distraction due to noticeable quality artifacts
These attention parameters can be customized to give greater
or less importance to each characteristic. This allows each measurement
using an attention model to be user-configurable. The model is especially
useful to evaluate the video process tuned to the specific application.
For example, if the content is sports programming, the viewer is expected
to have higher attention in limited regional areas of the scene. Highlighted
areas within the attention image map will show the areas of the image
drawing the eye's attention.
Artifact Detection Settings
Artifact Detection
Artifact Detection reports
a variety of different changes to the edges of the image:
- Loss of Edges or Blurring
- Addition of Edges or Ringing/Mosquito
Noise
- Rotation of Edges to Vertical and Horizontal or Edge
Blockiness
- Loss of Edges Within an Image Block or DC Blockiness
They work as weighting parameters for subjective and objective
measurements with any combination. The results of these different
measurement combinations can help to improve picture quality through
the system.
For example, artifact detection can help answer
questions such as: “Will the DMOS be improved with more de-blocking
filtering?” or, “Should less prefiltering be used?”
If edge-blocking
weighted DMOS is much greater than blurring-weighted DMOS, the edge-blocking
is the dominant artifact, and perhaps more de-blocking filtering should
be considered.
In some applications, it may be known that added
edges, such as ringing and mosquito noise, are more objectionable
than the other artifacts. These weightings can be customized by the
user and configured for the application to reflect this viewer preference,
thus improving DMOS prediction.
Likewise, PSNR can be measured
with these artifact weightings to determine how much of the error
contributing to the PSNR measurement comes from each artifact.
The Attention Model and Artifact Detection can also be used in
conjunction with any combination of picture quality measurements.
This allows, for example, evaluation of how much of a particular noticeable
artifact will be seen where a viewer is most likely to look.
Comprehensive Picture Quality Analysis
The PQA500 provides
full-reference (FR) picture quality measurements that compare the
luminance signal of reference and test videos. It also offers some
no-reference (NR) measurements on the luminance signal of the test
video only. Reduced reference (RR) measurements can be made manually
from differences in no-reference measurements. The suite of measurements
includes:
- Critical Viewing (Human Vision System Model-based,
Full-reference) Picture Quality
- Casual Viewing (Attention
Weighted, Full-reference, or No-reference) Picture Quality
- Peak Signal-to-Noise Ratio (PSNR, Full-reference)
- Focus
of Attention (Applied to Both Full-reference and No-reference Measurements)
- Artifact Detection (Full-reference, Except for DC Blockiness)
- DC Blockiness (Full-reference and No-reference)
The PQA500 supports these measurements through preset and user-defined
combinations of display type, viewing conditions, human vision response
(demographic), focus of attention and artifact detection, in addition
to the default ITU BT-500 conditions. The ability to configure measurement
conditions helps CODEC designers evaluate design trade-offs as they
optimize for different applications, and helps any user investigate
how different viewing conditions affect picture quality measurement
results. A user-defined measurement is created by modifying a pre-configured
measurement or creating a new one, then saving and recalling the user-defined
measurement from the Configure Measure dialog menu.
Configure Measure Dialog
Edit Measure Dialog
Easy-to-Use
Interface
The PQA500 has two modes: measurement and review.
The measurement mode is used to execute the measurement selected in
the Configure Dialog. During measurement execution, the summary data
and map results are displayed on-screen and saved to the system hard
disk. The review mode is used to view previously saved summary results
and maps created either with the measurement mode or XML script execution.
The user can choose multiple results in this mode and compare each
result side by side using the synchronous display in Tile Mode. Comparing
multiple results maps made with the different CODEC parameters and/or
different measurement configurations enables easy investigation of
the root cause of any difference.
Statistical Graph
Multiple Result Display
Resultant maps can
be displayed synchronously with the reference and test video in a
tiled or overlaid display. Individual videos can also be viewed at
full resolution, one at a time, to accommodate resolutions greater
than what the tiled display can accommodate. In Overlay Display, the
user can control the mixing ratio with the fader bar, enabling co-location
of difference map, reference, and impairments in test video sequences.
Summary measures of standard parameters and perceptual summation
metrics for each frame and overall video sequence are provided. Summary
measure results are displayed as data lists, maps, or graphs with
a bar chart during video playback.
Error logging and alarms
are available to help users efficiently track down the cause of video
quality problems.
The logging parameters are:
- Registration information found in automatic temporal and spatial
alignment: cropping, scale, shift in horizontal and vertical, Y gain
and DC offset.
- Alignment confidence (cross-correlation coefficient):
(1.0 is perfect match).
- Logs of when measurement values per
frame exceed either warning or error levels (configurable by user
through the summary node).
All results, data, and graphs
can be recalled to the display for examination.
Automatic
Temporal/Spatial Alignment
The PQA500 supports automatic temporal
and spatial alignment, as well as manual alignment.
The automatic
spatial alignment can measure the cropping, scale, and shift in each
dimension, even across different resolutions (for example, when aligning
SD to HD video). If extra blanking is present within the standard
active region, it is measured as cropping when this function is enabled.
Auto Spatial and Temporal Alignment Between CIF vs
HD Pictures
The automatic spatial and temporal
alignment allows picture quality measurement between reference and
test videos of different resolutions and frame rates.
Script Sample
Import/Export
script in Configure Measure Dialog
Result File Sample
Automation Test
with XML Scripting
In the CODEC debugging/optimizing process,
the designer can repeat several measurement routines as CODEC parameters
are revised. Automated regression testing with XML scripting can ease
the restrictions of manual operation by allowing the user to write
a series of measurement sequences within an XML script. The script
file can be exported from or imported to the measurement configuration
menu to create and manage the script files easily. Measurement results
of the script operation can be viewed by using either the PQA500 user
interface or any spreadsheet application that can read the created
.csv file format as a summary. Up to four scripts can be executed
simultaneously for faster measurement results.
Generation/Capture
Optional SD/HD
SDI Interface
An optional SD/HD SDI interface enables both
generation and capture of SDI video with 3 modes of simultaneous operations.
Simultaneous generation and capture lets the user playout the
reference video clips directly from the PQA500 into the device under
test. The test output from the device can then be simultaneously captured
by the PQA500. This saves the user from having to use an external
video source to apply any required video input to the device under
test. With this generation capability, files created by video editing
software can be directly used as reference and test sequences for
picture quality measurements.
Simultaneous 2-channel capture
lets the user capture two live signals to use as reference and test
videos in evaluating the device under test in operation.
To
accommodate equipment processing delay that may be present in the
system the user can use the Delay Start function when capturing video.
Using Delayed Start will minimize the number of unused overhead frames
in the test file and enable faster execution of the auto temporal
alignment in the measurement.
Simultaneous 2-channel generation
capability supports the 2 types of subjective testing. With 2 displays,
the user can check the reference and test video content on side-by-side
monitors.
With 1 display and swap-channel capability, the user
can check the reference and test video content without moving the
eye's focus point.
2-channel Generation
2-channel Capture
Supported
File Formats for SD/HD SDI Interface
The SD/HD SDI video option
can generate SDI video from files in the following formats (8-bit
unless otherwise stated):
- .yuv (UYVY, YUY2)
- .v210 (10 bit, UYVY, 3 components in 32 bits)
- .rgb (BGR24)
- .avi (uncompressed, BGR32 (discard alpha channel) / BGR24 /
UYVY / YUY2 / v210)
- .vcap (created by PQA500 video capture)
- .vcap10 (10 bit, created by PQA500 video capture)
|
Option
|
Supported Frame Geometry
|
Formats Supported by SD/HD SDI Interface
|
|
SD-SDI
|
720×486, 720×576
|
525i/59.94, 625i/50
|
|
HD-SDI
|
1280×720, 1920×1080
|
720p/50, 720p/59.94, 720p/60
|
|
1080i/50, 1080i/59.94, 1080i/60
|
|
1080p/23.98, 1080p/23.98SF, 1080p/24, 1080p/24SF, 1080p/25,
1080p/29.97, 1080p/30
|
Supported File Formats for Measurement
All
formats support 8-bit unless otherwise stated, and measurements use
8MSBs:
- .yuv (UYVY, YUY2, YUV4:4:4, YUV4:2:0_planar)
- .v210 (10 bit, UYVY, 3 components in 32 bits)
- .rgb
(BGR24, GBR24)
- .avi (uncompressed, BGR32 (discard alpha channel)
/ BGR24 / UYVY / YUY2 / v210)
- ARIB ITE format (4:2:0 planar
with 3 separate files (.yyy, .bbb, .rrr))
- .vcap (created
by PQA500 video capture)
- .vcap10 (10 bit, created by PQA500
video capture)
Preinstalled Video Sequences
|
Sequence
|
Resolution
|
Formats
|
Clips
|
|
Vclips
|
1920×1088
|
YUV4:2:0 planar
|
V031202_Eigth_Ave, V031255_TimeSquare, V031251_Stripy_jogger
|
|
1920×1080
|
UYVY
|
V031251_Stripy_jogger
|
|
1280×720
|
UYVY, YUV4:2:0 planar
|
V031002_Eigth_Ave, V031055_TimeSquare, V031051_Stripy_jogger
with 3/10/26 Mbps
|
|
864×486
|
YUV4:2:0 planar
|
Converted V031051_Stripy_jogger with 2/4/7 Mbps
|
|
320×180
|
YUV4:2:0 planar
|
Converted V031051_Stripy_jogger with 1000/1780/2850 kbps
|
|
PQA300 without Trigger
|
720×486
|
UYVY
|
Ferris, Flower, Tennis, Cheer with 2 Mbps_25 fps
|
|
720×576
|
UYVY
|
Auto, BBC, Ski, Soccer
|
|
PQA300 with Trigger
|
720×486
|
UYVY
|
Mobile with 3/6/9 Mbps
|
|
720×576
|
UYVY
|
Mobile with 3/6/9 Mbps
|
Related Products
Recent updates and more
details are available in each data sheet.
Video Stream
Analysis
MTS4EA Elementary Stream Analysis
Software for VC-1, H.264/AVC, MPEG-2, MPEG-4, H.263 and 3GPP Standards
Features & Benefits
- Next Generation (VC-1,
H.264/AVC, MPEG-4, and 3GPP) and Legacy (MPEG-2 and H.263) CODEC Support
- Frame-by-Frame and Block-by-Block Analysis to Allow Easy CODEC
Comparison
- Easy-to-Interpret Detailed Graphical Displays
(Requires User-installed Microsoft® Excel)
- Comprehensive
Semantic Trace File Output to Determine Block-by-Block Encoder Decision
Making
- AV Delay Measurement (option)
- Audio Decode
and Analysis (option)
- Synchronized Audio and Video Analysis
- Real-Time and Non-Real-Time Decoding and Analysis of Compressed
Video Streams (Dependant on PC Performance)
- Bit Stream Editing
- Batch Mode to Allow Automated Testing
- YUV Decoded Video
Output for Baseband Video Analysis and Picture Quality Analysis
- Extraction of Elementary Stream from Transport Stream
- Available as Single-user Local License for PC and Tektronix Instruments
or Server-based Floating License.
Picture Quality Testing
Vclips - For Video Testing and Evaluation
Features &
Benefits
|
Characteristic
|
Description
|
|
Vclips are a diverse set of short video clips
designed to test video encoders and decoders to the limits of their
abilities.
|
|
Video Sizes
|
Test with many different video sizes; Sub-QCIF, QCIF, CIF,
D1, HD (720p and 1080i)
|
|
Difficult Subjects
|
Test with fine detail, night time, areas of high contrast,
sharp borders, uniform areas, bright and dull colors
|
|
Visual Objects
|
People, buildings, vehicles, trees, landscapes, clouds, water,
and synthetic objects
|
|
Movement
|
Fast, slow, uniform, random, multiple moving objects. Also
pan, zoom, and rotate
|
|
Test Card Sequences
|
Precisely defined motion, bright colors, dull colors, lines,
patterns, and grids. Also strobing and white noise
|
