How libavif Maps Decoded YUV Data to RGB

This article explains how the libavif library converts decoded YUV image data back into the RGB color space for accurate display on screens. It covers the retrieval of color metadata, the application of mathematical transformation matrices based on color standards, the handling of chroma subsampling, and how color management systems use this data to ensure visual fidelity.

Step 1: Retrieving Color Metadata (CICP and ICC Profiles)

To accurately convert YUV to RGB, libavif must first understand the color properties of the encoded image. AVIF files store this information using Coding-Independent Code Points (CICP) or embedded ICC profiles. During the decoding phase, libavif extracts these values to determine: * Color Primaries: Define the color gamut (e.g., BT.709, BT.2020). * Transfer Characteristics: Define the gamma curve (e.g., sRGB, PQ, HLG). * Matrix Coefficients: Define the exact mathematical relationship used to encode RGB into YUV.

If an ICC profile is present, libavif exposes it to the calling application, allowing a color management system (like Little CMS) to handle the final color mapping.

Step 2: Handling Chroma Upsampling

AVIF images often use chroma subsampling (such as YUV 4:2:0 or 4:2:2) to save bandwidth by reducing color resolution. Before converting to RGB, libavif must scale the lower-resolution U and V channels to match the full resolution of the Y (luminance) channel.

The library supports multiple upsampling filters, ranging from simple bilinear interpolation to higher-quality, sharper algorithms. This step ensures that every pixel has a corresponding Y, U, and V value before the color space transformation matrix is applied.

Step 3: Applying the Transformation Matrix

Once the YUV channels are at full resolution, libavif maps the YUV values to RGB using a 3x3 transformation matrix. The exact coefficients of this matrix are determined by the Matrix Coefficients extracted in Step 1 (such as BT.601 for standard definition, BT.709 for high definition, or BT.2020 for wide color gamut).

The conversion formulas also account for the signal range: * Full Range: Uses the entire digital spectrum (0–255 for 8-bit depth). * Limited (Studio) Range: Restricts luminance (Y) to 16–235 and chrominance (U/V) to 16–240. libavif scales these values to the full 0–255 range during the matrix multiplication to prevent washed-out colors.

Step 4: Library Implementations and Hardware Acceleration

To perform these matrix calculations efficiently, libavif can utilize different backends. While it contains built-in C-based conversion routines for maximum compatibility, it highly prefers fast, hardware-accelerated libraries when compiled with them: * libyuv: A highly optimized library that uses SIMD assembly instructions (such as AVX2 on x86 or NEON on ARM) to perform YUV-to-RGB conversions in parallel, significantly reducing CPU overhead. * sharpYUV: Used specifically for high-quality, visually lossless chroma distribution during conversions.