H.264: What It Is and Why It Weighs 1,500 Times Less Than a PNG

⏱️ Reading time: 14 min

Record five seconds of 1080p video at 60 frames per second without compressing anything, and the file weighs almost 2 GB. The same clip encoded with the H.264 codec weighs 175 KB, 1,500 times less than a single PNG screenshot of those same pixels.

📑 En este artículo

1. TL;DR
2. What Is the H.264 Codec?
3. Why a 2003 Video Standard Still Matters
4. How It Works Under the Hood: Prediction, Frequency, and Entropy
   1. Inter-Frame Prediction: Encoding Only What Changed
   2. The Frequency Domain: The Discrete Cosine Transform
   3. Quantization: Deciding Which Detail Gets Discarded
   4. Entropy Coding: CABAC and CAVLC
5. Practical Examples and How to Get Started
6. Real-World Use Cases for H.264
7. Common Mistakes and Best Practices with H.264
8. Comparison: H.264 vs Other Video Codecs
9. Going Deeper: Macroblocks, Profiles, and the Deblocking Filter
10. Frequently Asked Questions
    1. What Does AVC Mean in H.264?
    2. Does H.264 Have a Licensing Cost?
    3. What’s the Difference Between H.264 and H.265/HEVC?
    4. Why Does YouTube Use VP9 or AV1 Instead of H.264?
    5. What Is a Baseline Profile in H.264?
11. References

That difference is no accident: it’s the result of chaining three compression techniques that attack different types of redundancy in the image, and understanding how they combine explains why Netflix, Blu-ray discs, your phone, and even a security camera still depend on a standard signed in 2003.

TL;DR

• The H.264 codec compresses a 300-frame video down to 175 KB: 1,500 times less than a single PNG screenshot of those pixels.
• Inter-frame prediction avoids repeating pixels that didn’t change from one frame to the next.
• Splitting each block into frequencies with the DCT makes it possible to discard detail the human eye doesn’t perceive.
• Quantization and CABAC compress what’s left of the block without inventing or repeating information.
• FFmpeg encodes and measures the real size of an H.264 video with a single command on Windows, macOS, or Linux.

What Is the H.264 Codec?

H.264 is a lossy video compression codec, standardized in 2003 by ITU-T and MPEG as MPEG-4 Part 10 (AVC). It reduces a video’s size by eliminating imperceptible detail and redundancy between frames, and it’s the format behind most of today’s streaming and security cameras.

The core idea can be explained with an analogy: think about building a drag race car. To gain speed, you strip out everything that doesn’t contribute to racing: the back seats, the subwoofer, the air conditioning. You’re left with only what matters, and the car weighs a fraction of the original.

That’s lossy compression: discarding information that isn’t needed for the final result to remain useful. A lossless format like PNG, on the other hand, keeps every original bit and allows the exact image to be reconstructed, but pays for it with a much heavier file. H.264 chooses the opposite route: bit by bit, it decides what to throw away.

Why a 2003 Video Standard Still Matters

Without compression, video is impossible to handle. An uncompressed 1080p signal at 60 Hz takes up close to 370 MB per second (1920 x 1080 pixels x 60 frames x 3 bytes of color). A 50 GB Blu-ray disc would hold less than two minutes of raw footage, and even today’s solid-state drives can’t sustain that write speed without stumbling.

That’s why, more than twenty years after its publication, the H.264 codec remains the format running internet streaming, Blu-ray discs, phone cameras, security cameras, and drones. Almost no corner of the video industry has fully left it behind, and even the newest standards use it as the mandatory benchmark.

How It Works Under the Hood: Prediction, Frequency, and Entropy

The H.264 codec doesn’t use a single trick to shed weight. It combines three independent layers, each squeezing out a different type of redundancy: the redundancy between one frame and the next, the redundancy within a single frame, and whatever is left over after the first two, in the form of statistical patterns.

flowchart LR
A["Original frame"] --> B["Prediction (I/P/B)"]
B --> C["Residual"]
C --> D["DCT per block"]
D --> E["Quantization"]
E --> F["CABAC or CAVLC"]
F --> G["H.264 bitstream"]

Inter-Frame Prediction: Encoding Only What Changed

Most frames in a video look a lot like the one before. If a fixed camera records an empty office, the desk and the wall occupy the same pixels frame after frame. Instead of saving that image again, H.264 saves just the difference relative to the reference frame, a technique called motion compensation.

To do this, the standard defines three frame types. I frames (intra) are encoded in full, without looking at other frames, and serve as a reference point. P frames (predictive) only store what changed relative to a previous frame. B frames (bidirectional) predict using both a previous and a later frame at the same time, which requires the decoder to receive frames out of order.

A group of frames that starts with an I frame and chains P and B frames until the next I frame is called a GOP (group of pictures). The longer the GOP, the cheaper the video gets on average, but the more expensive it becomes to skip to the middle, because the player has to go back to the last I frame and reconstruct the whole path up to the requested point.

flowchart LR
I1["Frame I1"] --> P1["Frame P1"]
I1 --> B1["Frame B1"]
P1 --> B1
P1 --> I2["Frame I2 (new GOP)"]

The Frequency Domain: The Discrete Cosine Transform

After prediction, a residual remains: the part of the image that couldn’t be guessed. That residual still has redundancy, but of a different kind, hidden in how brightness varies within a block of pixels. To expose it, H.264 splits the frame into blocks (from 4×4 to 16×16 pixels) and applies a discrete cosine transform (DCT) to each one, similar to the one JPEG uses.

The DCT doesn’t change the information, it reorganizes it. Instead of describing the block pixel by pixel, it describes it as a combination of frequency patterns: how much slow variation there is (flat areas of color) and how much fast variation there is (edges and fine texture). The human eye is much more sensitive to slow variation, and that’s the key to the next step.

Quantization: Deciding Which Detail Gets Discarded

Once the block is expressed in frequencies, quantization rounds the high-frequency values toward zero, and many end up being exactly zero. These are things like the mesh grille of a speaker or the fine noise of a texture, details almost nobody notices if they disappear but that, in an uncompressed image, take up as much space as the rest of the block.

💡 Tip: When encoding with FFmpeg, the CRF parameter controls exactly this quantization aggressiveness: low values (18 to 20) preserve almost all the detail, high values (28 or more) sacrifice more frequency in exchange for a lighter file.

The more aggressive the quantization, the more zeros appear and the lighter the file gets, but the more visible the loss becomes too, with blurry blocks, haloed edges, and texture that melts away. Finding the right balance between file size and quality is, in practice, the central job of anyone configuring a video pipeline.

Entropy Coding: CABAC and CAVLC

The last thing the codec does is take that sequence of numbers (mostly zeros, some small values) and compress it without losing anything more, using entropy coding. The standard defines two methods: CAVLC, simpler and faster, and CABAC (context-adaptive binary arithmetic coding), which compresses better but requires more computation to decode.

CABAC is the engine that really explains a good part of the final gain: instead of assigning each symbol a fixed number of bits, it adjusts each symbol’s probability based on the context of the previous symbols. It’s the same idea behind describing ten coin flips as “10 flips, all heads” instead of writing down each result one by one.

Practical Examples and How to Get Started

Everything above can be seen in action with a single tool: FFmpeg, the command-line video converter used by everyone from YouTube to personal projects. Before running any commands, install it for your system. On Debian or Ubuntu:

sudo apt update && sudo apt install -y ffmpeg

On macOS with Homebrew it’s brew install ffmpeg, and on Windows with winget it’s winget install ffmpeg. With that installed, try the simplest case first: converting an existing video to H.264 while controlling quantization with CRF.

ffmpeg -i input.mov -c:v libx264 -crf 23 -preset medium output.mp4

This command takes input.mov (any video you have on hand) and re-encodes it with libx264, the software implementation of H.264 that ships with FFmpeg. -crf 23 is the default value, a reasonable balance between size and quality, and -preset medium controls how much computation time to spend looking for better predictions. When it finishes, FFmpeg prints a summary like this:

frame=  300 fps= 45 q=-1.0 Lsize=     612kB time=00:00:05.00 bitrate= 1003.0kbits/s
video:598kB audio:10kB subtitle:0kB muxing overhead: 0.588603%

The number that matters is Lsize, the final size of the encoded file. To see which frame type (I, P, or B) each one got, FFmpeg includes ffprobe:

ffprobe -v error -select_streams v:0 -show_entries frame=pict_type -of csv output.mp4 | head

The output is a list of letters, one per frame, in the same order they appear in the file:

I
P
P
B
P
P
B
P

That pattern, an I followed by a mix of P and B, is direct proof of everything explained above: the first frame is encoded in full, and the rest are described as differences relative to their neighbors.

Real-World Use Cases for H.264

On-demand video streaming is the most visible case, but far from the only one. Blu-ray discs standardized H.264 (alongside VC-1) as a primary codec from launch. Video calls also depend on low-latency variants of the same standard, with short GOPs to minimize the delay between something happening in front of the camera and showing up on the screen on the other end.

Security cameras are another huge and rarely discussed use case: they record 24 hours a day, with mostly static scenes (ideal for inter-frame prediction), and need to store weeks of footage on drives that aren’t infinite. Drones face the opposite problem, streaming live video over a radio link with limited bandwidth, and use profiles optimized for low latency rather than maximum quality.

Common Mistakes and Best Practices with H.264

The most common mistake is picking a CRF blindly. A CRF that’s too low (below 18) is rarely noticeable to the naked eye but needlessly doubles or triples the file size; one that’s too high (above 30) introduces visible blocking in scenes with motion. The sensible approach is to test with your own content: 18 to 23 for a reference file, 23 to 28 for general distribution.

Another frequent mistake is ignoring the interval between I frames (keyframe interval). A GOP that’s too long saves space but makes seeking to a specific point in the video slower, because the player has to reconstruct the whole path from the last I frame. For video that will be played with seek controls, a GOP of 2 to 4 seconds is usually the right spot.

The third mistake is not paying attention to the profile. H.264 defines Baseline, Main, and High profiles, with different support for advanced features like B-frames and CABAC. A video encoded with the High profile doesn’t always play on old or budget hardware; certain embedded browsers or low-cost decoders only support Baseline. If the target device is unknown, explicitly declaring the profile avoids surprises.

⚠️ Heads up: B-frames improve compression but add latency, because the encoder needs to see future frames before finishing a current one. For video calls or low-latency live streaming, many encoders disable B-frames entirely.

Comparison: H.264 vs Other Video Codecs

The reason the H.264 codec still dominates isn’t that it’s the most efficient, it’s that it’s already everywhere: in every browser, every smart TV, and every camera chip with a hardware decoder for more than a decade now. HEVC and AV1 compress better, but they’re still filling that same ground device by device.

Going Deeper: Macroblocks, Profiles, and the Deblocking Filter

The basic unit of work in H.264 is the macroblock, a 16×16 pixel square. The standard doesn’t force it to be treated as a single block: each macroblock can be split into 16×8, 8×16, or 8×8 partitions, and each 8×8 partition can in turn be split into sub-partitions as small as 4×4. The smaller the partition, the more precise the motion it can describe, at the cost of more bits to describe the partition itself.

flowchart TD
A["16x16 macroblock"] --> B["16x8 or 8x16 partition"]
A --> C["8x8 partition"]
C --> D["8x4, 4x8 or 4x4 sub-partition"]

That granularity lets the encoder treat a flat wall differently (a single 16×16 block is enough) from the edge of a moving person, where it pays to drop down to 4×4 to avoid losing definition. Deciding which partition to use for each macroblock is, in practice, one of the highest computational costs in the encoding process.

After decoding a frame, H.264 still applies a deblocking filter to smooth out the edges between macroblocks, which would otherwise be visible as a grid. It’s a mandatory step in the standard, and it’s part of why compressed video looks smoother than the raw block-by-block math would suggest.

The profiles (Baseline, Main, High, and their variants) determine which combination of these tools is available. Baseline, designed for video calls and limited hardware, doesn’t include B-frames or CABAC. High, the most used in streaming and Blu-ray, adds both and is the one that truly approaches the compression limit the standard allows.

Your next step: take any video you have on your phone, run ffmpeg -i yourvideo.mp4 -c:v libx264 -crf 28 output.mp4, and compare the final file’s size against the original.

Frequently Asked Questions

What Does AVC Mean in H.264?

AVC stands for Advanced Video Coding, the technical name under which ITU-T and MPEG standardized H.264 in 2003 as MPEG-4 Part 10. Both names, H.264 and AVC, refer to exactly the same standard.

Does H.264 Have a Licensing Cost?

Yes, for certain commercial uses. The patent pool managed by MPEG LA charges royalties to hardware manufacturers and to some high-volume content distributors, although most developers who simply encode and play video with libraries like FFmpeg pay nothing directly.

What’s the Difference Between H.264 and H.265/HEVC?

HEVC is the direct successor to H.264, published in 2013, and it uses larger blocks and more prediction modes to compress better at the same quality level. In exchange, decoding it requires more computation, and its licensing scheme is more fragmented across several different patent pools.

Why Does YouTube Use VP9 or AV1 Instead of H.264?

Google pushed VP9 and later AV1 partly to avoid paying HEVC royalties at scale. Both are royalty-free and compete in efficiency with the licensed standards, though they coexist with H.264 rather than replacing it entirely.

What Is a Baseline Profile in H.264?

It’s the simplest subset of the standard’s features: it doesn’t include B-frames or CABAC, which makes it easier to decode on limited hardware, at the cost of worse compression than the Main or High profiles.

References

• sidbala.com: the original technical article that inspired this breakdown of H.264.
• ITU-T: official page for the H.264/AVC recommendation.
• Wikipedia: history and technical details of Advanced Video Coding.
• FFmpeg: official documentation for the tool used in this article’s examples.
• FFmpeg Wiki: official guide to H.264 encoding with libx264.

📱 Follow @programacion on Telegram for daily tech content in Spanish: quick summaries, fresh content every day.

Featured image: Foto de Egor Komarov en Unsplash