⏱️ Lectura: 12 min
Cloudflare just freed about 100 terabytes of memory across its global fleet by rewriting, field by field, the data structures behind the DNS cache for 1.1.1.1. The team behind Big Pineapple, the internal platform that also runs Gateway DNS, DNS Firewall, and AS112, applied five successive changes in Rust that reduced the weight of each cache entry by more than 50%.
📑 En este artículo
The result wasn’t just freed space. Insert throughput rose 43% and read latency dropped 19%, according to the technical post published by Cloudflare. Shrinking the structs, in this case, also made them faster.
TL;DR
- Cloudflare freed about 100 terabytes of memory across its fleet by optimizing the DNS cache of Big Pineapple, the platform behind 1.1.1.1.
- Big Pineapple simultaneously holds more than 250 billion DNS cache entries across Cloudflare’s entire infrastructure.
- Five successive changes to the Rust data structures reduced the weight per cache entry by more than 50%.
- Switching Vec and String to Box<[T]> and Box<str> eliminated the leftover capacity field, saving more than 15 terabytes across the fleet.
- Merging three record lists into one using 2-byte offsets instead of 8-byte pointers saved 28 bytes per entry.
- Cache insert throughput rose 43% and lookup latency dropped 19% after the five changes.
- The total savings equal the RAM of 130 of Cloudflare’s Gen 13 servers.
- Data centers with EDNS Client Subnet (ECS) enabled benefit the most, since they cache multiple versions of the same query.
Introduction
1.1.1.1 is one of the most widely used public DNS resolvers in the world, and behind every query there’s a cache layer that decides whether Cloudflare answers in microseconds or has to ask an authoritative server. That layer is called Big Pineapple and, according to Cloudflare, it constantly holds more than 250 billion DNS cache entries spread across its network. At that scale, wasting a single byte per entry costs more than 250 gigabytes of memory across the fleet.
That was the engineering team’s starting point: audit byte by byte the structures that represent each DNS cache entry and eliminate everything that added no value once the data was already stored.
What happened to the 1.1.1.1 DNS cache
Each cache entry is a key-value pair. The key identifies what was queried: the domain name, the record type, whether the response is authenticated with DNSSEC, and an optional tag for EDNS Client Subnet (ECS) variants. The value holds the full response: the answer, authority, and additional sections, plus metadata like the timestamp, a hit counter, and the TTL.
The team found that several of those fields used types meant for structures that grow and change, like Vec<T> and String, when in reality a DNS cache entry, once inserted, is never modified again. That mismatch between the chosen type and actual usage is exactly the kind of waste that can be measured and fixed.
Background
To measure the impact of each change without depending on real traffic, Cloudflare built a benchmark that fills the cache with randomly generated entries in a ratio similar to production: 56% A records, 25% AAAA, and 19% TXT, with between one and four records per entry. The TXT records stand in for every record type other than A/AAAA, with a random size between 64 and 224 bytes.
To count real bytes, not estimates, they used a custom allocator that wraps Rust’s System allocator and logs the count and size of every allocation per cache entry. That metric was combined with resident memory measurements on production instances during the rollout, since real traffic also depends on the query mix, cache occupancy, and allocator state, variables that a synthetic benchmark only approximates.
One factor that makes the problem worse is EDNS Client Subnet (ECS): when it’s enabled, authoritative servers return different responses depending on the client’s subnet, so Big Pineapple ends up caching multiple versions of the same query. That multiplies both the number of entries and the memory per entry, which makes these optimizations matter even more at locations with heavy ECS traffic.
Technical Details and Performance
The first change, and the one that freed the most memory, was replacing Vec<T> and String with Box<[T]> and Box<str>. A Vec<T> holds three fields: a pointer to the heap, the current length, and the reserved capacity. That capacity only exists so the vector can grow without reallocating memory, something an immutable DNS cache entry never needs. Box<[T]> can’t grow after it’s created, so it needs no capacity and reserves no extra space: it’s a fat pointer with just a pointer and a length.
use std::mem::size_of;
struct EntradaConVec {
respuestas: Vec<u8>,
ttl: u32,
}
struct EntradaConBox {
respuestas: Box<[u8]>,
ttl: u32,
}
fn main() {
println!("Vec<u8>: {} bytes", size_of::<EntradaConVec>());
println!("Box<[u8]>: {} bytes", size_of::<EntradaConBox>());
}
Running that example with cargo run shows the difference: the Vec<u8> field takes up 24 bytes on a 64-bit system (pointer, length, and capacity, 8 bytes each), while Box<[u8]> takes up 16 (pointer and length). Each Big Pineapple struct had eight fields of type Vec or String, so the change saved 64 bytes per cache entry, plus the heap space a Vec over-reserves for growth. Multiplied by 250 billion entries, the total exceeds 15 terabytes.
The second change targeted the three record lists (answer, authority, additional). Instead of storing three independent Box<[T]> values, each with its own 8-byte pointer and 8-byte length, Cloudflare switched to storing a single list with 2-byte offsets (u16) that mark where each section starts. Since the number of records per section always fits in a u16, two 2-byte offsets replace two full 16-byte lists, saving 28 bytes per entry.
The third change was packing several loose boolean fields into a single bitflag. In Rust, the compiler adds padding to respect each field’s alignment and rounds the struct’s total size up to a multiple of that alignment. Removing a small boolean field can eliminate extra padding, so the struct shrinks by more than the combined size of the individual booleans.
| Change | Before | After | Estimated Savings |
|---|---|---|---|
| Vec/String to Box | pointer + length + capacity (24 bytes) | pointer + length (16 bytes) | 64 bytes per entry, more than 15 TB across the fleet |
| 3 record lists to 1 list with offsets | 3 x (pointer 8B + length 8B) = 48 bytes | 1 pointer + length + 2 u16 offsets = 20 bytes | 28 bytes per entry |
| Loose booleans to bitflag | several 1-byte fields plus padding | 1 single byte | variable, reduces struct padding |
| Duplicated owner to shared owner | the domain name repeats in every record | shared reference to the queried name | variable, depending on the number of records per entry |
| Total of the five changes | original size per entry | more than 50% smaller | ~100 TB freed across the fleet |
A fifth change, complementary to the previous ones, took advantage of the fact that many DNS records share an owner (the domain that owns the record) with the original query. A query for example.com A returns records whose owner is also example.com, so instead of storing that name once per record, Big Pineapple references it a single time and shares it across every record in the entry.
Together, the five changes shrank each DNS cache entry by more than 50%. Across a fleet that holds 250 billion entries, that translates to about 100 terabytes freed, the equivalent of the RAM in 130 of Cloudflare’s Gen 13 servers. And since there are fewer allocations per insert and better memory locality, insert throughput rose 43% and lookup latency dropped 19%: the memory optimization didn’t cost any speed.
How to Start Applying This
You don’t need to run a DNS resolver at Cloudflare’s scale to apply the same pattern. If you maintain a cache in Rust (or any language with resizable collection types), the first step is to ask whether that collection gets modified again after insertion. If the answer is no, an immutable type like Box<[T]> almost always wins.
A realistic case, with the bitflag and the Box together:
use bitflags::bitflags;
bitflags! {
struct EntradaFlags: u8 {
const AUTENTICADO = 0b0000_0001;
const TIENE_ERRORES = 0b0000_0010;
const ES_NEGATIVA = 0b0000_0100;
}
}
struct CacheEntryOptimizada {
ttl: u32,
hits: u32,
flags: EntradaFlags,
respuestas: Box<[u8]>,
}
To use bitflags, add the dependency to your Cargo.toml:
[dependencies]
bitflags = "2"
To verify the savings in your own project, you don’t have to guess: std::mem::size_of::<T>() tells you the exact struct size before and after the change, and a heap profiler like valgrind --tool=massif or the dhat-rs crate shows you how many allocations and how many real bytes your cache consumes under load. Run it before and after each change, not just at the end: that way you can identify which optimization contributed what percentage, just like Cloudflare did with its five successive changes.
💡 Tip: Measure with std::mem::size_of::<T>() before touching anything. It’s free, requires no benchmark, and tells you upfront whether there are wasted bytes in the struct’s layout.
If you want to observe the ECS behavior Cloudflare mentions, you can simulate a query with an explicit subnet against 1.1.1.1:
# Linux / macOS / WSL
dig @1.1.1.1 example.com A +subnet=203.0.113.0/24
# Windows (PowerShell, with dig installed via winget or BIND tools)
dig @1.1.1.1 example.com A +subnet=203.0.113.0/24
Each different subnet you use can generate an additional cache entry on the resolver side, which is exactly the scenario that makes data centers with heavy ECS traffic heavier.
Impact and Analysis
The Big Pineapple case is a reminder that optimizing memory and optimizing speed aren’t competing goals when the problem is an inefficient data layout. Fewer bytes per entry means fewer allocations, better CPU cache locality, and, in this case, a DNS cache that responds faster with less hardware. For any team running a high-cardinality cache (a Redis proxy, a CDN edge cache, an API gateway) the pattern is transferable: audit immutable structs, strip out leftover capacity, merge parallel lists, and pack booleans.
⚠️ Watch out: Box<[T]> can’t grow after it’s created. If your struct needs a mutable field, like the hits counter that does change on every cache hit, that field has to stay outside the Box or use a type with interior mutability (Cell or AtomicU32); you can’t just wrap the entire struct.
The following diagram summarizes where the optimized entry fits into the flow of a DNS query against 1.1.1.1:
sequenceDiagram
participant C as Client
participant BP as Big Pineapple
participant A as Authoritative server
C->>BP: DNS query (example.com A)
BP->>BP: looks up CacheKey in the DNS cache
alt cache hit
BP-->>C: responds from optimized CacheEntry
else cache miss
BP->>A: forwards the query
A-->>BP: response with TTL
BP->>BP: inserts optimized CacheEntry
BP-->>C: responds to client
endThe honest limitation of this approach is that not every field in a cache lends itself to immutability. The hits counter, for example, changes on every hit, so it can’t live inside a frozen Box<[T]>: it has to stay in a separate, mutable field. Aggressive memory optimization only works if you first clearly separate which data is read-only once inserted and which keeps changing.
What’s Next
Cloudflare notes that the benchmark figures approximate, but don’t exactly reproduce, production behavior: process memory also depends on the traffic mix, cache occupancy, and allocator state. That’s why they measured resident memory on real instances during the rollout, in addition to the synthetic benchmark. It’s reasonable to expect this kind of data layout audit to repeat across other Cloudflare platforms that handle similar volumes of in-memory entries, not just the DNS cache.
📖 Summary on Telegram: View summary
Try it yourself: run cargo run --release with the two example structs from this article and compare the size_of result before deciding whether your own cache has bytes to reclaim.
Frequently Asked Questions
What is Big Pineapple?
It’s Cloudflare’s internal platform that powers 1.1.1.1, Gateway DNS, DNS Firewall, AS112, and other DNS services at the company. It maintains the DNS cache that answers most queries without having to forward them to an authoritative server.
Why does Vec<T> use more memory than Box<[T]>?
Because Vec<T> reserves an extra capacity field so it can grow without reallocating memory, and it also tends to reserve extra heap space for future insertions. Box<[T]> can’t grow after it’s created, so it needs neither.
Did these changes affect speed, or just memory?
They affected both. According to Cloudflare, cache insert throughput rose 43% and lookup latency dropped 19%, because there are fewer allocations per entry and better memory locality.
What is EDNS Client Subnet and why does it matter here?
It’s a DNS extension that lets an authoritative server return different responses depending on the client’s subnet. When it’s enabled, Big Pineapple caches multiple versions of the same query, which increases both the number of entries and the memory per entry.
Can I apply this same optimization in my own Rust cache?
Yes. The pattern (replacing Vec/String with Box when the data no longer changes, merging parallel lists with offsets, and packing booleans into a bitflag) applies to any high-cardinality immutable struct, not just a DNS cache.
Where is Cloudflare’s original technical post?
On Cloudflare’s official blog, with full detail on the five changes and the benchmark methodology used to measure each one.
References
- Cloudflare Blog: original technical post on the 1.1.1.1 DNS cache optimization.
- Rust std docs, Box: official documentation for the Box type and its layout guarantees.
- Rust std docs, Vec: official Vec documentation, including the capacity field.
- RFC 7871: specification for the EDNS Client Subnet (ECS) mechanism.
- docs.rs, bitflags: documentation for the crate used to pack boolean fields.
📱 Enjoying this content? Follow @programacion on Telegram for daily tech content in Spanish: quick summaries, fresh content every day.
0 Comments