When RAM rises without a crash
The service does not crash, but every new million cached entries increases its use of RAM, the server’s limited working memory. Eventually, the team needs a larger server or a more expensive cloud plan even though the stored answers themselves appear small.
Memory profiling shows which data and supporting structures consume a program’s RAM. Backend developers use it to find avoidable costs in caches and long-running services, then measure whether a proposed data-structure change actually helps.
On August 27, 2026, Cloudflare described five changes to its DNS cache. According to the company’s measurements, they reduced memory per entry by 56% and freed roughly 100 TB of RAM. This is an example of a method, not a forecast: another service will have different data, structures, and workloads.
What one cache entry contains
A cache is temporary storage for prepared answers. A DNS cache, for example, stores an answer about a domain and the period for which that answer remains valid. While the entry is valid, the service does not need to repeat the same lookup.
The useful answer is only part of the entry. Data-structure layout describes how its fields and related regions are arranged in memory. A growable collection may reserve empty space. A pointer stores the address of another region. Obtaining such a separate region is a memory allocation, and the component that distributes memory is the allocator. Gaps may also appear between fields because processors require certain values to be aligned.
Consider an educational entry:
- the answer uses 120 bytes and expiration metadata uses 8;
- spare capacity uses 64 bytes;
- pointers use 16 bytes;
- alignment padding uses 8 bytes.
The useful data and required metadata occupy 128 bytes, but the complete entry already uses 216 before possible additional allocator overhead. One million entries require 216 MB rather than 128 MB when a megabyte is counted as one million bytes. The remaining 88 MB is overhead from capacity, pointers, and alignment. This synthetic example does not estimate a particular language or library.
What to look for in your own structure
Cloudflare stores more than 250 billion cache entries. At that scale, one unnecessary byte per entry requires more than 250 GB of memory across the fleet. For a smaller service, the useful lesson is not the scale but the sequence of questions:
- Is spare capacity needed after creation? If the data will no longer grow, an immutable collection may be more compact.
- Can the number of allocations be reduced? Fewer separate regions often mean fewer pointers and less allocator bookkeeping.
- Can several lists be combined? One list with validated offsets or indexes may save space, but it also makes the code more complex.
- Does each field width match its valid range? Narrow fields and bit flags are safe only when they represent every permitted value and do not obscure the code’s meaning.
In its full-cache-path benchmark, Cloudflare also measured a 43% increase in insertion throughput and a 19% reduction in lookup latency. Those results belong to specific changes under a specific workload. They do not prove that making any structure smaller will also make it faster.
How to prove that a change helped
A controlled benchmark is the same repeatable test run before and after one change. Cloudflare filled a test cache with randomly generated entries that only approximated the distribution of its production traffic. A custom allocator counted the number and size of allocations. This test helps explain the mechanism, but it does not cover all memory used by the process.
The resident set size, or RSS, reports the amount of a process’s memory pages currently present in physical RAM. It may include code, shared pages, buffers, and memory retained by the allocator for reuse. RSS is closer to what a service operator sees, but it does not isolate one cache entry. Cloudflare therefore also observed resident memory on production instances during the rollout.
The two measurements work best together: the benchmark tests the hypothesis, while RSS shows whether the effect is visible in the complete process.
A safe AI-assisted cycle
AI can help at every stage, but a person must first understand the metrics and the experiment’s limits.
-
Understand. Give the assistant only the definitions and synthetic example above. Permit explanations and arithmetic checks; prohibit assumptions about your stack or promises of savings. Stop when a term or unit is undefined. The artifact is a useful-data-plus-overhead diagram; verify it by adding and multiplying the bytes yourself.
-
Inspect. Provide an anonymized profiling report, allocation summary, or test benchmark. Permit only read-only grouping of costs; prohibit production dumps, secrets, real DNS queries, command execution, and code changes. Stop if the input is sensitive, uses unknown units, or lacks complete starting data. The artifact is a note containing baseline figures and hypotheses; verify it against the original tool and repeat the run.
-
Plan. Provide a verified baseline, workload description, correctness test, and latency limit. Permit a one-change experiment with metrics and rollback; prohibit combining optimizations or planning a direct production deployment. Stop if the baseline is not reproducible. The artifact is an experiment card; a person must review the hypothesis, commands, and thresholds.
-
Act. Use synthetic data, a disposable environment, or an isolated test branch. Permit edits only to that branch and pre-approved tests; prohibit production access, secrets, live traffic, and destructive commands. Stop if a test fails, RAM rises, or latency crosses the agreed limit. The artifact is a small diff and run log; verify it by repeating the work in a clean environment.
-
Verify. Provide paired results produced under identical conditions. Permit calculations of differences, variance, and regressions; prohibit selecting only favorable runs or treating an AI response as proof. Stop if the data or environment differs or there are too few repetitions. The artifact is a table of RAM, allocations, latency, throughput, and correctness; verify it with a new run without the assistant and the complete test suite.
-
Explain. Provide verified tables, the diff, and the source. Permit a short decision record; prohibit applying Cloudflare’s 56% or 100 TB figures to your service. Stop when a claim lacks a measurement or source. The artifact is a keep, reject, or continue decision with limits and rollback; the service owner must compare its figures with the logs.
Checklist
- The baseline is reproducible under the same conditions.
- The experiment contains one change, stop conditions, and a rollback plan.
- RAM or allocation counts improved without an unacceptable loss of correctness, latency, or throughput.
- A person reviewed the changes and repeated the test independently.
A compact structure is only a hypothesis. Keep the optimization when equivalent repeated measurements confirm its value for this service.