Anycast origins and CDN: why a cache miss can take the long way around

One slow cache miss that reveals the whole topology

Imagine a platform team starting the morning with a strange production symptom. A user in Europe opens a page, the nearby edge cache answers quickly for most assets, but one cache miss takes much longer than expected. On a route map, the path looks like an unnecessary loop: the request starts close to the user, but the data appears to be pulled through a route that resembles a detour across another continent.

At first, this feels illogical. The CDN should be close to the user. The origin runs in a public cloud. Most responses are cached. So it is tempting to blame temporary load or random network delay. But this is where a common operational mistake hides: the team assumes that a nearby IP address means the real backend is nearby too.

The basic path: edge cache → upper tier → origin

For static assets and many API responses, a CDN behaves like several storage layers. The edge cache sees the request first. If the needed response is already there, the user gets it quickly. If there is a cache miss, the CDN may not go straight to the origin. It may first ask an upper tier.

The upper tier is useful because it collects cache misses from many edge locations. Instead of dozens of edge caches hitting the origin for the same file at the same time, they can ask one intermediate layer. This can improve cache hit ratio at the higher layer and reduce the number of origin requests.

But the upper tier must be chosen well. If the origin physically lives in one cloud region, the intermediate layer should be network-close to that region, not merely close to an address that appears nearby from a routing probe.

Why a nearby IP does not mean a nearby origin

Public clouds often hide complex topology behind simple addresses. A service can have anycast or a regional entry point while the real backend sits somewhere else. From the CDN point of view, latency checks to an IP can show several good candidates at once. The address may respond quickly from different data centers, but that does not guarantee the request is already near the correct origin.

A recent Cloudflare engineering post describes this problem for choosing an upper tier for origins in public clouds. When a cloud origin uses anycast or a regional front end, the CDN may not have a clear signal for which upper tier is truly closest to the origin region. In the bad case, a cache miss goes through an entry point that looks fine by latency, but the traffic then performs hairpin routing.

This is not a failure of the CDN idea. It is a reminder that a modern request path has several layers: DNS, CDN, the CDN provider’s internal network, the cloud front end, a load balancer, and the backend. If one layer does not know the real locality of another layer, automatic route choice can be ambiguous.

Runbook: how to check for a long route

Start with a map of facts, not with configuration changes. Write down where the origin really lives: for example, europe-west1, eu-central-1, or westeurope. Do not stop at the DNS name or public IP address. If the origin runs behind a global load balancer, record where that load balancer sends traffic during normal operation.

Next, open CDN logs or analytics. Filter cache misses by path, user country, response status, and time window. Compare fast and slow requests. Useful fields include edge region, cache status, time to origin, upstream status, response size, and retry count.

Then inspect origin pull latency. If users in Europe consistently have high time to origin while the origin is also in Europe, check the upper tier selection. If high latency appears only for cache misses while cache hits are fine, the problem is not the browser and not the general speed of the site.

Also check cache hit ratio and origin requests. A drop in cache hit ratio together with a rise in origin requests means the CDN is going to the origin more often. If origin pull latency rises at the same time, you may have two problems at once: more requests and a longer route for each miss.

Anti-patterns that slow down the investigation

The first anti-pattern is looking only at average latency. Averages hide regional problems. It is better to compare percentiles for cache miss separately from cache hit.

The second is treating anycast as magic that always leads to the best backend. Anycast helps traffic reach a nearby entry point, but it does not always describe the location of the server that will produce the response.

The third is immediately changing TTL values or disabling tiered cache entirely. That may reduce the symptom on one route, but it can increase origin requests and make the system more expensive and less stable.

A better action is to look for a way to give the CDN knowledge about the origin region. Different providers may use different names: cloud region hint, origin region, topology-aware routing, or upper tier configuration. The label matters less than the idea: the CDN needs to understand where it should really pull a cache miss from.

What to include in production design

If your site or API runs behind a CDN, document not only the origin domain but also the real cloud region, load balancer type, expected cache miss path, and metrics for verification. For a multi-region architecture, separately describe when traffic is supposed to go to another region and when that becomes a sign of hairpin routing.

A CDN reduces complexity for the user, but it does not remove topology. A slow cache miss is often the first signal that automatic route choice does not have enough information. The team’s job is not to guess. It is to collect facts: origin region, CDN logs, origin pull latency, cache hit ratio, and number of origin requests.

Sources

• Cloudflare Blog: Smart Tiered Cache for Public Cloud Regions — https://blog.cloudflare.com/smart-tiered-cache-for-public-clouds/