Operating DNS at the edge

DNS is often described as a naming system, but operating it at the edge is an exercise in controlling uncertainty. A query can arrive over IPv4 or IPv6, by UDP or TCP, at a server selected by routing policy rather than by a map. The answer may be authoritative data, cached data, or an error produced by a health policy. A node can be reachable while its zone is stale, or healthy in a local probe while a large set of networks cannot reach it.

Anycast can make a distributed DNS service look like one address to clients. It does not make the Internet choose the physically nearest machine, does not remove the need to publish consistent zone data, and does not turn a record TTL into a route-failover timer. The reliable way to operate at the edge is to treat routing, DNS software, data distribution, and measurement as one system with explicit boundaries.

This article explains the engineering practice without assuming a particular provider, footprint, or availability guarantee. The examples are patterns to evaluate and test, not claims about how XQO operates a network.

Open the full-size diagram

Start with the DNS service you are actually building

Before choosing a routing model, identify whether the service is authoritative, recursive, or both. An authoritative server answers from zone data for which it is an authority. A recursive resolver accepts a client's request and follows referrals, using a cache to assemble an answer. The distinction is foundational: an authoritative anycast node should be able to answer its zones locally, while a recursive node has cache state, upstream reachability, validation state, and policy decisions to keep healthy.

RFC 1034 describes non-recursive service as using local information to return an answer, an error, or a referral. Recursive service is optional and is requested with the RD bit; a server indicates that it offers recursion with RA. RFC 1035 defines the AA, RD, RA, TC, and RCODE fields that make those behaviors visible in DNS messages. In operational terms, do not use a recursive success check to prove that an authoritative zone is ready, and do not use an authoritative answer to prove that a recursive resolver can reach its dependencies.

The service boundary also determines shared state. An authoritative fleet needs matching zone contents, serials, delegation data, and response policy. A recursive fleet may have different caches, but still needs consistent software policy, trust anchors, access controls, forwarding rules, and telemetry. Anycast is most natural for short, independent transactions; RFC 4786 cautions against long-running flows whose packets must stay with one node.

A useful design document starts with a query matrix: authoritative versus recursive answers, allowed recursion, UDP and TCP, IPv4 and IPv6, DNSSEC, large responses, cache misses, upstream timeouts, and policy errors. Judge readiness against these cases rather than against a single ping.

Anycast is a routing arrangement, not a location service

In an anycast design, multiple nodes advertise reachability for the same service address or prefix. The routing system delivers a packet to one of those nodes. A resolver does not need a list of node addresses, and the operator can withdraw a node's advertisement when that node should no longer receive new traffic. Cloudflare's educational overview is a useful explanation of the shared-address model, but its descriptions of its own network and DDoS posture are provider-specific and should not be treated as general properties of anycast.

The critical correction is to replace “closest” with “selected by routing.” BGP path selection is shaped by local policy, attributes, topology, business relationships, and implementation choices. RFC 4786 notes that the global routing system is generally controlled by other parties. A route that wins at one vantage point may lose at another. A node that is geographically nearby may be reached through a longer or less preferred path, while a farther node may be the preferred exit. Physical distance and round-trip latency are useful measurements, but they are not the selection rule.

The route decision is not a promise that every packet will use one node forever. Convergence, maintenance, equal-cost behavior, and failures can change the selection. RFC 4786 explains why per-packet paths that split a multi-packet exchange are risky. UDP usually carries one DNS request and response, while truncation and TCP fallback make node consistency more important.

Authoritative and recursive edge behavior

Authoritative anycast is often easier to reason about because the answer should come from local zone data. That simplicity is conditional. Every node must have the intended version of each served zone, answer with the right SOA serial, load the same DNSSEC material, and apply the same response policy. If one node has an old record or an invalid signature, routing may expose only a subset of clients to the problem, making the incident appear intermittent.

Recursive anycast has a different failure surface. A resolver can accept a query but fail while contacting an authoritative server, validating a DNSSEC chain, or serving a policy response. A node can also have a warm cache while its network path to an upstream is broken. Health checks must test the behavior that matters to clients. If the product promise is recursive resolution, a check of the local listening socket is necessary but not sufficient; controlled names should exercise cache-hit and cache-miss paths without creating an uncontrolled dependency on a third party.

For either model, avoid accidental recursion on authoritative service addresses. RFC 1034 notes that a name server should not perform recursion unless it is requested and available under its policy. In modern deployments, that becomes an access-control requirement: restrict recursion to intended clients, and make the authoritative and recursive roles visibly distinct in configuration, metrics, and incident procedures.

DNS transport matters at the edge. UDP is efficient for ordinary queries, but response size, EDNS behavior, fragmentation, and middlebox handling can affect delivery. A truncated response can require TCP. Test both protocols and both IP versions, and test the actual answer classes that matter. A node that passes a small UDP A query but drops TCP/53 or produces malformed DNSSEC responses is not ready for production.

Route choice is policy-shaped

A route advertisement says that a node is willing to receive traffic; it does not say that the node is the best destination for every client. BGP selection happens independently in many autonomous systems. Operators can influence their own announcements and upstream relationships, but they cannot dictate every downstream decision. That is why a city-by-city map of nodes is not a latency model.

The practical question is not “Which node is closest?” It is “Which node does each important network select, and does that selection meet the service objective?” Build a view by resolver network, autonomous system, address family, and time. Record the answering node when possible, then correlate it with response time, timeout, and error data. The same node may look excellent from one AS and poor from another because the path into the node is different.

A route can also be accepted before the service is actually ready. RFC 4786 recommends coupling service availability to route advertisement and withdrawal: a node should advertise only when it can accept the service traffic associated with that route. The coupling may be implemented by a routing process, a local health agent, or a controlled orchestration system. The mechanism is less important than the invariant: “announced” must mean “safe to serve,” not merely “the host is powered on.”

Placement is a coverage and failure-domain problem

The purpose of adding a node is not to fill a world map. It is to improve measured paths or remove a shared failure domain. Start with important resolver networks, group observations by AS, region, address family, and peering path, and look for persistent latency, loss, timeouts, or route problems. Geographic proximity alone is not evidence that a new node will change those outcomes.

RFC 2182 gives closely related guidance for authoritative secondaries: geographic and topological dispersion reduces the chance that one site, link, or provider failure makes every server unreachable. It also warns that adding more servers has costs, including more configuration to get wrong. Anycast does not relax this principle. A dozen nodes in one failure domain can be less resilient than fewer nodes with independent power, transit, management, and data paths.

An AFRINIC deployment guide is a concrete example of scoping these concerns. It calls for separate management and peering networks and assigns distinct responsibilities for host organizations and the service operator. That is an example of one program's deployment contract, not a universal requirement for every anycast service. The general lesson is to separate operational access from the traffic and routing plane where feasible, document ownership, and test the boundaries instead of assuming that “distributed” means independent.

Capacity planning must follow observed route distribution, not an average across nodes. A route change can move a large resolver population to one site. Size for normal traffic plus a defensible failure scenario, including DNSSEC work, cache memory, packet rate, and TCP handling. Define the result of a node withdrawal or upstream isolation; otherwise route redundancy can hide a capacity incident until the next failure.

Capacity and DDoS planning are related but not synonymous. Anycast may spread a flood when routing paths distribute it, but policy can also concentrate traffic on one node or link, and a distributed address does not prevent application-layer abuse. RFC 4786 treats denial-of-service mitigation as an operational concern. Plan packet-rate limits, upstream filtering, spare capacity, and a tested route response without assuming that withdrawing one node makes the attack disappear.

Readiness, withdrawals, damping, and flapping

A production readiness check should be layered. At the process layer, verify that the DNS daemon is running and listening on the intended addresses. At the protocol layer, query representative zones over UDP and TCP, through IPv4 and IPv6, with and without EDNS as appropriate. At the data layer, compare SOA serials, DNSSEC signatures, DNSKEY and DS relationships, and policy outputs with the intended source. At the network layer, confirm that the node can reach required upstreams or transfer sources. At the routing layer, confirm that the advertisement is present only after the earlier checks pass.

The route controller should have hysteresis. If a check fails for one packet, immediately withdrawing and then re-advertising can make the network oscillate. RFC 4786 warns that rapid advertisement and withdrawal can cause operational problems and recommends avoiding rapid oscillations, such as by imposing a delay before re-advertising after withdrawal. The delay should be chosen from observed recovery behavior and the service's risk tolerance, then tested under load.

BGP flap dampening is an additional network behavior, not a replacement for local hysteresis. Frequent announcements and withdrawals can consume router resources and lead remote routers to suppress a path. A node that repeatedly flaps may therefore remain absent after its local check turns green. Alert on advertisement state, withdrawal count, time since last announcement, and the route as seen from multiple external collectors. Record whether the problem is service health, the local routing session, or propagation.

Be explicit about the failure action. A conservative policy may withdraw on a failed authoritative data check, a listener failure, a broken address family, or a critical DNSSEC inconsistency. A more complex node might keep one service address advertised while withdrawing another. RFC 4786 discusses multi-service nodes and why a single covering route can make per-service coupling less precise. If the route cannot express the health boundary you need, split the service or adopt a different routing design.

Data synchronization and DNSSEC consistency

The synchronization path should be separate from the query path when practical. It needs authentication, authorization, replay protection, and observability. A zone transfer or artifact pipeline can succeed at the transport level while delivering the wrong content, so compare serials and selected record digests. For DNSSEC, validate that signatures are within their validity windows and that key rollovers are staged consistently. A node with a syntactically loaded zone can still serve a validation failure.

RFC 4786 emphasizes secure zone synchronization and keeping nodes autonomous. Autonomy means a node can continue serving a known-good dataset if a control link fails, while it does not continue to advertise indefinitely after its data becomes unsafe. Define maximum data age, stale-serving policy, and the condition that removes the route. The right answer depends on the zone's change and safety requirements; it should never be an accidental consequence of a timeout in an update script.

See the node, not just the address

When several servers share an address, an ordinary query result does not necessarily tell you which instance answered. RFC 5001 defines the EDNS NSID option: a resolver can request that the responding name server identify its instance. The payload is an opaque byte string whose meaning is chosen by the operator. NSID is a troubleshooting aid, not a requirement that an operator reveal a physical address, hostname, or origin detail.

Choose an identifier stable enough to correlate measurements but restrained enough not to expose sensitive topology. Keep a private mapping to the node, software version, site, and change history. Test that NSID is returned only when requested, and use routing collectors or site telemetry when an identity should not be disclosed.

For active measurement, use a controlled query that the service can answer locally. A random internet name can measure cache warmth, upstream performance, or recursive work rather than path quality. The RIPE Labs methodology for measuring anycast DNS with RIPE Atlas illustrates the distinction: distribute probes, issue a locally answerable query, measure response behavior, and identify the serving instance when possible. Its important caveat is that the selected path reflects network topology and policy, not necessarily physical distance or minimum latency.

What to measure every day

A single global uptime number can conceal a regional routing failure. At minimum, measure from multiple regions and ASes, with enough diversity to include major access networks, cloud networks, and transit paths relevant to the service. Track the distribution of answering node identities, not just aggregate success. A sudden change in the distribution can be an early sign of a route leak, a withdrawn path, a peering change, or a site that is attracting more traffic than expected.

Measure latency distributions rather than only averages. Record timeouts, connection failures, truncated responses, and DNS RCODEs separately. SERVFAIL, REFUSED, NXDOMAIN, and NOERROR mean different things and should not be collapsed into “DNS failed.” For authoritative service, compare answers and SOA serials across nodes. For recursive service, separate cache-hit and cache-miss probes, and record validation outcomes.

Split the matrix by IPv4 and IPv6, UDP and TCP, and ordinary and DNSSEC-aware queries. Include EDNS behavior and response sizes that are representative of production. A node can have healthy IPv4 while IPv6 is unreachable, or healthy UDP while TCP fallback fails. DNSSEC validation can fail only for larger responses or for a particular key state. These are operational defects even if a basic A query succeeds.

Correlate DNS metrics with BGP state and capacity. When latency rises, ask whether the answering node changed, whether the path changed, whether the node is overloaded, or whether recursive work increased. Preserve raw observations long enough to compare a suspected incident with a normal baseline. Use alerts that identify scope: “timeouts from these ASes over IPv6” is actionable; “DNS latency high” is a starting point, not a diagnosis.

Anycast, unicast, GeoDNS, and response-based failover

Anycast offers one service address and lets routing select among nodes. It can reduce path length for some networks and distribute short transactions, but it makes path selection less visible and route convergence part of the failure model. It fits when nodes can serve requests independently and route outcomes are measurable.

GeoDNS chooses responses based on the resolver address, an EDNS Client Subnet signal when available, or provider-defined geography. It changes DNS data returned to a client; it does not itself reroute packets to a different authoritative node. A geolocation decision can be stale, coarse, or based on the location of a recursive resolver rather than the end user. Use it when response steering is the objective, and do not describe it as a substitute for route diversity.

Response-based failover returns one endpoint normally and another when a health policy marks the first unhealthy. This can be appropriate for application addresses, but DNS caching means a prior response remains usable until its TTL and resolver behavior permit another query. It also requires careful semantics for an all-unhealthy state. Google Cloud DNS documents provider-specific routing policies in which health checks can select backup endpoints, while some all-unhealthy cases still return configured endpoints because an authoritative service must return a value. That behavior is not a DNS standard and must not be generalized to other providers.

TTLs are cache controls, not route timers

A DNS record TTL tells a caching resolver how long the record may be retained under DNS rules. It does not instruct BGP to withdraw a route, and it does not guarantee that all clients will query again when the TTL expires. Resolver implementation, negative caching, application behavior, and upstream caches all affect what a user observes.

This distinction matters during incidents. Lowering a record TTL in advance can reduce the lifetime of cached response data, but it cannot serve as a precise route-failover control. A planned migration should therefore coordinate route changes, record changes, cache expectations, and monitoring. Do not promise a fixed failover interval solely because a record has a particular TTL.

A staged deployment plan

First, establish the baseline. Run the service on its current architecture and collect node-level identity, route visibility, latency, error, timeout, transport, and answer-consistency data. Define the networks and regions that matter. Document the current TTLs, DNSSEC state, transfer path, and rollback mechanism.

Second, build one candidate node off the serving path. Load the exact zone set or recursive policy, but do not advertise the production service prefix. Test management isolation, software hardening, synchronization, restart behavior, IPv4, IPv6, UDP, TCP, EDNS, DNSSEC, and diagnostic identity. Compare answers and serials with existing nodes.

Third, announce narrowly. Use a controlled BGP session or test prefix where possible, and observe route propagation from multiple collectors. Start with a small, intentional exposure rather than changing every path at once. Compare node distribution, latency, RCODEs, and timeouts. Confirm that the new node attracts only traffic it can serve.

Fourth, exercise withdrawal. Fail the DNS process, invalidate a test zone, block a required dependency, and disable each address family in turn. Verify that the readiness logic removes the intended route without flapping. Confirm that other nodes serve consistent answers and that capacity remains within limits. Re-advertise only after the configured hold-down and full readiness checks pass.

Fifth, expand and document. Add nodes only when measurements show a coverage or failure-domain benefit. Record the intended route policy, site dependencies, owner, configuration version, data age limit, and rollback action. Treat every node as a production change, even if its address is shared.

Incident runbook: make the next failure legible

When users report DNS trouble, first define the symptom by scope. Is it one zone, one record type, one transport, one address family, one resolver AS, or the whole service? Query known names from independent vantage points and record the full response, RCODE, flags, timing, source address, and requested transport.

Next, identify the answering node. Use NSID when it is enabled and permitted, or use the service's diagnostic endpoint and external route observations. Compare the answer and SOA serial with a known-good node. If only one identity is stale or failing, protect it by withdrawing or isolating its route according to the tested procedure; do not edit production data ad hoc on a single node unless the change is part of the source-of-truth process.

Then inspect routing. Check the local BGP session, the node's advertisement, upstream acceptance, and external visibility. Look for recent flaps, a route that is present despite a failed health check, or a withdrawal that has not propagated to the affected networks. Remember that a route can be healthy from one collector and absent from another.

Inspect capacity and transport after scope is clear. Check packets per second, CPU, memory, socket and TCP pressure, response size, fragmentation symptoms, and DNSSEC work. Compare UDP with TCP and IPv4 with IPv6. For recursive service, inspect upstream reachability, cache-miss latency, validation failures, and policy logs. For authoritative service, inspect zone load, transfer state, serials, and signature validity.

Use a reversible mitigation. A route withdrawal, rate limit, temporary policy change, or traffic shift should have an owner, a success signal, and a rollback condition. Do not repeatedly flap a node while trying to discover whether it is healthy. Once service is stable, preserve measurements and route evidence, then update the readiness check or deployment guard that failed to catch the condition.

The edge operating principle

An edge DNS service is not made dependable by a slogan about proximity. It is made dependable when every advertised node can serve the same intended data, every route state has a defined relationship to service readiness, and measurements reveal which networks are actually receiving which node. Anycast can be a useful distribution mechanism, but its benefits emerge only when routing policy, failure domains, capacity, synchronization, and observability are designed together.

For XQO teams evaluating such an architecture, the durable question is not how many locations can be named. It is whether the system can explain its behavior during an ordinary query, a route change, a stale-data event, a transport-specific failure, and a site outage. If the answer is visible in telemetry and rehearsed in a runbook, the edge becomes an engineering system rather than an unverified promise.

References and further reading

1. RFC 4786 — Operation of Anycast Services
2. RFC 5001 — DNS Name Server Identifier (NSID) Option
3. RFC 1034 — Domain Names: Concepts and Facilities
4. RFC 1035 — Domain Names: Implementation and Specification
5. RFC 2182 — Selection and Operation of Secondary DNS Servers
6. AFRINIC DNS Anycast Nodes Deployment Guide
7. Google Cloud DNS — Routing policies and health checks
8. RIPE NCC DNS overview
9. RIPE NCC DNSMON
10. RIPE Labs — Measuring Anycast DNS Services Using RIPE Atlas
11. Cloudflare — What is Anycast DNS?