Designing Fallback Logic For Reliable DNS Steering

DNS steering is useful when the system has enough information to choose a good answer.

But the internet is not perfect.

Signals can be missing.

Health checks can be delayed.

Policies can have gaps.

GeoIP data can be incomplete.

A resolver may hide the real user location.

An endpoint may fail.

A runtime snapshot may be stale.

This is why DNS steering needs fallback logic.

Fallback logic defines what answer should be returned when the preferred answer cannot be used.

A good fallback design protects users from broken, missing, or unsafe answers.

It also helps operators keep DNS behavior predictable during failure.

What Is Fallback Logic?

Fallback logic is the set of rules used when the best or preferred answer is not available.

Simple example:

Preferred endpoint:
Manila

Fallback endpoint:
Singapore

If Manila is healthy:
Return Manila.

If Manila is unhealthy:
Return Singapore.

This is a basic fallback rule.

In production systems, fallback logic can be more layered.

Example:

Try exact ASN policy.
If no match, try country policy.
If no match, try regional policy.
If no match, use global default.
If preferred endpoint is unhealthy, use backup endpoint.
If health data is missing, use last known safe state.

Fallback logic is not an afterthought.

It is part of the routing design.

Why DNS Steering Needs Fallbacks

DNS steering depends on available signals.

Those signals may include:

Requested name
Record type
Resolver IP
Country
Region
ASN
Network prefix
EDNS Client Subnet
Endpoint health
Policy rules
Runtime snapshot version

Any of these can be missing, stale, or wrong.

If the system has no fallback, a small data problem can become a user-facing outage.

Example:

Country lookup failed.
No fallback exists.
DNS returns no useful answer.
Users fail to connect.

That is a bad design.

A safer design says:

Country lookup failed.
Use global default endpoint.
Return useful answer.
Log the fallback.

The user still gets a working answer.

The operator still has a signal to investigate.

DNS Should Prefer Safe Answers

A DNS steering system should always try to return a safe answer.

Safe does not always mean perfect.

It means the answer is expected to work.

Example:

Best answer:
Manila endpoint with 20 ms latency.

Safe fallback:
Singapore endpoint with 55 ms latency.

Bad answer:
Manila endpoint that is known to be down.

A farther working endpoint is better than a nearby broken endpoint.

This principle matters in DNS steering.

The goal is not only to choose the closest endpoint.

The goal is to return the safest useful answer using the information available at query time.

Common Reasons Fallback Is Needed

Fallback logic is needed in many situations.

1. No Policy Match

A query may not match any specific routing rule.

Example:

Policy exists for PH and SG.
Requester is from ID.
No ID policy exists.

The fallback may be:

Use Southeast Asia default.

Or:

Use global endpoint.

Every policy tree should have a default answer.

2. Missing Country

The requester IP may not map to a country.

Possible reasons:

Unknown IP range
Private IP address
Incomplete GeoIP data
Bad input
Resolver not recognized

Fallback:

Use global default endpoint.

3. Missing ASN

The system may fail to identify the requester ASN.

Fallback:

Use country policy.

Example:

Country is PH.
ASN is unknown.
Use default PH endpoint.

4. Missing EDNS Client Subnet

EDNS Client Subnet, or ECS, may not be present.

Fallback:

Use resolver IP.

ECS is useful, but it is not always available.

A DNS steering system should still work without it.

ECS is defined in RFC 7871.

Source:

https://www.rfc-editor.org/rfc/rfc7871

5. Preferred Endpoint Is Unhealthy

The best endpoint for a requester may be unhealthy.

Fallback:

Use backup endpoint.

Example:

PH users prefer Manila.
Manila is unhealthy.
Return Singapore.

6. Health Data Is Stale

The system may have health data, but it may be too old.

Example:

Last health result:
15 minutes ago

Allowed health age:
2 minutes

The fallback depends on the design.

Possible fallback:

Use last known safe state.

Or:

Use backup endpoint with fresh health data.

The important part is that stale data should not be treated the same as fresh data without a clear rule.

7. Runtime Snapshot Is Missing Or Invalid

A data plane may need a prepared runtime snapshot to answer quickly.

If the latest snapshot is missing or invalid, fallback may be:

Continue using previous known good snapshot.

This is safer than loading a broken version.

8. Endpoint Pool Is Empty

A policy may point to a pool that has no available endpoint.

Example:

Manila pool exists.
All Manila endpoints are unhealthy.

Fallback:

Use regional backup pool.

If no backup exists, use global backup.

9. Data Source Is Unavailable

The control plane, database, health system, or network intelligence source may be unavailable.

The data plane should not stop answering if it already has a safe prepared state.

Fallback:

Keep using loaded runtime state.
Log control plane data delay.

This is why the data plane should avoid depending on live database calls for every DNS query.

Types Of Fallbacks

Fallback logic can be designed in layers.

1. Policy Fallback

Policy fallback handles missing or unmatched rules.

Example:

Exact prefix rule
    |
ASN rule
    |
Country rule
    |
Regional rule
    |
Global default

If the exact rule is missing, the system moves to a broader rule.

2. Health Fallback

Health fallback handles unhealthy endpoints.

Example:

Preferred endpoint
    |
Backup endpoint
    |
Regional backup
    |
Global backup

Health fallback should protect users from known bad endpoints.

3. Signal Fallback

Signal fallback handles missing classification signals.

Example:

Use ECS if present.
If ECS is missing, use resolver IP.
If resolver IP is unknown, use default policy.

4. Runtime Fallback

Runtime fallback handles broken or missing runtime state.

Example:

Use latest valid snapshot.
If latest snapshot fails validation, keep previous known good snapshot.
If no valid snapshot exists, use static emergency answer.

5. Answer Fallback

Answer fallback controls what DNS response should be sent when no normal answer is safe.

Possible answers:

Return global default endpoint.
Return static emergency endpoint.
Return last known safe answer.
Return SERVFAIL only when no safe answer exists.

SERVFAIL should not be used as the first fallback for normal missing data.

DNS response codes, including server failure behavior, are part of DNS protocol definitions in RFC 1035.

Source:

https://www.rfc-editor.org/rfc/rfc1035

Designing A Fallback Hierarchy

A fallback hierarchy defines the order of choices.

Example:

1. Match exact prefix policy.
2. If no match, match ASN policy.
3. If no match, match country policy.
4. If no match, match region policy.
5. If no match, use global default policy.
6. Check endpoint health.
7. If preferred endpoint is unhealthy, use backup.
8. If backup is unhealthy, use global healthy endpoint.
9. If no healthy endpoint exists, use last known safe answer.
10. If no safe answer exists, return controlled failure.

This hierarchy should be clear.

Operators should know what happens at every step.

A good fallback hierarchy avoids surprise behavior.

Example: Country Fallback

Assume this policy:

PH users:
Return Manila.

SG users:
Return Singapore.

JP users:
Return Tokyo.

Unknown country:
Return Singapore.

Query:

Requester country:
Unknown

Answer:

Return Singapore.

Why Singapore?

Because it was defined as the default regional endpoint.

The important thing is not that Singapore is always correct.

The important thing is that the behavior is known and intentional.

Example: ASN Fallback

Assume this policy:

PH ASN 64501:
Return Manila.

PH ASN 64502:
Return Singapore.

PH unknown ASN:
Return Manila.

Query:

Country:
PH

ASN:
Unknown

Answer:

Return Manila.

This fallback says:

If the requester is in the Philippines but the ASN is unknown, use the normal Philippine endpoint.

That is safer than failing the query.

Example: ECS Fallback

Assume this classification logic:

If valid ECS exists, classify using ECS.
If ECS is missing, classify using resolver IP.
If resolver IP is unknown, use default.

Query:

Resolver IP:
Singapore

ECS:
Missing

Classification:

Use resolver IP.
Classify as Singapore.

Answer:

Return Singapore endpoint.

If ECS later appears with a Philippine client prefix, the answer may change.

But the system still works when ECS is missing.

That is the key.

Example: Health Fallback

Assume this routing policy:

PH users prefer Manila.
Fallback is Singapore.
Global backup is Tokyo.

Health state:

Manila:
Unhealthy

Singapore:
Healthy

Tokyo:
Healthy

Answer:

Return Singapore.

The system should not return Manila only because the policy prefers it.

Health must protect the final answer.

Example: Stale Health Data

Assume this health state:

Manila:
Healthy

Last checked:
20 minutes ago

Allowed age:
2 minutes

This health state is stale.

Possible rule:

If preferred endpoint health is stale, use backup endpoint with fresh healthy status.

Another possible rule:

If all health data is stale, use last known safe answer and lower the TTL.

Both can be valid depending on the service.

The main point is this:

Stale health data needs a defined rule.

Do not let the system guess.

Last Known Good State

Last known good state is a useful fallback pattern.

It means the system keeps the last version that passed validation and worked safely.

Example:

New policy snapshot:
Invalid

Previous policy snapshot:
Valid

Action:
Keep using previous policy snapshot.

This protects the data plane from bad updates.

Last known good state can apply to:

Policy snapshots
Endpoint lists
Health summaries
Routing maps
Default answer sets

It should not be used blindly forever.

Operators should monitor its age.

Example:

Last known good snapshot age:
3 hours

If the snapshot is too old, the system should raise an alert.

Static Emergency Answer

A static emergency answer is a final safety layer.

Example:

emergency.example.net
203.0.113.250

Or:

Global emergency endpoint:
192.0.2.250

This answer should be simple, tested, and documented.

It can be used when normal routing data is not available.

A static emergency answer is not always the best endpoint.

It is the endpoint that is expected to work when the normal decision system cannot make a safe choice.

Avoiding Fallback Loops

Fallback logic must not create loops.

Bad example:

Manila fallback is Singapore.
Singapore fallback is Manila.
Both are unhealthy.

This can cause confusing behavior if the system keeps moving between the two.

Better design:

Manila fallback is Singapore.
Singapore fallback is Tokyo.
Tokyo fallback is global emergency endpoint.

Fallback paths should be checked during policy validation.

The control plane should reject fallback chains that loop.

Avoiding Silent Bad Fallbacks

A fallback should not hide serious problems forever.

Example:

Manila is unhealthy.
System returns Singapore.
Users keep working.
No alert is sent.
Operators do not notice Manila is down.

This is dangerous.

Fallback should keep users connected, but it should also create operational visibility.

The system should log and alert:

Fallback used.
Preferred endpoint unhealthy.
Traffic shifted to backup.

Fallback is not only a routing behavior.

It is also an operational signal.

TTL During Fallback

TTL matters during fallback.

Example:

Normal TTL:
300 seconds

Fallback TTL:
60 seconds

A shorter fallback TTL can help the system recover faster when the preferred endpoint becomes healthy again.

But lower TTL also increases DNS query volume.

There is no single correct TTL for every service.

A practical rule:

Use lower TTL during unstable states only when the authoritative DNS system can handle the extra query load.

DNS TTL and caching behavior are part of standard DNS operation.

Sources:

https://www.rfc-editor.org/rfc/rfc1034
https://www.rfc-editor.org/rfc/rfc1035

Should DNS Return SERVFAIL?

SERVFAIL means the DNS server failed to complete the request.

It should be used carefully.

For many DNS steering systems, normal missing policy data should not cause SERVFAIL.

Example:

Country unknown:
Do not return SERVFAIL.
Use default answer.

Example:

ASN unknown:
Do not return SERVFAIL.
Use country policy.

SERVFAIL may be reasonable only when the system truly cannot return a safe answer.

Example:

No valid zone data exists.
No safe default exists.
No known good answer exists.
The request cannot be answered safely.

Even then, operators should treat this as a serious failure.

Should DNS Return NXDOMAIN?

NXDOMAIN means the name does not exist.

Do not use NXDOMAIN as a fallback for temporary routing problems.

Bad example:

Endpoint unhealthy.
Return NXDOMAIN.

This is usually wrong.

The domain still exists.

The service may only need a backup answer.

NXDOMAIN should mean the name does not exist, not that the preferred endpoint is down.

This distinction matters because resolvers and clients may cache negative answers.

DNS negative caching behavior is described in RFC 2308.

Source:

https://www.rfc-editor.org/rfc/rfc2308

Fallback And Serving Stale Data

Some DNS resilience designs use stale data when fresh data is unavailable.

RFC 8767 discusses serving stale DNS data to improve resilience when authoritative answers are unavailable.

Source:

https://www.rfc-editor.org/rfc/rfc8767

For DNS steering, the related design idea is this:

A slightly old safe answer may be better than no answer.

But stale answers need limits.

Example:

Use stale safe answer for up to 10 minutes.
After that, raise critical alert.
If stale data becomes too old, use emergency default or controlled failure.

Stale data should be intentional, bounded, and visible.

Fallback And The Control Plane

The control plane should define and validate fallback rules.

It should check:

Does every policy have a default?
Does every pool have a backup path?
Are fallback paths free from loops?
Do fallback endpoints exist?
Are fallback endpoints allowed for this service?
Are TTL values valid?
Can the policy still answer when signals are missing?

The control plane should catch these issues before publishing the policy.

The data plane should not discover them during live DNS queries.

Fallback And The Data Plane

The data plane should apply fallback rules quickly.

It should already have the needed state loaded.

Example data plane logic:

Receive query.
Find policy.
Classify requester.
Select preferred endpoint.
Check prepared health state.
If preferred endpoint is valid and healthy, return it.
If not, select fallback endpoint.
If fallback is valid and healthy, return it.
If no healthy endpoint exists, use last known safe answer.
If no safe answer exists, return controlled failure.

This logic should be fast and predictable.

The data plane should not call many external systems before answering.

Logging Fallback Decisions

Every fallback decision should be explainable.

Useful log fields:

Requested name
Record type
Resolver IP
ECS prefix, if present
Classified country
Classified ASN
Matched policy
Preferred endpoint
Preferred endpoint health
Fallback endpoint
Fallback reason
Returned answer
TTL
Policy version
Snapshot version

These logs help operators answer:

Why did this query receive the backup endpoint?
Was the requester unclassified?
Was ECS missing?
Was the preferred endpoint unhealthy?
Was health data stale?
Which fallback rule was used?

Without logs, fallback behavior becomes hard to trust.

Metrics For Fallback Logic

Fallback logic should be measured.

Useful metrics:

Fallback rate
Fallback rate by domain
Fallback rate by country
Fallback rate by ASN
Fallback rate by endpoint
Unknown country rate
Unknown ASN rate
Missing ECS rate
Unhealthy endpoint fallback count
Stale health fallback count
Default policy usage
SERVFAIL count
NXDOMAIN count
Last known good state usage
Emergency answer usage

These metrics tell operators when the system is no longer using the ideal path.

A rising fallback rate may mean:

Endpoint failure
GeoIP issue
ASN mapping issue
Policy gap
Health check problem
Snapshot delay
Resolver behavior change

Fallback metrics should be part of DNS steering observability.

Testing Fallback Logic

Fallback logic should be tested before production use.

Test cases should include:

Known country with healthy endpoint
Known country with unhealthy endpoint
Unknown country
Known ASN
Unknown ASN
ECS present
ECS missing
Policy missing
Pool empty
Health stale
All regional endpoints unhealthy
Snapshot invalid
Fallback loop
Emergency answer path

The goal is to know exactly what the system will do before the failure happens.

A fallback that has never been tested is only a guess.

Example Fallback Matrix

A fallback matrix can make behavior easier to review.

This table should be adapted per service.

The important part is that every failure case has a planned response.

Good Fallback Design Principles

A good fallback design follows these principles:

Every policy should have a default.
Every preferred endpoint should have a backup.
Health should override preference.
Missing signals should not break answers.
Stale data should have limits.
Fallback paths should not loop.
Fallback use should be logged.
Fallback use should be measured.
Emergency answers should be tested.
Bad policy updates should not replace known good state.

These principles keep DNS steering safer.

Common Fallback Mistakes

Mistake 1: No Default Answer

Every policy needs a default.

Without a default, unknown requesters may receive poor answers or no answer.

Mistake 2: Returning An Unhealthy Preferred Endpoint

Preference should not override health.

If the preferred endpoint is known bad, use fallback.

Mistake 3: Treating Missing Data As Failure

Missing ASN, missing ECS, or unknown country should not automatically break DNS answers.

Use safe defaults.

Mistake 4: Using SERVFAIL Too Quickly

SERVFAIL should be a last option, not the normal response to missing classification data.

Mistake 5: Returning NXDOMAIN For Temporary Problems

NXDOMAIN means the name does not exist.

Do not use it for temporary endpoint failure.

Mistake 6: No Alert On Fallback

Fallback without alerting can hide real problems.

Mistake 7: No Limit On Stale State

Old data may become unsafe.

Last known good state should have age limits and alerts.

Mistake 8: Untested Emergency Path

Emergency fallback should be tested before it is needed.

Practical Design Rule

Use this rule:

Fallback logic should keep DNS answers useful, safe, predictable, and explainable when the preferred path is not available.

This rule covers the main goal.

Useful means the user can still connect.

Safe means the answer avoids known bad endpoints.

Predictable means the system follows defined rules.

Explainable means operators can understand why the answer was returned.

How This Fits The Article Series

The earlier articles covered the foundation:

Article 1:
How authoritative DNS influences traffic direction.

Article 2:
The control plane and data plane of DNS steering.

Article 3:
Why DNS steering is not load balancing.

Article 4:
Why geography, latency, and network distance are different.

Article 5:
How requester classification works.

Article 6:
How EDNS Client Subnet can help or hurt DNS steering.

This article focuses on reliability.

Even with good classification, good policy, and useful signals, failures will happen.

Fallback logic decides what the DNS system should do when the ideal answer is not safe or not available.

Key Takeaways

Fallback logic is required for reliable DNS steering.

A fallback is the planned answer when the preferred answer cannot be used.

Fallback may be needed because of missing policy, unknown country, unknown ASN, missing ECS, unhealthy endpoints, stale health data, invalid snapshots, or unavailable data sources.

A good fallback hierarchy moves from specific rules to broader rules.

Health should override preference.

Missing data should lead to safe defaults, not automatic failure.

SERVFAIL should be used carefully.

NXDOMAIN should not be used for temporary endpoint problems.

Last known good state can protect the data plane from bad updates.

Static emergency answers can provide a final safety layer.

Fallback use should be logged, measured, tested, and visible to operators.

The best fallback logic does not try to be perfect.

It tries to keep DNS answers safe and useful when the internet, the data, or the service state is imperfect.

In the next article, we will discuss Health Checks In DNS Steering: From Origin Status To Answer Policy.

Sources

RFC 1034, Domain Names, Concepts and Facilities:
https://www.rfc-editor.org/rfc/rfc1034

RFC 1035, Domain Names, Implementation and Specification:
https://www.rfc-editor.org/rfc/rfc1035

RFC 2308, Negative Caching of DNS Queries:
https://www.rfc-editor.org/rfc/rfc2308

RFC 7871, Client Subnet in DNS Queries:
https://www.rfc-editor.org/rfc/rfc7871

RFC 8767, Serving Stale Data to Improve DNS Resiliency:
https://www.rfc-editor.org/rfc/rfc8767