Summary: A retry limit is not a storm defense. NResilience's budget bounds retries as a fraction of traffic instead, funds them from a token bucket that successful calls feed, refuses amplification without ever refusing a first attempt, exempts the refusals your own process issues, holds half a bucket back for work someone is waiting on, and reports the nested loops no single layer can see. Here is each piece and the reasoning behind it.

The multiplication nobody sees

Retries compose multiplicatively. A frontend that allows three retries calls a backend that allows three, which calls a database that allows three: one user action that fails at the bottom becomes 4 x 4 x 4 = 64 database attempts. Each layer's configuration looks reasonable in isolation - three retries is a modest number - and every layer is blind to the aggregate, because no single layer can see the whole call chain.

This is what makes a retry storm different from ordinary overload. It is not one misbehaving client. It is a fleet of clients, each independently behaving exactly as configured, delivering hammer blows to a dependency at the one moment it can least afford them. The mechanism every layer installed to survive the failure is the thing that finishes the dependency off.

The obvious fix - lower the attempt counts - does not work, and it is worth seeing why before reaching for it. Attempts compose multiplicatively: whatever fixed limit each layer picks, the product across layers grows faster than any of them. Two retries per layer is still 27 attempts at the bottom of a three-hop chain. No per-call number fixes a cross-call problem.

The fix is a fraction, not a count

A budget expressed as a fraction of traffic composes differently. If every client independently funds retries at 10% of its successful traffic, fleet-wide amplification is bounded at 1.1 times the original traffic - whatever the topology, however deep the call chain, with no coordination protocol anywhere.

That bound comes from arithmetic, not cooperation. Each client holds its own retries at one-tenth of its own successes, and the fleet's total is the sum of those ratios - so the fleet's aggregate amplification is 1.1x too, however the clients differ in size. Google SRE uses the same figure; Finagle and Envoy budget 0.2.

This is why the retry budget is on by default in NResilience and the attempt count is only a maximum. Attempts answers "how many times may this one call retry". The budget answers "how much traffic may this process add to a struggling dependency" - which is the question the dependency actually cares about.

The token bucket

The budget is a token bucket with two sources of tokens and one rule about spending:

Deposits. Every successful attempt deposits fraction tokens - 0.1 by default. Ten successes fund one retry. In steady state, retries can add at most 10% on top of your successful traffic, whatever your traffic is.
A floor rate. minimumPerSecond (3 per second by default) refills the bucket at a constant rate regardless of traffic, so a quiet client whose successes are too sparse to fund anything can still retry. Set it to zero and the fraction becomes the only source of tokens.
A burst bound. The bucket's capacity is ten seconds of the floor rate - 30 tokens at the defaults. This caps the burst a recovering client can spend at once without changing the sustained rate the floor and deposits set.
A full bucket at cold start. A new process begins with a full bucket. Throttling a fresh deployment's first retries would penalize deployment rather than a storm, and the budget's job is the opposite.

One charging rule sits above all of this: only retries are charged. The first attempt of every call always executes. The budget is not a rate limiter - it refuses amplification, not traffic. A dependency cannot be starved by its clients' budgets, because no budget can stop the calls the caller actually wanted to make.

The lock holds arithmetic and nothing else

Deposit runs on every successful attempt, which makes it the one gate the library takes on the success path. Hold one static policy - the shape every application settles into - and all of that traffic funnels through a single lock inside the budget.

So the lock's critical section is arithmetic and nothing else. The clock read happens before the lock is taken; the timestamp arrives with the caller. A thread can arrive holding a timestamp another thread has already refilled past, and the guard for that already existed for non-monotonic clocks: a refill with no elapsed time accrues nothing rather than rewinding the refill point. The lost accrual is the sub-microsecond gap between two clock reads, which no per-second rate can observe - the lock remains a thin layer of arithmetic on the success path.

It is a small thing, and it is the difference between a bucket that costs nothing on the hot path and one that shows up in a flame graph.

The hint that stays honest

When a retry is refused, the call stops with StopReason.BudgetExhausted after the library's short guarded rejection pause, and CallRejectedException.RetryAfter says when to come back. The number is worth trusting, because of when it is computed.

The executor asks the budget for the hint after serving the 100ms rejection pause, and the hint refills the bucket first. Get that ordering wrong and the answer is the shortfall as it stood 100ms ago - an overstatement, which is the one thing a RetryAfter must not be. A caller that schedules its own polling on the hint gets the wait the floor rate actually needs to accrue the next token, not a padded version of it.

And when the floor is zero, the hint is null. With no floor rate, only successful traffic can refill the bucket, and no one can honestly predict when your next success arrives. The budget would rather say nothing than make something up.

What the budget exempts

Two kinds of retry never draw on the bucket, and both exemptions are about protecting the mechanism from itself.

A refusal this process issued. When your own rate limiter refuses an attempt, the outcome is classified Throttled with a self-imposed bit set - the dependency was never contacted, so the retry costs it nothing. Charging that retry against the budget would let a burst of self-throttling quietly drain the capacity that real transient failures need, and it would drain it at exactly the moment a client is under load, which is when that capacity matters most. So a self-imposed refusal neither spends nor deposits; it is not evidence in either direction.

The breaker needed a subtler version of the same care. Its recording is still called for a self-imposed refusal - it records nothing, but it returns the in-flight probe slot the refused call occupied. Skip the call to save the work and the probe count leaks downward until a half-open breaker has no slots left to give and never recovers.

Half the bucket, for work nobody is waiting on. A nightly backfill and a checkout draw on the same bucket, and by default the executor cannot tell them apart. But a budget is a fraction of traffic, not a queue - the only way to guarantee capacity for the checkout is refusing to hand the last of it to work nobody is waiting for. A Sheddable call must leave half the bucket behind: while the dependency is healthy, nothing is refused; once the bucket drains, the backfill stops first.

Every other criticality level spends the bucket exactly as before. The refusal is the ordinary one - same pause, same StopReason.BudgetExhausted - because criticality changes when a refusal happens, not what it is.

Sharing, and where buckets go to die

Budget state is per process and unshared. There is no coordination between pods, and that is not a defect - the statistical argument from the fraction is what makes coordination unnecessary. But it decides where a bucket can live:

RetryBudget.Automatic, the default, resolves to a bucket private to each policy instance - or to each key when the policy is scoped. The privacy is the point: it is why a storm against payments cannot throttle retries to search. Policies derived from the same preset deliberately do not share a bucket.
RetryBudget.Shared("payments") is a process-wide budget looked up by name. Two policies naming the same string share it, and the first caller's parameters win. Share one across the policies that hit the same dependency, so three clients of the same API do not each get their own 10% allowance.
RetryBudget.None is the one spelling of off, and it is right only when the dependency is known not to be shared.

The per-process rule has a consequence worth stating bluntly: a budget allocated per HttpClient instance, or resolved from a scoped DI container, is worthless. It is discarded before it observes enough traffic to bound anything. Hold one instance, or name it.

Two lifetime details keep the mechanism intact where you would expect it to break. In DI, the bucket is pinned to the registration name, so a configuration reload creates a new policy instance but keeps the traffic history. And the bucket an Automatic marker resolves to lives in a ConditionalWeakTable keyed by policy identity, so lazy initialization does not affect record equality - two equal policies still compare equal after one of them has warmed a bucket.

Reading the budget

A budget near capacity is the library telling you it is preventing a storm right now:

if (result.Exception is CallRejectedException rejection)
{
    Console.WriteLine(rejection.Reason);     // BudgetExhausted
    Console.WriteLine(rejection.RetryAfter); // when the floor funds the next token
}

Utilization reads 0 to 1 for dashboards. A budget sitting near 1 is a client whose retries are being refused - a symptom to alert on, not a steady state, because it means your dependency is failing and the budget is absorbing what would otherwise be your contribution to the incident. The ratio of nresilience.attempts to nresilience.calls is the same story from the other side: it is the retry fraction you actually ran, and it is the one number that says whether the amplification you designed for is the amplification you got.

The loop no single layer can see

The budget bounds what this process spends. The other half of the amplification problem is structural: your frontend retries onto a backend that retries onto a database, and every layer sees only its own three attempts - the same blindness that makes fixed limits useless.

NResilience's HTTP handler detects nesting two ways: a flag in the execution context catches it within a process, and an HTTP header catches it across process boundaries. The handler stamps X-NResilience-Retrying: 1 on every request it can retry, and the stamp is on the first attempt, not the retries - because by the time a retry goes out, the amplification has already happened. The gRPC interceptor carries the same marker as x-nresilience-retrying: 1, because the chain that amplifies is rarely all one protocol.

In an ASP.NET Core app, receiving the marker is one line:

app.UseResilienceNestedRetry();

Registered before anything that makes an outbound call, it publishes the flag this request carries, so the middle hop of a chain reports NestedRetry events itself and can see an amplification it is part of.

Detection reports and does not intervene: silently dropping configured retries would be surprising and hard to debug. The right response is architectural - the layer closest to the failing dependency should do most of the retrying, and the layers above it should pass the call through. The library finds the loops; you decide what they mean.

The pacing around it

The budget decides whether a retry happens. The backoff decides when, and its defaults each carry a reason:

Two bases, because two verdicts. A Transient failure waits 100ms before the first retry; a Throttled one waits 1s. A delay tuned for connection resets is too aggressive for a rate limiter - a 429 is the dependency telling you about its token bucket, not about how long its queue is.
Full jitter, always on. A brief outage ends and every client's accumulated retries arrive together - a second traffic spike on top of the recovery. Jitter removes the correlation; without it, a fleet turns a two-second blip into a thundering herd.
Server pushback wins verbatim. A Retry-After header on a 429 or 503 replaces the computed delay exactly - no jitter, no curve - capped only by the configured maximum and the remaining deadline. Where the server knows, it says so.
The measured base, opt-in. Backoff.Measured(1) sets the transient base to one recent normal call's duration instead of a millisecond count that is only right for one dependency. It is the one measured term the library does not turn on for you, because it is the one that can lengthen a delay rather than only shorten one - so it is clamped to a factor of 10 either side of the constant you wrote, and the constant stays the anchor.

The trade, priced

None of this is free. A budget is a claim that surviving a dependency's struggles matters more than the availability your retries would have bought during them - and that trade can be measured before shipping. A five-minute simulation of a dependency that goes eight times slower for one minute, at 500 calls a second, puts the worst second the dependency feels at 2.1x load with the budget off and 1.1x with it on, for six points of this client's availability.

That is the whole idea in two numbers. The budget converts your worst second into the dependency's survivable one, and it is on by default because the alternative - every client independently being reasonable - is how the storm starts.

For more information, see Retry budget internals, which has the full token-bucket mechanism and the reasoning for bounding by traffic rather than by attempts.

A deep dive into retries and the retry budget