Summary: A circuit breaker is not a failure counter with a door on it. NResilience's breaker measures what your dependency considers normal, trips on deviations from that normal rather than on magic thresholds, refuses calls without melting your CPU, hands traffic back over a ramp, and de-correlates a fleet's probes with equal jitter. Here is each piece and the reasoning behind it.

Stop calling it

When a dependency starts failing, the instinct is to retry until it goes through. If a hundred instances of your app all do that at once, you have not fixed the outage - you have organized a distributed denial of service attack against your own infrastructure. That is how a database hiccup becomes a system-wide outage.

The kindest thing you can do for a struggling dependency is to stop calling it. That is the breaker's whole job.

The breaker is a state machine with three states you will actually see in operation:

Closed - normal operation. The circuit is complete, traffic flows, and the breaker quietly samples every attempt.
Open - the breaker has tripped. Every call is refused with a CallRejectedException without touching the network. The dependency gets silence, which is the one thing it needs.
HalfOpen - after the break duration elapses, a small number of trial calls are let through. One concurrent probe by default. Two successful probes close the breaker; a failed probe re-opens it immediately.

That last number is worth pausing on. Closing after a single lucky probe is how you get oscillation: a dependency that is only intermittently available passes one probe just as a fleet of accumulated retries arrives, and the cycle repeats. Two consecutive successes prove the dependency actually recovered.

There is a fourth state, Recovering, but it is opt-in - we get to it later.

The break doubles

If a probe fails, the breaker re-opens - and the next break is longer. The break duration doubles on each consecutive open, up to a configured maximum (15 seconds by default, 2 minutes at the top). It is exponential backoff applied to the breaker itself.

This is what bounds a flapping dependency. A service that comes back, accepts its probe, and dies again does not get probed every 15 seconds forever: 15, 30, 60, 120. Unstable dependencies sit in the timeout corner longer, preventing an endless open/half-open loop.

Tripping without guessing a rate

The classic breaker asks you for a flat error threshold: trip at 5%. That number cannot fit two dependencies. For a payments API that normally runs at a 0.02% error rate, 5% is a page-everyone catastrophe. For a flaky third-party search backend that has always failed 30% of calls, 5% is a quiet Tuesday. Hardcoding a threshold before the code has seen production traffic is guesswork either way.

NResilience measures instead. By default the breaker tracks the dependency's own error rate over a rolling 5-minute baseline and trips when the current window's rate exceeds 5 times that baseline. A payments API spiking to 0.1% - five times its flawless normal - trips, even though no absolute rule would notice. The search backend's 30% is accepted as its normal; only a significant deterioration trips anything.

Both relative trips come with guards, because a relative measurement against a tiny sample is a way to trip on noise:

A floor of 5% absolute. Below that, nothing is wrong no matter how quiet the baseline was. Five times a baseline of nearly zero is still nearly zero, so without the floor the first error of the day would open the circuit.
At least two failures in the trip window. One failure is not a rate, it is an event. At a 5% floor and 20 minimum calls, a single transient error would be exactly the threshold - so the relative trip needs a second one.
Enough samples. The error baseline needs 100 outcomes before the relative trip arms (Failures.MinimumSamples); the latency baseline needs 20 successful calls. Until then the breaker behaves exactly like a consecutive-failures breaker - which is also exactly what it does at cold start, because the consecutive-failure trip is on and needs no history.

The consecutive-failure trip (5 by default) is the default's backbone for a related reason: a rate-based trip needs throughput to mean anything, and a service doing two calls a minute would never arm one. The breaker samples attempts, not logical calls, so a first attempt that trips the breaker stops the second attempt immediately.

Brownouts: failing slowly

A dependency does not have to return errors to be broken. A 30-second 200 OK is often more dangerous than a 10ms 500 Internal Server Error. The hard failure is fast and frees your threads; the brownout ties up your thread pool and connection pool while claiming everything is fine.

The breaker tracks a latency baseline next to the error baseline: the median of successful call duration over the last 5 minutes. An attempt counts as slow when it exceeds 3 times that median, and half the trip window being slow trips the breaker. Nothing here asks you for a millisecond figure.

Why the median and not a high quantile? Because a threshold defined as "the p99 of recent latency" has 1% of calls above it by construction, whatever the dependency does - it can never trip. The median survives a brownout until it accounts for more than half the window it is measured over, and the baseline lives on a 5-minute clock against the trip window's 30 seconds, so the trip window fills with slow calls long before the baseline forgets what healthy looked like.

The baseline is fed only by successful attempts, and it survives an open, a close, and a Reset. It measures the dependency; it does not judge it. That is what makes the first probe against a still-degraded dependency recognizable as slow - the breaker remembers the pre-brownout latency.

The refusal that breaks the spin

Here is a failure mode the breaker itself can cause. A developer wraps a resilient call in a while loop - a critical background job polling until the service is back. The breaker opens, the call is refused... instantly. No network latency, no timeout, just an exception at the speed of a method call. The loop catches it, calls again, gets refused again. Thousands of iterations a second, CPU at 100%.

The mechanism built to shed load has become a load generator - self-inflicted, and invisible on the network dashboards, because outbound traffic is zero. Your system looks safe while the client's CPU melts.

NResilience fixes this with guarded rejection: a refused call pauses 100ms before returning the refusal. 100ms is imperceptible to a user, but it's a speed bump that caps a polling loop at 10 iterations a second instead of 10,000.

The pause is not configurable, on purpose: its job is to set a floor under the rate of a rejection loop, not to be a tuning knob. It is bounded by the time remaining on the call's deadline and observes the cancellation token, so a refusal never pushes a call past the caller's budget and cancelling aborts it immediately. If you schedule your own retries, CallRejectedException.RetryAfter says when to come back - the break actually being served, not the nominal figure.

if (result.Exception is CallRejectedException rejection)
{
    Console.WriteLine(rejection.Reason);   // DependencyUnavailable, or BudgetExhausted
    Console.WriteLine(rejection.RetryAfter); // when to come back, when there is an answer
}

This is the general truth the breaker makes visible: you have to protect the dependency from the client, and the client from its own protective mechanisms.

Hand the traffic back over a ramp

Two successful probes prove the dependency can serve two calls. A naive breaker reads that as proof it can serve two thousand, closes, and the entire backed-up load arrives in one millisecond. The recovering dependency has a cold cache and an empty connection pool. It fails, the breaker re-opens with a doubled break, and you have the crash/wait cycle all over again.

Recovery adds a transitional Recovering state between half-open and closed. It admits a small fraction of traffic - 5% to start - and refuses the rest. The fraction is the lower of two terms:

A clock, climbing linearly across a ramp whose length is a fraction of the break just served (default 0.25, so a 15-second break ramps over about 4 seconds).
Evidence, which is AIMD - the same algorithm TCP uses for congestion control. Admitted calls coming back fast add a step, slowly and linearly. A call comes back slow: the admitted fraction is slashed in half on the spot. Aggressive on stress, cautious on recovery.

One real failure during the ramp abandons the hypothesis and re-opens the breaker with the grown break. The ramp is opt-in because a call refused during it would have been served by a breaker that closed immediately - but the alternative is not "all served". It is "all served once, then all refused again for twice as long".

The synchronized probe, and equal jitter

The ramp solves the pod-level cliff. There is a fleet-level problem just before it: 200 pods watching the same dependency all trip within a second of each other, all set the same break, and all fire their one probe in the same second. The recovering gateway receives a synchronized pulse of 200 probes, fails them, and all 200 breakers double their backoff together. Fleet-wide oscillation.

The standard fix is jitter - randomizing the break so the pods do not wake together. But Jitter.Full has a flaw: the random delay could be anywhere from zero to the full break, so a pod could probe 200ms into a 15-second break. The break duration is not just de-correlation - it is the guaranteed rest period the dependency gets.

So NResilience defaults to Jitter.Equal: at least half the break is strictly enforced, and only the second half is random.

var breaker = new Breaker(settings: new BreakerSettings
{
    Name = "search",
    BreakDuration = TimeSpan.FromSeconds(15), // now half of that, plus up to half again
    BreakJitter = Jitter.Equal,               // the default
});

Every pod waits at least 7.5 seconds, no exceptions, then adds a random 0 to 7.5 on top. The probes spread over a wide window, and the rest period survives.

Who the breaker trips for

One more operational question: the blast radius. If you wire one global breaker for a dependency, tenant A's malformed requests trip it, and tenant B - sending perfectly valid traffic - finds the door locked too. Tenant A has weaponized your breaker against tenant B.

NResilience isolates state by default where it can: the HTTP handler scopes breakers per host, so a dead analytics API does not trip calls to the auth API; the gRPC integration scopes per service. For the multi-tenant case specifically, PolicyScope<TKey> gives every tenant, shard, or channel its own breaker, retry budget, and latency estimate:

private static readonly PolicyScope<string> Tenants = new(
    Resilience.Default with { Breaker = new Breaker() });

await Tenants.For(tenantId).RunAsync(
    attempt => calls.NextAsync(attempt), cancellationToken);

Tenant A's failures only accumulate in tenant A's breaker. One bound to know: the scope holds at most maximumKeys entries (default 1024), because every entry carries a breaker and a budget, and an evicted tenant that returns gets a fresh breaker that does not remember being open.

What counts as evidence

One subtlety that keeps all of the above honest: the breaker does not count every non-200 as a failure. Outcomes are classified into verdicts, and only Transient outcomes are evidence against the dependency. A 500 is a genuine failure. A 429 - or a 503 carrying a Retry-After header - is classified Throttled: the dependency is up and defending itself with its own rate limiter. The breaker ignores throttled responses entirely, because tripping on a functioning rate limiter turns a working defense into a hard outage. And Permanent outcomes are usually client-side facts, not dependency health.

The breaker as a mirror

One last detail. When a breaker opens, the health check reports Degraded, not Unhealthy. That is deliberate: the dependency is down, but this process is doing exactly what it was designed to do - shedding load and failing fast. Reporting Unhealthy invites Kubernetes to restart the pod, which discards the breaker's state and unleashes the traffic storm all over again. Restarting the pod does not fix the database.

That visibility is worth more than it first appears. When a breaker opens and a seemingly unrelated part of your application degrades with it, the breaker has done more than stop a crash - it has revealed the true blast radius of your architecture: how it actually works, not how you drew it on the whiteboard. It is worth putting breaker state on a dashboard for that reason alone.

For more information, see Breaker internals has the full state machine, the arithmetic behind every default, and the recovery-ramp failure modes.

A deep dive into the circuit breaker