Composing patterns

Resilience patterns wrap a call in layers. Each layer adds its own behavior, and the order of the layers changes what they mean. A timeout outside a retry bounds every attempt together. The same timeout inside the retry bounds one attempt.

A Stack takes the patterns and applies them in the safe order, so the order stops being something to get right at every call site.

The order

Fallback → Retry → CircuitBreaker → RateLimiter → Bulkhead → Timeout → call

Read it as the path of one call. The call travels left to right, and the result or the error travels back right to left. Whatever touches call is innermost.

Layer	Reason it sits where it does
Fallback	Outermost. Catches whatever still escapes the inner layers and returns a safe value.
Retry	Above the breaker. Retries transient errors, and stops as soon as the breaker refuses.
CircuitBreaker	Above the limiter. Once the dependency is failing, refusing here spends no tokens and no permits.
RateLimiter	Above the bulkhead. A call waiting for tokens holds no permit while it waits.
Bulkhead	Above the timeout. Caps concurrency before the call enters its deadline.
Timeout	Innermost. Bounds one attempt.
call	The function.

Stack

Pass the patterns in any order. A Stack applies them in the order above:

import httpx

from grelmicro.resilience import (
    Bulkhead,
    CircuitBreaker,
    Fallback,
    RateLimiter,
    Retry,
    Stack,
    Timeout,
)

SYSTEM = "recs-api"
CALL = "recs-list"

# Named for the system, so every call site shares one circuit and one budget.
breaker = CircuitBreaker(SYSTEM)
limiter = RateLimiter.token_bucket(SYSTEM, capacity=20, refill_rate=10)
pool = Bulkhead(SYSTEM, max_concurrent=10)

recs_list = Stack(
    CALL,
    patterns=[
        # Named for this call: its default and its deadline are its own.
        Fallback(CALL, when=Exception, default=[]),
        Retry.exponential(CALL, when=httpx.HTTPError, attempts=3),
        breaker,
        limiter(key_template="user:{user_id}"),
        pool,
        Timeout(CALL, seconds=1.0),
    ],
)


@recs_list
async def get_recommendations(
    client: httpx.AsyncClient, user_id: str
) -> list[str]:
    response = await client.get(f"/recs/{user_id}")
    response.raise_for_status()
    return response.json()

Naming: system-scoped and call-scoped

A name is not a label, it is the configuration identity of the thing it names. It drives the environment prefix (GREL_TIMEOUT_RECS_LIST_SECONDS, GREL_CIRCUITBREAKER_RECS_API_ERROR_THRESHOLD) and the registration for live reconfiguration. Which name a pattern takes follows from what it protects.

Pattern	Scope	Name it after
`CircuitBreaker`	The external system	The system. Every call site shares one circuit, so one failing dependency trips once
`RateLimiter`	The external system	The system. The quota belongs to the system, not to whoever calls it
`Bulkhead`	Usually the external system	The system, when it caps how hard you may hit it. The call, when it protects your own workers from one slow endpoint
`Retry`	The call	The call. What is safe to repeat depends on the endpoint
`Timeout`	The call	The call. A listing and a report do not share a deadline
`Fallback`	The call	The call. The safe value is the caller's, not the system's

So one external system usually has several stacks, one per call site, sharing the system-scoped instances between them:

import httpx

from grelmicro.resilience import CircuitBreaker, Fallback, Stack, Timeout

SYSTEM = "recs-api"

# One breaker for the system, shared by every call site that reaches it.
breaker = CircuitBreaker(SYSTEM)

recs_list = Stack(
    "recs-list",
    patterns=[
        Fallback("recs-list", when=Exception, default=[]),
        breaker,
        Timeout("recs-list", seconds=1.0),
    ],
)

recs_report = Stack(
    "recs-report",
    patterns=[
        breaker,
        Timeout("recs-report", seconds=30.0),
    ],
)


@recs_list
async def get_recommendations(client: httpx.AsyncClient) -> list[str]:
    return (await client.get("/recs")).json()


@recs_report
async def build_report(client: httpx.AsyncClient) -> bytes:
    return (await client.get("/recs/report")).content

Sharing is by instance, not by name: build the breaker once and put the same object in every stack that reaches that system. Two breakers with the same name and the same backend also share state, but one object is the plain way to say it.

The mistake to avoid is a per-call-site circuit breaker. Give recs-list and recs-report a breaker each and the system has two circuits that must each fail on their own before either opens, so the first failing call site keeps hammering a dependency the second one already gave up on.

A Stack is reusable. Decorate several functions with the same one and they share its breaker, its bucket, and its permits, which is what you want when they call the same dependency.

Two patterns of the same kind are refused, because that needs an order only you know. So is a stack with no patterns, because it would return the function unchanged.

What a Stack guarantees

A refusal by the stack's own rate limiter, bulkhead, or circuit breaker is not an outcome of the call. The call never happened. Inside a Stack those refusals are read that way:

Refusal by the stack's own	Inside a Stack
`CircuitBreakerError`	its `Retry` never retries it
`RateLimitExceededError`	its `CircuitBreaker` records no outcome
`BulkheadFullError`	its `CircuitBreaker` records no outcome

Nothing that happens while the stack is admitting a call counts against its breaker either. A key= function that raises, a cost above the limiter's capacity, and a limiter backend that is down are all refused without recording a dependency outcome, so a mistake in one tenant's key cannot open the circuit for every caller.

Everything else reaches every pattern exactly as that pattern's own when= decides. The stack changes no configuration.

The refusal itself is unchanged. A Fallback at the top still catches the CircuitBreakerError, and a caller still reads retry_after off the RateLimitExceededError.

This is the one place a Stack is more than the hand-written layers below. Stacking the same patterns by hand gives you the raw composition, where a broad when= retries through an open breaker and a quota refusal counts as a dependency failure.

Budgets add up

Each layer holds its own budget, and one attempt can spend all of them in turn. The worst case for a single attempt is the sum:

rate limiter max_wait + bulkhead max_wait + timeout seconds

The Timeout is innermost, so it bounds the call and nothing above it. A Bulkhead(max_wait=30.0) queues for a permit outside that deadline, and a limiter with a wait budget queues outside both. To bound one attempt end to end, budget the parts. To bound a whole logical call including its retries, put a deadline above the stack:

fragment

async with asyncio.timeout(5.0):
    await get_recommendations(user_id)

Both wait budgets default to no waiting at all, so a stack refuses rather than queues until you ask otherwise.

A retry spends the budgets again. Every attempt re-enters the limiter and takes a fresh token, including an attempt the bulkhead then refuses, so a broad Retry(when=Exception) above a saturated bulkhead drains the quota on calls that never leave the process. This is the case the narrow when= advice exists for: the usual when=httpx.HTTPError never matches a BulkheadFullError, so the default is already right.

The imperative form

For a call site that cannot be decorated, run applies the same stack:

import asyncio

import httpx

from grelmicro.resilience import Retry, Stack, Timeout

NAME = "recs"

recs = Stack(
    NAME,
    patterns=[
        Retry.exponential(NAME, when=httpx.HTTPError, attempts=3),
        Timeout(NAME, seconds=1.0),
    ],
)


def recs_api(request: httpx.Request) -> httpx.Response:
    """Stand in for the recommendations API, so the example runs offline."""
    return httpx.Response(200, json={"items": []})


async def main() -> None:
    transport = httpx.MockTransport(recs_api)
    async with httpx.AsyncClient(
        base_url="https://recs.example", transport=transport
    ) as client:
        response = await recs.run(client.get, "/recs/42")
        print(response.status_code)


if __name__ == "__main__":
    asyncio.run(main())

recs.run(fn, *args, **kwargs) calls fn(*args, **kwargs) under the same patterns, so the same breaker, bucket, and permits are shared with the decorated functions.

The stack builds its chain once, when you build the stack, so run costs about a microsecond more than a decorated call, which is what checking the target it was handed costs.

Sync functions

Fallback, Retry, and CircuitBreaker wrap def functions as well as async def. RateLimiter, Bulkhead, and Timeout are async only, so a stack that holds one of them refuses a sync function where it is written:

Stack 'recs' only decorates async functions, because Timeout does.
Make refresh_prices async, or drop that pattern from the stack.

A sync stack that holds a CircuitBreaker runs from a worker thread. The breaker keeps its state on the backend and reaches it through the event loop the backend runs on, so calling it from that loop would wait on the loop that has to do the work. That is refused with a message rather than left to hang, and the refusal travels the way a cancellation does, past every Retry and Fallback, so a wiring mistake is never stood in for by a default. Reach it with asyncio.to_thread(...), the same way CircuitBreaker is reached from sync code on its own.

An object whose __call__ is async def counts as an async function everywhere a Stack looks, in the decorator and in run.

Building the list conditionally

A None entry is skipped, the way it is in Grelmicro(uses=[...]), so a pattern that applies to one deployment stays a plain expression: patterns=[retrier, breaker if shared else None].

For a list built beforehand, annotate it with Pattern:

import httpx

from grelmicro.resilience import CircuitBreaker, Pattern, Retry, Stack, Timeout

NAME = "recs"

shared_backend = False

patterns: list[Pattern] = [
    Retry.exponential(NAME, when=httpx.HTTPError, attempts=3),
    Timeout(NAME, seconds=1.0),
]
if shared_backend:
    patterns.append(CircuitBreaker(NAME))

recs = Stack(NAME, patterns=patterns)

What goes in a Stack

A Stack composes the six patterns that wrap a call. Anything else is refused at construction, with where it actually belongs:

Reached for	Where it goes
`Shield`	On its own. It is already a stack, with its own timeout, retries, and adaptive throttling.
`@cached`	Above the Stack, so a hit answers without entering it.
A registering decorator (`@tasks.every`, `@tasks.cron`, `@health.check`, `@outbox.handler`)	Above the Stack. It records the function it is handed and returns that same one, so a Stack below it would wrap only direct calls and leave every registered run unprotected. Written the wrong way round, every grelmicro decorator refuses.
`LeaderElection`	Nowhere in the order. It runs as a service. Gate the work on `leader.is_leader`.
`Lock`, `ReadWriteLock`	Inside the function. Both are keyed and held around a block.
A generator function	Around the code that consumes it, or around the call inside its body. A generator runs its body while it is iterated, so a Stack would wrap building it and nothing else. Refused.

Read outside-in, a fully wired call site looks like this:

@tasks.every(interval=60)
@cached(prices, ttl=30)
@recs
async def refresh_prices() -> list[Price]: ...

Stacking by hand

The order is enforced by Stack, not by the decorators. Stack them yourself when you want a different one:

import httpx

from grelmicro.resilience import CircuitBreaker, Retry, Timeout, fallback

breaker = CircuitBreaker("recs")
retrier = Retry.exponential("recs", when=httpx.HTTPError, attempts=3)
call_timeout = Timeout("recs", seconds=1.0)


@fallback(when=Exception, default=[])
@retrier
@breaker
@call_timeout
async def get_recommendations(
    client: httpx.AsyncClient, user_id: str
) -> list[str]:
    response = await client.get(f"/recs/{user_id}")
    response.raise_for_status()
    return response.json()

Read top to bottom: the call enters Fallback, which delegates to Retry, which delegates to CircuitBreaker, which delegates to Timeout, which finally runs the function.

Hand-stacked layers get none of the guarantees above, so the filters have to carry them:

Keep when= on the retry narrow. A broad Retry(when=Exception) retries a CircuitBreakerError, so an open breaker spends every attempt and every backoff before failing.
Keep the breaker's when= clear of admission errors, for example when=Match.not_exception(BulkheadFullError, RateLimitExceededError). Without it a full bulkhead or a spent quota counts as a dependency failure and helps open the circuit.

Picking `when=`

Every pattern uses the same when= keyword for its outcome filter, fed by the Match DSL:

retrier = Retry.exponential("recs", when=httpx.HTTPError, attempts=3)

A broad fallback(when=Exception, default=...) swallows every error inside the layers below. That is usually what you want at a graceful-degradation boundary. Pair it with a narrower retry so transient errors get a second chance before the fallback fires.