Skip to content

Bulkhead

A Bulkhead caps how many calls run at once. Rate limiting bounds requests per unit time. A bulkhead bounds concurrent in-flight work, so one slow dependency cannot consume every worker and starve the rest of the app.

Why

  • Bound concurrent in-flight business operations.
  • Fail fast when saturated instead of queueing unboundedly.
  • Keep blocking work on a dedicated thread pool, off the event loop and off the shared executor.

Usage

Bulkhead works as an async context manager and as a decorator on async functions. When the limit is reached, a caller waits up to max_wait seconds for a permit, then is rejected with BulkheadFullError.

from grelmicro.resilience import Bulkhead, BulkheadFullError


async def process(order_id: str) -> str:
    return order_id


# Bound concurrent checkouts to 50. A caller waits up to 2s for a free
# permit, then is rejected.
checkout = Bulkhead("checkout", max_concurrent=50, max_wait=2.0)


async def handle(order_id: str) -> str:
    try:
        async with checkout:
            return await process(order_id)
    except BulkheadFullError:
        return "busy"

The default fails fast: with no max_wait, a full bulkhead rejects immediately. Set max_wait to let callers queue briefly for a permit.

The limit counts the calls through one Bulkhead object in one process. Build it once and share it: two objects named payments each admit max_concurrent calls, and so does every process, so three replicas running four workers each with max_concurrent=10 run up to 120 calls at once. To run one call at a time across replicas, use a Lock.

Bounded blocking work

to_thread runs a blocking function on the bulkhead's own thread pool when max_workers is set, otherwise on the event loop's shared executor.

import time

from grelmicro.resilience import Bulkhead

# A dedicated 4-thread pool keeps blocking work off the event loop and
# off the default executor shared by the rest of the app.
reports = Bulkhead("reports", max_workers=4)


def render_pdf(report_id: str) -> bytes:
    time.sleep(0.1)  # blocking work
    return report_id.encode()


@reports
async def build_report(report_id: str) -> bytes:
    return await reports.to_thread(render_pdf, report_id)

Failure-domain isolation

uses= scopes Providers and Components to the bulkhead, in the same shape as Grelmicro(uses=[...]). Inside the scope, a Pattern that resolves its default backend (a bare Lock("k"), a cache.get(...), ...) picks up the bulkhead's Component instead of the app's. A Pattern with an explicit backend= is unaffected, so explicit choices always win. This isolates a business context (checkout, reporting) onto its own connection pool, so one context cannot exhaust another's.

from grelmicro.coordination import Lock
from grelmicro.providers.redis import RedisProvider
from grelmicro.resilience import Bulkhead

# A dedicated Redis for checkout, isolated from the app's default
# backends. The bulkhead opens it on first entry, scopes every kind it
# serves, and closes it at app shutdown.
checkout_redis = RedisProvider("redis://localhost:6379/1")
checkout = Bulkhead(
    "checkout",
    max_concurrent=50,
    uses=[checkout_redis],
)

cart_lock = Lock("cart")


async def handle_checkout(cart_id: str) -> None:
    # `cart_lock` has no explicit backend, so inside the scope it
    # resolves to the bulkhead's dedicated lock backend.
    async with checkout, cart_lock:
        ...

The bulkhead opens its uses= on first entry and closes them when the app shuts down, so an active Grelmicro app is required. Order inside uses= does not matter: a Provider listed after the Component that borrows it is moved ahead of it. The scope never adopts a Provider you left out, where Grelmicro(uses=[...]) would, because it opens on the app's exit stack and would close a Provider the app still holds. The exit stack is the list of things the app closes at shutdown, in reverse order.

The scope belongs to one app run: whichever enters the bulkhead first. A later run in the same process opens every item again from the start. A run that overlaps the owner borrows the open items instead of opening a second set, and gives them up when the owner closes them, so give overlapping apps their own Bulkhead when their lifetimes differ.

Something has to open the Provider a Component borrows: list it here, list it on the app, or open it yourself. A Component that opens ahead of the Provider it borrows says which one to move. A scope that opens during startup sits below whatever the app enters after it. So a Provider the app opens later closes first, and a Component in the scope that borrows it closes against a dead client. List that Provider ahead of whatever enters the bulkhead.

Listing an item in both places is fine: whichever opens it first owns it. That holds within one app run, for the same object, a Provider, or a bare backend. It does not reach across runs: a second run that lists an item, in uses= or on the app itself, opens it again while the first still holds it open. Two Components you build separately around one backend are two items too, so share the Component or list the backend bare. A Component in the list claims its whole kind, so a bare Provider beside it fills only the other kinds it serves, exactly as on the app.

micro.fake() and micro.override(...) swap the app's Components, and a scope answers before the app does, so neither reaches inside uses=. Build the bulkhead with the backends the test wants when a test needs the scope faked. A Provider that only the scope's Components borrow is left closed by fake(), which cannot see them, so pass it as micro.fake(keep=[provider]) when the scope should run on the real one.

These Components are not registered on the app, so the backend check cannot run at startup for them. It runs on first entry instead, with the same rules.

Configuration

Configure Bulkhead with keyword arguments, GREL_BULKHEAD_* environment variables, or a mounted file, as every component is. See Configuration.

Environmental

Prefix: GREL_BULKHEAD_{NAME_UPPER}_. The default instance drops the name segment and reads GREL_BULKHEAD_*.

Environment variables are read by default

A GREL_* variable that fills a component field is read unless GREL_ENV_LOAD=0. With GREL_ENV_LOAD=0, a variable that is set is ignored, the default applies, and startup warns with its name. Passing env_load=False to a component turns its reads off and stays quiet. See How a value is resolved for the three ways to configure a component, including a local .env.

Env var Field Type Default
GREL_BULKHEAD_{NAME_UPPER}_MAX_CONCURRENT max_concurrent PositiveInt unbounded
GREL_BULKHEAD_{NAME_UPPER}_MAX_WAIT max_wait NonNegativeFloat fail fast
GREL_BULKHEAD_{NAME_UPPER}_MAX_WORKERS max_workers PositiveInt shared executor

Composition

The recommended outside-in order is Fallback → Retry → CircuitBreaker → RateLimiter → Bulkhead → Timeout → call. Read more in Composing patterns. Placing the bulkhead above the timeout caps concurrency before a call enters its timeout window.

Live reconfiguration

Bulkhead inherits Reconfigurable[BulkheadConfig]. Calling bulkhead.reconfigure(new_config) applies a new max_concurrent to calls admitted after the swap. Calls already inside keep their permit. Changing max_workers rebuilds the private thread pool. See Live reconfiguration.

Reference

See the API reference for every option.