Skip to content

Bulkhead

A Bulkhead caps how many calls run at once. Rate limiting bounds requests per unit time. A bulkhead bounds concurrent in-flight work, so one slow dependency cannot consume every worker and starve the rest of the app.

Why

  • Bound concurrent in-flight business operations.
  • Fail fast when saturated instead of queueing unboundedly.
  • Keep blocking work on a dedicated thread pool, off the event loop and off the shared executor.

Usage

Bulkhead works as an async context manager and as a decorator on async functions. When the limit is reached, a caller waits up to max_wait seconds for a permit, then is rejected with BulkheadFullError.

from grelmicro.resilience import Bulkhead, BulkheadFullError


async def process(order_id: str) -> str:
    return order_id


# Bound concurrent checkouts to 50. A caller waits up to 2s for a free
# permit, then is rejected.
checkout = Bulkhead("checkout", max_concurrent=50, max_wait=2.0)


async def handle(order_id: str) -> str:
    try:
        async with checkout:
            return await process(order_id)
    except BulkheadFullError:
        return "busy"

The default fails fast: with no max_wait, a full bulkhead rejects immediately. Set max_wait to let callers queue briefly for a permit.

Bounded blocking work

to_thread runs a blocking function on the bulkhead's own thread pool when max_workers is set, otherwise on the event loop's shared executor.

import time

from grelmicro.resilience import Bulkhead

# A dedicated 4-thread pool keeps blocking work off the event loop and
# off the default executor shared by the rest of the app.
reports = Bulkhead("reports", max_workers=4)


def render_pdf(report_id: str) -> bytes:
    time.sleep(0.1)  # blocking work
    return report_id.encode()


@reports
async def build_report(report_id: str) -> bytes:
    return await reports.to_thread(render_pdf, report_id)

Failure-domain isolation

uses= scopes Providers and Components to the bulkhead, in the same shape as Grelmicro(uses=[...]). Inside the scope, a Pattern that resolves its default backend (a bare Lock("k"), a cache.get(...), ...) picks up the bulkhead's Component instead of the app's. A Pattern with an explicit backend= is unaffected, so explicit choices always win. This isolates a business context (checkout, reporting) onto its own connection pool, so one context cannot exhaust another's.

from grelmicro.coordination import Coordination, Lock
from grelmicro.providers.redis import RedisProvider
from grelmicro.resilience import Bulkhead

# A dedicated Redis for checkout, isolated from the app's default lock
# backend. The bulkhead opens it on first entry and closes it at app
# shutdown.
checkout_redis = RedisProvider("redis://localhost:6379/1")
checkout = Bulkhead(
    "checkout",
    max_concurrent=50,
    uses=[Coordination(checkout_redis)],
)

cart_lock = Lock("cart")


async def handle_checkout(cart_id: str) -> None:
    # `cart_lock` has no explicit backend, so inside the scope it
    # resolves to the bulkhead's dedicated lock backend.
    async with checkout, cart_lock:
        ...

The bulkhead opens its uses= on first entry and closes them when the app shuts down, so an active Grelmicro app is required. List the Provider before the Component that borrows it, exactly as in Grelmicro(uses=[...]).

These Components are not registered on the app, so the backend check cannot run at startup for them. It runs on first entry instead, with the same rules.

Configuration

Bulkhead follows the three-paths configuration contract.

Environmental

Prefix: GREL_BULKHEAD_{NAME_UPPER}_. The default instance drops the name segment and reads GREL_BULKHEAD_*.

Environment variables are opt-in

A GREL_* variable that fills a component field is read only when GREL_ENV_LOAD is truthy (1, true, yes, on), the flag itself and GREL_ENVIRONMENT excepted. Without it the variable is ignored and the default applies. Setting one while the flag is unset warns at startup, so the mistake is not silent. Passing env_load=False is a deliberate opt-out and stays quiet. See How a value is resolved for the three ways to configure a component, including a local .env.

Env var Field Type Default
GREL_BULKHEAD_{NAME_UPPER}_MAX_CONCURRENT max_concurrent PositiveInt unbounded
GREL_BULKHEAD_{NAME_UPPER}_MAX_WAIT max_wait NonNegativeFloat fail fast
GREL_BULKHEAD_{NAME_UPPER}_MAX_WORKERS max_workers PositiveInt shared executor

Composition

The recommended outside-in order is Fallback → Retry → CircuitBreaker → Bulkhead → Timeout → call. Read more in Composing patterns. Placing the bulkhead above the timeout caps concurrency before a call enters its timeout window.

Live reconfiguration

Bulkhead inherits Reconfigurable[BulkheadConfig]. Calling bulkhead.reconfigure(new_config) applies a new max_concurrent to calls admitted after the swap. Calls already inside keep their permit. Changing max_workers rebuilds the private thread pool. See Live reconfiguration.

Reference

See the API reference for every option.