7
Architecture
Bulkheads vs a shared connection pool after a slow dependency took down checkout
Failure isolation ↔ Resource utilization
Context
Stop one slow downstream service from exhausting threads for every other call.
Last month a slow reporting API held every thread in our shared 200-thread HTTP pool, and checkout was down for 25 minutes, although checkout never called reporting directly.
Option A: bulkheads, a separate, bounded pool (or concurrency limit) per downstream dependency, so one slow service can only exhaust its own share.
Option B: keep a shared pool, but add timeouts and raise the size.
Isolation wastes some capacity. Is it worth it?
Constraints
- Today
- One 200-thread pool for 6 downstream services
- Incident
- Slow reporting API blocked checkout for 25 min
- Runtime
- Java 21
Bulkheads (separate pool per dependency)vsShared resource pool
Community verdict
Bulkheads (separate pool per dependency) 80%Shared resource pool 20%
10 engineers · 5 opinions
With these constraints, what would you choose?
One choice per engineer. You can change it any time.
Bulkheads (separate pool per dependency)
Shared resource pool
Resource utilisation →
Trade-offs
Dimensions
Bulkheads (separate pool per dependency)Shared resource poolout of 5
- Failure isolation
- 5Bulkheads (separate pool per dependency) scores 5 of 51Shared resource pool scores 1 of 5
- Resource utilisation
- 3Bulkheads (separate pool per dependency) scores 3 of 55Shared resource pool scores 5 of 5
- Tuning effort
Community discussion
5 comments
Have you made this decision in production? Share your reasoning.
Sign in to commentBulkheads, sized by criticality: checkout and payment dependencies get reserved capacity; reporting gets a small pool and is allowed to fail.
Timeouts alone don't save you. With a 10 s timeout and enough slow calls, the shared pool still fills.
Exactly what happened: our timeout was 30 s. It just delayed the collapse.
Six separately tuned pools is six numbers someone has to keep right. Start with two: critical and best-effort.
On Java 21, virtual threads make threads cheap, but you still need a per-dependency semaphore, or a slow service exhausts its connections and memory instead of threads.
Concurrency limits per dependency is the portable version of a bulkhead. Same idea in Go with a buffered channel.