Circuit Breaker

Ithil’s circuit breaker protects your downstream API from cascading failures. When a downstream service starts returning errors, the gateway isolates it automatically rather than allowing a flood of failing requests.

States


          failure threshold reached
Closed ────────────────────────────► Open
  ▲                                    │
  │                                    │ cooldown period
  │                                    ▼
  └──────────────────────────── Half-Open
         probe succeeds                │
                                       │ probe fails
                                       ▼
                                     Open (reset cooldown)

State	Behavior
Closed	Normal operation. Requests flow through to the downstream API. Failures are counted.
Open	Downstream is isolated. The gateway returns `503` immediately for all tool calls without contacting the downstream API.
Half-Open	After the cooldown period, one probe request is allowed through. If it succeeds, the circuit closes. If it fails, the circuit re-opens and the cooldown resets.

What triggers the circuit opening

The circuit opens when the downstream API reaches the consecutive failure threshold (default: 5 failures within a rolling window). A failure is any response with a 5xx status code or a network timeout.

What happens when open

While the circuit is open:

The gateway returns 503 Service Unavailable for every tool call targeting the isolated cluster
The agent receives a JSON-RPC error with code -32003
No requests reach the downstream API
Budget is not consumed (the call was not executed)

Half-open probing

After the cooldown period (default: 30 seconds), the circuit transitions to Half-Open. The gateway allows exactly one request through:

Success → circuit transitions to Closed, normal operation resumes
Failure → circuit transitions back to Open, cooldown resets

Dashboard view

The current circuit state per cluster is visible in the Ithil Dashboard under the Circuit tab. The dashboard shows:

Current state (Closed / Open / Half-Open)
Time since last state change
Failure count in the current window

Configuration

Key	Type	Default	Description
`Ithil:CircuitBreaker:FailureThreshold`	int	`5`	Number of consecutive failures required to open the circuit.
`Ithil:CircuitBreaker:CooldownSeconds`	int	`30`	Seconds to wait in Open state before transitioning to Half-Open.
`Ithil:CircuitBreaker:TimeoutSeconds`	int	`10`	Request timeout in seconds. Timeouts count as failures.