Patterns for agents that act safely

Safety

Verification-gated Actuation

The AI node proposes, a deterministic verifier disposes. The actuator is unreachable from the AI node. It can only fire via the verifier. The pattern works equally well with a classical ML model, an SLM, or a frontier LLM.

The model proposes a setpoint, a deterministic rule engine verifies it's within physical safety bounds, and only then does the heater fire. The actuator is wired to the verifier, not to the AI node, so the policy is part of the architecture, not bolted on afterwards.

Safest AI use

AI-as-Classifier-Only

The AI node only classifies into a finite set of branches. All action logic is deterministic. The safest possible AI use in a control loop. For the classification itself, a classical ML model or an SLM is often the better fit than a frontier LLM.

The node assigns the situation to one of {normal | warning | critical | unknown}. Every downstream action is deterministic code. For exactly this step, a classical ML model or an SLM running on your own hardware is often the better tradeoff than a frontier LLM: lower running cost, lower latency, and no data leaves the device. The contextual reasoning stays with the model, the safety stays with the graph.

Engineering

Replay Testing

Every model call is a node with structured input and output. Production runs are recorded and replayed in CI against a mocked model, whether the node is a classical ML model, an SLM, or an LLM.

Production runs are recorded as (input, model output) pairs at each node boundary. In CI the mocked model returns the recorded output and every downstream branch is exercised deterministically. Regressions are caught before they ship to a device, including the case where an AI node is later swapped from a cloud LLM to an on-device SLM.

Cost

Confidence-routed Model Cascade

A cheap, fast model handles the easy cases. On low confidence the next tier takes over, and the rare hard cases escalate to a human. The cascade can read classical ML → SLM → LLM, or, as Anthropic frames it, Haiku → Sonnet → Opus. Same logic, different substrate.

A classical ML model or an SLM running on your own hardware handles the easy classifications, a larger model picks up when confidence drops, and only the rare hard cases reach a frontier LLM or a human. In our experience, roughly an order of magnitude in cost reduction with quality held on the tail, and a second, often underrated effect: for the bulk of cases, the data never leaves your hardware, because the lower tiers answer on-device.

Operations

Shadow-Mode Deployment

The new agent runs in parallel with the existing deterministic logic. Its output goes to a log instead of the actuator. After N days of comparison, flip one wire to promote.

The trigger fans out to both the existing deterministic logic and the new agent, while the agent's output goes to a log instead of the actuator. After the comparison window, one wire is changed to flip the agent into production: no parallel daemons, no external comparator.

Graph-native

State-Machine-augmented Agent

Most transitions are deterministic FSM edges. An AI node is invoked only for ambiguous transitions, often an SLM that answers fast on the customer's hardware. The graph combines both in one runtime.

Most state transitions are deterministic edges, and an AI node only fires when the rules can't decide. Then the graph routes to the next state. The topology enforces the invariants, the model provides the contextual judgement. Because most transitions need no model at all, the AI node itself can often be an SLM running on the customer's hardware.