Edge AI Agents on Microcontrollers

This page covers the embedded, microcontroller-specific case. For the general definition of the term and how it maps to industrial deployments, see the canonical pillar on edge agents.

What “Agent” Means on a Microcontroller

In 2024-2025 the phrase “AI agent” usually means an LLM that calls tools, keeps memory, and pursues goals — ChatGPT with function calling, a coding assistant, a multi-step reasoning system. That definition does not survive contact with a microcontroller. An ESP32 has on the order of 520 KB of SRAM. An STM32F4 has tens to a few hundred kilobytes. Neither runs a language model.

On a microcontroller an edge AI agent is something narrower and more concrete:

An autonomous sense-reason-act loop that runs continuously on the device, deciding about the physical world from local inference, with no human in the loop and no network dependency.

The “intelligence” is not a foundation model. It is the combination of:

• Small specialized models — anomaly detection, classification, keyword spotting — quantized to int8 and run with LiteRT for Microcontrollers (formerly TensorFlow Lite Micro) or CMSIS-NN.
• Rule-based decision logic — thresholds, state machines, conditional execution that encodes engineering knowledge.
• Physical actions — GPIO toggling, relay switching, motor control, an MQTT alert.
• Feedback — the outcome of an action changes the next sensing cycle (adaptive sampling, escalation).

This is closer to control engineering and robotics than to chatbots. Calling it an “agent” is justified by the loop, not by the model.

The Sense-Reason-Act Loop

Every embedded agent is a variation on three stages plus feedback. Most embedded-ML projects implement only the first.

        ┌──────────────────────────────────────────┐
        │                                          │
        ▼                                          │
   ┌────────┐    ┌─────────┐    ┌────────┐    ┌────────┐
   │ SENSE  │ ─► │ REASON  │ ─► │  ACT   │ ─► │ STATE  │
   │ ADC,   │    │ model + │    │ GPIO,  │    │ history│
   │ I2C,   │    │ logic + │    │ relay, │    │ trend  │
   │ SPI    │    │ state   │    │ MQTT   │    │        │
   └────────┘    └─────────┘    └────────┘    └────────┘
                                                  │
                          feedback (sampling rate)│
        ◄─────────────────────────────────────────┘

Sense

The sensing task runs on a fixed schedule — typically 10 Hz to a few kHz depending on the modality — reading raw ADC, I2C, or SPI data, applying calibration, and writing into a shared ring buffer:

typedef struct {
    float    vibration_rms;
    float    temperature_c;
    float    current_a;
    uint32_t timestamp_ms;
} sensor_reading_t;

When an agent fuses multiple sensors, this layer also handles synchronization so the reasoning stage sees temporally aligned data.

Reason

The reasoning stage is where an agent differs from plain inference. It runs the model, then applies a policy that incorporates state and trend:

agent_decision_t agent_reason(sensor_reading_t *r, int n) {
    float spectrum[128];
    compute_fft(r, spectrum, n);

    float score = run_anomaly_model(spectrum);   // int8 LiteRT model
    push_history(score);                          // keep last N scores
    float trend = compute_trend();                // rising / falling

    if (score > CRITICAL_THRESHOLD)            return DECISION_CRITICAL;
    if (score > WARNING_THRESHOLD && trend > 0) return DECISION_WARNING;
    return DECISION_NORMAL;
}

The decision is not a bare threshold on a model output. It mixes the current score, the trend, and (in multi-sensor agents) cross-references between modalities.

Act

The action stage turns a decision into a physical effect and, optionally, changes the agent’s own behavior:

Decision	Action	Hardware path
NORMAL	Periodic status	MQTT publish over Wi-Fi
WARNING	Alert, raise sampling rate	MQTT + timer reconfiguration
CRITICAL	Local alarm, notify, log	GPIO relay + MQTT + flash log

Feedback closes the loop: raise the sampling rate when an anomaly is developing, drop back to a low duty cycle when conditions normalize. On a battery node like the STM32L4 running anomaly detection, adaptive sampling is the difference between weeks and months of runtime.

Architecture: Tasks, Arenas, and a State Machine

A single agent on an MCU is usually three or four RTOS tasks coordinated through queues:

1. Sensor task — fixed-rate acquisition into a ring buffer.
2. Inference task — pulls a window, runs the model into a statically allocated tensor arena, emits a score.
3. Policy task — runs the state machine, decides, dispatches actions.
4. Comms task — handles MQTT/Modbus/UART without blocking the control path.

On the ESP32, FreeRTOS (bundled with ESP-IDF) provides the task and queue primitives directly. On STM32, the same structure runs on FreeRTOS, Zephyr, or a bare super-loop with a timer-driven scheduler. ST’s X-CUBE-AI converts and optimizes the model for the target; LiteRT for Microcontrollers and CMSIS-NN are the common runtimes.

The state machine is the spine of the agent. A predictive-maintenance agent built on the ESP32 with Edge Impulse might look like:

MONITORING --(score > 0.7)----------------► ALERT
ALERT      --(score < 0.5 for 1 h)--------► MONITORING
ALERT      --(score > 0.9 OR trend > 2 h)-► CRITICAL
CRITICAL   --(ack via MQTT)---------------► MONITORING

Each state changes both the sampling rate and the actions taken — the behavior adapts to what the device observes rather than reacting identically every cycle.

Constraints That Define the Design

Memory is the binding constraint. Each model, tensor arena, sensor buffer, and history ring consumes SRAM that does not grow. A single int8 model with its arena is roughly 30-100 KB. An ESP32-S3 (512 KB SRAM plus external PSRAM) can host two or three small models as separate tasks; an ESP32 anomaly-detection node leaves modest headroom for application logic. Budget memory first, then design the agent to fit.

Compute is fixed and modest. A 240 MHz Xtensa or a 168 MHz Cortex-M4 runs a small model in single-digit to low-tens of milliseconds. That is fine for control loops at tens to hundreds of Hz. It is not enough for large vision transformers — those belong on a gateway or in the cloud.

Determinism matters. The reason to put the loop on the device is bounded, repeatable latency. Network calls reintroduce jitter. If an agent must call out, the call belongs off the hard real-time path (for example, escalation messaging), never inside the control decision.

On-device learning is limited. Full retraining needs backpropagation and optimizer state that exceed MCU memory. What is practical on-device: threshold adaptation from confirmed feedback, slow baseline-drift compensation, and incremental statistics for non-neural models. Real retraining happens off-device and ships as a firmware update.

When to Use an Edge AI Agent on an MCU

Reach for an on-device agent when most of these hold:

• The control decision needs deterministic sub-100 ms latency — vibration response, motor adjustment, safety interlock.
• The device must keep working offline — remote, industrial, or intermittently connected sites.
• Raw data cannot leave the device for privacy or bandwidth reasons. A 3.2 kHz accelerometer produces tens of megabytes per hour; on-device reasoning reduces that to kilobytes of events.
• You are deploying many low-cost nodes where per-inference cloud cost would dominate.

Do not force it on-device when the model exceeds MCU memory, the reasoning genuinely needs a language model, or you retrain weekly. Those cases want a gateway or cloud tier — often in a hybrid split where the MCU runs the fast local loop and a server handles heavier reasoning. The trade-offs are laid out in detail in edge agents vs cloud agents and edge AI vs cloud AI.

How ForestHub Fits

The patterns above are typically hand-written C firmware: custom RTOS tasks, hand-rolled state machines, manual MQTT plumbing. That is the work that stalls most teams between a working inference demo and a deployable agent.

ForestHub is the edge AI agents orchestration platform that targets exactly that gap. It runs on your Linux edge gateway, above the MCUs: you design the sense-reason-act loop as a graph — sensor and inference results arriving over MQTT, Modbus, and OPC-UA, decision and state nodes, an LLM as one reasoning node among many, escalation and actuation back out over the same protocols. ForestHub orchestrates that graph deterministically and keeps it inspectable, replayable, and auditable: it ingests device results, holds state across the fleet, adds reasoning where a step benefits from a language model rather than an on-device pattern matcher, and acts over industrial protocols. The device owns on-device sensing and actuation; the Linux-gateway platform owns the orchestration that turns models into agents.

If you are moving from a single inference loop toward a coordinated agent, the companion guide build an AI agent for embedded systems walks through the architecture in more depth, and how to build agentic edge AI gives the step-by-step build path.