How to Build Agentic Edge AI

This is the practical build path. For the definition of the term and how it maps to industrial deployments, see the canonical pillar on edge agents; for the architecture in depth, see build an AI agent for embedded systems.

What You Are Building

Agentic edge AI is a sense-reason-act loop running on the device. Most embedded-ML tutorials stop at “model runs, prints a result.” An agent continues: it decides, acts on the physical world, holds state across cycles, and feeds outcomes back into sensing. The steps below take you from a validated model to a deployable agent on an ESP32 or STM32, and then to a fleet.

Before you start, get the boundaries right:

• The MCU does the fast, deterministic, local work. A language model does not run on it.
• Memory is the binding constraint. Budget it before adding models.
• Keep network calls off the decision path; escalation is asynchronous.

Step 1 — Define the Loop and Pick the Hardware

Write down the four corners of the loop before any code: what it senses, what decision it must make, what action it takes, and what state it must remember. That single paragraph determines the hardware.

• Vibration/predictive maintenance maps well to an ESP32 with Edge Impulse or, for heavier DSP, an STM32H7.
• Vision (object detection, people counting) needs PSRAM and a camera — an ESP32-S3 running object detection.
• Motion/gesture on a low-power node fits the Arduino Nano 33 BLE.
• Anomaly detection on a constrained MCU runs on an STM32F4.

Pick the cheapest part whose SRAM and clock comfortably hold your model plus the loop. Use the MCU compatibility checker to sanity-check the fit.

Step 2 — Train and Validate the Model Offline

Build the model before the firmware. Collect representative data, train (Edge Impulse or TensorFlow), and quantize to int8 for the MCU. Validate accuracy on a held-out set on a PC first — debugging a bad model inside firmware is far harder than fixing it in a notebook.

Convert to the target runtime: LiteRT for Microcontrollers (formerly TensorFlow Lite Micro) or CMSIS-NN, with ST X-CUBE-AI for STM32. Record the model’s RAM footprint (arena size) and per-inference latency now — both feed the next steps.

Step 3 — Build the Sensing Task

Implement acquisition as a fixed-rate task that writes calibrated readings into a ring buffer. Do not couple sampling to inference timing — they have different rates.

void sensor_task(void *arg) {
    const TickType_t period = pdMS_TO_TICKS(10);   // 100 Hz
    TickType_t last = xTaskGetTickCount();
    for (;;) {
        sensor_reading_t s = read_calibrated_sensors();
        ring_push(&g_ring, &s);
        vTaskDelayUntil(&last, period);
    }
}

For multi-sensor agents, timestamp every reading so the reasoning step operates on aligned data.

Step 4 — Run Inference On-Device

Statically allocate the tensor arena (never allocate it in the loop) and run the model on a window pulled from the ring buffer:

static uint8_t tensor_arena[ARENA_SIZE];   // sized in Step 2

float run_inference(const float *window, int n) {
    fill_input_tensor(window, n);
    if (interpreter->Invoke() != kTfLiteOk) return NAN;
    return read_output_score();
}

Confirm the on-device latency matches your Step 2 estimate. If it is too slow, shrink the model or move to a faster part — do not paper over it with a longer loop period that breaks the control requirement.

Step 5 — Add Decision and State Logic

This is the step that turns inference into agency. Combine the model output with state (history) and a state machine, not a bare threshold:

agent_decision_t decide(float score) {
    push_history(score);
    float trend = compute_trend();

    switch (g_state) {
    case MONITORING:
        if (score > 0.7f) g_state = ALERT;
        break;
    case ALERT:
        if (score > 0.9f || trend > RISING_2H) g_state = CRITICAL;
        else if (score < 0.5f) g_state = MONITORING;
        break;
    case CRITICAL:
        if (ack_received()) g_state = MONITORING;
        break;
    }
    return decision_for_state(g_state);
}

The agent now escalates on trend, remembers what it has seen, and behaves differently in each state.

Step 6 — Drive Actuation

Map each decision to a physical action, and let state change the agent’s own behavior (adaptive sampling, model cascade):

State	Sample rate	Action
MONITORING	100 Hz	Periodic status via MQTT
ALERT	500 Hz	Alert via MQTT, warning LED
CRITICAL	1 kHz	Alarm relay, continuous MQTT, raw-data log

Keep actuation deterministic and local. Anything that needs the network (an alert, a log upload) goes through the comms task so it never blocks the decision.

Step 7 — Orchestrate State, Networking, and Escalation

A single device is an agent; a deployment is a system. The orchestration layer handles state persistence across reboots, network reconnection and buffering, escalation to a gateway or cloud reasoning step, and coordination between devices. This is where hand-rolled firmware accumulates the most accidental complexity.

For decisions that genuinely need a larger model or an LLM, escalate off the real-time path: the edge agent makes the fast local decision, and asynchronously asks a server for the harder judgment, then acts on the response. The trade-offs of where each step runs are covered in edge agents vs cloud agents.

Step 8 — Deploy and Update the Fleet

Ship with OTA so you can push model and logic updates without touching hardware. Version models and decision logic independently, validate each update on a staging device, and roll out gradually. Centralize device health and event logs so you can see what every agent is deciding — local-only logs do not scale past a handful of devices.

Tools Landscape

Layer	Common options
Model training/conversion	Edge Impulse, TensorFlow + int8 quantization
On-device runtime	LiteRT for Microcontrollers, CMSIS-NN
Vendor optimization	ST X-CUBE-AI (STM32), ESP-DL (ESP32)
Scheduling	FreeRTOS (ESP-IDF), Zephyr, bare super-loop
Connectivity	MQTT (Wi-Fi), Modbus RTU, UART, CAN
Orchestration	ForestHub (edge AI agents orchestration platform)

How ForestHub Fits

Steps 5 through 8 — decision/state logic, orchestration, escalation, and fleet deployment — are where most of the custom firmware lives, and where projects stall after the model works.

ForestHub is the edge AI agents orchestration platform that targets that layer. It runs on your Linux edge gateway, above the devices: you wire the loop as a graph — sensor and inference results coming in over MQTT, Modbus, and OPC-UA, decision and state nodes, an LLM as one reasoning node among many, escalation and actuation back out over the same industrial protocols. ForestHub orchestrates that graph deterministically and keeps it inspectable, replayable, and auditable: state handling, escalation to cloud or on-premise reasoning, and versioned management across the fleet. The device keeps owning sensing and actuation; the platform owns the orchestration between a working model and a deployable, manageable agent. For the conceptual grounding, start with edge AI agents on microcontrollers.