Embodied systems · Explore this field ↗ · Architecture · 2 min read
The embodied-agent stack
Build the interaction loop before polishing the face.
See the source / a related case
Seven coordinated layers
The stack contains character and rig; input and speech recognition; dialogue and retrieval; memory and persona; behavior planning; animation and rendering; and operations. Lip-sync is only one bridge. A behavior layer should convert conversational intent into timed expression, gaze, gesture, and actions while respecting the current animation state. The dialogue layer should know what the body can actually perform.
Time is the integration surface
The system must align end-of-speech detection, model response, synthesis, visemes, facial expression, gaze, gesture onset, and world action. An emotionally upbeat gesture arriving after a refusal feels broken even when every component works independently. Record timestamps across the path and evaluate synchronization, interruption, and recovery.
Operator note
Prototype with one character, one room, one task, and one gesture vocabulary. Make text fallback available. Instrument every stage before increasing visual fidelity. Compare the embodied version against text or voice-only service on task outcome, not just preference.
Prototype the seams
Build an instrumentation scene before a showcase scene. Display partial speech recognition, dialogue state, selected behavior intent, synthesis timing, viseme queue, gesture state, and tool status next to the character. Interrupt at every stage and verify cancellation. Simulate a slow model, missing animation, and tool refusal. This lab view makes mismatches observable before artistic polish hides them. Once the loop is stable, the debug surface can remain available to developers and evaluators while the public experience becomes visually quiet.
Primary reading
Sources and limits
These links support the architecture, policy, or product behavior discussed above. Vendor documentation describes vendor features; it is not independent proof of performance. Current details should be rechecked before a production decision.