Embodied systems · Explore this field ↗ · Governance · 2 min read
Embodied safety: disclosure, likeness, and dependency
The closer the system looks and sounds like a person, the more clearly it must show its boundaries.
See the source / a related case
Disclosure belongs in the experience
A footer is not enough when a character looks human. Identify the system as AI at the start and at meaningful transitions. Distinguish scripted, generated, and human-controlled behavior where confusion is plausible. Article 50 of the EU AI Act includes transparency duties for systems intended to interact directly with people, subject to its scope and exceptions; legal application requires qualified review.
Rights are a supply chain
Document rights for face, voice, motion capture, character design, training material, music, environments, and user-generated adaptations. Consent for one production does not automatically grant model training, cloning, resale, or perpetual interactive use. Build revocation and replacement into asset and model contracts.
Design against dependency
Companion-like systems should not claim consciousness, exclusivity, therapy, or human need. Avoid guilt, streak pressure, jealous language, and manipulative re-engagement. Provide memory controls, quiet periods, easy exit, and age-appropriate defaults. Health and crisis flows need qualified resources and tested escalation, not improvised empathy.
Operator note
Run red-team scenarios where the user mistakes the character for a person, requests impersonation, discloses self-harm, asks for diagnosis, or is a minor. Evaluate words, voice, expression, gesture, memory, and proactive messaging together—the harm can arise from their combination.
Review the whole performance
Safety review should play the same words through different voices, faces, gaze patterns, gestures, and relationship histories. A neutral sentence can become coercive when paired with pleading expression or a cloned familiar voice. Test proactive messages after absence, refusal reactions, memory references, and requests to keep secrets. Record who owns each expressive behavior and which layer can suppress it. A text moderation pass cannot see the combined social signal; embodied systems need scenario review of the complete timed performance.
Primary reading
Sources and limits
These links support the architecture, policy, or product behavior discussed above. Vendor documentation describes vendor features; it is not independent proof of performance. Current details should be rechecked before a production decision.