AvatarLayer — pluggable SDK for realtime conversational avatars

The AvatarControl schema defines the full state of an avatar and its scene. It is based on the avatar-runtime v0.2 contract and is used for direct renderer control via session.updateControl() or renderer.update().

Overview

interface AvatarControl {
  avatar: {
    face: AvatarFace;
    emotion: AvatarEmotion;
    body: AvatarBody;
  };
  scene: AvatarScene;
}

All fields in update() are partial — you only need to send the fields you want to change.

Face

Pose

Head orientation in degrees:

interface FacePose {
  yaw: number;    // left/right rotation
  pitch: number;  // up/down rotation
  roll: number;   // tilt
}

Eyes

interface FaceEyes {
  blinkL: number;   // 0 (open) to 1 (closed)
  blinkR: number;
  gazeX: number;    // horizontal gaze direction
  gazeY: number;    // vertical gaze direction
}

Brows

interface FaceBrows {
  browInner: number;   // inner brow raise
  browOuterL: number;  // left outer brow
  browOuterR: number;  // right outer brow
}

Mouth

interface FaceMouth {
  jawOpen: number;      // 0 (closed) to 1 (fully open)
  smile: number;        // smile intensity
  mouthPucker: number;  // pucker/kiss shape
}

Emotion

Emotional state of the avatar, used to drive expression presets:

interface AvatarEmotion {
  valence: number;    // -1 (negative) to 1 (positive)
  arousal: number;    // -1 (calm) to 1 (excited)
  label: string;      // "happy", "sad", "angry", "surprised", "relaxed", "neutral"
  intensity: number;  // 0 to 1
}

The VRM renderer maps label to VRM expression presets:

Label	VRM Preset
`"happy"`	Happy
`"sad"`	Sad
`"angry"`	Angry
`"surprised"`	Surprised
`"relaxed"`	Relaxed
`"neutral"`	Neutral

Body

interface AvatarBody {
  preset: string;
  skeleton?: Record<string, Vec3>;
  ik?: Record<string, { position: Vec3; weight: number }>;
}

Field	Description
`preset`	Named body pose preset
`skeleton`	Direct bone rotations as `{ boneName: { x, y, z } }`
`ik`	Inverse kinematics targets with position and weight

Scene

Camera

interface SceneCamera {
  position?: Vec3;  // Camera position in world space
  target?: Vec3;    // Look-at target
  fov?: number;     // Field of view in degrees
}

World

interface SceneWorld {
  background?: string;      // CSS color or gradient
  ambientLight?: number;    // Ambient light intensity
  keyLight?: {
    intensity: number;
    direction?: Vec3;
  };
}

Usage example

session.updateControl({
  avatar: {
    face: {
      pose: { yaw: 5, pitch: -2, roll: 0 },
      eyes: { blinkL: 0, blinkR: 0, gazeX: 0.1, gazeY: 0 },
      brows: { browInner: 0.3, browOuterL: 0, browOuterR: 0 },
      mouth: { jawOpen: 0, smile: 0.6, mouthPucker: 0 },
    },
    emotion: {
      valence: 0.8,
      arousal: 0.4,
      label: "happy",
      intensity: 0.7,
    },
    body: { preset: "idle" },
  },
  scene: {
    camera: {
      position: { x: 0, y: 1.35, z: 1.5 },
      target: { x: 0, y: 1.2, z: 0 },
      fov: 30,
    },
    world: {
      ambientLight: 0.6,
    },
  },
});

Remote renderers (LemonSlice, Atlas, HeyGen) typically ignore update() calls since the avatar is controlled server-side. The control schema is most useful with local renderers like VRM.

Avatar Control