AvatarLayer — pluggable SDK for realtime conversational avatars

The VRMLocalRenderer renders a 3D VRM model in the browser using Three.js and @pixiv/three-vrm. It includes automatic blink animation, expression presets, and lip-sync (RMS-based or viseme-based).

Installation

npm install three @pixiv/three-vrm

Usage

import { VRMLocalRenderer } from "avatarlayer";

const renderer = new VRMLocalRenderer({
  modelUrl: "/models/avatar.vrm",
  idleAnimationUrl: "/animations/idle.fbx",  // optional
  visemeLipSync: true,                        // optional
});

Constructor options

Option	Type	Default	Description
`modelUrl`	`string`	required	URL to the .vrm model file
`idleAnimationUrl`	`string`	—	Optional URL to an idle animation (FBX/GLB)
`visemeLipSync`	`boolean`	—	Enable viseme-based lip-sync instead of RMS amplitude mapping

How it works

When mounted, the renderer:

Creates a Three.js scene with camera, lighting, and WebGL renderer
Loads the VRM model via GLTFLoader with the VRM plugin
Starts an automatic blink loop (random interval, 3-6 seconds)
Optionally loads an idle animation from the provided URL
Begins a render loop at screen refresh rate

When speak(audio) is called:

The audio blob is played through an <audio> element
A LipSyncEngine analyzes the audio in realtime via AnalyserNode
Audio is mapped to the VRM mouth-open expression — either via RMS amplitude (Aa preset) or viseme weights when visemeLipSync is enabled
The promise resolves when the audio ends

Avatar control

The VRM renderer responds to update() calls for fine-grained control:

session.updateControl({
  avatar: {
    face: {
      mouth: { jawOpen: 0.5, smile: 0, mouthPucker: 0 },
      eyes: { blinkL: 0, blinkR: 0, gazeX: 0, gazeY: 0 },
    },
    emotion: {
      label: "happy",
      intensity: 0.8,
      valence: 0.7,
      arousal: 0.5,
    },
  },
});

Supported expression presets: happy, sad, angry, surprised, relaxed, neutral.

Resize handling

The renderer automatically responds to container resize via ResizeObserver, updating the camera aspect ratio and renderer dimensions.

VRM (Local)