AvatarLayer — pluggable SDK for realtime conversational avatars

LiveAvatarRenderer connects to LiveAvatar (formerly HeyGen Interactive Avatar) LITE Mode for real-time lip-synced video avatars. You provide TTS audio and LiveAvatar renders synchronized video, streamed back through a LiveKit room.

Migration from HeyGen

HeyGen's Streaming Avatar API was sunset on March 31, 2026. LiveAvatar LITE Mode is the direct replacement. The deprecated HeyGenRenderer class is still exported as an alias for backwards compatibility, but new integrations should use LiveAvatarRenderer.

Installation

npm install livekit-client

Usage

import { LiveAvatarRenderer } from "avatarlayer/renderers";

const renderer = new LiveAvatarRenderer({
  createSession: async () => {
    const resp = await fetch("/api/heygen", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        liveAvatarApiKey: "...",
        avatarId: "...",  // LiveAvatar avatar UUID
      }),
    });
    return resp.json();
  },
  stopSession: async (sessionId) => {
    await fetch(`/api/heygen/${sessionId}`, {
      method: "DELETE",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ liveAvatarApiKey: "..." }),
    });
  },
});

Constructor options

Option	Type	Description
`createSession`	`() => Promise<LiveAvatarSession>`	Required. Creates a LiveAvatar LITE session and returns LiveKit + WebSocket connection info.
`stopSession`	`(sessionId: string) => Promise<void>`	Recommended. Stops the session when done. Called automatically on `unmount()`.
`onVideoStream`	`(stream: MediaStream \| null) => void`	Called when video track is received/lost.
`onAudioStream`	`(stream: MediaStream \| null) => void`	Called when audio track is received/lost.
`onStateChange`	`(state: string, detail?: string) => void`	Called on connection state changes.

LiveAvatarSession

Your createSession callback must return this shape:

interface LiveAvatarSession {
  sessionId: string;
  livekitUrl: string;
  livekitToken: string;
  wsUrl: string;
}

Your backend calls POST /v1/sessions/token (with mode: "LITE") then POST /v1/sessions/start on the LiveAvatar API and combines the results.

How it works

mount() calls createSession to get LiveKit credentials and a WebSocket URL
Connects to the LiveKit room for video/audio streaming
Opens a WebSocket to the LiveAvatar control endpoint
Waits for session.state_updated with state "connected" before accepting commands
When speak(audio) is called, the audio blob is decoded to PCM 16-bit 24 kHz, split into ~1-second chunks, base64-encoded, and sent as agent.speak WebSocket messages
An agent.speak_end message is sent after the last chunk
LiveAvatar renders lip-synced video from the audio in real-time
interrupt() sends agent.interrupt via the WebSocket to clear the avatar's playback buffer
A keep-alive timer sends session.keep_alive every 2 minutes to prevent the 5-minute idle timeout

Gapless playback

Because audio chunks are pushed directly to LiveAvatar's buffer via WebSocket (without waiting for playback confirmation), multiple utterances are played back-to-back with no gaps. The SpeechQueue pre-buffers TTS blobs while earlier ones are still being sent, eliminating pauses between sentences.

Session configuration

LiveAvatar LITE Mode requires an external TTS provider — unlike the old HeyGen Streaming API, LiveAvatar does not handle TTS internally:

import { AvatarSession } from "avatarlayer";
import { OpenAIAdapter } from "avatarlayer/llm";
import { ElevenLabsAdapter } from "avatarlayer/tts";

const session = new AvatarSession({
  llm: new OpenAIAdapter({ apiKey: "..." }),
  tts: new ElevenLabsAdapter({ apiKey: "...", voiceId: "..." }),
  renderer: liveAvatarRenderer,
  systemPrompt: "You are a helpful assistant.",
});

Server-side setup

Your backend needs two endpoints:

Create session — calls LiveAvatar POST /v1/sessions/token with mode: "LITE" and your avatar_id, then POST /v1/sessions/start with the session token. Returns the combined connection info.
Stop session — calls LiveAvatar POST /v1/sessions/stop with the session ID.

You need a LiveAvatar API key from app.liveavatar.com/developers.

Credits

LITE Mode costs 1 credit per minute. See the LiveAvatar pricing docs for details.

LiveAvatar (HeyGen)