Atlas

AtlasRenderer connects to North Model Labs Atlas for realtime lip-synced video avatars. Atlas manages its own LiveKit room — you only need an Atlas API key (no separate LiveKit credentials).

Two modes are supported:

• Passthrough — you provide TTS audio via speak(blob), Atlas renders lip-synced video. Use your own LLM + TTS pipeline.
• Conversation — Atlas handles STT, LLM, TTS, and avatar rendering end-to-end. Use speakText(text) for text input and autoEnableMic for voice input.

Installation

npm install livekit-client

Usage — passthrough mode

import { AtlasRenderer } from "avatarlayer/renderers";

const renderer = new AtlasRenderer({
  createSession: async () => {
    const resp = await fetch("/api/atlas", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        atlasApiKey: "...",
        faceUrl: "https://...",  // optional face image
        mode: "passthrough",
      }),
    });
    const data = await resp.json();
    return {
      sessionId: data.session_id,
      livekitUrl: data.livekit_url,
      token: data.token,
    };
  },
  deleteSession: async (sessionId) => {
    await fetch(`/api/atlas/${sessionId}`, {
      method: "DELETE",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ atlasApiKey: "..." }),
    });
  },
});

Usage — conversation mode

import { AtlasRenderer } from "avatarlayer/renderers";

const renderer = new AtlasRenderer({
  createSession: async () => {
    const resp = await fetch("/api/atlas", {
      method: "POST",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({
        atlasApiKey: "...",
        faceUrl: "https://...",
        mode: "conversation",
      }),
    });
    const data = await resp.json();
    return {
      sessionId: data.session_id,
      livekitUrl: data.livekit_url,
      token: data.token,
    };
  },
  deleteSession: async (sessionId) => {
    await fetch(`/api/atlas/${sessionId}`, {
      method: "DELETE",
      headers: { "Content-Type": "application/json" },
      body: JSON.stringify({ atlasApiKey: "..." }),
    });
  },
  autoEnableMic: true,
  onTranscript: (msg) => console.log(`[${msg.role}] ${msg.text}`),
});

Constructor options

AtlasSessionInfo

Your createSession callback must return this shape:

interface AtlasSessionInfo {
  sessionId: string;
  livekitUrl: string;
  token: string;
}

Your backend proxy calls POST /v1/realtime/session on the Atlas API and maps the response (session_id → sessionId, livekit_url → livekitUrl).

AtlasTranscriptMessage

interface AtlasTranscriptMessage {
  id: string;
  role: "user" | "agent";
  text: string;
  final: boolean;
  timestamp: number;
}

How it works

Passthrough mode

Your TTS audio drives the avatar:

1. mount() calls createSession to get LiveKit credentials
2. Connects to the LiveKit room
3. On the first speak(audio) call, a persistent AudioContext, MediaStreamDestination, and LocalAudioTrack (named "tts-audio") are created and published to the room — one-time setup
4. Each subsequent speak() decodes the audio blob and plays it through the same persistent track, avoiding per-sentence WebRTC renegotiation
5. Atlas receives the audio and renders lip-synced video in realtime
6. interrupt() stops playback, unpublishes the track, and closes the AudioContext

If the mic was enabled, it is automatically muted when the persistent track is created and restored when the track is torn down.

Conversation mode

Atlas handles the full pipeline:

1. mount() calls createSession and connects to LiveKit
2. If autoEnableMic is true, the microphone is enabled
3. Atlas performs STT on the mic audio, runs LLM inference, generates TTS, and streams lip-synced video back
4. Transcript segments arrive via onTranscript
5. Use speakText(text) to send text to the agent via the LiveKit data channel

speakText vs sendChat

In conversation mode (autoEnableMic: true), the renderer exposes speakText on the instance. AvatarSession detects this and routes text directly to Atlas via the data channel instead of running external TTS → speak(blob).

In passthrough mode (autoEnableMic: false, the default), speakText is intentionally absent so AvatarSession uses the normal TTS → speak(blob) pipeline to publish audio to the room.

If you need to send data-channel messages in either mode (e.g. from outside AvatarSession), use sendChat:

const renderer = new AtlasRenderer({ ... });
renderer.sendChat("Hello!"); // always available

Session lifecycle

When unmount() is called, the renderer:

1. Cancels any in-flight TTS audio playback
2. Disconnects from the LiveKit room
3. Removes video elements from the container and audio elements from the document
4. Calls deleteSession if a session ID exists