AvatarLayer — pluggable SDK for realtime conversational avatars

Character cards define an avatar's personality, backstory, scenario, and behavior examples in a structured format. AvatarLayer supports the V3 character card specification and can load cards from PNG images or JSON.

Using a character card

Set characterCard in the session config. When present, it takes precedence over systemPrompt:

import {
  AvatarSession,
  OpenAIAdapter,
  ElevenLabsAdapter,
  VRMLocalRenderer,
  parseCharacterCard,
} from "avatarlayer";

const response = await fetch("/characters/assistant.json");
const card = parseCharacterCard(await response.text());

const session = new AvatarSession({
  llm: new OpenAIAdapter({ apiKey: "sk-..." }),
  tts: new ElevenLabsAdapter({ apiKey: "..." }),
  renderer: new VRMLocalRenderer({ modelUrl: "/models/avatar.vrm" }),
  characterCard: card,
});

Parsing character cards

From JSON

import { parseCharacterCard } from "avatarlayer";

const json = await fetch("/characters/my-char.json").then(r => r.text());
const card = parseCharacterCard(json);

From PNG (embedded metadata)

Character cards can be embedded in PNG image metadata (following the V3 spec). Pass the raw bytes:

import { parseCharacterCard } from "avatarlayer";

const buffer = await fetch("/characters/my-char.png").then(r => r.arrayBuffer());
const card = parseCharacterCard(new Uint8Array(buffer));

Updating at runtime

session.setCharacterCard(newCard);

Building prompt messages

Under the hood, AvatarSession uses buildCardMessages to construct the LLM messages from a card. You can also use this directly:

import { buildCardMessages } from "avatarlayer";

const messages = buildCardMessages(card, conversationHistory, {
  charName: "Luna",     // defaults to card.name
  userName: "User",     // default: "User"
  emotionSuffix: true,  // append emotion marker instructions
});

This produces a message array with:

A system message built from the card's description, personality, scenario, and system_prompt fields
Matched lorebook entries injected as context
Message examples from the card
The provided conversation history

Lorebook matching

Lorebook entries are automatically matched against the conversation history:

import { matchLorebookEntries } from "avatarlayer";

const matched = matchLorebookEntries(card.data.character_book, messages);

Each entry's keys are tested against message content. Matched entries have their content injected into the LLM context.

CharacterCard type

interface CharacterCard {
  spec: "chara_card_v3";
  spec_version: "3.0";
  data: CharacterCardV3Data;
}

interface CharacterCardV3Data {
  name: string;
  description: string;
  personality: string;
  scenario: string;
  first_mes: string;
  mes_example: string;
  system_prompt: string;
  creator_notes: string;
  tags: string[];
  creator: string;
  character_version: string;
  character_book?: CharacterBook;
  assets?: CharacterCardAsset[];
  // ...additional V3 fields
}

Character Cards