Avi - System Documentation

A personal agent system that runs on any machine. The agent shares the host's terminal, filesystem, browser, and apps, operating like a human user. Everything is extensible through plugins.

Core Principles

• Single Agent - one agent per org (named "avi" by default). No multi-agent support. Configured at the org level, operates across all projects.
• Zero infrastructure - embedded database (SQLite via better-sqlite3), no Docker, no external services.
• Agent uses real apps - no sandboxes. The agent runs as a native process with access to the host machine's terminal, filesystem, and apps.
• Database as config - agent, projects, plugins, and instructions live in the embedded database. No config files on disk. The web UI manages everything.
• Plugin = extension - all external capabilities are plugins. Channels handle messaging; MCP servers handle tools. Any existing MCP server can be wrapped with zero code.
• Multi-model - Vercel AI SDK (latest) for provider abstraction. Anthropic, OpenAI, Google, AWS Bedrock - swap models per org in Org Settings → AI.
• Project isolation - nested project tree with scoped data access. The agent sees the target project and all its children, never parents or siblings.
• Remote UI - running the web frontend on a different machine than the core. The UI is a pure viewer - all execution (tools, agent, plugins, MCP, database) stays on the core host.

Glossary

Architecture

Orchestrator
  │
  ├── Plugins (loaded once at startup, one plugin = one service)
  │     ├── PluginAPI         - env vars, version, log, scoped store
  │     ├── MCP Server        - wraps existing MCP server (command + args) and/or inline tools
  │     ├── Channels          - bidirectional conversations on a service, bound to projects
  │     └── Lifecycle         - onDisconnect hooks
  │
  ├── Projects (nested tree - data access flows down, channels do not)
  │     └── general/                    ← default project, always exists
  │           ├── personal/
  │           │     ├── work/
  │           │     │     ├── acme-corp/
  │           │     │     │     ├── Directory, Database (scoped)
  │           │     │     │     ├── Channels:  slack:channels:acme-team (via channels table)
  │           │     │     │     └── Tasks:     scheduled jobs (via tasks table)
  │           │     │     └── side-project/
  │           │     └── family/
  │           └── ...
  │
  │     Data access rule: the agent invoked for a project can read/write
  │     that project AND all its children. Cannot see parents or siblings.
  │
  ├── Tool Registry (all tools registered at boot, permission-gated per project)
  │     ├── System tools:        system_skills-*, system_tasks-*, system_notes-*, system_project-*, etc.
  │     └── Plugin tools:        chrome_page, computer_file-read, github_pr-list, etc.
  │
  └── Agent (one per org, "avi" by default - invoked per message, scoped to a project)
        ├── Data access:       scoped DB for this project + all children
        ├── Context:           channel summary + recent messages
        ├── Model:             Vercel AI SDK (any provider)
        ├── Response:          automatically sent back to the originating channel
        │
        ├── All permitted tools loaded directly into streamText as native AI SDK tools.
        │   No wrapper/dispatch layer. Tool names use underscores (provider-safe).
        │
        └── Tool Permissions:  per-project (via project_tool_permissions table)
              Tools are filtered by permissions before invocation.
              Only permitted tools are passed to streamText.

              Flow:  Agent → streamText(tools) → tool.execute()
                     (tools pre-filtered by permissions at invocation time)

MCP servers are spawned at plugin load time via stdio (@ai-sdk/mcp). After plugin.setup() completes, the system spawns each registered MCP server using createMCPClient + Experimental_StdioMCPTransport, discovers tools via mcpClient.tools(), and bridges them as regular PluginToolDefinition entries. MCP tools are prefixed with the plugin name using underscores (e.g., github_create_issue) and are indistinguishable from inline plugin tools at runtime. Tool calls have a 30s timeout. Spawn failures warn and skip - other plugins continue. Code: src/mcp/index.ts.

System Config

All configuration lives in the database - no config files on disk. The system scaffolds only a ~/.avi/data/ directory and a server_secret file (auto-generated).

AI settings are org-scoped — each org has its own provider keys, chat model, embedding model, and summarization model. These are stored directly on the orgs table: ai_model_chat, ai_model_summarization, ai_model_embedding (plain text model IDs), and ai_provider_keys (AES-256-GCM encrypted JSON blob containing all provider API keys). AI settings are managed via Org Settings → AI tab. The system blocks access until at least one provider key, a chat model, an embedding model, and a summarization model are configured (setup screen shown after registration).

Provider registry is centralized in config/models.ts. Supported providers: Anthropic, OpenAI, Google, AWS Bedrock. Each provider declares a secretKeys array (e.g. provider.anthropic, provider.awsBedrock.accessKeyId). A provider is considered "available" for an org only when all its secretKeys are present in that org's ai_provider_keys. The Org Settings → AI tab renders input fields dynamically from secretKeys.

AI API endpoints are org-scoped under /api/v1/accounts/:accountId/orgs/:orgId/ai/...:

• GET/PUT .../ai — read/write model selections (chatModel, embeddingModel, summarizationModel)
• GET .../ai/models — list providers with availability and models
• GET/PUT/DELETE .../ai/keys[/:key] — manage provider API keys (masked on read)
• GET .../ai/embeddings/count, POST .../ai/embeddings/reembed — embedding management

The agent, projects, and plugins are stored in database tables and managed through:

• Web UI - settings pages with instructions editor
• HTTP API - versioned at /api/v1/, with hierarchical URL scoping: /api/v1/accounts/:accountId/orgs/:orgId/...

Pagination

All list endpoints return a consistent paginated envelope:

{ "data": [...], "total": 42, "limit": 50, "offset": 0 }

Query params: ?limit=N&offset=N. Defaults: limit=50, offset=0, max limit 1000.

Startup Sequence

1. Scaffold home directory (~/.avi/data/) if not present
2. Initialize SQLite database + run migrations (schema only, no data seeding)
3. Generate server secret for JWT signing + secret encryption if not present (~/.avi/server_secret)
4. Register system tool metadata in the tool registry
5. Start web UI + HTTP API on configured port
6. Start task scheduler
7. First request: user registers → account + org + defaults seeded → setup modal shown
8. Subsequent requests: auth middleware resolves user + org → org runtime loaded on demand (config, plugins, tools)

All names (plugin, agent, project) are lowercase alphanumeric + hyphens only: /^[a-z0-9][a-z0-9-]*$/.

Logging

All core modules log through createLogger(source) from src/utils/index.ts. Output goes to console.error by default (visible in dev terminals). Embedders (e.g. Electron) can call setLogSink(sink) before start() to route core logs to a custom destination (file, structured collector, etc.). The sink receives (level, source, message, data?) — the same fields as the formatted console output but without ANSI escapes. The packaged desktop app routes core logs to ~/Library/Application Support/Avi/logs/main.log, while npm run dev uses ~/Library/Application Support/Avi Dev/logs/main.log.

Remote Core

The desktop app can run as a thin client, connecting to an Avi core on another machine (e.g. a Mac Mini or home server) instead of starting its own embedded core. When connected, the Electron window loads directly from the remote core's URL — the entire UI is served from the remote, so same-origin WebSocket and relative API URLs work automatically with zero frontend changes. All agents, tools, plugins, and data stay on the remote host.

Config file: ~/.avi/avi.json (or $AVI_HOME/avi.json if AVI_HOME is set). The ~/.avi/ directory is created automatically if it doesn't exist. Read/written via IPC through the preload bridge (window.desktop.getRemoteConfig, window.desktop.setRemoteConfig).

Two config blocks:

remoteHost — for the machine running the core (e.g. Mac Mini). Binds to 0.0.0.0 so other devices can connect.

{ "remoteHost": { "enabled": true } }

remoteClient — for the machine running only the GUI (e.g. MacBook). Skips the local core, connects to the remote.

{ "remoteClient": { "enabled": true, "url": "http://100.64.1.5:4200" } }

Setup:

• Host machine: Add remoteHost.enabled: true to ~/.avi/avi.json, restart Avi. The Remote tab in Admin Tools will show "Remote Core: Host Mode Enabled".
• Client machine: Open Admin Tools → Remote tab, enter the host URL, click Connect. Or add remoteClient to ~/.avi/avi.json directly.
• Disconnect: Via Admin Tools → Remote tab or the system tray menu ("Disconnect from Remote Core"). Clears remoteClient, starts a local core.

UI indicators: A badge in the bottom-left sidebar shows the current mode ("Remote Core: Client Mode" or "Remote Core: Host Mode") when either mode is active. The Admin Tools → Remote tab shows the current state and connection details.

Networking: Tailscale recommended for secure access — provides encrypted WireGuard connectivity between devices without exposing ports to the internet. Works across any network. Plain LAN also works (same network only, unencrypted).

Health check: GET /api/v1/health — unauthenticated endpoint returning { ok: true, version }. Used by the desktop app to verify the remote core is reachable before loading.

Startup: On launch, the desktop app reads ~/.avi/avi.json. If remoteClient is enabled, it skips the embedded core and loads the remote URL. If unreachable, shows retry/local/quit options. If remoteHost is enabled, the local core binds to 0.0.0.0 instead of 127.0.0.1.

Secure context: The remote UI is served over plain HTTP (not HTTPS). Browser APIs that require a secure context (e.g. crypto.randomUUID) are not available. The frontend uses a uuid() helper (packages/web/src/lib/uuid.ts) with a fallback for insecure contexts.

Data Architecture

SQLite (via better-sqlite3). Single database file (~/.avi/data/core.db). All data is scoped by org_id. No config files on disk. Most tables use created_at/updated_at timestamps. Tables that omit updated_at: accounts, orgs, account_members, org_members, messages.

The query() function from client.ts accepts Postgres-style $1, $2 placeholders and converts them to SQLite ? internally. initDb() and closeDb() are synchronous.

Migrations: Schema evolution is managed by numbered migrations in src/models/migrations.ts. A _migrations table tracks which migrations have been applied. On startup, runMigrations() runs any pending migrations in a transaction. To add a schema change: create a new migration function (e.g. migration002), add it to the migrations array with the next sequential ID. Never modify an already-applied migration.

Foreign keys: All org_id references enforce REFERENCES orgs(id) ON DELETE CASCADE. Project-scoped tables cascade from projects(id). Foreign keys are enforced via PRAGMA foreign_keys = ON.

WAL & checkpointing: WAL mode enabled for concurrent read/write performance. Auto-checkpoint at 1000 pages. On graceful shutdown, PRAGMA wal_checkpoint(TRUNCATE) flushes the WAL back into the main DB file.

Backups: Daily automatic backups via VACUUM INTO to ~/.avi/backups/core-{timestamp}.db. Keeps the 10 most recent. Also runs on graceful shutdown. backupDatabase() is exported for manual triggers.

Identity & Multi-tenancy

users

device_codes

Short-lived codes for device authorization flow (RFC 8628). Schema: avi.

auth_codes

One-time codes for exchanging Google OAuth callback for JWT. Schema: avi.

accounts

Billing unit. Auto-created personal accounts cannot be deleted.

orgs

Container for all data. Auto-created personal orgs cannot be deleted or renamed. AI configuration is org-scoped — each org has its own model selections and provider API keys.

account_members / org_members

Auth

Authentication uses Google OAuth only with custom JWT sessions. No password auth. No helper libraries (no BetterAuth, no Passport).

Backend (packages/backend):

• Google OAuth: Direct integration with Google OAuth2 endpoints. Code: src/auth/google.ts
• JWT: HMAC-SHA256 tokens, 30-day expiry. Secret via JWT_SECRET env var. Payload: { sub, email, name, iat, exp }. Code: src/auth/jwt.ts
• Middleware: requireAuth validates JWT Bearer token. Code: src/middleware/require-auth.ts
• User table: "user" table with google_sub column for Google identity linking
• Migration: Existing password users are seamlessly migrated on first Google sign-in by matching email addresses

Auth routes (all under /api/v1):

• GET /auth/google — Redirects to Google OAuth consent screen
• GET /auth/google/callback — Handles Google callback, upserts user, creates one-time auth code, redirects to site
• POST /auth/token — Exchanges one-time auth code for JWT (avoids token-in-URL exposure)
• GET /auth/me — Returns current user from JWT
• POST /auth/device/code — Generates device code + user code for device auth flow
• POST /auth/device/token — Poll endpoint for device auth (returns JWT once approved)
• POST /auth/device/approve — Approves a device code (requires auth)

Dashboard routes (all under /api/v1, require auth):

• GET /dashboard/usage — Current billing period usage total
• GET /dashboard/billing — Current plan (free/pro)
• POST /dashboard/billing/upgrade — Switch plan

Device Code Flow (RFC 8628): Used by packages/web and packages/desktop to authenticate. The user signs in via packages/site (the only surface with Google OAuth), enters a code, and the device receives a JWT.

Login surface: Only packages/site (Next.js) handles sign-in. packages/web and packages/desktop use the device code flow. There is no local email/password auth anywhere — packages/core no longer has register, login, or password change routes.

Core (packages/core):

• No local auth routes (register/login/password removed)
• Bridge route POST /api/v1/auth/bridge remains — exchanges a remote JWT for a local session token
• Local JWT signing/verification via src/auth/token.ts (for bridge sessions)
• Remote identity sync via src/auth/remote.ts (fetches bootstrap from backend, upserts user locally)
• src/auth/password.ts deleted — no password hashing in the codebase

Site dashboard (packages/site):

• /login — Google OAuth redirect

• /auth/callback — Exchanges one-time auth code for JWT, stores in localStorage, redirects to dashboard

• /activate — Device code entry page (preserves code across Google login redirect)

• /dashboard — Usage summary (current period cost)

• /dashboard/billing — Plan management (free/pro)

• Scoped routes: /api/v1/accounts/:accountId/orgs/:orgId/...

• Global routes: GET /api/v1/settings, PUT /api/v1/settings/:key

• Signup flow: Google sign-in creates User → Account (personal=true, plan=free) → Org (personal=true) → seeds default agent ("avi") + project + plugins

Core Tables

All org-scoped via org_id.

agents

One agent per org (named "avi" by default). Configured via config.agent (singular object with id and model). The chat model is stored in app_settings as ai.chatModel, not on the agent row.

projects

Hierarchy is defined by parent_project_id, not by the name. Slashes in the "New Project" dialog (e.g. work/acme) are a creation convenience — each segment becomes a separate project with the correct parent. Names are freely renamable without affecting hierarchy.

project_tool_permissions

Junction table - one row per permitted tool per project. No wildcards.

project_skill_permissions

Junction table - one row per permitted skill per project.

channels

External channel bindings - maps a plugin channel ID to a project.

channel_instances

Per-project plugin integrations with credentials and config.

• Slack channel instances require project secret refs for bot_token and app_token, plus raw workspace_id and channel_id values.
• Telegram channel instances require a project secret ref for bot_token.
• Raw secrets are stored in the secrets table. channel_instances.credentials stores secret refs and provider IDs, not decrypted secret values.

tasks

Scheduled/managed tasks within projects. Supports 2-level hierarchy (parent tasks with subtasks).

notes

Project-scoped markdown documents. Flat structure (no hierarchy).

skills

contacts

Org-scoped (shared across all projects).

contact_tags

events

General-purpose event log. The type field follows a dotted convention: entity.action (e.g. update, webhook.received).

messages

channel_summaries

plugins

plugin_store

secrets

Encrypted secrets (AES-256-GCM). Scoped by project_id - NULL for system-level secrets.

embeddings

Vectors are stored in a separate vectors.db file via sqlite-vec, not in the main embeddings table. When no embedding model is configured, the system falls back to LIKE text search. Supported embedding providers: OpenAI (text-embedding-3-small, text-embedding-3-large) and AWS Bedrock (cohere.embed-english-v3 — 1024 dimensions). When switching embedding models, the Org Settings → AI tab has a "Re-embed all data" button that regenerates vectors for all stored content using the new model (old vectors are dropped since dimensions may differ).

Vector resilience: If sqlite-vec loads but the vec0 module is unavailable (observed on some platforms), ensureVecTable catches the error, permanently sets vecDb = null, and logs a one-time WARN. Both writes (storeVectors) and reads (searchEmbeddings) check this flag — when disabled, writes are skipped and searches fall back to text search without generating query embeddings. Embedding input text is truncated to 2000 chars before sending to the model (Cohere embed-english-v3 rejects inputs over 2048 chars).

app_settings

Global settings (non-AI). App UI preferences like app.autoStart, app.startMinimized, app.showDockIcon, app.remoteAccess, app.port. AI model settings have moved to the orgs table (see below).

usage_daily

Daily token usage rollups per (account, org, project, agent, model). Upserted after each agent invocation.

Code Architecture

packages/
  ├── core/                         - @avirun/core: backend + API
  │     ├── src/
  │     │     ├── index.ts          - Public API: exports start()
  │     │     ├── cli.ts            - CLI entry point (arg parsing, signal handlers)
  │     │     ├── bin/avi.ts        - npx/bin entry point
  │     │     ├── auth/
  │     │     │     ├── token.ts     - JWT sign/verify (HMAC-SHA256)
  │     │     │     ├── remote.ts    - Remote auth bridge (fetchRemoteBootstrap, sync)
  │     │     │     └── middleware.ts - Request auth + org resolution
  │     │     ├── orchestrator/
  │     │     │     └── index.ts    - Core process: loads config, starts plugins, routes messages
  │     │     ├── config/
  │     │     │     ├── index.ts    - loadConfigFromDb()
  │     │     │     └── models.ts   - Provider registry, model pricing
  │     │     ├── router/
  │     │     │     └── index.ts    - Message routing: channel → project
  │     │     ├── agent/
  │     │     │     └── index.ts    - Agent invocation via Vercel AI SDK
  │     │     ├── memory/
  │     │     │     └── index.ts    - Summary rolling
  │     │     ├── tools/
  │     │     │     ├── index.ts    - Tool registry, permission matching, text search
  │     │     │     └── system-tools.ts - All system tool definitions
  │     │     ├── mcp/
  │     │     │     └── index.ts    - MCP server spawning, tool discovery, bridging
  │     │     ├── plugins/
  │     │     │     ├── index.ts    - Plugin discovery, validation, lifecycle
  │     │     │     ├── context.ts  - PluginAPI implementation
  │     │     │     └── web/index.ts - Built-in web UI + HTTP API + WebSocket
  │     │     ├── channels/
  │     │     │     └── index.ts    - Channel message handling, summary management
  │     │     ├── scheduler/
  │     │     │     └── index.ts    - Interval-based task scheduling
  │     │     ├── server/
  │     │     │     └── index.ts    - HTTP server + WebSocket + all API routes
  │     │     ├── models/           - Database models (ALL database access goes through here)
  │     │     │     ├── client.ts   - SQLite instance, migrations, seeding, seedOrgDefaults()
  │     │     │     ├── migrations.ts - Schema definitions
  │     │     │     ├── user.ts, account.ts, org.ts - Identity
  │     │     │     ├── agent.ts, project.ts - Agent and project config
  │     │     │     ├── channel-instance.ts - Plugin channel instances
  │     │     │     ├── channel-summary.ts - Rolling summaries
  │     │     │     ├── message.ts  - Message history
  │     │     │     ├── task.ts, note.ts, contact.ts - Managed entities
  │     │     │     ├── skill.ts    - Skill templates
  │     │     │     ├── plugin-store.ts - Plugin K/V store
  │     │     │     ├── embedding.ts - Vector embeddings (sqlite-vec)
  │     │     │     ├── secrets.ts  - Encrypted secrets (AES-256-GCM)
  │     │     │     ├── settings.ts - App settings (global)
  │     │     │     ├── usage.ts    - Token/cost tracking
  │     │     │     └── event.ts    - Event log
  │     │     ├── utils/
  │     │     │     └── index.ts    - Shared utilities (logging, hashing, path helpers)
  │     │     └── types/
  │     │           └── index.ts    - Shared type definitions
  │     └── tests/
  │
  ├── web/                          - @avirun/web: React frontend (Vite)
  │
  ├── desktop/                      - @avirun/desktop: Electron desktop app
  │
  ├── site/                         - Landing page (Next.js, static)
  │
  └── plugins/
        ├── chrome/                 - Chrome browser automation (Puppeteer/CDP)
        ├── computer/               - File operations, terminal access
        ├── github/                 - GitHub API + git CLI
        ├── google/                 - Gmail, Calendar, Contacts, Drive
        ├── freshdesk/              - Freshdesk support tickets, contacts, companies
        ├── hubspot/                - HubSpot CRM sales tools
        └── mongodb/                - MongoDB driver wrapper

Rule: No file outside src/models/ may import the database client or execute queries directly.

Routing & Messages

Channel ID Format

• Native channels: project:{projectId}:user:{userId} - each user gets their own conversation per project
• External channels: {plugin}:{type}:{id} (e.g., slack:channels:acme-team) - bound to projects via the channels table

Routing Flow

1. Message arrives (WebSocket or scheduled task)
2. Router parses channel ID, performs async DB lookup via getProjectByChannelId
3. The single org agent is always used - no agent selection needed
4. Result: { projectId, channelId }

System Prompt Assembly

When an agent is invoked, the system prompt is assembled in this order:

1. Agent instructions - from agents table (static, cacheable)
2. Project instructions - from projects table (static, cacheable)
3. Channel summary - rolling summary of the conversation history
4. Context - user name, model name, project files directory, permitted tool/skill names

Recent messages since the last summary are appended as user/assistant conversation turns, not in the system prompt. Prompt caching is enabled via Vercel AI SDK's providerOptions to cache the static prefix.

Concurrency

The orchestrator serializes agent invocations per channel (projectId:channelId) - one invocation at a time for a given conversation. Different channels, including multiple conversations inside the same project, can run concurrently.

Post-response work such as usage logging and rolling summaries runs on a separate per-channel background queue after stream-end, so the next user turn on that conversation does not wait for summary generation before it can start.

Memory & Summarization

Rolling summary system compresses older messages to keep context manageable:

• Threshold: 4000 tokens accumulated after last summary
• Window: Keep 20 most recent messages unsummarized
• Cap: Summary capped at 6000 tokens; older sections merged if exceeded
• Stored in channel_summaries with last_message_id checkpoint
• Model context: the agent loads only messages after last_message_id and receives older context via ## Conversation Summary in the system prompt
• UI history: the full UIMessage transcript remains persisted and is still returned by the paginated /projects/:id/messages API for chat history browsing
• Tool-heavy chats: rolling-summary token estimation includes bounded text extracted from tool parts, so large tool-call/tool-result histories still trigger summarization instead of growing forever
• Archived channel fallback: when loadMessagesPaginated queries a specific channel that has 0 active messages (all archived by rolling summary), it falls back to returning the channel summary as a synthetic assistant message with ID summary:{channelId}. Only fires at offset=0. This prevents task channels that ran and were fully summarized from showing "No messages yet"
• Non-fatal: summary generation is wrapped in a top-level try/catch — failures are logged but never surface to the user
• Timeout: generateText calls use AbortController with a 120s timeout to prevent indefinite hangs on oversized context
• Truncation: conversation text is capped at 80K chars before sending to the summarization model
• Streaming persistence: the new user message is appended to the database before streaming begins so reconnects can see it; the full transcript (including completed tool calls) is rewritten when the response completes or is aborted. On abort, only completed tool calls with results are persisted — in-flight calls are excluded. The ActiveRunStore (in-memory, src/active-runs.ts) tracks whether a run is active for the /status endpoint and archive safety. Reconnecting clients see a "working" spinner via /status and receive live content via WebSocket; on stream-end the client refetches canonical history from the DB
• Tool output safety net: before persisting UIMessages, the orchestrator caps any tool output part exceeding 100K chars. The truncated output includes a note with the original size. This prevents any single oversized tool result from making the channel's history exceed the model's context window on the next turn. Plugin-level caps (e.g. chrome response trimming) handle known cases; this is the general safety net
• History recovery: at invocation time, recoverPersistedMessages validates persisted assistant tool parts against AI SDK v6 state requirements — input is required for all completed states (input-available, approval-requested, approval-responded, output-available, output-error, output-denied), output for output-available, either output or errorText for output-error, and per-state approval validation (approval-requested requires { id } with no approved/reason, approval-responded requires { id, approved: boolean }, output-denied requires { id, approved: false }). Parts that fail are dropped while surrounding text is preserved. If SDK validation still fails after recovery, an aggressive fallback (stripUnvalidatableToolParts) removes all tool parts whose tool name is not in the current runtime registry. Empty assistant message shells are removed. This ensures one poisoned historical row cannot permanently block the channel

System Tools

All system tools are defined in src/tools/system-tools.ts, registered at boot, and permission-gated per project via project_tool_permissions.

All permitted tools (system + plugin) are loaded directly into streamText as native AI SDK tools. No wrapper/dispatch layer. Tool names use underscores natively — providers reject colons.

Skills

Tasks

Notes

Patch operations: notes-update accepts an operations array for surgical edits. Operations: replace, insert_after, insert_before, delete, append, prepend. Heading-scoped operations: replace_section, insert_after_section, delete_section (target sections by heading line, e.g. ## April 2026, with optional occurrence for repeated headings). Line-number operations: replace_lines, insert_at_line, delete_lines (1-indexed; apply in reverse document order when issuing multiple line-number ops to avoid offset drift).

Contacts

Project

Diagnostics

Updates

Plugins

A plugin can provide channels (bidirectional messaging), an MCP server (tools), or both. One plugin = one service.

Plugins declare env vars they need. The system provides them at startup along with a PluginAPI:

interface PluginAPI {
  env: Record<string, string>;          // only their declared vars
  log: PluginLogger;                    // scoped logger (plugin:{name})
  globalStore: PluginStore;             // plugin-global storage (not project-scoped)
  version: string;                      // system framework version

  addTool(name: string, definition: PluginToolDefinition): void;
  addChannel(name: string, handlers: PluginChannelHandlers, options?: PluginChannelOptions): void;
  addMcp(name: string, config: McpServerConfig): void;
  onDisconnect(fn: () => Promise<void> | void): void;
}

// Channel handlers receive an emit callback for inbound messages:
interface PluginChannelHandlers {
  onMessage: (emit: (msg: ChannelEmitMessage) => void) => void;
  sendMessage: (id: string, text: string) => Promise<void> | void;
}

// Tool functions receive a project-scoped context:
interface PluginToolContext {
  projectId: string;
  store: PluginStore;                   // scoped to current plugin + current project
  files: PluginFiles;                   // scoped filesystem access
  secrets: PluginSecrets;               // explicit secret resolution API
}

interface PluginSecrets {
  getValue(keyName: string): string | null;
  requireValue(keyName: string): string;
  getOptionalValue(keyName: string): string | undefined;
}

interface PluginFiles {
  write(filename: string, data: Buffer): Promise<void>;
  read(filename: string): Promise<Buffer>;
  list(): Promise<string[]>;
  path(filename: string): string;       // returns absolute path
}

interface PluginStore {
  get(key: string): unknown;
  set(key: string, value: unknown): void;
  del(key: string): void;
  list(prefix: string, options?: { sortBy?, order?, limit?, offset?, filter? }): { data, total, limit, offset };
}

Data isolation: Plugin store data is isolated by both plugin name and project_id. A plugin cannot access another plugin's data. The orchestrator controls scoping - plugins cannot override it. globalStore uses project_id = '' for plugin lifecycle data only.

Plugin files: context.files provides filesystem access at <projectDir>/files/. Plugins share the project's files directory. Paths must be relative (subdirectories allowed) and must not contain .. - prevents directory traversal.

Secret handling: Plugin tools resolve secrets explicitly through context.secrets. The model sees secret key names and descriptions but never secret values. Tools accept secret key names as input parameters and resolve actual values server-side inside execute via context.secrets.requireValue(keyName). No magic substitution — secret use is visible in the tool schema and implementation. Channel runtimes (Slack, Telegram) use api.resolveProjectSecrets() for credential resolution at connection time.

Plugins cannot access other plugins, the database, the agent, or projects. They emit messages and the orchestrator handles routing. A plugin's MCP server can be an existing MCP server (wrapped by command + args), inline tool definitions, or both.

Built-in Plugins

chrome - Chrome browser automation via Puppeteer/CDP. No env vars. Connects to a real headed Chrome (not headless) to avoid bot detection. Env overrides: AVI_BROWSER_CDP_PORT (default 9222), AVI_BROWSER_USER_DATA_DIR (default /tmp/avi-browser). All chrome tools trim responses by default to prevent context window overflow (pruned a11y trees, capped console/network output, screenshots saved to file instead of inline base64). Pass raw: true for full unprocessed output (hard-capped at 25K chars, paginated via offset). Tools:

• browser-start — Launch Chrome/Brave/Edge with CDP debugging enabled
• browser-stop — Close the browser and free resources
• page — Navigate, create, list, select, or resize pages. Action: navigate | new | list | select | resize
• interact — Click, fill forms, select dropdowns, type, press keys, scroll, run JS, handle dialogs, upload files, wait. Action: click | fill | select | fill_form | type | press_key | hover | scroll | evaluate | handle_dialog | upload | wait
• snapshot — Take screenshots (base64 or save to file), accessibility snapshots, or memory metrics. Action: screenshot | accessibility | memory
• network — Inspect captured network requests. Action: list | get
• console — Inspect captured console messages. Action: list | get
• performance — Profile page performance (tracing, metrics, Lighthouse). Action: start_trace | stop_trace | analyze_insight | lighthouse

computer - File operations and terminal access. No env vars. Has its own database - does not write to the main database. Tools:

• file-read, file-write, file-edit, file-delete, file-copy, file-move, file-search (mode="files" for glob, mode="content" for grep) — all restricted to the project directory; paths outside it are rejected
• terminal-run (synchronous), terminal-start (background, 5000-line rolling buffer), terminal-list, terminal-read (offset-based pagination), terminal-kill

slack - Slack workspace integration. Channel handler uses Socket Mode (bot token + app token) for real-time bidirectional messaging. Tools use the Slack Web API and work with either user tokens (xoxp-) or bot tokens (xoxb-) — the agent picks the right secret based on context. User tokens act as you; bot tokens act as the bot.

• conversations-list — List channels, DMs, and group DMs with unread status. Sorted unread-first. Resolves DM user names.
• conversations-history — Read recent messages from a channel or DM. Returns messages oldest-first with user names resolved.
• chat-send — Send a message to a channel or DM. Supports thread replies via thread_ts.

github - GitHub API (Octokit) + git CLI (execFile('git', ...)). Tools:

• whoami, repo-list, repo-create, repo-get, repo-delete
• branch-create, branch-delete, branch-rename
• pr-create, pr-list, pr-update
• issue-create, issue-list, issue-update
• actions-trigger, actions-list
• release-list, release-upsert
• git-clone, git-push, git-pull, git-commit, git-status, git-log, git-checkout, git-rebase, git-diff — all restricted to the project directory

google - Multi-service via REST APIs (no Google SDK, all fetch). One OAuth client and one refresh token covers all services. Required secret keys: client_id, client_secret, refresh_token. OAuth 2.0 with automatic token refresh.

• Gmail (14 tools): gmail-review-inbox, gmail-search, gmail-get-message, gmail-list-threads, gmail-get-thread, gmail-list-labels, gmail-manage-label, gmail-get-attachment, gmail-send, gmail-reply, gmail-forward, gmail-create-draft, gmail-modify-labels, gmail-trash
  • gmail-review-inbox: Thorough paginated inbox review — finds every non-draft, non-sent thread in the mailbox within a precise timespan. Uses Unix timestamps internally (UTC for date-only inputs). Discovers ALL threads via threads.list with exhaustive pagination, no caps. Default query: broad discovery (-in:sent -in:drafts -in:spam -in:trash) + sent query for reply detection. With include_spam_trash: true, runs additional targeted in:spam and in:trash queries (Gmail excludes these by default, so they must be explicitly searched). Thread IDs frozen in an immutable stateless base64url cursor on first call; agent pages with next_cursor until hasMore: false. Ordering: unreplied first, then replied; within each group, Gmail's threads.list order (most recent activity). Review pages are AI-safe by default: page_size now defaults to 10, message bodies are returned as bounded previews by default, bodyTruncated marks previews, and callers should use gmail-get-thread or include_full_bodies: true only for small pages when full content is required. Returns totalThreads (frozen at discovery, stable across pages), unrepliedCount, repliedCount, offset, returnedCount, remainingCount, bodyMode, and bodyCharLimit. Total fetches per call capped at page_size (retries get priority, remaining budget goes to new threads). Failed thread fetches retried 3x per attempt, carried forward in cursor with cumulative failure count; after 3 cumulative cross-page failures a thread moves to permanentlyFailed and stops retrying. partialFailures shows threads still being retried (with retryCount), permanentlyFailed shows threads that exhausted retries. threadIds is never mutated; offset advances by new-thread slice size. page_size validated to 1-200. Snapshot consistency: thread IDs frozen at discovery; thread contents fetched live per page. Concurrency-throttled (15 parallel).
  • gmail-search and gmail-list-threads: Auto-paginate internally (up to 100). Default max_results bumped to 25. gmail-search returns full messages grouped by thread.
• Contacts (6 tools): contacts-list, contacts-search, contacts-get, contacts-other, contacts-create, contacts-delete
• Calendar (6 tools): calendar-list, calendar-events, calendar-get-event, calendar-create-event, calendar-update-event, calendar-delete-event
• Drive (5 tools): drive-list, drive-search, drive-get, drive-download, drive-upload

hubspot - HubSpot CRM sales tools via REST API v3 (no SDK, all fetch). Auth: either access_token directly, or client_id + client_secret + refresh_token for OAuth 2.0 with automatic token refresh.

• Contacts (5 tools): contact-list, contact-get, contact-create, contact-update, contact-search
• Deals (6 tools): deal-list, deal-get, deal-create, deal-update, deal-search, pipeline-list
• Companies (5 tools): company-list, company-get, company-create, company-update, company-search
• Engagements (4 tools): note-create, task-create, task-list, engagement-list

freshdesk - Freshdesk support tools via REST API v2 (no SDK, all fetch). Auth: API key via HTTP Basic Auth. Required secrets: FRESHDESK_API_KEY (from Freshdesk → Profile Settings → Your API Key), FRESHDESK_DOMAIN (the subdomain only — e.g. if helpdesk URL is acme.freshdesk.com, set domain to acme). The API always uses {domain}.freshdesk.com, not any custom portal domain. The API key inherits the permissions of the Freshdesk user that generated it; to restrict access, create a dedicated agent with a limited role.

• Tickets (8 tools): ticket-list, ticket-get, ticket-create, ticket-update, ticket-delete, ticket-search, ticket-reply, ticket-note
• Contacts (5 tools): contact-list, contact-get, contact-create, contact-update, contact-search
• Companies (5 tools): company-list, company-get, company-create, company-update, company-search

mongodb - MongoDB database operations and Atlas cloud management. Two auth modes:

• Database tools (via MONGODB_URI): list-databases, list-collections, collection-schema, find, count, aggregate, insert, update, delete, list-indexes, create-index
• Atlas Admin tools (via ATLAS_PUBLIC_KEY + ATLAS_PRIVATE_KEY): atlas-list-projects, atlas-list-clusters, atlas-get-cluster, atlas-list-db-users

Web UI

A minimal web app ships with the system on localhost:{port} (default 4200). Messages flow through the same channel system as external plugins. The port is a CLI option (--port 4200).

• Chat interface - send messages, see responses with tool call rendering, view conversation history. On opening a project, the last 30 messages are loaded with infinite scroll for older batches. Real-time WebSocket messages merge with loaded history (deduped by ID). Stream-delta updates are batched via RAF (~15 fps) to avoid per-token re-renders. Auto-scroll is RAF-throttled. The Markdown component is memoized (React.memo + hoisted config) to avoid unnecessary re-parsing. Clearing history uses echo-suppression (localClearsRef) so the originating client ignores the server's WS echo; post-clear fetches replace local state while normal fetches merge to preserve live streaming. The orchestrator uses generation-based cancellation (cancelGeneration counter per channel) so stopping an agent only skips invocations queued before the stop.
• Project sidebar - navigate between projects, access settings via gear icon. "Agent Settings" link at bottom.
• Agent settings - /settings/agent/:tab?. General + Instructions tabs. Edit model and instructions.
• Project settings - tabs: General, Instructions, Tools, Skills. Tool and skill permissions managed per-project.
• Org settings - General (name, delete) and Analytics tabs. Analytics shows monthly cost chart and daily usage table, filterable by project.
• Admin tools - Database tab for browsing all tables: view paginated rows, inspect records, delete records.
• Right sidebar - icon rail with Tasks, Notes, and Contacts. Only one open at a time. Resizable (200-600px). Each has list + detail view with real-time WebSocket updates.
• Rich text editor - InstructionsEditor wraps Milkdown (ProseMirror-based WYSIWYG). Used in project instructions, agent instructions, skill editing, task notes, and note content.
• Character rendering - the Three.js character renders at display refresh rate (typically 60 fps) and uses adaptive renderer pixel ratio based on canvas size, with small surfaces allowed to render up to DPR 2-2.25 so the robot stays crisp on HiDPI displays. Project chat explicitly caps the character at DPR 2, while larger hero/profile renders can use the same or higher adaptive budget when device DPR allows. Shadow maps stay cached for normal idle rendering, but fast full-body intro animations refresh shadows every frame so self-shadowing does not visibly detach from the model. The current quality/perf balance keeps PCFShadowMap but restores a higher 1024 shadow-map budget, larger shadow radius, and more blur samples than the earlier low-quality pass, which improves softness without returning to the full pre-optimization 2048 cost. Animation timing uses THREE.Timer so the UI avoids THREE.Clock deprecation warnings and does not accumulate a giant delta after the page is hidden.

Tasks REST API

Notes REST API

Contacts REST API

Channel Archive REST API

Events REST API

WebSocket broadcasts on mutations: task-changed, note-changed, contact-changed, config-changed, channel-history-cleared. The WebSocket URL auto-selects ws: or wss: based on window.location.protocol for remote/TLS compatibility.

Secret Management

Secrets are stored encrypted (AES-256-GCM). The encryption key is derived from ~/.avi/server_secret. Secrets are never sent to the client in plaintext — the API returns masked values only.

• Org provider keys: Stored as an encrypted JSON blob in orgs.ai_provider_keys. Managed via Org Settings → AI tab. At runtime, resolveModel() reads provider keys from the org (with a 1-minute cache). Code: src/models/org.ts.
• Project secrets: Stored in the secrets table scoped by project_id. Plugin credentials (e.g., Google OAuth tokens), managed via project settings. Code: src/models/secrets.ts.

Plugin env vars are separate from the encrypted secrets store. Plugins declare env var names; the orchestrator reads them from process.env at startup and passes only declared vars into PluginAPI.env.

Configuration

Minimal filesystem footprint:

~/.avi/
  ├── server_secret          - JWT signing + secret encryption key (auto-generated)
  ├── data/                 - SQLite database (core.db)
  ├── backups/              - Daily database backups
  └── accounts/{accountId}/
        └── orgs/{orgId}/
              └── projects/{projectId}/files/   - project files on disk

Config type: Config = { agent: AgentConfig; projects: Record<string, ProjectConfig> } where AgentConfig = { id, model } and ProjectConfig = { id }.

The AVI_HOME environment variable overrides the default ~/.avi path. Use this to run separate dev/test/prod instances side-by-side.

Development

Running Dev and Production Side-by-Side

Dev and production paths are determined automatically — no manual AVI_HOME override is needed:

• Dev (npm run dev): Uses ~/.avi-dev. The desktop app detects dev mode via !app.isPackaged and the core detects it via NODE_ENV !== 'production'. The desktop sets process.env.AVI_HOME so the core always agrees. Web on port 5173 (Vite), core on port 4201. Electron renderer storage and logs are isolated under ~/Library/Application Support/Avi Dev.
• Production (packaged Electron app): Uses ~/.avi. The desktop detects production via app.isPackaged and propagates AVI_HOME=~/.avi to the core process via process.env.AVI_HOME.
• Tests: Each test creates a temp directory and sets AVI_HOME per test. Tests that call real AI APIs read ANTHROPIC_API_KEY from env and seed it into the secrets store.

Electron

• Spawns core process at startup, embeds React UI via BrowserWindow
• Dev mode: loads from Vite dev server (http://localhost:5173) and uses a separate Electron userData directory (~/Library/Application Support/Avi Dev)
• Prod mode: serves built web dist
• Tray icon with menu, single instance lock, graceful shutdown
• CSS note: Tailwind v4's Vite plugin strips -webkit-app-region. Apply it via inline React styles, not CSS files.
• Linux support: electron-builder targets AppImage and deb. Icon sourced from packages/web/public/app-icon.png. GPU hardware acceleration is auto-disabled on Linux (app.disableHardwareAcceleration()) to avoid blank/black window issues with incomplete GPU drivers.

CI/CD — GitHub Actions

• Workflow: .github/workflows/release.yml, triggered manually via workflow_dispatch
• Build matrix: macOS ARM (macos-14), Linux x64 (ubuntu-latest), Linux ARM (ubuntu-24.04-arm)
• Release job: creates a GitHub Release tagged v{version} using the next version derived from existing release tags and the stable S3 publish path, then attaches all artifacts (.dmg, .AppImage, .deb)
• macOS code signing: not configured yet — users must right-click → Open to bypass Gatekeeper

Theme System

All colors are CSS custom properties in index.css, exposed to Tailwind via @theme inline. Both :root (light) and .dark define every variable. Use Tailwind classes (bg-background, text-foreground, border-border) - never hardcode colors.

Key semantic roles: background/foreground, sidebar/sidebar-foreground/sidebar-border, primary/primary-foreground, secondary, muted/muted-foreground, border/input, popover.

Dedicated macOS Machine Setup

Guide for running Avi on a dedicated macOS machine (Mac Mini, Mac Studio, etc.) as a headless agent host. This is the recommended setup for Remote UI mode, where the core runs on the Mac and the UI is accessed from another device.

Initial Setup

1. Install Avi — follow the standard installation steps.
2. Enable Remote Core — add {"remoteHost":{"enabled":true}} to ~/.avi/avi.json and restart.
3. Network access — use Tailscale for secure remote access. The Remote UI connects to http://<tailscale-hostname>:4200.

Disable macOS Keychain GUI Prompts

On a headless machine, macOS Keychain dialogs block tool calls indefinitely because there's no one to click "Allow." Run these commands once to prevent all Keychain popups:

# Prevent the security daemon from ever showing GUI dialogs
defaults write com.apple.security PKEnableSecuritydUI -bool false

# Lock the login keychain — access attempts fail immediately instead of prompting
security lock-keychain ~/Library/Keychains/login.keychain-db

# Remove any existing credentials that trigger prompts
security delete-internet-password -s github.com 2>/dev/null
security delete-generic-password -s github.com 2>/dev/null

# Disable git's Keychain integration at all levels
git config --global credential.helper ""
sudo git config --system credential.helper "" 2>/dev/null

Why this matters: Any process that touches the Keychain (git, curl, gh CLI, security frameworks) will pop a dialog on macOS. On a headless machine with no display, this dialog hangs forever and stalls the agent's tool call.

Credential Management

Do not store credentials in the macOS Keychain. Use Avi's built-in project secrets instead:

1. Add API keys, tokens, and credentials via Org Settings → AI (for AI providers) or Project Settings → Secrets (for project-specific credentials like GitHub PATs, database URIs).
2. Secrets are stored encrypted (AES-256-GCM) in Avi's database — not in the macOS Keychain.
3. When the agent runs a tool call, project secrets are automatically resolved and injected as environment variables.

For GitHub specifically:

# Add your GitHub PAT as a project secret in Avi's UI, then:
# The agent will use GH_TOKEN env var automatically in tool calls.
# No need for `gh auth login` or Keychain-stored credentials.

Defense-in-Depth

The computer plugin automatically injects these environment variables into every spawned process:

These are set automatically — no user configuration needed. They prevent any subprocess from triggering a Keychain prompt even if the system-level settings are not configured.

macOS Permissions (TCC)

macOS requires explicit permission grants for apps to access protected resources. On a headless machine, TCC prompts block tool calls indefinitely. Grant these permissions once during initial setup while you have screen access:

1. Full Disk Access — System Settings → Privacy & Security → Full Disk Access → add Avi.app (and/or Terminal if running from CLI). Required for the agent to read/write files across the filesystem.
2. Accessibility — System Settings → Privacy & Security → Accessibility → add Avi.app. Required for UI automation tools (AppleScript, clicking, typing).
3. Automation — Granted automatically when the first AppleScript command runs. Click "OK" on the one-time prompt.
4. Screen Recording (optional) — System Settings → Privacy & Security → Screen Recording → add Avi.app. Only needed if you want the agent to take screenshots.

After granting these, Avi's tool calls inherit the permissions and no further popups appear.

Note: On macOS Tahoe 26+, TCC permissions cannot be granted programmatically without MDM enrollment. The one-time interactive setup is required.

Auto-Start on Boot

For packaged macOS apps, the preferred path is the in-app Launch Avi at login setting in Avi's user settings. This supports both Avi and Avi Beta and is disabled for local dev runs.

If you need a manual fallback for an installed app in /Applications, use a per-user LaunchAgent. Do not use these instructions for npm run dev or other unpackaged local runs.

To start Avi automatically when you sign in on macOS:

1. Open System Settings → General → Login Items
2. Add the Avi app
3. Alternatively, create a LaunchAgent:

cat > ~/Library/LaunchAgents/com.avi.agent.plist << 'EOF'
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.avi.agent</string>
    <key>ProgramArguments</key>
    <array>
        <string>/Applications/Avi.app/Contents/MacOS/Avi</string>
    </array>
    <key>RunAtLoad</key>
    <true/>
    <key>KeepAlive</key>
    <true/>
    <key>ProcessType</key>
    <string>Interactive</string>
</dict>
</plist>
EOF
launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.avi.agent.plist
launchctl enable gui/$(id -u)/com.avi.agent
launchctl kickstart -k gui/$(id -u)/com.avi.agent

For Avi Beta, update ProgramArguments to:

<string>/Applications/Avi Beta.app/Contents/MacOS/Avi Beta</string>

Notes:

• LaunchAgents run in the logged-in user session and are appropriate for Avi's tray/window behavior.
• LaunchDaemons are not supported for Avi's current Electron architecture.
• Avoid enabling both the built-in login item and a manual LaunchAgent unless you intentionally want redundant startup mechanisms.

Recommended System Settings

• Energy Saver → Prevent automatic sleeping — keep the machine awake so scheduled tasks and remote connections work
• Sharing → Screen Sharing — enable for occasional maintenance via VNC/Screen Sharing
• Automatic login — enable for the user account running Avi so LaunchAgents start on boot without requiring a password