vbot-cf — Cloudflare-Native Voice Bot

Facts

Fact	Value
Status	dormant
Hosting platform	cloudflare
Repository	https://github.com/meta4m/vbot-cf
Tech stack	language: typescript, runtime: cloudflare-workers, framework: cloudflare-agents, voice_sdk: cloudflare-voice, dashboard: nextjs, ui: react, database: cloudflare-d1, vector_database: cloudflare-vectorize, object_storage: cloudflare-r2, queue: cloudflare-queues, ai: workers-ai, package_manager: pnpm, build_tool: vite, testing: vitest
Tags	voice, voice-bot, cloudflare, multi-tenant, multilingual, agents, rag, byok, durable-objects, websocket
Last audited	2026-09-15
Upstream dependencies	None declared
Downstream dependencies	None declared

Secrets manifest

Key names and metadata only. Encrypted records are available in this project’s vault below.

Key	Description	Environments	Required
`ENCRYPTION_SECRET`	AES-256-GCM master secret (openssl rand -base64 32) used to encrypt/decrypt per-bot BYOK provider API keys at rest in the D1 api_keys table	production, development	Yes
`ADMIN_API_KEY`	Shared key checked against the X-Admin-Key header to authorize the /api/* admin REST surface (bot CRUD, keys, rules, knowledge, analytics)	production, development	Yes
`CLOUDFLARE_API_TOKEN`	Cloudflare API token used by the GitHub Actions CI/CD workflow for D1 migrations, Worker deploys (dev + production envs), and Pages deploys of the vbot-admin dashboard	production, development	Yes
`OPENAI_API_KEY`	Optional BYOK OpenAI key for STT/LLM stages during local development, listed in packages/core/.dev.vars.example	development	No
`GROQ_API_KEY`	Optional BYOK Groq key for the LLM stage during local development, listed in packages/core/.dev.vars.example	development	No
`DEEPGRAM_API_KEY`	Optional BYOK Deepgram key for the STT stage during local development, listed in packages/core/.dev.vars.example	development	No
`ELEVENLABS_API_KEY`	Optional BYOK ElevenLabs key for the TTS stage during local development, listed in packages/core/.dev.vars.example	development	No

Overview

Project Overview: vbot-cf

1. Executive Summary

Mission: A multi-tenant, multi-lingual voice bot built entirely on Cloudflare — a Cloudflare Agents VBotAgent Durable Object streaming voice over WebSockets (STT → LLM → TTS with barge-in), with per-bot BYOK providers, RAG knowledge bases, canned-response rules, jailbreak filtering, tool calling, and an embeddable widget for customer sites. main’s README describes it as “based entirely on CloudFare infrastructure and APIs”; plan.md targets a $0 free-tier MVP.
Target Audience: Site owners wanting a voice assistant embedded on their page (script-tag widget, themeable, Urdu/English), with the operator as fleet admin via a Next.js dashboard.
Primary Maintainer: The operator (meta4m) — solo developer; authored all 14 commits in a 3-day sprint burst (2026-05-28 → 2026-05-30).

2. Current Status & Roadmap

Lifecycle State: dormant — all code lives on branch dev (sprints 1–3.5: CI/CD, REST API, dashboard rewrite, canned responses + RAG + knowledge UI, BYOK routing, language detection, widget polish); last commit 2026-05-30 with nothing since (~3.5 months). Branch main holds only the 2-line README, so the production CI path (push to main/master) currently has no code to deploy.
Ongoing work: none evidenced — plan.md is worked through sprint 3.5 and the repo has no issues or TODOs. Re-audit if development resumes.

3. Core Capabilities

Voice + text agent: VBotAgent extends the Agents SDK Agent via the @cloudflare/voice withVoice mixin — binary 16kHz PCM WebSocket audio, continuous STT, streamed TTS playback, barge-in/VAD, plus text chat; conversation history persisted in DO SQLite.
Per-bot pipeline with BYOK: each bot config picks provider + model per stage — defaults all-Cloudflare (Whisper large-v3-turbo STT, Llama-3.1-8B LLM, MMS TTS via Workers AI), overridable with OpenAI / Groq / Deepgram / ElevenLabs keys stored encrypted (AES-256-GCM under ENCRYPTION_SECRET) in D1.
RAG knowledge: dashboard upload with client-side PDF extraction (PDF.js) → sentence-aware chunking → Workers AI embeddings → per-bot Vectorize namespace; originals archived in R2; enrichWithRAG injects context pre-LLM.
Safety pipeline: jailbreak detection (block patterns/languages, sensitivity) and pre/post-LLM canned-response rules with hit counters, seeded with defaults on bot creation.
Agent tools: sendEmail (Gmail API), checkCalendar (Google Calendar API), CRM lookup, transferToHuman — credentials passed per-bot via tool config.
Multi-tenant admin: /api/* REST (bots, keys, rules, knowledge, analytics) behind X-Admin-Key; Next.js static dashboard on Cloudflare Pages; analytics from sessions_log / transcripts (total calls, avg duration, tool calls, transcript viewer).
Embeddable widget: self-contained /embed?bot=… HTML served by the Worker (theme/avatar via query params, Urdu RTL + Noto Nastaliq rendering) plus loader.js script tag from @vbot/embed.

4. Key References

Repository: meta4m/vbot-cf — code on branch dev; main is a README-only initial commit.
Production URLs: dev Worker https://vbot-cf-dev.irfan-064.workers.dev (dashboard’s default API base); dashboard https://vbot-admin.pages.dev (CI environment URL). No production Worker URL is stated in the repo — CI smoke-tests a VBOT_PROD_URL variable.
Architecture Docs: architecture.md · Deployment Guide: deployment.md

Architecture

Architecture & System Design: vbot-cf

1. System Topology

graph TD
    Visitor["Visitor on customer site"] -->|iframe / script tag| Embed["/embed widget + loader.js<br/>packages/embed"]
    Embed -->|"WebSocket · 16kHz PCM / text"| Agent["VBotAgent Durable Object<br/>agents SDK + @cloudflare/voice"]
    Admin["Operator → vbot-admin dashboard<br/>Cloudflare Pages (Next.js static)"] -->|"X-Admin-Key"| API["/api/* admin REST · api.ts"]
    API --> D1[("D1: bot_configs · api_keys<br/>sessions_log · transcripts · canned rules")]
    Agent --> D1
    Agent -->|"default STT/LLM/TTS"| WAI["Workers AI<br/>whisper-large-v3-turbo · llama-3.1-8b · mms-tts"]
    Agent -->|"BYOK stages"| Ext["OpenAI · Groq · Deepgram · ElevenLabs"]
    Agent --> Vec[("Vectorize vbot-cf-knowledge<br/>per-bot RAG namespaces")]
    API -->|"ingest originals"| R2[("R2 vbot-cf · bots/<id>/documents/")]
    Agent --> Q[("Queue vbot-cf-post-call")]

2. Core Components & Subsystems

packages/core — Worker vbot-cf (src/index.ts): routes /health, /embed (server-rendered widget HTML with query-param theme injection), /api/*, and hands the rest to routeAgentRequest (Agents SDK WebSocket routing to /agents/VBotAgent/:id).
VBotAgent Durable Object (src/agent.ts): withVoice(Agent) base; WorkersAIFluxSTT + WorkersAITTS defaults, per-bot overrides; pipeline hooks — pre-LLM jailbreak check and canned rules, RAG enrichment, post-LLM rules; tool calling via @vbot/tools; session logging to D1; in-code fallback config when D1 is unavailable locally.
Provider abstraction (src/providers/): factory.ts plus adapters (cf-whisper, cf-llama, cf-mms-tts, openai, groq, deepgram, elevenlabs); BYOK keys resolved from D1 api_keys (by key id, or bot + provider fallback) and decrypted with ENCRYPTION_SECRET (config/crypto.ts: AES-256-GCM, SHA-256 key derivation, IV-prefixed ciphertext).
RAG subsystem (src/rag/): sentence-aware chunking with token overlap, Workers AI embeddings, Vectorize upsert/search per bot namespace (kb-<botId8>).
Admin REST API (src/api.ts): bot CRUD, key add/delete, canned-rules CRUD, knowledge ingest (multipart text + file) and search, analytics summary + sessions, dynamic provider model listing (OpenAI-compatible /models with per-provider parsing and stage filters).
packages/dashboard: Next.js 15 static export (output: 'export' → out/), create/manage pages, analytics, PDF.js client-side extraction; talks to the Worker with NEXT_PUBLIC_API_URL + NEXT_PUBLIC_ADMIN_KEY.
Supporting packages: @vbot/embed (script-tag loader with data-vbot-* attributes), @vbot/client-sdk (VBotClient), @vbot/shared (types/utils), @vbot/tools (Gmail/Calendar/CRM/transfer).

3. Data Flow & Storage Contracts

D1 (local vbot-cf-db, remote vbot-db-dev / vbot-db-prod): bot_configs (JSON config per bot), api_keys (per-bot provider/stage keys), sessions_log + transcripts (analytics), canned rules — migrations 0001–0004, applied by CI before each deploy.
Vectorize vbot-cf-knowledge: one namespace per bot; vector metadata carries botId/docId/title/chunkIndex.
R2 vbot-cf: original uploaded documents at bots/<botId>/documents/<docId>/<filename>.
DO SQLite: conversation history per agent instance (agents addressed by bot id).
Caveat (flagged): POST /api/bots/:id/keys writes body.key directly into api_keys.encrypted_key, and no client in the repo performs the encryption — callers must pre-encrypt with ENCRYPTION_SECRET or plaintext lands in a column named encrypted_key.

4. Architectural Decision Records (ADRs)

ADR-01: Cloudflare Agents as the entire runtime backbone

Context: A voice bot needs bidirectional streaming, VAD/barge-in, state, and tool calling; plan.md targets a $0 free-tier MVP.
Decision: build on the agents SDK + @cloudflare/voice mixin over a SQLite-backed Durable Object; every other service (D1, Vectorize, R2, Queues, Workers AI) is a Cloudflare binding.
Consequences: near-zero infra cost and one vendor; beta-stage SDKs caused several sprint-3 WebSocket/message-parsing fixes, and non-Cloudflare deployment is effectively impossible.

ADR-02: BYOK per stage with keys encrypted at rest in D1

Context: A multi-tenant bot shouldn’t hold shared provider keys; free Workers AI quotas are the fallback.
Decision: defaults run on Workers AI; each bot overrides STT/LLM/TTS with its own provider + model + key, keys AES-256-GCM-encrypted with ENCRYPTION_SECRET and decrypted at call time.
Consequences: per-tenant cost isolation and provider choice; adds a key-management surface and the pre-encryption caveat above.

ADR-03: Safety and canned responses as pipeline middleware

Decision: pre-LLM (jailbreak detection, canned matches) and post-LLM (response augmentation) hooks in the voice pipeline rather than prompt-only guards; rules live in D1, dashboard-editable, seeded on bot creation.
Consequences: deterministic safety and instant answers with hit-count analytics; one more moving part per turn.

ADR-04: Static Next.js dashboard on Pages, not a Worker route

Decision: the admin dashboard is a static export shipped to Cloudflare Pages (vbot-admin), calling the Worker’s REST API with a build-time admin key.
Consequences: simple CI artifact deploys and per-branch previews; no server-side auth — the admin key ships in the client bundle, acceptable only for a single-operator admin tool.

Deployment

Deployment & Operational Runbook: vbot-cf

1. Hosting & Environments

Environment	Platform	URL / Endpoint	Auto-Deploy Branch
Dev (Worker)	Cloudflare Workers, env `dev` (`vbot-cf-dev`)	`https://vbot-cf-dev.irfan-064.workers.dev` (dashboard default API base)	`dev` (push → CI)
Dev (Dashboard)	Cloudflare Pages `vbot-admin`, dev-branch preview	preview URL per deploy	`dev` (push → CI)
Production (Worker)	Cloudflare Workers, env `production` (`vbot-cf`)	not stated in repo — CI smoke-tests a `VBOT_PROD_URL` variable	`master` / `main` (push → CI)
Production (Dashboard)	Cloudflare Pages `vbot-admin`	`https://vbot-admin.pages.dev`	`master` / `main` (push → CI)
Local dev	`wrangler dev` (local D1 SQLite + `.dev.vars`)	`http://localhost:8787`	N/A

Caveat: main contains only the README, so the production CI path has no code to deploy — a real production deploy today requires a manual pnpm deploy:prod from a dev checkout (or merging dev → main, which has never happened).

2. Prerequisites & Toolchain

Node.js ≥ 22, pnpm 9.15.4 (pinned via packageManager; CI uses pnpm/action-setup), Git.
Wrangler v4 (devDependency) and a Cloudflare account owning: Workers vbot-cf / vbot-cf-dev, D1 vbot-db-dev (id c4ec5640-…) / vbot-db-prod (id 6cf9bbd8-…), Vectorize index vbot-cf-knowledge, R2 bucket vbot-cf, Queue vbot-cf-post-call, Pages project vbot-admin.

3. Local Development Setup

git clone https://github.com/meta4m/vbot-cf.git && cd vbot-cf
git checkout dev                                    # all code lives on dev
corepack enable && pnpm install
cp packages/core/.dev.vars.example packages/core/.dev.vars   # set ENCRYPTION_SECRET
pnpm dev                              # @vbot/core → wrangler dev (local D1)
pnpm --filter @vbot/dashboard dev     # dashboard → next dev
pnpm test                             # vitest across packages

4. Environment & Secrets Provisioning

Refer to spec.yml for the full manifest (ENCRYPTION_SECRET, ADMIN_API_KEY, CLOUDFLARE_API_TOKEN, optional BYOK keys). Retrieve ciphertext from 2Brain vault and decrypt locally into .env.local.

Worker runtime: ENCRYPTION_SECRET (wrangler secret per env; .dev.vars locally) and ADMIN_API_KEY (a dev placeholder ships in wrangler.toml [vars] — must be rotated for any real deployment).
Dashboard build: NEXT_PUBLIC_API_URL (defaults to the dev Worker) and NEXT_PUBLIC_ADMIN_KEY (must match ADMIN_API_KEY; the repo default is the public dev placeholder).
CI (GitHub environment CloudFlare): secrets CLOUDFLARE_API_TOKEN + CLOUDFLARE_ACCOUNT_ID (the latter is a config value, excluded from the manifest), plus environment vars VBOT_DEV_URL / VBOT_PROD_URL for smoke tests.

5. Deployment Commands

pnpm deploy:dev        # wrangler deploy --env dev
pnpm deploy:prod       # wrangler deploy --env production
pnpm db:migrate:dev    # wrangler d1 migrations apply vbot-db-dev --remote --env dev
pnpm db:migrate:prod   # wrangler d1 migrations apply vbot-db-prod --remote --env production
pnpm pages:dev         # build dashboard → Pages deploy, project vbot-admin, branch dev
pnpm pages:prod        # same, branch main

CI (.github/workflows/deploy.yml) runs typecheck → tests → build assets → D1 migrations → Worker deploy → Pages deploy → /health smoke test on every push to dev / master / main.

6. Verification & Health Checks

Worker: GET /health → {"status":"ok"} (CI smoke test retries 5× with 5 s sleeps).
Widget: open https://<worker>/embed?bot=<id> — connection status, mic permission, interim transcripts, voice round-trip.
Dashboard: bot list renders; create a bot, upload a knowledge doc, check analytics counters increment.

7. Incident Response & Rollback

Rollback: redeploy the previous commit (git checkout <sha> && pnpm deploy:dev); Pages keeps per-branch previews and deployment history for one-click rollback in the Cloudflare dashboard.
Bad bot config: PUT /api/bots/:id with corrected JSON, or delete and recreate (canned rules re-seed automatically).
Key exposure: DELETE /api/bots/:id/keys/:keyId, rotate at the provider, re-add pre-encrypted; rotating ENCRYPTION_SECRET invalidates every stored api_keys.encrypted_key value.
Known gaps to watch: the admin-key placeholder is public in the repo (rotate before any real use); the keys API expects pre-encrypted values (see architecture.md §3).

Project vault

Cloudflare Access authorization and your passphrase are required to retrieve and decrypt stored values.

Client-Side Encrypted Secret Vault

Zero-knowledge architecture. Secrets are encrypted client-side using Web Crypto AES-256-GCM before leaving this browser tab. Edge databases store only ciphertext, IV, and salt.

Vault Passphrase (PBKDF2-SHA256, 100,000 iterations):

Encrypted Records for `vbot-cf` (0)

Encrypted records have not been loaded.