Varolio DocsApplication Architecture
The services, data stores, and AI providers that make up the Varolio platform
Application architecture
Varolio is a TypeScript monorepo deployed as a set of cooperating services on Kubernetes, with a managed data layer and an abstracted AI surface. The same components run in every deployment scenario — only their placement changes.
Amazon EKS cluster
Services
| Service | Role |
|---|---|
| Studio | Main user web application (Next.js), served at app.varolio.io |
| Engine | API gateway (NestJS): terminates all HTTP/tRPC traffic, authenticates requests, and coordinates the other services |
| Agent | LLM orchestration (NestJS): runs the dialogue layer, case resolution, extraction/enrichment, and external API plugins |
| Worker | Temporal worker executing asynchronous workflows (email processing, case lifecycle, alerts) |
| MCP Server | Model Context Protocol server exposing workspace data to AI tools, served at mcp.varolio.io — see the MCP docs |
The backend services — Engine, Agent, Worker, and MCP Server — each run as a Kubernetes workload in their own namespace on the Amazon EKS cluster. The separation is deliberate: the Engine is the only service with access to the control plane (Supabase), the Agent is the only service that talks to the retrieval layer (Elasticsearch) and is reached exclusively through the Engine, and the Worker executes Temporal workflows.
Edge and ingress
The public API surface sits behind AWS WAF — AWS managed rule groups (IP reputation, common rule set, known bad inputs, SQLi) plus per-IP rate limiting with blocked-request alarms — in front of an internet-facing Application Load Balancer that enforces TLS 1.2+ with an ACM certificate. Traffic is then routed through an Istio ingress gateway to the engine inside the cluster's private subnets.
Data layer
Operational data stores run as externally managed services — Supabase, Redis, and Elastic Cloud — while object storage (Amazon S3) lives in the Varolio AWS account. Workflow orchestration is handled by Temporal Cloud, also externally managed.
| Store | Purpose |
|---|---|
| Supabase (PostgreSQL) | Operational database: accounts, workspaces, configuration, workflow state |
| Elasticsearch (Elastic Cloud) | Retrieval layer: threads, messages, cases, knowledge index, embeddings — an Elastic-hosted deployment |
| Amazon S3 | Large objects: message content and attachments, encrypted at rest (SSE-S3) and namespaced per workspace — in the Varolio AWS account |
| Google Cloud Storage | Attachment replicas and meeting media |
| Redis | Cache and application-level rate limiting — externally managed |
All stores are encrypted at rest (AES-256) and workspace-scoped at the application layer — see Security.
Workflow system
Inbound events (mail webhooks, meeting artifacts, messages) enter through the engine and are executed as Temporal workflows by the worker, orchestrated by Temporal Cloud (externally managed). Workflows are composed of nodes (function executions) connected by edges, with templated arguments evaluated against previous node outputs. Integrations (Microsoft Graph, Gmail, WhatsApp, CRM systems) are implemented as plugins with per-tenant OAuth connectors.
AI providers
All model access goes through a single dialogue abstraction that separates the LLM interface from the retrieval interface — which is also what makes the BYO Data Store model possible.
| Provider | Use |
|---|---|
| Google Vertex AI (Gemini) | Primary LLM, served from europe-west3 (Frankfurt) in the default deployment |
| Amazon Bedrock (Anthropic Claude) | Secondary / fallback LLM |
| OpenAI | Supported provider via the same dialogue abstraction |
| Google Document AI | Attachment OCR |
Channel and integration sub-processors
| Provider | Use |
|---|---|
| Microsoft 365 / Graph API | Outlook mailboxes: email ingestion via webhooks — read-only access (Mail.Read), customer-authorized OAuth |
| Google Gmail | Gmail mailboxes — customer-authorized OAuth |
| Recall.ai | Varolio Notetaker: meeting recording and artifacts, delivered via Svix-signed webhooks |
| Meta WhatsApp Cloud API | WhatsApp channel: inbound webhooks (HMAC-verified) |
Identity and access
Users authenticate through the customer's identity provider via OIDC SSO (for example Microsoft Entra ID). API requests carry JWTs with account-, workspace-, and inbox-scoped RBAC. Integration credentials are stored per tenant in AWS Secrets Manager and synced into the cluster with the Secrets Store CSI driver.