By James Aspinwall, co-written by Alfred Pennyworth (my trusted AI) — March 1, 2026, 14:30
This is a comprehensive tour of every feature built into the WorkingAgents MCP application — an Elixir OTP system that serves as both an AI integration platform and a personal productivity suite. What started as an MCP server experiment has grown into a full-stack application with CRM, task management, messaging, monitoring, and multi-provider LLM chat, all unified by a single permission system and accessible through four interfaces: web UI, REST API, MCP tools, and IEx console.
Architecture at a Glance
The application runs on Elixir/OTP with a supervision tree managing ~25 processes. HTTPS on port 8443 (Bandit + Plug) handles web traffic. SQLite (via Exqlite) provides persistence across 12+ database instances. The MCP protocol layer (Hermes) exposes ~80 tools to AI agents. Everything is permission-gated through a custom AccessControl system.
Key design principles:
- Separate process management from business logic — Server modules handle GenServer lifecycle, functional modules contain pure logic
- Modules own their data — each module manages its own Sqler instance, permission key, and process lifecycle
- Permissions in logic layer, not transport — Web, REST, and MCP all pass permissions down; the business logic decides access
1. Authentication & User Management
Two authentication flows serve different clients:
Cookie-based sessions for browsers — email/password login with Argon2 hashing, encrypted cookies, account lockout after 5 failed attempts (15-minute cooldown), audit logging of every login event.
OAuth 2.1 / PKCE for MCP clients — full authorization code flow with dynamic client registration (RFC 7591), bearer token issuance, and PKCE challenge verification. MCP clients like Claude Code authenticate through this flow.
User accounts track login history, failed attempts, password change timestamps, and support metadata extensions via a JSON column. Two-factor authentication fields exist in the schema for future activation.
2. Access Control — Roles, Permissions, Delegation
The heart of the security model. AccessControl is a singleton GenServer managing integer-based permission keys, role definitions, and an audit trail — all encrypted at rest with AES-256-CTR.
Permission keys are simple integers mapped to domains:
| Key | Domain |
|---|---|
| 10,001 | Platform (basic access) |
| 20,001 | |
| 30,001 | Utility |
| 40,001 | Summaries |
| 50,001 | Blogs |
| 60,001 | Admin |
| 70,001 | Chat |
| 80,001 | NIS (CRM + Tasks) |
| 90,001 | Monitor |
| 100,001 | YouTube |
Roles bundle keys into named sets (e.g., “viewer” = platform + summaries + blogs). Roles can be assigned/revoked per user. Temporary keys support TTL-based grants — useful for time-limited access. Permission delegation lets any user share a subset of their own keys to another user temporarily.
Every grant, revoke, and role change is logged to an audit trail. The web UI at /access-control provides full management: role CRUD, per-user permission views, and audit log browsing.
The AccessControlled macro generates boilerplate for any module that needs permission gating — define @permission, @module_name, @module_description, then use AccessControlled to get allowed?/1, permission_key/0, and automatic registration with the admin system.
3. MCP Protocol Server — ~80 Tools
The MCP server implements the Hermes.Server behaviour, exposing tools and resources over JSON-RPC 2.0 with SSE streaming. Tools are filtered per-user based on their permission keys — you only see tools you’re authorized to use.
Time-sensitive tools (task creation, scheduling, follow-ups) automatically inject the user’s timezone, local time, and Unix timestamp into their descriptions so the LLM can correctly interpret relative dates like “tomorrow” or “next Monday.”
Tool domains span every module in the system: platform utilities, WhatsApp messaging, task management, CRM operations, article summarization, blog search, system monitoring, access control administration, YouTube subscriptions, push notifications, TTS synthesis, and file operations.
4. LLM Chat with Tool Integration
Per-user chat sessions backed by three pluggable LLM providers:
- Anthropic — Claude models via the Messages API with full tool-use loop
- OpenRouter — any model available on the OpenRouter marketplace via OpenAI-compatible API
- Perplexity — web-grounded responses with citations via the Responses API
Each user gets their own ServerChat GenServer with conversation history persisted to SQLite. Sessions idle-timeout after 30 minutes. Providers are switchable at runtime without clearing history. The chat session discovers MCP tools dynamically and can call them on behalf of the user during conversation.
The web UI at /chat provides a clean conversation interface. The REST API supports programmatic access for external integrations.
5. NIS — Personal CRM
A complete customer relationship management system, per-user, with its own SQLite database.
Six entity types:
- Contacts — full contact cards with background, location, contact medium, birthday, priority, stage, cadence tracking, interests, and tags
- Companies — organization records with deal value, expected close date, pipeline stage
- Websites — URL tracking linked to contacts/companies
- Activities — recurring or one-time actions linked to any entity
- Notes — timestamped notes attachable to any entity, with channel and reason tracking
- Tasks — full task management linked to CRM entities (see next section)
Key workflows: pipeline view for staged deals, due follow-ups based on contact cadence, birthday tracking, bulk stage updates, full-text search across all entities via FTS5, interaction logging, and CRM statistics.
The web UI at /nis is a full single-page app with a dark theme. REST API provides complete CRUD plus specialized endpoints for search, pipeline, follow-ups, and bulk operations.
6. Task Management
A task system with recurrence, natural language capture, subtask hierarchy, and 60+ query functions across 11 categories.
Task fields: title, priority (integer), note, deferred-until time, expiration, completion timestamp, waiting-for text, status (todo/in_progress/blocked/completed), due date, tags (JSON array), parent ID for subtask hierarchy, and recurrence pattern.
Recurrence engine handles: daily, weekly, monthly, weekdays, every N days, every N weeks, and custom day-of-week patterns. Completing a recurring task automatically spawns the next instance.
Natural language capture parses plain text like “review PR tomorrow #work p2” into structured tasks, extracting tags, due dates, and priority hints.
60+ named queries organized into: status filters, due date ranges, waiting/expiry, priority ordering, completion analytics, tag grouping, subtask navigation, blocking analysis, dashboard aggregation, and data integrity checks.
The web UI provides a dashboard, task forms, detail views, and a query browser. Tasks integrate with the NIS CRM via entity linking — associate any task with contacts, companies, activities, or websites.
7. Article Summaries
An article summarization pipeline: submit a URL, get back a ~100-word summary generated by your choice of LLM provider.
URLs arrive via browser extension or MCP tool. The system fetches content, sends it to the configured provider (Anthropic, OpenRouter, or Perplexity — switchable at runtime), and stores the result. YouTube URLs are stored with metadata but not summarized.
Two search modes:
-
Semantic vector search — VoyageAI embeddings (1024-dimension,
voyage-3model), cosine similarity ranking - Full-text search — SQLite FTS5 keyword matching
Backfill endpoints let you embed or index existing summaries that predate the search features.
8. Blog System
A dual-mode blog platform: file-based authoring with a WYSIWYG editor, plus a SQLite-backed vector store for semantic search.
File-based blog: Markdown and HTML files in asset/blogs/, edited via a Summernote WYSIWYG editor in the browser, with a Medium-style reader view supporting dark/light theme toggle. Blog posts are publicly readable without authentication.
Blog vector store: BlogStore imports blog files into SQLite, chunks the text for embedding (target 2000 characters, 3-sentence overlap), and indexes via both vector search (vec0 extension, 1024-dimension cosine) and FTS5 full-text search. A file watcher triggers re-import on save.
9. System Monitor
Real-time BEAM VM monitoring with anomaly detection.
Metrics collected: uptime, OTP/Elixir versions, memory breakdown (total, processes, ETS, binary, atoms, code), process/atom/port counts and limits, scheduler utilization, I/O throughput, per-database health (table/row counts), service status for all registered GenServers, and task statistics.
MonitorServer wraps this in a GenServer with configurable polling intervals, a 60-snapshot circular buffer, and threshold-based anomaly detection (memory, OS memory, process/atom/port utilization percentages).
The web UI at /monitor is a dark-themed dashboard with auto-refresh and resource utilization charts.
10. WhatsApp Integration
Full WhatsApp messaging via a Node.js bridge managed by MuonTrap, communicating over ZeroMQ.
Capabilities: send/receive text and media (image, video, audio, document), contact and group listing, group creation, message history retrieval, and recent message polling with JID/since/limit filters.
WhatsAppClaude orchestrator bridges incoming messages to an LLM for auto-response. Supports four modes: Anthropic API direct, Google Gemini API, Gemini CLI, and Claude CLI. Per-JID conversation history, debounced message handling (3-second batching), and full MCP tool access — the AI can query your CRM, check tasks, or look up information while composing a reply.
Important constraints: IP-gated to local network (192.168.*), messages to different contacts spaced by 15+ seconds to avoid rate limits, and never sends identical text to multiple contacts (WhatsApp spam detection).
11. Text-to-Speech
ElevenLabs TTS integration via the eleven_multilingual_v2 model. Tts.speak(text, opts) synthesizes speech and saves MP3 files to asset/t2s/. Configurable voice, speed, stability, similarity, and style parameters. Max 4096 characters per request with retry logic. Available as the MCP tool tts_speak.
12. YouTube Integration
YouTube Data API v3 client using OAuth2 refresh-token flow. YouTube.list_subscriptions() fetches all subscribed channels with automatic pagination (YouTube caps at 50 per page). Returns channel ID, title, description, thumbnail URL, and subscription date. Available as the MCP tool youtube_subscriptions.
13. Push Notifications
Pushover API integration for mobile push notifications. Send immediately or schedule via natural language time parsing (“tomorrow at 9am”, “in 2 hours”). Integrated with the Alarm system — scheduled alarms fire through Pushover automatically.
14. Alarm & Scheduling
Persistent alarm scheduling with crash recovery. Natural language time parsing via Chronic, SQLite-backed persistence, and automatic restoration of pending alarms on restart. The Timer process manages Process.send_after references and fires messages to target GenServers at the scheduled time. Soft-delete pattern preserves history — alarms are marked completed or cancelled, never hard-deleted.
15. Database Layer
SQLite everywhere, via Sqler — a GenServer wrapper with prepared statement caching, auto-generated millisecond timestamp IDs, and optimistic locking.
12+ database instances serve different domains: users, chat history, tasks, summaries, blogs, compile logs, scheduler entries, A2A tasks, page scraper extractions, per-user NIS databases, alarms, and general-purpose term storage.
Vector search via SQLite’s vec0 extension — 1024-dimension float embeddings with cosine distance for blog chunks and article summaries.
Full-text search via SQLite FTS5 — keyword matching across blog content, NIS entities (contacts, companies, websites), and article summaries.
VoyageAI embeddings power the vector search — the voyage-3 model produces 1024-dimension embeddings, stored as binary float arrays in SQLite.
16. A2A Protocol
Google’s Agent-to-Agent protocol implementation. JSON-RPC 2.0 endpoints for tasks/send, tasks/get, and tasks/cancel. Agent card discovery at /.well-known/agent.json. A2A tasks persist to SQLite with a state machine (submitted → working → completed/canceled). The A2A server can execute MCP tools in the context of an A2A task, enabling agent interoperability.
17. Browser DOM Extraction (PageScraper)
Server-driven extraction from Chrome browser pages. URL pattern matching triggers extraction when you visit matching pages. The server sends DOM commands (query, click, wait, watch) to a Chrome content script via WebSocket, and the content script returns results.
Extraction types: snapshot (on page arrival), poll (periodic re-query), sequence (multi-step workflows), and mutation (DOM change watching).
A GitHub PR handler demonstrates the pattern — visiting a GitHub PR page automatically extracts title, state, author, body, file count, and commit count.
18. WebSocket & Real-time
WsRegistry manages per-user WebSocket connections for two process types: WsHandler (data channel) and WsPage (page rendering). WsHandler.rpc(username, fn_name, args) sends RPC to connected users. Used by PageScraper for DOM commands and by AutoCompile for browser reload notifications.
19. Developer Experience
Auto-compilation watches lib/ for file changes and hot-reloads .ex files without restarting. JS file changes trigger WebSocket-based browser reload. Compilation events are logged with git diffs for change tracking.
12 IEx manuals cover every module with copy-paste examples: User, NIS, Tasks, Monitor, WhatsApp, Summaries, Blogs, AccessControl, Platform (TTS/Pushover), YouTube, Chat, and Utility.
DocAudit tool scans lib/ for documentation coverage: @moduledoc presence, public function doc coverage, and whether companion .md files exist in asset/docs/.
Four access methods for every feature: web UI, REST API, MCP tools, and IEx console. Each interface passes user permissions down to the same business logic layer.
The Numbers
| Metric | Count |
|---|---|
| MCP tools | ~80 |
| SQLite databases | 12+ |
| Permission domains | 10 |
| Task query functions | 60+ |
| LLM providers | 3 (Anthropic, OpenRouter, Perplexity) |
| IEx manuals | 12 |
| External integrations | 7 (WhatsApp, YouTube, ElevenLabs, Pushover, VoyageAI, Google A2A, ZeroMQ/Python) |
| Web UI pages | 15+ |
| REST API endpoint groups | 8 |
What’s Next
Near-term priorities include fixing WebSocket security for external access (auth tokens currently in URL query params), implementing permission cache invalidation (push notification on grant/revoke), and wiring up the dormant SsePush module for MCP server-to-client notifications.
The foundation is solid — every feature shares the same permission model, the same database patterns, and the same four-interface access strategy. Adding a new domain module means defining a permission key, writing the business logic, creating a Permissions.* wrapper, and wiring it into the MCP server. The AccessControlled macro and Sqler GenServer handle the rest.
Built with Elixir, OTP, SQLite, and a healthy disregard for sleep.