Agent Service Architecture — Per-Client Sandboxed Deployments

Author: Brad (Hermes Agent) Date: 2026-05-04 Status: Proposal — for founder review Cross-ref: ARCHITECTURE.md, TECHNICAL_SCOPES.md, docs/features/agentic-tasks.md

⚠️ Runtime Note (2026-05-07, updated 2026-05-12): The current production runtime is Hermes CLI (agent/hermes_client.py). POST /v1/chat shells out to the hermes chat CLI; LangGraph rollback and /graph/invoke have been removed and deferred. The per-client sandboxed deployment model described in this doc is the target architecture (Phase 2+), not the current state. See IMPLEMENTATION_STATUS.md § 15 for current AI agent runtime status.

0. Design Constraints (non-negotiable)

No database access. Agent instances never connect to PostgreSQL, Redis, or any platform data store. Zero connection strings, zero query capability.
API-only communication. Agents fetch all context they need from the humanwork Platform API using a scoped auth token. The token restricts the agent to a single org — it is structurally impossible for an agent to read another org's data.
Per-client isolation. Each client org gets 1–N dedicated agent instances. Agents are never shared across orgs. Each runs in its own sandbox (container/pod).
Data security is paramount. Even though these are internal systems, the architecture must enforce segregation at every layer — not rely on "trust" or "we won't do that."
Audit everything. Every agent action (LLM call, tool execution, callback) must be traceable for diagnosis and compliance.

1. Problem Statement

Today the agent is a single shared FastAPI service with direct API calls using env-var credentials. This violates all five constraints above:

One process serves all orgs — no isolation
Agent reads Shopify/Amazon credentials from env vars — credentials for all orgs in one process
Agent calls GET /memory/search on the platform but nothing prevents an agent bug from querying the wrong org
No audit trail of what the agent did or why

2. Use Case Enumeration

Every interaction between the platform and an AI agent, extracted from the codebase:

2.1 Synchronous Request→Response (Platform → Agent)

#	Use Case	Description
UC-1	Draft a reply	Client sends message. Platform calls agent with message + history + specialist persona. Agent returns `{reply, confidence, risk_level, flag_for_review}`. Platform routes to expert queue or auto-sends.
UC-2	Stream a reply	Same as UC-1, SSE streaming. Frontend renders tokens as they arrive.
UC-3	Classify task mode	Agent-internal: decides "domain" (bounded tool call) vs "agentic" (multi-step plan).
UC-4	Score confidence	Agent-internal: second LLM call to self-rate confidence 1–10 → 0.0–1.0.
UC-5	Classify risk	Agent-internal: LOW/MEDIUM/HIGH/CRITICAL. Platform uses for routing.

2.2 Tool Execution (Agent → External APIs, via Platform)

#	Use Case	Current State	Target State
UC-6	Query Shopify	Agent reads env vars, calls Shopify directly	Agent calls Platform API proxy: `GET /v1/agent-api/tools/shopify?action=...` — platform resolves credentials from encrypted store
UC-7	Query Amazon	Same — direct API call	Same pattern — platform proxy
UC-8	Browse web	Playwright in agent container	Stays in agent — no credential issue
UC-9	Retrieve memory/context	Agent calls `GET /memory/search`	Platform retrieves via RAGFlow and exposes `GET /v1/agent-api/context?query=...` for org-scoped results only
UC-10	Finance/IR tools	Function signatures only (stubs)	Same proxy pattern when implemented

Key change: Agents never hold third-party credentials. They call the Platform API, which resolves credentials from the encrypted store (IntegrationCredential entity, AES-256-GCM) and proxies the request. The agent gets results, never keys.

2.3 Agentic Tasks — Multi-Step with HITL Gates (Bidirectional)

#	Use Case	Description
UC-11	Agent proposes a plan	Agent decomposes complex request into steps with risk levels. Pushes `task.plan_ready` event to platform.
UC-12	Expert approves/rejects step	Expert reviews in UI. Platform sends `POST /v1/tasks/`{id}`/steps/`{n}`/go` or `/no` to agent.
UC-13	Agent executes approved step	Agent calls platform tool proxy to execute. Reports result via `task.step_completed` callback.
UC-14	Agent reports step result	Push `task.step_completed` or `task.step_failed` to platform callback.
UC-15	Agent requests human input	Push `task.needs_input` — agent pauses until expert responds.

2.4 Agent Lifecycle (Platform manages instances)

#	Use Case	Description
UC-16	Provision agent	Onboarding creates org → platform provisions agent container with org-scoped token.
UC-17	Configure agent	Platform pushes config (prompt, model, tools, threshold) via `POST /v1/configure`.
UC-18	Health check	Platform polls `GET /v1/health`.
UC-19	Scale agent	Platform scales container replicas based on load.
UC-20	Teardown agent	Org deactivated → platform destroys container + revokes token.

2.5 Learning & Feedback (Platform → Agent)

#	Use Case	Description
UC-21	Expert correction	Platform pushes corrections via `POST /v1/corrections`. Agent ingests for prompt refinement.
UC-22	Knowledge base update	Platform ingests documents through `/api/kb/documents` into per-org RAGFlow. Agent-side ingest endpoints are deprecated under platform-as-RAG-owner.
UC-23	Prompt refinement	Platform pushes few-shot examples extracted from correction patterns.

3. Security Model

3.1 The Scoped Agent Token

Each agent instance receives a Platform API token at provisioning time. This token:

{
  "sub": "agent-acme-001",          // agent instance ID
  "org_id": "uuid-of-acme",        // the ONLY org this agent can access
  "role": "agent",                  // platform role — NOT expert, NOT admin
  "scopes": [
    "conversations:read",           // read message history for its org
    "context:read",                 // search memory/KB for its org
    "tools:execute",                // call tool proxies for its org
    "callbacks:write"               // push events back to platform
  ],
  "iat": 1746360000,
  "exp": 1746446400                 // 24h expiry, auto-rotated
}

Enforcement on Platform side:

All /v1/agent-api/* endpoints check req.agent.org_id matches the resource's org_id
RLS interceptor applies to agent tokens the same way it applies to user tokens
An agent token for Org A literally cannot query Org B's data — the WHERE clause filters it out at the DB level

Token lifecycle:

Created during agent provisioning
Auto-rotated every 24h (agent calls POST /v1/agent-api/token/refresh)
Revoked instantly when org is deactivated or agent is torn down
Never stored in agent — passed as env var PLATFORM_API_TOKEN at container start

3.2 What the Agent CANNOT Do

Action	Blocked by
Query another org's conversations	Token `org_id` + RLS
Read another org's integration credentials	Token `org_id` + service-layer check
Connect to PostgreSQL	No `DATABASE_URL` — not in env, not in config, not discoverable
Connect to Redis	Same
Call admin/superadmin endpoints	Token role is `agent` — guards reject
Modify org settings, users, billing	Token scopes don't include write access to these resources
Read raw credentials (Shopify tokens, etc.)	Tool proxy returns results, never keys. Agent sends `{tool: "shopify", action: "getOrder", params: {id: "123"}}` and gets the order back

3.3 What the Agent CAN Do

Action	How
Read conversation history for its org	`GET /v1/agent-api/conversations/`{id}`/messages`
Search org knowledge base / memory	`GET /v1/agent-api/context?query=...` backed by RAGFlow
Execute tools (Shopify, Amazon, etc.)	`POST /v1/agent-api/tools/`{tool_name}`` — platform proxies with credentials
Push events to platform	`POST /v1/agent-events` with HMAC signature
Call LLM APIs (OpenAI, Anthropic, etc.)	Direct — agent holds its own LLM API key (or org-specific key)

3.4 Network Isolation

┌─────────────────────────────┐     ┌─────────────────────────────┐
│  Platform Network            │     │  Agent Sandbox (per-org)    │
│                              │     │                             │
│  ┌────────┐  ┌────────────┐ │     │  ┌──────────────────────┐  │
│  │ API    │  │ PostgreSQL │ │     │  │ Agent Instance        │  │
│  │ Server │  │ + Redis    │ │     │  │                       │  │
│  │        │◄─┤            │ │     │  │  - No DB access       │  │
│  │        │  └────────────┘ │     │  │  - Platform API token │  │
│  │        │                 │     │  │  - LLM API key        │  │
│  │        │◄────────────────┼─────┼──│  - Callback secret    │  │
│  │        │  HTTPS only     │     │  │                       │  │
│  │        ├─────────────────┼─────┼─►│  (agent-api + events) │  │
│  └────────┘                 │     │  └──────────────────────┘  │
│                              │     │                             │
│  Agent CANNOT reach:         │     │  Agent CAN reach:          │
│  - PostgreSQL (port 5432)    │     │  - Platform API (HTTPS)    │
│  - Redis (port 6379)         │     │  - LLM providers (HTTPS)   │
│  - Internal services         │     │  - Tool target APIs (via   │
│                              │     │    platform proxy)          │
└─────────────────────────────┘     └─────────────────────────────┘

In K8s: NetworkPolicy whitelists only HTTPS egress to Platform API + LLM providers. All other egress blocked.

4. Architecture

4.1 System Overview

┌──────────────────────────────────────────────────────────────────┐
│                     PLATFORM API (NestJS)                         │
│                                                                   │
│  ┌─────────────────────────────────────────────────────────────┐ │
│  │              Agent Gateway Module                            │ │
│  │                                                              │ │
│  │  ┌─────────────┐  ┌──────────────┐  ┌───────────────────┐  │ │
│  │  │ Registry    │  │ Router       │  │ Lifecycle Mgr     │  │ │
│  │  │ (org→agent  │  │ (dispatch to │  │ (provision,       │  │ │
│  │  │  mapping)   │  │  correct     │  │  configure,       │  │ │
│  │  │             │  │  instance)   │  │  scale, teardown) │  │ │
│  │  └─────────────┘  └──────┬───────┘  └───────────────────┘  │ │
│  │                          │                                   │ │
│  │  ┌──────────────────┐  ┌┴─────────────────┐                │ │
│  │  │ Agent API        │  │ Callback Handler  │                │ │
│  │  │ /v1/agent-api/*  │  │ POST /agent-events│                │ │
│  │  │ (context, tools, │  │ (HMAC verified)   │                │ │
│  │  │  conversations)  │  │                   │                │ │
│  │  └──────────────────┘  └───────────────────┘                │ │
│  └─────────────────────────────────────────────────────────────┘ │
└───────────────────────────┬──────────────────────────────────────┘
                            │ HTTPS (scoped token)
              ┌─────────────┼─────────────┐
              │             │             │
              ▼             ▼             ▼
     ┌────────────┐  ┌────────────┐  ┌────────────┐
     │ Agent:     │  │ Agent:     │  │ Agent:     │
     │ Acme Corp  │  │ Meridian   │  │ GlobalTech │
     │            │  │ Logistics  │  │            │
     │ Token:     │  │ Token:     │  │ Token:     │
     │  org=acme  │  │  org=merid │  │  org=gtech │
     │ Model: 4o  │  │ Model: 4o  │  │ Model:     │
     │ No DB.     │  │ No DB.     │  │  Claude    │
     │ No creds.  │  │ No creds.  │  │ No DB.     │
     └────────────┘  └────────────┘  └────────────┘

4.2 Platform → Agent Interface (Commands)

POST   /v1/chat                     # UC-1: Draft reply (sync)
POST   /v1/chat/stream              # UC-2: Draft reply (SSE stream)
POST   /v1/tasks                    # UC-11: Request plan for complex task
POST   /v1/tasks/{id}/steps/{n}/go  # UC-12: Approve step, resume execution
POST   /v1/tasks/{id}/steps/{n}/no  # UC-12: Reject step, halt
POST   /v1/tasks/{id}/cancel        # Cancel running task
POST   /v1/configure                # UC-17: Hot-reload config (prompt, model, tools, threshold)
POST   /v1/knowledge/ingest         # UC-22: Push KB documents
POST   /v1/knowledge/remove         # Remove KB documents
POST   /v1/corrections              # UC-21: Push expert corrections for learning
GET    /v1/health                   # UC-18: Health + readiness
GET    /v1/status                   # Running tasks, model info, uptime

4.3 Agent → Platform Interface (Data Fetching + Callbacks)

Data fetching — agent calls platform to get context it needs (never DB-direct):

GET  /v1/agent-api/conversations/{id}/messages   # Conversation history
GET  /v1/agent-api/context?query=...             # RAGFlow KB search (org-scoped)
POST /v1/agent-api/tools/{tool_name}             # Execute tool via platform proxy
POST /v1/agent-api/token/refresh                 # Rotate token before expiry

Event callbacks — agent pushes events to platform:

POST /v1/agent-events

Event types:
  task.plan_ready        # Agent produced a plan, needs expert approval
  task.step_started      # Agent began executing step N
  task.step_completed    # Step N succeeded + results
  task.step_failed       # Step N failed + error
  task.completed         # All steps done
  task.needs_input       # Agent paused, needs expert clarification
  tool.called            # Audit: agent invoked a tool (name, params, result summary)
  llm.called             # Audit: agent made an LLM call (model, tokens, cost, latency)
  error                  # Unrecoverable error

Every event is signed (HMAC-SHA256 with per-instance secret) and carries:

{
  "event_type": "task.step_completed",
  "agent_id": "agent-acme-001",
  "org_id": "uuid",
  "task_id": "uuid",
  "step_id": 2,
  "timestamp": "2026-05-04T14:30:00Z",
  "payload": { ... },
  "idempotency_key": "uuid"
}

4.4 Audit Trail

Every agent interaction is logged to agent_audit_log on the platform side:

What	When	What's Logged
Chat request	Platform → Agent	org_id, conversation_id, message length, timestamp
Chat response	Agent → Platform	confidence, risk_level, flag_for_review, token usage, cost, latency
Tool execution	Agent → Platform (proxy)	tool_name, parameters (sanitized), result summary, duration
LLM call	Agent → callback	model, input/output tokens, cost, latency, prompt version
Task lifecycle	Agent → callback	task_id, step transitions, approver_id, execution results
Error	Agent → callback	error type, stack trace (sanitized), context
Config change	Platform → Agent	what changed, who triggered it, before/after
Token rotation	Platform internal	old token hash, new token hash, expiry

Platform-side — all logged to agent_audit_log table (new entity). Not logged by the agent itself — the agent has no DB.

4.5 Tool Proxy Pattern (how agents call tools without holding credentials)

Agent wants to query Shopify:

Agent                          Platform API                      Shopify
  │                               │                                │
  ├── POST /v1/agent-api/tools/shopify                             │
  │   { action: "getOrder",       │                                │
  │     params: { id: "1234" } }  │                                │
  │   Authorization: Bearer <token>                                │
  │                               │                                │
  │                               ├── Verify token (org_id=acme)   │
  │                               ├── Lookup IntegrationCredential │
  │                               │   for org=acme, type=shopify   │
  │                               ├── Decrypt credentials (AES-256)│
  │                               ├── GET /admin/api/orders/1234 ──┤
  │                               │   X-Shopify-Access-Token: xxx  │
  │                               │                                │
  │                               │◄── { order: { ... } } ────────┤
  │                               │                                │
  │◄── { result: { order: ... } } │                                │
  │    (no credentials exposed)   │                                │
  │                               ├── Log to agent_audit_log       │
  │                               │                                │

The agent never sees credentials. It sends {tool, action, params} and gets results.

5. Agent Instance Configuration

Agent receives config at startup (env vars) + hot-reload via POST /v1/configure:

# Injected as env vars at container start
PLATFORM_API_URL: "https://api.h.work"
PLATFORM_API_TOKEN: "<scoped JWT, auto-rotated 24h>"
CALLBACK_URL: "https://api.h.work/v1/agent-events"
CALLBACK_SECRET: "<HMAC signing key>"
LLM_API_KEY: "<OpenRouter or direct provider key>"
AGENT_MODEL: "openai/gpt-4o"

# NO DATABASE_URL. NO REDIS_URL. NO SHOPIFY_TOKEN. NO AMAZON_KEY.

Config pushed via POST /v1/configure (hot-reload without restart):

{
  "specialist": {
    "name": "Bob",
    "system_prompt": "You are Bob, a senior e-commerce operations specialist..."
  },
  "model": "openai/gpt-4o",
  "temperature": 0.3,
  "confidence_threshold": 70,
  "enabled_tools": ["shopify_query", "amazon_query"],
  "max_concurrent_tasks": 3
}

5.1 Agent Internal Architecture

Agent Instance (per-org sandbox)
│
├── /v1/chat           → ChatHandler
│                         ├── PromptBuilder (specialist persona, injected via configure)
│                         ├── ContextFetcher (calls Platform: /v1/agent-api/context)
│                         ├── LLMClient (OpenRouter/direct — agent holds LLM key only)
│                         ├── ToolCaller (calls Platform: /v1/agent-api/tools/*)
│                         ├── ConfidenceScorer
│                         ├── RiskClassifier
│                         └── AuditEmitter (pushes llm.called, tool.called events)
│
├── /v1/tasks          → TaskHandler
│                         ├── Planner (decompose → steps)
│                         ├── Checkpointer (local SQLite or file — NOT platform DB)
│                         ├── StepExecutor (calls Platform tool proxy per step)
│                         └── CallbackEmitter (pushes task.* events)
│
├── /v1/configure      → ConfigHandler (hot-reload model/prompt/tools)
├── /v1/knowledge/*    → Deprecated under platform-as-RAG-owner; NestJS owns RAGFlow ingestion
├── /v1/corrections    → LearningHandler (ingest → local few-shot example store)
├── /v1/health         → HealthHandler
└── /v1/status         → StatusHandler

Local state: Agent maintains Hermes session mappings and workspace files. KB retrieval is platform-owned. NestJS injects retrieved KB context into conversation history and also serves /v1/agent-api/context for agent callers. RAGFlow credentials never leave the platform.

6. Deployment Model

6.1 Container per Org (launch default)

Each org gets a dedicated container:

Railway / K8s:
├── agent-acme-001       (256MB RAM, 0.25 vCPU)
├── agent-meridian-001   (256MB RAM, 0.25 vCPU)
├── agent-globaltech-001 (512MB RAM, 0.5 vCPU)  ← enterprise tier
└── ...

One universal Docker image. Config-driven specialization via env vars + /v1/configure.

Cost: ~$5–15/mo per agent at idle. 100 orgs = $500–1,500/mo. Acceptable given $500+/mo plan pricing.

6.2 Multiple Agents per Org

Enterprise clients can have multiple specialized agents:

Acme Corp:
├── agent-acme-ecommerce   (Shopify + Amazon, ecommerce_ops persona)
├── agent-acme-finance      (QuickBooks + Xero, finance_ops persona)
└── agent-acme-ir           (HKEX, ir_reporting persona)

Each gets its own scoped token (same org_id, different agent_id). The registry supports 1:N org→agent mapping. Router picks the right agent based on conversation role or specialist assignment.

6.3 Future: Serverless (low-volume orgs)

For orgs with < 100 messages/month, run agent as serverless function:

Cold start: 2–5s (acceptable — client is waiting for AI reply)
Trade-off: no persistent in-agent planner state — must re-hydrate from platform on each invocation

7. Migration Path

Phase 1: Define the interface (1–2 weeks)

No deployment changes. Keep single shared agent. But:

Write OpenAPI spec for ACP (/v1/* endpoints)
Build agent-api/ endpoints on platform (context fetch, tool proxy)
Replace AgentClient → AgentGateway (initially returns shared URL for all orgs)
Add POST /v1/agent-events callback handler on platform
Add agent_audit_log entity and logging

Result: Clean interface defined. Agent still shared, but protocol ready.

Phase 2: Per-org provisioning (2–3 weeks)

Build agent_instances table + lifecycle management
Implement scoped token issuance (JWT with org_id claim)
During onboarding, auto-provision agent container
Migrate agent to fetch context via platform API instead of direct calls
Migrate tool execution to platform proxy pattern
Remove all DB connection code from agent

Result: New orgs get dedicated sandboxed agents. Existing orgs migrated one at a time.

Phase 3: Agentic execution (3–4 weeks)

Implement Hermes-backed tool execution using the platform tool proxy
Persist Hermes session state through the workspace/session mapping
Step-level approval flow via callback protocol
Expert task UI (approve/reject steps, view results, chat with agent)

Result: Full plan→gate→execute with real tool execution, HITL approval, complete audit trail.

8. Decision Points

#	Question	Recommendation
1	Container runtime	Railway containers (Phase 1–2), K8s pods (Phase 3+)
2	Agent state storage	Hermes session mapping and workspace state (persistent volume, re-hydratable from platform)
3	Credential delivery to agent	None — agent never holds third-party credentials. Tool proxy pattern.
4	LLM API key	Agent holds its own LLM key (or per-org key for cost tracking). Only key the agent gets.
5	Knowledge base storage	Per-org RAGFlow via NestJS (`api/src/ragflow/`, `/api/kb/`). Agent-side local index is deprecated.
6	Callback security	HMAC-SHA256 per instance (consistent with channel webhook pattern)
7	Token rotation	24h auto-rotation. Agent calls `/v1/agent-api/token/refresh` before expiry.
8	Agent image	One universal image, config-driven. No per-domain images.

9. Files to Create/Modify

Action	Path	Description
Create	`api/src/agent-gateway/`	New module: registry, router, callback handler, lifecycle
Create	`api/src/agent-gateway/agent-gateway.service.ts`	Replaces `AgentClient` — org-aware routing
Create	`api/src/agent-gateway/agent-events.controller.ts`	`POST /v1/agent-events` — HMAC-verified callback receiver
Create	`api/src/agent-gateway/agent-api.controller.ts`	`/v1/agent-api/*` — context fetch, tool proxy for agents
Create	`api/src/agent-gateway/agent-registry.ts`	`agent_instances` table CRUD
Create	`api/src/agent-gateway/agent-lifecycle.ts`	Provision/configure/scale/teardown
Create	`api/src/agent-gateway/agent-audit.service.ts`	`agent_audit_log` entity + logging
Create	`agent/config.py`	Config loader (env vars + hot-reload via `/v1/configure`)
Create	`agent/platform_client.py`	HTTP client for Platform API (context fetch, tool proxy, token refresh)
Create	`agent/callbacks.py`	Event emitter to platform (HMAC-signed)
Create	`agent/handlers/`	Extracted handlers: `chat.py`, `tasks.py`, `knowledge.py`
Create	`docs/design/AGENT_CONTROL_PROTOCOL.md`	OpenAPI spec for ACP
Modify	`api/src/conversations/conversations.service.ts`	Replace `AgentClient` with `AgentGateway`
Modify	`api/src/common/entities.ts`	Add `AgentInstance`, `AgentAuditLog` entities
Delete	`api/src/common/agent.client.ts`	Replaced by `agent-gateway/`
Delete	Agent env vars: `DATABASE_URL`, `REDIS_URL`, `SHOPIFY_`, `AMAZON_`	Agent must not have these

0. Design Constraints (non-negotiable)​

1. Problem Statement​

2. Use Case Enumeration​

2.1 Synchronous Request→Response (Platform → Agent)​

2.2 Tool Execution (Agent → External APIs, via Platform)​

2.3 Agentic Tasks — Multi-Step with HITL Gates (Bidirectional)​

2.4 Agent Lifecycle (Platform manages instances)​

2.5 Learning & Feedback (Platform → Agent)​

3. Security Model​

3.1 The Scoped Agent Token​

3.2 What the Agent CANNOT Do​

3.3 What the Agent CAN Do​

3.4 Network Isolation​

4. Architecture​

4.1 System Overview​

4.2 Platform → Agent Interface (Commands)​

4.3 Agent → Platform Interface (Data Fetching + Callbacks)​

4.4 Audit Trail​

4.5 Tool Proxy Pattern (how agents call tools without holding credentials)​

5. Agent Instance Configuration​

5.1 Agent Internal Architecture​

6. Deployment Model​

6.1 Container per Org (launch default)​

6.2 Multiple Agents per Org​

6.3 Future: Serverless (low-volume orgs)​

7. Migration Path​

Phase 1: Define the interface (1–2 weeks)​

Phase 2: Per-org provisioning (2–3 weeks)​

Phase 3: Agentic execution (3–4 weeks)​

8. Decision Points​

9. Files to Create/Modify​