Skip to main content

Feature: Auto-Reply Behaviour

How humanwork decides what (if anything) to send to a client when their inbound message arrives, before an Expert reviews it.

Default for new orgs: autoReplyMode = 'none' (no autoreply). New client orgs do NOT auto-send anything to the client — every inbound message goes straight to the Expert queue. An AM or SuperAdmin must explicitly opt the org into system_reply or agent_reply from the SuperAdmin/AM org settings page.

The three modes​

Configured per-org on organizations.autoReplyMode (varchar). Possible values:

ModeWhat the client sees on a fresh inbound turnWhen the AI draft is held
none (default for new orgs)Nothing. Complete silence on the channel.Always. Expert composes every reply from the workspace.
system_replyA static template (per-org custom OR platform default) — "Thanks, an expert will be with you shortly."Always. AI draft is saved with heldForReview=true and surfaced in the Expert workspace.
agent_replyThe AI draft, IF and ONLY IF the org's autoRespondThreshold is met. Otherwise: behaves like system_reply (template sent, queued).When autoRespondThreshold is NOT met.

How modes interact with autoRespondThreshold​

autoRespondThreshold is a 0–101 integer on organizations (and optionally per-Specialist on org_specialist_assignments.confidence_threshold):

ValueMeaning
101 (default)Never auto-respond — every reply, even confident ones, goes to the Expert queue.
0Always auto-respond (no quality floor).
1–100Auto-respond when round(confidence * 100) >= threshold. UI labels: Never=101, Sometimes=90, Often=70, Always=0.

The threshold is only consulted when autoReplyMode = 'agent_reply'. In system_reply and none modes the threshold is ignored — the AI draft is always held regardless of confidence.

The 9 cases (3 modes × 3 confidence/risk bands)​

For every inbound client turn, ConversationsService.addMessage calls resolveAutoReplyPolicy(orgId, confidence) exactly once, then routes through one of these:

none mode​

ConditionsOutcome
Any confidence, any riskAI draft held (heldForReview=true). No client-visible reply on any channel. Queue item created with escalationReason=auto_reply_mode_none (or risk_* if risk is high/critical). Expert composes the response.

system_reply mode​

ConditionsOutcome
Any confidence, any riskAI draft held. Static template sent to the client (organizations.autoReplyTemplate, falling back to the platform default text if null). Queue item created with escalationReason=auto_reply_mode_system (or risk_* if applicable). Expert can override the template or correct the AI draft from the workspace.

agent_reply mode​

ConditionsOutcome
confidence >= threshold AND risk < high AND flagForReview = falseAI draft auto-sent to client. autoResponded=true on the conversation. No queue item. No template.
confidence < threshold (any reason — autoResponded=false)AI draft HELD (heldForReview=true) and queued for expert review. escalationReason=agent_reply_threshold_not_met. The expert workspace surfaces the held draft on the queue item; the expert edits and sends. The client sees nothing automatic.
flagForReview = true (any confidence)Held + queued. escalationReason=agent_flagged.
risk = high or risk = criticalHeld + queued. escalationReason=risk_high / risk_critical.

Why strict? Eusden flagged this directly: "for agent_reply it should ONLY reply if it's on that mode" — i.e. an AI draft must only reach the client when (a) the AM explicitly opted into agent_reply, AND (b) the confidence threshold is met. Before this fix, an agent_reply org with the default autoRespondThreshold=101 would silently emit every unreviewed AI draft to the client. Now every unmet-threshold turn holds the draft and queues for expert review.

Outbound channel gating​

Every channel checks !agentMsg.heldForReview before sending. Channels covered:

ChannelSource
In-app websocket (emitAgentMessageSent)api/src/conversations/conversations.service.ts
Slackapi/src/channels/channels.controller.ts
Email (Resend)api/src/channels/channels.controller.ts
Telegramapi/src/channels/telegram/telegram.service.ts
WhatsApp (Twilio)api/src/channels/channels.controller.ts
Teamsapi/src/channels/channels.controller.ts

When a channel skips delivery because the draft is held, it logs <Channel> reply held for expert review: conv=<id>.

Held-draft visibility (Issue #347 + #350)​

heldForReview=true on a Message means it is invisible to clients — but is still surfaced to internal roles. The visibility rule lives in ConversationsService and is enforced on every read path that returns message content:

Read pathFilters held drafts for clients?
GET /conversations/:id → findOne✅ Yes (#347)
GET /conversations/search?q=… → searchMessages✅ Yes (#350)
POST /conversations/:id/messages/stream → streamMessage✅ Yes (#350) — delegates to sendMessage for system_reply/none orgs, never streams AI tokens
Real-time WebSocket agent_message_sent emit✅ Yes (#345) — gated on !agentMsg.heldForReview
Channel send (Slack/Email/Telegram/WhatsApp/Teams)✅ Yes (#345) — same gate
RoleSees held AI drafts?
org_admin, org_member, client_admin, client_member (clients)No — filtered out of all read paths
expert, account_manager, superadmin, system (internal / webhooks)Yes — needed so the expert workspace can render the pending draft

Without these filters, a client refreshing or polling the conversation after a system_reply turn would see BOTH the held AI draft AND the system template (#347 fix), or could surface the AI draft text via full-text search (#350 fix). The websocket emit path (emitAgentMessageSent) is gated on !agentMsg.heldForReview so the real-time push is correct.

Fail-safe defaults​

resolveAutoReplyPolicy defaults to mode = 'none' when no orgId is resolvable (#350). This matters for stub channel adapters (WeChat) and any future ingestion path that hasn't wired org resolution yet — fail-open to agent_reply would silently leak unreviewed AI drafts; fail-closed to none keeps the HITL invariant intact.

Configuring an org​

There is no public client-facing setting. Only AMs and SuperAdmins can change auto-reply config, via:

  • UI: Ops → Clients → <Org> → Settings → Auto-reply
  • API: PATCH /organizations/:orgId/auto-reply-config with { mode, template, confidenceThreshold } (DTO: UpdateAutoReplyConfigDto)

Reading the current config: GET /organizations/:orgId/auto-reply-config. The response includes defaultTemplate so the UI can show the fallback text for system_reply mode.

Audit log​

Every inbound turn writes an audit_log entry with action=message.send and a payload containing:

{
  "conversationId": "...",
  "confidence": 0.84,
  "flagForReview": false,
  "riskLevel": "low",
  "escalated": true,
  "autoResponded": false,
  "autoReplyMode": "system_reply",      // configured value
  "autoReplyApplied": "system_reply",   // actually applied (after overrides)
  "systemReplyMessageId": "msg_..."     // when a template was sent
}

This is recorded on every turn, not just escalated ones, so the policy decision is auditable end-to-end.

Database​

ColumnTypeDefaultNotes
organizations.auto_reply_modevarchar(20)'none' (Issue #346)Check constraint: IN ('system_reply', 'agent_reply', 'none')
organizations.auto_reply_templatetextnullPer-org system_reply template. Null → platform default text.
organizations.auto_reply_confidence_thresholddecimal(3,2)nullOptional fallback hook for resolveAutoReplyPolicy (separate from autoRespondThreshold).
organizations.auto_respond_thresholdint1010–101. Only consulted in agent_reply mode.

History​

Issue / PRChange
#250 / #345Bug fix: autoReplyMode was only respected on the escalated path. Every outbound path (in-app + 5 channels) is now gated on the policy. 6 integration tests in api/test/auto-reply-mode.spec.ts.
#346Default autoReplyMode for new orgs flipped from system_reply (and original agent_reply from migration 1714000000034) to none. Existing orgs are NOT migrated — they keep whatever value they had. AMs flip per-org as needed.
#347Bug fix: clients on system_reply orgs were seeing BOTH the held AI draft and the system template on refresh / poll, because GET /conversations/:id returned heldForReview=true rows to client roles. findOne now filters those rows for non-internal roles. Two new integration tests cover the client-vs-expert visibility split.
#350Audit-sweep fixes for three additional bypass paths: (1) POST /conversations/:id/messages/stream (SSE) now delegates to sendMessage for system_reply/none orgs so AI tokens never reach the client; (2) GET /conversations/search now filters held_for_review=true for client roles (the FTS sibling of #347's findOne fix); (3) resolveAutoReplyPolicy defaults to 'none' (not 'agent_reply') when orgId is unresolvable — protects stub channels (WeChat) and any new ingestion path that hasn't wired org resolution yet. Two new integration tests.
#352 (PR #368)Audit-log entry written on every PATCH /orgs/:orgId/auto-reply-config change. Records changedFields, before, after, actorPlatformRole. No-op PATCHes (caller submitted the same values that were already set) skip the audit write so the log stays signal-heavy.
#353 (PR #366)Tightened PATCH /orgs/:orgId/auto-reply-config role gate from superadmin / AM / expert → superadmin / AM only. autoReplyMode controls whether AI drafts reach the client — that's an AM/SuperAdmin decision, not a per-conversation expert decision. Experts retain read access via GET.
#354 (PR #367)Frontend defense-in-depth: client portal now filters heldForReview=true rows out of the agent_message_sent websocket payloads, so a held draft can't slip through even if a backend regression re-emits it.
(strict agent_reply)Bug fix: agent_reply mode silently emitted unreviewed AI drafts whenever the confidence threshold wasn't met. Now agent_reply ONLY auto-sends when autoResponded=true (threshold met) — every other turn in agent_reply mode holds the draft and queues for expert review with escalationReason=agent_reply_threshold_not_met. Three new integration tests covering the default-101 threshold, sub-threshold confidence, and the happy-path regression.
Last updated on by ethumanity