Agentic Runtime Streaming Design

Context

Issue #248 asks for true incremental SSE from POST /v1/chat/stream instead of the current single final data: event. Issue #172 originally described a LangGraph execute_node gap, but dev now contains Paul's Hermes-backed task runtime: /v1/tasks/*, JSON checkpoints, task-level locking, step approval endpoints, agent event callbacks, and the /workspace/tasks review surface. The remaining work is therefore a focused streaming implementation plus a verification pass against #172, not a LangGraph resurrection.

Current Hermes CLI on this machine exposes hermes chat -q ... -Q for quiet programmatic output and does not expose the issue body's claimed --stream flag. The design must support token-by-token deltas where Hermes emits them, while preserving the existing stable quiet-mode contract in environments that have not gained a stream flag.

Goals

• Stream incremental assistant text as SSE event: token frames from both /chat/stream and /v1/chat/stream.
• Emit one final event: done frame containing the same normalized metadata the existing single-event endpoint returned.
• Keep the NestJS proxy as a transparent byte stream so frontend callers can receive tokens without a second buffering layer.
• Preserve the existing /chat and /v1/chat non-streaming contracts.
• Verify #172 against current dev and implement only actual gaps found in the Hermes task lifecycle, API, or workspace UI.

Non-Goals

• Do not reintroduce LangGraph or agent/graph.py; current project docs and issue comments say Hermes is the active runtime.
• Do not invent frontend streaming UX beyond the existing conversation stream proxy unless verification shows it is required for #248.
• Do not replace the local JSON checkpointer for #172 unless a concrete missing requirement is found.

Architecture

Agent Streaming

Add HermesClient.chat_stream() beside the existing HermesClient.chat():

• Reuse the same command construction, environment merge, working directory creation, model/provider/toolsets, --source tool, --yolo, and optional --resume.
• Default stream mode should run without -Q so stdout can surface incremental output.
• Allow deployment-specific stream flags through HERMES_STREAM_ARGS, because current local Hermes lacks --stream but upstream may add a stable flag.
• Parse structured JSON lines such as {"type":"message.delta","delta":"..."} into token events.
• Treat plain stdout lines as fallback token text so a streaming-capable non-JSON CLI still produces incremental frames.
• Retain a final stdout transcript and stderr so the existing parse_hermes_output() and extract_hermes_payload() logic can produce the final metadata.

The stream event interface is intentionally tiny:

@dataclass(frozen=True)
class HermesStreamEvent:
    event: str
    token: str | None = None
    result: HermesRunResult | None = None

SSE Contract

Agent SSE frames:

event: token
data: {"token":"Hello"}

event: token
data: {"token":" there"}

event: done
data: {"reply":"Hello there","confidence":0.9,"risk_level":"low","flagForReview":false,"session_id":"..."}

data: [DONE]

Errors should be SSE-framed instead of raising after bytes may already have been sent:

event: error
data: {"error":"Hermes execution timed out"}

data: [DONE]

/chat/stream

main._stream_chat_sse() should use the same setup as chat():

• cost cap check before invoking Hermes
• workspace materialization and inbound artifact fetch
• resume session lookup
• Hermes streaming invocation
• token frame forwarding as chunks arrive
• final parse, confidence/risk normalization, session persistence, outbound artifact persistence, audit, metrics, and cost tracking

To avoid duplicating too much logic, the first implementation may extract small helpers only where they reduce real duplication. A large refactor of chat() is not required for #248.

/v1/chat/stream

agent/routers/chat.py should convert the v1 request to the legacy request and delegate to main._stream_chat_sse(), mapping the final payload shape if needed. This keeps one streaming implementation responsible for Hermes behavior.

NestJS Proxy

api/src/common/agent.client.ts already pipes the agent stream directly. Keep it as a raw stream proxy. Update comments/tests only if they assume single final events. The proxy should not parse, buffer, or reframe token events.

#172 Verification

Verify current implementation against the issue checklist:

• real approved-step Hermes execution in agent/task_runtime.py
• JSON checkpoint snapshots under runtime task paths
• approval resumes the next step after the previous approved step completes
• step failure halts the task and emits a failure event
• task result persistence in checkpoint JSON
• NestJS step approval endpoints and task events
• frontend task page with plan, per-step approve/reject, risk, progress, and audit/history enough for P2

Only implement gaps that are still real on dev. Based on the current inspection, likely #172 work is documentation/test alignment rather than a second runtime rewrite.

Testing Strategy

• Add Hermes client subprocess tests for streaming delta lines, final transcript, timeout, and fallback plain text.
• Add FastAPI endpoint tests proving token frames precede event: done.
• Keep existing single-final endpoint tests updated to the new event: done contract.
• Run focused agent pytest files first, then broader agent tests if the focused suite passes.
• Run API/frontend focused tests only if verification changes those layers.

Risks

• The installed Hermes CLI lacks --stream; token streaming depends on the actual stream-capable runtime emitting incremental stdout. The implementation must be explicit about this and provide HERMES_STREAM_ARGS.
• Plain stdout fallback can accidentally stream banners or tool previews. Parser should suppress recognized structured non-token lines, but this may need hardening after real Hermes stream smoke.
• Duplicating chat() finalization logic in _stream_chat_sse() can drift. Keep helper extraction small and test the final metadata path.