Hermes-agent

This commit is contained in:
Zakaria
2026-06-14 14:30:48 -04:00
commit dac4b88b94
5058 changed files with 1884848 additions and 0 deletions
+146
View File
@@ -0,0 +1,146 @@
# Profile Builder — Dashboard-Native, Full-Featured Profile Creation
Status: design proposal (not yet implemented)
Author: drafted for Teknium
Supersedes: PR #31781 (prompt_toolkit `hermes profile wizard`)
## Why this, not the CLI wizard
PR #31781 added a keyboard-driven `hermes profile wizard` in the terminal.
The decision is to **not** build the profile-creation experience in the CLI.
The dashboard already owns mature, separate pages for every element a profile
needs, and a profile is just a HERMES_HOME directory — so the dashboard is the
right home for a full-featured builder, and it can reuse everything that
already exists.
A profile = a full `~/.hermes/profiles/<name>/` directory with its own:
- `config.yaml` — holds `model`/`provider`, `mcp_servers`, enabled skills
- `skills/` — physical SKILL.md files (built-in seed + optional + hub installs)
- `.env` — secrets
- `SOUL.md` / `USER.md` — identity
So per-profile scoping of Model, MCPs, and Skills is **native** — no data-model
change needed. The gap is purely UX: creation today is a thin modal
(name + clone + model + description), and you can only compose skills/MCPs
*after* the profile exists, by visiting other pages and remembering to scope
them.
## What already exists (reuse, don't rebuild)
| Element | Existing page | Existing API | Profile-scopable? |
|---|---|---|---|
| Name / Description | ProfilesPage create modal | `POST /api/profiles` (`create_profile`) | yes (args) |
| Model + Provider | ModelsPage | `_write_profile_model(profile_dir, …)` | yes — HERMES_HOME override, already wired into create endpoint |
| MCPs | McpPage | `mcp_config._save_mcp_server` + `/api/mcp/catalog` | yes — wrap with HERMES_HOME override |
| Skills (built-in/optional) | SkillsPage | `GET /api/skills`, `/api/skills/toggle` | yes — config write |
| Skills (hub) | SkillsPage | `/api/skills/hub/search`, `/api/skills/hub/install` | **only via subprocess** — see seam #1 |
## Two architectural seams found while grounding this design
These are load-bearing — they change the implementation, not just the polish.
### Seam #1 — hub-skill install cannot use the HERMES_HOME override
`tools/skills_hub.py` binds `SKILLS_DIR = HERMES_HOME / "skills"` at **module
import time**. The context-local `set_hermes_home_override()` swap (which makes
`_write_profile_model` and the MCP write land in the target profile) does NOT
retroactively rebind that already-imported module global. So a data-layer wrap
of hub install would write into the dashboard's *own* active profile, not the
new one.
The correct mechanism is the existing subprocess path: `_spawn_hermes_action`
runs `python -m hermes_cli.main <subcommand>`, and `_apply_profile_override()`
re-reads `sys.argv` at import in the fresh child. Prepend `-p <profile>`:
```python
_spawn_hermes_action(["-p", profile, "skills", "install", identifier], "skills-install")
```
A fresh subprocess re-imports `skills_hub` with the profile's HERMES_HOME bound
from the start, so `SKILLS_DIR` resolves to `<profile>/skills/`. Correct by
construction.
### Seam #2 — hub installs are async, so create cannot be fully atomic
Built-in/optional skill enabling and MCP writes are **synchronous config ops**
and can be part of the create call. Hub installs are long-running git fetches
spawned detached (`_spawn_hermes_action` returns a PID immediately). So the
create flow is:
1. `create_profile()` — make the dir (synchronous)
2. write model (synchronous, HERMES_HOME override)
3. write selected MCP servers (synchronous, HERMES_HOME override)
4. seed/enable selected built-in + optional skills (synchronous)
5. spawn `hermes -p <profile> skills install <id>` per hub skill (async, returns PIDs)
Steps 14 commit before the response; step 5 returns a list of action PIDs the
UI polls (same pattern as today's SkillsPage hub install). The builder's
"Review → Create" returns `{ok, name, path, hub_installs: [{id, pid}]}` and the
final screen shows live install progress for the hub skills.
## Proposed backend change (small, follows existing patterns)
Extend `ProfileCreate` and the create endpoint — no new endpoints, no rewrite:
```python
class ProfileCreate(BaseModel):
name: str
clone_from: Optional[str] = None
# Backward compatibility for older dashboard/desktop clients.
clone_from_default: bool = False
clone_all: bool = False
no_skills: bool = False
description: Optional[str] = None
provider: Optional[str] = None
model: Optional[str] = None
# NEW — all optional, all best-effort post-create (profile already exists)
mcp_servers: List[MCPServerCreate] = [] # synchronous, HERMES_HOME override
builtin_skills: List[str] = [] # synchronous enable/seed
hub_skills: List[str] = [] # async spawn, returns PIDs
```
The endpoint already does best-effort post-create steps (`seed_profile_skills`,
`_write_profile_model`). Add two more best-effort blocks (MCP write, hub-skill
spawn) in the same style — a failure in any of them must not 500 the create,
since the profile dir already exists and the user can fix it from the relevant
page afterward. Mirror `_write_profile_model`'s HERMES_HOME-override helper for
the MCP write (`_write_profile_mcp_servers(profile_dir, servers)`).
## Proposed frontend — dedicated builder page `/profiles/new`
A full page (not the cramped modal), stepped, each step reusing the existing
page's component + API, targeted at the new profile:
```
① Identity Name + Description (+ optional clone-from existing profile)
② Model Provider + model picker (reuse ModelsPage picker)
③ Skills Tabs: Built-in · Optional · Hub-search
multi-select; "Start from default bundle" preset button
④ MCPs Tabs: Catalog browse · Manual add (reuse McpPage form)
⑤ Review Blueprint preview → Create
→ progress screen for async hub installs
```
Nothing writes to disk until ⑤.
## Open product decisions (need Teknium)
1. **Skills seeding default.** Fresh profiles auto-seed the default bundle
today. In the builder, should the skill step **replace** the bundle (pick
exactly what you want; offer a "start from default bundle" preset) or
**augment** it? Recommendation: replace + preset button.
2. **Page vs richer modal.** Dedicated `/profiles/new` page (room to grow:
SOUL editing, multi-agent fleets later) vs a bigger create modal on
ProfilesPage. Recommendation: dedicated page — matches "full-featured / way
more options."
## Verification plan (when built)
- Backend E2E with isolated HERMES_HOME: POST a full create body
(name + model + 2 MCPs + 3 builtin skills + 1 hub skill), assert the new
profile dir has the model in config.yaml, both MCP servers in config.yaml,
the builtin skills enabled, and a spawned PID for the hub skill. Negative:
a bad MCP entry must not 500 the create.
- `cd web && npm run build` (no JS test suite in web/).
- Targeted: `pytest tests/<web_server profile tests> -k profile_create`.
Binary file not shown.
+39
View File
@@ -0,0 +1,39 @@
# Multi-gateway deployment
Hermes supports multiple gateway processes running concurrently — one per profile
(default, writer, admin, coder, researcher). Each gateway opens its own connection
to platform APIs and delivers messages for its profile's subscribers.
## Single-dispatcher posture
Only one gateway owns the kanban dispatcher. The owning gateway keeps
`kanban.dispatch_in_gateway: true` (the default); every other gateway sets it
to `false`.
**Why this matters:** a gateway with `dispatch_in_gateway: true` opens per-board
SQLite connections for both the dispatcher and the notifier watcher. Multiple
gateways doing this concurrently multiplies the open file descriptors on each
`kanban.db` and amplifies WAL `-shm` reader contention. Gating both paths on the
same flag means exactly one process touches the kanban DBs.
## Configuration
On the dispatch-owning gateway (typically the `default` profile), no change is
needed. On every other profile gateway, add to `~/.hermes/config.yaml`:
```yaml
kanban:
dispatch_in_gateway: false
```
Or set the env var: `HERMES_KANBAN_DISPATCH_IN_GATEWAY=false`
## What each gateway does
| Gateway role | dispatch_in_gateway | Opens per-board DBs? | Runs dispatcher + notifier? |
|---|---|---|---|
| default (dispatch owner) | true (default) | yes | yes |
| writer, admin, coder, etc. | false | no | no |
Non-dispatch gateways still deliver messages for their own platform adapters
(Telegram, Discord, etc.) — they just don't poll kanban boards.
+260
View File
@@ -0,0 +1,260 @@
# Hermes Middleware
Hermes middleware is the behavior-changing companion to observer hooks.
Observer hooks report what happened. Middleware can change what happens by
rewriting a request before execution or by wrapping the execution callback
itself.
This contract is intentionally backend-neutral. A plugin can use it for local
policy, request shaping, tracing, adaptive routing, cache control, sandbox
selection, or handoff to runtimes such as NeMo Relay without changing Hermes'
planner, model provider adapters, tool registry, memory, or CLI UX.
With middleware enabled, plugins can:
- Rewrite LLM provider request kwargs before Hermes calls the provider.
- Rewrite tool arguments before guardrails, approval checks, hooks, and tool
execution see them.
- Wrap the actual LLM execution callback while preserving Hermes retry,
streaming, interrupt, and hook behavior.
- Wrap the actual tool execution callback while preserving Hermes guardrails,
approval, post-tool hooks, and tool-result transformation.
## Contract
Plugins register middleware from `register(ctx)`:
```python
def register(ctx):
ctx.register_middleware("llm_request", on_llm_request)
ctx.register_middleware("llm_execution", on_llm_execution)
ctx.register_middleware("tool_request", on_tool_request)
ctx.register_middleware("tool_execution", on_tool_execution)
```
Every middleware callback receives:
- `telemetry_schema_version`: currently `hermes.observer.v1`
- `middleware_schema_version`: currently `hermes.middleware.v1`
- Runtime context such as `session_id`, `task_id`, `turn_id`,
`api_request_id`, `provider`, `model`, `api_mode`, `tool_name`, and
`tool_call_id` when applicable.
Supported middleware kinds:
| Kind | Payload | Return shape | Purpose |
| --- | --- | --- | --- |
| `llm_request` | `request`, `original_request` | `{"request": {...}}` | Replace effective provider kwargs before provider execution. |
| `tool_request` | `tool_name`, `args`, `original_args` | `{"args": {...}}` | Replace effective tool args before hooks, guardrails, approvals, and execution. |
| `llm_execution` | `request`, `original_request`, `next_call` | Any provider response | Wrap or replace the actual provider call. |
| `tool_execution` | `tool_name`, `args`, `original_args`, `next_call` | Any tool result | Wrap or replace the actual tool call. |
Request middleware can return optional trace fields:
```python
return {
"request": updated_request,
"source": "my-plugin",
"reason": "selected fallback model",
}
```
Hermes stores those trace entries in later observer hook payloads as
`middleware_trace`.
Execution middleware receives a `next_call` callback. Call it to continue the
chain:
```python
def on_tool_execution(**kwargs):
result = kwargs["next_call"](kwargs["args"])
return result
```
If multiple plugins register the same execution middleware kind, Hermes runs
them as a nested chain in registration order. Middleware failures are fail-open:
Hermes logs a warning and continues with the next middleware or the base
runtime path.
## Execution Order
### LLM Calls
For each provider request, Hermes applies middleware in this order:
1. Build provider kwargs from the current conversation.
2. Apply `llm_request` middleware.
3. Emit `pre_api_request` observer hooks with the effective request.
4. Run provider execution through `llm_execution` middleware.
5. Emit `post_api_request` or `api_request_error` observer hooks.
Request middleware sees the full provider kwargs, including `messages` or
Responses API `input`, model settings, tool definitions, stream options, and
provider-specific options. Execution middleware receives the same effective
request plus `next_call`.
### Tool Calls
For each tool call, Hermes applies middleware in this order:
1. Parse and coerce model-provided tool arguments.
2. Apply `tool_request` middleware.
3. Run the normal Hermes pre-execution path against the effective arguments:
tool availability checks, observer block directives, guardrails, and
approval checks.
4. Run tool execution through `tool_execution` middleware.
5. Emit `post_tool_call` observer hooks.
6. Apply `transform_tool_result` hooks before the result is appended back into
conversation context.
Tool request middleware runs before approval checks. Use it carefully: a
rewritten path, command, or URL is the value downstream policy will evaluate.
## Enablement
Middleware only runs for enabled plugins. For a bundled plugin:
```bash
hermes plugins enable <plugin-name>
```
For isolated local testing, use one `HERMES_HOME` for plugin enablement and the
agent run:
```bash
export HERMES_HOME=/tmp/hermes-middleware-test
mkdir -p "$HERMES_HOME"
hermes plugins enable <plugin-name>
hermes chat --query 'Reply exactly ok'
```
For source checkouts, prefer the source command so the runtime sees plugins and
middleware from the working tree:
```bash
uv sync
uv run hermes plugins enable <plugin-name>
uv run hermes chat --query 'Reply exactly ok'
```
## Generic Plugin Examples
The examples below are intentionally small. They show the middleware contract
shape without depending on NeMo Relay.
### LLM Request Middleware
This plugin tags provider requests and records a middleware trace entry:
```python
def register(ctx):
ctx.register_middleware("llm_request", tag_llm_request)
def tag_llm_request(**kwargs):
request = dict(kwargs["request"])
extra_body = dict(request.get("extra_body") or {})
extra_body.setdefault("metadata", {})["hermes_middleware_demo"] = True
request["extra_body"] = extra_body
return {
"request": request,
"source": "middleware-demo",
"reason": "tagged provider request",
}
```
The effective request is passed to `pre_api_request`, provider execution, and
`post_api_request`.
### Tool Request Middleware
This plugin constrains `terminal` calls to a known working directory:
```python
def register(ctx):
ctx.register_middleware("tool_request", normalize_terminal_workdir)
def normalize_terminal_workdir(**kwargs):
if kwargs.get("tool_name") != "terminal":
return None
args = dict(kwargs["args"])
args.setdefault("workdir", "/tmp/hermes-middleware-demo")
return {
"args": args,
"source": "middleware-demo",
"reason": "defaulted terminal workdir",
}
```
Because this runs before hooks and approvals, downstream telemetry and policy
observe the rewritten `workdir`.
### LLM Execution Middleware
This plugin wraps the provider call and preserves the raw provider response:
```python
import time
def register(ctx):
ctx.register_middleware("llm_execution", time_llm_execution)
def time_llm_execution(**kwargs):
started = time.monotonic()
response = kwargs["next_call"](kwargs["request"])
elapsed_ms = int((time.monotonic() - started) * 1000)
print(f"llm_execution elapsed_ms={elapsed_ms}")
return response
```
Return the same response shape Hermes expects from the provider adapter. Do not
wrap the response in a plugin-specific envelope unless the rest of the runtime
expects that envelope.
### Tool Execution Middleware
This plugin wraps tool execution while preserving the tool result:
```python
def register(ctx):
ctx.register_middleware("tool_execution", annotate_tool_execution)
def annotate_tool_execution(**kwargs):
result = kwargs["next_call"](kwargs["args"])
# Metrics, logging, or external routing can happen here.
return result
```
Execution middleware may call `next_call(modified_args)` to pass a changed
payload to later middleware and the base tool dispatcher.
Plugin-specific examples should live with the plugin that owns the behavior.
For NeMo Relay adaptive execution middleware, see
[`plugins/observability/nemo_relay/README.md`](../../plugins/observability/nemo_relay/README.md).
## Safety Notes
- Middleware should be deterministic for the same input unless it is explicitly
routing to a dynamic external system.
- Request middleware should return complete replacement payloads, not partial
patches.
- Execution middleware should call `next_call(...)` exactly once unless it is
intentionally short-circuiting execution.
- If execution middleware raises before calling `next_call(...)`, Hermes treats
that as middleware failure and continues with the remaining middleware chain
and base execution.
- If execution middleware calls `next_call(...)` successfully and then raises
during post-processing, Hermes preserves the downstream result and does not
run the provider or tool a second time.
- If downstream provider or tool execution fails, middleware may let that error
propagate or translate it deliberately. Hermes does not convert downstream
failure into a successful `None` result.
- Tool request middleware runs before approvals. If it mutates file paths,
commands, URLs, or arguments, the mutated values are what guardrails and
approvals evaluate.
- Observer hooks remain the right place for read-only telemetry. Use middleware
only when a plugin needs to alter or wrap behavior.
+316
View File
@@ -0,0 +1,316 @@
# Hermes Observer Hooks
Hermes observer hooks are the read-only telemetry contract for plugins that
need to reconstruct agent execution without changing runtime behavior. This
contract supports trace, metrics, audit, replay, and export integrations such
as Langfuse, OpenTelemetry-style collectors, and NeMo Relay.
Observer hooks are intentionally backend-neutral. They expose stable lifecycle
events, correlation IDs, sanitized payloads, timing, status, and error fields.
They do not replace Hermes' planner, model providers, memory, tool registry,
approval UX, CLI, gateway behavior, or execution semantics.
Behavior-changing request or execution wrappers are outside this observer
contract. Observer hooks should report what happened; they should not replace
provider requests, tool arguments, or execution callbacks.
## Contract
Plugins register observer callbacks from `register(ctx)`:
```python
def register(ctx):
ctx.register_hook("pre_api_request", on_pre_api_request)
ctx.register_hook("post_api_request", on_post_api_request)
ctx.register_hook("pre_tool_call", on_pre_tool_call)
ctx.register_hook("post_tool_call", on_post_tool_call)
```
Every hook callback receives keyword arguments. Plugins should accept
`**kwargs` so additive fields remain backward-compatible:
```python
def on_post_tool_call(**kwargs):
tool_name = kwargs.get("tool_name")
status = kwargs.get("status")
result = kwargs.get("result")
```
The plugin manager injects this field into every hook payload:
```text
telemetry_schema_version = "hermes.observer.v1"
```
Hook callbacks are fail-open. Hermes catches callback exceptions, logs a
warning, and keeps the agent loop running.
Most observer hook return values are ignored. The exceptions are older
behavior-affecting hooks:
| Hook | Return behavior |
| --- | --- |
| `pre_llm_call` | May return a string or `{"context": "..."}` to inject ephemeral context into the current user message. |
| `pre_tool_call` | May return `{"action": "block", "message": "..."}` to block a tool before execution. |
| `transform_tool_result` | May return a replacement tool result string after `post_tool_call`. |
| `transform_llm_output` | May return a replacement final assistant text string. |
Telemetry plugins should treat these behavior-affecting returns as optional
compatibility features, not as observability requirements.
## Correlation IDs
Observer payloads use stable IDs so plugins can join events without relying on
callback order alone.
| Field | Meaning |
| --- | --- |
| `session_id` | Conversation/session identity. |
| `task_id` | Task identity, especially useful for subagents and isolated execution. |
| `turn_id` | User-turn identity shared by API attempts and tool calls in a turn. |
| `api_request_id` | Opaque provider-attempt identity. Do not parse its string format. |
| `api_call_count` | Numeric API attempt count within the agent loop. |
| `tool_call_id` | Provider-supplied tool call ID when available. |
| `parent_session_id` / `child_session_id` | Session link for delegated subagents. |
| `parent_subagent_id` / `child_subagent_id` | Subagent link when available. |
| `parent_turn_id` | Parent turn that spawned delegated work. |
Consumers should prefer explicit fields over parsing compound IDs. In
particular, `api_request_id` is an opaque correlation value.
## Event Families
### Session Lifecycle
Session hooks describe conversation boundaries and resets:
| Hook | When it fires |
| --- | --- |
| `on_session_start` | A brand-new session starts after the system prompt is built. |
| `on_session_end` | A `run_conversation` call ends, including interrupted or incomplete turns. |
| `on_session_finalize` | CLI or gateway tears down an active session identity. |
| `on_session_reset` | CLI or gateway moves from an old session identity to a new one. |
Common fields include `session_id`, `completed`, `interrupted`, `reason`,
`old_session_id`, and `new_session_id` where available.
`on_session_end` is turn/run scoped. It is not necessarily the final lifetime
boundary for a chat identity. Use `on_session_finalize` and `on_session_reset`
for lifecycle cleanup that must happen once per session identity.
### Turn-Scoped LLM Hooks
These hooks frame the user turn, not individual provider API attempts:
| Hook | When it fires |
| --- | --- |
| `pre_llm_call` | Before the tool loop begins for a user turn. |
| `post_llm_call` | After the turn completes with final assistant output. |
Common `pre_llm_call` fields include `session_id`, `turn_id`,
`user_message`, `conversation_history`, `is_first_turn`, `model`, `platform`,
and `sender_id`.
Common `post_llm_call` fields include `session_id`, `turn_id`,
`user_message`, `assistant_response`, `conversation_history`, `model`, and
`platform`.
Use request-scoped API hooks for LLM span telemetry. Use `pre_llm_call` and
`post_llm_call` for turn-level context, compatibility, and final turn summary.
### Request-Scoped API Hooks
API hooks describe provider attempts inside the agent loop:
| Hook | When it fires |
| --- | --- |
| `pre_api_request` | Immediately before a provider API request. |
| `post_api_request` | After a successful provider response. |
| `api_request_error` | After a failed provider request or retryable error path. |
`pre_api_request` includes:
- identity: `session_id`, `task_id`, `turn_id`, `api_request_id`
- runtime: `platform`, `model`, `provider`, `base_url`, `api_mode`
- attempt metadata: `api_call_count`, `message_count`, `tool_count`,
`approx_input_tokens`, `request_char_count`, `max_tokens`
- timing: `started_at`
- sanitized request payload: `request`
`post_api_request` includes the same identity/runtime fields plus:
- `api_duration`, `started_at`, `ended_at`
- `finish_reason`, `message_count`, `response_model`
- `usage`
- `assistant_content_chars`, `assistant_tool_call_count`
- sanitized response payload: `response`
- compatibility object: `assistant_message`
`api_request_error` includes the same identity/runtime fields plus:
- `api_duration`, `started_at`, `ended_at`
- `status_code`, `retry_count`, `max_retries`, `retryable`, `reason`
- structured `error = {"type": ..., "message": ...}`
- sanitized failed request payload: `request`
The sanitized `request`, `response`, and `error` fields are the canonical
observer inputs for new consumers.
### Tool Lifecycle
Tool hooks describe individual tool calls:
| Hook | When it fires |
| --- | --- |
| `pre_tool_call` | Before guardrail-approved tool dispatch. |
| `post_tool_call` | After tool dispatch, cancellation, block, or error completion. |
| `transform_tool_result` | After `post_tool_call`, before the result is appended to model context. |
`pre_tool_call` includes `tool_name`, `args`, `task_id`, `session_id`,
`tool_call_id`, `turn_id`, and `api_request_id`.
`post_tool_call` includes the same identity fields plus `result`,
`duration_ms`, `status`, `error_type`, and `error_message`.
`status` is the observer-grade lifecycle outcome. Common values include:
| Status | Meaning |
| --- | --- |
| `ok` | Tool completed normally. |
| `error` | Tool ran and returned or raised an error outcome. |
| `blocked` | A `pre_tool_call` hook blocked execution. |
| `cancelled` | Execution was cancelled before normal completion. |
`post_tool_call` is emitted for blocked and cancelled paths so telemetry
plugins can close spans cleanly.
### Approval Lifecycle
Approval hooks describe dangerous-command approval prompts:
| Hook | When it fires |
| --- | --- |
| `pre_approval_request` | Before the approval request is shown or sent. |
| `post_approval_response` | After the user responds or the request times out. |
Common fields include `command`, `description`, `pattern_key`,
`pattern_keys`, `session_key`, and `surface`.
`post_approval_response` also includes `choice`, with values such as `once`,
`session`, `always`, `deny`, and `timeout`.
Approval hooks are observer-only. Plugins cannot pre-answer or veto approvals
from these hooks. To prevent a tool from reaching approval, use
`pre_tool_call` blocking.
### Subagent Lifecycle
Subagent hooks describe delegated child-agent work:
| Hook | When it fires |
| --- | --- |
| `subagent_start` | A delegated child agent is created. |
| `subagent_stop` | A delegated child agent returns or fails. |
`subagent_start` fields include `parent_session_id`, `parent_turn_id`,
`parent_subagent_id`, `child_session_id`, `child_subagent_id`, `child_role`,
and `child_goal`.
`subagent_stop` fields include parent/child session IDs, role/status fields,
`child_summary`, and `duration_ms`.
Observers can use these hooks to model nested trajectories while keeping child
agent execution linked to the parent turn that spawned it.
## Payload Safety
Observer payloads are designed for telemetry consumers, not raw object access.
New consumers should use the sanitized API payloads:
- `pre_api_request.request`
- `post_api_request.response`
- `api_request_error.request`
- `api_request_error.error`
Sanitization converts provider objects to JSON-compatible structures, bounds
large payloads, redacts sensitive keys, and avoids exposing raw response
objects in sanitized fields.
Legacy compatibility fields such as `request_messages`, `conversation_history`,
and `assistant_message` may still be present for existing plugins. New
observability consumers should prefer the sanitized payloads.
## Performance
The default uninstrumented path should stay cheap. Expensive request/response
payload construction is gated behind `has_hook(...)`, so Hermes only builds
sanitized API telemetry payloads when at least one plugin registered the
relevant hook.
Plugin authors should preserve this property:
- Register only hooks the plugin actually consumes.
- Avoid deep-copying or re-sanitizing already sanitized payloads.
- Keep hook callbacks fast and fail-open.
- Offload network export or batch writes when practical.
## Writing An Observer Plugin
Minimal observer plugin:
```python
def register(ctx):
ctx.register_hook("pre_api_request", on_pre_api_request)
ctx.register_hook("post_api_request", on_post_api_request)
ctx.register_hook("pre_tool_call", on_pre_tool_call)
ctx.register_hook("post_tool_call", on_post_tool_call)
def on_pre_api_request(**kwargs):
start_llm_span(
request_id=kwargs.get("api_request_id"),
turn_id=kwargs.get("turn_id"),
request=kwargs.get("request"),
model=kwargs.get("model"),
)
def on_post_api_request(**kwargs):
finish_llm_span(
request_id=kwargs.get("api_request_id"),
response=kwargs.get("response"),
usage=kwargs.get("usage"),
duration=kwargs.get("api_duration"),
)
def on_pre_tool_call(**kwargs):
start_tool_span(
call_id=kwargs.get("tool_call_id"),
name=kwargs.get("tool_name"),
args=kwargs.get("args"),
)
def on_post_tool_call(**kwargs):
finish_tool_span(
call_id=kwargs.get("tool_call_id"),
result=kwargs.get("result"),
status=kwargs.get("status"),
duration_ms=kwargs.get("duration_ms"),
)
```
Use `session_id`, `turn_id`, `api_request_id`, and `tool_call_id` for span
correlation. Use subagent and approval hooks when the export format supports
nested agent work or security lifecycle events.
## Existing Consumers
The bundled Langfuse plugin demonstrates direct hook-based observability for
turns, provider requests, and tool calls.
The bundled NeMo Relay plugin maps the same generic observer contract to NeMo
Relay scopes, LLM spans, tool spans, marks, ATOF streams, and ATIF exports.
NeMo Relay-specific configuration and examples live in
[`plugins/observability/nemo_relay/README.md`](../../plugins/observability/nemo_relay/README.md).
@@ -0,0 +1,240 @@
---
title: "fix: Prevent Telegram streamed replies from ending after first overflow chunk"
status: active
date: 2026-06-09
type: fix
target_repo: hermes-agent
origin: user-reported Telegram topic screenshot
---
# fix: Prevent Telegram streamed replies from ending after first overflow chunk
## Summary
Fix a Telegram gateway bug where a long streamed assistant reply can appear to stop mid-answer in a topic after the first overflow chunk. The reported screenshot shows a long Hermes response in the `Nehemiah - Coding` Telegram topic ending at `- The visible tool-call summary`, followed by the user noting that the previous message did not finish streaming to that Telegram topic.
The plan targets the streamed edit overflow path, not general model generation. A completed assistant response must either reach Telegram in full across all continuation messages or leave enough state for the gateway fallback path to deliver the remaining content instead of marking the turn complete after a partial delivery.
---
## Problem Frame
Telegram limits message text to 4096 UTF-16 code units. Hermes streams gateway responses by editing a message and, when a streamed message grows past the limit, splitting the overflow into additional Telegram messages. The adapter already has a split-and-deliver path for oversized edits, but the partial-continuation failure contract is weak: if chunk 1 is edited successfully and a later continuation fails, the adapter can still report success for the operation. The stream consumer may then mark the final response delivered even though the visible topic only contains the first part.
This is especially visible in Telegram forum topics because a long final response can be split below tool-progress bubbles, and a missing continuation looks exactly like the stream stopped mid-answer.
---
## Requirements
- R1. Long streamed Telegram replies must preserve all final content across overflow chunks.
- R2. If any continuation chunk fails after the first overflow edit lands, the gateway must not mark the final response as fully delivered.
- R3. Continuation chunks must remain routed to the same Telegram topic/thread as the original response.
- R4. The fix must avoid duplicate full-answer sends when all overflow chunks were delivered successfully.
- R5. Tests must cover the reported failure shape: a final streamed reply that exceeds Telegram's limit, succeeds on the first edit, fails on a continuation, and must not be treated as complete.
---
## Key Technical Decisions
- Treat overflow delivery as all-or-not-complete. `_edit_overflow_split` should only return a successful final-delivery result when every planned chunk reaches Telegram. Partial delivery is a distinct outcome that downstream code can recover from.
- Carry partial-overflow metadata through `SendResult.raw_response` rather than adding a new public dataclass field unless implementation proves the existing result shape is insufficient. The stream consumer already inspects `SendResult` after adapter edits, so a small raw response contract can keep the change contained.
- Make the stream consumer responsible for final-delivery truth. The adapter knows which chunks landed, but the consumer owns `_final_response_sent`, `_final_content_delivered`, `_fallback_prefix`, and fallback final-send behaviour.
- Keep routing inside Telegram adapter helpers. Continuation sends should continue to use `_thread_kwargs_for_send(...)` with metadata-derived `message_thread_id` and reply anchors so forum topic behaviour stays consistent.
---
## High-Level Technical Design
```mermaid
sequenceDiagram
participant C as GatewayStreamConsumer
participant T as TelegramAdapter.edit_message
participant B as Telegram Bot API
C->>T: finalize/edit long accumulated response
T->>B: edit original message with chunk 1
loop remaining chunks
T->>B: send continuation in same topic/thread
end
alt all chunks delivered
T-->>C: success, last message id, continuation ids
C->>C: mark final response delivered
else any continuation failed
T-->>C: partial overflow failure with delivered prefix metadata
C->>C: do not mark final delivered
C->>B: fallback sends missing tail or full final response safely
end
```
---
## Implementation Units
### U1. Add a partial-overflow contract for Telegram edit splits
**Goal:** Make `TelegramAdapter._edit_overflow_split` distinguish complete overflow delivery from partial delivery.
**Requirements:** R1, R2, R4
**Dependencies:** None
**Files:**
- `gateway/platforms/telegram.py`
- `tests/gateway/test_telegram_send.py` or the existing Telegram adapter test module that already covers `edit_message` overflow behaviour
**Approach:**
- Keep the successful path unchanged when every chunk is delivered: return `SendResult(success=True, message_id=<last chunk>, continuation_message_ids=(...))`.
- When a continuation fails after the first edit, return a result that clearly indicates partial delivery instead of plain success. Prefer `success=False`, `retryable=True`, and `raw_response` metadata such as delivered chunk count, total chunk count, last delivered message id, and the visible delivered prefix.
- Preserve logging, but do not rely on logs as the only signal. The caller must be able to tell partial delivery happened.
- Ensure the first edited chunk and all successful continuation chunks still include the existing Markdown/plain-text fallback behaviour.
**Patterns to follow:**
- Existing overflow handling in `TelegramAdapter.edit_message` and `_edit_overflow_split`.
- Existing `SendResult` semantics in `gateway/platforms/base.py`, especially `retryable`, `raw_response`, and `continuation_message_ids`.
**Test scenarios:**
- Oversized finalized edit where all continuations succeed returns success, the last continuation id, and all continuation ids.
- Oversized finalized edit where the first continuation send fails returns a partial-overflow failure and does not report success.
- Oversized finalized edit where one continuation succeeds and a later continuation fails reports the last delivered continuation id and delivered count in raw metadata.
- A continuation MarkdownV2 formatting failure still retries plain text before being treated as a delivery failure.
**Verification:** Adapter tests prove complete overflow remains successful and partial overflow is observable by the caller.
### U2. Teach the stream consumer to recover from partial overflow
**Goal:** Ensure a partial Telegram overflow does not set `_final_response_sent` or `_final_content_delivered` unless the full response reached the user.
**Requirements:** R1, R2, R4, R5
**Dependencies:** U1
**Files:**
- `gateway/stream_consumer.py`
- `tests/gateway/test_stream_consumer.py` or a focused new `tests/gateway/test_stream_consumer_telegram_overflow.py`
**Approach:**
- In `_send_or_edit`, when `adapter.edit_message(...)` returns a partial-overflow failure, update consumer state to reflect the last visible prefix/message and enter fallback delivery for the missing content.
- Avoid treating `_already_sent` as final delivery. A partial visible message can be true while final delivery is false.
- Use the delivered-prefix metadata if available so `_send_fallback_final(...)` sends only the missing tail. If implementation finds the prefix is unreliable after Markdown formatting, prefer sending the complete final response as a fresh fallback message rather than silently dropping the tail.
- Keep the existing success handling for `continuation_message_ids` when the adapter delivered all chunks.
**Patterns to follow:**
- Existing fallback mode in `GatewayStreamConsumer._send_or_edit` and `_send_fallback_final`.
- Existing comments around `_final_response_sent`, `_final_content_delivered`, and `_fallback_prefix` for prior partial-delivery regressions.
**Test scenarios:**
- A final streamed response that overflows and receives a complete-success edit split sets final-delivery flags and does not invoke fallback.
- A final streamed response whose adapter reports partial overflow does not set final-delivery flags immediately.
- After partial overflow, fallback delivery sends the remaining tail and then marks final content delivered only if the fallback send succeeds.
- If fallback delivery also fails, the consumer leaves final-delivery false so the gateway's non-streaming final-send safety path can still run.
**Verification:** Stream consumer tests reproduce the screenshot shape by simulating first chunk visible and continuation failure, then assert the final answer is not suppressed.
### U3. Preserve Telegram topic/thread routing for overflow and fallback continuations
**Goal:** Ensure overflow recovery messages land in the same Telegram forum topic or DM topic fallback context.
**Requirements:** R3
**Dependencies:** U1, U2
**Files:**
- `gateway/platforms/telegram.py`
- `gateway/stream_consumer.py`
- `tests/gateway/test_stream_consumer_thread_routing.py`
- Relevant Telegram adapter routing tests, if existing coverage is closer there
**Approach:**
- Keep passing `metadata` through every overflow continuation and fallback send.
- Keep reply anchors where valid, but do not let a missing reply anchor drop the `message_thread_id` for normal forum topics.
- For private DM topic fallback metadata, preserve the existing stricter anchor behaviour documented in the adapter comments.
**Patterns to follow:**
- `TelegramAdapter._thread_kwargs_for_send(...)`.
- Existing tests around Telegram topic recovery and stream consumer thread routing.
**Test scenarios:**
- Overflow continuations include `message_thread_id` for a forum topic.
- A continuation retry after `reply message not found` keeps forum topic routing when allowed.
- Partial-overflow fallback sends receive the same metadata passed to the original stream consumer.
**Verification:** Thread-routing assertions inspect fake bot calls and confirm all continuation/fallback messages carry the expected topic metadata.
### U4. Add issue evidence and PR body traceability
**Goal:** Make the upstream issue and PR clearly trace the user-visible bug and verification evidence.
**Requirements:** R5
**Dependencies:** U1, U2, U3
**Files:**
- GitHub issue body created via `gh issue create`
- PR body using `.github/PULL_REQUEST_TEMPLATE.md`
**Approach:**
- Create a GitHub issue with the screenshot evidence: the long message in the `Nehemiah - Coding` Telegram topic stops at `- The visible tool-call summary`, and the user's reply says the previous message did not finish streaming to that Telegram topic.
- Reference affected component as Gateway and platform as Telegram.
- In the PR body, link the issue with `Fixes #...`, describe the split-delivery contract change, and include the screenshot or attach it if GitHub upload is available.
- Follow `CONTRIBUTING.md` and the repository PR template exactly.
**Patterns to follow:**
- `.github/ISSUE_TEMPLATE/bug_report.yml`
- `.github/PULL_REQUEST_TEMPLATE.md`
**Test scenarios:**
- Test expectation: none, this is tracker and PR documentation work.
**Verification:** The GitHub issue exists with screenshot evidence or an explicit screenshot reference, and the PR body links the issue and lists the tests run.
---
## Scope Boundaries
### In Scope
- Telegram streamed response overflow splitting and recovery.
- Stream consumer final-delivery truth for partial overflow delivery.
- Topic/thread metadata preservation for overflow and fallback continuation sends.
- Focused unit tests around adapter and stream consumer behaviour.
### Out of Scope
- Changing model streaming semantics in `run_agent.py`.
- Reworking Telegram draft streaming, which is DM-only and not the forum-topic path in the screenshot.
- Changing general platform message splitting for Discord, Slack, WhatsApp, or Matrix unless a shared helper must be corrected for the Telegram fix.
- Altering tool-progress display settings or terminal progress rendering.
### Deferred to Follow-Up Work
- Broader observability for gateway delivery completeness across all messaging platforms.
- A user-facing resend/recover command for a previous truncated response.
---
## Risks & Mitigations
- Risk: fallback recovery duplicates already-visible first chunks. Mitigation: use delivered-prefix metadata where reliable and add tests for no-duplicate complete-success behaviour.
- Risk: preserving forum topic routing while dropping invalid reply anchors is easy to regress. Mitigation: include fake bot call assertions for `message_thread_id` and reply behaviour.
- Risk: MarkdownV2 formatting can alter visible/raw prefix comparisons. Mitigation: keep fallback conservative; duplicate content is preferable to silently missing content, but tests should keep the common path tail-only.
---
## Sources & Research
- User-provided screenshot at `/root/.hermes/image_cache/img_f664e68f6ddf.jpg`.
- `gateway/stream_consumer.py` streamed edit, overflow, fallback, and final-delivery state handling.
- `gateway/platforms/telegram.py` Telegram send/edit overflow splitting and topic routing helpers.
- `gateway/platforms/base.py` `SendResult` contract and shared message chunking helper.
- `tests/gateway/test_stream_consumer.py`, `tests/gateway/test_stream_consumer_thread_routing.py`, and Telegram adapter tests for focused regression coverage.
---
## Verification Strategy
- Run focused Telegram adapter overflow tests.
- Run focused stream consumer overflow/fallback tests.
- Run topic-routing tests affected by metadata changes.
- Run the gateway test subset around Telegram send/edit, stream consumer, and run progress if touched.
- Before PR creation, ensure `git diff` contains only the plan, implementation, tests, and PR/issue-relevant documentation for this bug.
+54
View File
@@ -0,0 +1,54 @@
# RCA: SSL CA cert bundle corruption after `hermes update`
**Status:** resolved by `fix(ssl): surface broken CA bundles before provider calls`
**Severity:** P2 — degrades the agent into opaque provider/client failures until the user repairs deps or CA configuration.
## Summary
A partial `hermes update`, interrupted venv repair, or stale CA-bundle environment variable can leave Python TLS configuration pointing at a missing, empty, or unloadable CA bundle. The first outbound HTTPS client creation or request can then fail with a raw `FileNotFoundError: [Errno 2] No such file or directory` or a low-level SSL error that does not name the broken CA path.
## Root cause
Hermes uses OpenAI/httpx and requests-based clients for provider calls, model metadata, gateway delivery, and web tools. Those clients inherit CA bundle settings from:
- `HERMES_CA_BUNDLE`
- `SSL_CERT_FILE`
- `REQUESTS_CA_BUNDLE`
- `CURL_CA_BUNDLE`
- the bundled `certifi` package's `cacert.pem`
When the venv is partially refreshed, or when one of those env vars points at a file that no longer exists, provider client construction can fail before Hermes has enough context to produce a useful message.
## Fix
`agent/ssl_guard.py` validates CA bundle configuration before the OpenAI-compatible provider client is created in `agent/agent_init.py`. It:
1. Checks explicit CA bundle env vars and reports the exact broken variable/path,
2. Verifies `certifi` is importable,
3. Verifies `certifi.where()` points at an existing file of plausible size,
4. Builds an `ssl.SSLContext` from each checked bundle,
5. Raises a typed `SSLConfigurationError` with a repair hint before httpx/OpenAI can raise a raw low-level error.
`hermes_cli doctor` exposes the same check under `SSL / CA Certificates`, so users can diagnose the problem without starting a model session.
## Recovery
When the guard fires during agent init, the user sees a message like:
```text
Failed to initialize OpenAI client: SSL_CERT_FILE points to a missing CA bundle: C:\path\to\missing\cacert.pem
Repair: python -m pip install --force-reinstall certifi openai httpx
If you configured a custom corporate CA bundle, fix or unset the broken CA bundle environment variable.
```
For a normal corrupted Hermes venv, reinstall the affected client dependencies:
```bash
python -m pip install --force-reinstall certifi openai httpx
```
For a custom/corporate CA setup, fix the env var so it points at a real PEM bundle, or unset it if Hermes should use the bundled `certifi` store.
## Environment escape hatch
Set `HERMES_SKIP_SSL_GUARD=1` to bypass the preflight check. This is intended only for sandboxed or managed-trust environments where the Python CA path looks unusual but downstream clients are known to work.
+195
View File
@@ -0,0 +1,195 @@
# Network Egress Isolation for Docker Deployments
When running Hermes inside Docker, the default `network_mode: host` gives the
agent process unrestricted outbound network access. This guide shows how to
segment traffic so the agent core can only reach the services it needs, while
blocking arbitrary outbound connections.
This is primarily a defense against prompt injection attacks that attempt to
exfiltrate data via `curl`, `wget`, or raw HTTP from tool-generated shell
commands.
## Threat Model
The Hermes [SECURITY.md](../../SECURITY.md) §2 defines the trust model. The
terminal backend is the primary execution boundary. However, when running with
`network_mode: host`, any command the agent executes can reach any endpoint on
the network, including external ones.
Network egress isolation adds a second layer: even if a malicious command
executes inside the container, it cannot reach endpoints outside the
explicitly allowlisted set.
## Architecture
```
┌─────────────────────────────────────────────┐
│ Docker Network: internal (no internet) │
│ │
│ ┌──────────────┐ ┌──────────────────┐ │
│ │ hermes-agent │ │ hermes-dashboard │ │
│ └──────┬───────┘ └────────┬─────────┘ │
│ │ │ │
│ ▼ │ │
│ ┌──────────────┐ │ │
│ │ hermes-gtw │◄───────────┘ │
│ └──────┬───────┘ │
│ │ │
└──────────┼───────────────────────────────────┘
┌──────────┼───────────────────────────────────┐
│ Docker Network: egress (internet-capable) │
│ │ │
│ ▼ │
│ ┌─────────────────┐ │
│ │ egress-proxy │──► allowlisted hosts │
│ │ (squid / envoy) │ │
│ └─────────────────┘ │
└──────────────────────────────────────────────┘
```
Two Docker networks:
- **`internal`** — no default route, no internet access. The agent, dashboard,
and gateway run here.
- **`egress`** — has internet access. Only services that need to reach external
APIs are attached to this network.
The gateway service is dual-homed (attached to both networks) so it can
receive inbound messages from Telegram/Slack/etc. and forward them to the
agent on the internal network.
## Compose Configuration
Override the default `docker-compose.yml` with a
`docker-compose.override.yml`:
```yaml
# docker-compose.override.yml
# Network egress isolation for production deployments.
#
# Usage:
# HERMES_UID=$(id -u) HERMES_GID=$(id -g) docker compose up -d
#
# This overrides network_mode: host with isolated Docker networks.
networks:
internal:
driver: bridge
internal: true # no default route, no internet
egress:
driver: bridge
services:
gateway:
network_mode: "" # clear the host-mode default
networks:
- internal
- egress # needs outbound for Telegram, LLM APIs
ports:
- "127.0.0.1:9119:9119" # dashboard proxy, localhost only
dashboard:
network_mode: ""
networks:
- internal # internal only, no egress needed
```
### With an Egress Proxy (Recommended)
For tighter control, route all outbound traffic through an HTTP proxy with
an explicit allowlist:
```yaml
# docker-compose.override.yml (with egress proxy)
networks:
internal:
driver: bridge
internal: true
egress:
driver: bridge
services:
gateway:
network_mode: ""
networks:
- internal
- egress
environment:
- HTTP_PROXY=http://egress-proxy:3128
- HTTPS_PROXY=http://egress-proxy:3128
- NO_PROXY=hermes,hermes-dashboard,localhost
dashboard:
network_mode: ""
networks:
- internal
egress-proxy:
image: ubuntu/squid:6.10-24.04_edge
networks:
- egress
volumes:
- ./config/squid-allowlist.conf:/etc/squid/conf.d/allowlist.conf:ro
restart: unless-stopped
```
Example `config/squid-allowlist.conf`:
```
# Only allow HTTPS CONNECT to these hosts
acl allowed_hosts dstdomain api.openai.com
acl allowed_hosts dstdomain api.anthropic.com
acl allowed_hosts dstdomain openrouter.ai
acl allowed_hosts dstdomain generativelanguage.googleapis.com
acl allowed_hosts dstdomain api.telegram.org
acl allowed_hosts dstdomain api.github.com
acl allowed_hosts dstdomain discord.com
http_access allow CONNECT allowed_hosts
http_access deny all
```
Adjust the allowlist to match your LLM provider and messaging platform.
## Validating the Setup
After bringing up the stack, verify isolation:
```bash
# From the agent container: this should FAIL (no egress)
docker compose exec gateway \
curl -sf --max-time 5 https://example.com && echo "FAIL: egress not blocked" || echo "OK: egress blocked"
# From the agent container: this should SUCCEED (internal network)
docker compose exec gateway \
curl -sf --max-time 5 http://hermes-dashboard:9119/health && echo "OK: internal reachable" || echo "FAIL"
# If using egress proxy: this should SUCCEED (allowlisted)
docker compose exec gateway \
curl -sf --max-time 5 --proxy http://egress-proxy:3128 https://api.openai.com/v1/models && echo "OK" || echo "FAIL"
```
## Limitations
- **DNS resolution:** The `internal` network can still resolve external DNS
names unless you also run a local DNS resolver that blocks external queries.
For most threat models this is acceptable since DNS resolution alone does not
exfiltrate meaningful data.
- **Not a substitute for sandbox backends:** This guide isolates the agent
*container's* network. If you use the default local terminal backend, tool
commands execute inside the same container. For stronger isolation, combine
network segmentation with a sandboxed terminal backend (Docker, Modal,
Daytona).
- **Platform adapters need egress:** The gateway service needs outbound access
to reach messaging platform APIs. If you add new platform adapters, add their
API endpoints to the proxy allowlist.
## Related
- [SECURITY.md](../../SECURITY.md) — Hermes trust model and vulnerability reporting
- [Terminal backends](../../README.md) — sandboxed execution targets
- [docker-compose.yml](../../docker-compose.yml) — default compose configuration