Zakaria/hermes-agent
1
0
Fork 1
Code Issues Pull Requests 1 Actions Packages Projects Releases Wiki Activity

Hermes-agent

Browse Source
This commit is contained in:
Zakaria
2026-06-14 14:30:48 -04:00
commit dac4b88b94
5058 changed files with 1884848 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
website/docs/user-guide/features/_category_.json
+8
View File
@@ -0,0 +1,8 @@
{
"label": "Features",
"position": 4,
"link": {
"type": "generated-index",
"description": "Explore the powerful features of Hermes Agent."
}
}
website/docs/user-guide/features/acp.md
+277
View File
@@ -0,0 +1,277 @@
---
sidebar_position: 11
title: "ACP Editor Integration"
description: "Use Hermes Agent inside ACP-compatible editors such as VS Code, Zed, and JetBrains"
---
# ACP Editor Integration
Hermes Agent can run as an ACP server, letting ACP-compatible editors talk to Hermes over stdio and render:
- chat messages
- tool activity
- file diffs
- terminal commands
- approval prompts
- streamed thinking / response chunks
ACP is a good fit when you want Hermes to behave like an editor-native coding agent instead of a standalone CLI or messaging bot.
## What Hermes exposes in ACP mode
Hermes runs with a curated `hermes-acp` toolset designed for editor workflows. It includes:
- file tools: `read_file`, `write_file`, `patch`, `search_files`
- terminal tools: `terminal`, `process`
- web/browser tools
- memory, todo, session search
- skills
- execute_code and delegate_task
- vision
It intentionally excludes things that do not fit typical editor UX, such as messaging delivery and cronjob management.
## Installation
Install Hermes normally, then add the ACP extra:
```bash
pip install -e '.[acp]'
```
This installs the `agent-client-protocol` dependency and enables:
- `hermes acp`
- `hermes-acp`
- `python -m acp_adapter`
For Zed registry installs, Zed launches Hermes through the official ACP Registry entry. That entry uses a `uvx` distribution that runs:
```bash
uvx --from 'hermes-agent[acp]==<version>' hermes-acp
```
Make sure `uv` is available on `PATH` before using the registry install path.
## Launching the ACP server
Any of the following starts Hermes in ACP mode:
```bash
hermes acp
```
```bash
hermes-acp
```
```bash
python -m acp_adapter
```
Hermes logs to stderr so stdout remains reserved for ACP JSON-RPC traffic.
For non-interactive checks:
```bash
hermes acp --version
hermes acp --check
```
### Browser tools (optional)
Browser tools (`browser_navigate`, `browser_click`, etc.) depend on the
`agent-browser` npm package and Chromium, which aren't part of the Python
wheel. Install them with:
```bash
hermes acp --setup-browser # interactive (prompts before ~400 MB download)
hermes acp --setup-browser --yes # accept the download non-interactively
```
This is the standalone command. The Zed registry's terminal-auth flow (`hermes acp --setup`) also offers the browser bootstrap as a follow-up question after model selection, so most users never need to run `--setup-browser` directly.
What it does:
- Installs Node.js 22 LTS into `~/.hermes/node/` if missing
- `npm install -g agent-browser @askjo/camofox-browser` into that prefix (no sudo needed — `npm`'s `--prefix` points at the user-writable Hermes-managed Node)
- Installs Playwright Chromium, or uses a detected system Chrome/Chromium when available
The bootstrap is idempotent — re-running it is fast and skips work that's already done.
## Editor setup
### VS Code
Install the [ACP Client](https://marketplace.visualstudio.com/items?itemName=formulahendry.acp-client) extension.
To connect:
1. Open the ACP Client panel from the Activity Bar.
2. Select **Hermes Agent** from the built-in agent list.
3. Connect and start chatting.
If you want to define Hermes manually, add it through VS Code settings under `acp.agents`:
```json
{
"acp.agents": {
"Hermes Agent": {
"command": "hermes",
"args": ["acp"]
}
}
}
```
### Zed
Zed v0.221.x and newer installs external agents through the official ACP Registry.
1. Open the Agent Panel.
2. Click **Add Agent**, or run the `zed: acp registry` command.
3. Search for **Hermes Agent**.
4. Install it and start a new Hermes external-agent thread.
Prerequisites:
- Configure Hermes provider credentials first with `hermes model`, or set them in `~/.hermes/.env` / `~/.hermes/config.yaml`.
- Install `uv` so the registry launcher can run `uvx --from 'hermes-agent[acp]==<version>' hermes-acp`.
For local development before the registry entry is available, use a custom agent server in Zed settings:
```json
{
"agent_servers": {
"hermes-agent": {
"type": "custom",
"command": "hermes",
"args": ["acp"]
}
}
}
```
### JetBrains
Use an ACP-compatible plugin and point it at:
```text
/path/to/hermes-agent/acp_registry
```
## Registry manifest
The source copy of Hermes' official ACP Registry metadata lives at:
```text
acp_registry/agent.json
acp_registry/icon.svg
```
The upstream registry PR copies those files into the top-level `hermes-agent/` directory in `agentclientprotocol/registry`.
The registry entry uses a `uvx` distribution that points directly at the `hermes-agent` PyPI release:
```text
uvx --from 'hermes-agent[acp]==<version>' hermes-acp
```
The registry CI verifies that the pinned version exists on PyPI, so the manifest's `version` and uvx `package` pin must always match `pyproject.toml`. `scripts/release.py` keeps them in lockstep automatically.
## Configuration and credentials
ACP mode uses the same Hermes configuration as the CLI:
- `~/.hermes/.env`
- `~/.hermes/config.yaml`
- `~/.hermes/skills/`
- `~/.hermes/state.db`
Provider resolution uses Hermes' normal runtime resolver, so ACP inherits the currently configured provider and credentials. Hermes also advertises a terminal auth method (`--setup`) for first-run registry clients; this opens Hermes' interactive model/provider setup.
## Session behavior
ACP sessions are tracked by the ACP adapter's in-memory session manager while the server is running.
Each session stores:
- session ID
- working directory
- selected model
- current conversation history
- cancel event
The underlying `AIAgent` still uses Hermes' normal persistence/logging paths, but ACP `list/load/resume/fork` are scoped to the currently running ACP server process.
## Working directory behavior
ACP sessions bind the editor's cwd to the Hermes task ID so file and terminal tools run relative to the editor workspace, not the server process cwd.
## Approvals
Dangerous terminal commands can be routed back to the editor as approval prompts. ACP approval options are simpler than the CLI flow:
- allow once
- allow always
- deny
On timeout or error, the approval bridge denies the request.
### Session-scoped edit auto-approval
ACP exposes a third tier between *allow once* and *allow always*: **Allow for session**. Picking it from the editor's permission prompt records the approval inside the current ACP session only — every subsequent matching command in that session goes through without prompting, but a new ACP session (or restarting the editor) resets the slate and re-prompts the first time.
| Option | Editor label | Scope | Persisted across restarts |
|---|---|---|---|
| `allow_once` | Allow once | This one tool call | No |
| `allow_session` | Allow for session | All matching calls in this ACP session | No — cleared when the session ends |
| `allow_always` | Allow always | All future sessions | Yes (written to the Hermes permanent allowlist) |
| `deny` | Deny | This one tool call | No |
`allow_session` is the right default for an editor workflow where you trust an agent for the duration of a task but don't want to grant a long-lived allowlist entry. The safety trade-off is straightforward: the broader the scope, the less the editor will interrupt you, and the more damage a misbehaving agent (or prompt injection) can do before you notice. Start with `allow_once` for unfamiliar commands; promote to `allow_session` once you've seen the agent run the same pattern correctly a few times; reserve `allow_always` for truly idempotent commands you trust forever (e.g. `git status`).
The ACP bridge maps these options onto Hermes' internal approval semantics — `allow_always` writes a permanent allowlist entry the same way the CLI does, while `allow_session` only affects the in-process approval cache for the current ACP session.
## Troubleshooting
### ACP agent does not appear in the editor
Check:
- In Zed, open the ACP Registry with `zed: acp registry` and search for **Hermes Agent**.
- For manual/local development, verify the custom `agent_servers` command points to `hermes acp`.
- Hermes is installed and on your PATH.
- The ACP extra is installed (`pip install -e '.[acp]'`).
- `uv` is installed if launching from the official Zed registry entry.
### ACP starts but immediately errors
Try these checks:
```bash
hermes acp --version
hermes acp --check
hermes doctor
hermes status
```
### Missing credentials
ACP mode uses Hermes' existing provider setup. Configure credentials with:
```bash
hermes model
```
or by editing `~/.hermes/.env`. Registry clients can also trigger Hermes' terminal auth flow, which runs the same interactive provider/model setup.
### Zed registry launcher cannot find uv
Install `uv` from the official uv installation docs, then retry the Hermes Agent thread from Zed.
## See also
- [ACP Internals](../../developer-guide/acp-internals.md)
- [Provider Runtime Resolution](../../developer-guide/provider-runtime.md)
- [Tools Runtime](../../developer-guide/tools-runtime.md)
website/docs/user-guide/features/api-server.md
+503
View File
@@ -0,0 +1,503 @@
---
sidebar_position: 14
title: "API Server"
description: "Expose hermes-agent as an OpenAI-compatible API for any frontend"
---
# API Server
The API server exposes hermes-agent as an OpenAI-compatible HTTP endpoint. Any frontend that speaks the OpenAI format — Open WebUI, LobeChat, LibreChat, NextChat, ChatBox, and hundreds more — can connect to hermes-agent and use it as a backend.
Your agent handles requests with its full toolset (terminal, file operations, web search, memory, skills) and returns the final response. When streaming, tool progress indicators appear inline so frontends can show what the agent is doing.
:::tip One backend covers models + tools
Hermes itself needs a configured provider and tool backends for the API server to be useful. A [Nous Portal](/user-guide/features/tool-gateway) subscription handles both — 300+ models plus web/image/TTS/browser via the Tool Gateway. Run `hermes setup --portal` once before starting the API server and frontends like Open WebUI or LobeChat get a fully tool-equipped backend.
:::
## Quick Start
### 1. Enable the API server
Add to `~/.hermes/.env`:
```bash
API_SERVER_ENABLED=true
API_SERVER_KEY=change-me-local-dev
# Optional: only if a browser must call Hermes directly
# API_SERVER_CORS_ORIGINS=http://localhost:3000
```
### 2. Start the gateway
```bash
hermes gateway
```
You'll see:
```
[API Server] API server listening on http://127.0.0.1:8642
```
### 3. Connect a frontend
Point any OpenAI-compatible client at `http://localhost:8642/v1`:
```bash
# Test with curl
curl http://localhost:8642/v1/chat/completions \
-H "Authorization: Bearer change-me-local-dev" \
-H "Content-Type: application/json" \
-d '{"model": "hermes-agent", "messages": [{"role": "user", "content": "Hello!"}]}'
```
Or connect Open WebUI, LobeChat, or any other frontend — see the [Open WebUI integration guide](/user-guide/messaging/open-webui) for step-by-step instructions.
## Endpoints
### POST /v1/chat/completions
Standard OpenAI Chat Completions format. Stateless — the full conversation is included in each request via the `messages` array.
**Request:**
```json
{
"model": "hermes-agent",
"messages": [
{"role": "system", "content": "You are a Python expert."},
{"role": "user", "content": "Write a fibonacci function"}
],
"stream": false
}
```
**Response:**
```json
{
"id": "chatcmpl-abc123",
"object": "chat.completion",
"created": 1710000000,
"model": "hermes-agent",
"choices": [{
"index": 0,
"message": {"role": "assistant", "content": "Here's a fibonacci function..."},
"finish_reason": "stop"
}],
"usage": {"prompt_tokens": 50, "completion_tokens": 200, "total_tokens": 250}
}
```
**Inline image input:** user messages may send `content` as an array of `text` and `image_url` parts. Both remote `http(s)` URLs and `data:image/...` URLs are supported:
```json
{
"model": "hermes-agent",
"messages": [
{
"role": "user",
"content": [
{"type": "text", "text": "What is in this image?"},
{"type": "image_url", "image_url": {"url": "https://example.com/cat.png", "detail": "high"}}
]
}
]
}
```
Uploaded files (`file` / `input_file` / `file_id`) and non-image `data:` URLs return `400 unsupported_content_type`.
**Streaming** (`"stream": true`): Returns Server-Sent Events (SSE) with token-by-token response chunks. For **Chat Completions**, the stream uses standard `chat.completion.chunk` events plus Hermes' custom `hermes.tool.progress` event for tool-start UX. For **Responses**, the stream uses OpenAI Responses event types such as `response.created`, `response.output_text.delta`, `response.output_item.added`, `response.output_item.done`, and `response.completed`.
**Tool progress in streams**:
- **Chat Completions**: Hermes emits `event: hermes.tool.progress` for tool-start visibility without polluting persisted assistant text.
- **Responses**: Hermes emits spec-native `function_call` and `function_call_output` output items during the SSE stream, so clients can render structured tool UI in real time.
### POST /v1/responses
OpenAI Responses API format. Supports server-side conversation state via `previous_response_id` — the server stores full conversation history (including tool calls and results) so multi-turn context is preserved without the client managing it.
**Request:**
```json
{
"model": "hermes-agent",
"input": "What files are in my project?",
"instructions": "You are a helpful coding assistant.",
"store": true
}
```
**Response:**
```json
{
"id": "resp_abc123",
"object": "response",
"status": "completed",
"model": "hermes-agent",
"output": [
{"type": "function_call", "name": "terminal", "arguments": "{\"command\": \"ls\"}", "call_id": "call_1"},
{"type": "function_call_output", "call_id": "call_1", "output": "README.md src/ tests/"},
{"type": "message", "role": "assistant", "content": [{"type": "output_text", "text": "Your project has..."}]}
],
"usage": {"input_tokens": 50, "output_tokens": 200, "total_tokens": 250}
}
```
**Inline image input:** `input[].content` can contain `input_text` and `input_image` parts. Both remote URLs and `data:image/...` URLs are supported:
```json
{
"model": "hermes-agent",
"input": [
{
"role": "user",
"content": [
{"type": "input_text", "text": "Describe this screenshot."},
{"type": "input_image", "image_url": "data:image/png;base64,iVBORw0K..."}
]
}
]
}
```
Uploaded files (`input_file` / `file_id`) and non-image `data:` URLs return `400 unsupported_content_type`.
#### Multi-turn with previous_response_id
Chain responses to maintain full context (including tool calls) across turns:
```json
{
"input": "Now show me the README",
"previous_response_id": "resp_abc123"
}
```
The server reconstructs the full conversation from the stored response chain — all previous tool calls and results are preserved. Chained requests also share the same session, so multi-turn conversations appear as a single entry in the dashboard and session history.
#### Named conversations
Use the `conversation` parameter instead of tracking response IDs:
```json
{"input": "Hello", "conversation": "my-project"}
{"input": "What's in src/?", "conversation": "my-project"}
{"input": "Run the tests", "conversation": "my-project"}
```
The server automatically chains to the latest response in that conversation. Like the `/title` command for gateway sessions.
### GET /v1/responses/\{id\}
Retrieve a previously stored response by ID.
### DELETE /v1/responses/\{id\}
Delete a stored response.
### GET /v1/models
Lists the agent as an available model. The advertised model name defaults to the [profile](/user-guide/profiles) name (or `hermes-agent` for the default profile). Required by most frontends for model discovery.
### GET /v1/capabilities
Returns a machine-readable description of the API server's stable surface for external UIs, orchestrators, and plugin bridges.
```json
{
"object": "hermes.api_server.capabilities",
"platform": "hermes-agent",
"model": "hermes-agent",
"auth": {"type": "bearer", "required": true},
"features": {
"chat_completions": true,
"responses_api": true,
"run_submission": true,
"run_status": true,
"run_events_sse": true,
"run_stop": true
}
}
```
Use this endpoint when integrating dashboards, browser UIs, or control planes so they can discover whether the running Hermes version supports runs, streaming, cancellation, and session continuity without depending on private Python internals.
### GET /health
Health check. Returns `{"status": "ok"}`. Also available at **GET /v1/health** for OpenAI-compatible clients that expect the `/v1/` prefix.
### GET /health/detailed
Extended health check that also reports active sessions, running agents, and resource usage. Useful for monitoring/observability tooling.
## Runs API (streaming-friendly alternative)
In addition to `/v1/chat/completions` and `/v1/responses`, the server exposes a **runs** API for long-form sessions where the client wants to subscribe to progress events instead of managing streaming themselves.
### POST /v1/runs
Create a new agent run. Returns a `run_id` that can be used to subscribe to progress events.
```json
{
"run_id": "run_abc123",
"status": "started"
}
```
Runs accept a simple `input` string and optional `session_id`, `instructions`, `conversation_history`, or `previous_response_id`. When `session_id` is provided, Hermes surfaces it in the run status so external UIs can correlate runs with their own conversation IDs.
### GET /v1/runs/\{run_id\}
Poll the current run state. This is useful for dashboards that need status without holding an SSE connection open, or for UIs that reconnect after navigation.
```json
{
"object": "hermes.run",
"run_id": "run_abc123",
"status": "completed",
"session_id": "space-session",
"model": "hermes-agent",
"output": "Done.",
"usage": {"input_tokens": 50, "output_tokens": 200, "total_tokens": 250}
}
```
Statuses are retained briefly after terminal states (`completed`, `failed`, or `cancelled`) for polling and UI reconciliation.
### GET /v1/runs/\{run_id\}/events
Server-Sent Events stream of the run's tool-call progress, token deltas, and lifecycle events. Designed for dashboards and thick clients that want to attach/detach without losing state.
### POST /v1/runs/\{run_id\}/stop
Interrupt a running agent turn. The endpoint returns immediately with `{"status": "stopping"}` while Hermes asks the active agent to stop at the next safe interruption point.
### POST /v1/runs/\{run_id\}/approval
Resolve a pending approval for a run that is waiting on a human decision (for example, a tool call gated behind an approval policy). The body carries the approval decision; the run resumes once the decision is recorded. This endpoint is advertised in `/v1/capabilities` as the `run_approval` feature so external UIs can detect support before surfacing an approval prompt.
## Jobs API (background scheduled work)
The server exposes a lightweight jobs CRUD surface for managing scheduled / background agent runs from a remote client. All endpoints are gated behind the same bearer auth.
### GET /api/jobs
List all scheduled jobs.
### POST /api/jobs
Create a new scheduled job. Body accepts the same shape as `hermes cron` — prompt, schedule, skills, provider override, delivery target.
### GET /api/jobs/\{job_id\}
Fetch a single job's definition and last-run state.
### PATCH /api/jobs/\{job_id\}
Update fields on an existing job (prompt, schedule, etc.). Partial updates are merged.
### DELETE /api/jobs/\{job_id\}
Remove a job. Also cancels any in-flight run.
### POST /api/jobs/\{job_id\}/pause
Pause a job without deleting it. Next-scheduled-run timestamps are suspended until resumed.
### POST /api/jobs/\{job_id\}/resume
Resume a previously paused job.
### POST /api/jobs/\{job_id\}/run
Trigger the job to run immediately, out of schedule.
## Sessions API (session control over REST)
External UIs can manage Hermes sessions over REST without standing up the dashboard. All endpoints are gated by `API_SERVER_KEY` and live under `/api/sessions/*`.
| Method | Path | Description |
|--------|------|-------------|
| `GET` | `/api/sessions` | List sessions (paginated — `limit`, `offset`, `source`, `include_children`) |
| `POST` | `/api/sessions` | Create an empty session |
| `GET` | `/api/sessions/{id}` | Read session metadata |
| `PATCH` | `/api/sessions/{id}` | Update title or `end_reason` |
| `DELETE` | `/api/sessions/{id}` | Delete a session |
| `GET` | `/api/sessions/{id}/messages` | Message history for a session |
| `POST` | `/api/sessions/{id}/fork` | Branch the session via `SessionDB` lineage (matches CLI `/branch` semantics) |
| `POST` | `/api/sessions/{id}/chat` | Run one synchronous agent turn |
| `POST` | `/api/sessions/{id}/chat/stream` | SSE wrapper over a single turn — emits `assistant.delta`, `tool.started`, `tool.completed`, `run.completed` events |
`/v1/capabilities` advertises the full surface via `session_*` feature flags and `endpoints.session_*` entries so external UIs can detect support and fall back safely. Inline images are supported in `chat` and `chat/stream` payloads (multimodal-aware path).
```bash
# fork a session and run one turn
curl -X POST http://localhost:8642/api/sessions/$ID/fork \
-H "Authorization: Bearer $API_SERVER_KEY" \
-d '{"title": "explore alt path"}'
# stream a turn over SSE
curl -N -X POST http://localhost:8642/api/sessions/$ID/chat/stream \
-H "Authorization: Bearer $API_SERVER_KEY" \
-d '{"input": "what files changed in the last hour?"}'
```
## Skills and toolsets discovery
`GET /v1/skills` and `GET /v1/toolsets` let external clients enumerate the agent's capabilities deterministically over REST instead of asking the model. Both are read-only and gated by `API_SERVER_KEY`.
```bash
curl http://localhost:8642/v1/skills \
-H "Authorization: Bearer $API_SERVER_KEY"
# → [{"name": "github-pr-workflow", "description": "...", "category": "..."}, ...]
curl http://localhost:8642/v1/toolsets \
-H "Authorization: Bearer $API_SERVER_KEY"
# → [{"name": "core", "label": "...", "description": "...", "enabled": true,
# "configured": true, "tools": ["read_file", "write_file", ...]}, ...]
```
`/v1/skills` returns the same metadata the skills hub uses internally. `/v1/toolsets` returns toolsets resolved for the `api_server` platform with the concrete `tools` list each one expands to. Both are advertised under `endpoints.*` in `/v1/capabilities`.
## Long-term memory scoping (`X-Hermes-Session-Key`)
Multi-user frontends like Open WebUI need a stable per-channel identifier for long-term memory (Honcho, etc.) that is **independent** of the transcript-scoped `X-Hermes-Session-Id` (which rotates on `/new`). Pass `X-Hermes-Session-Key` on `/v1/chat/completions`, `/v1/responses`, or `/v1/runs` and Hermes threads it through to `AIAgent(gateway_session_key=...)`, where the Honcho memory provider uses it to derive a stable scope.
```http
POST /v1/chat/completions HTTP/1.1
Authorization: Bearer ***
X-Hermes-Session-Id: transcript-alpha
X-Hermes-Session-Key: agent:main:webui:dm:user-42
```
Rules: max 256 chars, control characters (`\r`, `\n`, `\x00`) are rejected, and the value is echoed back on responses (JSON + SSE). `/v1/capabilities` advertises support via `"session_key_header": "X-Hermes-Session-Key"`. Without the key, Honcho's `per-session` strategy produces a different scope per `session_id` — exactly the behavior Hermes had before.
## System Prompt Handling
When a frontend sends a `system` message (Chat Completions) or `instructions` field (Responses API), hermes-agent **layers it on top** of its core system prompt. Your agent keeps all its tools, memory, and skills — the frontend's system prompt adds extra instructions.
This means you can customize behavior per-frontend without losing capabilities:
- Open WebUI system prompt: "You are a Python expert. Always include type hints."
- The agent still has terminal, file tools, web search, memory, etc.
## Authentication
Bearer token auth via the `Authorization` header:
```
Authorization: Bearer ***
```
Configure the key via `API_SERVER_KEY` env var. If you need a browser to call Hermes directly, also set `API_SERVER_CORS_ORIGINS` to an explicit allowlist.
:::warning Security
The API server gives full access to hermes-agent's toolset, **including terminal commands**. `API_SERVER_KEY` is **required for every deployment**, including the default loopback bind on `127.0.0.1`. Keep `API_SERVER_CORS_ORIGINS` narrow to control browser access when you explicitly allow browser callers.
:::
## Configuration
### Environment Variables
| Variable | Default | Description |
|----------|---------|-------------|
| `API_SERVER_ENABLED` | `false` | Enable the API server |
| `API_SERVER_PORT` | `8642` | HTTP server port |
| `API_SERVER_HOST` | `127.0.0.1` | Bind address (localhost only by default) |
| `API_SERVER_KEY` | _(required)_ | Bearer token for auth |
| `API_SERVER_CORS_ORIGINS` | _(none)_ | Comma-separated allowed browser origins |
| `API_SERVER_MODEL_NAME` | _(profile name)_ | Model name on `/v1/models`. Defaults to profile name, or `hermes-agent` for default profile. |
### config.yaml
```yaml
# Not yet supported — use environment variables.
# config.yaml support coming in a future release.
```
## Security Headers
All responses include security headers:
- `X-Content-Type-Options: nosniff` — prevents MIME type sniffing
- `Referrer-Policy: no-referrer` — prevents referrer leakage
## CORS
The API server does **not** enable browser CORS by default.
For direct browser access, set an explicit allowlist:
```bash
API_SERVER_CORS_ORIGINS=http://localhost:3000,http://127.0.0.1:3000
```
When CORS is enabled:
- **Preflight responses** include `Access-Control-Max-Age: 600` (10 minute cache)
- **SSE streaming responses** include CORS headers so browser EventSource clients work correctly
- **`Idempotency-Key`** is an allowed request header — clients can send it for deduplication (responses are cached by key for 5 minutes)
Most documented frontends such as Open WebUI connect server-to-server and do not need CORS at all.
## Compatible Frontends
Any frontend that supports the OpenAI API format works. Tested/documented integrations:
| Frontend | Stars | Connection |
|----------|-------|------------|
| [Open WebUI](/user-guide/messaging/open-webui) | 126k | Full guide available |
| LobeChat | 73k | Custom provider endpoint |
| LibreChat | 34k | Custom endpoint in librechat.yaml |
| AnythingLLM | 56k | Generic OpenAI provider |
| NextChat | 87k | BASE_URL env var |
| ChatBox | 39k | API Host setting |
| Jan | 26k | Remote model config |
| HF Chat-UI | 8k | OPENAI_BASE_URL |
| big-AGI | 7k | Custom endpoint |
| OpenAI Python SDK | — | `OpenAI(base_url="http://localhost:8642/v1")` |
| curl | — | Direct HTTP requests |
## Multi-User Setup with Profiles
To give multiple users their own isolated Hermes instance (separate config, memory, skills), use [profiles](/user-guide/profiles):
```bash
# Create a profile per user
hermes profile create alice
hermes profile create bob
# Configure each profile's API server on a different port. API_SERVER_* are env
# vars (not config.yaml keys), so write them to each profile's .env:
cat >> ~/.hermes/profiles/alice/.env <<EOF
API_SERVER_ENABLED=true
API_SERVER_PORT=8643
API_SERVER_KEY=alice-secret
EOF
cat >> ~/.hermes/profiles/bob/.env <<EOF
API_SERVER_ENABLED=true
API_SERVER_PORT=8644
API_SERVER_KEY=bob-secret
EOF
# Start each profile's gateway
hermes -p alice gateway &
hermes -p bob gateway &
```
Each profile's API server automatically advertises the profile name as the model ID:
- `http://localhost:8643/v1/models` → model `alice`
- `http://localhost:8644/v1/models` → model `bob`
In Open WebUI, add each as a separate connection. The model dropdown shows `alice` and `bob` as distinct models, each backed by a fully isolated Hermes instance. See the [Open WebUI guide](/user-guide/messaging/open-webui#multi-user-setup-with-profiles) for details.
## Limitations
- **Response storage** — stored responses (for `previous_response_id`) are persisted in SQLite and survive gateway restarts. Max 100 stored responses (LRU eviction).
- **No file upload** — inline images are supported on both `/v1/chat/completions` and `/v1/responses`, but uploaded files (`file`, `input_file`, `file_id`) and non-image document inputs are not supported through the API.
- **Model field is cosmetic** — the `model` field in requests is accepted but the actual LLM model used is configured server-side in config.yaml.
## Proxy Mode
The API server also serves as the backend for **gateway proxy mode**. When another Hermes gateway instance is configured with `GATEWAY_PROXY_URL` pointing at this API server, it forwards all messages here instead of running its own agent. This enables split deployments — for example, a Docker container handling Matrix E2EE that relays to a host-side agent.
See [Matrix Proxy Mode](/user-guide/messaging/matrix#proxy-mode-e2ee-on-macos) for the full setup guide.
website/docs/user-guide/features/batch-processing.md
+230
View File
@@ -0,0 +1,230 @@
---
sidebar_position: 12
title: "Batch Processing"
description: "Generate agent trajectories at scale — parallel processing, checkpointing, and toolset distributions"
---
# Batch Processing
Batch processing lets you run the Hermes agent across hundreds or thousands of prompts in parallel, generating structured trajectory data. This is primarily used for **training data generation** — producing ShareGPT-format trajectories with tool usage statistics that can be used for fine-tuning or evaluation.
## Overview
The batch runner (`batch_runner.py`) processes a JSONL dataset of prompts, running each through a full agent session with tool access. Each prompt gets its own isolated environment. The output is structured trajectory data with full conversation history, tool call statistics, and reasoning coverage metrics.
## Quick Start
```bash
# Basic batch run
python batch_runner.py \
--dataset_file=data/prompts.jsonl \
--batch_size=10 \
--run_name=my_first_run \
--model=anthropic/claude-sonnet-4.6 \
--num_workers=4
# Resume an interrupted run
python batch_runner.py \
--dataset_file=data/prompts.jsonl \
--batch_size=10 \
--run_name=my_first_run \
--resume
# List available toolset distributions
python batch_runner.py --list_distributions
```
:::tip Predictable cost at scale
Batch runs spin up many concurrent agent sessions, each making model calls and tool calls. A [Nous Portal](/user-guide/features/tool-gateway) subscription bundles model access plus web search, image gen, TTS, and cloud browsers under one bill — useful when you want stable cost-per-trajectory without juggling rate limits across five vendor accounts. Set up with `hermes setup --portal`, then point `--model` at a Nous model.
:::
## Dataset Format
The input dataset is a JSONL file (one JSON object per line). Each entry must have a `prompt` field:
```jsonl
{"prompt": "Write a Python function that finds the longest palindromic substring"}
{"prompt": "Create a REST API endpoint for user authentication using Flask"}
{"prompt": "Debug this error: TypeError: cannot unpack non-iterable NoneType object"}
```
Entries can optionally include:
- `image` or `docker_image`: A container image to use for this prompt's sandbox (works with Docker, Modal, and Singularity backends)
- `cwd`: Working directory override for the task's terminal session
## Configuration Options
| Parameter | Default | Description |
|-----------|---------|-------------|
| `--dataset_file` | (required) | Path to JSONL dataset |
| `--batch_size` | (required) | Prompts per batch |
| `--run_name` | (required) | Name for this run (used for output dir and checkpointing) |
| `--distribution` | `"default"` | Toolset distribution to sample from |
| `--model` | `claude-sonnet-4.6` | Model to use |
| `--base_url` | `https://openrouter.ai/api/v1` | API base URL |
| `--api_key` | (env var) | API key for model |
| `--max_turns` | `10` | Maximum tool-calling iterations per prompt |
| `--num_workers` | `4` | Parallel worker processes |
| `--resume` | `false` | Resume from checkpoint |
| `--verbose` | `false` | Enable verbose logging |
| `--max_samples` | all | Only process first N samples from dataset |
| `--max_tokens` | model default | Maximum tokens per model response |
### Provider Routing (OpenRouter)
| Parameter | Description |
|-----------|-------------|
| `--providers_allowed` | Comma-separated providers to allow (e.g., `"anthropic,openai"`) |
| `--providers_ignored` | Comma-separated providers to ignore (e.g., `"together,deepinfra"`) |
| `--providers_order` | Comma-separated preferred provider order |
| `--provider_sort` | Sort by `"price"`, `"throughput"`, or `"latency"` |
### Reasoning Control
| Parameter | Description |
|-----------|-------------|
| `--reasoning_effort` | Effort level: `none`, `minimal`, `low`, `medium`, `high`, `xhigh` |
| `--reasoning_disabled` | Completely disable reasoning/thinking tokens |
### Advanced Options
| Parameter | Description |
|-----------|-------------|
| `--ephemeral_system_prompt` | System prompt used during execution but NOT saved to trajectories |
| `--log_prefix_chars` | Characters to show in log previews (default: 100) |
| `--prefill_messages_file` | Path to JSON file with prefill messages for few-shot priming |
## Toolset Distributions
Each prompt gets a randomly sampled set of toolsets from a **distribution**. This ensures training data covers diverse tool combinations. Use `--list_distributions` to see all available distributions.
In the current implementation, distributions assign a probability to **each individual toolset**. The sampler flips each toolset independently, then guarantees that at least one toolset is enabled. This is different from a hand-authored table of prebuilt combinations.
## Output Format
All output goes to `data/<run_name>/`:
```text
data/my_run/
├── trajectories.jsonl # Combined final output (all batches merged)
├── batch_0.jsonl # Individual batch results
├── batch_1.jsonl
├── ...
├── checkpoint.json # Resume checkpoint
└── statistics.json # Aggregate tool usage stats
```
### Trajectory Format
Each line in `trajectories.jsonl` is a JSON object:
```json
{
"prompt_index": 42,
"conversations": [
{"from": "human", "value": "Write a function..."},
{"from": "gpt", "value": "I'll create that function...",
"tool_calls": [...]},
{"from": "tool", "value": "..."},
{"from": "gpt", "value": "Here's the completed function..."}
],
"metadata": {
"batch_num": 2,
"timestamp": "2026-01-15T10:30:00",
"model": "anthropic/claude-sonnet-4.6"
},
"completed": true,
"partial": false,
"api_calls": 3,
"toolsets_used": ["terminal", "file"],
"tool_stats": {
"terminal": {"count": 2, "success": 2, "failure": 0},
"read_file": {"count": 1, "success": 1, "failure": 0}
},
"tool_error_counts": {
"terminal": 0,
"read_file": 0
}
}
```
The `conversations` field uses a ShareGPT-like format with `from` and `value` fields. Tool stats are normalized to include all possible tools with zero defaults, ensuring consistent schema across entries for HuggingFace datasets compatibility.
## Checkpointing
The batch runner has robust checkpointing for fault tolerance:
- **Checkpoint file:** Saved after each batch completes, tracking which prompt indices are done
- **Content-based resume:** On `--resume`, the runner scans existing batch files and matches completed prompts by their actual text content (not just indices), enabling recovery even if the dataset order changes
- **Failed prompts:** Only successfully completed prompts are marked as done — failed prompts will be retried on resume
- **Batch merging:** On completion, all batch files (including from previous runs) are merged into a single `trajectories.jsonl`
### How Resume Works
1. Scan all `batch_*.jsonl` files for completed prompts (by content matching)
2. Filter the dataset to exclude already-completed prompts
3. Re-batch the remaining prompts
4. Process only the remaining prompts
5. Merge all batch files (old + new) into final output
## Quality Filtering
The batch runner applies automatic quality filtering:
- **No-reasoning filter:** Samples where zero assistant turns contain reasoning (no `<REASONING_SCRATCHPAD>` or native thinking tokens) are discarded
- **Corrupted entry filter:** Entries with hallucinated tool names (not in the valid tool list) are filtered out during the final merge
- **Reasoning statistics:** Tracks percentage of turns with/without reasoning across the entire run
## Statistics
After completion, the runner prints comprehensive statistics:
- **Tool usage:** Call counts, success/failure rates per tool
- **Reasoning coverage:** Percentage of assistant turns with reasoning
- **Samples discarded:** Count of samples filtered for lacking reasoning
- **Duration:** Total processing time
Statistics are also saved to `statistics.json` for programmatic analysis.
## Use Cases
### Training Data Generation
Generate diverse tool-use trajectories for fine-tuning:
```bash
python batch_runner.py \
--dataset_file=data/coding_prompts.jsonl \
--batch_size=20 \
--run_name=coding_v1 \
--model=anthropic/claude-sonnet-4.6 \
--num_workers=8 \
--distribution=default \
--max_turns=15
```
### Model Evaluation
Evaluate how well a model uses tools across standardized prompts:
```bash
python batch_runner.py \
--dataset_file=data/eval_suite.jsonl \
--batch_size=10 \
--run_name=eval_gpt4 \
--model=openai/gpt-4o \
--num_workers=4 \
--max_turns=10
```
### Per-Prompt Container Images
For benchmarks requiring specific environments, each prompt can specify its own container image:
```jsonl
{"prompt": "Install numpy and compute eigenvalues of a 3x3 matrix", "image": "python:3.11-slim"}
{"prompt": "Compile this Rust program and run it", "image": "rust:1.75"}
{"prompt": "Set up a Node.js Express server", "image": "node:20-alpine", "cwd": "/app"}
```
The batch runner verifies Docker images are accessible before running each prompt.
website/docs/user-guide/features/browser.md
+661
View File
@@ -0,0 +1,661 @@
---
title: Browser Automation
description: Control browsers with multiple providers, local Chromium-family browsers via CDP, or cloud browsers for web interaction, form filling, scraping, and more.
sidebar_label: Browser
sidebar_position: 5
---
# Browser Automation
Hermes Agent includes a full browser automation toolset with multiple backend options:
- **Browserbase cloud mode** via [Browserbase](https://browserbase.com) for managed cloud browsers and anti-bot tooling
- **Browser Use cloud mode** via [Browser Use](https://browser-use.com) as an alternative cloud browser provider
- **Firecrawl cloud mode** via [Firecrawl](https://firecrawl.dev) for cloud browsers with built-in scraping
- **Camofox local mode** via [Camofox](https://github.com/jo-inc/camofox-browser) for local anti-detection browsing (Firefox-based fingerprint spoofing)
- **Local Chromium-family CDP** — connect browser tools to your own Chrome, Brave, Chromium, or Edge instance using `/browser connect`
- **Local browser mode** via the `agent-browser` CLI and a local Chromium installation
In all modes, the agent can navigate websites, interact with page elements, fill forms, and extract information.
## Overview
Pages are represented as **accessibility trees** (text-based snapshots), making them ideal for LLM agents. Interactive elements get ref IDs (like `@e1`, `@e2`) that the agent uses for clicking and typing.
Key capabilities:
- **Multi-provider cloud execution** — Browserbase, Browser Use, or Firecrawl — no local browser needed
- **Local Chromium-family integration** — attach to your running Chrome, Brave, Chromium, or Edge browser via CDP for hands-on browsing
- **Built-in stealth** — random fingerprints, CAPTCHA solving, residential proxies (Browserbase)
- **Session isolation** — each task gets its own browser session
- **Automatic cleanup** — inactive sessions are closed after a timeout
- **Vision analysis** — screenshot + AI analysis for visual understanding
## Setup
:::tip Nous Subscribers
If you have a paid [Nous Portal](https://portal.nousresearch.com) subscription, you can use browser automation through the **[Tool Gateway](tool-gateway.md)** without any separate API keys. New installs can run `hermes setup --portal` to log in and turn on every gateway tool at once; existing installs can pick **Nous Subscription** as the browser provider via `hermes model` or `hermes tools`.
:::
### Browserbase cloud mode
To use Browserbase-managed cloud browsers, add:
```bash
# Add to ~/.hermes/.env
BROWSERBASE_API_KEY=***
BROWSERBASE_PROJECT_ID=your-project-id-here
```
Get your credentials at [browserbase.com](https://browserbase.com).
### Browser Use cloud mode
To use Browser Use as your cloud browser provider, add:
```bash
# Add to ~/.hermes/.env
BROWSER_USE_API_KEY=***
```
Get your API key at [browser-use.com](https://browser-use.com). Browser Use provides a cloud browser via its REST API. If both Browserbase and Browser Use credentials are set, Browserbase takes priority.
### Firecrawl cloud mode
To use Firecrawl as your cloud browser provider, add:
```bash
# Add to ~/.hermes/.env
FIRECRAWL_API_KEY=fc-***
```
Get your API key at [firecrawl.dev](https://firecrawl.dev). Then select Firecrawl as your browser provider:
```bash
hermes setup tools
# → Browser Automation → Firecrawl
```
Optional settings:
```bash
# Self-hosted Firecrawl instance (default: https://api.firecrawl.dev)
FIRECRAWL_API_URL=http://localhost:3002
# Session TTL in seconds (default: 300)
FIRECRAWL_BROWSER_TTL=600
```
### Hybrid routing: cloud for public URLs, local for LAN/localhost
When a cloud provider is configured, Hermes auto-spawns a **local Chromium sidecar**
for URLs that resolve to a private/loopback/LAN address (`localhost`, `127.0.0.1`,
`192.168.x.x`, `10.x.x.x`, `172.16-31.x.x`, `*.local`, `*.lan`, `*.internal`,
IPv6 loopback `::1`, link-local `169.254.x.x`). Public URLs continue to use the
cloud provider in the same conversation.
This solves the common "I'm developing locally but using Browserbase" workflow —
the agent can screenshot your dashboard at `http://localhost:3000` AND scrape
`https://github.com` without you switching providers or disabling the SSRF guard.
The cloud provider never sees the private URL.
The feature is **on by default**. To disable it (all URLs go to the configured
cloud provider, as before):
```yaml
# ~/.hermes/config.yaml
browser:
cloud_provider: browserbase
auto_local_for_private_urls: false
```
With auto-routing disabled, private URLs are rejected with
`"Blocked: URL targets a private or internal address"` unless you also set
`browser.allow_private_urls: true` (which lets the cloud provider attempt them —
usually won't work since Browserbase etc. can't reach your LAN).
Requirements: the local sidecar uses the same `agent-browser` CLI as pure local
mode, so you need it installed (`hermes setup tools → Browser Automation`
auto-installs it). Post-navigation redirects from a public URL onto a private
address are still blocked (you can't use a redirect-to-internal trick to reach
your LAN through the public path).
### Camofox local mode
[Camofox](https://github.com/jo-inc/camofox-browser) is a self-hosted Node.js server wrapping Camoufox (a Firefox fork with C++ fingerprint spoofing). It provides local anti-detection browsing without cloud dependencies.
```bash
# Clone the Camofox browser server first
git clone https://github.com/jo-inc/camofox-browser
cd camofox-browser
# Build and start with Docker using the default container settings
# (auto-detects arch: aarch64 on M1/M2, x86_64 on Intel)
make up
# Stop and remove the default container
make down
# Force a clean rebuild (for example, after upgrading VERSION/RELEASE)
make reset
# Just download binaries without building
make fetch
# Override arch or version explicitly
make up ARCH=x86_64
make up VERSION=135.0.1 RELEASE=beta.24
```
`make up` starts the default container immediately. If you want custom runtime settings such as a larger Node heap, VNC, or a persistent profile directory, build the image first and then run it yourself:
```bash
# Build the image without starting the default container
make build
# Start with persistence, VNC live view, and a larger Node heap
mkdir -p ~/.camofox-docker
docker run -d \
--name camofox-browser \
--restart unless-stopped \
-p 9377:9377 \
-p 6080:6080 \
-p 5901:5900 \
-e CAMOFOX_PORT=9377 \
-e ENABLE_VNC=1 \
-e VNC_BIND=0.0.0.0 \
-e VNC_RESOLUTION=1920x1080 \
-e MAX_OLD_SPACE_SIZE=2048 \
-v ~/.camofox-docker:/root/.camofox \
camofox-browser:135.0.1-aarch64
```
With VNC enabled, the browser runs in headed mode and can be watched live in your browser at `http://localhost:6080` (noVNC). You can also connect a native VNC client to `localhost:5901`.
If you already ran `make up`, stop and remove that default container before starting the custom one:
```bash
make down
# then run the custom docker run command above
```
Then set in `~/.hermes/.env`:
```bash
CAMOFOX_URL=http://localhost:9377
```
If Camofox is running in Docker and you want it to open web apps served from the host machine, enable loopback rewriting. `CAMOFOX_URL` should still point at the host-published control API, but page URLs such as `http://127.0.0.1:3000` must be opened from inside the container as `http://host.docker.internal:3000`:
```yaml
# ~/.hermes/config.yaml
browser:
camofox:
rewrite_loopback_urls: true
loopback_host_alias: host.docker.internal # default; use a LAN IP if needed
```
Equivalent env vars:
```bash
CAMOFOX_REWRITE_LOOPBACK_URLS=true
CAMOFOX_LOOPBACK_HOST_ALIAS=host.docker.internal
```
The rewrite only applies to page navigation URLs with loopback hosts (`localhost`, `127.0.0.1`, `::1`). It does not change `CAMOFOX_URL`. Leave it disabled for non-Docker Camofox installs, where the browser already runs on the host and loopback URLs are correct.
Or configure via `hermes tools` → Browser Automation → Camofox.
When `CAMOFOX_URL` is set, all browser tools automatically route through Camofox instead of Browserbase or agent-browser.
#### Persistent browser sessions
By default, each Camofox session gets a random identity — cookies and logins don't survive across agent restarts. To enable persistent browser sessions, add the following to `~/.hermes/config.yaml`:
```yaml
browser:
camofox:
managed_persistence: true
```
Then fully restart Hermes so the new config is picked up.
:::warning Nested path matters
Hermes reads `browser.camofox.managed_persistence`, **not** a top-level `managed_persistence`. A common mistake is writing:
```yaml
# ❌ Wrong — Hermes ignores this
managed_persistence: true
```
If the flag is placed at the wrong path, Hermes silently falls back to a random ephemeral `userId` and your login state will be lost on every session.
:::
##### What Hermes does
- Sends a deterministic profile-scoped `userId` to Camofox so the server can reuse the same Firefox profile across sessions.
- Skips server-side context destruction on cleanup, so cookies and logins survive between agent tasks.
- Scopes the `userId` to the active Hermes profile, so different Hermes profiles get different browser profiles (profile isolation).
##### What Hermes does not do
- It does not force persistence on the Camofox server. Hermes only sends a stable `userId`; the server must honor it by mapping that `userId` to a persistent Firefox profile directory.
- If your Camofox server build treats every request as ephemeral (e.g. always calls `browser.newContext()` without loading a stored profile), Hermes cannot make those sessions persist. Make sure you are running a Camofox build that implements userId-based profile persistence.
##### Verify it's working
1. Start Hermes and your Camofox server.
2. Open Google (or any login site) in a browser task and sign in manually.
3. End the browser task normally.
4. Start a new browser task.
5. Open the same site again — you should still be signed in.
If step 5 logs you out, the Camofox server isn't honoring the stable `userId`. Double-check your config path, confirm you fully restarted Hermes after editing `config.yaml`, and verify your Camofox server version supports persistent per-user profiles.
##### Where state lives
Hermes derives the stable `userId` from the profile-scoped directory `~/.hermes/browser_auth/camofox/` (or the equivalent under `$HERMES_HOME` for non-default profiles). The actual browser profile data lives on the Camofox server side, keyed by that `userId`. To fully reset a persistent profile, clear it on the Camofox server and remove the corresponding Hermes profile's state directory.
#### Externally managed Camofox sessions
When another app drives the visible Camofox browser (a desktop assistant, a custom integration, another agent), configure Hermes to operate inside that same identity instead of spawning its own isolated profile.
Three knobs control the behavior:
| Setting | Env var | Effect |
|---------|---------|--------|
| `browser.camofox.user_id` | `CAMOFOX_USER_ID` | Camofox `userId` Hermes uses when creating tabs. Setting this opts the session into "externally managed" mode. |
| `browser.camofox.session_key` | `CAMOFOX_SESSION_KEY` | `sessionKey` (a.k.a. `listItemId`) sent on tab creation. Used to match an existing tab during adoption. Defaults to a per-task value if unset. |
| `browser.camofox.adopt_existing_tab` | `CAMOFOX_ADOPT_EXISTING_TAB` | When true, Hermes calls `GET /tabs?userId=<user_id>` on first use and reuses an existing tab before creating a new one. |
Env vars take precedence over `config.yaml`. Either form works:
```yaml
browser:
camofox:
user_id: shared-camofox
session_key: visible-tab
adopt_existing_tab: true
```
```bash
CAMOFOX_USER_ID=shared-camofox
CAMOFOX_SESSION_KEY=visible-tab
CAMOFOX_ADOPT_EXISTING_TAB=true
```
**What changes when `user_id` is set:**
- Hermes skips destructive cleanup at task end (same as `managed_persistence: true`). The other app's tab/cookies/profile survive.
- Hermes does **not** call `DELETE /sessions/<user_id>` — that endpoint wipes all user data, so it would nuke the external app's session if it fired.
**How tab adoption works (when `adopt_existing_tab: true`):**
1. On the first browser tool call after a process start, Hermes issues `GET /tabs?userId=<user_id>` (5-second timeout).
2. If any tab in the response has `listItemId == session_key`, Hermes adopts the most recently created one in that group.
3. Otherwise, Hermes adopts the most recently created tab for the user (any `listItemId`).
4. If no tabs exist or the request fails, Hermes falls back to creating a new tab on the next operation.
Adoption only fires until `tab_id` is populated for the session. If the external app closes the adopted tab mid-run, the next browser tool call will surface a Camofox error — Hermes does not re-poll for a fresh tab on every call.
**Picking `session_key`:** if you want Hermes to reliably attach to a *specific* existing tab, set `session_key` to the `listItemId` the external app used when creating it. If you leave `session_key` unset and only set `user_id`, Hermes generates a per-task `session_key` (`task_<id>`) — Hermes will share cookies and the profile with the external app, but will open its own tab alongside instead of reusing one.
**Concurrency note:** the external app and Hermes can drive the same Camofox `userId` simultaneously, but Camofox does not coordinate per-tab focus between clients. Coordinate ownership at the application layer (e.g. the external app pauses while Hermes runs).
#### VNC live view
When Camofox runs in headed mode (with a visible browser window), it exposes a VNC port in its health check response. Hermes automatically discovers this and includes the VNC URL in navigation responses, so the agent can share a link for you to watch the browser live.
### Local Chromium-family browser via CDP (`/browser connect`)
Instead of a cloud provider, you can attach Hermes browser tools to your own running Chrome, Brave, Chromium, or Edge instance via the Chrome DevTools Protocol (CDP). This is useful when you want to see what the agent is doing in real-time, interact with pages that require your own cookies/sessions, or avoid cloud browser costs.
:::note
`/browser connect` is an **interactive-CLI slash command** — it is not dispatched by the gateway. If you try to run it inside a WebUI, Telegram, Discord, or other gateway chat, the message will be sent to the agent as plain text and the command will not execute. Start Hermes from the terminal (`hermes` or `hermes chat`) and issue `/browser connect` there.
:::
In the CLI, use:
```
/browser connect # Auto-launch/connect to a local Chromium-family browser at http://127.0.0.1:9222
/browser connect ws://host:port # Connect to a specific CDP endpoint
/browser status # Check current connection
/browser disconnect # Detach and return to cloud/local mode
```
If a browser isn't already running with remote debugging, Hermes will attempt to auto-launch a supported Chromium-family browser with `--remote-debugging-port=9222`. Detection includes Brave, Google Chrome, Chromium, and Microsoft Edge, with common Linux install paths such as `/opt/brave-bin/brave` and `/snap/bin/brave`.
:::tip
To start a Chromium-family browser manually with CDP enabled, use a dedicated user-data-dir so the debug port actually comes up even if the browser is already running with your normal profile:
```bash
# Linux — Brave
brave-browser \
--remote-debugging-port=9222 \
--user-data-dir=$HOME/.hermes/chrome-debug \
--no-first-run \
--no-default-browser-check &
# Linux — Google Chrome
google-chrome \
--remote-debugging-port=9222 \
--user-data-dir=$HOME/.hermes/chrome-debug \
--no-first-run \
--no-default-browser-check &
# macOS — Brave
"/Applications/Brave Browser.app/Contents/MacOS/Brave Browser" \
--remote-debugging-port=9222 \
--user-data-dir="$HOME/.hermes/chrome-debug" \
--no-first-run \
--no-default-browser-check &
# macOS — Google Chrome
"/Applications/Google Chrome.app/Contents/MacOS/Google Chrome" \
--remote-debugging-port=9222 \
--user-data-dir="$HOME/.hermes/chrome-debug" \
--no-first-run \
--no-default-browser-check &
```
Then launch the Hermes CLI and run `/browser connect`.
**Why `--user-data-dir`?** Without it, launching a Chromium-family browser while a regular instance is already running typically opens a new window on the existing process — and that existing process was not started with `--remote-debugging-port`, so port 9222 never opens. A dedicated user-data-dir forces a fresh browser process where the debug port actually listens. `--no-first-run --no-default-browser-check` skips the first-launch wizard for the fresh profile.
:::
When connected via CDP, all browser tools (`browser_navigate`, `browser_click`, etc.) operate on your live browser instance instead of spinning up a cloud session.
### WSL2 + Windows Chrome: prefer MCP over `/browser connect`
If Hermes runs inside WSL2 but the Chrome window you want to control runs on the Windows host, `/browser connect` is often not the best path.
Why:
- `/browser connect` expects Hermes itself to reach a usable CDP endpoint
- modern Chrome live-debugging sessions often expose a host-local endpoint that is not directly reachable from WSL the same way a classic `9222` port is
- even when Windows Chrome is debuggable, the cleanest integration is often to let a Windows-side browser MCP server attach to Chrome and let Hermes talk to that MCP server
For that setup, prefer `chrome-devtools-mcp` through Hermes MCP support.
See the MCP guide for the practical setup:
- [Use MCP with Hermes](../../guides/use-mcp-with-hermes.md#wsl2-bridge-hermes-in-wsl-to-windows-chrome)
### Local browser mode
If you do **not** set any cloud credentials and don't use `/browser connect`, Hermes can still use the browser tools through a local Chromium install driven by `agent-browser`.
### Optional Environment Variables
```bash
# Residential proxies for better CAPTCHA solving (default: "true")
BROWSERBASE_PROXIES=true
# Advanced stealth with custom Chromium — requires Scale Plan (default: "false")
BROWSERBASE_ADVANCED_STEALTH=false
# Session reconnection after disconnects — requires paid plan (default: "true")
BROWSERBASE_KEEP_ALIVE=true
# Custom session timeout in seconds (max 21600 = 6 hours) (default: project default)
# Examples: 600 (10min), 1800 (30min), 21600 (6h max)
BROWSERBASE_SESSION_TIMEOUT=1800
# Inactivity timeout before auto-cleanup in seconds (default: 120)
BROWSER_INACTIVITY_TIMEOUT=120
# Extra Chromium launch flags (comma- or newline-separated). Hermes auto-injects
# `--no-sandbox,--disable-dev-shm-usage` when it detects root or AppArmor-restricted
# unprivileged user namespaces (Ubuntu 23.10+, DGX Spark, many container images),
# so most users don't need to set this. Set it manually only if you need a flag
# Hermes doesn't add automatically; setting it disables the auto-injection.
AGENT_BROWSER_ARGS=--no-sandbox
```
### Install agent-browser CLI
```bash
npm install -g agent-browser
# Or install locally in the repo:
npm install
```
:::info
The `browser` toolset must be included in your config's `toolsets` list or enabled via `hermes config set toolsets '["hermes-cli", "browser"]'`.
:::
## Available Tools
### `browser_navigate`
Navigate to a URL. Must be called before any other browser tool. Initializes the Browserbase session.
```
Navigate to https://github.com/NousResearch
```
:::tip
For simple information retrieval, prefer `web_search` or `web_extract` — they are faster and cheaper. Use browser tools when you need to **interact** with a page (click buttons, fill forms, handle dynamic content).
:::
### `browser_snapshot`
Get a text-based snapshot of the current page's accessibility tree. Returns interactive elements with ref IDs like `@e1`, `@e2` for use with `browser_click` and `browser_type`.
- **`full=false`** (default): Compact view showing only interactive elements
- **`full=true`**: Complete page content
Snapshots over 8000 characters are automatically summarized by an LLM.
### `browser_click`
Click an element identified by its ref ID from the snapshot.
```
Click @e5 to press the "Sign In" button
```
### `browser_type`
Type text into an input field. Clears the field first, then types the new text.
```
Type "hermes agent" into the search field @e3
```
### `browser_scroll`
Scroll the page up or down to reveal more content.
```
Scroll down to see more results
```
### `browser_press`
Press a keyboard key. Useful for submitting forms or navigation.
```
Press Enter to submit the form
```
Supported keys: `Enter`, `Tab`, `Escape`, `ArrowDown`, `ArrowUp`, and more.
### `browser_back`
Navigate back to the previous page in browser history.
### `browser_get_images`
List all images on the current page with their URLs and alt text. Useful for finding images to analyze.
### `browser_vision`
Take a screenshot and analyze it with vision AI. Use this when text snapshots don't capture important visual information — especially useful for CAPTCHAs, complex layouts, or visual verification challenges.
The screenshot is saved persistently and the file path is returned alongside the AI analysis. On messaging platforms (Telegram, Discord, Slack, WhatsApp), you can ask the agent to share the screenshot — it will be sent as a native photo attachment via the `MEDIA:` mechanism.
```
What does the chart on this page show?
```
Screenshots are stored in `~/.hermes/cache/screenshots/` and automatically cleaned up after 24 hours.
### `browser_console`
Get browser console output (log/warn/error messages) and uncaught JavaScript exceptions from the current page. Essential for detecting silent JS errors that don't appear in the accessibility tree.
```
Check the browser console for any JavaScript errors
```
Use `clear=True` to clear the console after reading, so subsequent calls only show new messages.
`browser_console` also evaluates JavaScript when called with an `expression` argument — same shape as DevTools console, the result comes back parsed (JSON-serialized objects become dicts; primitive values stay primitive).
```
browser_console(expression="document.querySelector('h1').textContent")
browser_console(expression="JSON.stringify(performance.timing)")
```
When a CDP supervisor is active for the current session (typical for any session that's run `browser_navigate` against a CDP-capable backend), evaluation runs over the supervisor's persistent WebSocket — no subprocess startup cost. Falls through to the standard agent-browser CLI path otherwise. Behaviour is identical either way; only latency changes.
### `browser_cdp`
Raw Chrome DevTools Protocol passthrough — the escape hatch for browser operations not covered by the other tools. Use for native dialog handling, iframe-scoped evaluation, cookie/network control, or any CDP verb the agent needs.
**Only available when a CDP endpoint is reachable at session start** — meaning `/browser connect` has attached to a running Chrome, Brave, Chromium, or Edge browser, or `browser.cdp_url` is set in `config.yaml`. The default local agent-browser mode, Camofox, and cloud providers (Browserbase, Browser Use, Firecrawl) do not currently expose CDP to this tool — cloud providers have per-session CDP URLs but live-session routing is a follow-up.
**CDP method reference:** https://chromedevtools.github.io/devtools-protocol/ — the agent can `web_extract` a specific method's page to look up parameters and return shape.
Common patterns:
```
# List tabs (browser-level, no target_id)
browser_cdp(method="Target.getTargets")
# Handle a native JS dialog on a tab
browser_cdp(method="Page.handleJavaScriptDialog",
params={"accept": true, "promptText": ""},
target_id="<tabId>")
# Evaluate JS in a specific tab
browser_cdp(method="Runtime.evaluate",
params={"expression": "document.title", "returnByValue": true},
target_id="<tabId>")
# Get all cookies
browser_cdp(method="Network.getAllCookies")
```
Browser-level methods (`Target.*`, `Browser.*`, `Storage.*`) omit `target_id`. Page-level methods (`Page.*`, `Runtime.*`, `DOM.*`, `Emulation.*`) require a `target_id` from `Target.getTargets`. Each stateless call is independent — sessions do not persist between calls.
**Cross-origin iframes:** pass `frame_id` (from `browser_snapshot.frame_tree.children[]` where `is_oopif=true`) to route the CDP call through the supervisor's live session for that iframe. This is how `Runtime.evaluate` inside a cross-origin iframe works on Browserbase, where stateless CDP connections would hit signed-URL expiry. Example:
```
browser_cdp(
method="Runtime.evaluate",
params={"expression": "document.title", "returnByValue": True},
frame_id="<frame_id from browser_snapshot>",
)
```
Same-origin iframes don't need `frame_id` — use `document.querySelector('iframe').contentDocument` from a top-level `Runtime.evaluate` instead.
### `browser_dialog`
Responds to a native JS dialog (`alert` / `confirm` / `prompt` / `beforeunload`). Before this tool existed, dialogs would silently block the page's JavaScript thread and subsequent `browser_*` calls would hang or throw; now the agent sees pending dialogs in `browser_snapshot` output and responds explicitly.
**Workflow:**
1. Call `browser_snapshot`. If a dialog is blocking the page, it shows up as `pending_dialogs: [{"id": "d-1", "type": "alert", "message": "..."}]`.
2. Call `browser_dialog(action="accept")` or `browser_dialog(action="dismiss")`. For `prompt()` dialogs, pass `prompt_text="..."` to supply the response.
3. Re-snapshot — `pending_dialogs` is empty; the page's JS thread has resumed.
**Detection happens automatically** via a persistent CDP supervisor — one WebSocket per task that subscribes to Page/Runtime/Target events. The supervisor also populates a `frame_tree` field in the snapshot so the agent can see the iframe structure of the current page, including cross-origin (OOPIF) iframes.
**Availability matrix:**
| Backend | Detection via `pending_dialogs` | Response (`browser_dialog` tool) |
|---|---|---|
| Local Chrome via `/browser connect` or `browser.cdp_url` | ✓ | ✓ full workflow |
| Browserbase | ✓ | ✓ full workflow (via injected XHR bridge) |
| Camofox / default local agent-browser | ✗ | ✗ (no CDP endpoint) |
**How it works on Browserbase.** Browserbase's CDP proxy auto-dismisses real native dialogs server-side within ~10ms, so we can't use `Page.handleJavaScriptDialog`. The supervisor injects a small script via `Page.addScriptToEvaluateOnNewDocument` that overrides `window.alert`/`confirm`/`prompt` with a synchronous XHR. We intercept those XHRs via `Fetch.enable` — the page's JS thread stays blocked on the XHR until we call `Fetch.fulfillRequest` with the agent's response. `prompt()` return values round-trip back into page JS unchanged.
**Dialog policy** is configured in `config.yaml` under `browser.dialog_policy`:
| Policy | Behavior |
|--------|----------|
| `must_respond` (default) | Capture, surface in snapshot, wait for explicit `browser_dialog()` call. Safety auto-dismiss after `browser.dialog_timeout_s` (default 300s) so a buggy agent can't stall forever. |
| `auto_dismiss` | Capture, dismiss immediately. Agent still sees the dialog in `browser_state` history but doesn't have to act. |
| `auto_accept` | Capture, accept immediately. Useful when navigating pages with aggressive `beforeunload` prompts. |
**Frame tree** inside `browser_snapshot.frame_tree` is capped to 30 frames and OOPIF depth 2 to keep payloads bounded on ad-heavy pages. A `truncated: true` flag surfaces when limits were hit; agents needing the full tree can use `browser_cdp` with `Page.getFrameTree`.
## Practical Examples
### Filling Out a Web Form
```
User: Sign up for an account on example.com with my email john@example.com
Agent workflow:
1. browser_navigate("https://example.com/signup")
2. browser_snapshot() → sees form fields with refs
3. browser_type(ref="@e3", text="john@example.com")
4. browser_type(ref="@e5", text="SecurePass123")
5. browser_click(ref="@e8") → clicks "Create Account"
6. browser_snapshot() → confirms success
```
### Researching Dynamic Content
```
User: What are the top trending repos on GitHub right now?
Agent workflow:
1. browser_navigate("https://github.com/trending")
2. browser_snapshot(full=true) → reads trending repo list
3. Returns formatted results
```
## Session Recording
Automatically record browser sessions as WebM video files:
```yaml
browser:
record_sessions: true # default: false
```
When enabled, recording starts automatically on the first `browser_navigate` and saves to `~/.hermes/browser_recordings/` when the session closes. Works in both local and cloud (Browserbase) modes. Recordings older than 72 hours are automatically cleaned up.
## Stealth Features
Browserbase provides automatic stealth capabilities:
| Feature | Default | Notes |
|---------|---------|-------|
| Basic Stealth | Always on | Random fingerprints, viewport randomization, CAPTCHA solving |
| Residential Proxies | On | Routes through residential IPs for better access |
| Advanced Stealth | Off | Custom Chromium build, requires Scale Plan |
| Keep Alive | On | Session reconnection after network hiccups |
:::note
If paid features aren't available on your plan, Hermes automatically falls back — first disabling `keepAlive`, then proxies — so browsing still works on free plans.
:::
## Session Management
- Each task gets an isolated browser session via Browserbase
- Sessions are automatically cleaned up after inactivity (default: 2 minutes)
- A background thread checks every 30 seconds for stale sessions
- Emergency cleanup runs on process exit to prevent orphaned sessions
- Sessions are released via the Browserbase API (`REQUEST_RELEASE` status)
## Limitations
- **Text-based interaction** — relies on accessibility tree, not pixel coordinates
- **Snapshot size** — large pages may be truncated or LLM-summarized at 8000 characters
- **Session timeout** — cloud sessions expire based on your provider's plan settings
- **Cost** — cloud sessions consume provider credits; sessions are automatically cleaned up when the conversation ends or after inactivity. Use `/browser connect` for free local browsing.
- **No file downloads** — cannot download files from the browser
website/docs/user-guide/features/built-in-plugins.md
+302
View File
@@ -0,0 +1,302 @@
---
sidebar_position: 12
sidebar_label: "Built-in Plugins"
title: "Built-in Plugins"
description: "Plugins shipped with Hermes Agent that run automatically via lifecycle hooks — disk-cleanup and friends"
---
# Built-in Plugins
Hermes ships a small set of plugins bundled with the repository. They live under `<repo>/plugins/<name>/` and load automatically alongside user-installed plugins in `~/.hermes/plugins/`. They use the same plugin surface as third-party plugins — hooks, tools, slash commands — just maintained in-tree.
See the [Plugins](/user-guide/features/plugins) page for the general plugin system, and [Build a Hermes Plugin](/guides/build-a-hermes-plugin) to write your own.
## How discovery works
The `PluginManager` scans four sources, in order:
1. **Bundled** — `<repo>/plugins/<name>/` (what this page documents)
2. **User** — `~/.hermes/plugins/<name>/`
3. **Project** — `./.hermes/plugins/<name>/` (requires `HERMES_ENABLE_PROJECT_PLUGINS=1`)
4. **Pip entry points** — `hermes_agent.plugins`
On name collision, later sources win — a user plugin named `disk-cleanup` would replace the bundled one.
`plugins/memory/` and `plugins/context_engine/` are deliberately excluded from bundled scanning. Those directories use their own discovery paths because memory providers and context engines are single-select providers configured through `hermes memory setup` / `context.engine` in config.
## Bundled plugins are opt-in
Bundled plugins ship disabled. Discovery finds them (they appear in `hermes plugins list` and the interactive `hermes plugins` UI), but none load until you explicitly enable them:
```bash
hermes plugins enable disk-cleanup
```
Or via `~/.hermes/config.yaml`:
```yaml
plugins:
enabled:
- disk-cleanup
```
This is the same mechanism user-installed plugins use. Bundled plugins are never auto-enabled — not on fresh install, not for existing users upgrading to a newer Hermes. You always opt in explicitly.
To turn a bundled plugin off again:
```bash
hermes plugins disable disk-cleanup
# or: remove it from plugins.enabled in config.yaml
```
## Currently shipped
The repo ships these bundled plugins under `plugins/`. All are opt-in — enable them via `hermes plugins enable <name>`.
| Plugin | Kind | Purpose |
|---|---|---|
| `disk-cleanup` | hooks + slash command | Auto-track ephemeral files and clean them on session end |
| `security-guidance` | hooks | Pattern-match dangerous code on `write_file`/`patch` and append a security warning (or block) — 25 rules (Apache-2.0 fork of Anthropic's `claude-plugins-official` patterns) |
| `observability/langfuse` | hooks | Trace turns / LLM calls / tools to [Langfuse](https://langfuse.com) |
| `observability/nemo_relay` | hooks | Relay observability events (turns / LLM calls / tools) to an NVIDIA NeMo endpoint |
| `teams_pipeline` | standalone | Microsoft Teams meeting pipeline — Graph-backed, transcript-first meeting summaries |
| `spotify` | backend (7 tools) | Native Spotify playback, queue, search, playlists, albums, library |
| `google_meet` | standalone | Join Meet calls, live-caption transcription, optional realtime duplex audio |
| `image_gen/openai` | image backend | OpenAI `gpt-image-2` image generation backend (alternative to FAL) |
| `image_gen/openai-codex` | image backend | OpenAI image generation via Codex OAuth |
| `image_gen/xai` | image backend | xAI `grok-2-image` backend |
| `hermes-achievements` | dashboard tab | Steam-style collectible badges generated from your real Hermes session history |
| `kanban/dashboard` | dashboard tab | Kanban board UI for the multi-agent dispatcher — tasks, comments, fan-out, board switching. See [Kanban Multi-Agent](./kanban.md). |
Memory providers (`plugins/memory/*`) and context engines (`plugins/context_engine/*`) are listed separately on [Memory Providers](./memory-providers.md) — they're managed through `hermes memory` and `hermes plugins` respectively. The full per-plugin detail for the two long-running hooks-based plugins follows.
### disk-cleanup
Auto-tracks and removes ephemeral files created during sessions — test scripts, temp outputs, cron logs, stale chrome profiles — without requiring the agent to remember to call a tool.
**How it works:**
| Hook | Behaviour |
|---|---|
| `post_tool_call` | When `write_file` / `terminal` / `patch` creates a file matching `test_*`, `tmp_*`, or `*.test.*` inside `HERMES_HOME` or `/tmp/hermes-*`, track it silently as `test` / `temp` / `cron-output`. |
| `on_session_end` | If any test files were auto-tracked during the turn, run the safe `quick` cleanup and log a one-line summary. Stays silent otherwise. |
**Deletion rules:**
| Category | Threshold | Confirmation |
|---|---|---|
| `test` | every session end | Never |
| `temp` | >7 days since tracked | Never |
| `cron-output` | >14 days since tracked | Never |
| empty dirs under HERMES_HOME | always | Never |
| `research` | >30 days, beyond 10 newest | Always (deep only) |
| `chrome-profile` | >14 days since tracked | Always (deep only) |
| files >500 MB | never auto | Always (deep only) |
**Slash command** — `/disk-cleanup` available in both CLI and gateway sessions:
```
/disk-cleanup status # breakdown + top-10 largest
/disk-cleanup dry-run # preview without deleting
/disk-cleanup quick # run safe cleanup now
/disk-cleanup deep # quick + list items needing confirmation
/disk-cleanup track <path> <category> # manual tracking
/disk-cleanup forget <path> # stop tracking (does not delete)
```
**State** — everything lives at `$HERMES_HOME/disk-cleanup/`:
| File | Contents |
|---|---|
| `tracked.json` | Tracked paths with category, size, and timestamp |
| `tracked.json.bak` | Atomic-write backup of the above |
| `cleanup.log` | Append-only audit trail of every track / skip / reject / delete |
**Safety** — cleanup only ever touches paths under `HERMES_HOME` or `/tmp/hermes-*`. Windows mounts (`/mnt/c/...`) are rejected. Well-known top-level state dirs (`logs/`, `memories/`, `sessions/`, `cron/`, `cache/`, `skills/`, `plugins/`, `disk-cleanup/` itself) are never removed even when empty — a fresh install does not get gutted on first session end.
**Enabling:** `hermes plugins enable disk-cleanup` (or check the box in `hermes plugins`).
**Disabling again:** `hermes plugins disable disk-cleanup`.
### security-guidance
Fast pattern-matched security warnings on file writes. When the agent's `write_file` / `patch` / `skill_manage` calls carry content matching a known-dangerous code pattern — `pickle.load`, `yaml.load` without `SafeLoader`, `eval(`, `os.system`, `subprocess(..., shell=True)`, JS `child_process.exec`, React `dangerouslySetInnerHTML`, raw `.innerHTML =` / `.outerHTML =` / `document.write`, Node `crypto.createCipher`, AES ECB mode, TLS verification disabled, XXE-prone `xml.etree` / `minidom` parsers, `<script src="//..." >` without SRI, `torch.load` without `weights_only=True`, GitHub Actions `${{ github.event.* }}` injection — the plugin appends a `⚠️ Security guidance` block to the tool's result.
The file is still written. The model reads the warning in the next turn's tool message and can either fix the code or document why the construct is safe in this context. Pattern matching has a non-trivial false-positive rate, which is why warn (not block) is the default.
**Coverage:** 25 rules total, covering unsafe deserialization, command injection, XSS sinks, crypto footguns, XXE, supply-chain (SRI), and CI/CD workflow injection. The pattern data is a verbatim Apache-2.0 fork of [Anthropic's `claude-plugins-official`](https://github.com/anthropics/claude-plugins-official/tree/main/plugins/security-guidance/hooks) — see the plugin's `LICENSE` and `NOTICE` files for attribution.
**Modes:**
| Env var | Effect |
|---|---|
| (unset) | **warn mode** (default) — file is written, warning appended to result |
| `SECURITY_GUIDANCE_BLOCK=1` | **block mode** — write refused, warning returned as the block reason |
| `SECURITY_GUIDANCE_DISABLE=1` | kill switch — plugin loads but does nothing |
**Enabling:** `hermes plugins enable security-guidance` (or check the box in `hermes plugins`).
**Disabling again:** `hermes plugins disable security-guidance`.
**What it does not do (yet):** the upstream Anthropic plugin has two more layers — an LLM diff review on each agent turn that touched files, and an agentic commit-time review that traces data flow across files. Neither is ported. The agent can already run those reviews on demand via `delegate_task`.
### observability/langfuse
Traces Hermes turns, LLM calls, and tool invocations to [Langfuse](https://langfuse.com) — an open-source LLM observability platform. One span per turn, one generation per API call, one tool observation per tool call. Usage totals, per-type token counts, and cost estimates come out of Hermes' canonical `agent.usage_pricing` numbers, so the Langfuse dashboard sees the same breakdown (input / output / `cache_read_input_tokens` / `cache_creation_input_tokens` / `reasoning_tokens`) that appears in `hermes logs`.
The plugin is fail-open: no SDK installed, no credentials, or a transient Langfuse error — all turn into a silent no-op in the hook. The agent loop is never impacted.
**Setup (interactive — recommended):**
```bash
hermes tools # → Langfuse Observability → Cloud or Self-Hosted
```
The wizard collects your keys, `pip install`s the `langfuse` SDK, and adds `observability/langfuse` to `plugins.enabled` for you. Restart Hermes and the next turn ships a trace.
**Setup (manual):**
```bash
pip install langfuse
hermes plugins enable observability/langfuse
```
Then put the credentials in `~/.hermes/.env`:
```bash
HERMES_LANGFUSE_PUBLIC_KEY=pk-lf-...
HERMES_LANGFUSE_SECRET_KEY=sk-lf-...
HERMES_LANGFUSE_BASE_URL=https://cloud.langfuse.com # or your self-hosted URL
```
**How it works:**
| Hook | Behaviour |
|---|---|
| `pre_api_request` / `pre_llm_call` | Open (or reuse) a per-turn root span "Hermes turn". Start a `generation` child observation for this API call with serialized recent messages as input. |
| `post_api_request` / `post_llm_call` | Close the generation, attach `usage_details`, `cost_details`, `finish_reason`, assistant output + tool calls. If no tool calls and non-empty content, close the turn. |
| `pre_tool_call` | Start a `tool` child observation with sanitized `args`. |
| `post_tool_call` | Close the tool observation with sanitized `result`. `read_file` payloads get summarized (head + tail + omitted-line count) so a huge file read stays under `HERMES_LANGFUSE_MAX_CHARS`. |
Session grouping keys off the Hermes session ID (or task ID for sub-agents) via `langfuse.propagate_attributes`, so everything in a single `hermes chat` session lives under one Langfuse session.
**Verify:**
```bash
hermes plugins list # observability/langfuse should show "enabled"
hermes chat -q "hello" # check the Langfuse UI for a "Hermes turn" trace
```
**Optional tuning** (in `.env`):
| Variable | Default | Purpose |
|---|---|---|
| `HERMES_LANGFUSE_ENV` | — | Environment tag on traces (`production`, `staging`, …) |
| `HERMES_LANGFUSE_RELEASE` | — | Release/version tag |
| `HERMES_LANGFUSE_SAMPLE_RATE` | `1.0` | Sampling rate passed to the SDK (0.0–1.0) |
| `HERMES_LANGFUSE_MAX_CHARS` | `12000` | Per-field truncation for message content / tool args / tool results |
| `HERMES_LANGFUSE_DEBUG` | `false` | Verbose plugin logging to `agent.log` |
Hermes-prefixed and standard SDK env vars (`LANGFUSE_PUBLIC_KEY`, `LANGFUSE_SECRET_KEY`, `LANGFUSE_BASE_URL`) are both accepted — Hermes-prefixed wins when both are set.
**Performance:** the Langfuse client is cached after the first hook call. If credentials or SDK are missing, that decision is also cached — subsequent hooks fast-return without re-checking env vars or reloading config.
**Disabling:** `hermes plugins disable observability/langfuse`. The plugin module is still discovered, but no module code runs until you re-enable.
### google_meet
Lets the agent **join, transcribe, and participate in Google Meet calls** — take notes on a meeting, summarize the back-and-forth after, follow up on specific points, and (optionally) speak replies back into the call via TTS.
**What it adds:**
- A headless virtual participant that joins a Meet URL using browser automation
- Live transcription of the meeting audio via the configured STT provider
- A `meet_summarize` / `meet_speak` / `meet_followup` toolset the agent invokes to act on what it heard
- Post-meeting artifacts (transcript, speaker-attributed notes, action items) saved under `~/.hermes/cache/google_meet/<meeting_id>/`
**Setup:**
```bash
hermes plugins enable google_meet
# Prompts you to sign in via the plugin's OAuth flow on first use —
# needs a Google account with Meet access. Host approval may be required
# if the meeting enforces "only invited participants can join".
```
Usage from chat:
> "Join meet.google.com/abc-defg-hij and take notes. After the call, send me a summary with action items."
The agent kicks off the meeting join, streams the transcription back into its context as the call proceeds, and produces a structured summary when the meeting ends (or when you tell it to stop).
**When to use it:** recurring standups where you want a bot to transcribe + summarize for async attendees; deposition-style interviews where you want structured notes; any case where you'd otherwise need Fireflies / Otter / Grain. When you'd rather not have an AI listening in — don't enable it.
**Disabling:** `hermes plugins disable google_meet`. Any cached transcripts and recordings stay in `~/.hermes/cache/google_meet/` until you remove them.
### hermes-achievements
Adds a **Steam-style achievements tab to the dashboard** — 60+ collectible, tiered badges generated from your real Hermes session history. Tool-chain feats, debugging patterns, vibe-coding streaks, skill/memory usage, model/provider variety, lifestyle quirks (weekend and night sessions). Originally authored by [@PCinkusz](https://github.com/PCinkusz) as an external plugin; brought in-tree so it stays in lockstep with Hermes feature changes.
**How it works:**
- Scans your entire `~/.hermes/state.db` session history on the dashboard backend
- Per-session stats are cached by `(started_at, last_active)` fingerprint, so only new or changed sessions re-analyze on subsequent scans
- First-ever scan runs in a background thread — the dashboard never blocks waiting for it, even on databases with thousands of sessions
- Unlock state is persisted to `$HERMES_HOME/plugins/hermes-achievements/state.json`
**Tier progression:** Copper → Silver → Gold → Diamond → Olympian. Each card exposes a "What counts" section listing the exact metric being tracked.
**Achievement states:**
| State | Meaning |
|---|---|
| Unlocked | At least one tier achieved |
| Discovered | Known achievement, progress visible, not yet earned |
| Secret | Hidden until Hermes detects the first related signal in your history |
**API** — routes mount under `/api/plugins/hermes-achievements/`:
| Endpoint | Purpose |
|---|---|
| `GET /achievements` | Full catalog with per-badge unlock state (returns a pending placeholder while the first cold scan is running) |
| `GET /scan-status` | State of the background scanner: `idle` / `running` / `failed`, last duration, run count |
| `GET /recent-unlocks` | Twenty most recently unlocked badges, newest first |
| `GET /sessions/{id}/badges` | Badges earned primarily in one specific session |
| `POST /rescan` | Manual synchronous rescan (blocks; use when the user clicks the rescan button) |
| `POST /reset-state` | Clear unlock history and cached snapshot |
**State files** — live under `$HERMES_HOME/plugins/hermes-achievements/`:
| File | Contents |
|---|---|
| `state.json` | Unlock history: which badges you've earned and when. Stable across Hermes updates. |
| `scan_snapshot.json` | Last completed scan payload (served immediately on dashboard load) |
| `scan_checkpoint.json` | Per-session stats cache keyed by fingerprint (makes warm rescans fast) |
**Performance notes:**
- Cold scan on ~8,000 sessions takes a few minutes. It runs in a background thread on first dashboard request; the UI sees a pending placeholder and polls `/scan-status`.
- **Incremental results during a cold scan** — the scanner publishes a partial snapshot every ~250 sessions so each dashboard refresh shows more badges unlocked as the scan progresses. No minute-long stare at zeros.
- Warm rescan reuses per-session stats for every session whose `started_at` + `last_active` fingerprint matches the checkpoint — completes in seconds even on large histories.
- The in-memory snapshot TTL is 120s; stale requests serve the old snapshot immediately and kick a background refresh. You never wait on a spinner just because TTL expired.
**Enabling:** Nothing to enable — `hermes-achievements` is a dashboard-only plugin (no lifecycle hooks, no model-visible tools). It auto-registers as a tab in `hermes dashboard` on first launch. The `plugins.enabled` config only gates lifecycle/tool plugins; dashboard plugins are discovered purely via their `dashboard/manifest.json`.
**Opting out:** Delete or rename `plugins/hermes-achievements/dashboard/manifest.json`, or override it with a user plugin of the same name in `~/.hermes/plugins/hermes-achievements/` that ships no dashboard. The plugin's state files under `$HERMES_HOME/plugins/hermes-achievements/` survive — reinstalling preserves your unlock history.
## Adding a bundled plugin
Bundled plugins are written exactly like any other Hermes plugin — see [Build a Hermes Plugin](/guides/build-a-hermes-plugin). The only differences are:
- Directory lives at `<repo>/plugins/<name>/` instead of `~/.hermes/plugins/<name>/`
- Manifest source is reported as `bundled` in `hermes plugins list`
- User plugins with the same name override the bundled version
A plugin is a good candidate for bundling when:
- It has no optional dependencies (or they're already `pip install .[all]` deps)
- The behaviour benefits most users and is opt-out rather than opt-in
- The logic ties into lifecycle hooks that the agent would otherwise have to remember to invoke
- It complements a core capability without expanding the model-visible tool surface
Counter-examples — things that should stay as user-installable plugins, not bundled: third-party integrations with API keys, niche workflows, large dependency trees, anything that would meaningfully change agent behaviour by default.
website/docs/user-guide/features/code-execution.md
+296
View File
@@ -0,0 +1,296 @@
---
sidebar_position: 8
title: "Code Execution"
description: "Programmatic Python execution with RPC tool access — collapse multi-step workflows into a single turn"
---
# Code Execution (Programmatic Tool Calling)
The `execute_code` tool lets the agent write Python scripts that call Hermes tools programmatically, collapsing multi-step workflows into a single LLM turn. The script runs in a child process on the agent host, communicating with Hermes over a Unix domain socket RPC.
## How It Works
1. The agent writes a Python script using `from hermes_tools import ...`
2. Hermes generates a `hermes_tools.py` stub module with RPC functions
3. Hermes opens a Unix domain socket and starts an RPC listener thread
4. The script runs in a child process — tool calls travel over the socket back to Hermes
5. Only the script's `print()` output is returned to the LLM; intermediate tool results never enter the context window
```python
# The agent can write scripts like:
from hermes_tools import web_search, web_extract
results = web_search("Python 3.13 features", limit=5)
for r in results["data"]["web"]:
content = web_extract([r["url"]])
# ... filter and process ...
print(summary)
```
**Available tools inside scripts:** `web_search`, `web_extract`, `read_file`, `write_file`, `search_files`, `patch`, `terminal` (foreground only).
## When the Agent Uses This
The agent uses `execute_code` when there are:
- **3+ tool calls** with processing logic between them
- Bulk data filtering or conditional branching
- Loops over results
The key benefit: intermediate tool results never enter the context window — only the final `print()` output comes back, dramatically reducing token usage.
## Practical Examples
### Data Processing Pipeline
```python
from hermes_tools import search_files, read_file
import json
# Find all config files and extract database settings
matches = search_files("database", path=".", file_glob="*.yaml", limit=20)
configs = []
for match in matches.get("matches", []):
content = read_file(match["path"])
configs.append({"file": match["path"], "preview": content["content"][:200]})
print(json.dumps(configs, indent=2))
```
### Multi-Step Web Research
```python
from hermes_tools import web_search, web_extract
import json
# Search, extract, and summarize in one turn
results = web_search("Rust async runtime comparison 2025", limit=5)
summaries = []
for r in results["data"]["web"]:
page = web_extract([r["url"]])
for p in page.get("results", []):
if p.get("content"):
summaries.append({
"title": r["title"],
"url": r["url"],
"excerpt": p["content"][:500]
})
print(json.dumps(summaries, indent=2))
```
### Bulk File Refactoring
```python
from hermes_tools import search_files, read_file, patch
# Find all Python files using deprecated API and fix them
matches = search_files("old_api_call", path="src/", file_glob="*.py")
fixed = 0
for match in matches.get("matches", []):
result = patch(
path=match["path"],
old_string="old_api_call(",
new_string="new_api_call(",
replace_all=True
)
if "error" not in str(result):
fixed += 1
print(f"Fixed {fixed} files out of {len(matches.get('matches', []))} matches")
```
### Build and Test Pipeline
```python
from hermes_tools import terminal, read_file
import json
# Run tests, parse results, and report
result = terminal("cd /project && python -m pytest --tb=short -q 2>&1", timeout=120)
output = result.get("output", "")
# Parse test output
passed = output.count(" passed")
failed = output.count(" failed")
errors = output.count(" error")
report = {
"passed": passed,
"failed": failed,
"errors": errors,
"exit_code": result.get("exit_code", -1),
"summary": output[-500:] if len(output) > 500 else output
}
print(json.dumps(report, indent=2))
```
## Execution Mode
`execute_code` has two execution modes controlled by `code_execution.mode` in `~/.hermes/config.yaml`:
| Mode | Working directory | Python interpreter |
|------|-------------------|--------------------|
| **`project`** (default) | The session's working directory (same as `terminal()`) | Active `VIRTUAL_ENV` / `CONDA_PREFIX` python, falling back to Hermes's own python |
| `strict` | A temp staging directory isolated from the user's project | `sys.executable` (Hermes's own python) |
**When to leave it on `project`:** you want `import pandas`, `from my_project import foo`, or relative paths like `open(".env")` to work the same way they do in `terminal()`. This is almost always what you want.
**When to flip to `strict`:** you need maximum reproducibility — you want the same interpreter every session regardless of which venv the user activated, and you want scripts quarantined from the project tree (no risk of accidentally reading project files through a relative path).
```yaml
# ~/.hermes/config.yaml
code_execution:
mode: project # or "strict"
```
Fallback behavior in `project` mode: if `VIRTUAL_ENV` / `CONDA_PREFIX` is unset, broken, or points at a Python older than 3.8, the resolver falls back cleanly to `sys.executable` — it never leaves the agent without a working interpreter.
Security-critical invariants are identical across both modes:
- environment scrubbing (API keys, tokens, credentials stripped)
- tool whitelist (scripts cannot call `execute_code` recursively, `delegate_task`, or MCP tools)
- resource limits (timeout, stdout cap, tool-call cap)
Switching mode changes where scripts run and which interpreter runs them, not what credentials they can see or which tools they can call.
## Resource Limits
| Resource | Limit | Notes |
|----------|-------|-------|
| **Timeout** | 5 minutes (300s) | Script is killed with SIGTERM, then SIGKILL after 5s grace |
| **Stdout** | 50 KB | Output truncated with `[output truncated at 50KB]` notice |
| **Stderr** | 10 KB | Included in output on non-zero exit for debugging |
| **Tool calls** | 50 per execution | Error returned when limit reached |
All limits are configurable via `config.yaml`:
```yaml
# In ~/.hermes/config.yaml
code_execution:
mode: project # project (default) | strict
timeout: 300 # Max seconds per script (default: 300)
max_tool_calls: 50 # Max tool calls per execution (default: 50)
```
## How Tool Calls Work Inside Scripts
When your script calls a function like `web_search("query")`:
1. The call is serialized to JSON and sent over a Unix domain socket to the parent process
2. The parent dispatches through the standard `handle_function_call` handler
3. The result is sent back over the socket
4. The function returns the parsed result
This means tool calls inside scripts behave identically to normal tool calls — same rate limits, same error handling, same capabilities. The only restriction is that `terminal()` is foreground-only (no `background` or `pty` parameters).
## Error Handling
When a script fails, the agent receives structured error information:
- **Non-zero exit code**: stderr is included in the output so the agent sees the full traceback
- **Timeout**: Script is killed and the agent sees `"Script timed out after 300s and was killed."`
- **Interruption**: If the user sends a new message during execution, the script is terminated and the agent sees `[execution interrupted — user sent a new message]`
- **Tool call limit**: When the 50-call limit is hit, subsequent tool calls return an error message
The response always includes `status` (success/error/timeout/interrupted), `output`, `tool_calls_made`, and `duration_seconds`.
## Security
:::danger Security Model
The child process runs with a **minimal environment**. API keys, tokens, and credentials are stripped by default. The script accesses tools exclusively via the RPC channel — it cannot read secrets from environment variables unless explicitly allowed.
:::
Environment variables containing `KEY`, `TOKEN`, `SECRET`, `PASSWORD`, `CREDENTIAL`, `PASSWD`, or `AUTH` in their names are excluded. Only safe system variables (`PATH`, `HOME`, `LANG`, `SHELL`, `PYTHONPATH`, `VIRTUAL_ENV`, etc.) are passed through.
### Skill Environment Variable Passthrough
When a skill declares `required_environment_variables` in its frontmatter, those variables are **automatically passed through** to both `execute_code` and `terminal` child processes after the skill is loaded. This lets skills use their declared API keys without weakening the security posture for arbitrary code.
For non-skill use cases, you can explicitly allowlist variables in `config.yaml`:
```yaml
terminal:
env_passthrough:
- MY_CUSTOM_KEY
- ANOTHER_TOKEN
```
See the [Security guide](/user-guide/security#environment-variable-passthrough) for full details.
### `HERMES_*` variables in the child
The child process receives only a small, fixed set of operational `HERMES_*`
variables by exact name:
- `HERMES_HOME`
- `HERMES_PROFILE`
- `HERMES_CONFIG`
- `HERMES_ENV`
(plus `HERMES_RPC_DIR` / `HERMES_RPC_SOCKET` / `TZ` / `HOME`, which Hermes
injects explicitly so the RPC channel works).
:::note Behavior change
Earlier versions passed **any** variable whose name began with `HERMES_`
through to the child. That broad prefix was removed for security hardening: it
could leak `HERMES_*`-named configuration that doesn't match a secret substring
(for example `HERMES_BASE_URL`, `HERMES_KANBAN_DB`, or a `HERMES_*_WEBHOOK`
endpoint) into arbitrary sandboxed code.
If an `execute_code` script — or a repo/plugin module it imports at import time
— relied on a `HERMES_*` variable outside the four operational names above, it
will now find that variable **unset** in the child. The drop is intentional,
not a bug.
:::
**Workaround — opt the variable back in explicitly.** Both routes pass the
variable through `execute_code` *and* `terminal` children, and neither weakens
the secret-stripping guarantee (Hermes-managed provider credentials can never
be re-allowed this way):
1. **Per-machine, in `config.yaml`** — add the exact variable name to the
passthrough allowlist:
```yaml
terminal:
env_passthrough:
- HERMES_KANBAN_DB
- HERMES_BASE_URL
```
2. **Per-skill, in the skill's frontmatter** — declare it so it is registered
automatically whenever that skill is loaded:
```yaml
required_environment_variables:
- HERMES_KANBAN_DB
```
**Diagnosing it.** When the child drops one or more non-allowlisted `HERMES_*`
variables, Hermes emits a one-line `debug` log naming them and pointing at the
`env_passthrough` escape hatch. Run with debug logging (`hermes logs --level
DEBUG`, or check `~/.hermes/logs/agent.log`) and look for
`execute_code: dropped N non-allowlisted HERMES_* var(s)` if a script behaves
as though a `HERMES_*` variable is missing.
Hermes always writes the script and the auto-generated `hermes_tools.py` RPC stub into a temp staging directory that is cleaned up after execution. In `strict` mode the script also *runs* there; in `project` mode it runs in the session's working directory (the staging directory stays on `PYTHONPATH` so imports still resolve). The child process runs in its own process group so it can be cleanly killed on timeout or interruption.
## execute_code vs terminal
| Use Case | execute_code | terminal |
|----------|-------------|----------|
| Multi-step workflows with tool calls between | ✅ | ❌ |
| Simple shell command | ❌ | ✅ |
| Filtering/processing large tool outputs | ✅ | ❌ |
| Running a build or test suite | ❌ | ✅ |
| Looping over search results | ✅ | ❌ |
| Interactive/background processes | ❌ | ✅ |
| Needs API keys in environment | ⚠️ Only via [passthrough](/user-guide/security#environment-variable-passthrough) | ✅ (most pass through) |
**Rule of thumb:** Use `execute_code` when you need to call Hermes tools programmatically with logic between calls. Use `terminal` for running shell commands, builds, and processes.
## Platform Support
Code execution requires Unix domain sockets and is available on **Linux and macOS only**. It is automatically disabled on Windows — the agent falls back to regular sequential tool calls.
website/docs/user-guide/features/codex-app-server-runtime.md
+445
View File
@@ -0,0 +1,445 @@
---
title: Codex App-Server Runtime (optional)
sidebar_label: Codex App-Server Runtime
---
# Codex App-Server Runtime
Hermes can optionally hand `openai/*` and `openai-codex/*` turns to the [Codex CLI app-server](https://github.com/openai/codex) instead of running its own tool loop. When enabled, terminal commands, file edits, sandboxing, and MCP tool calls all execute inside Codex's runtime — Hermes becomes the shell around it (sessions DB, slash commands, gateway, memory and skill review).
This is **opt-in only**. Default Hermes behavior is unchanged unless you flip the flag. Hermes never auto-routes you onto this runtime.
:::tip
Not using OpenAI Codex? `hermes setup --portal` configures a non-Codex backend with Claude/Gemini/etc. in one step. See [Nous Portal](/integrations/nous-portal).
:::
## Why
- Run OpenAI agent turns against your **ChatGPT subscription** (no API key required) using the same auth flow Codex CLI uses.
- Use **Codex's own toolset and sandbox** — `shell` for terminal/read/write/search, `apply_patch` for structured edits, `update_plan` for planning, all running inside seatbelt/landlock sandboxing.
- **Native Codex plugins** — Linear, GitHub, Gmail, Calendar, Canva, etc. — installed via `codex plugin` are auto-migrated and active in your Hermes session.
- **Hermes' richer tools come along** — web_search, web_extract, browser automation, vision, image generation, skills, and TTS work via an MCP callback. Codex calls back into Hermes for tools it doesn't have built in.
- **Memory and skill nudges keep working** — Codex's events are projected into Hermes' message shape so the self-improvement loop sees a normal-looking transcript.
## What tools the model actually has
This is the part most users want to know up front. When this runtime is on, the model running your turn has three independent sources of tools:
### 1. Codex's built-in toolset (always on)
These ship with `codex app-server` itself — no Hermes involvement, no MCP, no plugins. All five are available the moment the runtime starts:
- **`shell`** — runs arbitrary shell commands inside the sandbox. This is how the model reads files (`cat`, `head`, `tail`), writes them (`echo > foo`, heredocs), searches them (`find`, `rg`, `grep`), navigates directories (`ls`, `cd`), runs builds, manages processes, and anything else you'd do in bash.
- **`apply_patch`** — applies a structured multi-file diff in Codex's patch format. The model uses this for non-trivial code edits (adding a function, refactoring across files); shell heredocs are still available for one-off writes.
- **`update_plan`** — codex's internal todo / plan tracker. Equivalent of Hermes' `todo` tool, but managed entirely inside codex's runtime.
- **`view_image`** — load a local image file into the conversation so the model can see it.
- **`web_search`** — codex has its own built-in web search when configured. Hermes also exposes `web_search` (Firecrawl-backed) via the callback below; the model picks whichever it prefers.
So **anything you'd do via terminal — read/write/search/find/run — codex does natively**. The sandbox profile (`:workspace` by default when you enable the runtime) controls what's writable.
### 2. Native Codex plugins (auto-migrated from your `codex plugin` install)
When you enable the runtime, Hermes queries codex's `plugin/list` RPC and writes a `[plugins."<name>@openai-curated"]` entry for every plugin you have installed. The plugins themselves are managed by codex and authorized once via codex's own UI.
Examples (the ones the OpenClaw thread highlighted as "YouTube-video-worthy"):
- **Linear** — find/update issues
- **GitHub** — search code, view PRs, comment
- **Gmail** — read/send mail
- **Google Calendar** — create/find events
- **Outlook calendar/email** — same shape via the Microsoft connector
- **Canva** — design generation
- ...whatever else you've installed via `codex plugin marketplace add openai-curated` + `codex plugin install ...`
What's NOT migrated:
- Plugins you haven't installed yet — install them in Codex first.
- ChatGPT app marketplace entries (`app/list`) — these are already enabled inside codex by virtue of your account auth.
### 3. Hermes tool callback (MCP server, registered in `~/.codex/config.toml`)
Hermes registers itself as an MCP server so codex can call back for tools codex doesn't ship with. Available via the callback:
- **`web_search`** / **`web_extract`** — Firecrawl-backed; tends to be cleaner than scraping for structured content.
- **`browser_navigate` / `browser_click` / `browser_type` / `browser_press` / `browser_snapshot` / `browser_scroll` / `browser_back` / `browser_get_images` / `browser_console` / `browser_vision`** — full browser automation via Camofox or Browserbase.
- **`vision_analyze`** — call a separate vision model to inspect an image (different from codex's `view_image` which loads it into the conversation).
- **`image_generate`** — image generation through Hermes' image_gen plugin chain.
- **`skill_view` / `skills_list`** — read from Hermes' skill library.
- **`text_to_speech`** — TTS through Hermes' configured provider.
When the model wants one of these, codex spawns the `hermes_tools_mcp_server` subprocess via stdio MCP, the call is dispatched through `model_tools.handle_function_call()` (same code path as Hermes' default runtime), and the result is returned to codex like any other MCP response.
### What's NOT available on this runtime
These four Hermes tools require the running AIAgent context (mid-loop state) to dispatch, and a stateless MCP callback can't drive them. Switch back to the default runtime (`/codex-runtime auto`) when you need any of them:
- **`delegate_task`** — spawn subagents
- **`memory`** — Hermes' persistent memory store
- **`session_search`** — cross-session search
- **`todo`** — Hermes' todo store (codex's `update_plan` is the in-runtime equivalent)
## Workflow features (`/goal`, kanban, cron)
### `/goal` (the Ralph loop)
**Works on this runtime.** Goals persist in `state_meta` keyed by session id, the continuation prompt feeds back as a normal user message through `run_conversation()`, and codex executes the next turn natively. The goal judge runs via the auxiliary client (configured via `auxiliary.goal_judge` in config.yaml), independent of which runtime is active. The judge's "blocked, needs user input" verdict is a clean escape if codex stalls on approvals.
**One thing to be aware of:** each continuation prompt is a fresh codex turn, which means codex re-evaluates command approval policy from scratch. If you're doing a long-running goal with lots of writes, expect more approval prompts than you'd see on a single in-session task. Set `default_permissions = ":workspace"` (which Hermes does automatically when you enable the runtime) so simple workspace writes don't require prompting.
### Kanban (multi-agent worktree dispatch)
**Works on this runtime, with one subtle dependency.** The kanban dispatcher spawns each worker as a separate `hermes chat -q` subprocess that reads the user's config — which means if `model.openai_runtime: codex_app_server` is set globally, workers also come up on the codex runtime.
What works inside a codex-runtime worker:
- Codex's full toolset (shell, apply_patch, update_plan, view_image, web_search) — the worker does its actual task work natively
- The migrated codex plugins — Linear, GitHub, etc.
- The Hermes tool callback for browser_*, vision, image_gen, skills, TTS
What also works because the MCP callback exposes them:
- **`kanban_complete` / `kanban_block` / `kanban_comment` / `kanban_heartbeat`** — the worker handoff tools. These read `HERMES_KANBAN_TASK` from env (set by the dispatcher), gate access correctly, and write to the per-board SQLite DB pinned by `HERMES_KANBAN_DB`. Without these in the callback, a worker on this runtime could do its task but couldn't report back, hanging until the dispatcher's timeout.
- **`kanban_show` / `kanban_list`** — read-only board queries for the worker to check its own context.
- **`kanban_create` / `kanban_unblock` / `kanban_link`** — orchestrator-only operations. Available for orchestrator agents running on the codex runtime that need to dispatch new tasks.
The kanban tools are gated by `HERMES_KANBAN_TASK` env var the dispatcher sets — that var is propagated to the codex subprocess (codex inherits env) and from there to the spawned `hermes-tools` MCP server subprocess. So the tools see the right task id and gate correctly. For Codex app-server workers, Hermes also passes narrow app-server sandbox overrides when `HERMES_KANBAN_TASK` is present: keep `workspace-write` sandboxing, add the **board DB directory plus every Kanban path the dispatcher pinned** as extra writable roots (`HERMES_KANBAN_WORKSPACES_ROOT`, `HERMES_KANBAN_WORKSPACE`, legacy `HERMES_KANBAN_ROOT` — deduplicated, DB-dir first), and keep network disabled by default. This avoids the brittle `:danger-no-sandbox` workaround while letting `kanban_complete` / `kanban_block` update the board DB **and** letting workers write reports/artifacts under workspace mounts that live outside the DB directory (e.g. `/media/.../kanban-workspaces/...` on a separate drive — [issue #27941](https://github.com/NousResearch/hermes-agent/issues/27941)).
### Cron jobs
**Not specifically tested.** Cron jobs run via `cronjob` → `AIAgent.run_conversation`, the same code path as the CLI. If the cron job's config has `openai_runtime: codex_app_server` it'll run on codex. The same tool-availability rules apply — codex built-ins + plugins + MCP callback work, agent-loop tools (delegate_task, memory, session_search, todo) don't. If your cron job relies on those, scope the cron to a profile that uses the default runtime.
## Trade-offs
| | Hermes default runtime | Codex app-server (opt-in) |
|---|---|---|
| `delegate_task` subagents | yes | not available — needs agent loop context |
| `memory`, `session_search`, `todo` | yes | not available — needs agent loop context |
| `web_search`, `web_extract` | yes | yes (via MCP callback) |
| Browser automation (Camofox/Browserbase) | yes | yes (via MCP callback) |
| `vision_analyze`, `image_generate` | yes | yes (via MCP callback) |
| `skill_view`, `skills_list` | yes | yes (via MCP callback) |
| `text_to_speech` | yes | yes (via MCP callback) |
| Codex `shell` (terminal/read/write/search/find/run) | — | yes (Codex built-in) |
| Codex `apply_patch` (structured multi-file edits) | — | yes (Codex built-in) |
| Codex `update_plan` (in-runtime todo) | — | yes (Codex built-in) |
| Codex `view_image` (load image into conversation) | — | yes (Codex built-in) |
| Codex sandbox (seatbelt/landlock, profiles) | — | yes (Codex built-in) |
| ChatGPT subscription auth | — | yes (via `openai-codex` provider) |
| Native Codex plugins (Linear, GitHub, etc.) | — | yes (auto-migrated) |
| User MCP servers | yes | yes (auto-migrated to codex) |
| Memory + skill review (background) | yes | yes (via item projection) |
| Multi-turn conversations | yes | yes |
| `/goal` (Ralph loop) | yes | yes |
| Kanban worker dispatch | yes | yes (via callback) |
| Kanban orchestrator tools | yes | yes (via callback) |
| All gateway platforms | yes | yes |
| Non-OpenAI providers | yes | n/a — OpenAI/Codex-scoped |
## Prerequisites
1. **Codex CLI installed:**
```bash
npm i -g @openai/codex
codex --version # 0.130.0 or newer
```
2. **Codex OAuth login.** The codex subprocess reads `~/.codex/auth.json`. Two ways to populate it:
```bash
codex login # writes tokens to ~/.codex/auth.json
```
Hermes' own `hermes auth login codex` writes to `~/.hermes/auth.json` — that's a separate session. **Run `codex login` separately** if you haven't.
3. **(Optional) Install the Codex plugins you want.** When you enable the runtime, Hermes auto-migrates whichever curated plugins you've already installed via Codex CLI:
```bash
codex plugin marketplace add openai-curated
# then via codex's TUI, install Linear / GitHub / Gmail / etc.
```
Hermes will discover them and write `[plugins."<name>@openai-curated"]` entries to `~/.codex/config.toml` automatically.
## Enabling
In a Hermes session:
```
/codex-runtime codex_app_server
```
That command:
- Verifies the `codex` CLI is installed (blocks with an install hint if not).
- Persists `model.openai_runtime: codex_app_server` to your config.yaml.
- Migrates user MCP servers from `~/.hermes/config.yaml` to `~/.codex/config.toml`.
- **Discovers and migrates installed native Codex plugins** (Linear, GitHub, Gmail, Calendar, Canva, etc.) by querying Codex's `plugin/list` RPC.
- **Registers Hermes' own tools as an MCP server** so the codex subprocess can call back for tools codex doesn't ship with.
- **Writes `default_permissions = ":workspace"`** so the sandbox allows writes within the workspace without prompting for every operation.
- Tells you what was migrated. Takes effect on the **next** session — the current cached agent keeps the prior runtime so prompt caches stay valid.
Synonyms: `/codex-runtime on`, `/codex-runtime off`, `/codex-runtime auto`.
To check current state without changing anything:
```
/codex-runtime
```
You can also set it manually in `~/.hermes/config.yaml`:
```yaml
model:
openai_runtime: codex_app_server # default is "auto" (= Hermes runtime)
```
## Self-improvement loop (memory + skill nudges)
Hermes' background self-improvement fires on counter thresholds:
- Every 10 user prompts → a forked review agent looks at the conversation and decides whether anything should be saved to memory.
- Every 10 tool iterations within a single turn → same idea but for skills (`skill_manage` writes).
**Both keep working on the codex runtime.** The codex path projects each completed `commandExecution` / `fileChange` / `mcpToolCall` / `dynamicToolCall` item into a synthetic `assistant tool_call` + `tool` result message, so by the time the review runs it sees the same shape it sees on the default Hermes runtime.
How the wiring stays equivalent:
| | Default runtime | Codex runtime |
|---|---|---|
| `_turns_since_memory` increments | per user prompt, in run_conversation pre-loop | same code path, before the early-return |
| `_iters_since_skill` increments | per tool iteration in the chat-completions loop | by `turn.tool_iterations` after the codex turn returns |
| Memory trigger (`_turns_since_memory >= _memory_nudge_interval`) | computed in pre-loop, fires after response | computed in pre-loop, passed through to codex helper |
| Skill trigger (`_iters_since_skill >= _skill_nudge_interval`) | computed after the loop | computed after the codex turn |
| `_spawn_background_review(messages_snapshot=..., review_memory=..., review_skills=...)` | called when either trigger fires | called identically when either trigger fires |
One detail: the review fork itself needs to call Hermes' agent-loop tools (`memory`, `skill_manage`), which require Hermes' own dispatch. So when the parent agent is on `codex_app_server`, the review fork is **downgraded to `codex_responses`** — same OAuth credentials, same `openai-codex` provider, but talks to OpenAI's Responses API directly so Hermes owns the loop and the agent-loop tools work. This is invisible to the user.
Net effect: enable the codex runtime and your memory + skill nudges keep firing exactly as they would otherwise.
## How approvals work
Codex requests approval before executing commands or applying patches. These get translated into Hermes' standard "Dangerous Command" prompt:
```
╭───────────────────────────────────────╮
│ Dangerous Command │
│ │
│ /bin/bash -lc 'echo hello > foo.txt' │
│ │
│ ❯ 1. Allow once │
│ 2. Allow for this session │
│ 3. Deny │
│ │
│ Codex requests exec in /your/cwd │
╰───────────────────────────────────────╯
```
- **Allow once** → approve this single command.
- **Allow for this session** → Codex won't re-prompt for similar commands.
- **Deny** → command is rejected; Codex continues in read-only mode.
For `apply_patch` (file edit) approvals, Hermes shows a summary of what changed (`1 add, 1 update: /tmp/new.py, /tmp/old.py`) when codex provides the data via the corresponding `fileChange` item.
## Permission profiles
Codex has three built-in permission profiles:
- `:read-only` — no writes; every shell command requires approval
- `:workspace` — writes within the current workspace allowed without prompts (Hermes' default when you enable the runtime)
- `:danger-no-sandbox` — no sandbox at all (don't use this unless you understand it)
You can override the default in `~/.codex/config.toml` outside Hermes' managed block:
```toml
default_permissions = ":read-only"
```
(Hermes will preserve your override on re-migration as long as it lives outside the `# managed by hermes-agent` markers.)
## Auxiliary tasks and ChatGPT subscription token cost
When this runtime is on with the `openai-codex` provider, **auxiliary tasks (title generation, context compression, vision auto-detect, the background self-improvement review fork) also flow through your ChatGPT subscription by default**, because Hermes' auxiliary client uses the main provider/model when no per-task override is set.
This isn't specific to `codex_app_server` — it's true for the existing `codex_responses` path too — but it's more visible here because you're explicitly opting in for the subscription billing.
To route specific aux tasks to a cheaper / different model, set explicit overrides in `~/.hermes/config.yaml`:
```yaml
auxiliary:
title_generation:
provider: openrouter
model: google/gemini-3-flash-preview
compression:
provider: openrouter
model: google/gemini-3-flash-preview
vision:
provider: openrouter
model: google/gemini-3-flash-preview
goal_judge:
provider: openrouter
model: google/gemini-3-flash-preview
```
The self-improvement review fork inherits the main runtime via `_current_main_runtime()` and Hermes downgrades it from `codex_app_server` to `codex_responses` automatically (so the fork can actually call `memory` and `skill_manage` — Hermes' own agent-loop tools). That fork still uses your subscription auth unless you've routed aux tasks elsewhere.
## Editing `~/.codex/config.toml` safely
Hermes wraps everything it manages between two marker comments:
```toml
# managed by hermes-agent — `hermes codex-runtime migrate` regenerates this section
default_permissions = ":workspace"
[mcp_servers.filesystem]
...
[plugins."github@openai-curated"]
...
# end hermes-agent managed section
```
Anything **outside** that block is yours. Re-running migration (via `/codex-runtime codex_app_server` or whenever you toggle the runtime on) replaces the managed block in place but preserves user content above and below it verbatim. This means you can:
- Add your own MCP servers Hermes doesn't know about
- Override `default_permissions` to `:read-only` if you prefer to be prompted
- Configure codex-only options (model, providers, otel, etc.)
- Add user-defined permission profiles in `[permissions.<name>]` tables
Anything you add **inside** the managed block will get clobbered on the next migration. If you need a tweak that requires editing the managed block, file an issue and we'll add the knob.
## Multi-profile / multi-tenant setups
By default, Hermes points the codex subprocess at `~/.codex/` regardless of which Hermes profile is active. This means `hermes -p work` and `hermes -p personal` share the same Codex auth, plugins, and config. For most users this is the right behavior — it matches what running `codex` CLI directly would do.
If you want per-profile Codex isolation (separate auth, separate installed plugins, separate config), set `CODEX_HOME` explicitly per profile. The cleanest way is to point at a directory under your `HERMES_HOME`:
```bash
# Inside the work profile, you might wrap hermes:
CODEX_HOME=~/.hermes/profiles/work/codex hermes chat
```
You'll need to re-run `codex login` once with that `CODEX_HOME` set so the OAuth tokens land in the profile-scoped location. After that, `hermes -p work` will operate on isolated Codex state.
We don't auto-scope this because moving an existing user's `~/.codex/` would silently invalidate their Codex CLI auth — anyone who already ran `codex login` would have to re-authenticate. Opt-in feels safer than surprising users.
## HOME environment variable passthrough
Hermes does NOT rewrite `HOME` when spawning the codex app-server subprocess (we use `os.environ.copy()` and only overlay `CODEX_HOME` and `RUST_LOG`). This means:
- Commands codex runs via its `shell` tool see the real user `HOME` and find `~/.gitconfig`, `~/.gh/`, `~/.aws/`, `~/.npmrc`, etc. correctly.
- Codex's internal state stays isolated through `CODEX_HOME` (which points at `~/.codex/` by default).
This matches the boundary OpenClaw arrived at after some early experimentation: isolate Codex's state, leave the user's home alone. (Cf. openclaw/openclaw#81562.)
## MCP server migration
Hermes' `mcp_servers` config is auto-translated to the TOML format Codex expects. The migration runs every time you enable the runtime and is idempotent — re-runs replace the managed section but preserve any user-edited Codex config.
What translates:
| Hermes (`config.yaml`) | Codex (`config.toml`) |
|---|---|
| `command` + `args` + `env` | stdio transport |
| `url` + `headers` | streamable_http transport |
| `timeout` | `tool_timeout_sec` |
| `connect_timeout` | `startup_timeout_sec` |
| `enabled: false` | `enabled = false` |
What's not migrated:
- Hermes-specific keys like `sampling` (Codex's MCP client has no equivalent — these are dropped with a per-server warning).
## Native Codex plugin migration
Plugins installed via `codex plugin` (Linear, GitHub, Gmail, Calendar, Canva, etc.) are discovered through Codex's `plugin/list` RPC. For each plugin where `installed: true`, Hermes writes a `[plugins."<name>@openai-curated"]` block enabling it in your Hermes session.
This means: when your friend says "I have Calendar and GitHub set up in my Codex CLI" and they enable Hermes' codex runtime, Hermes activates those automatically. No re-configuration needed.
What's NOT migrated:
- Plugins you haven't installed yet — install them in Codex first.
- Plugins where codex reports `availability != AVAILABLE` (broken install, expired OAuth, removed from marketplace, etc.). These are skipped to avoid writing config that would fail at activation time.
- ChatGPT app marketplace entries (the per-account `app/list` results — these are already enabled inside codex by virtue of your account auth).
- Plugin OAuth — you authorize each plugin once in Codex itself; Hermes doesn't touch credentials.
## Hermes tool callback (the new MCP server)
Codex's built-in toolset covers shell/file ops/patches but doesn't have web search, browser automation, vision, image generation, etc. To keep those usable in a codex turn, Hermes registers itself as an MCP server in `~/.codex/config.toml`:
```toml
[mcp_servers.hermes-tools]
command = "/path/to/python"
args = ["-m", "agent.transports.hermes_tools_mcp_server"]
env = { HERMES_HOME = "/your/.hermes", PYTHONPATH = "...", HERMES_QUIET = "1" }
startup_timeout_sec = 30.0
tool_timeout_sec = 600.0
```
When the model calls `web_search` (or another exposed Hermes tool), codex spawns the `hermes_tools_mcp_server` subprocess via stdio, the request is dispatched through `model_tools.handle_function_call()`, and the result is projected back to codex like any other MCP response.
**Tools available via the callback:** `web_search`, `web_extract`, `browser_navigate`, `browser_click`, `browser_type`, `browser_press`, `browser_snapshot`, `browser_scroll`, `browser_back`, `browser_get_images`, `browser_console`, `browser_vision`, `vision_analyze`, `image_generate`, `skill_view`, `skills_list`, `text_to_speech`.
**Tools NOT available:** `delegate_task`, `memory`, `session_search`, `todo`. These need the running AIAgent context to dispatch (mid-loop state) and a stateless MCP callback can't drive them. Use the default Hermes runtime (`/codex-runtime auto`) when you need these.
## Disabling
Switch back at any time:
```
/codex-runtime auto
```
Effective on the next session. The Codex managed block stays in `~/.codex/config.toml` so you can re-enable later without losing config — or remove it manually if you prefer.
## Limitations
This runtime is **opt-in beta**. Working as of Hermes Agent 2026.5 + Codex CLI 0.130.0:
- Multi-turn conversations
- `commandExecution` and `fileChange` (apply_patch) approvals via Hermes UI
- MCP tool calls (verified against `@modelcontextprotocol/server-filesystem` and the new `hermes-tools` callback)
- Native Codex plugin migration (verified against Linear / GitHub / Calendar inventory)
- Deny/cancel paths
- Toggle on/off cycle
- Memory and skill nudge counters (verified live via integration tests)
- Hermes web_search through codex (verified live: "OpenAI Codex CLI – Getting Started" returned end-to-end)
Known limitations:
- **Hermes auth and codex auth are separate sessions.** You need both `codex login` AND `hermes auth login codex` for the cleanest UX (the runtime uses codex's session for the LLM call). This is a deliberate design choice in Hermes' `_import_codex_cli_tokens` — Hermes won't share OAuth state with codex CLI to avoid clobbering each other on token refresh.
- **`delegate_task`, `memory`, `session_search`, `todo` are unavailable on this runtime.** They need the running AIAgent context which a stateless MCP callback can't provide. Use `/codex-runtime auto` when you need these.
- **No inline patch preview in approval prompts when codex doesn't track the changeset.** Codex's `fileChange` approval params don't always carry the changeset. Hermes caches the data from the corresponding `item/started` notification when possible, but if approval arrives before the item has streamed, the prompt falls back to whatever `reason` codex provides.
- **Sub-second cancellation isn't guaranteed.** Mid-stream interrupts (Ctrl+C while codex is responding) are sent via `turn/interrupt`, but if codex has already flushed the final message, you get the response anyway.
If you find a bug, [open an issue](https://github.com/NousResearch/hermes-agent/issues) with the output of `hermes logs --since 5m`. Mention `codex-runtime` in the title so it's easy to triage.
## Architecture
```
┌─── Hermes shell (CLI / TUI / gateway) ───┐
│ sessions DB · slash commands · memory │
│ & skill review · cron · session pickers │
└──┬──────────────────────────────────────┬┘
│ user_message final │
▼ text + │
┌──────────────────────────────────┐ projected │
│ AIAgent.run_conversation() │ messages │
│ if api_mode == codex_app_server │ │
│ → CodexAppServerSession │ │
│ else: chat_completions / codex_responses (default)
└────┬─────────────────────────────┘ │
│ JSON-RPC over stdio │
▼ │
┌──────────────────────────────────┐ │
│ codex app-server (subprocess) │──────────────┘
│ thread/start, turn/start │
│ item/* notifications │
│ shell + apply_patch + update_plan│
│ view_image + sandbox │
│ ┌─────────────────────────┐ │
│ │ MCP client │ │
│ │ ├─ user MCP servers │ │
│ │ ├─ native plugins │ │
│ │ │ (linear, github, │ │
│ │ │ gmail, calendar, │ │
│ │ │ canva, ...) │ │
│ │ └─ hermes-tools ───────┼─────────────────┐
│ │ (callback to │ │ │
│ │ Hermes' richer │ │ │
│ │ tools) │ │ │
│ └─────────────────────────┘ │ │
└──────────────────────────────────┘ │
│
▼
┌──────────────────────────────────────────────────────────┐
│ hermes_tools_mcp_server.py (subprocess on demand) │
│ web_search, web_extract, browser_*, vision_analyze, │
│ image_generate, skill_view, skills_list, text_to_speech│
└──────────────────────────────────────────────────────────┘
```
For implementation details, see [PR #24182](https://github.com/NousResearch/hermes-agent/pull/24182) and the [Codex app-server protocol README](https://github.com/openai/codex/blob/main/codex-rs/app-server/README.md).
website/docs/user-guide/features/computer-use.md
+201
View File
@@ -0,0 +1,201 @@
---
title: Computer Use
sidebar_position: 16
---
# Computer Use (macOS)
Hermes Agent can drive your Mac's desktop — clicking, typing, scrolling,
dragging — in the **background**. Your cursor doesn't move, keyboard focus
doesn't change, and macOS doesn't switch Spaces on you. You and the agent
co-work on the same machine.
Unlike most computer-use integrations, this works with **any tool-capable
model** — Claude, GPT, Gemini, or an open model on a local vLLM endpoint.
There's no Anthropic-native schema to worry about.
## How it works
The `computer_use` toolset speaks MCP over stdio to [`cua-driver`](https://github.com/trycua/cua),
a macOS driver that uses SkyLight private SPIs (`SLEventPostToPid`,
`SLPSPostEventRecordTo`) and the `_AXObserverAddNotificationAndCheckRemote`
accessibility SPI to:
- Post synthesized events directly to target processes — no HID event tap,
no cursor warp.
- Flip AppKit active-state without raising windows — no Space switching.
- Keep Chromium/Electron accessibility trees alive when windows are
occluded.
That combination is what OpenAI's Codex "background computer-use" ships.
cua-driver is the open-source equivalent.
## Enabling
Pick whichever path is most convenient — both run the same upstream installer:
**Option 1: dedicated CLI command (most direct).**
```
hermes computer-use install
```
This fetches and runs the upstream cua-driver installer:
`curl -fsSL https://raw.githubusercontent.com/trycua/cua/main/libs/cua-driver/scripts/install.sh`.
Use `hermes computer-use status` to verify the install.
**Option 2: enable the toolset interactively.**
1. Run `hermes tools`, pick `🖱️ Computer Use (macOS)` → `cua-driver (background)`.
2. The setup runs the upstream installer (same as Option 1).
After installing, regardless of which path you took:
3. Grant macOS permissions when prompted:
- **System Settings → Privacy & Security → Accessibility** → allow the
terminal (or Hermes app).
- **System Settings → Privacy & Security → Screen Recording** → allow
the same.
4. Start a session with the toolset enabled:
```
hermes -t computer_use chat
```
or add `computer_use` to your enabled toolsets in `~/.hermes/config.yaml`.
## Keeping cua-driver up to date
The cua-driver project ships fixes regularly (e.g. v0.1.6 fixed a Safari
window-focus bug for UTM workflows). Hermes refreshes the binary in two
places so you don't get stuck on a stale release:
- **`hermes update`** — when you update Hermes itself, if `cua-driver` is
on PATH the upstream installer re-runs at the end of the update.
No-op for non-macOS users and for users without cua-driver installed.
- **`hermes computer-use install --upgrade`** — manual force-refresh.
Re-runs the upstream installer regardless of whether cua-driver is
already installed. Use this when you want the latest fix without
waiting for the next agent update.
`hermes computer-use status` shows the installed version next to the
binary path.
## Quick example
User prompt: *"Find my latest email from Stripe and summarise what they want me to do."*
The agent's plan:
1. `computer_use(action="capture", mode="som", app="Mail")` — gets a
screenshot of Mail with every sidebar item, toolbar button, and message
row numbered.
2. `computer_use(action="click", element=14)` — clicks the search field
(element #14 from the capture).
3. `computer_use(action="type", text="from:stripe")`
4. `computer_use(action="key", keys="return", capture_after=True)` — submit
and get the new screenshot.
5. Click the top result, read the body, summarise.
During all of this, your cursor stays wherever you left it and Mail never
comes to front.
## Provider compatibility
| Provider | Vision? | Works? | Notes |
|---|---|---|---|
| Anthropic (Claude Sonnet/Opus 3+) | ✅ | ✅ | Best overall; SOM + raw coordinates. |
| OpenRouter (any vision model) | ✅ | ✅ | Multi-part tool messages supported. |
| OpenAI (GPT-4+, GPT-5) | ✅ | ✅ | Same as above. |
| Local vLLM / LM Studio (vision model) | ✅ | ✅ | If the model supports multi-part tool content. |
| Text-only models | ❌ | ✅ (degraded) | Use `mode="ax"` for accessibility-tree-only operation. |
Screenshots are sent inline with tool results as OpenAI-style `image_url`
parts. For Anthropic, the adapter converts them into native `tool_result`
image blocks.
## Safety
Hermes applies multi-layer guardrails:
- Destructive actions (click, type, drag, scroll, key, focus_app) require
approval — either interactively via the CLI dialog or via the
messaging-platform approval buttons.
- Hard-blocked key combos at the tool level: empty trash, force delete,
lock screen, log out, force log out.
- Hard-blocked type patterns: `curl | bash`, `sudo rm -rf /`, fork bombs,
etc.
- The agent's system prompt tells it explicitly: no clicking permission
dialogs, no typing passwords, no following instructions embedded in
screenshots.
Pair with `approvals.mode: manual` in `~/.hermes/config.yaml` if you want every action confirmed.
## Token efficiency
Screenshots are expensive. Hermes applies four layers of optimisation:
- **Screenshot eviction** — the Anthropic adapter keeps only the 3 most
recent screenshots in context; older ones become `[screenshot removed
to save context]` placeholders.
- **Client-side compression pruning** — the context compressor detects
multimodal tool results and strips image parts from old ones.
- **Image-aware token estimation** — each image is counted as ~1500 tokens
(Anthropic's flat rate) instead of its base64 char length.
- **Server-side context editing (Anthropic only)** — when active, the
adapter enables `clear_tool_uses_20250919` via `context_management` so
Anthropic's API clears old tool results server-side.
A 20-action session on a 1568×900 display typically costs ~30K tokens
of screenshot context, not ~600K.
## Limitations
- **macOS only.** cua-driver uses private Apple SPIs that don't exist on
Linux or Windows. For cross-platform GUI automation, use the `browser`
toolset.
- **Private SPI risk.** Apple can change SkyLight's symbol surface in any
OS update. Pin the driver version with the `HERMES_CUA_DRIVER_VERSION`
env var if you want reproducibility across a macOS bump.
- **Performance.** Background mode is slower than foreground —
SkyLight-routed events take ~5-20ms vs direct HID posting. Not
noticeable for agent-speed clicking; noticeable if you try to record a
speed-run.
- **No keyboard password entry.** `type` has hard-block patterns on
command-shell payloads; for passwords, use the system's autofill.
## Configuration
Override the driver binary path (tests / CI):
```
HERMES_CUA_DRIVER_CMD=/opt/homebrew/bin/cua-driver
HERMES_CUA_DRIVER_VERSION=0.5.0 # optional pin
```
Swap the backend entirely (for testing):
```
HERMES_COMPUTER_USE_BACKEND=noop # records calls, no side effects
```
## Troubleshooting
**`computer_use backend unavailable: cua-driver is not installed`** — Run
`hermes computer-use install` to fetch the cua-driver binary, or run
`hermes tools` and enable the Computer Use toolset.
**Clicks seem to have no effect** — Capture and verify. A modal you
didn't see may be blocking input. Dismiss it with `escape` or the close
button.
**Element indices are stale** — SOM indices are only valid until the
next `capture`. Re-capture after any state-changing action.
**"blocked pattern in type text"** — The text you tried to `type`
matches the dangerous-shell-pattern list. Break the command up or
reconsider.
## See also
- [Universal skill: `macos-computer-use`](https://github.com/NousResearch/hermes-agent/blob/main/skills/apple/macos-computer-use/SKILL.md)
- [cua-driver source (trycua/cua)](https://github.com/trycua/cua)
- [Browser automation](./browser.md) for cross-platform web tasks.
website/docs/user-guide/features/context-files.md
+218
View File
@@ -0,0 +1,218 @@
---
sidebar_position: 8
title: "Context Files"
description: "Project context files — .hermes.md, AGENTS.md, CLAUDE.md, global SOUL.md, and .cursorrules — automatically injected into every conversation"
---
# Context Files
Hermes Agent automatically discovers and loads context files that shape how it behaves. Some are project-local and discovered from your working directory. `SOUL.md` is now global to the Hermes instance and is loaded from `HERMES_HOME` only.
## Supported Context Files
| File | Purpose | Discovery |
|------|---------|-----------|
| **.hermes.md** / **HERMES.md** | Project instructions (highest priority) | Walks to git root |
| **AGENTS.md** | Project instructions, conventions, architecture | CWD at startup + subdirectories progressively |
| **CLAUDE.md** | Claude Code context files (also detected) | CWD at startup + subdirectories progressively |
| **SOUL.md** | Global personality and tone customization for this Hermes instance | `HERMES_HOME/SOUL.md` only |
| **.cursorrules** | Cursor IDE coding conventions | CWD only |
| **.cursor/rules/*.mdc** | Cursor IDE rule modules | CWD only |
:::info Priority system
Only **one** project context type is loaded per session (first match wins): `.hermes.md` → `AGENTS.md` → `CLAUDE.md` → `.cursorrules`. **SOUL.md** is always loaded independently as the agent identity (slot #1).
:::
## AGENTS.md
`AGENTS.md` is the primary project context file. It tells the agent how your project is structured, what conventions to follow, and any special instructions.
### Progressive Subdirectory Discovery
At session start, Hermes loads the `AGENTS.md` from your working directory into the system prompt. As the agent navigates into subdirectories during the session (via `read_file`, `terminal`, `search_files`, etc.), it **progressively discovers** context files in those directories and injects them into the conversation at the moment they become relevant.
```
my-project/
├── AGENTS.md ← Loaded at startup (system prompt)
├── frontend/
│ └── AGENTS.md ← Discovered when agent reads frontend/ files
├── backend/
│ └── AGENTS.md ← Discovered when agent reads backend/ files
└── shared/
└── AGENTS.md ← Discovered when agent reads shared/ files
```
This approach has two advantages over loading everything at startup:
- **No system prompt bloat** — subdirectory hints only appear when needed
- **Prompt cache preservation** — the system prompt stays stable across turns
Each subdirectory is checked at most once per session. The discovery also walks up parent directories, so reading `backend/src/main.py` will discover `backend/AGENTS.md` even if `backend/src/` has no context file of its own.
:::info
Subdirectory context files go through the same [security scan](#security-prompt-injection-protection) as startup context files. Malicious files are blocked.
:::
### Example AGENTS.md
```markdown
# Project Context
This is a Next.js 14 web application with a Python FastAPI backend.
## Architecture
- Frontend: Next.js 14 with App Router in `/frontend`
- Backend: FastAPI in `/backend`, uses SQLAlchemy ORM
- Database: PostgreSQL 16
- Deployment: Docker Compose on a Hetzner VPS
## Conventions
- Use TypeScript strict mode for all frontend code
- Python code follows PEP 8, use type hints everywhere
- All API endpoints return JSON with `{data, error, meta}` shape
- Tests go in `__tests__/` directories (frontend) or `tests/` (backend)
## Important Notes
- Never modify migration files directly — use Alembic commands
- The `.env.local` file has real API keys, don't commit it
- Frontend port is 3000, backend is 8000, DB is 5432
```
## SOUL.md
`SOUL.md` controls the agent's personality, tone, and communication style. See the [Personality](/user-guide/features/personality) page for full details.
**Location:**
- `~/.hermes/SOUL.md`
- or `$HERMES_HOME/SOUL.md` if you run Hermes with a custom home directory
Important details:
- Hermes seeds a default `SOUL.md` automatically if one does not exist yet
- Hermes loads `SOUL.md` only from `HERMES_HOME`
- Hermes does not probe the working directory for `SOUL.md`
- If the file is empty, nothing from `SOUL.md` is added to the prompt
- If the file has content, the content is injected verbatim after scanning and truncation
## .cursorrules
Hermes is compatible with Cursor IDE's `.cursorrules` file and `.cursor/rules/*.mdc` rule modules. If these files exist in your project root and no higher-priority context file (`.hermes.md`, `AGENTS.md`, or `CLAUDE.md`) is found, they're loaded as the project context.
This means your existing Cursor conventions automatically apply when using Hermes.
## How Context Files Are Loaded
### At startup (system prompt)
Context files are loaded by `build_context_files_prompt()` in `agent/prompt_builder.py`:
1. **Scan working directory** — checks for `.hermes.md` → `AGENTS.md` → `CLAUDE.md` → `.cursorrules` (first match wins)
2. **Content is read** — each file is read as UTF-8 text
3. **Security scan** — content is checked for prompt injection patterns
4. **Truncation** — files exceeding 20,000 characters are head/tail truncated (70% head, 20% tail, with a marker in the middle)
5. **Assembly** — all sections are combined under a `# Project Context` header
6. **Injection** — the assembled content is added to the system prompt
### During the session (progressive discovery)
`SubdirectoryHintTracker` in `agent/subdirectory_hints.py` watches tool call arguments for file paths:
1. **Path extraction** — after each tool call, file paths are extracted from arguments (`path`, `workdir`, shell commands)
2. **Ancestor walk** — the directory and up to 5 parent directories are checked (stopping at already-visited directories)
3. **Hint loading** — if an `AGENTS.md`, `CLAUDE.md`, or `.cursorrules` is found, it's loaded (first match per directory)
4. **Security scan** — same prompt injection scan as startup files
5. **Truncation** — capped at 8,000 characters per file
6. **Injection** — appended to the tool result, so the model sees it in context naturally
The final prompt section looks roughly like:
```text
# Project Context
The following project context files have been loaded and should be followed:
## AGENTS.md
[Your AGENTS.md content here]
## .cursorrules
[Your .cursorrules content here]
[Your SOUL.md content here]
```
Notice that SOUL content is inserted directly, without extra wrapper text.
## Security: Prompt Injection Protection
All context files are scanned for potential prompt injection before being included. The scanner checks for:
- **Instruction override attempts**: "ignore previous instructions", "disregard your rules"
- **Deception patterns**: "do not tell the user"
- **System prompt overrides**: "system prompt override"
- **Hidden HTML comments**: `<!-- ignore instructions -->`
- **Hidden div elements**: `<div style="display:none">`
- **Credential exfiltration**: `curl ... $API_KEY`
- **Secret file access**: `cat .env`, `cat credentials`
- **Invisible characters**: zero-width spaces, bidirectional overrides, word joiners
If any threat pattern is detected, the file is blocked:
```
[BLOCKED: AGENTS.md contained potential prompt injection (prompt_injection). Content not loaded.]
```
:::warning
This scanner protects against common injection patterns, but it's not a substitute for reviewing context files in shared repositories. Always validate AGENTS.md content in projects you didn't author.
:::
## Size Limits
| Limit | Value |
|-------|-------|
| Max chars per file | 20,000 (~7,000 tokens) |
| Head truncation ratio | 70% |
| Tail truncation ratio | 20% |
| Truncation marker | 10% (shows char counts and suggests using file tools) |
When a file exceeds 20,000 characters, the truncation message reads:
```
[...truncated AGENTS.md: kept 14000+4000 of 25000 chars. Use file tools to read the full file.]
```
## Tips for Effective Context Files
:::tip Best practices for AGENTS.md
1. **Keep it concise** — stay well under 20K chars; the agent reads it every turn
2. **Structure with headers** — use `##` sections for architecture, conventions, important notes
3. **Include concrete examples** — show preferred code patterns, API shapes, naming conventions
4. **Mention what NOT to do** — "never modify migration files directly"
5. **List key paths and ports** — the agent uses these for terminal commands
6. **Update as the project evolves** — stale context is worse than no context
:::
### Per-Subdirectory Context
For monorepos, put subdirectory-specific instructions in nested AGENTS.md files:
```markdown
<!-- frontend/AGENTS.md -->
# Frontend Context
- Use `pnpm` not `npm` for package management
- Components go in `src/components/`, pages in `src/app/`
- Use Tailwind CSS, never inline styles
- Run tests with `pnpm test`
```
```markdown
<!-- backend/AGENTS.md -->
# Backend Context
- Use `poetry` for dependency management
- Run the dev server with `poetry run uvicorn main:app --reload`
- All endpoints need OpenAPI docstrings
- Database models are in `models/`, schemas in `schemas/`
```
website/docs/user-guide/features/context-references.md
+142
View File
@@ -0,0 +1,142 @@
---
sidebar_position: 9
sidebar_label: "Context References"
title: "Context References"
description: "Inline @-syntax for attaching files, folders, git diffs, and URLs directly into your messages"
---
# Context References
Type `@` followed by a reference to inject content directly into your message. Hermes expands the reference inline and appends the content under an `--- Attached Context ---` section.
## Supported References
| Syntax | Description |
|--------|-------------|
| `@file:path/to/file.py` | Inject file contents |
| `@file:path/to/file.py:10-25` | Inject specific line range (1-indexed, inclusive) |
| `@folder:path/to/dir` | Inject directory tree listing with file metadata |
| `@diff` | Inject `git diff` (unstaged working tree changes) |
| `@staged` | Inject `git diff --staged` (staged changes) |
| `@git:5` | Inject last N commits with patches (max 10) |
| `@url:https://example.com` | Fetch and inject web page content |
## Usage Examples
```text
Review @file:src/main.py and suggest improvements
What changed? @diff
Compare @file:old_config.yaml and @file:new_config.yaml
What's in @folder:src/components?
Summarize this article @url:https://arxiv.org/abs/2301.00001
```
Multiple references work in a single message:
```text
Check @file:main.py, and also @file:test.py.
```
Trailing punctuation (`,`, `.`, `;`, `!`, `?`) is automatically stripped from reference values.
## CLI Tab Completion
In the interactive CLI, typing `@` triggers autocomplete:
- `@` shows all reference types (`@diff`, `@staged`, `@file:`, `@folder:`, `@git:`, `@url:`)
- `@file:` and `@folder:` trigger filesystem path completion with file size metadata
- Bare `@` followed by partial text shows matching files and folders from the current directory
## Line Ranges
The `@file:` reference supports line ranges for precise content injection:
```text
@file:src/main.py:42 # Single line 42
@file:src/main.py:10-25 # Lines 10 through 25 (inclusive)
```
Lines are 1-indexed. Invalid ranges are silently ignored (full file is returned).
## Size Limits
Context references are bounded to prevent overwhelming the model's context window:
| Threshold | Value | Behavior |
|-----------|-------|----------|
| Soft limit | 25% of context length | Warning appended, expansion proceeds |
| Hard limit | 50% of context length | Expansion refused, original message returned unchanged |
| Folder entries | 200 files max | Excess entries replaced with `- ...` |
| Git commits | 10 max | `@git:N` clamped to range [1, 10] |
## Security
### Sensitive Path Blocking
These paths are always blocked from `@file:` references to prevent credential exposure:
- SSH keys and config: `~/.ssh/id_rsa`, `~/.ssh/id_ed25519`, `~/.ssh/authorized_keys`, `~/.ssh/config`
- Shell profiles: `~/.bashrc`, `~/.zshrc`, `~/.profile`, `~/.bash_profile`, `~/.zprofile`
- Credential files: `~/.netrc`, `~/.pgpass`, `~/.npmrc`, `~/.pypirc`
- Hermes env: `$HERMES_HOME/.env`
These directories are fully blocked (any file inside):
- `~/.ssh/`, `~/.aws/`, `~/.gnupg/`, `~/.kube/`, `$HERMES_HOME/skills/.hub/`
### Path Traversal Protection
All paths are resolved relative to the working directory. References that resolve outside the allowed workspace root are rejected.
### Binary File Detection
Binary files are detected via MIME type and null-byte scanning. Known text extensions (`.py`, `.md`, `.json`, `.yaml`, `.toml`, `.js`, `.ts`, etc.) bypass MIME-based detection. Binary files are rejected with a warning.
## Platform Availability
Context references are primarily a **CLI feature**. They work in the interactive CLI where `@` triggers tab completion and references are expanded before the message is sent to the agent.
In **messaging platforms** (Telegram, Discord, etc.), the `@` syntax is not expanded by the gateway — messages are passed through as-is. The agent itself can still reference files via the `read_file`, `search_files`, and `web_extract` tools.
## Interaction with Context Compression
When conversation context is compressed, the expanded reference content is included in the compression summary. This means:
- Large file contents injected via `@file:` contribute to context usage
- If the conversation is later compressed, the file content is summarized (not preserved verbatim)
- For very large files, consider using line ranges (`@file:main.py:100-200`) to inject only relevant sections
## Common Patterns
```text
# Code review workflow
Review @diff and check for security issues
# Debug with context
This test is failing. Here's the test @file:tests/test_auth.py
and the implementation @file:src/auth.py:50-80
# Project exploration
What does this project do? @folder:src @file:README.md
# Research
Compare the approaches in @url:https://arxiv.org/abs/2301.00001
and @url:https://arxiv.org/abs/2301.00002
```
## Error Handling
Invalid references produce inline warnings rather than failures:
| Condition | Behavior |
|-----------|----------|
| File not found | Warning: "file not found" |
| Binary file | Warning: "binary files are not supported" |
| Folder not found | Warning: "folder not found" |
| Git command fails | Warning with git stderr |
| URL returns no content | Warning: "no content extracted" |
| Sensitive path | Warning: "path is a sensitive credential file" |
| Path outside workspace | Warning: "path is outside the allowed workspace" |
website/docs/user-guide/features/credential-pools.md
+259
View File
@@ -0,0 +1,259 @@
---
title: Credential Pools
description: Pool multiple API keys or OAuth tokens per provider for automatic rotation and rate limit recovery.
sidebar_label: Credential Pools
sidebar_position: 9
---
# Credential Pools
Credential pools let you register multiple API keys or OAuth tokens for the same provider. When one key hits a rate limit or billing quota, Hermes automatically rotates to the next healthy key — keeping your session alive without switching providers.
This is different from [fallback providers](./fallback-providers.md), which switch to a *different* provider entirely. Credential pools are same-provider rotation; fallback providers are cross-provider failover. Pools are tried first — if all pool keys are exhausted, *then* the fallback provider activates.
:::tip
Credential pools are mainly for API-key providers (OpenRouter, Anthropic). A single [Nous Portal](/integrations/nous-portal) OAuth covers 300+ models, so most users don't need a pool when on Portal.
:::
## How It Works
```
Your request
→ Pick key from pool (round_robin / least_used / fill_first / random)
→ Send to provider
→ 429 rate limit?
→ Plan/usage limit reached (e.g. ChatGPT/Codex "usage limit reached")?
→ Rotate to next pool key immediately (no retry — the cap won't clear on retry)
→ Generic / transient 429?
→ Retry same key once (transient blip)
→ Second 429 → rotate to next pool key
→ All keys exhausted → fallback_model (different provider)
→ 402 billing error?
→ Immediately rotate to next pool key (24h cooldown)
→ 401 auth expired?
→ Try refreshing the token (OAuth)
→ Refresh failed → rotate to next pool key
→ Success → continue normally
```
## Quick Start
If you already have an API key set in `.env`, Hermes auto-discovers it as a 1-key pool. To benefit from pooling, add more keys:
```bash
# Add a second OpenRouter key
hermes auth add openrouter --api-key sk-or-v1-your-second-key
# Add a second Anthropic key
hermes auth add anthropic --type api-key --api-key sk-ant-api03-your-second-key
# Add an Anthropic OAuth credential (requires Claude Max plan + extra usage credits)
hermes auth add anthropic --type oauth
# Opens browser for OAuth login
```
Check your pools:
```bash
hermes auth list
```
Output:
```
openrouter (2 credentials):
#1 OPENROUTER_API_KEY api_key env:OPENROUTER_API_KEY ←
#2 backup-key api_key manual
anthropic (3 credentials):
#1 hermes_pkce oauth hermes_pkce ←
#2 claude_code oauth claude_code
#3 ANTHROPIC_API_KEY api_key env:ANTHROPIC_API_KEY
```
The `←` marks the currently selected credential.
## Interactive Management
Run `hermes auth` with no subcommand for an interactive wizard:
```bash
hermes auth
```
This shows your full pool status and offers a menu:
```
What would you like to do?
1. Add a credential
2. Remove a credential
3. Reset cooldowns for a provider
4. Set rotation strategy for a provider
5. Exit
```
For providers that support both API keys and OAuth (Anthropic, Nous, Codex), the add flow asks which type:
```
anthropic supports both API keys and OAuth login.
1. API key (paste a key from the provider dashboard)
2. OAuth login (authenticate via browser)
Type [1/2]:
```
## CLI Commands
| Command | Description |
|---------|-------------|
| `hermes auth` | Interactive pool management wizard |
| `hermes auth list` | Show all pools and credentials |
| `hermes auth list <provider>` | Show a specific provider's pool |
| `hermes auth add <provider>` | Add a credential (prompts for type and key) |
| `hermes auth add <provider> --type api-key --api-key <key>` | Add an API key non-interactively |
| `hermes auth add <provider> --type oauth` | Add an OAuth credential via browser login |
| `hermes auth remove <provider> <index>` | Remove credential by 1-based index |
| `hermes auth reset <provider>` | Clear all cooldowns/exhaustion status |
## Rotation Strategies
Configure via `hermes auth` → "Set rotation strategy" or in `config.yaml`:
```yaml
credential_pool_strategies:
openrouter: round_robin
anthropic: least_used
```
| Strategy | Behavior |
|----------|----------|
| `fill_first` (default) | Use the first healthy key until it's exhausted, then move to the next |
| `round_robin` | Cycle through keys evenly, rotating after each selection |
| `least_used` | Always pick the key with the lowest request count |
| `random` | Random selection among healthy keys |
## Error Recovery
The pool handles different errors differently:
| Error | Behavior | Cooldown |
|-------|----------|----------|
| **429 Rate Limit** | Retry same key once (transient). Second consecutive 429 rotates to next key | 1 hour |
| **402 Billing/Quota** | Immediately rotate to next key | 24 hours |
| **401 Auth Expired** | Try refreshing the OAuth token first. Rotate only if refresh fails | — |
| **All keys exhausted** | Fall through to `fallback_model` if configured | — |
The `has_retried_429` flag resets on every successful API call, so a single transient 429 doesn't trigger rotation.
## Custom Endpoint Pools
Custom OpenAI-compatible endpoints (Together.ai, RunPod, local servers) get their own pools, keyed by the endpoint name from `custom_providers` in config.yaml.
When you set up a custom endpoint via `hermes model`, it auto-generates a name like "Together.ai" or "Local (localhost:8080)". This name becomes the pool key.
```bash
# After setting up a custom endpoint via hermes model:
hermes auth list
# Shows:
# Together.ai (1 credential):
# #1 config key api_key config:Together.ai ←
# Add a second key for the same endpoint:
hermes auth add Together.ai --api-key sk-together-second-key
```
Custom endpoint pools are stored in `auth.json` under `credential_pool` with a `custom:` prefix:
```json
{
"credential_pool": {
"openrouter": [...],
"custom:together.ai": [...]
}
}
```
## Auto-Discovery
Hermes automatically discovers credentials from multiple sources and seeds the pool on startup:
| Source | Example | Auto-seeded? |
|--------|---------|-------------|
| Environment variables | `OPENROUTER_API_KEY`, `ANTHROPIC_API_KEY` | Yes |
| OAuth tokens (auth.json) | Codex device code, Nous device code | Yes |
| Claude Code credentials | `~/.claude/.credentials.json` | Yes (Anthropic) |
| Hermes PKCE OAuth | `~/.hermes/auth.json` | Yes (Anthropic) |
| Custom endpoint config | `model.api_key` in config.yaml | Yes (custom endpoints) |
| Manual entries | Added via `hermes auth add` | Persisted in auth.json |
Auto-seeded entries are updated on each pool load — if you remove an env var, its pool entry is automatically pruned. Manual entries (added via `hermes auth add`) are never auto-pruned.
Borrowed runtime secrets (for example env vars, Bitwarden/Vault/keyring/systemd references, and custom config values) are reference-only at the `auth.json` boundary. Hermes can use the resolved value in memory for the current run, but it persists only metadata such as the source ref, label, status, request counters, and a non-reversible fingerprint. Manual entries and Hermes-owned OAuth/device-code state keep the durable tokens they need to refresh.
## Delegation & Subagent Sharing
When the agent spawns subagents via `delegate_task`, the parent's credential pool is automatically shared with children:
- **Same provider** — the child receives the parent's full pool, enabling key rotation on rate limits
- **Different provider** — the child loads that provider's own pool (if configured)
- **No pool configured** — the child falls back to the inherited single API key
This means subagents benefit from the same rate-limit resilience as the parent, with no extra configuration needed. Per-task credential leasing ensures children don't conflict with each other when rotating keys concurrently.
## Thread Safety
The credential pool uses a threading lock for all state mutations (`select()`, `mark_exhausted_and_rotate()`, `try_refresh_current()`, `mark_used()`). This ensures safe concurrent access when the gateway handles multiple chat sessions simultaneously.
## Architecture
For the full data flow diagram, see [`docs/credential-pool-flow.excalidraw`](https://excalidraw.com/#json=2Ycqhqpi6f12E_3ITyiwh,c7u9jSt5BwrmiVzHGbm87g) in the repository.
The credential pool integrates at the provider resolution layer:
1. **`agent/credential_pool.py`** — Pool manager: storage, selection, rotation, cooldowns
2. **`hermes_cli/auth_commands.py`** — CLI commands and interactive wizard
3. **`hermes_cli/runtime_provider.py`** — Pool-aware credential resolution
4. **`run_agent.py`** — Error recovery: 429/402/401 → pool rotation → fallback
## Storage
Pool state is stored in `~/.hermes/auth.json` under the `credential_pool` key:
```json
{
"version": 1,
"credential_pool": {
"openrouter": [
{
"id": "abc123",
"label": "OPENROUTER_API_KEY",
"auth_type": "api_key",
"priority": 0,
"source": "env:OPENROUTER_API_KEY",
"secret_source": "bitwarden",
"secret_fingerprint": "sha256:12ab34cd56ef7890",
"last_status": "ok",
"request_count": 142
}
],
"anthropic": [
{
"id": "manual1",
"label": "personal-api-key",
"auth_type": "api_key",
"priority": 0,
"source": "manual",
"access_token": "sk-ant-api03-..."
}
]
}
}
```
The OpenRouter entry above was borrowed from an external source, so the raw key is not stored in `auth.json`. The manual Anthropic entry was intentionally added to Hermes' credential store, so its token remains persistable.
Strategies are stored in `config.yaml` (not `auth.json`):
```yaml
credential_pool_strategies:
openrouter: round_robin
anthropic: least_used
```
website/docs/user-guide/features/cron.md
+661
View File
@@ -0,0 +1,661 @@
---
sidebar_position: 5
title: "Scheduled Tasks (Cron)"
description: "Schedule automated tasks with natural language, manage them with one cron tool, and attach one or more skills"
---
# Scheduled Tasks (Cron)
Schedule tasks to run automatically with natural language or cron expressions. Hermes exposes cron management through a single `cronjob` tool with action-style operations instead of separate schedule/list/remove tools.
## What cron can do now
Cron jobs can:
- schedule one-shot or recurring tasks
- pause, resume, edit, trigger, and remove jobs
- attach zero, one, or multiple skills to a job
- deliver results back to the origin chat, local files, or configured platform targets
- run in fresh agent sessions with the normal static tool list
- run in **no-agent mode** — a script on a schedule, its stdout delivered verbatim, zero LLM involvement (see the [no-agent mode](#no-agent-mode-script-only-jobs) section below)
All of this is available to Hermes itself through the `cronjob` tool, so you can create, pause, edit, and remove jobs by asking in plain language — no CLI required.
:::tip
Cron jobs use whatever provider `hermes model` selected. `hermes setup --portal` is the lowest-friction option for unattended runs since OAuth refresh is automatic. See [Nous Portal](/integrations/nous-portal).
:::
:::warning
Cron-run sessions cannot recursively create more cron jobs. Hermes disables cron management tools inside cron executions to prevent runaway scheduling loops.
:::
## Creating scheduled tasks
### In chat with `/cron`
```bash
/cron add 30m "Remind me to check the build"
/cron add "every 2h" "Check server status"
/cron add "every 1h" "Summarize new feed items" --skill blogwatcher
/cron add "every 1h" "Use both skills and combine the result" --skill blogwatcher --skill maps
```
### From the standalone CLI
```bash
hermes cron create "every 2h" "Check server status"
hermes cron create "every 1h" "Summarize new feed items" --skill blogwatcher
hermes cron create "every 1h" "Use both skills and combine the result" \
--skill blogwatcher \
--skill maps \
--name "Skill combo"
```
### Through natural conversation
Ask Hermes normally:
```text
Every morning at 9am, check Hacker News for AI news and send me a summary on Telegram.
```
Hermes will use the unified `cronjob` tool internally.
## Skill-backed cron jobs
A cron job can load one or more skills before it runs the prompt.
### Single skill
```python
cronjob(
action="create",
skill="blogwatcher",
prompt="Check the configured feeds and summarize anything new.",
schedule="0 9 * * *",
name="Morning feeds",
)
```
### Multiple skills
Skills are loaded in order. The prompt becomes the task instruction layered on top of those skills.
```python
cronjob(
action="create",
skills=["blogwatcher", "maps"],
prompt="Look for new local events and interesting nearby places, then combine them into one short brief.",
schedule="every 6h",
name="Local brief",
)
```
This is useful when you want a scheduled agent to inherit reusable workflows without stuffing the full skill text into the cron prompt itself.
## Running a job inside a project directory
Cron jobs default to running detached from any repo — no `AGENTS.md`, `CLAUDE.md`, or `.cursorrules` is loaded, and the terminal / file / code-exec tools run from whatever working directory the gateway started in. Pass `--workdir` (CLI) or `workdir=` (tool call) to change that:
```bash
# Standalone CLI (schedule and prompt are positional)
hermes cron create "every 1d at 09:00" \
"Audit open PRs, summarize CI health, and post to #eng" \
--workdir /home/me/projects/acme
```
```python
# From a chat, via the cronjob tool
cronjob(
action="create",
schedule="every 1d at 09:00",
workdir="/home/me/projects/acme",
prompt="Audit open PRs, summarize CI health, and post to #eng",
)
```
When `workdir` is set:
- `AGENTS.md`, `CLAUDE.md`, and `.cursorrules` from that directory are injected into the system prompt (same discovery order as the interactive CLI)
- `terminal`, `read_file`, `write_file`, `patch`, `search_files`, and `execute_code` all use that directory as their working directory
- The path must be an absolute directory that exists — relative paths and missing directories are rejected at create / update time
- Pass `--workdir ""` (or `workdir=""` via the tool) on edit to clear it and restore the old behaviour
:::note Serialization
Jobs with a `workdir` run sequentially on the scheduler tick, not in the parallel pool. This is deliberate: the cron worker applies the job workdir through process-global terminal state, so two workdir jobs running at the same time would corrupt each other's cwd. Workdir-less jobs still run in parallel as before.
:::
## Editing jobs
You do not need to delete and recreate jobs just to change them.
:::tip Job reference
The `<job_id>` placeholder below (and in [Lifecycle actions](#lifecycle-actions)) also accepts the job's name (case-insensitive) — handy when you remember `morning-digest` but not the hex ID. An exact job ID takes precedence over name matches; if the reference is not an ID and a name matches more than one job, the command refuses and prints the candidate IDs so you can disambiguate.
:::
### Chat
```bash
/cron edit <job_id> --schedule "every 4h"
/cron edit <job_id> --prompt "Use the revised task"
/cron edit <job_id> --skill blogwatcher --skill maps
/cron edit <job_id> --remove-skill blogwatcher
/cron edit <job_id> --clear-skills
```
### Standalone CLI
```bash
hermes cron edit <job_id> --schedule "every 4h"
hermes cron edit <job_id> --prompt "Use the revised task"
hermes cron edit <job_id> --skill blogwatcher --skill maps
hermes cron edit <job_id> --add-skill maps
hermes cron edit <job_id> --remove-skill blogwatcher
hermes cron edit <job_id> --clear-skills
```
Notes:
- repeated `--skill` replaces the job's attached skill list
- `--add-skill` appends to the existing list without replacing it
- `--remove-skill` removes specific attached skills
- `--clear-skills` removes all attached skills
## Lifecycle actions
Cron jobs now have a fuller lifecycle than just create/remove.
### Chat
```bash
/cron list
/cron pause <job_id>
/cron resume <job_id>
/cron run <job_id>
/cron remove <job_id>
```
### Standalone CLI
```bash
hermes cron list
hermes cron pause <job_id_or_name>
hermes cron resume <job_id_or_name>
hermes cron run <job_id_or_name>
hermes cron remove <job_id_or_name>
hermes cron edit <job_id_or_name> [...flags]
hermes cron status
hermes cron tick
```
What they do:
- `pause` — keep the job but stop scheduling it
- `resume` — re-enable the job and compute the next future run
- `run` — trigger the job on the next scheduler tick
- `remove` — delete it entirely
- `edit` — modify schedule, prompt, delivery, etc.
**Name-based lookup.** All four mutating verbs (`pause`, `resume`, `run`, `remove`, `edit`) plus the agent's `cronjob` tool now accept a job **name** (case-insensitive) in place of the hex ID. The agent and CLI both prefer an exact ID match if one exists; ambiguous name matches (multiple jobs sharing the same name) are refused with the full list of candidate IDs so you can pick one explicitly. Names are not unique, so this guard is load-bearing — it prevents silently mutating the wrong job when two share a name.
## How it works
**Cron execution is handled by the gateway daemon.** The gateway ticks the scheduler every 60 seconds, running any due jobs in isolated agent sessions.
```bash
hermes gateway install # Install as a user service
sudo hermes gateway install --system # Linux: boot-time system service for servers
hermes gateway # Or run in foreground
hermes cron list
hermes cron status
```
### Gateway scheduler behavior
On each tick Hermes:
1. loads jobs from `~/.hermes/cron/jobs.json`
2. checks `next_run_at` against the current time
3. starts a fresh `AIAgent` session for each due job
4. optionally injects one or more attached skills into that fresh session
5. runs the prompt to completion
6. delivers the final response
7. updates run metadata and the next scheduled time
A file lock at `~/.hermes/cron/.tick.lock` prevents overlapping scheduler ticks from double-running the same job batch.
## Delivery options
When scheduling jobs, you specify where the output goes:
| Option | Description | Example |
|--------|-------------|---------|
| `"origin"` | Back to where the job was created | Default on messaging platforms |
| `"local"` | Save to local files only (`~/.hermes/cron/output/`) | Default on CLI |
| `"telegram"` | Telegram home channel | Uses `TELEGRAM_HOME_CHANNEL` |
| `"telegram:123456"` | Specific Telegram chat by ID | Direct delivery |
| `"telegram:-100123:17585"` | Specific Telegram topic | `chat_id:thread_id` format |
| `"discord"` | Discord home channel | Uses `DISCORD_HOME_CHANNEL` |
| `"discord:#engineering"` | Specific Discord channel | By channel name |
| `"slack"` | Slack home channel | |
| `"whatsapp"` | WhatsApp home | |
| `"signal"` | Signal | |
| `"matrix"` | Matrix home room | |
| `"mattermost"` | Mattermost home channel | |
| `"email"` | Email | |
| `"sms"` | SMS via Twilio | |
| `"homeassistant"` | Home Assistant | |
| `"dingtalk"` | DingTalk | |
| `"feishu"` | Feishu/Lark | |
| `"wecom"` | WeCom | |
| `"weixin"` | Weixin (WeChat) | |
| `"bluebubbles"` | BlueBubbles (iMessage) | |
| `"qqbot"` | QQ Bot (Tencent QQ) | |
| `"all"` | Fan out to every connected home channel | Resolved at fire time |
| `"telegram,discord"` | Fan out to a specific set of channels | Comma-separated list |
| `"origin,all"` | Deliver to the origin **plus** every other connected channel | Combine any tokens |
The agent's final response is automatically delivered. You do not need to call `send_message` in the cron prompt.
### Routing intent (`all`)
`all` lets you ship one cron job to every messaging channel you have configured, without having to enumerate them by name. It is **resolved at fire time**, so a job created before you wired up Telegram will pick up Telegram on the next tick after you set `TELEGRAM_HOME_CHANNEL`.
Semantics: `all` expands to every platform with a configured home channel. Zero is fine; the job simply produces no delivery targets and is recorded as a delivery failure upstream.
`all` composes with explicit targets. `origin,all` delivers to the origin chat *plus* every other connected home channel, de-duplicating by `(platform, chat_id, thread_id)`.
### Telegram cron topic (`TELEGRAM_CRON_THREAD_ID`)
When Telegram topic mode is enabled, the root DM is reserved as a system lobby — replies sent there are rebuffed with a lobby reminder and `reply_to_message_id` is dropped, so you cannot reply to a cron message that landed in the main chat.
Point cron at a dedicated forum topic instead:
1. In Telegram, open the bot DM and create a topic named e.g. `Cron`. Long-press the topic header → **Copy link**; the trailing integer is the topic's `message_thread_id`.
2. Set `TELEGRAM_CRON_THREAD_ID=<that id>` in your `.env`.
This applies only to cron deliveries. `TELEGRAM_HOME_CHANNEL_THREAD_ID` (used elsewhere, e.g. restart notifications) is unchanged. Explicit `deliver="telegram:chat_id:thread_id"` targets continue to win over the env var. Replies to cron messages now arrive in the existing topic session, so you can act on them directly.
### Response wrapping
By default, delivered cron output is wrapped with a header and footer so the recipient knows it came from a scheduled task:
```
Cronjob Response: Morning feeds
-------------
<agent output here>
Note: The agent cannot see this message, and therefore cannot respond to it.
```
To deliver the raw agent output without the wrapper, set `cron.wrap_response` to `false`:
```yaml
# ~/.hermes/config.yaml
cron:
wrap_response: false
```
### Silent suppression
If the agent's final response contains `[SILENT]`, delivery is suppressed entirely. The output is still saved locally for audit (in `~/.hermes/cron/output/`), but no message is sent to the delivery target.
This is useful for monitoring jobs that should only report when something is wrong:
```text
Check if nginx is running. If everything is healthy, respond with only [SILENT].
Otherwise, report the issue.
```
Failed jobs always deliver regardless of the `[SILENT]` marker — only successful runs can be silenced. For quiet monitoring jobs, prompt the agent to reply with only `[SILENT]` when there is nothing to report.
## Script timeout
Pre-run scripts (attached via the `script` parameter) have a default timeout of 120 seconds. If your scripts need longer — for example, to include randomized delays that avoid bot-like timing patterns — you can increase this:
```yaml
# ~/.hermes/config.yaml
cron:
script_timeout_seconds: 300 # 5 minutes
```
Or set the `HERMES_CRON_SCRIPT_TIMEOUT` environment variable. The resolution order is: env var → config.yaml → 120s default.
## No-agent mode (script-only jobs)
For recurring jobs that don't need LLM reasoning — classic watchdogs, disk/memory alerts, heartbeats, CI pings — pass `no_agent=True` at creation time. The scheduler runs your script on schedule and delivers its stdout directly, skipping the agent entirely:
```bash
hermes cron create "every 5m" \
--no-agent \
--script memory-watchdog.sh \
--deliver telegram \
--name "memory-watchdog"
```
Semantics:
- Script stdout (trimmed) → delivered verbatim as the message.
- **Empty stdout → silent tick**, no delivery. This is the watchdog pattern: "only say something when something is wrong".
- Non-zero exit or timeout → an error alert is delivered, so a broken watchdog can't fail silently.
- `{"wakeAgent": false}` on the last line → silent tick (same gate LLM jobs use).
- No tokens, no model, no provider fallback — the job never touches the inference layer.
`.sh` / `.bash` files run under `/bin/bash`; anything else under the current Python interpreter (`sys.executable`). Scripts must live in `~/.hermes/scripts/` (same sandboxing rule as the pre-run script gate).
### The agent sets these up for you
The `cronjob` tool's schema exposes `no_agent` to Hermes directly, so you can describe a watchdog in chat and let the agent wire it up:
```text
Ping me on Telegram if RAM is over 85%, every 5 minutes.
```
Hermes will write the check script to `~/.hermes/scripts/` via `write_file`, then call:
```python
cronjob(action="create", schedule="every 5m",
script="memory-watchdog.sh", no_agent=True,
deliver="telegram", name="memory-watchdog")
```
It picks `no_agent=True` automatically when the message content is fully determined by the script (watchdogs, threshold alerts, heartbeats). The same tool also lets the agent pause, resume, edit, and remove jobs — so the whole lifecycle is chat-driven without anyone touching the CLI.
See the [Script-Only Cron Jobs guide](/guides/cron-script-only) for worked examples.
## Chaining jobs with `context_from`
Cron jobs run in isolated sessions with no memory of previous runs. But sometimes one job's output is exactly what the next job needs. The `context_from` parameter wires that connection automatically — Job B's prompt gets Job A's most recent output prepended as context at runtime.
```python
# Job 1: Collect raw data
cronjob(
action="create",
prompt="Fetch the top 10 AI/ML stories from Hacker News. Save them to ~/.hermes/data/briefs/raw.md in markdown format with title, URL, and score.",
schedule="0 7 * * *",
name="AI News Collector",
)
# Job 2: Triage — receives Job 1's output as context
# Get Job 1's ID from: cronjob(action="list")
cronjob(
action="create",
prompt="Read ~/.hermes/data/briefs/raw.md. Score each story 1–10 for engagement potential and novelty. Output the top 5 to ~/.hermes/data/briefs/ranked.md.",
schedule="30 7 * * *",
context_from="<job1_id>",
name="AI News Triage",
)
# Job 3: Ship — receives Job 2's output as context
cronjob(
action="create",
prompt="Read ~/.hermes/data/briefs/ranked.md. Write 3 tweet drafts (hook + body + hashtags). Deliver to telegram:7976161601.",
schedule="0 8 * * *",
context_from="<job2_id>",
name="AI News Brief",
)
```
**How it works:**
- When Job 2 fires, Hermes reads Job 1's most recent output from `~/.hermes/cron/output/{job1_id}/*.md`
- That output is prepended to Job 2's prompt automatically
- Job 2 doesn't need to hardcode "read this file" — it receives the content as context
- The chain can be any length: Job 1 → Job 2 → Job 3 → ...
**What `context_from` accepts:**
| Format | Example |
|--------|---------|
| Single job ID (string) | `context_from="a1b2c3d4"` |
| Multiple job IDs (list) | `context_from=["job_a", "job_b"]` |
Outputs are concatenated in the order listed.
**When to use it:**
- Multi-stage pipelines (collect → filter → format → deliver)
- Dependent tasks where step N's work depends on step N−1's output
- Fan-out/fan-in patterns where one job aggregates results from several others
## Provider recovery
Cron jobs inherit your configured fallback providers and credential pool rotation. If the primary API key is rate-limited or the provider returns an error, the cron agent can:
- **Fall back to an alternate provider** if you have `fallback_providers` (or the legacy `fallback_model`) configured in `config.yaml`
- **Rotate to the next credential** in your [credential pool](/user-guide/configuration#credential-pool-strategies) for the same provider
This means cron jobs that run at high frequency or during peak hours are more resilient — a single rate-limited key won't fail the entire run.
## Schedule formats
The agent's final response is automatically delivered — you do **not** need to include `send_message` in the cron prompt for that same destination. If a cron run calls `send_message` to the exact target the scheduler will already deliver to, Hermes skips that duplicate send and tells the model to put the user-facing content in the final response instead. Use `send_message` only for additional or different targets.
### Relative delays (one-shot)
```text
30m → Run once in 30 minutes
2h → Run once in 2 hours
1d → Run once in 1 day
```
### Intervals (recurring)
```text
every 30m → Every 30 minutes
every 2h → Every 2 hours
every 1d → Every day
```
### Cron expressions
```text
0 9 * * * → Daily at 9:00 AM
0 9 * * 1-5 → Weekdays at 9:00 AM
0 */6 * * * → Every 6 hours
30 8 1 * * → First of every month at 8:30 AM
0 0 * * 0 → Every Sunday at midnight
```
### ISO timestamps
```text
2026-03-15T09:00:00 → One-time at March 15, 2026 9:00 AM
```
## Repeat behavior
| Schedule type | Default repeat | Behavior |
|--------------|----------------|----------|
| One-shot (`30m`, timestamp) | 1 | Runs once |
| Interval (`every 2h`) | forever | Runs until removed |
| Cron expression | forever | Runs until removed |
You can override it:
```python
cronjob(
action="create",
prompt="...",
schedule="every 2h",
repeat=5,
)
```
## Managing jobs programmatically
The agent-facing API is one tool:
```python
cronjob(action="create", ...)
cronjob(action="list")
cronjob(action="update", job_id="...")
cronjob(action="pause", job_id="...")
cronjob(action="resume", job_id="...")
cronjob(action="run", job_id="...")
cronjob(action="remove", job_id="...")
```
For `update`, pass `skills=[]` to remove all attached skills.
## Toolsets available to cron jobs
Cron runs each job in a fresh agent session with no chat platform attached. By default the cron agent gets **the toolset you configured for the `cron` platform in `hermes tools`** — not the CLI default, not everything under the sun.
```bash
hermes tools
# → pick the "cron" platform in the curses UI
# → toggle toolsets on/off just like you would for Telegram/Discord/etc.
```
Tighter per-job control is available via the `enabled_toolsets` field on `cronjob.create` (or on an existing job via `cronjob.update`):
```text
cronjob(action="create", name="weekly-news-summary",
schedule="every sunday 9am",
enabled_toolsets=["web", "file"], # just web + file, no terminal/browser/etc.
prompt="Summarize this week's AI news: ...")
```
When `enabled_toolsets` is set on a job it wins; otherwise the `hermes tools` cron-platform config wins; otherwise Hermes falls back to the built-in defaults. This matters for cost control: carrying `moa`, `browser`, `delegation` into every tiny "fetch news" job bloats the tool-schema prompt on every LLM call.
### Skipping the agent entirely: `wakeAgent`
If your cron job attaches a pre-check script (via `script=`), the script can decide at runtime whether Hermes should even invoke the agent. Emit a final stdout line of the form:
```text
{"wakeAgent": false}
```
…and cron skips the agent run entirely for this tick. Useful for frequent polls (every 1–5 min) that only need to wake the LLM when state actually changed — otherwise you pay for zero-content agent turns over and over.
```python
# pre-check script
import json, sys
latest = fetch_latest_issue_count()
prev = read_state("issue_count")
if latest == prev:
print(json.dumps({"wakeAgent": False})) # skip this tick
sys.exit(0)
write_state("issue_count", latest)
print(json.dumps({"wakeAgent": True, "context": {"new_issues": latest - prev}}))
```
When `wakeAgent` is omitted, the default is `true` (wake the agent as usual).
#### Recipes: cheap pre-run gates
The `wakeAgent` gate gives you a $0 way to decide whether a scheduled job should spend any LLM tokens at all. Three patterns cover most use cases.
**File-change gate** — only run when a watched file has new content since the last successful tick. The scheduler records each job's `last_run_at`; compare it against the file's mtime.
```bash
#!/bin/bash
# ~/.hermes/scripts/feed-changed.sh
FEED="$HOME/data/feed.json"
STATE="$HOME/.hermes/scripts/.feed-changed.last"
test -f "$FEED" || { echo '{"wakeAgent": false}'; exit 0; }
mtime=$(stat -c %Y "$FEED")
last=$(cat "$STATE" 2>/dev/null || echo 0)
if [ "$mtime" -le "$last" ]; then
echo '{"wakeAgent": false}'
else
echo "$mtime" > "$STATE"
echo '{"wakeAgent": true}'
fi
```
```text
cronjob(action="create", name="process-feed",
schedule="every 30m",
script="feed-changed.sh",
prompt="A new ~/data/feed.json has landed. Summarize what changed.")
```
**External-flag gate** — only run when some other process has signalled readiness (e.g. a deploy hook drops a file, a CI job sets a value in your state store).
```bash
#!/bin/bash
# ~/.hermes/scripts/flag-ready.sh
if test -f /tmp/new-data-ready; then
rm -f /tmp/new-data-ready
echo '{"wakeAgent": true}'
else
echo '{"wakeAgent": false}'
fi
```
```text
cronjob(action="create", name="nightly-analysis",
schedule="0 9 * * *",
script="flag-ready.sh",
prompt="Run the nightly analysis over today's batch.")
```
**SQL-count gate** — only run when there are new rows to process in your own database. The script can also pass the count through to the agent via `context`, so the agent knows how much it's looking at without re-querying.
```python
#!/usr/bin/env python
# ~/.hermes/scripts/new-rows.py
import json, sqlite3
conn = sqlite3.connect("/home/me/data/app.db")
n = conn.execute(
"SELECT COUNT(*) FROM messages WHERE ts > strftime('%s','now','-2 hours')"
).fetchone()[0]
if n < 1:
print(json.dumps({"wakeAgent": False}))
else:
print(json.dumps({"wakeAgent": True, "context": {"new_rows": n}}))
```
```text
cronjob(action="create", name="summarize-new-msgs",
schedule="every 2h",
script="new-rows.py",
prompt="Summarize the new messages from the last 2 hours.")
```
The same pattern works for any data source you can query from a script — Postgres, an HTTP API, your own state store — without baking a SQL evaluator into the cron subsystem.
:::tip
Hermes's own `~/.hermes/state.db` is an internal schema that changes between releases. Don't query it from a pre-run gate — point at your own database or feed instead.
:::
Credit: this recipe set was prompted by @iankar8's exploration in [#2654](https://github.com/NousResearch/hermes-agent/pull/2654), which proposed adding sql/file/command triggers as a parallel mechanism. The `script` + `wakeAgent` gate already covers all three cases at $0, so the work landed as documentation instead.
### Chaining jobs: `context_from`
A cron job can consume the most recent successful output of one or more other jobs by listing their names (or IDs) in `context_from`:
```text
cronjob(action="create", name="daily-digest",
schedule="every day 7am",
context_from=["ai-news-fetch", "github-prs-fetch"],
prompt="Write the daily digest using the outputs above.")
```
The referenced jobs' most recent completed outputs are injected above the prompt as context for this run. Each upstream entry must be a valid job ID or name (see `cronjob action="list"`). Note: chaining reads the *most recent completed* output — it does not wait for upstream jobs that are running in the same tick.
## Job storage
Jobs are stored in `~/.hermes/cron/jobs.json`. Output from job runs is saved to `~/.hermes/cron/output/{job_id}/{timestamp}.md`.
Jobs may store `model` and `provider` as `null`. When those fields are omitted, Hermes resolves them at execution time from the global configuration. They only appear in the job record when a per-job override is set.
The storage uses atomic file writes so interrupted writes do not leave a partially written job file behind.
## Self-contained prompts still matter
:::warning Important
Cron jobs run in a completely fresh agent session. The prompt must contain everything the agent needs that is not already provided by attached skills.
:::
**BAD:** `"Check on that server issue"`
**GOOD:** `"SSH into server 192.168.1.100 as user 'deploy', check if nginx is running with 'systemctl status nginx', and verify https://example.com returns HTTP 200."`
## Security
Scheduled task prompts are scanned for prompt-injection and credential-exfiltration patterns at creation and update time. Prompts containing invisible Unicode tricks, SSH backdoor attempts, or obvious secret-exfiltration payloads are blocked.
website/docs/user-guide/features/curator.md
+278
View File
@@ -0,0 +1,278 @@
---
sidebar_position: 3
title: "Curator"
description: "Background maintenance for agent-created skills — usage tracking, staleness, archival, and LLM-driven review"
---
# Curator
The curator is a background maintenance pass for **agent-created skills**. It tracks how often each skill is viewed, used, and patched, moves long-unused skills through `active → stale → archived` states, and periodically spawns a short auxiliary-model review that proposes consolidations or patches drift.
It exists so that skills created via the [self-improvement loop](/user-guide/features/skills#agent-managed-skills-skill_manage-tool) don't pile up forever. Every time the agent solves a novel problem and saves a skill, that skill lands in `~/.hermes/skills/`. Without maintenance, you end up with dozens of narrow near-duplicates that pollute the catalog and waste tokens.
By default (`prune_builtins: true`) the curator can archive **unused bundled built-in skills** (shipped with the repo) after `archive_after_days` of non-use, alongside the agent-created skills it primarily manages. Hub-installed skills (from [agentskills.io](https://agentskills.io)) are always off-limits. Set `curator.prune_builtins: false` to restore the old agent-created-only behavior, where bundled skills are never touched. The curator also **never auto-deletes** — the worst outcome is archival into `~/.hermes/skills/.archive/`, which is recoverable.
Tracks [issue #7816](https://github.com/NousResearch/hermes-agent/issues/7816).
## How it runs
The curator is triggered by an inactivity check, not a cron daemon. On CLI session start, and on a recurring tick inside the gateway's cron-ticker thread, Hermes checks whether:
1. Enough time has passed since the last curator run (`interval_hours`, default **7 days**), and
2. The agent has been idle long enough (`min_idle_hours`, default **2 hours**).
If both are true, it spawns a background fork of `AIAgent` — the same pattern used by the memory/skill self-improvement nudges. The fork runs in its own prompt cache and never touches the active conversation.
:::info First-run behavior
On a brand-new install (or the first time a pre-curator install ticks after `hermes update`), the curator **does not run immediately**. The first observation seeds `last_run_at` to "now" and defers the first real pass by one full `interval_hours`. This gives you a full interval to review your skill library, pin anything important, or opt out entirely before the curator ever touches it.
If you want to see what the curator *would* do before it runs for real, run `hermes curator run --dry-run` — it produces the same review report without mutating the library.
:::
A run has two phases:
1. **Automatic transitions** (deterministic, no LLM). Skills unused for `stale_after_days` (30) become `stale`; skills unused for `archive_after_days` (90) are moved to `~/.hermes/skills/.archive/`.
2. **LLM review** (single aux-model pass, `max_iterations=8`). The forked agent surveys the agent-created skills, can read any of them with `skill_view`, and decides per-skill whether to keep, patch (via `skill_manage`), consolidate overlapping ones, or archive via the terminal tool. Consolidation treats a skill as a full package: if a skill has `references/`, `templates/`, `scripts/`, `assets/`, or relative links to those paths, the curator must either keep it standalone, re-home the needed support files and rewrite paths, or archive the entire package unchanged — not flatten only `SKILL.md` into another skill's `references/` file.
Pinned skills are off-limits to both the curator's auto-transitions and the agent's own `skill_manage` tool. See [Pinning a skill](#pinning-a-skill) below.
## Configuration
All settings live in `config.yaml` under `curator:` (not `.env` — this isn't a secret). Defaults:
```yaml
curator:
enabled: true
interval_hours: 168 # 7 days
min_idle_hours: 2
stale_after_days: 30
archive_after_days: 90
prune_builtins: true # archive unused bundled built-in skills too (hub skills always exempt)
```
To disable entirely, set `curator.enabled: false`.
### Running the review on a cheaper aux model
The curator's LLM review pass is a regular auxiliary task slot — `auxiliary.curator` — alongside Vision, Compression, Session Search, etc. "Auto" means "use my main chat model"; override the slot to pin a specific provider + model for the review pass instead.
**Easiest — `hermes model`:**
```bash
hermes model # → "Auxiliary models — side-task routing"
# → pick "Curator" → pick provider → pick model
```
The same picker is available in the web dashboard under the **Models** tab.
**Direct config.yaml (equivalent):**
```yaml
auxiliary:
curator:
provider: openrouter
model: google/gemini-3-flash-preview
timeout: 600 # generous — reviews can take several minutes
```
Leaving `provider: auto` (the default) routes the review pass through whatever your main chat model is, matching the behavior of every other auxiliary task.
:::note Legacy config
Earlier releases used a one-off `curator.auxiliary.{provider,model}` block. That path still works but emits a deprecation log line — please migrate to `auxiliary.curator` above so the curator shares the same plumbing (`hermes model`, dashboard Models tab, `base_url`, `api_key`, `timeout`, `extra_body`) as every other aux task.
:::
## CLI
```bash
hermes curator status # last run, counts, pinned list, LRU top 5
hermes curator run # trigger a review now (blocks until the LLM pass finishes)
hermes curator run --background # fire-and-forget: start the LLM pass in a background thread
hermes curator run --dry-run # preview only — report without any mutations
hermes curator backup # take a manual snapshot of ~/.hermes/skills/
hermes curator rollback # restore from the newest snapshot
hermes curator rollback --list # list available snapshots
hermes curator rollback --id <ts> # restore a specific snapshot
hermes curator rollback -y # skip the confirmation prompt
hermes curator pause # stop runs until resumed
hermes curator resume
hermes curator pin <skill> # never auto-transition this skill
hermes curator unpin <skill>
hermes curator restore <skill> # move an archived skill back to active
hermes curator list-archived # list skills currently in ~/.hermes/skills/.archive/
hermes curator archive <skill> # manually archive a single skill now
hermes curator prune [--days N] # bulk-archive agent-created skills idle >= N days (default 90)
```
## Backups and rollback
Before every real curator pass, Hermes takes a tar.gz snapshot of `~/.hermes/skills/` at `~/.hermes/skills/.curator_backups/<utc-iso>/skills.tar.gz`. If a pass archives or consolidates something you didn't want touched, you can undo the whole run with one command:
```bash
hermes curator rollback # restore newest snapshot (with confirmation)
hermes curator rollback -y # skip the prompt
hermes curator rollback --list # see all snapshots with reason + size
```
The rollback itself is reversible: before replacing the skills tree, Hermes takes another snapshot tagged `pre-rollback to <target-id>`, so a mistaken rollback can be undone by rolling forward to that one with `--id`.
You can also take manual snapshots at any time with `hermes curator backup --reason "before-refactor"`. The `--reason` string lands in the snapshot's `manifest.json` and is shown in `--list`.
Snapshots are pruned to `curator.backup.keep` (default 5) to keep disk usage bounded:
```yaml
curator:
backup:
enabled: true
keep: 5
```
Set `curator.backup.enabled: false` to disable automatic snapshotting. The manual `hermes curator backup` command still works when backups are disabled only if you set `enabled: true` first — the flag gates both paths symmetrically so there's no way to accidentally skip the pre-run snapshot on mutating runs.
`hermes curator status` also lists the five least-recently-used skills — a quick way to see what's likely to become stale next.
The same subcommands are available as the `/curator` slash command inside a running session (CLI or gateway platforms).
## What "agent-created" means
The curator only manages skills explicitly marked as **agent-created** in
`~/.hermes/skills/.usage.json`. A skill qualifies when ALL of the following
are true:
1. Its name is **not** in `~/.hermes/skills/.bundled_manifest` (bundled skills shipped with the repo).
2. Its name is **not** in `~/.hermes/skills/.hub/lock.json` (hub-installed skills).
3. Its `.usage.json` entry has `"created_by": "agent"` or `"agent_created": true`.
Currently, only the **background self-improvement review fork** sets this marker
— when it creates a new umbrella skill during its periodic review pass (~every 10
agent turns). The background fork runs with a write origin of `"background_review"`
(via `tools/skill_provenance.py`), which is the only path that triggers the
`mark_agent_created()` call in `skill_manage`.
Skills the foreground agent creates via `skill_manage(action="create")` during a
conversation are **not** marked as agent-created — they are considered
user-directed and the curator intentionally leaves them alone.
:::warning Your hand-written skills are NOT curated
If you manually created a `SKILL.md` or pointed Hermes at an external skill
directory, that skill will have a `.usage.json` entry with `created_by: null`
(or the field absent). The curator will not touch it. The same applies to
skills the foreground agent created at your request.
**To see which skills the curator actually manages**, run `hermes curator status`.
If the agent-created count is 0, no skills are currently in the curator's
jurisdiction — the LLM review pass is skipped and the report will show
`Model: (not resolved) via (not resolved)` with `Duration: 0s`.
:::
Skills that ARE agent-created follow the full lifecycle:
- `active` → (30d unused) `stale` → (90d unused) `archived`
- Pinned skills bypass all auto-transitions
- Archives are recoverable via `hermes curator restore <name>`
If you want to protect a specific skill from ever being touched — for example a
hand-authored skill you rely on — use `hermes curator pin <name>`. See the next
section.
## Pinning a skill
Pinning protects a skill from deletion — both the curator's automated archive passes and the agent's `skill_manage(action="delete")` tool call. Once a skill is pinned:
- The **curator** skips it during auto-transitions (`active → stale → archived`), and its LLM review pass is instructed to leave it alone.
- The **agent's `skill_manage` tool** refuses `delete` on it, pointing the user at `hermes curator unpin <name>`. Patches and edits still go through, so the agent can improve a pinned skill's content as pitfalls come up without a pin/unpin/re-pin dance.
Pin and unpin with:
```bash
hermes curator pin <skill>
hermes curator unpin <skill>
```
The flag is stored as `"pinned": true` on the skill's entry in `~/.hermes/skills/.usage.json`, so it survives across sessions.
Only **agent-created** skills can be pinned — `hermes curator pin` refuses on bundled and hub-installed skills with an explanatory message if you try. Hub-installed skills are never subject to curator mutation. Bundled built-in skills are only touched when `curator.prune_builtins: true` (the default), and even then only archived after `archive_after_days` of non-use — never patched, consolidated, or deleted. Set `curator.prune_builtins: false` to exempt bundled skills entirely.
A small set of **protected built-ins** is hardcoded as never-archivable and never-consolidatable, regardless of `curator.prune_builtins`, pin state, or LLM judgment. These back load-bearing UX — for example, `plan` powers the `/plan` slash-command flow — so silently archiving one would turn its slash command into an "Unknown command" error with no signal to you. Protected built-ins are filtered out of the curator's candidate list entirely, so the consolidation pass never sees them.
If you want a stronger guarantee than "no deletion" — for instance, freezing a skill's content entirely while the agent still reads it — edit `~/.hermes/skills/<name>/SKILL.md` directly with your editor. The pin guards tool-driven deletion, not your own filesystem access.
## Usage telemetry
The curator maintains a sidecar at `~/.hermes/skills/.usage.json` with one entry per skill:
```json
{
"my-skill": {
"use_count": 12,
"view_count": 34,
"last_used_at": "2026-04-24T18:12:03Z",
"last_viewed_at": "2026-04-23T09:44:17Z",
"patch_count": 3,
"last_patched_at": "2026-04-20T22:01:55Z",
"created_at": "2026-03-01T14:20:00Z",
"state": "active",
"pinned": false,
"archived_at": null
}
}
```
Counters increment when:
- `view_count`: the agent calls `skill_view` on the skill.
- `use_count`: the skill is loaded into a conversation's prompt.
- `patch_count`: `skill_manage patch/edit/write_file/remove_file` runs on the skill.
Bundled and hub-installed skills are explicitly excluded from telemetry writes.
## Per-run reports
Every curator run writes a timestamped directory under `~/.hermes/logs/curator/`:
```
~/.hermes/logs/curator/
└── 20260429-111512/
├── run.json # machine-readable: full fidelity, stats, LLM output
└── REPORT.md # human-readable summary
```
`REPORT.md` is a quick way to see what a given run did — which skills transitioned, what the LLM reviewer said, which skills it patched. Good for auditing without having to grep `agent.log`.
:::note No candidates? Report shows `(not resolved)`
When the curator has **no agent-created skills** to review, the LLM review pass
is skipped entirely. The report header will show
`Model: (not resolved) via (not resolved)` with `Duration: 0s` — this does **not**
indicate a configuration error or model resolution failure. It simply means there
were no candidates, so no model was ever invoked. The auto-transition phase still
runs and reports its counts normally.
:::
### Rename map in the summary
If a run consolidated multiple skills under an umbrella (or merged near-duplicates), the user-visible summary printed at the end of the run includes an explicit rename map showing every `old-name → new-name` pair the curator applied. This is in addition to per-skill transition lines, so when a wave of renames lands you can spot them at a glance without diffing the JSON report. The hint also surfaces under `hermes curator pin` so you can pin the umbrella name immediately if you want to lock the new label in.
## Restoring an archived skill
If the curator archived something you still want:
```bash
hermes curator restore <skill-name>
```
This moves the skill back from `~/.hermes/skills/.archive/` to the active tree and resets its state to `active`. The restore refuses if a bundled or hub-installed skill has since been installed under the same name (would shadow upstream).
## Disabling per environment
The curator is on by default. To turn it off:
- **For one profile only:** edit `~/.hermes/config.yaml` (or the active profile's config) and set `curator.enabled: false`.
- **For just one run:** `hermes curator pause` — the pause persists across sessions; use `resume` to re-enable.
The curator also refuses to run if `min_idle_hours` hasn't elapsed, so on an active dev machine it naturally only runs during quiet stretches.
## See also
- [Skills System](/user-guide/features/skills) — how skills work in general and the self-improvement loop that creates them
- [Memory](/user-guide/features/memory) — a parallel background review that maintains long-term memory
- [Bundled Skills Catalog](/reference/skills-catalog)
- [Issue #7816](https://github.com/NousResearch/hermes-agent/issues/7816) — original proposal and design discussion
website/docs/user-guide/features/delegation.md
+290
View File
@@ -0,0 +1,290 @@
---
sidebar_position: 7
title: "Subagent Delegation"
description: "Spawn isolated child agents for parallel workstreams with delegate_task"
---
# Subagent Delegation
The `delegate_task` tool spawns child AIAgent instances with isolated context, restricted toolsets, and their own terminal sessions. Each child gets a fresh conversation and works independently — only its final summary enters the parent's context.
## Single Task
```python
delegate_task(
goal="Debug why tests fail",
context="Error: assertion in test_foo.py line 42",
toolsets=["terminal", "file"]
)
```
## Parallel Batch
Up to 3 concurrent subagents by default (configurable, no hard ceiling):
```python
delegate_task(tasks=[
{"goal": "Research topic A", "toolsets": ["web"]},
{"goal": "Research topic B", "toolsets": ["web"]},
{"goal": "Fix the build", "toolsets": ["terminal", "file"]}
])
```
## How Subagent Context Works
:::warning Critical: Subagents Know Nothing
Subagents start with a **completely fresh conversation**. They have zero knowledge of the parent's conversation history, prior tool calls, or anything discussed before delegation. The subagent's only context comes from the `goal` and `context` fields the parent agent populates when it calls `delegate_task`.
:::
This means the parent agent must pass **everything** the subagent needs in the call:
```python
# BAD - subagent has no idea what "the error" is
delegate_task(goal="Fix the error")
# GOOD - subagent has all context it needs
delegate_task(
goal="Fix the TypeError in api/handlers.py",
context="""The file api/handlers.py has a TypeError on line 47:
'NoneType' object has no attribute 'get'.
The function process_request() receives a dict from parse_body(),
but parse_body() returns None when Content-Type is missing.
The project is at /home/user/myproject and uses Python 3.11."""
)
```
The subagent receives a focused system prompt built from your goal and context, instructing it to complete the task and provide a structured summary of what it did, what it found, any files modified, and any issues encountered.
## Practical Examples
### Parallel Research
Research multiple topics simultaneously and collect summaries:
```python
delegate_task(tasks=[
{
"goal": "Research the current state of WebAssembly in 2025",
"context": "Focus on: browser support, non-browser runtimes, language support",
"toolsets": ["web"]
},
{
"goal": "Research the current state of RISC-V adoption in 2025",
"context": "Focus on: server chips, embedded systems, software ecosystem",
"toolsets": ["web"]
},
{
"goal": "Research quantum computing progress in 2025",
"context": "Focus on: error correction breakthroughs, practical applications, key players",
"toolsets": ["web"]
}
])
```
### Code Review + Fix
Delegate a review-and-fix workflow to a fresh context:
```python
delegate_task(
goal="Review the authentication module for security issues and fix any found",
context="""Project at /home/user/webapp.
Auth module files: src/auth/login.py, src/auth/jwt.py, src/auth/middleware.py.
The project uses Flask, PyJWT, and bcrypt.
Focus on: SQL injection, JWT validation, password handling, session management.
Fix any issues found and run the test suite (pytest tests/auth/).""",
toolsets=["terminal", "file"]
)
```
### Multi-File Refactoring
Delegate a large refactoring task that would flood the parent's context:
```python
delegate_task(
goal="Refactor all Python files in src/ to replace print() with proper logging",
context="""Project at /home/user/myproject.
Use the 'logging' module with logger = logging.getLogger(__name__).
Replace print() calls with appropriate log levels:
- print(f"Error: ...") -> logger.error(...)
- print(f"Warning: ...") -> logger.warning(...)
- print(f"Debug: ...") -> logger.debug(...)
- Other prints -> logger.info(...)
Don't change print() in test files or CLI output.
Run pytest after to verify nothing broke.""",
toolsets=["terminal", "file"]
)
```
## Batch Mode Details
When you provide a `tasks` array, subagents run in **parallel** using a thread pool:
- **Maximum concurrency:** 3 tasks by default (configurable via `delegation.max_concurrent_children` or the `DELEGATION_MAX_CONCURRENT_CHILDREN` env var; floor of 1, no hard ceiling). Batches larger than the limit return a tool error rather than being silently truncated.
- **Thread pool:** Uses `ThreadPoolExecutor` with the configured concurrency limit as max workers
- **Progress display:** In CLI mode, a tree-view shows tool calls from each subagent in real-time with per-task completion lines. In gateway mode, progress is batched and relayed to the parent's progress callback
- **Result ordering:** Results are sorted by task index to match input order regardless of completion order
- **Interrupt propagation:** Interrupting the parent (e.g., sending a new message) interrupts all active children
Single-task delegation runs directly without thread pool overhead.
## Model Override
You can configure a different model for subagents via `config.yaml` — useful for delegating simple tasks to cheaper/faster models:
```yaml
# In ~/.hermes/config.yaml
delegation:
model: "google/gemini-flash-2.0" # Cheaper model for subagents
provider: "openrouter" # Optional: route subagents to a different provider
```
If omitted, subagents use the same model as the parent.
## Toolset Selection Tips
The `toolsets` parameter controls what tools the subagent has access to. Choose based on the task:
| Toolset Pattern | Use Case |
|----------------|----------|
| `["terminal", "file"]` | Code work, debugging, file editing, builds |
| `["web"]` | Research, fact-checking, documentation lookup |
| `["terminal", "file", "web"]` | Full-stack tasks (default) |
| `["file"]` | Read-only analysis, code review without execution |
| `["terminal"]` | System administration, process management |
Certain toolsets are blocked for subagents regardless of what you specify:
- `delegation` — blocked for leaf subagents (the default). Retained for `role="orchestrator"` children, bounded by `max_spawn_depth` — see [Depth Limit and Nested Orchestration](#depth-limit-and-nested-orchestration) below.
- `clarify` — subagents cannot interact with the user
- `memory` — no writes to shared persistent memory
- `code_execution` — children should reason step-by-step
- `send_message` — no cross-platform side effects (e.g., sending Telegram messages)
## Max Iterations
Each subagent has an iteration limit (default: 50) that controls how many tool-calling turns it can take:
```python
delegate_task(
goal="Quick file check",
context="Check if /etc/nginx/nginx.conf exists and print its first 10 lines",
max_iterations=10 # Simple task, don't need many turns
)
```
## Child Timeout
By default there is **no wall-clock timeout** on subagents. Children fail only from what they're actually doing — API errors, tool errors, or hitting their iteration budget — never from a delegation-level stopwatch. Earlier releases shipped a hard cap (300s, later 600s), which kept killing legitimately busy children mid-task: deep code reviews, large research fan-outs, and slow reasoning models routinely need more than 10 minutes while making steady progress the whole time.
Genuinely stuck children are still detected: the heartbeat staleness monitor stops refreshing the parent's activity when a child makes no progress (no API calls, no tool starts), letting the gateway inactivity timeout fire on a truly wedged worker.
If you want a hard cap anyway (e.g. cost control on unattended cron-driven delegation), opt in per-install:
```yaml
delegation:
child_timeout_seconds: 0 # default: 0 = no timeout
# child_timeout_seconds: 1800 # opt-in hard cap (floor 30s)
```
A positive value enforces a hard wall-clock limit on each child; `0` or a negative value disables it.
:::tip Diagnostic dump on zero-call timeout
With a hard cap configured, if a subagent times out having made **zero** API calls (usually: provider unreachable, auth failure, or tool-schema rejection), `delegate_task` writes a structured diagnostic to `~/.hermes/logs/subagent-timeout-<session>-<timestamp>.log` containing the subagent's config snapshot, credential-resolution trace, and any early error messages. Much easier to root-cause than the previous silent-timeout behavior.
:::
## Monitoring Running Subagents (`/agents`)
The TUI ships a `/agents` overlay (alias `/tasks`) that turns recursive `delegate_task` fan-out into a first-class audit surface:
- Live tree view of running and recently-finished subagents, grouped by parent
- Per-branch cost, token, and file-touched rollups
- Kill and pause controls — cancel a specific subagent mid-flight without interrupting its siblings
- Post-hoc review: step through each subagent's turn-by-turn history even after they've returned to the parent
The classic CLI just prints `/agents` as a text summary; the TUI is where the overlay shines. See [TUI — Slash commands](/user-guide/tui#slash-commands).
## Depth Limit and Nested Orchestration
By default, delegation is **flat**: a parent (depth 0) spawns children (depth 1), and those children cannot delegate further. This prevents runaway recursive delegation.
For multi-stage workflows (research → synthesis, or parallel orchestration over sub-problems), a parent can spawn **orchestrator** children that *can* delegate their own workers:
```python
delegate_task(
goal="Survey three code review approaches and recommend one",
role="orchestrator", # Allows this child to spawn its own workers
context="...",
)
```
- `role="leaf"` (default): child cannot delegate further — identical to the flat-delegation behavior.
- `role="orchestrator"`: child retains the `delegation` toolset. Gated by `delegation.max_spawn_depth` (default **1** = flat, so `role="orchestrator"` is a no-op at defaults). Raise `max_spawn_depth` to 2 to allow orchestrator children to spawn leaf grandchildren; 3+ for deeper trees. There is no upper ceiling — cost is the practical limit.
- `delegation.orchestrator_enabled: false`: global kill switch that forces every child to `leaf` regardless of the `role` parameter.
**Cost warning:** With `max_spawn_depth: 3` and `max_concurrent_children: 3`, the tree can reach 3×3×3 = 27 concurrent leaf agents. Each extra level multiplies spend — raise `max_spawn_depth` intentionally.
## Lifetime and Durability
:::warning delegate_task is synchronous — not durable
`delegate_task` runs **inside the parent's current turn**. It blocks the parent until every child finishes (or is cancelled). It is **not** a background job queue:
- If the parent is interrupted (user sends a new message, `/stop`, `/new`), all active children are cancelled and return `status="interrupted"`. Their in-progress work is discarded.
- Children do **not** continue running after the parent turn ends.
- Cancelled children return a structured result (`status="interrupted"`, `exit_reason="interrupted"`), but because the parent was interrupted too, that result often never makes it into a user-visible reply.
For **durable long-running work** that must survive interrupts or outlive the current turn, use:
- `cronjob` (action=`create`) — schedules a separate agent run; immune to parent-turn interrupts.
- `terminal(background=True, notify_on_complete=True)` — long-running shell commands that keep running while the agent does other things.
:::
## Key Properties
- Each subagent gets its **own terminal session** (separate from the parent)
- **Nested delegation is opt-in** — only `role="orchestrator"` children can delegate further, and only when `max_spawn_depth` is raised from its default of 1 (flat). Disable globally with `orchestrator_enabled: false`.
- Leaf subagents **cannot** call: `delegate_task`, `clarify`, `memory`, `send_message`, `execute_code`. Orchestrator subagents retain `delegate_task` but still cannot use the other four.
- **Interrupt propagation** — interrupting the parent interrupts all active children (including grandchildren under orchestrators)
- Only the final summary enters the parent's context, keeping token usage efficient
- Subagents inherit the parent's **API key, provider configuration, and credential pool** (enabling key rotation on rate limits)
## Delegation vs execute_code
| Factor | delegate_task | execute_code |
|--------|--------------|-------------|
| **Reasoning** | Full LLM reasoning loop | Just Python code execution |
| **Context** | Fresh isolated conversation | No conversation, just script |
| **Tool access** | All non-blocked tools with reasoning | 7 tools via RPC, no reasoning |
| **Parallelism** | 3 concurrent subagents by default (configurable) | Single script |
| **Best for** | Complex tasks needing judgment | Mechanical multi-step pipelines |
| **Token cost** | Higher (full LLM loop) | Lower (only stdout returned) |
| **User interaction** | None (subagents can't clarify) | None |
**Rule of thumb:** Use `delegate_task` when the subtask requires reasoning, judgment, or multi-step problem solving. Use `execute_code` when you need mechanical data processing or scripted workflows.
## Configuration
```yaml
# In ~/.hermes/config.yaml
delegation:
max_iterations: 50 # Max turns per child (default: 50)
# max_concurrent_children: 3 # Parallel children per batch (default: 3)
# max_spawn_depth: 1 # Tree depth (floor 1, no ceiling, default 1 = flat). Raise to 2 to allow orchestrator children to spawn leaves; 3+ for deeper trees.
# orchestrator_enabled: true # Disable to force all children to leaf role.
model: "google/gemini-3-flash-preview" # Optional provider/model override
provider: "openrouter" # Optional built-in provider
api_mode: anthropic_messages # optional; auto-detected from base_url for anthropic_messages endpoints
# Or use a direct custom endpoint instead of provider:
delegation:
model: "qwen2.5-coder"
base_url: "http://localhost:1234/v1"
api_key: "local-key"
# api_mode: "anthropic_messages" # Optional. Wire protocol override for base_url ("chat_completions", "codex_responses", or "anthropic_messages"). Empty = auto-detect from URL (e.g. /anthropic suffix). Set explicitly for endpoints the heuristic can't classify (Azure AI Foundry, MiniMax, Zhipu GLM, LiteLLM proxies, …).
```
When `base_url` points at an Anthropic-compatible endpoint — for example a path ending in `/anthropic`, an Azure Foundry Claude route, or a MiniMax `/anthropic` proxy — `api_mode` is auto-detected as `anthropic_messages` so the subagent uses the right wire format without you setting anything. Set `api_mode` explicitly when the auto-detection guess is wrong (rare).
:::tip
The agent handles delegation automatically based on the task complexity. You don't need to explicitly ask it to delegate — it will do so when it makes sense.
:::
website/docs/user-guide/features/deliverable-mode.md
+132
View File
@@ -0,0 +1,132 @@
---
title: Deliverable Mode (Artifacts in Chat)
sidebar_label: Deliverable Mode
description: How the agent ships generated charts, PDFs, spreadsheets, and other files as native attachments in messaging platforms.
---
# Deliverable Mode
When Hermes Agent runs inside a messaging gateway (Slack, Discord, Telegram,
WhatsApp, Signal, etc.), it can deliver generated files directly into the
chat — not as paths the user has to copy, but as native attachments.
A chart shows up as an inline image. A PDF report shows up as a file
download. A spreadsheet uploads as `.xlsx`. The agent does not need to
write a `MEDIA:` tag or do anything special — it just generates the file
and mentions its absolute path in the response. The gateway picks the path
out of the text, removes it from the visible message, and uploads the
file natively.
## How it works
Three pieces fit together:
1. **The agent has tools that produce files.** `execute_code` for charts via
matplotlib, the `latex-pdf-report` skill for PDFs, the `powerpoint` skill
for decks, `image_generate` for images, `text_to_speech` for audio, and so
on.
2. **The gateway scans agent responses for file paths.** Any absolute path
(`/tmp/...`) or home-relative path (`~/...`) ending in a supported
extension gets extracted. Paths inside code blocks and inline code are
ignored so code samples are never mutilated.
3. **The gateway dispatches by file type.** Images embed inline where the
platform supports it; videos embed inline; audio routes to voice/audio
attachments; everything else uploads as a file attachment.
## Supported file extensions
| Category | Extensions | Delivery |
|---|---|---|
| Images | `.png .jpg .jpeg .gif .webp .bmp .tiff .svg` | Inline embed |
| Video | `.mp4 .mov .avi .mkv .webm` | Inline embed (where supported) |
| Audio | `.mp3 .wav .ogg .m4a .flac` | Voice / audio attachment |
| Documents | `.pdf .docx .doc .odt .rtf .txt .md` | File upload |
| Data | `.xlsx .xls .csv .tsv .json .xml .yaml .yml` | File upload |
| Presentations | `.pptx .ppt .odp` | File upload |
| Archives | `.zip .tar .gz .tgz .bz2 .7z` | File upload |
| Web | `.html .htm` | File upload |
`.py`, `.log`, and other source-file extensions are intentionally excluded so
the agent doesn't auto-ship arbitrary source files; if you want to send code
to the user, use a code block.
## Encouraging the agent to produce artifacts
The agent doesn't reach for artifacts by default — it has to know to.
Two ways to nudge it:
**Per-session:** ask explicitly ("send me the comparison as a chart",
"return the data as a CSV") or write your own custom-instructions /
personality entry that biases toward artifact-style replies on
messaging platforms.
**Project-level:** add the bias to `AGENTS.md` / `CLAUDE.md` /
`.cursorrules` in a project the agent works from, to your global
persona in `~/.hermes/SOUL.md`, or as a named preset under
`agent.personalities` in `~/.hermes/config.yaml` (switchable per session
via `/personality`).
The mechanic the agent has to use is simple: render the file to an
absolute path (e.g. `/tmp/q3-revenue.png`) and mention that path as
plain text in the reply. The gateway does the rest. Paths inside
fenced code blocks or backticks are ignored so code samples are never
mutilated.
## Kanban: artifacts ride completion notifications
If you use Hermes' kanban multi-agent workflow, workers can attach
deliverable files to their `kanban_complete` call:
```python
kanban_complete(
summary="rendered Q3 revenue chart and report",
artifacts=[
"/tmp/q3-revenue.png",
"/tmp/q3-report.pdf",
],
)
```
When the gateway notifier delivers the "task completed" message to whoever
subscribed to the task in Slack/Telegram/etc., it also uploads each artifact
as a native attachment to that chat. The human gets the deliverable and the
summary in one place.
Files that don't exist on disk when the notifier runs are silently skipped.
## Connecting more services with MCP
Beyond the artifact-delivery pipeline, the agent can reach into other
services via MCP (Model Context Protocol). The MCP ecosystem ships
community servers for most popular tools — install whichever you need:
| Service | What it unlocks |
|---|---|
| **Notion** | Read/write Notion pages, databases, query workspace |
| **GitHub** | Issues, PRs, comments, repo search beyond the gh CLI |
| **Linear** | Tickets, projects, cycles |
| **Slack** | Workspace-wide search, read other channels |
| **Gmail** | Inbox triage, send mail, label management |
| **Salesforce** | Leads, opportunities, account data |
| **Snowflake / BigQuery** | SQL against data warehouses |
| **Google Drive** | File search, contents, share management |
Install MCP servers via `~/.hermes/config.yaml` under the `mcp_servers`
section. See [MCP integration](./mcp.md) for the full setup guide.
## Comparison to Perplexity Computer in Slack
Perplexity Computer's Slack integration is built around the same idea:
the agent generates a deliverable (chart, PDF, slide deck) and posts it
back into the thread as a native attachment. Hermes Agent's deliverable
mode provides the same user-facing pattern locally:
- Generation happens in the user's own venv / sandbox (no remote tenant).
- Files land in the chat via the same Slack `files.uploadV2` API.
- Connector breadth comes via MCP rather than a curated catalog of 400
hosted integrations — install the ones you actually use.
OAuth tokens stay on the user's machine in `auth.json` / `.env`. No hosted
token storage. No multi-tenant microVM. Same end result.
website/docs/user-guide/features/extending-the-dashboard.md
+923
View File
@@ -0,0 +1,923 @@
---
sidebar_position: 17
title: "Extending the Dashboard"
description: "Build themes and plugins for the Hermes web dashboard — palettes, typography, layouts, custom tabs, shell slots, page-scoped slots, and backend API routes"
---
# Extending the Dashboard
The Hermes web dashboard (`hermes dashboard`) is built to be reskinned and extended without forking the codebase. Three layers are exposed:
1. **Themes** — YAML files that repaint the dashboard's palette, typography, layout, and per-component chrome. Drop a file in `~/.hermes/dashboard-themes/`; it appears in the theme switcher.
2. **UI plugins** — a directory with `manifest.json` + a JavaScript bundle that registers a tab, replaces a built-in page, augments one via page-scoped slots, or injects components into named shell slots.
3. **Backend plugins** — a Python file inside that plugin directory that exposes a FastAPI `router`; routes are mounted under `/api/plugins/<name>/` and called from the plugin's UI.
All three are **drop-in at runtime**: no repo clone, no `npm run build`, no patching the dashboard source. This page is the canonical reference for all three.
If you just want to use the dashboard, see [Web Dashboard](./web-dashboard). If you want to reskin the terminal CLI (not the web dashboard), see [Skins & Themes](./skins) — the CLI skin system is unrelated to dashboard themes.
:::note How the pieces compose
Themes and plugins are independent but synergistic. A theme can stand alone (just a YAML file). A plugin can stand alone (just a tab). Together they let you build a complete visual reskin with custom HUDs — the example `strike-freedom-cockpit` demo (lives in the `hermes-example-plugins` companion repo — see [Combined theme + plugin demo](#combined-theme--plugin-demo) for install steps) does exactly that.
:::
---
## Table of contents
- [Themes](#themes)
- [Quick start — your first theme](#quick-start--your-first-theme)
- [Palette, typography, layout](#palette-typography-layout)
- [Layout variants](#layout-variants)
- [Theme assets (images as CSS vars)](#theme-assets-images-as-css-vars)
- [Component chrome overrides](#component-chrome-overrides)
- [Color overrides](#color-overrides)
- [Raw `customCSS`](#raw-customcss)
- [Built-in themes](#built-in-themes)
- [Full theme YAML reference](#full-theme-yaml-reference)
- [Plugins](#plugins)
- [Quick start — your first plugin](#quick-start--your-first-plugin)
- [Directory layout](#directory-layout)
- [Manifest reference](#manifest-reference)
- [The Plugin SDK](#the-plugin-sdk)
- [Shell slots](#shell-slots)
- [Replacing built-in pages (`tab.override`)](#replacing-built-in-pages-taboverride)
- [Augmenting built-in pages (page-scoped slots)](#augmenting-built-in-pages-page-scoped-slots)
- [Slot-only plugins (`tab.hidden`)](#slot-only-plugins-tabhidden)
- [Backend API routes](#backend-api-routes)
- [Custom CSS per plugin](#custom-css-per-plugin)
- [Plugin discovery & reload](#plugin-discovery--reload)
- [Combined theme + plugin demo](#combined-theme--plugin-demo)
- [API reference](#api-reference)
- [Troubleshooting](#troubleshooting)
---
## Themes
Themes are YAML files stored in `~/.hermes/dashboard-themes/`. The file name doesn't matter (the theme's `name:` field is what the system uses), but convention is `<name>.yaml`. Every field is optional — missing keys fall back to the built-in `default` theme, so a theme can be as small as one color.
### Quick start — your first theme
```bash
mkdir -p ~/.hermes/dashboard-themes
```
```yaml
# ~/.hermes/dashboard-themes/neon.yaml
name: neon
label: Neon
description: Pure magenta on black
palette:
background: "#000000"
midground: "#ff00ff"
```
Refresh the dashboard. Click the palette icon in the header and pick **Neon**. The background goes black, text and accents go magenta, and every derived color (card, border, muted, ring, etc.) is recomputed from that 2-color triplet via `color-mix()` in CSS.
That's the whole onboarding: one file, two colors. Everything below is optional refinement.
### Palette, typography, layout
These three blocks are the heart of a theme. Each is independent — override one, leave the others.
#### Palette (3-layer)
The palette is a triplet of color layers plus a warm-glow vignette color and a noise-grain multiplier. The dashboard's design-system cascade derives every shadcn-compatible token (card, popover, muted, border, primary, destructive, ring, etc.) from this triplet via CSS `color-mix()`. Overriding three colors cascades into the whole UI.
| Key | Description |
|-----|-------------|
| `palette.background` | Deepest canvas color — typically near-black. Drives the page background and card fill. |
| `palette.midground` | Primary text and accent. Most UI chrome reads this (foreground text, button outlines, focus rings). |
| `palette.foreground` | Top-layer highlight. The default theme sets this to white at alpha 0 (invisible); themes that want a bright accent on top can raise its alpha. |
| `palette.warmGlow` | `rgba(...)` string used as the vignette color by `<Backdrop />`. |
| `palette.noiseOpacity` | 0–1.2 multiplier on the grain overlay. Lower = softer, higher = grittier. |
Each layer accepts either `{hex: "#RRGGBB", alpha: 0.0–1.0}` or a bare hex string (alpha defaults to 1.0).
```yaml
palette:
background:
hex: "#05091a"
alpha: 1.0
midground: "#d8f0ff" # bare hex, alpha = 1.0
foreground:
hex: "#ffffff"
alpha: 0 # invisible top layer
warmGlow: "rgba(255, 199, 55, 0.24)"
noiseOpacity: 0.7
```
#### Typography
| Key | Type | Description |
|-----|------|-------------|
| `fontSans` | string | CSS font-family stack for body copy (applied to `html`, `body`). |
| `fontMono` | string | CSS font-family stack for code blocks, `<code>`, `.font-mono` utilities. |
| `fontDisplay` | string | Optional heading/display stack. Falls back to `fontSans`. |
| `fontUrl` | string | Optional external stylesheet URL. Injected as `<link rel="stylesheet">` in `<head>` on theme switch. Same URL is never injected twice. Works with Google Fonts, Bunny Fonts, self-hosted `@font-face` sheets — anything linkable. |
| `baseSize` | string | Root font size — controls the rem scale. E.g. `"14px"`, `"16px"`. |
| `lineHeight` | string | Default line-height. E.g. `"1.5"`, `"1.65"`. |
| `letterSpacing` | string | Default letter-spacing. E.g. `"0"`, `"0.01em"`, `"-0.01em"`. |
```yaml
typography:
fontSans: '"Orbitron", "Eurostile", "Impact", sans-serif'
fontMono: '"Share Tech Mono", ui-monospace, monospace'
fontDisplay: '"Orbitron", "Eurostile", sans-serif'
fontUrl: "https://fonts.googleapis.com/css2?family=Orbitron:wght@400;500;600;700&family=Share+Tech+Mono&display=swap"
baseSize: "14px"
lineHeight: "1.5"
letterSpacing: "0.04em"
```
##### Changing the font from the UI (no YAML)
The theme picker in the dashboard header has a **Font** section below the
theme list. Pick any font there and it overrides the body font of whatever
theme is active — the choice is independent of the theme and persists across
theme switches (stored in `config.yaml` under `dashboard.font`). Choose
**Theme default** to clear the override and fall back to the active theme's
own `fontSans`.
The picker offers a curated catalog (system stacks plus a set of Google-Fonts
families across sans / serif / mono). It deliberately does **not** accept a
free-text font URL — the font's stylesheet is injected as a `<link>`, so the
catalog keeps the injected origins fixed. For a fully custom face, set
`fontSans` + `fontUrl` in a theme YAML as shown above. The theme's `fontMono`
(code blocks, terminal) is always left untouched by the UI override.
#### Layout
| Key | Values | Description |
|-----|--------|-------------|
| `radius` | any CSS length (`"0"`, `"0.25rem"`, `"0.5rem"`, `"1rem"`, ...) | Corner-radius token. Maps to `--radius` and cascades into `--radius-sm/md/lg/xl` — every rounded element shifts together. |
| `density` | `compact` \| `comfortable` \| `spacious` | Spacing multiplier applied as the `--spacing-mul` CSS var. `compact = 0.85×`, `comfortable = 1.0×` (default), `spacious = 1.2×`. Scales Tailwind's base spacing, so padding, gap, and space-between utilities all shift proportionally. |
```yaml
layout:
radius: "0"
density: compact
```
### Layout variants
`layoutVariant` picks the overall shell layout. Defaults to `"standard"` when absent.
| Variant | Behaviour |
|---------|-----------|
| `standard` | Single column, 1600px max-width (default). |
| `cockpit` | Left sidebar rail (260px) + main content. Populated by plugins via the `sidebar` slot — see [Shell slots](#shell-slots). Without a plugin the rail shows a placeholder. |
| `tiled` | Drops the max-width clamp so pages can use the full viewport width. |
```yaml
layoutVariant: cockpit
```
The current variant is exposed as `document.documentElement.dataset.layoutVariant`, so raw CSS in `customCSS` can target it via `:root[data-layout-variant="cockpit"] ...`.
### Theme assets (images as CSS vars)
Ship artwork URLs with a theme. Each named slot becomes a CSS var (`--theme-asset-<name>`) that the built-in shell and any plugin can read. The `bg` slot is automatically wired into the backdrop; other slots are plugin-facing.
```yaml
assets:
bg: "https://example.com/hero-bg.jpg" # auto-wired into <Backdrop />
hero: "/my-images/strike-freedom.png" # for plugin sidebars
crest: "/my-images/crest.svg" # for header-left plugins
logo: "/my-images/logo.png"
sidebar: "/my-images/rail.png"
header: "/my-images/header-art.png"
custom:
scanLines: "/my-images/scanlines.png" # → --theme-asset-custom-scanLines
```
Values accept:
- Bare URLs — wrapped in `url(...)` automatically.
- Pre-wrapped `url(...)`, `linear-gradient(...)`, `radial-gradient(...)` expressions — used as-is.
- `"none"` — explicit opt-out.
Every asset is also emitted as `--theme-asset-<name>-raw` (the unwrapped URL), in case a plugin needs to pass it to `<img src>` instead of `background-image`.
Plugins read these with plain CSS or JS:
```javascript
// In a plugin slot
const hero = getComputedStyle(document.documentElement)
.getPropertyValue("--theme-asset-hero").trim();
```
### Component chrome overrides
`componentStyles` restyles individual shell components without writing CSS selectors. Each bucket's entries become CSS vars (`--component-<bucket>-<kebab-property>`) that the shell's shared components read. So `card:` overrides apply to every `<Card>`, `header:` to the app bar, etc.
```yaml
componentStyles:
card:
clipPath: "polygon(12px 0, 100% 0, 100% calc(100% - 12px), calc(100% - 12px) 100%, 0 100%, 0 12px)"
background: "linear-gradient(180deg, rgba(10, 22, 52, 0.85), rgba(5, 9, 26, 0.92))"
boxShadow: "inset 0 0 0 1px rgba(64, 200, 255, 0.28)"
header:
background: "linear-gradient(180deg, rgba(16, 32, 72, 0.95), rgba(5, 9, 26, 0.9))"
tab:
clipPath: "polygon(6px 0, 100% 0, calc(100% - 6px) 100%, 0 100%)"
sidebar: {}
backdrop: {}
footer: {}
progress: {}
badge: {}
page: {}
```
Supported buckets: `card`, `header`, `footer`, `sidebar`, `tab`, `progress`, `badge`, `backdrop`, `page`.
Property names use camelCase (`clipPath`) and are emitted as kebab (`clip-path`). Values are plain CSS strings — anything CSS accepts (`clip-path`, `border-image`, `background`, `box-shadow`, `animation`, ...).
### Color overrides
Most themes won't need this — the 3-layer palette derives every shadcn token. Use `colorOverrides` when you want a specific accent the derivation won't produce (a softer destructive red for a pastel theme, a specific success green for a brand).
```yaml
colorOverrides:
primary: "#ffce3a"
primaryForeground: "#05091a"
accent: "#3fd3ff"
ring: "#3fd3ff"
destructive: "#ff3a5e"
border: "rgba(64, 200, 255, 0.28)"
```
Supported keys: `card`, `cardForeground`, `popover`, `popoverForeground`, `primary`, `primaryForeground`, `secondary`, `secondaryForeground`, `muted`, `mutedForeground`, `accent`, `accentForeground`, `destructive`, `destructiveForeground`, `success`, `warning`, `border`, `input`, `ring`.
Each key maps 1:1 to the `--color-<kebab>` CSS var (e.g. `primaryForeground` → `--color-primary-foreground`). Any key set here wins over the palette cascade for the active theme only — switching to another theme clears the overrides.
### Raw `customCSS`
For selector-level chrome that `componentStyles` can't express — pseudo-elements, animations, media queries, theme-scoped overrides — drop raw CSS into `customCSS`:
```yaml
customCSS: |
/* Scanline overlay — only visible when cockpit variant is active. */
:root[data-layout-variant="cockpit"] body::before {
content: "";
position: fixed;
inset: 0;
pointer-events: none;
z-index: 100;
background: repeating-linear-gradient(to bottom,
transparent 0px, transparent 2px,
rgba(64, 200, 255, 0.035) 3px, rgba(64, 200, 255, 0.035) 4px);
mix-blend-mode: screen;
}
```
The CSS is injected as a single scoped `<style data-hermes-theme-css>` tag on theme apply and cleaned up on theme switch. **Capped at 32 KiB per theme.**
### Built-in themes
Each built-in ships its own palette, typography, and layout — switching produces visible changes beyond color alone.
| Theme | Palette | Typography | Layout |
|-------|---------|------------|--------|
| **Hermes Teal** (`default`) | Dark teal + cream | System stack, 15px | 0.5rem radius, comfortable |
| **Hermes Teal (Large)** (`default-large`) | Same as default | System stack, 18px, line-height 1.65 | 0.5rem radius, spacious |
| **Midnight** (`midnight`) | Deep blue-violet | Inter + JetBrains Mono, 14px | 0.75rem radius, comfortable |
| **Ember** (`ember`) | Warm crimson + bronze | Spectral (serif) + IBM Plex Mono, 15px | 0.25rem radius, comfortable |
| **Mono** (`mono`) | Grayscale | IBM Plex Sans + IBM Plex Mono, 13px | 0 radius, compact |
| **Cyberpunk** (`cyberpunk`) | Neon green on black | Share Tech Mono everywhere, 14px | 0 radius, compact |
| **Rosé** (`rose`) | Pink + ivory | Fraunces (serif) + DM Mono, 16px | 1rem radius, spacious |
Themes that reference Google Fonts (all except Hermes Teal) load the stylesheet on demand — the first time you switch to them a `<link>` tag is injected into `<head>`.
### Full theme YAML reference
Every knob in one file — copy and trim what you don't need:
```yaml
# ~/.hermes/dashboard-themes/ocean.yaml
name: ocean
label: Ocean Deep
description: Deep sea blues with coral accents
# 3-layer palette (accepts {hex, alpha} or bare hex)
palette:
background:
hex: "#0a1628"
alpha: 1.0
midground:
hex: "#a8d0ff"
alpha: 1.0
foreground:
hex: "#ffffff"
alpha: 0.0
warmGlow: "rgba(255, 107, 107, 0.35)"
noiseOpacity: 0.7
typography:
fontSans: "Poppins, system-ui, sans-serif"
fontMono: "Fira Code, ui-monospace, monospace"
fontDisplay: "Poppins, system-ui, sans-serif" # optional
fontUrl: "https://fonts.googleapis.com/css2?family=Poppins:wght@400;500;600&family=Fira+Code:wght@400;500&display=swap"
baseSize: "15px"
lineHeight: "1.6"
letterSpacing: "-0.003em"
layout:
radius: "0.75rem"
density: comfortable
layoutVariant: standard # standard | cockpit | tiled
assets:
bg: "https://example.com/ocean-bg.jpg"
hero: "/my-images/kraken.png"
crest: "/my-images/anchor.svg"
logo: "/my-images/logo.png"
custom:
pattern: "/my-images/waves.svg"
componentStyles:
card:
boxShadow: "inset 0 0 0 1px rgba(168, 208, 255, 0.18)"
header:
background: "linear-gradient(180deg, rgba(10, 22, 40, 0.95), rgba(5, 9, 26, 0.9))"
colorOverrides:
destructive: "#ff6b6b"
ring: "#ff6b6b"
customCSS: |
/* Any additional selector-level tweaks */
```
Refresh the dashboard after creating the file. Switch themes live from the header bar — click the palette icon. Selection persists to `config.yaml` under `dashboard.theme` and is restored on reload.
---
## Plugins
A dashboard plugin is a directory with a `manifest.json`, a pre-built JS bundle, and optionally a CSS file and a Python file with FastAPI routes. Plugins live next to other Hermes plugins in `~/.hermes/plugins/<name>/` — the dashboard extension is a `dashboard/` subfolder inside that plugin directory, so one plugin can extend both the CLI/gateway and the dashboard from a single install.
Plugins don't bundle React or UI components. They use the **Plugin SDK** exposed on `window.__HERMES_PLUGIN_SDK__`. This keeps plugin bundles tiny (typically a few KB) and avoids version conflicts.
### Quick start — your first plugin
Create the directory structure:
```bash
mkdir -p ~/.hermes/plugins/my-plugin/dashboard/dist
```
Write the manifest:
```json
// ~/.hermes/plugins/my-plugin/dashboard/manifest.json
{
"name": "my-plugin",
"label": "My Plugin",
"icon": "Sparkles",
"version": "1.0.0",
"tab": {
"path": "/my-plugin",
"position": "after:skills"
},
"entry": "dist/index.js"
}
```
Write the JS bundle (a plain IIFE — no build step needed):
```javascript
// ~/.hermes/plugins/my-plugin/dashboard/dist/index.js
(function () {
"use strict";
const SDK = window.__HERMES_PLUGIN_SDK__;
const { React } = SDK;
const { Card, CardHeader, CardTitle, CardContent } = SDK.components;
function MyPage() {
return React.createElement(Card, null,
React.createElement(CardHeader, null,
React.createElement(CardTitle, null, "My Plugin"),
),
React.createElement(CardContent, null,
React.createElement("p", { className: "text-sm text-muted-foreground" },
"Hello from my custom dashboard tab.",
),
),
);
}
window.__HERMES_PLUGINS__.register("my-plugin", MyPage);
})();
```
Refresh the dashboard — your tab appears in the nav bar, after **Skills**.
:::tip Skip React.createElement
If you prefer JSX, use any bundler (esbuild, Vite, rollup) with React as an external and IIFE output. The only hard requirement is that the final file is a single JS file loadable via `<script>`. React is never bundled; it comes from `SDK.React`.
:::
### Directory layout
```
~/.hermes/plugins/my-plugin/
├── plugin.yaml # optional — existing CLI/gateway plugin manifest
├── __init__.py # optional — existing CLI/gateway hooks
└── dashboard/ # dashboard extension
├── manifest.json # required — tab config, icon, entry point
├── dist/
│ ├── index.js # required — pre-built JS bundle (IIFE)
│ └── style.css # optional — custom CSS
└── plugin_api.py # optional — backend API routes (FastAPI)
```
A single plugin directory can carry three orthogonal extensions:
- `plugin.yaml` + `__init__.py` — CLI/gateway plugin ([see plugins page](./plugins)).
- `dashboard/manifest.json` + `dashboard/dist/index.js` — dashboard UI plugin.
- `dashboard/plugin_api.py` — dashboard backend routes.
None of them are required; include only the layers you need.
### Manifest reference
```json
{
"name": "my-plugin",
"label": "My Plugin",
"description": "What this plugin does",
"icon": "Sparkles",
"version": "1.0.0",
"tab": {
"path": "/my-plugin",
"position": "after:skills",
"override": "/",
"hidden": false
},
"slots": ["sidebar", "header-left"],
"entry": "dist/index.js",
"css": "dist/style.css",
"api": "plugin_api.py"
}
```
| Field | Required | Description |
|-------|----------|-------------|
| `name` | Yes | Unique plugin identifier. Lowercase, hyphens ok. Used in URLs and registration. |
| `label` | Yes | Display name shown in the nav tab. |
| `description` | No | Short description (shown in dashboard admin surfaces). |
| `icon` | No | Lucide icon name. Defaults to `Puzzle`. Unknown names fall back to `Puzzle`. |
| `version` | No | Semver string. Defaults to `0.0.0`. |
| `tab.path` | Yes | URL path for the tab (e.g. `/my-plugin`). |
| `tab.position` | No | Where to insert the tab. `"end"` (default), `"after:<path>"`, or `"before:<path>"` — value after the colon is the **path segment** of the target tab (no leading slash). Examples: `"after:skills"`, `"before:config"`. |
| `tab.override` | No | Set to a built-in route path (`"/"`, `"/sessions"`, `"/config"`, ...) to **replace** that page instead of adding a new tab. See [Replacing built-in pages](#replacing-built-in-pages-taboverride). |
| `tab.hidden` | No | When true, register the component and any slots without adding a tab to the nav. Used by slot-only plugins. See [Slot-only plugins](#slot-only-plugins-tabhidden). |
| `slots` | No | Named shell slots this plugin populates. **Documentation aid only** — actual registration happens from the JS bundle via `registerSlot()`. Listing slots here makes discovery surfaces more informative. |
| `entry` | Yes | Path to the JS bundle relative to `dashboard/`. Defaults to `dist/index.js`. |
| `css` | No | Path to a CSS file to inject as a `<link>` tag. |
| `api` | No | Path to a Python file with FastAPI routes. Mounted at `/api/plugins/<name>/`. |
#### Available icons
Plugins use Lucide icon names. The dashboard maps these by name — unknown names silently fall back to `Puzzle`.
Currently mapped: `Activity`, `BarChart3`, `Clock`, `Code`, `Database`, `Eye`, `FileText`, `Globe`, `Heart`, `KeyRound`, `MessageSquare`, `Package`, `Puzzle`, `Settings`, `Shield`, `Sparkles`, `Star`, `Terminal`, `Wrench`, `Zap`.
Need a different icon? Open a PR to `web/src/App.tsx`'s `ICON_MAP` — pure additive change.
### The Plugin SDK
Everything a plugin needs is on `window.__HERMES_PLUGIN_SDK__`. Plugins should never import React directly.
```javascript
const SDK = window.__HERMES_PLUGIN_SDK__;
// React + hooks
SDK.React // the React instance
SDK.hooks.useState
SDK.hooks.useEffect
SDK.hooks.useCallback
SDK.hooks.useMemo
SDK.hooks.useRef
SDK.hooks.useContext
SDK.hooks.createContext
// UI components (shadcn/ui primitives)
SDK.components.Card
SDK.components.CardHeader
SDK.components.CardTitle
SDK.components.CardContent
SDK.components.Badge
SDK.components.Button
SDK.components.Input
SDK.components.Label
SDK.components.Select
SDK.components.SelectOption
SDK.components.Separator
SDK.components.Tabs
SDK.components.TabsList
SDK.components.TabsTrigger
SDK.components.PluginSlot // render a named slot (useful for nested plugin UIs)
// Hermes API client + raw fetcher
SDK.api // typed client — getStatus, getSessions, getConfig, ...
SDK.fetchJSON // raw fetch for custom endpoints (plugin-registered routes)
// Utilities
SDK.utils.cn // Tailwind class merger (clsx + twMerge)
SDK.utils.timeAgo // "5m ago" from unix timestamp
SDK.utils.isoTimeAgo // "5m ago" from ISO string
// Hooks
SDK.useI18n // i18n hook for multi-language plugins
```
#### Calling your plugin's backend
```javascript
SDK.fetchJSON("/api/plugins/my-plugin/data")
.then((data) => console.log(data))
.catch((err) => console.error("API call failed:", err));
```
`fetchJSON` injects the session auth token, surfaces errors as thrown exceptions, and parses JSON automatically.
#### Calling built-in Hermes endpoints
```javascript
// Agent status
SDK.api.getStatus().then((s) => console.log("Version:", s.version));
// Recent sessions
SDK.api.getSessions(10).then((resp) => console.log(resp.sessions.length));
```
See [Web Dashboard → REST API](./web-dashboard#rest-api) for the full list.
### Shell slots
Slots let a plugin inject components into named locations of the app shell — the cockpit sidebar, the header, the footer, an overlay layer — without claiming a whole tab. Multiple plugins can populate the same slot; they render stacked in registration order.
Register from inside the plugin bundle:
```javascript
window.__HERMES_PLUGINS__.registerSlot("my-plugin", "sidebar", MySidebar);
window.__HERMES_PLUGINS__.registerSlot("my-plugin", "header-left", MyCrest);
```
#### Slot catalogue
**Shell-wide slots** (render anywhere in the app chrome):
| Slot | Location |
|------|----------|
| `backdrop` | Inside the `<Backdrop />` layer stack, above the noise layer. |
| `header-left` | Before the Hermes brand in the top bar. |
| `header-right` | Before the theme/language switchers in the top bar. |
| `header-banner` | Full-width strip below the nav. |
| `sidebar` | Cockpit sidebar rail — **only rendered when `layoutVariant === "cockpit"`**. |
| `pre-main` | Above the route outlet (inside `<main>`). |
| `post-main` | Below the route outlet (inside `<main>`). |
| `footer-left` | Footer cell content (replaces default). |
| `footer-right` | Footer cell content (replaces default). |
| `overlay` | Fixed-position layer above everything else. Useful for chrome (scanlines, vignettes) `customCSS` can't achieve alone. |
**Page-scoped slots** (render only on the named built-in page — use these to inject widgets, cards, or toolbars into an existing page without overriding the whole route):
| Slot | Where it renders |
|------|------------------|
| `sessions:top` / `sessions:bottom` | Top / bottom of the `/sessions` page. |
| `analytics:top` / `analytics:bottom` | Top / bottom of the `/analytics` page. |
| `logs:top` / `logs:bottom` | Top (above filter toolbar) / bottom (below log viewer) of `/logs`. |
| `cron:top` / `cron:bottom` | Top / bottom of the `/cron` page. |
| `skills:top` / `skills:bottom` | Top / bottom of the `/skills` page. |
| `config:top` / `config:bottom` | Top / bottom of the `/config` page. |
| `env:top` / `env:bottom` | Top / bottom of the `/env` (Keys) page. |
| `docs:top` / `docs:bottom` | Top (above the iframe) / bottom of `/docs`. |
| `chat:top` / `chat:bottom` | Top / bottom of `/chat` (only active when embedded chat is enabled). |
Example — add a banner card to the top of the Sessions page:
```javascript
function PinnedSessionsBanner() {
return React.createElement(Card, null,
React.createElement(CardContent, { className: "py-2 text-xs" },
"Pinned note injected by my-plugin"),
);
}
window.__HERMES_PLUGINS__.registerSlot("my-plugin", "sessions:top", PinnedSessionsBanner);
```
Combine page-scoped slots with `tab.hidden: true` if your plugin only augments existing pages and doesn't need a sidebar tab of its own.
The shell only renders `<PluginSlot name="..." />` for the slots above. Additional names are accepted by the registry for nested plugin UIs — a plugin can expose its own slots via `SDK.components.PluginSlot`.
#### Re-registration and HMR
If the same `(plugin, slot)` pair is registered twice, the later call replaces the earlier one — this matches how React HMR expects plugin re-mounts to behave.
### Replacing built-in pages (`tab.override`)
Setting `tab.override` to a built-in route path makes the plugin's component replace that page instead of adding a new tab. Useful when a theme wants a custom home page (`/`) but wants to keep the rest of the dashboard intact.
```json
{
"name": "my-home",
"label": "Home",
"tab": {
"path": "/my-home",
"override": "/",
"position": "end"
},
"entry": "dist/index.js"
}
```
With `override` set:
- The original page component at `/` is removed from the router.
- Your plugin renders at `/` instead.
- No nav tab is added for `tab.path` (the override is the point).
Only one plugin can override a given path. If two plugins claim the same override, the first wins and the second is ignored with a dev-mode warning.
If you only need to add a card or toolbar to an existing page without taking it over, use [page-scoped slots](#augmenting-built-in-pages-page-scoped-slots) instead.
### Augmenting built-in pages (page-scoped slots)
Full replacement via `tab.override` is heavy — your plugin now owns the entire page, including any future updates we ship to it. Most of the time you just want to add a banner, card, or toolbar to an existing page. That's what **page-scoped slots** are for.
Every built-in page exposes `<page>:top` and `<page>:bottom` slots rendered at the top and bottom of its content area. Your plugin populates one by calling `registerSlot()` — the built-in page keeps working normally, and your component renders alongside it.
Available slots: `sessions:*`, `analytics:*`, `logs:*`, `cron:*`, `skills:*`, `config:*`, `env:*`, `docs:*`, `chat:*` (each with `:top` and `:bottom`). See the full catalogue in [Shell slots → Slot catalogue](#slot-catalogue).
Minimal example — pin a banner to the top of the Sessions page:
```json
// ~/.hermes/plugins/session-notes/dashboard/manifest.json
{
"name": "session-notes",
"label": "Session Notes",
"tab": { "path": "/session-notes", "hidden": true },
"slots": ["sessions:top"],
"entry": "dist/index.js"
}
```
```javascript
// ~/.hermes/plugins/session-notes/dashboard/dist/index.js
(function () {
const SDK = window.__HERMES_PLUGIN_SDK__;
const { React } = SDK;
const { Card, CardContent } = SDK.components;
function Banner() {
return React.createElement(Card, null,
React.createElement(CardContent, { className: "py-2 text-xs" },
"Remember to label important sessions before archiving."),
);
}
// Placeholder for the hidden tab.
window.__HERMES_PLUGINS__.register("session-notes", function () { return null; });
// The real work.
window.__HERMES_PLUGINS__.registerSlot("session-notes", "sessions:top", Banner);
})();
```
Key points:
- `tab.hidden: true` keeps the plugin out of the sidebar — it has no standalone page.
- The `slots` manifest field is documentation only. The actual binding happens in the JS bundle via `registerSlot()`.
- Multiple plugins can claim the same page-scoped slot. They render stacked in registration order.
- Zero footprint when no plugin registers: the built-in page renders exactly as before.
A reference plugin (`example-dashboard` in [`hermes-example-plugins`](https://github.com/NousResearch/hermes-example-plugins/tree/main/example-dashboard)) ships a live demo that injects a banner into `sessions:top` — install it to see the pattern end-to-end.
### Slot-only plugins (`tab.hidden`)
When `tab.hidden: true`, the plugin registers its component (for direct URL visits) and any slots, but never adds a tab to the navigation. Used by plugins that only exist to inject into slots — a header crest, a sidebar HUD, an overlay.
```json
{
"name": "header-crest",
"label": "Header Crest",
"tab": {
"path": "/header-crest",
"position": "end",
"hidden": true
},
"slots": ["header-left"],
"entry": "dist/index.js"
}
```
The bundle still calls `register()` with a placeholder component (good practice in case someone hits the URL directly) and then `registerSlot()` to do the real work.
### Backend API routes
Plugins can register FastAPI routes by setting `api` in the manifest. Create the file and export a `router`:
```python
# ~/.hermes/plugins/my-plugin/dashboard/plugin_api.py
from fastapi import APIRouter
router = APIRouter()
@router.get("/data")
async def get_data():
return {"items": ["one", "two", "three"]}
@router.post("/action")
async def do_action(body: dict):
return {"ok": True, "received": body}
```
Routes are mounted under `/api/plugins/<name>/`, so the above becomes:
- `GET /api/plugins/my-plugin/data`
- `POST /api/plugins/my-plugin/action`
Plugin API routes bypass session-token authentication since the dashboard server binds to localhost by default. **Don't expose the dashboard on a public interface with `--host 0.0.0.0` if you run untrusted plugins** — their routes become reachable too.
#### Accessing Hermes internals
Backend routes run inside the dashboard process, so they can import from the hermes-agent codebase directly:
```python
from fastapi import APIRouter
from hermes_state import SessionDB
from hermes_cli.config import load_config
router = APIRouter()
@router.get("/session-count")
async def session_count():
db = SessionDB()
try:
count = len(db.list_sessions(limit=9999))
return {"count": count}
finally:
db.close()
@router.get("/config-snapshot")
async def config_snapshot():
cfg = load_config()
return {"model": cfg.get("model", {})}
```
### Custom CSS per plugin
If your plugin needs styles beyond Tailwind classes and inline `style=`, add a CSS file and reference it in the manifest:
```json
{
"css": "dist/style.css"
}
```
The file is injected as a `<link>` tag on plugin load. Use specific class names to avoid conflicts with the dashboard's styles, and reference the dashboard's CSS vars to stay theme-aware:
```css
/* dist/style.css */
.my-plugin-chart {
border: 1px solid var(--color-border);
background: var(--color-card);
color: var(--color-card-foreground);
padding: 1rem;
}
.my-plugin-chart:hover {
border-color: var(--color-ring);
}
```
The dashboard exposes every shadcn token as `--color-*` plus theme extras (`--theme-asset-*`, `--component-<bucket>-*`, `--radius`, `--spacing-mul`). Reference those and your plugin automatically reskins with the active theme.
### Plugin discovery & reload
The dashboard scans three directories for `dashboard/manifest.json`:
| Priority | Directory | Source label |
|----------|-----------|--------------|
| 1 (wins on conflict) | `~/.hermes/plugins/<name>/dashboard/` | `user` |
| 2 | `<repo>/plugins/memory/<name>/dashboard/` | `bundled` |
| 2 | `<repo>/plugins/<name>/dashboard/` | `bundled` |
| 3 | `./.hermes/plugins/<name>/dashboard/` | `project` — only when `HERMES_ENABLE_PROJECT_PLUGINS` is set |
Discovery results are cached per dashboard process. After adding a new plugin, either:
```bash
# Force a rescan without restart
curl http://127.0.0.1:9119/api/dashboard/plugins/rescan
```
…or restart `hermes dashboard`.
#### Plugin load lifecycle
1. Dashboard loads. `main.tsx` exposes the SDK on `window.__HERMES_PLUGIN_SDK__` and the registry on `window.__HERMES_PLUGINS__`.
2. `App.tsx` calls `usePlugins()` → fetches `GET /api/dashboard/plugins`.
3. For each manifest: CSS `<link>` is injected (if declared), then a `<script>` tag loads the JS bundle.
4. The plugin's IIFE runs and calls `window.__HERMES_PLUGINS__.register(name, Component)` — and optionally `.registerSlot(name, slot, Component)` for each slot.
5. The dashboard resolves the registered component against the manifest, adds the tab to navigation (unless `hidden`), and mounts the component as a route.
Plugins have up to **2 seconds** after their script loads to call `register()`. After that the dashboard stops waiting and finishes initial render. If a plugin later registers, it still appears — the nav is reactive.
If a plugin's script fails to load (404, syntax error, exception during IIFE), the dashboard logs a warning to the browser console and continues without it.
---
## Combined theme + plugin demo
The [`strike-freedom-cockpit`](https://github.com/NousResearch/hermes-example-plugins/tree/main/strike-freedom-cockpit) plugin (companion repo `hermes-example-plugins`) is a complete reskin demo. It pairs a theme YAML with a slot-only plugin to produce a cockpit-style HUD without forking the dashboard.
**What it demonstrates:**
- A full theme using palette, typography, `fontUrl`, `layoutVariant: cockpit`, `assets`, `componentStyles` (notched card corners, gradient backgrounds), `colorOverrides`, and `customCSS` (scanline overlay).
- A slot-only plugin (`tab.hidden: true`) that registers into three slots:
- `sidebar` — an MS-STATUS panel with live telemetry bars driven by `SDK.api.getStatus()`.
- `header-left` — a faction crest that reads `--theme-asset-crest` from the active theme.
- `footer-right` — a custom tagline replacing the default org line.
- The plugin reads theme-supplied artwork via CSS vars, so swapping themes changes the hero/crest without plugin code changes.
**Install:**
```bash
git clone https://github.com/NousResearch/hermes-example-plugins.git
# Theme
cp hermes-example-plugins/strike-freedom-cockpit/theme/strike-freedom.yaml \
~/.hermes/dashboard-themes/
# Plugin
cp -r hermes-example-plugins/strike-freedom-cockpit ~/.hermes/plugins/
```
Open the dashboard, pick **Strike Freedom** from the theme switcher. The cockpit sidebar appears, the crest shows in the header, the tagline replaces the footer. Switch back to **Hermes Teal** and the plugin remains installed but invisible (the `sidebar` slot only renders under the `cockpit` layout variant).
Read the plugin source (`strike-freedom-cockpit/dashboard/dist/index.js` in the companion repo) to see how it reads CSS vars, guards against older dashboards without slot support, and registers three slots from one bundle.
---
## API reference
### Theme endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/dashboard/themes` | GET | List available themes + active name. Built-ins return `{name, label, description}`; user themes also include a `definition` field with the full normalised theme object. |
| `/api/dashboard/theme` | PUT | Set active theme. Body: `{"name": "midnight"}`. Persists to `config.yaml` under `dashboard.theme`. |
### Plugin endpoints
| Endpoint | Method | Description |
|----------|--------|-------------|
| `/api/dashboard/plugins` | GET | List discovered plugins (with manifests, minus internal fields). |
| `/api/dashboard/plugins/rescan` | GET | Force re-scan the plugin directories without restarting. |
| `/dashboard-plugins/<name>/<path>` | GET | Serve static assets from a plugin's `dashboard/` directory. Path traversal is blocked. |
| `/api/plugins/<name>/*` | * | Plugin-registered backend routes. |
### SDK on `window`
| Global | Type | Provider |
|--------|------|----------|
| `window.__HERMES_PLUGIN_SDK__` | object | `registry.ts` — React, hooks, UI components, API client, utils. |
| `window.__HERMES_PLUGINS__.register(name, Component)` | function | Register a plugin's main component. |
| `window.__HERMES_PLUGINS__.registerSlot(name, slot, Component)` | function | Register into a named shell slot. |
---
## Troubleshooting
**My theme doesn't appear in the picker.**
Check that the file is in `~/.hermes/dashboard-themes/` and ends in `.yaml` or `.yml`. Refresh the page. Run `curl http://127.0.0.1:9119/api/dashboard/themes` — your theme should be in the response. If the YAML has a parse error, the dashboard logs to `errors.log` under `~/.hermes/logs/`.
**My plugin's tab doesn't show up.**
1. Check the manifest is at `~/.hermes/plugins/<name>/dashboard/manifest.json` (note the `dashboard/` subdirectory).
2. `curl http://127.0.0.1:9119/api/dashboard/plugins/rescan` to force re-discovery.
3. Open browser dev tools → Network — confirm `manifest.json`, `index.js`, and any CSS loaded without 404s.
4. Open browser dev tools → Console — look for errors during the IIFE or `window.__HERMES_PLUGINS__ is undefined` (indicates the SDK didn't initialize, usually a React render crash earlier).
5. Verify your bundle calls `window.__HERMES_PLUGINS__.register(...)` with the **same name** as `manifest.json:name`.
**Slot-registered components don't render.**
The `sidebar` slot only renders when the active theme has `layoutVariant: cockpit`. Other slots always render. If you're registering into a slot with no hits, add `console.log` inside `registerSlot` to confirm the plugin bundle ran at all.
**Plugin backend routes return 404.**
1. Confirm the manifest has `"api": "plugin_api.py"` pointing to an existing file inside `dashboard/`.
2. Restart `hermes dashboard` — plugin API routes are mounted once at startup, **not** on rescan.
3. Check that `plugin_api.py` exports a module-level `router = APIRouter()`. Other export names are not picked up.
4. Tail `~/.hermes/logs/errors.log` for `Failed to load plugin <name> API routes` — import errors are logged there.
**Theme change drops my color overrides.**
`colorOverrides` are scoped to the active theme and cleared on theme switch — that's by design. If you want overrides that persist, put them in your theme's YAML, not in the live switcher.
**Theme customCSS gets truncated.**
The `customCSS` block is capped at 32 KiB per theme. Split large stylesheets across multiple themes, or switch to a plugin that injects a full stylesheet via its `css` field (no size cap).
**I want to ship a plugin on PyPI.**
Dashboard plugins are installed by directory layout, not by pip entry point. The cleanest distribution path today is a git repo the user clones into `~/.hermes/plugins/`. A pip-based installer for dashboard plugins is not currently wired up.
website/docs/user-guide/features/fallback-providers.md
+413
View File
@@ -0,0 +1,413 @@
---
title: Fallback Providers
description: Configure automatic failover to backup LLM providers when your primary model is unavailable.
sidebar_label: Fallback Providers
sidebar_position: 8
---
# Fallback Providers
Hermes Agent has three layers of resilience that keep your sessions running when providers hit issues:
1. **[Credential pools](./credential-pools.md)** — rotate across multiple API keys for the *same* provider (tried first)
2. **Primary model fallback** — automatically switches to a *different* provider:model when your main model fails
3. **Auxiliary task fallback** — independent provider resolution for side tasks like vision, compression, and web extraction
Credential pools handle same-provider rotation (e.g., multiple OpenRouter keys). This page covers cross-provider fallback. Both are optional and work independently.
## Primary Model Fallback
When your main LLM provider encounters errors — rate limits, server overload, auth failures, connection drops — Hermes can automatically switch to a backup provider:model pair mid-session without losing your conversation.
### Configuration
The easiest path is the interactive manager:
```bash
hermes fallback
```
`hermes fallback` reuses the provider picker from `hermes model` — same provider list, same credential prompts, same validation. Use the subcommands `add`, `list` (alias `ls`), `remove` (alias `rm`), and `clear` to manage the chain. Changes persist under the top-level `fallback_providers:` list in `config.yaml`.
If you'd rather edit the YAML directly, add a top-level `fallback_providers` list to `~/.hermes/config.yaml`:
```yaml
fallback_providers:
- provider: openrouter
model: anthropic/claude-sonnet-4
```
Each entry requires both `provider` and `model`. Entries missing either field are ignored.
:::note `fallback_model` vs `fallback_providers`
`fallback_providers` (plural, list) is the current config shape and supports multiple fallbacks tried in order. `fallback_model` (singular) is the legacy single-fallback key — Hermes still honors it for back-compat, but `hermes fallback` writes the current `fallback_providers` key and migrates legacy config on write. When both are set, `fallback_providers` takes priority.
:::
### Supported Providers
| Provider | Value | Requirements |
|----------|-------|-------------|
| OpenRouter | `openrouter` | `OPENROUTER_API_KEY` |
| Nous Portal | `nous` | `hermes setup --portal` (fresh) or `hermes auth add nous` (OAuth) |
| OpenAI Codex | `openai-codex` | `hermes model` (ChatGPT OAuth) |
| GitHub Copilot | `copilot` | `COPILOT_GITHUB_TOKEN`, `GH_TOKEN`, or `GITHUB_TOKEN` |
| GitHub Copilot ACP | `copilot-acp` | External process (editor integration) |
| Anthropic | `anthropic` | `ANTHROPIC_API_KEY` or Claude Code credentials |
| z.ai / GLM | `zai` | `GLM_API_KEY` |
| Kimi / Moonshot | `kimi-coding` | `KIMI_API_KEY` |
| MiniMax | `minimax` | `MINIMAX_API_KEY` |
| MiniMax (China) | `minimax-cn` | `MINIMAX_CN_API_KEY` |
| DeepSeek | `deepseek` | `DEEPSEEK_API_KEY` |
| NVIDIA NIM | `nvidia` | `NVIDIA_API_KEY` (optional: `NVIDIA_BASE_URL`) |
| GMI Cloud | `gmi` | `GMI_API_KEY` (optional: `GMI_BASE_URL`) |
| StepFun | `stepfun` | `STEPFUN_API_KEY` (optional: `STEPFUN_BASE_URL`) |
| Ollama Cloud | `ollama-cloud` | `OLLAMA_API_KEY` |
| Google Gemini (OAuth) | `google-gemini-cli` | `hermes model` (Google OAuth; optional: `HERMES_GEMINI_PROJECT_ID`) |
| Google AI Studio | `gemini` | `GOOGLE_API_KEY` (alias: `GEMINI_API_KEY`) |
| xAI (Grok) | `xai` (alias `grok`) | `XAI_API_KEY` (optional: `XAI_BASE_URL`) |
| xAI Grok OAuth (SuperGrok) | `xai-oauth` (alias `grok-oauth`) | `hermes model` → xAI Grok OAuth (browser login; SuperGrok subscription) |
| AWS Bedrock | `bedrock` | Standard boto3 auth (`AWS_REGION` + `AWS_PROFILE` or `AWS_ACCESS_KEY_ID`) |
| Qwen Portal (OAuth) | `qwen-oauth` | `hermes model` (Qwen Portal OAuth; optional: `HERMES_QWEN_BASE_URL`) |
| MiniMax (OAuth) | `minimax-oauth` | `hermes model` (MiniMax portal OAuth) |
| OpenCode Zen | `opencode-zen` | `OPENCODE_ZEN_API_KEY` |
| OpenCode Go | `opencode-go` | `OPENCODE_GO_API_KEY` |
| Kilo Code | `kilocode` | `KILOCODE_API_KEY` |
| Xiaomi MiMo | `xiaomi` | `XIAOMI_API_KEY` |
| Arcee AI | `arcee` | `ARCEEAI_API_KEY` |
| GMI Cloud | `gmi` | `GMI_API_KEY` |
| Alibaba / DashScope | `alibaba` | `DASHSCOPE_API_KEY` |
| Alibaba Coding Plan | `alibaba-coding-plan` | `ALIBABA_CODING_PLAN_API_KEY` (falls back to `DASHSCOPE_API_KEY`) |
| Kimi / Moonshot (China) | `kimi-coding-cn` | `KIMI_CN_API_KEY` |
| StepFun | `stepfun` | `STEPFUN_API_KEY` |
| Tencent TokenHub | `tencent-tokenhub` | `TOKENHUB_API_KEY` |
| Microsoft Foundry | `azure-foundry` | `AZURE_FOUNDRY_API_KEY` + `AZURE_FOUNDRY_BASE_URL` |
| LM Studio (local) | `lmstudio` | `LM_API_KEY` (or none for local) + `LM_BASE_URL` |
| Hugging Face | `huggingface` | `HF_TOKEN` |
| Custom endpoint | `custom` | `base_url` + `key_env` (see below) |
### Custom Endpoint Fallback
For a custom OpenAI-compatible endpoint, add `base_url` and optionally `key_env`:
```yaml
fallback_providers:
- provider: custom
model: my-local-model
base_url: http://localhost:8000/v1
key_env: MY_LOCAL_KEY # env var name containing the API key
```
### When Fallback Triggers
The fallback activates automatically when the primary model fails with:
- **Rate limits** (HTTP 429) — after exhausting retry attempts
- **Server errors** (HTTP 500, 502, 503) — after exhausting retry attempts
- **Auth failures** (HTTP 401, 403) — immediately (no point retrying)
- **Not found** (HTTP 404) — immediately
- **Invalid responses** — when the API returns malformed or empty responses repeatedly
When triggered, Hermes:
1. Resolves credentials for the fallback provider
2. Builds a new API client
3. Swaps the model, provider, and client in-place
4. Resets the retry counter and continues the conversation
The switch is seamless — your conversation history, tool calls, and context are preserved. The agent continues from exactly where it left off, just using a different model.
:::info Per-Turn, Not Per-Session
Fallback is **turn-scoped**: each new user message starts with the primary model restored. If the primary fails mid-turn, fallback activates for that turn only. On the next message, Hermes tries the primary again. Within a single turn, fallback activates at most once — if the fallback also fails, normal error handling takes over (retries, then error message). This prevents cascading failover loops within a turn while giving the primary model a fresh chance every turn.
:::
### Examples
**OpenRouter as fallback for Anthropic native:**
```yaml
model:
provider: anthropic
default: claude-sonnet-4-6
fallback_providers:
- provider: openrouter
model: anthropic/claude-sonnet-4
```
**Nous Portal as fallback for OpenRouter:**
```yaml
model:
provider: openrouter
default: anthropic/claude-opus-4
fallback_providers:
- provider: nous
model: nous-hermes-3
```
**Local model as fallback for cloud:**
```yaml
fallback_providers:
- provider: custom
model: llama-3.1-70b
base_url: http://localhost:8000/v1
key_env: LOCAL_API_KEY
```
**Codex OAuth as fallback:**
```yaml
fallback_providers:
- provider: openai-codex
model: gpt-5.3-codex
```
### Where Fallback Works
| Context | Fallback Supported |
|---------|-------------------|
| CLI sessions | ✔ |
| Messaging gateway (Telegram, Discord, etc.) | ✔ |
| Subagent delegation | ✔ (subagents inherit the parent fallback chain) |
| Cron jobs | ✔ (cron agents inherit configured fallback providers) |
| Auxiliary tasks (vision, compression) | ✘ (use their own provider chain — see below) |
:::tip
There are no environment variables for the primary fallback chain — configure it exclusively through `config.yaml` or `hermes fallback`. This is intentional: fallback configuration is a deliberate choice, not something a stale shell export should override.
:::
---
## Auxiliary Task Fallback
Hermes uses separate lightweight models for side tasks. Each task has its own provider resolution chain that acts as a built-in fallback system.
### Tasks with Independent Provider Resolution
| Task | What It Does | Config Key |
|------|-------------|-----------|
| Vision | Image analysis, browser screenshots | `auxiliary.vision` |
| Web Extract | Web page summarization | `auxiliary.web_extract` |
| Compression | Context compression summaries | `auxiliary.compression` |
| Skills Hub | Skill search and discovery | `auxiliary.skills_hub` |
| MCP | MCP helper operations | `auxiliary.mcp` |
| Approval | Smart command-approval classification | `auxiliary.approval` |
| Title Generation | Session title summaries | `auxiliary.title_generation` |
| Triage Specifier | `hermes kanban specify` / dashboard ✨ button — fleshes out a one-liner triage task into a real spec | `auxiliary.triage_specifier` |
### Auto-Detection Chain
When a task's provider is set to `"auto"` (the default), Hermes tries providers in order until one works:
**For text tasks (compression, web extract, etc.):**
```text
OpenRouter → Nous Portal → Custom endpoint → Codex OAuth →
API-key providers (z.ai, Kimi, MiniMax, Xiaomi MiMo, Hugging Face, Anthropic) → give up
```
**For vision tasks:**
```text
Main provider (if vision-capable) → OpenRouter → Nous Portal →
Codex OAuth → Anthropic → Custom endpoint → give up
```
If the resolved provider fails at call time, Hermes also has an internal retry: if the provider is not OpenRouter and no explicit `base_url` is set, it tries OpenRouter as a last-resort fallback.
### Configuring Auxiliary Providers
Each task can be configured independently in `config.yaml`:
```yaml
auxiliary:
vision:
provider: "auto" # auto | openrouter | nous | codex | main | anthropic
model: "" # e.g. "openai/gpt-4o"
base_url: "" # direct endpoint (takes precedence over provider)
api_key: "" # API key for base_url
web_extract:
provider: "auto"
model: ""
compression:
provider: "auto"
model: ""
skills_hub:
provider: "auto"
model: ""
mcp:
provider: "auto"
model: ""
```
Every task above follows the same **provider / model / base_url** pattern. Context compression is configured under `auxiliary.compression`:
```yaml
auxiliary:
compression:
provider: main # Same provider options as other auxiliary tasks
model: google/gemini-3-flash-preview
base_url: null # Custom OpenAI-compatible endpoint
```
And the primary fallback chain uses:
```yaml
fallback_providers:
- provider: openrouter
model: anthropic/claude-sonnet-4
# base_url: http://localhost:8000/v1 # Optional custom endpoint
```
All three — auxiliary, compression, fallback — work the same way: set `provider` to pick who handles the request, `model` to pick which model, and `base_url` to point at a custom endpoint (overrides provider).
### Provider Options for Auxiliary Tasks
These options apply to `auxiliary:`, `compression:`, and `fallback_providers:` entries only — `"main"` is **not** a valid value for your top-level `model.provider`. For custom endpoints, use `provider: custom` in your `model:` section (see [AI Providers](/integrations/providers)).
| Provider | Description | Requirements |
|----------|-------------|-------------|
| `"auto"` | Try providers in order until one works (default) | At least one provider configured |
| `"openrouter"` | Force OpenRouter | `OPENROUTER_API_KEY` |
| `"nous"` | Force Nous Portal | `hermes auth` |
| `"codex"` | Force Codex OAuth | `hermes model` → Codex |
| `"main"` | Use whatever provider the main agent uses (auxiliary tasks only) | Active main provider configured |
| `"anthropic"` | Force Anthropic native | `ANTHROPIC_API_KEY` or Claude Code credentials |
### Direct Endpoint Override
For any auxiliary task, setting `base_url` bypasses provider resolution entirely and sends requests directly to that endpoint:
```yaml
auxiliary:
vision:
base_url: "http://localhost:1234/v1"
api_key: "local-key"
model: "qwen2.5-vl"
```
`base_url` takes precedence over `provider`. Hermes uses the configured `api_key` for authentication, falling back to `OPENAI_API_KEY` if not set. It does **not** reuse `OPENROUTER_API_KEY` for custom endpoints.
---
## Auxiliary Capacity-Error Fallback
When you set an explicit auxiliary provider (e.g. `auxiliary.vision.provider: glm`), Hermes treats that as your preferred choice — but if the provider literally cannot serve the request because of a **capacity error** (HTTP 402 payment required, HTTP 429 daily-quota exhaustion, connection failure), Hermes falls back through a layered chain instead of failing silently:
1. **Primary aux provider** — the one you configured (tried first, always)
2. **`auxiliary.<task>.fallback_chain`** — your per-task override list, if you wrote one
3. **Main agent provider + model** — last-resort safety net (always tried, even if you didn't write a chain)
4. **Warn + re-raise** — if every layer fails, Hermes logs `Auxiliary <task>: ... all fallbacks exhausted` at WARNING level and re-raises the original error
Transient HTTP 429 rate limits (`Retry-After: ...`) are treated as request constraints, not capacity problems — they respect your explicit provider choice and do **not** trigger the fallback ladder. Only daily/monthly quota exhaustion, payment errors, and connection failures bypass the explicit-provider gate.
For users on `provider: auto` (no explicit aux provider), the existing auto-detection chain runs in place of steps 2–3. Its first step is already the main agent model, so `auto` users get the same outcome with zero config.
### Optional: per-task fallback chain
If you want a different fallback ordering than "main agent model first", configure `fallback_chain` explicitly. Each entry needs at least `provider`; `model`, `base_url`, and `api_key` are optional.
```yaml
auxiliary:
vision:
provider: glm
model: glm-4v-flash
fallback_chain:
- provider: openrouter
model: google/gemini-3-flash-preview
- provider: nous
model: anthropic/claude-sonnet-4
compression:
provider: openrouter
fallback_chain:
- provider: openai
model: gpt-4o-mini
```
You do **not** need to configure `fallback_chain` to get fallback — the main-agent safety net runs regardless. Use it only when you specifically want a different order than the default.
### Provider quota errors that trigger fallback
Hermes recognizes these as capacity-equivalent to 402 credit exhaustion (not transient rate limits):
- Bedrock / LiteLLM: `Too many tokens per day`, `daily limit`, `tokens per day`
- Vertex AI / GCP: `quota exceeded`, `resource exhausted`, `RESOURCE_EXHAUSTED`
- Generic: `daily quota`, `quota_exceeded`
If your provider returns a different phrase for daily-quota exhaustion and Hermes doesn't trigger fallback, that's a bug — open an issue with the exact error string.
---
## Context Compression Fallback
Context compression uses the `auxiliary.compression` config block to control which model and provider handles summarization:
```yaml
auxiliary:
compression:
provider: "auto" # auto | openrouter | nous | main
model: "google/gemini-3-flash-preview"
```
:::info Legacy migration
Older configs with `compression.summary_model` / `compression.summary_provider` / `compression.summary_base_url` are automatically migrated to `auxiliary.compression.*` on first load (config version 17).
:::
If no provider is available for compression, Hermes drops middle conversation turns without generating a summary rather than failing the session.
---
## Delegation Provider Override
Subagents spawned by `delegate_task` inherit the parent agent's primary fallback chain. You can still route subagents to a different primary provider:model pair for cost optimization:
```yaml
delegation:
provider: "openrouter" # override provider for all subagents
model: "google/gemini-3-flash-preview" # override model
# base_url: "http://localhost:1234/v1" # or use a direct endpoint
# api_key: "local-key"
```
See [Subagent Delegation](/user-guide/features/delegation) for full configuration details.
---
## Cron Job Providers
Cron jobs inherit your configured `fallback_providers` chain (or legacy `fallback_model`) when they create an agent. To use a different primary provider for a cron job, configure `provider` and `model` overrides on the cron job itself:
```python
cronjob(
action="create",
schedule="every 2h",
prompt="Check server status",
provider="openrouter",
model="google/gemini-3-flash-preview"
)
```
See [Scheduled Tasks (Cron)](/user-guide/features/cron) for full configuration details.
---
## Summary
| Feature | Fallback Mechanism | Config Location |
|---------|-------------------|----------------|
| Main agent model | `fallback_providers` in config.yaml — per-turn failover on errors (primary restored each turn) | `fallback_providers:` (top-level list) |
| Auxiliary tasks (any) — auto users | Full auto-detection chain (main agent model first, then provider chain) on capacity errors | `auxiliary.<task>.provider: auto` |
| Auxiliary tasks (any) — explicit provider | `fallback_chain` (if set) → main agent model → warn + raise, on capacity errors only | `auxiliary.<task>.fallback_chain` |
| Vision | Layered (see above) + internal OpenRouter retry | `auxiliary.vision` |
| Web extraction | Layered (see above) + internal OpenRouter retry | `auxiliary.web_extract` |
| Context compression | Layered (see above); degrades to no-summary if all layers unavailable | `auxiliary.compression` |
| Skills hub | Layered (see above) | `auxiliary.skills_hub` |
| MCP helpers | Layered (see above) | `auxiliary.mcp` |
| Approval classification | Layered (see above) | `auxiliary.approval` |
| Title generation | Layered (see above) | `auxiliary.title_generation` |
| Triage specifier | Layered (see above) | `auxiliary.triage_specifier` |
| Delegation | Provider override only (no automatic fallback) | `delegation.provider` / `delegation.model` |
| Cron jobs | Per-job provider override only (no automatic fallback) | Per-job `provider` / `model` |
website/docs/user-guide/features/goals.md
+180
View File
@@ -0,0 +1,180 @@
---
sidebar_position: 16
title: "Persistent Goals"
description: "Set a standing goal and let Hermes keep working across turns until it's done. Our take on the Ralph loop."
---
# Persistent Goals (`/goal`)
`/goal` gives Hermes a standing objective that survives across turns. After every turn a lightweight judge model checks whether the goal is satisfied by the assistant's last response. If not, Hermes automatically feeds a continuation prompt back into the same session and keeps working — until the goal is achieved, you pause or clear it, or the turn budget runs out.
It's our take on the **Ralph loop**, directly inspired by [Codex CLI 0.128.0's `/goal`](https://github.com/openai/codex) by Eric Traut (OpenAI). The core idea — keep a goal alive across turns and don't stop until it's achieved — is theirs. The implementation here is independent and adapted to Hermes' architecture.
## When to use it
Use `/goal` for tasks where you want Hermes to iterate on its own without you re-prompting every turn:
- "Fix every lint error in `src/` and verify `ruff check` passes"
- "Port feature X from repo Y, including tests, and get CI green"
- "Investigate why session IDs sometimes drift on mid-run compression and write up a report"
- "Build a small CLI to rename files by their EXIF dates, then test it against the photos/ folder"
Tasks where the agent does one turn and stops don't need `/goal`. Tasks where *you'd otherwise have to say "keep going" three times* are where this shines.
## Quick start
```
/goal Fix every failing test in tests/hermes_cli/ and make sure scripts/run_tests.sh passes for that directory
```
What you'll see:
1. **Goal accepted** — `⊙ Goal set (20-turn budget): <your goal>`
2. **Turn 1 runs** — Hermes starts working as if you'd sent the goal as a normal message.
3. **Judge runs** — after the turn, the judge model decides `done` or `continue`.
4. **Loop fires if needed** — if `continue`, you'll see `↻ Continuing toward goal (1/20): <judge's reason>` and Hermes takes the next step automatically.
5. **Terminates** — eventually you see either `✓ Goal achieved: <reason>` or `⏸ Goal paused — N/20 turns used`.
## Commands
| Command | What it does |
|---|---|
| `/goal <text>` | Set (or replace) the standing goal. Kicks off the first turn immediately so you don't need to send a separate message. |
| `/goal` or `/goal status` | Show the current goal, its status, and turns used. |
| `/goal pause` | Stop the auto-continuation loop without clearing the goal. |
| `/goal resume` | Resume the loop (resets the turn counter back to zero). |
| `/goal clear` | Drop the goal entirely. |
Works identically on the CLI and every gateway platform (Telegram, Discord, Slack, Matrix, Signal, WhatsApp, SMS, iMessage, Webhook, API server, and the web dashboard).
## Adding criteria mid-goal: `/subgoal`
While a goal is active you can append extra acceptance criteria with `/subgoal <text>` without resetting the loop. Each call adds one numbered item to the goal's subgoal list; the **continuation prompt** the agent sees on the next turn includes the original goal plus an "Additional criteria the user added mid-loop" block, and the **judge prompt** is rewritten so the verdict must consider every subgoal — the goal isn't marked done until the original objective **and** every subgoal are met.
| Command | What it does |
|---|---|
| `/subgoal <text>` | Append a new criterion to the active goal. Requires an active `/goal`. |
| `/subgoal` (no args) | Show the current numbered subgoal list. |
| `/subgoal remove <N>` | Remove the Nth subgoal (1-based). |
| `/subgoal clear` | Drop every subgoal but keep the original goal intact. |
Subgoals are persisted alongside the goal in `SessionDB.state_meta`, so they survive `/resume`. Setting a new `/goal <text>` replaces the goal and clears the subgoal list; `/goal clear` does the same.
Use this when you start a loop ("fix the failing tests") and notice partway through that you also want it to "and add a regression test for the bug you just patched" — `/subgoal add a regression test` tightens the success criteria without breaking the running loop.
## Behavior details
### The judge
After every turn, Hermes calls an auxiliary model with:
- The standing goal text
- The agent's most recent final response (last ~4 KB of text)
- A system prompt telling the judge to reply with strict JSON: `{"done": <bool>, "reason": "<one-sentence rationale>"}`
The judge is deliberately conservative: it marks a goal `done` only when the response **explicitly** confirms the goal is complete, when the final deliverable is clearly produced, or when the goal is unachievable/blocked (treated as DONE with a block reason so we don't burn budget on impossible tasks).
### Fail-open semantics
If the judge errors (network blip, malformed response, unavailable aux client), Hermes treats the verdict as `continue` — a broken judge never wedges progress. The **turn budget** is the real backstop.
### Turn budget
Default is 20 continuation turns (`goals.max_turns` in `config.yaml`). When the budget is hit, Hermes auto-pauses and tells you exactly how to proceed:
```
⏸ Goal paused — 20/20 turns used. Use /goal resume to keep going, or /goal clear to stop.
```
`/goal resume` resets the counter to zero, so you can keep going in measured chunks.
### User messages always preempt
Any real message you send while a goal is active takes priority over the continuation loop. On the CLI your message lands in `_pending_input` ahead of the queued continuation; on the gateway it goes through the adapter FIFO the same way. The judge runs again after your turn — so if your message happens to complete the goal, the judge will catch it and stop.
### Mid-run safety (gateway)
While an agent is already running, `/goal status`, `/goal pause`, and `/goal clear` are safe to run — they only touch control-plane state and don't interrupt the current turn. Setting a **new** goal mid-run (`/goal <new text>`) is rejected with a message telling you to `/stop` first, so the old continuation can't race the new one.
### Persistence
Goal state lives in `SessionDB.state_meta` keyed by `goal:<session_id>`. That means `/resume` picks up right where you left off — set a goal, close your laptop, come back tomorrow, `/resume`, and the goal is still standing exactly as you left it (active, paused, or done).
### Prompt cache
The continuation prompt is a plain user-role message appended to history. It does **not** mutate the system prompt, swap toolsets, or touch the conversation in any way that invalidates Hermes' prompt cache. Running a 20-turn goal costs the same cache-wise as 20 turns of normal conversation.
## Configuration
Add to `~/.hermes/config.yaml`:
```yaml
goals:
# Max continuation turns before Hermes auto-pauses and asks you to
# /goal resume. Default 20. Lower this if you want tighter loops;
# raise it for long-running refactors.
max_turns: 20
```
### Choosing the judge model
The judge uses the `goal_judge` auxiliary task. By default it resolves to your main model (see [Auxiliary Models](/user-guide/configuration#auxiliary-models)). If you want to route the judge to a cheap fast model to keep costs down, add an override:
```yaml
auxiliary:
goal_judge:
provider: openrouter
model: google/gemini-3-flash-preview
```
The judge call is small (~200 output tokens) and runs once per turn, so a cheap fast model is usually the right call.
## Example walkthrough
```
You: /goal Create four files /tmp/note_{1..4}.txt, one per turn, each containing its number as text
⊙ Goal set (20-turn budget): Create four files /tmp/note_{1..4}.txt, one per turn, each containing its number as text
Hermes: Creating /tmp/note_1.txt now.
💻 echo "1" > /tmp/note_1.txt (0.1s)
I've created /tmp/note_1.txt with the content "1". I'll continue with the remaining files on the next turn as you specified.
↻ Continuing toward goal (1/20): Only 1 of 4 files has been created; 3 files remain.
Hermes: [Continuing toward your standing goal]
💻 echo "2" > /tmp/note_2.txt (0.1s)
Created /tmp/note_2.txt. Two more to go.
↻ Continuing toward goal (2/20): 2 of 4 files created; 2 remain.
Hermes: [Continuing toward your standing goal]
💻 echo "3" > /tmp/note_3.txt (0.1s)
Created /tmp/note_3.txt.
↻ Continuing toward goal (3/20): 3 of 4 files created; 1 remains.
Hermes: [Continuing toward your standing goal]
💻 echo "4" > /tmp/note_4.txt (0.1s)
All four files have been created: /tmp/note_1.txt through /tmp/note_4.txt, each containing its number.
✓ Goal achieved: All four files were created with the specified content, completing the goal.
You: _
```
Four turns, one `/goal` invocation, zero "keep going" prompts from you.
## When the judge gets it wrong
No judge is perfect. Two failure modes to watch for:
**False negative — judge says continue when the goal is actually done.** The turn budget catches this. You'll see `⏸ Goal paused` and can `/goal clear` or just send a new message.
**False positive — judge says done when work remains.** You'll see `✓ Goal achieved` but you know better. Send a follow-up message to continue, or re-set the goal more precisely: `/goal <more specific text>`. The judge's system prompt is deliberately conservative to make false positives rarer than false negatives.
If you find a judge verdict unconvincing, the reason text in the `↻ Continuing toward goal` or `✓ Goal achieved` line tells you exactly what the judge saw. That's usually enough to diagnose whether the goal text was ambiguous or the model's response was.
## Attribution
`/goal` is Hermes' take on the **Ralph loop** pattern. The user-facing design — keep a goal alive across turns, don't stop until it's achieved, with create/pause/resume/clear controls — was popularised and shipped in [Codex CLI 0.128.0](https://github.com/openai/codex) by Eric Traut on OpenAI's Codex team. Our implementation is independent (central `CommandDef` registry, `SessionDB.state_meta` persistence, auxiliary-client judge, adapter-FIFO continuation on the gateway side) but the idea is theirs. Credit where credit's due.
website/docs/user-guide/features/honcho.md
+238
View File
@@ -0,0 +1,238 @@
---
sidebar_position: 99
title: "Honcho Memory"
description: "AI-native persistent memory via Honcho — dialectic reasoning, multi-agent user modeling, and deep personalization"
---
# Honcho Memory
[Honcho](https://github.com/plastic-labs/honcho) is an AI-native memory backend that adds dialectic reasoning and deep user modeling on top of Hermes's built-in memory system. Instead of simple key-value storage, Honcho maintains a running model of who the user is — their preferences, communication style, goals, and patterns — by reasoning about conversations after they happen.
:::info Honcho is a Memory Provider Plugin
Honcho is integrated into the [Memory Providers](./memory-providers.md) system. All features below are available through the unified memory provider interface.
:::
## What Honcho Adds
| Capability | Built-in Memory | Honcho |
|-----------|----------------|--------|
| Cross-session persistence | ✔ File-based MEMORY.md/USER.md | ✔ Server-side with API |
| User profile | ✔ Manual agent curation | ✔ Automatic dialectic reasoning |
| Session summary | — | ✔ Session-scoped context injection |
| Multi-agent isolation | — | ✔ Per-peer profile separation |
| Observation modes | — | ✔ Unified or directional observation |
| Conclusions (derived insights) | — | ✔ Server-side reasoning about patterns |
| Search across history | ✔ FTS5 session search | ✔ Semantic search over conclusions |
**Dialectic reasoning**: After each conversation turn (gated by `dialecticCadence`), Honcho analyzes the exchange and derives insights about the user's preferences, habits, and goals. These accumulate over time, giving the agent a deepening understanding that goes beyond what the user explicitly stated. The dialectic supports multi-pass depth (1–3 passes) with automatic cold/warm prompt selection — cold start queries focus on general user facts while warm queries prioritize session-scoped context.
**Session-scoped context**: Base context now includes the session summary alongside the user representation and peer card. This gives the agent awareness of what has already been discussed in the current session, reducing repetition and enabling continuity.
**Multi-agent profiles**: When multiple Hermes instances talk to the same user (e.g., a coding assistant and a personal assistant), Honcho maintains separate "peer" profiles. Each peer sees only its own observations and conclusions, preventing cross-contamination of context.
## Setup
```bash
hermes memory setup # select "honcho" from the provider list
```
Or configure manually:
```yaml
# ~/.hermes/config.yaml
memory:
provider: honcho
```
```bash
echo 'HONCHO_API_KEY=***' >> ~/.hermes/.env
```
Get an API key at [honcho.dev](https://honcho.dev).
## Architecture
### Two-Layer Context Injection
Every turn (in `hybrid` or `context` mode), Honcho assembles two layers of context injected into the system prompt:
1. **Base context** — session summary, user representation, user peer card, AI self-representation, and AI identity card. Refreshed on `contextCadence`. This is the "who is this user" layer.
2. **Dialectic supplement** — LLM-synthesized reasoning about the user's current state and needs. Refreshed on `dialecticCadence`. This is the "what matters right now" layer.
Both layers are concatenated and truncated to the `contextTokens` budget (if set).
### Cold/Warm Prompt Selection
The dialectic automatically selects between two prompt strategies:
- **Cold start** (no base context yet): General query — "Who is this person? What are their preferences, goals, and working style?"
- **Warm session** (base context exists): Session-scoped query — "Given what's been discussed in this session so far, what context about this user is most relevant?"
This happens automatically based on whether base context has been populated.
### Three Orthogonal Config Knobs
Cost and depth are controlled by three independent knobs:
| Knob | Controls | Default |
|------|----------|---------|
| `contextCadence` | Turns between `context()` API calls (base layer refresh) | `1` |
| `dialecticCadence` | Turns between `peer.chat()` LLM calls (dialectic layer refresh) | `2` (recommended 1–5) |
| `dialecticDepth` | Number of `.chat()` passes per dialectic invocation (1–3) | `1` |
These are orthogonal — you can have frequent context refreshes with infrequent dialectic, or deep multi-pass dialectic at low frequency. Example: `contextCadence: 1, dialecticCadence: 5, dialecticDepth: 2` refreshes base context every turn, runs dialectic every 5 turns, and each dialectic run makes 2 passes.
### Dialectic Depth (Multi-Pass)
When `dialecticDepth` > 1, each dialectic invocation runs multiple `.chat()` passes:
- **Pass 0**: Cold or warm prompt (see above)
- **Pass 1**: Self-audit — identifies gaps in the initial assessment and synthesizes evidence from recent sessions
- **Pass 2**: Reconciliation — checks for contradictions between prior passes and produces a final synthesis
Each pass uses a proportional reasoning level (lighter early passes, base level for the main pass). Override per-pass levels with `dialecticDepthLevels` — e.g., `["minimal", "medium", "high"]` for a depth-3 run.
Passes bail out early if the prior pass returned strong signal (long, structured output), so depth 3 doesn't always mean 3 LLM calls.
### Session-Start Prewarm
On session init, Honcho fires a dialectic call in the background at the full configured `dialecticDepth` and hands the result directly to turn 1's context assembly. A single-pass prewarm on a cold peer often returns thin output — multi-pass depth runs the audit/reconcile cycle before the user ever speaks. If prewarm hasn't landed by turn 1, turn 1 falls back to a synchronous call with a bounded timeout.
### Query-Adaptive Reasoning Level
The auto-injected dialectic scales `dialecticReasoningLevel` by query length: +1 level at ≥120 chars, +2 at ≥400, clamped at `reasoningLevelCap` (default `"high"`). Disable with `reasoningHeuristic: false` to pin every auto call to `dialecticReasoningLevel`. Available levels: `minimal`, `low`, `medium`, `high`, `max`.
## Configuration Options
Honcho is configured in `~/.honcho/config.json` (global) or `$HERMES_HOME/honcho.json` (profile-local). The setup wizard handles this for you.
### Self-Hosted Honcho with Authentication
When pointing Hermes at a self-hosted Honcho server, `hermes honcho setup` (and `hermes memory setup`) ask for a **local JWT / bearer token** after the base URL. Paste a JWT signed with the server's `AUTH_JWT_SECRET` (the Honcho compose env var) to enable authenticated access; leave it blank for servers running with `AUTH_USE_AUTH=false`. The local token is stored under the host block (`hosts.<host>.apiKey` in `honcho.json`), separate from any cloud `apiKey`, so you can flip the `Cloud or local?` prompt back to `cloud` later without losing either credential.
### Full Config Reference
| Key | Default | Description |
|-----|---------|-------------|
| `contextTokens` | `null` (uncapped) | Token budget for auto-injected context per turn. Set to an integer (e.g. 1200) to cap. Truncates at word boundaries |
| `contextCadence` | `1` | Minimum turns between `context()` API calls (base layer refresh) |
| `dialecticCadence` | `2` | Minimum turns between `peer.chat()` LLM calls (dialectic layer). Recommended 1–5. In `tools` mode, irrelevant — model calls explicitly |
| `dialecticDepth` | `1` | Number of `.chat()` passes per dialectic invocation. Clamped to 1–3 |
| `dialecticDepthLevels` | `null` | Optional array of reasoning levels per pass, e.g. `["minimal", "low", "medium"]`. Overrides proportional defaults |
| `dialecticReasoningLevel` | `'low'` | Base reasoning level: `minimal`, `low`, `medium`, `high`, `max` |
| `dialecticDynamic` | `true` | When `true`, model can override reasoning level per-call via tool param |
| `dialecticMaxChars` | `600` | Max chars of dialectic result injected into system prompt |
| `recallMode` | `'hybrid'` | `hybrid` (auto-inject + tools), `context` (inject only), `tools` (tools only) |
| `writeFrequency` | `'async'` | When to flush messages: `async` (background thread), `turn` (sync), `session` (batch on end), or integer N |
| `saveMessages` | `true` | Whether to persist messages to Honcho API |
| `observationMode` | `'directional'` | `directional` (all on) or `unified` (shared pool). Override with `observation` object for granular control |
| `messageMaxChars` | `25000` | Max chars per message sent via `add_messages()`. Chunked if exceeded |
| `dialecticMaxInputChars` | `10000` | Max chars for dialectic query input to `peer.chat()` |
| `sessionStrategy` | `'per-directory'` | `per-directory`, `per-repo`, `per-session`, or `global` |
**Session strategy** controls how Honcho sessions map to your work:
- `per-session` — each `hermes` run gets a fresh session. Clean starts, memory via tools. Recommended for new users.
- `per-directory` — one Honcho session per working directory. Context accumulates across runs.
- `per-repo` — one session per git repository.
- `global` — single session across all directories.
**Recall mode** controls how memory flows into conversations:
- `hybrid` — context auto-injected into system prompt AND tools available (model decides when to query).
- `context` — auto-injection only, tools hidden.
- `tools` — tools only, no auto-injection. Agent must explicitly call `honcho_reasoning`, `honcho_search`, etc.
**Settings per recall mode:**
| Setting | `hybrid` | `context` | `tools` |
|---------|----------|-----------|---------|
| `writeFrequency` | flushes messages | flushes messages | flushes messages |
| `contextCadence` | gates base context refresh | gates base context refresh | irrelevant — no injection |
| `dialecticCadence` | gates auto LLM calls | gates auto LLM calls | irrelevant — model calls explicitly |
| `dialecticDepth` | multi-pass per invocation | multi-pass per invocation | irrelevant — model calls explicitly |
| `contextTokens` | caps injection | caps injection | irrelevant — no injection |
| `dialecticDynamic` | gates model override | N/A (no tools) | gates model override |
In `tools` mode, the model is fully in control — it calls `honcho_reasoning` when it wants, at whatever `reasoning_level` it picks. Cadence and budget settings only apply to modes with auto-injection (`hybrid` and `context`).
## Observation (Directional vs. Unified)
Honcho models a conversation as peers exchanging messages. Each peer has two observation toggles that map 1:1 to Honcho's `SessionPeerConfig`:
| Toggle | Effect |
|--------|--------|
| `observeMe` | Honcho builds a representation of this peer from its own messages |
| `observeOthers` | This peer observes the other peer's messages (feeds cross-peer reasoning) |
Two peers × two toggles = four flags. `observationMode` is a shorthand preset:
| Preset | User flags | AI flags | Semantics |
|--------|-----------|----------|-----------|
| `"directional"` (default) | me: on, others: on | me: on, others: on | Full mutual observation. Enables cross-peer dialectic — "what does the AI know about the user, based on what the user said and the AI replied." |
| `"unified"` | me: on, others: off | me: off, others: on | Shared-pool semantics — the AI observes the user's messages only, the user peer only self-models. Single-observer pool. |
Override the preset with an explicit `observation` block for per-peer control:
```json
"observation": {
"user": { "observeMe": true, "observeOthers": true },
"ai": { "observeMe": true, "observeOthers": false }
}
```
Common patterns:
| Intent | Config |
|--------|--------|
| Full observation (most users) | `"observationMode": "directional"` |
| AI shouldn't re-model the user from its own replies | `"ai": {"observeMe": true, "observeOthers": false}` |
| Strong persona the AI peer shouldn't update from self-observation | `"ai": {"observeMe": false, "observeOthers": true}` |
Server-side toggles set via the [Honcho dashboard](https://app.honcho.dev) win over local defaults — Hermes syncs them back at session init.
## Tools
When Honcho is active as the memory provider, five tools become available:
| Tool | Purpose |
|------|---------|
| `honcho_profile` | Read or update peer card — pass `card` (list of facts) to update, omit to read |
| `honcho_search` | Semantic search over context — raw excerpts, no LLM synthesis |
| `honcho_context` | Full session context — summary, representation, card, recent messages |
| `honcho_reasoning` | Synthesized answer from Honcho's LLM — pass `reasoning_level` (minimal/low/medium/high/max) to control depth |
| `honcho_conclude` | Create or delete conclusions — pass `conclusion` to create, `delete_id` to remove (PII only) |
## CLI Commands
The `hermes honcho` subcommand is **only registered when Honcho is the active memory provider** (`memory.provider: honcho` in `config.yaml`). On a fresh install, configure Honcho directly with `hermes memory setup honcho` (or run `hermes memory setup` and pick it from the list); the `hermes honcho` subcommand then appears on the next invocation.
```bash
hermes memory setup honcho # Configure Honcho directly (works before activation)
hermes honcho status # Connection status, config, and key settings
hermes honcho setup # Redirects to `hermes memory setup` (post-activation alias)
hermes honcho strategy # Show or set session strategy (per-session/per-directory/per-repo/global)
hermes honcho peer # Show or update peer names + dialectic reasoning level
hermes honcho mode # Show or set recall mode (hybrid/context/tools)
hermes honcho tokens # Show or set token budget for context and dialectic
hermes honcho identity # Seed or show the AI peer's Honcho identity
hermes honcho sync # Sync Honcho config to all existing profiles
hermes honcho peers # Show peer identities across all profiles
hermes honcho sessions # List known Honcho session mappings
hermes honcho map # Map current directory to a Honcho session name
hermes honcho enable # Enable Honcho for the active profile
hermes honcho disable # Disable Honcho for the active profile
hermes honcho migrate # Step-by-step migration guide from openclaw-honcho
```
## Migrating from `hermes honcho`
If you previously used the standalone `hermes honcho setup`:
1. Your existing configuration (`honcho.json` or `~/.honcho/config.json`) is preserved
2. Your server-side data (memories, conclusions, user profiles) is intact
3. Set `memory.provider: honcho` in config.yaml to reactivate
No re-login or re-setup needed. Run `hermes memory setup` and select "honcho" — the wizard detects your existing config.
## Full Documentation
See [Memory Providers — Honcho](./memory-providers.md#honcho) for the complete reference.
website/docs/user-guide/features/hooks.md
+1336
View File
File diff suppressed because it is too large Load Diff
website/docs/user-guide/features/image-generation.md
+158
View File
@@ -0,0 +1,158 @@
---
title: Image Generation
description: Generate images via FAL.ai — 11 models including FLUX 2, GPT Image (1.5 & 2), Nano Banana Pro, Ideogram, Recraft V4 Pro, Krea 2, and more, selectable via `hermes tools`.
sidebar_label: Image Generation
sidebar_position: 6
---
# Image Generation
Hermes Agent generates images from text prompts via FAL.ai. Eleven models are supported out of the box, each with different speed, quality, and cost tradeoffs. The active model is user-configurable via `hermes tools` and persists in `config.yaml`.
## Supported Models
| Model | Speed | Strengths | Price |
|---|---|---|---|
| `fal-ai/flux-2/klein/9b` *(default)* | `<1s` | Fast, crisp text | $0.006/MP |
| `fal-ai/flux-2-pro` | ~6s | Studio photorealism | $0.03/MP |
| `fal-ai/z-image/turbo` | ~2s | Bilingual EN/CN, 6B params | $0.005/MP |
| `fal-ai/nano-banana-pro` | ~8s | Gemini 3 Pro, reasoning depth, text rendering | $0.15/image (1K) |
| `fal-ai/gpt-image-1.5` | ~15s | Prompt adherence | $0.034/image |
| `fal-ai/gpt-image-2` | ~20s | SOTA text rendering + CJK, world-aware photorealism | $0.04–0.06/image |
| `fal-ai/ideogram/v3` | ~5s | Best typography | $0.03–0.09/image |
| `fal-ai/recraft/v4/pro/text-to-image` | ~8s | Design, brand systems, production-ready | $0.25/image |
| `fal-ai/qwen-image` | ~12s | LLM-based, complex text | $0.02/MP |
| `fal-ai/krea/v2/medium/text-to-image` | ~15-25s | Illustration, anime, painting, expressive/artistic styles | $0.030–0.035/image |
| `fal-ai/krea/v2/large/text-to-image` | ~25-60s | Photorealism, raw textured looks (motion blur, grain, film) | $0.060–0.065/image |
Prices are FAL's pricing at time of writing; check [fal.ai](https://fal.ai/) for current numbers.
## Setup
:::tip Nous Subscribers
If you have a paid [Nous Portal](https://portal.nousresearch.com) subscription, you can use image generation through the **[Tool Gateway](tool-gateway.md)** without a FAL API key. Your model selection persists across both paths. New installs can run `hermes setup --portal` to log in and turn on every gateway tool at once; existing installs can pick **Nous Subscription** as the image-gen backend via `hermes tools`.
If the managed gateway returns `HTTP 4xx` for a specific model, that model isn't yet proxied on the portal side — the agent will tell you so, with remediation steps (set `FAL_KEY` for direct access, or pick a different model).
:::
### Get a FAL API Key
1. Sign up at [fal.ai](https://fal.ai/)
2. Generate an API key from your dashboard
### Configure and Pick a Model
Run the tools command:
```bash
hermes tools
```
Navigate to **🎨 Image Generation**, pick your backend (Nous Subscription or FAL.ai), then the picker shows all supported models in a column-aligned table — arrow keys to navigate, Enter to select:
```
Model Speed Strengths Price
fal-ai/flux-2/klein/9b <1s Fast, crisp text $0.006/MP ← currently in use
fal-ai/flux-2-pro ~6s Studio photorealism $0.03/MP
fal-ai/z-image/turbo ~2s Bilingual EN/CN, 6B $0.005/MP
...
```
Your selection is saved to `config.yaml`:
```yaml
image_gen:
model: fal-ai/flux-2/klein/9b
use_gateway: false # true if using Nous Subscription
```
### GPT-Image Quality
The `fal-ai/gpt-image-1.5` and `fal-ai/gpt-image-2` request quality is pinned to `medium` (~$0.034–$0.06/image at 1024×1024). We don't expose the `low` / `high` tiers as a user-facing option so that Nous Portal billing stays predictable across all users — the cost spread between tiers is 3–22×. If you want a cheaper option, pick Klein 9B or Z-Image Turbo; if you want higher quality, use Nano Banana Pro or Recraft V4 Pro.
## Usage
The agent-facing schema is intentionally minimal — the model picks up whatever you've configured:
```
Generate an image of a serene mountain landscape with cherry blossoms
```
```
Create a square portrait of a wise old owl — use the typography model
```
```
Make me a futuristic cityscape, landscape orientation
```
## Aspect Ratios
Every model accepts the same three aspect ratios from the agent's perspective. Internally, each model's native size spec is filled in automatically:
| Agent input | image_size (flux/z-image/qwen/recraft/ideogram) | aspect_ratio (nano-banana-pro) | image_size (gpt-image-1.5) | image_size (gpt-image-2) |
|---|---|---|---|---|
| `landscape` | `landscape_16_9` | `16:9` | `1536x1024` | `landscape_4_3` (1024×768) |
| `square` | `square_hd` | `1:1` | `1024x1024` | `square_hd` (1024×1024) |
| `portrait` | `portrait_16_9` | `9:16` | `1024x1536` | `portrait_4_3` (768×1024) |
GPT Image 2 maps to 4:3 presets rather than 16:9 because its minimum pixel count is 655,360 — the `landscape_16_9` preset (1024×576 = 589,824) would be rejected.
This translation happens in `_build_fal_payload()` — agent code never has to know about per-model schema differences.
## Automatic Upscaling
Upscaling via FAL's **Clarity Upscaler** is gated per-model:
| Model | Upscale? | Why |
|---|---|---|
| `fal-ai/flux-2-pro` | ✓ | Backward-compat (was the pre-picker default) |
| All others | ✗ | Fast models would lose their sub-second value prop; hi-res models don't need it |
When upscaling runs, it uses these settings:
| Setting | Value |
|---|---|
| Upscale factor | 2× |
| Creativity | 0.35 |
| Resemblance | 0.6 |
| Guidance scale | 4 |
| Inference steps | 18 |
If upscaling fails (network issue, rate limit), the original image is returned automatically.
## How It Works Internally
1. **Model resolution** — `_resolve_fal_model()` reads `image_gen.model` from `config.yaml`, falls back to the `FAL_IMAGE_MODEL` env var, then to `fal-ai/flux-2/klein/9b`.
2. **Payload building** — `_build_fal_payload()` translates your `aspect_ratio` into the model's native format (preset enum, aspect-ratio enum, or GPT literal), merges the model's default params, applies any caller overrides, then filters to the model's `supports` whitelist so unsupported keys are never sent.
3. **Submission** — `_submit_fal_request()` routes via direct FAL credentials or the managed Nous gateway.
4. **Upscaling** — runs only if the model's metadata has `upscale: True`.
5. **Delivery** — final image URL returned to the agent, which emits a `MEDIA:<url>` tag that platform adapters convert to native media.
## Debugging
Enable debug logging:
```bash
export IMAGE_TOOLS_DEBUG=true
```
Debug logs go to `./logs/image_tools_debug_<session_id>.json` with per-call details (model, parameters, timing, errors).
## Platform Delivery
| Platform | Delivery |
|---|---|
| **CLI** | Image URL printed as markdown `![](url)` — click to open |
| **Telegram** | Photo message with the prompt as caption |
| **Discord** | Embedded in a message |
| **Slack** | URL unfurled by Slack |
| **WhatsApp** | Media message |
| **Others** | URL in plain text |
## Limitations
- **Requires FAL credentials** (direct `FAL_KEY` or Nous Subscription)
- **Text-to-image only** — no inpainting, img2img, or editing via this tool
- **Temporary URLs** — FAL returns hosted URLs that expire after hours/days; save locally if needed
- **Per-model constraints** — some models don't support `seed`, `num_inference_steps`, etc. The `supports` filter silently drops unsupported params; this is expected behavior
website/docs/user-guide/features/kanban-tutorial.md
+310
View File
@@ -0,0 +1,310 @@
# Kanban tutorial
A walkthrough of the four use-cases the Hermes Kanban system was designed for, with the dashboard open in a browser. If you haven't read the [Kanban overview](./kanban) yet, start there — this assumes you know what a task, run, assignee, and dispatcher are.
## Setup
```bash
hermes kanban init # optional; first `hermes kanban <anything>` auto-inits
hermes dashboard # opens http://127.0.0.1:9119 in your browser
# click Kanban in the left nav
```
The dashboard is the most comfortable place for **you** to watch the system. Agent workers the dispatcher spawns never see the dashboard or the CLI — they drive the board through a dedicated `kanban_*` [toolset](./kanban#how-workers-interact-with-the-board) (`kanban_show`, `kanban_list`, `kanban_complete`, `kanban_block`, `kanban_heartbeat`, `kanban_comment`, `kanban_create`, `kanban_link`, `kanban_unblock`). All three surfaces — dashboard, CLI, worker tools — route through the same per-board SQLite DB (`~/.hermes/kanban.db` for the default board, `~/.hermes/kanban/boards/<slug>/kanban.db` for any board you create later), so each board is consistent no matter which side of the fence a change came from.
This tutorial uses the `default` board throughout. If you want multiple isolated queues (one per project / repo / domain), see [Boards (multi-project)](./kanban#boards-multi-project) in the overview — the same CLI / dashboard / worker flows apply per board, and workers physically cannot see tasks on other boards.
Throughout the tutorial, **code blocks labelled `bash` are commands *you* run.** Code blocks labelled `# worker tool calls` are what the spawned worker's model emits as tool calls — shown here so you can see the loop end-to-end, not because you'd ever run them yourself.
## The board at a glance
![Kanban board overview](/img/kanban-tutorial/01-board-overview.png)
Six columns, left to right:
- **Triage** — raw ideas. By default the dispatcher auto-runs the **decomposer** on tasks here: the built-in decomposer uses `auxiliary.kanban_decomposer`, reads your profile roster + descriptions, and produces a graph of child tasks routed to the best-fit specialists. The original task is held alive as the parent so its assignee (`kanban.orchestrator_profile`, or the active default profile when unset) wakes back up to judge completion when everything finishes. Flip the **Orchestration: Auto/Manual** pill at the top of the kanban page to switch modes. In Manual mode click **⚗ Decompose** on a card, or run `hermes kanban decompose <id>` / `/kanban decompose <id>`. For single tasks that don't need fan-out, **✨ Specify** does a one-shot spec rewrite (goal, approach, acceptance criteria) and promotes to `todo`. Configure the models under `auxiliary.kanban_decomposer` and `auxiliary.triage_specifier` in `config.yaml`. See [Auto vs Manual orchestration](./kanban#auto-vs-manual-orchestration) in the main Kanban guide.
- **Todo** — created but waiting on dependencies, or not yet assigned.
- **Ready** — assigned and waiting for the dispatcher to claim.
- **In progress** — a worker is actively running the task. With "Lanes by profile" on (the default), this column sub-groups by assignee so you can see at a glance what each worker is doing.
- **Blocked** — a worker asked for human input, or the circuit breaker tripped.
- **Done** — completed.
The top bar has filters for search, tenant, and assignee, plus a `Lanes by profile` toggle and a `Nudge dispatcher` button that runs one dispatch tick right now instead of waiting for the daemon's next interval. Clicking any card opens its drawer on the right.
### Flat view
If the profile lanes are noisy, toggle "Lanes by profile" off and the In Progress column collapses to a single flat list ordered by claim time:
![Board with lanes by profile off](/img/kanban-tutorial/02-board-flat.png)
## Story 1 — Solo dev shipping a feature
You're building a feature. Classic flow: design a schema, implement the API, write the tests. Three tasks with parent→child dependencies.
```bash
SCHEMA=$(hermes kanban create "Design auth schema" \
--assignee backend-dev --tenant auth-project --priority 2 \
--body "Design the user/session/token schema for the auth module." \
--json | jq -r .id)
API=$(hermes kanban create "Implement auth API endpoints" \
--assignee backend-dev --tenant auth-project --priority 2 \
--parent $SCHEMA \
--body "POST /register, POST /login, POST /refresh, POST /logout." \
--json | jq -r .id)
hermes kanban create "Write auth integration tests" \
--assignee qa-dev --tenant auth-project --priority 2 \
--parent $API \
--body "Cover happy path, wrong password, expired token, concurrent refresh."
```
Because `API` has `SCHEMA` as its parent, and `tests` has `API` as its parent, only `SCHEMA` starts in `ready`. The other two sit in `todo` until their parents complete. This is the dependency promotion engine doing its job — no other worker will pick up the test-writing until there's an API to test.
On the next dispatcher tick (60s by default, or immediately if you hit **Nudge dispatcher**) the `backend-dev` profile spawns as a worker with `HERMES_KANBAN_TASK=$SCHEMA` in its env. Here's what the worker's tool-call loop looks like from inside the agent:
```python
# worker tool calls — NOT commands you run
kanban_show()
# → returns title, body, worker_context, parents, prior attempts, comments
# (worker reads worker_context, uses terminal/file tools to design the schema,
# write migrations, run its own checks, commit — the real work happens here)
kanban_heartbeat(note="schema drafted, writing migrations now")
kanban_complete(
summary="users(id, email, pw_hash), sessions(id, user_id, jti, expires_at); "
"refresh tokens stored as sessions with type='refresh'",
metadata={
"changed_files": ["migrations/001_users.sql", "migrations/002_sessions.sql"],
"decisions": ["bcrypt for hashing", "JWT for session tokens",
"7-day refresh, 15-min access"],
},
)
```
`kanban_show` defaults `task_id` to `$HERMES_KANBAN_TASK`, so the worker doesn't need to know its own id. `kanban_complete` writes the summary + metadata onto the current `task_runs` row, closes that run, and transitions the task to `done` — all in one atomic hop through `kanban_db`.
When `SCHEMA` hits `done`, the dependency engine promotes `API` to `ready` automatically. The API worker, when it picks up, will call `kanban_show()` and see `SCHEMA`'s summary and metadata attached to the parent handoff — so it knows the schema decisions without re-reading a long design doc.
Click the completed schema task on the board and the drawer shows everything:
![Solo dev — completed schema task drawer](/img/kanban-tutorial/03-drawer-schema-task.png)
The Run History section at the bottom is the key addition. One attempt: outcome `completed`, worker `@backend-dev`, duration, timestamp, and the handoff summary in full. The metadata blob (`changed_files`, `decisions`) is stored on the run too and surfaced to any downstream worker that reads this parent.
You can inspect the same data from your terminal at any time — these commands are **you** peeking at the board, not the worker:
```bash
hermes kanban show $SCHEMA
hermes kanban runs $SCHEMA
# # OUTCOME PROFILE ELAPSED STARTED
# 1 completed backend-dev 0s 2026-04-27 19:34
# → users(id, email, pw_hash), sessions(id, user_id, jti, expires_at); refresh tokens ...
```
## Story 2 — Fleet farming
You have three workers (a translator, a transcriber, a copywriter) and a pile of independent tasks. You want all three pulling in parallel and making visible progress. This is the simplest kanban use-case and the one the original design optimized for.
Create the work:
```bash
for lang in Spanish French German; do
hermes kanban create "Translate homepage to $lang" \
--assignee translator --tenant content-ops
done
for i in 1 2 3 4 5; do
hermes kanban create "Transcribe Q3 customer call #$i" \
--assignee transcriber --tenant content-ops
done
for sku in 1001 1002 1003 1004; do
hermes kanban create "Generate product description: SKU-$sku" \
--assignee copywriter --tenant content-ops
done
```
Start the gateway and walk away — it hosts the embedded dispatcher
that picks up all three specialist profiles' tasks on the same
kanban.db:
```bash
hermes gateway start
```
Now filter the board to `content-ops` (or just search for "Transcribe") and you get this:
![Fleet view filtered to transcribe tasks](/img/kanban-tutorial/07-fleet-transcribes.png)
Two transcribes done, one running, two ready waiting for the next dispatcher tick. The In Progress column is grouped by profile (the "Lanes by profile" default) so you see each worker's active task without scanning a mixed list. The dispatcher will promote the next ready task to running as soon as the current one completes. With three daemons working on three assignee pools in parallel, the whole content queue drains without further human input.
**Everything Story 1 said about structured handoff still applies here.** A translator worker completing a call emits `kanban_complete(summary="translated 4 pages, style matched existing marketing voice", metadata={"duration_seconds": 720, "tokens_used": 2100})` — useful for analytics and for any downstream task that depends on this one.
## Story 3 — Role pipeline with retry
This is where Kanban earns its keep over a flat TODO list. A PM writes a spec. An engineer implements it. A reviewer rejects the first attempt. The engineer tries again with changes. The reviewer approves.
The dashboard view, filtered by `auth-project`:
![Pipeline view for a multi-role feature](/img/kanban-tutorial/08-pipeline-auth.png)
Three-stage chain visible at once: `Spec: password reset flow` (DONE, pm), `Implement password reset flow` (DONE, backend-dev), `Review password reset PR` (READY, reviewer). Each has its parent in green at the bottom and children as dependencies.
The interesting one is the implementation task, because it was blocked and retried. Here's the full three-agent choreography, shown as the tool calls each worker's model makes:
```python
# --- PM worker spawns on $SPEC and writes the acceptance criteria ---
# worker tool calls
kanban_show()
kanban_complete(
summary="spec approved; POST /forgot-password sends email, "
"GET /reset/:token renders form, POST /reset applies new password",
metadata={"acceptance": [
"expired token returns 410",
"reused last-3 password returns 400 with message",
"successful reset invalidates all active sessions",
]},
)
# → $SPEC is done; $IMPL auto-promotes from todo to ready
# --- Engineer worker spawns on $IMPL (first attempt) ---
# worker tool calls
kanban_show() # reads $SPEC's summary + acceptance metadata in worker_context
# (engineer writes code, runs tests, opens PR)
# Reviewer feedback arrives — engineer decides the concerns are valid and blocks
kanban_block(
reason="Review: password strength check missing, reset link isn't "
"single-use (can be replayed within 30min)",
)
# → $IMPL transitions to blocked; run 1 closes with outcome='blocked'
```
Now you (the human, or a separate reviewer profile) read the block reason, decide the fix direction is clear, and unblock from the dashboard's "Unblock" button — or from the CLI / slash command:
```bash
hermes kanban unblock $IMPL
# or from a chat: /kanban unblock $IMPL
```
The dispatcher promotes `$IMPL` back to `ready` and, on the next tick, respawns the `backend-dev` worker. This second spawn is a **new run** on the same task:
```python
# --- Engineer worker spawns on $IMPL (second attempt) ---
# worker tool calls
kanban_show()
# → worker_context now includes the run 1 block reason, so this worker knows
# which two things to fix instead of re-reading the whole spec
# (engineer adds zxcvbn check, makes reset tokens single-use, re-runs tests)
kanban_complete(
summary="added zxcvbn strength check, reset tokens are now single-use "
"(stored + deleted on success)",
metadata={
"changed_files": [
"auth/reset.py",
"auth/tests/test_reset.py",
"migrations/003_single_use_reset_tokens.sql",
],
"tests_run": 11,
"review_iteration": 2,
},
)
```
Click the implementation task. The drawer shows **two attempts**:
![Implementation task with two runs — blocked then completed](/img/kanban-tutorial/04b-drawer-retry-history-scrolled.png)
- **Run 1** — `blocked` by `@backend-dev`. The review feedback sits right under the outcome: "password strength check missing, reset link isn't single-use (can be replayed within 30min)".
- **Run 2** — `completed` by `@backend-dev`. Fresh summary, fresh metadata.
Each run is a row in `task_runs` with its own outcome, summary, and metadata. Retry history is not a conceptual afterthought layered on top of a "latest state" task — it's the primary representation. When a retrying worker opens the task, `build_worker_context` shows it the prior attempts, so the second-pass worker sees why the first pass was blocked and addresses those specific findings instead of re-running from scratch.
The reviewer picks up next. When they open `Review password reset PR`, they see:
![Reviewer's drawer view of the pipeline](/img/kanban-tutorial/09-drawer-pipeline-review.png)
The parent link is the completed implementation. When the reviewer's worker spawns on `Review password reset PR` and calls `kanban_show()`, the returned `worker_context` includes the parent's most-recent-completed-run summary + metadata — so the reviewer reads "added zxcvbn strength check, reset tokens are now single-use" and has the list of changed files in hand before looking at a diff.
## Story 4 — Circuit breaker and crash recovery
Real workers fail. Missing credentials, OOM kills, transient network errors. The dispatcher has two lines of defense: a **circuit breaker** that auto-blocks after N consecutive failures so the board doesn't thrash forever, and **crash detection** that reclaims a task whose worker PID went away before its TTL expired.
### Circuit breaker — permanent-looking failure
A deploy task that can't spawn its worker because `AWS_ACCESS_KEY_ID` isn't set in the profile's environment:
```bash
hermes kanban create "Deploy to staging (missing creds)" \
--assignee deploy-bot --tenant ops \
--max-retries 3
```
The dispatcher tries to spawn the worker. Spawn fails (`RuntimeError: AWS_ACCESS_KEY_ID not set`). The dispatcher releases the claim, increments a failure counter, and tries again next tick. Because this example sets `--max-retries 3`, the circuit trips after three consecutive failures: the task goes to `blocked` with outcome `gave_up`. If you omit the flag, Hermes uses `kanban.failure_limit` (default: 2). No more retries until a human unblocks it.
Click the blocked task:
![Circuit breaker — 2 spawn_failed + 1 gave_up](/img/kanban-tutorial/11-drawer-gave-up.png)
Three runs, all with the same error on the `error` field. The first two are `spawn_failed` (retryable), the third is `gave_up` (terminal). The event log above shows the full sequence: `created → claimed → spawn_failed → claimed → spawn_failed → claimed → gave_up`.
On the terminal:
```bash
hermes kanban runs t_ef5d
# # OUTCOME PROFILE ELAPSED STARTED
# 1 spawn_failed deploy-bot 0s 2026-04-27 19:34
# ! AWS_ACCESS_KEY_ID not set in deploy-bot env
# 2 spawn_failed deploy-bot 0s 2026-04-27 19:34
# ! AWS_ACCESS_KEY_ID not set in deploy-bot env
# 3 gave_up deploy-bot 0s 2026-04-27 19:34
# ! AWS_ACCESS_KEY_ID not set in deploy-bot env
```
If Telegram / Discord / Slack is wired in, a gateway notification fires on the `gave_up` event so you hear about the outage without having to check the board.
### Crash recovery — worker dies mid-flight
Sometimes the spawn succeeds but the worker process dies later — segfault, OOM, `systemctl stop`. The dispatcher polls `kill(pid, 0)` and detects the dead pid; the claim releases, the task goes back to `ready`, and the next tick gives it to a fresh worker.
The example in the seed data is a migration that was running out of memory:
```bash
# Worker claims, starts scanning 2.4M rows, OOM kills it at ~2.3M
# Dispatcher detects dead pid, releases claim, increments attempt counter
# Retry with a chunked strategy succeeds
```
The drawer shows the full two-attempt history:
![Crash and recovery — 1 crashed + 1 completed](/img/kanban-tutorial/06-drawer-crash-recovery.png)
Run 1 — `crashed`, with the error `OOM kill at row 2.3M (process 99999 gone)`. Run 2 — `completed`, with `"strategy": "chunked with LIMIT + WHERE id > last_id"` in its metadata. The retrying worker saw the crash of run 1 in its context and picked a safer strategy; the metadata makes it obvious to a future observer (or postmortem writer) what changed.
## Structured handoff — why `summary` and `metadata` matter
In every story above, workers called `kanban_complete(summary=..., metadata=...)` at the end. That's not decoration — it's the primary handoff channel between stages of a workflow.
When a worker on task B is spawned and calls `kanban_show()`, the `worker_context` it gets back includes:
- B's **prior attempts** (previous runs: outcome, summary, error, metadata) so a retrying worker doesn't repeat a failed path.
- **Parent task results** — for each parent, the most-recent completed run's summary and metadata — so downstream workers see why and how the upstream work was done.
This replaces the "dig through comments and the work output" dance that plagues flat kanban systems. A PM writes acceptance criteria in the spec's metadata, and the engineer's worker sees them structurally in the parent handoff. An engineer records which tests they ran and how many passed, and the reviewer's worker has that list in hand before opening a diff.
The bulk-close guard exists because this data is per-run. `hermes kanban complete a b c --summary X` (you, from the CLI) is refused — copy-pasting the same summary to three tasks is almost always wrong. Bulk close without the handoff flags still works for the common "I finished a pile of admin tasks" case. The tool surface doesn't expose a bulk variant at all; `kanban_complete` is always single-task-at-a-time for the same reason.
## Inspecting a task currently running
For completeness — here's the drawer of a task still in flight (the API implementation from Story 1, claimed by `backend-dev` but not yet complete):
![Claimed, in-flight task](/img/kanban-tutorial/10-drawer-in-flight.png)
Status is `Running`. The active run appears in the Run History section with outcome `active` and no `ended_at`. If this worker dies or times out, the dispatcher closes this run with the appropriate outcome and opens a new one on the next claim — the attempt row never disappears.
## Next steps
- [Kanban overview](./kanban) — the full data model, event vocabulary, and CLI reference.
- `hermes kanban --help` — every subcommand, every flag.
- `hermes kanban watch --kinds completed,gave_up,timed_out` — live stream terminal events across the whole board.
- `hermes kanban notify-subscribe <task> --platform telegram --chat-id <id>` — get a gateway ping when a specific task finishes.
website/docs/user-guide/features/kanban-worker-lanes.md
+114
View File
@@ -0,0 +1,114 @@
# Kanban worker lanes
A **worker lane** is a class of process that the kanban dispatcher can route tasks to. Each lane has an identity (the assignee string), a spawn mechanism, and a contract for what it must do with the task once spawned.
This page is the contract. It exists for two audiences:
- **Operators** picking which lanes to wire into a board (which profiles to create, which assignees to use).
- **Plugin / integration authors** wanting to add a new lane shape (a CLI worker that wraps Codex / Claude Code / OpenCode, a containerised review worker, a non-Hermes service that pulls tasks via the API).
If you're writing the worker code itself — the agent that runs *inside* a lane — the [`kanban-worker`](https://github.com/NousResearch/hermes-agent/blob/main/skills/devops/kanban-worker/SKILL.md) skill is the deeper procedural detail.
## The hierarchy
```text
Hermes Kanban = canonical task lifecycle + audit trail
Worker lane = implementation executor for one assigned card
Reviewer = human or human-proxy that gates "done"
GitHub PR = upstreamable artifact (optional, for code lanes)
```
Hermes Kanban owns lifecycle truth — `ready` → `running` → `blocked` / `done` / `archived`. Worker lanes execute work but never own that truth; everything they do flows back through the kanban kernel via the `kanban_*` tools (or, for non-Hermes external workers, via the API). Reviewers gate the transition from "code change written" to "task done."
## What a lane provides
To be a kanban worker lane, an integration must provide three things:
### 1. An assignee string
The dispatcher matches `task.assignee` against either a Hermes profile name (the default lane shape) or a registered non-spawnable identifier (the plugin lane shape — see [Adding an external CLI worker lane](#adding-an-external-cli-worker-lane) below). Tasks whose assignee doesn't resolve are left on `ready` with a `skipped_nonspawnable` event so a board operator can fix them; they are not silently dropped or executed by an arbitrary fallback.
### 2. A spawn mechanism
For Hermes profile lanes, the dispatcher's `_default_spawn` runs `hermes -p <assignee> chat -q <prompt>` (or the equivalent module form when the `hermes` shim isn't on `$PATH`) inside the task's pinned workspace, with these env vars set:
| Variable | Carries |
|---|---|
| `HERMES_KANBAN_TASK` | the task id the worker is operating on |
| `HERMES_KANBAN_DB` | absolute path to the per-board SQLite file |
| `HERMES_KANBAN_BOARD` | board slug |
| `HERMES_KANBAN_WORKSPACES_ROOT` | root of the board's workspace tree |
| `HERMES_KANBAN_WORKSPACE` | absolute path to *this* task's workspace |
| `HERMES_KANBAN_RUN_ID` | the current run's id (for the lifecycle gate) |
| `HERMES_KANBAN_CLAIM_LOCK` | the claim lock string (`<host>:<pid>:<uuid>`) |
| `HERMES_PROFILE` | the worker's own profile name (for `kanban_comment` author attribution) |
| `HERMES_TENANT` | tenant namespace, if the task has one |
For non-Hermes lanes (registered via a plugin), the plugin supplies its own `spawn_fn` callable that gets `task`, `workspace`, and `board` and returns an optional pid for crash detection.
### 3. A lifecycle terminator
Every claim must end in exactly one of:
- `kanban_complete(summary=..., metadata=...)` — task succeeds, status flips to `done`.
- `kanban_block(reason=...)` — task waits for human input, status flips to `blocked`. The dispatcher respawns when `kanban_unblock` runs.
- The worker process exits without a tool call. The kernel reaps it and emits `crashed` (PID died) or `gave_up` (consecutive-failure breaker tripped) or `timed_out` (max_runtime exceeded). This is the failure path; healthy workers don't end here.
The kanban kernel enforces that exactly one of these terminates each run. A worker that calls neither and exits normally is treated as crashed.
## Outputs and the review-required convention
For most code-changing tasks, the work isn't truly *done* the moment the worker finishes — it needs a human reviewer. The kanban kernel doesn't enforce this distinction (a "code-changing task" is fuzzy and forcing block-instead-of-complete on every code worker would break flows where no review is wanted). It's a convention layered on top:
- **Block instead of complete**, with `reason` prefixed `review-required: ` so the dashboard / `hermes kanban show` surfaces the row as awaiting review.
- **Drop structured metadata into a `kanban_comment` first** since `kanban_block` only carries the human-readable `reason`. Comments are the durable annotation channel — every audit-relevant field (changed_files, tests_run, diff_path or PR url, decisions) belongs there.
- **Reviewer either approves and unblocks**, which respawns the worker with the comment thread for follow-ups; or asks for changes via another comment, which the next worker run sees as part of `kanban_show`'s context.
The [`kanban-worker`](https://github.com/NousResearch/hermes-agent/blob/main/skills/devops/kanban-worker/SKILL.md) skill has worked examples for both `kanban_complete` (truly terminal tasks — typo fixes, docs changes, research writeups) and the `review-required` block pattern.
## Logs and audit trail
The dispatcher writes per-task worker stdout/stderr to `<board-root>/logs/<task_id>.log`. Logs are auditable from kanban metadata:
- `task_runs` rows carry the `log_path`, exit code (where available), summary, and metadata.
- `task_events` rows carry every state transition (`promoted`, `claimed`, `heartbeat`, `completed`, `blocked`, `gave_up`, `crashed`, `timed_out`, `reclaimed`, `claim_extended`).
- `kanban_show` returns both, so a reviewer (or a follow-up worker) reading the task gets the full history without needing dashboard access.
The dashboard renders run history with summaries, metadata blocks, and exit-status badges. CLI users can run `hermes kanban tail <task_id>` to follow live, or `hermes kanban runs <task_id>` for the historical attempt list.
## Existing lane shapes
### Hermes profile lane (default)
The shape every kanban worker takes today: the assignee is a profile name, the dispatcher spawns `hermes -p <profile>`, the worker auto-loads the [`kanban-worker`](https://github.com/NousResearch/hermes-agent/blob/main/skills/devops/kanban-worker/SKILL.md) skill plus the `KANBAN_GUIDANCE` system-prompt block, and uses the `kanban_*` tools to terminate the run. No setup beyond defining the profile.
When you create profiles for your fleet, choose names that match the *role* you want the orchestrator to route to. The orchestrator (when there is one) discovers your profile names via `hermes profile list` — there's no fixed roster the system assumes (see the [`kanban-orchestrator`](https://github.com/NousResearch/hermes-agent/blob/main/skills/devops/kanban-orchestrator/SKILL.md) skill for the orchestrator side of the contract).
### Orchestrator profile lane
A specialisation of the profile lane: an orchestrator is a Hermes profile whose toolset includes `kanban` but excludes `terminal` / `file` / `code` / `web` for implementation. Its job is decomposing a high-level goal into child tasks via `kanban_create` + `kanban_link` and stepping back. The orchestrator skill encodes the anti-temptation rules.
## Adding an external CLI worker lane
Wiring a non-Hermes CLI tool (Codex CLI, Claude Code CLI, OpenCode CLI, a local coding-model runner, etc.) as a kanban worker lane is *not yet a paved path*. The dispatcher's spawn function is pluggable (`spawn_fn` is a parameter on `dispatch_once`), and a plugin could register its own `spawn_fn` for a non-Hermes assignee, but the surrounding integration work — wrapping the CLI's exit code into `kanban_complete` / `kanban_block` calls, mapping the CLI's workspace/sandbox conventions onto the dispatcher's `HERMES_KANBAN_WORKSPACE` env, handling auth and per-CLI policy — is still per-integration design work.
If you're considering adding a CLI lane, open an issue describing the specific CLI and the workflow you're trying to enable. The contract above is the constraints any such lane must satisfy; the implementation shape (one plugin per CLI vs a generic CLI-runner plugin parameterised by config) is open.
The historical issue for this is [#19931](https://github.com/NousResearch/hermes-agent/issues/19931) and the closed-not-merged Codex-specific PR [#19924](https://github.com/NousResearch/hermes-agent/pull/19924) — those describe the original architecture proposal but didn't land a runner.
## Failure modes the dispatcher handles
So lane authors don't have to reimplement these:
- **Stale claim TTL** — a worker that claims and then never heartbeats / completes / blocks gets reclaimed after `DEFAULT_CLAIM_TTL_SECONDS` (15 min default) — but only if the worker process has actually died. A live worker (slow model spending 20+ min in one tool-free LLM call) gets the claim *extended* instead of killed; only a dead PID is reclaimed.
- **Crashed worker** — a worker whose host-local PID has vanished is detected by `detect_crashed_workers` and reaped; the task increments `consecutive_failures` and may auto-block when the breaker trips.
- **Run-level retry** — when a task is retried (post-block, post-crash, post-reclaim), the worker can use the `expected_run_id` parameter on terminating tools to fail fast if its own run was already superseded.
- **Per-task max runtime** — `task.max_runtime_seconds` hard-caps wall-clock time per run, regardless of PID liveness. Catches genuinely-deadlocked workers that the live-PID extension would otherwise keep running.
- **Stranded-task detection** — a ready task whose assignee never produces a claim within `kanban.stranded_threshold_seconds` (default 30 min) shows up in `hermes kanban diagnostics` as a `stranded_in_ready` warning. Severity escalates to error at 2x the threshold and critical at 6x. Catches typo'd assignees, deleted profiles, and down external worker pools in one signal — identity-agnostic, no per-board allowlist to curate.
## Related
- [Kanban overview](./kanban) — the user-facing intro.
- [Kanban tutorial](./kanban-tutorial) — walkthrough with the dashboard open.
- [`kanban-worker`](https://github.com/NousResearch/hermes-agent/blob/main/skills/devops/kanban-worker/SKILL.md) — the skill the worker process loads.
- [`kanban-orchestrator`](https://github.com/NousResearch/hermes-agent/blob/main/skills/devops/kanban-orchestrator/SKILL.md) — the orchestrator side.
website/docs/user-guide/features/kanban.md
+964
View File
@@ -0,0 +1,964 @@
---
sidebar_position: 12
title: "Kanban (Multi-Agent Board)"
description: "Durable SQLite-backed task board for coordinating multiple Hermes profiles"
---
# Kanban — Multi-Agent Profile Collaboration
> **Want a walkthrough?** Read the [Kanban tutorial](./kanban-tutorial) — four user stories (solo dev, fleet farming, role pipeline with retry, circuit breaker) with dashboard screenshots of each. This page is the reference; the tutorial is the narrative.
Hermes Kanban is a durable task board, shared across all your Hermes profiles, that lets multiple named agents collaborate on work without fragile in-process subagent swarms. Every task is a row in `~/.hermes/kanban.db`; every handoff is a row anyone can read and write; every worker is a full OS process with its own identity.
### Two surfaces: the model talks through tools, you talk through the CLI
The board has two front doors, both backed by the same `~/.hermes/kanban.db`:
- **Agents drive the board through a dedicated `kanban_*` toolset** — `kanban_show`, `kanban_list`, `kanban_complete`, `kanban_block`, `kanban_heartbeat`, `kanban_comment`, `kanban_create`, `kanban_link`, `kanban_unblock`. The dispatcher spawns each worker with these tools already in its schema; orchestrator profiles can also enable the `kanban` toolset explicitly. The model reads and routes tasks by calling tools directly, *not* by shelling out to `hermes kanban`. See [How workers interact with the board](#how-workers-interact-with-the-board) below.
- **You (and scripts, and cron) drive the board through `hermes kanban …`** on the CLI, `/kanban …` as a slash command, or the dashboard. These are for humans and automation — the places without a tool-calling model behind them.
Both surfaces route through the same `kanban_db` layer, so reads see a consistent view and writes can't drift. The rest of this page shows CLI examples because they're easy to copy-paste, but every CLI verb has a tool-call equivalent the model uses.
This is the shape that covers the workloads `delegate_task` can't:
- **Research triage** — parallel researchers + analyst + writer, human-in-the-loop.
- **Scheduled ops** — recurring daily briefs that build a journal over weeks.
- **Digital twins** — persistent named assistants (`inbox-triage`, `ops-review`) that accumulate memory over time.
- **Engineering pipelines** — decompose → implement in parallel worktrees → review → iterate → PR.
- **Fleet work** — one specialist managing N subjects (50 social accounts, 12 monitored services).
For the full design rationale, comparative analysis against Cline Kanban / Paperclip / NanoClaw / Google Gemini Enterprise, and the eight canonical collaboration patterns, see `docs/hermes-kanban-v1-spec.pdf` in the repository.
## Kanban vs. `delegate_task`
They look similar; they are not the same primitive.
| | `delegate_task` | Kanban |
|---|---|---|
| Shape | RPC call (fork → join) | Durable message queue + state machine |
| Parent | Blocks until child returns | Fire-and-forget after `create` |
| Child identity | Anonymous subagent | Named profile with persistent memory |
| Resumability | None — failed = failed | Block → unblock → re-run; crash → reclaim |
| Human in the loop | Not supported | Comment / unblock at any point |
| Agents per task | One call = one subagent | N agents over task's life (retry, review, follow-up) |
| Audit trail | Lost on context compression | Durable rows in SQLite forever |
| Coordination | Hierarchical (caller → callee) | Peer — any profile reads/writes any task |
**One-sentence distinction:** `delegate_task` is a function call; Kanban is a work queue where every handoff is a row any profile (or human) can see and edit.
**Use `delegate_task` when** the parent agent needs a short reasoning answer before continuing, no humans involved, result goes back into the parent's context.
**Use Kanban when** work crosses agent boundaries, needs to survive restarts, might need human input, might be picked up by a different role, or needs to be discoverable after the fact.
They coexist: a kanban worker may call `delegate_task` internally during its run.
## Core concepts
- **Board** — a standalone queue of tasks with its own SQLite DB, workspaces
directory, and dispatcher loop. A single install can have many boards
(e.g. one per project, repo, or domain); see [Boards (multi-project)](#boards-multi-project)
below. Single-project users stay on the `default` board and never see the
word "board" outside this docs section.
- **Task** — a row with title, optional body, one assignee (a profile name), status (`triage | todo | ready | running | blocked | done | archived`), optional tenant namespace, optional idempotency key (dedup for retried automation).
- **Link** — `task_links` row recording a parent → child dependency. The dispatcher promotes `todo → ready` when all parents are `done`.
- **Comment** — the inter-agent protocol. Agents and humans append comments; when a worker is (re-)spawned it reads the full comment thread as part of its context.
- **Workspace** — the directory a worker operates in. Three kinds:
- `scratch` (default) — fresh tmp dir under `~/.hermes/kanban/workspaces/<id>/` (or `~/.hermes/kanban/boards/<slug>/workspaces/<id>/` on non-default boards). **Deleted when the task completes** — scratch is ephemeral by design, so the dir is wiped the moment the worker (or `hermes kanban complete <id>`) marks the task done. If you want to keep the worker's output, use `worktree:` or `dir:<path>` instead. The first time a scratch workspace is created on an install, the dispatcher logs a warning and emits a `tip_scratch_workspace` event on the task (visible via `hermes kanban show <id>`).
- `dir:<path>` — an existing shared directory (Obsidian vault, mail ops dir, per-account folder). **Must be an absolute path.** Relative paths like `dir:../tenants/foo/` are rejected at dispatch because they'd resolve against whatever CWD the dispatcher happens to be in, which is ambiguous and a confused-deputy escape vector. The path is otherwise trusted — it's your box, your filesystem, the worker runs with your uid. This is the trusted-local-user threat model; kanban is single-host by design. **Preserved on completion.**
- `worktree` — a git worktree under `.worktrees/<id>/` for coding tasks. Use `worktree:<path>` to pin the exact target path. Worker-side `git worktree add` creates it, using `--branch` when provided. **Preserved on completion.**
- **Dispatcher** — a long-lived loop that, every N seconds (default 60): reclaims stale claims, reclaims crashed workers (PID gone but TTL not yet expired), promotes ready tasks, atomically claims, spawns assigned profiles. Runs **inside the gateway** by default (`kanban.dispatch_in_gateway: true`). One dispatcher sweeps all boards per tick; workers are spawned with `HERMES_KANBAN_BOARD` pinned so they can't see other boards. After `kanban.failure_limit` consecutive spawn failures on the same task (default: 2) the dispatcher auto-blocks it with the last error as the reason — prevents thrashing on tasks whose profile doesn't exist, workspace can't mount, etc.
- **Tenant** — optional string namespace *within* a board. One specialist fleet can serve multiple businesses (`--tenant business-a`) with data isolation by workspace path and memory key prefix. Tenants are a soft filter; boards are the hard isolation boundary.
## Boards (multi-project)
Boards let you separate unrelated streams of work — one per project, repo,
or domain — into isolated queues. A new install has exactly one board
called `default` (DB at `~/.hermes/kanban.db` for back-compat). Users who
only want one stream of work never need to know about boards; the feature
is opt-in.
Per-board isolation is absolute:
- Separate SQLite DB per board (`~/.hermes/kanban/boards/<slug>/kanban.db`).
- Separate `workspaces/` and `logs/` directories.
- Workers spawned for a task see **only** their board's tasks — the
dispatcher sets `HERMES_KANBAN_BOARD` in the child env and every
`kanban_*` tool the worker has access to reads it.
- Linking tasks across boards is not allowed (keeps the schema simple; if
you really need cross-project refs, use free-text mentions and look
them up by id manually).
### Managing boards from the CLI
```bash
# See what's on disk. Fresh installs show only "default".
hermes kanban boards list
# Create a new board.
hermes kanban boards create atm10-server \
--name "ATM10 Server" \
--description "Minecraft modded server ops" \
--icon 🎮 \
--switch # optional: make it the active board
# Operate on a specific board without switching.
hermes kanban --board atm10-server list
hermes kanban --board atm10-server create "Restart ATM server" --assignee ops
# Change which board is "current" for subsequent calls.
hermes kanban boards switch atm10-server
hermes kanban boards show # who's active right now?
# Rename the display name (the slug is immutable — it's the directory name).
hermes kanban boards rename atm10-server "ATM10 (Prod)"
# Archive (default) — moves the board's dir to boards/_archived/<slug>-<ts>/.
# Recoverable by moving the dir back.
hermes kanban boards rm atm10-server
# Hard delete — `rm -rf` the board dir. No recovery.
hermes kanban boards rm atm10-server --delete
```
Board resolution order (highest precedence first):
1. Explicit `--board <slug>` on the CLI call.
2. `HERMES_KANBAN_BOARD` env var (set by the dispatcher when spawning a
worker, so workers can't see other boards).
3. `~/.hermes/kanban/current` — the slug persisted by `hermes kanban
boards switch`.
4. `default`.
Slugs are validated: lowercase alphanumerics + hyphens + underscores, 1-64
chars, must start with alphanumeric. Uppercase input is auto-downcased.
Anything else (slashes, spaces, dots, `..`) is rejected at the CLI layer
so path-traversal tricks can't name a board.
### Managing boards from the dashboard
`hermes dashboard` → Kanban tab shows a board switcher at the top as soon
as more than one board exists (or any board has tasks). Single-board users
see only a small `+ New board` button; the switcher is hidden until it
matters.
- **Board dropdown** — pick the active board. Your selection is saved to
the browser's `localStorage` so it persists across reloads without
shifting the CLI's `current` pointer out from under a terminal you left
open.
- **+ New board** — opens a modal asking for slug, display name,
description, and icon. Option to auto-switch to the new board.
- **Archive** — only shown on non-`default` boards. Confirms, then moves
the board dir to `boards/_archived/`.
All dashboard API endpoints accept `?board=<slug>` for board scoping. The
events WebSocket is pinned to a board at connection time; switching in
the UI opens a fresh WS against the new board.
## File attachments
Tasks can carry file attachments — PDFs, images, source documents — so a
worker has the source material it needs without you pasting paths into the
body and hoping it finds them.
- **Upload** — open a task in the dashboard drawer and use the
**Attachments** section's *Upload file* button (multiple files at once
are fine). Each upload is capped at 25 MB.
- **Storage** — files land under
`<hermes-home>/kanban/attachments/<task_id>/` for the default board, or
`<hermes-home>/kanban/boards/<slug>/attachments/<task_id>/` for a named
board. Set `HERMES_KANBAN_ATTACHMENTS_ROOT` to pin a custom location.
- **What the worker sees** — when the dispatcher hands a task to a worker,
the worker's context includes an **Attachments** section listing each
file's name and its **absolute path**. The worker has full file/terminal
tool access, so it reads attachments directly (`read_file`, or shell
tools like `pdftotext`).
- **Download / remove** — the drawer lists each attachment with a download
link and a remove (×) control. Removing an attachment deletes both the
metadata row and the on-disk file.
:::note Remote terminal backends
Attachment paths resolve directly on the **local** terminal backend, which
is the default for Kanban workers. If you run workers on a remote backend
(Docker, Modal), mount the board's `attachments/` directory into the
sandbox so the absolute paths in the worker context are reachable.
:::
## Quick start
The commands below are **you** (the human) setting up the board and creating tasks. Once a task is assigned, the dispatcher spawns the assigned profile as a worker, and from there **the model drives the task through `kanban_*` tool calls, not CLI commands** — see [How workers interact with the board](#how-workers-interact-with-the-board).
```bash
# 1. Create the board (you)
hermes kanban init
# 2. Start the gateway (hosts the embedded dispatcher)
hermes gateway start
# 3. Create a task (you — or an orchestrator agent via kanban_create)
hermes kanban create "research AI funding landscape" --assignee researcher
# 4. Watch activity live (you)
hermes kanban watch
# 5. See the board (you)
hermes kanban list
hermes kanban stats
```
When the dispatcher picks up `t_abcd` and spawns the `researcher` profile, the very first thing that worker's model does is call `kanban_show()` to read its task. It doesn't run `hermes kanban show t_abcd`.
### Gateway-embedded dispatcher (default)
The dispatcher runs inside the gateway process. Nothing to install, no
separate service to manage — if the gateway is up, ready tasks get picked
up on the next tick (60s by default).
```yaml
# config.yaml
kanban:
dispatch_in_gateway: true # default
dispatch_interval_seconds: 60 # default
```
Override the config flag at runtime via `HERMES_KANBAN_DISPATCH_IN_GATEWAY=0`
for debugging. Standard gateway supervision applies: run `hermes gateway
start` directly, or wire the gateway up as a systemd user unit (see the
gateway docs). Without a running gateway, `ready` tasks stay where they are
until one comes up — `hermes kanban create` warns about this at creation
time.
Running `hermes kanban daemon` as a separate process is **deprecated**;
use the gateway. If you truly cannot run the gateway (headless host
policy forbids long-lived services, etc.) a `--force` escape hatch keeps
the old standalone daemon alive for one release cycle, but running both
a gateway-embedded dispatcher AND a standalone daemon against the same
`kanban.db` causes claim races and is not supported.
### Idempotent create (for automation / webhooks)
```bash
# First call creates the task. Any subsequent call with the same key
# returns the existing task id instead of duplicating.
hermes kanban create "nightly ops review" \
--assignee ops \
--idempotency-key "nightly-ops-$(date -u +%Y-%m-%d)" \
--json
```
### Bulk CLI verbs
All the lifecycle verbs accept multiple ids so you can clean up a batch
in one command:
```bash
hermes kanban complete t_abc t_def t_hij --result "batch wrap"
hermes kanban archive t_abc t_def t_hij
hermes kanban unblock t_abc t_def
hermes kanban block t_abc "need input" --ids t_def t_hij
```
## How workers interact with the board
**Workers do not shell out to `hermes kanban`.** When the dispatcher spawns a worker it sets `HERMES_KANBAN_TASK=t_abcd` in the child's env, and that env var flips on a dedicated **kanban toolset** in the model's schema. The same toolset is also available to orchestrator profiles that enable `kanban` in their toolsets config. These tools read and mutate the board directly via the Python `kanban_db` layer, same as the CLI does. A running worker calls these like any other tool; it never sees or needs the `hermes kanban` CLI.
| Tool | Purpose | Required params |
|---|---|---|
| `kanban_show` | Read the current task (title, body, prior attempts, parent handoffs, comments, full pre-formatted `worker_context`). Defaults to the env's task id. | — |
| `kanban_list` | List task summaries with filters for `assignee`, `status`, `tenant`, archived visibility, and limit. Intended for orchestrators discovering board work. | — |
| `kanban_complete` | Finish with `summary` + `metadata` structured handoff. | at least one of `summary` / `result` |
| `kanban_block` | Escalate for human input with a `reason`. | `reason` |
| `kanban_heartbeat` | Signal liveness during long operations. Pure side-effect. | — |
| `kanban_comment` | Append a durable note to the task thread. | `task_id`, `body` |
| `kanban_create` | (Orchestrators) fan out into child tasks with an `assignee`, optional `parents`, `skills`, etc. | `title`, `assignee` |
| `kanban_link` | (Orchestrators) add a `parent_id → child_id` dependency edge after the fact. | `parent_id`, `child_id` |
| `kanban_unblock` | (Orchestrators) move a blocked task back to `ready`. | `task_id` |
A typical worker turn looks like:
```
# Model's tool calls, in order:
kanban_show() # no args — uses HERMES_KANBAN_TASK
# (model reads the returned worker_context, does the work via terminal/file tools)
kanban_heartbeat(note="halfway through — 4 of 8 files transformed")
# (more work)
kanban_complete(
summary="migrated limiter.py to token-bucket; added 14 tests, all pass",
metadata={"changed_files": ["limiter.py", "tests/test_limiter.py"], "tests_run": 14},
)
```
An **orchestrator** worker fans out instead:
```
kanban_show()
kanban_create(
title="research ICP funding 2024-2026",
assignee="researcher-a",
body="focus on seed + series A, North America, AI-adjacent",
)
# → returns {"task_id": "t_r1", ...}
kanban_create(title="research ICP funding — EU angle", assignee="researcher-b", body="…")
# → returns {"task_id": "t_r2", ...}
kanban_create(
title="synthesize findings into launch brief",
assignee="writer",
parents=["t_r1", "t_r2"], # promotes to ready when both complete
body="one-pager, 300 words, neutral tone",
)
kanban_complete(summary="decomposed into 2 research tasks + 1 writer; linked dependencies")
```
The "(Orchestrators)" tools — `kanban_list`, `kanban_create`, `kanban_link`, `kanban_unblock`, and `kanban_comment` on foreign tasks — are available through the same toolset; the convention (enforced by the `kanban-orchestrator` skill) is that worker profiles don't fan out or route unrelated work, and orchestrator profiles don't execute implementation work. Dispatcher-spawned workers are still task-scoped for destructive lifecycle operations and cannot mutate unrelated tasks.
### Why tools instead of shelling to `hermes kanban`
Three reasons:
1. **Backend portability.** Workers whose terminal tool points at a remote backend (Docker / Modal / Singularity / SSH) would run `hermes kanban complete` *inside* the container, where `hermes` isn't installed and `~/.hermes/kanban.db` isn't mounted. The kanban tools run in the agent's own Python process and always reach `~/.hermes/kanban.db` regardless of terminal backend.
2. **No shell-quoting fragility.** Passing `--metadata '{"files": [...]}'` through shlex + argparse is a latent footgun. Structured tool args skip it entirely.
3. **Better errors.** Tool results are structured JSON the model can reason about, not stderr strings it has to parse.
**Zero schema footprint on normal sessions.** A regular `hermes chat` session has zero `kanban_*` tools in its schema unless the active profile explicitly enables the `kanban` toolset for orchestrator work. Dispatcher-spawned task workers get task-scoped tools because `HERMES_KANBAN_TASK` is set; orchestrator profiles get the broader routing surface through config. No tool bloat for users who never touch kanban.
The `kanban-worker` and `kanban-orchestrator` skills teach the model which tool to call when and in what order.
### Recommended handoff evidence
`kanban_complete(summary=..., metadata={...})` is intentionally flexible:
the summary is the human-readable closeout, and `metadata` is the
machine-readable handoff that downstream agents, reviewers, or dashboards can
reuse without scraping prose.
For engineering and review tasks, prefer this optional metadata shape:
```json
{
"changed_files": ["path/to/file.py"],
"verification": ["pytest tests/hermes_cli/test_kanban_db.py -q"],
"dependencies": ["parent task id or external issue, if any"],
"blocked_reason": null,
"retry_notes": "what failed before, if this was a retry",
"residual_risk": ["what was not tested or still needs human review"]
}
```
These keys are a convention, not a schema requirement. The useful property is
that every worker leaves enough evidence for the next reader to answer four
questions quickly:
1. What changed?
2. How was it verified?
3. What can unblock or retry this if it fails?
4. What risk is still deliberately left open?
Keep secrets, raw logs, tokens, OAuth material, and unrelated transcripts out of
`metadata`. Store pointers and summaries instead. If a task has no files or
tests, say so explicitly in `summary` and use `metadata` for the evidence that
does exist, such as source URLs, issue ids, or manual review steps.
### The worker skill
Any profile that should be able to work kanban tasks must load the `kanban-worker` skill. It teaches the worker the full lifecycle in **tool calls**, not CLI commands:
1. On spawn, call `kanban_show()` to read title + body + parent handoffs + prior attempts + full comment thread.
2. `cd $HERMES_KANBAN_WORKSPACE` (via the terminal tool) and do the work there.
3. Call `kanban_heartbeat(note="...")` every few minutes during long operations. **If your work may run longer than 1 hour, call `kanban_heartbeat` at least once an hour** — the dispatcher reclaims tasks that have been running past `kanban.dispatch_stale_timeout_seconds` (default 4 h) with no heartbeat in the last hour, on the assumption the worker crashed without cleanup. A reclaim is benign (the task goes back to `ready` for re-dispatch without a failure-counter tick) but you lose your current run's progress.
4. Complete with `kanban_complete(summary="...", metadata={...})`, or `kanban_block(reason="...")` if stuck.
That final `kanban_complete` / `kanban_block` call is part of the worker
protocol. If the worker process exits with status 0 while the task is still
`running`, the dispatcher treats that as a protocol violation, emits a
`protocol_violation` event, and auto-blocks the task on the next tick instead
of respawning it into the same loop. This usually means the model wrote a
plain-text answer and exited without using the Kanban tool surface.
`kanban-worker` is a bundled skill, synced into every profile during install and
update — there is no separate Skills Hub install step. Verify it is present in
whichever profile you use for kanban workers (`researcher`, `writer`, `ops`,
etc.):
```bash
hermes -p <your-worker-profile> skills list | grep kanban-worker
```
If the bundled copy is missing, restore it for that profile:
```bash
hermes -p <your-worker-profile> skills reset kanban-worker --restore
```
The dispatcher also auto-passes `--skills kanban-worker` when spawning every worker, so the worker always has the pattern library available even if a profile's default skills config doesn't include it.
### Pinning extra skills to a specific task
Sometimes a single task needs specialist context the assignee profile doesn't carry by default — a translation job that needs the `translation` skill, a review task that needs `github-code-review`, a security audit that needs `security-pr-audit`. Rather than editing the assignee's profile every time, attach the skills directly to the task.
**From an orchestrator agent** (the usual case — one agent routing work to another), use the `kanban_create` tool's `skills` array:
```
kanban_create(
title="translate README to Japanese",
assignee="linguist",
skills=["translation"],
)
kanban_create(
title="audit auth flow",
assignee="reviewer",
skills=["security-pr-audit", "github-code-review"],
)
```
**From a human (CLI / slash command)**, repeat `--skill` for each one:
```bash
hermes kanban create "translate README to Japanese" \
--assignee linguist \
--skill translation
hermes kanban create "audit auth flow" \
--assignee reviewer \
--skill security-pr-audit \
--skill github-code-review
```
**From the dashboard**, type the skills comma-separated into the **skills** field of the inline create form.
These skills are **additive** to the built-in `kanban-worker` — the dispatcher emits one `--skills <name>` flag for each (and for the built-in), so the worker spawns with all of them loaded. The skill names must match skills that are actually installed on the assignee's profile (run `hermes skills list` to see what's available); there's no runtime install.
### Goal-mode cards (`--goal`)
By default each worker gets **one shot** at its card — do the work, call `kanban_complete`/`kanban_block`, exit. Pass `--goal` (CLI) or `goal_mode=True` (the `kanban_create` tool / dashboard) to instead run that worker in a **goal loop**, the same Ralph-style engine behind the `/goal` slash command: after every turn an auxiliary judge checks the worker's output against the card's title + body (treated as the acceptance criteria), and if the work isn't done — and the turn budget remains — the worker keeps going **in the same session** until the judge agrees, the worker terminates the task itself, or the budget runs out (which **blocks** the card for human review rather than exiting silently).
```bash
hermes kanban create "Translate the docs site to French" \
--body "Acceptance: every page translated, no English left, links intact." \
--assignee linguist \
--goal \
--goal-max-turns 15 # optional; default 20
```
Use it for open-ended, multi-step, or "keep going until X is true" cards. Skip it for cheap one-shot work — the per-turn judge overhead isn't worth it, and the dispatcher's existing retry/circuit-breaker already handles transient worker failures. The judge is only as good as your goal text, so write the body as **explicit acceptance criteria**.
### The orchestrator skill
A **well-behaved orchestrator does not do the work itself.** It decomposes the user's goal into tasks, links them, assigns each to one of the profiles you've set up, and steps back. The `kanban-orchestrator` skill encodes this as tool-call patterns: anti-temptation rules, a Step-0 profile-discovery prompt (the dispatcher silently fails on unknown assignee names, so the orchestrator must ground every card in profiles that actually exist on your machine), and a decomposition playbook keyed on `kanban_create` / `kanban_link` / `kanban_comment`.
A canonical orchestrator turn (two parallel researchers handing off to a writer):
```
# Goal from user: "draft a launch post on the ICP funding landscape"
kanban_create(title="research ICP funding, NA angle", assignee="researcher-a", body="…") # → t_r1
kanban_create(title="research ICP funding, EU angle", assignee="researcher-b", body="…") # → t_r2
kanban_create(
title="synthesize ICP funding research into launch post draft",
assignee="writer",
parents=["t_r1", "t_r2"], # promoted to 'ready' when both researchers complete
body="one-pager, neutral tone, cite sources inline",
) # → t_w1
# Optional: add cross-cutting deps discovered later without re-creating tasks
kanban_link(parent_id="t_r1", child_id="t_followup")
kanban_complete(
summary="decomposed into 2 parallel research tasks → 1 synthesis task; writer starts when both researchers finish",
)
```
`kanban-orchestrator` is a bundled skill. It is synced into each profile during
install and update, so there is no separate Skills Hub install step. Verify it is
present in your orchestrator profile:
```bash
hermes -p orchestrator skills list | grep kanban-orchestrator
```
If the bundled copy is missing, restore it for that profile:
```bash
hermes -p orchestrator skills reset kanban-orchestrator --restore
```
For best results, pair it with a profile whose toolsets are restricted to board operations (`kanban`, `gateway`, `memory`) so the orchestrator literally cannot execute implementation tasks even if it tries.
## Dashboard (GUI)
The `/kanban` CLI and slash command are enough to run the board headlessly, but a visual board is often the right interface for humans-in-the-loop: triage, cross-profile supervision, reading comment threads, and dragging cards between columns. Hermes ships this as a **bundled dashboard plugin** at `plugins/kanban/` — not a core feature, not a separate service — following the model laid out in [Extending the Dashboard](./extending-the-dashboard).
Open it with:
```bash
hermes kanban init # one-time: create kanban.db if not already present
hermes dashboard # "Kanban" tab appears in the nav, after "Skills"
```
### What the plugin gives you
- A **Kanban** tab showing one column per status: `triage`, `todo`, `ready`, `running`, `blocked`, `done` (plus `archived` when the toggle is on).
- `triage` is the parking column for rough ideas. By default (`kanban.auto_decompose: true`), the dispatcher auto-runs the **decomposer** on tasks that land here. The built-in decomposer uses the `auxiliary.kanban_decomposer` model path, reads your profile roster (with descriptions), and fans the task out into a small graph of child tasks routed to the best-fit specialists. The original task stays alive as the parent of every child so its assignee (`kanban.orchestrator_profile`, or the active default profile when unset) wakes back up to judge completion when everything finishes. Flip the **Orchestration: Auto/Manual** pill at the top of the page (emerald = Auto, muted gray = Manual), or by editing `config.yaml` directly. Both modes coexist with `hermes kanban specify` - that's still available as a single-task spec rewrite when you don't want fan-out.
- Cards show the task id, title, priority badge, tenant tag, assigned profile, comment/link counts, a **progress pill** (`N/M` children done when the task has dependents), and "created N ago". A per-card checkbox enables multi-select.
- **Per-profile lanes inside Running** — toolbar checkbox toggles sub-grouping of the Running column by assignee.
- **Live updates via WebSocket** — the plugin tails the append-only `task_events` table on a short poll interval; the board reflects changes the instant any profile (CLI, gateway, or another dashboard tab) acts. Reloads are debounced so a burst of events triggers a single refetch.
- **Drag-drop** cards between columns to change status. The drop sends `PATCH /api/plugins/kanban/tasks/:id` which routes through the same `kanban_db` code the CLI uses — the three surfaces can never drift. Moves into destructive statuses (`done`, `archived`, `blocked`) prompt for confirmation. Touch devices use a pointer-based fallback so the board is usable from a tablet.
- **Inline create** — click `+` on any column header to type a title, assignee, priority, and (optionally) a parent task from a dropdown over every existing task. Press Enter to create the task, Shift+Enter to insert a newline in the title field, or Escape to cancel. Creating from the Triage column automatically parks the new task in triage.
- **Multi-select with bulk actions** — shift/ctrl-click a card or tick its checkbox to add it to the selection. A bulk action bar appears at the top with batch status transitions, archive, and reassign (by profile dropdown, or "(unassign)"). Destructive batches confirm first. Per-id partial failures are reported without aborting the rest.
- **Click a card** (without shift/ctrl) to open a side drawer (Escape or click-outside closes) with:
- **Editable title** — click the heading to rename.
- **Editable assignee / priority** — click the meta row to rewrite.
- **Editable description** — markdown-rendered by default (headings, bold, italic, inline code, fenced code, `http(s)` / `mailto:` links, bullet lists), with an "edit" button that swaps in a textarea. Markdown rendering is a tiny, XSS-safe renderer — every substitution runs on HTML-escaped input, only `http(s)` / `mailto:` links pass through, and `target="_blank"` + `rel="noopener noreferrer"` are always set.
- **Dependency editor** — chip list of parents and children, each with an `×` to unlink, plus dropdowns over every other task to add a new parent or child. Cycle attempts are rejected server-side with a clear message.
- **Status action row** (→ triage / → ready / → running / block / unblock / complete / archive) with confirm prompts for destructive transitions. For cards in the **Triage** column the row also exposes two LLM-driven actions: **⚗ Decompose** fans the task out into a graph of child tasks routed to specialist profiles by description, and **✨ Specify** does a single-task spec rewrite. Decompose falls back to specify-style promotion when the LLM decides the task doesn't benefit from fan-out, so it's a strict superset. Both are reachable from the CLI (`hermes kanban decompose <id>` / `specify <id>` / `--all`), from any gateway platform (`/kanban decompose <id>`), and programmatically via `POST /api/plugins/kanban/tasks/:id/decompose` and `…/specify`. Configure the models under `auxiliary.kanban_decomposer` and `auxiliary.triage_specifier` in `config.yaml`.
- Result section (also markdown-rendered), comment thread with Enter-to-submit, the last 20 events.
- **Toolbar filters** — free-text search, tenant dropdown (defaults to `dashboard.kanban.default_tenant` from `config.yaml`), assignee dropdown, "show archived" toggle, "lanes by profile" toggle, and a **Nudge dispatcher** button so you don't have to wait for the next 60 s tick.
Visually the target is the familiar Linear / Fusion layout: dark theme, column headers with counts, coloured status dots, pill chips for priority and tenant. The plugin reads only theme CSS vars (`--color-*`, `--radius`, `--font-mono`, ...), so it reskins automatically with whichever dashboard theme is active.
### Auto vs Manual orchestration
The kanban board has two ways to handle a task you drop into the Triage column:
**Auto (default)** — `kanban.auto_decompose: true`. The gateway-embedded dispatcher runs the **decomposer** on each tick, capped by `kanban.auto_decompose_per_tick` (default 3 tasks per tick) so a bulk-load of triage tasks doesn't burst-spend the auxiliary LLM. The decomposer uses the built-in decomposition prompt plus the `auxiliary.kanban_decomposer` model path, reads your installed profiles + their descriptions, and asks the LLM to produce a JSON task graph: which tasks to spawn, who they go to, and which depend on which. The original triage task becomes the parent of every leaf in the graph, so it stays alive until the whole graph completes - and then promotes back to `ready` so its assignee (`kanban.orchestrator_profile`, or the active default profile when unset) can judge completion and add more tasks if the work isn't done. This is the "drop a one-liner, walk away" flow.
**Manual** — `kanban.auto_decompose: false`. Triage tasks stay in triage until you act. Click the **⚗ Decompose** button on a card, run `hermes kanban decompose <id>` (or `--all`), or use `/kanban decompose <id>` from a chat. This matches the pre-decomposer behavior of the board, useful when you want full control over what runs when.
Flip between the two modes from the **Orchestration: Auto/Manual** pill at the top of the kanban page (emerald = Auto, muted gray = Manual), or by editing `config.yaml` directly. Both modes coexist with `hermes kanban specify` — that's still available as a single-task spec rewrite when you don't want fan-out.
The decomposer's routing decisions depend on profile descriptions, which is a per-profile labeling primitive you set with `hermes profile create --description "..."`, `hermes profile describe <name> --text "..."`, `hermes profile describe <name> --auto` (LLM-generates from the profile's installed skills + model), or the dashboard's per-profile editor in the expanded **Orchestration settings** panel. Profiles without a description still appear in the roster — they're routable by name, just less precisely. The decomposer NEVER lands a child task with `assignee=None`: when the LLM picks an unknown profile, the child gets routed to `kanban.default_assignee` (or the active default profile if that's unset).
`kanban.orchestrator_profile` does not load that profile's prompt, skills, or custom logic into the decomposition call. It controls who owns the root/orchestration task after fan-out. To change the decomposer's model/provider, configure `auxiliary.kanban_decomposer`. To use a profile's custom task-splitting logic instead of the built-in decomposer, switch to Manual mode and have that profile create or decompose tasks explicitly.
Config knobs (all under `kanban:` in `~/.hermes/config.yaml`):
| Key | Default | Purpose |
|---|---|---|
| `auto_decompose` | `true` | Dispatcher auto-runs the decomposer every tick. |
| `auto_decompose_per_tick` | `3` | Cap on decompositions per dispatcher tick. Excess defers to the next tick. |
| `orchestrator_profile` | `""` | Profile assigned to the root/orchestration task after decomposition. Empty = fall back to active default profile. |
| `default_assignee` | `""` | Where a child task lands when the LLM picks an unknown profile. Empty = fall back to active default. |
And the two auxiliary LLM slots:
| Key | Purpose |
|---|---|
| `auxiliary.kanban_decomposer` | Model that produces the task graph (called by Decompose). Set `provider`/`model` to override the main chat model. |
| `auxiliary.profile_describer` | Model that auto-generates profile descriptions (called by `hermes profile describe --auto`). |
### Architecture
The GUI is strictly a **read-through-the-DB + write-through-kanban_db** layer with no domain logic of its own:
<!-- ascii-guard-ignore -->
```
┌────────────────────────┐ WebSocket (tails task_events)
│ React SPA (plugin) │ ◀──────────────────────────────────┐
│ HTML5 drag-and-drop │ │
└──────────┬─────────────┘ │
│ REST over fetchJSON │
▼ │
┌────────────────────────┐ writes call kanban_db.* │
│ FastAPI router │ directly — same code path │
│ plugins/kanban/ │ the CLI /kanban verbs use │
│ dashboard/plugin_api.py │
└──────────┬─────────────┘ │
│ │
▼ │
┌────────────────────────┐ │
│ ~/.hermes/kanban.db │ ───── append task_events ──────────┘
│ (WAL, shared) │
└────────────────────────┘
```
<!-- ascii-guard-ignore-end -->
### REST surface
All routes are mounted under `/api/plugins/kanban/` and protected by the dashboard's ephemeral session token:
| Method | Path | Purpose |
|---|---|---|
| `GET` | `/board?tenant=<name>&include_archived=…` | Full board grouped by status column, plus tenants + assignees for filter dropdowns |
| `GET` | `/tasks/:id` | Task + comments + events + links |
| `POST` | `/tasks` | Create (wraps `kanban_db.create_task`, accepts `triage: bool` and `parents: [id, …]`) |
| `PATCH` | `/tasks/:id` | Status / assignee / priority / title / body / result |
| `POST` | `/tasks/bulk` | Apply the same patch (status / archive / assignee / priority) to every id in `ids`. Per-id failures reported without aborting siblings |
| `POST` | `/tasks/:id/comments` | Append a comment |
| `POST` | `/tasks/:id/specify` | Run the triage specifier — auxiliary LLM fleshes out the task body and promotes it from `triage` to `todo`. Returns `{ok, task_id, reason, new_title}`; `ok=false` with a human-readable reason on "not in triage" / no aux client / LLM error is a 200, not a 4xx |
| `POST` | `/tasks/:id/decompose` | Run the kanban decomposer — auxiliary LLM produces a task graph and the helper atomically creates the children + links the root + flips `triage → todo`. Returns `{ok, task_id, reason, fanout, child_ids, new_title}`. Same 200-on-LLM-error convention as `/specify`. |
| `GET` | `/profiles` | List installed profiles with their descriptions (consumed by the dashboard's profile-description editor and the orchestrator picker). |
| `PATCH` | `/profiles/:name` | Set or clear a profile's description (user-authored — `description_auto: false`). Returns `{ok, profile, description}`. |
| `POST` | `/profiles/:name/describe-auto` | Generate a description for a profile via `auxiliary.profile_describer`. Persists with `description_auto: true` so the dashboard can surface a "review" badge. |
| `GET` | `/orchestration` | Read the kanban orchestration settings (`orchestrator_profile`, `default_assignee`, `auto_decompose`) plus the *resolved* effective values after fallbacks. |
| `PUT` | `/orchestration` | Update one or more of the three orchestration keys in `config.yaml`. Validates that non-empty profile names actually exist. |
| `POST` | `/links` | Add a dependency (`parent_id` → `child_id`) |
| `DELETE` | `/links?parent_id=…&child_id=…` | Remove a dependency |
| `POST` | `/dispatch?max=…&dry_run=…` | Nudge the dispatcher — skip the 60 s wait |
| `GET` | `/config` | Read `dashboard.kanban` preferences from `config.yaml` — `default_tenant`, `lane_by_profile`, `include_archived_by_default`, `render_markdown` |
| `WS` | `/events?since=<event_id>` | Live stream of `task_events` rows |
Every handler is a thin wrapper — the plugin is ~700 lines of Python (router + WebSocket tail + bulk batcher + config reader) and adds no new business logic. A tiny `_conn()` helper auto-initializes `kanban.db` on every read and write, so a fresh install works whether the user opened the dashboard first, hit the REST API directly, or ran `hermes kanban init`.
### Dashboard config
Any of these keys under `dashboard.kanban` in `~/.hermes/config.yaml` changes the tab's defaults — the plugin reads them at load time via `GET /config`:
```yaml
dashboard:
kanban:
default_tenant: acme # preselects the tenant filter
lane_by_profile: true # default for the "lanes by profile" toggle
include_archived_by_default: false
render_markdown: true # set false for plain <pre> rendering
```
Each key is optional and falls back to the shown default.
### Security model
The dashboard's HTTP auth middleware [explicitly skips `/api/plugins/`](./extending-the-dashboard#backend-api-routes) — plugin routes are unauthenticated by design because the dashboard binds to localhost by default. That means the kanban REST surface is reachable from any process on the host.
The WebSocket takes one additional step: it requires the dashboard's ephemeral session token as a `?token=…` query parameter (browsers can't set `Authorization` on an upgrade request), matching the pattern used by the in-browser PTY bridge.
If you run `hermes dashboard --host 0.0.0.0`, every plugin route — kanban included — becomes reachable from the network. **Don't do that on a shared host.** The board contains task bodies, comments, and workspace paths; an attacker reaching these routes gets read access to your entire collaboration surface and can also create / reassign / archive tasks.
Tasks in `~/.hermes/kanban.db` are profile-agnostic on purpose (that's the coordination primitive). If you open the dashboard with `hermes -p <profile> dashboard`, the board still shows tasks created by any other profile on the host. Same user owns all profiles, but this is worth knowing if multiple personas coexist.
### Live updates
`task_events` is an append-only SQLite table with a monotonic `id`. The WebSocket endpoint holds each client's last-seen event id and pushes new rows as they land. When a burst of events arrives, the frontend reloads the (very cheap) board endpoint — simpler and more correct than trying to patch local state from every event kind. WAL mode means the read loop never blocks the dispatcher's `BEGIN IMMEDIATE` claim transactions.
### Extending it
The plugin uses the standard Hermes dashboard plugin contract — see [Extending the Dashboard](./extending-the-dashboard) for the full manifest reference, shell slots, page-scoped slots, and the Plugin SDK. Extra columns, custom card chrome, tenant-filtered layouts, or full `tab.override` replacements are all expressible without forking this plugin.
To disable without removing: add `dashboard.plugins.kanban.enabled: false` to `config.yaml` (or delete `plugins/kanban/dashboard/manifest.json`).
### Scope boundary
The GUI is deliberately thin. Everything the plugin does is reachable from the CLI; the plugin just makes it comfortable for humans. Auto-assignment, budgets, governance gates, and org-chart views remain user-space — a router profile, another plugin, or a reuse of `tools/approval.py` — exactly as listed in the out-of-scope section of the design spec.
## CLI command reference
This is the surface **you** (or scripts, cron, the dashboard) use to drive the board. Workers running inside the dispatcher use the `kanban_*` [tool surface](#how-workers-interact-with-the-board) for the same operations — the CLI here and the tools there both route through `kanban_db`, so the two surfaces agree by construction.
```
hermes kanban init # create kanban.db + print daemon hint
hermes kanban create "<title>" [--body ...] [--assignee <profile>]
[--parent <id>]... [--tenant <name>]
[--workspace scratch|worktree|worktree:<path>|dir:<path>]
[--branch <name>]
[--priority N] [--triage] [--idempotency-key KEY]
[--max-runtime 30m|2h|1d|<seconds>]
[--max-retries N]
[--goal] [--goal-max-turns N]
[--skill <name>]...
[--json]
hermes kanban list [--mine] [--assignee P] [--status S] [--tenant T] [--archived]
[--workflow-template-id <id>] [--current-step-key <key>]
[--sort created|created-desc|priority|priority-desc|status|assignee|title|updated]
[--json]
hermes kanban show <id> [--json]
hermes kanban assign <id> <profile> # or 'none' to unassign
hermes kanban reassign <id>... <profile> # bulk re-assign tasks to a profile
hermes kanban edit <id> [--title ...] [--body ...] # edit task title / body / priority in place
[--priority N]
hermes kanban promote <id>... # move todo/blocked tasks to ready (recovery)
hermes kanban schedule <id> --at <ISO8601> # set/clear a task's scheduled_at start time
hermes kanban diagnostics [--json] # board health snapshot (alias: diag)
hermes kanban link <parent_id> <child_id>
hermes kanban unlink <parent_id> <child_id>
hermes kanban claim <id> [--ttl SECONDS]
hermes kanban comment <id> "<text>" [--author NAME]
# Bulk verbs — accept multiple ids:
hermes kanban complete <id>... [--result "..."]
hermes kanban block <id> "<reason>" [--ids <id>...]
hermes kanban unblock <id>...
hermes kanban archive <id>...
hermes kanban tail <id> # follow a single task's event stream
hermes kanban watch [--assignee P] [--tenant T] # live stream ALL events to the terminal
[--kinds completed,blocked,…] [--interval SECS]
hermes kanban heartbeat <id> [--note "..."] # worker liveness signal for long ops
hermes kanban runs <id> [--json] # attempt history (one row per run)
hermes kanban assignees [--json] # profiles on disk + per-assignee task counts
hermes kanban dispatch [--dry-run] [--max N] # one-shot pass
[--failure-limit N] [--json]
hermes kanban daemon --force # DEPRECATED — standalone dispatcher (use `hermes gateway start` instead)
[--failure-limit N] [--pidfile PATH] [-v]
hermes kanban stats [--json] # per-status + per-assignee counts
hermes kanban log <id> [--tail BYTES] # worker log from ~/.hermes/kanban/logs/
hermes kanban notify-subscribe <id> # gateway bridge hook (used by /kanban in the gateway)
--platform <name> --chat-id <id> [--thread-id <id>] [--user-id <id>]
hermes kanban notify-list [<id>] [--json]
hermes kanban notify-unsubscribe <id>
--platform <name> --chat-id <id> [--thread-id <id>]
hermes kanban context <id> # what a worker sees
hermes kanban specify [<id> | --all] [--tenant T] # flesh out a triage-column idea
[--author NAME] [--json] # into a full spec and promote to todo
hermes kanban gc [--event-retention-days N] # workspaces + old events + old logs
[--log-retention-days N]
```
All commands are also available as a slash command in the interactive CLI and in the messaging gateway (see [`/kanban` slash command](#kanban-slash-command) below).
`--max-retries` is a per-task circuit-breaker override for the dispatcher. `--max-retries 1` blocks the task on the first non-successful attempt, while `--max-retries 3` allows two retries and blocks on the third failure. Omit it to use `kanban.failure_limit` from `config.yaml`, then the built-in default.
### Concurrency, scheduling, and child promotion config
| Config key | Default | What it does |
|------------|---------|--------------|
| `kanban.max_in_progress` | unset (unlimited) | Caps the number of simultaneously running tasks. When the board already has N running, the dispatcher skips spawning more — useful for slow workers (local LLMs, resource-constrained hosts) so they finish what they have before more pile up and time out. Invalid or below-1 values log a warning and behave as unlimited. |
| `kanban.max_in_progress_per_profile` | unset (unlimited) | Per-profile variant of `max_in_progress` — caps how many tasks any single assignee profile may run concurrently. Useful when one profile is slow or rate-limited but others should keep flowing. Applies alongside the board-wide `max_in_progress`; both must allow a spawn for it to proceed. |
| `kanban.auto_promote_children` | `true` | After `decompose_triage_task()` produces children with no parent-blocker dependencies, they're automatically promoted to `ready` so the dispatcher can pick them up. Set to `false` to require manual review — children stay in `todo` until you promote them. |
| `kanban.default_workdir` | unset | Board-level default working directory applied to new tasks when neither `--workspace` nor the task itself overrides it. Per-task `workspace:` still wins. |
```yaml
kanban:
max_in_progress: 2
auto_promote_children: false
default_workdir: ~/work/active-project
```
### Scheduled task starts (`scheduled_at`)
Set `scheduled_at` on a task to delay dispatch until a specific time. The dispatcher skips ready tasks whose `scheduled_at` is in the future and picks them up on the first tick after that timestamp.
```bash
hermes kanban create "nightly backup audit" \
--assignee ops --scheduled-at "2026-06-01T03:00:00Z"
```
### Respawn guard
The dispatcher refuses to re-spawn a ready task when it hit a quota/auth/429 error on the previous run (`blocker_auth`), or completed a run successfully within the guard window (`recent_success`), or a recent task comment links to a GitHub PR (`active_pr`). This prevents repeat worker storms on the same bug or task while a human catches up. See the `respawn_guarded` row in the [event reference](#event-reference).
### Drag-to-delete and bulk delete (dashboard)
The dashboard exposes a **trash drop zone** on the kanban page — drag any card into it to delete the task (cascades through `task_events`, child links, and subscriptions). A confirmation prompt protects against accidents. Bulk delete is also reachable via `DELETE /api/plugins/kanban/tasks` with a JSON body `{"ids": ["t_abc", "t_def", ...]}`.
### Worker visibility endpoints
The dashboard plugin API now exposes these read-only endpoints (plus a run-control verb) for external monitors:
| Endpoint | Returns |
|----------|---------|
| `GET /api/plugins/kanban/workers/active` | Currently spawned workers with PID, profile, task id, started-at, last heartbeat |
| `GET /api/plugins/kanban/runs/{id}` | Single-run detail — task id, status, started/ended, exit code, log path |
| `POST /api/plugins/kanban/runs/{run_id}/terminate` | Terminate a reclaimable run — stops the worker and frees the task for re-dispatch |
| `GET /api/plugins/kanban/inspect` | Combined dispatcher snapshot — backlog, in-progress count vs. `max_in_progress`, recent events |
All of these are gated by the same dashboard plugin auth as the rest of the kanban plugin API.
### Kanban Swarm topology helper
`hermes kanban swarm` creates a durable **Kanban Swarm v1** graph in one shot: a completed root/blackboard card, N parallel worker cards, a verifier card gated on all workers, and a synthesizer card gated on the verifier. Shared swarm context (the "blackboard") is stored as structured JSON comments on the root card so any worker can read it.
```bash
hermes kanban swarm "Design a multi-region failover plan" \
--workers researcher,architect,sre \
--verifier reviewer --synthesizer writer
```
The resulting graph dispatches normally — workers run in parallel, the verifier wakes after they all finish, the synthesizer wakes after the verifier marks the work clean.
## `/kanban` slash command {#kanban-slash-command}
Every `hermes kanban <action>` verb is also reachable as `/kanban <action>` — from inside an interactive `hermes chat` session **and** from any gateway platform (Telegram, Discord, Slack, WhatsApp, Signal, Matrix, Mattermost, email, SMS). Both surfaces call the exact same `hermes_cli.kanban.run_slash()` entry point that reuses the `hermes kanban` argparse tree, so the argument surface, flags, and output format are identical across CLI, `/kanban`, and `hermes kanban`. You don't have to leave the chat to drive the board.
```
/kanban list
/kanban show t_abcd
/kanban create "write launch post" --assignee writer --parent t_research
/kanban comment t_abcd "looks good, ship it"
/kanban unblock t_abcd
/kanban dispatch --max 3
/kanban specify t_abcd # flesh out a triage one-liner into a real spec
/kanban specify --all --tenant engineering # sweep every triage task in one tenant
```
Quote multi-word arguments the same way you would on a shell — `run_slash` parses the rest of the line with `shlex.split`, so `"..."` and `'...'` both work.
### Mid-run usage: `/kanban` bypasses the running-agent guard
The gateway normally queues slash commands and user messages while an agent is still thinking — that's what stops you from accidentally starting a second turn while the first is in flight. **`/kanban` is explicitly exempted from this guard.** The board lives in `~/.hermes/kanban.db`, not in the running agent's state, so reads (`list`, `show`, `context`, `tail`, `watch`, `stats`, `runs`) and writes (`comment`, `unblock`, `block`, `assign`, `archive`, `create`, `link`, …) all go through immediately, even mid-turn.
This is the whole point of the separation:
- A worker blocks waiting on a peer → you send `/kanban unblock t_abcd` from your phone and the dispatcher picks the peer up on its next tick. The blocked worker isn't interrupted — it just stops being blocked.
- You spot a card that needs human context → `/kanban comment t_xyz "use the 2026 schema, not 2025"` lands on the task thread and the *next* run of that task will read it in `kanban_show()`.
- You want to know what your fleet is doing without stopping the orchestrator → `/kanban list --mine` or `/kanban stats` inspects the board without touching your main conversation.
### Auto-subscribe on `/kanban create` (gateway only)
When you create a task from the gateway with `/kanban create "…"`, the originating chat (platform + chat id + thread id) is automatically subscribed to that task's terminal events (`completed`, `blocked`, `gave_up`, `crashed`, `timed_out`). You'll get one message back per terminal event — including the first line of the worker's result summary on `completed` — without having to poll or remember the task id.
```
you> /kanban create "transcribe today's podcast" --assignee transcriber
bot> Created t_9fc1a3 (ready, assignee=transcriber)
(subscribed — you'll be notified when t_9fc1a3 completes or blocks)
… ~8 minutes later …
bot> ✓ t_9fc1a3 completed by transcriber
transcribed 42 minutes, saved to podcast/2026-05-04.md
```
Subscriptions auto-remove themselves once the task reaches `done` or `archived`. If you script a create with `--json` (machine output) the auto-subscribe is skipped — the assumption is that scripted callers want to manage subscriptions explicitly via `/kanban notify-subscribe`.
### Output truncation in messaging
Gateway platforms have practical message-length caps. If `/kanban list`, `/kanban show`, or `/kanban tail` produce more than ~3800 characters of output, the response is truncated with a `… (truncated; use \`hermes kanban …\` in your terminal for full output)` footer. The CLI surface has no such cap.
### Autocomplete
In the interactive CLI, typing `/kanban ` and hitting Tab cycles through the built-in subcommand list (`list`, `ls`, `show`, `create`, `assign`, `link`, `unlink`, `claim`, `comment`, `complete`, `block`, `unblock`, `archive`, `tail`, `dispatch`, `context`, `init`, `gc`). The remaining verbs listed in the CLI reference above (`watch`, `stats`, `runs`, `log`, `assignees`, `heartbeat`, `notify-subscribe`, `notify-list`, `notify-unsubscribe`, `daemon`) also work — they're just not in the autocomplete hint list yet.
## Collaboration patterns
The board supports these eight patterns without any new primitives:
| Pattern | Shape | Example |
|---|---|---|
| **P1 Fan-out** | N siblings, same role | "research 5 angles in parallel" |
| **P2 Pipeline** | role chain: scout → editor → writer | daily brief assembly |
| **P3 Voting / quorum** | N siblings + 1 aggregator | 3 researchers → 1 reviewer picks |
| **P4 Long-running journal** | same profile + shared dir + cron | Obsidian vault |
| **P5 Human-in-the-loop** | worker blocks → user comments → unblock | ambiguous decisions |
| **P6 `@mention`** | inline routing from prose | `@reviewer look at this` |
| **P7 Thread-scoped workspace** | `/kanban here` in a thread | per-project gateway threads |
| **P8 Fleet farming** | one profile, N subjects | 50 social accounts |
| **P9 Triage specifier** | rough idea → `triage` → `hermes kanban specify` expands body → `todo` | "turn this one-liner into a spec'd task" |
For worked examples of each, see `docs/hermes-kanban-v1-spec.pdf`.
## Multi-tenant usage
When one specialist fleet serves multiple businesses, tag each task with a tenant:
```bash
hermes kanban create "monthly report" \
--assignee researcher \
--tenant business-a \
--workspace dir:~/tenants/business-a/data/
```
Workers receive `$HERMES_TENANT` and namespace their memory writes by prefix. The board, the dispatcher, and the profile definitions are all shared; only the data is scoped.
## Gateway notifications
When you run `/kanban create …` from the gateway (Telegram, Discord, Slack, etc.), the originating chat is automatically subscribed to the new task. The gateway's background notifier polls `task_events` every few seconds and delivers one message per terminal event (`completed`, `blocked`, `gave_up`, `crashed`, `timed_out`) to that chat. Completed tasks also send the first line of the worker's `--result` so you see the outcome without having to `/kanban show`.
You can manage subscriptions explicitly from the CLI — useful when a script / cron job wants to notify a chat it didn't originate from:
```bash
hermes kanban notify-subscribe t_abcd \
--platform telegram --chat-id 12345678 --thread-id 7
hermes kanban notify-list
hermes kanban notify-unsubscribe t_abcd \
--platform telegram --chat-id 12345678 --thread-id 7
```
A subscription removes itself automatically once the task reaches `done` or `archived`; no cleanup needed.
## Runs — one row per attempt
A task is a logical unit of work; a **run** is one attempt to execute it. When the dispatcher claims a ready task it creates a row in `task_runs` and points `tasks.current_run_id` at it. When that attempt ends — completed, blocked, crashed, timed out, spawn-failed, reclaimed — the run row closes with an `outcome` and the task's pointer clears. A task that's been attempted three times has three `task_runs` rows.
Why two tables instead of just mutating the task: you need **full attempt history** for real-world postmortems ("the second reviewer attempt got to approve, the third merged"), and you need a clean place to hang per-attempt metadata — which files changed, which tests ran, which findings a reviewer noted. Those are run facts, not task facts.
Runs are also where **structured handoff** lives. When a worker completes a task (via `kanban_complete(...)`) it can pass:
- `summary` (tool param) / `--summary` (CLI) — human handoff; goes on the run; downstream children see it in their `build_worker_context`.
- `metadata` (tool param) / `--metadata` (CLI) — free-form JSON dict on the run; children see it serialized alongside the summary.
- `result` (tool param) / `--result` (CLI) — short log line that goes on the task row (legacy field, kept for back-compat).
Downstream children read the most recent completed run's summary + metadata for each parent. Retrying workers read the prior attempts on their own task (outcome, summary, error) so they don't repeat a path that already failed.
```
# What a worker actually does — a tool call, from inside the agent loop:
kanban_complete(
summary="implemented token bucket, keys on user_id with IP fallback, all tests pass",
metadata={"changed_files": ["limiter.py", "tests/test_limiter.py"], "tests_run": 14},
result="rate limiter shipped",
)
```
The same handoff is reachable from the CLI when you (the human) need to close out a task a worker can't — e.g. a task that was abandoned, or one you marked done manually from the dashboard:
```bash
hermes kanban complete t_abcd \
--result "rate limiter shipped" \
--summary "implemented token bucket, keys on user_id with IP fallback, all tests pass" \
--metadata '{"changed_files": ["limiter.py", "tests/test_limiter.py"], "tests_run": 14}'
# Review the attempt history on a retried task:
hermes kanban runs t_abcd
# # OUTCOME PROFILE ELAPSED STARTED
# 1 blocked worker 12s 2026-04-27 14:02
# → BLOCKED: need decision on rate-limit key
# 2 completed worker 8m 2026-04-27 15:18
# → implemented token bucket, keys on user_id with IP fallback
```
Runs are exposed on the dashboard (Run History section in the drawer, one coloured row per attempt) and on the REST API (`GET /api/plugins/kanban/tasks/:id` returns a `runs[]` array). `PATCH /api/plugins/kanban/tasks/:id` with `{status: "done", summary, metadata}` forwards both to the kernel, so the dashboard's "mark done" button is CLI-equivalent. `task_events` rows carry the `run_id` they belong to so the UI can group them by attempt, and the `completed` event embeds the first-line summary in its payload (capped at 400 chars) so gateway notifiers can render structured handoffs without a second SQL round-trip.
**Bulk close caveat.** `hermes kanban complete a b c --summary X` is refused — structured handoff is per-run, so copy-pasting the same summary to N tasks is almost always wrong. Bulk close *without* `--summary` / `--metadata` still works for the common "I finished a pile of admin tasks" case.
**Reclaimed runs from status changes.** If you drag a running task off `running` in the dashboard (back to `ready`, or straight to `todo`), or archive a task that was still running, the in-flight run closes with `outcome='reclaimed'` rather than being orphaned. The `task_runs` row is always in a terminal state when `tasks.current_run_id` is `NULL`, and vice versa — that invariant holds across CLI, dashboard, dispatcher, and notifier.
**Synthetic runs for never-claimed completions.** Completing or blocking a task that was never claimed (e.g. a human closes a `ready` task from the dashboard with a summary, or a CLI user runs `hermes kanban complete <ready-task> --summary X`) would otherwise drop the handoff. Instead the kernel inserts a zero-duration run row (`started_at == ended_at`) carrying the summary / metadata / reason so attempt history stays complete. The `completed` / `blocked` event's `run_id` points at that row.
**Live drawer refresh.** When the dashboard's WebSocket event stream reports new events for the task the user is currently viewing, the drawer reloads itself (via a per-task event counter threaded into its `useEffect` dependency list). Closing and reopening is no longer required to see a run's new row or updated outcome.
### Forward compatibility
Two nullable columns on `tasks` are reserved for v2 workflow routing: `workflow_template_id` (which template this task belongs to) and `current_step_key` (which step in that template is active). The v1 kernel ignores them for routing but lets clients write them, so a v2 release can add the routing machinery without another schema migration.
## Event reference
Every transition appends a row to `task_events`. Each row carries an optional `run_id` so UIs can group events by attempt. Kinds group into three clusters so filtering is easy (`hermes kanban watch --kinds completed,gave_up,timed_out`):
**Lifecycle** (what changed about the task as a logical unit):
| Kind | Payload | When |
|---|---|---|
| `created` | `{assignee, status, parents, tenant}` | Task inserted. `run_id` is `NULL`. |
| `promoted` | — | `todo → ready` because all parents hit `done`. `run_id` is `NULL`. |
| `claimed` | `{lock, expires, run_id}` | Dispatcher atomically claimed a `ready` task for spawn. |
| `completed` | `{result_len, summary?}` | Worker wrote `--result` / `--summary` and task hit `done`. `summary` is the first-line handoff (400-char cap); full version lives on the run row. If `complete_task` is called on a never-claimed task with handoff fields, a zero-duration run is synthesized so `run_id` still points at something. |
| `blocked` | `{reason}` | Worker or human flipped the task to `blocked`. Synthesizes a zero-duration run when called on a never-claimed task with `--reason`. |
| `unblocked` | — | `blocked → ready`, either manually or via `/unblock`. `run_id` is `NULL`. |
| `archived` | — | Hidden from the default board. If the task was still running, carries the `run_id` of the run that was reclaimed as a side effect. |
**Edits** (human-driven changes that aren't transitions):
| Kind | Payload | When |
|---|---|---|
| `assigned` | `{assignee}` | Assignee changed (including unassignment). |
| `edited` | `{fields}` | Title or body updated. |
| `reprioritized` | `{priority}` | Priority changed. |
| `status` | `{status}` | Dashboard drag-drop wrote a status directly (e.g. `todo → ready`). Carries the `run_id` of the run that was reclaimed when dragging off `running`; otherwise `run_id` is NULL. |
**Worker telemetry** (about the execution process, not the logical task):
| Kind | Payload | When |
|---|---|---|
| `spawned` | `{pid}` | Dispatcher successfully started a worker process. |
| `heartbeat` | `{note?}` | Worker called `hermes kanban heartbeat $TASK` to signal liveness during long operations. |
| `reclaimed` | `{stale_lock}` | Claim TTL expired without a completion; task goes back to `ready`. |
| `crashed` | `{pid, claimer}` | Worker PID no longer alive but TTL hadn't expired yet. |
| `timed_out` | `{pid, elapsed_seconds, limit_seconds, sigkill}` | `max_runtime_seconds` exceeded; dispatcher SIGTERM'd (then SIGKILL'd after 5 s grace) and re-queued. |
| `stale` | `{elapsed_seconds, last_heartbeat_at, heartbeat_age_seconds, timeout_seconds, pid, terminated}` | Task ran longer than `kanban.dispatch_stale_timeout_seconds` (default 4 h) AND no `kanban_heartbeat` arrived in the last hour. Dispatcher SIGTERM'd the host-local worker (if any), reset the task to `ready` for re-dispatch. Does NOT tick the failure counter (stale is dispatcher-side absence detection, not a worker fault). Workers running long operations should call `kanban_heartbeat` at least once an hour to avoid this. |
| `respawn_guarded` | `{reason}` | Dispatcher refused to re-spawn this ready task this tick. Reasons: `blocker_auth` (last failure was a quota/auth/429 error — wait for the rate window to reset), `recent_success` (a completed run happened in the last hour — wait for review before re-running), `active_pr` (a GitHub PR URL appears in a recent comment — a prior worker already opened a PR). The task stays in `ready`; the next tick gets another chance to spawn. If the underlying condition persists, the normal `consecutive_failures` circuit breaker will auto-block via `gave_up` after `failure_limit` failures. |
| `spawn_failed` | `{error, failures}` | One spawn attempt failed (missing PATH, workspace unmountable, …). Counter increments; task returns to `ready` for retry. |
| `protocol_violation` | `{pid, claimer, exit_code}` | Worker exited successfully while the task was still `running`, usually because it answered without calling `kanban_complete` or `kanban_block`. The dispatcher also emits `gave_up` and auto-blocks immediately instead of retrying. |
| `gave_up` | `{failures, effective_limit, limit_source, error}` | Circuit breaker fired after N consecutive non-successful attempts. Task auto-blocks with the last error. The effective limit resolves as task `max_retries`, then dispatcher `failure_limit` / `kanban.failure_limit`, then the built-in default. |
`hermes kanban tail <id>` shows these for a single task. `hermes kanban watch` streams them board-wide.
## Out of scope
Kanban is deliberately single-host. `~/.hermes/kanban.db` is a local SQLite file and the dispatcher spawns workers on the same machine. Running a shared board across two hosts is not supported — there's no coordination primitive for "worker X on host A, worker Y on host B," and the crash-detection path assumes PIDs are host-local. If you need multi-host, run an independent board per host and use `delegate_task` / a message queue to bridge them.
## Design spec
The complete design — architecture, concurrency correctness, comparison with other systems, implementation plan, risks, open questions — lives in `docs/hermes-kanban-v1-spec.pdf`. Read that before filing any behavior-change PR.
website/docs/user-guide/features/lsp.md
+254
View File
@@ -0,0 +1,254 @@
---
sidebar_position: 16
title: "LSP — Semantic Diagnostics"
description: "Real language servers (pyright, gopls, rust-analyzer, …) wired into the post-write lint check used by write_file and patch."
---
# Language Server Protocol (LSP)
Hermes runs full language servers — pyright, gopls, rust-analyzer,
typescript-language-server, clangd, and ~20 more — as background
subprocesses and feeds their semantic diagnostics into the post-write
lint check used by `write_file` and `patch`. When the agent edits a
file, it sees exactly the errors that edit introduced — not just
syntax errors, but **type errors, undefined names, missing imports,
and project-wide semantic issues** the language server detects.
This is the same architecture top-tier coding agents use. Hermes
ships it self-contained: no editor host required, no plugins to
install, no separate daemon to manage.
## When LSP runs
LSP is gated on **git workspace detection**. When the agent's working
directory (or the file being edited) is inside a git repository, LSP
runs against that workspace. When neither is in a git repo, LSP
stays dormant — useful for messaging gateways where the cwd is the
user's home directory and there's no project to diagnose.
The check is layered: in-process syntax check first (microseconds),
then LSP diagnostics second when syntax is clean. A flaky or missing
language server can never break a write — every LSP failure path
falls back silently to the syntax-only result.
Concretely, on every successful `write_file` or `patch`:
1. Hermes captures a baseline of current diagnostics for the file.
2. Performs the write.
3. Re-queries the language server, filters out diagnostics that were
already in the baseline, and surfaces only the new ones.
The agent sees output like:
```
{
"bytes_written": 42,
"dirs_created": false,
"lint": {"status": "ok", "output": ""},
"lsp_diagnostics": "LSP diagnostics introduced by this edit:\n<diagnostics file=\"/path/to/foo.py\">\nERROR [42:5] Cannot find name 'foo' [reportUndefinedVariable] (Pyright)\nERROR [50:1] Argument of type \"str\" is not assignable to \"int\" [reportArgumentType] (Pyright)\n</diagnostics>"
}
```
The `lint` field carries the syntax-check result (microsecond
in-process parse via `ast.parse`, `json.loads`, etc.); the
`lsp_diagnostics` field carries the semantic diagnostics from the
real language server. Two channels, independent signals — the
agent sees a syntax-clean file with semantic problems as
``lint: ok`` plus a populated ``lsp_diagnostics``.
## Supported languages
| Language | Server | Auto-install |
|----------|--------|--------------|
| Python | `pyright-langserver` | npm |
| TypeScript / JavaScript / JSX / TSX | `typescript-language-server` | npm |
| Vue | `@vue/language-server` | npm |
| Svelte | `svelte-language-server` | npm |
| Astro | `@astrojs/language-server` | npm |
| Go | `gopls` | `go install` |
| Rust | `rust-analyzer` | manual (rustup) |
| C / C++ | `clangd` | manual (LLVM) |
| Bash / Zsh | `bash-language-server` | npm |
| YAML | `yaml-language-server` | npm |
| Lua | `lua-language-server` | manual (GitHub releases) |
| PHP | `intelephense` | npm |
| OCaml | `ocaml-lsp` | manual (opam) |
| Dockerfile | `dockerfile-language-server-nodejs` | npm |
| Terraform | `terraform-ls` | manual |
| Dart | `dart language-server` | manual (dart sdk) |
| Haskell | `haskell-language-server` | manual (ghcup) |
| Julia | `julia` + LanguageServer.jl | manual |
| Clojure | `clojure-lsp` | manual |
| Nix | `nixd` | manual |
| Zig | `zls` | manual |
| Gleam | `gleam lsp` | manual (gleam install) |
| Elixir | `elixir-ls` | manual |
| Prisma | `prisma language-server` | manual |
| Kotlin | `kotlin-language-server` | manual |
| Java | `jdtls` | manual |
For "manual" entries, install the server through whatever toolchain
manager makes sense for that language (rustup, ghcup, opam, brew,
…). Hermes auto-detects the binary on PATH or in
`<HERMES_HOME>/lsp/bin/`.
A few servers are installed alongside a peer dependency that npm
won't auto-pull. The current case is `typescript-language-server`,
which requires the `typescript` SDK importable from the same
`node_modules` tree — Hermes installs both packages together when you
run `hermes lsp install typescript` or auto-install fires on first
use.
## CLI
```
hermes lsp status # service state + per-server install status
hermes lsp list # registry, optionally --installed-only
hermes lsp install <id> # eagerly install one server
hermes lsp install-all # try every server with a known recipe
hermes lsp restart # tear down running clients
hermes lsp which <id> # print resolved binary path
```
`hermes lsp status` is the best starting point — it shows which
languages will get semantic diagnostics today and which need a
binary installed.
## Configuration
The defaults work for typical setups; nothing to set if the binaries
are on PATH.
```yaml
# config.yaml
lsp:
# Master toggle. Disabling skips the entire subsystem — no servers
# spawn, no background event loop runs.
enabled: true
# How long to wait for diagnostics after each write.
wait_mode: document # "document" or "full"
wait_timeout: 5.0
# How to handle missing server binaries.
# auto — install via npm/pip/go install into <HERMES_HOME>/lsp/bin
# manual — only use binaries already on PATH
install_strategy: auto
# Per-server overrides (all optional).
servers:
pyright:
disabled: false
command: ["/abs/path/to/pyright-langserver", "--stdio"]
env: { PYRIGHT_LOG_LEVEL: "info" }
initialization_options:
python:
analysis:
typeCheckingMode: "strict"
typescript:
disabled: true # skip TS even when its extensions match
```
### Per-server keys
* `disabled: true` — skip this server entirely even when its
extensions match a file.
* `command: [bin, ...args]` — pin a custom binary path. Bypasses
auto-install.
* `env: {KEY: value}` — extra env vars passed to the spawned process.
* `initialization_options: {...}` — merged into the LSP
`initializationOptions` payload sent in the `initialize`
handshake. Server-specific; consult the language server's docs.
## Installation locations
When `install_strategy: auto`, Hermes installs binaries into
`<HERMES_HOME>/lsp/bin/`. NPM packages land in
`<HERMES_HOME>/lsp/node_modules/` with bin symlinks one level up.
Go binaries come from `go install` with `GOBIN` pointed at the
staging dir.
Nothing is ever installed to `/usr/local/`, `~/.local/`, or any other
shared location — the staging dir is fully Hermes-owned and is
removed when you reset the profile.
## Performance characteristics
LSP servers are **lazy-spawned** on first use. Editing a Python file
in a project that's never seen `.py` traffic spawns pyright; the
spawn takes 1-3 seconds for most servers (rust-analyzer can take 10+
on a cold project). Subsequent edits in the same workspace re-use
the running server.
The LSP layer adds a few milliseconds to clean writes when no
diagnostics are emitted. When diagnostics are emitted, the wait
budget is `wait_timeout` seconds — typically the server responds in
tens of milliseconds for pyright/tsserver and a few seconds for
rust-analyzer mid-indexing.
Servers are kept alive for the life of the Hermes process. There's
no idle-timeout reaper — the cost of restarting the server's index
on every write would be far higher than holding the daemon.
## Disabling
Set `lsp.enabled: false` in `config.yaml` to disable the entire
subsystem. The post-write check falls back to the in-process syntax
check (`ast.parse` for Python, `json.loads` for JSON, etc.) which
ships unchanged from earlier versions.
To disable a single language without disabling the whole layer:
```yaml
lsp:
servers:
rust-analyzer:
disabled: true
```
## Troubleshooting
**`hermes lsp status` shows a server as "missing"**
The binary isn't on PATH and isn't in `<HERMES_HOME>/lsp/bin/`. Run
`hermes lsp install <server_id>` to attempt an auto-install, or
install the binary manually through the language's normal toolchain.
**`Backend warnings` section in `hermes lsp status`**
Some servers ship as thin wrappers around an external CLI for actual
diagnostics — they spawn cleanly and accept requests but never emit
errors when the sidecar binary is missing. The most common case is
`bash-language-server`, which delegates diagnostics to `shellcheck`.
When `hermes lsp status` shows a `Backend warnings` section, install
the named tool through your OS package manager:
```
apt install shellcheck # Debian / Ubuntu
brew install shellcheck # macOS
scoop install shellcheck # Windows
```
The same warning is logged once at server spawn time in
`~/.hermes/logs/agent.log`.
**Server starts but never returns diagnostics**
Check `~/.hermes/logs/agent.log` for `[agent.lsp.client]` entries —
both stderr from the language server and protocol errors land
there. Some servers (rust-analyzer especially) need to finish a
project-wide index before they emit per-file diagnostics; the first
edit after server start may complete with no diagnostics, with
subsequent edits picking them up.
**Server crashed**
A crashed server is added to the broken-set and won't be retried for
the rest of the session. Run `hermes lsp restart` to clear the set;
the next edit re-spawns.
**Editing a file outside any git repo**
By design, LSP only runs inside a git repository. If the project isn't
yet initialized, run `git init` to enable LSP diagnostics. Otherwise the
in-process syntax-only fallback applies.
website/docs/user-guide/features/mcp.md
+784
View File
@@ -0,0 +1,784 @@
---
sidebar_position: 4
title: "MCP (Model Context Protocol)"
description: "Connect Hermes Agent to external tool servers via MCP — and control exactly which MCP tools Hermes loads"
---
# MCP (Model Context Protocol)
MCP lets Hermes Agent connect to external tool servers so the agent can use tools that live outside Hermes itself — GitHub, databases, file systems, browser stacks, internal APIs, and more.
If you have ever wanted Hermes to use a tool that already exists somewhere else, MCP is usually the cleanest way to do it.
## What MCP gives you
- Access to external tool ecosystems without writing a native Hermes tool first
- Local stdio servers and remote HTTP MCP servers in the same config
- Automatic tool discovery and registration at startup
- Utility wrappers for MCP resources and prompts when supported by the server
- Per-server filtering so you can expose only the MCP tools you actually want Hermes to see
## Quick start
1. Install MCP support (already included if you used the standard install script):
```bash
cd ~/.hermes/hermes-agent
uv pip install -e ".[mcp]"
```
2. Add an MCP server to `~/.hermes/config.yaml`:
```yaml
mcp_servers:
filesystem:
command: "npx"
args: ["-y", "@modelcontextprotocol/server-filesystem", "/home/user/projects"]
```
3. Start Hermes:
```bash
hermes chat
```
4. Ask Hermes to use the MCP-backed capability.
For example:
```text
List the files in /home/user/projects and summarize the repo structure.
```
Hermes will discover the MCP server's tools and use them like any other tool.
## Catalog: one-click install for Nous-approved MCPs
Hermes ships a curated catalog of MCP servers that Nous staff has reviewed
and merged. They're disabled by default — install only what you actually
want.
```bash
hermes mcp # interactive picker (default)
hermes mcp catalog # plain-text list, scriptable
hermes mcp install n8n # install a catalog entry by name
```
The picker shows each entry with its current status:
```
n8n available Manage and inspect n8n workflows from Hermes
linear enabled Linear issue/project management (remote OAuth)
github installed (disabled) GitHub repo + PR tools
```
Hit `Enter` on a row to install (and walk through any required credentials),
enable, disable, or uninstall. Catalog entries are stored under
`optional-mcps/` in the hermes-agent repo — presence in that directory means
Nous approval. There is no community submission tier; entries are added by
merging a PR.
Catalog entries can require:
- **API key** — Hermes prompts at install time and writes the value to
`~/.hermes/.env`. Non-secret values (base URLs) go to the same file.
- **OAuth** (remote MCP) — written as `auth: oauth` in your config; the MCP
client opens a browser on first connection.
- **OAuth** (third-party provider like Google/GitHub) — Hermes points you at
`hermes auth <provider>` if you haven't authenticated already.
### Tool selection at install time
After credentials are configured, Hermes probes the MCP server to list every
tool it exposes and presents a checklist:
```
Select tools for 'linear' (SPACE toggle, ENTER confirm)
[x] find_issues Find issues matching a query
[x] get_issue Get a single issue
[x] create_issue Create a new issue
[ ] delete_workspace Delete a Linear workspace
...
```
The pre-checked rows come from:
1. **Your prior selection** if you've installed this entry before (reinstalls
preserve what you had — the manifest's defaults don't override it)
2. **The manifest's `tools.default_enabled`** if the entry declares one (some
catalog entries pre-prune mutating or rarely-useful tools)
3. **Everything** if neither applies
Submit the checklist with ENTER. Only the checked tools end up in
`mcp_servers.<name>.tools.include`. If you select everything, no filter is
written (cleanest config shape, identical behavior).
**If the probe fails** (server unreachable, OAuth not yet completed,
backing service not running), the install still succeeds: the manifest's
`tools.default_enabled` is applied directly (if declared), or no filter is
written (if not). Re-run `hermes mcp configure <name>` once the server is
reachable to refine.
### Trust model
Installing a catalog entry runs whatever the manifest specifies — `git clone`,
the entry's `bootstrap` commands (`pip install`, `npm install`, etc.), and
ultimately the MCP server's own code. Manifests are gated by PR review into
the hermes-agent repo, so Nous has reviewed each entry before it shipped —
**but you should still read the manifest before installing**, especially the
`source:` field's repository, the `install.bootstrap:` commands, and any
`transport.command:` invocation.
Manifests live at
[`optional-mcps/<name>/manifest.yaml`](https://github.com/NousResearch/hermes-agent/tree/main/optional-mcps)
on GitHub. The picker also prints the manifest's `source:` URL at install
time so you can quickly verify the upstream repo.
### Manifest version compatibility
Manifests pin a `manifest_version`. The catalog is forward-compatible: if a
PR adds an entry with a newer `manifest_version` than your installed Hermes
understands, the picker will surface a warning (`⚠ '<name>' requires a newer
Hermes`) for that entry instead of silently hiding it. Run `hermes update`
to install the latest Hermes when you see that.
### Runtime `${ENV_VAR}` substitution
Inside an entry's `transport.command`, `transport.args`, `transport.url`,
and `headers`, `${VAR}` placeholders are resolved at server-connect time
from environment variables (which include everything in `~/.hermes/.env`).
This is useful when a catalog entry wants to reference a value the user
configured elsewhere — e.g. `${HOME}/foo` or `${MY_PROVIDER_TOKEN}`.
Note this is distinct from `${INSTALL_DIR}` in catalog manifests, which is
substituted at install-time with the path the catalog cloned the entry's
repo into.
### Updating tool selection later
```bash
hermes mcp configure linear
```
Reopens the same checklist with your current selection pre-checked. Use this
when you want more tools enabled, or when the server has added new tools that
you want to opt into.
### Updating the catalog manifest
MCPs are never auto-updated. Re-run `hermes mcp install <name>` to refresh
after a Hermes update if a manifest version changed.
To add an MCP to the catalog, open a PR against
[`optional-mcps/`](https://github.com/NousResearch/hermes-agent/tree/main/optional-mcps).
## Two kinds of MCP servers
### Stdio servers
Stdio servers run as local subprocesses and talk over stdin/stdout.
```yaml
mcp_servers:
github:
command: "npx"
args: ["-y", "@modelcontextprotocol/server-github"]
env:
GITHUB_PERSONAL_ACCESS_TOKEN: "***"
```
Use stdio servers when:
- the server is installed locally
- you want low-latency access to local resources
- you are following MCP server docs that show `command`, `args`, and `env`
### HTTP servers
HTTP MCP servers are remote endpoints Hermes connects to directly.
```yaml
mcp_servers:
remote_api:
url: "https://mcp.example.com/mcp"
headers:
Authorization: "Bearer ***"
```
Use HTTP servers when:
- the MCP server is hosted elsewhere
- your organization exposes internal MCP endpoints
- you do not want Hermes spawning a local subprocess for that integration
### OAuth-authenticated HTTP servers
Most hosted MCP servers (Linear, Sentry, Atlassian, Asana, Figma, Stripe, …) require OAuth 2.1 instead of a static bearer token. Set `auth: oauth` and Hermes handles discovery, dynamic client registration, PKCE, token exchange, refresh, and step-up auth via the MCP Python SDK.
```yaml
mcp_servers:
linear:
url: "https://mcp.linear.app/mcp"
auth: oauth
```
On first connect, Hermes prints an authorize URL, opens your browser when possible, and waits for the OAuth callback on a local loopback port. Tokens are cached at `~/.hermes/mcp-tokens/<server>.json` with 0o600 perms; subsequent runs reuse them silently until refresh fails.
**Remote / headless hosts.** When Hermes runs on a different machine than your browser, the loopback callback can't reach your laptop. Two ways to complete the flow:
- **Paste-back (no setup):** on an interactive terminal Hermes prints "Or paste the redirect URL here…" alongside the authorize URL. Open the URL in your browser, approve, copy the full URL the browser ends up on (the redirect will show a connection error — that's expected), paste it at the prompt. Bare `?code=…&state=…` query strings work too.
- **SSH port forward:** `ssh -N -L <port>:127.0.0.1:<port> user@host` in a separate terminal, then let the redirect flow normally.
See [OAuth over SSH / Remote Hosts](../../guides/oauth-over-ssh.md#mcp-servers) for the full walkthrough, including DCR-less servers (e.g. Slack), pre-registered `client_id`/`client_secret`, scope customization, and re-auth via `hermes mcp login <server>`.
**Pitfall — providers that don't support automatic registration (Google Drive, Atlassian).** Some servers reject the dynamic client registration step (RFC 7591) that bare `auth: oauth` relies on — Google's official Drive server (`https://drivemcp.googleapis.com/mcp/v1`) returns a `400 Bad Request`, so no OAuth client is created and no token is acquired. The symptom is subtle: these servers also serve `tools/list` *without* auth, so `hermes mcp login` can list the tools and look like it worked, but every real tool call later times out. `hermes mcp login` now detects this (it checks that a token actually landed on disk) and tells you to supply your own OAuth client. Create one in the provider's console and add it to config:
```yaml
mcp_servers:
googledrive:
url: "https://drivemcp.googleapis.com/mcp/v1"
auth: oauth
oauth:
client_id: "<your-oauth-client-id>"
client_secret: "<your-oauth-client-secret>"
```
Then run `hermes mcp login googledrive` — with the pre-registered client, Hermes skips registration and runs the normal browser authorization flow.
**Pitfall — config auto-reload race.** When you edit `~/.hermes/config.yaml` from inside a running Hermes session, the CLI auto-reloads MCP connections with a 30s timeout. That's not enough for an interactive OAuth flow. Add the entry, then run `hermes mcp login <server>` from a fresh terminal — it waits the full 5 minutes for you to complete auth.
## mTLS / client certificates
Remote HTTP MCP servers that require mutual TLS (client-certificate authentication) are supported via `client_cert` / `client_key`. Hermes passes the resolved certificate to the underlying HTTP client for the TLS handshake.
`client_cert` accepts three shapes:
- **A single combined PEM path** — one file holding both the certificate and the private key:
```yaml
mcp_servers:
internal_api:
url: "https://mcp.internal.example.com/mcp"
client_cert: "~/.certs/mcp-client.pem"
```
- **A `[cert, key]` 2-tuple** — certificate and key in separate files (equivalent to setting `client_cert` + `client_key`):
```yaml
mcp_servers:
internal_api:
url: "https://mcp.internal.example.com/mcp"
client_cert: ["~/.certs/mcp-client.crt", "~/.certs/mcp-client.key"]
```
- **A `[cert, key, password]` 3-tuple** — when the private key is encrypted, the third element is the key passphrase:
```yaml
mcp_servers:
internal_api:
url: "https://mcp.internal.example.com/mcp"
client_cert: ["~/.certs/mcp-client.crt", "~/.certs/mcp-client.key", "${MCP_KEY_PASSWORD}"]
```
You can also keep the cert and key fully separate via `client_cert` (combined PEM) plus an explicit `client_key`. Paths support `~` expansion; a missing file raises a clear, server-scoped error rather than an opaque TLS handshake failure.
## Basic configuration reference
Hermes reads MCP config from `~/.hermes/config.yaml` under `mcp_servers`.
### Common keys
| Key | Type | Meaning |
|---|---|---|
| `command` | string | Executable for a stdio MCP server |
| `args` | list | Arguments for the stdio server |
| `env` | mapping | Environment variables passed to the stdio server |
| `url` | string | HTTP MCP endpoint |
| `headers` | mapping | HTTP headers for remote servers |
| `client_cert` | string \| list | Client certificate for mTLS — a combined PEM path, or `[cert, key]` / `[cert, key, password]` |
| `client_key` | string | Client private-key PEM path (when separate from `client_cert`) |
| `timeout` | number | Tool call timeout |
| `connect_timeout` | number | Initial connection timeout |
| `enabled` | bool | If `false`, Hermes skips the server entirely |
| `supports_parallel_tool_calls` | bool | If `true`, tools from this server may run concurrently |
| `tools` | mapping | Per-server tool filtering and utility policy |
### Minimal stdio example
```yaml
mcp_servers:
filesystem:
command: "npx"
args: ["-y", "@modelcontextprotocol/server-filesystem", "/tmp"]
```
### Minimal HTTP example
```yaml
mcp_servers:
company_api:
url: "https://mcp.internal.example.com"
headers:
Authorization: "Bearer ***"
```
## Built-in presets
For well-known MCP servers, `hermes mcp add` accepts a `--preset` flag that fills in the transport details so you don't have to look up the command and args. The preset only supplies defaults — anything else (env vars, headers, filtering) you pass on the same command line still wins.
| Preset | What it wires up |
|---|---|
| `codex` | The Codex CLI's MCP server (`codex mcp-server` over stdio). Requires the `codex` CLI on PATH. |
```bash
# Add Codex CLI as an MCP server in one line
hermes mcp add codex --preset codex
```
That writes the equivalent of:
```yaml
mcp_servers:
codex:
command: "codex"
args: ["mcp-server"]
```
You can pick any local name (`hermes mcp add my-codex --preset codex` is fine); the preset only provides the `command`/`args` defaults.
## How Hermes registers MCP tools
Hermes prefixes MCP tools so they do not collide with built-in names:
```text
mcp_<server_name>_<tool_name>
```
Examples:
| Server | MCP tool | Registered name |
|---|---|---|
| `filesystem` | `read_file` | `mcp_filesystem_read_file` |
| `github` | `create-issue` | `mcp_github_create_issue` |
| `my-api` | `query.data` | `mcp_my_api_query_data` |
In practice, you usually do not need to call the prefixed name manually — Hermes sees the tool and chooses it during normal reasoning.
## MCP utility tools
When supported, Hermes also registers utility tools around MCP resources and prompts:
- `list_resources`
- `read_resource`
- `list_prompts`
- `get_prompt`
These are registered per server with the same prefix pattern, for example:
- `mcp_github_list_resources`
- `mcp_github_get_prompt`
### Important
These utility tools are now capability-aware:
- Hermes only registers resource utilities if the MCP session actually supports resource operations
- Hermes only registers prompt utilities if the MCP session actually supports prompt operations
So a server that exposes callable tools but no resources/prompts will not get those extra wrappers.
## Per-server filtering
You can control which tools each MCP server contributes to Hermes, allowing fine-grained management of your tool namespace.
### Disable a server entirely
```yaml
mcp_servers:
legacy:
url: "https://mcp.legacy.internal"
enabled: false
```
If `enabled: false`, Hermes skips the server completely and does not even attempt a connection.
### Whitelist server tools
```yaml
mcp_servers:
github:
command: "npx"
args: ["-y", "@modelcontextprotocol/server-github"]
env:
GITHUB_PERSONAL_ACCESS_TOKEN: "***"
tools:
include: [create_issue, list_issues]
```
Only those MCP server tools are registered.
### Blacklist server tools
```yaml
mcp_servers:
stripe:
url: "https://mcp.stripe.com"
tools:
exclude: [delete_customer]
```
All server tools are registered except the excluded ones.
### Precedence rule
If both are present:
```yaml
tools:
include: [create_issue]
exclude: [create_issue, delete_issue]
```
`include` wins.
### Filter utility tools too
You can also separately disable Hermes-added utility wrappers:
```yaml
mcp_servers:
docs:
url: "https://mcp.docs.example.com"
tools:
prompts: false
resources: false
```
That means:
- `tools.resources: false` disables `list_resources` and `read_resource`
- `tools.prompts: false` disables `list_prompts` and `get_prompt`
### Full example
```yaml
mcp_servers:
github:
command: "npx"
args: ["-y", "@modelcontextprotocol/server-github"]
env:
GITHUB_PERSONAL_ACCESS_TOKEN: "***"
tools:
include: [create_issue, list_issues, search_code]
prompts: false
stripe:
url: "https://mcp.stripe.com"
headers:
Authorization: "Bearer ***"
tools:
exclude: [delete_customer]
resources: false
legacy:
url: "https://mcp.legacy.internal"
enabled: false
```
## What happens if everything is filtered out?
If your config filters out all callable tools and disables or omits all supported utilities, Hermes does not create an empty runtime MCP toolset for that server.
That keeps the tool list clean.
## Runtime behavior
### Discovery time
Hermes discovers MCP servers at startup and registers their tools into the normal tool registry.
### Dynamic Tool Discovery
MCP servers can notify Hermes when their available tools change at runtime by sending a `notifications/tools/list_changed` notification. When Hermes receives this notification, it automatically re-fetches the server's tool list and updates the registry — no manual `/reload-mcp` required.
This is useful for MCP servers whose capabilities change dynamically (e.g. a server that adds tools when a new database schema is loaded, or removes tools when a service goes offline).
The refresh is lock-protected so rapid-fire notifications from the same server don't cause overlapping refreshes. Prompt and resource change notifications (`prompts/list_changed`, `resources/list_changed`) are received but not yet acted on.
### Reloading
If you change MCP config, use:
```text
/reload-mcp
```
This reloads MCP servers from config and refreshes the available tool list. For runtime tool changes pushed by the server itself, see [Dynamic Tool Discovery](#dynamic-tool-discovery) above.
### Toolsets
Each configured MCP server also creates a runtime toolset when it contributes at least one registered tool:
```text
mcp-<server>
```
That makes MCP servers easier to reason about at the toolset level.
## Security model
### Stdio env filtering
For stdio servers, Hermes does not blindly pass your full shell environment.
Only explicitly configured `env` plus a safe baseline are passed through. This reduces accidental secret leakage.
### Config-level exposure control
The new filtering support is also a security control:
- disable dangerous tools you do not want the model to see
- expose only a minimal whitelist for a sensitive server
- disable resource/prompt wrappers when you do not want that surface exposed
## Example use cases
### GitHub server with a minimal issue-management surface
```yaml
mcp_servers:
github:
command: "npx"
args: ["-y", "@modelcontextprotocol/server-github"]
env:
GITHUB_PERSONAL_ACCESS_TOKEN: "***"
tools:
include: [list_issues, create_issue, update_issue]
prompts: false
resources: false
```
Use it like:
```text
Show me open issues labeled bug, then draft a new issue for the flaky MCP reconnection behavior.
```
### Stripe server with dangerous actions removed
```yaml
mcp_servers:
stripe:
url: "https://mcp.stripe.com"
headers:
Authorization: "Bearer ***"
tools:
exclude: [delete_customer, refund_payment]
```
Use it like:
```text
Look up the last 10 failed payments and summarize common failure reasons.
```
### Filesystem server for a single project root
```yaml
mcp_servers:
project_fs:
command: "npx"
args: ["-y", "@modelcontextprotocol/server-filesystem", "/home/user/my-project"]
```
Use it like:
```text
Inspect the project root and explain the directory layout.
```
## Troubleshooting
### MCP server not connecting
Check:
```bash
# Verify MCP deps are installed (already included in standard install)
cd ~/.hermes/hermes-agent && uv pip install -e ".[mcp]"
node --version
npx --version
```
Then verify your config and restart Hermes.
### Tools not appearing
Possible causes:
- the server failed to connect
- discovery failed
- your filter config excluded the tools
- the utility capability does not exist on that server
- the server is disabled with `enabled: false`
If you are intentionally filtering, this is expected.
### Why didn't resource or prompt utilities appear?
Because Hermes now only registers those wrappers when both are true:
1. your config allows them
2. the server session actually supports the capability
This is intentional and keeps the tool list honest.
## Parallel Tool Calls
By default, MCP tools run sequentially — one at a time. If your MCP server exposes tools that are safe to run concurrently (e.g. read-only queries, independent API calls), you can opt-in to parallel execution:
```yaml
mcp_servers:
docs:
command: "docs-server"
supports_parallel_tool_calls: true
```
When `supports_parallel_tool_calls` is `true`, Hermes may execute multiple tools from that server at the same time within a single tool-call batch, just like it does for built-in read-only tools (web_search, read_file, etc.).
:::caution
Only enable parallel calls for MCP servers whose tools are safe to run at the same time. If tools read and write shared state, files, databases, or external resources, review the read/write race conditions before enabling this setting.
:::
## MCP Sampling Support
MCP servers can request LLM inference from Hermes via the `sampling/createMessage` protocol. This allows an MCP server to ask Hermes to generate text on its behalf — useful for servers that need LLM capabilities but don't have their own model access.
Sampling is **enabled by default** for all MCP servers (when the MCP SDK supports it). Configure it per-server under the `sampling` key:
```yaml
mcp_servers:
my_server:
command: "my-mcp-server"
sampling:
enabled: true # Enable sampling (default: true)
model: "openai/gpt-4o" # Override model for sampling requests (optional)
max_tokens_cap: 4096 # Max tokens per sampling response (default: 4096)
timeout: 30 # Timeout in seconds per request (default: 30)
max_rpm: 10 # Rate limit: max requests per minute (default: 10)
max_tool_rounds: 5 # Max tool-use rounds in sampling loops (default: 5)
allowed_models: [] # Allowlist of model names the server may request (empty = any)
log_level: "info" # Audit log level: debug, info, or warning (default: info)
```
The sampling handler includes a sliding-window rate limiter, per-request timeouts, and tool-loop depth limits to prevent runaway usage. Metrics (request count, errors, tokens used) are tracked per server instance.
To disable sampling for a specific server:
```yaml
mcp_servers:
untrusted_server:
url: "https://mcp.example.com"
sampling:
enabled: false
```
## Running Hermes as an MCP server
In addition to connecting **to** MCP servers, Hermes can also **be** an MCP server. This lets other MCP-capable agents (Claude Code, Cursor, Codex, or any MCP client) use Hermes's messaging capabilities — list conversations, read message history, and send messages across all your connected platforms.
### When to use this
- You want Claude Code, Cursor, or another coding agent to send and read Telegram/Discord/Slack messages through Hermes
- You want a single MCP server that bridges to all of Hermes's connected messaging platforms at once
- You already have a running Hermes gateway with connected platforms
### Quick start
```bash
hermes mcp serve
```
This starts a stdio MCP server. The MCP client (not you) manages the process lifecycle.
### MCP client configuration
Add Hermes to your MCP client config. For example, in Claude Code's `~/.claude/claude_desktop_config.json`:
```json
{
"mcpServers": {
"hermes": {
"command": "hermes",
"args": ["mcp", "serve"]
}
}
}
```
Or if you installed Hermes in a specific location:
```json
{
"mcpServers": {
"hermes": {
"command": "/home/user/.hermes/hermes-agent/venv/bin/hermes",
"args": ["mcp", "serve"]
}
}
}
```
### Available tools
The MCP server exposes 10 tools, matching OpenClaw's channel bridge surface plus a Hermes-specific channel browser:
| Tool | Description |
|------|-------------|
| `conversations_list` | List active messaging conversations. Filter by platform or search by name. |
| `conversation_get` | Get detailed info about one conversation by session key. |
| `messages_read` | Read recent message history for a conversation. |
| `attachments_fetch` | Extract non-text attachments (images, media) from a specific message. |
| `events_poll` | Poll for new conversation events since a cursor position. |
| `events_wait` | Long-poll / block until the next event arrives (near-real-time). |
| `messages_send` | Send a message through a platform (e.g. `telegram:123456`, `discord:#general`). |
| `channels_list` | List available messaging targets across all platforms. |
| `permissions_list_open` | List pending approval requests observed during this bridge session. |
| `permissions_respond` | Allow or deny a pending approval request. |
### Event system
The MCP server includes a live event bridge that polls Hermes's session database for new messages. This gives MCP clients near-real-time awareness of incoming conversations:
```
# Poll for new events (non-blocking)
events_poll(after_cursor=0)
# Wait for next event (blocks up to timeout)
events_wait(after_cursor=42, timeout_ms=30000)
```
Event types: `message`, `approval_requested`, `approval_resolved`
The event queue is in-memory and starts when the bridge connects. Older messages are available through `messages_read`.
### Options
```bash
hermes mcp serve # Normal mode
hermes mcp serve --verbose # Debug logging on stderr
```
### How it works
The MCP server reads conversation data directly from Hermes's session store (`~/.hermes/sessions/sessions.json` and the SQLite database). A background thread polls the database for new messages and maintains an in-memory event queue. For sending messages, it uses the same `send_message` infrastructure as the Hermes agent itself.
The gateway does NOT need to be running for read operations (listing conversations, reading history, polling events). It DOES need to be running for send operations, since the platform adapters need active connections.
### Current limits
- The embedded `hermes mcp serve` exposes a **stdio-only** MCP server today. If you need an HTTP MCP server, run a separate adapter — or, much more commonly, use the MCP **client** side of Hermes, which already speaks both stdio and HTTP (`url` + `headers` in `mcp_servers.yaml` / `config.yaml`; see [HTTP servers](#http-servers) above).
- Event polling at ~200ms intervals via mtime-optimized DB polling (skips work when files are unchanged)
- No `claude/channel` push notification protocol yet
- Text-only sends (no media/attachment sending through `messages_send`)
## Related docs
- [Use MCP with Hermes](/guides/use-mcp-with-hermes)
- [CLI Commands](/reference/cli-commands)
- [Slash Commands](/reference/slash-commands)
- [FAQ](/reference/faq)
website/docs/user-guide/features/memory-providers.md
+571
View File
@@ -0,0 +1,571 @@
---
sidebar_position: 4
title: "Memory Providers"
description: "External memory provider plugins — Honcho, OpenViking, Mem0, Hindsight, Holographic, RetainDB, ByteRover, Supermemory"
---
# Memory Providers
Hermes Agent ships with 8 external memory provider plugins that give the agent persistent, cross-session knowledge beyond the built-in MEMORY.md and USER.md. Only **one** external provider can be active at a time — the built-in memory is always active alongside it.
## Quick Start
```bash
hermes memory setup # interactive picker + configuration
hermes memory status # check what's active
hermes memory off # disable external provider
```
You can also select the active memory provider via `hermes plugins` → Provider Plugins → Memory Provider.
Or set manually in `~/.hermes/config.yaml`:
```yaml
memory:
provider: openviking # or honcho, mem0, hindsight, holographic, retaindb, byterover, supermemory
```
## How It Works
When a memory provider is active, Hermes automatically:
1. **Injects provider context** into the system prompt (what the provider knows)
2. **Prefetches relevant memories** before each turn (background, non-blocking)
3. **Syncs conversation turns** to the provider after each response
4. **Extracts memories on session end** (for providers that support it)
5. **Mirrors built-in memory writes** to the external provider
6. **Adds provider-specific tools** so the agent can search, store, and manage memories
The built-in memory (MEMORY.md / USER.md) continues to work exactly as before. The external provider is additive.
## Available Providers
### Honcho
AI-native cross-session user modeling with dialectic reasoning, session-scoped context injection, semantic search, and persistent conclusions. Base context now includes the session summary alongside user representation and peer cards, giving the agent awareness of what has already been discussed.
| | |
|---|---|
| **Best for** | Multi-agent systems with cross-session context, user-agent alignment |
| **Requires** | `pip install honcho-ai` + [API key](https://app.honcho.dev) or self-hosted instance |
| **Data storage** | Honcho Cloud or self-hosted |
| **Cost** | Honcho pricing (cloud) / free (self-hosted) |
**Tools (5):** `honcho_profile` (read/update peer card), `honcho_search` (semantic search), `honcho_context` (session context — summary, representation, card, messages), `honcho_reasoning` (LLM-synthesized), `honcho_conclude` (create/delete conclusions)
**Architecture:** Two-layer context injection — a base layer (session summary + representation + peer card, refreshed on `contextCadence`) plus a dialectic supplement (LLM reasoning, refreshed on `dialecticCadence`). The dialectic automatically selects cold-start prompts (general user facts) vs. warm prompts (session-scoped context) based on whether base context exists.
**Three orthogonal config knobs** control cost and depth independently:
- `contextCadence` — how often the base layer refreshes (API call frequency)
- `dialecticCadence` — how often the dialectic LLM fires (LLM call frequency)
- `dialecticDepth` — how many `.chat()` passes per dialectic invocation (1–3, depth of reasoning)
**Setup Wizard:**
```bash
hermes memory setup # select "honcho" — runs the Honcho-specific post-setup
```
The legacy `hermes honcho setup` command still works (it now redirects to `hermes memory setup`), but is only registered after Honcho is selected as the active memory provider.
**Config:** `$HERMES_HOME/honcho.json` (profile-local) or `~/.honcho/config.json` (global). Resolution order: `$HERMES_HOME/honcho.json` > `~/.hermes/honcho.json` > `~/.honcho/config.json`. See the [config reference](https://github.com/NousResearch/hermes-agent/blob/main/plugins/memory/honcho/README.md) and the [Honcho integration guide](https://docs.honcho.dev/v3/guides/integrations/hermes).
<details>
<summary>Full config reference</summary>
| Key | Default | Description |
|-----|---------|-------------|
| `apiKey` | -- | API key from [app.honcho.dev](https://app.honcho.dev) |
| `baseUrl` | -- | Base URL for self-hosted Honcho |
| `peerName` | -- | User peer identity |
| `aiPeer` | host key | AI peer identity (one per profile) |
| `workspace` | host key | Shared workspace ID |
| `contextTokens` | `null` (uncapped) | Token budget for auto-injected context per turn. Truncates at word boundaries |
| `contextCadence` | `1` | Minimum turns between `context()` API calls (base layer refresh) |
| `dialecticCadence` | `2` | Minimum turns between `peer.chat()` LLM calls. Recommended 1–5. Only applies to `hybrid`/`context` modes |
| `dialecticDepth` | `1` | Number of `.chat()` passes per dialectic invocation. Clamped 1–3. Pass 0: cold/warm prompt, pass 1: self-audit, pass 2: reconciliation |
| `dialecticDepthLevels` | `null` | Optional array of reasoning levels per pass, e.g. `["minimal", "low", "medium"]`. Overrides proportional defaults |
| `dialecticReasoningLevel` | `'low'` | Base reasoning level: `minimal`, `low`, `medium`, `high`, `max` |
| `dialecticDynamic` | `true` | When `true`, model can override reasoning level per-call via tool param |
| `dialecticMaxChars` | `600` | Max chars of dialectic result injected into system prompt |
| `recallMode` | `'hybrid'` | `hybrid` (auto-inject + tools), `context` (inject only), `tools` (tools only) |
| `writeFrequency` | `'async'` | When to flush messages: `async` (background thread), `turn` (sync), `session` (batch on end), or integer N |
| `saveMessages` | `true` | Whether to persist messages to Honcho API |
| `observationMode` | `'directional'` | `directional` (all on) or `unified` (shared pool). Override with `observation` object |
| `messageMaxChars` | `25000` | Max chars per message (chunked if exceeded) |
| `dialecticMaxInputChars` | `10000` | Max chars for dialectic query input to `peer.chat()` |
| `sessionStrategy` | `'per-directory'` | `per-directory`, `per-repo`, `per-session`, `global` |
</details>
<details>
<summary>Minimal honcho.json (cloud)</summary>
```json
{
"apiKey": "your-key-from-app.honcho.dev",
"hosts": {
"hermes": {
"enabled": true,
"aiPeer": "hermes",
"peerName": "your-name",
"workspace": "hermes"
}
}
}
```
</details>
<details>
<summary>Minimal honcho.json (self-hosted)</summary>
```json
{
"baseUrl": "http://localhost:8000",
"hosts": {
"hermes": {
"enabled": true,
"aiPeer": "hermes",
"peerName": "your-name",
"workspace": "hermes"
}
}
}
```
</details>
:::tip Migrating from `hermes honcho`
If you previously used `hermes honcho setup`, your config and all server-side data are intact. Just re-enable through the setup wizard again or manually set `memory.provider: honcho` to reactivate via the new system.
:::
**Multi-peer setup:**
Honcho models conversations as peers exchanging messages — one user peer plus one AI peer per Hermes profile, all sharing a workspace. The workspace is the shared environment: the user peer is global across profiles, each AI peer is its own identity. Every AI peer builds an independent representation / card from its own observations, so a `coder` profile stays code-oriented while a `writer` profile stays editorial against the same user.
The mapping:
| Concept | What it is |
|---------|-----------|
| **Workspace** | Shared environment. All Hermes profiles under one workspace see the same user identity. |
| **User peer** (`peerName`) | The human. Shared across profiles in the workspace. |
| **AI peer** (`aiPeer`) | One per Hermes profile. Host key `hermes` → default; `hermes.<profile>` for others. |
| **Observation** | Per-peer toggles controlling what Honcho models from whose messages. `directional` (default, all four on) or `unified` (single-observer pool). |
### New profile, fresh Honcho peer
```bash
hermes profile create coder --clone
```
`--clone` creates a `hermes.coder` host block in `honcho.json` with `aiPeer: "coder"`, shared `workspace`, inherited `peerName`, `recallMode`, `writeFrequency`, `observation`, etc. The AI peer is eagerly created in Honcho so it exists before the first message.
### Existing profiles, backfill Honcho peers
```bash
hermes honcho sync
```
Scans every Hermes profile, creates host blocks for any profile without one, inherits settings from the default `hermes` block, and creates the new AI peers eagerly. Idempotent — skips profiles that already have a host block.
### Per-profile observation
Each host block can override the observation config independently. Example: a code-focused profile where the AI peer observes the user but doesn't self-model:
```json
"hermes.coder": {
"aiPeer": "coder",
"observation": {
"user": { "observeMe": true, "observeOthers": true },
"ai": { "observeMe": false, "observeOthers": true }
}
}
```
**Observation toggles (one set per peer):**
| Toggle | Effect |
|--------|--------|
| `observeMe` | Honcho builds a representation of this peer from its own messages |
| `observeOthers` | This peer observes the other peer's messages (feeds cross-peer reasoning) |
Presets via `observationMode`:
- **`"directional"`** (default) — all four flags on. Full mutual observation; enables cross-peer dialectic.
- **`"unified"`** — user `observeMe: true`, AI `observeOthers: true`, rest false. Single-observer pool; AI models the user but not itself, user peer only self-models.
Server-side toggles set via the [Honcho dashboard](https://app.honcho.dev) win over local defaults — synced back at session init.
See the [Honcho page](./honcho.md#observation-directional-vs-unified) for the full observation reference.
<details>
<summary>Full honcho.json example (multi-profile)</summary>
```json
{
"apiKey": "your-key",
"workspace": "hermes",
"peerName": "eri",
"hosts": {
"hermes": {
"enabled": true,
"aiPeer": "hermes",
"workspace": "hermes",
"peerName": "eri",
"recallMode": "hybrid",
"writeFrequency": "async",
"sessionStrategy": "per-directory",
"observation": {
"user": { "observeMe": true, "observeOthers": true },
"ai": { "observeMe": true, "observeOthers": true }
},
"dialecticReasoningLevel": "low",
"dialecticDynamic": true,
"dialecticCadence": 2,
"dialecticDepth": 1,
"dialecticMaxChars": 600,
"contextCadence": 1,
"messageMaxChars": 25000,
"saveMessages": true
},
"hermes.coder": {
"enabled": true,
"aiPeer": "coder",
"workspace": "hermes",
"peerName": "eri",
"recallMode": "tools",
"observation": {
"user": { "observeMe": true, "observeOthers": false },
"ai": { "observeMe": true, "observeOthers": true }
}
},
"hermes.writer": {
"enabled": true,
"aiPeer": "writer",
"workspace": "hermes",
"peerName": "eri"
}
},
"sessions": {
"/home/user/myproject": "myproject-main"
}
}
```
</details>
See the [config reference](https://github.com/NousResearch/hermes-agent/blob/main/plugins/memory/honcho/README.md) and [Honcho integration guide](https://docs.honcho.dev/v3/guides/integrations/hermes).
---
### OpenViking
Context database by Volcengine (ByteDance) with filesystem-style knowledge hierarchy, tiered retrieval, and automatic memory extraction into 6 categories.
| | |
|---|---|
| **Best for** | Self-hosted knowledge management with structured browsing |
| **Requires** | `pip install openviking` + running server |
| **Data storage** | Self-hosted (local or cloud) |
| **Cost** | Free (open-source, AGPL-3.0) |
**Tools:** `viking_search` (semantic search), `viking_read` (tiered: abstract/overview/full), `viking_browse` (filesystem navigation), `viking_remember` (store facts), `viking_add_resource` (ingest URLs/docs)
**Setup:**
```bash
# Start the OpenViking server first
pip install openviking
openviking-server
# Then configure Hermes
hermes memory setup # select "openviking"
# Or manually:
hermes config set memory.provider openviking
echo "OPENVIKING_ENDPOINT=http://localhost:1933" >> ~/.hermes/.env
```
**Key features:**
- Tiered context loading: L0 (~100 tokens) → L1 (~2k) → L2 (full)
- Automatic memory extraction on session commit (profile, preferences, entities, events, cases, patterns)
- `viking://` URI scheme for hierarchical knowledge browsing
---
### Mem0
Server-side LLM fact extraction with semantic search, reranking, and automatic deduplication.
| | |
|---|---|
| **Best for** | Hands-off memory management — Mem0 handles extraction automatically |
| **Requires** | `pip install mem0ai` + API key |
| **Data storage** | Mem0 Cloud |
| **Cost** | Mem0 pricing |
**Tools:** `mem0_profile` (all stored memories), `mem0_search` (semantic search + reranking), `mem0_conclude` (store verbatim facts)
**Setup:**
```bash
hermes memory setup # select "mem0"
# Or manually:
hermes config set memory.provider mem0
echo "MEM0_API_KEY=your-key" >> ~/.hermes/.env
```
**Config:** `$HERMES_HOME/mem0.json`
| Key | Default | Description |
|-----|---------|-------------|
| `user_id` | `hermes-user` | User identifier |
| `agent_id` | `hermes` | Agent identifier |
---
### Hindsight
Long-term memory with knowledge graph, entity resolution, and multi-strategy retrieval. The `hindsight_reflect` tool provides cross-memory synthesis that no other provider offers. Automatically retains full conversation turns (including tool calls) with session-level document tracking.
| | |
|---|---|
| **Best for** | Knowledge graph-based recall with entity relationships |
| **Requires** | Cloud: API key from [ui.hindsight.vectorize.io](https://ui.hindsight.vectorize.io). Local: LLM API key (OpenAI, Groq, OpenRouter, etc.) |
| **Data storage** | Hindsight Cloud or local embedded PostgreSQL |
| **Cost** | Hindsight pricing (cloud) or free (local) |
**Tools:** `hindsight_retain` (store with entity extraction), `hindsight_recall` (multi-strategy search), `hindsight_reflect` (cross-memory synthesis)
**Setup:**
```bash
hermes memory setup # select "hindsight"
# Or manually:
hermes config set memory.provider hindsight
echo "HINDSIGHT_API_KEY=your-key" >> ~/.hermes/.env
```
The setup wizard installs dependencies automatically and only installs what's needed for the selected mode (`hindsight-client` for cloud, `hindsight-all` for local). Requires `hindsight-client >= 0.4.22` (auto-upgraded on session start if outdated).
**Local mode UI:** `hindsight-embed -p hermes ui start`
**Config:** `$HERMES_HOME/hindsight/config.json`
| Key | Default | Description |
|-----|---------|-------------|
| `mode` | `cloud` | `cloud` or `local` |
| `bank_id` | `hermes` | Memory bank identifier |
| `recall_budget` | `mid` | Recall thoroughness: `low` / `mid` / `high` |
| `memory_mode` | `hybrid` | `hybrid` (context + tools), `context` (auto-inject only), `tools` (tools only) |
| `auto_retain` | `true` | Automatically retain conversation turns |
| `auto_recall` | `true` | Automatically recall memories before each turn |
| `retain_async` | `true` | Process retain asynchronously on the server |
| `retain_context` | `conversation between Hermes Agent and the User` | Context label for retained memories |
| `retain_tags` | — | Default tags applied to retained memories; merged with per-call tool tags |
| `retain_source` | — | Optional `metadata.source` attached to retained memories |
| `retain_user_prefix` | `User` | Label used before user turns in auto-retained transcripts |
| `retain_assistant_prefix` | `Assistant` | Label used before assistant turns in auto-retained transcripts |
| `recall_tags` | — | Tags to filter on recall |
See [plugin README](https://github.com/NousResearch/hermes-agent/blob/main/plugins/memory/hindsight/README.md) for the full configuration reference.
---
### Holographic
Local SQLite fact store with FTS5 full-text search, trust scoring, and HRR (Holographic Reduced Representations) for compositional algebraic queries.
| | |
|---|---|
| **Best for** | Local-only memory with advanced retrieval, no external dependencies |
| **Requires** | Nothing (SQLite is always available). NumPy optional for HRR algebra. |
| **Data storage** | Local SQLite |
| **Cost** | Free |
**Tools:** `fact_store` (9 actions: add, search, probe, related, reason, contradict, update, remove, list), `fact_feedback` (helpful/unhelpful rating that trains trust scores)
**Setup:**
```bash
hermes memory setup # select "holographic"
# Or manually:
hermes config set memory.provider holographic
```
**Config:** `config.yaml` under `plugins.hermes-memory-store`
| Key | Default | Description |
|-----|---------|-------------|
| `db_path` | `$HERMES_HOME/memory_store.db` | SQLite database path |
| `auto_extract` | `false` | Auto-extract facts at session end |
| `default_trust` | `0.5` | Default trust score (0.0–1.0) |
**Unique capabilities:**
- `probe` — entity-specific algebraic recall (all facts about a person/thing)
- `reason` — compositional AND queries across multiple entities
- `contradict` — automated detection of conflicting facts
- Trust scoring with asymmetric feedback (+0.05 helpful / -0.10 unhelpful)
---
### RetainDB
Cloud memory API with hybrid search (Vector + BM25 + Reranking), 7 memory types, and delta compression.
| | |
|---|---|
| **Best for** | Teams already using RetainDB's infrastructure |
| **Requires** | RetainDB account + API key |
| **Data storage** | RetainDB Cloud |
| **Cost** | $20/month |
**Tools:** `retaindb_profile` (user profile), `retaindb_search` (semantic search), `retaindb_context` (task-relevant context), `retaindb_remember` (store with type + importance), `retaindb_forget` (delete memories)
**Setup:**
```bash
hermes memory setup # select "retaindb"
# Or manually:
hermes config set memory.provider retaindb
echo "RETAINDB_API_KEY=your-key" >> ~/.hermes/.env
```
---
### ByteRover
Persistent memory via the `brv` CLI — hierarchical knowledge tree with tiered retrieval (fuzzy text → LLM-driven search). Local-first with optional cloud sync.
| | |
|---|---|
| **Best for** | Developers who want portable, local-first memory with a CLI |
| **Requires** | ByteRover CLI (`npm install -g byterover-cli` or [install script](https://byterover.dev)) |
| **Data storage** | Local (default) or ByteRover Cloud (optional sync) |
| **Cost** | Free (local) or ByteRover pricing (cloud) |
**Tools:** `brv_query` (search knowledge tree), `brv_curate` (store facts/decisions/patterns), `brv_status` (CLI version + tree stats)
**Setup:**
```bash
# Install the CLI first
curl -fsSL https://byterover.dev/install.sh | sh
# Then configure Hermes
hermes memory setup # select "byterover"
# Or manually:
hermes config set memory.provider byterover
```
**Key features:**
- Automatic pre-compression extraction (saves insights before context compression discards them)
- Knowledge tree stored at `$HERMES_HOME/byterover/` (profile-scoped)
- SOC2 Type II certified cloud sync (optional)
---
### Supermemory
Semantic long-term memory with profile recall, semantic search, explicit memory tools, and session-end conversation ingest via the Supermemory graph API.
| | |
|---|---|
| **Best for** | Semantic recall with user profiling and session-level graph building |
| **Requires** | `pip install supermemory` + [API key](https://supermemory.ai) |
| **Data storage** | Supermemory Cloud |
| **Cost** | Supermemory pricing |
**Tools:** `supermemory_store` (save explicit memories), `supermemory_search` (semantic similarity search), `supermemory_forget` (forget by ID or best-match query), `supermemory_profile` (persistent profile + recent context)
**Setup:**
```bash
hermes memory setup # select "supermemory"
# Or manually:
hermes config set memory.provider supermemory
echo 'SUPERMEMORY_API_KEY=***' >> ~/.hermes/.env
```
**Config:** `$HERMES_HOME/supermemory.json`
| Key | Default | Description |
|-----|---------|-------------|
| `container_tag` | `hermes` | Container tag used for search and writes. Supports `{identity}` template for profile-scoped tags. |
| `auto_recall` | `true` | Inject relevant memory context before turns |
| `auto_capture` | `true` | Store cleaned user-assistant turns after each response |
| `max_recall_results` | `10` | Max recalled items to format into context |
| `profile_frequency` | `50` | Include profile facts on first turn and every N turns |
| `capture_mode` | `all` | Skip tiny or trivial turns by default |
| `search_mode` | `hybrid` | Search mode: `hybrid`, `memories`, or `documents` |
| `api_timeout` | `5.0` | Timeout for SDK and ingest requests |
**Environment variables:** `SUPERMEMORY_API_KEY` (required), `SUPERMEMORY_CONTAINER_TAG` (overrides config).
**Key features:**
- Automatic context fencing — strips recalled memories from captured turns to prevent recursive memory pollution
- Full-session ingest — the entire conversation is sent once at session boundaries
- Session-end conversation ingest (to `/v4/conversations`) for richer profile + graph building in Supermemory
- Profile facts injected on first turn and at configurable intervals
- **Profile-scoped containers** — use `{identity}` in `container_tag` (e.g. `hermes-{identity}` → `hermes-coder`) to isolate memories per Hermes profile
- **Multi-container mode** — enable `enable_custom_container_tags` with a `custom_containers` list to let the agent read/write across named containers. Automatic operations stay on the primary container.
<details>
<summary>Multi-container example</summary>
```json
{
"container_tag": "hermes",
"enable_custom_container_tags": true,
"custom_containers": ["project-alpha", "shared-knowledge"],
"custom_container_instructions": "Use project-alpha for coding context."
}
```
</details>
**Support:** [Discord](https://supermemory.link/discord) · [support@supermemory.com](mailto:support@supermemory.com)
### Memori
Structured long-term memory using Memori Cloud, with background completed-turn capture, tool-aware turn context, and explicit recall tools for facts, summaries, quota, signup, and feedback.
| | |
|---|---|
| **Best for** | Agent-controlled recall with structured project and session attribution |
| **Requires** | `pip install hermes-memori` + `hermes-memori install` + [Memori API key](https://app.memorilabs.ai/signup) |
| **Data storage** | Memori Cloud |
| **Cost** | Memori pricing |
**Tools:** `memori_recall` (search long-term memory), `memori_recall_summary` (summarized context), `memori_quota` (usage/quota), `memori_signup` (request signup email), `memori_feedback` (send integration feedback)
**Setup:**
```bash
pip install hermes-memori
hermes-memori install
hermes config set memory.provider memori
hermes memory setup
```
---
## Provider Comparison
| Provider | Storage | Cost | Tools | Dependencies | Unique Feature |
|----------|---------|------|-------|-------------|----------------|
| **Honcho** | Cloud | Paid | 5 | `honcho-ai` | Dialectic user modeling + session-scoped context |
| **OpenViking** | Self-hosted | Free | 5 | `openviking` + server | Filesystem hierarchy + tiered loading |
| **Mem0** | Cloud | Paid | 3 | `mem0ai` | Server-side LLM extraction |
| **Hindsight** | Cloud/Local | Free/Paid | 3 | `hindsight-client` | Knowledge graph + reflect synthesis |
| **Holographic** | Local | Free | 2 | None | HRR algebra + trust scoring |
| **RetainDB** | Cloud | $20/mo | 5 | `requests` | Delta compression |
| **ByteRover** | Local/Cloud | Free/Paid | 3 | `brv` CLI | Pre-compression extraction |
| **Supermemory** | Cloud | Paid | 4 | `supermemory` | Context fencing + session graph ingest + multi-container |
| **Memori** | Cloud | Free/Paid | 5 | `hermes-memori` | Tool-aware memory + structured recall |
## Profile Isolation
Each provider's data is isolated per [profile](/user-guide/profiles):
- **Local storage providers** (Holographic, ByteRover) use `$HERMES_HOME/` paths which differ per profile
- **Config file providers** (Honcho, Mem0, Hindsight, Supermemory) store config in `$HERMES_HOME/` so each profile has its own credentials
- **Cloud providers** (RetainDB) auto-derive profile-scoped project names
- **Env var providers** (OpenViking) are configured via each profile's `.env` file
## Building a Memory Provider
See the [Developer Guide: Memory Provider Plugins](/developer-guide/memory-provider-plugin) for how to create your own.
website/docs/user-guide/features/memory.md
+287
View File
@@ -0,0 +1,287 @@
---
sidebar_position: 3
title: "Persistent Memory"
description: "How Hermes Agent remembers across sessions — MEMORY.md, USER.md, and session search"
---
# Persistent Memory
Hermes Agent has bounded, curated memory that persists across sessions. This lets it remember your preferences, your projects, your environment, and things it has learned.
## How It Works
Two files make up the agent's memory:
| File | Purpose | Char Limit |
|------|---------|------------|
| **MEMORY.md** | Agent's personal notes — environment facts, conventions, things learned | 2,200 chars (~800 tokens) |
| **USER.md** | User profile — your preferences, communication style, expectations | 1,375 chars (~500 tokens) |
Both are stored in `~/.hermes/memories/` and are injected into the system prompt as a frozen snapshot at session start. The agent manages its own memory via the `memory` tool — it can add, replace, or remove entries.
:::info
Character limits keep memory focused. Memory does **not** auto-compact: when a
write would exceed the limit, the `memory` tool returns an error instead of
silently dropping entries. The agent then makes room itself — consolidating or
removing entries in the same turn before retrying (see [What Happens When Memory
is Full](#what-happens-when-memory-is-full)). Note that `replace` is also bound
by the limit: swapping an entry for a longer one can still overflow, so the new
content must be shortened (or another entry removed) to fit.
:::
## How Memory Appears in the System Prompt
At the start of every session, memory entries are loaded from disk and rendered into the system prompt as a frozen block:
```
══════════════════════════════════════════════
MEMORY (your personal notes) [67% — 1,474/2,200 chars]
══════════════════════════════════════════════
User's project is a Rust web service at ~/code/myapi using Axum + SQLx
§
This machine runs Ubuntu 22.04, has Docker and Podman installed
§
User prefers concise responses, dislikes verbose explanations
```
The format includes:
- A header showing which store (MEMORY or USER PROFILE)
- Usage percentage and character counts so the agent knows capacity
- Individual entries separated by `§` (section sign) delimiters
- Entries can be multiline
**Frozen snapshot pattern:** The system prompt injection is captured once at session start and never changes mid-session. This is intentional — it preserves the LLM's prefix cache for performance. When the agent adds/removes memory entries during a session, the changes are persisted to disk immediately but won't appear in the system prompt until the next session starts. Tool responses always show the live state.
## Memory Tool Actions
The agent uses the `memory` tool with these actions:
- **add** — Add a new memory entry
- **replace** — Replace an existing entry with updated content (uses substring matching via `old_text`)
- **remove** — Remove an entry that's no longer relevant (uses substring matching via `old_text`)
There is no `read` action — memory content is automatically injected into the system prompt at session start. The agent sees its memories as part of its conversation context.
### Substring Matching
The `replace` and `remove` actions use short unique substring matching — you don't need the full entry text. The `old_text` parameter just needs to be a unique substring that identifies exactly one entry:
```python
# If memory contains "User prefers dark mode in all editors"
memory(action="replace", target="memory",
old_text="dark mode",
content="User prefers light mode in VS Code, dark mode in terminal")
```
If the substring matches multiple entries, an error is returned asking for a more specific match.
## Two Targets Explained
### `memory` — Agent's Personal Notes
For information the agent needs to remember about the environment, workflows, and lessons learned:
- Environment facts (OS, tools, project structure)
- Project conventions and configuration
- Tool quirks and workarounds discovered
- Completed task diary entries
- Skills and techniques that worked
### `user` — User Profile
For information about the user's identity, preferences, and communication style:
- Name, role, timezone
- Communication preferences (concise vs detailed, format preferences)
- Pet peeves and things to avoid
- Workflow habits
- Technical skill level
## What to Save vs Skip
### Save These (Proactively)
The agent saves automatically — you don't need to ask. It saves when it learns:
- **User preferences:** "I prefer TypeScript over JavaScript" → save to `user`
- **Environment facts:** "This server runs Debian 12 with PostgreSQL 16" → save to `memory`
- **Corrections:** "Don't use `sudo` for Docker commands, user is in docker group" → save to `memory`
- **Conventions:** "Project uses tabs, 120-char line width, Google-style docstrings" → save to `memory`
- **Completed work:** "Migrated database from MySQL to PostgreSQL on 2026-01-15" → save to `memory`
- **Explicit requests:** "Remember that my API key rotation happens monthly" → save to `memory`
### Skip These
- **Trivial/obvious info:** "User asked about Python" — too vague to be useful
- **Easily re-discovered facts:** "Python 3.12 supports f-string nesting" — can web search this
- **Raw data dumps:** Large code blocks, log files, data tables — too big for memory
- **Session-specific ephemera:** Temporary file paths, one-off debugging context
- **Information already in context files:** SOUL.md and AGENTS.md content
## Capacity Management
Memory has strict character limits to keep system prompts bounded:
| Store | Limit | Typical entries |
|-------|-------|----------------|
| memory | 2,200 chars | 8-15 entries |
| user | 1,375 chars | 5-10 entries |
### What Happens When Memory is Full
When you try to add an entry that would exceed the limit, the tool returns an error:
```json
{
"success": false,
"error": "Memory at 2,100/2,200 chars. Adding this entry (250 chars) would exceed the limit. Consolidate now: use 'replace' to merge overlapping entries into shorter ones or 'remove' stale or less important entries (see current_entries below), then retry this add — all in this turn.",
"current_entries": ["..."],
"usage": "2,100/2,200"
}
```
The agent should then:
1. Read the current entries (shown in the error response)
2. Identify entries that can be removed or consolidated
3. Use `replace` to merge related entries into shorter versions
4. Then `add` the new entry
**Best practice:** When memory is above 80% capacity (visible in the system prompt header), consolidate entries before adding new ones. For example, merge three separate "project uses X" entries into one comprehensive project description entry.
### Practical Examples of Good Memory Entries
**Compact, information-dense entries work best:**
```
# Good: Packs multiple related facts
User runs macOS 14 Sonoma, uses Homebrew, has Docker Desktop and Podman. Shell: zsh with oh-my-zsh. Editor: VS Code with Vim keybindings.
# Good: Specific, actionable convention
Project ~/code/api uses Go 1.22, sqlc for DB queries, chi router. Run tests with 'make test'. CI via GitHub Actions.
# Good: Lesson learned with context
The staging server (10.0.1.50) needs SSH port 2222, not 22. Key is at ~/.ssh/staging_ed25519.
# Bad: Too vague
User has a project.
# Bad: Too verbose
On January 5th, 2026, the user asked me to look at their project which is
located at ~/code/api. I discovered it uses Go version 1.22 and...
```
## Duplicate Prevention
The memory system automatically rejects exact duplicate entries. If you try to add content that already exists, it returns success with a "no duplicate added" message.
## Security Scanning
Memory entries are scanned for injection and exfiltration patterns before being accepted, since they're injected into the system prompt. Content matching threat patterns (prompt injection, credential exfiltration, SSH backdoors) or containing invisible Unicode characters is blocked.
## Session Search
Beyond MEMORY.md and USER.md, the agent can search its past conversations using the `session_search` tool:
- All CLI and messaging sessions are stored in SQLite (`~/.hermes/state.db`) with FTS5 full-text search
- Search queries return actual messages from the DB — no LLM summarization, no truncation
- The agent can find things it discussed weeks ago, even if they're not in its active memory
- The agent can also scroll forward/backward inside any session it finds
```bash
hermes sessions list # Browse past sessions
```
See [Session Search Tool](/user-guide/sessions#session-search-tool) for the three calling shapes (discovery / scroll / browse) and the response format.
### session_search vs memory
| Feature | Persistent Memory | Session Search |
|---------|------------------|----------------|
| **Capacity** | ~1,300 tokens total | Unlimited (all sessions) |
| **Speed** | Instant (in system prompt) | ~20ms FTS5 query, ~1ms scroll |
| **Cost** | Token cost in every prompt | Free — no LLM calls |
| **Use case** | Key facts always available | Finding specific past conversations |
| **Management** | Manually curated by agent | Automatic — all sessions stored |
| **Token cost** | Fixed per session (~1,300 tokens) | On-demand (searched when needed) |
**Memory** is for critical facts that should always be in context. **Session search** is for "did we discuss X last week?" queries where the agent needs to recall specifics from past conversations.
## Configuration
```yaml
# In ~/.hermes/config.yaml
memory:
memory_enabled: true
user_profile_enabled: true
memory_char_limit: 2200 # ~800 tokens
user_char_limit: 1375 # ~500 tokens
write_approval: false # false = write freely (default) | true = require approval
```
## Controlling memory writes (`write_approval`)
By default the agent saves memory freely — including from the background
self-improvement review that runs after a turn. If you'd rather approve saves
first, set `memory.write_approval: true`. It's a simple on/off gate applied to
**both** foreground turns and the background review:
| `write_approval` | Behaviour |
|------------------|-----------|
| `false` (default) | Write freely — the gate is off (the pre-gate behaviour). |
| `true` | Require approval before anything is saved. In the interactive CLI, foreground writes prompt you inline (entries are small enough to read in full). Everywhere else — messaging platforms, scripts, and the background self-improvement review — writes are **staged** for review with `/memory pending`. |
> To turn memory off entirely (not just gate it), set `memory_enabled: false`.
Review staged writes from the CLI or any messaging platform:
```
/memory pending # list staged memory writes (auto ones tagged [auto])
/memory approve <id> # apply one (or 'all')
/memory reject <id> # drop one (or 'all')
/memory approval on # turn the gate on (or 'off') and persist it
```
This is the answer to "the agent saved a wrong assumption about me": set
`write_approval: true`, and every save — especially the unprompted background
ones — waits for your yes/no before it ever enters your profile.
## Controlling skill writes (`skills.write_approval`)
Skills use the same on/off gate, but the review UX differs because a
`SKILL.md` is far too large to read in a chat bubble:
```yaml
skills:
write_approval: false # false = write freely (default) | true = require approval
```
When `write_approval: true`, skill writes (create / edit / patch / write_file /
delete) always **stage** regardless of origin. You review the one-line gist
inline, but the full diff stays out-of-band:
```
/skills pending # list staged skill writes + a one-line gist each
/skills diff <id> # full unified diff (best viewed in CLI or dashboard)
/skills approve <id> # apply it (or 'all')
/skills reject <id> # drop it (or 'all')
/skills approval on # turn the gate on (or 'off') and persist it
```
On a messaging platform, approve a skill from its gist + metadata, or open
`/skills diff` on the CLI / dashboard / the staged file under
`~/.hermes/pending/skills/<id>.json` when you want to read the whole change.
Full details in [Gating agent skill writes](/user-guide/features/skills#gating-agent-skill-writes-skillswrite_approval).
## External Memory Providers
For deeper, persistent memory that goes beyond MEMORY.md and USER.md, Hermes ships with 8 external memory provider plugins — including Honcho, OpenViking, Mem0, Hindsight, Holographic, RetainDB, ByteRover, and Supermemory.
External providers run **alongside** built-in memory (never replacing it) and add capabilities like knowledge graphs, semantic search, automatic fact extraction, and cross-session user modeling.
```bash
hermes memory setup # pick a provider and configure it
hermes memory status # check what's active
```
See the [Memory Providers](./memory-providers.md) guide for full details on each provider, setup instructions, and comparison.
website/docs/user-guide/features/overview.md
+56
View File
@@ -0,0 +1,56 @@
---
title: "Features Overview"
sidebar_label: "Overview"
sidebar_position: 1
---
# Features Overview
Hermes Agent includes a rich set of capabilities that extend far beyond basic chat. From persistent memory and file-aware context to browser automation and voice conversations, these features work together to make Hermes a powerful autonomous assistant.
:::tip Don't know where to start?
`hermes setup --portal` covers a model provider plus all four Tool Gateway tools (web search, image generation, TTS, browser) in one command. See [Nous Portal](/integrations/nous-portal).
:::
## Core
- **[Tools & Toolsets](tools.md)** — Tools are functions that extend the agent's capabilities. They're organized into logical toolsets that can be enabled or disabled per platform, covering web search, terminal execution, file editing, memory, delegation, and more.
- **[Skills System](skills.md)** — On-demand knowledge documents the agent can load when needed. Skills follow a progressive disclosure pattern to minimize token usage and are compatible with the [agentskills.io](https://agentskills.io/specification) open standard.
- **[Persistent Memory](memory.md)** — Bounded, curated memory that persists across sessions. Hermes remembers your preferences, projects, environment, and things it has learned via `MEMORY.md` and `USER.md`.
- **[Context Files](context-files.md)** — Hermes automatically discovers and loads project context files (`.hermes.md`, `AGENTS.md`, `CLAUDE.md`, `SOUL.md`, `.cursorrules`) that shape how it behaves in your project.
- **[Context References](context-references.md)** — Type `@` followed by a reference to inject files, folders, git diffs, and URLs directly into your messages. Hermes expands the reference inline and appends the content automatically.
- **[Checkpoints](../checkpoints-and-rollback.md)** — Hermes automatically snapshots your working directory before making file changes, giving you a safety net to roll back with `/rollback` if something goes wrong.
## Automation
- **[Scheduled Tasks (Cron)](cron.md)** — Schedule tasks to run automatically with natural language or cron expressions. Jobs can attach skills, deliver results to any platform, and support pause/resume/edit operations.
- **[Subagent Delegation](delegation.md)** — The `delegate_task` tool spawns child agent instances with isolated context, restricted toolsets, and their own terminal sessions. Run 3 concurrent subagents by default (configurable) for parallel workstreams.
- **[Code Execution](code-execution.md)** — The `execute_code` tool lets the agent write Python scripts that call Hermes tools programmatically, collapsing multi-step workflows into a single LLM turn via sandboxed RPC execution.
- **[Event Hooks](hooks.md)** — Run custom code at key lifecycle points. Gateway hooks handle logging, alerts, and webhooks; plugin hooks handle tool interception, metrics, and guardrails.
- **[Batch Processing](batch-processing.md)** — Run the Hermes agent across hundreds or thousands of prompts in parallel, generating structured ShareGPT-format trajectory data for training data generation or evaluation.
## Media & Web
- **[Voice Mode](voice-mode.md)** — Full voice interaction across CLI and messaging platforms. Talk to the agent using your microphone, hear spoken replies, and have live voice conversations in Discord voice channels.
- **[Browser Automation](browser.md)** — Full browser automation with multiple backends: Browserbase cloud, Browser Use cloud, local Chrome/Brave/Chromium/Edge via CDP, or local Chromium. Navigate websites, fill forms, and extract information.
- **[Vision & Image Paste](vision.md)** — Multimodal vision support. Paste images from your clipboard into the CLI and ask the agent to analyze, describe, or work with them using any vision-capable model.
- **[Image Generation](image-generation.md)** — Generate images from text prompts using FAL.ai. Nine models supported (FLUX 2 Klein/Pro, GPT-Image 1.5/2, Nano Banana Pro, Ideogram V3, Recraft V4 Pro, Qwen, Z-Image Turbo); pick one via `hermes tools`.
- **[Voice & TTS](tts.md)** — Text-to-speech output and voice message transcription across all messaging platforms, with ten native provider options: Edge TTS (free), ElevenLabs, OpenAI TTS, MiniMax, Mistral Voxtral, Google Gemini, xAI, NeuTTS, KittenTTS, and Piper — plus custom command providers for any local TTS CLI.
## Integrations
- **[MCP Integration](mcp.md)** — Connect to any MCP server via stdio or HTTP transport. Access external tools from GitHub, databases, file systems, and internal APIs without writing native Hermes tools. Includes per-server tool filtering and sampling support.
- **[Provider Routing](provider-routing.md)** — Fine-grained control over which AI providers handle your requests. Optimize for cost, speed, or quality with sorting, whitelists, blacklists, and priority ordering.
- **[Fallback Providers](fallback-providers.md)** — Automatic failover to backup LLM providers when your primary model encounters errors, including independent fallback for auxiliary tasks like vision and compression.
- **[Credential Pools](credential-pools.md)** — Distribute API calls across multiple keys for the same provider. Automatic rotation on rate limits or failures.
- **[Prompt caching](../configuration#prompt-caching)** — Built-in cross-session 1-hour prefix cache for Claude on native Anthropic, OpenRouter, and Nous Portal. Always-on; no configuration required.
- **[Memory Providers](memory-providers.md)** — Plug in external memory backends (Honcho, OpenViking, Mem0, Hindsight, Holographic, RetainDB, ByteRover, Supermemory) for cross-session user modeling and personalization beyond the built-in memory system.
- **[API Server](api-server.md)** — Expose Hermes as an OpenAI-compatible HTTP endpoint. Connect any frontend that speaks the OpenAI format — Open WebUI, LobeChat, LibreChat, and more.
- **[IDE Integration (ACP)](acp.md)** — Use Hermes inside ACP-compatible editors such as VS Code, Zed, and JetBrains. Chat, tool activity, file diffs, and terminal commands render inside your editor.
- **[Batch Processing](batch-processing.md)** — Run the agent over many prompts or tasks in parallel from the CLI, with structured outputs and trajectory capture suitable for evals or downstream training pipelines.
## Customization
- **[Personality & SOUL.md](personality.md)** — Fully customizable agent personality. `SOUL.md` is the primary identity file — the first thing in the system prompt — and you can swap in built-in or custom `/personality` presets per session.
- **[Skins & Themes](skins.md)** — Customize the CLI's visual presentation: banner colors, spinner faces and verbs, response-box labels, branding text, and the tool activity prefix.
- **[Plugins](plugins.md)** — Add custom tools, hooks, and integrations without modifying core code. Three plugin types: general plugins (tools/hooks), memory providers (cross-session knowledge), and context engines (alternative context management). Managed via the unified `hermes plugins` interactive UI.
website/docs/user-guide/features/personality.md
+271
View File
@@ -0,0 +1,271 @@
---
sidebar_position: 9
title: "Personality & SOUL.md"
description: "Customize Hermes Agent's personality with a global SOUL.md, built-in personalities, and custom persona definitions"
---
# Personality & SOUL.md
Hermes Agent's personality is fully customizable. `SOUL.md` is the **primary identity** — it's the first thing in the system prompt and defines who the agent is.
- `SOUL.md` — a durable persona file that lives in `HERMES_HOME` and serves as the agent's identity (slot #1 in the system prompt)
- built-in or custom `/personality` presets — session-level system-prompt overlays
If you want to change who Hermes is — or replace it with an entirely different agent persona — edit `SOUL.md`.
## How SOUL.md works now
Hermes now seeds a default `SOUL.md` automatically in:
```text
~/.hermes/SOUL.md
```
More precisely, it uses the current instance's `HERMES_HOME`, so if you run Hermes with a custom home directory, it will use:
```text
$HERMES_HOME/SOUL.md
```
### Important behavior
- **SOUL.md is the agent's primary identity.** It occupies slot #1 in the system prompt, replacing the hardcoded default identity.
- Hermes creates a starter `SOUL.md` automatically if one does not exist yet
- Existing user `SOUL.md` files are never overwritten
- Hermes loads `SOUL.md` only from `HERMES_HOME`
- Hermes does not look in the current working directory for `SOUL.md`
- If `SOUL.md` exists but is empty, or cannot be loaded, Hermes falls back to a built-in default identity
- If `SOUL.md` has content, that content is injected verbatim after security scanning and truncation
- SOUL.md is **not** duplicated in the context files section — it appears only once, as the identity
That makes `SOUL.md` a true per-user or per-instance identity, not just an additive layer.
## Why this design
This keeps personality predictable.
If Hermes loaded `SOUL.md` from whatever directory you happened to launch it in, your personality could change unexpectedly between projects. By loading only from `HERMES_HOME`, the personality belongs to the Hermes instance itself.
That also makes it easier to teach users:
- "Edit `~/.hermes/SOUL.md` to change Hermes' default personality."
## Where to edit it
For most users:
```bash
~/.hermes/SOUL.md
```
If you use a custom home:
```bash
$HERMES_HOME/SOUL.md
```
## What should go in SOUL.md?
Use it for durable voice and personality guidance, such as:
- tone
- communication style
- level of directness
- default interaction style
- what to avoid stylistically
- how Hermes should handle uncertainty, disagreement, or ambiguity
Use it less for:
- one-off project instructions
- file paths
- repo conventions
- temporary workflow details
Those belong in `AGENTS.md`, not `SOUL.md`.
## Good SOUL.md content
A good SOUL file is:
- stable across contexts
- broad enough to apply in many conversations
- specific enough to materially shape the voice
- focused on communication and identity, not task-specific instructions
### Example
```markdown
# Personality
You are a pragmatic senior engineer with strong taste.
You optimize for truth, clarity, and usefulness over politeness theater.
## Style
- Be direct without being cold
- Prefer substance over filler
- Push back when something is a bad idea
- Admit uncertainty plainly
- Keep explanations compact unless depth is useful
## What to avoid
- Sycophancy
- Hype language
- Repeating the user's framing if it's wrong
- Overexplaining obvious things
## Technical posture
- Prefer simple systems over clever systems
- Care about operational reality, not idealized architecture
- Treat edge cases as part of the design, not cleanup
```
## What Hermes injects into the prompt
`SOUL.md` content goes directly into slot #1 of the system prompt — the agent identity position. No wrapper language is added around it.
The content goes through:
- prompt-injection scanning
- truncation if it is too large
If the file is empty, whitespace-only, or cannot be read, Hermes falls back to a built-in default identity ("You are Hermes Agent, an intelligent AI assistant created by Nous Research..."). This fallback also applies when `skip_context_files` is set (e.g., in subagent/delegation contexts).
## Security scanning
`SOUL.md` is scanned like other context-bearing files for prompt injection patterns before inclusion.
That means you should still keep it focused on persona/voice rather than trying to sneak in strange meta-instructions.
## SOUL.md vs AGENTS.md
This is the most important distinction.
### SOUL.md
Use for:
- identity
- tone
- style
- communication defaults
- personality-level behavior
### AGENTS.md
Use for:
- project architecture
- coding conventions
- tool preferences
- repo-specific workflows
- commands, ports, paths, deployment notes
A useful rule:
- if it should follow you everywhere, it belongs in `SOUL.md`
- if it belongs to a project, it belongs in `AGENTS.md`
## SOUL.md vs `/personality`
`SOUL.md` is your durable default personality.
`/personality` is a session-level overlay that changes or supplements the current system prompt.
So:
- `SOUL.md` = baseline voice
- `/personality` = temporary mode switch
Examples:
- keep a pragmatic default SOUL, then use `/personality teacher` for a tutoring conversation
- keep a concise SOUL, then use `/personality creative` for brainstorming
## Built-in personalities
Hermes ships with built-in personalities you can switch to with `/personality`.
| Name | Description |
|------|-------------|
| **helpful** | Friendly, general-purpose assistant |
| **concise** | Brief, to-the-point responses |
| **technical** | Detailed, accurate technical expert |
| **creative** | Innovative, outside-the-box thinking |
| **teacher** | Patient educator with clear examples |
| **kawaii** | Cute expressions, sparkles, and enthusiasm ★ |
| **catgirl** | Neko-chan with cat-like expressions, nya~ |
| **pirate** | Captain Hermes, tech-savvy buccaneer |
| **shakespeare** | Bardic prose with dramatic flair |
| **surfer** | Totally chill bro vibes |
| **noir** | Hard-boiled detective narration |
| **uwu** | Maximum cute with uwu-speak |
| **philosopher** | Deep contemplation on every query |
| **hype** | MAXIMUM ENERGY AND ENTHUSIASM!!! |
## Switching personalities with commands
### CLI
```text
/personality
/personality concise
/personality technical
```
### Messaging platforms
```text
/personality teacher
```
These are convenient overlays, but your global `SOUL.md` still gives Hermes its persistent default personality unless the overlay meaningfully changes it.
## Custom personalities in config
You can also define named custom personalities in `~/.hermes/config.yaml` under `agent.personalities`.
```yaml
agent:
personalities:
codereviewer: >
You are a meticulous code reviewer. Identify bugs, security issues,
performance concerns, and unclear design choices. Be precise and constructive.
```
Then switch to it with:
```text
/personality codereviewer
```
## Recommended workflow
A strong default setup is:
1. Keep a thoughtful global `SOUL.md` in `~/.hermes/SOUL.md`
2. Put project instructions in `AGENTS.md`
3. Use `/personality` only when you want a temporary mode shift
That gives you:
- a stable voice
- project-specific behavior where it belongs
- temporary control when needed
## How personality interacts with the full prompt
At a high level, the prompt stack includes:
1. **SOUL.md** (agent identity — or built-in fallback if SOUL.md is unavailable)
2. tool-aware behavior guidance
3. memory/user context
4. skills guidance
5. context files (`AGENTS.md`, `.cursorrules`)
6. timestamp
7. platform-specific formatting hints
8. optional system-prompt overlays such as `/personality`
`SOUL.md` is the foundation — everything else builds on top of it.
## Related docs
- [Context Files](/user-guide/features/context-files)
- [Configuration](/user-guide/configuration)
- [Tips & Best Practices](/guides/tips)
- [SOUL.md Guide](/guides/use-soul-with-hermes)
## CLI appearance vs conversational personality
Conversational personality and CLI appearance are separate:
- `SOUL.md`, `agent.system_prompt`, and `/personality` affect how Hermes speaks
- `display.skin` and `/skin` affect how Hermes looks in the terminal
For terminal appearance, see [Skins & Themes](./skins.md).
website/docs/user-guide/features/plugins.md
+346
View File
@@ -0,0 +1,346 @@
---
sidebar_position: 11
sidebar_label: "Plugins"
title: "Plugins"
description: "Extend Hermes with custom tools, hooks, and integrations via the plugin system"
---
# Plugins
Hermes has a plugin system for adding custom tools, hooks, and integrations without modifying core code.
If you want to create a custom tool for yourself, your team, or one project,
this is usually the right path. The developer guide's
[Adding Tools](/developer-guide/adding-tools) page is for built-in Hermes
core tools that live in `tools/` and `toolsets.py`.
**→ [Build a Hermes Plugin](/guides/build-a-hermes-plugin)** — step-by-step guide with a complete working example.
## Quick overview
Drop a directory into `~/.hermes/plugins/` with a `plugin.yaml` and Python code:
```
~/.hermes/plugins/my-plugin/
├── plugin.yaml # manifest
├── __init__.py # register() — wires schemas to handlers
├── schemas.py # tool schemas (what the LLM sees)
└── tools.py # tool handlers (what runs when called)
```
Start Hermes — your tools appear alongside built-in tools. The model can call them immediately.
### Minimal working example
Here is a complete plugin that adds a `hello_world` tool and logs every tool call via a hook.
**`~/.hermes/plugins/hello-world/plugin.yaml`**
```yaml
name: hello-world
version: "1.0"
description: A minimal example plugin
```
**`~/.hermes/plugins/hello-world/__init__.py`**
```python
"""Minimal Hermes plugin — registers a tool and a hook."""
import json
def register(ctx):
# --- Tool: hello_world ---
schema = {
"name": "hello_world",
"description": "Returns a friendly greeting for the given name.",
"parameters": {
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Name to greet",
}
},
"required": ["name"],
},
}
def handle_hello(params, **kwargs):
del kwargs
name = params.get("name", "World")
return json.dumps({"success": True, "greeting": f"Hello, {name}!"})
ctx.register_tool(
name="hello_world",
toolset="hello_world",
schema=schema,
handler=handle_hello,
description="Return a friendly greeting for the given name.",
)
# --- Hook: log every tool call ---
def on_tool_call(tool_name, params, result):
print(f"[hello-world] tool called: {tool_name}")
ctx.register_hook("post_tool_call", on_tool_call)
```
Drop both files into `~/.hermes/plugins/hello-world/`, restart Hermes, and the model can immediately call `hello_world`. The hook prints a log line after every tool invocation.
Project-local plugins under `./.hermes/plugins/` are disabled by default. Enable them only for trusted repositories by setting `HERMES_ENABLE_PROJECT_PLUGINS=true` before starting Hermes.
## What plugins can do
Every `ctx.*` API below is available inside a plugin's `register(ctx)` function.
| Capability | How |
|-----------|-----|
| Add tools | `ctx.register_tool(name=..., toolset=..., schema=..., handler=...)` |
| Add hooks | `ctx.register_hook("post_tool_call", callback)` |
| Add slash commands | `ctx.register_command(name, handler, description)` — adds `/name` in CLI and gateway sessions |
| Dispatch tools from commands | `ctx.dispatch_tool(name, args)` — invokes a registered tool with parent-agent context auto-wired |
| Add CLI commands | `ctx.register_cli_command(name, help, setup_fn, handler_fn)` — adds `hermes <plugin> <subcommand>` |
| Inject messages | `ctx.inject_message(content, role="user")` — see [Injecting Messages](#injecting-messages) |
| Ship data files | `Path(__file__).parent / "data" / "file.yaml"` |
| Bundle skills | `ctx.register_skill(name, path)` — namespaced as `plugin:skill`, loaded via `skill_view("plugin:skill")` |
| Gate on env vars | `requires_env: [API_KEY]` in plugin.yaml — prompted during `hermes plugins install` |
| Distribute via pip | `[project.entry-points."hermes_agent.plugins"]` |
| Register a gateway platform (Discord, Telegram, IRC, …) | `ctx.register_platform(name, label, adapter_factory, check_fn, ...)` — see [Adding Platform Adapters](/developer-guide/adding-platform-adapters) |
| Register an image-generation backend | `ctx.register_image_gen_provider(provider)` — see [Image Generation Provider Plugins](/developer-guide/image-gen-provider-plugin) |
| Register a video-generation backend | `ctx.register_video_gen_provider(provider)` — see [Video Generation Provider Plugins](/developer-guide/video-gen-provider-plugin) |
| Register a context-compression engine | `ctx.register_context_engine(engine)` — see [Context Engine Plugins](/developer-guide/context-engine-plugin) |
| Register a memory backend | Subclass `MemoryProvider` in `plugins/memory/<name>/__init__.py` — see [Memory Provider Plugins](/developer-guide/memory-provider-plugin) (uses a separate discovery system) |
| Run a host-owned LLM call | `ctx.llm.complete(...)` / `ctx.llm.complete_structured(...)` — borrow the user's active model + auth for a one-shot completion with optional JSON schema validation. See [Plugin LLM Access](/developer-guide/plugin-llm-access) |
| Register an inference backend (LLM provider) | `register_provider(ProviderProfile(...))` in `plugins/model-providers/<name>/__init__.py` — see [Model Provider Plugins](/developer-guide/model-provider-plugin) (uses a separate discovery system) |
## Plugin discovery
| Source | Path | Use case |
|--------|------|----------|
| Bundled | `<repo>/plugins/` | Ships with Hermes — see [Built-in Plugins](/user-guide/features/built-in-plugins) |
| User | `~/.hermes/plugins/` | Personal plugins |
| Project | `.hermes/plugins/` | Project-specific plugins (requires `HERMES_ENABLE_PROJECT_PLUGINS=true`) |
| pip | `hermes_agent.plugins` entry_points | Distributed packages |
| Nix | `services.hermes-agent.extraPlugins` / `extraPythonPackages` | NixOS declarative installs — see [Nix Setup](/getting-started/nix-setup#plugins) |
Later sources override earlier ones on name collision, so a user plugin with the same name as a bundled plugin replaces it.
### Plugin sub-categories
Within each source, Hermes also recognizes sub-category directories that route plugins to specialized discovery systems:
| Sub-directory | What it holds | Discovery system |
|---|---|---|
| `plugins/` (root) | General plugins — tools, hooks, slash commands, CLI commands, bundled skills | `PluginManager` (kind: `standalone` or `backend`) |
| `plugins/platforms/<name>/` | Gateway channel adapters (`ctx.register_platform()`) | `PluginManager` (kind: `platform`, one level deeper) |
| `plugins/image_gen/<name>/` | Image-generation backends (`ctx.register_image_gen_provider()`) | `PluginManager` (kind: `backend`, one level deeper) |
| `plugins/memory/<name>/` | Memory providers (subclass `MemoryProvider`) | **Own loader** in `plugins/memory/__init__.py` (kind: `exclusive` — one active at a time) |
| `plugins/context_engine/<name>/` | Context-compression engines (`ctx.register_context_engine()`) | **Own loader** in `plugins/context_engine/__init__.py` (one active at a time) |
| `plugins/model-providers/<name>/` | LLM provider profiles (`register_provider(ProviderProfile(...))`) | **Own loader** in `providers/__init__.py` (lazily scanned on first `get_provider_profile()` call) |
User plugins at `~/.hermes/plugins/model-providers/<name>/` and `~/.hermes/plugins/memory/<name>/` override bundled plugins of the same name — last-writer-wins in `register_provider()` / `register_memory_provider()`. Drop a directory in, and it replaces the built-in without any repo edits.
## Plugins are opt-in (with a few exceptions)
**General plugins and user-installed backends are disabled by default** — discovery finds them (so they show up in `hermes plugins` and `/plugins`), but nothing with hooks or tools loads until you add the plugin's name to `plugins.enabled` in `~/.hermes/config.yaml`. This stops third-party code from running without your explicit consent.
```yaml
plugins:
enabled:
- my-tool-plugin
- disk-cleanup
disabled: # optional deny-list — always wins if a name appears in both
- noisy-plugin
```
Three ways to flip state:
```bash
hermes plugins # interactive toggle (space to check/uncheck)
hermes plugins enable <name> # add to allow-list
hermes plugins disable <name> # remove from allow-list + add to disabled
```
After `hermes plugins install owner/repo`, you're asked `Enable 'name' now? [y/N]` — defaults to no. Skip the prompt for scripted installs with `--enable` or `--no-enable`.
### What the allow-list does NOT gate
Several categories of plugin bypass `plugins.enabled` — they're part of Hermes' built-in surface and would break basic functionality if gated off by default:
| Plugin kind | How it's activated instead |
|---|---|
| **Bundled platform plugins** (IRC, Teams, etc. under `plugins/platforms/`) | Auto-loaded so every shipped gateway channel is available. The actual channel turns on via `gateway.platforms.<name>.enabled` in `config.yaml`. |
| **Bundled backends** (image-gen providers under `plugins/image_gen/`, etc.) | Auto-loaded so the default backend "just works". Selection happens via `<category>.provider` in `config.yaml` (e.g. `image_gen.provider: openai`). |
| **Memory providers** (`plugins/memory/`) | All discovered; exactly one is active, chosen by `memory.provider` in `config.yaml`. |
| **Context engines** (`plugins/context_engine/`) | All discovered; one is active, chosen by `context.engine` in `config.yaml`. |
| **Model providers** (`plugins/model-providers/`) | All bundled providers under `plugins/model-providers/` discover and register at the first `get_provider_profile()` call. The user picks one at a time via `--provider` or `config.yaml`. |
| **Pip-installed `backend` plugins** | Opt-in via `plugins.enabled` (same as general plugins). |
| **User-installed platforms** (under `~/.hermes/plugins/platforms/`) | Opt-in via `plugins.enabled` — third-party gateway adapters need explicit consent. |
In short: **bundled "always-works" infrastructure loads automatically; third-party general plugins are opt-in.** The `plugins.enabled` allow-list is the gate specifically for arbitrary code a user drops into `~/.hermes/plugins/`.
### Migration for existing users
When you upgrade to a version of Hermes that has opt-in plugins (config schema v21+), any user plugins already installed under `~/.hermes/plugins/` that weren't already in `plugins.disabled` are **automatically grandfathered** into `plugins.enabled`. Your existing setup keeps working. Bundled standalone plugins are NOT grandfathered — even existing users have to opt in explicitly. (Bundled platform/backend plugins never needed grandfathering because they were never gated.)
## Available hooks
Plugins can register callbacks for these lifecycle events. See the **[Event Hooks page](/user-guide/features/hooks#plugin-hooks)** for full details, callback signatures, and examples.
| Hook | Fires when |
|------|-----------|
| [`pre_tool_call`](/user-guide/features/hooks#pre_tool_call) | Before any tool executes |
| [`post_tool_call`](/user-guide/features/hooks#post_tool_call) | After any tool returns |
| [`pre_llm_call`](/user-guide/features/hooks#pre_llm_call) | Once per turn, before the LLM loop — can return `{"context": "..."}` to [inject context into the user message](/user-guide/features/hooks#pre_llm_call) |
| [`post_llm_call`](/user-guide/features/hooks#post_llm_call) | Once per turn, after the LLM loop (successful turns only) |
| [`on_session_start`](/user-guide/features/hooks#on_session_start) | New session created (first turn only) |
| [`on_session_end`](/user-guide/features/hooks#on_session_end) | End of every `run_conversation` call + CLI exit handler |
| [`on_session_finalize`](/user-guide/features/hooks#on_session_finalize) | CLI/gateway tears down an active session (`/new`, GC, CLI quit) |
| [`on_session_reset`](/user-guide/features/hooks#on_session_reset) | Gateway swaps in a new session key (`/new`, `/reset`, `/clear`, idle rotation) |
| [`subagent_stop`](/user-guide/features/hooks#subagent_stop) | Once per child after `delegate_task` finishes |
| [`pre_gateway_dispatch`](/user-guide/features/hooks#pre_gateway_dispatch) | Gateway received a user message, before auth + dispatch. Return `{"action": "skip" \| "rewrite" \| "allow", ...}` to influence flow. |
## Plugin types
Hermes has four kinds of plugins:
| Type | What it does | Selection | Location |
|------|-------------|-----------|----------|
| **General plugins** | Add tools, hooks, slash commands, CLI commands | Multi-select (enable/disable) | `~/.hermes/plugins/` |
| **Memory providers** | Replace or augment built-in memory | Single-select (one active) | `plugins/memory/` |
| **Context engines** | Replace the built-in context compressor | Single-select (one active) | `plugins/context_engine/` |
| **Model providers** | Declare an inference backend (OpenRouter, Anthropic, …) | Multi-register, picked by `--provider` / `config.yaml` | `plugins/model-providers/` |
Memory providers and context engines are **provider plugins** — only one of each type can be active at a time. Model providers are also plugins, but many load simultaneously; the user picks one at a time via `--provider` or `config.yaml`. General plugins can be enabled in any combination.
## Pluggable interfaces — where to go for each
The table above shows the four plugin categories, but within "General plugins" the `PluginContext` exposes several distinct extension points — and Hermes also accepts extensions outside the Python plugin system (config-driven backends, shell-hooked commands, external servers, etc.). Use this table to find the right doc for what you want to build:
| Want to add… | How | Authoring guide |
|---|---|---|
| A **tool** the LLM can call | Python plugin — `ctx.register_tool()` | [Build a Hermes Plugin](/guides/build-a-hermes-plugin) · [Adding Tools](/developer-guide/adding-tools) |
| A **lifecycle hook** (pre/post LLM, session start/end, tool filter) | Python plugin — `ctx.register_hook()` | [Hooks reference](/user-guide/features/hooks) · [Build a Hermes Plugin](/guides/build-a-hermes-plugin) |
| A **slash command** for the CLI / gateway | Python plugin — `ctx.register_command()` | [Build a Hermes Plugin](/guides/build-a-hermes-plugin) · [Extending the CLI](/developer-guide/extending-the-cli) |
| A **subcommand** for `hermes <thing>` | Python plugin — `ctx.register_cli_command()` | [Extending the CLI](/developer-guide/extending-the-cli) |
| A bundled **skill** that your plugin ships | Python plugin — `ctx.register_skill()` | [Creating Skills](/developer-guide/creating-skills) |
| An **inference backend** (LLM provider: OpenAI-compat, Codex, Anthropic-Messages, Bedrock) | Provider plugin — `register_provider(ProviderProfile(...))` in `plugins/model-providers/<name>/` | **[Model Provider Plugins](/developer-guide/model-provider-plugin)** · [Adding Providers](/developer-guide/adding-providers) |
| A **gateway channel** (Discord / Telegram / IRC / Teams / etc.) | Platform plugin — `ctx.register_platform()` in `plugins/platforms/<name>/` | [Adding Platform Adapters](/developer-guide/adding-platform-adapters) |
| A **memory backend** (Honcho, Mem0, Supermemory, …) | Memory plugin — subclass `MemoryProvider` in `plugins/memory/<name>/` | [Memory Provider Plugins](/developer-guide/memory-provider-plugin) |
| A **context-compression strategy** | Context-engine plugin — `ctx.register_context_engine()` | [Context Engine Plugins](/developer-guide/context-engine-plugin) |
| An **image-generation backend** (DALL·E, SDXL, …) | Backend plugin — `ctx.register_image_gen_provider()` | [Image Generation Provider Plugins](/developer-guide/image-gen-provider-plugin) |
| A **video-generation backend** (Veo, Kling, Pixverse, Grok-Imagine, Runway, …) | Backend plugin — `ctx.register_video_gen_provider()` | [Video Generation Provider Plugins](/developer-guide/video-gen-provider-plugin) |
| A **TTS backend** (any CLI — Piper, VoxCPM, Kokoro, xtts, voice-cloning scripts, …) | Config-driven (recommended) — declare under `tts.providers.<name>` with `type: command` in `config.yaml`. OR Python backend plugin — `ctx.register_tts_provider()` for Python-SDK / streaming engines that need more than a shell template. | [TTS Setup](/user-guide/features/tts#custom-command-providers) · [Python plugin guide](/user-guide/features/tts#python-plugin-providers) |
| An **STT backend** (any CLI — whisper.cpp, custom whisper binary, local ASR CLI) | Config-driven (recommended) — declare under `stt.providers.<name>` with `type: command` in `config.yaml`, or set `HERMES_LOCAL_STT_COMMAND` for the legacy single-command escape hatch. OR Python backend plugin — `ctx.register_transcription_provider()` for Python-SDK engines (OpenRouter, SenseAudio, Gemini-STT, etc.). | [STT Setup](/user-guide/features/tts#stt-custom-command-providers) · [Python plugin guide](/user-guide/features/tts#python-plugin-providers-stt) |
| **External tools via MCP** (filesystem, GitHub, Linear, Notion, any MCP server) | Config-driven — declare `mcp_servers.<name>` with `command:` / `url:` in `config.yaml`. Hermes auto-discovers the server's tools and registers them alongside built-ins. | [MCP](/user-guide/features/mcp) |
| **Additional skill sources** (custom GitHub repos, private skill indexes) | CLI — `hermes skills tap add <repo>` | [Skills Hub](/user-guide/features/skills#skills-hub) · [Publishing a custom tap](/user-guide/features/skills#publishing-a-custom-skill-tap) |
| **Gateway event hooks** (fire on `gateway:startup`, `session:start`, `agent:end`, `command:*`) | Drop `HOOK.yaml` + `handler.py` into `~/.hermes/hooks/<name>/` | [Event Hooks](/user-guide/features/hooks#gateway-event-hooks) |
| **Shell hooks** (run a shell command on events — notifications, audit logs, desktop alerts) | Config-driven — declare under `hooks:` in `config.yaml` | [Shell Hooks](/user-guide/features/hooks#shell-hooks) |
:::note
Not everything is a Python plugin. Some extension surfaces intentionally use **config-driven shell commands** (TTS, STT, shell hooks) so any CLI you already have becomes a plugin without writing Python. Others are **external servers** (MCP) the agent connects to and auto-registers tools from. And some are **drop-in directories** (gateway hooks) with their own manifest format. Pick the right surface for the integration style that fits your use case; the authoring guides in the table above each cover placeholders, discovery, and examples.
:::
## NixOS declarative plugins
On NixOS, plugins can be installed declaratively via the module options — no `hermes plugins install` needed. See the **[Nix Setup guide](/getting-started/nix-setup#plugins)** for full details.
```nix
services.hermes-agent = {
# Directory plugin (source tree with plugin.yaml)
extraPlugins = [ (pkgs.fetchFromGitHub { ... }) ];
# Entry-point plugin (pip package)
extraPythonPackages = [ (pkgs.python312Packages.buildPythonPackage { ... }) ];
# Enable in config
settings.plugins.enabled = [ "my-plugin" ];
};
```
Declarative plugins are symlinked with a `nix-managed-` prefix — they coexist with manually installed plugins and are cleaned up automatically when removed from the Nix config.
## Managing plugins
```bash
hermes plugins # unified interactive UI
hermes plugins list # table: enabled / disabled / not enabled
hermes plugins install user/repo # install from Git, then prompt Enable? [y/N]
hermes plugins install user/repo --enable # install AND enable (no prompt)
hermes plugins install user/repo --no-enable # install but leave disabled (no prompt)
hermes plugins update my-plugin # pull latest
hermes plugins remove my-plugin # uninstall
hermes plugins enable my-plugin # add to allow-list
hermes plugins disable my-plugin # remove from allow-list + add to disabled
```
### Interactive UI
Running `hermes plugins` with no arguments opens a composite interactive screen:
```
Plugins
↑↓ navigate SPACE toggle ENTER configure/confirm ESC done
General Plugins
→ [✓] my-tool-plugin — Custom search tool
[ ] webhook-notifier — Event hooks
[ ] disk-cleanup — Auto-cleanup of ephemeral files [bundled]
Provider Plugins
Memory Provider ▸ honcho
Context Engine ▸ compressor
```
- **General Plugins section** — checkboxes, toggle with SPACE. Checked = in `plugins.enabled`, unchecked = in `plugins.disabled` (explicit off).
- **Provider Plugins section** — shows current selection. Press ENTER to drill into a radio picker where you choose one active provider.
- Bundled plugins appear in the same list with a `[bundled]` tag.
Provider plugin selections are saved to `config.yaml`:
```yaml
memory:
provider: "honcho" # empty string = built-in only
context:
engine: "compressor" # default built-in compressor
```
### Enabled vs. disabled vs. neither
Plugins occupy one of three states:
| State | Meaning | In `plugins.enabled`? | In `plugins.disabled`? |
|---|---|---|---|
| `enabled` | Loaded on next session | Yes | No |
| `disabled` | Explicitly off — won't load even if also in `enabled` | (irrelevant) | Yes |
| `not enabled` | Discovered but never opted in | No | No |
The default for a newly-installed or bundled plugin is `not enabled`. `hermes plugins list` shows all three distinct states so you can tell what's been explicitly turned off vs. what's just waiting to be enabled.
In a running session, `/plugins` shows which plugins are currently loaded.
## Injecting Messages
Plugins can inject messages into the active conversation using `ctx.inject_message()`:
```python
ctx.inject_message("New data arrived from the webhook", role="user")
```
**Signature:** `ctx.inject_message(content: str, role: str = "user") -> bool`
How it works:
- If the agent is **idle** (waiting for user input), the message is queued as the next input and starts a new turn.
- If the agent is **mid-turn** (actively running), the message interrupts the current operation — the same as a user typing a new message and pressing Enter.
- For non-`"user"` roles, the content is prefixed with `[role]` (e.g. `[system] ...`).
- Returns `True` if the message was queued successfully, `False` if no CLI reference is available (e.g. in gateway mode).
This enables plugins like remote control viewers, messaging bridges, or webhook receivers to feed messages into the conversation from external sources.
:::note
`inject_message` is only available in CLI mode. In gateway mode, there is no CLI reference and the method returns `False`.
:::
See the **[full guide](/guides/build-a-hermes-plugin)** for handler contracts, schema format, hook behavior, error handling, and common mistakes.
website/docs/user-guide/features/provider-routing.md
+204
View File
@@ -0,0 +1,204 @@
---
title: Provider Routing
description: Configure OpenRouter provider preferences to optimize for cost, speed, or quality.
sidebar_label: Provider Routing
sidebar_position: 7
---
# Provider Routing
When using [OpenRouter](https://openrouter.ai) as your LLM provider, Hermes Agent supports **provider routing** — fine-grained control over which underlying AI providers handle your requests and how they're prioritized.
OpenRouter routes requests to many providers (e.g., Anthropic, Google, AWS Bedrock, Together AI). Provider routing lets you optimize for cost, speed, quality, or enforce specific provider requirements.
:::tip
Traffic routed through [Nous Portal](/integrations/nous-portal) still respects per-model routing and priority configs — and Portal subscribers get 10% off token-billed providers.
:::
## Configuration
Add a `provider_routing` section to your `~/.hermes/config.yaml`:
```yaml
provider_routing:
sort: "price" # How to rank providers
only: [] # Whitelist: only use these providers
ignore: [] # Blacklist: never use these providers
order: [] # Explicit provider priority order
require_parameters: false # Only use providers that support all parameters
data_collection: null # Control data collection ("allow" or "deny")
```
:::info
Provider routing only applies when using OpenRouter. It has no effect with direct provider connections (e.g., connecting directly to the Anthropic API).
:::
## Options
### `sort`
Controls how OpenRouter ranks available providers for your request.
| Value | Description |
|-------|-------------|
| `"price"` | Cheapest provider first |
| `"throughput"` | Fastest tokens-per-second first |
| `"latency"` | Lowest time-to-first-token first |
```yaml
provider_routing:
sort: "price"
```
### `only`
Whitelist of provider names. When set, **only** these providers will be used. All others are excluded.
```yaml
provider_routing:
only:
- "Anthropic"
- "Google"
```
### `ignore`
Blacklist of provider names. These providers will **never** be used, even if they offer the cheapest or fastest option.
```yaml
provider_routing:
ignore:
- "Together"
- "DeepInfra"
```
### `order`
Explicit priority order. Providers listed first are preferred. Unlisted providers are used as fallbacks.
```yaml
provider_routing:
order:
- "Anthropic"
- "Google"
- "AWS Bedrock"
```
### `require_parameters`
When `true`, OpenRouter will only route to providers that support **all** parameters in your request (like `temperature`, `top_p`, `tools`, etc.). This avoids silent parameter drops.
```yaml
provider_routing:
require_parameters: true
```
### `data_collection`
Controls whether providers can use your prompts for training. Options are `"allow"` or `"deny"`.
```yaml
provider_routing:
data_collection: "deny"
```
## Practical Examples
### Optimize for Cost
Route to the cheapest available provider. Good for high-volume usage and development:
```yaml
provider_routing:
sort: "price"
```
### Optimize for Speed
Prioritize low-latency providers for interactive use:
```yaml
provider_routing:
sort: "latency"
```
### Optimize for Throughput
Best for long-form generation where tokens-per-second matters:
```yaml
provider_routing:
sort: "throughput"
```
### Lock to Specific Providers
Ensure all requests go through a specific provider for consistency:
```yaml
provider_routing:
only:
- "Anthropic"
```
### Avoid Specific Providers
Exclude providers you don't want to use (e.g., for data privacy):
```yaml
provider_routing:
ignore:
- "Together"
- "Lepton"
data_collection: "deny"
```
### Preferred Order with Fallbacks
Try your preferred providers first, fall back to others if unavailable:
```yaml
provider_routing:
order:
- "Anthropic"
- "Google"
require_parameters: true
```
## How It Works
Provider routing preferences are passed to the OpenRouter API via the `extra_body.provider` field on every API call. This applies to both:
- **CLI mode** — configured in `~/.hermes/config.yaml`, loaded at startup
- **Gateway mode** — same config file, loaded when the gateway starts
The routing config is read from `config.yaml` and passed as parameters when creating the `AIAgent`:
```
providers_allowed ← from provider_routing.only
providers_ignored ← from provider_routing.ignore
providers_order ← from provider_routing.order
provider_sort ← from provider_routing.sort
provider_require_parameters ← from provider_routing.require_parameters
provider_data_collection ← from provider_routing.data_collection
```
:::tip
You can combine multiple options. For example, sort by price but exclude certain providers and require parameter support:
```yaml
provider_routing:
sort: "price"
ignore: ["Together"]
require_parameters: true
data_collection: "deny"
```
:::
## Default Behavior
When no `provider_routing` section is configured (the default), OpenRouter uses its own default routing logic, which generally balances cost and availability automatically.
:::tip Provider Routing vs. Fallback Models
Provider routing controls which **sub-providers within OpenRouter** handle your requests. For automatic failover to an entirely different provider when your primary model fails, see [Fallback Providers](/user-guide/features/fallback-providers).
:::
website/docs/user-guide/features/skills.md
+852
View File
@@ -0,0 +1,852 @@
---
sidebar_position: 2
title: "Skills System"
description: "On-demand knowledge documents — progressive disclosure, agent-managed skills, and the Skills Hub"
---
# Skills System
Skills are on-demand knowledge documents the agent can load when needed. They follow a **progressive disclosure** pattern to minimize token usage and are compatible with the [agentskills.io](https://agentskills.io/specification) open standard.
All skills live in **`~/.hermes/skills/`** — the primary directory and source of truth. On fresh install, bundled skills are copied from the repo. Hub-installed and agent-created skills also go here. The agent can modify or delete any skill.
You can also point Hermes at **external skill directories** — additional folders scanned alongside the local one. See [External Skill Directories](#external-skill-directories) below.
See also:
- [Bundled Skills Catalog](/reference/skills-catalog)
- [Official Optional Skills Catalog](/reference/optional-skills-catalog)
## Starting with a blank slate
By default every profile is seeded with the bundled skill catalog, and each `hermes update` adds any newly bundled skills. If you want a profile with **no bundled skills** — and that stays empty across updates — you have two paths:
**At install time** (applies to the default `~/.hermes` profile):
```bash
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash -s -- --no-skills
```
**At profile-create time** (named profiles):
```bash
hermes profile create research --no-skills
```
**On an already-installed profile** (default or named), toggle it at runtime:
```bash
hermes skills opt-out # stop future seeding — nothing on disk is touched
hermes skills opt-out --remove # also delete UNMODIFIED bundled skills (confirms first)
hermes skills opt-in --sync # undo: remove the marker and re-seed now
```
All three paths write a `.no-bundled-skills` marker into the profile directory. While the marker is present, the installer, `hermes update`, and any skill sync all skip bundled-skill seeding for that profile. Delete the marker (or run `hermes skills opt-in`) to re-enable.
:::note Safe by default
`hermes skills opt-out` only stops *future* seeding — it never deletes anything already on disk. The optional `--remove` flag deletes bundled skills **only** when they are unmodified (byte-identical to the version Hermes installed). Skills you have edited, skills installed from the hub, and skills you wrote yourself are always kept.
:::
## Using Skills
Every installed skill is automatically available as a slash command:
```bash
# In the CLI or any messaging platform:
/gif-search funny cats
/axolotl help me fine-tune Llama 3 on my dataset
/github-pr-workflow create a PR for the auth refactor
/plan design a rollout for migrating our auth provider
# Just the skill name loads it and lets the agent ask what you need:
/excalidraw
```
The bundled `plan` skill is a good example. Running `/plan [request]` loads the skill's instructions, telling Hermes to inspect context if needed, write a markdown implementation plan instead of executing the task, and save the result under `.hermes/plans/` relative to the active workspace/backend working directory.
You can also interact with skills through natural conversation:
```bash
hermes chat --toolsets skills -q "What skills do you have?"
hermes chat --toolsets skills -q "Show me the axolotl skill"
```
## Progressive Disclosure
Skills use a token-efficient loading pattern:
```
Level 0: skills_list() → [{name, description, category}, ...] (~3k tokens)
Level 1: skill_view(name) → Full content + metadata (varies)
Level 2: skill_view(name, path) → Specific reference file (varies)
```
The agent only loads the full skill content when it actually needs it.
## SKILL.md Format
```markdown
---
name: my-skill
description: Brief description of what this skill does
version: 1.0.0
platforms: [macos, linux] # Optional — restrict to specific OS platforms
metadata:
hermes:
tags: [python, automation]
category: devops
fallback_for_toolsets: [web] # Optional — conditional activation (see below)
requires_toolsets: [terminal] # Optional — conditional activation (see below)
config: # Optional — config.yaml settings
- key: my.setting
description: "What this controls"
default: "value"
prompt: "Prompt for setup"
---
# Skill Title
## When to Use
Trigger conditions for this skill.
## Procedure
1. Step one
2. Step two
## Pitfalls
- Known failure modes and fixes
## Verification
How to confirm it worked.
```
### Platform-Specific Skills
Skills can restrict themselves to specific operating systems using the `platforms` field:
| Value | Matches |
|-------|---------|
| `macos` | macOS (Darwin) |
| `linux` | Linux |
| `windows` | Windows |
```yaml
platforms: [macos] # macOS only (e.g., iMessage, Apple Reminders, FindMy)
platforms: [macos, linux] # macOS and Linux
```
When set, the skill is automatically hidden from the system prompt, `skills_list()`, and slash commands on incompatible platforms. If omitted, the skill loads on all platforms.
## Skill output and media delivery
When a skill response (or any agent response) includes a bare absolute path to a media file — for example `/home/user/screenshots/diagram.png` — the gateway auto-detects it, strips it from the visible text, and delivers the file natively to the user's chat (Telegram photo, Discord attachment, etc.) instead of leaving the raw path in the message.
For audio specifically, the `[[audio_as_voice]]` directive promotes audio files to native voice-message bubbles on platforms that support them (Telegram, WhatsApp).
### Forcing document-style delivery: `[[as_document]]`
Sometimes you want the **opposite** of inline preview: you want the file delivered as a downloadable attachment, not a re-compressed image bubble. The classic example is a high-resolution screenshot or chart — Telegram's `sendPhoto` recompresses it to ~200 KB at 1280 px, destroying readability. A 1-2 MB PNG sent via `sendDocument` keeps the original bytes intact.
If a response (or any text inside it — typically the last line) contains the literal directive `[[as_document]]`, every media path extracted from that response is delivered as a document/file attachment rather than an image bubble:
```
Here is your rendered chart:
/home/user/.hermes/cache/chart-q4-2025.png
[[as_document]]
```
The directive is stripped before delivery, so users never see it. Granularity is intentionally all-or-nothing per response: emit `[[as_document]]` once and every image path in the same response is delivered as a document. This mirrors the scope of `[[audio_as_voice]]`.
Use it from a skill when:
- You produce screenshots or charts the user needs as files (for editing in another tool, archiving, sharing intact).
- The default lossy preview would obscure detail (small text, pixel-accurate diagrams, color-sensitive renders).
Platforms without a separate document path (e.g. SMS) fall back to whatever attachment mechanism they have.
### Conditional Activation (Fallback Skills)
Skills can automatically show or hide themselves based on which tools are available in the current session. This is most useful for **fallback skills** — free or local alternatives that should only appear when a premium tool is unavailable.
```yaml
metadata:
hermes:
fallback_for_toolsets: [web] # Show ONLY when these toolsets are unavailable
requires_toolsets: [terminal] # Show ONLY when these toolsets are available
fallback_for_tools: [web_search] # Show ONLY when these specific tools are unavailable
requires_tools: [terminal] # Show ONLY when these specific tools are available
```
| Field | Behavior |
|-------|----------|
| `fallback_for_toolsets` | Skill is **hidden** when the listed toolsets are available. Shown when they're missing. |
| `fallback_for_tools` | Same, but checks individual tools instead of toolsets. |
| `requires_toolsets` | Skill is **hidden** when the listed toolsets are unavailable. Shown when they're present. |
| `requires_tools` | Same, but checks individual tools. |
**Example:** The built-in `duckduckgo-search` skill uses `fallback_for_toolsets: [web]`. When you have `FIRECRAWL_API_KEY` set, the web toolset is available and the agent uses `web_search` — the DuckDuckGo skill stays hidden. If the API key is missing, the web toolset is unavailable and the DuckDuckGo skill automatically appears as a fallback.
Skills without any conditional fields behave exactly as before — they're always shown.
## Secure Setup on Load
Skills can declare required environment variables without disappearing from discovery:
```yaml
required_environment_variables:
- name: TENOR_API_KEY
prompt: Tenor API key
help: Get a key from https://developers.google.com/tenor
required_for: full functionality
```
When a missing value is encountered, Hermes asks for it securely only when the skill is actually loaded in the local CLI. You can skip setup and keep using the skill. Messaging surfaces never ask for secrets in chat — they tell you to use `hermes setup` or `~/.hermes/.env` locally instead.
Once set, declared env vars are **automatically passed through** to `execute_code` and `terminal` sandboxes — the skill's scripts can use `$TENOR_API_KEY` directly. For non-skill env vars, use the `terminal.env_passthrough` config option. See [Environment Variable Passthrough](/user-guide/security#environment-variable-passthrough) for details.
### Skill Config Settings
Skills can also declare non-secret config settings (paths, preferences) stored in `config.yaml`:
```yaml
metadata:
hermes:
config:
- key: myplugin.path
description: Path to the plugin data directory
default: "~/myplugin-data"
prompt: Plugin data directory path
```
Settings are stored under `skills.config` in your config.yaml. `hermes config migrate` prompts for unconfigured settings, and `hermes config show` displays them. When a skill loads, its resolved config values are injected into the context so the agent knows the configured values automatically.
See [Skill Settings](/user-guide/configuration#skill-settings) and [Creating Skills — Config Settings](/developer-guide/creating-skills#config-settings-configyaml) for details.
## Skill Directory Structure
```text
~/.hermes/skills/ # Single source of truth
├── mlops/ # Category directory
│ ├── axolotl/
│ │ ├── SKILL.md # Main instructions (required)
│ │ ├── references/ # Additional docs
│ │ ├── templates/ # Output formats
│ │ ├── scripts/ # Helper scripts callable from the skill
│ │ └── assets/ # Supplementary files
│ └── vllm/
│ └── SKILL.md
├── devops/
│ └── deploy-k8s/ # Agent-created skill
│ ├── SKILL.md
│ └── references/
├── .hub/ # Skills Hub state
│ ├── lock.json
│ ├── quarantine/
│ └── audit.log
└── .bundled_manifest # Tracks seeded bundled skills
```
## External Skill Directories
If you maintain skills outside of Hermes — for example, a shared `~/.agents/skills/` directory used by multiple AI tools — you can tell Hermes to scan those directories too.
Add `external_dirs` under the `skills` section in `~/.hermes/config.yaml`:
```yaml
skills:
external_dirs:
- ~/.agents/skills
- /home/shared/team-skills
- ${SKILLS_REPO}/skills
```
Paths support `~` expansion and `${VAR}` environment variable substitution.
### How it works
- **Create locally, update in place**: New agent-created skills are written to `~/.hermes/skills/`. Existing skills are modified where they are found, including skills under `external_dirs`, when the agent uses `skill_manage` actions such as `patch`, `edit`, `write_file`, `remove_file`, or `delete`.
- **External dirs are not a write-protection boundary**: If an external skill directory is writable by the Hermes process, agent-managed skill updates can change files in that directory. Use filesystem permissions or a separate profile/toolset setup if shared external skills must stay read-only.
- **Local precedence**: If the same skill name exists in both the local dir and an external dir, the local version wins.
- **Full integration**: External skills appear in the system prompt index, `skills_list`, `skill_view`, and as `/skill-name` slash commands — no different from local skills.
- **Non-existent paths are silently skipped**: If a configured directory doesn't exist, Hermes ignores it without errors. Useful for optional shared directories that may not be present on every machine.
### Example
```text
~/.hermes/skills/ # Local (primary, read-write)
├── devops/deploy-k8s/
│ └── SKILL.md
└── mlops/axolotl/
└── SKILL.md
~/.agents/skills/ # External (shared, mutable if writable)
├── my-custom-workflow/
│ └── SKILL.md
└── team-conventions/
└── SKILL.md
```
All four skills appear in your skill index. If you create a new skill called `my-custom-workflow` locally, it shadows the external version.
## Skill Bundles
Skill bundles are tiny YAML files that group several skills under a single slash command. When you run `/<bundle-name>`, every skill listed in the bundle loads at once — useful when a particular task always benefits from the same set of skills together.
### Quick example
```bash
# Create a bundle for backend feature work
hermes bundles create backend-dev \
--skill github-code-review \
--skill test-driven-development \
--skill github-pr-workflow \
-d "Backend feature work — review, test, PR workflow"
```
Then in the CLI or any gateway platform:
```
/backend-dev refactor the auth middleware
```
The agent receives all three skills loaded into one user message, with any text after the slash command attached as a user instruction.
### YAML schema
Bundles live in **`~/.hermes/skill-bundles/<slug>.yaml`** and look like this:
```yaml
name: backend-dev
description: Backend feature work — review, test, PR workflow.
skills:
- github-code-review
- test-driven-development
- github-pr-workflow
instruction: |
Always start by writing failing tests, then implement.
Open the PR through the standard workflow with co-author tags.
```
Fields:
- `name` (optional — defaults to the filename stem) — the bundle's display name. Normalized to a hyphen slug for the slash command (`Backend Dev` → `/backend-dev`).
- `description` (optional) — short text shown in `/bundles` and `hermes bundles list`.
- `skills` (required, non-empty list) — skill names or paths relative to your skills directory. Use the same identifier you'd pass to `/<skill-name>`.
- `instruction` (optional) — extra guidance prepended to the loaded skill content. Useful for codifying "how we always use these together."
### Managing bundles
```bash
# List all installed bundles
hermes bundles list
# Inspect one bundle
hermes bundles show backend-dev
# Create a bundle interactively (omit --skill flags to enter them one per line)
hermes bundles create research
# Overwrite an existing bundle
hermes bundles create backend-dev --skill ... --force
# Delete a bundle
hermes bundles delete backend-dev
# Re-scan ~/.hermes/skill-bundles/ and report changes
hermes bundles reload
```
From inside a chat session, `/bundles` lists every installed bundle and its skills.
### Behavior
- **Bundles take precedence over individual skills** when slugs collide. If you name a bundle `research` and you also have a skill called `research`, `/research` invokes the bundle. This is intentional — you opted into the bundle by naming it.
- **Missing skills are skipped, not fatal.** If a bundle lists `skill-foo` and you haven't installed it, the bundle still loads the skills that do resolve, and the agent gets a note listing what was skipped.
- **Bundles work in every surface** — interactive CLI, TUI, dashboard chat, and every gateway platform (Telegram, Discord, Slack, …) — because dispatch is centralized in the same place as individual skill commands.
- **Bundles do not invalidate the prompt cache.** They generate a fresh user message at invocation time, the same way `/<skill-name>` does — no system prompt mutation.
### When bundles beat installing each skill manually
Use a bundle when:
- You always pair the same skills for a recurring task (`/backend-dev`, `/release-prep`, `/incident-response`).
- You want a one-character-shorter mental model than typing several `/skill` invocations in a row.
- You want to ship a team-wide "task profile" by checking the bundle YAML into a shared dotfiles repo and symlinking it into `~/.hermes/skill-bundles/`.
A bundle is just a YAML alias — it doesn't install skills for you. The skills themselves must already be present (in `~/.hermes/skills/` or an external skill directory). Otherwise the bundle invocation just skips the missing ones.
## Agent-Managed Skills (skill_manage tool)
The agent can create, update, and delete its own skills via the `skill_manage` tool. This is the agent's **procedural memory** — when it figures out a non-trivial workflow, it saves the approach as a skill for future reuse.
### When the Agent Creates Skills
- After completing a complex task (5+ tool calls) successfully
- When it hit errors or dead ends and found the working path
- When the user corrected its approach
- When it discovered a non-trivial workflow
### Actions
| Action | Use for | Key params |
|--------|---------|------------|
| `create` | New skill from scratch | `name`, `content` (full SKILL.md), optional `category` |
| `patch` | Targeted fixes (preferred) | `name`, `old_string`, `new_string` |
| `edit` | Major structural rewrites | `name`, `content` (full SKILL.md replacement) |
| `delete` | Remove a skill entirely | `name` |
| `write_file` | Add/update supporting files | `name`, `file_path`, `file_content` |
| `remove_file` | Remove a supporting file | `name`, `file_path` |
:::tip
The `patch` action is preferred for updates — it's more token-efficient than `edit` because only the changed text appears in the tool call.
:::
### Gating agent skill writes (`skills.write_approval`)
By default the agent writes skills freely — including from the [background
self-improvement review](/user-guide/features/memory#controlling-memory-writes-write_approval)
that runs after a turn. If you'd rather approve every skill write first
(small models that misjudge what they learned, secure environments, or just
wanting eyes on the self-improvement loop), turn on the write-approval gate:
```yaml
skills:
write_approval: false # false = write freely (default) | true = require approval
```
When `write_approval: true`, every `skill_manage` write (create / edit /
patch / delete / write_file / remove_file) is **staged** instead of committed —
a SKILL.md is too large to review inline, so staging applies regardless of
whether the write came from a foreground turn or the background review.
Staged writes survive restarts under `~/.hermes/pending/skills/` and are
reviewed with the same familiar approve/deny flow as dangerous commands:
```
/skills pending # list staged skill writes + a one-line gist each
/skills diff <id> # full unified diff (best viewed in CLI or dashboard)
/skills approve <id> # apply it (or 'all')
/skills reject <id> # drop it (or 'all')
/skills approval on # turn the gate on (or 'off') and persist it
```
The review surface works in the interactive CLI and on messaging platforms
(diff output is truncated for chat bubbles — read the full diff on the CLI or
in the pending JSON file). Memory writes have the same gate under
`memory.write_approval` — see [Controlling memory writes](/user-guide/features/memory#controlling-memory-writes-write_approval).
> The separate `skills.guard_agent_created` setting is a content scanner
> (dangerous-pattern heuristics), not an approval gate — the two are
> independent. See [Guard on agent-created skill writes](/user-guide/configuration#guard-on-agent-created-skill-writes).
## Skills Hub
Browse, search, install, and manage skills from online registries, `skills.sh`, direct well-known skill endpoints, and official optional skills.
### Common commands
```bash
hermes skills browse # Browse all hub skills (official first)
hermes skills browse --source official # Browse only official optional skills
hermes skills search kubernetes # Search all sources
hermes skills search react --source skills-sh # Search the skills.sh directory
hermes skills search https://mintlify.com/docs --source well-known
hermes skills inspect openai/skills/k8s # Preview before installing
hermes skills install openai/skills/k8s # Install with security scan
hermes skills install official/security/1password
hermes skills install skills-sh/vercel-labs/json-render/json-render-react --force
hermes skills install well-known:https://mintlify.com/docs/.well-known/skills/mintlify
hermes skills install https://sharethis.chat/SKILL.md # Direct URL (single-file SKILL.md)
hermes skills install https://example.com/SKILL.md --name my-skill # Override name when frontmatter has none
hermes skills list --source hub # List hub-installed skills
hermes skills check # Check installed hub skills for upstream updates
hermes skills update # Reinstall hub skills with upstream changes when needed
hermes skills audit # Re-scan all hub skills for security
hermes skills uninstall k8s # Remove a hub skill
hermes skills reset google-workspace # Un-stick a bundled skill from "user-modified" (see below)
hermes skills reset google-workspace --restore # Also restore the bundled version, deleting your local edits
hermes skills publish skills/my-skill --to github --repo owner/repo
hermes skills snapshot export setup.json # Export skill config
hermes skills tap add myorg/skills-repo # Add a custom GitHub source
```
### Supported hub sources
| Source | Example | Notes |
|--------|---------|-------|
| `official` | `official/security/1password` | Optional skills shipped with Hermes. |
| `skills-sh` | `skills-sh/vercel-labs/agent-skills/vercel-react-best-practices` | Searchable via `hermes skills search <query> --source skills-sh`. Hermes resolves alias-style skills when the skills.sh slug differs from the repo folder. |
| `well-known` | `well-known:https://mintlify.com/docs/.well-known/skills/mintlify` | Skills served directly from `/.well-known/skills/index.json` on a website. Search using the site or docs URL. |
| `url` | `https://sharethis.chat/SKILL.md` | Direct HTTP(S) URL to a single-file `SKILL.md`. Name resolution: frontmatter → URL slug → interactive prompt → `--name` flag. |
| `github` | `openai/skills/k8s` | Direct GitHub repo/path installs and custom taps. |
| `clawhub`, `lobehub`, `browse-sh` | Source-specific identifiers | Community or marketplace integrations. |
### Integrated hubs and registries
Hermes currently integrates with these skills ecosystems and discovery sources:
#### 1. Official optional skills (`official`)
These are maintained in the Hermes repository itself and install with built-in trust.
- Catalog: [Official Optional Skills Catalog](../../reference/optional-skills-catalog)
- Source in repo: `optional-skills/`
- Example:
```bash
hermes skills browse --source official
hermes skills install official/security/1password
```
#### 2. skills.sh (`skills-sh`)
This is Vercel's public skills directory. Hermes can search it directly, inspect skill detail pages, resolve alias-style slugs, and install from the underlying source repo.
- Directory: [skills.sh](https://skills.sh/)
- CLI/tooling repo: [vercel-labs/skills](https://github.com/vercel-labs/skills)
- Official Vercel skills repo: [vercel-labs/agent-skills](https://github.com/vercel-labs/agent-skills)
- Example:
```bash
hermes skills search react --source skills-sh
hermes skills inspect skills-sh/vercel-labs/json-render/json-render-react
hermes skills install skills-sh/vercel-labs/json-render/json-render-react --force
```
#### 3. Well-known skill endpoints (`well-known`)
This is URL-based discovery from sites that publish `/.well-known/skills/index.json`. It is not a single centralized hub — it is a web discovery convention.
- Example live endpoint: [Mintlify docs skills index](https://mintlify.com/docs/.well-known/skills/index.json)
- Reference server implementation: [vercel-labs/skills-handler](https://github.com/vercel-labs/skills-handler)
- Example:
```bash
hermes skills search https://mintlify.com/docs --source well-known
hermes skills inspect well-known:https://mintlify.com/docs/.well-known/skills/mintlify
hermes skills install well-known:https://mintlify.com/docs/.well-known/skills/mintlify
```
#### 4. Direct GitHub skills (`github`)
Hermes can install directly from GitHub repositories and GitHub-based taps. This is useful when you already know the repo/path or want to add your own custom source repo.
Default taps (browsable without any setup):
- [openai/skills](https://github.com/openai/skills)
- [anthropics/skills](https://github.com/anthropics/skills)
- [huggingface/skills](https://github.com/huggingface/skills)
- [NVIDIA/skills](https://github.com/NVIDIA/skills) — NVIDIA-verified skills (signed `skill.oms.sig` + governance `skill-card.md`)
- [garrytan/gstack](https://github.com/garrytan/gstack)
- Example:
```bash
hermes skills install openai/skills/k8s
hermes skills tap add myorg/skills-repo
```
**Category groupings (`skills.sh.json`).** A GitHub tap may ship a
`skills.sh.json` file at its repo root following the
[skills.sh schema](https://skills.sh/schemas/skills.sh.schema.json). Its
`groupings` (each with a `title` and a list of skill names) are read at index
time and become the category labels shown in the
[Skills Hub](https://hermes-agent.nousresearch.com/docs) page — instead of a
tag-derived guess. This is generic: any tap that ships the file gets real
categorization, no Hermes-side changes required.
```json
{
"$schema": "https://skills.sh/schemas/skills.sh.schema.json",
"groupings": [
{ "title": "Inference AI", "skills": ["dynamo-recipe-runner", "dynamo-router-sla"] },
{ "title": "Decision Optimization", "skills": ["cuopt-developer", "cuopt-install"] }
]
}
```
#### 5. ClawHub (`clawhub`)
A third-party skills marketplace integrated as a community source.
- Site: [clawhub.ai](https://clawhub.ai/)
- Hermes source id: `clawhub`
#### 6. Claude marketplace-style repos (`claude-marketplace`)
Hermes supports marketplace repos that publish Claude-compatible plugin/marketplace manifests.
Known integrated sources include:
- [anthropics/skills](https://github.com/anthropics/skills)
- [aiskillstore/marketplace](https://github.com/aiskillstore/marketplace)
Hermes source id: `claude-marketplace`
#### 7. LobeHub (`lobehub`)
Hermes can search and convert agent entries from LobeHub's public catalog into installable Hermes skills.
- Site: [LobeHub](https://lobehub.com/)
- Public agents index: [chat-agents.lobehub.com](https://chat-agents.lobehub.com/)
- Backing repo: [lobehub/lobe-chat-agents](https://github.com/lobehub/lobe-chat-agents)
- Hermes source id: `lobehub`
#### 8. browse.sh (`browse-sh`)
Hermes integrates with [browse.sh](https://browse.sh), Browserbase's catalog of 200+ site-specific browser-automation SKILL.md files (Airbnb, Amazon, arXiv, 12306.cn, Etsy, Xero, and many more). Each skill describes how to drive one website end-to-end and is suitable for use with Hermes' browser tools and any browser-automation skills you already have installed.
- Site: [browse.sh](https://browse.sh/)
- Catalog API: `https://browse.sh/api/skills`
- Hermes source id: `browse-sh`
- Trust level: `community`
```bash
hermes skills search airbnb --source browse-sh
hermes skills inspect browse-sh/airbnb.com/search-listings-ddgioa
hermes skills install browse-sh/airbnb.com/search-listings-ddgioa
```
Identifiers use the form `browse-sh/<hostname>/<task-id>` and match the slug exposed by the browse.sh catalog. Content is resolved through the per-skill detail endpoint (`/api/skills/<slug>` → `skillMdUrl`), not through the catalog's GitHub `sourceUrl`.
#### 9. Direct URL (`url`)
Install a single-file `SKILL.md` directly from any HTTP(S) URL — useful when an author hosts a skill on their own site (no hub listing, no GitHub path to type). Hermes fetches the URL, parses the YAML frontmatter, security-scans it, and installs.
- Hermes source id: `url`
- Identifier: the URL itself (no prefix needed)
- Scope: **single-file `SKILL.md`** only. Multi-file skills with `references/` or `scripts/` need a manifest and should be published via one of the other sources above.
```bash
hermes skills install https://sharethis.chat/SKILL.md
hermes skills install https://example.com/my-skill/SKILL.md --category productivity
```
Name resolution, in order:
1. `name:` field in the SKILL.md YAML frontmatter (recommended — every well-formed skill has one).
2. Parent directory name from the URL path (e.g. `.../my-skill/SKILL.md` → `my-skill`, or `.../my-skill.md` → `my-skill`), when it's a valid identifier (`^[a-z][a-z0-9_-]*$`).
3. Interactive prompt on a terminal with a TTY.
4. On non-interactive surfaces (the `/skills install` slash command inside the TUI, gateway platforms, scripts), a clean error pointing at the `--name` override.
```bash
# Frontmatter has no name and the URL slug is unhelpful — supply one:
hermes skills install https://example.com/SKILL.md --name sharethis-chat
# Or inside a chat session:
/skills install https://example.com/SKILL.md --name sharethis-chat
```
Trust level is always `community` — the same security scan runs as for every other source. The URL is stored as the install identifier, so `hermes skills update` re-fetches from the same URL automatically when you want to refresh.
### Security scanning and `--force`
All hub-installed skills go through a **security scanner** that checks for data exfiltration, prompt injection, destructive commands, supply-chain signals, and other threats.
`hermes skills inspect ...` now also surfaces upstream metadata when available:
- repo URL
- skills.sh detail page URL
- install command
- weekly installs
- upstream security audit statuses
- well-known index/endpoint URLs
Use `--force` when you have reviewed a third-party skill and want to override a non-dangerous policy block:
```bash
hermes skills install skills-sh/anthropics/skills/pdf --force
```
Important behavior:
- `--force` can override policy blocks for caution/warn-style findings.
- `--force` does **not** override a `dangerous` scan verdict.
- Official optional skills (`official/...`) are treated as built-in trust and do not show the third-party warning panel.
### Trust levels
| Level | Source | Policy |
|-------|--------|--------|
| `builtin` | Ships with Hermes | Always trusted |
| `official` | `optional-skills/` in the repo | Built-in trust, no third-party warning |
| `trusted` | Trusted registries/repos such as `openai/skills`, `anthropics/skills`, `huggingface/skills`, `NVIDIA/skills` | More permissive policy than community sources |
| `community` | Everything else (`skills.sh`, well-known endpoints, custom GitHub repos, most marketplaces) | Non-dangerous findings can be overridden with `--force`; `dangerous` verdicts stay blocked |
### Update lifecycle
The hub now tracks enough provenance to re-check upstream copies of installed skills:
```bash
hermes skills check # Report which installed hub skills changed upstream
hermes skills update # Reinstall only the skills with updates available
hermes skills update react # Update one specific installed hub skill
```
This uses the stored source identifier plus the current upstream bundle content hash to detect drift.
:::tip GitHub rate limits
Skills hub operations use the GitHub API, which has a rate limit of 60 requests/hour for unauthenticated users. If you see rate-limit errors during install or search, set `GITHUB_TOKEN` in your `.env` file to increase the limit to 5,000 requests/hour. The error message includes an actionable hint when this happens.
:::
### Publishing a custom skill tap
If you want to share a curated set of skills — for your team, your org, or publicly — you can publish them as a **tap**: a GitHub repository other Hermes users add with `hermes skills tap add <owner/repo>`. No server, no registry sign-up, no release pipeline. Just a directory of `SKILL.md` files.
#### Repo layout
A tap is any GitHub repo (public or private — private needs `GITHUB_TOKEN`) laid out like this:
```
owner/repo
├── skills/ # default path; configurable per-tap
│ ├── my-workflow/
│ │ ├── SKILL.md # required
│ │ ├── references/ # optional supporting files
│ │ ├── templates/
│ │ └── scripts/
│ ├── another-skill/
│ │ └── SKILL.md
│ └── third-skill/
│ └── SKILL.md
└── README.md # optional but helpful
```
Rules:
- Each skill lives in its own directory under the tap's root path (default `skills/`).
- The directory name becomes the skill's install slug.
- Each skill directory must contain a `SKILL.md` with standard [SKILL.md frontmatter](#skillmd-format) (`name`, `description`, plus optional `metadata.hermes.tags`, `version`, `author`, `platforms`, `metadata.hermes.config`).
- Subdirectories like `references/`, `templates/`, `scripts/`, `assets/` are downloaded alongside `SKILL.md` at install time.
- Skills whose directory name starts with `.` or `_` are ignored.
Hermes discovers skills by listing every subdirectory of the tap path and probing each for `SKILL.md`.
#### Minimal tap example
```
my-org/hermes-skills
└── skills/
└── deploy-runbook/
└── SKILL.md
```
`skills/deploy-runbook/SKILL.md`:
```markdown
---
name: deploy-runbook
description: Our deployment runbook — services, rollback, Slack channels
version: 1.0.0
author: My Org Platform Team
metadata:
hermes:
tags: [deployment, runbook, internal]
---
# Deploy Runbook
Step 1: ...
```
After pushing that to GitHub, any Hermes user can subscribe and install:
```bash
hermes skills tap add my-org/hermes-skills
hermes skills search deploy
hermes skills install my-org/hermes-skills/deploy-runbook
```
#### Non-default paths
If your skills don't live under `skills/` (common when you're adding a `skills/` subtree to an existing project), edit the tap entry in `~/.hermes/.hub/taps.json`:
```json
{
"taps": [
{"repo": "my-org/platform-docs", "path": "internal/skills/"}
]
}
```
The `hermes skills tap add` CLI defaults new taps to `path: "skills/"`; edit the file directly if you need a different path. `hermes skills tap list` shows the effective path per tap.
#### Installing individual skills directly (without adding a tap)
Users can also install a single skill from any public GitHub repo without adding the whole repo as a tap:
```bash
hermes skills install owner/repo/skills/my-workflow
```
Useful when you want to share one skill without asking the user to subscribe to your whole registry.
#### Trust levels for taps
New taps are assigned `community` trust by default. Skills installed from them run through the standard security scan and show the third-party warning panel on first install. If your org or a widely-trusted source should get higher trust, add its repo to `TRUSTED_REPOS` in `tools/skills_hub.py` (requires a Hermes core PR).
#### Tap management
```bash
hermes skills tap list # show all configured taps
hermes skills tap add myorg/skills-repo # add (default path: skills/)
hermes skills tap remove myorg/skills-repo # remove
```
Inside a running session:
```
/skills tap list
/skills tap add myorg/skills-repo
/skills tap remove myorg/skills-repo
```
Taps are stored in `~/.hermes/.hub/taps.json` (created on demand).
## Bundled skill updates (`hermes skills reset`)
Hermes ships with a set of bundled skills in `skills/` inside the repo. On install and on every `hermes update`, a sync pass copies those into `~/.hermes/skills/` and records a manifest at `~/.hermes/skills/.bundled_manifest` mapping each skill name to the content hash at the time it was synced (the **origin hash**).
On each sync, Hermes recomputes the hash of your local copy and compares it to the origin hash:
- **Unchanged** → safe to pull upstream changes, copy the new bundled version in, record the new origin hash.
- **Changed** → treated as **user-modified** and skipped forever, so your edits never get stomped.
The protection is good, but it has one sharp edge. If you edit a bundled skill and then later want to abandon your changes and go back to the bundled version by just copy-pasting from `~/.hermes/hermes-agent/skills/`, the manifest still holds the *old* origin hash from whenever the last successful sync ran. Your fresh copy-paste contents (current bundled hash) won't match that stale origin hash, so sync keeps flagging it as user-modified.
`hermes skills reset` is the escape hatch:
```bash
# Safe: clears the manifest entry for this skill. Your current copy is preserved,
# but the next sync re-baselines against it so future updates work normally.
hermes skills reset google-workspace
# Full restore: also deletes your local copy and re-copies the current bundled
# version. Use this when you want the pristine upstream skill back.
hermes skills reset google-workspace --restore
# Non-interactive (e.g. in scripts or TUI mode) — skip the --restore confirmation.
hermes skills reset google-workspace --restore --yes
```
The same command works in chat as a slash command:
```text
/skills reset google-workspace
/skills reset google-workspace --restore
```
:::note Profiles
Each profile has its own `.bundled_manifest` under its own `HERMES_HOME`, so `hermes -p coder skills reset <name>` only affects that profile.
:::
### Slash commands (inside chat)
All the same commands work with `/skills`:
```text
/skills browse
/skills search react --source skills-sh
/skills search https://mintlify.com/docs --source well-known
/skills inspect skills-sh/vercel-labs/json-render/json-render-react
/skills install openai/skills/skill-creator --force
/skills check
/skills update
/skills reset google-workspace
/skills list
```
Official optional skills still use identifiers like `official/security/1password` and `official/migration/openclaw-migration`.
website/docs/user-guide/features/skins.md
+271
View File
@@ -0,0 +1,271 @@
---
sidebar_position: 10
title: "Skins & Themes"
description: "Customize the Hermes CLI with built-in and user-defined skins"
---
# Skins & Themes
Skins control the **visual presentation** of the Hermes CLI: banner colors, spinner faces and verbs, response-box labels, branding text, and the tool activity prefix.
Conversational style and visual style are separate concepts:
- **Personality** changes the agent's tone and wording.
- **Skin** changes the CLI's appearance.
## Change skins
```bash
/skin # show the current skin and list available skins
/skin ares # switch to a built-in skin
/skin mytheme # switch to a custom skin from ~/.hermes/skins/mytheme.yaml
```
Or set the default skin in `~/.hermes/config.yaml`:
```yaml
display:
skin: default
```
## Built-in skins
| Skin | Description | Agent branding | Visual character |
|------|-------------|----------------|------------------|
| `default` | Classic Hermes — gold and kawaii | `Hermes Agent` | Warm gold borders, cornsilk text, kawaii faces in spinners. The familiar caduceus banner. Clean and inviting. |
| `ares` | War-god theme — crimson and bronze | `Ares Agent` | Deep crimson borders with bronze accents. Aggressive spinner verbs ("forging", "marching", "tempering steel"). Custom sword-and-shield ASCII art banner. |
| `mono` | Monochrome — clean grayscale | `Hermes Agent` | All grays — no color. Borders are `#555555`, text is `#c9d1d9`. Ideal for minimal terminal setups or screen recordings. |
| `slate` | Cool blue — developer-focused | `Hermes Agent` | Royal blue borders (`#4169e1`), soft blue text. Calm and professional. No custom spinner — uses default faces. |
| `daylight` | Light theme for bright terminals with dark text and cool blue accents | `Hermes Agent` | Designed for white or bright terminals. Dark slate text with blue borders, pale status surfaces, and a light completion menu that stays readable in light terminal profiles. |
| `warm-lightmode` | Warm brown/gold text for light terminal backgrounds | `Hermes Agent` | Warm parchment tones for light terminals. Dark brown text with saddle-brown accents, cream-colored status surfaces. An earthy alternative to the cooler daylight theme. |
| `poseidon` | Ocean-god theme — deep blue and seafoam | `Poseidon Agent` | Deep blue to seafoam gradient. Ocean-themed spinners ("charting currents", "sounding the depth"). Trident ASCII art banner. |
| `sisyphus` | Sisyphean theme — austere grayscale with persistence | `Sisyphus Agent` | Light grays with stark contrast. Boulder-themed spinners ("pushing uphill", "resetting the boulder", "enduring the loop"). Boulder-and-hill ASCII art banner. |
| `charizard` | Volcanic theme — burnt orange and ember | `Charizard Agent` | Warm burnt orange to ember gradient. Fire-themed spinners ("banking into the draft", "measuring burn"). Dragon-silhouette ASCII art banner. |
## Complete list of configurable keys
### Colors (`colors:`)
Controls all color values throughout the CLI. Values are hex color strings.
| Key | Description | Default (`default` skin) |
|-----|-------------|--------------------------|
| `banner_border` | Panel border around the startup banner | `#CD7F32` (bronze) |
| `banner_title` | Title text color in the banner | `#FFD700` (gold) |
| `banner_accent` | Section headers in the banner (Available Tools, etc.) | `#FFBF00` (amber) |
| `banner_dim` | Muted text in the banner (separators, secondary labels) | `#B8860B` (dark goldenrod) |
| `banner_text` | Body text in the banner (tool names, skill names) | `#FFF8DC` (cornsilk) |
| `ui_accent` | General UI accent color (highlights, active elements) | `#FFBF00` |
| `ui_label` | UI labels and tags | `#4dd0e1` (teal) |
| `ui_ok` | Success indicators (checkmarks, completion) | `#4caf50` (green) |
| `ui_error` | Error indicators (failures, blocked) | `#ef5350` (red) |
| `ui_warn` | Warning indicators (caution, approval prompts) | `#ffa726` (orange) |
| `prompt` | Interactive prompt text color | `#FFF8DC` |
| `input_rule` | Horizontal rule above the input area | `#CD7F32` |
| `response_border` | Border around the agent's response box (ANSI escape) | `#FFD700` |
| `session_label` | Session label color | `#DAA520` |
| `session_border` | Session ID dim border color | `#8B8682` |
| `status_bar_bg` | Background color for the TUI status / usage bar | `#1a1a2e` |
| `voice_status_bg` | Background color for the voice-mode status badge | `#1a1a2e` |
| `selection_bg` | Background color for the TUI mouse-selection highlighter. Falls back to `completion_menu_current_bg` when unset. | `#333355` |
| `completion_menu_bg` | Background color for the completion menu list | `#1a1a2e` |
| `completion_menu_current_bg` | Background color for the active completion row | `#333355` |
| `completion_menu_meta_bg` | Background color for the completion meta column | `#1a1a2e` |
| `completion_menu_meta_current_bg` | Background color for the active completion meta column | `#333355` |
### Spinner (`spinner:`)
Controls the animated spinner shown while waiting for API responses.
| Key | Type | Description | Example |
|-----|------|-------------|---------|
| `waiting_faces` | list of strings | Faces cycled while waiting for API response | `["(⚔)", "(⛨)", "(▲)"]` |
| `thinking_faces` | list of strings | Faces cycled during model reasoning | `["(⚔)", "(⌁)", "(<>)"]` |
| `thinking_verbs` | list of strings | Verbs shown in spinner messages | `["forging", "plotting", "hammering plans"]` |
| `wings` | list of [left, right] pairs | Decorative brackets around the spinner | `[["⟪⚔", "⚔⟫"], ["⟪▲", "▲⟫"]]` |
When spinner values are empty (like in `default` and `mono`), hardcoded defaults from `display.py` are used.
### Branding (`branding:`)
Text strings used throughout the CLI interface.
| Key | Description | Default |
|-----|-------------|---------|
| `agent_name` | Name shown in banner title and status display | `Hermes Agent` |
| `welcome` | Welcome message shown at CLI startup | `Welcome to Hermes Agent! Type your message or /help for commands.` |
| `goodbye` | Message shown on exit | `Goodbye! ⚕` |
| `response_label` | Label on the response box header | ` ⚕ Hermes ` |
| `prompt_symbol` | Symbol before the user input prompt (bare token, renderers add a trailing space) | `❯` |
| `help_header` | Header text for the `/help` command output | `(^_^)? Available Commands` |
### Other top-level keys
| Key | Type | Description | Default |
|-----|------|-------------|---------|
| `tool_prefix` | string | Character prefixed to tool output lines in the CLI | `┊` |
| `tool_emojis` | dict | Per-tool emoji overrides for spinners and progress (`{tool_name: emoji}`) | `{}` |
| `banner_logo` | string | Rich-markup ASCII art logo (replaces the default HERMES_AGENT banner) | `""` |
| `banner_hero` | string | Rich-markup hero art (replaces the default caduceus art) | `""` |
## Custom skins
Create YAML files under `~/.hermes/skins/`. User skins inherit missing values from the built-in `default` skin, so you only need to specify the keys you want to change.
### Full custom skin YAML template
```yaml
# ~/.hermes/skins/mytheme.yaml
# Complete skin template — all keys shown. Delete any you don't need;
# missing values automatically inherit from the 'default' skin.
name: mytheme
description: My custom theme
colors:
banner_border: "#CD7F32"
banner_title: "#FFD700"
banner_accent: "#FFBF00"
banner_dim: "#B8860B"
banner_text: "#FFF8DC"
ui_accent: "#FFBF00"
ui_label: "#4dd0e1"
ui_ok: "#4caf50"
ui_error: "#ef5350"
ui_warn: "#ffa726"
prompt: "#FFF8DC"
input_rule: "#CD7F32"
response_border: "#FFD700"
session_label: "#DAA520"
session_border: "#8B8682"
status_bar_bg: "#1a1a2e"
voice_status_bg: "#1a1a2e"
selection_bg: "#333355"
completion_menu_bg: "#1a1a2e"
completion_menu_current_bg: "#333355"
completion_menu_meta_bg: "#1a1a2e"
completion_menu_meta_current_bg: "#333355"
spinner:
waiting_faces:
- "(⚔)"
- "(⛨)"
- "(▲)"
thinking_faces:
- "(⚔)"
- "(⌁)"
- "(<>)"
thinking_verbs:
- "processing"
- "analyzing"
- "computing"
- "evaluating"
wings:
- ["⟪⚡", "⚡⟫"]
- ["⟪●", "●⟫"]
branding:
agent_name: "My Agent"
welcome: "Welcome to My Agent! Type your message or /help for commands."
goodbye: "See you later! ⚡"
response_label: " ⚡ My Agent "
prompt_symbol: "⚡"
help_header: "(⚡) Available Commands"
tool_prefix: "┊"
# Per-tool emoji overrides (optional)
tool_emojis:
terminal: "⚔"
web_search: "🔮"
read_file: "📄"
# Custom ASCII art banners (optional, Rich markup supported)
# banner_logo: |
# [bold #FFD700] MY AGENT [/]
# banner_hero: |
# [#FFD700] Custom art here [/]
```
### Minimal custom skin example
Since everything inherits from `default`, a minimal skin only needs to change what's different:
```yaml
name: cyberpunk
description: Neon terminal theme
colors:
banner_border: "#FF00FF"
banner_title: "#00FFFF"
banner_accent: "#FF1493"
spinner:
thinking_verbs: ["jacking in", "decrypting", "uploading"]
wings:
- ["⟨⚡", "⚡⟩"]
branding:
agent_name: "Cyber Agent"
response_label: " ⚡ Cyber "
tool_prefix: "▏"
```
## Hermes Mod — Visual Skin Editor
[Hermes Mod](https://github.com/cocktailpeanut/hermes-mod) is a community-built web UI for creating and managing skins visually. Instead of writing YAML by hand, you get a point-and-click editor with live preview.
![Hermes Mod skin editor](https://raw.githubusercontent.com/cocktailpeanut/hermes-mod/master/nous.png)
**What it does:**
- Lists all built-in and custom skins
- Opens any skin into a visual editor with all Hermes skin fields (colors, spinner, branding, tool prefix, tool emojis)
- Generates `banner_logo` text art from a text prompt
- Converts uploaded images (PNG, JPG, GIF, WEBP) into `banner_hero` ASCII art with multiple render styles (braille, ASCII ramp, blocks, dots)
- Saves directly to `~/.hermes/skins/`
- Activates a skin by updating `~/.hermes/config.yaml`
- Shows the generated YAML and a live preview
### Install
**Option 1 — Pinokio (1-click):**
Find it on [pinokio.computer](https://pinokio.computer) and install with one click.
**Option 2 — npx (quickest from terminal):**
```bash
npx -y hermes-mod
```
**Option 3 — Manual:**
```bash
git clone https://github.com/cocktailpeanut/hermes-mod.git
cd hermes-mod/app
npm install
npm start
```
### Usage
1. Start the app (via Pinokio or terminal).
2. Open **Skin Studio**.
3. Choose a built-in or custom skin to edit.
4. Generate a logo from text and/or upload an image for hero art. Pick a render style and width.
5. Edit colors, spinner, branding, and other fields.
6. Click **Save** to write the skin YAML to `~/.hermes/skins/`.
7. Click **Activate** to set it as the current skin (updates `display.skin` in `config.yaml`).
Hermes Mod respects the `HERMES_HOME` environment variable, so it works with [profiles](/user-guide/profiles) too.
## Operational notes
- Built-in skins load from `hermes_cli/skin_engine.py`.
- Unknown skins automatically fall back to `default`.
- `/skin` updates the active CLI theme immediately for the current session.
- User skins in `~/.hermes/skins/` take precedence over built-in skins with the same name.
- Skin changes via `/skin` are session-only. To make a skin your permanent default, set it in `config.yaml`.
- The `banner_logo` and `banner_hero` fields support Rich console markup (e.g., `[bold #FF0000]text[/]`) for colored ASCII art.
website/docs/user-guide/features/spotify.md
+279
View File
@@ -0,0 +1,279 @@
# Spotify
Hermes can control Spotify directly — playback, queue, search, playlists, saved tracks/albums, and listening history — using Spotify's official Web API with PKCE OAuth. Tokens are stored in `~/.hermes/auth.json` and refreshed automatically on 401; you only log in once per machine.
Unlike Hermes' built-in OAuth integrations (Google, GitHub Copilot, Codex), Spotify requires every user to register their own lightweight developer app. Spotify does not let third parties ship a public OAuth app that anyone can use. It takes about two minutes and `hermes auth spotify` walks you through it.
## Prerequisites
- A Spotify account. **Free** works for search, playlist, library, and activity tools. **Premium** is required for playback control (play, pause, skip, seek, volume, queue add, transfer).
- Hermes Agent installed and running.
- For playback tools: an **active Spotify Connect device** — the Spotify app must be open on at least one device (phone, desktop, web player, speaker) so the Web API has something to control. If nothing is active you'll get a `403 Forbidden` with a "no active device" message; open Spotify on any device and retry.
## Setup
### One-shot: `hermes tools` or first-run setup
The fastest path. Run:
```bash
hermes tools
```
Scroll to `🎵 Spotify`, press space to toggle it on, then `s` to save. The same toggle is also available during the first-run `hermes setup` / `hermes setup tools` flow. Spotify stays opt-in, so enabling it there runs the same provider-aware configuration as `hermes tools`.
Hermes drops you straight into the OAuth flow — if you don't have a Spotify app yet, it walks you through creating one inline. Once you finish, the toolset is enabled AND authenticated in one pass.
If you prefer to do the steps separately (or you're re-authing later), use the two-step flow below.
### Two-step flow
#### 1. Enable the toolset
```bash
hermes tools
```
Toggle `🎵 Spotify` on, save, and when the inline wizard opens, dismiss it (Ctrl+C). The toolset stays on; only the auth step is deferred.
#### 2. Run the login wizard
```bash
hermes auth spotify
```
The 7 Spotify tools only appear in the agent's toolset after step 1 — they're off by default so users who don't want them don't ship extra tool schemas on every API call.
If no `HERMES_SPOTIFY_CLIENT_ID` is set, Hermes walks you through the app registration inline:
1. Opens `https://developer.spotify.com/dashboard` in your browser
2. Prints the exact values to paste into Spotify's "Create app" form
3. Prompts you for the Client ID you get back
4. Saves it to `~/.hermes/.env` so future runs skip this step
5. Continues straight into the OAuth consent flow
After you approve, tokens are written under `providers.spotify` in `~/.hermes/auth.json`. The active inference provider is NOT changed — Spotify auth is independent of your LLM provider.
### Creating the Spotify app (what the wizard asks for)
When the dashboard opens, click **Create app** and fill in:
| Field | Value |
|-------|-------|
| App name | anything (e.g. `hermes-agent`) |
| App description | anything (e.g. `personal Hermes integration`) |
| Website | leave blank |
| Redirect URI | `http://127.0.0.1:43827/spotify/callback` |
| Which API/SDKs? | check **Web API** |
Agree to the terms and click **Save**. On the next page click **Settings** → copy the **Client ID** and paste it into the Hermes prompt. That's the only value Hermes needs — PKCE doesn't use a client secret.
### Running over SSH / in a headless environment
If `SSH_CLIENT` or `SSH_TTY` is set, Hermes skips the automatic browser open during both the wizard and the OAuth step. Copy the dashboard URL and the authorization URL Hermes prints, open them in a browser on your local machine, and proceed normally — the local HTTP listener still runs on the remote host on port `43827`. Your laptop's browser can't reach the remote loopback without an SSH local-forward:
```bash
ssh -N -L 43827:127.0.0.1:43827 user@remote-host
```
For jump-box / bastion setups and other gotchas (mosh, tmux, port conflicts), see [OAuth over SSH / Remote Hosts](../../guides/oauth-over-ssh.md).
## Verify
```bash
hermes auth status spotify
```
Shows whether tokens are present and when the access token expires. Refresh is automatic: when any Spotify API call returns 401, the client exchanges the refresh token and retries once. Refresh tokens persist across Hermes restarts, so you only re-auth if you revoke the app in your Spotify account settings or run `hermes auth logout spotify`.
## Using it
Once logged in, the agent has access to 7 Spotify tools. You talk to the agent naturally — it picks the right tool and action. For the best behavior, the agent loads a companion skill that teaches canonical usage patterns (single-search-then-play, when not to preflight `get_state`, etc.).
```
> play some miles davis
> what am I listening to
> add this track to my Late Night Jazz playlist
> skip to the next song
> make a new playlist called "Focus 2026" and add the last three songs I played
> which of my saved albums are by Radiohead
> search for acoustic covers of Blackbird
> transfer playback to my kitchen speaker
```
### Tool reference
All playback-mutating actions accept an optional `device_id` to target a specific device. If omitted, Spotify uses the currently active device.
#### `spotify_playback`
Control and inspect playback, plus fetch recently played history.
| Action | Purpose | Premium? |
|--------|---------|----------|
| `get_state` | Full playback state (track, device, progress, shuffle/repeat) | No |
| `get_currently_playing` | Just the current track (returns empty on 204 — see below) | No |
| `play` | Start/resume playback. Optional: `context_uri`, `uris`, `offset`, `position_ms` | Yes |
| `pause` | Pause playback | Yes |
| `next` / `previous` | Skip track | Yes |
| `seek` | Jump to `position_ms` | Yes |
| `set_repeat` | `state` = `track` / `context` / `off` | Yes |
| `set_shuffle` | `state` = `true` / `false` | Yes |
| `set_volume` | `volume_percent` = 0-100 | Yes |
| `recently_played` | Last played tracks. Optional `limit`, `before`, `after` (Unix ms) | No |
#### `spotify_devices`
| Action | Purpose |
|--------|---------|
| `list` | Every Spotify Connect device visible to your account |
| `transfer` | Move playback to `device_id`. Optional `play: true` starts playback on transfer |
### Home Assistant-managed speakers
If Home Assistant manages speakers that already support Spotify Connect (for example Sonos, Echo, Nest, or other Connect-capable speakers), they appear in `spotify_devices list` automatically whenever Spotify can see them. Hermes does not need a Home Assistant ↔ Spotify bridge for this path — Spotify handles the device routing natively.
Ask Hermes to transfer playback by the speaker's display name (for example, “transfer Spotify to the kitchen speaker”), or call `spotify_devices list` and pass the exact `device_id` to `spotify_devices transfer` when scripting. If the speaker is missing, open the Spotify app or the speaker's Spotify integration once so Spotify registers it as an active Connect target.
#### `spotify_queue`
| Action | Purpose | Premium? |
|--------|---------|----------|
| `get` | Currently queued tracks | No |
| `add` | Append `uri` to the queue | Yes |
#### `spotify_search`
Search the catalog. `query` is required. Optional: `types` (array of `track` / `album` / `artist` / `playlist` / `show` / `episode`), `limit`, `offset`, `market`.
#### `spotify_playlists`
| Action | Purpose | Required args |
|--------|---------|---------------|
| `list` | User's playlists | — |
| `get` | One playlist + tracks | `playlist_id` |
| `create` | New playlist | `name` (+ optional `description`, `public`, `collaborative`) |
| `add_items` | Add tracks | `playlist_id`, `uris` (optional `position`) |
| `remove_items` | Remove tracks | `playlist_id`, `uris` (+ optional `snapshot_id`) |
| `update_details` | Rename / edit | `playlist_id` + any of `name`, `description`, `public`, `collaborative` |
#### `spotify_albums`
| Action | Purpose | Required args |
|--------|---------|---------------|
| `get` | Album metadata | `album_id` |
| `tracks` | Album track list | `album_id` |
#### `spotify_library`
Unified access to saved tracks and saved albums. Pick the collection with the `kind` arg.
| Action | Purpose |
|--------|---------|
| `list` | Paginated library listing |
| `save` | Add `ids` / `uris` to library |
| `remove` | Remove `ids` / `uris` from library |
Required: `kind` = `tracks` or `albums`, plus `action`.
### Feature matrix: Free vs Premium
Read-only tools work on Free accounts. Anything that mutates playback or the queue requires Premium.
| Works on Free | Premium required |
|---------------|------------------|
| `spotify_search` (all) | `spotify_playback` — play, pause, next, previous, seek, set_repeat, set_shuffle, set_volume |
| `spotify_playback` — get_state, get_currently_playing, recently_played | `spotify_queue` — add |
| `spotify_devices` — list | `spotify_devices` — transfer |
| `spotify_queue` — get | |
| `spotify_playlists` (all) | |
| `spotify_albums` (all) | |
| `spotify_library` (all) | |
## Scheduling: Spotify + cron
Because Spotify tools are regular Hermes tools, a cron job running in a Hermes session can trigger playback on any schedule. No new code needed.
### Morning wake-up playlist
```bash
hermes cron add \
--name "morning-commute" \
"0 7 * * 1-5" \
"Transfer playback to my kitchen speaker and start my 'Morning Commute' playlist. Volume to 40. Shuffle on."
```
What happens at 7am every weekday:
1. Cron spins up a headless Hermes session.
2. Agent reads the prompt, calls `spotify_devices list` to find "kitchen speaker" by name, then `spotify_devices transfer` → `spotify_playback set_volume` → `spotify_playback set_shuffle` → `spotify_search` + `spotify_playback play`.
3. Music starts on the target speaker. Total cost: one session, a few tool calls, no human input.
### Wind-down at night
```bash
hermes cron add \
--name "wind-down" \
"30 22 * * *" \
"Pause Spotify. Then set volume to 20 so it's quiet when I start it again tomorrow."
```
### Gotchas
- **An active device must exist when the cron fires.** If no Spotify client is running (phone/desktop/Connect speaker), playback actions return `403 no active device`. For morning playlists, the trick is to target a device that's always on (Sonos, Echo, a smart speaker) rather than your phone.
- **Premium required for anything that mutates playback** — play, pause, skip, volume, transfer. Read-only cron jobs (scheduled "email me my recently played tracks") work fine on Free.
- **The cron agent inherits your active toolsets.** Spotify must be enabled in `hermes tools` for the cron session to see the Spotify tools.
- **Cron jobs run with `skip_memory=True`** so they don't write to your memory store.
Full cron reference: [Cron Jobs](./cron).
## Sign out
```bash
hermes auth logout spotify
```
Removes tokens from `~/.hermes/auth.json`. To also clear the app config, delete `HERMES_SPOTIFY_CLIENT_ID` (and `HERMES_SPOTIFY_REDIRECT_URI` if you set it) from `~/.hermes/.env`, or run the wizard again.
To revoke the app on Spotify's side, visit [Apps connected to your account](https://www.spotify.com/account/apps/) and click **REMOVE ACCESS**.
## Troubleshooting
**`403 Forbidden — Player command failed: No active device found`** — You need Spotify running on at least one device. Open the Spotify app on your phone, desktop, or web player, start any track for a second to register it, and retry. `spotify_devices list` shows what's currently visible.
**`403 Forbidden — Premium required`** — You're on a Free account trying to use a playback-mutating action. See the feature matrix above.
**`204 No Content` on `get_currently_playing`** — nothing is currently playing on any device. This is Spotify's normal response, not an error; Hermes surfaces it as an explanatory empty result (`is_playing: false`).
**`INVALID_CLIENT: Invalid redirect URI`** — the redirect URI in your Spotify app settings doesn't match what Hermes is using. The default is `http://127.0.0.1:43827/spotify/callback`. Either add that to your app's allowed redirect URIs, or set `HERMES_SPOTIFY_REDIRECT_URI` in `~/.hermes/.env` to whatever you registered.
**`429 Too Many Requests`** — Spotify's rate limit. Hermes returns a friendly error; wait a minute and retry. If this persists, you're probably running a tight loop in a script — Spotify's quota resets roughly every 30 seconds.
**`401 Unauthorized` keeps coming back** — Your refresh token was revoked (usually because you removed the app from your account, or the app was deleted). Run `hermes auth spotify` again.
**Wizard doesn't open the browser** — If you're over SSH or in a container without a display, Hermes detects it and skips the auto-open. Copy the dashboard URL it prints and open it manually.
## Advanced: custom scopes
By default Hermes requests the scopes needed for every shipped tool. Override if you want to restrict access:
```bash
hermes auth spotify --scope "user-read-playback-state user-modify-playback-state playlist-read-private"
```
Scope reference: [Spotify Web API scopes](https://developer.spotify.com/documentation/web-api/concepts/scopes). If you request fewer scopes than a tool needs, that tool's calls will fail with 403.
## Advanced: custom client ID / redirect URI
```bash
hermes auth spotify --client-id <id> --redirect-uri http://localhost:3000/callback
```
Or set them permanently in `~/.hermes/.env`:
```
HERMES_SPOTIFY_CLIENT_ID=<your_id>
HERMES_SPOTIFY_REDIRECT_URI=http://localhost:3000/callback
```
The redirect URI must be allow-listed in your Spotify app's settings. The default works for almost everyone — only change it if port 43827 is taken.
## Where things live
| File | Contents |
|------|----------|
| `~/.hermes/auth.json` → `providers.spotify` | access token, refresh token, expiry, scope, redirect URI |
| `~/.hermes/.env` | `HERMES_SPOTIFY_CLIENT_ID`, optional `HERMES_SPOTIFY_REDIRECT_URI` |
| Spotify app | owned by you at [developer.spotify.com/dashboard](https://developer.spotify.com/dashboard); contains the Client ID and the redirect URI allow-list |
website/docs/user-guide/features/subscription-proxy.md
+203
View File
@@ -0,0 +1,203 @@
---
sidebar_position: 15
title: "Subscription Proxy"
description: "Use your Nous Portal subscription (or other OAuth provider) as an OpenAI-compatible endpoint for external apps"
---
# Subscription Proxy
The subscription proxy is a local HTTP server that lets external apps —
OpenViking, Karakeep, Open WebUI, anything that speaks OpenAI-compatible
chat completions — use your Hermes-managed provider subscription as their
LLM endpoint. The proxy attaches the right credentials (refreshing them
automatically) so the app never needs a static API key.
This is different from the [API server](./api-server.md):
| | API server | Subscription proxy |
|---|---|---|
| What it serves | Your agent (full toolset, memory, skills) | Raw model inference |
| Use case | "Use Hermes as a chat backend" | "Use my Portal sub from another app" |
| Auth | Your `API_SERVER_KEY` | Any bearer (proxy attaches the real one) |
| Tool calls | Yes — the agent runs tools | No — passthrough only |
Use the API server when you want the **agent** as a backend. Use the
proxy when you just want **the model** through your subscription.
## Quick Start
### 1. Log into your provider (one-time)
```bash
hermes portal
```
This opens your browser for the Nous Portal OAuth flow. Hermes stores
the refresh token in `~/.hermes/auth.json` — the same place all Hermes
provider logins live.
### 2. Start the proxy
```bash
hermes proxy start
```
```
Starting Hermes proxy for Nous Portal
Listening on: http://127.0.0.1:8645/v1
Forwarding to: (resolved per-request from your subscription)
Use any bearer token in the client — the proxy attaches your real credential.
```
Leave this running in the foreground. Use `tmux`, `nohup`, or a systemd
unit if you want it to survive logout.
### 3. Point your app at it
Any OpenAI-compatible app config takes the same triple:
```
Base URL: http://127.0.0.1:8645/v1
API key: anything (e.g. "sk-unused")
Model: Hermes-4-70B # or Hermes-4.3-36B, Hermes-4-405B
```
The proxy ignores the `Authorization` header from your app and attaches
your real Portal credential to the upstream request. Refreshes happen
automatically when the bearer approaches expiry.
## Available providers
```bash
hermes proxy providers
```
Currently shipped: `nous` (Nous Portal) and `xai` (xAI / Grok). More
OAuth providers can be added by implementing the `UpstreamAdapter`
interface in `hermes_cli/proxy/adapters/`.
## Check status
```bash
hermes proxy status
```
```
Hermes proxy upstream adapters
[nous ] Nous Portal — ready (bearer expires 2026-05-15T06:43:21Z)
```
If you see `not logged in`, run `hermes portal`. If you see
`credentials need attention`, your refresh token was revoked (rare —
happens if you signed out from the Portal web UI) — just re-run
`hermes portal`.
## Allowed paths
The proxy only forwards paths the upstream actually serves. For Nous
Portal:
| Path | Purpose |
|------|---------|
| `/v1/chat/completions` | Chat completions (streaming + non-streaming) |
| `/v1/completions` | Legacy text completions |
| `/v1/embeddings` | Embeddings |
| `/v1/models` | Model list |
Other paths (`/v1/images/generations`, `/v1/audio/speech`, etc.) return
404 with a clear error pointing at the allowed paths. This keeps stray
clients from leaking weird requests to the upstream.
## Configuring OpenViking to use Portal
[OpenViking](https://github.com/volcengine/OpenViking) is a context
database that needs an LLM provider for its VLM (vision/language model
used to extract memories) and embedding model. With the proxy, you can
point its `vlm.api_base` at your local proxy:
Edit `~/.openviking/ov.conf`:
```json
{
"vlm": {
"provider": "openai",
"model": "Hermes-4-70B",
"api_base": "http://127.0.0.1:8645/v1",
"api_key": "unused-proxy-attaches-real-creds"
}
}
```
Then start your proxy in a terminal alongside `openviking-server`:
```bash
# Terminal 1
hermes proxy start
# Terminal 2
openviking-server
```
OpenViking's VLM calls now flow through your Portal subscription. The
embedding model side still needs its own provider — Portal does serve
`/v1/embeddings` but the model selection depends on what your tier
supports; check `portal.nousresearch.com/models`.
## Configuring Karakeep (or any bookmark/summarizer app)
[Karakeep](https://karakeep.app/) takes an OpenAI-compatible API for
bookmark summarization. In its config:
```bash
# Karakeep .env
OPENAI_API_BASE_URL=http://127.0.0.1:8645/v1
OPENAI_API_KEY=any-non-empty-string
INFERENCE_TEXT_MODEL=Hermes-4-70B
```
Same pattern works for Open WebUI, LobeChat, NextChat, or any other
OpenAI-compatible client.
## Exposing on LAN
By default the proxy binds `127.0.0.1` (localhost only). To let other
machines on your network use it:
```bash
hermes proxy start --host 0.0.0.0 --port 8645
```
⚠ **Be aware:** anyone on your network can now use your Portal
subscription. The proxy has no auth of its own — it accepts any bearer.
Use a firewall, VPN, or reverse proxy with proper auth if you expose
this beyond your trusted network.
## Rate limits
Your Portal tier's RPM/TPM limits apply across the whole proxy. The
proxy doesn't fan out or pool — it's a single bearer with your full
subscription quota. Monitor usage at
[portal.nousresearch.com](https://portal.nousresearch.com).
## Architecture
The proxy is intentionally minimal. Per request:
1. Receive `POST /v1/chat/completions` from your app
2. Look up the adapter's current credential (refresh if expiring)
3. Forward the request body verbatim, with `Authorization: Bearer <minted-key>`
4. Stream the response back unchanged (SSE preserved)
No transformation. No logging of request bodies. No agent loop. The
proxy is a credential-attaching pass-through.
## Future: more OAuth providers
The adapter system is pluggable. Adding a new provider (e.g.
HuggingFace, GitHub Copilot's chat endpoint, Anthropic via OAuth)
requires implementing `UpstreamAdapter` in
`hermes_cli/proxy/adapters/<provider>.py` and registering it in
`adapters/__init__.py`. Providers that aren't OpenAI-compatible at the
protocol level (Anthropic Messages API, for example) would need a
transformation layer, which is out of scope for the current shape.
website/docs/user-guide/features/tool-gateway.md
+192
View File
@@ -0,0 +1,192 @@
---
title: "Nous Tool Gateway"
description: "One subscription, every tool. Web search, image generation, TTS, and cloud browsers — all routed through Nous Portal with no extra API keys."
sidebar_label: "Tool Gateway"
sidebar_position: 2
---
# Nous Tool Gateway
**One subscription. Every tool built in.**
The Tool Gateway is included with every paid [Nous Portal](https://portal.nousresearch.com) subscription. It routes Hermes' tool calls — web search, image generation, text-to-speech, and cloud browser automation — through infrastructure Nous already runs, so you don't have to sign up with Firecrawl, FAL, OpenAI, Browser Use, or anyone else just to make your agent useful.
<div style={{display: 'flex', gap: '1rem', flexWrap: 'wrap', margin: '1.5rem 0'}}>
<a href="https://portal.nousresearch.com/manage-subscription" style={{background: 'var(--ifm-color-primary)', color: 'white', padding: '0.75rem 1.5rem', borderRadius: '6px', textDecoration: 'none', fontWeight: 'bold'}}>Start or manage subscription →</a>
</div>
## What's included
| | Tool | What you get |
|---|---|---|
| 🔍 | **Web search & extract** | Agent-grade web search and full-page extraction via Firecrawl. No rate limits to worry about — the gateway handles scaling. |
| 🎨 | **Image generation** | Nine models under one endpoint: **FLUX 2 Klein 9B**, **FLUX 2 Pro**, **Z-Image Turbo**, **Nano Banana Pro** (Gemini 3 Pro Image), **GPT Image 1.5**, **GPT Image 2**, **Ideogram V3**, **Recraft V4 Pro**, **Qwen Image**. Pick per-generation with a flag, or let Hermes default to FLUX 2 Klein. |
| 🔊 | **Text-to-speech** | OpenAI TTS voices wired into the `text_to_speech` tool. Drop voice notes into Telegram, generate audio for pipelines, narrate anything. |
| 🌐 | **Cloud browser automation** | Headless Chromium sessions via Browser Use. `browser_navigate`, `browser_click`, `browser_type`, `browser_vision` — all the agent-driving primitives, no Browserbase account required. |
All four are pay-as-you-use billed against your Nous subscription. Use any combination — run the gateway for web and images while keeping your own ElevenLabs key for TTS, or route everything through Nous.
## Why it's here
Building an agent that can actually *do things* means stitching together 5+ API subscriptions — each with their own signup, rate limits, billing, and quirks. The gateway collapses that into one account:
- **One bill.** Pay Nous; we handle the rest.
- **One signup.** No Firecrawl, FAL, Browser Use, or OpenAI audio accounts to manage.
- **One key.** Your Nous Portal OAuth covers every tool.
- **Same quality.** Same backends the direct-key route uses — just fronted by us.
Bring your own keys anytime — per-tool, whenever you want to. The gateway isn't a lock-in, it's a shortcut.
## Get started
There are three ways in — pick whichever fits where you are:
```bash
hermes setup --portal # Fresh install: Nous OAuth + set Nous as provider + turn on the Tool Gateway in one go
```
```bash
hermes model # Switch your inference provider to Nous Portal — Hermes then offers to turn on the gateway for all tools
```
```bash
hermes tools # Enable the gateway per-tool — pick "Nous Subscription" for any tool you want
```
`hermes setup --portal` and `hermes model` are the all-at-once paths: log in once, optionally flip every tool to the gateway. `hermes tools` is the à la carte path — turn on just the tools you want, one at a time.
**You don't have to log in first.** With `hermes tools`, the Nous-managed backends (Web search, Image, Video, TTS, Browser) are always listed, even if you've never signed into Nous Portal. Select one and Hermes runs the Portal login right there if you aren't already authenticated — no need to run `hermes model` beforehand. If your Nous OAuth is already active, selecting the backend enables it immediately with no extra prompt. This path only logs you in and turns on the one tool you picked — it does **not** switch your inference provider, and it does **not** prompt you to enable the gateway for every other tool.
Check what's active at any time:
```bash
hermes portal info # Portal auth + Tool Gateway routing summary
hermes portal tools # Gateway catalog with current routing per tool
hermes status # Full system status (Tool Gateway is one section)
```
`hermes portal info` shows a section like:
```
◆ Nous Tool Gateway
Nous Portal ✓ managed tools available
Web tools ✓ active via Nous subscription
Image gen ✓ active via Nous subscription
TTS ✓ active via Nous subscription
Browser ○ active via Browser Use key
```
Tools marked "active via Nous subscription" are going through the gateway. Anything else is using your own keys.
## Eligibility
The Tool Gateway is a **paid-subscription** feature. Free-tier Nous accounts can use Portal for inference but don't include managed tools — [upgrade your plan](https://portal.nousresearch.com/manage-subscription) to unlock the gateway.
Some accounts are also entitled to a **free tool pool** — a small managed-tool allowance that covers gateway tool calls without a paid subscription. When a free pool is available, the gateway surfaces it and shows a setup prompt on first use, so you can opt in and start using managed tools right away.
## Mix and match
The gateway is per-tool. Turn it on for just what you want:
- **All tools through Nous** — easiest; one subscription, done.
- **Gateway for web + images, bring your own TTS** — keep your ElevenLabs voice, let Nous handle the rest.
- **Gateway only for things you don't have keys for** — "I already pay for Browserbase, but I don't want a Firecrawl account" works fine.
Switch any tool at any time via:
```bash
hermes tools # Interactive picker for each tool category
```
Select the tool, pick **Nous Subscription** as the provider (or any direct provider you prefer). No config editing required. If you aren't logged into Nous Portal yet, picking **Nous Subscription** kicks off the Portal login inline — you don't need to authenticate through `hermes model` first.
## Using individual image models
Image generation defaults to FLUX 2 Klein 9B for speed. Override per-call by passing the model ID to the `image_generate` tool:
| Model | ID | Best for |
|---|---|---|
| FLUX 2 Klein 9B | `fal-ai/flux-2/klein/9b` | Fast, good default |
| FLUX 2 Pro | `fal-ai/flux-2-pro` | Higher fidelity FLUX |
| Z-Image Turbo | `fal-ai/z-image/turbo` | Stylized, fast |
| Nano Banana Pro | `fal-ai/nano-banana-pro` | Google Gemini 3 Pro Image |
| GPT Image 1.5 | `fal-ai/gpt-image-1.5` | OpenAI image gen, text+image |
| GPT Image 2 | `fal-ai/gpt-image-2` | OpenAI latest |
| Ideogram V3 | `fal-ai/ideogram/v3` | Strong prompt adherence + typography |
| Recraft V4 Pro | `fal-ai/recraft/v4/pro/text-to-image` | Vector-style, graphic design |
| Qwen Image | `fal-ai/qwen-image` | Alibaba multimodal |
The set evolves — `hermes tools` → Image Generation shows the current live list.
---
## Configuration reference
Most users never need to touch this — `hermes model` and `hermes tools` cover every workflow interactively. This section is for writing config.yaml directly or scripting setups.
### Per-tool `use_gateway` flag
Each tool's config block takes a `use_gateway` boolean:
```yaml
web:
backend: firecrawl
use_gateway: true
image_gen:
use_gateway: true
tts:
provider: openai
use_gateway: true
browser:
cloud_provider: browser-use
use_gateway: true
```
Precedence: `use_gateway: true` routes through Nous regardless of any direct keys in `.env`. `use_gateway: false` (or absent) uses direct keys if available and only falls back to the gateway when none exist.
### Disabling the gateway
```yaml
web:
use_gateway: false # Hermes now uses FIRECRAWL_API_KEY from .env
```
`hermes tools` automatically clears the flag when you pick a non-gateway provider, so this usually happens for you.
### Self-hosted gateway (advanced)
Running your own Nous-compatible gateway? Override endpoints in `~/.hermes/.env`:
```bash
TOOL_GATEWAY_DOMAIN=your-domain.example.com
TOOL_GATEWAY_SCHEME=https
TOOL_GATEWAY_USER_TOKEN=your-token # normally auto-populated from Portal login
FIRECRAWL_GATEWAY_URL=https://... # override one endpoint specifically
```
These knobs exist for custom infrastructure setups (enterprise deployments, dev environments). Regular subscribers never set them.
## FAQ
### Does it work with Telegram / Discord / the other messaging gateways?
Yes. Tool Gateway operates at the tool-execution layer, not the CLI. Every interface that can call a tool — CLI, Telegram, Discord, Slack, IRC, Teams, the API server, anything — benefits from it transparently.
### What happens if my subscription expires?
Tools routed through the gateway stop working until you renew or swap in direct API keys via `hermes tools`. Hermes shows a clear error pointing at the portal.
### Can I see usage or costs per tool?
Yes — the [Nous Portal dashboard](https://portal.nousresearch.com) breaks usage down by tool so you can see what's driving your bill.
### Is Modal (serverless terminal) included?
Modal is available as an **optional add-on** through the Nous subscription, not part of the default Tool Gateway bundle. Configure it via `hermes setup terminal` or directly in `config.yaml` when you want a remote sandbox for shell execution.
### Do I need to delete my existing API keys when I enable the gateway?
No — keep them in `.env`. When `use_gateway: true`, Hermes skips direct keys and uses the gateway. Flip the flag back to `false` and your keys become the source again. The gateway isn't a lock-in.
website/docs/user-guide/features/tool-search.md
+159
View File
@@ -0,0 +1,159 @@
---
title: Tool Search
sidebar_position: 95
---
# Tool Search
When you have many MCP servers or non-core plugin tools attached to a
session, their JSON schemas can consume a substantial fraction of the
context window on every turn — even when only a few of them are relevant
to what the user actually asked for.
**Tool Search** is Hermes' opt-in progressive-disclosure layer for that
problem. When activated, MCP and plugin tools are replaced in the
model-visible tools array by three bridge tools, and the model loads each
specific tool's schema on demand.
:::info Built-in Hermes tools never defer
The tools that make up Hermes' core capability set (`terminal`,
`read_file`, `write_file`, `patch`, `search_files`, `todo`, `memory`,
`browser_*`, `web_search`, `web_extract`, `clarify`, `execute_code`,
`delegate_task`, `session_search`, `send_message`, and the rest of
`_HERMES_CORE_TOOLS`) are *always* loaded directly. Only MCP tools and
non-core plugin tools are eligible for deferral.
:::
## How it works
When Tool Search activates for a turn, the model sees three new tools in
place of the deferred ones:
```
tool_search(query, limit?) — search the deferred-tool catalog
tool_describe(name) — load the full schema for one tool
tool_call(name, arguments) — invoke a deferred tool
```
A typical interaction looks like:
```
Model: tool_search("create a github issue")
→ { matches: [{ name: "mcp_github_create_issue", ... }, ...] }
Model: tool_describe("mcp_github_create_issue")
→ { parameters: { type: "object", properties: { ... } } }
Model: tool_call("mcp_github_create_issue", { title: "...", body: "..." })
→ { ok: true, issue_number: 42 }
```
When the model invokes `tool_call`, Hermes **unwraps the bridge** and
dispatches the underlying tool exactly as if the model had called it
directly. Pre-tool-call hooks, guardrails, approval prompts, and
post-tool-call hooks all run against the real tool name — not against
`tool_call`. The activity feed in the CLI and gateway also unwraps so you
see the underlying tool, not the bridge.
## When does it activate?
By default Tool Search runs in `auto` mode: it activates only when the
deferrable tool schemas would consume at least 10% of the active model's
context window. Below that, the tools-array assembly is a pure
pass-through and you pay no overhead.
This decision is re-evaluated every time the tools array is built, so:
- A session with just a few MCP tools and a long context model never
activates Tool Search.
- A session with many MCP servers attached (15+ tools typically) starts
activating it.
- Removing MCP servers mid-session correctly returns to direct exposure
on the next assembly.
## Configuration
```yaml
tools:
tool_search:
enabled: auto # auto (default), on, or off
threshold_pct: 10 # percentage of context — only used in auto mode
search_default_limit: 5
max_search_limit: 20
```
| Key | Default | Meaning |
| --- | --- | --- |
| `enabled` | `auto` | `auto` activates above threshold; `on` always activates if there's at least one deferrable tool; `off` disables entirely. |
| `threshold_pct` | `10` | Percentage of context length at which `auto` mode kicks in. Range 0–100. |
| `search_default_limit` | `5` | Hits returned when the model calls `tool_search` without a `limit`. |
| `max_search_limit` | `20` | Hard upper bound the model can request via `limit`. Range 1–50. |
You can also flip the legacy boolean shape:
```yaml
tools:
tool_search: true # equivalent to {enabled: auto}
```
## When NOT to use it
Tool Search trades a fixed per-turn token cost (the three bridge tool
schemas, ~300 tokens) and at least one extra round trip (search →
describe → call) for the savings on the deferred schemas. It's a clear
win when you have many tools and use few per turn; it's overhead when
you have few tools total.
The `auto` default handles this for you. If you set `enabled: on`
unconditionally, expect a slight per-turn cost on small toolsets.
## Trade-offs that don't go away
These come from the prompt-cache integrity invariant — they are inherent
to any progressive-disclosure design, not specific to this implementation:
- **One extra round trip on cold tools.** The first time the model needs
a deferred tool, it spends one or two extra model calls to find and
load the schema. The token savings on the static side are real, but a
portion is paid back at runtime.
- **No cache benefit on deferred schemas.** A loaded `tool_describe`
result enters the conversation history (so it does get cached on
subsequent turns) but it never benefits from the system-prompt cache
prefix.
- **Model-quality dependence.** Tool Search assumes the model can write a
reasonable search query for the tool it wants. Smaller models do this
less well; the published Anthropic numbers (49% → 74% on Opus 4 with
vs. without tool search) show the upside but also that ~26 points of
accuracy is still retrieval failure.
- **Toolset edits invalidate cache.** Adding or removing a tool mid-
session changes the bridge tools' descriptions (which include the
count of deferred tools) and the catalog, so the prompt cache is
invalidated. This is the same trade-off as any toolset edit.
## Implementation details
- **Retrieval:** BM25 over tokenized tool name + description + parameter
names. Falls back to a literal substring match on the tool name when
BM25 returns no positive-score hits, which protects against
zero-IDF degenerate cases (e.g. searching `"github"` against a
catalog where every tool name contains "github").
- **Catalog is stateless across turns.** It rebuilds from the current
tool-defs list every assembly — no session-keyed `Map`. This avoids
the class of bug where a stored catalog drifts out of sync with the
live tool registry.
- **The catalog is scoped to the session's toolsets.** `tool_search`,
`tool_describe`, and `tool_call` only ever see and invoke tools the
session was actually granted. A subagent, kanban worker, or gateway
session restricted to a subset of toolsets cannot use the bridge to
discover or call a tool outside that subset — the deferred catalog is
the deferrable slice of the session's own enabled/disabled toolsets,
not the whole process registry.
- **No JS sandbox.** Hermes uses the simpler "structured tools" mode
(search / describe / call as plain functions). The JS-sandbox "code
mode" some other implementations offer is a large surface area; we
skip it.
## See also
- `tools/tool_search.py` — the implementation
- `tests/tools/test_tool_search.py` — the regression suite
- The `openclaw-tool-search-report` PDF in the original implementation
PR for the research that shaped the design
website/docs/user-guide/features/tools.md
+178
View File
@@ -0,0 +1,178 @@
---
sidebar_position: 1
title: "Tools & Toolsets"
description: "Overview of Hermes Agent's tools — what's available, how toolsets work, and terminal backends"
---
# Tools & Toolsets
Tools are functions that extend the agent's capabilities. They're organized into logical **toolsets** that can be enabled or disabled per platform.
## Available Tools
Hermes ships with a broad built-in tool registry covering web search, browser automation, terminal execution, file editing, memory, delegation, RL training, messaging delivery, Home Assistant, and more.
:::note
**Honcho cross-session memory** is available as a memory provider plugin (`plugins/memory/honcho/`), not as a built-in toolset. See [Plugins](./plugins.md) for installation.
:::
High-level categories:
| Category | Examples | Description |
|----------|----------|-------------|
| **Web** | `web_search`, `web_extract` | Search the web and extract page content. |
| **X Search** | `x_search` | Search X (Twitter) posts and threads via xAI's built-in `x_search` Responses tool — gated on xAI credentials (SuperGrok OAuth or `XAI_API_KEY`); off by default, opt in via `hermes tools` → 🐦 X (Twitter) Search. |
| **Terminal & Files** | `terminal`, `process`, `read_file`, `patch` | Execute commands and manipulate files. |
| **Browser** | `browser_navigate`, `browser_snapshot`, `browser_vision` | Interactive browser automation with text and vision support. |
| **Media** | `vision_analyze`, `image_generate`, `text_to_speech` | Multimodal analysis and generation. |
| **Agent orchestration** | `todo`, `clarify`, `execute_code`, `delegate_task` | Planning, clarification, code execution, and subagent delegation. |
| **Memory & recall** | `memory`, `session_search` | Persistent memory and session search. |
| **Automation & delivery** | `cronjob`, `send_message` | Scheduled tasks with create/list/update/pause/resume/run/remove actions, plus outbound messaging delivery. |
| **Integrations** | `ha_*`, MCP server tools | Home Assistant, MCP, and other integrations. |
For the authoritative code-derived registry, see [Built-in Tools Reference](/reference/tools-reference) and [Toolsets Reference](/reference/toolsets-reference).
:::tip Nous Tool Gateway
Paid [Nous Portal](https://portal.nousresearch.com) subscribers can use web search, image generation, TTS, and browser automation through the **[Tool Gateway](tool-gateway.md)** — no separate API keys needed. Run `hermes model` to enable it, or configure individual tools with `hermes tools`.
:::
## Using Toolsets
```bash
# Use specific toolsets
hermes chat --toolsets "web,terminal"
# See all available tools
hermes tools
# Configure tools per platform (interactive)
hermes tools
```
Common toolsets include `web`, `search`, `terminal`, `file`, `browser`, `vision`, `image_gen`, `moa`, `skills`, `tts`, `todo`, `memory`, `session_search`, `cronjob`, `code_execution`, `delegation`, `clarify`, `homeassistant`, `messaging`, `spotify`, `discord`, `discord_admin`, `debugging`, and `safe`.
See [Toolsets Reference](/reference/toolsets-reference) for the full set, including platform presets such as `hermes-cli`, `hermes-telegram`, and dynamic MCP toolsets like `mcp-<server>`.
## Terminal Backends
The terminal tool can execute commands in different environments:
| Backend | Description | Use Case |
|---------|-------------|----------|
| `local` | Run on your machine (default) | Development, trusted tasks |
| `docker` | Isolated containers | Security, reproducibility |
| `ssh` | Remote server | Sandboxing, keep agent away from its own code |
| `singularity` | HPC containers | Cluster computing, rootless |
| `modal` | Cloud execution | Serverless, scale |
| `daytona` | Cloud sandbox workspace | Persistent remote dev environments |
### Configuration
```yaml
# In ~/.hermes/config.yaml
terminal:
backend: local # or: docker, ssh, singularity, modal, daytona
cwd: "." # Working directory
timeout: 180 # Command timeout in seconds
```
### Docker Backend
```yaml
terminal:
backend: docker
docker_image: python:3.11-slim
```
**One persistent container, shared across the whole process.** Hermes starts a single long-lived container on first use (`docker run -d ... sleep 2h`) and routes every terminal, file, and `execute_code` call through `docker exec` into that same container. Working-directory changes, installed packages, environment tweaks, and files written to `/workspace` all carry over from one tool call to the next, across `/new`, `/reset`, and `delegate_task` subagents, for the lifetime of the Hermes process. The container is stopped and removed on shutdown.
This means the Docker backend behaves like a persistent sandbox VM, not a fresh container per command. If you `pip install foo` once, it's there for the rest of the session. If you `cd /workspace/project`, subsequent `ls` calls see that directory. See [Configuration → Docker Backend](../configuration.md#docker-backend) for the full lifecycle details and the `container_persistent` flag that controls whether `/workspace` and `/root` survive across Hermes restarts.
### SSH Backend
Recommended for security — agent can't modify its own code:
```yaml
terminal:
backend: ssh
```
```bash
# Set credentials in ~/.hermes/.env
TERMINAL_SSH_HOST=my-server.example.com
TERMINAL_SSH_USER=myuser
TERMINAL_SSH_KEY=~/.ssh/id_rsa
```
### Singularity/Apptainer
```bash
# Pre-build SIF for parallel workers
apptainer build ~/python.sif docker://python:3.11-slim
# Configure
hermes config set terminal.backend singularity
hermes config set terminal.singularity_image ~/python.sif
```
### Modal (Serverless Cloud)
```bash
uv pip install modal
modal setup
hermes config set terminal.backend modal
```
### Container Resources
Configure CPU, memory, disk, and persistence for all container backends:
```yaml
terminal:
backend: docker # or singularity, modal, daytona
container_cpu: 1 # CPU cores (default: 1)
container_memory: 5120 # Memory in MB (default: 5GB)
container_disk: 51200 # Disk in MB (default: 50GB)
container_persistent: true # Persist filesystem across sessions (default: true)
```
When `container_persistent: true`, installed packages, files, and config survive across sessions.
### Container Security
All container backends run with security hardening:
- Read-only root filesystem (Docker)
- All Linux capabilities dropped
- No privilege escalation
- PID limits (256 processes)
- Full namespace isolation
- Persistent workspace via volumes, not writable root layer
Docker can optionally receive an explicit env allowlist via `terminal.docker_forward_env`, but forwarded variables are visible to commands inside the container and should be treated as exposed to that session.
## Background Process Management
Start background processes and manage them:
```python
terminal(command="pytest -v tests/", background=true)
# Returns: {"session_id": "proc_abc123", "pid": 12345}
# Then manage with the process tool:
process(action="list") # Show all running processes
process(action="poll", session_id="proc_abc123") # Check status
process(action="wait", session_id="proc_abc123") # Block until done
process(action="log", session_id="proc_abc123") # Full output
process(action="kill", session_id="proc_abc123") # Terminate
process(action="write", session_id="proc_abc123", data="y") # Send input
```
PTY mode (`pty=true`) enables interactive CLI tools like Codex and Claude Code.
## Sudo Support
If a command needs sudo, you'll be prompted for your password (cached for the session). Or set `SUDO_PASSWORD` in `~/.hermes/.env`.
:::warning
On messaging platforms, if sudo fails, the output includes a tip to add `SUDO_PASSWORD` to `~/.hermes/.env`.
:::
website/docs/user-guide/features/tts.md
+672
View File
@@ -0,0 +1,672 @@
---
sidebar_position: 9
title: "Voice & TTS"
description: "Text-to-speech and voice message transcription across all platforms"
---
# Voice & TTS
Hermes Agent supports both text-to-speech output and voice message transcription across all messaging platforms.
:::tip Nous Subscribers
If you have a paid [Nous Portal](https://portal.nousresearch.com) subscription, OpenAI TTS is available through the **[Tool Gateway](tool-gateway.md)** without a separate OpenAI API key. New installs can run `hermes setup --portal` to log in and turn on every gateway tool at once; existing installs can pick **Nous Subscription** for just TTS via `hermes model` or `hermes tools`.
:::
## Text-to-Speech
Convert text to speech with ten providers:
| Provider | Quality | Cost | API Key |
|----------|---------|------|---------|
| **Edge TTS** (default) | Good | Free | None needed |
| **ElevenLabs** | Excellent | Paid | `ELEVENLABS_API_KEY` |
| **OpenAI TTS** | Good | Paid | `VOICE_TOOLS_OPENAI_KEY` |
| **MiniMax TTS** | Excellent | Paid | `MINIMAX_API_KEY` |
| **Mistral (Voxtral TTS)** | Excellent | Paid | `MISTRAL_API_KEY` |
| **Google Gemini TTS** | Excellent | Free tier | `GEMINI_API_KEY` |
| **xAI TTS** | Excellent | Paid | `XAI_API_KEY` |
| **NeuTTS** | Good | Free (local) | None needed |
| **KittenTTS** | Good | Free (local) | None needed |
| **Piper** | Good | Free (local) | None needed |
### Platform Delivery
| Platform | Delivery | Format |
|----------|----------|--------|
| Telegram | Voice bubble (plays inline) | Opus `.ogg` |
| Discord | Voice bubble (Opus/OGG), falls back to file attachment | Opus/MP3 |
| WhatsApp | Audio file attachment | MP3 |
| CLI | Saved to `~/.hermes/audio_cache/` | MP3 |
### Configuration
```yaml
# In ~/.hermes/config.yaml
tts:
provider: "edge" # "edge" | "elevenlabs" | "openai" | "minimax" | "mistral" | "gemini" | "xai" | "neutts" | "kittentts" | "piper"
speed: 1.0 # Global speed multiplier (provider-specific settings override this)
edge:
voice: "en-US-AriaNeural" # 322 voices, 74 languages
speed: 1.0 # Converted to rate percentage (+/-%)
elevenlabs:
voice_id: "pNInz6obpgDQGcFmaJgB" # Adam
model_id: "eleven_multilingual_v2"
openai:
model: "gpt-4o-mini-tts"
voice: "alloy" # alloy, echo, fable, onyx, nova, shimmer
base_url: "https://api.openai.com/v1" # Override for OpenAI-compatible TTS endpoints
speed: 1.0 # 0.25 - 4.0
minimax:
model: "speech-2.8-hd" # speech-2.8-hd (default), speech-2.8-turbo
voice_id: "English_Graceful_Lady" # See https://platform.minimax.io/faq/system-voice-id
speed: 1 # 0.5 - 2.0
vol: 1 # 0 - 10
pitch: 0 # -12 - 12
mistral:
model: "voxtral-mini-tts-2603"
voice_id: "c69964a6-ab8b-4f8a-9465-ec0925096ec8" # Paul - Neutral (default)
gemini:
model: "gemini-2.5-flash-preview-tts" # or gemini-3.1-flash-tts-preview
voice: "Kore" # 30 prebuilt voices: Zephyr, Puck, Kore, Enceladus, Gacrux, etc.
audio_tags: false # Enable hidden Gemini 3.1 TTS audio-tag insertion
persona_prompt_file: "" # Optional Markdown/text file with Gemini voice direction
xai:
voice_id: "eve" # or a custom voice ID — see docs below
language: "en" # ISO 639-1 code
sample_rate: 24000 # 22050 / 24000 (default) / 44100 / 48000
bit_rate: 128000 # MP3 bitrate; only applies when codec=mp3
# base_url: "https://api.x.ai/v1" # Override via XAI_BASE_URL env var
neutts:
ref_audio: ''
ref_text: ''
model: neuphonic/neutts-air-q4-gguf
device: cpu
kittentts:
model: KittenML/kitten-tts-nano-0.8-int8 # 25MB int8; also: kitten-tts-micro-0.8 (41MB), kitten-tts-mini-0.8 (80MB)
voice: Jasper # Jasper, Bella, Luna, Bruno, Rosie, Hugo, Kiki, Leo
speed: 1.0 # 0.5 - 2.0
clean_text: true # Expand numbers, currencies, units
piper:
voice: en_US-lessac-medium # voice name (auto-downloaded) OR absolute path to .onnx
# voices_dir: '' # default: ~/.hermes/cache/piper-voices/
# use_cuda: false # requires onnxruntime-gpu
# length_scale: 1.0 # 2.0 = twice as slow
# noise_scale: 0.667
# noise_w_scale: 0.8
# volume: 1.0 # 0.5 = half as loud
# normalize_audio: true
```
**Speed control**: The global `tts.speed` value applies to all providers by default. Each provider can override it with its own `speed` setting (e.g., `tts.openai.speed: 1.5`). Provider-specific speed takes precedence over the global value. Default is `1.0` (normal speed).
### Gemini Persona Prompts
Gemini TTS can follow natural-language performance direction. Set `tts.gemini.persona_prompt_file` to a local Markdown or text file that describes the voice persona. The file can include Gemini-style sections such as `AUDIO PROFILE`, `SCENE`, `DIRECTOR'S NOTES`, `SAMPLE CONTEXT`, and `TRANSCRIPT`.
If the file contains `{transcript}` or `{{ transcript }}`, Hermes replaces that placeholder with the live TTS text. Otherwise, Hermes appends a labeled `TRANSCRIPT` section automatically. The persona prompt stays local and is not shown in the chat reply.
```yaml
tts:
provider: gemini
gemini:
voice: Algieba
persona_prompt_file: ~/.hermes/tts/butler-voice.md
```
### Gemini Audio Tags
Gemini 3.1 Flash TTS supports freeform square-bracket audio tags such as `[whispers]`, `[excitedly]`, `[very slow]`, `[laughs]`, and other expressive delivery notes. Enable `tts.gemini.audio_tags` to have Hermes run a hidden rewrite pass before Gemini TTS. The rewrite inserts inline tags into the TTS script only; the visible chat reply stays unchanged.
```yaml
tts:
provider: gemini
gemini:
model: gemini-3.1-flash-tts-preview
audio_tags: true
```
The rewrite uses `auxiliary.tts_audio_tags` and defaults to your main chat model. Override that auxiliary task if you want tag insertion handled by a cheaper or faster model.
### Input length limits
Each provider has a documented per-request input-character cap. Hermes truncates text before calling the provider so requests never fail with a length error:
| Provider | Default cap (chars) |
|----------|---------------------|
| Edge TTS | 5000 |
| OpenAI | 4096 |
| xAI | 15000 |
| MiniMax | 10000 |
| Mistral | 4000 |
| Google Gemini | 32000 |
| ElevenLabs | Model-aware (see below) |
| NeuTTS | 2000 |
| KittenTTS | 2000 |
| Piper | 5000 |
**ElevenLabs** picks a cap from the configured `model_id`:
| `model_id` | Cap (chars) |
|------------|-------------|
| `eleven_flash_v2_5` | 40000 |
| `eleven_flash_v2` | 30000 |
| `eleven_multilingual_v2` (default), `eleven_multilingual_v1`, `eleven_english_sts_v2`, `eleven_english_sts_v1` | 10000 |
| `eleven_v3`, `eleven_ttv_v3` | 5000 |
| Unknown model | Falls back to provider default (10000) |
**Override per provider** with `max_text_length:` under the provider section of your TTS config:
```yaml
tts:
openai:
max_text_length: 8192 # raise or lower the provider cap
```
Only positive integers are honored. Zero, negative, non-numeric, or boolean values fall through to the provider default, so a broken config can't accidentally disable truncation.
### Telegram Voice Bubbles & ffmpeg
Telegram voice bubbles require Opus/OGG audio format:
- **OpenAI, ElevenLabs, and Mistral** produce Opus natively — no extra setup
- **Edge TTS** (default) outputs MP3 and needs **ffmpeg** to convert:
- **MiniMax TTS** outputs MP3 and needs **ffmpeg** to convert for Telegram voice bubbles
- **Google Gemini TTS** outputs raw PCM and uses **ffmpeg** to encode Opus directly for Telegram voice bubbles
- **xAI TTS** outputs MP3 and needs **ffmpeg** to convert for Telegram voice bubbles
- **NeuTTS** outputs WAV and also needs **ffmpeg** to convert for Telegram voice bubbles
- **KittenTTS** outputs WAV and also needs **ffmpeg** to convert for Telegram voice bubbles
- **Piper** outputs WAV and also needs **ffmpeg** to convert for Telegram voice bubbles
```bash
# Ubuntu/Debian
sudo apt install ffmpeg
# macOS
brew install ffmpeg
# Fedora
sudo dnf install ffmpeg
```
Without ffmpeg, Edge TTS, MiniMax TTS, NeuTTS, KittenTTS, and Piper audio are sent as regular audio files (playable, but shown as a rectangular player instead of a voice bubble).
:::tip
If you want voice bubbles without installing ffmpeg, switch to the OpenAI, ElevenLabs, or Mistral provider.
:::
### xAI Custom Voices (voice cloning)
xAI supports cloning your voice and using it with TTS. Create a custom voice in the [xAI Console](https://console.x.ai/team/default/voice/voice-library), then set the resulting `voice_id` in your config:
```yaml
tts:
provider: xai
xai:
voice_id: "nlbqfwie" # your custom voice ID
```
See the [xAI Custom Voices docs](https://docs.x.ai/developers/model-capabilities/audio/custom-voices) for details on recording, supported formats, and limits.
### Piper (local, 44 languages)
Piper is a fast, local neural TTS engine from the Open Home Foundation (the Home Assistant maintainers). It runs entirely on CPU, supports **44 languages** with pre-trained voices, and needs no API key.
**Install via `hermes tools`** → Voice & TTS → Piper — Hermes runs `pip install piper-tts` for you. Or install manually: `pip install piper-tts`.
**Switch to Piper:**
```yaml
tts:
provider: piper
piper:
voice: en_US-lessac-medium
```
On the first TTS call for a voice that isn't cached locally, Hermes runs `python -m piper.download_voices <name>` and downloads the model (~20-90MB depending on quality tier) into `~/.hermes/cache/piper-voices/`. Subsequent calls reuse the cached model.
**Picking a voice.** The [full voice catalog](https://github.com/OHF-Voice/piper1-gpl/blob/main/docs/VOICES.md) covers English, Spanish, French, German, Italian, Dutch, Portuguese, Russian, Polish, Turkish, Chinese, Arabic, Hindi, and more — each with `x_low` / `low` / `medium` / `high` quality tiers. Sample voices at [rhasspy.github.io/piper-samples](https://rhasspy.github.io/piper-samples/).
**Using a pre-downloaded voice.** Set `tts.piper.voice` to an absolute path ending in `.onnx`:
```yaml
tts:
piper:
voice: /path/to/my-custom-voice.onnx
```
**Advanced knobs** (`tts.piper.length_scale` / `noise_scale` / `noise_w_scale` / `volume` / `normalize_audio`, `use_cuda`) correspond 1:1 to Piper's `SynthesisConfig`. They're ignored on older `piper-tts` versions.
### Custom command providers
If a TTS engine you want isn't natively supported (VoxCPM, MLX-Kokoro, XTTS CLI, a voice-cloning script, anything else that exposes a CLI), you can wire it in as a **command-type provider** without writing any Python. Hermes writes the input text to a temp UTF-8 file, runs your shell command, and reads the audio file the command produced.
Declare one or more providers under `tts.providers.<name>` and switch between them with `tts.provider: <name>` — the same way you switch between built-ins like `edge` and `openai`.
```yaml
tts:
provider: voxcpm # pick any name under tts.providers
providers:
voxcpm:
type: command
command: "voxcpm --ref ~/voice.wav --text-file {input_path} --out {output_path}"
output_format: mp3
timeout: 180
voice_compatible: true # try to deliver as a Telegram voice bubble
mlx-kokoro:
type: command
command: "python -m mlx_kokoro --in {input_path} --out {output_path} --voice {voice}"
voice: af_sky
output_format: wav
piper-custom: # native Piper also supports custom .onnx via tts.piper.voice
type: command
command: "piper -m /path/to/custom.onnx -f {output_path} < {input_path}"
output_format: wav
```
#### Example: Doubao (Chinese seed-tts-2.0)
For high-quality Chinese TTS via ByteDance's [seed-tts-2.0](https://www.volcengine.com/docs/6561/1257544) bidirectional-streaming API, install the [`doubao-speech`](https://pypi.org/project/doubao-speech/) PyPI package and wire it in as a command provider:
```bash
pip install doubao-speech
export VOLCENGINE_APP_ID="your-app-id"
export VOLCENGINE_ACCESS_TOKEN="your-access-token"
```
```yaml
tts:
provider: doubao
providers:
doubao:
type: command
command: "doubao-speech say --text-file {input_path} --out {output_path}"
output_format: mp3
max_text_length: 1024
timeout: 30
```
Credentials come from your shell environment (`VOLCENGINE_APP_ID` / `VOLCENGINE_ACCESS_TOKEN`) or `~/.doubao-speech/config.yaml`. Pick a voice by adding `--voice zh-female-warm` (or any other alias from `doubao-speech list-voices`) to the command. `doubao-speech` also bundles streaming ASR — see the [STT section below](#example-doubao--volcengine-asr) for Hermes integration. Source and full docs: [github.com/Hypnus-Yuan/doubao-speech](https://github.com/Hypnus-Yuan/doubao-speech).
#### Placeholders
Your command template can reference these placeholders. Hermes substitutes them at render time and shell-quotes each value for the surrounding context (bare / single-quoted / double-quoted), so paths with spaces and other shell-sensitive characters are safe.
| Placeholder | Meaning |
|------------------|------------------------------------------------------|
| `{input_path}` | Path to the temp UTF-8 text file Hermes wrote |
| `{text_path}` | Alias for `{input_path}` |
| `{output_path}` | Path the command must write audio to |
| `{format}` | `mp3` / `wav` / `ogg` / `flac` |
| `{voice}` | `tts.providers.<name>.voice`, empty when unset |
| `{model}` | `tts.providers.<name>.model` |
| `{speed}` | Resolved speed multiplier (provider or global) |
Use `{{` and `}}` for literal braces.
#### Optional keys
| Key | Default | Meaning |
|--------------------|---------|------------------------------------------------------------------------------------------------------------|
| `timeout` | `120` | Seconds; the process tree is killed on expiry (Unix `killpg`, Windows `taskkill /T`). |
| `output_format` | `mp3` | One of `mp3` / `wav` / `ogg` / `flac`. Auto-inferred from the output extension if Hermes picks a path. |
| `voice_compatible` | `false` | When `true`, Hermes converts MP3/WAV output to Opus/OGG via ffmpeg so Telegram renders a voice bubble. |
| `max_text_length` | `5000` | Input is truncated to this length before rendering the command. |
| `voice` / `model` | empty | Passed to the command as placeholder values only. |
#### Behavior notes
- **Built-in names always win.** A `tts.providers.openai` entry never shadows the native OpenAI provider, so no user config can silently replace a built-in.
- **Default delivery is a document.** Command providers deliver as regular audio attachments on every platform. Opt in to voice-bubble delivery per-provider with `voice_compatible: true`.
- **Command failures surface to the agent.** Non-zero exit, empty output, or timeout all return an error with the command's stderr/stdout included so you can debug the provider from the conversation.
- **`type: command` is the default when `command:` is set.** Writing `type: command` explicitly is good practice but not required; an entry with a non-empty `command` string is treated as a command provider.
- **`{input_path}` / `{text_path}` are interchangeable.** Use whichever reads better in your command.
#### Security
Command-type providers run whatever shell command you configure, with your user's permissions. Hermes quotes placeholder values and enforces the configured timeout, but the command template itself is trusted local input — treat it the same way you would a shell script on your PATH.
### Python plugin providers
For TTS engines that can't be expressed as a single shell command — Python SDKs without a CLI, streaming engines, voice-listing APIs, OAuth-refreshing auth — register a Python plugin via `ctx.register_tts_provider()`. The plugin **coexists with** (does not replace) the [Custom command providers](#custom-command-providers) registry; pick the surface that fits your engine.
#### When to pick which
| Your backend has… | Use |
|---|---|
| A single CLI reading text from a file/stdin and writing audio to a file/stdout | **Command provider** (no Python needed) |
| Two or three CLIs chained with shell pipes | **Command provider** |
| A Python SDK only — no CLI | **Plugin** |
| Streaming bytes you want to deliver chunked (mid-generation voice bubbles) | **Plugin** (override `stream()`) |
| A voice-listing API used by `hermes setup` | **Plugin** (override `list_voices()`) |
| OAuth refresh flow (not a static bearer token) | **Plugin** |
Built-ins always win, and command providers win over a same-name plugin — so plugins are safe to register against any non-built-in name without worrying about shadowing your existing config.
#### Minimal plugin
Drop this in `~/.hermes/plugins/my-tts/`:
`plugin.yaml`:
```yaml
name: my-tts
version: 0.1.0
description: "My custom Python TTS backend"
```
`__init__.py`:
```python
from agent.tts_provider import TTSProvider
class MyTTSProvider(TTSProvider):
@property
def name(self) -> str:
return "my-tts" # what tts.provider matches against
@property
def display_name(self) -> str:
return "My Custom TTS"
def is_available(self) -> bool:
# Return False when credentials/deps are missing — picker skips
# this row but the dispatcher still routes here on explicit config.
import os
return bool(os.environ.get("MY_TTS_API_KEY"))
def synthesize(self, text, output_path, *, voice=None, model=None,
speed=None, format="mp3", **extra) -> str:
# Write audio bytes to output_path, return the path.
# Raise on failure — the dispatcher converts exceptions to a
# standard error envelope.
import my_tts_sdk
client = my_tts_sdk.Client()
audio_bytes = client.synthesize(text=text, voice=voice or "default")
with open(output_path, "wb") as f:
f.write(audio_bytes)
return output_path
def register(ctx):
ctx.register_tts_provider(MyTTSProvider())
```
Enable it (`hermes plugins enable my-tts`), point `tts.provider` at it (`tts.provider: my-tts` in `config.yaml`), and the `text_to_speech` tool will route through your plugin.
#### Optional hooks
Override these on your provider class for richer integration:
- `list_voices()` → list of `{id, display, language, gender, preview_url}` dicts shown in `hermes tools`.
- `list_models()` → list of `{id, display, languages, max_text_length}` dicts.
- `get_setup_schema()` → return `{name, badge, tag, env_vars: [{key, prompt, url}]}` to power the picker row in `hermes tools` / `hermes setup`. Without this, the plugin still works but its row in the picker is minimal.
- `stream(text, *, voice, model, format, **extra)` → iterator yielding audio bytes for streaming delivery (default raises `NotImplementedError`).
- `voice_compatible` property → set `True` if your output is Opus-compatible and the gateway should deliver it as a voice bubble (default `False` = regular audio attachment).
See `agent/tts_provider.py` for the full ABC including docstrings.
## Voice Message Transcription (STT)
Voice messages sent on Telegram, Discord, WhatsApp, Slack, or Signal are automatically transcribed and injected as text into the conversation. The agent sees the transcript as normal text.
| Provider | Quality | Cost | API Key |
|----------|---------|------|---------|
| **Local Whisper** (default) | Good | Free | None needed |
| **Groq Whisper API** | Good–Best | Free tier | `GROQ_API_KEY` |
| **OpenAI Whisper API** | Good–Best | Paid | `VOICE_TOOLS_OPENAI_KEY` or `OPENAI_API_KEY` |
:::info Zero Config
Local transcription works out of the box when `faster-whisper` is installed. If that's unavailable, Hermes can also use a local `whisper` CLI from common install locations (like `/opt/homebrew/bin`) or a custom command via `HERMES_LOCAL_STT_COMMAND`.
:::
### Configuration
```yaml
# In ~/.hermes/config.yaml
stt:
provider: "local" # "local" | "groq" | "openai" | "mistral" | "xai"
local:
model: "base" # tiny, base, small, medium, large-v3
openai:
model: "whisper-1" # whisper-1, gpt-4o-mini-transcribe, gpt-4o-transcribe
mistral:
model: "voxtral-mini-latest" # voxtral-mini-latest, voxtral-mini-2602
xai:
model: "grok-stt" # xAI Grok STT
```
### Provider Details
**Local (faster-whisper)** — Runs Whisper locally via [faster-whisper](https://github.com/SYSTRAN/faster-whisper). Uses CPU by default, GPU if available. Model sizes:
| Model | Size | Speed | Quality |
|-------|------|-------|---------|
| `tiny` | ~75 MB | Fastest | Basic |
| `base` | ~150 MB | Fast | Good (default) |
| `small` | ~500 MB | Medium | Better |
| `medium` | ~1.5 GB | Slower | Great |
| `large-v3` | ~3 GB | Slowest | Best |
**Groq API** — Requires `GROQ_API_KEY`. Good cloud fallback when you want a free hosted STT option.
**OpenAI API** — Accepts `VOICE_TOOLS_OPENAI_KEY` first and falls back to `OPENAI_API_KEY`. Supports `whisper-1`, `gpt-4o-mini-transcribe`, and `gpt-4o-transcribe`.
**Mistral API (Voxtral Transcribe)** — Requires `MISTRAL_API_KEY`. Uses Mistral's [Voxtral Transcribe](https://docs.mistral.ai/capabilities/audio/speech_to_text/) models. Supports 13 languages, speaker diarization, and word-level timestamps. Install with `pip install hermes-agent[mistral]`.
**xAI Grok STT** — Requires `XAI_API_KEY`. Posts to `https://api.x.ai/v1/stt` as multipart/form-data. Good choice if you're already using xAI for chat or TTS and want one API key for everything. Auto-detection order puts it after Groq — explicitly set `stt.provider: xai` to force it.
**Custom local CLI fallback** — Set `HERMES_LOCAL_STT_COMMAND` if you want Hermes to call a local transcription command directly. The command template supports `{input_path}`, `{output_dir}`, `{language}`, and `{model}` placeholders. Your command must write a `.txt` transcript somewhere under `{output_dir}`.
#### Example: Doubao / Volcengine ASR
If you use [`doubao-speech`](https://pypi.org/project/doubao-speech/) for Doubao TTS (see [above](#example-doubao-chinese-seed-tts-20)), the same package handles speech-to-text via the local-command STT surface:
```bash
pip install doubao-speech
export VOLCENGINE_APP_ID="your-app-id"
export VOLCENGINE_ACCESS_TOKEN="your-access-token"
export HERMES_LOCAL_STT_COMMAND='doubao-speech transcribe {input_path} --out {output_dir}/transcript.txt'
```
```yaml
stt:
provider: local_command
```
Hermes writes the incoming voice message to `{input_path}`, runs the command, and reads the `.txt` file produced under `{output_dir}`. Language is auto-detected by the Volcengine bigmodel endpoint.
### Fallback Behavior
If your configured provider isn't available, Hermes automatically falls back:
- **Local faster-whisper unavailable** → Tries a local `whisper` CLI or `HERMES_LOCAL_STT_COMMAND` before cloud providers
- **Groq key not set** → Falls back to local transcription, then OpenAI
- **OpenAI key not set** → Falls back to local transcription, then Groq
- **Mistral key/SDK not set** → Skipped in auto-detect; falls through to next available provider
- **Nothing available** → Voice messages pass through with an accurate note to the user
### STT custom command providers
If the STT engine you want isn't natively supported (Doubao ASR, NVIDIA Parakeet, a whisper.cpp build, an open-source SenseVoice CLI, anything else that exposes a shell command), wire it in as a **command-type provider** without writing any Python. Hermes runs your shell command against the audio file and reads back the transcript.
Declare one or more providers under `stt.providers.<name>` and switch between them with `stt.provider: <name>` — same shape as the TTS [command-provider registry](#custom-command-providers), adapted for the input=audio → output=transcript direction.
```yaml
stt:
provider: parakeet # pick any name under stt.providers
providers:
parakeet:
type: command
command: "parakeet-asr --model nvidia/parakeet-tdt-0.6b-v2 --in {input_path} --out {output_path}"
format: txt
language: en
timeout: 300
whispercpp:
type: command
command: "whisper-cli -m ~/models/ggml-large-v3.bin -f {input_path} -otxt -of {output_dir}/transcript"
format: txt
sensevoice:
type: command
command: "sensevoice-cli {input_path} --json | tee {output_path}"
format: json
```
This complements the legacy `HERMES_LOCAL_STT_COMMAND` escape hatch — that env var still works untouched via the built-in `local_command` path. Use `stt.providers.<name>` when you want **multiple** shell-driven STT engines, a name you can pick via `stt.provider`, or anything that needs per-provider `language` / `model` / `timeout`.
#### STT placeholders
Your command template can reference these placeholders. Hermes substitutes them at render time and shell-quotes each value for the surrounding context (bare / single-quoted / double-quoted), so paths with spaces are safe.
| Placeholder | Meaning |
|-------------------|----------------------------------------------------------------------|
| `{input_path}` | Absolute path to the input audio file (original location, read-only) |
| `{output_path}` | Absolute path the command should write the transcript to |
| `{output_dir}` | Parent directory of `{output_path}` (handy for whisper-style tools) |
| `{format}` | Configured output format: `txt` / `json` / `srt` / `vtt` |
| `{language}` | Configured language code (defaults to `en`) |
| `{model}` | `stt.providers.<name>.model`, empty when unset |
Use `{{` and `}}` for literal braces (handy when embedding JSON snippets in the command).
#### How the transcript is read back
After your command exits successfully:
1. If `{output_path}` exists and is non-empty → Hermes reads it as UTF-8 text.
2. Otherwise, if the command wrote to stdout → Hermes uses that.
3. Otherwise → error: "Command STT provider wrote no output file and produced no stdout".
This lets you use the registry for both file-writing CLIs (`whisper-cli`, `parakeet-asr`) and curl-style one-liners that emit transcript to stdout (`curl … | jq -r .text`).
For `format: json` / `srt` / `vtt`, Hermes returns the raw file content as the `transcript` field. Extracting `.text` from JSON is out of scope for the runner — either configure `format: txt`, or post-process JSON downstream.
#### STT command-provider optional keys
| Key | Default | Meaning |
|-----------------|---------|------------------------------------------------------------------------------------------------------|
| `timeout` | `300` | Seconds; the process tree is killed on expiry (Unix `start_new_session`, Windows `taskkill /T`). |
| `format` | `txt` | One of `txt` / `json` / `srt` / `vtt`. Sets the extension of `{output_path}`. |
| `language` | `en` | Forwarded to `{language}`. Defaults to `stt.language` then `en`. |
| `model` | empty | Forwarded to `{model}`. The `model=` argument to `transcribe_audio()` overrides this. |
#### STT command-provider behavior notes
- **Built-ins always win.** Declaring `stt.providers.openai: type: command` does NOT override the real OpenAI Whisper handler. The built-in name is short-circuited before the command-provider resolver runs.
- **Process-tree cleanup.** A command running over `timeout` has its entire process tree killed, not just the shell wrapper. Long-running ASR pipelines that fork model-loading subprocesses are reaped reliably.
- **Shell-quoting is automatic.** Placeholders inside `'…'` get single-quote-safe escaping; inside `"…"` get `$`/`` ` ``/`"` escaping; outside quotes get `shlex.quote`. Don't pre-quote placeholder values.
#### STT command-provider security
The shell command runs under the same user as Hermes with full filesystem access — same trust model as `tts.providers.<name>: type: command` and `HERMES_LOCAL_STT_COMMAND`. Only declare command providers from sources you trust.
### Python plugin providers (STT)
For STT engines that aren't built-in AND can't be expressed as a shell command (need a Python SDK, OAuth-refreshing auth, streaming chunks, etc.), register a Python plugin via `ctx.register_transcription_provider()`. The plugin **coexists with** the 6 built-in providers (`local`, `local_command`, `groq`, `openai`, `mistral`, `xai`) and the `stt.providers.<name>: type: command` registry — built-ins keep their native implementations and always win on name collision; command providers win over plugins of the same name (config is more local than plugin install).
#### When to pick which (STT)
| Backend has… | Use |
|--------------------------------------------------------------|------------------------------------------------------------------|
| A single shell command that takes an audio file and emits text | `stt.providers.<name>: type: command` (no Python needed) |
| Only the legacy single-command escape hatch is wanted | `HERMES_LOCAL_STT_COMMAND` env var (preserved for back-compat) |
| A Python SDK with no CLI | `register_transcription_provider()` plugin |
| OAuth-refreshing auth, streaming chunks, voice-list metadata | `register_transcription_provider()` plugin |
| A built-in already covers it (`local`, `groq`, `openai`, …) | Set `stt.provider: <name>` — built-ins are inline |
#### Resolution order
1. **`stt.provider` is a built-in name** → built-in dispatch. **Always wins.**
2. **`stt.provider` matches `stt.providers.<name>` with `command:` set** → command-provider runner (see [STT custom command providers](#stt-custom-command-providers)). Wins over a same-name plugin.
3. **`stt.provider` matches a plugin-registered `TranscriptionProvider`** → plugin dispatch:
- if the plugin's `is_available()` returns `False` (missing creds or SDK), the call surfaces an unavailability error envelope identifying the plugin — **not** the generic "No STT provider available" message.
- otherwise the plugin's `transcribe()` is called with `model` (from the public `model=` arg, falling back to `stt.<provider>.model`) and `language` (from `stt.<provider>.language`).
4. **No match** → "No STT provider available" error.
#### Per-provider config namespace
Plugins read their per-provider configuration from `stt.<provider>` in `config.yaml`, mirroring how built-ins read `stt.openai.model` / `stt.mistral.model`:
```yaml
stt:
provider: my-stt
my-stt:
model: whisper-large-v3
language: ja # forwarded as language= to transcribe()
# any other plugin-specific keys go here; read them via your
# own config.yaml access in __init__/is_available/transcribe
```
The dispatcher forwards `model` and `language` from this section; everything else, the plugin can read itself.
#### Minimal plugin
Drop this in `~/.hermes/plugins/my-stt/`:
`plugin.yaml`:
```yaml
name: my-stt
version: 0.1.0
description: "My custom Python STT backend"
```
`__init__.py`:
```python
from agent.transcription_provider import TranscriptionProvider
class MySTTProvider(TranscriptionProvider):
@property
def name(self) -> str:
return "my-stt" # what stt.provider matches against
@property
def display_name(self) -> str:
return "My Custom STT"
def is_available(self) -> bool:
# Return False when credentials/deps are missing — picker skips
# this row but the dispatcher still routes here on explicit config.
import os
return bool(os.environ.get("MY_STT_API_KEY"))
def transcribe(self, file_path, *, model=None, language=None, **extra):
# Return the standard transcribe envelope:
# {"success": bool, "transcript": str, "provider": str, "error": str}
# Do NOT raise — convert exceptions to the error envelope so the
# gateway/CLI caller sees a consistent shape on failure.
try:
import my_stt_sdk
client = my_stt_sdk.Client()
text = client.transcribe(open(file_path, "rb"))
return {
"success": True,
"transcript": text,
"provider": "my-stt",
}
except Exception as exc:
return {
"success": False,
"transcript": "",
"error": f"my-stt failed: {exc}",
"provider": "my-stt",
}
def register(ctx):
ctx.register_transcription_provider(MySTTProvider())
```
Enable it (`hermes plugins enable my-stt`), set `stt.provider: my-stt` in `config.yaml`, and voice-message transcription will route through your plugin.
#### Optional hooks
Override these on your provider class for richer integration:
- `list_models()` → list of `{id, display, languages, max_audio_seconds}` dicts.
- `default_model()` → string returned when the user doesn't override the model.
- `get_setup_schema()` → return `{name, badge, tag, env_vars: [{key, prompt, url}]}` to power picker rows in `hermes tools` / `hermes setup` (the picker category for STT is not yet shipped — this metadata is available to plugins for forward compatibility).
See `agent/transcription_provider.py` for the full ABC including docstrings.
website/docs/user-guide/features/vision.md
+214
View File
@@ -0,0 +1,214 @@
---
title: Vision & Image Paste
description: Paste images from your clipboard into the Hermes CLI for multimodal vision analysis.
sidebar_label: Vision & Image Paste
sidebar_position: 7
---
# Vision & Image Paste
Hermes Agent supports **multimodal vision** — you can paste images from your clipboard directly into the CLI and ask the agent to analyze, describe, or work with them. Images are sent to the model as base64-encoded content blocks, so any vision-capable model can process them.
:::tip
Portal subscribers get vision-capable models (Claude, GPT-5, Gemini) in the same catalog — no extra credentials needed. See [Nous Portal](/integrations/nous-portal).
:::
## How It Works
1. Copy an image to your clipboard (screenshot, browser image, etc.)
2. Attach it using one of the methods below
3. Type your question and press Enter
4. The image appears as a `[📎 Image #1]` badge above the input
5. On submit, the image is sent to the model as a vision content block
You can attach multiple images before sending — each gets its own badge. Press `Ctrl+C` to clear all attached images.
Images are saved to `~/.hermes/images/` as PNG files with timestamped filenames.
## Paste Methods
How you attach an image depends on your terminal environment. Not all methods work everywhere — here's the full breakdown:
### `/paste` Command
**The most reliable explicit image-attach fallback.**
```
/paste
```
Type `/paste` and press Enter. Hermes checks your clipboard for an image and attaches it. This is the safest option when your terminal rewrites `Cmd+V`/`Ctrl+V`, or when you copied only an image and there is no bracketed-paste text payload to inspect.
### Ctrl+V / Cmd+V
Hermes now treats paste as a layered flow:
- normal text paste first
- native clipboard / OSC52 text fallback if the terminal did not deliver text cleanly
- image attach when the clipboard or pasted payload resolves to an image or image path
This means pasted macOS screenshot temp paths and `file://...` image URIs can attach immediately instead of sitting in the composer as raw text.
:::warning
If your clipboard has **only an image** (no text), terminals still cannot send binary image bytes directly. Use `/paste` as the explicit image-attach fallback.
:::
### `/terminal-setup` for VS Code / Cursor / Windsurf
If you run the TUI inside a local VS Code-family integrated terminal on macOS, Hermes can install the recommended `workbench.action.terminal.sendSequence` bindings for better multiline and undo/redo parity:
```text
/terminal-setup
```
This is especially useful when `Cmd+Enter`, `Cmd+Z`, or `Shift+Cmd+Z` are being intercepted by the IDE. Run it on the local machine only — not inside an SSH session.
## Platform Compatibility
| Environment | `/paste` | Cmd/Ctrl+V | `/terminal-setup` | Notes |
|---|:---:|:---:|:---:|---|
| **macOS Terminal / iTerm2** | ✅ | ✅ | n/a | Best experience — native clipboard + screenshot-path recovery |
| **Apple Terminal** | ✅ | ✅ | n/a | If Cmd+←/→/⌫ gets rewritten, use Ctrl+A / Ctrl+E / Ctrl+U fallbacks |
| **Linux X11 desktop** | ✅ | ✅ | n/a | Requires `xclip` (`apt install xclip`) |
| **Linux Wayland desktop** | ✅ | ✅ | n/a | Requires `wl-paste` (`apt install wl-clipboard`) |
| **WSL2 (Windows Terminal)** | ✅ | ✅ | n/a | Uses `powershell.exe` — no extra install needed |
| **VS Code / Cursor / Windsurf (local)** | ✅ | ✅ | ✅ | Recommended for better Cmd+Enter / undo / redo parity |
| **VS Code / Cursor / Windsurf (SSH)** | ❌² | ❌² | ❌³ | Run `/terminal-setup` on the local machine instead |
| **SSH terminal (any)** | ❌² | ❌² | n/a | Remote clipboard not accessible |
² See [SSH & Remote Sessions](#ssh--remote-sessions) below
³ The command writes local IDE keybindings and should not be run from the remote host
## Platform-Specific Setup
### macOS
**No setup required.** Hermes uses `osascript` (built into macOS) to read the clipboard. For faster performance, optionally install `pngpaste`:
```bash
brew install pngpaste
```
### Linux (X11)
Install `xclip`:
```bash
# Ubuntu/Debian
sudo apt install xclip
# Fedora
sudo dnf install xclip
# Arch
sudo pacman -S xclip
```
### Linux (Wayland)
Modern Linux desktops (Ubuntu 22.04+, Fedora 34+) often use Wayland by default. Install `wl-clipboard`:
```bash
# Ubuntu/Debian
sudo apt install wl-clipboard
# Fedora
sudo dnf install wl-clipboard
# Arch
sudo pacman -S wl-clipboard
```
:::tip How to check if you're on Wayland
```bash
echo $XDG_SESSION_TYPE
# "wayland" = Wayland, "x11" = X11, "tty" = no display server
```
:::
### WSL2
**No extra setup required.** Hermes detects WSL2 automatically (via `/proc/version`) and uses `powershell.exe` to access the Windows clipboard through .NET's `System.Windows.Forms.Clipboard`. This is built into WSL2's Windows interop — `powershell.exe` is available by default.
The clipboard data is transferred as base64-encoded PNG over stdout, so no file path conversion or temp files are needed.
:::info WSLg Note
If you're running WSLg (WSL2 with GUI support), Hermes tries the PowerShell path first, then falls back to `wl-paste`. WSLg's clipboard bridge only supports BMP format for images — Hermes auto-converts BMP to PNG using Pillow (if installed) or ImageMagick's `convert` command.
:::
#### Verify WSL2 clipboard access
```bash
# 1. Check WSL detection
grep -i microsoft /proc/version
# 2. Check PowerShell is accessible
which powershell.exe
# 3. Copy an image, then check
powershell.exe -NoProfile -Command "Add-Type -AssemblyName System.Windows.Forms; [System.Windows.Forms.Clipboard]::ContainsImage()"
# Should print "True"
```
## SSH & Remote Sessions
**Clipboard image paste does not fully work over SSH.** When you SSH into a remote machine, the Hermes CLI runs on the remote host. Clipboard tools (`xclip`, `wl-paste`, `powershell.exe`, `osascript`) read the clipboard of the machine they run on — which is the remote server, not your local machine. Your local clipboard image is therefore inaccessible from the remote side.
Text can sometimes still bridge through terminal paste or OSC52, but image clipboard access and local screenshot temp paths remain tied to the machine running Hermes.
### Workarounds for SSH
1. **Upload the image file** — Save the image locally, upload it to the remote server via `scp`, VSCode's file explorer (drag-and-drop), or any file transfer method. Then reference it by path. *(A `/attach <filepath>` command is planned for a future release.)*
2. **Use a URL** — If the image is accessible online, just paste the URL in your message. The agent can use `vision_analyze` to look at any image URL directly.
3. **X11 forwarding** — Connect with `ssh -X` to forward X11. This lets `xclip` on the remote machine access your local X11 clipboard. Requires an X server running locally (XQuartz on macOS, built-in on Linux X11 desktops). Slow for large images.
4. **Use a messaging platform** — Send images to Hermes via Telegram, Discord, Slack, or WhatsApp. These platforms handle image upload natively and are not affected by clipboard/terminal limitations.
## Why Terminals Can't Paste Images
This is a common source of confusion, so here's the technical explanation:
Terminals are **text-based** interfaces. When you press Ctrl+V (or Cmd+V), the terminal emulator:
1. Reads the clipboard for **text content**
2. Wraps it in [bracketed paste](https://en.wikipedia.org/wiki/Bracketed-paste) escape sequences
3. Sends it to the application through the terminal's text stream
If the clipboard contains only an image (no text), the terminal has nothing to send. There is no standard terminal escape sequence for binary image data. The terminal simply does nothing.
This is why Hermes uses a separate clipboard check — instead of receiving image data through the terminal paste event, it calls OS-level tools (`osascript`, `powershell.exe`, `xclip`, `wl-paste`) directly via subprocess to read the clipboard independently.
## Supported Models
Image paste works with any vision-capable model. The image is sent as a base64-encoded data URL in the OpenAI vision content format:
```json
{
"type": "image_url",
"image_url": {
"url": "data:image/png;base64,..."
}
}
```
Most modern models support this format, including GPT-4 Vision, Claude (with vision), Gemini, and open-source multimodal models served through OpenRouter.
## Image Routing (Vision-Capable vs Text-Only Models)
When a user attaches an image — from the CLI clipboard, the gateway (Telegram/Discord photo), or any other entry point — Hermes routes it based on whether your current model actually supports vision:
| Your model | What happens to the image |
|---|---|
| **Vision-capable** (GPT-4V, Claude with vision, Gemini, Qwen-VL, MiMo-VL, etc.) | Sent as **real pixels** using the provider's native image content format above. No text summary layer. |
| **Text-only** (DeepSeek V3, smaller open-source models, older chat-only endpoints) | Routed through the `vision_analyze` auxiliary tool — an auxiliary vision model describes the image, and the text description is injected into the conversation. |
You don't configure this — Hermes looks up your current model's capability in the provider metadata and picks the right path automatically. The practical effect: you can switch between vision and non-vision models mid-session and image handling "just works" without changing your workflow. Text-only models get coherent context about the image rather than a broken multimodal payload they'd have to reject.
Which auxiliary model handles the text-description path is configurable under `auxiliary.vision` — see [Auxiliary Models](/user-guide/configuration#auxiliary-models).
### `vision_analyze` has the same dual behavior
The `vision_analyze` tool itself follows the same routing. When the active main model is vision-capable **and** its provider supports image content inside tool results (currently the Anthropic, OpenAI, Azure-OpenAI, and Gemini 3.x stacks), `vision_analyze` short-circuits the auxiliary describer and returns the raw image pixels as a multimodal tool-result envelope. The main model sees the image natively on its next turn — no aux call, no text-summary information loss, no extra latency.
For text-only main models (or providers whose tool-result channel doesn't carry images), `vision_analyze` falls back to the legacy path: it asks the configured auxiliary vision model to describe the image and returns the description as plain text. Either way the calling tool signature is the same — the tool decides which path to take at runtime based on the active model.
website/docs/user-guide/features/voice-mode.md
+524
View File
@@ -0,0 +1,524 @@
---
sidebar_position: 10
title: "Voice Mode"
description: "Real-time voice conversations with Hermes Agent — CLI, Telegram, Discord (DMs, text channels, and voice channels)"
---
# Voice Mode
Hermes Agent supports full voice interaction across CLI and messaging platforms. Talk to the agent using your microphone, hear spoken replies, and have live voice conversations in Discord voice channels.
If you want a practical setup walkthrough with recommended configurations and real usage patterns, see [Use Voice Mode with Hermes](/guides/use-voice-mode-with-hermes).
## Prerequisites
Before using voice features, make sure you have:
1. **Hermes Agent installed** — `pip install hermes-agent` (see [Installation](/getting-started/installation))
2. **An LLM provider configured** — run `hermes model` or set your preferred provider credentials in `~/.hermes/.env`
3. **A working base setup** — run `hermes` to verify the agent responds to text before enabling voice
:::tip
The `~/.hermes/` directory and default `config.yaml` are created automatically the first time you run `hermes`. You only need to create `~/.hermes/.env` manually for API keys.
:::
:::tip Nous Portal covers both
A paid [Nous Portal](/user-guide/features/tool-gateway) subscription supplies the LLM (step 2) **and** OpenAI TTS via the Tool Gateway — no separate OpenAI key needed. On a fresh install, `hermes setup --portal` wires both up at once.
:::
## Overview
| Feature | Platform | Description |
|---------|----------|-------------|
| **Interactive Voice** | CLI | Press Ctrl+B to record, agent auto-detects silence and responds |
| **Auto Voice Reply** | Telegram, Discord | Agent sends spoken audio alongside text responses |
| **Voice Channel** | Discord | Bot joins VC, listens to users speaking, speaks replies back |
## Requirements
### Python Packages
```bash
# CLI voice mode (microphone + audio playback)
pip install "hermes-agent[voice]"
# Discord + Telegram messaging (includes discord.py[voice] for VC support)
pip install "hermes-agent[messaging]"
# Premium TTS (ElevenLabs)
pip install "hermes-agent[tts-premium]"
# Local TTS (NeuTTS, optional)
python -m pip install -U neutts[all]
# Everything at once
pip install "hermes-agent[all]"
```
| Extra | Packages | Required For |
|-------|----------|-------------|
| `voice` | `sounddevice`, `numpy` | CLI voice mode |
| `messaging` | `discord.py[voice]`, `python-telegram-bot`, `aiohttp` | Discord & Telegram bots |
| `tts-premium` | `elevenlabs` | ElevenLabs TTS provider |
Optional local TTS provider: install `neutts` separately with `python -m pip install -U neutts[all]`. On first use it downloads the model automatically.
:::info
`discord.py[voice]` installs **PyNaCl** (for voice encryption) and **opus bindings** automatically. This is required for Discord voice channel support.
:::
### System Dependencies
```bash
# macOS
brew install portaudio ffmpeg opus
brew install espeak-ng # for NeuTTS
# Ubuntu/Debian
sudo apt install portaudio19-dev ffmpeg libopus0
sudo apt install espeak-ng # for NeuTTS
```
| Dependency | Purpose | Required For |
|-----------|---------|-------------|
| **PortAudio** | Microphone input and audio playback | CLI voice mode |
| **ffmpeg** | Audio format conversion (MP3 → Opus, PCM → WAV) | All platforms |
| **Opus** | Discord voice codec | Discord voice channels |
| **espeak-ng** | Phonemizer backend | Local NeuTTS provider |
### API Keys
Add to `~/.hermes/.env`:
```bash
# Speech-to-Text — local provider needs NO key at all
# pip install faster-whisper # Free, runs locally, recommended
GROQ_API_KEY=your-key # Groq Whisper — fast, free tier (cloud)
VOICE_TOOLS_OPENAI_KEY=your-key # OpenAI Whisper — paid (cloud)
# Text-to-Speech (optional — Edge TTS and NeuTTS work without any key)
ELEVENLABS_API_KEY=*** # ElevenLabs — premium quality
# VOICE_TOOLS_OPENAI_KEY above also enables OpenAI TTS
```
:::tip
If `faster-whisper` is installed, voice mode works with **zero API keys** for STT. The model (~150 MB for `base`) downloads automatically on first use.
:::
---
## CLI Voice Mode
Voice mode is available in both the **classic CLI** (`hermes chat`) and the **TUI** (`hermes --tui`). Behavior is identical across both — same slash commands, same VAD silence detection, same streaming TTS, same hallucination filter. The TUI additionally forwards crash-forensic logs to `~/.hermes/logs/` so push-to-talk failures on exotic audio backends can be reported with a full stack trace rather than disappearing silently.
### Quick Start
Start the CLI and enable voice mode:
```bash
hermes # Start the interactive CLI
```
Then use these commands inside the CLI:
```
/voice Toggle voice mode on/off
/voice on Enable voice mode
/voice off Disable voice mode
/voice tts Toggle TTS output
/voice status Show current state
```
### How It Works
1. Start the CLI with `hermes` and enable voice mode with `/voice on`
2. **Press Ctrl+B** — a beep plays (880Hz), recording starts
3. **Speak** — a live audio level bar shows your input: `● [▁▂▃▅▇▇▅▂] ❯`
4. **Stop speaking** — after 3 seconds of silence, recording auto-stops
5. **Two beeps** play (660Hz) confirming the recording ended
6. Audio is transcribed via Whisper and sent to the agent
7. If TTS is enabled, the agent's reply is spoken aloud
8. Recording **automatically restarts** — speak again without pressing any key
This loop continues until you press **Ctrl+B** during recording (exits continuous mode) or 3 consecutive recordings detect no speech.
:::tip
The record key is configurable via `voice.record_key` in `~/.hermes/config.yaml` (default: `ctrl+b`).
:::
### Silence Detection
Two-stage algorithm detects when you've finished speaking:
1. **Speech confirmation** — waits for audio above the RMS threshold (200) for at least 0.3s, tolerating brief dips between syllables
2. **End detection** — once speech is confirmed, triggers after 3.0 seconds of continuous silence
If no speech is detected at all for 15 seconds, recording stops automatically.
Both `silence_threshold` and `silence_duration` are configurable in `config.yaml`. You can also disable the record start/stop beeps with `voice.beep_enabled: false`.
### Streaming TTS
When TTS is enabled, the agent speaks its reply **sentence-by-sentence** as it generates text — you don't wait for the full response:
1. Buffers text deltas into complete sentences (min 20 chars)
2. Strips markdown formatting and `<think>` blocks
3. Generates and plays audio per sentence in real-time
### Hallucination Filter
Whisper sometimes generates phantom text from silence or background noise ("Thank you for watching", "Subscribe", etc.). The agent filters these out using a set of 26 known hallucination phrases across multiple languages, plus a regex pattern that catches repetitive variations.
---
## Gateway Voice Reply (Telegram & Discord)
If you haven't set up your messaging bots yet, see the platform-specific guides:
- [Telegram Setup Guide](../messaging/telegram.md)
- [Discord Setup Guide](../messaging/discord.md)
Start the gateway to connect to your messaging platforms:
```bash
hermes gateway # Start the gateway (connects to configured platforms)
hermes gateway setup # Interactive setup wizard for first-time configuration
```
### Discord: Channels vs DMs
The bot supports two interaction modes on Discord:
| Mode | How to Talk | Mention Required | Setup |
|------|------------|-----------------|-------|
| **Direct Message (DM)** | Open the bot's profile → "Message" | No | Works immediately |
| **Server Channel** | Type in a text channel where the bot is present | Yes (`@botname`) | Bot must be invited to the server |
**DM (recommended for personal use):** Just open a DM with the bot and type — no @mention needed. Voice replies and all commands work the same as in channels.
**Server channels:** The bot only responds when you @mention it (e.g. `@hermesbyt4 hello`). Make sure you select the **bot user** from the mention popup, not the role with the same name.
:::tip
To disable the mention requirement in server channels, add to `~/.hermes/.env`:
```bash
DISCORD_REQUIRE_MENTION=false
```
Or set specific channels as free-response (no mention needed):
```bash
DISCORD_FREE_RESPONSE_CHANNELS=123456789,987654321
```
:::
### Commands
These work in both Telegram and Discord (DMs and text channels):
```
/voice Toggle voice mode on/off
/voice on Voice replies only when you send a voice message
/voice tts Voice replies for ALL messages
/voice off Disable voice replies
/voice status Show current setting
```
### Modes
| Mode | Command | Behavior |
|------|---------|----------|
| `off` | `/voice off` | Text only (default) |
| `voice_only` | `/voice on` | Speaks reply only when you send a voice message |
| `all` | `/voice tts` | Speaks reply to every message |
Voice mode setting is persisted across gateway restarts.
### Platform Delivery
| Platform | Format | Notes |
|----------|--------|-------|
| **Telegram** | Voice bubble (Opus/OGG) | Plays inline in chat. ffmpeg converts MP3 → Opus if needed |
| **Discord** | Native voice bubble (Opus/OGG) | Plays inline like a user voice message. Falls back to file attachment if voice bubble API fails |
---
## Discord Voice Channels
The most immersive voice feature: the bot joins a Discord voice channel, listens to users speaking, transcribes their speech, processes through the agent, and speaks the reply back in the voice channel.
### Setup
#### 1. Discord Bot Permissions
If you already have a Discord bot set up for text (see [Discord Setup Guide](../messaging/discord.md)), you need to add voice permissions.
Go to the [Discord Developer Portal](https://discord.com/developers/applications) → your application → **Installation** → **Default Install Settings** → **Guild Install**:
**Add these permissions to the existing text permissions:**
| Permission | Purpose | Required |
|-----------|---------|----------|
| **Connect** | Join voice channels | Yes |
| **Speak** | Play TTS audio in voice channels | Yes |
| **Use Voice Activity** | Detect when users are speaking | Recommended |
**Updated Permissions Integer:**
| Level | Integer | What's Included |
|-------|---------|----------------|
| Text only | `274878286912` | View Channels, Send Messages, Read History, Embeds, Attachments, Threads, Reactions |
| Text + Voice | `274881432640` | All above + Connect, Speak |
**Re-invite the bot** with the updated permissions URL:
```
https://discord.com/oauth2/authorize?client_id=YOUR_APP_ID&scope=bot+applications.commands&permissions=274881432640
```
Replace `YOUR_APP_ID` with your Application ID from the Developer Portal.
:::warning
Re-inviting the bot to a server it's already in will update its permissions without removing it. You won't lose any data or configuration.
:::
#### 2. Privileged Gateway Intents
In the [Developer Portal](https://discord.com/developers/applications) → your application → **Bot** → **Privileged Gateway Intents**, enable all three:
| Intent | Purpose |
|--------|---------|
| **Presence Intent** | Detect user online/offline status |
| **Server Members Intent** | Resolve usernames in `DISCORD_ALLOWED_USERS` to numeric IDs (conditional) |
| **Message Content Intent** | Read text message content in channels |
**Message Content Intent** is required. **Server Members Intent** is only needed if your `DISCORD_ALLOWED_USERS` list uses usernames — if you use numeric user IDs, you can leave it OFF. Voice-channel SSRC → user_id mapping comes from Discord's SPEAKING opcode on the voice websocket and does **not** require the Server Members Intent.
#### 3. Opus Codec
The Opus codec library must be installed on the machine running the gateway:
```bash
# macOS (Homebrew)
brew install opus
# Ubuntu/Debian
sudo apt install libopus0
```
The bot auto-loads the codec from:
- **macOS:** `/opt/homebrew/lib/libopus.dylib`
- **Linux:** `libopus.so.0`
#### 4. Environment Variables
```bash
# ~/.hermes/.env
# Discord bot (already configured for text)
DISCORD_BOT_TOKEN=your-bot-token
DISCORD_ALLOWED_USERS=your-user-id
# STT — local provider needs no key (pip install faster-whisper)
# GROQ_API_KEY=your-key # Alternative: cloud-based, fast, free tier
# TTS — optional. Edge TTS and NeuTTS need no key.
# ELEVENLABS_API_KEY=*** # Premium quality
# VOICE_TOOLS_OPENAI_KEY=*** # OpenAI TTS / Whisper
```
### Start the Gateway
```bash
hermes gateway # Start with existing configuration
```
The bot should come online in Discord within a few seconds.
### Commands
Use these in the Discord text channel where the bot is present:
```
/voice join Bot joins your current voice channel
/voice channel Alias for /voice join
/voice leave Bot disconnects from voice channel
/voice status Show voice mode and connected channel
```
:::info
You must be in a voice channel before running `/voice join`. The bot joins the same VC you're in.
:::
### How It Works
When the bot joins a voice channel, it:
1. **Listens** to each user's audio stream independently
2. **Detects silence** — 1.5s of silence after at least 0.5s of speech triggers processing
3. **Transcribes** the audio via Whisper STT (local, Groq, or OpenAI)
4. **Processes** through the full agent pipeline (session, tools, memory)
5. **Speaks** the reply back in the voice channel via TTS
### Text Channel Integration
When the bot is in a voice channel:
- Transcripts appear in the text channel: `[Voice] @user: what you said`
- Agent responses are sent as text in the channel AND spoken in the VC
- The text channel is the one where `/voice join` was issued
### Echo Prevention
The bot automatically pauses its audio listener while playing TTS replies, preventing it from hearing and re-processing its own output.
### Access Control
Only users listed in `DISCORD_ALLOWED_USERS` can interact via voice. Other users' audio is silently ignored.
```bash
# ~/.hermes/.env
DISCORD_ALLOWED_USERS=284102345871466496
```
---
## Configuration Reference
### config.yaml
```yaml
# Voice recording (CLI)
voice:
record_key: "ctrl+b" # Key to start/stop recording
max_recording_seconds: 120 # Maximum recording length
auto_tts: false # Auto-enable TTS when voice mode starts
beep_enabled: true # Play record start/stop beeps
silence_threshold: 200 # RMS level (0-32767) below which counts as silence
silence_duration: 3.0 # Seconds of silence before auto-stop
# Speech-to-Text
stt:
enabled: true # set to false to skip auto-transcription —
# the gateway still caches the audio file and
# passes its path to the agent as part of the
# inbound message, useful for custom pipelines
# (diarization, alignment, archival, etc.)
provider: "local" # "local" (free) | "groq" | "openai" | "mistral" | "xai"
local:
model: "base" # tiny, base, small, medium, large-v3
# model: "whisper-1" # Legacy: used when provider is not set
# Text-to-Speech
tts:
provider: "edge" # "edge" (free) | "elevenlabs" | "openai" | "neutts" | "minimax" | "mistral" | "gemini" | "xai" | "kittentts" | "piper"
edge:
voice: "en-US-AriaNeural" # 322 voices, 74 languages
elevenlabs:
voice_id: "pNInz6obpgDQGcFmaJgB" # Adam
model_id: "eleven_multilingual_v2"
openai:
model: "gpt-4o-mini-tts"
voice: "alloy" # alloy, echo, fable, onyx, nova, shimmer
base_url: "https://api.openai.com/v1" # optional: override for self-hosted or OpenAI-compatible endpoints
neutts:
ref_audio: ''
ref_text: ''
model: neuphonic/neutts-air-q4-gguf
device: cpu
```
### Environment Variables
```bash
# Speech-to-Text providers (local needs no key)
# pip install faster-whisper # Free local STT — no API key needed
GROQ_API_KEY=... # Groq Whisper (fast, free tier)
VOICE_TOOLS_OPENAI_KEY=... # OpenAI Whisper (paid)
# STT advanced overrides (optional)
STT_GROQ_MODEL=whisper-large-v3-turbo # Override default Groq STT model
STT_OPENAI_MODEL=whisper-1 # Override default OpenAI STT model
GROQ_BASE_URL=https://api.groq.com/openai/v1 # Custom Groq endpoint
STT_OPENAI_BASE_URL=https://api.openai.com/v1 # Custom OpenAI STT endpoint
# Text-to-Speech providers (Edge TTS and NeuTTS need no key)
ELEVENLABS_API_KEY=*** # ElevenLabs (premium quality)
# VOICE_TOOLS_OPENAI_KEY above also enables OpenAI TTS
# Discord voice channel
DISCORD_BOT_TOKEN=...
DISCORD_ALLOWED_USERS=...
```
### STT Provider Comparison
| Provider | Model | Speed | Quality | Cost | API Key |
|----------|-------|-------|---------|------|---------|
| **Local** | `base` | Fast (depends on CPU/GPU) | Good | Free | No |
| **Local** | `small` | Medium | Better | Free | No |
| **Local** | `large-v3` | Slow | Best | Free | No |
| **Groq** | `whisper-large-v3-turbo` | Very fast (~0.5s) | Good | Free tier | Yes |
| **Groq** | `whisper-large-v3` | Fast (~1s) | Better | Free tier | Yes |
| **OpenAI** | `whisper-1` | Fast (~1s) | Good | Paid | Yes |
| **OpenAI** | `gpt-4o-transcribe` | Medium (~2s) | Best | Paid | Yes |
| **Mistral** | `voxtral-mini-latest` | Fast | Good | Paid | Yes |
| **xAI** | `grok-stt` | Fast | Good | Paid | Yes |
Provider priority (automatic fallback): **local** > **groq** > **openai**
### TTS Provider Comparison
| Provider | Quality | Cost | Latency | Key Required |
|----------|---------|------|---------|-------------|
| **Edge TTS** | Good | Free | ~1s | No |
| **ElevenLabs** | Excellent | Paid | ~2s | Yes |
| **OpenAI TTS** | Good | Paid | ~1.5s | Yes |
| **NeuTTS** | Good | Free | Depends on CPU/GPU | No |
NeuTTS uses the `tts.neutts` config block above.
---
## Troubleshooting
### "No audio device found" (CLI)
PortAudio is not installed:
```bash
brew install portaudio # macOS
sudo apt install portaudio19-dev # Ubuntu
```
If you are running Hermes inside Docker on a Linux desktop, the container also needs access to your host audio socket. See the [Docker audio bridge](/user-guide/docker#optional-linux-desktop-audio-bridge) notes for a PulseAudio/PipeWire-compatible setup.
### Bot doesn't respond in Discord server channels
The bot requires an @mention by default in server channels. Make sure you:
1. Type `@` and select the **bot user** (with the #discriminator), not the **role** with the same name
2. Or use DMs instead — no mention needed
3. Or set `DISCORD_REQUIRE_MENTION=false` in `~/.hermes/.env`
### Bot joins VC but doesn't hear me
- Check your Discord user ID is in `DISCORD_ALLOWED_USERS`
- Make sure you're not muted in Discord
- The bot needs a SPEAKING event from Discord before it can map your audio — start speaking within a few seconds of joining
### Bot hears me but doesn't respond
- Verify STT is available: install `faster-whisper` (no key needed) or set `GROQ_API_KEY` / `VOICE_TOOLS_OPENAI_KEY`
- Check the LLM model is configured and accessible
- Review gateway logs: `tail -f ~/.hermes/logs/gateway.log`
### Bot responds in text but not in voice channel
- TTS provider may be failing — check API key and quota
- Edge TTS (free, no key) is the default fallback
- Check logs for TTS errors
### Whisper returns garbage text
The hallucination filter catches most cases automatically. If you're still getting phantom transcripts:
- Use a quieter environment
- Adjust `silence_threshold` in config (higher = less sensitive)
- Try a different STT model
website/docs/user-guide/features/web-dashboard.md
+1105
View File
File diff suppressed because it is too large Load Diff
website/docs/user-guide/features/web-search.md
+446
View File
@@ -0,0 +1,446 @@
---
title: Web Search & Extract
description: Search the web and extract page content with multiple backend providers — including free self-hosted SearXNG.
sidebar_label: Web Search
sidebar_position: 6
---
# Web Search & Extract
Hermes Agent includes two model-callable web tools backed by multiple providers:
- **`web_search`** — search the web and return ranked results
- **`web_extract`** — fetch and extract readable content from one or more URLs
Both are configured through a single backend selection. Providers are chosen via `hermes tools` or set directly in `config.yaml`.
## Backends
| Provider | Env Var | Search | Extract | Free tier |
|----------|---------|--------|---------|-----------|
| **Firecrawl** (default) | `FIRECRAWL_API_KEY` | ✔ | ✔ | 500 credits/mo |
| **SearXNG** | `SEARXNG_URL` | ✔ | — | ✔ Free (self-hosted) |
| **Brave Search (free tier)** | `BRAVE_SEARCH_API_KEY` | ✔ | — | 2 000 queries/mo |
| **DDGS (DuckDuckGo)** | — (no key) | ✔ | — | ✔ Free |
| **Tavily** | `TAVILY_API_KEY` | ✔ | ✔ | 1 000 searches/mo |
| **Exa** | `EXA_API_KEY` | ✔ | ✔ | 1 000 searches/mo |
| **Parallel** | `PARALLEL_API_KEY` | ✔ | ✔ | Paid |
| **xAI (Grok)** | `XAI_API_KEY` or `hermes auth login xai-oauth` | ✔ | — | Paid (SuperGrok or per-token) |
Brave Search, DDGS, and xAI are **search-only** — pair any of them with Firecrawl/Tavily/Exa/Parallel when you also need `web_extract`. DDGS uses the [`ddgs` Python package](https://pypi.org/project/ddgs/) under the hood; if it isn't already installed, run `pip install ddgs` (or let Hermes lazy-install it on first use). xAI runs Grok's server-side `web_search` tool on the Responses API — results are LLM-generated rather than index-backed, so titles, descriptions, and URL choice are all model output (see the [trust-model caveat](#xai-grok) below).
**Per-capability split:** you can use different providers for search and extract independently — for example SearXNG (free) for search and Firecrawl for extract. See [Per-capability configuration](#per-capability-configuration) below.
:::tip Nous Subscribers
If you have a paid [Nous Portal](https://portal.nousresearch.com) subscription, web search and extract are available through the **[Tool Gateway](tool-gateway.md)** via managed Firecrawl — no API key needed. New installs can run `hermes setup --portal` to log in and turn on all gateway tools at once; existing installs can flip just web via `hermes tools`.
:::
---
## How `web_extract` handles long pages
Backends return raw page markdown, which can be huge (forum threads, docs sites, news articles with embedded comments). To keep your context window usable and your costs down, `web_extract` runs returned content through the **`web_extract` auxiliary model** before handing it to the agent. Behavior is purely size-driven:
| Page size (characters) | What happens |
|------------------------|--------------|
| Under 5 000 | Returned as-is — no LLM call, full markdown reaches the agent |
| 5 000 – 500 000 | Single-pass summary via the `web_extract` auxiliary model, capped at ~5 000 chars of output |
| 500 000 – 2 000 000 | Chunked: split into 100 k-char chunks, summarize each in parallel, then synthesize a final summary (~5 000 chars) |
| Over 2 000 000 | Refused with a hint to use a more focused source URL |
The summary keeps quotes, code blocks, and key facts in their original formatting — it's a content compressor, not a paraphraser. If summarization fails or times out, Hermes falls back to the first ~5 000 chars of raw content rather than a useless error.
### Which model does the summarizing?
The `web_extract` auxiliary task. By default (`auxiliary.web_extract.provider: "auto"`), this is your **main chat model** — same provider, same model as `hermes model`. That's fine for most setups, but on expensive reasoning models (Opus, MiniMax M2.7, etc.) every long-page extract adds meaningful cost.
To route extraction summaries to a cheap, fast model regardless of your main:
```yaml
# ~/.hermes/config.yaml
auxiliary:
web_extract:
provider: openrouter
model: google/gemini-3-flash-preview
timeout: 360 # seconds; raise if you hit summarization timeouts
```
Or pick interactively: `hermes model` → **Configure auxiliary models** → `web_extract`.
See [Auxiliary Models](/user-guide/configuration#auxiliary-models) for the full reference and per-task override patterns.
### When summarization gets in the way
If you specifically need raw, unsummarized page content — for example, you're scraping a structured page where the LLM summary would drop important fields — use `browser_navigate` + `browser_snapshot` instead. The browser tool returns the live accessibility tree without auxiliary-model rewriting (subject to its own 8 000-char snapshot cap on huge pages).
---
## Setup
### Quick setup via `hermes tools`
Run `hermes tools`, navigate to **Web Search & Extract**, and pick a provider. The wizard prompts for the required URL or API key and writes it to your config.
```bash
hermes tools
```
---
### Firecrawl (default)
Full-featured search and extract. Recommended for most users.
```bash
# ~/.hermes/.env
FIRECRAWL_API_KEY=fc-your-key-here
```
Get a key at [firecrawl.dev](https://firecrawl.dev). The free tier includes 500 credits/month.
**Self-hosted Firecrawl:** Point at your own instance instead of the cloud API:
```bash
# ~/.hermes/.env
FIRECRAWL_API_URL=http://localhost:3002
```
When `FIRECRAWL_API_URL` is set, the API key is optional (disable server auth with `USE_DB_AUTHENTICATION=false`).
---
### SearXNG (free, self-hosted)
SearXNG is a privacy-respecting, open-source metasearch engine that aggregates results from 70+ search engines. **No API key required** — just point Hermes at a running SearXNG instance.
SearXNG is **search-only** — `web_extract` requires a separate extract provider.
#### Option A — Self-host with Docker (recommended)
This gives you a private instance with no rate limits.
**1. Create a working directory:**
```bash
mkdir -p ~/searxng/searxng
cd ~/searxng
```
**2. Write a `docker-compose.yml`:**
```yaml
# ~/searxng/docker-compose.yml
services:
searxng:
image: searxng/searxng:latest
container_name: searxng
ports:
- "8888:8080"
volumes:
- ./searxng:/etc/searxng:rw
environment:
- SEARXNG_BASE_URL=http://localhost:8888/
restart: unless-stopped
```
**3. Start the container:**
```bash
docker compose up -d
```
**4. Enable the JSON API format:**
SearXNG ships with JSON output disabled by default. Copy the generated config and enable it:
```bash
# Copy the auto-generated config out of the container
docker cp searxng:/etc/searxng/settings.yml ~/searxng/searxng/settings.yml
```
Open `~/searxng/searxng/settings.yml` and find the `formats` block (around line 84):
```yaml
# Before (default — JSON disabled):
formats:
- html
# After (enable JSON for Hermes):
formats:
- html
- json
```
**5. Restart to apply:**
```bash
docker cp ~/searxng/searxng/settings.yml searxng:/etc/searxng/settings.yml
docker restart searxng
```
**6. Verify it works:**
```bash
curl -s "http://localhost:8888/search?q=test&format=json" | python3 -c \
"import sys,json; d=json.load(sys.stdin); print(f'{len(d[\"results\"])} results')"
```
You should see something like `10 results`. If you get a `403 Forbidden`, JSON format is still disabled — recheck step 4.
**7. Configure Hermes:**
```bash
# ~/.hermes/.env
SEARXNG_URL=http://localhost:8888
```
Then select SearXNG as the search backend in `~/.hermes/config.yaml`:
```yaml
web:
search_backend: "searxng"
```
Or set via `hermes tools` → Web Search & Extract → SearXNG.
---
#### Option B — Use a public instance
Public SearXNG instances are listed at [searx.space](https://searx.space/). Filter by instances that have **JSON format enabled** (shown in the table).
```bash
# ~/.hermes/.env
SEARXNG_URL=https://searx.example.com
```
:::caution Public instances
Public instances have rate limits, variable uptime, and may disable JSON format at any time. For production use, self-hosting is strongly recommended.
:::
---
#### Pair SearXNG with an extract provider
SearXNG handles search; you need a separate provider for `web_extract`. Use the per-capability keys:
```yaml
# ~/.hermes/config.yaml
web:
search_backend: "searxng"
extract_backend: "firecrawl" # or tavily, exa, parallel
```
With this config, Hermes uses SearXNG for all search queries and Firecrawl for URL extraction — combining free search with high-quality extraction.
---
### Tavily
AI-optimised search and extract with a generous free tier.
```bash
# ~/.hermes/.env
TAVILY_API_KEY=tvly-your-key-here
```
Get a key at [app.tavily.com](https://app.tavily.com/home). The free tier includes 1 000 searches/month.
---
### Exa
Neural search with semantic understanding. Good for research and finding conceptually related content.
```bash
# ~/.hermes/.env
EXA_API_KEY=your-exa-key-here
```
Get a key at [exa.ai](https://exa.ai). The free tier includes 1 000 searches/month.
---
### Parallel
AI-native search and extraction with deep research capabilities.
```bash
# ~/.hermes/.env
PARALLEL_API_KEY=your-parallel-key-here
```
Get access at [parallel.ai](https://parallel.ai).
---
### xAI (Grok) {#xai-grok}
Routes `web_search` through Grok's server-side [web_search tool](https://docs.x.ai/developers/tools/web-search) on the Responses API. Grok runs the actual searching and returns the top results as structured JSON.
Works with either credential path — no new env vars, no new setup wizard:
```bash
# ~/.hermes/.env (env-var path)
XAI_API_KEY=sk-xai-your-key-here
```
or for SuperGrok subscribers:
```bash
hermes auth login xai-oauth
```
Then select xAI as the search backend:
```yaml
# ~/.hermes/config.yaml
web:
backend: "xai"
```
**Optional knobs:**
```yaml
web:
backend: "xai"
xai:
model: grok-4.3 # reasoning model required by web_search (default)
allowed_domains: # optional, max 5 — mutex with excluded_domains
- arxiv.org
excluded_domains: # optional, max 5
- example-spam.com
timeout: 90 # seconds (default)
```
**Search-only** — pair with Firecrawl / Tavily / Exa / Parallel if you also need `web_extract`. On 401 the provider performs a single forced OAuth-token refresh and retries (covers mid-window revocation and opaque tokens the proactive expiry check can't decode); env-var credentials skip the retry.
:::caution Trust model
Unlike index-backed providers (Brave, Tavily, Exa) which return verbatim search-engine results, xAI is an LLM choosing which URLs to surface and writing the titles and descriptions itself. The *content* of the query influences the output, so a maliciously crafted query (e.g. injected via untrusted upstream input the agent picked up) can in principle steer Grok into emitting attacker-chosen URLs. Treat returned URLs the same way you'd treat any model-generated link — validate before fetching, especially if the query came from untrusted input.
:::
---
## Configuration
### Single backend
Set one provider for all web capabilities:
```yaml
# ~/.hermes/config.yaml
web:
backend: "searxng" # firecrawl | searxng | brave-free | ddgs | tavily | exa | parallel | xai
```
### Per-capability configuration
Use different providers for search vs extract. This lets you combine free search (SearXNG) with a paid extract provider, or vice versa:
```yaml
# ~/.hermes/config.yaml
web:
search_backend: "searxng" # used by web_search
extract_backend: "firecrawl" # used by web_extract
```
When per-capability keys are empty, both fall through to `web.backend`. When `web.backend` is also empty, the backend is auto-detected from whichever API key/URL is present.
**Priority order (per capability):**
1. `web.search_backend` / `web.extract_backend` (explicit per-capability)
2. `web.backend` (shared fallback)
3. Auto-detect from environment variables
### Auto-detection
If no backend is explicitly configured, Hermes picks the first available one based on which credentials are set:
| Credential present | Auto-selected backend |
|--------------------|-----------------------|
| `FIRECRAWL_API_KEY` or `FIRECRAWL_API_URL` | firecrawl |
| `PARALLEL_API_KEY` | parallel |
| `TAVILY_API_KEY` | tavily |
| `EXA_API_KEY` | exa |
| `SEARXNG_URL` | searxng |
xAI Web Search is **not** in the auto-detection chain — having `XAI_API_KEY` set (or being signed in via xAI Grok OAuth) does not automatically route web traffic through xAI, since those credentials are also used for inference / TTS / image gen and the user may want a different backend for web. Opt in explicitly with `web.backend: "xai"`.
---
## Verify your setup
Run `hermes setup` to see which web backend is detected:
```
✅ Web Search & Extract (searxng)
```
Or check via the CLI:
```bash
# Activate the venv and run the web tools module directly
source ~/.hermes/hermes-agent/.venv/bin/activate
python -m tools.web_tools
```
This prints the active backend and its status:
```
✅ Web backend: searxng
Using SearXNG (search only): http://localhost:8888
```
---
## Troubleshooting
### `web_search` returns `{"success": false}`
- Check `SEARXNG_URL` is reachable: `curl -s "http://localhost:8888/search?q=test&format=json"`
- If you get HTTP 403, JSON format is disabled — add `json` to the `formats` list in `settings.yml` and restart
- If you get a connection error, the container may not be running: `docker ps | grep searxng`
### `web_extract` says "search-only backend"
SearXNG cannot extract URL content. Set `web.extract_backend` to a provider that supports extraction:
```yaml
web:
search_backend: "searxng"
extract_backend: "firecrawl" # or tavily / exa / parallel
```
### SearXNG returns 0 results
Some public instances disable certain search engines or categories. Try:
- A different query
- A different public instance from [searx.space](https://searx.space/)
- Self-hosting your own instance for reliable results
### Rate limited on a public instance
Switch to a self-hosted instance (see [Option A](#option-a--self-host-with-docker-recommended) above). With Docker, your own instance has no rate limits.
### `web_extract` returns truncated content with a "summarization timed out" note
The auxiliary model didn't finish summarizing within the configured timeout. Either:
- Raise `auxiliary.web_extract.timeout` in `config.yaml` (default 360s on fresh installs, 30s if the key is missing)
- Switch the `web_extract` auxiliary task to a faster model (e.g. `google/gemini-3-flash-preview`) — see [How `web_extract` handles long pages](#how-web_extract-handles-long-pages)
- For pages where summarization is the wrong tool, use `browser_navigate` instead
---
## Optional skill: `searxng-search`
For agents that need to use SearXNG via `curl` directly (e.g. as a fallback when the web toolset isn't available), install the `searxng-search` optional skill:
```bash
hermes skills install official/research/searxng-search
```
This adds a skill that teaches the agent how to:
- Call the SearXNG JSON API via `curl` or Python
- Filter by category (`general`, `news`, `science`, etc.)
- Handle pagination and error cases
- Fall back gracefully when SearXNG is unreachable
website/docs/user-guide/features/x-search.md
+144
View File
@@ -0,0 +1,144 @@
---
title: X (Twitter) Search
description: Search X (Twitter) posts and threads from within the agent using xAI's built-in x_search Responses tool — works with either a SuperGrok OAuth login or an XAI_API_KEY.
sidebar_label: X (Twitter) Search
sidebar_position: 7
---
# X (Twitter) Search
The `x_search` tool lets the agent search X (Twitter) posts, profiles, and threads directly. It's backed by xAI's built-in `x_search` tool on the Responses API at `https://api.x.ai/v1/responses` — Grok itself runs the search server-side and returns synthesized results with citations to the originating posts.
**Use this instead of `web_search`** when you specifically want current discussion, reactions, or claims **on X**. For general web pages, keep using `web_search` / `web_extract`.
:::tip
If you're paying Portal for an xAI model anyway, Live Search calls bill against the same xAI key configured for chat. See [Nous Portal](/integrations/nous-portal).
:::
## Authentication
`x_search` registers when **either** xAI credential path is available:
| Credential | Source | Setup |
|------------|--------|-------|
| **SuperGrok / X Premium+ OAuth** (preferred) | Browser login at `accounts.x.ai`, refreshed automatically | `hermes auth add xai-oauth` — see [xAI Grok OAuth (SuperGrok / X Premium+)](../../guides/xai-grok-oauth.md) |
| **`XAI_API_KEY`** | Paid xAI API key | Set in `~/.hermes/.env` |
Both hit the same endpoint with the same payload — the only difference is the bearer token. **When both are configured, SuperGrok OAuth wins** so x_search runs against your subscription quota instead of paid API spend.
The tool's `check_fn` runs the xAI credential resolver every time the model's tool list is rebuilt. A `True` return means the bearer is fetchable AND non-empty AND (if it had expired) successfully refreshed. Revoked tokens with a failed refresh hide the tool from the schema; the model simply can't see it.
## Enabling the tool
Auto-enables when xAI credentials (OAuth token or `XAI_API_KEY`) are present. Disable explicitly via `hermes tools` → Search → x_search if you don't want this.
```bash
hermes tools
# → 🐦 X (Twitter) Search (press space to toggle on)
```
The picker offers two credential choices:
1. **xAI Grok OAuth (SuperGrok / Premium+)** — opens the browser to `accounts.x.ai` if you're not already logged in
2. **xAI API key** — prompts for `XAI_API_KEY`
Either choice satisfies the gating. You can pick whichever credentials you already have; the tool works identically with both. If both end up configured, OAuth is preferred at call time.
## Configuration
```yaml
# ~/.hermes/config.yaml
x_search:
# xAI model used for the Responses call.
# grok-4.20-reasoning is the recommended default; any Grok model
# with x_search tool access works.
model: grok-4.20-reasoning
# Request timeout in seconds. x_search can take 60–120s for
# complex queries — the default is generous. Minimum: 30.
timeout_seconds: 180
# Number of automatic retries on 5xx / ReadTimeout / ConnectionError.
# Each retry backs off (1.5x attempt seconds, capped at 5s).
retries: 2
```
## Tool parameters
The agent calls `x_search` with these arguments:
| Parameter | Type | Description |
|-----------|------|-------------|
| `query` | string (required) | What to look up on X. |
| `allowed_x_handles` | string array | Optional list of handles to include **exclusively** (max 10). Leading `@` is stripped. |
| `excluded_x_handles` | string array | Optional list of handles to exclude (max 10). Mutually exclusive with `allowed_x_handles`. |
| `from_date` | string | Optional `YYYY-MM-DD` start date. |
| `to_date` | string | Optional `YYYY-MM-DD` end date. |
| `enable_image_understanding` | boolean | Ask xAI to analyze images attached to matching posts. |
| `enable_video_understanding` | boolean | Ask xAI to analyze videos attached to matching posts. |
The tool returns JSON with:
- `answer` — synthesized text response from Grok
- `citations` — citations returned by the Responses API top-level field
- `inline_citations` — `url_citation` annotations extracted from the message body (each with `url`, `title`, `start_index`, `end_index`)
- `degraded` — `true` when any narrowing filter (`allowed_x_handles`, `excluded_x_handles`, `from_date`, `to_date`) was set AND both citation channels came back empty. In that case the `answer` was synthesized from the model's own knowledge rather than the X index, so treat it as unsourced. `false` otherwise (including the "no filters set" case — a broad unsourced answer is just an answer, not a filter miss)
- `degraded_reason` — short string naming which filters were active, or `null` when `degraded` is `false`
- `credential_source` — `"xai-oauth"` if OAuth resolved, `"xai"` if API key resolved
- `model`, `query`, `provider`, `tool`, `success`
### Date validation
`from_date` / `to_date` are validated client-side before the HTTP call:
- Both, if provided, must parse as `YYYY-MM-DD`.
- When both are set, `from_date` must be on or before `to_date`.
- `from_date` must not be later than today UTC — no posts can exist in a window that hasn't started yet, so the call would be guaranteed to return zero citations.
- `to_date` in the future is allowed (callers may legitimately request "from yesterday to tomorrow" to catch posts as they arrive).
Validation failures surface as a structured `{"error": "..."}` tool result, never as an HTTP call to xAI.
## Example
Talking to the agent:
> What are people on X saying about the new Grok image features? Focus on responses from @xai.
The agent will:
1. Call `x_search` with `query="reactions to new Grok image features"`, `allowed_x_handles=["xai"]`
2. Get back a synthesized answer plus a list of citations linking to specific posts
3. Reply with the answer and references
## Troubleshooting
### "No xAI credentials available"
The tool surfaces this when both auth paths fail. Either set `XAI_API_KEY` in `~/.hermes/.env` or run `hermes auth add xai-oauth` and complete the browser login. Then restart your session so the agent re-reads the tool registry.
### "`x_search` is not enabled for this model"
The configured `x_search.model` doesn't have access to the server-side `x_search` tool. Switch to `grok-4.20-reasoning` (the default) or another Grok model that supports it. Check the [xAI documentation](https://docs.x.ai/) for the current list.
### Tool doesn't appear in the schema
Two possible causes:
1. **Toolset not enabled.** Run `hermes tools` and confirm `🐦 X (Twitter) Search` is checked.
2. **No xAI credentials.** The check_fn returns False, so the schema stays hidden. Run `hermes auth status` to confirm xai-oauth login state, and check that `XAI_API_KEY` is set (if you're using the API-key path).
### `degraded: true` — answer with no citations
When you used `allowed_x_handles`, `excluded_x_handles`, or a date range and the response comes back with `degraded: true`, xAI's X index returned no matching posts but Grok still produced a synthesized answer from its own training data. The answer is unsourced — do not treat it as a real X result.
Causes worth checking:
- **Typo in the handle.** Strip the `@`, double-check spelling, and confirm the account exists.
- **Date range too narrow** or sliding past today's posts; widen and retry.
- **xAI index gap.** Some active accounts intermittently fail to surface in `x_search` even when they post regularly. Retry after a few minutes, or use the `xurl` skill for direct X API reads when you need an exact handle's timeline.
## See Also
- [xAI Grok OAuth (SuperGrok / Premium+)](../../guides/xai-grok-oauth.md) — the OAuth setup guide
- [Web Search & Extract](web-search.md) — for general (non-X) web search
- [Tools Reference](../../reference/tools-reference.md) — full tool catalog