Hermes-agent
This commit is contained in:
Zakaria
@@ -0,0 +1,156 @@
|
||||
---
|
||||
name: inference-sh-cli
|
||||
description: "Run 150+ AI apps via inference.sh CLI (infsh) — image generation, video creation, LLMs, search, 3D, social automation. Uses the terminal tool. Triggers: inference.sh, infsh, ai apps, flux, veo, image generation, video generation, seedream, seedance, tavily"
|
||||
version: 1.0.0
|
||||
author: okaris
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [AI, image-generation, video, LLM, search, inference, FLUX, Veo, Claude]
|
||||
related_skills: []
|
||||
---
|
||||
|
||||
# inference.sh CLI
|
||||
|
||||
Run 150+ AI apps in the cloud with a simple CLI. No GPU required.
|
||||
|
||||
All commands use the **terminal tool** to run `infsh` commands.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks to generate images (FLUX, Reve, Seedream, Grok, Gemini image)
|
||||
- User asks to generate video (Veo, Wan, Seedance, OmniHuman)
|
||||
- User asks about inference.sh or infsh
|
||||
- User wants to run AI apps without managing individual provider APIs
|
||||
- User asks for AI-powered search (Tavily, Exa)
|
||||
- User needs avatar/lipsync generation
|
||||
|
||||
## Prerequisites
|
||||
|
||||
The `infsh` CLI must be installed and authenticated. Check with:
|
||||
|
||||
```bash
|
||||
infsh me
|
||||
```
|
||||
|
||||
If not installed:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://cli.inference.sh | sh
|
||||
infsh login
|
||||
```
|
||||
|
||||
See `references/authentication.md` for full setup details.
|
||||
|
||||
## Workflow
|
||||
|
||||
### 1. Always Search First
|
||||
|
||||
Never guess app names — always search to find the correct app ID:
|
||||
|
||||
```bash
|
||||
infsh app list --search flux
|
||||
infsh app list --search video
|
||||
infsh app list --search image
|
||||
```
|
||||
|
||||
### 2. Run an App
|
||||
|
||||
Use the exact app ID from the search results. Always use `--json` for machine-readable output:
|
||||
|
||||
```bash
|
||||
infsh app run <app-id> --input '{"prompt": "your prompt here"}' --json
|
||||
```
|
||||
|
||||
### 3. Parse the Output
|
||||
|
||||
The JSON output contains URLs to generated media. Present these to the user with `MEDIA:<url>` for inline display.
|
||||
|
||||
## Common Commands
|
||||
|
||||
### Image Generation
|
||||
|
||||
```bash
|
||||
# Search for image apps
|
||||
infsh app list --search image
|
||||
|
||||
# FLUX Dev with LoRA
|
||||
infsh app run falai/flux-dev-lora --input '{"prompt": "sunset over mountains", "num_images": 1}' --json
|
||||
|
||||
# Gemini image generation
|
||||
infsh app run google/gemini-2-5-flash-image --input '{"prompt": "futuristic city", "num_images": 1}' --json
|
||||
|
||||
# Seedream (ByteDance)
|
||||
infsh app run bytedance/seedream-5-lite --input '{"prompt": "nature scene"}' --json
|
||||
|
||||
# Grok Imagine (xAI)
|
||||
infsh app run xai/grok-imagine-image --input '{"prompt": "abstract art"}' --json
|
||||
```
|
||||
|
||||
### Video Generation
|
||||
|
||||
```bash
|
||||
# Search for video apps
|
||||
infsh app list --search video
|
||||
|
||||
# Veo 3.1 (Google)
|
||||
infsh app run google/veo-3-1-fast --input '{"prompt": "drone shot of coastline"}' --json
|
||||
|
||||
# Seedance (ByteDance)
|
||||
infsh app run bytedance/seedance-1-5-pro --input '{"prompt": "dancing figure", "resolution": "1080p"}' --json
|
||||
|
||||
# Wan 2.5
|
||||
infsh app run falai/wan-2-5 --input '{"prompt": "person walking through city"}' --json
|
||||
```
|
||||
|
||||
### Local File Uploads
|
||||
|
||||
The CLI automatically uploads local files when you provide a path:
|
||||
|
||||
```bash
|
||||
# Upscale a local image
|
||||
infsh app run falai/topaz-image-upscaler --input '{"image": "/path/to/photo.jpg", "upscale_factor": 2}' --json
|
||||
|
||||
# Image-to-video from local file
|
||||
infsh app run falai/wan-2-5-i2v --input '{"image": "/path/to/image.png", "prompt": "make it move"}' --json
|
||||
|
||||
# Avatar with audio
|
||||
infsh app run bytedance/omnihuman-1-5 --input '{"audio": "/path/to/audio.mp3", "image": "/path/to/face.jpg"}' --json
|
||||
```
|
||||
|
||||
### Search & Research
|
||||
|
||||
```bash
|
||||
infsh app list --search search
|
||||
infsh app run tavily/tavily-search --input '{"query": "latest AI news"}' --json
|
||||
infsh app run exa/exa-search --input '{"query": "machine learning papers"}' --json
|
||||
```
|
||||
|
||||
### Other Categories
|
||||
|
||||
```bash
|
||||
# 3D generation
|
||||
infsh app list --search 3d
|
||||
|
||||
# Audio / TTS
|
||||
infsh app list --search tts
|
||||
|
||||
# Twitter/X automation
|
||||
infsh app list --search twitter
|
||||
```
|
||||
|
||||
## Pitfalls
|
||||
|
||||
1. **Never guess app IDs** — always run `infsh app list --search <term>` first. App IDs change and new apps are added frequently.
|
||||
2. **Always use `--json`** — raw output is hard to parse. The `--json` flag gives structured output with URLs.
|
||||
3. **Check authentication** — if commands fail with auth errors, run `infsh login` or verify `INFSH_API_KEY` is set.
|
||||
4. **Long-running apps** — video generation can take 30-120 seconds. The terminal tool timeout should be sufficient, but warn the user it may take a moment.
|
||||
5. **Input format** — the `--input` flag takes a JSON string. Make sure to properly escape quotes.
|
||||
|
||||
## Reference Docs
|
||||
|
||||
- `references/authentication.md` — Setup, login, API keys
|
||||
- `references/app-discovery.md` — Searching and browsing the app catalog
|
||||
- `references/running-apps.md` — Running apps, input formats, output handling
|
||||
- `references/cli-reference.md` — Complete CLI command reference
|
||||
@@ -0,0 +1,112 @@
|
||||
# Discovering Apps
|
||||
|
||||
## List All Apps
|
||||
|
||||
```bash
|
||||
infsh app list
|
||||
```
|
||||
|
||||
## Pagination
|
||||
|
||||
```bash
|
||||
infsh app list --page 2
|
||||
```
|
||||
|
||||
## Filter by Category
|
||||
|
||||
```bash
|
||||
infsh app list --category image
|
||||
infsh app list --category video
|
||||
infsh app list --category audio
|
||||
infsh app list --category text
|
||||
infsh app list --category other
|
||||
```
|
||||
|
||||
## Search
|
||||
|
||||
```bash
|
||||
infsh app search "flux"
|
||||
infsh app search "video generation"
|
||||
infsh app search "tts" -l
|
||||
infsh app search "image" --category image
|
||||
```
|
||||
|
||||
Or use the flag form:
|
||||
|
||||
```bash
|
||||
infsh app list --search "flux"
|
||||
infsh app list --search "video generation"
|
||||
infsh app list --search "tts"
|
||||
```
|
||||
|
||||
## Featured Apps
|
||||
|
||||
```bash
|
||||
infsh app list --featured
|
||||
```
|
||||
|
||||
## Newest First
|
||||
|
||||
```bash
|
||||
infsh app list --new
|
||||
```
|
||||
|
||||
## Detailed View
|
||||
|
||||
```bash
|
||||
infsh app list -l
|
||||
```
|
||||
|
||||
Shows table with app name, category, description, and featured status.
|
||||
|
||||
## Save to File
|
||||
|
||||
```bash
|
||||
infsh app list --save apps.json
|
||||
```
|
||||
|
||||
## Your Apps
|
||||
|
||||
List apps you've deployed:
|
||||
|
||||
```bash
|
||||
infsh app my
|
||||
infsh app my -l # detailed
|
||||
```
|
||||
|
||||
## Get App Details
|
||||
|
||||
```bash
|
||||
infsh app get falai/flux-dev-lora
|
||||
infsh app get falai/flux-dev-lora --json
|
||||
```
|
||||
|
||||
Shows full app info including input/output schema.
|
||||
|
||||
## Popular Apps by Category
|
||||
|
||||
### Image Generation
|
||||
- `falai/flux-dev-lora` - FLUX.2 Dev (high quality)
|
||||
- `falai/flux-2-klein-lora` - FLUX.2 Klein (fastest)
|
||||
- `infsh/sdxl` - Stable Diffusion XL
|
||||
- `google/gemini-3-pro-image-preview` - Gemini 3 Pro
|
||||
- `xai/grok-imagine-image` - Grok image generation
|
||||
|
||||
### Video Generation
|
||||
- `google/veo-3-1-fast` - Veo 3.1 Fast
|
||||
- `google/veo-3` - Veo 3
|
||||
- `bytedance/seedance-1-5-pro` - Seedance 1.5 Pro
|
||||
- `infsh/ltx-video-2` - LTX Video 2 (with audio)
|
||||
- `bytedance/omnihuman-1-5` - OmniHuman avatar
|
||||
|
||||
### Audio
|
||||
- `infsh/dia-tts` - Conversational TTS
|
||||
- `infsh/kokoro-tts` - Kokoro TTS
|
||||
- `infsh/fast-whisper-large-v3` - Fast transcription
|
||||
- `infsh/diffrythm` - Music generation
|
||||
|
||||
## Documentation
|
||||
|
||||
- [Browsing the Grid](https://inference.sh/docs/apps/browsing-grid) - Visual app browsing
|
||||
- [Apps Overview](https://inference.sh/docs/apps/overview) - Understanding apps
|
||||
- [Running Apps](https://inference.sh/docs/apps/running) - How to run apps
|
||||
@@ -0,0 +1,59 @@
|
||||
# Authentication & Setup
|
||||
|
||||
## Install the CLI
|
||||
|
||||
```bash
|
||||
curl -fsSL https://cli.inference.sh | sh
|
||||
```
|
||||
|
||||
## Login
|
||||
|
||||
```bash
|
||||
infsh login
|
||||
```
|
||||
|
||||
This opens a browser for authentication. After login, credentials are stored locally.
|
||||
|
||||
## Check Authentication
|
||||
|
||||
```bash
|
||||
infsh me
|
||||
```
|
||||
|
||||
Shows your user info if authenticated.
|
||||
|
||||
## Environment Variable
|
||||
|
||||
For CI/CD or scripts, set your API key:
|
||||
|
||||
```bash
|
||||
export INFSH_API_KEY=your-api-key
|
||||
```
|
||||
|
||||
The environment variable overrides the config file.
|
||||
|
||||
## Update CLI
|
||||
|
||||
```bash
|
||||
infsh update
|
||||
```
|
||||
|
||||
Or reinstall:
|
||||
|
||||
```bash
|
||||
curl -fsSL https://cli.inference.sh | sh
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
| Error | Solution |
|
||||
|-------|----------|
|
||||
| "not authenticated" | Run `infsh login` |
|
||||
| "command not found" | Reinstall CLI or add to PATH |
|
||||
| "API key invalid" | Check `INFSH_API_KEY` or re-login |
|
||||
|
||||
## Documentation
|
||||
|
||||
- [CLI Setup](https://inference.sh/docs/extend/cli-setup) - Complete CLI installation guide
|
||||
- [API Authentication](https://inference.sh/docs/api/authentication) - API key management
|
||||
- [Secrets](https://inference.sh/docs/secrets/overview) - Managing credentials
|
||||
@@ -0,0 +1,104 @@
|
||||
# CLI Reference
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
curl -fsSL https://cli.inference.sh | sh
|
||||
```
|
||||
|
||||
## Global Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `infsh help` | Show help |
|
||||
| `infsh version` | Show CLI version |
|
||||
| `infsh update` | Update CLI to latest |
|
||||
| `infsh login` | Authenticate |
|
||||
| `infsh me` | Show current user |
|
||||
|
||||
## App Commands
|
||||
|
||||
### Discovery
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `infsh app list` | List available apps |
|
||||
| `infsh app list --category <cat>` | Filter by category (image, video, audio, text, other) |
|
||||
| `infsh app search <query>` | Search apps |
|
||||
| `infsh app list --search <query>` | Search apps (flag form) |
|
||||
| `infsh app list --featured` | Show featured apps |
|
||||
| `infsh app list --new` | Sort by newest |
|
||||
| `infsh app list --page <n>` | Pagination |
|
||||
| `infsh app list -l` | Detailed table view |
|
||||
| `infsh app list --save <file>` | Save to JSON file |
|
||||
| `infsh app my` | List your deployed apps |
|
||||
| `infsh app get <app>` | Get app details |
|
||||
| `infsh app get <app> --json` | Get app details as JSON |
|
||||
|
||||
### Execution
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `infsh app run <app> --input <file>` | Run app with input file |
|
||||
| `infsh app run <app> --input '<json>'` | Run with inline JSON |
|
||||
| `infsh app run <app> --input <file> --no-wait` | Run without waiting for completion |
|
||||
| `infsh app sample <app>` | Show sample input |
|
||||
| `infsh app sample <app> --save <file>` | Save sample to file |
|
||||
|
||||
## Task Commands
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `infsh task get <task-id>` | Get task status and result |
|
||||
| `infsh task get <task-id> --json` | Get task as JSON |
|
||||
| `infsh task get <task-id> --save <file>` | Save task result to file |
|
||||
|
||||
### Development
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `infsh app init` | Create new app (interactive) |
|
||||
| `infsh app init <name>` | Create new app with name |
|
||||
| `infsh app test --input <file>` | Test app locally |
|
||||
| `infsh app deploy` | Deploy app |
|
||||
| `infsh app deploy --dry-run` | Validate without deploying |
|
||||
| `infsh app pull <id>` | Pull app source |
|
||||
| `infsh app pull --all` | Pull all your apps |
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Description |
|
||||
|----------|-------------|
|
||||
| `INFSH_API_KEY` | API key (overrides config) |
|
||||
|
||||
## Shell Completions
|
||||
|
||||
```bash
|
||||
# Bash
|
||||
infsh completion bash > /etc/bash_completion.d/infsh
|
||||
|
||||
# Zsh
|
||||
infsh completion zsh > "${fpath[1]}/_infsh"
|
||||
|
||||
# Fish
|
||||
infsh completion fish > ~/.config/fish/completions/infsh.fish
|
||||
```
|
||||
|
||||
## App Name Format
|
||||
|
||||
Apps use the format `namespace/app-name`:
|
||||
|
||||
- `falai/flux-dev-lora` - fal.ai's FLUX 2 Dev
|
||||
- `google/veo-3` - Google's Veo 3
|
||||
- `infsh/sdxl` - inference.sh's SDXL
|
||||
- `bytedance/seedance-1-5-pro` - ByteDance's Seedance
|
||||
- `xai/grok-imagine-image` - xAI's Grok
|
||||
|
||||
Version pinning: `namespace/app-name@version`
|
||||
|
||||
## Documentation
|
||||
|
||||
- [CLI Setup](https://inference.sh/docs/extend/cli-setup) - Complete CLI installation guide
|
||||
- [Running Apps](https://inference.sh/docs/apps/running) - How to run apps via CLI
|
||||
- [Creating an App](https://inference.sh/docs/extend/creating-app) - Build your own apps
|
||||
- [Deploying](https://inference.sh/docs/extend/deploying) - Deploy apps to the cloud
|
||||
@@ -0,0 +1,171 @@
|
||||
# Running Apps
|
||||
|
||||
## Basic Run
|
||||
|
||||
```bash
|
||||
infsh app run user/app-name --input input.json
|
||||
```
|
||||
|
||||
## Inline JSON
|
||||
|
||||
```bash
|
||||
infsh app run falai/flux-dev-lora --input '{"prompt": "a sunset over mountains"}'
|
||||
```
|
||||
|
||||
## Version Pinning
|
||||
|
||||
```bash
|
||||
infsh app run user/app-name@1.0.0 --input input.json
|
||||
```
|
||||
|
||||
## Local File Uploads
|
||||
|
||||
The CLI automatically uploads local files when you provide a file path instead of a URL. Any field that accepts a URL also accepts a local path:
|
||||
|
||||
```bash
|
||||
# Upscale a local image
|
||||
infsh app run falai/topaz-image-upscaler --input '{"image": "/path/to/photo.jpg", "upscale_factor": 2}'
|
||||
|
||||
# Image-to-video from local file
|
||||
infsh app run falai/wan-2-5-i2v --input '{"image": "./my-image.png", "prompt": "make it move"}'
|
||||
|
||||
# Avatar with local audio and image
|
||||
infsh app run bytedance/omnihuman-1-5 --input '{"audio": "/path/to/speech.mp3", "image": "/path/to/face.jpg"}'
|
||||
|
||||
# Post tweet with local media
|
||||
infsh app run x/post-create --input '{"text": "Check this out!", "media": "./screenshot.png"}'
|
||||
```
|
||||
|
||||
Supported paths:
|
||||
- Absolute paths: `/home/user/images/photo.jpg`
|
||||
- Relative paths: `./image.png`, `../data/video.mp4`
|
||||
- Home directory: `~/Pictures/photo.jpg`
|
||||
|
||||
## Generate Sample Input
|
||||
|
||||
Before running, generate a sample input file:
|
||||
|
||||
```bash
|
||||
infsh app sample falai/flux-dev-lora
|
||||
```
|
||||
|
||||
Save to file:
|
||||
|
||||
```bash
|
||||
infsh app sample falai/flux-dev-lora --save input.json
|
||||
```
|
||||
|
||||
Then edit `input.json` and run:
|
||||
|
||||
```bash
|
||||
infsh app run falai/flux-dev-lora --input input.json
|
||||
```
|
||||
|
||||
## Workflow Example
|
||||
|
||||
### Image Generation with FLUX
|
||||
|
||||
```bash
|
||||
# 1. Get app details
|
||||
infsh app get falai/flux-dev-lora
|
||||
|
||||
# 2. Generate sample input
|
||||
infsh app sample falai/flux-dev-lora --save input.json
|
||||
|
||||
# 3. Edit input.json
|
||||
# {
|
||||
# "prompt": "a cat astronaut floating in space",
|
||||
# "num_images": 1,
|
||||
# "image_size": "landscape_16_9"
|
||||
# }
|
||||
|
||||
# 4. Run
|
||||
infsh app run falai/flux-dev-lora --input input.json
|
||||
```
|
||||
|
||||
### Video Generation with Veo
|
||||
|
||||
```bash
|
||||
# 1. Generate sample
|
||||
infsh app sample google/veo-3-1-fast --save input.json
|
||||
|
||||
# 2. Edit prompt
|
||||
# {
|
||||
# "prompt": "A drone shot flying over a forest at sunset"
|
||||
# }
|
||||
|
||||
# 3. Run
|
||||
infsh app run google/veo-3-1-fast --input input.json
|
||||
```
|
||||
|
||||
### Text-to-Speech
|
||||
|
||||
```bash
|
||||
# Quick inline run
|
||||
infsh app run falai/kokoro-tts --input '{"text": "Hello, this is a test."}'
|
||||
```
|
||||
|
||||
## Task Tracking
|
||||
|
||||
When you run an app, the CLI shows the task ID:
|
||||
|
||||
```
|
||||
Running falai/flux-dev-lora
|
||||
Task ID: abc123def456
|
||||
```
|
||||
|
||||
For long-running tasks, you can check status anytime:
|
||||
|
||||
```bash
|
||||
# Check task status
|
||||
infsh task get abc123def456
|
||||
|
||||
# Get result as JSON
|
||||
infsh task get abc123def456 --json
|
||||
|
||||
# Save result to file
|
||||
infsh task get abc123def456 --save result.json
|
||||
```
|
||||
|
||||
### Run Without Waiting
|
||||
|
||||
For very long tasks, run in background:
|
||||
|
||||
```bash
|
||||
# Submit and return immediately
|
||||
infsh app run google/veo-3 --input input.json --no-wait
|
||||
|
||||
# Check later
|
||||
infsh task get <task-id>
|
||||
```
|
||||
|
||||
## Output
|
||||
|
||||
The CLI returns the app output directly. For file outputs (images, videos, audio), you'll receive URLs to download.
|
||||
|
||||
Example output:
|
||||
|
||||
```json
|
||||
{
|
||||
"images": [
|
||||
{
|
||||
"url": "https://cloud.inference.sh/...",
|
||||
"content_type": "image/png"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
| Error | Cause | Solution |
|
||||
|-------|-------|----------|
|
||||
| "invalid input" | Schema mismatch | Check `infsh app get` for required fields |
|
||||
| "app not found" | Wrong app name | Check `infsh app list --search` |
|
||||
| "quota exceeded" | Out of credits | Check account balance |
|
||||
|
||||
## Documentation
|
||||
|
||||
- [Running Apps](https://inference.sh/docs/apps/running) - Complete running apps guide
|
||||
- [Streaming Results](https://inference.sh/docs/api/sdk/streaming) - Real-time progress updates
|
||||
- [Setup Parameters](https://inference.sh/docs/apps/setup-parameters) - Configuring app inputs
|
||||
+281
@@ -0,0 +1,281 @@
|
||||
---
|
||||
name: docker-management
|
||||
description: Manage Docker containers, images, volumes, networks, and Compose stacks — lifecycle ops, debugging, cleanup, and Dockerfile optimization.
|
||||
version: 1.0.0
|
||||
author: sprmn24
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [docker, containers, devops, infrastructure, compose, images, volumes, networks, debugging]
|
||||
category: devops
|
||||
requires_toolsets: [terminal]
|
||||
---
|
||||
|
||||
# Docker Management
|
||||
|
||||
Manage Docker containers, images, volumes, networks, and Compose stacks using standard Docker CLI commands. No additional dependencies beyond Docker itself.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Run, stop, restart, remove, or inspect containers
|
||||
- Build, pull, push, tag, or clean up Docker images
|
||||
- Work with Docker Compose (multi-service stacks)
|
||||
- Manage volumes or networks
|
||||
- Debug a crashing container or analyze logs
|
||||
- Check Docker disk usage or free up space
|
||||
- Review or optimize a Dockerfile
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Docker Engine installed and running
|
||||
- User added to the `docker` group (or use `sudo`)
|
||||
- Docker Compose v2 (included with modern Docker installations)
|
||||
|
||||
Quick check:
|
||||
|
||||
```bash
|
||||
docker --version && docker compose version
|
||||
```
|
||||
|
||||
## Quick Reference
|
||||
|
||||
| Task | Command |
|
||||
|------|---------|
|
||||
| Run container (background) | `docker run -d --name NAME IMAGE` |
|
||||
| Stop + remove | `docker stop NAME && docker rm NAME` |
|
||||
| View logs (follow) | `docker logs --tail 50 -f NAME` |
|
||||
| Shell into container | `docker exec -it NAME /bin/sh` |
|
||||
| List all containers | `docker ps -a` |
|
||||
| Build image | `docker build -t TAG .` |
|
||||
| Compose up | `docker compose up -d` |
|
||||
| Compose down | `docker compose down` |
|
||||
| Disk usage | `docker system df` |
|
||||
| Cleanup dangling | `docker image prune && docker container prune` |
|
||||
|
||||
## Procedure
|
||||
|
||||
### 1. Identify the domain
|
||||
|
||||
Figure out which area the request falls into:
|
||||
|
||||
- **Container lifecycle** → run, stop, start, restart, rm, pause/unpause
|
||||
- **Container interaction** → exec, cp, logs, inspect, stats
|
||||
- **Image management** → build, pull, push, tag, rmi, save/load
|
||||
- **Docker Compose** → up, down, ps, logs, exec, build, config
|
||||
- **Volumes & networks** → create, inspect, rm, prune, connect
|
||||
- **Troubleshooting** → log analysis, exit codes, resource issues
|
||||
|
||||
### 2. Container operations
|
||||
|
||||
**Run a new container:**
|
||||
|
||||
```bash
|
||||
# Detached service with port mapping
|
||||
docker run -d --name web -p 8080:80 nginx
|
||||
|
||||
# With environment variables
|
||||
docker run -d -e POSTGRES_PASSWORD=secret -e POSTGRES_DB=mydb --name db postgres:16
|
||||
|
||||
# With persistent data (named volume)
|
||||
docker run -d -v pgdata:/var/lib/postgresql/data --name db postgres:16
|
||||
|
||||
# For development (bind mount source code)
|
||||
docker run -d -v $(pwd)/src:/app/src -p 3000:3000 --name dev my-app
|
||||
|
||||
# Interactive debugging (auto-remove on exit)
|
||||
docker run -it --rm ubuntu:22.04 /bin/bash
|
||||
|
||||
# With resource limits and restart policy
|
||||
docker run -d --memory=512m --cpus=1.5 --restart=unless-stopped --name app my-app
|
||||
```
|
||||
|
||||
Key flags: `-d` detached, `-it` interactive+tty, `--rm` auto-remove, `-p` port (host:container), `-e` env var, `-v` volume, `--name` name, `--restart` restart policy.
|
||||
|
||||
**Manage running containers:**
|
||||
|
||||
```bash
|
||||
docker ps # running containers
|
||||
docker ps -a # all (including stopped)
|
||||
docker stop NAME # graceful stop
|
||||
docker start NAME # start stopped container
|
||||
docker restart NAME # stop + start
|
||||
docker rm NAME # remove stopped container
|
||||
docker rm -f NAME # force remove running container
|
||||
docker container prune # remove ALL stopped containers
|
||||
```
|
||||
|
||||
**Interact with containers:**
|
||||
|
||||
```bash
|
||||
docker exec -it NAME /bin/sh # shell access (use /bin/bash if available)
|
||||
docker exec NAME env # view environment variables
|
||||
docker exec -u root NAME apt update # run as specific user
|
||||
docker logs --tail 100 -f NAME # follow last 100 lines
|
||||
docker logs --since 2h NAME # logs from last 2 hours
|
||||
docker cp NAME:/path/file ./local # copy file from container
|
||||
docker cp ./file NAME:/path/ # copy file to container
|
||||
docker inspect NAME # full container details (JSON)
|
||||
docker stats --no-stream # resource usage snapshot
|
||||
docker top NAME # running processes
|
||||
```
|
||||
|
||||
### 3. Image management
|
||||
|
||||
```bash
|
||||
# Build
|
||||
docker build -t my-app:latest .
|
||||
docker build -t my-app:prod -f Dockerfile.prod .
|
||||
docker build --no-cache -t my-app . # clean rebuild
|
||||
DOCKER_BUILDKIT=1 docker build -t my-app . # faster with BuildKit
|
||||
|
||||
# Pull and push
|
||||
docker pull node:20-alpine
|
||||
docker login ghcr.io
|
||||
docker tag my-app:latest registry/my-app:v1.0
|
||||
docker push registry/my-app:v1.0
|
||||
|
||||
# Inspect
|
||||
docker images # list local images
|
||||
docker history IMAGE # see layers
|
||||
docker inspect IMAGE # full details
|
||||
|
||||
# Cleanup
|
||||
docker image prune # remove dangling (untagged) images
|
||||
docker image prune -a # remove ALL unused images (careful!)
|
||||
docker image prune -a --filter "until=168h" # unused images older than 7 days
|
||||
```
|
||||
|
||||
### 4. Docker Compose
|
||||
|
||||
```bash
|
||||
# Start/stop
|
||||
docker compose up -d # start all services detached
|
||||
docker compose up -d --build # rebuild images before starting
|
||||
docker compose down # stop and remove containers
|
||||
docker compose down -v # also remove volumes (DESTROYS DATA)
|
||||
|
||||
# Monitoring
|
||||
docker compose ps # list services
|
||||
docker compose logs -f api # follow logs for specific service
|
||||
docker compose logs --tail 50 # last 50 lines all services
|
||||
|
||||
# Interaction
|
||||
docker compose exec api /bin/sh # shell into running service
|
||||
docker compose run --rm api npm test # one-off command (new container)
|
||||
docker compose restart api # restart specific service
|
||||
|
||||
# Validation
|
||||
docker compose config # validate and view resolved config
|
||||
```
|
||||
|
||||
**Minimal compose.yml example:**
|
||||
|
||||
```yaml
|
||||
services:
|
||||
api:
|
||||
build: .
|
||||
ports:
|
||||
- "3000:3000"
|
||||
environment:
|
||||
- DATABASE_URL=postgres://user:pass@db:5432/mydb
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
|
||||
db:
|
||||
image: postgres:16-alpine
|
||||
environment:
|
||||
POSTGRES_USER: user
|
||||
POSTGRES_PASSWORD: pass
|
||||
POSTGRES_DB: mydb
|
||||
volumes:
|
||||
- pgdata:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U user"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
volumes:
|
||||
pgdata:
|
||||
```
|
||||
|
||||
### 5. Volumes and networks
|
||||
|
||||
```bash
|
||||
# Volumes
|
||||
docker volume ls # list volumes
|
||||
docker volume create mydata # create named volume
|
||||
docker volume inspect mydata # details (mount point, etc.)
|
||||
docker volume rm mydata # remove (fails if in use)
|
||||
docker volume prune # remove unused volumes
|
||||
|
||||
# Networks
|
||||
docker network ls # list networks
|
||||
docker network create mynet # create bridge network
|
||||
docker network inspect mynet # details (connected containers)
|
||||
docker network connect mynet NAME # attach container to network
|
||||
docker network disconnect mynet NAME # detach container
|
||||
docker network rm mynet # remove network
|
||||
docker network prune # remove unused networks
|
||||
```
|
||||
|
||||
### 6. Disk usage and cleanup
|
||||
|
||||
Always start with a diagnostic before cleaning:
|
||||
|
||||
```bash
|
||||
# Check what's using space
|
||||
docker system df # summary
|
||||
docker system df -v # detailed breakdown
|
||||
|
||||
# Targeted cleanup (safe)
|
||||
docker container prune # stopped containers
|
||||
docker image prune # dangling images
|
||||
docker volume prune # unused volumes
|
||||
docker network prune # unused networks
|
||||
|
||||
# Aggressive cleanup (confirm with user first!)
|
||||
docker system prune # containers + images + networks
|
||||
docker system prune -a # also unused images
|
||||
docker system prune -a --volumes # EVERYTHING — named volumes too
|
||||
```
|
||||
|
||||
**Warning:** Never run `docker system prune -a --volumes` without confirming with the user. This removes named volumes with potentially important data.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
| Problem | Cause | Fix |
|
||||
|---------|-------|-----|
|
||||
| Container exits immediately | Main process finished or crashed | Check `docker logs NAME`, try `docker run -it --entrypoint /bin/sh IMAGE` |
|
||||
| "port is already allocated" | Another process using that port | `docker ps` or `lsof -i :PORT` to find it |
|
||||
| "no space left on device" | Docker disk full | `docker system df` then targeted prune |
|
||||
| Can't connect to container | App binds to 127.0.0.1 inside container | App must bind to `0.0.0.0`, check `-p` mapping |
|
||||
| Permission denied on volume | UID/GID mismatch host vs container | Use `--user $(id -u):$(id -g)` or fix permissions |
|
||||
| Compose services can't reach each other | Wrong network or service name | Services use service name as hostname, check `docker compose config` |
|
||||
| Build cache not working | Layer order wrong in Dockerfile | Put rarely-changing layers first (deps before source code) |
|
||||
| Image too large | No multi-stage build, no .dockerignore | Use multi-stage builds, add `.dockerignore` |
|
||||
|
||||
## Verification
|
||||
|
||||
After any Docker operation, verify the result:
|
||||
|
||||
- **Container started?** → `docker ps` (check status is "Up")
|
||||
- **Logs clean?** → `docker logs --tail 20 NAME` (no errors)
|
||||
- **Port accessible?** → `curl -s http://localhost:PORT` or `docker port NAME`
|
||||
- **Image built?** → `docker images | grep TAG`
|
||||
- **Compose stack healthy?** → `docker compose ps` (all services "running" or "healthy")
|
||||
- **Disk freed?** → `docker system df` (compare before/after)
|
||||
|
||||
## Dockerfile Optimization Tips
|
||||
|
||||
When reviewing or creating a Dockerfile, suggest these improvements:
|
||||
|
||||
1. **Multi-stage builds** — separate build environment from runtime to reduce final image size
|
||||
2. **Layer ordering** — put dependencies before source code so changes don't invalidate cached layers
|
||||
3. **Combine RUN commands** — fewer layers, smaller image
|
||||
4. **Use .dockerignore** — exclude `node_modules`, `.git`, `__pycache__`, etc.
|
||||
5. **Pin base image versions** — `node:20-alpine` not `node:latest`
|
||||
6. **Run as non-root** — add `USER` instruction for security
|
||||
7. **Use slim/alpine bases** — `python:3.12-slim` not `python:3.12`
|
||||
@@ -0,0 +1,178 @@
|
||||
---
|
||||
name: hermes-s6-container-supervision
|
||||
description: Modify, debug, or extend the s6-overlay supervision tree inside the Hermes Agent Docker image — adding new services, debugging profile gateways, understanding the Architecture B main-program pattern.
|
||||
version: 1.0.0
|
||||
author: Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux]
|
||||
environments: [s6]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [docker, s6, supervision, gateway, profiles]
|
||||
related_skills: [hermes-agent, hermes-agent-dev]
|
||||
---
|
||||
|
||||
# Hermes s6-overlay Container Supervision
|
||||
|
||||
## When to use this skill
|
||||
|
||||
Load this skill when you're working on:
|
||||
- Adding or removing a static service in the Hermes Docker image (something that should be supervised at every container start, like the dashboard)
|
||||
- Diagnosing why a per-profile gateway isn't starting, restarting, or surviving `docker restart`
|
||||
- Understanding why the container's CMD is `/opt/hermes/docker/main-wrapper.sh` and how leading-dash args reach the user's program
|
||||
- Modifying `cont-init.d` boot scripts (UID remap, volume seeding, profile reconciliation)
|
||||
- Changing the rendered run-script for per-profile gateways (Phase 4)
|
||||
|
||||
If you're just running the Hermes Agent and want to use Docker, see `website/docs/user-guide/docker.md` instead.
|
||||
|
||||
## Architecture at a glance
|
||||
|
||||
```
|
||||
/init ← PID 1 (s6-overlay v3.2.3.0)
|
||||
├── cont-init.d ← oneshot setup, runs as root
|
||||
│ ├── 01-hermes-setup ← docker/stage2-hook.sh
|
||||
│ │ ├── UID/GID remap
|
||||
│ │ ├── chown /opt/data
|
||||
│ │ ├── chown /opt/data/profiles (every boot)
|
||||
│ │ ├── seed .env / config.yaml / SOUL.md
|
||||
│ │ └── skills_sync.py
|
||||
│ └── 02-reconcile-profiles ← hermes_cli.container_boot
|
||||
│ ├── chown /run/service (hermes-writable for runtime register)
|
||||
│ └── walk $HERMES_HOME/profiles/<name>/gateway_state.json
|
||||
│ → recreate /run/service/gateway-<name>/
|
||||
│ → auto-start only those with prior_state == "running"
|
||||
│
|
||||
├── s6-rc.d (static services, in /etc/s6-overlay/s6-rc.d/)
|
||||
│ ├── main-hermes/run ← exec sleep infinity (no-op slot)
|
||||
│ └── dashboard/run ← if HERMES_DASHBOARD=1, runs `hermes dashboard`
|
||||
│
|
||||
├── /run/service (s6-svscan watches; tmpfs)
|
||||
│ ├── gateway-coder/ ← runtime-registered per-profile
|
||||
│ │ ├── type ("longrun")
|
||||
│ │ ├── run ("#!/command/with-contenv sh ... exec s6-setuidgid hermes hermes -p coder gateway run")
|
||||
│ │ ├── down (marker — present means "registered but don't auto-start")
|
||||
│ │ └── log/run (s6-log → $HERMES_HOME/logs/gateways/coder/current)
|
||||
│ └── ...
|
||||
│
|
||||
└── CMD ("main program") ← /opt/hermes/docker/main-wrapper.sh
|
||||
└── routes user args: bare exec | hermes subcommand | hermes (no args)
|
||||
— exec'd by /init with stdin/stdout/stderr inherited (TTY for --tui)
|
||||
```
|
||||
|
||||
## Key files
|
||||
|
||||
| Path | Role |
|
||||
|---|---|
|
||||
| `Dockerfile` | s6-overlay install + cont-init.d wiring + `ENTRYPOINT ["/init", "/opt/hermes/docker/main-wrapper.sh"]` |
|
||||
| `docker/stage2-hook.sh` | The "old entrypoint logic" — UID remap, chown, seed, skills sync. Runs as cont-init.d/01-hermes-setup. |
|
||||
| `docker/cont-init.d/02-reconcile-profiles` | Calls `hermes_cli.container_boot` on every boot to restore profile gateway slots from the persistent volume. |
|
||||
| `docker/main-wrapper.sh` | The container's CMD. Routes user args, drops to hermes via `s6-setuidgid`, exec's the chosen program. |
|
||||
| `docker/s6-rc.d/main-hermes/run` | No-op `sleep infinity` — slot exists so the s6-rc user bundle is valid; main hermes runs as the CMD, not as a supervised service. |
|
||||
| `docker/s6-rc.d/dashboard/run` | Conditional service — `exec sleep infinity` unless `HERMES_DASHBOARD` is truthy. |
|
||||
| `docker/entrypoint.sh` | Back-compat shim that `exec`s the stage2 hook. External scripts that hard-coded the old entrypoint path still work. |
|
||||
| `hermes_cli/service_manager.py` | `S6ServiceManager`: `register_profile_gateway`, `unregister_profile_gateway`, `start/stop/restart/is_running`, `list_profile_gateways`. |
|
||||
| `hermes_cli/container_boot.py` | `reconcile_profile_gateways()` — walks persistent profiles, regenerates s6 slots, emits `container-boot.log`. |
|
||||
| `hermes_cli/gateway.py::_dispatch_via_service_manager_if_s6` | Intercepts `hermes gateway start/stop/restart` and routes to s6 when running in a container. |
|
||||
|
||||
## Why Architecture B (CMD as main program, not s6-supervised)
|
||||
|
||||
The original plan (v1–v3) called for main hermes to run as a supervised s6-rc service. Two real s6-overlay v3 mechanics blocked that:
|
||||
|
||||
1. **cont-init.d scripts receive no CMD args** — so the stage2 hook can't parse `docker run <image> chat -q "hi"` to set `HERMES_ARGS` for a service `run` script to consume.
|
||||
2. **`/run/s6/basedir/bin/halt` does NOT propagate the exit code** written to `/run/s6-linux-init-container-results/exitcode`. Containers always exit 143 (SIGTERM) regardless. Confirmed by skarnet (s6 author) in [issue #477](https://github.com/just-containers/s6-overlay/issues/477): _"if you want a container shutdown, you need to either have your CMD exit, or, if you have no CMD, write the container exit code you want then call halt"_.
|
||||
|
||||
So we use the s6-overlay-native CMD pattern: `ENTRYPOINT ["/init", "/opt/hermes/docker/main-wrapper.sh"]`. /init prepends the wrapper to user args automatically — so `docker run <image> --version` becomes `/init main-wrapper.sh --version`, and `--version` doesn't get intercepted by /init's POSIX shell. The wrapper drops to hermes via `s6-setuidgid`, then exec's the chosen program. The program's exit code becomes the container exit code, exactly matching the pre-s6 tini contract.
|
||||
|
||||
Trade-off: main hermes is unsupervised under s6. That exactly matches its behavior under tini (the pre-s6 image). Dashboard supervision is the only **new** guarantee — and per-profile gateways under `/run/service/` get full supervision.
|
||||
|
||||
## Quick recipes
|
||||
|
||||
### Verify s6 is PID 1 in a running container
|
||||
|
||||
```sh
|
||||
docker exec <c> sh -c 'cat /proc/1/comm; readlink /proc/1/exe'
|
||||
# Expect: s6-svscan or init / /package/admin/s6/.../s6-svscan
|
||||
```
|
||||
|
||||
### Inspect a profile gateway service
|
||||
|
||||
```sh
|
||||
# /command/ isn't on docker-exec PATH — use absolute path
|
||||
docker exec <c> /command/s6-svstat /run/service/gateway-<name>
|
||||
# "up (pid …) … seconds" → running
|
||||
# "down (exitcode N) … seconds, normally up, want up, …" → s6 wants it up but the process keeps exiting (crash loop)
|
||||
# "down … normally up, ready …" → user stopped it
|
||||
```
|
||||
|
||||
### Bring a service up/down manually
|
||||
|
||||
```sh
|
||||
docker exec <c> /command/s6-svc -u /run/service/gateway-<name> # up
|
||||
docker exec <c> /command/s6-svc -d /run/service/gateway-<name> # down
|
||||
docker exec <c> /command/s6-svc -t /run/service/gateway-<name> # SIGTERM (restart)
|
||||
```
|
||||
|
||||
### Watch the cont-init reconciler log
|
||||
|
||||
```sh
|
||||
docker exec <c> tail -n 50 /opt/data/logs/container-boot.log
|
||||
# 2026-05-21T06:18:05+0000 profile=coder prior_state=running action=started
|
||||
# 2026-05-21T06:18:05+0000 profile=writer prior_state=stopped action=registered
|
||||
```
|
||||
|
||||
### Add a new static service
|
||||
|
||||
1. Create `docker/s6-rc.d/<name>/type` with `longrun\n` and `docker/s6-rc.d/<name>/run` (use `#!/command/with-contenv sh` + `# shellcheck shell=sh`).
|
||||
2. Drop to hermes via `s6-setuidgid hermes` at the top of run (unless you specifically need root).
|
||||
3. Create empty `docker/s6-rc.d/<name>/dependencies.d/base` so it waits for the base bundle.
|
||||
4. Create empty `docker/s6-rc.d/user/contents.d/<name>` so it joins the user bundle.
|
||||
5. The `COPY docker/s6-rc.d/` in the Dockerfile picks it up automatically — no other changes.
|
||||
|
||||
### Change the per-profile gateway run command
|
||||
|
||||
Edit `S6ServiceManager._render_run_script` in `hermes_cli/service_manager.py`. The function is also called by `hermes_cli/container_boot.py::_register_service` during boot reconciliation, so it's the single source of truth. Update the corresponding assertion in `tests/hermes_cli/test_service_manager.py::test_s6_register_creates_service_dir_and_triggers_scan`.
|
||||
|
||||
### Run the docker test harness
|
||||
|
||||
```sh
|
||||
docker build -t hermes-agent-harness:latest .
|
||||
HERMES_TEST_IMAGE=hermes-agent-harness:latest scripts/run_tests.sh tests/docker/ -v
|
||||
# Expect 19 passed, 0 xfailed against the s6 image
|
||||
```
|
||||
|
||||
The harness lives in `tests/docker/` and skips when Docker isn't available. The per-test timeout is bumped to 180s (see `tests/docker/conftest.py`).
|
||||
|
||||
## Common pitfalls
|
||||
|
||||
### "command not found" via `docker exec`
|
||||
|
||||
`/command/` (where s6-overlay puts its binaries) is on PATH only for processes spawned by the supervision tree — services, cont-init.d, main-wrapper.sh. `docker exec <c> s6-svstat …` will fail with "command not found"; always use the absolute path `/command/s6-svstat`. The `hermes` binary works because the Dockerfile adds `/opt/hermes/.venv/bin` to the runtime `ENV PATH`.
|
||||
|
||||
### Profile directory ownership
|
||||
|
||||
The cont-init reconciler runs as hermes (`s6-setuidgid hermes` in `02-reconcile-profiles`). If a profile dir ends up root-owned (e.g. because `docker exec <c> hermes profile create …` ran as root by default), the reconciler can't read SOUL.md and fails with `PermissionError`. Mitigation: `stage2-hook.sh` chowns `$HERMES_HOME/profiles` to hermes on **every** boot, idempotently. Don't remove that block.
|
||||
|
||||
### Files written by `docker exec` are root-owned
|
||||
|
||||
`docker exec` defaults to root. Either pass `--user hermes` or rely on the stage2 chown sweep next reboot. Don't write files under `$HERMES_HOME/profiles/<name>/` as root manually — the next reconcile pass will sweep them but in-flight operations may hit perm errors.
|
||||
|
||||
### Service slot exists but s6-svstat says "s6-supervise not running"
|
||||
|
||||
The service directory is on tmpfs and was wiped on container restart. Either the cont-init reconciler hasn't run yet (give it a moment after `docker restart`) or it failed. Check `docker logs <c> | grep '02-reconcile'`.
|
||||
|
||||
### Gateway starts then immediately exits (`down (exitcode 1)` in svstat)
|
||||
|
||||
Most likely the profile has no model or auth configured. The service slot is correct — the gateway itself is unconfigured. Run `hermes -p <profile> setup` first. The s6 supervisor will keep restarting it; that's the desired behavior (when you fix the config, the next attempt succeeds and stays up).
|
||||
|
||||
### Reconciler skipped a profile
|
||||
|
||||
The reconciler keys on the **presence of `SOUL.md`** as the "real profile" marker. `hermes profile create` always seeds it. If a profile dir is missing SOUL.md (stray directory, partial restore, backup-in-progress), the reconciler skips it intentionally. Add a `SOUL.md` (even empty) to opt back in.
|
||||
|
||||
### "Help, the container exits 143!"
|
||||
|
||||
Check whether something is invoking `s6-svscanctl -t` or `/run/s6/basedir/bin/halt` — both cause /init to begin stage 3 shutdown but return 143 (SIGTERM) rather than the desired exit code. This was the Phase 2 architecture pivot from A to B. For container shutdown with a real exit code, you must let the CMD (main-wrapper.sh) exit normally; do **not** try to control exit from a finish script.
|
||||
|
||||
## Related skills
|
||||
|
||||
- `hermes-agent-dev`: General hermes-agent codebase navigation
|
||||
- `hermes-tool-quirks`: Specific Hermes-tool workarounds (sed/grep/etc.) — load when debugging the s6 stack's interaction with hermes built-in tools.
|
||||
@@ -0,0 +1,309 @@
|
||||
---
|
||||
name: pinggy-tunnel
|
||||
description: Zero-install localhost tunnels over SSH via Pinggy.
|
||||
version: 0.1.0
|
||||
author: Teknium (teknium1), Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos, windows]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Pinggy, Tunnel, Networking, SSH, Webhook, Localhost]
|
||||
related_skills: [cloudflared-quick-tunnel, webhook-subscriptions]
|
||||
---
|
||||
|
||||
# Pinggy Tunnel Skill
|
||||
|
||||
Expose a local service (dev server, webhook receiver, MCP endpoint, demo) to the public internet using a Pinggy SSH reverse tunnel. No daemon to install — the user's stock SSH client connects to `a.pinggy.io:443` and Pinggy hands back a public HTTP/HTTPS URL.
|
||||
|
||||
Free tier: 60-minute tunnels, random subdomain, no signup. Pro tier ($3/mo) is an opt-in with a token.
|
||||
|
||||
## When to Use
|
||||
|
||||
- User asks to "expose this locally", "share my dev server", "make this URL public", "tunnel port N", "get a public URL for a webhook"
|
||||
- Need to receive a webhook callback during a local task (Stripe, GitHub, Discord, AgentMail)
|
||||
- Sharing a one-off HTTP demo (MCP server, Ollama/vLLM endpoint, dashboard) with a remote party
|
||||
- The host has SSH but no `cloudflared` / `ngrok` binary, and installing one would be overkill
|
||||
|
||||
If the host already has `cloudflared` configured, prefer the `cloudflared-quick-tunnel` skill — Cloudflare quick tunnels don't expire after 60 minutes.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- `ssh` on PATH (`ssh -V`). Default on Linux, macOS, and Windows 10+. No other install.
|
||||
- A local service listening on `127.0.0.1:<port>` before the tunnel starts. Pinggy will return URLs but they'll 502 until the local origin is up.
|
||||
|
||||
Optional:
|
||||
|
||||
- `PINGGY_TOKEN` env var for paid Pro features (persistent subdomain, custom domain, multiple tunnels, no 60-minute cap). Free tier needs no credentials.
|
||||
|
||||
## Quick Reference
|
||||
|
||||
```bash
|
||||
# Plain HTTP/HTTPS tunnel for port 8000 (free tier)
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -o ServerAliveInterval=30 \
|
||||
-R0:localhost:8000 free@a.pinggy.io
|
||||
|
||||
# TCP tunnel (databases, raw SSH, etc.)
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -R0:localhost:5432 tcp@a.pinggy.io
|
||||
|
||||
# TLS tunnel (Pinggy can't decrypt — bring your own certs at origin)
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -R0:localhost:443 tls@a.pinggy.io
|
||||
|
||||
# Basic auth gate (b:user:pass)
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -R0:localhost:8000 \
|
||||
"b:admin:secret+free@a.pinggy.io"
|
||||
|
||||
# Bearer token gate (k:token)
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -R0:localhost:8000 \
|
||||
"k:mysecrettoken+free@a.pinggy.io"
|
||||
|
||||
# IP whitelist (w:CIDR)
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -R0:localhost:8000 \
|
||||
"w:203.0.113.0/24+free@a.pinggy.io"
|
||||
|
||||
# Enable CORS + force HTTPS redirect
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -R0:localhost:8000 \
|
||||
"co+x:https+free@a.pinggy.io"
|
||||
|
||||
# Pro tier (persistent URL, no 60-min cap)
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -R0:localhost:8000 "$PINGGY_TOKEN+a.pinggy.io"
|
||||
```
|
||||
|
||||
## Procedure — Start a Tunnel and Get the URL
|
||||
|
||||
The model SHOULD use the `terminal` tool. The tunnel must stay alive for the duration of the share, so run it as a background process and parse the public URL from stdout.
|
||||
|
||||
### 1. Confirm a local origin is up
|
||||
|
||||
```bash
|
||||
curl -sI http://127.0.0.1:8000/ | head -1
|
||||
# expect HTTP/1.x 200 (or any non-connection-refused response)
|
||||
```
|
||||
|
||||
If nothing is listening yet, start it first (e.g. `python3 -m http.server 8000 --bind 127.0.0.1`). Pinggy will happily return a URL pointed at nothing — the user will see 502 until the origin comes up.
|
||||
|
||||
### 2. Launch the tunnel as a background process
|
||||
|
||||
Use `terminal(background=True)` and capture output to a logfile (Pinggy prints the URLs on stdout, then keeps the connection open):
|
||||
|
||||
```bash
|
||||
LOG=/tmp/pinggy-8000.log
|
||||
nohup ssh -p 443 \
|
||||
-o StrictHostKeyChecking=no \
|
||||
-o UserKnownHostsFile=/dev/null \
|
||||
-o ServerAliveInterval=30 \
|
||||
-o ServerAliveCountMax=3 \
|
||||
-R0:localhost:8000 free@a.pinggy.io \
|
||||
> "$LOG" 2>&1 &
|
||||
echo $! > /tmp/pinggy-8000.pid
|
||||
```
|
||||
|
||||
`StrictHostKeyChecking=no` + `UserKnownHostsFile=/dev/null` skips the first-run host-key prompt. `ServerAliveInterval=30` keeps the SSH session from getting torn down by an idle NAT.
|
||||
|
||||
### 3. Parse the URL out of the log
|
||||
|
||||
```bash
|
||||
sleep 4
|
||||
grep -oE 'https://[a-z0-9-]+\.[a-z]+\.pinggy\.link' /tmp/pinggy-8000.log | head -1
|
||||
```
|
||||
|
||||
Expected output looks like:
|
||||
|
||||
```
|
||||
You are not authenticated.
|
||||
Your tunnel will expire in 60 minutes.
|
||||
http://yqycl-98-162-69-48.a.free.pinggy.link
|
||||
https://yqycl-98-162-69-48.a.free.pinggy.link
|
||||
```
|
||||
|
||||
Hand the `https://...pinggy.link` URL to the user.
|
||||
|
||||
### 4. Verify
|
||||
|
||||
```bash
|
||||
curl -sI https://<the-url>/ | head -3
|
||||
# expect 200/302/whatever the local origin actually returns
|
||||
```
|
||||
|
||||
If you get `502 Bad Gateway`, the SSH session is up but the local origin isn't listening — fix step 1 first.
|
||||
|
||||
### 5. Teardown
|
||||
|
||||
```bash
|
||||
kill "$(cat /tmp/pinggy-8000.pid)"
|
||||
# or, if the pid file got lost:
|
||||
pkill -f 'ssh -p 443 .* free@a\.pinggy\.io'
|
||||
```
|
||||
|
||||
If you have a session_id from `terminal(background=True)`, prefer `process(action='kill', session_id=...)`.
|
||||
|
||||
## Access Control via Username Keywords
|
||||
|
||||
Pinggy stacks control flags into the SSH username separated by `+`. Always quote the whole `user@host` argument when it contains a `+`:
|
||||
|
||||
| Keyword | Effect |
|
||||
|---------|--------|
|
||||
| `b:user:pass` | HTTP Basic auth gate |
|
||||
| `k:token` | Bearer-token header gate (`Authorization: Bearer <token>`) |
|
||||
| `w:CIDR` | IP whitelist (single IP or CIDR, repeatable) |
|
||||
| `co` | Add `Access-Control-Allow-Origin: *` (CORS) |
|
||||
| `x:https` | Force HTTPS — auto-redirect HTTP to HTTPS |
|
||||
| `a:Name:Value` | Add request header |
|
||||
| `u:Name:Value` | Update request header |
|
||||
| `r:Name` | Remove request header |
|
||||
| `qr` | Print a QR code of the URL to stdout (handy for mobile sharing) |
|
||||
|
||||
Combine freely: `"b:admin:secret+co+x:https+free@a.pinggy.io"`.
|
||||
|
||||
## Web Debugger (optional)
|
||||
|
||||
Pinggy can mirror the inbound traffic to `localhost:4300` for inspection. Add a local forward to the SSH command:
|
||||
|
||||
```bash
|
||||
ssh -p 443 -L4300:localhost:4300 -R0:localhost:8000 free@a.pinggy.io
|
||||
```
|
||||
|
||||
Then open `http://localhost:4300` in a browser to see live request/response pairs.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **60-minute hard cap on the free tier.** The SSH session terminates at the 60-minute mark; the URL goes dead. For longer shares, either use `PINGGY_TOKEN` (Pro) or auto-restart with a shell loop (note that the URL changes on every restart for free-tier).
|
||||
- **Free-tier URL is random and changes on restart.** Don't bookmark it, don't paste it into a config file. Re-parse from the log each time.
|
||||
- **Concurrent free tunnels are limited to one per source IP.** Starting a second tunnel from the same machine usually kills the first. Pro tier lifts this.
|
||||
- **`+` in usernames must be quoted.** Bare `ssh ... b:admin:secret+free@a.pinggy.io` works in bash but breaks under shells that treat `+` specially or when assembled programmatically. Always wrap in double quotes.
|
||||
- **Don't tunnel anything sensitive without an access-control flag.** A bare HTTP tunnel is reachable by anyone with the URL. Use `b:`, `k:`, or `w:` for non-public services.
|
||||
- **`process(action='log')` may miss SSH banner output.** Pinggy prints the URLs and then the SSH session goes interactive. Always redirect to a logfile and `grep` the file directly — same pattern as `cloudflared-quick-tunnel`.
|
||||
- **Host-key prompt on first run.** Default OpenSSH config asks the user to accept Pinggy's host key. Always pass `-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null` for unattended runs.
|
||||
- **TCP and TLS tunnels return a `<subdomain>.a.pinggy.online:<port>` pair, not an https URL.** Parse with a different regex (`tcp://` and a port). Don't assume every Pinggy tunnel is HTTP.
|
||||
- **Pro mode requires the token as the username, not a flag.** Use `"$PINGGY_TOKEN+a.pinggy.io"` (no `free@`). With a token you can also add `:persistent` for a stable subdomain — see `pinggy.io/docs/`.
|
||||
|
||||
## Recipes
|
||||
|
||||
Composite patterns combining a local origin with a Pinggy tunnel. Each recipe is self-contained — start the origin, start the tunnel, parse the URL, hand it back to the user.
|
||||
|
||||
### Recipe 1 — Receive a webhook callback
|
||||
|
||||
Use this when an external service (Stripe, GitHub, Discord, AgentMail, etc.) needs to POST to a publicly reachable URL during a local task.
|
||||
|
||||
```bash
|
||||
# 1. Tiny capturing server: every request gets appended to /tmp/webhook-hits.log
|
||||
cat >/tmp/webhook-server.py <<'PY'
|
||||
import http.server, json, datetime, pathlib
|
||||
LOG = pathlib.Path("/tmp/webhook-hits.log")
|
||||
class H(http.server.BaseHTTPRequestHandler):
|
||||
def _capture(self):
|
||||
n = int(self.headers.get("content-length") or 0)
|
||||
body = self.rfile.read(n).decode("utf-8", "replace") if n else ""
|
||||
rec = {"t": datetime.datetime.utcnow().isoformat(), "path": self.path,
|
||||
"method": self.command, "headers": dict(self.headers), "body": body}
|
||||
with LOG.open("a") as f: f.write(json.dumps(rec) + "\n")
|
||||
self.send_response(200); self.send_header("content-type","application/json")
|
||||
self.end_headers(); self.wfile.write(b'{"ok":true}\n')
|
||||
def do_GET(self): self._capture()
|
||||
def do_POST(self): self._capture()
|
||||
def log_message(self,*a,**k): pass
|
||||
http.server.HTTPServer(("127.0.0.1", 18080), H).serve_forever()
|
||||
PY
|
||||
nohup python3 /tmp/webhook-server.py >/tmp/webhook-server.log 2>&1 &
|
||||
echo $! >/tmp/webhook-server.pid
|
||||
|
||||
# 2. Tunnel — bearer-token-gate so randos can't pollute the capture log
|
||||
nohup ssh -p 443 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
|
||||
-o ServerAliveInterval=30 \
|
||||
-R0:localhost:18080 "k:$(openssl rand -hex 12)+free@a.pinggy.io" \
|
||||
>/tmp/webhook-pinggy.log 2>&1 &
|
||||
echo $! >/tmp/webhook-pinggy.pid
|
||||
sleep 5
|
||||
URL=$(grep -oE 'https://[a-z0-9-]+\.[a-z]+\.pinggy\.link' /tmp/webhook-pinggy.log | head -1)
|
||||
echo "Webhook URL: $URL"
|
||||
|
||||
# 3. While the agent works, watch hits land
|
||||
tail -f /tmp/webhook-hits.log
|
||||
```
|
||||
|
||||
Hand `$URL` to the service that needs to call you. Teardown: `kill $(cat /tmp/webhook-server.pid) $(cat /tmp/webhook-pinggy.pid)`.
|
||||
|
||||
### Recipe 2 — Expose an MCP server over HTTP/SSE
|
||||
|
||||
Use when a remote MCP client (Claude Desktop on another machine, a teammate's editor, etc.) needs to reach an MCP server running on the local box. Only works for MCP servers that speak HTTP transport — stdio-mode servers can't be tunneled.
|
||||
|
||||
```bash
|
||||
# 1. Start the MCP server in HTTP mode (example: a FastMCP server on port 8765)
|
||||
nohup python3 my_mcp_server.py --transport http --port 8765 \
|
||||
>/tmp/mcp-server.log 2>&1 &
|
||||
echo $! >/tmp/mcp-server.pid
|
||||
|
||||
# 2. Tunnel with a bearer token — MCP traffic should not be open to the internet
|
||||
TOKEN=$(openssl rand -hex 16)
|
||||
nohup ssh -p 443 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
|
||||
-o ServerAliveInterval=30 \
|
||||
-R0:localhost:8765 "k:$TOKEN+free@a.pinggy.io" \
|
||||
>/tmp/mcp-pinggy.log 2>&1 &
|
||||
echo $! >/tmp/mcp-pinggy.pid
|
||||
sleep 5
|
||||
URL=$(grep -oE 'https://[a-z0-9-]+\.[a-z]+\.pinggy\.link' /tmp/mcp-pinggy.log | head -1)
|
||||
echo "MCP URL: $URL"
|
||||
echo "Bearer token: $TOKEN"
|
||||
```
|
||||
|
||||
The remote client connects to `$URL` with `Authorization: Bearer $TOKEN`. Hermes' own native MCP client config: `{"transport": "http", "url": "<URL>", "headers": {"Authorization": "Bearer <TOKEN>"}}`.
|
||||
|
||||
### Recipe 3 — Expose a local LLM endpoint (Ollama / vLLM / llama.cpp)
|
||||
|
||||
Share a local model with a remote caller (another agent, a phone, a teammate). Ollama listens on `:11434`, vLLM and llama.cpp typically on `:8000`.
|
||||
|
||||
```bash
|
||||
# Pre-req: the model server is already running on 127.0.0.1:11434 (Ollama default)
|
||||
TOKEN=$(openssl rand -hex 16)
|
||||
nohup ssh -p 443 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
|
||||
-o ServerAliveInterval=30 \
|
||||
-R0:localhost:11434 "k:$TOKEN+co+free@a.pinggy.io" \
|
||||
>/tmp/llm-pinggy.log 2>&1 &
|
||||
echo $! >/tmp/llm-pinggy.pid
|
||||
sleep 5
|
||||
URL=$(grep -oE 'https://[a-z0-9-]+\.[a-z]+\.pinggy\.link' /tmp/llm-pinggy.log | head -1)
|
||||
echo "Endpoint: $URL"
|
||||
echo "Token: $TOKEN"
|
||||
|
||||
# Verify
|
||||
curl -s "$URL/api/tags" -H "Authorization: Bearer $TOKEN" | head
|
||||
```
|
||||
|
||||
`co` enables CORS so a browser caller can hit the endpoint. Drop `co` for backend-only callers. For an OpenAI-compatible vLLM/llama.cpp endpoint, callers use base URL `$URL/v1` with `Authorization: Bearer $TOKEN` — but note Pinggy strips/replaces nothing in the body, so the model server itself sees Pinggy's token; the local server should be configured to ignore auth (it's already on `127.0.0.1`) and let Pinggy do the gating.
|
||||
|
||||
### Recipe 4 — Share a dev server with a one-shot password
|
||||
|
||||
The fastest "let a teammate poke at my running app" pattern. Random password, prints once, dies when you Ctrl-C.
|
||||
|
||||
```bash
|
||||
PASS=$(openssl rand -base64 12 | tr -d '+/=' | head -c 12)
|
||||
echo "Dev server password: $PASS"
|
||||
ssh -p 443 -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null \
|
||||
-o ServerAliveInterval=30 \
|
||||
-R0:localhost:3000 "b:dev:$PASS+co+x:https+free@a.pinggy.io"
|
||||
# URL prints to the terminal. Share URL + password. Ctrl-C to tear down.
|
||||
```
|
||||
|
||||
`b:dev:$PASS` gates the URL with HTTP Basic auth. `x:https` forces TLS. `co` adds CORS for SPA frontends.
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
# End-to-end: spin up a trivial origin, tunnel it, hit it, tear down
|
||||
python3 -m http.server 18000 --bind 127.0.0.1 >/tmp/origin.log 2>&1 &
|
||||
ORIGIN_PID=$!
|
||||
|
||||
nohup ssh -p 443 \
|
||||
-o StrictHostKeyChecking=no \
|
||||
-o UserKnownHostsFile=/dev/null \
|
||||
-R0:localhost:18000 free@a.pinggy.io >/tmp/pinggy-verify.log 2>&1 &
|
||||
SSH_PID=$!
|
||||
|
||||
sleep 5
|
||||
URL=$(grep -oE 'https://[a-z0-9-]+\.[a-z]+\.pinggy\.link' /tmp/pinggy-verify.log | head -1)
|
||||
echo "URL: $URL"
|
||||
curl -sI "$URL/" | head -1
|
||||
|
||||
kill "$SSH_PID" "$ORIGIN_PID"
|
||||
```
|
||||
|
||||
Expected: a `pinggy.link` URL and `HTTP/2 200` on the curl head.
|
||||
@@ -0,0 +1,112 @@
|
||||
---
|
||||
name: watchers
|
||||
description: Poll RSS, JSON APIs, and GitHub with watermark dedup.
|
||||
version: 1.0.0
|
||||
author: Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux, macos]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [cron, polling, rss, github, http, automation, monitoring]
|
||||
category: devops
|
||||
requires_toolsets: [terminal]
|
||||
related_skills: []
|
||||
---
|
||||
|
||||
# Watchers
|
||||
|
||||
Poll external sources on an interval and react only to new items. Three ready-made scripts plus a shared watermark helper; wire them into a cron job (or run them ad-hoc from the terminal).
|
||||
|
||||
## When to Use
|
||||
|
||||
- User wants to watch an RSS/Atom feed and be notified of new entries
|
||||
- User wants to watch a GitHub repo's issues / pulls / releases / commits
|
||||
- User wants to poll an arbitrary JSON endpoint and get notified on new items
|
||||
- User asks for "a watcher for X" or "notify me when X changes"
|
||||
|
||||
## Mental model
|
||||
|
||||
A watcher is just a script that:
|
||||
|
||||
1. Fetches data from the external source
|
||||
2. Compares against a watermark file of previously-seen IDs
|
||||
3. Writes the new watermark back
|
||||
4. Prints new items to stdout (or nothing on no-change)
|
||||
|
||||
The scripts below handle all three. The agent runs them via the terminal tool — from a cron job, a webhook, or an interactive chat — and reports what's new.
|
||||
|
||||
## Ready-made scripts
|
||||
|
||||
All three live in `$HERMES_HOME/skills/devops/watchers/scripts/` once the skill is installed. Each reads `WATCHER_STATE_DIR` (defaults to `$HERMES_HOME/watcher-state/`) for its state file, keyed by the `--name` argument.
|
||||
|
||||
| Script | What it watches | Dedup key |
|
||||
|---|---|---|
|
||||
| `watch_rss.py` | RSS 2.0 or Atom feed URL | `<guid>` / `<id>` |
|
||||
| `watch_http_json.py` | Any JSON endpoint returning a list of objects | Configurable id field |
|
||||
| `watch_github.py` | GitHub issues / pulls / releases / commits for a repo | `id` / `sha` |
|
||||
|
||||
All three:
|
||||
|
||||
- First run records a baseline — never replays existing feed
|
||||
- Watermark is a bounded ID set (max 500) to cap memory
|
||||
- Output format: `## <title>\n<url>\n\n<optional body>` per item
|
||||
- Empty stdout on no-new — the caller treats that as silent
|
||||
- Non-zero exit on fetch errors
|
||||
|
||||
## Usage
|
||||
|
||||
Run a watcher directly from the terminal tool:
|
||||
|
||||
```bash
|
||||
python $HERMES_HOME/skills/devops/watchers/scripts/watch_rss.py \
|
||||
--name hn --url https://news.ycombinator.com/rss --max 5
|
||||
```
|
||||
|
||||
Watch a GitHub repo (set `GITHUB_TOKEN` in `${HERMES_HOME:-~/.hermes}/.env` to avoid the 60 req/hr anonymous rate limit):
|
||||
|
||||
```bash
|
||||
python $HERMES_HOME/skills/devops/watchers/scripts/watch_github.py \
|
||||
--name hermes-issues --repo NousResearch/hermes-agent --scope issues
|
||||
```
|
||||
|
||||
Poll an arbitrary JSON API:
|
||||
|
||||
```bash
|
||||
python $HERMES_HOME/skills/devops/watchers/scripts/watch_http_json.py \
|
||||
--name api --url https://api.example.com/events \
|
||||
--id-field event_id --items-path data.events
|
||||
```
|
||||
|
||||
## Wiring into cron
|
||||
|
||||
Ask the agent to schedule a cron job with a prompt like:
|
||||
|
||||
> Every 15 minutes, run `watch_rss.py --name hn --url https://news.ycombinator.com/rss`. If it prints anything, summarize the headlines and deliver them. If it prints nothing, stay silent.
|
||||
|
||||
The agent invokes the script via the terminal tool inside the cron job's agent loop; no changes to cron's built-in `--script` flag are needed.
|
||||
|
||||
## State files
|
||||
|
||||
Every watcher writes `$HERMES_HOME/watcher-state/<name>.json`. Inspect:
|
||||
|
||||
```bash
|
||||
cat $HERMES_HOME/watcher-state/hn.json
|
||||
```
|
||||
|
||||
Force a replay (next run treated as first poll):
|
||||
|
||||
```bash
|
||||
rm $HERMES_HOME/watcher-state/hn.json
|
||||
```
|
||||
|
||||
## Writing your own
|
||||
|
||||
All three scripts use the same template: load watermark, fetch, diff, save, emit. `scripts/_watermark.py` is the shared helper; import it to get atomic writes + bounded ID set + first-run baseline for free. See any of the three reference scripts for how little boilerplate it takes.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
1. **Printing a "no new items" header every tick.** Callers rely on empty stdout = silent. If you print anything on an empty delta, you spam the channel. The shipped scripts handle this; custom scripts must too.
|
||||
2. **Expecting the first run to emit items.** It won't — first run records a baseline. If you need an initial digest, delete the state file after the first run or add a `--prime-with-latest N` flag in your own script.
|
||||
3. **Unbounded watermark growth.** The shared helper caps at 500 IDs. Raise it for high-churn feeds; lower it on constrained filesystems.
|
||||
4. **Putting the state dir where the agent's sandbox can't write.** `$HERMES_HOME/watcher-state/` is always writable. Docker/Modal backends may not see arbitrary host paths.
|
||||
|
||||
+148
@@ -0,0 +1,148 @@
|
||||
"""Shared watermark helper used by the three watcher scripts.
|
||||
|
||||
A watermark is just a JSON file that records the IDs we've seen on previous
|
||||
runs, so the next run only emits items we haven't seen before.
|
||||
|
||||
Contract:
|
||||
- First run: record all IDs from the fetched batch, emit nothing.
|
||||
- Subsequent runs: emit items whose ID isn't in the stored set.
|
||||
- Bounded: keep at most `max_seen` IDs (default 500).
|
||||
- Atomic: write to a .tmp file and rename, so a crashed script can't
|
||||
leave a half-written state file that permanently breaks dedup.
|
||||
|
||||
Import and use from any custom watcher script:
|
||||
|
||||
from _watermark import Watermark
|
||||
|
||||
wm = Watermark.load("my-feed-name")
|
||||
new_items = wm.filter_new(fetched_items, id_key="id")
|
||||
wm.save()
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, Iterable, List, Optional
|
||||
|
||||
|
||||
def _state_dir() -> Path:
|
||||
"""Where watermark files live — respects WATCHER_STATE_DIR override."""
|
||||
override = os.environ.get("WATCHER_STATE_DIR")
|
||||
if override:
|
||||
return Path(override)
|
||||
# Default: $HERMES_HOME/watcher-state/, falling back to ~/.hermes/watcher-state/.
|
||||
hermes_home = os.environ.get("HERMES_HOME") or str(Path.home() / ".hermes")
|
||||
return Path(hermes_home) / "watcher-state"
|
||||
|
||||
|
||||
class Watermark:
|
||||
"""Per-watcher state. Persisted to <state_dir>/<name>.json."""
|
||||
|
||||
def __init__(self, name: str, *, max_seen: int = 500) -> None:
|
||||
if not name or not name.replace("-", "").replace("_", "").isalnum():
|
||||
raise ValueError(
|
||||
f"watermark name must be alphanumeric + '-'/'_' (got {name!r})"
|
||||
)
|
||||
self.name = name
|
||||
self.max_seen = max_seen
|
||||
self._path = _state_dir() / f"{name}.json"
|
||||
self._data: Dict[str, Any] = {"seen_ids": [], "first_run": True}
|
||||
|
||||
@classmethod
|
||||
def load(cls, name: str, *, max_seen: int = 500) -> "Watermark":
|
||||
wm = cls(name, max_seen=max_seen)
|
||||
if wm._path.exists():
|
||||
try:
|
||||
wm._data = json.loads(wm._path.read_text(encoding="utf-8"))
|
||||
wm._data.setdefault("seen_ids", [])
|
||||
wm._data["first_run"] = False
|
||||
except (OSError, json.JSONDecodeError):
|
||||
# Corrupt state file — treat as a first run but don't crash.
|
||||
wm._data = {"seen_ids": [], "first_run": True}
|
||||
return wm
|
||||
|
||||
@property
|
||||
def is_first_run(self) -> bool:
|
||||
return bool(self._data.get("first_run", True))
|
||||
|
||||
@property
|
||||
def seen(self) -> List[str]:
|
||||
return list(self._data.get("seen_ids", []))
|
||||
|
||||
def filter_new(
|
||||
self, items: Iterable[Dict[str, Any]], *, id_key: str = "id"
|
||||
) -> List[Dict[str, Any]]:
|
||||
"""Return items whose id isn't in the stored set.
|
||||
|
||||
Side effect: updates the in-memory seen set with every id in the
|
||||
batch (so save() persists the full new watermark). On first run,
|
||||
records every id but returns an empty list (baseline, no replay).
|
||||
"""
|
||||
existing = set(str(x) for x in self._data.get("seen_ids", []))
|
||||
was_first_run = self.is_first_run
|
||||
|
||||
new_items: List[Dict[str, Any]] = []
|
||||
batch_ids: List[str] = []
|
||||
for item in items:
|
||||
ident = item.get(id_key)
|
||||
if ident is None:
|
||||
continue
|
||||
ident_str = str(ident)
|
||||
batch_ids.append(ident_str)
|
||||
if ident_str in existing:
|
||||
continue
|
||||
if was_first_run:
|
||||
continue # record but don't emit
|
||||
new_items.append(item)
|
||||
|
||||
combined = list(existing) + [i for i in batch_ids if i not in existing]
|
||||
if len(combined) > self.max_seen:
|
||||
combined = combined[-self.max_seen:]
|
||||
self._data["seen_ids"] = combined
|
||||
self._data["first_run"] = False
|
||||
return new_items
|
||||
|
||||
def save(self) -> None:
|
||||
self._path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = self._path.with_suffix(".tmp")
|
||||
tmp.write_text(
|
||||
json.dumps(self._data, indent=2, sort_keys=True),
|
||||
encoding="utf-8",
|
||||
)
|
||||
os.replace(tmp, self._path)
|
||||
|
||||
|
||||
def format_items_as_markdown(
|
||||
items: List[Dict[str, Any]],
|
||||
*,
|
||||
title_key: str = "title",
|
||||
url_key: str = "url",
|
||||
body_key: Optional[str] = None,
|
||||
max_body_chars: int = 500,
|
||||
) -> str:
|
||||
"""Render a list of items as Markdown for cron delivery.
|
||||
|
||||
One heading per item + its URL + optional snippet of body. Output is
|
||||
empty string when items is empty — cron will then treat stdout as
|
||||
silent and skip delivery (existing behavior).
|
||||
"""
|
||||
if not items:
|
||||
return ""
|
||||
lines: List[str] = []
|
||||
for item in items:
|
||||
title = (item.get(title_key) or "(no title)").strip()
|
||||
url = (item.get(url_key) or "").strip()
|
||||
lines.append(f"## {title}")
|
||||
if url:
|
||||
lines.append(url)
|
||||
if body_key:
|
||||
body = (item.get(body_key) or "").strip()
|
||||
if body:
|
||||
if len(body) > max_body_chars:
|
||||
body = body[:max_body_chars].rstrip() + "…"
|
||||
lines.append("")
|
||||
lines.append(body)
|
||||
lines.append("")
|
||||
return "\n".join(lines).rstrip() + "\n"
|
||||
+169
@@ -0,0 +1,169 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Watch GitHub activity — issues, pulls, releases, or commits — with dedup.
|
||||
|
||||
Usage (via cron with --no-agent):
|
||||
|
||||
hermes cron create hermes-issues \\
|
||||
--schedule "*/5 * * * *" --no-agent \\
|
||||
--script "$HERMES_HOME/skills/devops/watchers/scripts/watch_github.py" \\
|
||||
--script-args "--name hermes-issues --repo NousResearch/hermes-agent --scope issues"
|
||||
|
||||
Set GITHUB_TOKEN (or GH_TOKEN) in the Hermes .env file
|
||||
(``${HERMES_HOME:-~/.hermes}/.env``) to avoid the 60 req/hr
|
||||
anonymous rate limit.
|
||||
|
||||
Scopes: issues | pulls | releases | commits. Or pass --search QUERY to
|
||||
use the /search/issues endpoint instead of /repos/:owner/:repo/:scope.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import re
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _watermark import Watermark, format_items_as_markdown # type: ignore
|
||||
|
||||
|
||||
VALID_SCOPES = ("issues", "pulls", "releases", "commits")
|
||||
|
||||
|
||||
def _flatten_commit(item):
|
||||
"""Commit objects nest title/author/date under 'commit' — flatten for rendering."""
|
||||
commit = item.get("commit") or {}
|
||||
msg = (commit.get("message") or "").strip().splitlines()
|
||||
title = msg[0] if msg else ""
|
||||
body = "\n".join(msg[1:]).strip() if len(msg) > 1 else ""
|
||||
author = (item.get("author") or {}).get("login") or (commit.get("author") or {}).get("name", "")
|
||||
date = (commit.get("author") or {}).get("date", "")
|
||||
return {
|
||||
"id": item.get("sha", ""),
|
||||
"title": f"{title} ({author})" if author else title,
|
||||
"url": item.get("html_url"),
|
||||
"body": body,
|
||||
"created_at": date,
|
||||
}
|
||||
|
||||
|
||||
def _flatten_issue_or_release(item):
|
||||
return {
|
||||
"id": str(item.get("id", "")),
|
||||
"title": item.get("title") or item.get("name") or "",
|
||||
"url": item.get("html_url") or item.get("url"),
|
||||
"body": (item.get("body") or "").strip(),
|
||||
"state": item.get("state"),
|
||||
"author": (item.get("user") or {}).get("login")
|
||||
or (item.get("author") or {}).get("login"),
|
||||
"created_at": item.get("created_at"),
|
||||
}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description="Watch GitHub issues / pulls / releases / commits.")
|
||||
p.add_argument("--name", required=True, help="Watcher name (used for state file)")
|
||||
p.add_argument("--repo", default="",
|
||||
help="owner/name of the repo (one of --repo or --search is required)")
|
||||
p.add_argument("--scope", default="issues", choices=VALID_SCOPES,
|
||||
help="What to poll (default: issues)")
|
||||
p.add_argument("--search", default="",
|
||||
help="GitHub issues search query (alternative to --repo/--scope)")
|
||||
p.add_argument("--per-page", type=int, default=30,
|
||||
help="Results per page (default: 30, max: 100)")
|
||||
p.add_argument("--max", type=int, default=20,
|
||||
help="Max new items to emit per tick (default: 20)")
|
||||
p.add_argument("--with-body", action="store_true",
|
||||
help="Include issue/commit body as a snippet under each item")
|
||||
p.add_argument("--timeout", type=float, default=30.0,
|
||||
help="HTTP timeout in seconds (default: 30)")
|
||||
args = p.parse_args()
|
||||
|
||||
if not args.repo and not args.search:
|
||||
print("watch_github: one of --repo or --search is required", file=sys.stderr)
|
||||
return 2
|
||||
if args.repo and not re.fullmatch(r"[A-Za-z0-9._-]+/[A-Za-z0-9._-]+", args.repo):
|
||||
print(f"watch_github: --repo must be owner/name (got {args.repo!r})", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
# URL + flattening strategy.
|
||||
if args.search:
|
||||
url = (
|
||||
"https://api.github.com/search/issues"
|
||||
f"?q={urllib.parse.quote(args.search)}&per_page={args.per_page}"
|
||||
)
|
||||
flatten = _flatten_issue_or_release
|
||||
items_path = "items"
|
||||
elif args.scope == "commits":
|
||||
url = f"https://api.github.com/repos/{args.repo}/commits?per_page={args.per_page}"
|
||||
flatten = _flatten_commit
|
||||
items_path = ""
|
||||
else:
|
||||
url = (
|
||||
f"https://api.github.com/repos/{args.repo}/{args.scope}"
|
||||
f"?per_page={args.per_page}&state=all"
|
||||
)
|
||||
flatten = _flatten_issue_or_release
|
||||
items_path = ""
|
||||
|
||||
headers = {
|
||||
"Accept": "application/vnd.github+json",
|
||||
"User-Agent": "Hermes-Watcher/1.0",
|
||||
}
|
||||
token = os.environ.get("GITHUB_TOKEN") or os.environ.get("GH_TOKEN")
|
||||
if token:
|
||||
headers["Authorization"] = f"Bearer {token}"
|
||||
|
||||
req = urllib.request.Request(url)
|
||||
for k, v in headers.items():
|
||||
req.add_header(k, v)
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=args.timeout) as resp:
|
||||
raw = resp.read()
|
||||
except urllib.error.HTTPError as e:
|
||||
print(f"watch_github: HTTP {e.code} from {url}", file=sys.stderr)
|
||||
return 2
|
||||
except (urllib.error.URLError, TimeoutError, OSError) as e:
|
||||
print(f"watch_github: network error: {e}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
try:
|
||||
data = json.loads(raw.decode("utf-8"))
|
||||
except (UnicodeDecodeError, json.JSONDecodeError) as e:
|
||||
print(f"watch_github: response is not valid JSON: {e}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
# Drill into items_path if needed (search endpoint returns {"items":[...]}).
|
||||
if items_path:
|
||||
data = data.get(items_path) if isinstance(data, dict) else None
|
||||
if not isinstance(data, list):
|
||||
print(f"watch_github: expected a list of items; got {type(data).__name__}",
|
||||
file=sys.stderr)
|
||||
return 2
|
||||
|
||||
items = [flatten(i) for i in data if isinstance(i, dict)]
|
||||
# Drop any items that flattened without an ID (defensive).
|
||||
items = [i for i in items if i.get("id")]
|
||||
|
||||
wm = Watermark.load(args.name)
|
||||
new_items = wm.filter_new(items, id_key="id")
|
||||
wm.save()
|
||||
|
||||
if args.max > 0:
|
||||
new_items = new_items[: args.max]
|
||||
|
||||
body_key = "body" if args.with_body else None
|
||||
output = format_items_as_markdown(new_items, body_key=body_key)
|
||||
if output:
|
||||
sys.stdout.write(output)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
+131
@@ -0,0 +1,131 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Watch any JSON endpoint that returns a list of objects; dedup by ID field.
|
||||
|
||||
Usage (via cron with --no-agent):
|
||||
|
||||
hermes cron create api-events \\
|
||||
--schedule "*/1 * * * *" --no-agent \\
|
||||
--script "$HERMES_HOME/skills/devops/watchers/scripts/watch_http_json.py" \\
|
||||
--script-args "--name api --url https://api.example.com/events \\
|
||||
--id-field event_id --items-path data.events"
|
||||
|
||||
The response can be:
|
||||
- a top-level JSON list (default), or
|
||||
- a JSON object with a dotted ``--items-path`` pointing to the list.
|
||||
|
||||
Each item is deduped by ``--id-field`` (default "id").
|
||||
|
||||
Optional ``--header KEY:VALUE`` flags pass HTTP headers (repeatable).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import json
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _watermark import Watermark, format_items_as_markdown # type: ignore
|
||||
|
||||
|
||||
def _dig(obj, path: str):
|
||||
"""Dotted-path lookup: _dig({'a':{'b':[1,2]}}, 'a.b') → [1,2]."""
|
||||
if not path:
|
||||
return obj
|
||||
cur = obj
|
||||
for part in path.split("."):
|
||||
if isinstance(cur, dict) and part in cur:
|
||||
cur = cur[part]
|
||||
else:
|
||||
return None
|
||||
return cur
|
||||
|
||||
|
||||
def _parse_header(s: str):
|
||||
if ":" not in s:
|
||||
raise argparse.ArgumentTypeError(
|
||||
f"--header expects 'KEY: VALUE' (got {s!r})"
|
||||
)
|
||||
k, v = s.split(":", 1)
|
||||
return (k.strip(), v.strip())
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description="Poll a JSON endpoint.")
|
||||
p.add_argument("--name", required=True, help="Watcher name (used for state file)")
|
||||
p.add_argument("--url", required=True, help="JSON endpoint URL")
|
||||
p.add_argument("--id-field", default="id",
|
||||
help="Field used to dedup items (default: 'id')")
|
||||
p.add_argument("--items-path", default="",
|
||||
help="Dotted path to the list inside the JSON response (e.g. 'data.events')")
|
||||
p.add_argument("--title-field", default="title",
|
||||
help="Field used as the item title in the rendered output (default: 'title')")
|
||||
p.add_argument("--url-field", default="url",
|
||||
help="Field used as the item URL in the rendered output (default: 'url')")
|
||||
p.add_argument("--body-field", default="",
|
||||
help="Optional body field to include as a snippet under each item")
|
||||
p.add_argument("--max", type=int, default=20,
|
||||
help="Max new items to emit per tick (default: 20)")
|
||||
p.add_argument("--header", action="append", type=_parse_header, default=[],
|
||||
metavar="KEY: VALUE",
|
||||
help="HTTP header (repeatable)")
|
||||
p.add_argument("--timeout", type=float, default=20.0,
|
||||
help="HTTP timeout in seconds (default: 20)")
|
||||
args = p.parse_args()
|
||||
|
||||
req = urllib.request.Request(args.url, headers={"User-Agent": "Hermes-Watcher/1.0"})
|
||||
for k, v in args.header:
|
||||
req.add_header(k, v)
|
||||
|
||||
try:
|
||||
with urllib.request.urlopen(req, timeout=args.timeout) as resp:
|
||||
raw = resp.read()
|
||||
except urllib.error.HTTPError as e:
|
||||
print(f"watch_http_json: HTTP {e.code} from {args.url}", file=sys.stderr)
|
||||
return 2
|
||||
except (urllib.error.URLError, TimeoutError, OSError) as e:
|
||||
print(f"watch_http_json: network error: {e}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
try:
|
||||
data = json.loads(raw.decode("utf-8"))
|
||||
except (UnicodeDecodeError, json.JSONDecodeError) as e:
|
||||
print(f"watch_http_json: response is not valid JSON: {e}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
items = _dig(data, args.items_path) if args.items_path else data
|
||||
if not isinstance(items, list):
|
||||
print(
|
||||
f"watch_http_json: items_path={args.items_path!r} did not resolve to a list "
|
||||
f"(got {type(items).__name__})",
|
||||
file=sys.stderr,
|
||||
)
|
||||
return 2
|
||||
|
||||
# Keep only dicts — skip any bare strings / numbers so filter_new doesn't crash.
|
||||
items = [i for i in items if isinstance(i, dict)]
|
||||
|
||||
wm = Watermark.load(args.name)
|
||||
new_items = wm.filter_new(items, id_key=args.id_field)
|
||||
wm.save()
|
||||
|
||||
if args.max > 0:
|
||||
new_items = new_items[: args.max]
|
||||
|
||||
body_key = args.body_field or None
|
||||
output = format_items_as_markdown(
|
||||
new_items,
|
||||
title_key=args.title_field,
|
||||
url_key=args.url_field,
|
||||
body_key=body_key,
|
||||
)
|
||||
if output:
|
||||
sys.stdout.write(output)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
+121
@@ -0,0 +1,121 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Watch an RSS 2.0 or Atom feed; print new items to stdout, silent on empty.
|
||||
|
||||
Usage (via cron with --no-agent):
|
||||
|
||||
hermes cron create my-feed \\
|
||||
--schedule "*/15 * * * *" --no-agent \\
|
||||
--script "$HERMES_HOME/skills/devops/watchers/scripts/watch_rss.py" \\
|
||||
--script-args "--name hn --url https://news.ycombinator.com/rss"
|
||||
|
||||
First run records a baseline (emits nothing). Subsequent runs emit only
|
||||
items whose <guid> / <id> isn't in the watermark.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from xml.etree import ElementTree as ET
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
from _watermark import Watermark, format_items_as_markdown # type: ignore
|
||||
|
||||
|
||||
def _strip_ns(tag: str) -> str:
|
||||
return tag.split("}", 1)[1] if "}" in tag else tag
|
||||
|
||||
|
||||
def _parse_feed(xml_bytes: bytes):
|
||||
"""Return a list of {id, title, url, summary} dicts.
|
||||
|
||||
Handles both RSS 2.0 ``<item>`` and Atom ``<entry>``.
|
||||
"""
|
||||
try:
|
||||
root = ET.fromstring(xml_bytes)
|
||||
except ET.ParseError as e:
|
||||
print(f"watch_rss: invalid XML: {e}", file=sys.stderr)
|
||||
sys.exit(2)
|
||||
|
||||
entries = []
|
||||
for item in root.iter():
|
||||
tag = _strip_ns(item.tag)
|
||||
if tag not in {"item", "entry"}:
|
||||
continue
|
||||
# ElementTree Elements without children are *falsy* — use `is not None`.
|
||||
children = {_strip_ns(c.tag): c for c in item}
|
||||
|
||||
guid_el = children.get("guid")
|
||||
if guid_el is None:
|
||||
guid_el = children.get("id")
|
||||
link_el = children.get("link")
|
||||
if link_el is not None:
|
||||
href = link_el.attrib.get("href") or (link_el.text or "").strip()
|
||||
else:
|
||||
href = ""
|
||||
guid = (guid_el.text or "").strip() if guid_el is not None else ""
|
||||
guid = guid or href
|
||||
if not guid:
|
||||
continue
|
||||
|
||||
title_el = children.get("title")
|
||||
title = (title_el.text or "").strip() if title_el is not None else ""
|
||||
|
||||
summ_el = children.get("description")
|
||||
if summ_el is None:
|
||||
summ_el = children.get("summary")
|
||||
summary = (summ_el.text or "").strip() if summ_el is not None else ""
|
||||
|
||||
entries.append(
|
||||
{"id": guid, "title": title, "url": href, "summary": summary}
|
||||
)
|
||||
return entries
|
||||
|
||||
|
||||
def main() -> int:
|
||||
p = argparse.ArgumentParser(description="Watch an RSS/Atom feed.")
|
||||
p.add_argument("--name", required=True, help="Watcher name (used for state file)")
|
||||
p.add_argument("--url", required=True, help="Feed URL")
|
||||
p.add_argument("--max", type=int, default=10,
|
||||
help="Max new items to emit per tick (default: 10)")
|
||||
p.add_argument("--with-summary", action="store_true",
|
||||
help="Include <description>/<summary> snippet under each item")
|
||||
p.add_argument("--timeout", type=float, default=20.0,
|
||||
help="HTTP timeout in seconds (default: 20)")
|
||||
args = p.parse_args()
|
||||
|
||||
try:
|
||||
req = urllib.request.Request(args.url, headers={"User-Agent": "Hermes-Watcher/1.0"})
|
||||
with urllib.request.urlopen(req, timeout=args.timeout) as resp:
|
||||
xml_bytes = resp.read()
|
||||
except urllib.error.HTTPError as e:
|
||||
print(f"watch_rss: HTTP {e.code} from {args.url}", file=sys.stderr)
|
||||
return 2
|
||||
except (urllib.error.URLError, TimeoutError, OSError) as e:
|
||||
print(f"watch_rss: network error: {e}", file=sys.stderr)
|
||||
return 2
|
||||
|
||||
entries = _parse_feed(xml_bytes)
|
||||
|
||||
wm = Watermark.load(args.name)
|
||||
new_items = wm.filter_new(entries, id_key="id")
|
||||
wm.save()
|
||||
|
||||
# Cap emitted items (watermark still records all seen IDs so we don't
|
||||
# re-emit them next tick).
|
||||
if args.max > 0:
|
||||
new_items = new_items[: args.max]
|
||||
|
||||
body_key = "summary" if args.with_summary else None
|
||||
output = format_items_as_markdown(new_items, body_key=body_key)
|
||||
if output:
|
||||
sys.stdout.write(output)
|
||||
# Empty stdout on no-new — cron treats that as silent.
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user