first-commit
ci / Validate workspace (push) Has been cancelled
landing-page-ci / Validate landing page (push) Has been cancelled
landing-page-deploy / Deploy landing page (push) Has been cancelled
github-metrics / Generate repository metrics SVG (push) Has been cancelled
refresh-contributors-wall / Refresh contributors wall cache bust (push) Waiting to run
ci / Validate workspace (push) Has been cancelled
landing-page-ci / Validate landing page (push) Has been cancelled
landing-page-deploy / Deploy landing page (push) Has been cancelled
github-metrics / Generate repository metrics SVG (push) Has been cancelled
refresh-contributors-wall / Refresh contributors wall cache bust (push) Waiting to run
This commit is contained in:
Zakaria
@@ -0,0 +1,121 @@
|
||||
# UI 用例库
|
||||
|
||||
这个目录是 UI 自动化场景的来源库。
|
||||
|
||||
## 目的
|
||||
|
||||
用例库把这三层拆开：
|
||||
|
||||
- 场景设计
|
||||
- 自动化实现
|
||||
- 测试素材和运行数据
|
||||
|
||||
这样 Playwright spec 不会慢慢变成一堆写死的 prompt 和一次性断言。
|
||||
|
||||
## 当前目录结构
|
||||
|
||||
- [index.ts](/Users/mac/open-design/open-design/e2e/cases/index.ts)：用例定义
|
||||
- [types.ts](/Users/mac/open-design/open-design/e2e/cases/types.ts)：用例 schema
|
||||
- [modules/project-and-generation.md](/Users/mac/open-design/open-design/e2e/cases/modules/project-and-generation.md)：项目创建与生成链路用例
|
||||
- [modules/conversations.md](/Users/mac/open-design/open-design/e2e/cases/modules/conversations.md)：会话生命周期用例
|
||||
- [modules/files.md](/Users/mac/open-design/open-design/e2e/cases/modules/files.md)：文件上传、mention、预览恢复用例
|
||||
- [../reports/README.zh-CN.md](/Users/mac/open-design/open-design/e2e/reports/README.zh-CN.md)：测试结果与报告说明
|
||||
- [../specs/app.spec.ts](/Users/mac/open-design/open-design/e2e/specs/app.spec.ts)：执行已自动化用例的 Playwright 入口
|
||||
|
||||
## Schema 说明
|
||||
|
||||
每条用例都是一个 `UICase`。
|
||||
|
||||
- `id`：稳定的用例标识，用于 spec 和测试报告
|
||||
- `title`：人可读的用例名称
|
||||
- `kind`：项目类型，比如 `prototype`、`deck`、`workspace`
|
||||
- `flow`：Playwright 里对应的自动化流程分支
|
||||
- `automated`：当前是否会被 `pnpm run test:ui` 执行
|
||||
- `description`：覆盖目标和场景说明
|
||||
- `create`：创建项目时要用到的输入
|
||||
- `prompt`：主输入内容
|
||||
- `secondaryPrompt`：多步骤流程里的后续输入
|
||||
- `mockArtifact`：mock SSE 时预期生成的 artifact
|
||||
- `notes`：实现细节或维护备注
|
||||
|
||||
## 当前支持的 Flow
|
||||
|
||||
- `standard`：创建项目，发送 prompt，校验生成 artifact
|
||||
- `conversation-persistence`：创建多会话，刷新后恢复，再切换历史
|
||||
- `file-mention`：预置文件后通过 `@` mention 选中并校验 staged attachment
|
||||
- `deep-link-preview`：通过文件路由打开预览并校验恢复
|
||||
- `file-upload-send`：走真实文件选择器，校验上传和发送
|
||||
- `conversation-delete-recovery`：删除当前活跃会话后校验回退
|
||||
|
||||
## 文档拆分规则
|
||||
|
||||
- `README.zh-CN.md` 只保留总览、结构和维护规则
|
||||
- 具体用例清单按模块拆到 `modules/` 目录
|
||||
- 一个模块一个 Markdown，后面可以继续细分
|
||||
- 当单个模块内容变长时，再继续按子模块拆分
|
||||
|
||||
## 新增用例的方式
|
||||
|
||||
1. 在 [index.ts](/Users/mac/open-design/open-design/e2e/cases/index.ts) 里新增一条 `UICase`。
|
||||
2. 先把场景写进对应模块文档，如果只是设计阶段，保持 `automated: false`。
|
||||
3. 能复用已有 `flow` 就优先复用。
|
||||
4. 只有在确实需要新自动化路径时，才去 [types.ts](/Users/mac/open-design/open-design/e2e/cases/types.ts) 增加新的 `flow` 类型。
|
||||
5. 在 [app.spec.ts](/Users/mac/open-design/open-design/e2e/specs/app.spec.ts) 里实现这个流程。
|
||||
6. 用例稳定后，再把 `automated` 改成 `true`。
|
||||
|
||||
## 推荐工作流
|
||||
|
||||
1. 先用产品语言把场景写清楚。
|
||||
2. 先决定它归哪个模块文档。
|
||||
3. 判断它能不能归到已有的自动化 flow。
|
||||
4. 只在确实需要的节点补 `data-testid`。
|
||||
5. 优先 mock `/api/chat` 的 SSE，保证稳定性。
|
||||
6. 项目创建、路由、持久化、文件 API 尽量走真实链路。
|
||||
|
||||
## 适合放进来的范围
|
||||
|
||||
适合：
|
||||
|
||||
- 项目创建主流程
|
||||
- 生成与 artifact 预览流程
|
||||
- 会话生命周期流程
|
||||
- 文件上传、mention、重新打开流程
|
||||
- deep link 和刷新恢复流程
|
||||
|
||||
不建议优先放：
|
||||
|
||||
- 纯视觉、容易抖的检查
|
||||
- 模型质量评估
|
||||
- 强依赖真实外部 agent CLI 的测试
|
||||
|
||||
## 运行方式
|
||||
|
||||
```bash
|
||||
pnpm run test:ui
|
||||
```
|
||||
|
||||
也可以直接在独立测试包内运行：
|
||||
|
||||
```bash
|
||||
pnpm --filter @open-design/e2e test:ui
|
||||
```
|
||||
|
||||
运行完成后会自动生成：
|
||||
|
||||
- `e2e/reports/latest.md`
|
||||
- `e2e/reports/ui-test-report.html`
|
||||
- `e2e/reports/playwright-html-report/`
|
||||
- `e2e/reports/results.json`
|
||||
- `e2e/reports/junit.xml`
|
||||
|
||||
运行开始前会自动清理旧的 e2e 运行时数据和上一次报告，避免：
|
||||
|
||||
- `.od-data` 里累积空 project 目录
|
||||
- `e2e/reports/test-results` 混入旧失败截图
|
||||
- 报告内容和本次执行结果不一致
|
||||
|
||||
如果要带界面调试：
|
||||
|
||||
```bash
|
||||
pnpm run test:ui:headed
|
||||
```
|
||||
@@ -0,0 +1,405 @@
|
||||
import type { UICase } from './types';
|
||||
|
||||
export const uiCases: UICase[] = [
|
||||
{
|
||||
id: 'prototype-basic',
|
||||
title: 'Prototype project creates and previews a generated artifact',
|
||||
kind: 'prototype',
|
||||
flow: 'standard',
|
||||
automated: true,
|
||||
description:
|
||||
'Validates the primary happy path: create a prototype project, send one prompt, persist the generated HTML, and render it in the preview iframe.',
|
||||
create: {
|
||||
projectName: 'UI automation smoke',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Create a small test artifact',
|
||||
mockArtifact: {
|
||||
identifier: 'mock-artifact',
|
||||
title: 'Mock Artifact',
|
||||
fileName: 'mock-artifact.html',
|
||||
heading: 'Mock Artifact',
|
||||
html:
|
||||
'<!doctype html><html><body><main><h1>Mock Artifact</h1><p>Generated by Playwright.</p></main></body></html>',
|
||||
},
|
||||
notes: [
|
||||
'This is the seed smoke test and should stay fast.',
|
||||
'It uses mocked SSE so the UI path stays deterministic.',
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'deck-basic',
|
||||
title: 'Deck project renders a mocked slide artifact',
|
||||
kind: 'deck',
|
||||
flow: 'standard',
|
||||
automated: true,
|
||||
description:
|
||||
'Covers the deck tab in project creation and verifies that a deck artifact lands in the workspace preview.',
|
||||
create: {
|
||||
projectName: 'Deck automation smoke',
|
||||
tab: 'deck',
|
||||
},
|
||||
prompt: 'Create a short deck with two slides',
|
||||
mockArtifact: {
|
||||
identifier: 'mock-deck',
|
||||
title: 'Mock Deck',
|
||||
fileName: 'mock-deck.html',
|
||||
heading: 'Mock Deck',
|
||||
html:
|
||||
'<!doctype html><html><body><section class="slide"><h1>Mock Deck</h1></section></body></html>',
|
||||
},
|
||||
notes: [
|
||||
'Confirms the deck creation tab still routes into the same generation path.',
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'comment-attachment-flow',
|
||||
title: 'Preview comments attach to chat and send as structured context',
|
||||
kind: 'prototype',
|
||||
flow: 'comment-attachment-flow',
|
||||
automated: true,
|
||||
description:
|
||||
'Exercises V1 comment mode: save a latest element comment, attach/remove it from the composer, and send it as an empty visible prompt with structured comment context.',
|
||||
create: {
|
||||
projectName: 'Comment attachment flow',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Create a commentable preview artifact',
|
||||
mockArtifact: {
|
||||
identifier: 'commentable-artifact',
|
||||
title: 'Commentable Artifact',
|
||||
fileName: 'commentable-artifact.html',
|
||||
heading: 'Prototype headline',
|
||||
html:
|
||||
'<!doctype html><html><body><main data-od-id="hero-section"><h1 data-od-id="hero-title" data-screen-label="Hero title">Prototype headline</h1><p data-od-id="hero-copy">Preview copy for comment mode.</p></main></body></html>',
|
||||
},
|
||||
notes: [
|
||||
'The composer textarea stays empty; selected preview comments are sent through commentAttachments.',
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'design-system-selection',
|
||||
title: 'Selecting a design system carries through project creation',
|
||||
kind: 'prototype',
|
||||
flow: 'design-system-selection',
|
||||
automated: true,
|
||||
description:
|
||||
'Verifies that a chosen design system is selectable in the new-project panel and remains visible in project metadata after creation.',
|
||||
create: {
|
||||
projectName: 'Design system selection',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Create a small test artifact',
|
||||
notes: [
|
||||
'Uses a mocked design-system list so the picker stays deterministic across environments.',
|
||||
'Focuses on creation and metadata persistence instead of generation output.',
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'example-use-prompt',
|
||||
title: 'Using an example prompt creates a project with a seeded draft',
|
||||
kind: 'prototype',
|
||||
flow: 'example-use-prompt',
|
||||
automated: true,
|
||||
description:
|
||||
'Verifies the Examples tab fast path: click Use this prompt, create a project immediately, and carry the example prompt into the chat composer.',
|
||||
create: {
|
||||
projectName: 'Example prompt project',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Draft a warm utility landing page for a productivity app',
|
||||
notes: [
|
||||
'Uses a mocked skills list so the examples gallery stays deterministic.',
|
||||
'Targets the pendingPrompt fast-create path instead of the standard new-project form.',
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'conversation-persistence',
|
||||
title: 'Conversation history survives refresh and switching',
|
||||
kind: 'workspace',
|
||||
flow: 'conversation-persistence',
|
||||
automated: true,
|
||||
description:
|
||||
'Exercises conversation creation, persistence, refresh reload, and switching between threads in one project.',
|
||||
create: {
|
||||
projectName: 'Conversation persistence',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Create a small test artifact',
|
||||
secondaryPrompt: 'Create another artifact in a fresh conversation',
|
||||
mockArtifact: {
|
||||
identifier: 'mock-artifact',
|
||||
title: 'Mock Artifact',
|
||||
fileName: 'mock-artifact.html',
|
||||
heading: 'Mock Artifact',
|
||||
html:
|
||||
'<!doctype html><html><body><main><h1>Mock Artifact</h1><p>Generated by Playwright.</p></main></body></html>',
|
||||
},
|
||||
notes: [
|
||||
'Should use the same mock SSE flow as the prototype smoke path.',
|
||||
'Reload should keep the original conversation content available from the history menu.',
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'file-mention',
|
||||
title: 'Uploaded files can be mentioned and sent back to the agent',
|
||||
kind: 'workspace',
|
||||
flow: 'file-mention',
|
||||
automated: true,
|
||||
description:
|
||||
'Validates the upload, staged attachment, and @ mention flow inside the chat composer.',
|
||||
create: {
|
||||
projectName: 'File mention flow',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Review @reference.txt and use it as context',
|
||||
notes: [
|
||||
'Seeds a tiny text fixture through the project file API, then exercises the composer mention flow.',
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'deep-link-preview',
|
||||
title: 'Deep-linking to a file route opens the expected preview tab',
|
||||
kind: 'workspace',
|
||||
flow: 'deep-link-preview',
|
||||
automated: true,
|
||||
description:
|
||||
'Verifies that /projects/:id/files/:name restores the matching open tab and preview frame after navigation or refresh.',
|
||||
create: {
|
||||
projectName: 'Deep link preview',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Create a small test artifact',
|
||||
mockArtifact: {
|
||||
identifier: 'mock-artifact',
|
||||
title: 'Mock Artifact',
|
||||
fileName: 'mock-artifact.html',
|
||||
heading: 'Mock Artifact',
|
||||
html:
|
||||
'<!doctype html><html><body><main><h1>Mock Artifact</h1><p>Generated by Playwright.</p></main></body></html>',
|
||||
},
|
||||
notes: [
|
||||
'Can reuse the generated HTML from prototype-basic, then revisit with a routed URL.',
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'file-upload-send',
|
||||
title: 'Composer file picker uploads a file and sends it with the prompt',
|
||||
kind: 'workspace',
|
||||
flow: 'file-upload-send',
|
||||
automated: true,
|
||||
description:
|
||||
'Exercises the real attach button and hidden file input, then verifies the staged file is sent and shown back on the user message.',
|
||||
create: {
|
||||
projectName: 'File upload send flow',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Use the uploaded reference as context',
|
||||
notes: [
|
||||
'Uses Playwright setInputFiles on the hidden composer picker instead of seeding through the API.',
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'design-files-upload',
|
||||
title: 'Design Files panel uploads an image and opens it in the workspace',
|
||||
kind: 'workspace',
|
||||
flow: 'design-files-upload',
|
||||
automated: true,
|
||||
description:
|
||||
'Exercises the Design Files upload flow in the workspace, then verifies the uploaded image can be previewed and opened as a tab.',
|
||||
create: {
|
||||
projectName: 'Design files upload flow',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Upload an image through the design files browser',
|
||||
notes: [
|
||||
'Uses the FileWorkspace upload input rather than the chat composer upload path.',
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'design-files-delete',
|
||||
title: 'Design Files panel deletes an uploaded file and clears its tab',
|
||||
kind: 'workspace',
|
||||
flow: 'design-files-delete',
|
||||
automated: true,
|
||||
description:
|
||||
'Uploads a file through the Design Files panel, deletes it from the row menu, and verifies it disappears from both the list and open tabs.',
|
||||
create: {
|
||||
projectName: 'Design files delete flow',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Delete an uploaded image through the design files browser',
|
||||
notes: [
|
||||
'Builds on the same workspace file flow as design-files-upload, then verifies cleanup behavior.',
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'design-files-tab-persistence',
|
||||
title: 'Open file tabs survive refresh with the correct active tab',
|
||||
kind: 'workspace',
|
||||
flow: 'design-files-tab-persistence',
|
||||
automated: true,
|
||||
description:
|
||||
'Uploads multiple files through the Design Files flow, switches the active tab, reloads the page, and verifies both the tab set and selected tab are restored.',
|
||||
create: {
|
||||
projectName: 'Design files tab persistence',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Restore open file tabs after refresh',
|
||||
notes: [
|
||||
'Covers the persisted tabs state stored by ProjectView and restored by FileWorkspace.',
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'conversation-delete-recovery',
|
||||
title: 'Deleting the active conversation falls back cleanly',
|
||||
kind: 'workspace',
|
||||
flow: 'conversation-delete-recovery',
|
||||
automated: true,
|
||||
description:
|
||||
'Creates multiple conversations, deletes the active one, and verifies the UI falls back to the remaining thread instead of getting stuck.',
|
||||
create: {
|
||||
projectName: 'Conversation delete recovery',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Create a small test artifact',
|
||||
secondaryPrompt: 'Create another artifact before deleting this thread',
|
||||
mockArtifact: {
|
||||
identifier: 'mock-artifact',
|
||||
title: 'Mock Artifact',
|
||||
fileName: 'mock-artifact.html',
|
||||
heading: 'Mock Artifact',
|
||||
html:
|
||||
'<!doctype html><html><body><main><h1>Mock Artifact</h1><p>Generated by Playwright.</p></main></body></html>',
|
||||
},
|
||||
notes: [
|
||||
'Confirms the project still has a live conversation after deleting the current thread.',
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'question-form-selection-limit',
|
||||
title: 'Question form checkbox limits block selecting more than the allowed maximum',
|
||||
kind: 'workspace',
|
||||
flow: 'question-form-selection-limit',
|
||||
automated: true,
|
||||
description:
|
||||
'Verifies that a discovery-style checkbox question with maxSelections=2 cannot be pushed past two selected options.',
|
||||
create: {
|
||||
projectName: 'Question form selection limit',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Help me plan a restaurant homepage',
|
||||
notes: [
|
||||
'Mocks a question-form response instead of an artifact so the test can exercise the inline clarifying UI.',
|
||||
'Confirms both the interaction guard and the rendered checked state stay capped at two options.',
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'question-form-submit-persistence',
|
||||
title: 'Question form answers persist into chat history and reload in a locked state',
|
||||
kind: 'workspace',
|
||||
flow: 'question-form-submit-persistence',
|
||||
automated: true,
|
||||
description:
|
||||
'Verifies that answering a question form writes a user follow-up message, then rehydrates the form in an answered and locked state after reload.',
|
||||
create: {
|
||||
projectName: 'Question form submit persistence',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Plan a small restaurant homepage',
|
||||
notes: [
|
||||
'Mocks an inline question form on the first assistant turn and a plain acknowledgment on the follow-up turn.',
|
||||
'Confirms the answered state survives a full page reload instead of relying only on local submit state.',
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'generation-does-not-create-extra-file',
|
||||
title: 'Generated artifacts stay stable when no new prompt is sent',
|
||||
kind: 'workspace',
|
||||
flow: 'generation-does-not-create-extra-file',
|
||||
automated: true,
|
||||
description:
|
||||
'Generates one HTML artifact, then verifies reload and idle time do not create any additional project files without a new user prompt.',
|
||||
create: {
|
||||
projectName: 'No extra generated file',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Create one landing page artifact',
|
||||
mockArtifact: {
|
||||
identifier: 'stable-artifact',
|
||||
title: 'Stable Artifact',
|
||||
fileName: 'stable-artifact.html',
|
||||
heading: 'Stable Artifact',
|
||||
html:
|
||||
'<!doctype html><html><body><main><h1>Stable Artifact</h1><p>Only one file should exist.</p></main></body></html>',
|
||||
},
|
||||
notes: [
|
||||
'Targets the trust-sensitive bug where a project can appear to generate a fresh file on its own.',
|
||||
'Uses the files API after reload to assert the project file set is unchanged.',
|
||||
],
|
||||
},
|
||||
{
|
||||
id: 'deck-pagination-next-prev-correctness',
|
||||
title: 'Deck preview previous and next controls move in the correct direction',
|
||||
kind: 'deck',
|
||||
flow: 'deck-pagination-next-prev-correctness',
|
||||
automated: false,
|
||||
description:
|
||||
'Should verify that deck preview pagination moves to the actual previous and next slide instead of routing both actions to the same page.',
|
||||
create: {
|
||||
projectName: 'Deck pagination controls',
|
||||
tab: 'deck',
|
||||
},
|
||||
prompt: 'Review pagination behavior in a multi-slide deck preview',
|
||||
},
|
||||
{
|
||||
id: 'deck-pagination-per-file-isolated',
|
||||
title: 'Each HTML deck tab preserves its own pagination state',
|
||||
kind: 'deck',
|
||||
flow: 'deck-pagination-per-file-isolated',
|
||||
automated: false,
|
||||
description:
|
||||
'Should verify that switching between multiple deck HTML files does not leak page position across tabs or reset both files to page 1.',
|
||||
create: {
|
||||
projectName: 'Deck pagination isolation',
|
||||
tab: 'deck',
|
||||
},
|
||||
prompt: 'Keep pagination state isolated per generated deck file',
|
||||
},
|
||||
{
|
||||
id: 'uploaded-image-renders-in-preview',
|
||||
title: 'Uploaded reference images render correctly in generated deck preview',
|
||||
kind: 'workspace',
|
||||
flow: 'uploaded-image-renders-in-preview',
|
||||
automated: false,
|
||||
description:
|
||||
'Should verify that uploaded images resolve to loadable src paths inside generated HTML instead of rendering as broken images.',
|
||||
create: {
|
||||
projectName: 'Uploaded image preview render',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Use uploaded brand images inside a generated deck preview',
|
||||
},
|
||||
{
|
||||
id: 'python-source-preview',
|
||||
title: 'Python files should open with a readable inline source preview',
|
||||
kind: 'workspace',
|
||||
flow: 'python-source-preview',
|
||||
automated: false,
|
||||
description:
|
||||
'Should verify that opening a .py file in the main workspace renders a readable source/code preview instead of an unsupported blank state.',
|
||||
create: {
|
||||
projectName: 'Python source preview',
|
||||
tab: 'prototype',
|
||||
},
|
||||
prompt: 'Open a generated Python file and inspect its source inline',
|
||||
notes: [
|
||||
'Candidate follow-up to the Python preview gap in the file viewer.',
|
||||
'Likely automation shape: seed a .py file through the project files API, open it, and assert the viewer renders code text.',
|
||||
],
|
||||
},
|
||||
];
|
||||
|
||||
export function automatedCases(): UICase[] {
|
||||
return uiCases.filter((entry) => entry.automated);
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
# 会话生命周期
|
||||
|
||||
这个模块聚焦项目内聊天会话的生命周期：
|
||||
|
||||
- 新建会话
|
||||
- 切换会话
|
||||
- 刷新恢复
|
||||
- 删除会话
|
||||
- 后续可扩展重命名等场景
|
||||
|
||||
## 当前用例
|
||||
|
||||
### `conversation-persistence`
|
||||
|
||||
- 状态：已自动化
|
||||
- 对应 flow：`conversation-persistence`
|
||||
- 目标：覆盖会话创建、刷新恢复、历史切换
|
||||
- 核心步骤：
|
||||
1. 在第一个会话里发送 prompt
|
||||
2. 新建第二个会话
|
||||
3. 在第二个会话里发送新的 prompt
|
||||
4. 刷新页面
|
||||
5. 校验当前会话内容仍在
|
||||
6. 打开历史菜单切回第一个会话
|
||||
|
||||
### `conversation-delete-recovery`
|
||||
|
||||
- 状态：已自动化
|
||||
- 对应 flow：`conversation-delete-recovery`
|
||||
- 目标：覆盖删除当前活跃会话后的回退逻辑
|
||||
- 核心步骤：
|
||||
1. 创建两个会话
|
||||
2. 删除当前活跃会话
|
||||
3. 校验界面自动回退到剩余会话
|
||||
4. 校验项目仍然保有可用会话
|
||||
|
||||
### `question-form-selection-limit`
|
||||
|
||||
- 状态：已自动化
|
||||
- 对应 flow：`question-form-selection-limit`
|
||||
- 目标：覆盖快速确认里 checkbox 多选上限约束
|
||||
- 核心步骤：
|
||||
1. 创建项目并发送一条 prompt
|
||||
2. mock 返回带 `maxSelections: 2` 的 question form
|
||||
3. 连续点击三个视觉风格选项
|
||||
4. 校验始终只有两个选项处于选中态
|
||||
5. 校验第三个选项不会被错误选中
|
||||
|
||||
### `question-form-submit-persistence`
|
||||
|
||||
- 状态：已自动化
|
||||
- 对应 flow：`question-form-submit-persistence`
|
||||
- 目标：覆盖 question form 提交后的用户回答落盘、锁定态与刷新回填
|
||||
- 核心步骤：
|
||||
1. mock 返回一个带必填项的 question form
|
||||
2. 选择答案并点击提交
|
||||
3. 校验会话里写入了用户回答消息
|
||||
4. 校验原表单进入 answered / locked 状态
|
||||
5. 刷新页面后再次确认锁定态和已选答案仍然正确
|
||||
|
||||
## 推荐后续补充
|
||||
|
||||
- 会话重命名
|
||||
- 删除最后一个会话后的自动重建
|
||||
- 历史菜单关闭/重新打开后的状态一致性
|
||||
- 长会话列表滚动与选中态
|
||||
- 多轮对话后的会话标题生成或更新策略
|
||||
@@ -0,0 +1,121 @@
|
||||
# 文件链路
|
||||
|
||||
这个模块聚焦项目文件相关的主链路：
|
||||
|
||||
- 文件上传
|
||||
- 文件 mention
|
||||
- staged attachment
|
||||
- 文件路由打开
|
||||
- 预览恢复
|
||||
|
||||
## 当前用例
|
||||
|
||||
### `file-mention`
|
||||
|
||||
- 状态：已自动化
|
||||
- 对应 flow：`file-mention`
|
||||
- 目标：覆盖 `@` mention 选择文件并加入 staged attachment
|
||||
- 核心步骤：
|
||||
1. 通过项目文件 API 预置 `reference.txt`
|
||||
2. 在聊天输入框中输入 `@ref`
|
||||
3. 选择 mention popover 里的文件
|
||||
4. 校验输入框中插入 `@reference.txt`
|
||||
5. 校验 staged attachment 显示正确
|
||||
|
||||
### `file-upload-send`
|
||||
|
||||
- 状态：已自动化
|
||||
- 对应 flow：`file-upload-send`
|
||||
- 目标：覆盖聊天区真实上传文件并发送
|
||||
- 核心步骤：
|
||||
1. 通过 composer 的隐藏 file input 上传文件
|
||||
2. 校验 staged attachment 出现
|
||||
3. 发送 prompt
|
||||
4. 校验用户消息里带上上传文件
|
||||
|
||||
### `deep-link-preview`
|
||||
|
||||
- 状态：已自动化
|
||||
- 对应 flow：`deep-link-preview`
|
||||
- 目标：覆盖文件路由直达和预览恢复
|
||||
- 核心步骤：
|
||||
1. 生成 artifact
|
||||
2. 校验 URL 进入 `/projects/:id/files/:name`
|
||||
3. 离开项目文件路由
|
||||
4. 再次通过文件路由进入
|
||||
5. 校验预览 iframe 正常恢复
|
||||
|
||||
### `design-files-upload`
|
||||
|
||||
- 状态：已自动化
|
||||
- 对应 flow：`design-files-upload`
|
||||
- 目标：覆盖 Design Files 面板真实上传、预览与打开
|
||||
- 核心步骤：
|
||||
1. 通过 Design Files 面板的上传入口选择图片
|
||||
2. 校验文件行出现在列表中
|
||||
3. 校验右侧预览信息出现
|
||||
4. 双击文件行
|
||||
5. 校验文件以 tab 形式打开
|
||||
|
||||
### `design-files-delete`
|
||||
|
||||
- 状态：已自动化
|
||||
- 对应 flow：`design-files-delete`
|
||||
- 目标：覆盖 Design Files 面板删除文件以及打开 tab 的清理
|
||||
- 核心步骤：
|
||||
1. 先上传一张图片
|
||||
2. 回到 Design Files 面板
|
||||
3. 打开文件行菜单并执行删除
|
||||
4. 确认文件行从列表中消失
|
||||
5. 确认对应文件 tab 也被清理
|
||||
|
||||
### `design-files-tab-persistence`
|
||||
|
||||
- 状态：已自动化
|
||||
- 对应 flow：`design-files-tab-persistence`
|
||||
- 目标：覆盖多个打开文件 tab 在刷新后的恢复
|
||||
- 核心步骤：
|
||||
1. 先上传两张图片
|
||||
2. 确认两张图片都打开为 tab
|
||||
3. 切换当前 active tab
|
||||
4. 刷新页面
|
||||
5. 确认两个 tab 都被恢复
|
||||
6. 确认刷新前的 active tab 仍然是 active
|
||||
|
||||
## 推荐后续补充
|
||||
|
||||
### `deck-pagination-per-file-isolated`
|
||||
|
||||
- 状态：待自动化
|
||||
- 对应 flow：`deck-pagination-per-file-isolated`
|
||||
- 目标：覆盖多个 deck HTML 之间的分页状态隔离
|
||||
- 核心步骤：
|
||||
1. 打开两个多页 deck 文件
|
||||
2. 分别停留在不同页码
|
||||
3. 来回切换文件 tab
|
||||
4. 校验每个文件维持自己的页码
|
||||
|
||||
### `uploaded-image-renders-in-preview`
|
||||
|
||||
- 状态：待自动化
|
||||
- 对应 flow：`uploaded-image-renders-in-preview`
|
||||
- 目标：覆盖上传图片参与生成后，预览中的图片真实可加载
|
||||
- 核心步骤：
|
||||
1. 上传图片作为参考素材
|
||||
2. 生成引用该图片的 HTML artifact
|
||||
3. 进入预览 iframe
|
||||
4. 校验对应 `img` 的 `src` 可解析且不是 broken image
|
||||
|
||||
### `python-source-preview`
|
||||
|
||||
- 状态：待自动化
|
||||
- 对应 flow：`python-source-preview`
|
||||
- 目标：覆盖 `.py` 文件在主工作区中的源码预览能力
|
||||
- 核心步骤：
|
||||
1. 通过项目文件 API 预置一个 `.py` 文件
|
||||
2. 在主工作区打开该文件
|
||||
3. 校验文件查看器进入源码/文本预览模式
|
||||
4. 校验能看到 Python 源码内容，而不是空白或不支持状态
|
||||
|
||||
- 图片文件上传与缩略图展示
|
||||
- 刷新后 staged attachment 清理策略
|
||||
@@ -0,0 +1,88 @@
|
||||
# 项目创建与生成
|
||||
|
||||
这个模块聚焦主入口链路：
|
||||
|
||||
- 创建项目
|
||||
- 进入工作区
|
||||
- 发送 prompt
|
||||
- 生成 artifact
|
||||
- 打开预览
|
||||
|
||||
## 当前用例
|
||||
|
||||
### `prototype-basic`
|
||||
|
||||
- 状态：已自动化
|
||||
- 对应 flow：`standard`
|
||||
- 目标：覆盖 prototype 项目的主 happy path
|
||||
- 核心步骤：
|
||||
1. 创建 `prototype` 项目
|
||||
2. 输入 prompt
|
||||
3. mock `/api/chat` SSE 返回 HTML artifact
|
||||
4. 校验生成文件出现在工作区
|
||||
5. 校验 iframe 预览正常
|
||||
|
||||
### `deck-basic`
|
||||
|
||||
- 状态：已自动化
|
||||
- 对应 flow：`standard`
|
||||
- 目标：覆盖 deck 项目创建分支
|
||||
- 核心步骤：
|
||||
1. 切换到 `deck` 创建 tab
|
||||
2. 创建项目
|
||||
3. 发送 prompt
|
||||
4. mock 返回 deck artifact
|
||||
5. 校验预览正常
|
||||
|
||||
### `design-system-selection`
|
||||
|
||||
- 状态：已自动化
|
||||
- 对应 flow：`design-system-selection`
|
||||
- 目标：覆盖设计系统选择后创建项目，并确认项目元信息保留了该选择
|
||||
- 核心步骤：
|
||||
1. mock 设计系统列表
|
||||
2. 打开设计系统选择器
|
||||
3. 搜索并选择指定设计系统
|
||||
4. 创建项目
|
||||
5. 校验项目页 meta 中出现设计系统名称
|
||||
|
||||
### `example-use-prompt`
|
||||
|
||||
- 状态：已自动化
|
||||
- 对应 flow：`example-use-prompt`
|
||||
- 目标：覆盖 Examples 页的快捷创建链路
|
||||
- 核心步骤：
|
||||
1. mock skills 列表，提供一个示例卡片
|
||||
2. 切到 Examples 页
|
||||
3. 点击 `Use this prompt`
|
||||
4. 校验项目被直接创建
|
||||
5. 校验聊天输入框预填了 example prompt
|
||||
|
||||
### `generation-does-not-create-extra-file`
|
||||
|
||||
- 状态：已自动化
|
||||
- 对应 flow：`generation-does-not-create-extra-file`
|
||||
- 目标：覆盖“没有新 prompt 却自己多生成一个 HTML 文件”的回归风险
|
||||
- 核心步骤：
|
||||
1. 生成一个 mocked artifact
|
||||
2. 通过 files API 记录当前项目文件集合
|
||||
3. 刷新页面但不发送新 prompt
|
||||
4. 再次读取 files API
|
||||
5. 校验文件集合没有变化，也没有新增 HTML 文件
|
||||
|
||||
## 推荐后续补充
|
||||
|
||||
### `deck-pagination-next-prev-correctness`
|
||||
|
||||
- 状态：待自动化
|
||||
- 对应 flow：`deck-pagination-next-prev-correctness`
|
||||
- 目标：覆盖 deck 预览上一页 / 下一页按钮的方向正确性
|
||||
- 核心步骤：
|
||||
1. 打开多页 deck HTML
|
||||
2. 进入中间页
|
||||
3. 点击上一页并校验页码递减
|
||||
4. 点击下一页并校验页码递增
|
||||
|
||||
- template 项目创建
|
||||
- 创建项目后的刷新恢复
|
||||
- 创建失败或必填校验
|
||||
@@ -0,0 +1,134 @@
|
||||
export interface ReportCaseMetadata {
|
||||
module: string;
|
||||
assertions: string[];
|
||||
}
|
||||
|
||||
const caseMetadata: Record<string, ReportCaseMetadata> = {
|
||||
'prototype-basic': {
|
||||
module: '项目创建与生成',
|
||||
assertions: [
|
||||
'可以创建 prototype 项目并进入工作区',
|
||||
'发送 prompt 后会收到 mocked artifact',
|
||||
'生成文件会出现在工作区',
|
||||
'预览 iframe 中能看到期望标题',
|
||||
],
|
||||
},
|
||||
'deck-basic': {
|
||||
module: '项目创建与生成',
|
||||
assertions: [
|
||||
'可以通过 deck tab 创建项目',
|
||||
'发送 prompt 后会收到 deck artifact',
|
||||
'deck 文件会出现在工作区',
|
||||
'预览 iframe 中能看到期望标题',
|
||||
],
|
||||
},
|
||||
'design-system-selection': {
|
||||
module: '项目创建与生成',
|
||||
assertions: [
|
||||
'设计系统选择器可以搜索并选中目标设计系统',
|
||||
'创建项目后项目 meta 会保留设计系统名称',
|
||||
'项目成功进入工作区而不是停留在创建页',
|
||||
],
|
||||
},
|
||||
'example-use-prompt': {
|
||||
module: '项目创建与生成',
|
||||
assertions: [
|
||||
'Examples 页的 Use this prompt 可以直接创建项目',
|
||||
'创建后的项目标题与 meta 会带上对应 skill 名称',
|
||||
'聊天输入框会预填 example prompt',
|
||||
],
|
||||
},
|
||||
'conversation-persistence': {
|
||||
module: '会话生命周期',
|
||||
assertions: [
|
||||
'可以创建第二个会话并发送新的 prompt',
|
||||
'刷新后当前会话消息仍然存在',
|
||||
'历史菜单中可以切回旧会话',
|
||||
'切回后旧会话内容仍然正确显示',
|
||||
],
|
||||
},
|
||||
'conversation-delete-recovery': {
|
||||
module: '会话生命周期',
|
||||
assertions: [
|
||||
'删除当前活跃会话后不会卡死在空状态',
|
||||
'界面会回退到剩余会话',
|
||||
'被删除会话的消息不会继续显示',
|
||||
],
|
||||
},
|
||||
'question-form-selection-limit': {
|
||||
module: '会话生命周期',
|
||||
assertions: [
|
||||
'question form 中声明 maxSelections=2 的 checkbox 题目最多只能选中两个选项',
|
||||
'达到上限后新的未选项不会被选中',
|
||||
'界面中的已选数量会保持在约束范围内',
|
||||
],
|
||||
},
|
||||
'question-form-submit-persistence': {
|
||||
module: '会话生命周期',
|
||||
assertions: [
|
||||
'提交 question form 后会写入一条用户回答消息',
|
||||
'表单会立即进入 answered / locked 状态',
|
||||
'刷新页面后表单仍会根据历史答案正确回填并保持锁定',
|
||||
],
|
||||
},
|
||||
'generation-does-not-create-extra-file': {
|
||||
module: '项目创建与生成',
|
||||
assertions: [
|
||||
'第一次生成后项目中只出现预期的 artifact 文件',
|
||||
'在没有发送新 prompt 的情况下刷新页面不会新增文件',
|
||||
'files API 返回的文件集合在前后两次检查中保持一致',
|
||||
],
|
||||
},
|
||||
'file-mention': {
|
||||
module: '文件链路',
|
||||
assertions: [
|
||||
'预置文件后 mention popover 可以搜索并选中文件',
|
||||
'输入框会插入 @filename',
|
||||
'staged attachment 会显示对应文件',
|
||||
],
|
||||
},
|
||||
'file-upload-send': {
|
||||
module: '文件链路',
|
||||
assertions: [
|
||||
'聊天区 file input 可以上传文件',
|
||||
'上传后 staged attachment 会显示文件',
|
||||
'发送消息后用户消息中会保留该附件',
|
||||
],
|
||||
},
|
||||
'deep-link-preview': {
|
||||
module: '文件链路',
|
||||
assertions: [
|
||||
'生成 artifact 后 URL 会进入文件路由',
|
||||
'离开项目文件路由后可再次通过文件路由进入',
|
||||
'重新进入后预览 iframe 仍能恢复到正确文件',
|
||||
],
|
||||
},
|
||||
'design-files-upload': {
|
||||
module: '文件链路',
|
||||
assertions: [
|
||||
'Design Files 面板可以真实上传图片',
|
||||
'上传后文件行会出现在 Design Files 列表',
|
||||
'右侧预览面板会显示文件信息',
|
||||
'双击文件行会把文件打开成 tab',
|
||||
],
|
||||
},
|
||||
'design-files-delete': {
|
||||
module: '文件链路',
|
||||
assertions: [
|
||||
'Design Files 行级菜单可以触发删除',
|
||||
'删除确认后文件行会从列表消失',
|
||||
'如果文件已打开，对应 tab 也会被清理',
|
||||
],
|
||||
},
|
||||
'design-files-tab-persistence': {
|
||||
module: '文件链路',
|
||||
assertions: [
|
||||
'多个文件 tab 可以同时打开',
|
||||
'切换 active tab 后状态会被持久化',
|
||||
'刷新页面后 tab 集合会恢复',
|
||||
'刷新前选中的 active tab 仍然保持选中',
|
||||
],
|
||||
},
|
||||
} satisfies Record<string, ReportCaseMetadata>;
|
||||
|
||||
export default caseMetadata;
|
||||
@@ -0,0 +1,45 @@
|
||||
export type CaseKind = 'prototype' | 'deck' | 'template' | 'workspace';
|
||||
|
||||
export interface MockArtifactCase {
|
||||
identifier: string;
|
||||
title: string;
|
||||
html: string;
|
||||
fileName: string;
|
||||
heading: string;
|
||||
}
|
||||
|
||||
export interface UICase {
|
||||
id: string;
|
||||
title: string;
|
||||
kind: CaseKind;
|
||||
flow?:
|
||||
| 'standard'
|
||||
| 'design-system-selection'
|
||||
| 'example-use-prompt'
|
||||
| 'conversation-persistence'
|
||||
| 'file-mention'
|
||||
| 'deep-link-preview'
|
||||
| 'file-upload-send'
|
||||
| 'design-files-upload'
|
||||
| 'design-files-delete'
|
||||
| 'design-files-tab-persistence'
|
||||
| 'conversation-delete-recovery'
|
||||
| 'question-form-selection-limit'
|
||||
| 'question-form-submit-persistence'
|
||||
| 'generation-does-not-create-extra-file'
|
||||
| 'comment-attachment-flow'
|
||||
| 'deck-pagination-next-prev-correctness'
|
||||
| 'deck-pagination-per-file-isolated'
|
||||
| 'uploaded-image-renders-in-preview'
|
||||
| 'python-source-preview';
|
||||
automated: boolean;
|
||||
description: string;
|
||||
create: {
|
||||
projectName: string;
|
||||
tab?: 'prototype' | 'deck' | 'template' | 'other';
|
||||
};
|
||||
prompt: string;
|
||||
secondaryPrompt?: string;
|
||||
mockArtifact?: MockArtifactCase;
|
||||
notes?: string[];
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"name": "@open-design/e2e",
|
||||
"version": "0.3.0",
|
||||
"private": true,
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"test": "vitest run -c vitest.config.ts",
|
||||
"typecheck": "tsc -p tsconfig.json --noEmit && tsc -p ../scripts/tsconfig.json --noEmit",
|
||||
"test:ui:clean": "node --experimental-strip-types scripts/reset-artifacts.ts",
|
||||
"test:ui": "corepack pnpm run test:ui:clean && playwright test -c playwright.config.ts",
|
||||
"test:ui:headed": "corepack pnpm run test:ui:clean && playwright test -c playwright.config.ts --headed",
|
||||
"test:e2e:live": "corepack pnpm --filter @open-design/daemon build && node --experimental-strip-types --test scripts/runtime-adapter.e2e.live.test.ts"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@playwright/test": "^1.59.1",
|
||||
"@testing-library/react": "^16.3.2",
|
||||
"@types/node": "^20.17.10",
|
||||
"jsdom": "^29.1.0",
|
||||
"react": "^18.3.1",
|
||||
"react-dom": "^18.3.1",
|
||||
"typescript": "^5.6.3",
|
||||
"vitest": "^2.1.8"
|
||||
},
|
||||
"engines": {
|
||||
"node": "~24"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
import { defineConfig, devices } from '@playwright/test';
|
||||
|
||||
const daemonPort = Number(process.env.OD_PORT) || 17_456;
|
||||
const webPort = Number(process.env.OD_WEB_PORT) || 17_573;
|
||||
const baseURL = `http://127.0.0.1:${webPort}`;
|
||||
|
||||
export default defineConfig({
|
||||
testDir: './specs',
|
||||
outputDir: './reports/test-results',
|
||||
timeout: 30_000,
|
||||
expect: {
|
||||
timeout: 10_000,
|
||||
},
|
||||
fullyParallel: true,
|
||||
reporter: process.env.CI
|
||||
? [
|
||||
['github'],
|
||||
['list'],
|
||||
['html', { open: 'never', outputFolder: './reports/playwright-html-report' }],
|
||||
['json', { outputFile: './reports/results.json' }],
|
||||
['junit', { outputFile: './reports/junit.xml' }],
|
||||
['./reporters/markdown-reporter.ts', { outputFile: './reports/latest.md' }],
|
||||
]
|
||||
: [
|
||||
['list'],
|
||||
['html', { open: 'never', outputFolder: './reports/playwright-html-report' }],
|
||||
['json', { outputFile: './reports/results.json' }],
|
||||
['junit', { outputFile: './reports/junit.xml' }],
|
||||
['./reporters/markdown-reporter.ts', { outputFile: './reports/latest.md' }],
|
||||
],
|
||||
use: {
|
||||
baseURL,
|
||||
trace: 'on-first-retry',
|
||||
screenshot: 'only-on-failure',
|
||||
},
|
||||
webServer: {
|
||||
command:
|
||||
`OD_DATA_DIR=e2e/.od-data ` +
|
||||
`pnpm --dir .. tools-dev run web --daemon-port ${daemonPort} --web-port ${webPort}`,
|
||||
url: baseURL,
|
||||
reuseExistingServer: false,
|
||||
timeout: 120_000,
|
||||
},
|
||||
projects: [
|
||||
{
|
||||
name: 'chromium',
|
||||
use: { ...devices['Desktop Chrome'] },
|
||||
},
|
||||
],
|
||||
});
|
||||
@@ -0,0 +1,289 @@
|
||||
import fs from 'node:fs';
|
||||
import path from 'node:path';
|
||||
import type { FullConfig, Reporter, Suite, TestCase, TestResult } from '@playwright/test/reporter';
|
||||
import caseMetadata from '../cases/report-metadata.ts';
|
||||
|
||||
interface MarkdownReporterOptions {
|
||||
outputFile?: string;
|
||||
}
|
||||
|
||||
interface CaseRow {
|
||||
caseId: string;
|
||||
title: string;
|
||||
module: string;
|
||||
assertions: string[];
|
||||
status: string;
|
||||
durationMs: number;
|
||||
retries: number;
|
||||
file: string;
|
||||
line: number | null;
|
||||
attachments: Array<{ name: string; contentType: string; path: string }>;
|
||||
error: string | null;
|
||||
}
|
||||
|
||||
interface Summary {
|
||||
total: number;
|
||||
passed: number;
|
||||
failed: number;
|
||||
flaky: number;
|
||||
skipped: number;
|
||||
timedOut: number;
|
||||
interrupted: number;
|
||||
durationMs: number;
|
||||
}
|
||||
|
||||
interface MarkdownInput {
|
||||
startedAt: Date;
|
||||
finishedAt: Date;
|
||||
summary: Summary;
|
||||
rows: CaseRow[];
|
||||
outputFile: string;
|
||||
}
|
||||
|
||||
class MarkdownReporter implements Reporter {
|
||||
private rootSuite: Suite | null = null;
|
||||
private startedAt: Date | null = null;
|
||||
private readonly options: MarkdownReporterOptions;
|
||||
|
||||
constructor(options: MarkdownReporterOptions = {}) {
|
||||
this.options = options;
|
||||
}
|
||||
|
||||
onBegin(_config: FullConfig, suite: Suite): void {
|
||||
this.rootSuite = suite;
|
||||
this.startedAt = new Date();
|
||||
}
|
||||
|
||||
async onEnd(): Promise<void> {
|
||||
if (!this.rootSuite) return;
|
||||
|
||||
const rows: CaseRow[] = [];
|
||||
visitSuite(this.rootSuite, rows);
|
||||
rows.sort((a, b) => a.caseId.localeCompare(b.caseId));
|
||||
|
||||
const summary = summarize(rows);
|
||||
const startedAt = this.startedAt ?? new Date();
|
||||
const finishedAt = new Date();
|
||||
const outputFile = this.options.outputFile || './reports/latest.md';
|
||||
const resolvedOutput = path.resolve(process.cwd(), outputFile);
|
||||
|
||||
fs.mkdirSync(path.dirname(resolvedOutput), { recursive: true });
|
||||
fs.writeFileSync(
|
||||
resolvedOutput,
|
||||
buildMarkdown({
|
||||
startedAt,
|
||||
finishedAt,
|
||||
summary,
|
||||
rows,
|
||||
outputFile,
|
||||
}),
|
||||
'utf8',
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
function visitSuite(suite: Suite, rows: CaseRow[]): void {
|
||||
for (const child of suite.suites || []) {
|
||||
visitSuite(child, rows);
|
||||
}
|
||||
for (const test of suite.tests || []) {
|
||||
const finalResult = test.results[test.results.length - 1];
|
||||
if (!finalResult) continue;
|
||||
rows.push(buildCaseRow(test, finalResult));
|
||||
}
|
||||
}
|
||||
|
||||
function buildCaseRow(test: TestCase, finalResult: TestResult): CaseRow {
|
||||
const parsed = parseCaseTitle(test.title);
|
||||
const metadata = caseMetadata[parsed.caseId];
|
||||
return {
|
||||
caseId: parsed.caseId,
|
||||
title: parsed.title,
|
||||
module: metadata?.module || '未分组',
|
||||
assertions: metadata?.assertions || [],
|
||||
status: normalizeStatus(finalResult.status, test.outcome?.()),
|
||||
durationMs: finalResult.duration ?? 0,
|
||||
retries: Math.max(0, test.results.length - 1),
|
||||
file: test.location?.file ?? '',
|
||||
line: test.location?.line ?? null,
|
||||
attachments: (finalResult.attachments || [])
|
||||
.map((entry) => ({
|
||||
name: entry.name || '',
|
||||
contentType: entry.contentType || '',
|
||||
path: entry.path ? toRelative(entry.path) : '',
|
||||
}))
|
||||
.filter((entry) => entry.path.length > 0),
|
||||
error: compactError(finalResult.error),
|
||||
};
|
||||
}
|
||||
|
||||
function parseCaseTitle(title: string): { caseId: string; title: string } {
|
||||
const idx = title.indexOf(': ');
|
||||
if (idx === -1) {
|
||||
return { caseId: title, title };
|
||||
}
|
||||
return {
|
||||
caseId: title.slice(0, idx).trim(),
|
||||
title: title.slice(idx + 2).trim(),
|
||||
};
|
||||
}
|
||||
|
||||
function normalizeStatus(status: string | undefined, outcome: string | undefined): string {
|
||||
if (outcome === 'flaky') return 'flaky';
|
||||
return status || 'unknown';
|
||||
}
|
||||
|
||||
function compactError(error: TestResult['error']): string | null {
|
||||
if (!error) return null;
|
||||
const raw = [error.message, error.value, error.stack]
|
||||
.filter(Boolean)
|
||||
.join('\n')
|
||||
.trim();
|
||||
if (!raw) return null;
|
||||
return raw.split('\n').slice(0, 8).join('\n');
|
||||
}
|
||||
|
||||
function summarize(rows: CaseRow[]): Summary {
|
||||
const summary = {
|
||||
total: rows.length,
|
||||
passed: 0,
|
||||
failed: 0,
|
||||
flaky: 0,
|
||||
skipped: 0,
|
||||
timedOut: 0,
|
||||
interrupted: 0,
|
||||
durationMs: rows.reduce((sum, row) => sum + row.durationMs, 0),
|
||||
};
|
||||
|
||||
for (const row of rows) {
|
||||
if (row.status === 'passed') summary.passed += 1;
|
||||
else if (row.status === 'failed') summary.failed += 1;
|
||||
else if (row.status === 'flaky') summary.flaky += 1;
|
||||
else if (row.status === 'skipped') summary.skipped += 1;
|
||||
else if (row.status === 'timedOut') summary.timedOut += 1;
|
||||
else if (row.status === 'interrupted') summary.interrupted += 1;
|
||||
}
|
||||
|
||||
return summary;
|
||||
}
|
||||
|
||||
function buildMarkdown({ startedAt, finishedAt, summary, rows, outputFile }: MarkdownInput): string {
|
||||
const lines: string[] = [];
|
||||
lines.push('# UI 自动化测试报告');
|
||||
lines.push('');
|
||||
lines.push(`- 生成时间：${finishedAt.toISOString()}`);
|
||||
lines.push(`- 开始时间：${startedAt.toISOString()}`);
|
||||
lines.push(`- 结束时间：${finishedAt.toISOString()}`);
|
||||
lines.push(`- 报告文件：\`${outputFile}\``);
|
||||
lines.push(`- 执行结果：${summary.failed === 0 && summary.timedOut === 0 ? '通过' : '失败'}`);
|
||||
lines.push('');
|
||||
lines.push('## 汇总');
|
||||
lines.push('');
|
||||
lines.push(`- 总用例：${summary.total}`);
|
||||
lines.push(`- 通过：${summary.passed}`);
|
||||
lines.push(`- 失败：${summary.failed}`);
|
||||
lines.push(`- Flaky：${summary.flaky}`);
|
||||
lines.push(`- 跳过：${summary.skipped}`);
|
||||
lines.push(`- 超时：${summary.timedOut}`);
|
||||
lines.push(`- 中断：${summary.interrupted}`);
|
||||
lines.push(`- 总耗时：${formatDuration(summary.durationMs)}`);
|
||||
lines.push('');
|
||||
lines.push('## 用例结果');
|
||||
lines.push('');
|
||||
lines.push('| Case ID | 模块 | 标题 | 状态 | 耗时 | 重试 |');
|
||||
lines.push('| --- | --- | --- | --- | --- | --- |');
|
||||
for (const row of rows) {
|
||||
lines.push(
|
||||
`| \`${escapeCell(row.caseId)}\` | ${escapeCell(row.module)} | ${escapeCell(row.title)} | ${statusLabel(row.status)} | ${formatDuration(row.durationMs)} | ${row.retries} |`,
|
||||
);
|
||||
}
|
||||
|
||||
lines.push('');
|
||||
lines.push('## 关键断言');
|
||||
lines.push('');
|
||||
for (const row of rows) {
|
||||
lines.push(`### ${row.caseId}`);
|
||||
lines.push('');
|
||||
lines.push(`- 模块：${row.module}`);
|
||||
lines.push(`- 标题：${row.title}`);
|
||||
lines.push(`- 状态：${statusLabel(row.status)}`);
|
||||
if (row.assertions.length > 0) {
|
||||
lines.push('- 本次验证点：');
|
||||
for (const assertion of row.assertions) {
|
||||
lines.push(` - ${assertion}`);
|
||||
}
|
||||
} else {
|
||||
lines.push('- 本次验证点：未配置');
|
||||
}
|
||||
lines.push('');
|
||||
}
|
||||
|
||||
const problematic = rows.filter((row) => row.status !== 'passed');
|
||||
if (problematic.length > 0) {
|
||||
lines.push('');
|
||||
lines.push('## 异常详情');
|
||||
lines.push('');
|
||||
for (const row of problematic) {
|
||||
lines.push(`### ${row.caseId}`);
|
||||
lines.push('');
|
||||
lines.push(`- 标题：${row.title}`);
|
||||
lines.push(`- 状态：${statusLabel(row.status)}`);
|
||||
lines.push(`- 位置：\`${toRelative(row.file)}${row.line ? `:${row.line}` : ''}\``);
|
||||
if (row.error) {
|
||||
lines.push('- 错误：');
|
||||
lines.push('```text');
|
||||
lines.push(row.error);
|
||||
lines.push('```');
|
||||
}
|
||||
if (row.attachments.length > 0) {
|
||||
lines.push('- 附件：');
|
||||
for (const attachment of row.attachments) {
|
||||
lines.push(` - \`${attachment.name}\` · \`${attachment.path}\``);
|
||||
}
|
||||
}
|
||||
lines.push('');
|
||||
}
|
||||
}
|
||||
|
||||
lines.push('## 原始产物');
|
||||
lines.push('');
|
||||
lines.push('- HTML 报告入口：`e2e/reports/ui-test-report.html`');
|
||||
lines.push('- Playwright HTML 底层目录：`e2e/reports/playwright-html-report/`');
|
||||
lines.push('- JSON 结果：`e2e/reports/results.json`');
|
||||
lines.push('- JUnit 结果：`e2e/reports/junit.xml`');
|
||||
lines.push('- Playwright 附件：`e2e/reports/test-results/`');
|
||||
lines.push('');
|
||||
lines.push('## 说明');
|
||||
lines.push('');
|
||||
lines.push('- 这份报告记录的是本次实际执行到的 UI 自动化用例。');
|
||||
lines.push('- 用例设计来源见 `e2e/cases/` 以及各模块文档。');
|
||||
lines.push('- 如果用例失败，优先查看本报告中的附件路径和 HTML 报告。');
|
||||
lines.push('');
|
||||
return `${lines.join('\n')}\n`;
|
||||
}
|
||||
|
||||
function formatDuration(ms: number): string {
|
||||
if (ms < 1000) return `${ms}ms`;
|
||||
return `${(ms / 1000).toFixed(1)}s`;
|
||||
}
|
||||
|
||||
function statusLabel(status: string): string {
|
||||
if (status === 'passed') return 'passed';
|
||||
if (status === 'failed') return 'failed';
|
||||
if (status === 'flaky') return 'flaky';
|
||||
if (status === 'skipped') return 'skipped';
|
||||
if (status === 'timedOut') return 'timedOut';
|
||||
if (status === 'interrupted') return 'interrupted';
|
||||
return status;
|
||||
}
|
||||
|
||||
function toRelative(filePath: string): string {
|
||||
if (!filePath) return '';
|
||||
return path.relative(process.cwd(), filePath) || filePath;
|
||||
}
|
||||
|
||||
function escapeCell(value: string): string {
|
||||
return String(value).replace(/\|/g, '\\|');
|
||||
}
|
||||
|
||||
export default MarkdownReporter;
|
||||
@@ -0,0 +1,49 @@
|
||||
# UI 测试报告
|
||||
|
||||
这个目录存放 UI 自动化测试的运行结果和可读报告。
|
||||
|
||||
## 目录说明
|
||||
|
||||
- `latest.md`：最近一次测试运行的 Markdown 汇总报告
|
||||
- `ui-test-report.html`：给人直接打开的 HTML 报告入口
|
||||
- `playwright-html-report/`：Playwright 原生 HTML 报告目录，内部入口仍是 `index.html`
|
||||
- `results.json`：Playwright JSON 原始结果
|
||||
- `junit.xml`：JUnit 格式结果，方便接 CI
|
||||
- `test-results/`：失败用例的截图、trace、error-context 等原始附件
|
||||
|
||||
每次执行 `pnpm run test:ui`（或 `pnpm --filter @open-design/e2e test:ui`）前，系统会先自动清理旧的：
|
||||
|
||||
- `e2e/.od-data/`
|
||||
- `e2e/reports/test-results/`
|
||||
- `e2e/reports/playwright-html-report/`
|
||||
- `e2e/reports/results.json`
|
||||
- `e2e/reports/junit.xml`
|
||||
- `e2e/reports/latest.md`
|
||||
|
||||
这样报告和测试数据默认只反映最近一次执行结果，不会把上一次残留混进来。
|
||||
|
||||
## 怎么看
|
||||
|
||||
如果你想快速判断“这次到底测了什么、有没有过”，先看：
|
||||
|
||||
- [latest.md](/Users/mac/open-design/open-design/e2e/reports/latest.md)
|
||||
- [ui-test-report.html](/Users/mac/open-design/open-design/e2e/reports/ui-test-report.html)
|
||||
|
||||
它会包含：
|
||||
|
||||
- 本次执行时间
|
||||
- 总用例数、通过数、失败数
|
||||
- 每条 case 的结果、耗时、重试次数
|
||||
- 失败时对应的错误摘要和附件路径
|
||||
|
||||
如果你想看更细的失败上下文，再看：
|
||||
|
||||
- `e2e/reports/playwright-html-report/`
|
||||
- `e2e/reports/test-results/`
|
||||
|
||||
## 和用例库的关系
|
||||
|
||||
- `e2e/cases/`：定义“应该测什么”
|
||||
- `e2e/reports/`：记录“这次实际测了什么、结果如何”
|
||||
|
||||
这两层分开以后，既能看覆盖设计，也能看真实执行结果。
|
||||
@@ -0,0 +1,52 @@
|
||||
<!doctype html>
|
||||
<html lang="zh-CN">
|
||||
<head>
|
||||
<meta charset="utf-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1" />
|
||||
<title>Open Design UI Test Report</title>
|
||||
<meta http-equiv="refresh" content="0; url=./playwright-html-report/index.html" />
|
||||
<style>
|
||||
:root {
|
||||
color-scheme: light;
|
||||
font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", sans-serif;
|
||||
}
|
||||
body {
|
||||
margin: 0;
|
||||
min-height: 100vh;
|
||||
display: grid;
|
||||
place-items: center;
|
||||
background: #f6f3ee;
|
||||
color: #201d18;
|
||||
}
|
||||
main {
|
||||
width: min(560px, calc(100vw - 48px));
|
||||
padding: 28px 32px;
|
||||
border: 1px solid #ddd4c7;
|
||||
border-radius: 18px;
|
||||
background: #fffdfa;
|
||||
box-shadow: 0 16px 40px rgba(32, 29, 24, 0.08);
|
||||
}
|
||||
h1 {
|
||||
margin: 0 0 12px;
|
||||
font-size: 22px;
|
||||
}
|
||||
p {
|
||||
margin: 0;
|
||||
line-height: 1.6;
|
||||
}
|
||||
a {
|
||||
color: #b45b33;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<main>
|
||||
<h1>Open Design UI Test Report</h1>
|
||||
<p>
|
||||
正在跳转到 Playwright HTML 报告。
|
||||
如果没有自动跳转，请打开
|
||||
<a href="./playwright-html-report/index.html">playwright-html-report/index.html</a>。
|
||||
</p>
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,53 @@
|
||||
import { mkdir, readdir, rm } from 'node:fs/promises';
|
||||
import path from 'node:path';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
|
||||
const __dirname = path.dirname(fileURLToPath(import.meta.url));
|
||||
const e2eDir = path.resolve(__dirname, '..');
|
||||
|
||||
const targets = [
|
||||
path.join(e2eDir, '.od-data'),
|
||||
path.join(e2eDir, 'test-results'),
|
||||
path.join(e2eDir, 'reports', 'test-results'),
|
||||
path.join(e2eDir, 'reports', 'html'),
|
||||
path.join(e2eDir, 'reports', 'playwright-html-report'),
|
||||
path.join(e2eDir, 'reports', 'results.json'),
|
||||
path.join(e2eDir, 'reports', 'junit.xml'),
|
||||
path.join(e2eDir, 'reports', 'latest.md'),
|
||||
path.join(e2eDir, '.DS_Store'),
|
||||
];
|
||||
|
||||
for (const target of targets) {
|
||||
await rm(target, { recursive: true, force: true });
|
||||
}
|
||||
|
||||
await mkdir(path.join(e2eDir, 'reports'), { recursive: true });
|
||||
|
||||
// Recreate runtime roots so local inspection stays predictable even before
|
||||
// Playwright or the daemon materializes them.
|
||||
await mkdir(path.join(e2eDir, '.od-data'), { recursive: true });
|
||||
await mkdir(path.join(e2eDir, 'reports', 'test-results'), {
|
||||
recursive: true,
|
||||
});
|
||||
|
||||
// Best-effort removal of accidental empty directories directly under the
|
||||
// test data root. This keeps old project ids from piling up across runs.
|
||||
const projectsRoot = path.join(e2eDir, '.od-data', 'projects');
|
||||
try {
|
||||
const entries = await readdir(projectsRoot, { withFileTypes: true });
|
||||
await Promise.all(
|
||||
entries
|
||||
.filter((entry) => entry.isDirectory())
|
||||
.map((entry) =>
|
||||
rm(path.join(projectsRoot, entry.name), {
|
||||
recursive: true,
|
||||
force: true,
|
||||
}),
|
||||
),
|
||||
);
|
||||
} catch (error) {
|
||||
const code = error instanceof Error && 'code' in error ? error.code : undefined;
|
||||
if (code !== 'ENOENT') {
|
||||
console.warn('Failed to clean stale e2e project dirs:', error);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,319 @@
|
||||
import type http from 'node:http';
|
||||
import test from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import fs from 'node:fs/promises';
|
||||
import os from 'node:os';
|
||||
import path from 'node:path';
|
||||
|
||||
interface AgentInfo {
|
||||
id: string;
|
||||
name: string;
|
||||
bin: string;
|
||||
available: boolean;
|
||||
path?: string;
|
||||
version?: string | null;
|
||||
models?: Array<{ id: string; label?: string }>;
|
||||
streamFormat?: string;
|
||||
}
|
||||
|
||||
interface AgentsResponse {
|
||||
agents: AgentInfo[];
|
||||
}
|
||||
|
||||
interface ParsedSseEvent {
|
||||
event: string;
|
||||
data: Record<string, unknown>;
|
||||
}
|
||||
|
||||
type StartServer = (options: { port: number; returnServer: true }) => Promise<http.Server | undefined>;
|
||||
type CloseDatabase = () => void;
|
||||
|
||||
const liveTimeoutMs = Number(process.env.OD_RUNTIME_LIVE_TIMEOUT_MS || 180_000);
|
||||
const requestedRuntimeIds = parseRuntimeIds(process.env.OD_E2E_RUNTIMES);
|
||||
const maxRuntimeCount = 8;
|
||||
const marker = 'OD_RUNTIME_ADAPTER_LIVE_OK';
|
||||
|
||||
let baseUrl: string;
|
||||
let server: http.Server | undefined;
|
||||
let startServer: StartServer;
|
||||
let closeDatabase: CloseDatabase | undefined;
|
||||
let detectedAgents: AgentInfo[] | undefined;
|
||||
let dataDir: string;
|
||||
|
||||
test.before(async () => {
|
||||
dataDir = await fs.mkdtemp(path.join(os.tmpdir(), 'od-runtime-adapter-live-'));
|
||||
process.env.OD_DATA_DIR = dataDir;
|
||||
({ startServer } = await import('../../apps/daemon/dist/server.js') as { startServer: StartServer });
|
||||
({ closeDatabase } = await import('../../apps/daemon/dist/db.js') as { closeDatabase: CloseDatabase });
|
||||
const started = await startServer({ port: 0, returnServer: true });
|
||||
if (started == null) {
|
||||
throw new Error('startServer did not return a server handle');
|
||||
}
|
||||
const address = started.address();
|
||||
if (address == null || typeof address === 'string') {
|
||||
throw new Error('startServer did not bind to a TCP port');
|
||||
}
|
||||
server = started;
|
||||
baseUrl = `http://127.0.0.1:${address.port}`;
|
||||
});
|
||||
|
||||
test.after(async () => {
|
||||
if (server) {
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
server?.close((err) => (err ? reject(err) : resolve()));
|
||||
});
|
||||
}
|
||||
closeDatabase?.();
|
||||
if (dataDir) {
|
||||
await fs.rm(dataDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
test('runtime adapter live detection flow exposes installed runtimes', async () => {
|
||||
log('detect', 'starting runtime detection via /api/agents');
|
||||
const res = await fetch(`${baseUrl}/api/agents`);
|
||||
assert.equal(res.status, 200);
|
||||
const body = await readAgentsResponse(res);
|
||||
assert.ok(Array.isArray(body.agents));
|
||||
assert.ok(body.agents.length > 0);
|
||||
|
||||
detectedAgents = body.agents;
|
||||
const available = body.agents.filter((agent) => agent.available);
|
||||
|
||||
for (const agent of body.agents) {
|
||||
const status = agent.available ? 'available' : 'unavailable';
|
||||
const version = agent.version ? ` version=${agent.version}` : '';
|
||||
const resolvedPath = agent.path ? ` path=${agent.path}` : '';
|
||||
log(
|
||||
'detect',
|
||||
`${agent.id}: ${status}${version}${resolvedPath} models=${agent.models?.length ?? 0} stream=${agent.streamFormat}`,
|
||||
);
|
||||
}
|
||||
|
||||
assert.ok(
|
||||
available.length > 0,
|
||||
'Install at least one supported runtime CLI on PATH: claude, codex, gemini, opencode, hermes, kimi, cursor-agent, or qwen.',
|
||||
);
|
||||
|
||||
for (const agent of body.agents) {
|
||||
assert.equal(typeof agent.id, 'string');
|
||||
assert.equal(typeof agent.name, 'string');
|
||||
assert.equal(typeof agent.bin, 'string');
|
||||
assert.equal(typeof agent.available, 'boolean');
|
||||
assert.ok(Array.isArray(agent.models));
|
||||
assert.ok(agent.models.some((model) => model.id === 'default'));
|
||||
assert.equal(typeof agent.streamFormat, 'string');
|
||||
if (agent.available) {
|
||||
assert.equal(typeof agent.path, 'string');
|
||||
const resolvedPath = agent.path;
|
||||
assert.ok(resolvedPath && resolvedPath.length > 0);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
test('runtime adapter live run flow streams a successful response for every available runtime', { timeout: liveTimeoutMs * maxRuntimeCount + 30_000 }, async () => {
|
||||
if (!detectedAgents) {
|
||||
log('run', 'detection cache empty; fetching /api/agents before run flow');
|
||||
const res = await fetch(`${baseUrl}/api/agents`);
|
||||
detectedAgents = (await readAgentsResponse(res)).agents;
|
||||
}
|
||||
|
||||
const requestedSet = requestedRuntimeIds ? new Set(requestedRuntimeIds) : null;
|
||||
const availableAgents = detectedAgents.filter(
|
||||
(agent) => agent.available && (!requestedSet || requestedSet.has(agent.id)),
|
||||
);
|
||||
|
||||
if (requestedSet) {
|
||||
log('run', `runtime filter=${requestedRuntimeIds?.join(',')}`);
|
||||
for (const id of requestedSet) {
|
||||
assert.ok(
|
||||
detectedAgents.some((agent) => agent.id === id),
|
||||
`Requested runtime ${id} is missing from /api/agents.`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
for (const agent of detectedAgents) {
|
||||
if (agent.available) {
|
||||
if (!requestedSet || requestedSet.has(agent.id)) {
|
||||
log('run', `${agent.id}: queued`);
|
||||
} else {
|
||||
log('run', `${agent.id}: skipped by runtime filter`);
|
||||
}
|
||||
} else {
|
||||
log('run', `${agent.id}: skipped because runtime is unavailable`);
|
||||
}
|
||||
}
|
||||
assert.ok(
|
||||
availableAgents.length > 0,
|
||||
requestedSet
|
||||
? `Requested runtimes unavailable: ${requestedRuntimeIds?.join(',')}.`
|
||||
: 'Available runtime required from /api/agents.',
|
||||
);
|
||||
|
||||
for (const agent of availableAgents) {
|
||||
await runRuntime(agent);
|
||||
}
|
||||
});
|
||||
|
||||
async function runRuntime(agent: AgentInfo): Promise<void> {
|
||||
const startedAt = Date.now();
|
||||
log('run', `${agent.id}: starting /api/chat live run`);
|
||||
|
||||
const projectId = `runtime-adapter-live-${Date.now()}-${Math.random().toString(36).slice(2)}`;
|
||||
const events: ParsedSseEvent[] = [];
|
||||
const abort = AbortSignal.timeout(liveTimeoutMs);
|
||||
try {
|
||||
const res = await fetch(`${baseUrl}/api/chat`, {
|
||||
method: 'POST',
|
||||
headers: { 'content-type': 'application/json' },
|
||||
signal: abort,
|
||||
body: JSON.stringify({
|
||||
agentId: agent.id,
|
||||
projectId,
|
||||
model: 'default',
|
||||
message: `Reply with exactly this token and nothing else: ${marker}`,
|
||||
systemPrompt: [
|
||||
'You are running a local runtime-adapter live smoke test.',
|
||||
'Produce a minimal text-only response.',
|
||||
'Do not create, edit, delete, or inspect files.',
|
||||
].join('\n'),
|
||||
}),
|
||||
});
|
||||
|
||||
assert.equal(res.status, 200);
|
||||
assert.match(res.headers.get('content-type') || '', /text\/event-stream/);
|
||||
assert.ok(res.body, 'SSE response should include a readable body.');
|
||||
|
||||
await collectSseEvents(res, events, agent.id);
|
||||
} finally {
|
||||
await fs.rm(path.join(dataDir, 'projects', projectId), {
|
||||
recursive: true,
|
||||
force: true,
|
||||
});
|
||||
}
|
||||
|
||||
const start = events.find((event) => event.event === 'start');
|
||||
assert.ok(start, 'SSE stream should include a start event.');
|
||||
assert.equal(start.data.agentId, agent.id);
|
||||
assert.equal(start.data.projectId, projectId);
|
||||
log('run', `${agent.id}: start event cwd=${String(start.data.cwd ?? '')}`);
|
||||
|
||||
const end = events.find((event) => event.event === 'end');
|
||||
assert.ok(end, 'SSE stream should include an end event.');
|
||||
assert.equal(end.data.code, 0, renderEvents(events));
|
||||
log('run', `${agent.id}: end event code=${String(end.data.code)} signal=${String(end.data.signal ?? 'none')}`);
|
||||
|
||||
const text = events
|
||||
.map((event) => {
|
||||
if (event.event === 'stdout') return stringData(event.data.chunk);
|
||||
if (event.event === 'agent') return stringData(event.data.text) || stringData(event.data.delta);
|
||||
return '';
|
||||
})
|
||||
.join('');
|
||||
assert.match(text, new RegExp(marker), renderEvents(events));
|
||||
log('run', `${agent.id}: passed in ${Date.now() - startedAt}ms`);
|
||||
}
|
||||
|
||||
async function collectSseEvents(res: Response, events: ParsedSseEvent[], agentId: string): Promise<void> {
|
||||
const reader = res.body?.getReader();
|
||||
assert.ok(reader, 'SSE response should include a readable body.');
|
||||
const decoder = new TextDecoder();
|
||||
let buffer = '';
|
||||
const seen = new Set<string>();
|
||||
|
||||
while (true) {
|
||||
const { done, value } = await reader.read();
|
||||
if (done) break;
|
||||
buffer += decoder.decode(value, { stream: true });
|
||||
const chunks = buffer.split('\n\n');
|
||||
buffer = chunks.pop() || '';
|
||||
for (const chunk of chunks) {
|
||||
const parsed = parseSseEvent(chunk);
|
||||
if (parsed) {
|
||||
events.push(parsed);
|
||||
logSseProgress(agentId, parsed, seen);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
buffer += decoder.decode();
|
||||
if (buffer.trim()) {
|
||||
const parsed = parseSseEvent(buffer);
|
||||
if (parsed) {
|
||||
events.push(parsed);
|
||||
logSseProgress(agentId, parsed, seen);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function parseSseEvent(chunk: string): ParsedSseEvent | null {
|
||||
const lines = chunk.split('\n');
|
||||
if (lines.every((line) => line === '' || line.startsWith(':'))) return null;
|
||||
|
||||
const eventLine = lines.find((line) => line.startsWith('event: '));
|
||||
const dataLine = lines.find((line) => line.startsWith('data: '));
|
||||
if (!eventLine || !dataLine) return null;
|
||||
return {
|
||||
event: eventLine.slice('event: '.length),
|
||||
data: JSON.parse(dataLine.slice('data: '.length)) as Record<string, unknown>,
|
||||
};
|
||||
}
|
||||
|
||||
function renderEvents(events: ParsedSseEvent[]): string {
|
||||
return JSON.stringify(events, null, 2).slice(0, 8000);
|
||||
}
|
||||
|
||||
function parseRuntimeIds(value: string | undefined): string[] | null {
|
||||
if (!value) return null;
|
||||
const ids = value
|
||||
.split(',')
|
||||
.map((item) => item.trim())
|
||||
.filter(Boolean);
|
||||
return ids.length > 0 ? ids : null;
|
||||
}
|
||||
|
||||
function log(stage: string, message: string): void {
|
||||
console.log(`[runtime-adapter:e2e:${stage}] ${message}`);
|
||||
}
|
||||
|
||||
function logSseProgress(agentId: string, event: ParsedSseEvent, seen: Set<string>): void {
|
||||
if (event.event === 'start' && !seen.has('start')) {
|
||||
seen.add('start');
|
||||
log('run', `${agentId}: received start event`);
|
||||
return;
|
||||
}
|
||||
if (event.event === 'stdout' && !seen.has('stdout')) {
|
||||
seen.add('stdout');
|
||||
log('run', `${agentId}: received stdout stream`);
|
||||
return;
|
||||
}
|
||||
const type = stringData(event.data.type) || 'event';
|
||||
if (event.event === 'agent' && !seen.has(`agent:${type}`)) {
|
||||
seen.add(`agent:${type}`);
|
||||
log('run', `${agentId}: received agent event type=${type || 'unknown'}`);
|
||||
return;
|
||||
}
|
||||
if (event.event === 'stderr' && !seen.has('stderr')) {
|
||||
seen.add('stderr');
|
||||
log('run', `${agentId}: received stderr stream`);
|
||||
return;
|
||||
}
|
||||
if (event.event === 'error') {
|
||||
log('run', `${agentId}: received error event ${stringData(event.data.message)}`.trim());
|
||||
return;
|
||||
}
|
||||
if (event.event === 'end' && !seen.has('end')) {
|
||||
seen.add('end');
|
||||
log('run', `${agentId}: received end event`);
|
||||
}
|
||||
}
|
||||
|
||||
async function readAgentsResponse(res: Response): Promise<AgentsResponse> {
|
||||
const body = await res.json() as Partial<AgentsResponse>;
|
||||
return { agents: Array.isArray(body.agents) ? body.agents : [] };
|
||||
}
|
||||
|
||||
function stringData(value: unknown): string {
|
||||
return typeof value === 'string' ? value : '';
|
||||
}
|
||||
@@ -0,0 +1,883 @@
|
||||
import { expect, test } from '@playwright/test';
|
||||
import { automatedCases } from '../cases';
|
||||
import type { UICase } from '../cases/types';
|
||||
|
||||
const STORAGE_KEY = 'open-design:config';
|
||||
|
||||
test.beforeEach(async ({ page }) => {
|
||||
await page.addInitScript((key) => {
|
||||
window.localStorage.setItem(
|
||||
key,
|
||||
JSON.stringify({
|
||||
mode: 'daemon',
|
||||
apiKey: '',
|
||||
baseUrl: 'https://api.anthropic.com',
|
||||
model: 'claude-sonnet-4-5',
|
||||
agentId: 'mock',
|
||||
skillId: null,
|
||||
designSystemId: null,
|
||||
onboardingCompleted: true,
|
||||
agentModels: {},
|
||||
}),
|
||||
);
|
||||
}, STORAGE_KEY);
|
||||
});
|
||||
|
||||
for (const entry of automatedCases()) {
|
||||
test(`${entry.id}: ${entry.title}`, async ({ page }) => {
|
||||
await page.route('**/api/agents', async (route) => {
|
||||
await route.fulfill({
|
||||
json: {
|
||||
agents: [
|
||||
{
|
||||
id: 'mock',
|
||||
name: 'Mock Agent',
|
||||
bin: 'mock-agent',
|
||||
available: true,
|
||||
version: 'test',
|
||||
models: [{ id: 'default', label: 'Default' }],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
if (entry.flow === 'design-system-selection') {
|
||||
await page.route('**/api/design-systems', async (route) => {
|
||||
await route.fulfill({
|
||||
json: {
|
||||
designSystems: [
|
||||
{
|
||||
id: 'nexu-soft-tech',
|
||||
title: 'Nexu Soft Tech',
|
||||
category: 'Product',
|
||||
summary: 'Warm utility system for product interfaces.',
|
||||
swatches: ['#F7F4EE', '#D6CBBF', '#1F2937', '#D97757'],
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
if (entry.flow === 'example-use-prompt') {
|
||||
await page.route('**/api/skills', async (route) => {
|
||||
await route.fulfill({
|
||||
json: {
|
||||
skills: [
|
||||
{
|
||||
id: 'warm-utility-example',
|
||||
name: 'Warm Utility Example',
|
||||
description: 'A warm utility prototype example.',
|
||||
triggers: [],
|
||||
mode: 'prototype',
|
||||
platform: 'desktop',
|
||||
scenario: 'product',
|
||||
previewType: 'html',
|
||||
designSystemRequired: false,
|
||||
defaultFor: ['prototype'],
|
||||
upstream: null,
|
||||
featured: 1,
|
||||
fidelity: 'high-fidelity',
|
||||
speakerNotes: null,
|
||||
animations: null,
|
||||
hasBody: true,
|
||||
examplePrompt: entry.prompt,
|
||||
},
|
||||
],
|
||||
},
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
if (entry.mockArtifact) {
|
||||
await page.route('**/api/runs', async (route) => {
|
||||
await route.fulfill({ status: 202, contentType: 'application/json', body: '{"runId":"mock-run"}' });
|
||||
});
|
||||
await page.route('**/api/runs/*/events', async (route) => {
|
||||
const artifact =
|
||||
`<artifact identifier="${entry.mockArtifact!.identifier}" type="text/html" title="${entry.mockArtifact!.title}">` +
|
||||
entry.mockArtifact!.html +
|
||||
'</artifact>';
|
||||
const body = [
|
||||
'event: start',
|
||||
'data: {"bin":"mock-agent"}',
|
||||
'',
|
||||
'event: stdout',
|
||||
`data: ${JSON.stringify({ chunk: artifact })}`,
|
||||
'',
|
||||
'event: end',
|
||||
'data: {"code":0}',
|
||||
'',
|
||||
'',
|
||||
].join('\n');
|
||||
|
||||
await route.fulfill({
|
||||
status: 200,
|
||||
headers: {
|
||||
'content-type': 'text/event-stream',
|
||||
'cache-control': 'no-cache',
|
||||
},
|
||||
body,
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
if (entry.flow === 'question-form-selection-limit') {
|
||||
await page.route('**/api/runs', async (route) => {
|
||||
await route.fulfill({ status: 202, contentType: 'application/json', body: '{"runId":"mock-run"}' });
|
||||
});
|
||||
await page.route('**/api/runs/*/events', async (route) => {
|
||||
const form = [
|
||||
'<question-form id="discovery" title="Quick brief — 30 seconds">',
|
||||
JSON.stringify(
|
||||
{
|
||||
description: "I'll lock these in before building.",
|
||||
questions: [
|
||||
{
|
||||
id: 'tone',
|
||||
label: 'Visual tone (pick up to two)',
|
||||
type: 'checkbox',
|
||||
maxSelections: 2,
|
||||
options: ['Editorial / magazine', 'Modern minimal', 'Soft / warm'],
|
||||
required: true,
|
||||
},
|
||||
],
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
'</question-form>',
|
||||
].join('\n');
|
||||
const body = [
|
||||
'event: start',
|
||||
'data: {"bin":"mock-agent"}',
|
||||
'',
|
||||
'event: stdout',
|
||||
`data: ${JSON.stringify({ chunk: form })}`,
|
||||
'',
|
||||
'event: end',
|
||||
'data: {"code":0}',
|
||||
'',
|
||||
'',
|
||||
].join('\n');
|
||||
|
||||
await route.fulfill({
|
||||
status: 200,
|
||||
headers: {
|
||||
'content-type': 'text/event-stream',
|
||||
'cache-control': 'no-cache',
|
||||
},
|
||||
body,
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
if (entry.flow === 'question-form-submit-persistence') {
|
||||
let requestCount = 0;
|
||||
await page.route('**/api/runs', async (route) => {
|
||||
await route.fulfill({ status: 202, contentType: 'application/json', body: '{"runId":"mock-run"}' });
|
||||
});
|
||||
await page.route('**/api/runs/*/events', async (route) => {
|
||||
requestCount += 1;
|
||||
const chunk =
|
||||
requestCount === 1
|
||||
? [
|
||||
'<question-form id="discovery" title="Quick brief — 30 seconds">',
|
||||
JSON.stringify(
|
||||
{
|
||||
description: "I'll lock these in before building.",
|
||||
questions: [
|
||||
{
|
||||
id: 'tone',
|
||||
label: 'Visual tone (pick up to two)',
|
||||
type: 'checkbox',
|
||||
maxSelections: 2,
|
||||
options: ['Editorial / magazine', 'Modern minimal', 'Soft / warm'],
|
||||
required: true,
|
||||
},
|
||||
],
|
||||
},
|
||||
null,
|
||||
2,
|
||||
),
|
||||
'</question-form>',
|
||||
].join('\n')
|
||||
: 'Thanks — I will use these answers for the next draft.';
|
||||
const body = [
|
||||
'event: start',
|
||||
'data: {"bin":"mock-agent"}',
|
||||
'',
|
||||
'event: stdout',
|
||||
`data: ${JSON.stringify({ chunk })}`,
|
||||
'',
|
||||
'event: end',
|
||||
'data: {"code":0,"status":"succeeded"}',
|
||||
'',
|
||||
'',
|
||||
].join('\n');
|
||||
|
||||
await route.fulfill({
|
||||
status: 200,
|
||||
headers: {
|
||||
'content-type': 'text/event-stream',
|
||||
'cache-control': 'no-cache',
|
||||
},
|
||||
body,
|
||||
});
|
||||
});
|
||||
}
|
||||
|
||||
await page.goto('/');
|
||||
|
||||
if (entry.flow === 'design-system-selection') {
|
||||
await runDesignSystemSelectionFlow(page, entry);
|
||||
return;
|
||||
}
|
||||
if (entry.flow === 'example-use-prompt') {
|
||||
await runExampleUsePromptFlow(page, entry);
|
||||
return;
|
||||
}
|
||||
|
||||
await createProject(page, entry);
|
||||
await expectWorkspaceReady(page);
|
||||
|
||||
if (entry.flow === 'conversation-persistence') {
|
||||
await runConversationPersistenceFlow(page, entry);
|
||||
return;
|
||||
}
|
||||
if (entry.flow === 'file-mention') {
|
||||
await runFileMentionFlow(page, entry);
|
||||
return;
|
||||
}
|
||||
if (entry.flow === 'deep-link-preview') {
|
||||
await runDeepLinkPreviewFlow(page, entry);
|
||||
return;
|
||||
}
|
||||
if (entry.flow === 'file-upload-send') {
|
||||
await runFileUploadSendFlow(page, entry);
|
||||
return;
|
||||
}
|
||||
if (entry.flow === 'design-files-upload') {
|
||||
await runDesignFilesUploadFlow(page);
|
||||
return;
|
||||
}
|
||||
if (entry.flow === 'design-files-delete') {
|
||||
await runDesignFilesDeleteFlow(page);
|
||||
return;
|
||||
}
|
||||
if (entry.flow === 'design-files-tab-persistence') {
|
||||
await runDesignFilesTabPersistenceFlow(page);
|
||||
return;
|
||||
}
|
||||
if (entry.flow === 'conversation-delete-recovery') {
|
||||
await runConversationDeleteRecoveryFlow(page, entry);
|
||||
return;
|
||||
}
|
||||
if (entry.flow === 'question-form-selection-limit') {
|
||||
await runQuestionFormSelectionLimitFlow(page, entry);
|
||||
return;
|
||||
}
|
||||
if (entry.flow === 'question-form-submit-persistence') {
|
||||
await runQuestionFormSubmitPersistenceFlow(page, entry);
|
||||
return;
|
||||
}
|
||||
if (entry.flow === 'generation-does-not-create-extra-file') {
|
||||
await runGenerationDoesNotCreateExtraFileFlow(page, entry);
|
||||
return;
|
||||
}
|
||||
if (entry.flow === 'comment-attachment-flow') {
|
||||
await runCommentAttachmentFlow(page, entry);
|
||||
return;
|
||||
}
|
||||
|
||||
await sendPrompt(page, entry.prompt);
|
||||
|
||||
if (entry.mockArtifact) {
|
||||
await expectArtifactVisible(page, entry);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
async function createProject(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
entry: UICase,
|
||||
) {
|
||||
await createProjectNameOnly(page, entry);
|
||||
await page.getByTestId('create-project').click();
|
||||
}
|
||||
|
||||
async function expectWorkspaceReady(page: Parameters<typeof test>[0]['page']) {
|
||||
await expect(page).toHaveURL(/\/projects\//);
|
||||
await expect(page.getByTestId('chat-composer')).toBeVisible();
|
||||
await expect(page.getByTestId('file-workspace')).toBeVisible();
|
||||
await expect(page.getByText('Start a conversation')).toBeVisible();
|
||||
}
|
||||
|
||||
async function sendPrompt(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
prompt: string,
|
||||
) {
|
||||
const input = page.getByTestId('chat-composer-input');
|
||||
const sendButton = page.getByTestId('chat-send');
|
||||
for (let attempt = 0; attempt < 3; attempt++) {
|
||||
await input.click();
|
||||
await input.fill(prompt);
|
||||
try {
|
||||
await expect(input).toHaveValue(prompt, { timeout: 1500 });
|
||||
await expect(sendButton).toBeEnabled({ timeout: 1500 });
|
||||
const chatResponse = page.waitForResponse(
|
||||
(resp) => resp.url().includes('/api/runs') && resp.request().method() === 'POST',
|
||||
{ timeout: 2000 },
|
||||
);
|
||||
await sendButton.evaluate((button: HTMLButtonElement) => button.click());
|
||||
await chatResponse;
|
||||
return;
|
||||
} catch (error) {
|
||||
await input.click();
|
||||
await input.press(`${process.platform === 'darwin' ? 'Meta' : 'Control'}+A`);
|
||||
await input.press('Backspace');
|
||||
await input.pressSequentially(prompt);
|
||||
try {
|
||||
await expect(input).toHaveValue(prompt, { timeout: 1500 });
|
||||
await expect(sendButton).toBeEnabled({ timeout: 1500 });
|
||||
const chatResponse = page.waitForResponse(
|
||||
(resp) => resp.url().includes('/api/runs') && resp.request().method() === 'POST',
|
||||
{ timeout: 2000 },
|
||||
);
|
||||
await sendButton.evaluate((button: HTMLButtonElement) => button.click());
|
||||
await chatResponse;
|
||||
return;
|
||||
} catch (retryError) {
|
||||
if (attempt === 2) throw retryError;
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function runDesignSystemSelectionFlow(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
entry: UICase,
|
||||
) {
|
||||
await createProjectNameOnly(page, entry);
|
||||
await page.getByTestId('design-system-trigger').click();
|
||||
await expect(page.getByTestId('design-system-search')).toBeVisible();
|
||||
await page.getByTestId('design-system-search').fill('Nexu');
|
||||
await page.getByRole('option', { name: /Nexu Soft Tech/i }).click();
|
||||
await expect(page.getByTestId('design-system-trigger')).toContainText('Nexu Soft Tech');
|
||||
await page.getByTestId('create-project').click();
|
||||
|
||||
await expect(page).toHaveURL(/\/projects\//);
|
||||
await expect(page.getByTestId('project-meta')).toContainText('Nexu Soft Tech');
|
||||
await expect(page.getByTestId('chat-composer')).toBeVisible();
|
||||
}
|
||||
|
||||
async function runExampleUsePromptFlow(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
entry: UICase,
|
||||
) {
|
||||
await page.getByTestId('entry-tab-examples').click();
|
||||
await expect(page.getByTestId('example-card-warm-utility-example')).toBeVisible();
|
||||
await page.getByTestId('example-use-prompt-warm-utility-example').click();
|
||||
|
||||
await expect(page).toHaveURL(/\/projects\//);
|
||||
await expect(page.getByTestId('chat-composer')).toBeVisible();
|
||||
await expect(page.getByTestId('chat-composer-input')).toHaveValue(entry.prompt);
|
||||
await expect(page.getByTestId('project-title')).toContainText('Warm Utility Example');
|
||||
await expect(page.getByTestId('project-meta')).toContainText('Warm Utility Example');
|
||||
}
|
||||
|
||||
async function runQuestionFormSelectionLimitFlow(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
entry: UICase,
|
||||
) {
|
||||
await sendPrompt(page, entry.prompt);
|
||||
|
||||
const toneQuestion = page.locator('.qf-field', {
|
||||
has: page.getByText('Visual tone (pick up to two)'),
|
||||
});
|
||||
await expect(toneQuestion).toBeVisible();
|
||||
|
||||
const editorialChip = toneQuestion.locator('label.qf-chip', {
|
||||
has: page.getByText('Editorial / magazine'),
|
||||
});
|
||||
const modernChip = toneQuestion.locator('label.qf-chip', {
|
||||
has: page.getByText('Modern minimal'),
|
||||
});
|
||||
const softChip = toneQuestion.locator('label.qf-chip', {
|
||||
has: page.getByText('Soft / warm'),
|
||||
});
|
||||
const editorial = editorialChip.locator('input[type="checkbox"]');
|
||||
const modern = modernChip.locator('input[type="checkbox"]');
|
||||
const soft = softChip.locator('input[type="checkbox"]');
|
||||
|
||||
await editorialChip.click();
|
||||
await modernChip.click();
|
||||
|
||||
await expect(editorial).toBeChecked();
|
||||
await expect(modern).toBeChecked();
|
||||
await expect(soft).toBeDisabled();
|
||||
|
||||
const checkedOptions = toneQuestion.locator('input[type="checkbox"]:checked');
|
||||
await expect(checkedOptions).toHaveCount(2);
|
||||
await expect(soft).not.toBeChecked();
|
||||
await expect(checkedOptions).toHaveCount(2);
|
||||
}
|
||||
|
||||
async function runQuestionFormSubmitPersistenceFlow(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
entry: UICase,
|
||||
) {
|
||||
await sendPrompt(page, entry.prompt);
|
||||
|
||||
const form = page.locator('.question-form').first();
|
||||
await expect(form).toBeVisible();
|
||||
|
||||
const toneQuestion = form.locator('.qf-field', {
|
||||
has: page.getByText('Visual tone (pick up to two)'),
|
||||
});
|
||||
await toneQuestion.locator('label.qf-chip', { has: page.getByText('Editorial / magazine') }).click();
|
||||
await toneQuestion.locator('label.qf-chip', { has: page.getByText('Modern minimal') }).click();
|
||||
|
||||
await form.getByRole('button', { name: 'Send answers' }).click();
|
||||
|
||||
await expect(page.getByText('[form answers — discovery]', { exact: false })).toBeVisible();
|
||||
await expect(form.getByText('answered', { exact: true })).toBeVisible();
|
||||
await expect(form.getByText('Answers sent — agent is using these for the rest of the session.')).toBeVisible();
|
||||
|
||||
const { projectId, conversationId } = await getCurrentProjectContext(page);
|
||||
const messagesResponse = await page.request.get(
|
||||
`/api/projects/${projectId}/conversations/${conversationId}/messages`,
|
||||
);
|
||||
expect(messagesResponse.ok()).toBeTruthy();
|
||||
const { messages } = (await messagesResponse.json()) as { messages: Array<{ role: string; content: string }> };
|
||||
const formAnswerMessage = messages.find((message) => message.role === 'user' && message.content.includes('[form answers — discovery]'));
|
||||
expect(formAnswerMessage).toBeTruthy();
|
||||
|
||||
await page.reload();
|
||||
const restoredForm = page.locator('.question-form').first();
|
||||
await expect(restoredForm).toBeVisible();
|
||||
await expect(restoredForm.getByText('answered', { exact: true })).toBeVisible();
|
||||
await expect(restoredForm.locator('input[type="checkbox"]:checked')).toHaveCount(2);
|
||||
await expect(restoredForm.getByRole('button', { name: 'Send answers' })).toHaveCount(0);
|
||||
}
|
||||
|
||||
async function runGenerationDoesNotCreateExtraFileFlow(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
entry: UICase,
|
||||
) {
|
||||
await sendPrompt(page, entry.prompt);
|
||||
await expectArtifactVisible(page, entry);
|
||||
|
||||
const { projectId } = await getCurrentProjectContext(page);
|
||||
const initialFiles = await listProjectFilesFromApi(page, projectId);
|
||||
expect(initialFiles.map((file) => file.name)).toContain(entry.mockArtifact!.fileName);
|
||||
|
||||
await page.reload();
|
||||
await expect(page.getByTestId('file-workspace')).toBeVisible();
|
||||
|
||||
const reloadedFiles = await listProjectFilesFromApi(page, projectId);
|
||||
expect(reloadedFiles.map((file) => file.name)).toEqual(initialFiles.map((file) => file.name));
|
||||
await expect(page.getByText(entry.mockArtifact!.fileName, { exact: true })).toBeVisible();
|
||||
}
|
||||
|
||||
async function runCommentAttachmentFlow(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
entry: UICase,
|
||||
) {
|
||||
await sendPrompt(page, entry.prompt);
|
||||
await expectArtifactVisible(page, entry);
|
||||
|
||||
await page.getByTestId('comment-mode-toggle').click();
|
||||
const frame = page.frameLocator('[data-testid="artifact-preview-frame"]');
|
||||
await frame.locator('[data-od-id="hero-title"]').click();
|
||||
await expect(page.getByTestId('comment-popover')).toBeVisible();
|
||||
await page.getByTestId('comment-popover-input').fill('Make the headline more specific.');
|
||||
await page.getByTestId('comment-add-send').click();
|
||||
|
||||
await expect(page.getByTestId('staged-comment-attachments')).toBeVisible();
|
||||
await expect(page.getByTestId('staged-comment-attachments')).toContainText('hero-title');
|
||||
await expect(page.getByTestId('staged-comment-attachments')).toContainText('Make the headline more specific.');
|
||||
await expect(page.getByTestId('chat-composer-input')).toHaveValue('');
|
||||
await expect(page.getByTestId('comment-saved-marker-hero-title')).toBeVisible();
|
||||
|
||||
await frame.locator('[data-od-id="hero-copy"]').hover();
|
||||
await expect(page.getByTestId('comment-target-overlay')).toBeVisible();
|
||||
await expect(page.getByTestId('comment-target-overlay')).toContainText('hero-copy');
|
||||
|
||||
await page.getByTestId('comment-saved-marker-hero-title').getByRole('button').click();
|
||||
await expect(page.getByTestId('comment-popover')).toBeVisible();
|
||||
await expect(page.getByTestId('comment-popover-input')).toHaveValue('Make the headline more specific.');
|
||||
await page.getByTestId('comment-popover').getByRole('button', { name: 'Close' }).click();
|
||||
|
||||
await page.getByRole('tab', { name: 'Comments' }).click();
|
||||
await expect(page.getByTestId('comments-panel')).toBeVisible();
|
||||
await expect(page.getByTestId('comments-panel').getByRole('heading', { name: 'Attached to chat' })).toBeVisible();
|
||||
await expect(page.getByTestId('comments-panel').getByRole('heading', { name: 'Saved comments' })).toBeVisible();
|
||||
|
||||
await page.getByTestId('comments-panel')
|
||||
.locator('[data-testid="comment-card-hero-title"]')
|
||||
.getByRole('button', { name: 'Remove' })
|
||||
.click();
|
||||
await page.getByRole('tab', { name: 'Chat' }).click();
|
||||
await expect(page.getByTestId('staged-comment-attachments')).toHaveCount(0);
|
||||
await expect(page.getByTestId('chat-send')).toBeDisabled();
|
||||
|
||||
await page.getByRole('tab', { name: 'Comments' }).click();
|
||||
await page.getByTestId('comments-panel')
|
||||
.locator('[data-testid="comment-card-hero-title"]')
|
||||
.getByRole('button', { name: 'Add' })
|
||||
.click();
|
||||
await page.getByRole('tab', { name: 'Chat' }).click();
|
||||
await expect(page.getByTestId('staged-comment-attachments')).toContainText('hero-title');
|
||||
|
||||
const runRequest = page.waitForRequest(
|
||||
(request) => request.url().includes('/api/runs') && request.method() === 'POST',
|
||||
);
|
||||
await page.getByTestId('chat-send').click();
|
||||
const request = await runRequest;
|
||||
const body = request.postDataJSON() as {
|
||||
message?: string;
|
||||
commentAttachments?: Array<{ elementId?: string; comment?: string; filePath?: string }>;
|
||||
};
|
||||
|
||||
expect(body.message).toMatch(/\n\n## user\n$/);
|
||||
expect(body.message).not.toContain('Apply selected preview comments');
|
||||
expect(body.commentAttachments).toEqual([
|
||||
expect.objectContaining({
|
||||
elementId: 'hero-title',
|
||||
comment: 'Make the headline more specific.',
|
||||
filePath: 'commentable-artifact.html',
|
||||
}),
|
||||
]);
|
||||
}
|
||||
|
||||
async function createProjectNameOnly(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
entry: UICase,
|
||||
) {
|
||||
await expect(page.getByTestId('new-project-panel')).toBeVisible();
|
||||
if (entry.create.tab) {
|
||||
await page.getByTestId(`new-project-tab-${entry.create.tab}`).click();
|
||||
}
|
||||
await page.getByTestId('new-project-name').fill(entry.create.projectName);
|
||||
}
|
||||
|
||||
async function getCurrentProjectContext(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
): Promise<{ projectId: string; conversationId: string }> {
|
||||
const current = new URL(page.url());
|
||||
const [, projects, projectId, maybeConversations, conversationId] = current.pathname.split('/');
|
||||
if (projects !== 'projects' || !projectId) {
|
||||
throw new Error(`unexpected project route: ${current.pathname}`);
|
||||
}
|
||||
if (maybeConversations === 'conversations' && conversationId) {
|
||||
return { projectId, conversationId };
|
||||
}
|
||||
|
||||
const response = await page.request.get(`/api/projects/${projectId}/conversations`);
|
||||
expect(response.ok()).toBeTruthy();
|
||||
const { conversations } = (await response.json()) as {
|
||||
conversations: Array<{ id: string; updatedAt: number }>;
|
||||
};
|
||||
const active = [...conversations].sort((a, b) => b.updatedAt - a.updatedAt)[0];
|
||||
if (!active) throw new Error(`no conversations found for project ${projectId}`);
|
||||
return { projectId, conversationId: active.id };
|
||||
}
|
||||
|
||||
async function listProjectFilesFromApi(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
projectId: string,
|
||||
): Promise<Array<{ name: string; kind: string }>> {
|
||||
const response = await page.request.get(`/api/projects/${projectId}/files`);
|
||||
expect(response.ok()).toBeTruthy();
|
||||
const { files } = (await response.json()) as { files: Array<{ name: string; kind: string }> };
|
||||
return files;
|
||||
}
|
||||
|
||||
async function expectArtifactVisible(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
entry: UICase,
|
||||
) {
|
||||
const artifact = entry.mockArtifact!;
|
||||
await expect(page.getByText(artifact.fileName, { exact: true })).toBeVisible();
|
||||
await expect(page.getByTestId('artifact-preview-frame')).toBeVisible();
|
||||
const frame = page.frameLocator('[data-testid="artifact-preview-frame"]');
|
||||
await expect(frame.getByRole('heading', { name: artifact.heading })).toBeVisible();
|
||||
}
|
||||
|
||||
async function runConversationPersistenceFlow(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
entry: UICase,
|
||||
) {
|
||||
await sendPrompt(page, entry.prompt);
|
||||
await expect(page.getByText(entry.prompt, { exact: true })).toBeVisible();
|
||||
await expectArtifactVisible(page, entry);
|
||||
|
||||
await page.getByTestId('new-conversation').click();
|
||||
await expect(page.getByText('Start a conversation')).toBeVisible();
|
||||
|
||||
const nextPrompt = entry.secondaryPrompt!;
|
||||
await sendPrompt(page, nextPrompt);
|
||||
await expect(page.getByText(nextPrompt, { exact: true })).toBeVisible();
|
||||
|
||||
await page.reload();
|
||||
await expect(page.getByTestId('chat-composer')).toBeVisible();
|
||||
await expect(page.getByText(nextPrompt, { exact: true })).toBeVisible();
|
||||
|
||||
await page.getByTestId('conversation-history-trigger').click();
|
||||
const historyList = page.getByTestId('conversation-list');
|
||||
await expect(historyList).toBeVisible();
|
||||
await expect(historyList.locator('.chat-conv-item')).toHaveCount(2);
|
||||
await historyList
|
||||
.locator('.chat-conv-item')
|
||||
.filter({ hasText: entry.prompt })
|
||||
.first()
|
||||
.locator('[data-testid^="conversation-select-"]')
|
||||
.click();
|
||||
|
||||
await expect(page.getByText(entry.prompt, { exact: true })).toBeVisible();
|
||||
}
|
||||
|
||||
async function runFileMentionFlow(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
entry: UICase,
|
||||
) {
|
||||
const current = new URL(page.url());
|
||||
const [, projects, projectId] = current.pathname.split('/');
|
||||
if (projects !== 'projects' || !projectId) {
|
||||
throw new Error(`unexpected project route: ${current.pathname}`);
|
||||
}
|
||||
|
||||
const resp = await page.request.post(`/api/projects/${projectId}/files`, {
|
||||
data: {
|
||||
name: 'reference.txt',
|
||||
content: 'Reference content for mention flow.\n',
|
||||
},
|
||||
});
|
||||
expect(resp.ok()).toBeTruthy();
|
||||
|
||||
await page.reload();
|
||||
await expect(page.getByTestId('chat-composer')).toBeVisible();
|
||||
await expect(page.getByText('reference.txt', { exact: true })).toBeVisible();
|
||||
|
||||
await page.getByTestId('chat-composer-input').click();
|
||||
await page.getByTestId('chat-composer-input').pressSequentially('Review @ref');
|
||||
await expect(page.getByTestId('mention-popover')).toBeVisible();
|
||||
await page.getByTestId('mention-popover').getByRole('button', { name: /reference\.txt/i }).click();
|
||||
await expect(page.getByTestId('chat-composer-input')).toHaveValue('Review @reference.txt ');
|
||||
await expect(page.getByTestId('staged-attachments')).toBeVisible();
|
||||
await expect(page.getByTestId('staged-attachments').getByText('reference.txt', { exact: true })).toBeVisible();
|
||||
await expect(page.getByTestId('chat-send')).toBeEnabled();
|
||||
}
|
||||
|
||||
async function runDeepLinkPreviewFlow(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
entry: UICase,
|
||||
) {
|
||||
await sendPrompt(page, entry.prompt);
|
||||
await expectArtifactVisible(page, entry);
|
||||
|
||||
const fileName = entry.mockArtifact!.fileName;
|
||||
await expect(page).toHaveURL(new RegExp(`/projects/[^/]+/files/${fileName.replace('.', '\\.')}$`));
|
||||
|
||||
const current = new URL(page.url());
|
||||
const [, projects, projectId] = current.pathname.split('/');
|
||||
if (projects !== 'projects' || !projectId) {
|
||||
throw new Error(`unexpected project route: ${current.pathname}`);
|
||||
}
|
||||
|
||||
await page.goto(`/projects/${projectId}`);
|
||||
await expect(page.getByTestId('file-workspace')).toBeVisible();
|
||||
|
||||
await page.goto(`/projects/${projectId}/files/${fileName}`);
|
||||
await expect(page.getByTestId('artifact-preview-frame')).toBeVisible();
|
||||
const frame = page.frameLocator('[data-testid="artifact-preview-frame"]');
|
||||
await expect(frame.getByRole('heading', { name: entry.mockArtifact!.heading })).toBeVisible();
|
||||
}
|
||||
|
||||
async function runFileUploadSendFlow(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
entry: UICase,
|
||||
) {
|
||||
const uploadResponse = page.waitForResponse(
|
||||
(resp) => resp.url().includes('/upload') && resp.request().method() === 'POST',
|
||||
{ timeout: 5000 },
|
||||
);
|
||||
await page.getByTestId('chat-file-input').setInputFiles({
|
||||
name: 'reference.txt',
|
||||
mimeType: 'text/plain',
|
||||
buffer: Buffer.from('Reference content for upload flow.\n', 'utf8'),
|
||||
});
|
||||
await expect((await uploadResponse).ok()).toBeTruthy();
|
||||
|
||||
await expect(page.getByTestId('staged-attachments')).toBeVisible();
|
||||
await expect(
|
||||
page.getByTestId('staged-attachments').getByText('reference.txt', { exact: true }),
|
||||
).toBeVisible();
|
||||
await expect(page.getByText('reference.txt', { exact: true })).toBeVisible();
|
||||
|
||||
await sendPrompt(page, entry.prompt);
|
||||
await expect(page.getByText(entry.prompt, { exact: true })).toBeVisible();
|
||||
await expect(page.locator('.user-attachments').getByText('reference.txt', { exact: true })).toBeVisible();
|
||||
}
|
||||
|
||||
async function runDesignFilesUploadFlow(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
) {
|
||||
await page.getByTestId('design-files-upload-input').setInputFiles({
|
||||
name: 'moodboard.png',
|
||||
mimeType: 'image/png',
|
||||
buffer: Buffer.from(
|
||||
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO5W6McAAAAASUVORK5CYII=',
|
||||
'base64',
|
||||
),
|
||||
});
|
||||
|
||||
await expect(page.getByRole('tab', { name: /moodboard\.png/i })).toBeVisible();
|
||||
await page.getByTestId('design-files-tab').click();
|
||||
const fileRow = page.locator('[data-testid^="design-file-row-"]', {
|
||||
hasText: 'moodboard.png',
|
||||
});
|
||||
await expect(fileRow).toBeVisible();
|
||||
await fileRow.click();
|
||||
const preview = page.getByTestId('design-file-preview');
|
||||
await expect(preview).toBeVisible();
|
||||
await expect(preview.getByText(/moodboard\.png/i)).toBeVisible();
|
||||
|
||||
await fileRow.dblclick();
|
||||
await expect(page.getByRole('tab', { name: /moodboard\.png/i })).toBeVisible();
|
||||
}
|
||||
|
||||
async function runDesignFilesDeleteFlow(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
) {
|
||||
page.on('dialog', async (dialog) => {
|
||||
await dialog.accept();
|
||||
});
|
||||
|
||||
const pngBytes = Buffer.from(
|
||||
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO5W6McAAAAASUVORK5CYII=',
|
||||
'base64',
|
||||
);
|
||||
|
||||
// Upload a sibling file first so that, after deleting trash-me.png, there
|
||||
// is a fallback tab the buggy code would have navigated to. The fix must
|
||||
// keep the user in the Design Files panel instead.
|
||||
await page.getByTestId('design-files-upload-input').setInputFiles({
|
||||
name: 'keep-me.png',
|
||||
mimeType: 'image/png',
|
||||
buffer: pngBytes,
|
||||
});
|
||||
await expect(page.getByRole('tab', { name: /keep-me\.png/i })).toBeVisible();
|
||||
|
||||
await page.getByTestId('design-files-upload-input').setInputFiles({
|
||||
name: 'trash-me.png',
|
||||
mimeType: 'image/png',
|
||||
buffer: pngBytes,
|
||||
});
|
||||
|
||||
await expect(page.getByRole('tab', { name: /trash-me\.png/i })).toBeVisible();
|
||||
await page.getByTestId('design-files-tab').click();
|
||||
|
||||
const fileRow = page.locator('[data-testid^="design-file-row-"]', {
|
||||
hasText: 'trash-me.png',
|
||||
});
|
||||
await expect(fileRow).toBeVisible();
|
||||
await fileRow.hover();
|
||||
await fileRow.locator('[data-testid^="design-file-menu-"]').click();
|
||||
await expect(page.getByTestId('design-file-menu-popover')).toBeVisible();
|
||||
await page.locator('[data-testid^="design-file-delete-"]').click();
|
||||
|
||||
await expect(fileRow).toHaveCount(0);
|
||||
await expect(page.getByRole('tab', { name: /trash-me\.png/i })).toHaveCount(0);
|
||||
|
||||
// Bug #115: deleting from the Design Files panel must not navigate the
|
||||
// user into another tab. The Design Files tab should remain the active
|
||||
// view, and the sibling tab should still exist (just not auto-activated).
|
||||
await expect(page.getByTestId('design-files-tab')).toHaveAttribute(
|
||||
'aria-selected',
|
||||
'true',
|
||||
);
|
||||
await expect(page.getByRole('tab', { name: /keep-me\.png/i })).toBeVisible();
|
||||
}
|
||||
|
||||
async function runDesignFilesTabPersistenceFlow(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
) {
|
||||
const pngBytes = Buffer.from(
|
||||
'iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO5W6McAAAAASUVORK5CYII=',
|
||||
'base64',
|
||||
);
|
||||
|
||||
await page.getByTestId('design-files-upload-input').setInputFiles({
|
||||
name: 'first-tab.png',
|
||||
mimeType: 'image/png',
|
||||
buffer: pngBytes,
|
||||
});
|
||||
await expect(page.getByRole('tab', { name: /first-tab\.png/i })).toBeVisible();
|
||||
|
||||
await page.getByTestId('design-files-upload-input').setInputFiles({
|
||||
name: 'second-tab.png',
|
||||
mimeType: 'image/png',
|
||||
buffer: pngBytes,
|
||||
});
|
||||
const firstTab = page.getByRole('tab', { name: /first-tab\.png/i });
|
||||
const secondTab = page.getByRole('tab', { name: /second-tab\.png/i });
|
||||
await expect(firstTab).toBeVisible();
|
||||
await expect(secondTab).toBeVisible();
|
||||
|
||||
await firstTab.click();
|
||||
await expect(firstTab).toHaveAttribute('aria-selected', 'true');
|
||||
await expect(secondTab).toHaveAttribute('aria-selected', 'false');
|
||||
|
||||
await page.reload();
|
||||
|
||||
const restoredFirstTab = page.getByRole('tab', { name: /first-tab\.png/i });
|
||||
const restoredSecondTab = page.getByRole('tab', { name: /second-tab\.png/i });
|
||||
await expect(restoredFirstTab).toBeVisible();
|
||||
await expect(restoredSecondTab).toBeVisible();
|
||||
await expect(restoredFirstTab).toHaveAttribute('aria-selected', 'true');
|
||||
await expect(restoredSecondTab).toHaveAttribute('aria-selected', 'false');
|
||||
}
|
||||
|
||||
async function runConversationDeleteRecoveryFlow(
|
||||
page: Parameters<typeof test>[0]['page'],
|
||||
entry: UICase,
|
||||
) {
|
||||
page.on('dialog', async (dialog) => {
|
||||
await dialog.accept();
|
||||
});
|
||||
|
||||
await sendPrompt(page, entry.prompt);
|
||||
await expect(
|
||||
page.locator('.msg.user .user-text').filter({ hasText: entry.prompt }).first(),
|
||||
).toBeVisible();
|
||||
|
||||
await page.getByTestId('new-conversation').click();
|
||||
await expect(page.getByText('Start a conversation')).toBeVisible();
|
||||
|
||||
const nextPrompt = entry.secondaryPrompt!;
|
||||
await sendPrompt(page, nextPrompt);
|
||||
await expect(
|
||||
page.locator('.msg.user .user-text').filter({ hasText: nextPrompt }).first(),
|
||||
).toBeVisible();
|
||||
|
||||
await page.getByTestId('conversation-history-trigger').click();
|
||||
await expect(page.getByTestId('conversation-list')).toBeVisible();
|
||||
|
||||
const activeRow = page
|
||||
.getByTestId('conversation-list')
|
||||
.locator('.chat-conv-item.active')
|
||||
.first();
|
||||
await expect(activeRow).toBeVisible();
|
||||
await activeRow.getByTestId(/conversation-delete-/).click();
|
||||
|
||||
await expect(
|
||||
page.locator('.msg.user .user-text').filter({ hasText: entry.prompt }).first(),
|
||||
).toBeVisible();
|
||||
await expect(page.locator('.msg.user .user-text').filter({ hasText: nextPrompt })).toHaveCount(0);
|
||||
|
||||
await page.getByTestId('conversation-history-trigger').click();
|
||||
await expect(page.getByTestId('conversation-list').locator('.chat-conv-item')).toHaveCount(1);
|
||||
}
|
||||
@@ -0,0 +1,112 @@
|
||||
import { cleanup, fireEvent, render, screen, within } from '@testing-library/react';
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import { AssistantMessage } from '../../apps/web/src/components/AssistantMessage';
|
||||
import type { AgentEvent, ChatMessage } from '../../apps/web/src/types';
|
||||
|
||||
function messageWithEvents(events: AgentEvent[]): ChatMessage {
|
||||
return {
|
||||
id: 'assistant-1',
|
||||
role: 'assistant',
|
||||
content: '',
|
||||
events,
|
||||
startedAt: 1_000,
|
||||
endedAt: 3_000,
|
||||
};
|
||||
}
|
||||
|
||||
describe('AssistantMessage unfinished todo state', () => {
|
||||
afterEach(() => cleanup());
|
||||
|
||||
it('keeps Done for a completed latest TodoWrite fixture', () => {
|
||||
render(
|
||||
<AssistantMessage
|
||||
message={messageWithEvents([
|
||||
{
|
||||
kind: 'tool_use',
|
||||
id: 'todo-1',
|
||||
name: 'TodoWrite',
|
||||
input: { todos: [{ content: 'Ship layout', status: 'completed' }] },
|
||||
},
|
||||
])}
|
||||
streaming={false}
|
||||
projectId="project-1"
|
||||
isLast
|
||||
/>,
|
||||
);
|
||||
|
||||
expect(screen.getByText('Done')).toBeTruthy();
|
||||
expect(screen.queryByText('Stopped with unfinished work')).toBeNull();
|
||||
expect(screen.queryByRole('button', { name: 'Continue remaining tasks' })).toBeNull();
|
||||
});
|
||||
|
||||
it('shows unfinished state and passes unfinished todos to the continue callback', () => {
|
||||
const onContinue = vi.fn();
|
||||
render(
|
||||
<AssistantMessage
|
||||
message={messageWithEvents([
|
||||
{
|
||||
kind: 'tool_use',
|
||||
id: 'todo-1',
|
||||
name: 'TodoWrite',
|
||||
input: {
|
||||
todos: [
|
||||
{ content: 'Draft layout', status: 'completed' },
|
||||
{
|
||||
content: 'Build components',
|
||||
status: 'in_progress',
|
||||
activeForm: 'Building components',
|
||||
},
|
||||
{ content: 'Run QA', status: 'pending' },
|
||||
],
|
||||
},
|
||||
},
|
||||
])}
|
||||
streaming={false}
|
||||
projectId="project-1"
|
||||
isLast
|
||||
onContinueRemainingTasks={onContinue}
|
||||
/>,
|
||||
);
|
||||
|
||||
expect(screen.getByText('Stopped with unfinished work')).toBeTruthy();
|
||||
expect(screen.getByText('2 task(s) remain')).toBeTruthy();
|
||||
const remainingList = screen.getByText('2 task(s) remain').closest('.unfinished-todos');
|
||||
expect(remainingList).not.toBeNull();
|
||||
expect(within(remainingList as HTMLElement).getByText('Building components')).toBeTruthy();
|
||||
expect(within(remainingList as HTMLElement).getByText('Run QA')).toBeTruthy();
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: 'Continue remaining tasks' }));
|
||||
|
||||
expect(onContinue).toHaveBeenCalledWith([
|
||||
{
|
||||
content: 'Build components',
|
||||
status: 'in_progress',
|
||||
activeForm: 'Building components',
|
||||
},
|
||||
{ content: 'Run QA', status: 'pending', activeForm: undefined },
|
||||
]);
|
||||
});
|
||||
|
||||
it('hides the continue button on older assistant turns', () => {
|
||||
render(
|
||||
<AssistantMessage
|
||||
message={messageWithEvents([
|
||||
{
|
||||
kind: 'tool_use',
|
||||
id: 'todo-1',
|
||||
name: 'TodoWrite',
|
||||
input: { todos: [{ content: 'Run QA', status: 'pending' }] },
|
||||
},
|
||||
])}
|
||||
streaming={false}
|
||||
projectId="project-1"
|
||||
isLast={false}
|
||||
onContinueRemainingTasks={vi.fn()}
|
||||
/>,
|
||||
);
|
||||
|
||||
expect(screen.getByText('Stopped with unfinished work')).toBeTruthy();
|
||||
expect(screen.getByText('1 task(s) remain')).toBeTruthy();
|
||||
expect(screen.queryByRole('button', { name: 'Continue remaining tasks' })).toBeNull();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,77 @@
|
||||
import { cleanup, render, screen } from '@testing-library/react';
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import { ChatPane } from '../../apps/web/src/components/ChatPane';
|
||||
import type { ChatMessage } from '../../apps/web/src/types';
|
||||
|
||||
function renderChatPane(messages: ChatMessage[]) {
|
||||
return render(
|
||||
<ChatPane
|
||||
messages={messages}
|
||||
streaming={false}
|
||||
error={null}
|
||||
projectId="project-1"
|
||||
projectFiles={[]}
|
||||
onEnsureProject={async () => 'project-1'}
|
||||
onSend={() => {}}
|
||||
onStop={() => {}}
|
||||
conversations={[]}
|
||||
activeConversationId={null}
|
||||
onSelectConversation={() => {}}
|
||||
onDeleteConversation={() => {}}
|
||||
/>,
|
||||
);
|
||||
}
|
||||
|
||||
describe('conversation timestamps', () => {
|
||||
afterEach(() => {
|
||||
cleanup();
|
||||
vi.useRealTimers();
|
||||
});
|
||||
|
||||
it('shows inline relative message times with exact hover text', () => {
|
||||
vi.useFakeTimers();
|
||||
vi.setSystemTime(new Date('2025-01-15T14:00:00Z'));
|
||||
|
||||
renderChatPane([
|
||||
{
|
||||
id: 'user-1',
|
||||
role: 'user',
|
||||
content: 'Create a landing page',
|
||||
createdAt: Date.parse('2025-01-15T12:00:00Z'),
|
||||
},
|
||||
{
|
||||
id: 'assistant-1',
|
||||
role: 'assistant',
|
||||
content: 'Done',
|
||||
createdAt: Date.parse('2025-01-15T12:01:00Z'),
|
||||
},
|
||||
]);
|
||||
|
||||
const firstTime = screen.getByText('2h ago');
|
||||
expect(firstTime.tagName).toBe('TIME');
|
||||
expect(firstTime.getAttribute('title')).toContain('2025');
|
||||
expect(screen.getByText('1h ago').tagName).toBe('TIME');
|
||||
});
|
||||
|
||||
it('adds day separators when a conversation crosses days', () => {
|
||||
vi.useFakeTimers();
|
||||
vi.setSystemTime(new Date('2025-01-16T14:00:00Z'));
|
||||
|
||||
renderChatPane([
|
||||
{
|
||||
id: 'user-1',
|
||||
role: 'user',
|
||||
content: 'First request',
|
||||
createdAt: Date.parse('2025-01-15T12:00:00Z'),
|
||||
},
|
||||
{
|
||||
id: 'user-2',
|
||||
role: 'user',
|
||||
content: 'Follow-up',
|
||||
createdAt: Date.parse('2025-01-16T12:00:00Z'),
|
||||
},
|
||||
]);
|
||||
|
||||
expect(screen.getAllByRole('separator')).toHaveLength(2);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,109 @@
|
||||
import { act, cleanup, fireEvent, render } from '@testing-library/react';
|
||||
import { afterEach, describe, expect, it, vi } from 'vitest';
|
||||
import { PreviewModal } from '../../apps/web/src/components/PreviewModal';
|
||||
|
||||
// Regression coverage for nexu-io/open-design#141: pressing Esc in fullscreen
|
||||
// used to require two presses because the browser exits its native fullscreen
|
||||
// element on the first press without delivering a keydown to JS, leaving the
|
||||
// React `fullscreen` state stuck on. The fix listens to fullscreenchange and
|
||||
// mirrors the native state into React.
|
||||
|
||||
const baseProps = {
|
||||
title: 'Sample',
|
||||
views: [{ id: 'main', label: 'Main', html: '<p>hi</p>' }],
|
||||
exportTitleFor: (id: string) => id,
|
||||
};
|
||||
|
||||
function dispatchFullscreenChange() {
|
||||
act(() => {
|
||||
document.dispatchEvent(new Event('fullscreenchange'));
|
||||
});
|
||||
}
|
||||
|
||||
function setNativeFullscreenElement(el: Element | null) {
|
||||
Object.defineProperty(document, 'fullscreenElement', {
|
||||
configurable: true,
|
||||
get: () => el,
|
||||
});
|
||||
}
|
||||
|
||||
describe('PreviewModal fullscreen exit', () => {
|
||||
afterEach(() => {
|
||||
cleanup();
|
||||
setNativeFullscreenElement(null);
|
||||
});
|
||||
|
||||
it('drops the fullscreen overlay when the browser exits native fullscreen', () => {
|
||||
const onClose = vi.fn();
|
||||
const { container } = render(
|
||||
<PreviewModal {...baseProps} onClose={onClose} />,
|
||||
);
|
||||
|
||||
// Click the Fullscreen button. jsdom does not implement requestFullscreen
|
||||
// on plain elements, so PreviewModal's fallback path runs and just sets
|
||||
// the React state — exactly matching what happens after a successful
|
||||
// browser fullscreen request.
|
||||
const fsButton = container.querySelector(
|
||||
'button[title="Fullscreen"]',
|
||||
) as HTMLButtonElement;
|
||||
expect(fsButton).toBeTruthy();
|
||||
fireEvent.click(fsButton);
|
||||
const stage = container.querySelector('.ds-modal') as HTMLElement;
|
||||
expect(stage.classList.contains('ds-modal-fullscreen')).toBe(true);
|
||||
|
||||
// Simulate the user pressing Esc in browser fullscreen: the browser
|
||||
// exits its native fullscreen element and fires fullscreenchange, but
|
||||
// (in browsers like Firefox) does not deliver the keydown to JS.
|
||||
setNativeFullscreenElement(null);
|
||||
dispatchFullscreenChange();
|
||||
|
||||
expect(stage.classList.contains('ds-modal-fullscreen')).toBe(false);
|
||||
expect(onClose).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('keeps the modal mounted on Esc while fullscreen, and closes only on a second Esc', () => {
|
||||
const onClose = vi.fn();
|
||||
const { container } = render(
|
||||
<PreviewModal {...baseProps} onClose={onClose} />,
|
||||
);
|
||||
const fsButton = container.querySelector(
|
||||
'button[title="Fullscreen"]',
|
||||
) as HTMLButtonElement;
|
||||
fireEvent.click(fsButton);
|
||||
const stage = container.querySelector('.ds-modal') as HTMLElement;
|
||||
expect(stage.classList.contains('ds-modal-fullscreen')).toBe(true);
|
||||
|
||||
// First Esc — drops fullscreen, must not close the modal.
|
||||
fireEvent.keyDown(document, { key: 'Escape' });
|
||||
expect(stage.classList.contains('ds-modal-fullscreen')).toBe(false);
|
||||
expect(onClose).not.toHaveBeenCalled();
|
||||
|
||||
// Second Esc — closes the modal.
|
||||
fireEvent.keyDown(document, { key: 'Escape' });
|
||||
expect(onClose).toHaveBeenCalledTimes(1);
|
||||
});
|
||||
|
||||
it('ignores fullscreenchange when another element is still fullscreen', () => {
|
||||
const onClose = vi.fn();
|
||||
const { container } = render(
|
||||
<PreviewModal {...baseProps} onClose={onClose} />,
|
||||
);
|
||||
const fsButton = container.querySelector(
|
||||
'button[title="Fullscreen"]',
|
||||
) as HTMLButtonElement;
|
||||
fireEvent.click(fsButton);
|
||||
const stage = container.querySelector('.ds-modal') as HTMLElement;
|
||||
expect(stage.classList.contains('ds-modal-fullscreen')).toBe(true);
|
||||
|
||||
// Some other element is the active fullscreen target — our overlay must
|
||||
// not collapse to non-fullscreen on transitions that leave a different
|
||||
// element fullscreen.
|
||||
const other = document.createElement('div');
|
||||
document.body.appendChild(other);
|
||||
setNativeFullscreenElement(other);
|
||||
dispatchFullscreenChange();
|
||||
|
||||
expect(stage.classList.contains('ds-modal-fullscreen')).toBe(true);
|
||||
document.body.removeChild(other);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,92 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import { createClaudeStreamHandler } from '../../apps/daemon/src/claude-stream.js';
|
||||
import { createCopilotStreamHandler } from '../../apps/daemon/src/copilot-stream.js';
|
||||
import { mapPiRpcEvent } from '../../apps/daemon/src/pi-rpc.js';
|
||||
|
||||
describe('structured agent stream fixtures', () => {
|
||||
it('emits TodoWrite tool_use from Claude Code stream JSON', () => {
|
||||
const events: unknown[] = [];
|
||||
const handler = createClaudeStreamHandler((event: unknown) => events.push(event));
|
||||
handler.feed(`${JSON.stringify({
|
||||
type: 'assistant',
|
||||
message: {
|
||||
id: 'msg-1',
|
||||
content: [
|
||||
{
|
||||
type: 'tool_use',
|
||||
id: 'toolu-1',
|
||||
name: 'TodoWrite',
|
||||
input: {
|
||||
todos: [{ content: 'Run QA', status: 'pending' }],
|
||||
},
|
||||
},
|
||||
],
|
||||
},
|
||||
})}\n`);
|
||||
handler.flush();
|
||||
|
||||
expect(events).toContainEqual({
|
||||
type: 'tool_use',
|
||||
id: 'toolu-1',
|
||||
name: 'TodoWrite',
|
||||
input: {
|
||||
todos: [{ content: 'Run QA', status: 'pending' }],
|
||||
},
|
||||
});
|
||||
});
|
||||
|
||||
it('emits TodoWrite tool_use from Pi RPC tool_execution events', () => {
|
||||
const events: unknown[] = [];
|
||||
const send = (_channel: string, payload: unknown) => { events.push(payload); };
|
||||
const ctx = { runStartedAt: Date.now(), sentFirstToken: { value: false } };
|
||||
|
||||
mapPiRpcEvent(
|
||||
{ type: 'tool_execution_start', toolCallId: 'pi-call-1', toolName: 'TodoWrite', args: { todos: [{ content: 'Run QA', status: 'pending' }] } },
|
||||
send,
|
||||
ctx,
|
||||
);
|
||||
mapPiRpcEvent(
|
||||
{ type: 'tool_execution_end', toolCallId: 'pi-call-1', toolName: 'TodoWrite', result: { content: [{ type: 'text', text: 'written' }] }, isError: false },
|
||||
send,
|
||||
ctx,
|
||||
);
|
||||
|
||||
expect(events).toContainEqual({
|
||||
type: 'tool_use',
|
||||
id: 'pi-call-1',
|
||||
name: 'TodoWrite',
|
||||
input: { todos: [{ content: 'Run QA', status: 'pending' }] },
|
||||
});
|
||||
expect(events).toContainEqual({
|
||||
type: 'tool_result',
|
||||
toolUseId: 'pi-call-1',
|
||||
content: 'written',
|
||||
isError: false,
|
||||
});
|
||||
});
|
||||
|
||||
it('emits TodoWrite tool_use from GitHub Copilot CLI JSON stream', () => {
|
||||
const events: unknown[] = [];
|
||||
const handler = createCopilotStreamHandler((event: unknown) => events.push(event));
|
||||
handler.feed(`${JSON.stringify({
|
||||
type: 'tool.execution_start',
|
||||
data: {
|
||||
toolCallId: 'call-1',
|
||||
toolName: 'TodoWrite',
|
||||
arguments: {
|
||||
todos: [{ content: 'Run QA', status: 'pending' }],
|
||||
},
|
||||
},
|
||||
})}\n`);
|
||||
handler.flush();
|
||||
|
||||
expect(events).toContainEqual({
|
||||
type: 'tool_use',
|
||||
id: 'call-1',
|
||||
name: 'TodoWrite',
|
||||
input: {
|
||||
todos: [{ content: 'Run QA', status: 'pending' }],
|
||||
},
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,84 @@
|
||||
import { describe, expect, it } from 'vitest';
|
||||
import {
|
||||
latestTodosFromEvents,
|
||||
parseTodoWriteInput,
|
||||
unfinishedTodosFromEvents,
|
||||
} from '../../apps/web/src/runtime/todos';
|
||||
import type { AgentEvent } from '../../apps/web/src/types';
|
||||
|
||||
const firstTodoInput = {
|
||||
todos: [
|
||||
{ content: 'Draft layout', status: 'completed' },
|
||||
{ content: 'Build components', status: 'in_progress', activeForm: 'Building components' },
|
||||
{ content: 'Run QA', status: 'pending' },
|
||||
{ content: '', status: 'pending' },
|
||||
{ content: 'Unknown status defaults pending', status: 'blocked' },
|
||||
null,
|
||||
],
|
||||
};
|
||||
|
||||
describe('todo event helpers', () => {
|
||||
it('normalizes TodoWrite input and ignores malformed items', () => {
|
||||
expect(parseTodoWriteInput(firstTodoInput)).toEqual([
|
||||
{ content: 'Draft layout', status: 'completed', activeForm: undefined },
|
||||
{
|
||||
content: 'Build components',
|
||||
status: 'in_progress',
|
||||
activeForm: 'Building components',
|
||||
},
|
||||
{ content: 'Run QA', status: 'pending', activeForm: undefined },
|
||||
{
|
||||
content: 'Unknown status defaults pending',
|
||||
status: 'pending',
|
||||
activeForm: undefined,
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it('uses the latest TodoWrite event as the current todo truth', () => {
|
||||
const events: AgentEvent[] = [
|
||||
{ kind: 'tool_use', id: 'todo-1', name: 'TodoWrite', input: firstTodoInput },
|
||||
{ kind: 'text', text: 'Working...' },
|
||||
{ kind: 'tool_use', id: 'todo-empty', name: 'TodoWrite', input: { todos: [] } },
|
||||
{
|
||||
kind: 'tool_use',
|
||||
id: 'todo-2',
|
||||
name: 'TodoWrite',
|
||||
input: { todos: [{ content: 'Final polish', status: 'pending' }] },
|
||||
},
|
||||
];
|
||||
|
||||
expect(latestTodosFromEvents(events)).toEqual([
|
||||
{ content: 'Final polish', status: 'pending', activeForm: undefined },
|
||||
]);
|
||||
});
|
||||
|
||||
it('treats an empty latest TodoWrite event as authoritative', () => {
|
||||
const events: AgentEvent[] = [
|
||||
{ kind: 'tool_use', id: 'todo-1', name: 'TodoWrite', input: firstTodoInput },
|
||||
{ kind: 'text', text: 'All done.' },
|
||||
{ kind: 'tool_use', id: 'todo-empty', name: 'TodoWrite', input: { todos: [] } },
|
||||
];
|
||||
|
||||
expect(latestTodosFromEvents(events)).toEqual([]);
|
||||
expect(unfinishedTodosFromEvents(events)).toEqual([]);
|
||||
});
|
||||
|
||||
it('returns only pending and in-progress todos as unfinished', () => {
|
||||
expect(unfinishedTodosFromEvents([
|
||||
{ kind: 'tool_use', id: 'todo-1', name: 'TodoWrite', input: firstTodoInput },
|
||||
])).toEqual([
|
||||
{
|
||||
content: 'Build components',
|
||||
status: 'in_progress',
|
||||
activeForm: 'Building components',
|
||||
},
|
||||
{ content: 'Run QA', status: 'pending', activeForm: undefined },
|
||||
{
|
||||
content: 'Unknown status defaults pending',
|
||||
status: 'pending',
|
||||
activeForm: undefined,
|
||||
},
|
||||
]);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2022",
|
||||
"lib": ["ES2022", "DOM", "DOM.Iterable"],
|
||||
"module": "NodeNext",
|
||||
"moduleResolution": "NodeNext",
|
||||
"jsx": "react-jsx",
|
||||
"strict": true,
|
||||
"noUncheckedIndexedAccess": true,
|
||||
"exactOptionalPropertyTypes": true,
|
||||
"allowImportingTsExtensions": true,
|
||||
"noEmit": true,
|
||||
"isolatedModules": true,
|
||||
"esModuleInterop": true,
|
||||
"forceConsistentCasingInFileNames": true,
|
||||
"resolveJsonModule": true,
|
||||
"skipLibCheck": true,
|
||||
"types": ["node", "vitest"]
|
||||
},
|
||||
"include": [
|
||||
"playwright.config.ts",
|
||||
"vitest.config.ts",
|
||||
"cases/report-metadata.ts",
|
||||
"reporters/**/*.ts",
|
||||
"scripts/**/*.ts"
|
||||
],
|
||||
"exclude": ["node_modules", "reports", ".od-data"]
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
import { defineConfig } from 'vitest/config';
|
||||
|
||||
export default defineConfig({
|
||||
esbuild: {
|
||||
jsx: 'automatic',
|
||||
jsxImportSource: 'react',
|
||||
},
|
||||
test: {
|
||||
environment: 'jsdom',
|
||||
include: ['tests/**/*.test.{ts,tsx}'],
|
||||
},
|
||||
});
|
||||
Reference in New Issue
Block a user