perf(logs): 日志事件按窗口合并推送,抽屉改为虚拟滚动

- 后端 LogSink 把逐行事件按 16 KiB / 100 ms 合并成一条 log 事件,文件写入保持即时;后台 tick 与析构 flush 保证低输出和收尾都不丢尾部
- 前端新增 LogViewer:固定行高虚拟滚动、跟随底部/回到最新,增量按 rAF 合并,长日志 DOM 节点数恒定
- 日志抽屉改为行缓冲(lib/logLines)+ 可见窗口计算(lib/logWindow)
- 详情区改 flex 布局,日志面板吃掉剩余高度,回到最新按钮不再被裁到面板外
- 测试:Rust 单元 52→57,Vitest 13→28,Playwright 4→5(1500 行虚拟滚动)
- 文档:README 补充合并推送与虚拟滚动,新增 ADR-0008
This commit is contained in:
2026-10-27 22:38:53 +08:00
parent 14fb3563c5
commit 07ca14ea34
12 changed files with 867 additions and 59 deletions
+13 -4
View File
@@ -5,7 +5,7 @@ TaskPilot 是一个面向 **Windows 单机** 的本地定时任务管理平台
- 单用户、单实例,只监听 `127.0.0.1` - 单用户、单实例,只监听 `127.0.0.1`
- 三类任务:Python/Shell 脚本、HTTP 请求、二进制进程 - 三类任务:Python/Shell 脚本、HTTP 请求、二进制进程
- SQLite 持久化,重启后任务与历史仍在 - SQLite 持久化,重启后任务与历史仍在
- 运行日志实时推送(SSE),断线自动回退轮询 - 运行日志实时推送(SSE),按窗口合并事件;断线自动回退轮询,长日志用虚拟滚动展示
- 前端零外部依赖:样式与脚本随包自带;界面字体可选,未生成时自动回退系统字体 - 前端零外部依赖:样式与脚本随包自带;界面字体可选,未生成时自动回退系统字体
> 根目录的 `task_pilot.html` 只是最初的单文件视觉原型,正式入口是 React 应用。 > 根目录的 `task_pilot.html` 只是最初的单文件视觉原型,正式入口是 React 应用。
@@ -184,6 +184,11 @@ queued ──► running ──► succeeded / failed / timed_out / cancelled
- **单行 256 KiB** 上限:超出部分丢弃、继续读到行尾,避免内存无界增长。 - **单行 256 KiB** 上限:超出部分丢弃、继续读到行尾,避免内存无界增长。
- 保留期(默认 30 天)与总容量(默认 512 MiB)清理;**跳过正在写入的日志**。 - 保留期(默认 30 天)与总容量(默认 512 MiB)清理;**跳过正在写入的日志**。
- 读取按 64 KiB 分块且对齐 UTF-8 边界;`complete` 只在"运行结束且读到文件末尾"时为真。 - 读取按 64 KiB 分块且对齐 UTF-8 边界;`complete` 只在"运行结束且读到文件末尾"时为真。
- **合并推送**:子进程输出按行写入文件后,先在内存里按 **100 ms / 16 KiB** 窗口合并,
每个窗口只发一条 SSE `log` 事件(偏移仍严格连续);低输出时由后台任务按时推,
运行结束时在 `LogSink` 析构里兜底 flush,不会丢尾部。
- **虚拟滚动**:日志抽屉只渲染视口附近的行(固定行高 20 px),几万行时 DOM 节点数恒定;
增量先并进行缓冲、一帧最多重渲染一次。往上翻时停止跟随并显示「回到最新」。长行横向滚动、不折行。
- **输出编码**:默认 `auto` = 逐行先按 UTF-8 解码,失败回退到系统 ANSI 代码页(简体中文为 GBK)。 - **输出编码**:默认 `auto` = 逐行先按 UTF-8 解码,失败回退到系统 ANSI 代码页(简体中文为 GBK)。
可用 `TASK_PILOT_LOG_ENCODING=gbk|utf-8|windows-1251` 强制。 可用 `TASK_PILOT_LOG_ENCODING=gbk|utf-8|windows-1251` 强制。
- **脱敏**:环境变量/Header 名包含 `SECRET`、`TOKEN`、`PASSWORD`、`PASS`、`API_KEY`、 - **脱敏**:环境变量/Header 名包含 `SECRET`、`TOKEN`、`PASSWORD`、`PASS`、`API_KEY`、
@@ -422,6 +427,9 @@ React (Vite) ──fetch──► Axum router ──security::api_guard(Origin +
| `components/TaskEditorModal.tsx` | 三类动态表单、Cron 预览、虚拟环境状态条 | | `components/TaskEditorModal.tsx` | 三类动态表单、Cron 预览、虚拟环境状态条 |
| `components/DirectoryPickerDialog.tsx` | 页面内目录浏览器 | | `components/DirectoryPickerDialog.tsx` | 页面内目录浏览器 |
| `components/RunLogDrawer.tsx` | 运行历史、日志、取消、下载、SSE | | `components/RunLogDrawer.tsx` | 运行历史、日志、取消、下载、SSE |
| `components/LogViewer.tsx` | 日志虚拟滚动、跟随底部/回到最新 |
| `lib/logLines.ts` | 增量日志的行缓冲(残行拼接、CRLF 归一) |
| `lib/logWindow.ts` | 可见行窗口计算 |
| `api/client.ts` | 类型化客户端、令牌缓存、403 重试 | | `api/client.ts` | 类型化客户端、令牌缓存、403 重试 |
| `lib/taskForm.ts` | Zod 校验与表单 ↔ API 载荷映射 | | `lib/taskForm.ts` | Zod 校验与表单 ↔ API 载荷映射 |
@@ -483,10 +491,10 @@ uv run --with fonttools --with brotli tools/build-fonts.py
| 层 | 覆盖 | | 层 | 覆盖 |
| --- | --- | | --- | --- |
| Rust 单元(52) | Cron、虚拟环境解析与探测、执行器(编码/截断/清理/句柄归属)、安全、目录列举 | | Rust 单元(57) | Cron、虚拟环境解析与探测、执行器(编码/截断/清理/句柄归属/日志合并推送)、安全、目录列举 |
| Rust 集成(30) | 任务 CRUD 与校验、运行生命周期、取消与超时、日志分块、调度恢复、统计与调度接口、重启对账 | | Rust 集成(30) | 任务 CRUD 与校验、运行生命周期、取消与超时、日志分块、调度恢复、统计与调度接口、重启对账 |
| 前端 Vitest(13) | 仪表盘渲染、表单载荷、虚拟环境状态、目录浏览器(进入/取消/错误)、下次触发时间 | | 前端 Vitest(28) | 仪表盘渲染、表单载荷、虚拟环境状态、目录浏览器、下次触发时间、日志行切分/窗口计算/虚拟滚动 |
| Playwright(4) | 真实任务执行与日志、编辑/停用/删除、窄屏、**无外部网络渲染**(拦截所有外部请求) | | Playwright(5) | 真实任务执行与日志、编辑/停用/删除、窄屏、**无外部网络渲染**、1500 行日志的虚拟滚动 |
| Python 自检(14 项) | `tools/verify.py` 覆盖 API 全链路 | | Python 自检(14 项) | `tools/verify.py` 覆盖 API 全链路 |
--- ---
@@ -530,6 +538,7 @@ uv run --with fonttools --with brotli tools/build-fonts.py
- [ADR-0005:由后端提供系统文件夹选择](docs/decisions/0005-folder-picker.md) - [ADR-0005:由后端提供系统文件夹选择](docs/decisions/0005-folder-picker.md)
- [ADR-0006:视觉风格对齐主题并自托管字体](docs/decisions/0006-visual-theme-and-offline-fonts.md) - [ADR-0006:视觉风格对齐主题并自托管字体](docs/decisions/0006-visual-theme-and-offline-fonts.md)
- [ADR-0007:运行期可靠性优化](docs/decisions/0007-runtime-reliability.md) - [ADR-0007:运行期可靠性优化](docs/decisions/0007-runtime-reliability.md)
- [ADR-0008:日志合并推送与虚拟滚动](docs/decisions/0008-log-streaming-and-virtualization.md)
## 开发计划 ## 开发计划
+215 -29
View File
@@ -37,6 +37,10 @@ use crate::{
const MAX_LOG_BYTES: u64 = 5 * 1024 * 1024; const MAX_LOG_BYTES: u64 = 5 * 1024 * 1024;
const MAX_LOG_LINE_BYTES: usize = 256 * 1024; const MAX_LOG_LINE_BYTES: usize = 256 * 1024;
const MAX_HTTP_LOG_BODY_BYTES: usize = 64 * 1024; const MAX_HTTP_LOG_BODY_BYTES: usize = 64 * 1024;
/// 合并推送的日志事件上限:攒够这么多字节就立即发一个事件。
const LOG_EVENT_BATCH_BYTES: usize = 16 * 1024;
/// 合并推送的时间窗口:超过这个时间还没攒满也发出去,保证低输出时也有实时感。
const LOG_EVENT_BATCH_INTERVAL: Duration = Duration::from_millis(100);
const TERMINATION_WAIT_SECS: u64 = 5; const TERMINATION_WAIT_SECS: u64 = 5;
const CREATE_NO_WINDOW: u32 = 0x0800_0000; const CREATE_NO_WINDOW: u32 = 0x0800_0000;
const DEFAULT_RETENTION_DAYS: u64 = 30; const DEFAULT_RETENTION_DAYS: u64 = 30;
@@ -175,13 +179,28 @@ struct RunningHandle {
#[derive(Clone)] #[derive(Clone)]
struct LogSink { struct LogSink {
file: Arc<AsyncMutex<tokio::fs::File>>, inner: Arc<LogSinkInner>,
size: Arc<AtomicU64>, }
truncated: Arc<AtomicBool>,
struct LogSinkInner {
file: AsyncMutex<tokio::fs::File>,
size: AtomicU64,
truncated: AtomicBool,
run_id: i64, run_id: i64,
events: broadcast::Sender<ExecutionEvent>, events: broadcast::Sender<ExecutionEvent>,
secrets: Arc<Vec<String>>, secrets: Vec<String>,
encoding: OutputEncoding, encoding: OutputEncoding,
/// 已经写进文件、但还没通过 SSE 推送出去的字节。
pending: Mutex<PendingBatch>,
}
/// 等待合并推送的一段日志。`offset` 是这段内容在日志文件中的起始偏移。
#[derive(Default)]
struct PendingBatch {
offset: u64,
bytes: usize,
opened_at: Option<Instant>,
text: String,
} }
struct ScriptOutcome { struct ScriptOutcome {
@@ -466,7 +485,7 @@ impl TaskExecutor {
let secrets = sensitive_values(&task.action); let secrets = sensitive_values(&task.action);
let sink = match LogSink::create( let sink = match LogSink::create(
&log_path, &log_path,
task, &task.id,
run_id, run_id,
self.inner.events.clone(), self.inner.events.clone(),
secrets, secrets,
@@ -631,7 +650,7 @@ impl TaskExecutor {
let log_path = self.log_path(&task.id, run_id); let log_path = self.log_path(&task.id, run_id);
let sink = match LogSink::create( let sink = match LogSink::create(
&log_path, &log_path,
task, &task.id,
run_id, run_id,
self.inner.events.clone(), self.inner.events.clone(),
sensitive_values(&task.action), sensitive_values(&task.action),
@@ -812,7 +831,7 @@ impl TaskExecutor {
impl LogSink { impl LogSink {
async fn create( async fn create(
path: &Path, path: &Path,
task: &TaskResponse, task_id: &str,
run_id: i64, run_id: i64,
events: broadcast::Sender<ExecutionEvent>, events: broadcast::Sender<ExecutionEvent>,
secrets: Vec<String>, secrets: Vec<String>,
@@ -822,8 +841,7 @@ impl LogSink {
tokio::fs::create_dir_all(parent).await?; tokio::fs::create_dir_all(parent).await?;
} }
let header = format!( let header = format!(
"[system] TaskPilot run {run_id} for task {} started at {}\n", "[system] TaskPilot run {run_id} for task {task_id} started at {}\n",
task.id,
Utc::now().to_rfc3339() Utc::now().to_rfc3339()
); );
tokio::fs::write(path, header.as_bytes()).await?; tokio::fs::write(path, header.as_bytes()).await?;
@@ -834,45 +852,59 @@ impl LogSink {
let mut secrets = secrets; let mut secrets = secrets;
secrets.sort_by_key(|value| std::cmp::Reverse(value.len())); secrets.sort_by_key(|value| std::cmp::Reverse(value.len()));
Ok(Self { let inner = Arc::new(LogSinkInner {
file: Arc::new(AsyncMutex::new(file)), file: AsyncMutex::new(file),
size: Arc::new(AtomicU64::new(header.len() as u64)), size: AtomicU64::new(header.len() as u64),
truncated: Arc::new(AtomicBool::new(false)), truncated: AtomicBool::new(false),
run_id, run_id,
events, events,
secrets: Arc::new(secrets), secrets,
encoding, encoding,
}) pending: Mutex::new(PendingBatch::default()),
});
spawn_pending_flusher(&inner);
Ok(Self { inner })
} }
fn decode_output(&self, bytes: &[u8]) -> String { fn decode_output(&self, bytes: &[u8]) -> String {
self.encoding.decode(bytes) self.inner.encoding.decode(bytes)
} }
async fn append(&self, text: &str) { async fn append(&self, text: &str) {
let redacted = redact_text(text, &self.secrets); let redacted = redact_text(text, &self.inner.secrets);
let bytes = redacted.as_bytes(); let bytes = redacted.as_bytes();
let current = self.size.load(Ordering::Relaxed); let mut file = self.inner.file.lock().await;
let current = self.inner.size.load(Ordering::Relaxed);
if current + bytes.len() as u64 > MAX_LOG_BYTES { if current + bytes.len() as u64 > MAX_LOG_BYTES {
if !self.truncated.swap(true, Ordering::Relaxed) { if !self.inner.truncated.swap(true, Ordering::Relaxed) {
let notice = format!("[system] 日志超过 {MAX_LOG_BYTES} 字节,后续输出已丢弃\n"); let notice = format!("[system] 日志超过 {MAX_LOG_BYTES} 字节,后续输出已丢弃\n");
let mut file = self.file.lock().await;
let _ = file.write_all(notice.as_bytes()).await; let _ = file.write_all(notice.as_bytes()).await;
} }
return; return;
} }
let mut file = self.file.lock().await;
if file.write_all(bytes).await.is_err() { if file.write_all(bytes).await.is_err() {
return; return;
} }
let next = self.size.fetch_add(bytes.len() as u64, Ordering::Relaxed) + bytes.len() as u64; let next = current + bytes.len() as u64;
let _ = self.events.send(ExecutionEvent::Log { self.inner.size.store(next, Ordering::Relaxed);
run_id: self.run_id,
offset: current, // 文件写入是串行的,所以 pending 也在文件锁内更新,保证推送的偏移连续。
next_offset: next, let mut pending = lock_pending(&self.inner.pending);
content: redacted, if pending.bytes == 0 {
}); pending.offset = current;
pending.opened_at = Some(Instant::now());
}
pending.bytes += bytes.len();
pending.text.push_str(&redacted);
let batch_full = pending.bytes >= LOG_EVENT_BATCH_BYTES;
let window_elapsed = pending
.opened_at
.is_some_and(|opened| opened.elapsed() >= LOG_EVENT_BATCH_INTERVAL);
if batch_full || window_elapsed {
emit_pending(&self.inner, &mut pending);
}
} }
async fn system(&self, message: &str) { async fn system(&self, message: &str) {
@@ -884,6 +916,57 @@ impl LogSink {
} }
} }
impl Drop for LogSinkInner {
fn drop(&mut self) {
let mut pending = lock_pending(&self.pending);
emit_pending(self, &mut pending);
}
}
fn lock_pending(pending: &Mutex<PendingBatch>) -> std::sync::MutexGuard<'_, PendingBatch> {
pending
.lock()
.unwrap_or_else(|poisoned| poisoned.into_inner())
}
/// 把待推送的日志合并成一个事件发出去,并清空缓冲区。
fn emit_pending(inner: &LogSinkInner, pending: &mut PendingBatch) {
if pending.bytes == 0 {
return;
}
let batch = std::mem::take(pending);
let _ = inner.events.send(ExecutionEvent::Log {
run_id: inner.run_id,
offset: batch.offset,
next_offset: batch.offset + batch.bytes as u64,
content: batch.text,
});
}
fn flush_pending_if_due(inner: &LogSinkInner) {
let mut pending = lock_pending(&inner.pending);
let due = pending
.opened_at
.is_some_and(|opened| opened.elapsed() >= LOG_EVENT_BATCH_INTERVAL);
if due {
emit_pending(inner, &mut pending);
}
}
/// 低输出量时不会触发"攒满即发",靠这个后台任务按时间窗口把日志推出去。
fn spawn_pending_flusher(inner: &Arc<LogSinkInner>) {
let weak = Arc::downgrade(inner);
tokio::spawn(async move {
loop {
sleep(LOG_EVENT_BATCH_INTERVAL / 2).await;
let Some(inner) = weak.upgrade() else {
break;
};
flush_pending_if_due(&inner);
}
});
}
fn lock_running( fn lock_running(
running: &Mutex<HashMap<String, RunningHandle>>, running: &Mutex<HashMap<String, RunningHandle>>,
) -> std::sync::MutexGuard<'_, HashMap<String, RunningHandle>> { ) -> std::sync::MutexGuard<'_, HashMap<String, RunningHandle>> {
@@ -1273,4 +1356,107 @@ mod tests {
remove_running_if(&running, "task", Some(2)); remove_running_if(&running, "task", Some(2));
assert!(!lock_running(&running).contains_key("task")); assert!(!lock_running(&running).contains_key("task"));
} }
async fn test_sink() -> (
LogSink,
broadcast::Receiver<ExecutionEvent>,
tempfile::TempDir,
PathBuf,
) {
let temp = tempfile::TempDir::new().unwrap();
let path = temp.path().join("run.log");
let (events, receiver) = broadcast::channel(32);
let sink = LogSink::create(
&path,
"task",
7,
events,
Vec::new(),
OutputEncoding::from_env(),
)
.await
.unwrap();
(sink, receiver, temp, path)
}
async fn recv_log(receiver: &mut broadcast::Receiver<ExecutionEvent>) -> (u64, u64, String) {
let event = timeout(Duration::from_secs(2), receiver.recv())
.await
.expect("log event should arrive")
.expect("log channel should stay open");
match event {
ExecutionEvent::Log {
run_id,
offset,
next_offset,
content,
} => {
assert_eq!(run_id, 7);
(offset, next_offset, content)
}
other => panic!("expected log event, got {other:?}"),
}
}
#[tokio::test]
async fn coalesces_rapid_output_into_a_single_log_event() {
let (sink, mut receiver, _temp, path) = test_sink().await;
sink.append("first line\n").await;
sink.append("second line\n").await;
drop(sink);
let (offset, next_offset, content) = recv_log(&mut receiver).await;
assert_eq!(content, "first line\nsecond line\n");
assert_eq!(next_offset - offset, content.len() as u64);
let on_disk = std::fs::read_to_string(&path).unwrap();
assert!(on_disk.ends_with(&content));
}
#[tokio::test]
async fn flushes_the_pending_batch_when_the_sink_is_dropped() {
let (sink, mut receiver, _temp, _path) = test_sink().await;
sink.append("only line\n").await;
drop(sink);
let (offset, next_offset, content) = recv_log(&mut receiver).await;
assert_eq!(content, "only line\n");
assert_eq!(next_offset - offset, content.len() as u64);
}
#[tokio::test]
async fn flushes_a_full_batch_without_waiting_for_the_window() {
let (sink, mut receiver, _temp, _path) = test_sink().await;
sink.append(&"x".repeat(LOG_EVENT_BATCH_BYTES)).await;
let (_offset, _next_offset, content) = recv_log(&mut receiver).await;
assert_eq!(content.len(), LOG_EVENT_BATCH_BYTES);
}
#[tokio::test]
async fn flushes_low_volume_output_within_the_time_window() {
let (sink, mut receiver, _temp, _path) = test_sink().await;
sink.append("quiet output\n").await;
let (_offset, _next_offset, content) = recv_log(&mut receiver).await;
assert_eq!(content, "quiet output\n");
}
#[tokio::test]
async fn batches_keep_contiguous_offsets_across_flushes() {
let (sink, mut receiver, _temp, _path) = test_sink().await;
sink.append(&"x".repeat(LOG_EVENT_BATCH_BYTES)).await;
let (first_offset, first_next, first_content) = recv_log(&mut receiver).await;
sink.append("tail\n").await;
let (second_offset, second_next, second_content) = recv_log(&mut receiver).await;
assert_eq!(first_next - first_offset, first_content.len() as u64);
assert_eq!(second_offset, first_next);
assert_eq!(second_next - second_offset, second_content.len() as u64);
assert_eq!(second_content, "tail\n");
}
} }
@@ -0,0 +1,46 @@
# ADR-0008:日志合并推送与虚拟滚动
- 状态:已接受
- 日期:2026-10-27
## 背景
日志链路原本是"一行一个事件、一行一次渲染":
1. 子进程每输出一行,`LogSink::append` 就向 512 容量的 broadcast 通道发一条 `log` 事件。
输出密集时事件按行堆积,SSE 订阅者很容易 `Lagged`,触发 `lagged` → 前端重新拉取,
在日志最密集的时刻反而放大数据库与文件读取压力。
2. 前端 `RunLogOutput` 把每条事件字符串拼到 `content` state,React 每次都要重渲染整个
`<pre>` 文本;开销随行数线性增长,单次运行 5 MiB 上限意味着最坏情况下数万行 ×
数万次更新,是典型的 O(n²)。
3. 全部日志行都作为 DOM 文本存在,长日志下文本节点、布局与滚动都变得昂贵。
## 决策
1. **后端按窗口合并**:`LogSink` 新增 `PendingBatch`,行仍然**立即写文件**(轮询读到的偏移不受影响),
但 SSE 事件按"**攒满 16 KiB** 或 **满 100 ms**"合并成一条 `log` 事件,携带连续的
`offset..next_offset`。
2. **按时兜底**:每个 sink 起一个 50 ms tick 的后台任务,低输出量时也保证 100 ms 内推送;
任务只持有 `Weak<LogSinkInner>`,sink 释放后自行退出。
3. **析构 flush**:`Drop for LogSinkInner` 同步把残留批次发出去,运行结束时不会丢最后几行。
4. **前端按帧合并**:增量先进行缓冲,`requestAnimationFrame` 内最多触发一次重渲染,
并且在同一次渲染里把新行并入。
5. **虚拟滚动**:`LogViewer` 以固定行高 20 px 换算可见区间,只渲染视口 ± 24 行的窗口;
行内不折行(`white-space: pre`),超长行横向滚动,从而避免逐行测量高度。
6. **跟随策略**:默认贴底跟随;用户上滚超过 24 px 即停止跟随并显示「回到最新」,
点击后回到底部并恢复跟随。
## 原因
- 合并推送把事件数量从"每行一条"降到"每 100 ms 最多一条",SSE 背压与前端更新次数同时下降,
而实时感没有可感知的损失(100 ms 窗口)。
- 文件写入保持即时,轮询接口与下载接口的语义完全不变,合并只影响事件粒度。
- 固定行高让滚动位置与行号一一对应,虚拟滚动不需要动态测量,代码量与出错面都小。
- 行缓冲原地追加 + 一帧一次渲染,避免了复制整个字符串/数组的二次开销。
## 边界
- 合并意味着**事件粒度不再是行**:依赖"一条事件 = 一行"的消费者需要按 `content` 自行切分。
- 不折行是显式取舍:超过面板宽度的行需要横向滚动;折行会让行高不定,虚拟滚动需要测量。
- 单次运行仍受 5 MiB 总上限、单行 256 KiB 上限约束;虚拟滚动只解决渲染,不会保留更多日志。
- `Drop` 里的 flush 是同步发送,若此时没有订阅者,这批事件不会被补发(轮询仍然能读到文件)。
+60
View File
@@ -264,3 +264,63 @@ test('renders without external network access', async ({ page }) => {
fullPage: true, fullPage: true,
}) })
}) })
test('keeps a long run log scrollable without rendering every line', async ({
page,
request,
}) => {
const headers = await authHeaders(request)
await request.delete('/api/v1/tasks/e2e_long_log', { headers })
const response = await request.post('/api/v1/tasks', {
headers,
data: {
...taskPayload,
id: 'e2e_long_log',
name: 'Long log task',
timeout_secs: 60,
action: {
...taskPayload.action,
args: ['for /L %i in (1,1,1500) do @echo line-%i'],
},
},
})
expect(response.status()).toBe(201)
await page.goto('/')
await page
.getByRole('row', { name: /Long log task/ })
.getByRole('button', { name: '运行' })
.click()
const drawer = page.getByRole('dialog', { name: 'Long log task' })
await expect(drawer).toBeVisible()
await expect(drawer.locator('.status-succeeded').first()).toBeVisible({
timeout: 60_000,
})
// 自动跟随到底部,最后几行在视口内。
await expect(drawer.getByText('line-1500')).toBeVisible()
// 虚拟滚动:DOM 中只保留视口附近的少量行,而不是 1500 行全量渲染。
const rendered = await drawer.locator('.log-line').count()
expect(rendered).toBeLessThan(200)
expect(rendered).toBeGreaterThan(5)
await drawer.locator('.log-viewer').evaluate((node) => {
node.scrollTop = node.scrollHeight / 2
})
const jump = drawer.getByRole('button', { name: '回到最新 ↓' })
await expect(jump).toBeVisible()
const logBox = await drawer.locator('.log-viewer').boundingBox()
const jumpBox = await jump.boundingBox()
expect(logBox).not.toBeNull()
expect(jumpBox).not.toBeNull()
if (logBox && jumpBox) {
// 按钮必须落在日志面板可视范围内,而不是被面板裁剪到外面。
expect(jumpBox.y + jumpBox.height).toBeLessThanOrEqual(
logBox.y + logBox.height + 1,
)
}
await page.screenshot({
path: 'test-results/dashboard-log-drawer-long.png',
fullPage: true,
})
})
+94 -7
View File
@@ -1160,6 +1160,19 @@ code {
padding: 22px; padding: 22px;
} }
/* 日志面板吃掉详情区的剩余高度,自己内部滚动,"回到最新"按钮才不会被挤出可视区。 */
.run-detail-panel {
display: flex;
flex-direction: column;
}
.run-detail-panel > .section-heading,
.run-detail-panel > .run-metadata,
.run-detail-panel > .inline-alert,
.run-detail-panel > .empty-inline {
flex: none;
}
.run-history-panel { .run-history-panel {
border-right: 1px solid var(--border); border-right: 1px solid var(--border);
background: var(--surface-muted); background: var(--surface-muted);
@@ -1271,6 +1284,10 @@ code {
} }
.log-viewer-wrap { .log-viewer-wrap {
display: flex;
flex: 1;
flex-direction: column;
min-height: 240px;
overflow: hidden; overflow: hidden;
border: 1px solid rgba(255, 255, 255, 0.08); border: 1px solid rgba(255, 255, 255, 0.08);
border-radius: 14px; border-radius: 14px;
@@ -1279,6 +1296,7 @@ code {
.log-viewer-header { .log-viewer-header {
display: flex; display: flex;
flex: none;
align-items: center; align-items: center;
justify-content: space-between; justify-content: space-between;
border-bottom: 1px solid rgba(255, 255, 255, 0.08); border-bottom: 1px solid rgba(255, 255, 255, 0.08);
@@ -1296,18 +1314,73 @@ code {
color: var(--success); color: var(--success);
} }
.log-viewer-shell {
position: relative;
flex: 1;
min-height: 0;
}
.log-viewer { .log-viewer {
min-height: 340px; height: 100%;
max-height: calc(100vh - 410px); min-height: 0;
overflow: auto; overflow: auto;
margin: 0; margin: 0;
padding: 16px; padding: 0;
color: var(--text-light); color: var(--text-light);
font-family: var(--font-mono); font-family: var(--font-mono);
font-size: 12px; font-size: 12px;
line-height: 1.65; line-height: 20px;
white-space: pre-wrap; overscroll-behavior: contain;
word-break: break-word; }
.log-viewer:focus-visible {
outline: 2px solid var(--primary);
outline-offset: -2px;
}
.log-canvas {
position: relative;
}
.log-window {
position: absolute;
top: 0;
right: 0;
left: 0;
padding: 6px 16px;
will-change: transform;
}
.log-line {
width: max-content;
min-width: 100%;
height: 20px;
white-space: pre;
}
.log-empty {
margin: 0;
padding: 16px;
color: var(--text-tertiary);
}
.log-jump-latest {
position: absolute;
right: 16px;
bottom: 14px;
border: 1px solid rgba(255, 255, 255, 0.16);
border-radius: 999px;
padding: 6px 12px;
background: rgba(16, 18, 27, 0.88);
color: var(--text-light);
font-family: var(--font-mono);
font-size: 11px;
cursor: pointer;
box-shadow: 0 6px 18px rgba(0, 0, 0, 0.35);
}
.log-jump-latest:hover {
background: rgba(30, 34, 50, 0.95);
} }
.log-viewer-wrap .inline-alert { .log-viewer-wrap .inline-alert {
@@ -1443,6 +1516,19 @@ code {
overflow: visible; overflow: visible;
} }
.run-detail-panel {
display: block;
}
.log-viewer-wrap {
flex: none;
min-height: 0;
}
.log-viewer-shell {
flex: none;
}
.run-history-panel { .run-history-panel {
border-right: 0; border-right: 0;
border-bottom: 1px solid var(--border); border-bottom: 1px solid var(--border);
@@ -1453,8 +1539,9 @@ code {
} }
.log-viewer { .log-viewer {
height: auto;
min-height: 260px; min-height: 260px;
max-height: none; max-height: 60vh;
} }
.run-drawer { .run-drawer {
@@ -0,0 +1,33 @@
import { render, screen } from '@testing-library/react'
import { describe, expect, it } from 'vitest'
import { LogViewer } from './LogViewer'
describe('LogViewer', () => {
it('shows a placeholder before the first chunk arrives', () => {
render(<LogViewer lines={[]} version={0} />)
expect(screen.getByText('等待日志输出...')).toBeInTheDocument()
})
it('only renders the visible window of a long log', () => {
const lines = Array.from({ length: 5000 }, (_, index) => `line ${index}`)
const { container } = render(<LogViewer lines={lines} version={1} />)
const rendered = container.querySelectorAll('.log-line')
expect(rendered.length).toBeGreaterThan(0)
expect(rendered.length).toBeLessThan(100)
expect(screen.getByText('line 0')).toBeInTheDocument()
expect(screen.queryByText('line 4999')).toBeNull()
})
it('keeps the row height fixed so offsets stay computable', () => {
const lines = ['alpha', 'beta'] as const
const { container } = render(<LogViewer lines={lines} version={2} />)
const canvas = container.querySelector('.log-canvas')
expect(canvas).not.toBeNull()
expect((canvas as HTMLElement).style.height).toBe('52px')
})
})
+158
View File
@@ -0,0 +1,158 @@
import { useCallback, useEffect, useLayoutEffect, useRef, useState } from 'react'
import { computeLogWindow, isNearBottom } from '../lib/logWindow'
/** 每行固定高度(与 CSS 中的 .log-line 一致),虚拟滚动据此换算位置。 */
export const LOG_ROW_HEIGHT = 20
const LOG_VERTICAL_PADDING = 6
const OVERSCAN_ROWS = 24
const FOLLOW_THRESHOLD_PX = 24
interface LogViewerProps {
/** 已经切好的日志行;内容会被原地追加,靠 version 触发重渲染。 */
lines: readonly string[]
/** 每次内容变化递增,用于在"跟随中"时滚到底部。 */
version: number
emptyLabel?: string
}
/**
* 虚拟滚动的日志面板:只渲染视口附近的行,长日志下 DOM 节点数量恒定。
*
* 行高固定、不自动换行(长行横向滚动),这样滚动位置可以直接由行号换算,
* 不需要逐行测量高度。用户往上翻时停止跟随新日志,并给出"回到最新"按钮。
*/
export function LogViewer({
lines,
version,
emptyLabel = '等待日志输出...',
}: LogViewerProps) {
const viewportRef = useRef<HTMLDivElement>(null)
const frameRef = useRef(0)
const followingRef = useRef(true)
const [following, setFollowing] = useState(true)
const [metrics, setMetrics] = useState({ scrollTop: 0, height: 0 })
useEffect(() => {
const node = viewportRef.current
if (!node) {
return
}
const sync = () => {
setMetrics((previous) =>
previous.height === node.clientHeight
? previous
: { ...previous, height: node.clientHeight },
)
}
sync()
if (typeof ResizeObserver === 'undefined') {
return
}
const observer = new ResizeObserver(sync)
observer.observe(node)
return () => observer.disconnect()
}, [])
useEffect(() => {
return () => {
if (frameRef.current) {
cancelAnimationFrame(frameRef.current)
}
}
}, [])
const handleScroll = useCallback(() => {
if (frameRef.current) {
return
}
frameRef.current = requestAnimationFrame(() => {
frameRef.current = 0
const node = viewportRef.current
if (!node) {
return
}
const atBottom = isNearBottom(
node.scrollTop,
node.clientHeight,
node.scrollHeight,
FOLLOW_THRESHOLD_PX,
)
followingRef.current = atBottom
setFollowing(atBottom)
setMetrics({ scrollTop: node.scrollTop, height: node.clientHeight })
})
}, [])
// 有新内容时贴住底部;同步更新滚动位置,避免窗口还停在旧位置闪一下。
useLayoutEffect(() => {
if (!followingRef.current) {
return
}
const node = viewportRef.current
if (!node) {
return
}
node.scrollTop = node.scrollHeight
setMetrics({ scrollTop: node.scrollTop, height: node.clientHeight })
}, [version])
const jumpToLatest = useCallback(() => {
const node = viewportRef.current
if (!node) {
return
}
node.scrollTop = node.scrollHeight
followingRef.current = true
setFollowing(true)
setMetrics({ scrollTop: node.scrollTop, height: node.clientHeight })
}, [])
const { start, end } = computeLogWindow(
lines.length,
metrics.scrollTop,
metrics.height,
LOG_ROW_HEIGHT,
OVERSCAN_ROWS,
)
const canvasHeight =
lines.length * LOG_ROW_HEIGHT + LOG_VERTICAL_PADDING * 2
return (
<div className="log-viewer-shell">
<div
className="log-viewer"
ref={viewportRef}
onScroll={handleScroll}
role="log"
aria-label="运行日志"
tabIndex={0}
>
{lines.length === 0 ? (
<p className="log-empty">{emptyLabel}</p>
) : (
<div className="log-canvas" style={{ height: canvasHeight }}>
<div
className="log-window"
style={{ transform: `translateY(${start * LOG_ROW_HEIGHT}px)` }}
>
{lines.slice(start, end).map((line, index) => (
<div className="log-line" key={start + index}>
{line}
</div>
))}
</div>
</div>
)}
</div>
{lines.length > 0 && !following ? (
<button
type="button"
className="log-jump-latest"
onClick={jumpToLatest}
>
回到最新 ↓
</button>
) : null}
</div>
)
}
+54 -19
View File
@@ -1,5 +1,5 @@
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query' import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query'
import { useEffect, useRef, useState } from 'react' import { useCallback, useEffect, useRef, useState } from 'react'
import { import {
cancelRun, cancelRun,
getRun, getRun,
@@ -17,7 +17,9 @@ import {
formatDuration, formatDuration,
isTerminalRunStatus, isTerminalRunStatus,
} from '../lib/format' } from '../lib/format'
import { appendLogChunk, createLogBuffer, flushLogTail } from '../lib/logLines'
import { useFocusTrap } from '../lib/useFocusTrap' import { useFocusTrap } from '../lib/useFocusTrap'
import { LogViewer } from './LogViewer'
import { RunStatusBadge } from './StatusBadge' import { RunStatusBadge } from './StatusBadge'
interface RunLogDrawerProps { interface RunLogDrawerProps {
@@ -380,15 +382,48 @@ function RunLogOutput({
sseConnected, sseConnected,
runComplete, runComplete,
}: RunLogOutputProps) { }: RunLogOutputProps) {
const [content, setContent] = useState('') const [version, setVersion] = useState(0)
const [lines, setLines] = useState<readonly string[]>([])
const [pollComplete, setPollComplete] = useState(runComplete) const [pollComplete, setPollComplete] = useState(runComplete)
const [error, setError] = useState<string | null>(null) const [error, setError] = useState<string | null>(null)
const [reloadKey, setReloadKey] = useState(0) const [reloadKey, setReloadKey] = useState(0)
const [reconcileKey, setReconcileKey] = useState(0) const [reconcileKey, setReconcileKey] = useState(0)
const bufferRef = useRef(createLogBuffer())
const offsetRef = useRef(0) const offsetRef = useRef(0)
const outputRef = useRef<HTMLPreElement>(null) const frameRef = useRef(0)
const complete = runComplete || pollComplete const complete = runComplete || pollComplete
// 增量先并进缓冲区,一帧最多触发一次重渲染,避免逐行 setState 的 O(n²)。
const scheduleRender = useCallback(() => {
if (frameRef.current) {
return
}
frameRef.current = requestAnimationFrame(() => {
frameRef.current = 0
setLines(bufferRef.current.lines)
setVersion((value) => value + 1)
})
}, [])
const ingest = useCallback(
(text: string) => {
if (!text) {
return
}
appendLogChunk(bufferRef.current, text)
scheduleRender()
},
[scheduleRender],
)
useEffect(() => {
return () => {
if (frameRef.current) {
cancelAnimationFrame(frameRef.current)
}
}
}, [])
useEffect(() => { useEffect(() => {
let cancelled = false let cancelled = false
const controller = new AbortController() const controller = new AbortController()
@@ -404,12 +439,12 @@ function RunLogOutput({
if (offsetRef.current !== startOffset) { if (offsetRef.current !== startOffset) {
continue continue
} }
if (chunk.content) { ingest(chunk.content)
setContent((current) => current + chunk.content)
}
offsetRef.current = chunk.next_offset offsetRef.current = chunk.next_offset
if (chunk.complete) { if (chunk.complete) {
flushLogTail(bufferRef.current)
setPollComplete(true) setPollComplete(true)
scheduleRender()
return return
} }
await new Promise((resolve) => window.setTimeout(resolve, 600)) await new Promise((resolve) => window.setTimeout(resolve, 600))
@@ -431,7 +466,7 @@ function RunLogOutput({
cancelled = true cancelled = true
controller.abort() controller.abort()
} }
}, [reconcileKey, reloadKey, runId]) }, [ingest, reconcileKey, reloadKey, runId, scheduleRender])
useEffect(() => { useEffect(() => {
const onLog = (event: Event) => { const onLog = (event: Event) => {
@@ -443,9 +478,7 @@ function RunLogOutput({
return return
} }
if (payload.data.offset === offsetRef.current) { if (payload.data.offset === offsetRef.current) {
if (payload.data.content) { ingest(payload.data.content)
setContent((current) => current + payload.data.content)
}
offsetRef.current = payload.data.next_offset offsetRef.current = payload.data.next_offset
} else if (payload.data.offset > offsetRef.current) { } else if (payload.data.offset > offsetRef.current) {
setReconcileKey((value) => value + 1) setReconcileKey((value) => value + 1)
@@ -453,13 +486,15 @@ function RunLogOutput({
} }
logEvents.addEventListener('log', onLog) logEvents.addEventListener('log', onLog)
return () => logEvents.removeEventListener('log', onLog) return () => logEvents.removeEventListener('log', onLog)
}, [logEvents, runId]) }, [ingest, logEvents, runId])
useEffect(() => { useEffect(() => {
if (outputRef.current) { if (!complete) {
outputRef.current.scrollTop = outputRef.current.scrollHeight return
} }
}, [content]) flushLogTail(bufferRef.current)
scheduleRender()
}, [complete, scheduleRender])
return ( return (
<div className="log-viewer-wrap"> <div className="log-viewer-wrap">
@@ -482,10 +517,12 @@ function RunLogOutput({
type="button" type="button"
className="text-button" className="text-button"
onClick={() => { onClick={() => {
setContent('') bufferRef.current = createLogBuffer()
offsetRef.current = 0
setPollComplete(false) setPollComplete(false)
setError(null) setError(null)
offsetRef.current = 0 setLines(bufferRef.current.lines)
setVersion((value) => value + 1)
setReloadKey((value) => value + 1) setReloadKey((value) => value + 1)
}} }}
> >
@@ -493,9 +530,7 @@ function RunLogOutput({
</button> </button>
</div> </div>
) : null} ) : null}
<pre className="log-viewer" ref={outputRef} aria-live="polite"> <LogViewer lines={lines} version={version} />
{content || '等待日志输出...'}
</pre>
</div> </div>
) )
} }
+69
View File
@@ -0,0 +1,69 @@
import { describe, expect, it } from 'vitest'
import {
appendLogChunk,
createLogBuffer,
flushLogTail,
logLineCount,
} from './logLines'
describe('appendLogChunk', () => {
it('splits complete lines and keeps the unfinished tail', () => {
const buffer = createLogBuffer()
appendLogChunk(buffer, 'first\nsecond\npart')
expect(buffer.lines).toEqual(['first', 'second'])
expect(buffer.tail).toBe('part')
expect(logLineCount(buffer)).toBe(3)
})
it('joins chunks that split a line in the middle', () => {
const buffer = createLogBuffer()
appendLogChunk(buffer, 'hel')
appendLogChunk(buffer, 'lo\nnext')
expect(buffer.lines).toEqual(['hello'])
expect(buffer.tail).toBe('next')
})
it('drops the carriage return of Windows line endings', () => {
const buffer = createLogBuffer()
appendLogChunk(buffer, 'one\r\ntwo\r\n')
expect(buffer.lines).toEqual(['one', 'two'])
expect(buffer.tail).toBe('')
})
it('ignores empty chunks', () => {
const buffer = createLogBuffer()
appendLogChunk(buffer, '')
appendLogChunk(buffer, 'x\n')
expect(buffer.lines).toEqual(['x'])
})
})
describe('flushLogTail', () => {
it('promotes an unterminated last line when the run ends', () => {
const buffer = createLogBuffer()
appendLogChunk(buffer, 'line\nno-newline')
flushLogTail(buffer)
expect(buffer.lines).toEqual(['line', 'no-newline'])
expect(buffer.tail).toBe('')
})
it('does nothing when the log ends with a newline', () => {
const buffer = createLogBuffer()
appendLogChunk(buffer, 'line\n')
flushLogTail(buffer)
expect(buffer.lines).toEqual(['line'])
expect(logLineCount(buffer)).toBe(1)
})
})
+44
View File
@@ -0,0 +1,44 @@
/**
* 运行日志的行缓冲。
*
* 日志按块到达(SSE 合并推送或轮询分片),块边界可能落在任意字符之间,
* 所以只把完整行放进 `lines`,剩下的残行留在 `tail` 等下一块拼接。
* 直接改数组而不是每块复制一份,避免长日志上的 O(n²) 拷贝。
*/
export interface LogBuffer {
lines: string[]
tail: string
}
export function createLogBuffer(): LogBuffer {
return { lines: [], tail: '' }
}
/** 把一段增量日志并进缓冲区,只切出完整的行。 */
export function appendLogChunk(buffer: LogBuffer, chunk: string): void {
if (!chunk) {
return
}
const combined = buffer.tail + chunk
const parts = combined.split('\n')
buffer.tail = parts.pop() ?? ''
for (const line of parts) {
// Windows 子进程输出以 \r\n 结尾,去掉行尾的 \r 免得显示成方框。
buffer.lines.push(line.endsWith('\r') ? line.slice(0, -1) : line)
}
}
/** 运行结束:没有以换行符收尾的残行也要作为最后一行显示。 */
export function flushLogTail(buffer: LogBuffer): void {
if (buffer.tail) {
buffer.lines.push(
buffer.tail.endsWith('\r') ? buffer.tail.slice(0, -1) : buffer.tail,
)
buffer.tail = ''
}
}
/** 供界面展示的日志行数(包含尚未换行的最后一行)。 */
export function logLineCount(buffer: LogBuffer): number {
return buffer.lines.length + (buffer.tail ? 1 : 0)
}
+39
View File
@@ -0,0 +1,39 @@
import { describe, expect, it } from 'vitest'
import { computeLogWindow, isNearBottom } from './logWindow'
describe('computeLogWindow', () => {
it('renders a bounded window for a long log', () => {
const window = computeLogWindow(1000, 0, 400, 20, 5)
expect(window.start).toBe(0)
expect(window.end).toBe(25)
})
it('moves the window with the scroll position', () => {
const window = computeLogWindow(1000, 400, 400, 20, 5)
expect(window.start).toBe(15)
expect(window.end).toBe(45)
})
it('clamps the window at the end of the log', () => {
const window = computeLogWindow(1000, 20_000, 400, 20, 5)
expect(window.start).toBe(995)
expect(window.end).toBe(1000)
})
it('returns an empty window for an empty log', () => {
expect(computeLogWindow(0, 0, 400, 20, 5)).toEqual({ start: 0, end: 0 })
})
})
describe('isNearBottom', () => {
it('treats the bottom edge as following', () => {
expect(isNearBottom(1600, 400, 2000, 24)).toBe(true)
})
it('reports scrolling up as not following', () => {
expect(isNearBottom(1200, 400, 2000, 24)).toBe(false)
})
})
+42
View File
@@ -0,0 +1,42 @@
/** 虚拟滚动的可见窗口(左闭右开)。 */
export interface LogWindow {
start: number
end: number
}
/**
* 计算需要渲染的行区间。
*
* `scrollTop` 以像素计,行高固定;`overscan` 是窗口上下额外渲染的行数,
* 用来遮住快速滚动时的空白。
*/
export function computeLogWindow(
total: number,
scrollTop: number,
viewportHeight: number,
rowHeight: number,
overscan: number,
): LogWindow {
if (total <= 0 || rowHeight <= 0) {
return { start: 0, end: 0 }
}
const first = clamp(Math.floor(Math.max(0, scrollTop) / rowHeight), 0, total)
const visible = Math.max(1, Math.ceil(Math.max(0, viewportHeight) / rowHeight))
const start = Math.max(0, first - overscan)
const end = Math.min(total, first + visible + overscan)
return { start, end }
}
/** 视口是否已经贴着底部(留一点像素容差)。 */
export function isNearBottom(
scrollTop: number,
clientHeight: number,
scrollHeight: number,
threshold: number,
): boolean {
return scrollHeight - scrollTop - clientHeight <= threshold
}
function clamp(value: number, min: number, max: number): number {
return Math.min(max, Math.max(min, value))
}