perf(logs): 日志事件按窗口合并推送,抽屉改为虚拟滚动
- 后端 LogSink 把逐行事件按 16 KiB / 100 ms 合并成一条 log 事件,文件写入保持即时;后台 tick 与析构 flush 保证低输出和收尾都不丢尾部 - 前端新增 LogViewer:固定行高虚拟滚动、跟随底部/回到最新,增量按 rAF 合并,长日志 DOM 节点数恒定 - 日志抽屉改为行缓冲(lib/logLines)+ 可见窗口计算(lib/logWindow) - 详情区改 flex 布局,日志面板吃掉剩余高度,回到最新按钮不再被裁到面板外 - 测试:Rust 单元 52→57,Vitest 13→28,Playwright 4→5(1500 行虚拟滚动) - 文档:README 补充合并推送与虚拟滚动,新增 ADR-0008
This commit is contained in:
@@ -5,7 +5,7 @@ TaskPilot 是一个面向 **Windows 单机** 的本地定时任务管理平台
|
|||||||
- 单用户、单实例,只监听 `127.0.0.1`
|
- 单用户、单实例,只监听 `127.0.0.1`
|
||||||
- 三类任务:Python/Shell 脚本、HTTP 请求、二进制进程
|
- 三类任务:Python/Shell 脚本、HTTP 请求、二进制进程
|
||||||
- SQLite 持久化,重启后任务与历史仍在
|
- SQLite 持久化,重启后任务与历史仍在
|
||||||
- 运行日志实时推送(SSE),断线自动回退轮询
|
- 运行日志实时推送(SSE),按窗口合并事件;断线自动回退轮询,长日志用虚拟滚动展示
|
||||||
- 前端零外部依赖:样式与脚本随包自带;界面字体可选,未生成时自动回退系统字体
|
- 前端零外部依赖:样式与脚本随包自带;界面字体可选,未生成时自动回退系统字体
|
||||||
|
|
||||||
> 根目录的 `task_pilot.html` 只是最初的单文件视觉原型,正式入口是 React 应用。
|
> 根目录的 `task_pilot.html` 只是最初的单文件视觉原型,正式入口是 React 应用。
|
||||||
@@ -184,6 +184,11 @@ queued ──► running ──► succeeded / failed / timed_out / cancelled
|
|||||||
- **单行 256 KiB** 上限:超出部分丢弃、继续读到行尾,避免内存无界增长。
|
- **单行 256 KiB** 上限:超出部分丢弃、继续读到行尾,避免内存无界增长。
|
||||||
- 保留期(默认 30 天)与总容量(默认 512 MiB)清理;**跳过正在写入的日志**。
|
- 保留期(默认 30 天)与总容量(默认 512 MiB)清理;**跳过正在写入的日志**。
|
||||||
- 读取按 64 KiB 分块且对齐 UTF-8 边界;`complete` 只在"运行结束且读到文件末尾"时为真。
|
- 读取按 64 KiB 分块且对齐 UTF-8 边界;`complete` 只在"运行结束且读到文件末尾"时为真。
|
||||||
|
- **合并推送**:子进程输出按行写入文件后,先在内存里按 **100 ms / 16 KiB** 窗口合并,
|
||||||
|
每个窗口只发一条 SSE `log` 事件(偏移仍严格连续);低输出时由后台任务按时推,
|
||||||
|
运行结束时在 `LogSink` 析构里兜底 flush,不会丢尾部。
|
||||||
|
- **虚拟滚动**:日志抽屉只渲染视口附近的行(固定行高 20 px),几万行时 DOM 节点数恒定;
|
||||||
|
增量先并进行缓冲、一帧最多重渲染一次。往上翻时停止跟随并显示「回到最新」。长行横向滚动、不折行。
|
||||||
- **输出编码**:默认 `auto` = 逐行先按 UTF-8 解码,失败回退到系统 ANSI 代码页(简体中文为 GBK)。
|
- **输出编码**:默认 `auto` = 逐行先按 UTF-8 解码,失败回退到系统 ANSI 代码页(简体中文为 GBK)。
|
||||||
可用 `TASK_PILOT_LOG_ENCODING=gbk|utf-8|windows-1251` 强制。
|
可用 `TASK_PILOT_LOG_ENCODING=gbk|utf-8|windows-1251` 强制。
|
||||||
- **脱敏**:环境变量/Header 名包含 `SECRET`、`TOKEN`、`PASSWORD`、`PASS`、`API_KEY`、
|
- **脱敏**:环境变量/Header 名包含 `SECRET`、`TOKEN`、`PASSWORD`、`PASS`、`API_KEY`、
|
||||||
@@ -422,6 +427,9 @@ React (Vite) ──fetch──► Axum router ──security::api_guard(Origin +
|
|||||||
| `components/TaskEditorModal.tsx` | 三类动态表单、Cron 预览、虚拟环境状态条 |
|
| `components/TaskEditorModal.tsx` | 三类动态表单、Cron 预览、虚拟环境状态条 |
|
||||||
| `components/DirectoryPickerDialog.tsx` | 页面内目录浏览器 |
|
| `components/DirectoryPickerDialog.tsx` | 页面内目录浏览器 |
|
||||||
| `components/RunLogDrawer.tsx` | 运行历史、日志、取消、下载、SSE |
|
| `components/RunLogDrawer.tsx` | 运行历史、日志、取消、下载、SSE |
|
||||||
|
| `components/LogViewer.tsx` | 日志虚拟滚动、跟随底部/回到最新 |
|
||||||
|
| `lib/logLines.ts` | 增量日志的行缓冲(残行拼接、CRLF 归一) |
|
||||||
|
| `lib/logWindow.ts` | 可见行窗口计算 |
|
||||||
| `api/client.ts` | 类型化客户端、令牌缓存、403 重试 |
|
| `api/client.ts` | 类型化客户端、令牌缓存、403 重试 |
|
||||||
| `lib/taskForm.ts` | Zod 校验与表单 ↔ API 载荷映射 |
|
| `lib/taskForm.ts` | Zod 校验与表单 ↔ API 载荷映射 |
|
||||||
|
|
||||||
@@ -483,10 +491,10 @@ uv run --with fonttools --with brotli tools/build-fonts.py
|
|||||||
|
|
||||||
| 层 | 覆盖 |
|
| 层 | 覆盖 |
|
||||||
| --- | --- |
|
| --- | --- |
|
||||||
| Rust 单元(52) | Cron、虚拟环境解析与探测、执行器(编码/截断/清理/句柄归属)、安全、目录列举 |
|
| Rust 单元(57) | Cron、虚拟环境解析与探测、执行器(编码/截断/清理/句柄归属/日志合并推送)、安全、目录列举 |
|
||||||
| Rust 集成(30) | 任务 CRUD 与校验、运行生命周期、取消与超时、日志分块、调度恢复、统计与调度接口、重启对账 |
|
| Rust 集成(30) | 任务 CRUD 与校验、运行生命周期、取消与超时、日志分块、调度恢复、统计与调度接口、重启对账 |
|
||||||
| 前端 Vitest(13) | 仪表盘渲染、表单载荷、虚拟环境状态、目录浏览器(进入/取消/错误)、下次触发时间 |
|
| 前端 Vitest(28) | 仪表盘渲染、表单载荷、虚拟环境状态、目录浏览器、下次触发时间、日志行切分/窗口计算/虚拟滚动 |
|
||||||
| Playwright(4) | 真实任务执行与日志、编辑/停用/删除、窄屏、**无外部网络渲染**(拦截所有外部请求) |
|
| Playwright(5) | 真实任务执行与日志、编辑/停用/删除、窄屏、**无外部网络渲染**、1500 行日志的虚拟滚动 |
|
||||||
| Python 自检(14 项) | `tools/verify.py` 覆盖 API 全链路 |
|
| Python 自检(14 项) | `tools/verify.py` 覆盖 API 全链路 |
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -530,6 +538,7 @@ uv run --with fonttools --with brotli tools/build-fonts.py
|
|||||||
- [ADR-0005:由后端提供系统文件夹选择](docs/decisions/0005-folder-picker.md)
|
- [ADR-0005:由后端提供系统文件夹选择](docs/decisions/0005-folder-picker.md)
|
||||||
- [ADR-0006:视觉风格对齐主题并自托管字体](docs/decisions/0006-visual-theme-and-offline-fonts.md)
|
- [ADR-0006:视觉风格对齐主题并自托管字体](docs/decisions/0006-visual-theme-and-offline-fonts.md)
|
||||||
- [ADR-0007:运行期可靠性优化](docs/decisions/0007-runtime-reliability.md)
|
- [ADR-0007:运行期可靠性优化](docs/decisions/0007-runtime-reliability.md)
|
||||||
|
- [ADR-0008:日志合并推送与虚拟滚动](docs/decisions/0008-log-streaming-and-virtualization.md)
|
||||||
|
|
||||||
## 开发计划
|
## 开发计划
|
||||||
|
|
||||||
|
|||||||
+215
-29
@@ -37,6 +37,10 @@ use crate::{
|
|||||||
const MAX_LOG_BYTES: u64 = 5 * 1024 * 1024;
|
const MAX_LOG_BYTES: u64 = 5 * 1024 * 1024;
|
||||||
const MAX_LOG_LINE_BYTES: usize = 256 * 1024;
|
const MAX_LOG_LINE_BYTES: usize = 256 * 1024;
|
||||||
const MAX_HTTP_LOG_BODY_BYTES: usize = 64 * 1024;
|
const MAX_HTTP_LOG_BODY_BYTES: usize = 64 * 1024;
|
||||||
|
/// 合并推送的日志事件上限:攒够这么多字节就立即发一个事件。
|
||||||
|
const LOG_EVENT_BATCH_BYTES: usize = 16 * 1024;
|
||||||
|
/// 合并推送的时间窗口:超过这个时间还没攒满也发出去,保证低输出时也有实时感。
|
||||||
|
const LOG_EVENT_BATCH_INTERVAL: Duration = Duration::from_millis(100);
|
||||||
const TERMINATION_WAIT_SECS: u64 = 5;
|
const TERMINATION_WAIT_SECS: u64 = 5;
|
||||||
const CREATE_NO_WINDOW: u32 = 0x0800_0000;
|
const CREATE_NO_WINDOW: u32 = 0x0800_0000;
|
||||||
const DEFAULT_RETENTION_DAYS: u64 = 30;
|
const DEFAULT_RETENTION_DAYS: u64 = 30;
|
||||||
@@ -175,13 +179,28 @@ struct RunningHandle {
|
|||||||
|
|
||||||
#[derive(Clone)]
|
#[derive(Clone)]
|
||||||
struct LogSink {
|
struct LogSink {
|
||||||
file: Arc<AsyncMutex<tokio::fs::File>>,
|
inner: Arc<LogSinkInner>,
|
||||||
size: Arc<AtomicU64>,
|
}
|
||||||
truncated: Arc<AtomicBool>,
|
|
||||||
|
struct LogSinkInner {
|
||||||
|
file: AsyncMutex<tokio::fs::File>,
|
||||||
|
size: AtomicU64,
|
||||||
|
truncated: AtomicBool,
|
||||||
run_id: i64,
|
run_id: i64,
|
||||||
events: broadcast::Sender<ExecutionEvent>,
|
events: broadcast::Sender<ExecutionEvent>,
|
||||||
secrets: Arc<Vec<String>>,
|
secrets: Vec<String>,
|
||||||
encoding: OutputEncoding,
|
encoding: OutputEncoding,
|
||||||
|
/// 已经写进文件、但还没通过 SSE 推送出去的字节。
|
||||||
|
pending: Mutex<PendingBatch>,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 等待合并推送的一段日志。`offset` 是这段内容在日志文件中的起始偏移。
|
||||||
|
#[derive(Default)]
|
||||||
|
struct PendingBatch {
|
||||||
|
offset: u64,
|
||||||
|
bytes: usize,
|
||||||
|
opened_at: Option<Instant>,
|
||||||
|
text: String,
|
||||||
}
|
}
|
||||||
|
|
||||||
struct ScriptOutcome {
|
struct ScriptOutcome {
|
||||||
@@ -466,7 +485,7 @@ impl TaskExecutor {
|
|||||||
let secrets = sensitive_values(&task.action);
|
let secrets = sensitive_values(&task.action);
|
||||||
let sink = match LogSink::create(
|
let sink = match LogSink::create(
|
||||||
&log_path,
|
&log_path,
|
||||||
task,
|
&task.id,
|
||||||
run_id,
|
run_id,
|
||||||
self.inner.events.clone(),
|
self.inner.events.clone(),
|
||||||
secrets,
|
secrets,
|
||||||
@@ -631,7 +650,7 @@ impl TaskExecutor {
|
|||||||
let log_path = self.log_path(&task.id, run_id);
|
let log_path = self.log_path(&task.id, run_id);
|
||||||
let sink = match LogSink::create(
|
let sink = match LogSink::create(
|
||||||
&log_path,
|
&log_path,
|
||||||
task,
|
&task.id,
|
||||||
run_id,
|
run_id,
|
||||||
self.inner.events.clone(),
|
self.inner.events.clone(),
|
||||||
sensitive_values(&task.action),
|
sensitive_values(&task.action),
|
||||||
@@ -812,7 +831,7 @@ impl TaskExecutor {
|
|||||||
impl LogSink {
|
impl LogSink {
|
||||||
async fn create(
|
async fn create(
|
||||||
path: &Path,
|
path: &Path,
|
||||||
task: &TaskResponse,
|
task_id: &str,
|
||||||
run_id: i64,
|
run_id: i64,
|
||||||
events: broadcast::Sender<ExecutionEvent>,
|
events: broadcast::Sender<ExecutionEvent>,
|
||||||
secrets: Vec<String>,
|
secrets: Vec<String>,
|
||||||
@@ -822,8 +841,7 @@ impl LogSink {
|
|||||||
tokio::fs::create_dir_all(parent).await?;
|
tokio::fs::create_dir_all(parent).await?;
|
||||||
}
|
}
|
||||||
let header = format!(
|
let header = format!(
|
||||||
"[system] TaskPilot run {run_id} for task {} started at {}\n",
|
"[system] TaskPilot run {run_id} for task {task_id} started at {}\n",
|
||||||
task.id,
|
|
||||||
Utc::now().to_rfc3339()
|
Utc::now().to_rfc3339()
|
||||||
);
|
);
|
||||||
tokio::fs::write(path, header.as_bytes()).await?;
|
tokio::fs::write(path, header.as_bytes()).await?;
|
||||||
@@ -834,45 +852,59 @@ impl LogSink {
|
|||||||
let mut secrets = secrets;
|
let mut secrets = secrets;
|
||||||
secrets.sort_by_key(|value| std::cmp::Reverse(value.len()));
|
secrets.sort_by_key(|value| std::cmp::Reverse(value.len()));
|
||||||
|
|
||||||
Ok(Self {
|
let inner = Arc::new(LogSinkInner {
|
||||||
file: Arc::new(AsyncMutex::new(file)),
|
file: AsyncMutex::new(file),
|
||||||
size: Arc::new(AtomicU64::new(header.len() as u64)),
|
size: AtomicU64::new(header.len() as u64),
|
||||||
truncated: Arc::new(AtomicBool::new(false)),
|
truncated: AtomicBool::new(false),
|
||||||
run_id,
|
run_id,
|
||||||
events,
|
events,
|
||||||
secrets: Arc::new(secrets),
|
secrets,
|
||||||
encoding,
|
encoding,
|
||||||
})
|
pending: Mutex::new(PendingBatch::default()),
|
||||||
|
});
|
||||||
|
spawn_pending_flusher(&inner);
|
||||||
|
|
||||||
|
Ok(Self { inner })
|
||||||
}
|
}
|
||||||
|
|
||||||
fn decode_output(&self, bytes: &[u8]) -> String {
|
fn decode_output(&self, bytes: &[u8]) -> String {
|
||||||
self.encoding.decode(bytes)
|
self.inner.encoding.decode(bytes)
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn append(&self, text: &str) {
|
async fn append(&self, text: &str) {
|
||||||
let redacted = redact_text(text, &self.secrets);
|
let redacted = redact_text(text, &self.inner.secrets);
|
||||||
let bytes = redacted.as_bytes();
|
let bytes = redacted.as_bytes();
|
||||||
let current = self.size.load(Ordering::Relaxed);
|
let mut file = self.inner.file.lock().await;
|
||||||
|
let current = self.inner.size.load(Ordering::Relaxed);
|
||||||
if current + bytes.len() as u64 > MAX_LOG_BYTES {
|
if current + bytes.len() as u64 > MAX_LOG_BYTES {
|
||||||
if !self.truncated.swap(true, Ordering::Relaxed) {
|
if !self.inner.truncated.swap(true, Ordering::Relaxed) {
|
||||||
let notice = format!("[system] 日志超过 {MAX_LOG_BYTES} 字节,后续输出已丢弃\n");
|
let notice = format!("[system] 日志超过 {MAX_LOG_BYTES} 字节,后续输出已丢弃\n");
|
||||||
let mut file = self.file.lock().await;
|
|
||||||
let _ = file.write_all(notice.as_bytes()).await;
|
let _ = file.write_all(notice.as_bytes()).await;
|
||||||
}
|
}
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
let mut file = self.file.lock().await;
|
|
||||||
if file.write_all(bytes).await.is_err() {
|
if file.write_all(bytes).await.is_err() {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
let next = self.size.fetch_add(bytes.len() as u64, Ordering::Relaxed) + bytes.len() as u64;
|
let next = current + bytes.len() as u64;
|
||||||
let _ = self.events.send(ExecutionEvent::Log {
|
self.inner.size.store(next, Ordering::Relaxed);
|
||||||
run_id: self.run_id,
|
|
||||||
offset: current,
|
// 文件写入是串行的,所以 pending 也在文件锁内更新,保证推送的偏移连续。
|
||||||
next_offset: next,
|
let mut pending = lock_pending(&self.inner.pending);
|
||||||
content: redacted,
|
if pending.bytes == 0 {
|
||||||
});
|
pending.offset = current;
|
||||||
|
pending.opened_at = Some(Instant::now());
|
||||||
|
}
|
||||||
|
pending.bytes += bytes.len();
|
||||||
|
pending.text.push_str(&redacted);
|
||||||
|
|
||||||
|
let batch_full = pending.bytes >= LOG_EVENT_BATCH_BYTES;
|
||||||
|
let window_elapsed = pending
|
||||||
|
.opened_at
|
||||||
|
.is_some_and(|opened| opened.elapsed() >= LOG_EVENT_BATCH_INTERVAL);
|
||||||
|
if batch_full || window_elapsed {
|
||||||
|
emit_pending(&self.inner, &mut pending);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
async fn system(&self, message: &str) {
|
async fn system(&self, message: &str) {
|
||||||
@@ -884,6 +916,57 @@ impl LogSink {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
impl Drop for LogSinkInner {
|
||||||
|
fn drop(&mut self) {
|
||||||
|
let mut pending = lock_pending(&self.pending);
|
||||||
|
emit_pending(self, &mut pending);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn lock_pending(pending: &Mutex<PendingBatch>) -> std::sync::MutexGuard<'_, PendingBatch> {
|
||||||
|
pending
|
||||||
|
.lock()
|
||||||
|
.unwrap_or_else(|poisoned| poisoned.into_inner())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 把待推送的日志合并成一个事件发出去,并清空缓冲区。
|
||||||
|
fn emit_pending(inner: &LogSinkInner, pending: &mut PendingBatch) {
|
||||||
|
if pending.bytes == 0 {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let batch = std::mem::take(pending);
|
||||||
|
let _ = inner.events.send(ExecutionEvent::Log {
|
||||||
|
run_id: inner.run_id,
|
||||||
|
offset: batch.offset,
|
||||||
|
next_offset: batch.offset + batch.bytes as u64,
|
||||||
|
content: batch.text,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
fn flush_pending_if_due(inner: &LogSinkInner) {
|
||||||
|
let mut pending = lock_pending(&inner.pending);
|
||||||
|
let due = pending
|
||||||
|
.opened_at
|
||||||
|
.is_some_and(|opened| opened.elapsed() >= LOG_EVENT_BATCH_INTERVAL);
|
||||||
|
if due {
|
||||||
|
emit_pending(inner, &mut pending);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 低输出量时不会触发"攒满即发",靠这个后台任务按时间窗口把日志推出去。
|
||||||
|
fn spawn_pending_flusher(inner: &Arc<LogSinkInner>) {
|
||||||
|
let weak = Arc::downgrade(inner);
|
||||||
|
tokio::spawn(async move {
|
||||||
|
loop {
|
||||||
|
sleep(LOG_EVENT_BATCH_INTERVAL / 2).await;
|
||||||
|
let Some(inner) = weak.upgrade() else {
|
||||||
|
break;
|
||||||
|
};
|
||||||
|
flush_pending_if_due(&inner);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
fn lock_running(
|
fn lock_running(
|
||||||
running: &Mutex<HashMap<String, RunningHandle>>,
|
running: &Mutex<HashMap<String, RunningHandle>>,
|
||||||
) -> std::sync::MutexGuard<'_, HashMap<String, RunningHandle>> {
|
) -> std::sync::MutexGuard<'_, HashMap<String, RunningHandle>> {
|
||||||
@@ -1273,4 +1356,107 @@ mod tests {
|
|||||||
remove_running_if(&running, "task", Some(2));
|
remove_running_if(&running, "task", Some(2));
|
||||||
assert!(!lock_running(&running).contains_key("task"));
|
assert!(!lock_running(&running).contains_key("task"));
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async fn test_sink() -> (
|
||||||
|
LogSink,
|
||||||
|
broadcast::Receiver<ExecutionEvent>,
|
||||||
|
tempfile::TempDir,
|
||||||
|
PathBuf,
|
||||||
|
) {
|
||||||
|
let temp = tempfile::TempDir::new().unwrap();
|
||||||
|
let path = temp.path().join("run.log");
|
||||||
|
let (events, receiver) = broadcast::channel(32);
|
||||||
|
let sink = LogSink::create(
|
||||||
|
&path,
|
||||||
|
"task",
|
||||||
|
7,
|
||||||
|
events,
|
||||||
|
Vec::new(),
|
||||||
|
OutputEncoding::from_env(),
|
||||||
|
)
|
||||||
|
.await
|
||||||
|
.unwrap();
|
||||||
|
(sink, receiver, temp, path)
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn recv_log(receiver: &mut broadcast::Receiver<ExecutionEvent>) -> (u64, u64, String) {
|
||||||
|
let event = timeout(Duration::from_secs(2), receiver.recv())
|
||||||
|
.await
|
||||||
|
.expect("log event should arrive")
|
||||||
|
.expect("log channel should stay open");
|
||||||
|
match event {
|
||||||
|
ExecutionEvent::Log {
|
||||||
|
run_id,
|
||||||
|
offset,
|
||||||
|
next_offset,
|
||||||
|
content,
|
||||||
|
} => {
|
||||||
|
assert_eq!(run_id, 7);
|
||||||
|
(offset, next_offset, content)
|
||||||
|
}
|
||||||
|
other => panic!("expected log event, got {other:?}"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn coalesces_rapid_output_into_a_single_log_event() {
|
||||||
|
let (sink, mut receiver, _temp, path) = test_sink().await;
|
||||||
|
|
||||||
|
sink.append("first line\n").await;
|
||||||
|
sink.append("second line\n").await;
|
||||||
|
drop(sink);
|
||||||
|
|
||||||
|
let (offset, next_offset, content) = recv_log(&mut receiver).await;
|
||||||
|
assert_eq!(content, "first line\nsecond line\n");
|
||||||
|
assert_eq!(next_offset - offset, content.len() as u64);
|
||||||
|
let on_disk = std::fs::read_to_string(&path).unwrap();
|
||||||
|
assert!(on_disk.ends_with(&content));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn flushes_the_pending_batch_when_the_sink_is_dropped() {
|
||||||
|
let (sink, mut receiver, _temp, _path) = test_sink().await;
|
||||||
|
|
||||||
|
sink.append("only line\n").await;
|
||||||
|
drop(sink);
|
||||||
|
|
||||||
|
let (offset, next_offset, content) = recv_log(&mut receiver).await;
|
||||||
|
assert_eq!(content, "only line\n");
|
||||||
|
assert_eq!(next_offset - offset, content.len() as u64);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn flushes_a_full_batch_without_waiting_for_the_window() {
|
||||||
|
let (sink, mut receiver, _temp, _path) = test_sink().await;
|
||||||
|
|
||||||
|
sink.append(&"x".repeat(LOG_EVENT_BATCH_BYTES)).await;
|
||||||
|
|
||||||
|
let (_offset, _next_offset, content) = recv_log(&mut receiver).await;
|
||||||
|
assert_eq!(content.len(), LOG_EVENT_BATCH_BYTES);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn flushes_low_volume_output_within_the_time_window() {
|
||||||
|
let (sink, mut receiver, _temp, _path) = test_sink().await;
|
||||||
|
|
||||||
|
sink.append("quiet output\n").await;
|
||||||
|
|
||||||
|
let (_offset, _next_offset, content) = recv_log(&mut receiver).await;
|
||||||
|
assert_eq!(content, "quiet output\n");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[tokio::test]
|
||||||
|
async fn batches_keep_contiguous_offsets_across_flushes() {
|
||||||
|
let (sink, mut receiver, _temp, _path) = test_sink().await;
|
||||||
|
|
||||||
|
sink.append(&"x".repeat(LOG_EVENT_BATCH_BYTES)).await;
|
||||||
|
let (first_offset, first_next, first_content) = recv_log(&mut receiver).await;
|
||||||
|
sink.append("tail\n").await;
|
||||||
|
let (second_offset, second_next, second_content) = recv_log(&mut receiver).await;
|
||||||
|
|
||||||
|
assert_eq!(first_next - first_offset, first_content.len() as u64);
|
||||||
|
assert_eq!(second_offset, first_next);
|
||||||
|
assert_eq!(second_next - second_offset, second_content.len() as u64);
|
||||||
|
assert_eq!(second_content, "tail\n");
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,46 @@
|
|||||||
|
# ADR-0008:日志合并推送与虚拟滚动
|
||||||
|
|
||||||
|
- 状态:已接受
|
||||||
|
- 日期:2026-10-27
|
||||||
|
|
||||||
|
## 背景
|
||||||
|
|
||||||
|
日志链路原本是"一行一个事件、一行一次渲染":
|
||||||
|
|
||||||
|
1. 子进程每输出一行,`LogSink::append` 就向 512 容量的 broadcast 通道发一条 `log` 事件。
|
||||||
|
输出密集时事件按行堆积,SSE 订阅者很容易 `Lagged`,触发 `lagged` → 前端重新拉取,
|
||||||
|
在日志最密集的时刻反而放大数据库与文件读取压力。
|
||||||
|
2. 前端 `RunLogOutput` 把每条事件字符串拼到 `content` state,React 每次都要重渲染整个
|
||||||
|
`<pre>` 文本;开销随行数线性增长,单次运行 5 MiB 上限意味着最坏情况下数万行 ×
|
||||||
|
数万次更新,是典型的 O(n²)。
|
||||||
|
3. 全部日志行都作为 DOM 文本存在,长日志下文本节点、布局与滚动都变得昂贵。
|
||||||
|
|
||||||
|
## 决策
|
||||||
|
|
||||||
|
1. **后端按窗口合并**:`LogSink` 新增 `PendingBatch`,行仍然**立即写文件**(轮询读到的偏移不受影响),
|
||||||
|
但 SSE 事件按"**攒满 16 KiB** 或 **满 100 ms**"合并成一条 `log` 事件,携带连续的
|
||||||
|
`offset..next_offset`。
|
||||||
|
2. **按时兜底**:每个 sink 起一个 50 ms tick 的后台任务,低输出量时也保证 100 ms 内推送;
|
||||||
|
任务只持有 `Weak<LogSinkInner>`,sink 释放后自行退出。
|
||||||
|
3. **析构 flush**:`Drop for LogSinkInner` 同步把残留批次发出去,运行结束时不会丢最后几行。
|
||||||
|
4. **前端按帧合并**:增量先进行缓冲,`requestAnimationFrame` 内最多触发一次重渲染,
|
||||||
|
并且在同一次渲染里把新行并入。
|
||||||
|
5. **虚拟滚动**:`LogViewer` 以固定行高 20 px 换算可见区间,只渲染视口 ± 24 行的窗口;
|
||||||
|
行内不折行(`white-space: pre`),超长行横向滚动,从而避免逐行测量高度。
|
||||||
|
6. **跟随策略**:默认贴底跟随;用户上滚超过 24 px 即停止跟随并显示「回到最新」,
|
||||||
|
点击后回到底部并恢复跟随。
|
||||||
|
|
||||||
|
## 原因
|
||||||
|
|
||||||
|
- 合并推送把事件数量从"每行一条"降到"每 100 ms 最多一条",SSE 背压与前端更新次数同时下降,
|
||||||
|
而实时感没有可感知的损失(100 ms 窗口)。
|
||||||
|
- 文件写入保持即时,轮询接口与下载接口的语义完全不变,合并只影响事件粒度。
|
||||||
|
- 固定行高让滚动位置与行号一一对应,虚拟滚动不需要动态测量,代码量与出错面都小。
|
||||||
|
- 行缓冲原地追加 + 一帧一次渲染,避免了复制整个字符串/数组的二次开销。
|
||||||
|
|
||||||
|
## 边界
|
||||||
|
|
||||||
|
- 合并意味着**事件粒度不再是行**:依赖"一条事件 = 一行"的消费者需要按 `content` 自行切分。
|
||||||
|
- 不折行是显式取舍:超过面板宽度的行需要横向滚动;折行会让行高不定,虚拟滚动需要测量。
|
||||||
|
- 单次运行仍受 5 MiB 总上限、单行 256 KiB 上限约束;虚拟滚动只解决渲染,不会保留更多日志。
|
||||||
|
- `Drop` 里的 flush 是同步发送,若此时没有订阅者,这批事件不会被补发(轮询仍然能读到文件)。
|
||||||
@@ -264,3 +264,63 @@ test('renders without external network access', async ({ page }) => {
|
|||||||
fullPage: true,
|
fullPage: true,
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
|
|
||||||
|
test('keeps a long run log scrollable without rendering every line', async ({
|
||||||
|
page,
|
||||||
|
request,
|
||||||
|
}) => {
|
||||||
|
const headers = await authHeaders(request)
|
||||||
|
await request.delete('/api/v1/tasks/e2e_long_log', { headers })
|
||||||
|
const response = await request.post('/api/v1/tasks', {
|
||||||
|
headers,
|
||||||
|
data: {
|
||||||
|
...taskPayload,
|
||||||
|
id: 'e2e_long_log',
|
||||||
|
name: 'Long log task',
|
||||||
|
timeout_secs: 60,
|
||||||
|
action: {
|
||||||
|
...taskPayload.action,
|
||||||
|
args: ['for /L %i in (1,1,1500) do @echo line-%i'],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
})
|
||||||
|
expect(response.status()).toBe(201)
|
||||||
|
|
||||||
|
await page.goto('/')
|
||||||
|
await page
|
||||||
|
.getByRole('row', { name: /Long log task/ })
|
||||||
|
.getByRole('button', { name: '运行' })
|
||||||
|
.click()
|
||||||
|
const drawer = page.getByRole('dialog', { name: 'Long log task' })
|
||||||
|
await expect(drawer).toBeVisible()
|
||||||
|
await expect(drawer.locator('.status-succeeded').first()).toBeVisible({
|
||||||
|
timeout: 60_000,
|
||||||
|
})
|
||||||
|
// 自动跟随到底部,最后几行在视口内。
|
||||||
|
await expect(drawer.getByText('line-1500')).toBeVisible()
|
||||||
|
|
||||||
|
// 虚拟滚动:DOM 中只保留视口附近的少量行,而不是 1500 行全量渲染。
|
||||||
|
const rendered = await drawer.locator('.log-line').count()
|
||||||
|
expect(rendered).toBeLessThan(200)
|
||||||
|
expect(rendered).toBeGreaterThan(5)
|
||||||
|
|
||||||
|
await drawer.locator('.log-viewer').evaluate((node) => {
|
||||||
|
node.scrollTop = node.scrollHeight / 2
|
||||||
|
})
|
||||||
|
const jump = drawer.getByRole('button', { name: '回到最新 ↓' })
|
||||||
|
await expect(jump).toBeVisible()
|
||||||
|
const logBox = await drawer.locator('.log-viewer').boundingBox()
|
||||||
|
const jumpBox = await jump.boundingBox()
|
||||||
|
expect(logBox).not.toBeNull()
|
||||||
|
expect(jumpBox).not.toBeNull()
|
||||||
|
if (logBox && jumpBox) {
|
||||||
|
// 按钮必须落在日志面板可视范围内,而不是被面板裁剪到外面。
|
||||||
|
expect(jumpBox.y + jumpBox.height).toBeLessThanOrEqual(
|
||||||
|
logBox.y + logBox.height + 1,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
await page.screenshot({
|
||||||
|
path: 'test-results/dashboard-log-drawer-long.png',
|
||||||
|
fullPage: true,
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|||||||
+94
-7
@@ -1160,6 +1160,19 @@ code {
|
|||||||
padding: 22px;
|
padding: 22px;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* 日志面板吃掉详情区的剩余高度,自己内部滚动,"回到最新"按钮才不会被挤出可视区。 */
|
||||||
|
.run-detail-panel {
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
}
|
||||||
|
|
||||||
|
.run-detail-panel > .section-heading,
|
||||||
|
.run-detail-panel > .run-metadata,
|
||||||
|
.run-detail-panel > .inline-alert,
|
||||||
|
.run-detail-panel > .empty-inline {
|
||||||
|
flex: none;
|
||||||
|
}
|
||||||
|
|
||||||
.run-history-panel {
|
.run-history-panel {
|
||||||
border-right: 1px solid var(--border);
|
border-right: 1px solid var(--border);
|
||||||
background: var(--surface-muted);
|
background: var(--surface-muted);
|
||||||
@@ -1271,6 +1284,10 @@ code {
|
|||||||
}
|
}
|
||||||
|
|
||||||
.log-viewer-wrap {
|
.log-viewer-wrap {
|
||||||
|
display: flex;
|
||||||
|
flex: 1;
|
||||||
|
flex-direction: column;
|
||||||
|
min-height: 240px;
|
||||||
overflow: hidden;
|
overflow: hidden;
|
||||||
border: 1px solid rgba(255, 255, 255, 0.08);
|
border: 1px solid rgba(255, 255, 255, 0.08);
|
||||||
border-radius: 14px;
|
border-radius: 14px;
|
||||||
@@ -1279,6 +1296,7 @@ code {
|
|||||||
|
|
||||||
.log-viewer-header {
|
.log-viewer-header {
|
||||||
display: flex;
|
display: flex;
|
||||||
|
flex: none;
|
||||||
align-items: center;
|
align-items: center;
|
||||||
justify-content: space-between;
|
justify-content: space-between;
|
||||||
border-bottom: 1px solid rgba(255, 255, 255, 0.08);
|
border-bottom: 1px solid rgba(255, 255, 255, 0.08);
|
||||||
@@ -1296,18 +1314,73 @@ code {
|
|||||||
color: var(--success);
|
color: var(--success);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.log-viewer-shell {
|
||||||
|
position: relative;
|
||||||
|
flex: 1;
|
||||||
|
min-height: 0;
|
||||||
|
}
|
||||||
|
|
||||||
.log-viewer {
|
.log-viewer {
|
||||||
min-height: 340px;
|
height: 100%;
|
||||||
max-height: calc(100vh - 410px);
|
min-height: 0;
|
||||||
overflow: auto;
|
overflow: auto;
|
||||||
margin: 0;
|
margin: 0;
|
||||||
padding: 16px;
|
padding: 0;
|
||||||
color: var(--text-light);
|
color: var(--text-light);
|
||||||
font-family: var(--font-mono);
|
font-family: var(--font-mono);
|
||||||
font-size: 12px;
|
font-size: 12px;
|
||||||
line-height: 1.65;
|
line-height: 20px;
|
||||||
white-space: pre-wrap;
|
overscroll-behavior: contain;
|
||||||
word-break: break-word;
|
}
|
||||||
|
|
||||||
|
.log-viewer:focus-visible {
|
||||||
|
outline: 2px solid var(--primary);
|
||||||
|
outline-offset: -2px;
|
||||||
|
}
|
||||||
|
|
||||||
|
.log-canvas {
|
||||||
|
position: relative;
|
||||||
|
}
|
||||||
|
|
||||||
|
.log-window {
|
||||||
|
position: absolute;
|
||||||
|
top: 0;
|
||||||
|
right: 0;
|
||||||
|
left: 0;
|
||||||
|
padding: 6px 16px;
|
||||||
|
will-change: transform;
|
||||||
|
}
|
||||||
|
|
||||||
|
.log-line {
|
||||||
|
width: max-content;
|
||||||
|
min-width: 100%;
|
||||||
|
height: 20px;
|
||||||
|
white-space: pre;
|
||||||
|
}
|
||||||
|
|
||||||
|
.log-empty {
|
||||||
|
margin: 0;
|
||||||
|
padding: 16px;
|
||||||
|
color: var(--text-tertiary);
|
||||||
|
}
|
||||||
|
|
||||||
|
.log-jump-latest {
|
||||||
|
position: absolute;
|
||||||
|
right: 16px;
|
||||||
|
bottom: 14px;
|
||||||
|
border: 1px solid rgba(255, 255, 255, 0.16);
|
||||||
|
border-radius: 999px;
|
||||||
|
padding: 6px 12px;
|
||||||
|
background: rgba(16, 18, 27, 0.88);
|
||||||
|
color: var(--text-light);
|
||||||
|
font-family: var(--font-mono);
|
||||||
|
font-size: 11px;
|
||||||
|
cursor: pointer;
|
||||||
|
box-shadow: 0 6px 18px rgba(0, 0, 0, 0.35);
|
||||||
|
}
|
||||||
|
|
||||||
|
.log-jump-latest:hover {
|
||||||
|
background: rgba(30, 34, 50, 0.95);
|
||||||
}
|
}
|
||||||
|
|
||||||
.log-viewer-wrap .inline-alert {
|
.log-viewer-wrap .inline-alert {
|
||||||
@@ -1443,6 +1516,19 @@ code {
|
|||||||
overflow: visible;
|
overflow: visible;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
.run-detail-panel {
|
||||||
|
display: block;
|
||||||
|
}
|
||||||
|
|
||||||
|
.log-viewer-wrap {
|
||||||
|
flex: none;
|
||||||
|
min-height: 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
.log-viewer-shell {
|
||||||
|
flex: none;
|
||||||
|
}
|
||||||
|
|
||||||
.run-history-panel {
|
.run-history-panel {
|
||||||
border-right: 0;
|
border-right: 0;
|
||||||
border-bottom: 1px solid var(--border);
|
border-bottom: 1px solid var(--border);
|
||||||
@@ -1453,8 +1539,9 @@ code {
|
|||||||
}
|
}
|
||||||
|
|
||||||
.log-viewer {
|
.log-viewer {
|
||||||
|
height: auto;
|
||||||
min-height: 260px;
|
min-height: 260px;
|
||||||
max-height: none;
|
max-height: 60vh;
|
||||||
}
|
}
|
||||||
|
|
||||||
.run-drawer {
|
.run-drawer {
|
||||||
|
|||||||
@@ -0,0 +1,33 @@
|
|||||||
|
import { render, screen } from '@testing-library/react'
|
||||||
|
import { describe, expect, it } from 'vitest'
|
||||||
|
import { LogViewer } from './LogViewer'
|
||||||
|
|
||||||
|
describe('LogViewer', () => {
|
||||||
|
it('shows a placeholder before the first chunk arrives', () => {
|
||||||
|
render(<LogViewer lines={[]} version={0} />)
|
||||||
|
|
||||||
|
expect(screen.getByText('等待日志输出...')).toBeInTheDocument()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('only renders the visible window of a long log', () => {
|
||||||
|
const lines = Array.from({ length: 5000 }, (_, index) => `line ${index}`)
|
||||||
|
|
||||||
|
const { container } = render(<LogViewer lines={lines} version={1} />)
|
||||||
|
|
||||||
|
const rendered = container.querySelectorAll('.log-line')
|
||||||
|
expect(rendered.length).toBeGreaterThan(0)
|
||||||
|
expect(rendered.length).toBeLessThan(100)
|
||||||
|
expect(screen.getByText('line 0')).toBeInTheDocument()
|
||||||
|
expect(screen.queryByText('line 4999')).toBeNull()
|
||||||
|
})
|
||||||
|
|
||||||
|
it('keeps the row height fixed so offsets stay computable', () => {
|
||||||
|
const lines = ['alpha', 'beta'] as const
|
||||||
|
|
||||||
|
const { container } = render(<LogViewer lines={lines} version={2} />)
|
||||||
|
|
||||||
|
const canvas = container.querySelector('.log-canvas')
|
||||||
|
expect(canvas).not.toBeNull()
|
||||||
|
expect((canvas as HTMLElement).style.height).toBe('52px')
|
||||||
|
})
|
||||||
|
})
|
||||||
@@ -0,0 +1,158 @@
|
|||||||
|
import { useCallback, useEffect, useLayoutEffect, useRef, useState } from 'react'
|
||||||
|
import { computeLogWindow, isNearBottom } from '../lib/logWindow'
|
||||||
|
|
||||||
|
/** 每行固定高度(与 CSS 中的 .log-line 一致),虚拟滚动据此换算位置。 */
|
||||||
|
export const LOG_ROW_HEIGHT = 20
|
||||||
|
const LOG_VERTICAL_PADDING = 6
|
||||||
|
const OVERSCAN_ROWS = 24
|
||||||
|
const FOLLOW_THRESHOLD_PX = 24
|
||||||
|
|
||||||
|
interface LogViewerProps {
|
||||||
|
/** 已经切好的日志行;内容会被原地追加,靠 version 触发重渲染。 */
|
||||||
|
lines: readonly string[]
|
||||||
|
/** 每次内容变化递增,用于在"跟随中"时滚到底部。 */
|
||||||
|
version: number
|
||||||
|
emptyLabel?: string
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 虚拟滚动的日志面板:只渲染视口附近的行,长日志下 DOM 节点数量恒定。
|
||||||
|
*
|
||||||
|
* 行高固定、不自动换行(长行横向滚动),这样滚动位置可以直接由行号换算,
|
||||||
|
* 不需要逐行测量高度。用户往上翻时停止跟随新日志,并给出"回到最新"按钮。
|
||||||
|
*/
|
||||||
|
export function LogViewer({
|
||||||
|
lines,
|
||||||
|
version,
|
||||||
|
emptyLabel = '等待日志输出...',
|
||||||
|
}: LogViewerProps) {
|
||||||
|
const viewportRef = useRef<HTMLDivElement>(null)
|
||||||
|
const frameRef = useRef(0)
|
||||||
|
const followingRef = useRef(true)
|
||||||
|
const [following, setFollowing] = useState(true)
|
||||||
|
const [metrics, setMetrics] = useState({ scrollTop: 0, height: 0 })
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
const node = viewportRef.current
|
||||||
|
if (!node) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
const sync = () => {
|
||||||
|
setMetrics((previous) =>
|
||||||
|
previous.height === node.clientHeight
|
||||||
|
? previous
|
||||||
|
: { ...previous, height: node.clientHeight },
|
||||||
|
)
|
||||||
|
}
|
||||||
|
sync()
|
||||||
|
if (typeof ResizeObserver === 'undefined') {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
const observer = new ResizeObserver(sync)
|
||||||
|
observer.observe(node)
|
||||||
|
return () => observer.disconnect()
|
||||||
|
}, [])
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
return () => {
|
||||||
|
if (frameRef.current) {
|
||||||
|
cancelAnimationFrame(frameRef.current)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}, [])
|
||||||
|
|
||||||
|
const handleScroll = useCallback(() => {
|
||||||
|
if (frameRef.current) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
frameRef.current = requestAnimationFrame(() => {
|
||||||
|
frameRef.current = 0
|
||||||
|
const node = viewportRef.current
|
||||||
|
if (!node) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
const atBottom = isNearBottom(
|
||||||
|
node.scrollTop,
|
||||||
|
node.clientHeight,
|
||||||
|
node.scrollHeight,
|
||||||
|
FOLLOW_THRESHOLD_PX,
|
||||||
|
)
|
||||||
|
followingRef.current = atBottom
|
||||||
|
setFollowing(atBottom)
|
||||||
|
setMetrics({ scrollTop: node.scrollTop, height: node.clientHeight })
|
||||||
|
})
|
||||||
|
}, [])
|
||||||
|
|
||||||
|
// 有新内容时贴住底部;同步更新滚动位置,避免窗口还停在旧位置闪一下。
|
||||||
|
useLayoutEffect(() => {
|
||||||
|
if (!followingRef.current) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
const node = viewportRef.current
|
||||||
|
if (!node) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
node.scrollTop = node.scrollHeight
|
||||||
|
setMetrics({ scrollTop: node.scrollTop, height: node.clientHeight })
|
||||||
|
}, [version])
|
||||||
|
|
||||||
|
const jumpToLatest = useCallback(() => {
|
||||||
|
const node = viewportRef.current
|
||||||
|
if (!node) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
node.scrollTop = node.scrollHeight
|
||||||
|
followingRef.current = true
|
||||||
|
setFollowing(true)
|
||||||
|
setMetrics({ scrollTop: node.scrollTop, height: node.clientHeight })
|
||||||
|
}, [])
|
||||||
|
|
||||||
|
const { start, end } = computeLogWindow(
|
||||||
|
lines.length,
|
||||||
|
metrics.scrollTop,
|
||||||
|
metrics.height,
|
||||||
|
LOG_ROW_HEIGHT,
|
||||||
|
OVERSCAN_ROWS,
|
||||||
|
)
|
||||||
|
const canvasHeight =
|
||||||
|
lines.length * LOG_ROW_HEIGHT + LOG_VERTICAL_PADDING * 2
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="log-viewer-shell">
|
||||||
|
<div
|
||||||
|
className="log-viewer"
|
||||||
|
ref={viewportRef}
|
||||||
|
onScroll={handleScroll}
|
||||||
|
role="log"
|
||||||
|
aria-label="运行日志"
|
||||||
|
tabIndex={0}
|
||||||
|
>
|
||||||
|
{lines.length === 0 ? (
|
||||||
|
<p className="log-empty">{emptyLabel}</p>
|
||||||
|
) : (
|
||||||
|
<div className="log-canvas" style={{ height: canvasHeight }}>
|
||||||
|
<div
|
||||||
|
className="log-window"
|
||||||
|
style={{ transform: `translateY(${start * LOG_ROW_HEIGHT}px)` }}
|
||||||
|
>
|
||||||
|
{lines.slice(start, end).map((line, index) => (
|
||||||
|
<div className="log-line" key={start + index}>
|
||||||
|
{line}
|
||||||
|
</div>
|
||||||
|
))}
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</div>
|
||||||
|
{lines.length > 0 && !following ? (
|
||||||
|
<button
|
||||||
|
type="button"
|
||||||
|
className="log-jump-latest"
|
||||||
|
onClick={jumpToLatest}
|
||||||
|
>
|
||||||
|
回到最新 ↓
|
||||||
|
</button>
|
||||||
|
) : null}
|
||||||
|
</div>
|
||||||
|
)
|
||||||
|
}
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query'
|
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query'
|
||||||
import { useEffect, useRef, useState } from 'react'
|
import { useCallback, useEffect, useRef, useState } from 'react'
|
||||||
import {
|
import {
|
||||||
cancelRun,
|
cancelRun,
|
||||||
getRun,
|
getRun,
|
||||||
@@ -17,7 +17,9 @@ import {
|
|||||||
formatDuration,
|
formatDuration,
|
||||||
isTerminalRunStatus,
|
isTerminalRunStatus,
|
||||||
} from '../lib/format'
|
} from '../lib/format'
|
||||||
|
import { appendLogChunk, createLogBuffer, flushLogTail } from '../lib/logLines'
|
||||||
import { useFocusTrap } from '../lib/useFocusTrap'
|
import { useFocusTrap } from '../lib/useFocusTrap'
|
||||||
|
import { LogViewer } from './LogViewer'
|
||||||
import { RunStatusBadge } from './StatusBadge'
|
import { RunStatusBadge } from './StatusBadge'
|
||||||
|
|
||||||
interface RunLogDrawerProps {
|
interface RunLogDrawerProps {
|
||||||
@@ -380,15 +382,48 @@ function RunLogOutput({
|
|||||||
sseConnected,
|
sseConnected,
|
||||||
runComplete,
|
runComplete,
|
||||||
}: RunLogOutputProps) {
|
}: RunLogOutputProps) {
|
||||||
const [content, setContent] = useState('')
|
const [version, setVersion] = useState(0)
|
||||||
|
const [lines, setLines] = useState<readonly string[]>([])
|
||||||
const [pollComplete, setPollComplete] = useState(runComplete)
|
const [pollComplete, setPollComplete] = useState(runComplete)
|
||||||
const [error, setError] = useState<string | null>(null)
|
const [error, setError] = useState<string | null>(null)
|
||||||
const [reloadKey, setReloadKey] = useState(0)
|
const [reloadKey, setReloadKey] = useState(0)
|
||||||
const [reconcileKey, setReconcileKey] = useState(0)
|
const [reconcileKey, setReconcileKey] = useState(0)
|
||||||
|
const bufferRef = useRef(createLogBuffer())
|
||||||
const offsetRef = useRef(0)
|
const offsetRef = useRef(0)
|
||||||
const outputRef = useRef<HTMLPreElement>(null)
|
const frameRef = useRef(0)
|
||||||
const complete = runComplete || pollComplete
|
const complete = runComplete || pollComplete
|
||||||
|
|
||||||
|
// 增量先并进缓冲区,一帧最多触发一次重渲染,避免逐行 setState 的 O(n²)。
|
||||||
|
const scheduleRender = useCallback(() => {
|
||||||
|
if (frameRef.current) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
frameRef.current = requestAnimationFrame(() => {
|
||||||
|
frameRef.current = 0
|
||||||
|
setLines(bufferRef.current.lines)
|
||||||
|
setVersion((value) => value + 1)
|
||||||
|
})
|
||||||
|
}, [])
|
||||||
|
|
||||||
|
const ingest = useCallback(
|
||||||
|
(text: string) => {
|
||||||
|
if (!text) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
appendLogChunk(bufferRef.current, text)
|
||||||
|
scheduleRender()
|
||||||
|
},
|
||||||
|
[scheduleRender],
|
||||||
|
)
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
return () => {
|
||||||
|
if (frameRef.current) {
|
||||||
|
cancelAnimationFrame(frameRef.current)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}, [])
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
let cancelled = false
|
let cancelled = false
|
||||||
const controller = new AbortController()
|
const controller = new AbortController()
|
||||||
@@ -404,12 +439,12 @@ function RunLogOutput({
|
|||||||
if (offsetRef.current !== startOffset) {
|
if (offsetRef.current !== startOffset) {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if (chunk.content) {
|
ingest(chunk.content)
|
||||||
setContent((current) => current + chunk.content)
|
|
||||||
}
|
|
||||||
offsetRef.current = chunk.next_offset
|
offsetRef.current = chunk.next_offset
|
||||||
if (chunk.complete) {
|
if (chunk.complete) {
|
||||||
|
flushLogTail(bufferRef.current)
|
||||||
setPollComplete(true)
|
setPollComplete(true)
|
||||||
|
scheduleRender()
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
await new Promise((resolve) => window.setTimeout(resolve, 600))
|
await new Promise((resolve) => window.setTimeout(resolve, 600))
|
||||||
@@ -431,7 +466,7 @@ function RunLogOutput({
|
|||||||
cancelled = true
|
cancelled = true
|
||||||
controller.abort()
|
controller.abort()
|
||||||
}
|
}
|
||||||
}, [reconcileKey, reloadKey, runId])
|
}, [ingest, reconcileKey, reloadKey, runId, scheduleRender])
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
const onLog = (event: Event) => {
|
const onLog = (event: Event) => {
|
||||||
@@ -443,9 +478,7 @@ function RunLogOutput({
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
if (payload.data.offset === offsetRef.current) {
|
if (payload.data.offset === offsetRef.current) {
|
||||||
if (payload.data.content) {
|
ingest(payload.data.content)
|
||||||
setContent((current) => current + payload.data.content)
|
|
||||||
}
|
|
||||||
offsetRef.current = payload.data.next_offset
|
offsetRef.current = payload.data.next_offset
|
||||||
} else if (payload.data.offset > offsetRef.current) {
|
} else if (payload.data.offset > offsetRef.current) {
|
||||||
setReconcileKey((value) => value + 1)
|
setReconcileKey((value) => value + 1)
|
||||||
@@ -453,13 +486,15 @@ function RunLogOutput({
|
|||||||
}
|
}
|
||||||
logEvents.addEventListener('log', onLog)
|
logEvents.addEventListener('log', onLog)
|
||||||
return () => logEvents.removeEventListener('log', onLog)
|
return () => logEvents.removeEventListener('log', onLog)
|
||||||
}, [logEvents, runId])
|
}, [ingest, logEvents, runId])
|
||||||
|
|
||||||
useEffect(() => {
|
useEffect(() => {
|
||||||
if (outputRef.current) {
|
if (!complete) {
|
||||||
outputRef.current.scrollTop = outputRef.current.scrollHeight
|
return
|
||||||
}
|
}
|
||||||
}, [content])
|
flushLogTail(bufferRef.current)
|
||||||
|
scheduleRender()
|
||||||
|
}, [complete, scheduleRender])
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<div className="log-viewer-wrap">
|
<div className="log-viewer-wrap">
|
||||||
@@ -482,10 +517,12 @@ function RunLogOutput({
|
|||||||
type="button"
|
type="button"
|
||||||
className="text-button"
|
className="text-button"
|
||||||
onClick={() => {
|
onClick={() => {
|
||||||
setContent('')
|
bufferRef.current = createLogBuffer()
|
||||||
|
offsetRef.current = 0
|
||||||
setPollComplete(false)
|
setPollComplete(false)
|
||||||
setError(null)
|
setError(null)
|
||||||
offsetRef.current = 0
|
setLines(bufferRef.current.lines)
|
||||||
|
setVersion((value) => value + 1)
|
||||||
setReloadKey((value) => value + 1)
|
setReloadKey((value) => value + 1)
|
||||||
}}
|
}}
|
||||||
>
|
>
|
||||||
@@ -493,9 +530,7 @@ function RunLogOutput({
|
|||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
) : null}
|
) : null}
|
||||||
<pre className="log-viewer" ref={outputRef} aria-live="polite">
|
<LogViewer lines={lines} version={version} />
|
||||||
{content || '等待日志输出...'}
|
|
||||||
</pre>
|
|
||||||
</div>
|
</div>
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,69 @@
|
|||||||
|
import { describe, expect, it } from 'vitest'
|
||||||
|
import {
|
||||||
|
appendLogChunk,
|
||||||
|
createLogBuffer,
|
||||||
|
flushLogTail,
|
||||||
|
logLineCount,
|
||||||
|
} from './logLines'
|
||||||
|
|
||||||
|
describe('appendLogChunk', () => {
|
||||||
|
it('splits complete lines and keeps the unfinished tail', () => {
|
||||||
|
const buffer = createLogBuffer()
|
||||||
|
|
||||||
|
appendLogChunk(buffer, 'first\nsecond\npart')
|
||||||
|
|
||||||
|
expect(buffer.lines).toEqual(['first', 'second'])
|
||||||
|
expect(buffer.tail).toBe('part')
|
||||||
|
expect(logLineCount(buffer)).toBe(3)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('joins chunks that split a line in the middle', () => {
|
||||||
|
const buffer = createLogBuffer()
|
||||||
|
|
||||||
|
appendLogChunk(buffer, 'hel')
|
||||||
|
appendLogChunk(buffer, 'lo\nnext')
|
||||||
|
|
||||||
|
expect(buffer.lines).toEqual(['hello'])
|
||||||
|
expect(buffer.tail).toBe('next')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('drops the carriage return of Windows line endings', () => {
|
||||||
|
const buffer = createLogBuffer()
|
||||||
|
|
||||||
|
appendLogChunk(buffer, 'one\r\ntwo\r\n')
|
||||||
|
|
||||||
|
expect(buffer.lines).toEqual(['one', 'two'])
|
||||||
|
expect(buffer.tail).toBe('')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('ignores empty chunks', () => {
|
||||||
|
const buffer = createLogBuffer()
|
||||||
|
|
||||||
|
appendLogChunk(buffer, '')
|
||||||
|
appendLogChunk(buffer, 'x\n')
|
||||||
|
|
||||||
|
expect(buffer.lines).toEqual(['x'])
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('flushLogTail', () => {
|
||||||
|
it('promotes an unterminated last line when the run ends', () => {
|
||||||
|
const buffer = createLogBuffer()
|
||||||
|
appendLogChunk(buffer, 'line\nno-newline')
|
||||||
|
|
||||||
|
flushLogTail(buffer)
|
||||||
|
|
||||||
|
expect(buffer.lines).toEqual(['line', 'no-newline'])
|
||||||
|
expect(buffer.tail).toBe('')
|
||||||
|
})
|
||||||
|
|
||||||
|
it('does nothing when the log ends with a newline', () => {
|
||||||
|
const buffer = createLogBuffer()
|
||||||
|
appendLogChunk(buffer, 'line\n')
|
||||||
|
|
||||||
|
flushLogTail(buffer)
|
||||||
|
|
||||||
|
expect(buffer.lines).toEqual(['line'])
|
||||||
|
expect(logLineCount(buffer)).toBe(1)
|
||||||
|
})
|
||||||
|
})
|
||||||
@@ -0,0 +1,44 @@
|
|||||||
|
/**
|
||||||
|
* 运行日志的行缓冲。
|
||||||
|
*
|
||||||
|
* 日志按块到达(SSE 合并推送或轮询分片),块边界可能落在任意字符之间,
|
||||||
|
* 所以只把完整行放进 `lines`,剩下的残行留在 `tail` 等下一块拼接。
|
||||||
|
* 直接改数组而不是每块复制一份,避免长日志上的 O(n²) 拷贝。
|
||||||
|
*/
|
||||||
|
export interface LogBuffer {
|
||||||
|
lines: string[]
|
||||||
|
tail: string
|
||||||
|
}
|
||||||
|
|
||||||
|
export function createLogBuffer(): LogBuffer {
|
||||||
|
return { lines: [], tail: '' }
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 把一段增量日志并进缓冲区,只切出完整的行。 */
|
||||||
|
export function appendLogChunk(buffer: LogBuffer, chunk: string): void {
|
||||||
|
if (!chunk) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
const combined = buffer.tail + chunk
|
||||||
|
const parts = combined.split('\n')
|
||||||
|
buffer.tail = parts.pop() ?? ''
|
||||||
|
for (const line of parts) {
|
||||||
|
// Windows 子进程输出以 \r\n 结尾,去掉行尾的 \r 免得显示成方框。
|
||||||
|
buffer.lines.push(line.endsWith('\r') ? line.slice(0, -1) : line)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 运行结束:没有以换行符收尾的残行也要作为最后一行显示。 */
|
||||||
|
export function flushLogTail(buffer: LogBuffer): void {
|
||||||
|
if (buffer.tail) {
|
||||||
|
buffer.lines.push(
|
||||||
|
buffer.tail.endsWith('\r') ? buffer.tail.slice(0, -1) : buffer.tail,
|
||||||
|
)
|
||||||
|
buffer.tail = ''
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 供界面展示的日志行数(包含尚未换行的最后一行)。 */
|
||||||
|
export function logLineCount(buffer: LogBuffer): number {
|
||||||
|
return buffer.lines.length + (buffer.tail ? 1 : 0)
|
||||||
|
}
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
import { describe, expect, it } from 'vitest'
|
||||||
|
import { computeLogWindow, isNearBottom } from './logWindow'
|
||||||
|
|
||||||
|
describe('computeLogWindow', () => {
|
||||||
|
it('renders a bounded window for a long log', () => {
|
||||||
|
const window = computeLogWindow(1000, 0, 400, 20, 5)
|
||||||
|
|
||||||
|
expect(window.start).toBe(0)
|
||||||
|
expect(window.end).toBe(25)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('moves the window with the scroll position', () => {
|
||||||
|
const window = computeLogWindow(1000, 400, 400, 20, 5)
|
||||||
|
|
||||||
|
expect(window.start).toBe(15)
|
||||||
|
expect(window.end).toBe(45)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('clamps the window at the end of the log', () => {
|
||||||
|
const window = computeLogWindow(1000, 20_000, 400, 20, 5)
|
||||||
|
|
||||||
|
expect(window.start).toBe(995)
|
||||||
|
expect(window.end).toBe(1000)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('returns an empty window for an empty log', () => {
|
||||||
|
expect(computeLogWindow(0, 0, 400, 20, 5)).toEqual({ start: 0, end: 0 })
|
||||||
|
})
|
||||||
|
})
|
||||||
|
|
||||||
|
describe('isNearBottom', () => {
|
||||||
|
it('treats the bottom edge as following', () => {
|
||||||
|
expect(isNearBottom(1600, 400, 2000, 24)).toBe(true)
|
||||||
|
})
|
||||||
|
|
||||||
|
it('reports scrolling up as not following', () => {
|
||||||
|
expect(isNearBottom(1200, 400, 2000, 24)).toBe(false)
|
||||||
|
})
|
||||||
|
})
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
/** 虚拟滚动的可见窗口(左闭右开)。 */
|
||||||
|
export interface LogWindow {
|
||||||
|
start: number
|
||||||
|
end: number
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 计算需要渲染的行区间。
|
||||||
|
*
|
||||||
|
* `scrollTop` 以像素计,行高固定;`overscan` 是窗口上下额外渲染的行数,
|
||||||
|
* 用来遮住快速滚动时的空白。
|
||||||
|
*/
|
||||||
|
export function computeLogWindow(
|
||||||
|
total: number,
|
||||||
|
scrollTop: number,
|
||||||
|
viewportHeight: number,
|
||||||
|
rowHeight: number,
|
||||||
|
overscan: number,
|
||||||
|
): LogWindow {
|
||||||
|
if (total <= 0 || rowHeight <= 0) {
|
||||||
|
return { start: 0, end: 0 }
|
||||||
|
}
|
||||||
|
const first = clamp(Math.floor(Math.max(0, scrollTop) / rowHeight), 0, total)
|
||||||
|
const visible = Math.max(1, Math.ceil(Math.max(0, viewportHeight) / rowHeight))
|
||||||
|
const start = Math.max(0, first - overscan)
|
||||||
|
const end = Math.min(total, first + visible + overscan)
|
||||||
|
return { start, end }
|
||||||
|
}
|
||||||
|
|
||||||
|
/** 视口是否已经贴着底部(留一点像素容差)。 */
|
||||||
|
export function isNearBottom(
|
||||||
|
scrollTop: number,
|
||||||
|
clientHeight: number,
|
||||||
|
scrollHeight: number,
|
||||||
|
threshold: number,
|
||||||
|
): boolean {
|
||||||
|
return scrollHeight - scrollTop - clientHeight <= threshold
|
||||||
|
}
|
||||||
|
|
||||||
|
function clamp(value: number, min: number, max: number): number {
|
||||||
|
return Math.min(max, Math.max(min, value))
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user