Compare commits
39
Commits
83dcc1c3ee
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
70654e45f7 | ||
|
|
fc17ae10d0 | ||
|
|
2f5a72a6ab | ||
|
|
7a194dd4a7 | ||
|
|
a41a5440c6 | ||
|
|
864b21f894 | ||
|
|
b396298819 | ||
|
|
53a50ecab9 | ||
|
|
425229f412 | ||
|
|
7e063eb3d5 | ||
|
|
406f180cab | ||
|
|
86323f1691 | ||
|
|
0f5b84b919 | ||
|
|
0a19b29fd3 | ||
|
|
1bf59e7175 | ||
|
|
cafa038110 | ||
|
|
527bc25c11 | ||
|
|
9a7404856e | ||
|
|
1f2f2a2bb1 | ||
|
|
0653cf0d8d | ||
|
|
95ba43a1c7 | ||
|
|
5365add657 | ||
|
|
573082df26 | ||
|
|
5a87c0655a | ||
|
|
0b47905a1c | ||
|
|
0d522dd782 | ||
|
|
d60cbcdb3c | ||
|
|
979a5814c1 | ||
|
|
c2732fb3cd | ||
|
|
7fe1e5e0b3 | ||
|
|
3c4bc6c8ab | ||
|
|
4a59ca84f6 | ||
|
|
faf6dd2748 | ||
|
|
b1cf4126fd | ||
|
|
e431862793 | ||
|
|
c65b9986ca | ||
|
|
db0bbbc15d | ||
|
|
21ad209747 | ||
|
|
c143b56232 |
+9
-2
@@ -25,7 +25,7 @@
|
||||
|---|---|
|
||||
| R9(真查看器、训练好的模型、30 道贴柱题) | 稳态 `planWarmP99` < 45 ms、运动中饥饿 < 1%、认证超时占比 < 5%;冷启动 P99、刹停率、`tickGapMax` 只报 |
|
||||
| R9 不过 | 几何查询(特征、投影、认证)搬进 Worker,主线程只执行 |
|
||||
| APG | batch 48、steps 8000(总任务量同原 128×3000),进程显存硬帽 `VRAM_FRAC=0.78` |
|
||||
| APG | batch 48、steps 8000(总任务量同原 128×3000),本进程显存上限浮动(整卡 − 已用 − 1 GB 余量,见 `ml/common/vram.py`) |
|
||||
| 离线 ort | 只用入库本地副本,缺失即报错,不退回 CDN |
|
||||
| 训练余量 3 mm | `stats.json` 间隙 p10 ≥ 3 mm 保持,否则降到 p10 |
|
||||
| `R2_STRICT` | `pool_check.independent=true` 且截距 < 0.10 时打开 |
|
||||
@@ -40,12 +40,19 @@
|
||||
4. APG 收敛与机时(batch 48 下首测)→ APG 日志。
|
||||
5. 真模型 R9 → `r9_browser.json`。
|
||||
|
||||
## 三′、首次正式预训练失败(2026-10-18,21850 步)与修正
|
||||
|
||||
现象:闭环到达 0–0.2%、穿柱 77–97%;MF 残差 2 万步停在 7–8、17k–19k 尖峰到 96;梯度 100% 被裁剪。用 last.pt 实测定位:
|
||||
1. **多峰头塌成单峰(主因)**:同一状态 8 个候选间标准差 0.030、动作幅度 0.151。FK / 平滑损失加在 `one_step(随机噪声)` 上去对齐这一条老师 Y,最优解是条件均值 E[Y|X],左绕 / 右绕取平均正好穿柱。→ 改加在 r = t、t ≤ 0.3 的配对去噪估计 x̂ = z − t·u 上。
|
||||
2. **MeanFlow 目标失控**:∂u/∂t 均方根 6–12,而 v 约 1.1,目标被自激的 JVP 项主导。时间嵌入 sin(1000·t) 太陡,JVP 又在 bf16 下算。→ 最高角频率 50;MF 分支 fp32;前 5000 步 p_neq = 0。
|
||||
3. **闭环分布偏移被低估(推断)**:增广只有 1°。→ 增广 40% / 3° / 12 步收回;新增 DAgger 阶段(策略走到的状态请老师重规划,2 轮);开训前加老师轨迹回放自检;每次评估加开环误差与候选多样性。
|
||||
|
||||
## 四、正式训练(三段,每段跑完回传)
|
||||
|
||||
| 段 | 命令 | 耗时 | 回传 |
|
||||
|---|---|---|---|
|
||||
| 数据 | `FROM=kin TO=prep bash ml/pipeline.sh all` | 数小时 | `ml/data/stats.json`、`ml/data/r2.json`、`ml/runs/r4.json` |
|
||||
| 训练 | `FROM=train TO=m5 bash ml/pipeline.sh all` | 约 3–6 小时 | `ml/runs/m5_gate_*.json` |
|
||||
| 训练(含 DAgger) | `FRESH_FROM=prep FROM=prep TO=m5 bash ml/pipeline.sh all`(增广改了,样本要重打包;gen 不重跑) | 约 9–11 小时(未实测:prep 0.5 + 预训练约 5(fp32 变慢)+ DAgger 约 3.5 + m5 0.3) | `ml/runs/replay_teacher.json`、`ml/runs/base/log.jsonl`、`ml/runs/m5_gate_*.json` |
|
||||
| 后训练与评测 | `FROM=apg TO=accept bash ml/pipeline.sh all` | 约 10–20 小时 | `ml/runs/acceptance.json`、`policy_bench.json`、`r9_browser.json` |
|
||||
|
||||
中途出错:把出错信息发我,修完用 `FROM=<出错阶段>` 接着跑,前面的不重做。
|
||||
|
||||
+245
@@ -0,0 +1,245 @@
|
||||
# 本机执行清单(给本地 AI 逐条执行)
|
||||
|
||||
本文件是 Hark 对本机计算的唯一要求清单。按顺序执行,每一步把「回传」里的文件或输出原样交给用户,不要跳步,不要自行改代码。
|
||||
所有命令在仓库根目录(G:\Robot_v2\机器人手臂)的 Git Bash 里运行。完成的条目由 Hark 在下一次提交里删掉。
|
||||
|
||||
## 已知结论(截至 2026-10-19 05:35 回传)
|
||||
|
||||
- 现有模型 `ml/runs/base/last.pt` 闭环 0% 到达:1 段采样碰撞 99.6%;4 段采样 94%;4 段 + 仿真安全层碰撞 0.78%,到达仍 0%。consist / value / first 三种选法结果几乎一样,问题出在候选本身,不在挑哪条。
|
||||
- 三份代码审查逐行核对了滚动索引、前缀、提交、标签与执行的对齐,没有找到错位;但之前没有任何测试真正走过 `rollout()`。下面第 1 步的 H 段把「仿真错了」「前缀照抄」「子目标切换规则」「多峰」几种解释逐个分开。
|
||||
- 之前「前 1 s 速度比 2.31×」按 6 轴角度直接相加,被 J5/J6 抖动和 pos 任务冻结的 J6 放大。H 段改为分轴、净位移、TCP 三种口径。
|
||||
|
||||
## 第 0 步:同步代码
|
||||
|
||||
```bash
|
||||
git status --short # 有未提交的改动先 commit(git add -A && git commit -m "本机改动"),不要丢
|
||||
git pull --rebase gitea main
|
||||
git log --oneline -1 # 应是 Hark 的「diag: H 段(老师 oracle / 前缀消融 / 子目标规则 / 分轴指标)…」
|
||||
```
|
||||
|
||||
- 训练控制台若开着,先点「停止」,确认没有 python 训练进程在跑。
|
||||
- 不要启动训练、DAgger、M5。
|
||||
|
||||
## 第 1 步:闭环失败定位 H 段(约 35–50 分钟)
|
||||
|
||||
```bash
|
||||
PYTHONUTF8=1 python -m ml.eval.diag_cl --ckpt ml/runs/base/last.pt --n 256 --out ml/runs/diag_cl.json
|
||||
```
|
||||
|
||||
- 专用显存超过 22 GB 就 Ctrl+C,回传终端最后 30 行和已写出的 `ml/runs/diag_cl.json`(每段跑完就写一次,中断也保留已完成的段)。
|
||||
- 现有 checkpoint 没有 EMA、没有 `act_scale`:脚本按原始权重、动作尺度 1.0 载入(输出里 `weights`=`model`、`act_scale_ckpt`=1.0),与之前各轮同口径。
|
||||
- 回传:`ml/runs/diag_cl.json`。
|
||||
|
||||
### 输出键
|
||||
|
||||
| 键 | 内容 |
|
||||
|---|---|
|
||||
| `A_*`、`B_act_diff_deg` | 起点特征 Python 与数据集逐组差;两份特征下的动作差(与之前相同) |
|
||||
| `C` | 旧部署口径(1 段采样、子目标旧规则 timeout)的失败形态。到达与碰撞改为只计每题自己的时限内,数字可能比上一轮略低 |
|
||||
| `H1_oracle` | 老师当策略走完整 `rollout()`。`oracle` 字段:`max_dev_deg_before_dls`(DLS 接管前与老师轨迹的最大关节偏差,p50 / p99 / max)、`reach`、`collision`、`finish_minus_teacher_s`(完成时刻 − 老师时长)、`dls_used` |
|
||||
| `H1_oracle_noise_0.05` / `_0.1` / `_0.2` | 老师动作每次规划加 N(0, σ²)(σ 以 STEP 为单位) |
|
||||
| `H2_open_loop` | r = 3 的 val 行:`s{1,4}_prefix_orig`(前缀原样,执行窗口 3..5 的误差,另含 `prefix_copy_err_deg_mean`)vs `s{1,4}_prefix_zeroed`(前缀置零) |
|
||||
| `H2_replan_empty_s1` / `_s4` | 队列空了才规划(每次 r = 0),其余同新部署默认 |
|
||||
| `H3_headsub_cross_s{1,4}` | 子目标头 + 越过顶点规则(新部署默认) |
|
||||
| `H3_headsub_timeout_s{1,4}` | 子目标头 + 旧规则(< 5° 或 1.5 s 超时,旧部署) |
|
||||
| `H3_teachersub_cross_s{1,4}` / `H3_teachersub_timeout_s{1,4}` | 老师真实途经点全表,两种切换规则 |
|
||||
| `H3_nosub_s{1,4}` | 不用子目标 |
|
||||
| `H4_open_loop` | 按前缀长度 r = 0..3 分组的执行窗口误差 \|Σ(act − Y)·STEP\|(度),1 段与 4 段采样 |
|
||||
|
||||
每个闭环变体(H1–H3)都带同一组指标:`reach`、`collision`(cat ≠ D、时限内)、`goal_ratio_2s_p50`、`end_goal_err_mm_p50`、`first_hit_s_p50`、`t_done_over_teacher_p50`、`brakes_p50`、`joint_amp_ratio`(6 轴前 1 s 关节行程 策略/老师)、`joint_net_ratio`(6 轴前 1 s 净位移 策略/老师)、`j6_pos_amp_deg_p50`、`tcp_path_ratio_p50`、`tcp_net_ratio_p50`、`sign_flip_rate` 与 `sign_flip_rate_teacher`(6 轴前 2 s 相邻 3 步块变号比例)、`by_cat_mode`(各类别 n / reach / collision / goal_ratio_2s_p50)、`D`(D 类单列)。`_s1` = 1 段采样(部署),`_s4` = 4 段采样。
|
||||
|
||||
### 判读
|
||||
|
||||
| 现象 | 结论 |
|
||||
|---|---|
|
||||
| `H1_oracle` 的 reach < 0.95、collision > 0.01、或 `max_dev_deg_before_dls.p99` > 0.1° | 仿真 / 滚动本身有错,之前所有闭环 0% 都不可信。先修仿真,不看下面各项 |
|
||||
| `H1_oracle` 全部正常 | 仿真没问题,失败在策略输出。再对照 `H1_oracle_noise_*`:σ = 0.05 时 reach 就大幅下降,说明闭环对残差噪声极其敏感,单靠降噪不够 |
|
||||
| `H2_open_loop` 的 `prefix_zeroed` 执行窗口误差明显小于 `prefix_orig`,或 `H2_replan_empty` 明显好于同段数的 `H3_headsub_cross`(reach > 0,或 `goal_ratio_2s_p50` 明显更低、`sign_flip_rate` 明显更低) | 前缀照抄(copycat)是主因 |
|
||||
| `H3_teachersub_cross` 明显好于 `H3_teachersub_timeout`(或 `H3_headsub_cross` 好于 `H3_headsub_timeout`) | 子目标切换规则是主因之一 |
|
||||
| `H3_teachersub_*` 明显好于 `H3_headsub_*` | 子目标头给的途经点不可用 |
|
||||
| `H3_nosub` 不比 `H3_headsub_cross` 差 | 子目标通道可以整体去掉 |
|
||||
| `by_cat_mode` 里 `A/pos`(老师就是一条直线)reach 也是 0 | 失败与多峰无关 |
|
||||
| `joint_amp_ratio` 远大于 `joint_net_ratio`,且 `sign_flip_rate` 远高于 `sign_flip_rate_teacher` | 「动得多、走得少」是来回抖,不是速度律学快了 |
|
||||
|
||||
重新打包和重训(`FRESH_FROM=prep`)要等 Hark 看完这一步的结果再定,这一步之后不要自行启动。
|
||||
|
||||
## 回传汇总
|
||||
|
||||
| 步骤 | 回传内容 |
|
||||
|---|---|
|
||||
| 1 | ml/runs/diag_cl.json |
|
||||
| 任一步出错 | 终端最后 30 行输出 |
|
||||
|
||||
---
|
||||
|
||||
## 执行回传(2026-10-19 19:55,本地 AI)
|
||||
|
||||
### 第 0 步:同步代码
|
||||
|
||||
`ml/eval/diag_collision.py` 归档提交(本机改动 `7e063eb`);根目录用户个人 html 为个人文件未入仓,原位保留。log 顶部 = `406f180`「diag: H 段 …」✓。无训练进程。
|
||||
|
||||
### 第 1 步:H 段(完整 JSON 已更新到 ml/runs/diag_cl.json;weights=model、act_scale_ckpt=1.0,同口径;H_tasks 256、missing_teacher 0)
|
||||
|
||||
**判读表逐行核对:**
|
||||
|
||||
1. **`H1_oracle`:仿真完全正常。** reach **1.0**(243/243)、collision **0.0**、`max_dev_deg_before_dls` p50 6.9e-5° / p99 4.6e-4°(阈值 0.1°)、`finish−teacher` p50 −0.02 s、`dls_used` 94.9%。→ **滚动/仿真没有错,之前所有闭环 0% 都可信,失败在策略输出本身。**
|
||||
2. **`H1_oracle_noise`:对残差噪声极其敏感。** σ=0.05 时 reach 1.0→**0.218**(碰撞 9.1%);σ=0.1→0.058;σ=0.2→0.012。→ 单靠降采样噪声不够(判读表第二行命中)。
|
||||
3. **`H2_open_loop`:前缀照抄排除。** prefix_orig vs zeroed:s1 2.315° vs **2.375**(置零不更好);s4 0.251° vs **0.705**(置零明显更差)。`H4` 按 r=0..3 分组误差持平(2.31–2.33°)。`H2_replan_empty`:s1 0%/79.4%/0.836、s4 0%/84.8%/0.603——不比同段数 `H3_headsub_cross` 好。
|
||||
4. **`H3`:子目标三个变体逐位相同(新异常)。** `headsub_cross` / `headsub_timeout` / `nosub` 在 s1、s4 下从 reach 到 end_err 到 flip 数组**完全一致**(s1: 0% / 96.3% / 0.7504 / 414.2091;s4: 0% / 79.8% / 0.5869 / 428.0032)。`teachersub_*`(真值途经点)仅微动(s1 95.9%/0.726、s4 79.4%/0.578)。→ **子目标头通道在这条闭环里零响应**——比「可以整体去掉」更强;可能是 need 概率全程 <0.5 从不触发,或子目标输入对旧模型输出无影响,建议核一下(D0 曾测过 need 标签占比 28.1%)。
|
||||
5. **`by_cat_mode`:A/pos(老师直线)reach 也是 0**(s1 碰撞 94%)→ **失败与多峰无关**。
|
||||
6. **分轴指标:s1 的"动得多"含抖、s4 后基本消抖。** J5/J6 幅度/净位移:s1 3.08/2.57、2.55/2.16;s4 1.83/1.84、0.69/0.69,`sign_flip_rate` 从 0.026–0.062 降到 0.0006–0.0013(≈老师 0.0005–0.004 水平)。→ 4 段采样把来回抖消掉了,但净位移仍 J5 1.8×,且到达仍 0%。
|
||||
|
||||
**C 段**(新时限口径):0% 到达 / 96.5% 碰撞 / 2 s 误差比 0.755。
|
||||
|
||||
### 本轮无出错,无终端报错段。
|
||||
|
||||
---
|
||||
|
||||
## 本地自主排查(2026-10-19 20:40,用户授权「继续排查」)
|
||||
|
||||
### 1. 闭环噪声容忍阈值(老师动作 + N(0,σ²),σ 以 STEP 为单位,n=256,其余同 H1_oracle)
|
||||
|
||||
| σ | 0 | 0.005 | 0.01 | 0.02 | 0.03 | 0.04 | 0.05 |
|
||||
|---|---|---|---|---|---|---|---|
|
||||
| reach | 1.0 | 1.0 | 0.971 | 0.716 | 0.457 | 0.309 | 0.218 |
|
||||
| collision | 0 | 0.8% | 2.5% | 4.9% | 6.2% | 7.4% | 9.1% |
|
||||
| ratio2s | 0.047 | 0.049 | 0.049 | 0.054 | 0.059 | 0.063 | — |
|
||||
|
||||
→ **闭环能容忍的每步噪声上限 ≈ 0.01–0.02×STEP**(约 0.02–0.04°/步)。策略 s1 残差 ~0.77°/步、s4 ~0.05–0.08°/步,都在崩溃区右侧。
|
||||
|
||||
### 2. 8 段采样闭环(steps=8,同 H3_headsub_cross 口径)
|
||||
|
||||
reach **0%** / collision **79.0%** / ratio2s 0.574 / end 408 mm;`sign_flip_rate` 0.0004–0.0015(≈老师 0.0005–0.004,抖已完全消掉)、`amp ≈ net`(1.00/1.19/0.97/1.19/1.36/0.76,无抖动成分)。
|
||||
|
||||
→ 对比 s4(reach 0 / 79.8% / 0.587):**s4→s8 几乎无改善**。按上表的白噪声口径,s8 的残差幅度(16 步 0.56° ≈ 0.035°/步,σ_eff≈0.0175)应给出 reach≈0.75——实测 0。**策略误差不是白噪声,是有方向的系统偏差**:幅度不大但同向、闭环里逐步累积,重规划抵消不掉。降采样噪声(1→4→8 段)已到天花板。
|
||||
|
||||
### 3. 子目标通道:need 头恒不触发(新发现,修正「子目标排除」的理由)
|
||||
|
||||
- 冷启动(子目标特征抹零,与闭环同口径)need 概率:mean 0.0048、p50 0.0030、p90 0.0079、**max 0.0819**、>0.5 占比 **0**;43 道 need=True 题召回 **0/43**。(D0 的 0.71875 准确率恰等于"全预测 False"基线,当时已有此暗示。)
|
||||
- 因此闭环里 `sub_flag` 恒 0、子目标从不激活——这才是 H3 三变体逐位相同的直接原因。
|
||||
- 但模型对子目标特征**高度敏感**(同一噪声下真值子目标 vs 置零:|Δact| mean 0.177、max 3.09)——训练时 80% 样本 sf=1,而闭环 100% 是 sf=0,属训练/闭环分布错配。
|
||||
- 不过 H3_teachersub(强制真值 need + 真值途经点)仍 0% / 95.9%——**子目标不是闭环全灭的主因**,但 need 头失效 + 闭环恒走少数分支值得修。
|
||||
|
||||
### 4. 汇总判断
|
||||
|
||||
- 仿真正常(H1_oracle 已证);采样噪声已压到老师抖动水平仍 0% 到达 → **剩余障碍是策略的系统性动作偏差 + 闭环对误差的极低容忍度(误差复利)**。
|
||||
- 继续调采样段数、子目标规则、候选选择都无解;正确方向是**在闭环状态上训练(DAgger)**,让老师纠正分布偏移,或提高闭环鲁棒性。
|
||||
- 未自行启动 DAgger / 重训,等指示。
|
||||
|
||||
### 5. 补充:系统偏差的方向结构(4096 行开环 + 256 题闭环,4 段采样)
|
||||
|
||||
**开环动作误差(deg,有符号)**:
|
||||
|
||||
- 每轴均值 [0.010, −0.012, −0.001, −0.029, 0.034, 0.018],|mean|/std ≤ 0.17 → **无固定轴向偏置**(不是某轴恒偏)。
|
||||
- 每轴标准差 0.16–0.21°;每步平均 |误差| 0.016–0.018°,**16 步内不随步数增长**。
|
||||
- **lag-1 自相关 0.931–0.967(极高)** → 误差是**慢变/连贯**的,不是白噪声。这解释了为什么白噪声 σ 扫描会低估策略误差的危害:闭环重规划抵消不掉持续同向的偏差。
|
||||
- 误差与老师动作方向:cos mean 0.046 / p50 0.072(不超调也不欠调);投影比 e·Y/|Y|² mean −0.038 / p50 0.006(无系统性缩放)。
|
||||
|
||||
**闭环误差增长(s4,n=256,done=0)**,goal_err / 起点误差:
|
||||
|
||||
| t (s) | 0.25 | 0.50 | 0.75 | 1.00 | 1.50 | 2.00 |
|
||||
|---|---|---|---|---|---|---|
|
||||
| p50 | 0.959 | 0.880 | 0.811 | 0.741 | 0.645 | 0.591 |
|
||||
| p90 | 1.024 | 1.072 | 1.119 | 1.162 | 1.222 | **1.318** |
|
||||
|
||||
→ p50 缓慢靠近(2 s 缩 41%),**p90 持续发散**——≥10% 的题误差随时间增长(误差复利进碰撞轨道),其余也远不够快。
|
||||
|
||||
**补充结论**:策略每步误差很小(~0.017°)但**时间相关性极强(0.93–0.97)**,闭环对"慢变误差"的敏感度远超等幅白噪声;失败是「慢变误差 × 闭环误差复利」。推理侧降噪(段数/选择器)已无空间,训练侧需要降低慢变误差(DAgger / 闭环状态训练)。
|
||||
|
||||
### 6. 「2 s 误差比 >1 的发散题」能否提前筛掉(per-task 分组,s4,n=256;明细 ml/runs/task_divergence.json)
|
||||
|
||||
| 组(2 s 误差比) | 题数 | 碰撞 | 到达 | cat/mode | 起点误差 p50 | 老师时长 p50 | need 率 | 老师到达 |
|
||||
|---|---|---|---|---|---|---|---|---|
|
||||
| 发散 r2>1.0 | 48 | 81% | 0% | A/pos 14、A/pose 14、C/pose 5、C/pos 5、B/pos 3、B/pose 3、D/pos 2、Dr/pos 2 | 374 mm | 1.80 s | 21% | 96% |
|
||||
| 平 0.7–1.0 | 48 | 83% | 0% | 混合 | 539 mm | 2.12 s | 27% | 90% |
|
||||
| 收敛 ≤0.7 | 160 | 79% | 0% | 混合 | 642 mm | 2.18 s | 31% | 96% |
|
||||
|
||||
- **三组碰撞率相同(79–83%)、到达全 0**——不存在"一小撮可提前去掉的坏题";发散题散布在所有类别,老师都能到达。
|
||||
- r2 与起点误差秩相关 **−0.384**(起点误差小 → 比值易大),是**分母效应**:短任务(老师 1.8 s、e0 374 mm)比值波动大,不代表题目特殊;r2 与老师时长/行程几乎无关(−0.09 / −0.03)。
|
||||
- 结论:误差放大是**全体题目上的普遍行为**,筛题无效(只让指标好看),必须训练侧修正。
|
||||
|
||||
### 7. DAgger 小样本试跑(用户授权;未跑全量、未微调)
|
||||
|
||||
**链路实测**(ckpt `ml/runs/base/last.pt`,旧口径 act_scale=1.0):
|
||||
|
||||
| 步骤 | 规模 | 实测 |
|
||||
|---|---|---|
|
||||
| `ml.dagger.collect`(16 题,parts=2) | 16 题 | **12.7 s**(含加载),29 个起点(全撞,拒 1 个 gap) |
|
||||
| `ml.dagger.collect --n 64`(显存采样) | 64 题 | 成功,112 个起点(拒 10 gap),**显存峰值 7.98 GB**(24 GB 卡,无 OOM) |
|
||||
| `node ml/gen/gen_dataset.cjs --tasks tasks_01.jsonl` | 15 个起点 | **7.6 s**,shard 含 qPrev / v0 / traj / cert |
|
||||
| `ACT_SCALE=1.0 node ml/prep/build_dataset.cjs …` | 13 题 | **2.3 s**,453 样本(175 增广),meta `act_scale:1, data_ver:2` |
|
||||
|
||||
**关键口径提醒**:`build_dataset.cjs` 现在默认 **ACT_SCALE=1.25**(DATA_VER 2),而现有主数据 `ml/data/samples` 无 meta → 训练按 **1.0** 读。两者不一致时训练直接报错(README 第 120 行)。全量 DAgger 前二选一:
|
||||
1. 打包 DAgger 时显式 `ACT_SCALE=1.0`(与旧主数据、旧 ckpt 兼容);
|
||||
2. 先按新流程 `FRESH_FROM=prep` 重打主数据为 1.25、重训后再跑 DAgger。
|
||||
本次试跑全部用 1.0;产物在 `ml/data/dag_try*/`(gitignored)。
|
||||
|
||||
**全量耗时外推(估算,以首次实测为准)**:collect 8000 题约 0.8–1 小时(单进程);老师重规划 1.6 万起点 / NPROC≈12 约 10 分钟;打包约 3 分钟;微调 15000 步未试。
|
||||
|
||||
**结论**:DAgger 收集链路可用、显存安全(峰值 8 GB),可以全量启动——但先定 act_scale 口径。
|
||||
|
||||
### 8. 「加大数据能否力挽狂澜」——闭环状态到训练分布的距离(s4,n=256,9408 个闭环状态)
|
||||
|
||||
方法:闭环轨迹每 10 步取一个 q(9408 点),对训练集 191 万老师状态求最近邻(6 维关节角 L∞,±360 回绕);对照 = 每题起点(老师分布内)。
|
||||
|
||||
| 状态 | p10 | p50 | p90 | p99 |
|
||||
|---|---|---|---|---|
|
||||
| 闭环轨迹状态 | 12.10° | **21.46°** | 32.50° | 44.56° |
|
||||
| 每题起点(老师) | 8.71° | 13.21° | 17.91° | 21.53° |
|
||||
|
||||
- 闭环状态离最近老师状态中位 **21.5°**(起点 13.2° 的 1.6×;p90 32.5°、p99 44.6°)。
|
||||
- 训练集本身在关节空间稀疏:连老师起点最近邻都有 13.2°——数据是"轨迹线"而非密集体。
|
||||
- 按密度缩放(6 维 d∝N^(−1/6)):数据翻 4 倍最近邻仅降 ~21%(21.5°→~17°),而策略误差尺度是 0.02°;差距三个数量级,且同类数据不改变误差的时间相关结构(0.94)。
|
||||
- **结论:扩大同类老师数据救不了闭环**;需要「不同的数据」——在策略实际到达的状态上标注(DAgger),把 21.5° 这些分布外点变成训练样本。
|
||||
|
||||
### 9. 方案 A 实测:合成扰动状态样本微调(两轮配方)
|
||||
|
||||
做法:从训练任务老师轨迹的 15–85% 处取状态,加 N(0,σ²) 关节扰动(σ 混合 0.5/1/2/4°;实测起点偏移 p50 1.5°),老师重规划(**2000/2000 到达**、cert 1951/2000),打包 36k 样本(act_scale=1.0,为主数据的 1.9%),从 `base/last.pt` 微调 3000 步,2000 步时以 EMA 权重做 512 题闭环评估。
|
||||
|
||||
| 配方 | success | collision | goal_ratio_2s | ol_fk_mm | ol_best_mm | ol_div |
|
||||
|---|---|---|---|---|---|---|
|
||||
| base/last.pt(基线) | 0 | 0.996 | 0.755 | 5.98 | 2.92 | 0.655 |
|
||||
| +扰动样本 lr 3e-4 | 0 | **0.484** | 0.927 | **15.27** | 10.32 | 0.966 |
|
||||
| +扰动样本 lr 1e-4 | 0 | 0.604 | 0.893 | **16.42** | 11.45 | 0.980 |
|
||||
|
||||
- 碰撞率首次大幅下降(99.6%→48–60%),但 **success 仍 0**;开环精度退化 2.5–2.7 倍(6→15/16 mm)、多样性升高——小步微调在两种分布间折中,把主分布带偏了。
|
||||
- **结论**:1.5° 量级的合成扰动 + 3000 步微调不能解决闭环。若继续这条线,需从**预训练开始**按课程混入扰动样本(并监控开环不退化),成本高;或转向结构改动(方案 B:绝对目标参数化/参考轨迹跟踪)或真实 DAgger。
|
||||
- 产物:`ml/data/samples_perturb/`、`ml/runs/base_perturb/`、`ml/runs/base_perturb2/`(均在 gitignored 目录)。
|
||||
|
||||
### 10. 方案 B 判别实验:模型预测的"累计目标"是否指向参考轨迹
|
||||
|
||||
方法:2000 个扰动起点(原轨迹状态 + σ 0.5–4° 扰动,L∞ 实测 p50 **3.19°**),模型预测 16 步增量并累积为绝对目标 `Z_model = q_off + cumsum(â·STEP)`,对比原老师轨迹未来 16 步绝对位置 `Z_ref`;`ρ = |Z_model − Z_ref| / |q_off − q_ref|`(ρ≈1 = 目标随状态平移、无纠偏;ρ≈0 = 目标稳定指向轨迹)。
|
||||
|
||||
| 模型 | 状态偏移 p50 | 目标误差 p50 | ρ p50 |
|
||||
|---|---|---|---|
|
||||
| base/last.pt | 3.19° | 4.86° | **1.54** |
|
||||
| base_perturb(扰动微调 lr 3e-4) | 3.19° | 5.85° | 1.90 |
|
||||
| base_perturb2(lr 1e-4) | 3.19° | 6.25° | 1.97 |
|
||||
| 对照:未扰动行开环 | — | **0.46°** | — |
|
||||
|
||||
- 未扰动时 16 步累计目标误差仅 0.46°;状态偏移 3.19° 后目标偏离 4.86°(ρ=1.54)——**当前模型是"从当前状态外推增量",不是"指向参考位置"**。扰动微调让 ρ 更差(1.9–2.0),与开环退化一致。
|
||||
- **结论:方案 B 不能靠改执行层实现,必须改训练标签**——把标签从"16 步增量"改为"16 步绝对目标关节角(参考轨迹位置)",让模型学"目标在哪"而不是"从这怎么走"。执行层每步朝目标走,偏移状态下即天然纠偏。
|
||||
- 实施成本:改 `build_dataset.cjs` 标签口径(Y = 绝对目标),同步 FK/平滑/价值等基于增量的辅助损失、rollout 与 `ik-policy.js` 的增量假设,然后从头训练(约 1 小时)。属结构性改动,未擅自开始。
|
||||
|
||||
### 11. 方案 B 初版实测(分支 `feat/abs-target`,保存点 tag `pre-b-abs-target`)
|
||||
|
||||
**改动**(全部在分支,main 未动):
|
||||
- `build_dataset.cjs`:`ABS_TARGET=1` → Y = 16 步绝对目标关节角(/180 归一化),prefix 仍存增量(度),meta 记录 `abs_target`、签名加 `a1`。
|
||||
- `train.py`:读 `abs_target`;每单位动作=180°;FK 损失用绝对位置;平滑项换算到增量量纲;开环指标用第 16 步绝对位置;ckpt 记录 `abs_target`。
|
||||
- `closed_loop.py`:模型输出绝对目标 → 队列存目标序列、执行 `q=clamp(target)`、前缀由目标差分、安全层中点与 DLS 增量转绝对目标。
|
||||
- `ckpt.py` / `data.py`:abs_target 透传与一致性检查。
|
||||
- **过程 bug(已修)**:前缀噪声增广把 eps 直接加到绝对目标标签上(≈18° 污染,一半样本);改为累计偏移后第 r 步扣回。
|
||||
|
||||
**数据**:84 shard 重打 → 201 万样本、abs_target=1;标签连续(步间差 p50 0.30°、p99 1.8°、无 ±180 跳变)。
|
||||
|
||||
**训练结果**(从零 9950 步;p_neq 预热 5000 步):
|
||||
|
||||
| step | ol_fk_mm | ol_best_mm | 闭环 success | 碰撞 |
|
||||
|---|---|---|---|---|
|
||||
| 4000 | 74.6 | 33.4 | 0 | 100% |
|
||||
| 6000 | 140.4 | 70.3 | 0 | 100% |
|
||||
| 8000 | 139.0 | 68.5 | 0 | 100% |
|
||||
|
||||
- 一步采样动作误差 **31°**(4/8 段采样 2.3°/2.8°)——**绝对目标的一步流匹配映射没学会**;闭环 1 段 0%/100%、4 段 0%/100%(ratio2s 1.045)。
|
||||
- 过预热后 5000 步无下降趋势。可能原因:一步样本仅 `p_edge=10%`、`ada_p=1` 逆误差加权压制难样本、FK/平滑辅助项在绝对位置空间主导。
|
||||
- **结论:此配方不成立**。B 若继续,需改训练配方(p_edge↑、w_fk↓、ada_p=0)或换输出头(确定性回归);当前证据不支持继续投入。分支代码保留可复现。
|
||||
+34
-8
@@ -24,7 +24,8 @@
|
||||
| 3 老师门禁 | `bash ml/pipeline.sh gate1` | 25–30 分钟(单核) | `ml/data/consistency.json` |
|
||||
| 4 生成数据 | `bash ml/pipeline.sh gen` | 不共识约 1 小时;共识约 4–5 小时 | `ml/data/shard_*.jsonl`、`stats.json` |
|
||||
| 5 打包样本 | `bash ml/pipeline.sh prep` | 15–30 分钟 | `ml/data/samples/`、`ml/data/r2.json` |
|
||||
| 6 预训练 | `bash ml/pipeline.sh train` | 测速 2000 步约 6 分钟,正式约 3 小时 | `ml/runs/base/best.pt` |
|
||||
| 6 预训练 | `bash ml/pipeline.sh train` | 测速 2000 步约 6 分钟,正式 24000 步约 1.5–2.5 小时(未实测:fp32 MF 分支后的吞吐以测速为准) | `ml/runs/base/best.pt` |
|
||||
| 6′ DAgger 纠偏 | `bash ml/pipeline.sh dagger` | 未实测,估 3–4 小时:2 轮 ×(采起点 5 分钟 + 老师重规划约 30 分钟 + 打包 + 微调 15000 步) | `ml/runs/base_dagger.ckpt`、`ml/runs/base_dag2/best.pt` |
|
||||
| 7 M5 门禁 | `bash ml/pipeline.sh m5` | 估算 10–20 分钟 | `ml/runs/m5_gate_base.json` |
|
||||
| 8 APG 微调 | `bash ml/pipeline.sh apg` | 未实测:展开长度 60–150 次推理随批变化,以监控页前 50 步换算的 ETA 为准 | `ml/runs/apg/best.pt` |
|
||||
| 9 挖错循环 | `bash ml/pipeline.sh mine` | 未实测:每轮 = 5 万题作答 + APG,以第 1 轮实际耗时 × 轮数为准 | `ml/runs/current.pt` |
|
||||
@@ -114,7 +115,9 @@ bash ml/pipeline.sh gen # 默认 5 万条、进程数 = CPU
|
||||
bash ml/pipeline.sh prep
|
||||
```
|
||||
|
||||
每条轨迹按 50 Hz 时间律重放,每 3 步取一个样本(特征与部署同一份 JS 代码计算),30% 样本加 σ = 1° 噪声增广(加噪后间隙 < 3 mm 的不用)。
|
||||
每条轨迹按 50 Hz 时间律重放,每 3 步取一个样本(特征与部署同一份 JS 代码计算)。40% 样本对 q 加 σ = 3° 噪声增广,标签改为 12 步内线性收回老师轨迹(加噪后间隙 < 3 mm 或收回途中有碰撞的不用)。
|
||||
- **动作标签**:Y = 每步增量 / (act_scale · STEP),夹在 ±1,`act_scale` = 1.25(`ACT_SCALE`)。老师巡航时主导轴每步正好 1 个 STEP,标签最大 0.8,执行端的 ±1 只防越界;被 ±1 截掉的收回量在下一步补上。运行时增量 = clamp(act, ±1) · act_scale · STEP,`act_scale` 写在 checkpoint 和 ONNX meta 里,仿真、APG、浏览器运行时都按它反归一化;没有这个字段的旧模型按 1.0 处理。
|
||||
- 每个 shard 打包产物带 `<shard>.meta.json`(数据版本 `data_ver`=2、`act_scale`、增广参数、签名 `sig`)。`ACT_SCALE`、`AUG_P`、`AUG_SIGMA`、`AUG_BACK`、`MIN_GAP_MM` 任一变化,签名就变,`run_prep.sh` 只重打签名不符的 shard;没有 meta 的旧打包一律重打。训练读 meta 取 `act_scale`,同一次训练的各目录(主数据 + DAgger)必须一致,否则报错。
|
||||
认证最小间隙 < 3 mm 的老师轨迹不进训练集(`MIN_GAP_MM=3`,与 `stats.json` 的 `gapDrop` 同一口径):模型只学离障碍 ≥ 3 mm 的走法,运行时投影护栏仍保持 5 mm。
|
||||
检查 `ml/data/samples/index.json`:`train_samples` 约 170 万、`val_samples` 约 9 万(5% 任务划为 val)。
|
||||
`MIN_GAP_MM` 必须与生成阶段相同(`stats.json` 的 `minGapFilterMm`),不一致时这一步直接报错(返回码 8)。
|
||||
@@ -127,11 +130,32 @@ bash ml/pipeline.sh prep
|
||||
bash ml/pipeline.sh train # MeanFlow 头 → ml/runs/bench_base/、ml/runs/base/
|
||||
```
|
||||
|
||||
先跑 2000 步测吞吐(`ml/runs/bench_base/log.jsonl` 的 `samples_per_s`,`eta_h` 是剩余小时数),再正式训练到 60000 步或闭环到达率连续 5 次不升为止。
|
||||
开训前先跑老师轨迹回放自检(`ml/runs/replay_teacher.json`,约 1–3 分钟),两部分都过才开训:
|
||||
- 运动学口径:老师 50 Hz 轨迹直接放进 PyTorch 运动学 / 碰撞 / 到达判据,到达 ≥ 95%、碰撞 ≤ 1%;
|
||||
- 滚动口径:老师当策略(`TeacherOracle`)走完整的闭环仿真 `rollout()`(队列、前缀、提交 3 步、标签 clamp、DLS、时限都与策略同一条路径),时限内到达 ≥ 95%、碰撞 ≤ 1%、DLS 接管前与老师轨迹的最大关节偏差 p99 ≤ 0.1°。
|
||||
流程读 json 的 `pass` 字段,并核对 `fp`(当前 `kin.json`)与 `data_fp`(数据 shard 记录的碰撞模型指纹);任一不符就重跑一次回放,仍不过 rc 5 停。
|
||||
|
||||
- **通过线(R10):`samples_per_s` ≥ 5000,即 60000 步 ≤ 3.5 小时**。参考:3090、batch 1024 实测 6.2 k 样本/s ≈ 2.8 小时。MeanFlow 的 JVP 双前向使训练受算力限制,加大 batch 不会提高样本/s;低于通过线时把 `configs/base.json` 的 `fk_caps` 从 96 降到 48,仍不够再把 `steps` 降到 40000
|
||||
- 训练中每 2000 步打印一行 `EVAL {...}`:val 任务闭环(部署节奏、不开安全层)`success`、`collision_rate`、`time_ok_1p3`、`start_fail`。时限 = 老师时长 × 2(至少 3 s)
|
||||
- 显存预算按 **19–20 GB** 算(Windows + 4K 桌面常驻占 4–5 GB),训练与 APG 进程启动时自设硬帽 `VRAM_FRAC=0.78`(≈18.7 GiB),日志 `vram` 字段记峰值;预训练 `batch=1024` 实测峰值 2.76 GiB。显存不够把 `batch` 改成 512。中断后再跑 `bash ml/pipeline.sh train`(或 `all`)会自动从 `ml/runs/<头>/last.pt` 续训(每 10 分钟存一次),测速那 2000 步做过就跳过。训练发散时不会用发散的权重覆盖 `last.pt` / `best.pt`。
|
||||
闭环指标长期 0% 而开环正常时,跑 `PYTHONUTF8=1 python -m ml.eval.diag_cl --ckpt ml/runs/base/last.pt --n 256`(约 35–50 分钟)→ `ml/runs/diag_cl.json`:A 起点特征 Python 与 JS 逐组差、B 两份特征下动作差、C 旧部署口径的失败形态、H 老师 oracle / 前缀消融 / 子目标规则 / 分轴与分类别指标(各键含义见 `ml/eval/diag_cl.py` 文件头与 `docs/本机执行清单.md`)。
|
||||
|
||||
先跑 2000 步测吞吐(`ml/runs/bench_base/log.jsonl` 的 `samples_per_s`,`eta_h` 是剩余小时数),再正式训练 24000 步(`configs/base.json` 的 `steps`,余弦学习率按这个预算走完)。
|
||||
|
||||
- **通过线(R10):`samples_per_s` ≥ 5000**(24000 步 × batch 1024 ≈ 1.4 小时)。MeanFlow 的 JVP 双前向使训练受算力限制,加大 batch 不会提高样本/s;低于通过线时把 `configs/base.json` 的 `fk_caps` 从 96 降到 48
|
||||
- **EMA**:训练同时维护权重的指数滑动平均(`ema` 0.999),checkpoint 里存 `model`(原始权重)与 `ema`。评估、DAgger、APG 起点、M5、挖错、验收、导出一律用 EMA 权重(没有 `ema` 的旧 checkpoint 用 `model`)
|
||||
- 训练中每 2000 步用 EMA 权重打印一行 `EVAL {...}`:val 任务闭环(部署节奏、不开安全层)`success`、`collision_rate`、`goal_ratio_2s`(2 s 时目标误差 / 起点误差的中位数)、`time_ok_1p3`、`start_fail`,时限 = 老师时长 × 2(至少 3 s),碰撞与到达只计时限内;开环三项(val 样本、老师状态上):`ol_fk_mm` 候选 0 的 16 步末端误差、`ol_best_mm` 8 个候选里最好的一条、`ol_div` 候选多样性。
|
||||
- **选模**:`score` = success + 0.05·(1 − collision_rate) + 0.05·(1 − min(1, goal_ratio_2s)) − 1e-4·min(ol_best_mm, 500),到达率全 0 时闭环指标仍能排序。p_neq 预热期内和预热后第一次评估之前不存 best.pt(一步映射还没训过);早停只在 max(预热 + 2 次评估, 总步数一半) 之后计数,连续 5 次不升即停
|
||||
- 损失:FK / 平滑只加在 MF 分支 r = t 且 t ≤ 0.3 的配对去噪估计 x̂ = z − t·u 上;时间嵌入最高角频率 50;MF 分支与 JVP 走 fp32(TF32 矩阵乘);前 5000 步 p_neq = 0,之后 p_neq = 0.5,另有 10% 样本固定在部署用的 (r = 0, t = 1)(`p_edge`)
|
||||
- 子目标头只用把 sub 特征置零的输入训练(sub 存在时子目标标签与输入是同一个量,原样抄就能答对);运行时子目标头只在冷启动、sub 为空时使用,口径一致。价值头用完整输入
|
||||
- 条件输入增广(训练时现做,`aug_pre_drop` 0.15 / `aug_pre_sigma` 0.1 / `aug_dq_sigma` 0.3):前缀随机清空(变成 r = 0 样本,标签不变);前缀加噪声,标签前 r 步同步加、第 r 步扣回;速度特征 dq 加噪声。目的是让模型不能靠照抄上一块和上一帧速度出动作
|
||||
|
||||
## 6′. DAgger 纠偏
|
||||
|
||||
```bash
|
||||
bash ml/pipeline.sh dagger
|
||||
```
|
||||
|
||||
起点模型:预训练 best.pt 的闭环 success > 0 时用它,否则用 last.pt(`python -m ml.common.ckpt pick ml/runs/<头>`),都取 EMA 权重;之后每轮按同一规则取上一轮的结果。微调时只有起点已走过自己的 p_neq 预热才不再预热。
|
||||
用起点模型在 train 任务上闭环跑(关安全层),每题取「首次碰撞前 0.2 s」和一个随机时刻的状态做起点(须无碰、间隙 > `MIN_GAP_MM`、离目标 > 5 mm),同时记下该时刻的上一帧。老师从这些起点重新规划(`gen_dataset.cjs --tasks`,与主数据同认证),时间律从当前速度在首段方向上的分量起步(`v0`),t = 0 样本的速度特征用真实上一帧;打包后与主数据一起微调。微调的 val 只取主数据,DAgger 样本全部算 train。默认 2 轮(`DAG_ROUNDS`),每轮 8000 题(`DAG_N`)、微调 15000 步(`DAG_STEPS`,lr 3e-4)。老师只用于训练标注,部署仍无老师。M5 门禁读 `ml/runs/<头>_dagger.ckpt` 指向的模型;APG / 挖错的 BC 项也用 DAgger 样本。可断点续跑(采集、老师 shard、打包、微调各有完成标记)。
|
||||
- 显存预算**浮动**:整卡留 1 GB 给系统(`VRAM_RESERVE_GIB`,默认 1),训练与 APG 进程启动时按"整卡 − nvidia-smi 已用 − 余量"和"CUDA 空闲 − 余量"两者取小自设本进程上限(`ml/common/vram.py`)。占显存的应用多,上限就自动变小,不用改配置;想按定值对拍给 `VRAM_FRAC`(旧口径 0.78)。日志 `vram` 字段记峰值;预训练 `batch=1024` 实测峰值 2.76 GiB、APG `batch=48` 13.08 GiB。显存不够把 `batch` 改成 512。中断后再跑 `bash ml/pipeline.sh train`(或 `all`)会自动从 `ml/runs/<头>/last.pt` 续训(每 10 分钟存一次),测速那 2000 步做过就跳过。训练发散时不会用发散的权重覆盖 `last.pt` / `best.pt`。
|
||||
|
||||
## 7. M5 门禁:选定动作头
|
||||
|
||||
@@ -177,7 +201,7 @@ bash ml/pipeline.sh mine # 默认 6 轮;ROUNDS=3 APG_STEPS=1000 MIN
|
||||
## 10. 导出
|
||||
|
||||
```bash
|
||||
bash ml/pipeline.sh export # fp32 ONNX + meta.json → webapp/models/(约 20 MB)
|
||||
bash ml/pipeline.sh export # fp32 ONNX + meta.json(含 act_scale)→ webapp/models/(约 20 MB),导出 EMA 权重
|
||||
```
|
||||
|
||||
默认 fp32:部署走 wasm 与 Node CPU,fp16 在 CPU 后端上不保证更快。只有第 13 步用 `&ep=webgpu` 实测 `__POLICY.latency()` 明显受益时,才另导出 fp16 做对比:
|
||||
@@ -220,6 +244,7 @@ python ml/serve.py # 带跨源隔离头的本地服务,wasm
|
||||
- 控制台 `__POLICY.latency()` 查看推理、前瞻、安全层、整次规划的 P50/P99 与节拍;`__POLICY.stats()` 查看刹停、换向、精修次数
|
||||
- `&autotest=30` 自动跑与 bench 同一批贴柱题,结果在 `__POLICY.autotest`
|
||||
- 运行时投影护栏 5 mm:候选动作离障碍 < 5 mm 时沿间隙梯度外推(训练数据的过滤线是 3 mm,护栏比数据更保守)
|
||||
- 动作尺度读模型 meta 的 `act_scale`;子目标按「越过顶点」切换(与训练标签同口径:沿拐角平分面越过当前途经点即换下一个,没有超时),与仿真 `closed_loop.sub_advance` 同一规则
|
||||
- 工具偏移 ≠ 0 时自动交回原规划器
|
||||
- 离线环境:`bash ml/prep/vendor_ort.sh` 把 onnxruntime-web 放到 `webapp/vendor/ort/`,地址加 `&ortLocal=1`(WebGPU 同样可用)
|
||||
|
||||
@@ -300,7 +325,8 @@ ml/
|
||||
sim/closed_loop.py 部署节奏闭环仿真(验证 / 挖错 / 门禁)
|
||||
apg/apg_finetune.py APG 可微微调
|
||||
mining/ tasks.py(无老师出题)、mine_round.py、gate.py(含 R7)、loop.sh
|
||||
eval/ dump_parity.cjs + check_parity.py(JS↔PyTorch 对拍)、check_r2.py(R2)、r4_probe.py(R4 无头浏览器探针)、acceptance.py(验收汇总)
|
||||
eval/ dump_parity.cjs + check_parity.py(JS↔PyTorch 对拍)、check_r2.py(R2)、r4_probe.py(R4 无头浏览器探针)、acceptance.py(验收汇总)、
|
||||
replay_teacher.py(开训前回放自检,含老师 oracle 走 rollout)、diag_cl.py(闭环失败定位 A/B/C/H)
|
||||
toy/toy_bimodal.py 2D 双模态对照实验(动作头选型依据)
|
||||
export_onnx.py serve.py
|
||||
configs/ base.json(MeanFlow)、wta.json(回退)、small.json(R4 不过时的 d=128)、apg.json
|
||||
|
||||
+27
-21
@@ -6,7 +6,10 @@
|
||||
2026-10-18:展开长度不再定长 60 次推理(3.6 s,会截掉老师时长 > 3.6 s 的绕远题),改为覆盖本批 exec_s p90×1.2,上限 plans_max;
|
||||
"到达"仍是每 seg_plans 段末的距离项(无到达即停),属有意设计:段末项给出持续的接近梯度。
|
||||
margin_train 3 mm:与训练数据过滤线一致(认证最小间隙 < 3 mm 的老师轨迹不进训练集),碰撞项不与 BC 标签对抗;
|
||||
运行时投影护栏独立保持 5 mm。端点与中点两个铰链取均值,与只查端点同量纲,w_col 无需重标。"""
|
||||
运行时投影护栏独立保持 5 mm。端点与中点两个铰链取均值,与只查端点同量纲,w_col 无需重标。
|
||||
2026-10-19:起点取 EMA 权重(ckpt.load_policy);动作反归一化乘 act_scale(与数据同一尺度,不一致报错),产物存 act_scale;
|
||||
子目标推进改用 closed_loop.sub_advance(默认「越过顶点」,与标签同口径);子目标头 BC 项用 sub 特征置零的输入(同 train.py);
|
||||
val 只取主数据目录(第一个 --samples)。"""
|
||||
import argparse, json, math, os, random, sys, time
|
||||
import torch
|
||||
|
||||
@@ -15,7 +18,7 @@ from ml.common.features import features, STEP, H, wrap180
|
||||
from ml.data import Samples, load_tasks
|
||||
from ml.model.policy import build, wta_loss
|
||||
from ml.model.meanflow import meanflow_loss
|
||||
from ml.sim.closed_loop import evaluate, tasks_to_tensors, goal_err
|
||||
from ml.sim.closed_loop import evaluate, tasks_to_tensors, goal_err, sub_advance
|
||||
from ml.common.ckpt import Saver, atomic_save, rng_state, set_rng_state
|
||||
from ml.common import vram
|
||||
|
||||
@@ -41,18 +44,23 @@ def main():
|
||||
dev = a.device; torch.manual_seed(a.seed); random.seed(a.seed)
|
||||
vram.limit()
|
||||
os.makedirs(a.out, exist_ok=True)
|
||||
ck = torch.load(a.init, map_location=dev)
|
||||
mcfg = ck['cfg']
|
||||
model = build(mcfg).to(dev); model.load_state_dict(ck['model'])
|
||||
from ml.common.ckpt import load_policy
|
||||
model, ck = load_policy(a.init, dev); mcfg = ck['cfg']
|
||||
kin = Kin(device=dev)
|
||||
data = Samples(a.samples, device=dev, split='train')
|
||||
data = Samples(a.samples, device=dev, split='train', val_dirs=a.samples[:1])
|
||||
act_scale = float(ck.get('act_scale', 1.0))
|
||||
if abs(act_scale - data.act_scale) > 1e-9:
|
||||
raise SystemExit(f'起点 act_scale={act_scale} 与样本 act_scale={data.act_scale} 不一致(BC 项标签尺度不同),换同一版数据训练的模型')
|
||||
model.act_scale = act_scale
|
||||
all_tasks = load_tasks(a.samples)
|
||||
train_tasks = [t for t in all_tasks if t['split'] == 'train'] + load_pool(a.pool)
|
||||
val_tasks = [t for t in all_tasks if t['split'] == 'val'][:C['eval_tasks']]
|
||||
main_tid = {t['tid'] for t in load_tasks(a.samples[:1]) if t['split'] == 'val'}
|
||||
train_tasks = [t for t in all_tasks if t['tid'] not in main_tid] + load_pool(a.pool)
|
||||
val_tasks = [t for t in load_tasks(a.samples[:1]) if t['split'] == 'val'][:C['eval_tasks']]
|
||||
w = [C['cat_weights'].get(t.get('cat', 'A'), 1.0) * (2.0 if t.get('hard') else 1.0) for t in train_tasks]
|
||||
opt = torch.optim.AdamW(model.parameters(), lr=C['lr'], betas=(0.9, 0.95), weight_decay=0.0)
|
||||
amp = dev.startswith('cuda') and C.get('amp') == 'bf16'
|
||||
step = torch.tensor(STEP, device=dev)
|
||||
astep = step * act_scale # 动作 → 度
|
||||
it0 = 0
|
||||
if a.resume and os.path.exists(a.resume):
|
||||
rk = torch.load(a.resume, map_location=dev); model.load_state_dict(rk['model'])
|
||||
@@ -68,11 +76,11 @@ def main():
|
||||
base, _ = evaluate(model.eval(), kin, val_tasks, dev)
|
||||
print('EVAL0', json.dumps(base), flush=True); log.write(json.dumps({'eval': base, 'step': 0}) + '\n')
|
||||
best = base['success']
|
||||
atomic_save({'model': model.state_dict(), 'cfg': mcfg, 'step': 0, 'fp': kin.fp, 'eval': base}, os.path.join(a.out, 'best.pt'))
|
||||
atomic_save({'model': model.state_dict(), 'cfg': mcfg, 'step': 0, 'fp': kin.fp, 'eval': base, 'act_scale': act_scale}, os.path.join(a.out, 'best.pt'))
|
||||
M = C['margin_train_mm']; t0 = time.time()
|
||||
saver = Saver(C.get('ckpt_min', 10)) # 2026-10-18:每 10 分钟 + 收到停止请求时存 last.pt(不再只在每 500 次验证时存)
|
||||
def save_last(n, ev=None):
|
||||
atomic_save({'model': model.state_dict(), 'opt': opt.state_dict(), 'cfg': mcfg, 'step': n, 'fp': kin.fp, 'eval': ev, 'rng': rng_state()}, os.path.join(a.out, 'last.pt')); saver.mark()
|
||||
atomic_save({'model': model.state_dict(), 'opt': opt.state_dict(), 'cfg': mcfg, 'step': n, 'fp': kin.fp, 'eval': ev, 'rng': rng_state(), 'act_scale': act_scale}, os.path.join(a.out, 'last.pt')); saver.mark()
|
||||
for it in range(it0, C['steps']):
|
||||
model.train()
|
||||
batch = random.choices(train_tasks, weights=w, k=C['batch'])
|
||||
@@ -81,7 +89,7 @@ def main():
|
||||
z = torch.randn(B, H, 6, device=dev)
|
||||
q, qp = q0.clone(), q0.clone()
|
||||
queue = torch.zeros(B, 0, 6, device=dev)
|
||||
subs = torch.zeros(B, 2, 6, device=dev); nsub = torch.zeros(B, dtype=torch.long, device=dev); sub_t = torch.zeros(B, device=dev)
|
||||
subs = torch.zeros(B, 2, 6, device=dev); nsub = torch.zeros(B, dtype=torch.long, device=dev); sub_t = torch.zeros(B, device=dev); sub_prev = q0.clone()
|
||||
opt.zero_grad(set_to_none=True)
|
||||
stats = {'col': 0.0, 'tgt': 0.0, 'smooth': 0.0}
|
||||
# 展开长度随本批任务时长:覆盖本批 exec_s 的 p90 × 1.2(无老师参考题按 6 s),夹在 [plans, plans_max]
|
||||
@@ -95,11 +103,7 @@ def main():
|
||||
pre = torch.zeros(B, 3, 6, device=dev); pre[:, :r] = queue[:, :r]
|
||||
qe = q + queue.sum(1)
|
||||
with torch.no_grad():
|
||||
if (nsub > 0).any():
|
||||
cur = subs[:, 0]; dd = torch.where(kin.cont, wrap180(cur - qe), cur - qe).abs().max(-1).values
|
||||
adv = (nsub > 0) & ((dd < 5) | (tnow - sub_t > 1.5))
|
||||
subs = torch.where(adv[:, None, None], torch.cat([subs[:, 1:], subs[:, 1:]], 1), subs)
|
||||
nsub = torch.where(adv, nsub - 1, nsub); sub_t = torch.where(adv, torch.full_like(sub_t, tnow), sub_t)
|
||||
subs, nsub, sub_prev, sub_t = sub_advance(kin, subs, nsub, sub_prev, sub_t, qe.detach(), tnow)
|
||||
feat = features(kin, q, qp, T, R, pre, torch.full((B,), r, device=dev), subs[:, 0], (nsub > 0).float())
|
||||
with torch.autocast('cuda', dtype=torch.bfloat16, enabled=amp):
|
||||
act, val, sub = model.sample_k(feat, 1, z)
|
||||
@@ -108,8 +112,8 @@ def main():
|
||||
with torch.no_grad():
|
||||
need = sub[:, 12].float() > 0.5
|
||||
subs = q.detach()[:, None] + sub[:, :12].float().view(B, 2, 6) * 180
|
||||
nsub = torch.where(need, torch.full_like(nsub, 2), torch.zeros_like(nsub))
|
||||
inc = act.clamp(-1, 1) * step # [B,16,6]
|
||||
nsub = torch.where(need, torch.full_like(nsub, 2), torch.zeros_like(nsub)); sub_prev = q.detach().clone()
|
||||
inc = act.clamp(-1, 1) * astep # [B,16,6]
|
||||
commit = inc[:, r:r + 3]
|
||||
if r == 0: # 冷启动:先提交一块,下一轮才开始执行
|
||||
queue = commit; continue
|
||||
@@ -118,7 +122,7 @@ def main():
|
||||
d = queue[:, k]
|
||||
qp = q; q = kin.clamp_box(q + d)
|
||||
# 碰撞铰链:端点 + 中点(与仿真/运行时的离散检查一致)
|
||||
G = kin.gaps(torch.cat([q, 0.5 * (q + qp)], 0), M)
|
||||
G = kin.gaps(torch.cat([q, 0.5 * (q + qp)], 0), M, pairs=True)
|
||||
hinge = torch.relu(M - G['gap'])
|
||||
col = torch.zeros(2 * B, device=dev).index_add(0, G['b'], hinge).view(2, B).mean(0) # 端点与中点取均值:与只查端点时同量纲,w_col 无需重标
|
||||
sm = ((d / step) ** 2).sum(-1) + (((d - prev_d) / step) ** 2).sum(-1)
|
||||
@@ -141,7 +145,9 @@ def main():
|
||||
with torch.autocast('cuda', dtype=torch.bfloat16, enabled=amp):
|
||||
mem = model.encode(X)
|
||||
l_bc = wta_loss(model, mem, Y)[0] if model.head == 'wta' else meanflow_loss(model, mem, Y)[0]
|
||||
v, s = model.heads(mem)
|
||||
v, _ = model.heads(mem)
|
||||
Xs = X.clone(); Xs[:, 68:75] = 0
|
||||
_, s = model.heads(model.encode(Xs))
|
||||
l_bc = l_bc + 0.1 * (v.float() - V).abs().mean() + 0.1 * (s.float() - SG).abs().mean()
|
||||
(C['w_bc'] * l_bc).backward()
|
||||
gn = torch.nn.utils.clip_grad_norm_(model.parameters(), C['grad_clip'])
|
||||
@@ -160,7 +166,7 @@ def main():
|
||||
res, _ = evaluate(model.eval(), kin, val_tasks, dev)
|
||||
res['it'] = it + 1; res['t'] = time.time()
|
||||
print('EVAL', json.dumps(res), flush=True); log.write(json.dumps({'eval': res}) + '\n'); log.flush()
|
||||
sd = {'model': model.state_dict(), 'cfg': mcfg, 'step': it + 1, 'fp': kin.fp, 'eval': res}
|
||||
sd = {'model': model.state_dict(), 'cfg': mcfg, 'step': it + 1, 'fp': kin.fp, 'eval': res, 'act_scale': act_scale}
|
||||
save_last(it + 1, res) # last 带优化器状态供续跑;best 不带(导出/门禁用)
|
||||
if res['success'] > best + 1e-4:
|
||||
best = res['success']; atomic_save(sd, os.path.join(a.out, 'best.pt'))
|
||||
|
||||
@@ -60,3 +60,33 @@ def set_rng_state(d):
|
||||
try: torch.cuda.set_rng_state_all([t.cpu() for t in d['cuda']])
|
||||
except Exception: pass
|
||||
return True
|
||||
|
||||
|
||||
def load_policy(path, device, prefer_ema=True):
|
||||
"""载入策略(2026-10-19,审查 review_model_train P0-2):优先取 EMA 权重 ck['ema'](train.py 起存),没有就用 ck['model'](旧 checkpoint、APG 产物)。
|
||||
model.act_scale = ck['act_scale'](动作标签归一化倍数,见 ml/prep/build_dataset.cjs ACT_SCALE);旧 checkpoint 缺省 1.0 = 原口径。
|
||||
闭环仿真 / DAgger / APG 起点 / 挖错 / 门禁 / 验收 / 导出都经这里载入,保证同一份权重、同一个动作尺度。返回 (model.eval(), ck)。"""
|
||||
from ml.model.policy import build
|
||||
ck = torch.load(path, map_location=device)
|
||||
m = build(ck['cfg']).to(device)
|
||||
sd = ck.get('ema') if prefer_ema and ck.get('ema') is not None else ck['model']
|
||||
m.load_state_dict(sd)
|
||||
m.act_scale = float(ck.get('act_scale', 1.0))
|
||||
return m.eval(), ck
|
||||
|
||||
|
||||
def pick_start(run_dir):
|
||||
"""DAgger 起点(2026-10-19,审查 P0-2):best.pt 的闭环 success > 0 才用 best.pt;否则用 last.pt
|
||||
(success 全 0 时 best.pt 只由开环误差决定,可能落在 p_neq 预热期,一步映射没训过)。载入时取 EMA。"""
|
||||
best, last = os.path.join(run_dir, 'best.pt'), os.path.join(run_dir, 'last.pt')
|
||||
if os.path.exists(best):
|
||||
ev = torch.load(best, map_location='cpu').get('eval') or {}
|
||||
if (ev.get('success') or 0) > 0 or not os.path.exists(last): return best
|
||||
return last if os.path.exists(last) else best
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
# python -m ml.common.ckpt pick <运行目录> → 打印 DAgger 起点 checkpoint 路径
|
||||
import sys
|
||||
if len(sys.argv) == 3 and sys.argv[1] == 'pick': print(pick_start(sys.argv[2]))
|
||||
else: raise SystemExit('usage: python -m ml.common.ckpt pick <run_dir>')
|
||||
|
||||
+59
-5
@@ -118,19 +118,72 @@ class Kin:
|
||||
return a, b
|
||||
|
||||
# ---------------- 间隙 ----------------
|
||||
def gaps(self, q, cap=40.0, xf=None, max_pairs=None):
|
||||
# 2026-10-19(用户实测 dagger/诊断吃满 24 GB 显存):本模型 1936 条胶囊 / 491 组 / 67256 粗对 / 104.7 万细对。
|
||||
# 显存有两段峰值,都随点数线性涨:
|
||||
# 粗筛段 cc[:,cp_a]-cc[:,cp_b] 的广播中间量 ≈ 36 B × 点数 × 粗对数(4096 点实测 9.3 GiB,与 cap 无关);
|
||||
# 精算段 ≈ 每对几百字节 × 幸存细对数(4096 点 cap 40 幸存 2.4 亿对;旧版不分块实测 36.9 GiB)。
|
||||
# 所以先按 COARSE_ENTRIES 把点切块(粗筛每块 ≤ ~1 GiB),块内再按幸存细对数 PAIR_BUDGET 切(精算每块 ≤ ~1 GiB)。
|
||||
# 两段不同时存在,峰值与调用方 batch 大小无关。结果与不分块逐位一致(梯度经 cat 保留)。
|
||||
COARSE_ENTRIES = 30_000_000
|
||||
PAIR_BUDGET = 2_000_000
|
||||
|
||||
@staticmethod
|
||||
def _cat_gaps(parts, starts):
|
||||
out = {'gm': torch.cat([p.pop('gm') for p in parts], 0)}
|
||||
if 'b' in parts[0]:
|
||||
for p, s in zip(parts, starts): p['b'] += s
|
||||
for k in ('b', 'i', 'j', 'gap'): out[k] = torch.cat([p.pop(k) for p in parts]) # 逐键拼、拼完即释放
|
||||
return out
|
||||
|
||||
def gaps(self, q, cap=40.0, xf=None, max_pairs=None, pairs=False):
|
||||
"""返回 dict:
|
||||
gm [B,49]:组对最小间隙 min(gap, cap)(gA*7+gB,gA<gB;其余为 cap)
|
||||
b, i, j, gap:幸存胶囊对(gap 可微)
|
||||
pairs=True 时另有 b, i, j, gap:幸存胶囊对(gap 可微;只有 APG 的 hinge 与 check_parity 需要)
|
||||
2026-10-19(用户实测 4096 点 cap 40 峰值 4.46 GiB 全在拼接 8.6e7 对的 b/i/j/gap):
|
||||
默认只返回 gm,每块算完就丢掉胶囊对,峰值只剩单块。
|
||||
"""
|
||||
xf = xf if xf is not None else self.cum_xf(q)
|
||||
Rs, Ts = xf
|
||||
B = q.shape[0]
|
||||
step = max(1, self.COARSE_ENTRIES // int(self.cp_a.shape[0]))
|
||||
if B <= step:
|
||||
return self._gaps_split(q, cap, xf, pairs)
|
||||
st = list(range(0, B, step))
|
||||
return self._cat_gaps([self._gaps_split(q[s:s + step], cap, (Rs[s:s + step], Ts[s:s + step]), pairs) for s in st], st)
|
||||
|
||||
def _coarse(self, q, cap, xf):
|
||||
Rs, Ts = xf
|
||||
cc = torch.einsum('bkij,kj->bki', Rs[:, self.cl_g], self.cl_c.to(q.dtype)) + Ts[:, self.cl_g]
|
||||
d2 = (cc[:, self.cp_a] - cc[:, self.cp_b]).pow(2).sum(-1) # [B,M]
|
||||
del cc
|
||||
return d2 <= (self.cp_lim + cap).pow(2)
|
||||
|
||||
def _gaps_split(self, q, cap, xf, pairs):
|
||||
"""粗筛已在显存预算内:按幸存细对数再切块做精算"""
|
||||
Rs, Ts = xf
|
||||
B = q.shape[0]
|
||||
with torch.no_grad():
|
||||
cc = torch.einsum('bkij,kj->bki', Rs[:, self.cl_g], self.cl_c.to(q.dtype)) + Ts[:, self.cl_g]
|
||||
d2 = (cc[:, self.cp_a] - cc[:, self.cp_b]).pow(2).sum(-1) # [B,M]
|
||||
keep = d2 <= (self.cp_lim + cap).pow(2)
|
||||
per = (self._coarse(q, cap, xf).long() * self.cp_cnt).sum(1) # [B] 每点幸存细对数
|
||||
if B <= 1 or int(per.sum()) <= self.PAIR_BUDGET:
|
||||
bounds = [0, B]
|
||||
else:
|
||||
cs = torch.cumsum(per, 0).tolist(); bounds = [0]; base = 0
|
||||
for k, c in enumerate(cs):
|
||||
if c - base > self.PAIR_BUDGET and k > bounds[-1]:
|
||||
bounds.append(k); base = cs[k - 1]
|
||||
bounds.append(B)
|
||||
del per
|
||||
if len(bounds) == 2:
|
||||
return self._gaps_one(q, cap, xf, pairs)
|
||||
return self._cat_gaps([self._gaps_one(q[s:e], cap, (Rs[s:e], Ts[s:e]), pairs) for s, e in zip(bounds[:-1], bounds[1:])], bounds[:-1])
|
||||
|
||||
def _gaps_one(self, q, cap, xf, pairs):
|
||||
Rs, Ts = xf
|
||||
B = q.shape[0]
|
||||
with torch.no_grad():
|
||||
keep = self._coarse(q, cap, xf)
|
||||
bb, mm = keep.nonzero(as_tuple=True)
|
||||
del keep
|
||||
cnt = self.cp_cnt[mm]
|
||||
rep_b = torch.repeat_interleave(bb, cnt)
|
||||
off = torch.repeat_interleave(self.cp_start[mm] - torch.cumsum(cnt, 0) + cnt, cnt) + torch.arange(int(cnt.sum()), device=q.device)
|
||||
@@ -145,6 +198,7 @@ class Kin:
|
||||
gap = d - self.pair_need[pidx].to(q.dtype)
|
||||
gm = torch.full((B * 49,), float(cap), dtype=q.dtype, device=q.device)
|
||||
gm = gm.scatter_reduce(0, rep_b * 49 + self.pair_gp[pidx], gap.clamp(max=cap), reduce='amin', include_self=True)
|
||||
if not pairs: return {'gm': gm.view(B, 49)}
|
||||
return {'gm': gm.view(B, 49), 'b': rep_b, 'i': pi, 'j': pj, 'gap': gap}
|
||||
|
||||
def min_gap(self, q, cap=40.0):
|
||||
|
||||
+36
-9
@@ -1,19 +1,46 @@
|
||||
"""显存预算(2026-10-18 本机实测后加)。
|
||||
用户定的预算:这台 3090 只有 19–20 GB 能给训练用——Windows + 4K 桌面常驻就占掉 4–5 GB(空载 nvidia-smi 4.4 GB)。
|
||||
limit() 给本进程设硬帽(VRAM_FRAC,默认 0.78 ≈ 18.7 GiB)。超限时在自己的反向里早失败,
|
||||
而不是把桌面挤到没有显存(整机卡顿、别的应用崩),这也是 APG batch=128 原先 OOM 时把显存顶到 23.6 GB 的原因。
|
||||
peak_gib() 记进日志的 vram 字段:网页指标卡与 ml/selftest.sh 都按它核对预算,不用再去猜。
|
||||
实测参考(d=192 同结构、1 步即到位):预训练 batch=1024 → 2.76;APG batch=48/plans=150 → 13.08、batch=128 → OOM。"""
|
||||
"""显存预算(2026-10-18 本机实测后加;同日两次修正)。
|
||||
口径:整卡留 1 GB 给系统(VRAM_RESERVE_GIB),本进程上限 = CUDA 可用 − 余量。
|
||||
为什么不能按"整卡 − nvidia-smi 已用"扣:WDDM 下 nvidia-smi 的"已用"含**驱动可以换出**的量,
|
||||
把它当硬占用会把自己卡死——实测预训练第一次 evaluate 峰值要 ~15 GiB,那样算出的上限只有
|
||||
14.45 GiB,torch 就报 OOM,而同一时刻卡上其实还有 9.98 GiB 空闲(train.py step 2000 的评估里挂掉)。
|
||||
CUDA 的 mem_get_info 才是"我实际能拿到多少":它已经把可换出的部分算进去了。
|
||||
limit() 上限 = min(CUDA 可用, 整卡) − 余量;显式给 VRAM_FRAC 时再取较小值(跟旧口径 0.78 对拍用)。
|
||||
别的应用占多少照样打印,但只作参考、不从上限里扣。
|
||||
peak_gib() 记进日志的 vram 字段:网页指标卡与 ml/selftest.sh 都按它核对,不用再去猜。
|
||||
实测参考(d=192 同结构):预训练 batch=1024 的训练步峰值只有 2.76 GiB,但第一次 evaluate 就要 ~15 GiB
|
||||
(kin.gaps 一次开 2.35 GiB);APG batch=48/plans=150 → 13.08 GiB。"""
|
||||
import os
|
||||
import subprocess
|
||||
import torch
|
||||
|
||||
|
||||
def _smi():
|
||||
"""(整卡已用 GiB, 整卡总量 GiB);拿不到就 (None, None)。"""
|
||||
try:
|
||||
q = 'memory.used,memory.total'
|
||||
r = subprocess.run(['nvidia-smi', f'--query-gpu={q}', '--format=csv,noheader,nounits'],
|
||||
capture_output=True, text=True, timeout=3)
|
||||
u, t = [float(x) for x in r.stdout.strip().splitlines()[0].split(',')[:2]]
|
||||
return u / 1024, t / 1024
|
||||
except Exception:
|
||||
return None, None
|
||||
|
||||
|
||||
def limit():
|
||||
if not torch.cuda.is_available(): return False
|
||||
f = float(os.environ.get('VRAM_FRAC', '0.78'))
|
||||
tot = torch.cuda.get_device_properties(0).total_memory / 2**30
|
||||
torch.cuda.set_per_process_memory_fraction(f)
|
||||
print('显存上限:本进程 %.1f GiB(整卡 %.1f GiB × %.2f,其余留给桌面)' % (f * tot, tot, f), flush=True)
|
||||
torch.empty(1, device='cuda') # 让本进程的 CUDA 上下文先落地,再读可用(否则会把自家上下文算进可用)
|
||||
free = torch.cuda.mem_get_info()[0] / 2**30
|
||||
res = float(os.environ.get('VRAM_RESERVE_GIB', '1'))
|
||||
cap = min(free, tot) - res
|
||||
if os.environ.get('VRAM_FRAC'): # 显式硬帽(对拍旧口径用)
|
||||
cap = min(cap, tot * float(os.environ['VRAM_FRAC']))
|
||||
cap = max(1.0, min(cap, tot - res))
|
||||
torch.cuda.set_per_process_memory_fraction(cap / tot)
|
||||
used, _ = _smi()
|
||||
note = '' if used is None else ';别的应用当前占 %.1f GiB,需要时驱动自行换出,不从上限里扣' % used
|
||||
print('显存上限:本进程 %.1f GiB(CUDA 可用 %.1f − 系统余量 %.1f,= 整卡 %.1f 的 %.0f%%)%s'
|
||||
% (cap, free, res, tot, 100 * cap / tot, note), flush=True)
|
||||
return True
|
||||
|
||||
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
{
|
||||
"model": {"d": 192, "heads": 8, "ff": 768, "enc": 4, "dec": 4},
|
||||
"train": {
|
||||
"batch": 1024, "lr": 1e-3, "warmup": 1000, "steps": 60000, "wd": 0.01, "betas": [0.9, 0.95],
|
||||
"batch": 1024, "lr": 1e-3, "warmup": 1000, "steps": 24000, "wd": 0.01, "betas": [0.9, 0.95],
|
||||
"amp": "bf16", "grad_clip": 1.0,
|
||||
"w_mf": 1.0, "w_fk": 0.5, "w_smooth": 0.05, "w_value": 0.1, "w_sub": 0.1,
|
||||
"p_neq": 0.25, "fk_caps": 96,
|
||||
"p_neq": 0.5, "p_edge": 0.1, "p_neq_warm": 5000, "fk_tmax": 0.3, "fk_caps": 96,
|
||||
"ema": 0.999, "aug_pre_drop": 0.15, "aug_pre_sigma": 0.1, "aug_dq_sigma": 0.3,
|
||||
"ckpt_min": 10, "eval_every": 2000, "eval_tasks": 512, "log_every": 50, "patience_evals": 5
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,10 +1,11 @@
|
||||
{
|
||||
"model": {"d": 128, "heads": 8, "ff": 512, "enc": 4, "dec": 4},
|
||||
"train": {
|
||||
"batch": 1024, "lr": 1e-3, "warmup": 1000, "steps": 60000, "wd": 0.01, "betas": [0.9, 0.95],
|
||||
"batch": 1024, "lr": 1e-3, "warmup": 1000, "steps": 24000, "wd": 0.01, "betas": [0.9, 0.95],
|
||||
"amp": "bf16", "grad_clip": 1.0,
|
||||
"w_mf": 1.0, "w_fk": 0.5, "w_smooth": 0.05, "w_value": 0.1, "w_sub": 0.1,
|
||||
"p_neq": 0.25, "fk_caps": 96,
|
||||
"p_neq": 0.5, "p_edge": 0.1, "p_neq_warm": 5000, "fk_tmax": 0.3, "fk_caps": 96,
|
||||
"ema": 0.999, "aug_pre_drop": 0.15, "aug_pre_sigma": 0.1, "aug_dq_sigma": 0.3,
|
||||
"ckpt_min": 10, "eval_every": 2000, "eval_tasks": 512, "log_every": 50, "patience_evals": 5
|
||||
}
|
||||
}
|
||||
|
||||
+27
-6
@@ -29,15 +29,36 @@
|
||||
"eval_every": 100,
|
||||
"eval_tasks": 16,
|
||||
"log_every": 10,
|
||||
"patience_evals": 5
|
||||
"patience_evals": 5,
|
||||
"p_neq_warm": 100,
|
||||
"fk_tmax": 0.3
|
||||
},
|
||||
"apg": {
|
||||
"batch": 8, "lr": 0.0001, "steps": 10, "grad_clip": 1.0, "amp": "bf16",
|
||||
"plans": 20, "plans_max": 30, "seg_plans": 5,
|
||||
"batch": 8,
|
||||
"lr": 0.0001,
|
||||
"steps": 10,
|
||||
"grad_clip": 1.0,
|
||||
"amp": "bf16",
|
||||
"plans": 20,
|
||||
"plans_max": 30,
|
||||
"seg_plans": 5,
|
||||
"margin_train_mm": 3.0,
|
||||
"w_tgt": 1.0, "w_rot": 0.3, "w_col": 2.0, "w_smooth": 0.02, "w_len": 0.001, "w_bc": 0.5,
|
||||
"cat_weights": {"A": 0.4, "B": 1.0, "C": 1.6, "D": 0.6, "Dr": 0.6},
|
||||
"ckpt_min": 10, "eval_every": 1000, "eval_tasks": 4
|
||||
"w_tgt": 1.0,
|
||||
"w_rot": 0.3,
|
||||
"w_col": 2.0,
|
||||
"w_smooth": 0.02,
|
||||
"w_len": 0.001,
|
||||
"w_bc": 0.5,
|
||||
"cat_weights": {
|
||||
"A": 0.4,
|
||||
"B": 1.0,
|
||||
"C": 1.6,
|
||||
"D": 0.6,
|
||||
"Dr": 0.6
|
||||
},
|
||||
"ckpt_min": 10,
|
||||
"eval_every": 1000,
|
||||
"eval_tasks": 4
|
||||
},
|
||||
"_note": "冒烟与自检专用(bash ml/pipeline.sh selftest / smoke):只验证流程能跑通,数值无意义。apg 段必须有:selftest.sh 第 6 项原先不带 --config,走的是正式 apg.json 的 batch=128,本机实测第一次反向就 OOM;改成正式值 48 也要约 13 GiB,自检不必吃这么多。"
|
||||
}
|
||||
+3
-2
@@ -1,10 +1,11 @@
|
||||
{
|
||||
"model": {"d": 192, "heads": 8, "ff": 768, "enc": 4, "dec": 4, "head": "wta", "K": 8},
|
||||
"train": {
|
||||
"batch": 1024, "lr": 1e-3, "warmup": 1000, "steps": 60000, "wd": 0.01, "betas": [0.9, 0.95],
|
||||
"batch": 1024, "lr": 1e-3, "warmup": 1000, "steps": 24000, "wd": 0.01, "betas": [0.9, 0.95],
|
||||
"amp": "bf16", "grad_clip": 1.0,
|
||||
"w_mf": 1.0, "w_fk": 0.5, "w_smooth": 0.05, "w_value": 0.1, "w_sub": 0.1,
|
||||
"p_neq": 0.25, "fk_caps": 96,
|
||||
"p_neq": 0.5, "p_edge": 0.1, "p_neq_warm": 5000, "fk_tmax": 0.3, "fk_caps": 96,
|
||||
"ema": 0.999, "aug_pre_drop": 0.15, "aug_pre_sigma": 0.1, "aug_dq_sigma": 0.3,
|
||||
"ckpt_min": 10, "eval_every": 2000, "eval_tasks": 512, "log_every": 50, "patience_evals": 5
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,87 @@
|
||||
"""DAgger 状态采集(2026-10-18 首次正式预训练后新增):用当前策略在 train 任务上闭环跑(关安全层),
|
||||
从它自己走到的状态里挑起点,交给老师重新规划 → 这些"策略会去、老师数据里没有"的状态拿到正确标签。
|
||||
python -m ml.dagger.collect --ckpt ml/runs/base/best.pt --samples ml/data/samples --n 8000 --seed 1 --parts 28 --out ml/data/dag1
|
||||
每题最多取 2 个起点:
|
||||
- 撞了的题:首次碰撞前 0.2 s 的状态(策略"快要撞上"的地方,最有用);
|
||||
- 另取一个 [0.3 s, 首撞前或结束] 内均匀随机时刻的状态(偏离老师的一般状态)。
|
||||
起点必须无碰且间隙 > MIN_GAP_MM(老师从碰撞态出发规划不出来,与 build_dataset 的增广接受同一地板),且离目标 > 5 mm(否则 DLS 接管,不需要老师)。
|
||||
输出 tasks_XX.jsonl(按 --parts 均分,供 gen_dataset.cjs --tasks 并行),每行 {q0, qPrev, T, R, cat, src}。
|
||||
2026-10-19(审查 F7 / C10):同时记录策略在该时刻的上一帧 qPrev;老师重规划的时间律从当前速度(v0)起步,
|
||||
t=0 样本的 dq 特征用真实上一帧——原来一律按静止起步标注,DAgger 样本不是策略真正到过的 (q, dq) 状态。
|
||||
为什么需要:纯行为克隆在闭环里会偏出老师分布(首次 21850 步闭环到达 0%),原方案指望 APG/挖错修,但它们没法从 0% 起步。
|
||||
老师只用于训练标注,部署仍然无老师。"""
|
||||
import argparse, json, os, random
|
||||
import torch
|
||||
|
||||
from ml.common.kin_torch import Kin
|
||||
from ml.data import load_tasks
|
||||
from ml.model.policy import build
|
||||
from ml.sim.closed_loop import rollout, goal_err, tasks_to_tensors, DT
|
||||
|
||||
GAP_CHUNK = 1024 # 每次 kin.gaps 的查询点数上限
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument('--ckpt', required=True); ap.add_argument('--samples', nargs='+', default=['ml/data/samples'])
|
||||
ap.add_argument('--n', type=int, default=8000); ap.add_argument('--seed', type=int, default=1)
|
||||
ap.add_argument('--parts', type=int, default=8); ap.add_argument('--out', required=True)
|
||||
ap.add_argument('--bs', type=int, default=64)
|
||||
ap.add_argument('--device', default='cuda' if torch.cuda.is_available() else 'cpu')
|
||||
a = ap.parse_args()
|
||||
gap_min = float(os.environ.get('MIN_GAP_MM') or 3)
|
||||
os.makedirs(a.out, exist_ok=True)
|
||||
kin = Kin(device=a.device)
|
||||
from ml.common.ckpt import load_policy # 2026-10-19:EMA 权重 + act_scale
|
||||
model, ck = load_policy(a.ckpt, a.device)
|
||||
assert ck.get('fp') in (None, kin.fp), '碰撞模型指纹与 checkpoint 不符'
|
||||
pool = [t for t in load_tasks(a.samples[:1]) if t['split'] == 'train' and t.get('cat') not in ('D',)]
|
||||
rng = random.Random(a.seed); rng.shuffle(pool); tasks = pool[:a.n]
|
||||
out, stat = [], {'tasks': len(tasks), 'collided': 0, 'states': 0, 'rej_gap': 0, 'rej_near': 0}
|
||||
for i in range(0, len(tasks), a.bs):
|
||||
tb = tasks[i:i + a.bs]
|
||||
with torch.no_grad():
|
||||
_, per = rollout(model, kin, tb, a.device, K=8, safety=False, seed=a.seed * 100003 + i, return_traj=True)
|
||||
tr = per['traj'].to(a.device) # [B,T,6]
|
||||
B, Tn, _ = tr.shape
|
||||
_, T, R, _, _ = tasks_to_tensors(tb, a.device)
|
||||
# 2026-10-19 修(用户实测):原来一次喂 64 题 × 64 步 = 4096 点、cap=40 mm 给 kin.gaps,40 mm 内幸存的胶囊对太多,
|
||||
# 峰值把显存 / 内存吃满。这里只需判断"≤0(碰撞)"和"> gap_min(起点地板)",cap 取 gap_min+1 即可,
|
||||
# 且按 GAP_CHUNK 个点分块,用完即释放。
|
||||
cap = gap_min + 1.0
|
||||
flat = tr.reshape(-1, 6)
|
||||
with torch.no_grad():
|
||||
g = torch.cat([kin.gaps(flat[s:s + GAP_CHUNK], cap)['gm'].min(-1).values for s in range(0, flat.shape[0], GAP_CHUNK)]).view(B, Tn)
|
||||
del flat
|
||||
for b, t in enumerate(tb):
|
||||
hit = (g[b] <= 0).nonzero()
|
||||
c = int(hit[0]) if hit.numel() else Tn
|
||||
if hit.numel(): stat['collided'] += 1
|
||||
cand = []
|
||||
if hit.numel(): cand.append(max(0, c - int(round(0.2 / DT))))
|
||||
lo = int(round(0.3 / DT))
|
||||
if c - 1 > lo: cand.append(rng.randint(lo, c - 1))
|
||||
for k in cand:
|
||||
if g[b, k] <= gap_min: stat['rej_gap'] += 1; continue
|
||||
q = tr[b, k:k + 1]; qprev = tr[b, max(0, k - 1)]
|
||||
e, oe = goal_err(kin, q, T[b:b + 1], R[b:b + 1])
|
||||
if e.item() < 5 and (not t.get('R') or oe.item() < 5): stat['rej_near'] += 1; continue
|
||||
out.append({'q0': [round(v, 3) for v in q[0].tolist()], 'qPrev': [round(v, 3) for v in qprev.tolist()], 'T': t['T'], 'R': t.get('R'), 'cat': t.get('cat', 'A'),
|
||||
'src': {'tid': t['tid'], 'k': k, 'col': bool(hit.numel())}})
|
||||
del tr, g, per
|
||||
if a.device.startswith('cuda'): torch.cuda.empty_cache()
|
||||
print(f'[dagger] {min(i + a.bs, len(tasks))}/{len(tasks)} 题,起点 {len(out)}', flush=True)
|
||||
stat['states'] = len(out)
|
||||
rng.shuffle(out)
|
||||
per_part = (len(out) + a.parts - 1) // max(1, a.parts)
|
||||
for p in range(a.parts):
|
||||
chunk = out[p * per_part:(p + 1) * per_part]
|
||||
if not chunk: continue
|
||||
fn = os.path.join(a.out, 'tasks_%02d.jsonl' % (p + 1))
|
||||
open(fn + '.tmp', 'w', encoding='utf-8').write(''.join(json.dumps(x) + '\n' for x in chunk)); os.replace(fn + '.tmp', fn)
|
||||
json.dump(stat, open(os.path.join(a.out, 'collect.json'), 'w', encoding='utf-8'), indent=1) # 最后写:它存在 = 采集完成
|
||||
print(json.dumps(stat, ensure_ascii=False), flush=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,65 @@
|
||||
#!/usr/bin/env bash
|
||||
# DAgger 纠偏(2026-10-18 首次正式预训练后新增):bash ml/dagger/run.sh <头标签> <起点 ckpt>
|
||||
# 每轮:策略闭环采起点(ml.dagger.collect)→ 老师从这些起点重新规划(gen_dataset.cjs --tasks,NPROC 并行)
|
||||
# → 打包成样本(build_dataset.cjs,与主数据同口径)→ 主数据 + 历轮 DAgger 样本一起微调 DAG_STEPS 步。
|
||||
# 环境变量:DAG_ROUNDS(默认 2)、DAG_N(每轮采样题数,默认 8000,每题最多 2 个起点)、DAG_STEPS(每轮微调步数,默认 15000)、
|
||||
# DAG_LR(默认 3e-4)、NPROC、MIN_GAP_MM。
|
||||
# 产物:ml/data/dag_<tag>_r<k>/(起点与老师 shard)、ml/data/samples_dag_<tag>_r<k>/、ml/runs/<tag>_dag<k>/;
|
||||
# 最终模型路径写进 ml/runs/<tag>_dagger.ckpt(m5 读它)。
|
||||
# 断点续跑:采集以 collect.json 为完成标记;老师 shard 先写 .part 再改名、未写完的从断点续写;打包以 .tasks.jsonl 为完成标记;
|
||||
# 微调用该轮 last.pt 续训;整轮完成打 ml/runs/<tag>_dag<k>/done。
|
||||
# 2026-10-19(审查 review_model_train P0-2):每轮起点 = python -m ml.common.ckpt pick <目录>:best.pt 的闭环 success > 0 才用它,
|
||||
# 否则用 last.pt(success 全 0 时 best.pt 只由开环误差决定、可能落在 p_neq 预热期);train.py --init 载入 EMA 权重,
|
||||
# 只有起点已走过自己的 p_neq 预热时才不再预热。微调的 val 只取主数据(DAgger 样本全部算 train)。
|
||||
set -e -o pipefail
|
||||
cd "$(dirname "$0")/../.."
|
||||
PY=${PYTHON:-python}
|
||||
TAG=$1; CK=$2
|
||||
[ -n "$TAG" ] && [ -f "$CK" ] || { echo "usage: run.sh <tag> <ckpt>($CK 不存在)"; exit 1; }
|
||||
R=${DAG_ROUNDS:-2}; N=${DAG_N:-8000}; STEPS=${DAG_STEPS:-15000}; LR=${DAG_LR:-3e-4}; P=${NPROC:-$(nproc)}
|
||||
CFGF=ml/configs/$TAG.json
|
||||
SAMPLES="ml/data/samples"
|
||||
for ((r=1; r<=R; r++)); do
|
||||
D=ml/data/dag_${TAG}_r$r; SD=ml/data/samples_dag_${TAG}_r$r; OUT=ml/runs/${TAG}_dag$r
|
||||
SAMPLES="$SAMPLES $SD"
|
||||
if [ -f $OUT/done ]; then echo "[dagger] 第 $r 轮已完成,跳过"; CK=$($PY -m ml.common.ckpt pick $OUT); continue; fi
|
||||
if [ -f $D/collect.json ]; then echo "[dagger] 第 $r 轮起点已采集,跳过"
|
||||
else echo "[dagger] 第 $r 轮:用 $CK 闭环采起点($N 题)"; $PY -m ml.dagger.collect --ckpt $CK --samples ml/data/samples --n $N --seed $r --parts $P --out $D; fi
|
||||
cat $D/collect.json
|
||||
pids=()
|
||||
for f in $D/tasks_*.jsonl; do
|
||||
[ -e "$f" ] || continue
|
||||
k=$(basename $f .jsonl); k=${k#tasks_}; sh=$D/shard_dag_${TAG}_r${r}_$k.jsonl
|
||||
[ -f $sh ] && continue
|
||||
( node ml/gen/gen_dataset.cjs --tasks $f --seed $((9000 + 100 * r + 10#$k)) --out $sh.part 2>> $D/log_$k.txt && mv $sh.part $sh ) &
|
||||
pids+=($!)
|
||||
done
|
||||
if [ ${#pids[@]} -gt 0 ]; then
|
||||
echo "[dagger] 第 $r 轮:老师重新规划(${#pids[@]} 进程)"
|
||||
t0=$(date +%s)
|
||||
while :; do
|
||||
alive=0; for p in "${pids[@]}"; do kill -0 $p 2>/dev/null && alive=$((alive+1)); done
|
||||
[ $alive = 0 ] && break; sleep 30
|
||||
echo "[dagger 节拍] $(( ($(date +%s) - t0) / 60 )) 分钟 运行中 $alive/${#pids[@]} 已写 $(cat $D/*.part $D/shard_*.jsonl 2>/dev/null | wc -l) 条"
|
||||
done
|
||||
for p in "${pids[@]}"; do wait $p || true; done # 判定按产物(同 run_parallel.sh:Git Bash 长跑后 wait 返回码不可靠)
|
||||
for f in $D/tasks_*.jsonl; do
|
||||
k=$(basename $f .jsonl); k=${k#tasks_}; sh=$D/shard_dag_${TAG}_r${r}_$k.jsonl
|
||||
if [ ! -f $sh ]; then
|
||||
n=0; [ -f $sh.part ] && n=$(wc -l < $sh.part); need=$(wc -l < $f)
|
||||
if [ "$n" -ge "$need" ]; then mv $sh.part $sh; else echo "[dagger] $sh 未写完($n/$need),见 $D/log_$k.txt;重跑本阶段从断点续写"; exit 6; fi
|
||||
fi
|
||||
done
|
||||
fi
|
||||
mkdir -p $SD
|
||||
SIG=$(node ml/prep/build_dataset.cjs --sig) # 2026-10-19:与主数据同一套打包签名,参数变了重打(见 run_prep.sh)
|
||||
todo=(); for sh in $D/shard_dag_*.jsonl; do [ -e "$sh" ] || continue; b=$(basename $sh .jsonl); grep -qF "\"sig\":\"$SIG\"" $SD/$b.meta.json 2>/dev/null && [ -f $SD/$b.tasks.jsonl ] || todo+=("$sh"); done
|
||||
[ ${#todo[@]} = 0 ] || printf '%s\n' "${todo[@]}" | xargs -P "$P" -I{} node ml/prep/build_dataset.cjs {} $SD
|
||||
$PY -m ml.data --index $SD | tail -n 3
|
||||
res=""; [ -f $OUT/last.pt ] && res="--resume $OUT/last.pt"
|
||||
echo "[dagger] 第 $r 轮:微调 $STEPS 步(样本:$SAMPLES)"
|
||||
$PY -m ml.train --samples $SAMPLES --out $OUT --config $CFGF --init $CK --steps $STEPS --lr $LR --warmup 500 $res
|
||||
touch $OUT/done; CK=$($PY -m ml.common.ckpt pick $OUT)
|
||||
done
|
||||
echo "$CK" > ml/runs/${TAG}_dagger.ckpt
|
||||
echo "[dagger] 完成:后续阶段使用 $CK"
|
||||
+66
-7
@@ -1,12 +1,17 @@
|
||||
"""样本加载:ml/data/samples/*.{X,Y,V,SG,TID}.npy + *.tasks.jsonl(ml/prep/build_dataset.cjs 产出)
|
||||
"""样本加载:ml/data/samples/*.{X,Y,V,SG,TID}.npy + *.tasks.jsonl + *.meta.json(ml/prep/build_dataset.cjs 产出)
|
||||
python -m ml.data --index ml/data/samples # 汇总样本/任务数,写 index.json
|
||||
全部载入内存(200 万样本约 1.5 GB),按任务 split 字段划分 train / val。"""
|
||||
全部载入内存(200 万样本约 1.5 GB),按任务 split 字段划分 train / val。
|
||||
动作尺度(2026-10-19):Y = 每步增量 / (act_scale·STEP)。act_scale 取自各 shard 的 meta.json(DATA_VER 2 起写;
|
||||
旧数据没有 meta → act_scale 1.0、data_ver 1,即原口径)。同一次训练的所有目录(主数据 + DAgger)必须同一个 act_scale。"""
|
||||
import argparse, glob, json, os
|
||||
import numpy as np
|
||||
import torch
|
||||
|
||||
PREFIX_SL, MASK_SL, DQ_SL, SUB_SL = slice(47, 65), slice(65, 68), slice(15, 21), slice(68, 75) # 特征分组(features.py 拼接顺序)
|
||||
|
||||
|
||||
def load_tasks(dirs):
|
||||
if isinstance(dirs, str): dirs = [dirs]
|
||||
tasks = []
|
||||
for d in dirs:
|
||||
for f in sorted(glob.glob(os.path.join(d, '*.tasks.jsonl'))):
|
||||
@@ -16,10 +21,28 @@ def load_tasks(dirs):
|
||||
return tasks
|
||||
|
||||
|
||||
def read_meta(d):
|
||||
"""目录内各 shard 的打包参数(data_ver、act_scale、增广参数、sig)。没有 meta 的旧打包 → {'data_ver': 1, 'act_scale': 1.0}。
|
||||
一部分有、一部分没有,或 act_scale 不一致 → 拒绝(新旧标签尺度混在一起会让动作幅度错 25%)。"""
|
||||
xs = sorted(glob.glob(os.path.join(d, '*.X.npy')))
|
||||
ms = [json.load(open(f[:-6] + '.meta.json', encoding='utf-8')) for f in xs if os.path.exists(f[:-6] + '.meta.json')]
|
||||
if not ms: return {'data_ver': 1, 'act_scale': 1.0}
|
||||
if len(ms) != len(xs): raise SystemExit(f'{d}:{len(xs) - len(ms)} 个 shard 缺 meta.json(新旧打包混在一起),重跑 prep')
|
||||
sc = {float(m.get('act_scale', 1.0)) for m in ms}
|
||||
if len(sc) != 1: raise SystemExit(f'{d}:各 shard 的 act_scale 不一致 {sorted(sc)},重跑 prep')
|
||||
return ms[0]
|
||||
|
||||
|
||||
class Samples:
|
||||
def __init__(self, dirs, device='cpu', split=None):
|
||||
def __init__(self, dirs, device='cpu', split=None, val_dirs=None):
|
||||
"""val_dirs:只用这些目录的 val 任务划 val(默认 = 全部 dirs)。2026-10-19(审查 review_closedloop F11):
|
||||
DAgger 目录的任务起点带速度、来自策略状态,不该混进预训练的 val;train.py / apg 传主数据目录,DAgger 样本全部算 train。"""
|
||||
if isinstance(dirs, str): dirs = [dirs]
|
||||
xs, ys, vs, ss, ts = [], [], [], [], []
|
||||
metas = [read_meta(d) for d in dirs]
|
||||
sc = {float(m['act_scale']) for m in metas}
|
||||
if len(sc) > 1: raise SystemExit(f'样本目录的 act_scale 不一致 {sorted(sc)}({dirs}):DAgger 样本须与主数据同一版打包')
|
||||
self.act_scale = sc.pop(); self.data_ver = int(metas[0].get('data_ver', 1)) if metas else 1
|
||||
for d in dirs:
|
||||
for fx in sorted(glob.glob(os.path.join(d, '*.X.npy'))):
|
||||
b = fx[:-6]
|
||||
@@ -27,7 +50,7 @@ class Samples:
|
||||
ss.append(np.load(b + '.SG.npy')); ts.append(np.load(b + '.TID.npy'))
|
||||
if not xs: raise SystemExit(f'没有样本:{dirs}(先跑 ml/prep/run_prep.sh)')
|
||||
self.tasks = load_tasks(dirs)
|
||||
val_tid = {t['tid'] for t in self.tasks if t['split'] == 'val'}
|
||||
val_tid = {t['tid'] for t in load_tasks(val_dirs if val_dirs is not None else dirs) if t['split'] == 'val'}
|
||||
X, Y, V, SG, TID = map(np.concatenate, (xs, ys, vs, ss, ts))
|
||||
is_val = np.isin(TID, np.fromiter(val_tid, dtype=np.int64)) if val_tid else np.zeros(len(TID), bool)
|
||||
sel = {'train': ~is_val, 'val': is_val, None: np.ones(len(TID), bool)}[split]
|
||||
@@ -35,9 +58,43 @@ class Samples:
|
||||
self.X, self.Y, self.V, self.SG, self.TID = t(X), t(Y), t(V), t(SG), t(TID)
|
||||
self.n = self.X.shape[0]
|
||||
|
||||
def batch(self, bs, gen=None):
|
||||
def batch(self, bs, gen=None, aug=None):
|
||||
"""aug:训练配置(train 段),按 aug_pre_drop / aug_pre_sigma / aug_dq_sigma 做条件输入增广(见 augment);None = 原样。"""
|
||||
i = torch.randint(0, self.n, (bs,), device=self.X.device, generator=gen)
|
||||
return self.X[i], self.Y[i], self.V[i], self.SG[i]
|
||||
X, Y = self.X[i], self.Y[i]
|
||||
if aug: X, Y = self.augment(X.clone(), Y.clone(), aug, gen)
|
||||
return X, Y, self.V[i], self.SG[i]
|
||||
|
||||
def augment(self, X, Y, cfg, gen=None):
|
||||
"""条件输入增广(2026-10-19,审查 review_data_actions C3 / review_model_train P1-4:前缀与 dq 的 copycat 捷径)。
|
||||
训练集里 r=3 时要执行的 Y[3:6] 几乎就是前缀第 3 步的延续,dq 又永远是老师的干净速度,模型容易学成「照抄上一块」,
|
||||
闭环里自己的误差经前缀 / dq 原样带回。三种扰动,标签都保持自洽:
|
||||
- 前缀丢弃(aug_pre_drop):r>0 的行以该概率清空前缀与掩码 → 变成 r=0 样本。Y 本来就从当前 q 起算,r=0 时标签不变;
|
||||
- 前缀噪声(aug_pre_sigma,STEP 为单位):r>0 的行一半加噪声 ε。前缀 = 已提交、一定会执行的步,所以标签前 r 步同样 +ε,
|
||||
第 r 步减去 Σε 把位置拉回老师轨迹(亏空在下一步补齐,与增广收回同理);
|
||||
- dq 噪声(aug_dq_sigma,STEP 为单位):一半的行给速度特征加噪声,状态与标签不变。"""
|
||||
B, dev = X.shape[0], X.device
|
||||
rnd = lambda *s: torch.rand(*s, device=dev, generator=gen)
|
||||
rn = lambda *s: torch.randn(*s, device=dev, generator=gen)
|
||||
pl = X[:, MASK_SL].sum(-1).round().long()
|
||||
pd = float(cfg.get('aug_pre_drop', 0.0))
|
||||
if pd > 0:
|
||||
m = (rnd(B) < pd) & (pl > 0)
|
||||
X[:, 47:68] = torch.where(m[:, None], torch.zeros_like(X[:, 47:68]), X[:, 47:68]); pl = torch.where(m, torch.zeros_like(pl), pl)
|
||||
ps = float(cfg.get('aug_pre_sigma', 0.0))
|
||||
if ps > 0:
|
||||
m = (rnd(B) < 0.5) & (pl > 0)
|
||||
km = (torch.arange(3, device=dev)[None] < pl[:, None]) & m[:, None] # [B,3] 前缀有效步
|
||||
eps = rn(B, 3, 6) * ps * km[..., None].to(X.dtype) # STEP 为单位
|
||||
X[:, PREFIX_SL] = X[:, PREFIX_SL] + eps.reshape(B, 18)
|
||||
Y[:, :3] = Y[:, :3] + eps / self.act_scale # Y 以 act_scale·STEP 为单位
|
||||
Y[torch.arange(B, device=dev), pl.clamp(max=Y.shape[1] - 1)] -= eps.sum(1) / self.act_scale
|
||||
Y.clamp_(-1, 1)
|
||||
ds = float(cfg.get('aug_dq_sigma', 0.0))
|
||||
if ds > 0:
|
||||
m = (rnd(B) < 0.5).to(X.dtype)
|
||||
X[:, DQ_SL] = X[:, DQ_SL] + rn(B, 6) * ds * m[:, None]
|
||||
return X, Y
|
||||
|
||||
|
||||
def main():
|
||||
@@ -46,7 +103,9 @@ def main():
|
||||
tasks = load_tasks([a.index])
|
||||
cats = {}
|
||||
for t in tasks: cats[t['cat'] + '/' + t['mode']] = cats.get(t['cat'] + '/' + t['mode'], 0) + 1
|
||||
info = {'train_samples': tr.n, 'val_samples': va.n, 'tasks': len(tasks), 'val_tasks': sum(t['split'] == 'val' for t in tasks), 'by_cat_mode': cats}
|
||||
meta = read_meta(a.index)
|
||||
info = {'train_samples': tr.n, 'val_samples': va.n, 'tasks': len(tasks), 'val_tasks': sum(t['split'] == 'val' for t in tasks), 'by_cat_mode': cats,
|
||||
'data_ver': int(meta.get('data_ver', 1)), 'act_scale': float(meta.get('act_scale', 1.0)), 'sig': meta.get('sig')}
|
||||
json.dump(info, open(os.path.join(a.index, 'index.json'), 'w'), indent=1)
|
||||
print(json.dumps(info, indent=1))
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
其余项读已有文件(缺文件记为 missing,不算通过):
|
||||
ml/runs/policy_bench.json 部署形态:各集合穿模帧 = 0;near 30/30(pos、pose);route 全部到达;nc1000 0 失败;刹停率 < 2%;R8 恢复 ≥ 95%、活锁 = 0
|
||||
ml/data/r2.json R2 近邻标签 std < 10% 步长
|
||||
ml/runs/bench_*/log.jsonl R10 吞吐 ≥ 5000 样本/s(≈ 60000 步 ≤ 3.5 h)
|
||||
ml/runs/bench_*/log.jsonl R10 吞吐 ≥ 5000 样本/s(24000 步 batch 1024 ≈ 1.4 h)
|
||||
ml/runs/m5_gate_*.json M5 头门禁
|
||||
ml/runs/mine/r*/gate.json R7 最后一轮左右比例 / 行程门禁
|
||||
ml/runs/r4.json R4 浏览器推理 batch=8 P99 < 15 ms(r4 探针)
|
||||
@@ -37,8 +37,8 @@ def main():
|
||||
items.append({'item': name, 'value': value, 'pass': (None if ok is None else bool(ok)), 'rule': rule})
|
||||
|
||||
# 1) held-out 闭环(仿真安全层)
|
||||
ck = torch.load(a.ckpt, map_location=a.device)
|
||||
model = build(ck['cfg']).to(a.device); model.load_state_dict(ck['model']); model.eval()
|
||||
from ml.common.ckpt import load_policy # 2026-10-19:EMA 权重 + act_scale
|
||||
model, ck = load_policy(a.ckpt, a.device)
|
||||
kin = Kin(device=a.device)
|
||||
val = [t for t in load_tasks([a.samples]) if t['split'] == 'val']
|
||||
reach = [t for t in val if t.get('reached') and t['cat'] != 'D']
|
||||
@@ -97,7 +97,7 @@ def main():
|
||||
for f in sorted(glob.glob('ml/runs/bench_*/log.jsonl')):
|
||||
recs = [json.loads(l) for l in open(f) if '"samples_per_s"' in l]
|
||||
if recs: sps = max(sps or 0, recs[-1]['samples_per_s'])
|
||||
add('R10 训练吞吐(样本/s)', sps, sps is not None and sps >= 5000, '≥ 5000(≈ 60000 步 ≤ 3.5 h)')
|
||||
add('R10 训练吞吐(样本/s)', sps, sps is not None and sps >= 5000, '≥ 5000(24000 步 ≈ 1.4 h)')
|
||||
m5 = [jload(f) for f in sorted(glob.glob('ml/runs/m5_gate_*.json'))]
|
||||
m5 = [m for m in m5 if m]
|
||||
add('M5 头门禁', [{'head': m['head'], 'collision_rate': m['policy_only']['collision_rate']} for m in m5] or 'missing',
|
||||
|
||||
+3
-2
@@ -60,14 +60,15 @@ def measure(d, a, pool):
|
||||
# 与池无关的口径:全部有效近邻对(取距离较近的一半,排除远处非线性段)上回归 std ~ a + b·dist,截距 a
|
||||
dd = best_d[fin]; jj = best_j[fin]; qf = torch.nonzero(fin).squeeze(1)
|
||||
half = dd <= torch.quantile(dd, 0.5)
|
||||
Ya, Yb = d.Y[qi][qf[half], :3].float(), d.Y[pi][jj[half], :3].float()
|
||||
# 2026-10-19:Y 以 act_scale·STEP 为单位(DATA_VER 2 起 1.25),乘回 act_scale 让判据仍是「STEP 的比例」,与旧数据可比
|
||||
Ya, Yb = d.Y[qi][qf[half], :3].float() * d.act_scale, d.Y[pi][jj[half], :3].float() * d.act_scale
|
||||
sa = (Ya - Yb).pow(2).mean(-1).mean(-1).sqrt() / math.sqrt(2)
|
||||
A = torch.stack([torch.ones_like(dd[half]), dd[half]], 1)
|
||||
coef = torch.linalg.lstsq(A.cpu(), sa.cpu()[:, None]).solution.squeeze(1)
|
||||
std_norm = max(0.0, coef[0].item())
|
||||
thr = torch.quantile(best_d[fin], a.frac)
|
||||
m = fin & (best_d <= thr)
|
||||
Yq, Yp = d.Y[qi][m, :3].float(), d.Y[pi][best_j[m], :3].float()
|
||||
Yq, Yp = d.Y[qi][m, :3].float() * d.act_scale, d.Y[pi][best_j[m], :3].float() * d.act_scale
|
||||
rms = (Yq - Yp).pow(2).mean(-1).mean(-1).sqrt() # 每对、6 轴 × 3 步平均
|
||||
std = (rms / math.sqrt(2))
|
||||
return {'pairs': int(m.sum()), 'pool': P, 'queries': Q, 'nn_dist_threshold': thr.item(),
|
||||
|
||||
@@ -0,0 +1,316 @@
|
||||
#!/usr/bin/env python
|
||||
"""闭环失败定位(只读,手动运行,n=256 约 35–50 分钟,显存约 3–6 GB)。
|
||||
PYTHONUTF8=1 python -m ml.eval.diag_cl --ckpt ml/runs/base/last.pt --n 256 --out ml/runs/diag_cl.json
|
||||
任务 = val 集前 n 题;老师轨迹按任务 id 从 ml/data/shard_*.jsonl 取(取不到的题不进 H 段,数量记在 H_tasks_missing_teacher)。
|
||||
模型经 ckpt.load_policy 载入(有 EMA 用 EMA;act_scale 随 checkpoint,旧 checkpoint = 1.0)。
|
||||
|
||||
段落:
|
||||
A. 特征口径:val 任务起点(非增广、老师状态)上,Python features() 与数据集 X 的逐组最大差 / 平均差。
|
||||
B. 同一噪声下两份特征得到的动作差(16 步末端关节差,度)。
|
||||
C. 旧部署口径(1 段采样、一致性选择、子目标旧规则 timeout)的失败形态:首撞时刻、碰撞步数、首撞组对、最小目标误差、
|
||||
目标误差进度比、开安全层的到达 / 碰撞 / 刹停。到达 = 时限内到达,碰撞只计时限内(2026-10-19 起)。
|
||||
H(2026-10-19,三份审查 review_closedloop / review_model_train / review_data_actions 的决定性实验):
|
||||
H1 老师 oracle 走完整 rollout()(closed_loop.TeacherOracle;select='first'、use_sub=False):
|
||||
H1_oracle:额外含 oracle 字段——DLS 接管前与老师轨迹的最大关节偏差、到达 / 碰撞、完成时刻 − 老师时长;
|
||||
H1_oracle_noise_<σ>(σ = 0.05 / 0.1 / 0.2,STEP 为单位,每次规划重采):闭环对残差噪声的敏感度。
|
||||
H2 前缀消融:H2_open_loop(r=3 的 val 行,前缀 47:68 原样 vs 置零,执行窗口 3..5 的误差);
|
||||
H2_replan_empty_s{1,4}:队列空了才规划(每次 r=0、提交 3 步),其余同部署。
|
||||
H3 子目标:H3_headsub_cross_s{1,4}(子目标头 + 越过顶点规则 = 新部署默认)、H3_headsub_timeout_s{1,4}(旧规则 = 旧部署)、
|
||||
H3_teachersub_cross_s{1,4} / H3_teachersub_timeout_s{1,4}(老师真实途经点全表)、H3_nosub_s{1,4}(不用子目标)。
|
||||
H4 指标(上面每个闭环变体都输出同一组键,见 metrics());H4_open_loop:按前缀长度 r=0..3 的执行窗口误差。
|
||||
_s1 / _s4 = MeanFlow 1 段(部署)/ 4 段采样。"""
|
||||
import argparse, json, math, os
|
||||
|
||||
import torch
|
||||
|
||||
from ml.common.ckpt import load_policy
|
||||
from ml.common.kin_torch import Kin
|
||||
from ml.common.features import features, STEP, H
|
||||
from ml.data import Samples, load_tasks, read_meta
|
||||
from ml.sim.closed_loop import rollout, goal_err, DT, TeacherOracle
|
||||
from ml.eval.replay_teacher import load_teacher, pad_traj, oracle_raw, oracle_summary
|
||||
|
||||
# 83 维特征分组(与 features.py 拼接顺序一致)
|
||||
GROUPS = [('sincos', 0, 12), ('joint_norm', 12, 15), ('dq', 15, 21), ('p', 21, 24), ('Rq', 24, 30), ('T', 30, 33),
|
||||
('R', 33, 39), ('T-p', 39, 42), ('T-p_clamp', 42, 45), ('mode', 45, 47), ('prefix', 47, 65), ('mask', 65, 68),
|
||||
('sub', 68, 74), ('sub_flag', 74, 75), ('gaps', 75, 83)]
|
||||
|
||||
|
||||
PROG_T = (0.5, 1.0, 2.0) # 目标误差 / 起点误差 的取样时刻(s)
|
||||
K1, K2 = int(round(1.0 / DT)), int(round(2.0 / DT)) # 1 s / 2 s 对应的步数
|
||||
|
||||
|
||||
def pct(v, p):
|
||||
v = sorted(x for x in v if x is not None and x == x)
|
||||
return v[min(len(v) - 1, int(len(v) * p))] if v else None
|
||||
|
||||
|
||||
def qt(t, p):
|
||||
t = t[torch.isfinite(t)]
|
||||
return round(float(t.kthvalue(max(1, min(len(t), int(math.ceil(p * len(t)))))).values), 4) if len(t) else None
|
||||
|
||||
|
||||
def rnd(x, n=4):
|
||||
if isinstance(x, list): return [rnd(v, n) for v in x]
|
||||
return None if x is None or x != x else round(float(x), n)
|
||||
|
||||
|
||||
def pad_len(tr, n):
|
||||
"""轨迹 [B,T,6] 末帧重复补到至少 n 帧(到达后冻结,补帧与冻结等价)。"""
|
||||
if tr.shape[1] >= n: return tr
|
||||
return torch.cat([tr, tr[:, -1:].expand(-1, n - tr.shape[1], -1)], 1)
|
||||
|
||||
|
||||
def batch_stats(kin, ch, per, TR):
|
||||
"""一批的逐题量(CPU)。窗口:幅度 / 净位移 / TCP 用前 1 s,变号率用前 2 s 的 3 步块。"""
|
||||
dev = TR.device
|
||||
stp = torch.tensor(STEP, device=dev)
|
||||
tr = pad_len(per['traj'].to(dev), K2 + 1); te = pad_len(TR, K2 + 1)
|
||||
B = tr.shape[0]
|
||||
amp = lambda x: (x[:, 1:K1 + 1] - x[:, :K1]).abs().sum(1) # [B,6] 前 1 s 关节行程
|
||||
net = lambda x: (x[:, K1] - x[:, 0]).abs() # [B,6] 前 1 s 净位移
|
||||
def tcp(x):
|
||||
p = kin.fk(x[:, :K1 + 1].reshape(-1, 6)).view(B, K1 + 1, 3)
|
||||
return (p[:, 1:] - p[:, :-1]).norm(dim=-1).sum(1), (p[:, -1] - p[:, 0]).norm(dim=-1)
|
||||
def flips(x):
|
||||
idx = torch.arange(0, K2 + 1, 3, device=dev)
|
||||
b = x[:, idx[1:]] - x[:, idx[:-1]] # [B,nb,6] 3 步块增量
|
||||
big = b.abs() > 0.1 * stp
|
||||
v = big[:, 1:] & big[:, :-1]
|
||||
f = v & (torch.sign(b[:, 1:]) != torch.sign(b[:, :-1]))
|
||||
return f.sum(1).float(), v.sum(1).float()
|
||||
tl_p, tn_p = tcp(tr); tl_t, tn_t = tcp(te)
|
||||
fp_, vp_ = flips(tr); ft_, vt_ = flips(te)
|
||||
ex = torch.tensor([t.get('exec_s') or float('nan') for t in ch])
|
||||
return {'ok': per['ok'], 'col': per['col'] > 0, 'isD': torch.tensor([t.get('cat') == 'D' for t in ch]),
|
||||
'pose': torch.tensor([bool(t.get('R')) for t in ch]), 'key': [t.get('cat', '?') + '/' + t.get('mode', '?') for t in ch],
|
||||
'r2': per['goal_ratio_2s'], 'err': per['err'], 'first_hit': per['first_hit'], 'brakes': per['brakes'],
|
||||
'tratio': per['t_done'] / ex,
|
||||
'amp_p': amp(tr).cpu(), 'amp_t': amp(te).cpu(), 'net_p': net(tr).cpu(), 'net_t': net(te).cpu(),
|
||||
'tl_p': tl_p.cpu(), 'tl_t': tl_t.cpu(), 'tn_p': tn_p.cpu(), 'tn_t': tn_t.cpu(),
|
||||
'fl_p': fp_.cpu(), 'fv_p': vp_.cpu(), 'fl_t': ft_.cpu(), 'fv_t': vt_.cpu()}
|
||||
|
||||
|
||||
def metrics(stats):
|
||||
"""H4 指标(每个闭环变体同一组键):
|
||||
reach / collision:cat≠D、时限内(D 类目标不可达,单列在 D);
|
||||
goal_ratio_2s_p50:2 s 时目标误差 / 起点误差;end_goal_err_mm_p50;first_hit_s_p50(撞了的题);t_done_over_teacher_p50(到达的题);
|
||||
joint_amp_ratio[6]:前 1 s 各轴关节行程 Σ|Δq| 策略 / 老师(全体题合计;J6 只算 pose 题,pos 题老师冻结 J6);
|
||||
joint_net_ratio[6]:前 1 s 各轴净位移 |q(1s) − q(0)| 策略 / 老师(同上);j6_pos_amp_deg_p50:pos 题策略 J6 前 1 s 行程(度,老师为 0);
|
||||
tcp_path_ratio_p50 / tcp_net_ratio_p50:前 1 s TCP 路程 / 净位移 策略 / 老师(逐题比,老师 > 1 mm 的题);
|
||||
sign_flip_rate[6] / sign_flip_rate_teacher[6]:前 2 s 相邻 3 步块增量变号的比例(两块都 > 0.1·STEP 才计);
|
||||
by_cat_mode:各 cat/mode 的 n、reach、collision、goal_ratio_2s_p50(不含 D);D:n、collision、end_goal_err_mm_p50、goal_ratio_2s_p50。"""
|
||||
c = {k: (sum((s[k] for s in stats), []) if isinstance(stats[0][k], list) else torch.cat([s[k] for s in stats])) for k in stats[0]}
|
||||
keys = c['key']
|
||||
m = ~c['isD']; pose = c['pose']
|
||||
def jr(a, b):
|
||||
out = []
|
||||
for j in range(6):
|
||||
sel = m & pose if j == 5 else m
|
||||
den = float(b[sel, j].sum()); out.append(float(a[sel, j].sum()) / den if den > 1e-6 else None)
|
||||
return rnd(out)
|
||||
tl_ok = m & (c['tl_t'] > 1); tn_ok = m & (c['tn_t'] > 1)
|
||||
out = {'n': int(m.sum()),
|
||||
'reach': rnd(c['ok'][m].float().mean()) if m.any() else None,
|
||||
'collision': rnd(c['col'][m].float().mean()) if m.any() else None,
|
||||
'goal_ratio_2s_p50': qt(c['r2'][m], .5), 'end_goal_err_mm_p50': qt(c['err'][m], .5),
|
||||
'first_hit_s_p50': qt(c['first_hit'][m], .5), 't_done_over_teacher_p50': qt(c['tratio'][m & c['ok']], .5),
|
||||
'brakes_p50': qt(c['brakes'][m], .5),
|
||||
'joint_amp_ratio': jr(c['amp_p'], c['amp_t']), 'joint_net_ratio': jr(c['net_p'], c['net_t']),
|
||||
'j6_pos_amp_deg_p50': qt(c['amp_p'][m & ~pose, 5], .5),
|
||||
'tcp_path_ratio_p50': qt(c['tl_p'][tl_ok] / c['tl_t'][tl_ok], .5), 'tcp_net_ratio_p50': qt(c['tn_p'][tn_ok] / c['tn_t'][tn_ok], .5),
|
||||
'sign_flip_rate': rnd((c['fl_p'][m].sum(0) / c['fv_p'][m].sum(0).clamp(min=1)).tolist()),
|
||||
'sign_flip_rate_teacher': rnd((c['fl_t'][m].sum(0) / c['fv_t'][m].sum(0).clamp(min=1)).tolist())}
|
||||
by = {}
|
||||
for i, k in enumerate(keys):
|
||||
if not bool(m[i]): continue
|
||||
by.setdefault(k, []).append(i)
|
||||
out['by_cat_mode'] = {k: {'n': len(ix), 'reach': rnd(c['ok'][ix].float().mean()), 'collision': rnd(c['col'][ix].float().mean()),
|
||||
'goal_ratio_2s_p50': qt(c['r2'][ix], .5)} for k, ix in sorted(by.items())}
|
||||
d = c['isD']
|
||||
out['D'] = {'n': int(d.sum()), 'collision': rnd(c['col'][d].float().mean()) if d.any() else None,
|
||||
'end_goal_err_mm_p50': qt(c['err'][d], .5), 'goal_ratio_2s_p50': qt(c['r2'][d], .5)}
|
||||
return out
|
||||
|
||||
|
||||
def teacher_subs(recs, device):
|
||||
"""老师真实途经点全表(path 去掉起终点)→ (subs [B,M,6], n [B]),供 rollout(sub_override=...)。"""
|
||||
vs = [r['path'][1:-1] for r in recs]
|
||||
M = max(1, max(len(v) for v in vs))
|
||||
subs = torch.zeros(len(vs), M, 6, device=device); n = torch.zeros(len(vs), dtype=torch.long, device=device)
|
||||
for i, v in enumerate(vs):
|
||||
if v:
|
||||
x = torch.tensor(v, dtype=torch.float32, device=device); subs[i, :len(v)] = x; subs[i, len(v):] = x[-1]; n[i] = len(v)
|
||||
return subs, n
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument('--ckpt', required=True); ap.add_argument('--samples', default='ml/data/samples')
|
||||
ap.add_argument('--shards', default='ml/data/shard_*.jsonl')
|
||||
ap.add_argument('--n', type=int, default=256); ap.add_argument('--bs', type=int, default=64)
|
||||
ap.add_argument('--device', default='cuda' if torch.cuda.is_available() else 'cpu'); ap.add_argument('--out', default='ml/runs/diag_cl.json')
|
||||
a = ap.parse_args()
|
||||
dev = a.device
|
||||
model, ck = load_policy(a.ckpt, dev)
|
||||
kin = Kin(device=dev)
|
||||
astep = torch.tensor(STEP, device=dev) * model.act_scale # 动作 → 度
|
||||
data_scale = float(read_meta(a.samples).get('act_scale', 1.0))
|
||||
out = {'ckpt': a.ckpt, 'step': ck.get('step'), 'weights': 'ema' if ck.get('ema') is not None else 'model',
|
||||
'act_scale_ckpt': model.act_scale, 'act_scale_data': data_scale}
|
||||
tasks = [t for t in load_tasks([a.samples]) if t['split'] == 'val'][:a.n]
|
||||
by_tid = {t['tid']: t for t in tasks}
|
||||
def save():
|
||||
s = json.dumps(out, indent=1, ensure_ascii=False)
|
||||
os.makedirs(os.path.dirname(a.out) or '.', exist_ok=True)
|
||||
open(a.out + '.tmp', 'w', encoding='utf-8').write(s); os.replace(a.out + '.tmp', a.out)
|
||||
return s
|
||||
|
||||
# ---------- A / B:起点特征口径 ----------
|
||||
vd = Samples(a.samples, device=dev, split='val')
|
||||
tid = vd.TID.long()
|
||||
first = torch.ones_like(tid, dtype=torch.bool); first[1:] = tid[1:] != tid[:-1] # 每题第一行 = t=0 样本
|
||||
idx = [i for i in first.nonzero().flatten().tolist() if int(tid[i]) in by_tid]
|
||||
X0 = vd.X[idx].float()
|
||||
tk = [by_tid[int(tid[i])] for i in idx]
|
||||
q0 = torch.tensor([t['q0'] for t in tk], dtype=torch.float32, device=dev)
|
||||
qx = torch.atan2(X0[:, 0:12:2], X0[:, 1:12:2]) * 180 / torch.pi
|
||||
dqx = torch.remainder(qx - q0 + 180, 360) - 180
|
||||
clean = dqx.abs().max(-1).values < 1e-2 # 未被噪声增广的起点
|
||||
out['A_rows'] = int(clean.sum()); out['A_rows_aug_skipped'] = int((~clean).sum())
|
||||
if clean.any():
|
||||
Xc = X0[clean]; qc = q0[clean]; tc = [t for t, c in zip(tk, clean.tolist()) if c]
|
||||
T = torch.tensor([t['T'] for t in tc], dtype=torch.float32, device=dev)
|
||||
R = torch.tensor([t['R'] if t.get('R') else [0.] * 9 for t in tc], dtype=torch.float32, device=dev)
|
||||
B = qc.shape[0]
|
||||
qpc = torch.tensor([t.get('qp0') or t['q0'] for t in tc], dtype=torch.float32, device=dev)
|
||||
pre = Xc[:, 47:65].view(B, 3, 6) * torch.tensor(STEP, device=dev) # 前缀 / 子目标直接取数据集里的,只比状态相关部分
|
||||
pl = Xc[:, 65:68].sum(-1).round()
|
||||
sf = Xc[:, 74]
|
||||
sub = qc + Xc[:, 68:74] * 180
|
||||
Fp = features(kin, qc, qpc, T, R, pre, pl, sub, sf)
|
||||
out['A_maxdiff'] = {g: (Fp[:, s:e] - Xc[:, s:e]).abs().max().item() for g, s, e in GROUPS}
|
||||
out['A_meandiff'] = {g: (Fp[:, s:e] - Xc[:, s:e]).abs().mean().item() for g, s, e in GROUPS}
|
||||
with torch.no_grad():
|
||||
z = torch.randn(B * 8, H, 6, device=dev, generator=torch.Generator(device=dev).manual_seed(7))
|
||||
a1, _, _ = model.sample_k(Xc, 8, z); a2, _, _ = model.sample_k(Fp, 8, z)
|
||||
d = ((a1.clamp(-1, 1) - a2.clamp(-1, 1)) * astep).sum(2).abs().max(-1).values # [B,K] 16 步末端关节差(度)
|
||||
mag = (a1.clamp(-1, 1) * astep).sum(2).abs().max(-1).values
|
||||
out['B_act_diff_deg'] = {'mean': d.mean().item(), 'p90': d.flatten().kthvalue(max(1, int(0.9 * d.numel()))).values.item(),
|
||||
'ref_motion_deg_mean': mag.mean().item()}
|
||||
|
||||
# ---------- H4 / H2 开环:执行窗口误差(val 行,老师状态) ----------
|
||||
# 闭环稳态 r=3,真正执行的是 act[r:r+3];原来的 ol_fk_mm 是 16 步终点误差,前缀那几步几乎不会错,看不出执行窗口。
|
||||
with torch.no_grad():
|
||||
pl_all = vd.X[:, 65:68].sum(-1).round().long()
|
||||
gsel = torch.Generator(device=dev); gsel.manual_seed(321)
|
||||
def rows_with(r, n=1024):
|
||||
ix = (pl_all == r).nonzero().flatten()
|
||||
return ix[torch.randperm(len(ix), device=dev, generator=gsel)[:n]] if len(ix) else ix
|
||||
def win_err(X, Y, r, steps, k0=None):
|
||||
k0 = r if k0 is None else k0
|
||||
z = torch.randn(X.shape[0] * 8, H, 6, device=dev, generator=torch.Generator(device=dev).manual_seed(11))
|
||||
act, _, _ = model.sample_k(X, 8, z, steps=steps); act = act.float().clamp(-1, 1)
|
||||
e = ((act[:, :, k0:k0 + 3] - Y[:, None, k0:k0 + 3]) * astep).sum(2).abs() # [N,K,6] 窗口累计误差(度)
|
||||
emax = e.max(-1).values.flatten()
|
||||
res = {'n_rows': int(X.shape[0]), 'deg_mean': rnd(emax.mean()), 'deg_p90': qt(emax, .9), 'per_joint_deg_mean': rnd(e.mean((0, 1)).tolist())}
|
||||
if r > 0 and k0 == r:
|
||||
ce = ((act[:, :, :r] - Y[:, None, :r]) * astep).abs().max(-1).values.max(-1).values.flatten()
|
||||
res['prefix_copy_err_deg_mean'] = rnd(ce.mean())
|
||||
return res
|
||||
ol4, ol2 = {}, {}
|
||||
for steps in (1, 4):
|
||||
for r in range(4):
|
||||
ix = rows_with(r)
|
||||
if len(ix): ol4['s%d_r%d' % (steps, r)] = win_err(vd.X[ix].float(), vd.Y[ix].float(), r, steps)
|
||||
ix = rows_with(3)
|
||||
if len(ix):
|
||||
X3 = vd.X[ix].float(); Y3 = vd.Y[ix].float(); X3z = X3.clone(); X3z[:, 47:68] = 0
|
||||
ol2['s%d_prefix_orig' % steps] = win_err(X3, Y3, 3, steps)
|
||||
ol2['s%d_prefix_zeroed' % steps] = win_err(X3z, Y3, 0, steps, k0=3)
|
||||
out['H4_open_loop'] = ol4; out['H2_open_loop'] = ol2
|
||||
del vd
|
||||
if dev.startswith('cuda'): torch.cuda.empty_cache()
|
||||
|
||||
# ---------- C:旧部署口径的闭环失败形态(子目标旧规则,便于与前几轮对比) ----------
|
||||
first_t, col_steps, pairs, min_err, reach = [], [], {}, [], 0
|
||||
prog = {ts: [] for ts in PROG_T}
|
||||
reach_s, brakes_s, col_s = 0, [], 0
|
||||
for i in range(0, len(tasks), a.bs):
|
||||
ch = tasks[i:i + a.bs]
|
||||
with torch.no_grad():
|
||||
_, per = rollout(model, kin, ch, dev, K=8, safety=False, seed=i, return_traj=True, sub_rule='timeout')
|
||||
tr = per['traj'].to(dev) # [B,Tn,6]
|
||||
B, Tn, _ = tr.shape
|
||||
flat = tr.reshape(-1, 6) # 分块 ≤1024 点(同 dagger/collect.py 的显存修复)
|
||||
gm = torch.cat([kin.gaps(flat[s:s + 1024], 0.5)['gm'] for s in range(0, flat.shape[0], 1024)]).view(B, Tn, 49)
|
||||
inlim = (torch.arange(Tn, device=dev)[None] * DT) <= per['limit'].to(dev)[:, None] + 1e-6
|
||||
hit = (gm.min(-1).values <= 0) & inlim # [B,Tn] 只计时限内
|
||||
Tt = torch.tensor([t['T'] for t in ch], dtype=torch.float32, device=dev)
|
||||
Rt = torch.tensor([t['R'] if t.get('R') else [0.] * 9 for t in ch], dtype=torch.float32, device=dev)
|
||||
e, _ = goal_err(kin, tr.reshape(-1, 6), Tt.repeat_interleave(Tn, 0), Rt.repeat_interleave(Tn, 0))
|
||||
e = e.view(B, Tn)
|
||||
for b in range(B):
|
||||
for ts in PROG_T:
|
||||
k = min(Tn - 1, int(round(ts / DT))); prog[ts].append((e[b, k] / e[b, 0].clamp(min=1e-3)).item())
|
||||
for b in range(B):
|
||||
if per['ok'][b]: reach += 1
|
||||
min_err.append(e[b].min().item()); col_steps.append(int(per['col'][b]))
|
||||
h = hit[b].nonzero().flatten()
|
||||
if len(h):
|
||||
k = int(h[0]); first_t.append(k * DT)
|
||||
pr = int(gm[b, k].argmin()); key = '%d-%d' % (pr // 7, pr % 7); pairs[key] = pairs.get(key, 0) + 1
|
||||
rs, ps = rollout(model, kin, ch, dev, K=8, safety=True, seed=i, sub_rule='timeout')
|
||||
reach_s += int(ps['ok'].sum()); col_s += int((ps['col'] > 0).sum()); brakes_s += ps['brakes'].tolist()
|
||||
n = len(tasks)
|
||||
out['C'] = {'n': n, 'reach': reach / n, 'collided': len(first_t) / n,
|
||||
'first_hit_s': {'p10': pct(first_t, .1), 'p50': pct(first_t, .5), 'p90': pct(first_t, .9),
|
||||
'within_0.1s': sum(t <= 0.1 for t in first_t) / max(1, len(first_t))},
|
||||
'col_steps': {'p50': pct(col_steps, .5), 'p90': pct(col_steps, .9)},
|
||||
'first_hit_pair(group a-b)': dict(sorted(pairs.items(), key=lambda kv: -kv[1])[:8]),
|
||||
'min_goal_err_mm': {'p10': pct(min_err, .1), 'p50': pct(min_err, .5), 'p90': pct(min_err, .9)},
|
||||
'safety_on': {'reach': reach_s / n, 'collided': col_s / n, 'brakes_p50': pct(brakes_s, .5)}}
|
||||
out['C']['goal_err_ratio_p50'] = {'%.1fs' % ts: pct(v, .5) for ts, v in prog.items()} # <1 = 在靠近目标
|
||||
print(save(), flush=True)
|
||||
|
||||
# ---------- H:闭环变体(同一批 val 题,老师轨迹按 id 取) ----------
|
||||
teach = load_teacher(a.shards, [t['id'] for t in tasks])
|
||||
tH = [t for t in tasks if t['id'] in teach]
|
||||
out['H_tasks'] = len(tH); out['H_tasks_missing_teacher'] = len(tasks) - len(tH)
|
||||
|
||||
def variant(oracle_sigma=None, teacher_sub=False, **kw):
|
||||
stats, raws = [], []
|
||||
for i in range(0, len(tH), a.bs):
|
||||
ch = tH[i:i + a.bs]; recs = [teach[t['id']] for t in ch]
|
||||
TR = pad_traj([r['traj'] for r in recs], dev)
|
||||
kw2 = dict(kw)
|
||||
if oracle_sigma is not None:
|
||||
kw2.update(oracle=TeacherOracle(TR, act_scale=data_scale, sigma=oracle_sigma), select='first', use_sub=False)
|
||||
if teacher_sub: kw2['sub_override'] = teacher_subs(recs, dev)
|
||||
with torch.no_grad():
|
||||
_, per = rollout(None if oracle_sigma is not None else model, kin, ch, dev, K=8, seed=i, return_traj=True, **kw2)
|
||||
stats.append(batch_stats(kin, ch, per, TR))
|
||||
if oracle_sigma is not None: raws.append(oracle_raw(per, TR, [len(r['traj']) for r in recs], ch))
|
||||
del per
|
||||
res = metrics(stats)
|
||||
if oracle_sigma is not None and oracle_sigma == 0: res['oracle'] = oracle_summary(raws)
|
||||
if dev.startswith('cuda'): torch.cuda.empty_cache()
|
||||
return res
|
||||
|
||||
plan = [('H1_oracle', dict(oracle_sigma=0.0))]
|
||||
plan += [('H1_oracle_noise_%g' % s, dict(oracle_sigma=s)) for s in (0.05, 0.1, 0.2)]
|
||||
for st in (1, 4):
|
||||
plan += [('H3_headsub_cross_s%d' % st, dict(steps=st, sub_rule='cross')),
|
||||
('H3_headsub_timeout_s%d' % st, dict(steps=st, sub_rule='timeout')),
|
||||
('H3_teachersub_cross_s%d' % st, dict(steps=st, sub_rule='cross', teacher_sub=True)),
|
||||
('H3_teachersub_timeout_s%d' % st, dict(steps=st, sub_rule='timeout', teacher_sub=True)),
|
||||
('H3_nosub_s%d' % st, dict(steps=st, use_sub=False)),
|
||||
('H2_replan_empty_s%d' % st, dict(steps=st, sub_rule='cross', replan='empty'))]
|
||||
for name, kw in plan:
|
||||
out[name] = variant(**kw)
|
||||
save(); print('[diag] %s 完成' % name, flush=True)
|
||||
print(save(), flush=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,136 @@
|
||||
#!/usr/bin/env python
|
||||
"""闭环碰撞定位:第一次碰撞发生在第几秒、碰撞时最近的是哪两条连杆(J几-J几)。
|
||||
|
||||
用法(仓库根目录):
|
||||
PYTHONUTF8=1 python -m ml.eval.diag_collision --ckpt ml/runs/base/last.pt --n 512 --out ml/runs/diag_collision.json
|
||||
|
||||
口径与 closed_loop.rollout 一致:每步同时检查当前状态 q 与相邻状态中点 mid 的间隙(cap=0.5 mm);
|
||||
间隙组对索引 = gA*7+gB(gA<gB,g 为连杆组 0..5 → J1..J6)。
|
||||
"""
|
||||
import argparse, json, math, os, sys
|
||||
from collections import Counter
|
||||
|
||||
import torch
|
||||
|
||||
from ml.common.kin_torch import Kin
|
||||
from ml.data import load_tasks
|
||||
from ml.model.policy import build
|
||||
from ml.sim.closed_loop import DT, rollout
|
||||
|
||||
|
||||
def pair_names():
|
||||
# kin.json 有 7 个连杆组:g=0 是固定基座/立柱,g=1..6 是 J1..J6
|
||||
def nm(g):
|
||||
return '基座' if g == 0 else f'J{g}'
|
||||
return {a * 7 + b: f'{nm(a)}-{nm(b)}' for a in range(7) for b in range(a + 1, 7)}
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument('--ckpt', required=True)
|
||||
ap.add_argument('--samples', default='ml/data/samples')
|
||||
ap.add_argument('--split', default='val')
|
||||
ap.add_argument('--n', type=int, default=512)
|
||||
ap.add_argument('--bs', type=int, default=64)
|
||||
ap.add_argument('--chunk', type=int, default=64, help='回放轨迹分块步数(显存小就调小)')
|
||||
ap.add_argument('--K', type=int, default=8)
|
||||
ap.add_argument('--cap', type=float, default=0.5)
|
||||
ap.add_argument('--device', default='cuda' if torch.cuda.is_available() else 'cpu')
|
||||
ap.add_argument('--out', default=None)
|
||||
a = ap.parse_args()
|
||||
|
||||
ck = torch.load(a.ckpt, map_location=a.device)
|
||||
model = build(ck['cfg']).to(a.device)
|
||||
model.load_state_dict(ck['model'])
|
||||
model.eval()
|
||||
kin = Kin(device=a.device)
|
||||
if ck.get('fp') and ck['fp'] != kin.fp:
|
||||
sys.exit(f'checkpoint 指纹 {ck["fp"]} 与 kin.json {kin.fp} 不符')
|
||||
tasks = [t for t in load_tasks([a.samples]) if t['split'] == a.split][:a.n]
|
||||
PN = pair_names()
|
||||
rows = []
|
||||
tau = [0, 0.5, 1.0, 2.0, 3.0, 5.0, 1e9]
|
||||
hist = Counter()
|
||||
pair_first = Counter(); pair_steps = Counter()
|
||||
with torch.no_grad():
|
||||
for i in range(0, len(tasks), a.bs):
|
||||
chunk = tasks[i:i + a.bs]
|
||||
res, per = rollout(model, kin, chunk, a.device, K=a.K, safety=False, seed=i, return_traj=True)
|
||||
traj = per['traj'].to(a.device) # [B,T,6]
|
||||
B, Tn, _ = traj.shape
|
||||
first_t = torch.full((B,), float('nan'), device=a.device)
|
||||
first_pair = torch.full((B,), -1, dtype=torch.long, device=a.device)
|
||||
first_gap = torch.full((B,), float('nan'), device=a.device)
|
||||
min_gap = torch.full((B,), float('inf'), device=a.device)
|
||||
col_steps = torch.zeros(B, device=a.device)
|
||||
for k in range(Tn):
|
||||
qk = traj[:, k]
|
||||
mid = 0.5 * (traj[:, max(0, k - 1)] + qk)
|
||||
pts = torch.stack([qk, mid], 1).reshape(2 * B, 6) # 每帧只喂 2B 个点(与 rollout 同量级;一次 2048 点会爆显存)
|
||||
gm = kin.gaps(pts, a.cap)['gm'].view(B, 2, 49)
|
||||
gmin, gpair = gm.min(-1) # [B,2]
|
||||
g1 = gmin.min(1).values
|
||||
min_gap = torch.minimum(min_gap, g1)
|
||||
newly = (g1 <= 0) & torch.isnan(first_t)
|
||||
if bool(newly.any()):
|
||||
for b in newly.nonzero().flatten().tolist():
|
||||
pos = 0 if float(gmin[b, 0]) <= 0 else 1
|
||||
first_t[b] = k * DT
|
||||
first_pair[b] = gpair[b, pos]
|
||||
first_gap[b] = gmin[b, pos]
|
||||
if not bool(torch.isnan(first_t).any()):
|
||||
break
|
||||
for b, t in enumerate(chunk):
|
||||
ft = float(first_t[b])
|
||||
fp = int(first_pair[b])
|
||||
name = PN.get(fp, '—')
|
||||
if not math.isnan(ft):
|
||||
pair_first[name] += 1
|
||||
for j in range(len(tau) - 1):
|
||||
if tau[j] <= ft < tau[j + 1]:
|
||||
hist[f'{tau[j]}–{tau[j + 1]} s' if tau[j + 1] < 1e8 else '> 5 s'] += 1
|
||||
break
|
||||
rows.append({
|
||||
'tid': t.get('tid'), 'cat': t.get('cat'), 'mode': t.get('mode'),
|
||||
'collided': bool(per['col'][b] > 0),
|
||||
'first_collision_s': None if math.isnan(ft) else round(ft, 2),
|
||||
'first_pair': None if name == '—' else name, 'first_pair_idx': None if fp < 0 else fp,
|
||||
'first_gap_mm': None if math.isnan(float(first_gap[b])) else round(float(first_gap[b]), 3),
|
||||
'min_gap_mm': round(float(min_gap[b]), 3),
|
||||
'collision_steps': int(per['col'][b]),
|
||||
})
|
||||
print(f'[diag] {min(i + a.bs, len(tasks))}/{len(tasks)} 已回放', flush=True)
|
||||
|
||||
collided = [r for r in rows if r['collided']]
|
||||
def pct(v, p):
|
||||
v = sorted(v); return v[min(len(v) - 1, int(len(v) * p))] if v else None
|
||||
times = [r['first_collision_s'] for r in collided]
|
||||
out = {
|
||||
'ckpt': a.ckpt, 'step': ck.get('step'), 'n': len(rows), 'n_collided': len(collided),
|
||||
'collision_rate': len(collided) / max(1, len(rows)),
|
||||
'first_time_s': {'p10': pct(times, .1), 'p50': pct(times, .5), 'p90': pct(times, .9)},
|
||||
'first_time_hist': dict(hist),
|
||||
'first_pair_top': pair_first.most_common(),
|
||||
'min_gap_mm_p10/p50': [pct([r['min_gap_mm'] for r in rows], .1), pct([r['min_gap_mm'] for r in rows], .5)],
|
||||
'by_cat': {},
|
||||
'tasks': rows,
|
||||
}
|
||||
for k in sorted({f"{r['cat']}/{r['mode']}" for r in rows}):
|
||||
g = [r for r in rows if f"{r['cat']}/{r['mode']}" == k]
|
||||
out['by_cat'][k] = {'n': len(g), 'first_t_p50': pct([r['first_collision_s'] for r in g if r['collided']], .5),
|
||||
'first_pairs': Counter([r['first_pair'] for r in g if r['first_pair']]).most_common(3)}
|
||||
s = json.dumps(out, indent=1, ensure_ascii=False)
|
||||
if a.out:
|
||||
open(a.out + '.tmp', 'w', encoding='utf-8').write(s); os.replace(a.out + '.tmp', a.out)
|
||||
print('== 汇总 ==')
|
||||
print('题数 %d,撞过 %d(%.1f%%)' % (len(rows), len(collided), 100 * out['collision_rate']))
|
||||
print('首次碰撞时间 p10/p50/p90 = %s / %s / %s s' % (
|
||||
out['first_time_s']['p10'], out['first_time_s']['p50'], out['first_time_s']['p90']))
|
||||
print('首次碰撞时间分布:', out['first_time_hist'])
|
||||
print('首次碰撞的连杆对 Top:', pair_first.most_common(8))
|
||||
print('全程最小间隙 p10/p50 = %s / %s mm' % tuple(out['min_gap_mm_p10/p50']))
|
||||
if a.out: print('已写', a.out)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,128 @@
|
||||
#!/usr/bin/env python
|
||||
"""闭环末位误差诊断(只读,手动运行;不在流水线里)。
|
||||
|
||||
回答一个问题:策略"差一点没进 1 mm 门"还是"根本没朝目标走"。
|
||||
输出每题:
|
||||
- 最终位置误差 err_mm / 最终姿态误差 oerr_deg
|
||||
- 全程最小位置误差 min_err_mm、最小综合误差时是否进过 5 mm & 5° 窗口(DLS 接管门槛)
|
||||
- done(是否到达)、碰撞步数、用时
|
||||
以及按 cat/mode 分组的汇总与分位数。
|
||||
|
||||
用法(仓库根目录):
|
||||
PYTHONUTF8=1 python ml/eval/diag_tail.py --ckpt ml/runs/base/last.pt --n 512 --out ml/runs/diag_tail.json
|
||||
"""
|
||||
import argparse, json, math, os, sys
|
||||
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))))
|
||||
|
||||
import torch
|
||||
|
||||
|
||||
def pct(v, p):
|
||||
if not v:
|
||||
return None
|
||||
v = sorted(v)
|
||||
return v[min(len(v) - 1, int(len(v) * p))]
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument('--ckpt', required=True)
|
||||
ap.add_argument('--samples', default='ml/data/samples')
|
||||
ap.add_argument('--tasks', default=None, help='任务 jsonl(覆盖 --samples/--split)')
|
||||
ap.add_argument('--split', default='val')
|
||||
ap.add_argument('--n', type=int, default=512)
|
||||
ap.add_argument('--bs', type=int, default=64, help='每批题数(显存小就调小)')
|
||||
ap.add_argument('--K', type=int, default=8)
|
||||
ap.add_argument('--device', default='cuda' if torch.cuda.is_available() else 'cpu')
|
||||
ap.add_argument('--out', default=None)
|
||||
a = ap.parse_args()
|
||||
|
||||
from ml.common.kin_torch import Kin, D2R
|
||||
from ml.data import load_tasks
|
||||
from ml.model.policy import build
|
||||
from ml.sim.closed_loop import rollout, goal_err
|
||||
|
||||
from ml.common.ckpt import load_policy # 2026-10-19:EMA 权重 + act_scale
|
||||
model, ck = load_policy(a.ckpt, a.device)
|
||||
kin = Kin(device=a.device)
|
||||
if ck.get('fp') and ck['fp'] != kin.fp:
|
||||
sys.exit('checkpoint 的碰撞模型指纹 %s 与当前 kin.json %s 不符' % (ck['fp'], kin.fp))
|
||||
|
||||
if a.tasks:
|
||||
tasks = [json.loads(l) for l in open(a.tasks, encoding='utf-8') if l.strip()]
|
||||
else:
|
||||
tasks = [t for t in load_tasks([a.samples]) if t['split'] == a.split]
|
||||
tasks = tasks[:a.n]
|
||||
|
||||
per_task = []
|
||||
with torch.no_grad():
|
||||
for i in range(0, len(tasks), a.bs):
|
||||
chunk = tasks[i:i + a.bs]
|
||||
res, per = rollout(model, kin, chunk, a.device, K=a.K, safety=False, seed=i, return_traj=True)
|
||||
traj = per['traj'] # [B, T, 6]
|
||||
B, Tn, _ = traj.shape
|
||||
Tt = torch.tensor([t['T'] for t in chunk], dtype=torch.float32, device=a.device)
|
||||
Rt = torch.tensor([t['R'] if t.get('R') else [0.] * 9 for t in chunk], dtype=torch.float32, device=a.device)
|
||||
min_e = torch.full((B,), float('inf'), device=a.device)
|
||||
min_o = torch.full((B,), float('inf'), device=a.device)
|
||||
entered = torch.zeros(B, dtype=torch.bool, device=a.device) # 曾同时满足 e<5mm & (无姿态要求或 oe<5°)
|
||||
step = 256
|
||||
for t0 in range(0, Tn, step):
|
||||
q = traj[:, t0:t0 + step].reshape(-1, 6)
|
||||
n = q.shape[0] // B
|
||||
e, oe = goal_err(kin, q, Tt.repeat_interleave(n, 0), Rt.repeat_interleave(n, 0))
|
||||
e = e.view(B, n); oe = oe.view(B, n)
|
||||
min_e = torch.minimum(min_e, e.min(1).values)
|
||||
min_o = torch.minimum(min_o, oe.min(1).values)
|
||||
pose = (Rt.abs().sum(-1) > 0)[:, None]
|
||||
window = (e < 5) & (~pose | (oe < 5))
|
||||
entered |= window.any(1)
|
||||
done = per['done'].to(a.device); col = per['col'].to(a.device); td = per['t_done'].to(a.device)
|
||||
end_e = per['err'].to(a.device)
|
||||
# 末态姿态误差:traj 最后一帧
|
||||
qend = traj[:, -1]
|
||||
ee, eo = goal_err(kin, qend, Tt, Rt)
|
||||
for j, t in enumerate(chunk):
|
||||
per_task.append({
|
||||
'tid': t.get('tid'), 'cat': t.get('cat'), 'mode': t.get('mode'),
|
||||
'err_mm': float(end_e[j]), 'oerr_deg': float(eo[j]),
|
||||
'min_err_mm': float(min_e[j]), 'min_oerr_deg': float(min_o[j]),
|
||||
'entered_5mm5deg': bool(entered[j]),
|
||||
'done': bool(done[j]), 'collision_steps': float(col[j]),
|
||||
't_done_s': None if math.isnan(float(td[j])) else float(td[j]),
|
||||
})
|
||||
print('[diag] %d/%d done=%d entered=%d min_err<=5mm=%d' % (
|
||||
min(i + a.bs, len(tasks)), len(tasks), int(done.sum()), int(entered.sum()),
|
||||
sum(1 for p in per_task if p['min_err_mm'] <= 5)), flush=True)
|
||||
|
||||
def summ(rows):
|
||||
if not rows:
|
||||
return {'n': 0}
|
||||
e = [r['err_mm'] for r in rows]; me = [r['min_err_mm'] for r in rows]
|
||||
o = [r['oerr_deg'] for r in rows]
|
||||
return {'n': len(rows),
|
||||
'done': sum(r['done'] for r in rows),
|
||||
'entered_5mm5deg': sum(r['entered_5mm5deg'] for r in rows),
|
||||
'min_err_mm<=5': sum(1 for v in me if v <= 5),
|
||||
'min_err_mm<=10': sum(1 for v in me if v <= 10),
|
||||
'err_mm_p10/p50/p90': [pct(e, .1), pct(e, .5), pct(e, .9)],
|
||||
'min_err_mm_p10/p50/p90': [pct(me, .1), pct(me, .5), pct(me, .9)],
|
||||
'oerr_deg_p50/p90': [pct(o, .5), pct(o, .9)],
|
||||
'collision_steps_mean': sum(r['collision_steps'] for r in rows) / len(rows)}
|
||||
|
||||
out = {'ckpt': a.ckpt, 'step': ck.get('step'), 'fp': ck.get('fp'), 'n': len(per_task),
|
||||
'overall': summ(per_task),
|
||||
'by_cat_mode': {k: summ([r for r in per_task if '%s/%s' % (r['cat'], r['mode']) == k])
|
||||
for k in sorted({('%s/%s' % (r['cat'], r['mode'])) for r in per_task})},
|
||||
'tasks': per_task}
|
||||
s = json.dumps(out, indent=1, ensure_ascii=False)
|
||||
print(s[:2000])
|
||||
if a.out:
|
||||
open(a.out + '.tmp', 'w', encoding='utf-8').write(s)
|
||||
os.replace(a.out + '.tmp', a.out)
|
||||
print('[diag] 已写', a.out)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
@@ -0,0 +1,144 @@
|
||||
"""老师轨迹回放自检(train 阶段开训前自动跑,约 1–3 分钟):
|
||||
python -m ml.eval.replay_teacher --n 512 --out ml/runs/replay_teacher.json
|
||||
两部分,都过才 pass(pipeline.sh 读 pass 字段与指纹,不过就停 rc 5):
|
||||
1. 运动学口径(2026-10-18):gen 写出的老师 50 Hz 轨迹(keep=true、cat≠D)直接放进 PyTorch 运动学 / 碰撞 / 到达判据:
|
||||
到达率 ≥ 0.95、碰撞率 ≤ 0.01。验证的是「仿真口径(kin.json、间隙、到达门槛)与 JS 认证过的数据一致」。
|
||||
2. 滚动口径(2026-10-19,审查 review_closedloop F1/D1、review_data_actions C1):上面那条完全不经过 rollout(),
|
||||
队列 / 前缀 / 提交 3 步 / qe / 标签 clamp / DLS / done / 时限上的错位它都发现不了。这里让老师当策略(closed_loop.TeacherOracle:
|
||||
每次规划从当前时间下标取老师 16 步增量,按标签口径 clamp)原样走 rollout()(select='first'、use_sub=False):
|
||||
到达(时限内)≥ 0.95、碰撞 ≤ 0.01、DLS 接管前与老师轨迹的最大关节偏差 p99 ≤ 0.1°(shard 轨迹按 1e-3° 取整,标签 clamp 会截掉取整的余量)。
|
||||
输出另带 fp(当前 kin.json)、data_fp(shard 里记录的碰撞模型指纹)、ver=2;pipeline 三者都核对。
|
||||
load_teacher / pad_traj / oracle_raw / oracle_summary 供 ml/eval/diag_cl.py 的 H1 段复用。"""
|
||||
import argparse, glob, json, math, os, sys
|
||||
import torch
|
||||
|
||||
from ml.common.kin_torch import Kin
|
||||
from ml.sim.closed_loop import goal_err, rollout, TeacherOracle, DT
|
||||
|
||||
DEV_TOL_DEG = 0.1
|
||||
|
||||
|
||||
def load_teacher(shards, ids):
|
||||
"""按任务 id 从 shard jsonl 取老师记录(traj、path、exec_s…)。只解析 id 命中的行。"""
|
||||
want, out = set(ids), {}
|
||||
for f in sorted(glob.glob(shards)):
|
||||
for line in open(f, encoding='utf-8'):
|
||||
k = line.find('"id":"')
|
||||
if k < 0: continue
|
||||
rid = line[k + 6:line.find('"', k + 6)]
|
||||
if rid in want and rid not in out:
|
||||
r = json.loads(line)
|
||||
if r.get('traj'): out[rid] = r
|
||||
if len(out) == len(want): break
|
||||
return out
|
||||
|
||||
|
||||
def pad_traj(trajs, device, extra=20):
|
||||
"""[B,Tmax+extra,6]:各题老师轨迹末帧重复补齐(oracle 在轨迹末尾之后给零增量)。"""
|
||||
Tm = max(len(t) for t in trajs) + extra
|
||||
out = torch.zeros(len(trajs), Tm, 6, device=device)
|
||||
for i, t in enumerate(trajs):
|
||||
x = torch.tensor(t, dtype=torch.float32, device=device)
|
||||
out[i, :len(t)] = x; out[i, len(t):] = x[-1]
|
||||
return out
|
||||
|
||||
|
||||
def oracle_raw(per, TR, lens, tasks):
|
||||
"""oracle 滚动的逐题量:DLS 接管前(且到达前、老师轨迹结束前)与老师的最大关节偏差 mdev(度)、时限内到达 ok、
|
||||
时限内碰撞 col、是否计入到达统计(cat≠D)、完成时刻 − 老师时长 dt(s)、是否用过 DLS、2 s 目标误差比。"""
|
||||
tr = per['traj'].to(TR.device); B, Tn, _ = tr.shape
|
||||
L = min(Tn, TR.shape[1])
|
||||
dev = (tr[:, :L] - TR[:, :L]).abs().max(-1).values # [B,L]
|
||||
k = torch.arange(L, device=TR.device)[None]
|
||||
kd = torch.tensor(lens, device=TR.device)[:, None] - 1
|
||||
dls = per['dls_it'].to(TR.device)[:, None]
|
||||
td = per['t_done'].to(TR.device)
|
||||
kdone = torch.where(torch.isfinite(td), (td / DT).round(), torch.full_like(td, 1e9)).long()[:, None]
|
||||
valid = (k <= kd) & ((dls < 0) | (k <= dls)) & (k <= kdone)
|
||||
ex = torch.tensor([t.get('exec_s') or float('nan') for t in tasks])
|
||||
return {'mdev': torch.where(valid, dev, torch.zeros_like(dev)).max(-1).values.cpu(), 'ok': per['ok'], 'col': per['col'] > 0,
|
||||
'reach_m': torch.tensor([t.get('cat') != 'D' for t in tasks]), 'dt': per['t_done'] - ex,
|
||||
'dls': per['dls_it'] >= 0, 'r2': per['goal_ratio_2s']}
|
||||
|
||||
|
||||
def _q(v, p):
|
||||
v = v[torch.isfinite(v)]
|
||||
return float(v.kthvalue(max(1, min(len(v), int(math.ceil(p * len(v)))))).values) if len(v) else None
|
||||
|
||||
|
||||
def oracle_summary(raws):
|
||||
c = {k: torch.cat([r[k] for r in raws]) for k in raws[0]}
|
||||
m = c['reach_m']; ok = c['ok']
|
||||
dt = c['dt'][ok & m]
|
||||
return {'n': int(len(m)), 'n_reach_tasks': int(m.sum()),
|
||||
'reach': float(ok[m].float().mean()) if m.any() else None,
|
||||
'collision': float(c['col'][m].float().mean()) if m.any() else None,
|
||||
'D': {'n': int((~m).sum()), 'collision': float(c['col'][~m].float().mean()) if (~m).any() else None},
|
||||
'max_dev_deg_before_dls': {'p50': _q(c['mdev'], .5), 'p99': _q(c['mdev'], .99), 'max': float(c['mdev'].max())},
|
||||
'dls_used': float(c['dls'].float().mean()),
|
||||
'finish_minus_teacher_s': {'p50': _q(dt, .5), 'p90': _q(dt, .9), 'max': float(dt.max()) if len(dt) else None},
|
||||
'goal_ratio_2s_p50': _q(c['r2'], .5)}
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument('--shards', default='ml/data/shard_*.jsonl'); ap.add_argument('--n', type=int, default=512)
|
||||
ap.add_argument('--samples', default='ml/data/samples', help='读打包 meta 的 act_scale(oracle 按同一标签口径 clamp);没有就按 1.0')
|
||||
ap.add_argument('--bs', type=int, default=128)
|
||||
ap.add_argument('--out', default='ml/runs/replay_teacher.json')
|
||||
ap.add_argument('--device', default='cuda' if torch.cuda.is_available() else 'cpu')
|
||||
a = ap.parse_args()
|
||||
kin = Kin(device=a.device)
|
||||
recs = []
|
||||
for f in sorted(glob.glob(a.shards)):
|
||||
for line in open(f, encoding='utf-8'):
|
||||
r = json.loads(line)
|
||||
if r.get('keep') and r.get('traj') and r.get('cat') != 'D': recs.append(r)
|
||||
if len(recs) >= a.n: break
|
||||
if len(recs) >= a.n: break
|
||||
if not recs: sys.exit('没有可回放的老师轨迹(先跑 gen)')
|
||||
# ---- 1. 运动学口径
|
||||
reach = col = 0; worst_gap = []
|
||||
for r in recs:
|
||||
q = torch.tensor(r['traj'], dtype=torch.float32, device=a.device)
|
||||
mid = 0.5 * (q[1:] + q[:-1])
|
||||
g = kin.gaps(torch.cat([q, mid], 0), 0.5)['gm'].min(-1).values
|
||||
hit = bool((g <= 0).any()); col += hit; worst_gap.append(g.min().item())
|
||||
T = torch.tensor([r['T']], dtype=torch.float32, device=a.device)
|
||||
R = torch.tensor([r['R'] if r.get('R') else [0.] * 9], dtype=torch.float32, device=a.device)
|
||||
e, oe = goal_err(kin, q[-1:], T, R)
|
||||
reach += bool((e < 1).item() and (oe < 0.5).item())
|
||||
n = len(recs)
|
||||
kin_ok = reach / n >= 0.95 and col / n <= 0.01
|
||||
# ---- 2. 滚动口径:老师 oracle 走完整 rollout()
|
||||
try:
|
||||
from ml.data import read_meta
|
||||
act_scale = float(read_meta(a.samples).get('act_scale', 1.0)) if os.path.isdir(a.samples) else 1.0
|
||||
except SystemExit:
|
||||
act_scale = 1.0
|
||||
tasks = [{'q0': r['q0'], 'T': r['T'], 'R': r.get('R'), 'exec_s': r.get('exec_s') or (len(r['traj']) - 1) * DT,
|
||||
'travel': r.get('travel_deg'), 'cat': r.get('cat')} for r in recs]
|
||||
raws = []
|
||||
for i in range(0, n, a.bs):
|
||||
tb, rb = tasks[i:i + a.bs], recs[i:i + a.bs]
|
||||
TR = pad_traj([r['traj'] for r in rb], a.device)
|
||||
_, per = rollout(None, kin, tb, a.device, K=1, select='first', use_sub=False, return_traj=True, seed=i,
|
||||
oracle=TeacherOracle(TR, act_scale=act_scale))
|
||||
raws.append(oracle_raw(per, TR, [len(r['traj']) for r in rb], tb))
|
||||
orc = oracle_summary(raws); orc['act_scale'] = act_scale
|
||||
orc_ok = (orc['reach'] or 0) >= 0.95 and (orc['collision'] or 0) <= 0.01 and orc['max_dev_deg_before_dls']['p99'] <= DEV_TOL_DEG
|
||||
data_fps = {r.get('fp') for r in recs}
|
||||
data_fp = data_fps.pop() if len(data_fps) == 1 else 'mixed'
|
||||
res = {'ver': 2, 'n': n, 'reach': reach / n, 'collision': col / n, 'min_gap_mm_p10': sorted(worst_gap)[n // 10], 'kin_pass': kin_ok,
|
||||
'oracle': orc, 'oracle_pass': orc_ok, 'fp': kin.fp, 'data_fp': data_fp,
|
||||
'pass': bool(kin_ok and orc_ok and data_fp in (None, kin.fp))}
|
||||
os.makedirs(os.path.dirname(a.out) or '.', exist_ok=True)
|
||||
json.dump(res, open(a.out, 'w', encoding='utf-8'), indent=1, ensure_ascii=False)
|
||||
print('老师轨迹回放:到达 %.1f%%、碰撞 %.1f%%(%d 条);oracle 走 rollout:到达 %.1f%%、碰撞 %.1f%%、DLS 前最大偏差 p99 %.4f°;指纹 %s' % (
|
||||
100 * res['reach'], 100 * res['collision'], n, 100 * orc['reach'], 100 * orc['collision'],
|
||||
orc['max_dev_deg_before_dls']['p99'], '一致' if data_fp in (None, kin.fp) else '不符'), flush=True)
|
||||
sys.exit(0 if res['pass'] else 5)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
main()
|
||||
+6
-4
@@ -1,7 +1,7 @@
|
||||
"""导出部署包(方案 §8.4):
|
||||
python -m ml.export_onnx --ckpt ml/runs/current.pt --out webapp/models/traj_policy.onnx [--fp16]
|
||||
输出 traj_policy.onnx(输入 feat[1 或 K,83]、z[K,16,6];输出 act[K,16,6] 归一化增量、value[K]、sub[K,13])
|
||||
与 traj_policy.meta.json(碰撞模型指纹、特征版本、维度、enc_once)。导出后用 onnxruntime 与 PyTorch 对拍。
|
||||
与 traj_policy.meta.json(碰撞模型指纹、特征版本、维度、enc_once、act_scale)。导出后用 onnxruntime 与 PyTorch 对拍。
|
||||
2026-10-18 第五轮:改为"编码一次、解码 K 次"(Policy.sample_shared),meta.enc_once=true;运行时据此只送 1 行特征。
|
||||
2026-10-18 第六轮:feat 轴改为动态(1 行 = 主推理编码一次;K 行 = 前瞻逐行编码),meta.feat_dyn=true。
|
||||
第五轮的 enc_once 模型 feat 固定 1 行,运行时前瞻只能逐行推(ik-policy.js ortInfer 已兼容),重新导出即可。
|
||||
@@ -32,8 +32,8 @@ def main():
|
||||
torch.manual_seed(0); ck = {'cfg': json.load(open(a.random)), 'random_init': True}
|
||||
m = build(ck['cfg']); m.eval()
|
||||
elif a.ckpt:
|
||||
ck = torch.load(a.ckpt, map_location='cpu')
|
||||
m = build(ck['cfg']); m.load_state_dict(ck['model']); m.eval()
|
||||
from ml.common.ckpt import load_policy # 2026-10-19:导出 EMA 权重(审查 P1-5:原来导的是原始权重)
|
||||
m, ck = load_policy(a.ckpt, 'cpu')
|
||||
else: raise SystemExit('需要 --ckpt 或 --random <配置>')
|
||||
kin = Kin()
|
||||
if ck.get('fp') and ck['fp'] != kin.fp: raise SystemExit(f"checkpoint 指纹 {ck['fp']} ≠ 当前碰撞模型 {kin.fp}:须重训")
|
||||
@@ -62,8 +62,10 @@ def main():
|
||||
err_rows = max(float(np.abs(o8[i] - r8[i].numpy()).max()) for i in range(3))
|
||||
assert err_rows < 1e-3, f'feat 为 K 行时与 sample 不一致:{err_rows}'
|
||||
err = max(err, err_rows)
|
||||
# 2026-10-19:act_scale = 动作标签归一化倍数,运行时 inc = clamp(act, ±1)·act_scale·STEP(ik-policy.js Runner 的 actScale 读它)
|
||||
meta = {'fp': kin.fp, 'tcp0': kin.tcp0.tolist(), 'head': m.head, 'feat_ver': FEAT_VER, 'f_dim': F_DIM, 'h': H, 'step': STEP, 'fp16': a.fp16,
|
||||
'ckpt': os.path.abspath(a.ckpt) if a.ckpt else None, 'random_init': bool(a.random), 'enc_once': True, 'feat_dyn': True, 'eval': ck.get('eval'), 'max_abs_err_vs_torch': err}
|
||||
'ckpt': os.path.abspath(a.ckpt) if a.ckpt else None, 'random_init': bool(a.random), 'enc_once': True, 'feat_dyn': True,
|
||||
'act_scale': float(ck.get('act_scale', 1.0)), 'eval': ck.get('eval'), 'max_abs_err_vs_torch': err}
|
||||
json.dump(meta, open(a.out.replace('.onnx', '.meta.json'), 'w'), indent=1)
|
||||
print(json.dumps({'onnx': a.out, 'MB': os.path.getsize(a.out) / 1e6, 'max_abs_err_vs_torch': err}))
|
||||
|
||||
|
||||
@@ -5,7 +5,9 @@
|
||||
* 每行一条任务:采样 → 老师 → 距离下界认证过滤 → 局部时间律(50 Hz)→ 写出。
|
||||
* 认证不过的样本也写出(keep=false + 原因),供 stats 统计丢弃率;训练只用 keep=true。
|
||||
* 2026-10-18 断点续写:第 i 条任务的随机数只由 (seed, i) 决定(sampler.reseed);输出文件已存在时保留其中完整的行、
|
||||
* 截掉写了一半的最后一行,从下一条接着写 —— 生成几小时被中断,重跑只补剩下的,不重来、不重复。 */
|
||||
* 截掉写了一半的最后一行,从下一条接着写 —— 生成几小时被中断,重跑只补剩下的,不重来、不重复。
|
||||
* 2026-10-19:--tasks 的任务带 qPrev(DAgger 采集时策略的上一帧)时,记录 qPrev 与起步速度 v0(timelaw.initSpeed),
|
||||
* 时间律从 v0 起步;build_dataset.cjs 用它们做 t=0 样本的 dq 特征与标签。 */
|
||||
'use strict';
|
||||
const fs = require('fs'), path = require('path');
|
||||
const args = {}; for (let i = 2; i < process.argv.length; i += 2) args[process.argv[i].replace(/^--/, '')] = process.argv[i + 1];
|
||||
@@ -19,7 +21,7 @@ const K_CONS = +args.consensus || 0;
|
||||
const GIVEN = args.tasks ? fs.readFileSync(args.tasks, 'utf8').split('\n').filter(Boolean).map(l => JSON.parse(l)) : null;
|
||||
const NT = GIVEN ? GIVEN.length : N;
|
||||
const cert = require('./cert.cjs')(S, {});
|
||||
const { timeParam } = require('./timelaw.cjs');
|
||||
const { timeParam, initSpeed } = require('./timelaw.cjs');
|
||||
const fp = S.modelFingerprint ? S.modelFingerprint() : null;
|
||||
fs.mkdirSync(path.dirname(OUT), { recursive: true });
|
||||
let done = 0;
|
||||
@@ -63,7 +65,11 @@ for (let i = done; i < NT; i++) {
|
||||
if (valid >= 2 && agree * 2 < valid) { rec.keep = false; rec.why = 'consensus'; }
|
||||
}
|
||||
if (t.poseRelaxed && task.mode === 'pose') { rec.mode = 'pos'; rec.R = null; rec.demoted = true; rec.reached = t.err < 1; }
|
||||
if (rec.keep) { const tp = timeParam(t.pts); rec.traj = tp.traj.map(q => q.map(r3)); rec.exec_s = r3(tp.steps * 0.02); }
|
||||
if (rec.keep && task.qPrev) {
|
||||
rec.qPrev = task.qPrev.map(r3);
|
||||
rec.v0 = r3(initSpeed(t.pts, task.qPrev, S.wrap180, S.ORDER.map(id => !!S.LIM[id].cont)));
|
||||
}
|
||||
if (rec.keep) { const tp = timeParam(t.pts, { v0: rec.v0 || 0 }); rec.traj = tp.traj.map(q => q.map(r3)); rec.exec_s = r3(tp.steps * 0.02); }
|
||||
fs.writeSync(fd, JSON.stringify(rec) + '\n');
|
||||
if ((i + 1) % 25 === 0) process.stderr.write(`[seed ${SEED}] ${i + 1}/${NT} ${((performance.now() - t00) / 1000 / (i + 1 - done)).toFixed(2)} s/条\n`);
|
||||
}
|
||||
|
||||
+13
-8
@@ -31,20 +31,25 @@ while :; do
|
||||
done
|
||||
# 判定按产物而不是 wait 的退出码:2026-10-18 实测 28 个种子跑了 5 小时后全部 wait 报失败,
|
||||
# 但每个 .part 都是完整的 1786 行(node 打了 done、mv 却没执行)——Git Bash 长跑后 wait 的返回码不可靠。
|
||||
fail=0
|
||||
# 短产物分两种:网页点「停止」把 node 杀掉(.part 是断点,续跑接着写,不是失败)和 node 自己报错。
|
||||
# 只有日志尾部有报错特征才算失败;143 让网页显示「已停止」而不是「失败(返回码 6)」(monitor.html:508 同一口径)。
|
||||
crashed=0; stopped=0; sn=0
|
||||
for k in "${!pids[@]}"; do
|
||||
s=${seeds[$k]}; f=ml/data/shard_$(printf %04d $s).jsonl
|
||||
wait ${pids[$k]} || true
|
||||
w=0; wait ${pids[$k]} || w=$? # 记下来但不当判据:长跑后 Git Bash 的 wait 返回码不可靠
|
||||
if [ -f "$f" ]; then continue; fi
|
||||
if [ -f "$f.part" ]; then
|
||||
n=$(wc -l < "$f.part")
|
||||
if [ "$n" -ge "$N" ]; then mv "$f.part" "$f" && echo "[gen] 种子 $s 产物完整($n 行),补改名" && continue; fi
|
||||
n=0; [ -f "$f.part" ] && n=$(wc -l < "$f.part")
|
||||
if [ "$n" -ge "$N" ]; then mv "$f.part" "$f" && echo "[gen] 种子 $s 产物完整($n 行),补改名" && continue; fi
|
||||
if tail -n 4 ml/data/log_$s.txt 2>/dev/null | grep -qE '^(Error|.*Error:|.*SyntaxError|.*heap out of memory|Traceback|.*throw )'; then
|
||||
echo "[gen] 种子 $s 失败:.part 只有 $n/$N 行,最后几行:"; tail -n 5 ml/data/log_$s.txt || true
|
||||
crashed=1
|
||||
else
|
||||
echo "[gen] 种子 $s 失败:没有产物,最后几行:"; tail -n 5 ml/data/log_$s.txt || true
|
||||
echo "[gen] 种子 $s 已停止:写好 $n/$N 条,断点保留在 $f.part,点「继续训练」从第 $((n + 1)) 条接着写(子进程返回码 $w)"
|
||||
stopped=1; sn=$((sn + n))
|
||||
fi
|
||||
fail=1
|
||||
done
|
||||
[ $fail = 0 ] || exit 6
|
||||
[ $crashed = 0 ] || exit 6
|
||||
# 全被停止时不跑 stats.cjs:分片还没改名,统计口径没变,重算一遍只会让「补量」看起来原地不动
|
||||
[ $stopped = 0 ] || { echo "[gen] 本轮是被停止的,没有报错:$stopped 个种子共保住 $sn 条,续跑不重复生成"; exit 143; }
|
||||
node ml/gen/stats.cjs ml/data/shard_*.jsonl > ml/data/stats.json
|
||||
cat ml/data/stats.json
|
||||
|
||||
+20
-3
@@ -1,7 +1,10 @@
|
||||
/* 局部可观测时间律(2026-10-17,方案 §4.5)
|
||||
* 折线按"巡航时间" τ 参数化:每段 τ = max_j |Δq_j| / VMAX_j(各轴同步、主导轴满速)。
|
||||
* 归一化速度 v∈[0,1](1 = 巡航):v ← min(1, v + A·dt, sqrt(2·A·剩余 τ)),只依赖当前速度与剩余路程。
|
||||
* 输出 50 Hz(dt = 20 ms)状态序列,末点精确落在终点。 */
|
||||
* 输出 50 Hz(dt = 20 ms)状态序列,末点精确落在终点。
|
||||
* 2026-10-19(审查 review_closedloop F7 / review_data_actions C10):opt.v0 = 起步归一化速度(默认 0 = 静止起步)。
|
||||
* DAgger 重新标注的起点来自策略闭环、本来就在运动,原来一律从静止加速,标签与策略真正到达的 (q, dq) 状态不符。
|
||||
* initSpeed(P, qPrev) 把当前速度投影到首段方向上得到 v0(gen_dataset.cjs 用)。 */
|
||||
'use strict';
|
||||
const VMAX = [60, 60, 90, 90, 120, 180]; // °/s,与 tests/route_bench.cjs 同(轴速比 0.6/0.6/0.9/0.9/1.2/1.8)
|
||||
const A = 5.0; // 1/s²:0→巡航约 0.2 s(a_max 待真机参数)
|
||||
@@ -19,7 +22,7 @@ function timeParam(P, opt) {
|
||||
const u = segT[i] > 0 ? (tau - cum[i]) / segT[i] : 1;
|
||||
return P[i].map((v, j) => v + (P[i + 1][j] - v) * u);
|
||||
};
|
||||
const out = [P[0].slice()], taus = [0]; let tau = 0, v = 0, guard = 0;
|
||||
const out = [P[0].slice()], taus = [0]; let tau = 0, v = Math.max(0, Math.min(1, opt.v0 || 0)), guard = 0;
|
||||
while (tau < total - 1e-9 && guard++ < 100000) {
|
||||
const rem = total - tau;
|
||||
const v1 = Math.min(1, v + a * dt, Math.sqrt(2 * a * Math.max(0, rem - 0.5 * v * dt)) + 1e-9);
|
||||
@@ -30,4 +33,18 @@ function timeParam(P, opt) {
|
||||
}
|
||||
return { traj: out, taus, cum, total, steps: out.length - 1 };
|
||||
}
|
||||
module.exports = { timeParam, VMAX, A, DT };
|
||||
/* 起步速度:w_j = 当前每步增量 / STEP_j(qPrev → P[0]),u_j = 巡航时首段每步增量 / STEP_j(max|u_j| = 1),
|
||||
v0 = clamp(⟨w,u⟩/⟨u,u⟩, 0, 1)——当前速度在首段方向上的分量;反向或首段为零时 0。wrap:连续轴差值的最短弧函数(可缺省)。 */
|
||||
function initSpeed(P, qPrev, wrap, cont) {
|
||||
if (!qPrev || !P || P.length < 2) return 0;
|
||||
const dd = (a, b, j) => (wrap && cont && cont[j]) ? wrap(a - b) : a - b;
|
||||
let seg = 0; for (let j = 0; j < 6; j++) seg = Math.max(seg, Math.abs(P[1][j] - P[0][j]) / VMAX[j]);
|
||||
if (seg < 1e-9) return 0;
|
||||
let wu = 0, uu = 0;
|
||||
for (let j = 0; j < 6; j++) {
|
||||
const u = (P[1][j] - P[0][j]) / VMAX[j] / seg, w = dd(P[0][j], qPrev[j], j) / (VMAX[j] * DT);
|
||||
wu += w * u; uu += u * u;
|
||||
}
|
||||
return Math.max(0, Math.min(1, wu / uu));
|
||||
}
|
||||
module.exports = { timeParam, initSpeed, VMAX, A, DT };
|
||||
|
||||
+2
-2
@@ -12,8 +12,8 @@ from ml.sim.closed_loop import evaluate
|
||||
|
||||
|
||||
def load(path, dev):
|
||||
ck = torch.load(path, map_location=dev)
|
||||
m = build(ck['cfg']).to(dev); m.load_state_dict(ck['model']); return m.eval()
|
||||
from ml.common.ckpt import load_policy # 2026-10-19:EMA 权重 + act_scale
|
||||
return load_policy(path, dev)[0]
|
||||
|
||||
|
||||
def per_task_success(pers):
|
||||
|
||||
+4
-3
@@ -12,13 +12,14 @@ ROUNDS=${1:-6}; STEPS=${2:-1000}; N=${3:-50000}
|
||||
mkdir -p ml/runs/mine
|
||||
[ -f ml/runs/gate_tasks.jsonl ] || $PY -m ml.mining.mine_round --make-gate ${MINE_GATE_N:-2000} --seed 777 --out ml/runs
|
||||
[ -f ml/runs/current.pt ] || cp ml/runs/apg/best.pt ml/runs/current.pt
|
||||
SAMPLES="ml/data/samples"; FIXED=""; small=0
|
||||
BASE_S="ml/data/samples $(ls -d ml/data/samples_dag_* 2>/dev/null | tr '\n' ' ')" # 2026-10-18:BC 正则也用 DAgger 样本
|
||||
SAMPLES="$BASE_S"; FIXED=""; small=0
|
||||
for ((r=1; r<=ROUNDS; r++)); do
|
||||
D=ml/runs/mine/r$r; mkdir -p $D
|
||||
if [ -f $D/gate.json ]; then
|
||||
gain=$($PY -c "import json;print(json.load(open('$D/gate.json'))['gain_pp'])")
|
||||
echo "round $r 已完成(gain ${gain} pp),跳过"; FIXED="--fixed $D/gate_fixed.jsonl"
|
||||
[ -s ml/data/e2/shard_e2_r$r.jsonl ] && SAMPLES="ml/data/samples ml/data/samples_e2"
|
||||
[ -s ml/data/e2/shard_e2_r$r.jsonl ] && SAMPLES="$BASE_S ml/data/samples_e2"
|
||||
if $PY -c "import sys;sys.exit(0 if $gain < 0.5 else 1)"; then small=$((small+1)); else small=0; fi
|
||||
if [ $small -ge 2 ]; then echo "连续 2 轮提升 < 0.5 pp,停止"; break; fi
|
||||
continue
|
||||
@@ -35,7 +36,7 @@ for ((r=1; r<=ROUNDS; r++)); do
|
||||
if [ -s $D/e2_tasks.jsonl ]; then
|
||||
[ -f $E2F ] || { node ml/gen/gen_dataset.cjs --tasks $D/e2_tasks.jsonl --seed $((5000+r)) --out $E2F.part && mv $E2F.part $E2F; }
|
||||
node ml/prep/build_dataset.cjs ml/data/e2/shard_e2_r$r.jsonl ml/data/samples_e2
|
||||
SAMPLES="ml/data/samples ml/data/samples_e2"
|
||||
SAMPLES="$BASE_S ml/data/samples_e2"
|
||||
fi
|
||||
res=""; [ -f ml/runs/apg_r$r/last.pt ] && res="--resume ml/runs/apg_r$r/last.pt"
|
||||
$PY -m ml.apg.apg_finetune --init ml/runs/current.pt --samples $SAMPLES --pool $D/pool.jsonl --out ml/runs/apg_r$r --steps $STEPS $res
|
||||
|
||||
@@ -28,8 +28,8 @@ def main():
|
||||
with open(os.path.join(a.out, a.name), 'w') as f:
|
||||
for t in ts: f.write(json.dumps(t) + '\n')
|
||||
print('gate tasks', len(ts)); return
|
||||
ck = torch.load(a.ckpt, map_location=a.device)
|
||||
model = build(ck['cfg']).to(a.device); model.load_state_dict(ck['model']); model.eval()
|
||||
from ml.common.ckpt import load_policy # 2026-10-19:EMA 权重 + act_scale
|
||||
model, ck = load_policy(a.ckpt, a.device)
|
||||
tasks = sample_tasks(kin, a.n, a.seed)
|
||||
g = torch.Generator(); g.manual_seed(a.seed)
|
||||
cls = {k: [] for k in ('E1', 'E2', 'E3', 'E4', 'E5', 'ok')}
|
||||
|
||||
+31
-14
@@ -1,31 +1,48 @@
|
||||
"""MeanFlow 损失(Geng et al. 2025;MP1 配方:25% 样本 r≠t,自适应损失加权)。
|
||||
约定:t=0 为数据、t=1 为噪声;z_t = (1−t)·x + t·ε,瞬时速度 v = ε − x;网络输出平均速度 u(z,r,t)。
|
||||
目标 u_tgt = v − (t − r)·du/dt,du/dt = v·∂_z u + ∂_t u 用 torch.func.jvp 一次前向得到(切向量 (v, 0, 1))。"""
|
||||
目标 u_tgt = v − (t − r)·du/dt,du/dt = v·∂_z u + ∂_t u 用 torch.func.jvp 一次前向得到(切向量 (v, 0, 1))。
|
||||
2026-10-18 首次正式预训练后改:
|
||||
- 整个 MF 分支(解码器前向 + JVP)在 fp32 下算:目标里含网络自己的导数,bf16 下 JVP 误差直接变成目标噪声;
|
||||
- return_aux=True 时返回 (z, t, r, u),训练用 r==t 且 t 小的样本做"配对去噪估计" x̂ = z − t·u 来加 FK / 平滑损失。
|
||||
原来把 FK 加在 one_step(随机噪声) 上、对齐这一条样本的老师 Y:对任意噪声都回归同一个 Y,最优解是条件均值 E[Y|X],
|
||||
多峰头被训成单峰(实测 last.pt 8 个候选间标准差 0.030,动作幅度 0.151),左绕/右绕取平均正好穿柱。
|
||||
x̂ 由 z_t 里带着的那一条 x 决定(t 小时 z_t 几乎就是 x),不会把不同模式拉到一起。"""
|
||||
import torch
|
||||
from torch.func import jvp
|
||||
|
||||
|
||||
def sample_rt(B, device, dtype, p_neq=0.25, mu=-0.4, sigma=1.0):
|
||||
def sample_rt(B, device, dtype, p_neq=0.25, mu=-0.4, sigma=1.0, p_edge=0.0):
|
||||
"""2026-10-19(审查 review_model_train P0-3):部署只在 (r=0, t=1) 推理一次,而 logit-normal 取 max/min 后
|
||||
P(t>0.95) 约 0.08%、P(r<0.05) 约 0.5%,这个角几乎从未被训练到(一步残差与噪声相关 −0.86 即外推失准)。
|
||||
p_neq > 0 时另以 p_edge 的概率把样本直接放在 (r=0, t=1);预热期(p_neq=0)不放,那时一步映射本就不训。"""
|
||||
t = torch.sigmoid(torch.randn(B, device=device, dtype=dtype) * sigma + mu)
|
||||
r = torch.sigmoid(torch.randn(B, device=device, dtype=dtype) * sigma + mu)
|
||||
t, r = torch.maximum(t, r), torch.minimum(t, r)
|
||||
eq = torch.rand(B, device=device) >= p_neq
|
||||
r = torch.where(eq, t, r)
|
||||
if p_neq > 0 and p_edge > 0:
|
||||
edge = torch.rand(B, device=device) < p_edge
|
||||
t = torch.where(edge, torch.ones_like(t), t); r = torch.where(edge, torch.zeros_like(r), r)
|
||||
return r, t
|
||||
|
||||
|
||||
def meanflow_loss(model, mem, x, p_neq=0.25, ada_p=1.0, ada_c=1e-3):
|
||||
B = x.shape[0]
|
||||
r, t = sample_rt(B, x.device, x.dtype, p_neq)
|
||||
e = torch.randn_like(x)
|
||||
z = (1 - t)[:, None, None] * x + t[:, None, None] * e
|
||||
v = e - x
|
||||
fn = lambda z_, r_, t_: model.decode(mem, z_, r_, t_)
|
||||
u, dudt = jvp(fn, (z, r, t), (v, torch.zeros_like(r), torch.ones_like(t)))
|
||||
u_tgt = (v - (t - r)[:, None, None] * dudt).detach()
|
||||
err = (u - u_tgt).pow(2).mean(dim=(1, 2))
|
||||
w = 1.0 / (err.detach() + ada_c).pow(ada_p)
|
||||
return (w * err).mean(), err.mean().detach()
|
||||
def meanflow_loss(model, mem, x, p_neq=0.25, ada_p=1.0, ada_c=1e-3, return_aux=False, p_edge=0.0):
|
||||
mem, x = mem.float(), x.float()
|
||||
with torch.autocast('cuda', enabled=False):
|
||||
B = x.shape[0]
|
||||
r, t = sample_rt(B, x.device, x.dtype, p_neq, p_edge=p_edge)
|
||||
e = torch.randn_like(x)
|
||||
z = (1 - t)[:, None, None] * x + t[:, None, None] * e
|
||||
v = e - x
|
||||
fn = lambda z_, r_, t_: model.decode(mem, z_, r_, t_)
|
||||
u, dudt = jvp(fn, (z, r, t), (v, torch.zeros_like(r), torch.ones_like(t)))
|
||||
u_tgt = (v - (t - r)[:, None, None] * dudt).detach()
|
||||
err = (u - u_tgt).pow(2).mean(dim=(1, 2))
|
||||
w = 1.0 / (err.detach() + ada_c).pow(ada_p)
|
||||
loss = (w * err).mean()
|
||||
if return_aux:
|
||||
return loss, err.mean().detach(), {'z': z, 't': t, 'r': r, 'u': u}
|
||||
return loss, err.mean().detach()
|
||||
|
||||
|
||||
def one_step(model, mem, e):
|
||||
|
||||
+16
-5
@@ -55,10 +55,17 @@ class DecBlock(nn.Module):
|
||||
return x + self.ff(self.n3(x))
|
||||
|
||||
|
||||
TEMB_WMAX = 50.0 # 时间嵌入最高角频率(rad / 单位 t)
|
||||
|
||||
|
||||
def time_emb(t, dim):
|
||||
"""2026-10-18 首次正式预训练后改(step 21850 闭环到达 0%):原实现照扩散模型写成 sin(1000·t·f),f∈[1e-3, 1],
|
||||
最高角频率 1000 —— t∈[0,1] 上网络对 t 极其敏感,实测 last.pt 的 ∂u/∂t 均方根 6–12,而 v 只有约 1.1,
|
||||
MeanFlow 目标 u_tgt = v − (t−r)·du/dt 被自激的 JVP 项主导:MF 残差 2 万步一直停在 7–8、17k–19k 尖峰到 96、梯度 100% 被裁剪。
|
||||
改为角频率在 [0.5, 50] 几何分布(最快一维在 [0,1] 上约转 8 圈,足够分辨 t,导数量级降 20 倍)。"""
|
||||
half = dim // 2
|
||||
fr = torch.exp(-math.log(1000.0) * torch.arange(half, device=t.device, dtype=t.dtype) / half)
|
||||
a = t[:, None] * 1000.0 * fr[None]
|
||||
w = torch.exp(torch.linspace(math.log(0.5), math.log(TEMB_WMAX), half, device=t.device, dtype=t.dtype))
|
||||
a = t[:, None] * w[None]
|
||||
return torch.cat([torch.sin(a), torch.cos(a)], -1)
|
||||
|
||||
|
||||
@@ -144,8 +151,9 @@ class Policy(nn.Module):
|
||||
x = z - self.decode(m, z, torch.zeros_like(one), one)
|
||||
return x, v.expand(K), s.expand(K, -1)
|
||||
|
||||
def sample_k(self, feat, K, z=None):
|
||||
"""仿真 / APG 用:每个观测 K 条候选 → act [B,K,16,6], value [B], sub [B,13]"""
|
||||
def sample_k(self, feat, K, z=None, steps=1):
|
||||
"""仿真 / APG 用:每个观测 K 条候选 → act [B,K,16,6], value [B], sub [B,13]
|
||||
steps>1(诊断用,2026-10-19):把 [0,1] 均分成 steps 段,x ← x − (t−r)·u(x, r, t) 逐段走;steps=1 即一步生成。"""
|
||||
mem = self.encode(feat)
|
||||
v, s = self.heads(mem)
|
||||
B = feat.shape[0]
|
||||
@@ -158,7 +166,10 @@ class Policy(nn.Module):
|
||||
z = torch.randn(B * K, self.h, 6, device=feat.device, dtype=feat.dtype) if z is None else z.reshape(B * K, self.h, 6)
|
||||
m = mem.repeat_interleave(K, 0)
|
||||
one = torch.ones(B * K, device=feat.device, dtype=feat.dtype)
|
||||
x = z - self.decode(m, z, torch.zeros_like(one), one)
|
||||
x = z
|
||||
for i in range(steps):
|
||||
t, r = 1 - i / steps, 1 - (i + 1) / steps
|
||||
x = x - (t - r) * self.decode(m, x, one * r, one * t)
|
||||
return x.view(B, K, self.h, 6), v, s
|
||||
|
||||
|
||||
|
||||
+209
-62
@@ -56,7 +56,16 @@
|
||||
.step.done .ic { background: #34d39922; border-color: var(--ok); color: var(--ok); }
|
||||
.step.active .ic { background: #5aa9ff22; border-color: var(--accent); color: var(--accent); box-shadow: 0 0 0 4px #5aa9ff1f; }
|
||||
.step.fail .ic { background: #f8717122; border-color: var(--bad); color: var(--bad); }
|
||||
.step.fail .ds { color: var(--bad); font-weight: 650; font-size: 12px; }
|
||||
.alert { border-radius: 12px; padding: 13px 16px; margin: 2px 0 0; border: 1px solid; }
|
||||
.alert.bad { background: linear-gradient(180deg,#7f1d1d66,#450a0a55); border-color: #f8717199; box-shadow: 0 0 0 1px #f8717122, 0 8px 26px #00000066; }
|
||||
.alert.warn { background: #78350f55; border-color: #fbbf2488; }
|
||||
.alert .alerth { font-size: 17px; font-weight: 700; color: #ffe3e3; letter-spacing: .2px; }
|
||||
.alert.warn .alerth { color: #ffeccd; }
|
||||
.alert .alerterr { font: 500 12.5px/1.5 var(--mono); color: #ffb4b4; margin-top: 7px; word-break: break-all; }
|
||||
.alert .alerttip { font-size: 12.5px; color: #f0d9d9cc; margin-top: 7px; }
|
||||
.step .nm { font-size: 12.5px; color: var(--text); white-space: nowrap; } .step .ds { font-size: 11px; color: var(--dim2); white-space: nowrap; }
|
||||
.step .ds.big { font-size: 14px; font-weight: 650; color: var(--accent); }
|
||||
.step.pending .nm { color: var(--dim); }
|
||||
.step.active .ds { color: var(--accent); }
|
||||
|
||||
@@ -85,7 +94,8 @@
|
||||
.ring .c b { font: 700 24px/1 var(--mono); display: block; } .ring .c span { font-size: 11px; color: var(--dim); }
|
||||
.gpurows { flex: 1; display: flex; flex-direction: column; gap: 10px; min-width: 0; }
|
||||
.cores { display: flex; gap: 2px; height: 26px; margin-top: 2px; background: #1b2433; border-radius: 3px; padding: 2px; }
|
||||
.gr .l { display: flex; justify-content: space-between; font-size: 12px; color: var(--dim); margin-bottom: 4px; } .gr .l b { color: var(--text); font: 500 12.5px var(--mono); }
|
||||
.gr .l { display: flex; justify-content: space-between; gap: 8px; font-size: 12px; color: var(--dim); margin-bottom: 4px; } .gr .l b { color: var(--text); font: 500 12.5px var(--mono); }
|
||||
.gr .l span { flex: none; white-space: nowrap; } .gr .l b { white-space: nowrap; overflow: hidden; text-overflow: ellipsis; min-width: 0; } /* 值太长就省略号,别把整行挤成两行 */
|
||||
.mini { height: 6px; background: #0d131d; border-radius: 99px; overflow: hidden; } .mini > div { height: 100%; border-radius: 99px; transition: width .6s; }
|
||||
.gname { font-size: 12px; color: var(--dim2); margin-top: 12px; }
|
||||
|
||||
@@ -127,25 +137,41 @@
|
||||
button.danger { color: var(--bad); border-color: #f8717166; } button.danger:hover { background: #f8717114; }
|
||||
.overall { display: flex; align-items: flex-end; justify-content: space-between; gap: 16px; flex-wrap: wrap; }
|
||||
.overall .now { font-size: 13px; color: var(--dim); } .overall .now b { display: block; color: var(--text); font-size: 18px; font-weight: 650; margin: 2px 0 2px; }
|
||||
.sublabel { font-size: 12.5px; color: var(--dim); display: flex; justify-content: space-between; margin-top: 4px; }
|
||||
.bar.thin { height: 8px; margin: 8px 0 4px; } .bar.ind .fill { width: 30% !important; animation: ind 1.6s ease-in-out infinite; }
|
||||
.sublabel { font-size: 15px; color: var(--dim); display: flex; justify-content: space-between; align-items: baseline; margin-top: 6px; }
|
||||
.sublabel b { font: 700 20px/1 var(--mono); color: var(--text); }
|
||||
.bar.thin { height: 14px; margin: 10px 0 4px; } .bar.ind .fill { width: 30% !important; animation: ind 1.6s ease-in-out infinite; }
|
||||
@keyframes ind { 0% { margin-left: 0; } 50% { margin-left: 70%; } 100% { margin-left: 0; } }
|
||||
.ctlrow { display: flex; gap: 10px; margin-top: 14px; align-items: center; flex-wrap: wrap; }
|
||||
.ctlrow .hint { font-size: 12px; color: var(--dim2); }
|
||||
.term { background: #070a10; border: 1px solid var(--line); border-radius: 10px; padding: 10px 12px; height: 260px; overflow: auto; font: 12px/1.55 var(--mono); color: #c9d4e3; white-space: pre-wrap; word-break: break-all; }
|
||||
.term .e { color: var(--bad); } .term .c { color: var(--accent); } .term .g { color: var(--ok); }
|
||||
.modal { position: fixed; inset: 0; background: #0009; backdrop-filter: blur(3px); display: none; align-items: center; justify-content: center; z-index: 40; }
|
||||
.modal { position: fixed; inset: 0; background: #05080ed9; backdrop-filter: blur(7px); display: none; align-items: flex-start; justify-content: center; z-index: 40; padding: 5vh 16px 24px; overflow: auto; }
|
||||
.modal.show { display: flex; }
|
||||
.dlg { width: min(560px, 92vw); background: #131a27; border: 1px solid var(--line2); border-radius: 16px; padding: 20px 22px; box-shadow: 0 30px 80px #000a; }
|
||||
.dlg h3 { margin: 0 0 4px; font-size: 16px; } .dlg .d { color: var(--dim); font-size: 12.5px; margin-bottom: 14px; }
|
||||
.form { display: grid; grid-template-columns: 1fr 1fr; gap: 12px 14px; }
|
||||
.form label { display: flex; flex-direction: column; gap: 5px; font-size: 12px; color: var(--dim); }
|
||||
.form input, .form select { background: #0f1520; color: var(--text); border: 1px solid var(--line2); border-radius: 8px; padding: 7px 9px; font: 13px var(--mono); }
|
||||
.dlg { width: min(780px, 96vw); background: linear-gradient(180deg, #161f2e 0%, #121926 100%); border: 1px solid var(--line2); border-radius: 18px; box-shadow: 0 40px 100px #000c, 0 1px 0 #ffffff0a inset; overflow: hidden; }
|
||||
.dlg .hd { display: flex; gap: 13px; align-items: flex-start; padding: 18px 22px 15px; border-bottom: 1px solid var(--line); background: #ffffff04; }
|
||||
.dlg .hd .ic { flex: none; width: 34px; height: 34px; border-radius: 10px; display: grid; place-items: center; color: #fff; background: linear-gradient(135deg, var(--accent), var(--accent2)); box-shadow: 0 4px 16px #5aa9ff33; font-size: 13px; }
|
||||
.dlg .hd h3 { margin: 2px 0 0; font-size: 16px; } .dlg .hd .d2 { color: var(--dim2); font-size: 12px; margin-top: 4px; line-height: 1.55; }
|
||||
.dlg .hd .d2 code { color: var(--dim); }
|
||||
.dlg .bd { padding: 16px 22px 4px; }
|
||||
.sec { display: flex; align-items: center; gap: 10px; margin: 17px 0 11px; font-size: 11px; letter-spacing: .1em; color: var(--dim2); }
|
||||
.bd .sec:first-child { margin-top: 2px; }
|
||||
.sec:after { content: ''; flex: 1; height: 1px; background: var(--line); }
|
||||
.form { display: grid; grid-template-columns: 1fr 1fr; gap: 13px 16px; }
|
||||
.form label { display: flex; flex-direction: column; gap: 6px; font-size: 12px; color: var(--dim); min-width: 0; }
|
||||
.form .hint { font-size: 11px; color: var(--dim2); line-height: 1.45; }
|
||||
.form input, .form select { width: 100%; background: #0c1320; color: var(--text); border: 1px solid var(--line2); border-radius: 9px; padding: 9px 11px; font: 13px/1.35 var(--mono); outline: none; transition: border-color .12s, box-shadow .12s, background .12s; }
|
||||
.form input:hover, .form select:hover { border-color: #3d4c68; }
|
||||
.form input:focus, .form select:focus { border-color: var(--accent); box-shadow: 0 0 0 3px #5aa9ff22; background: #0e1626; }
|
||||
.form select { appearance: none; -webkit-appearance: none; padding-right: 32px; background-image: url("data:image/svg+xml;utf8,<svg xmlns='http://www.w3.org/2000/svg' width='12' height='8' viewBox='0 0 12 8'><path d='M1 1.5l5 5 5-5' fill='none' stroke='%238c99ad' stroke-width='1.6' stroke-linecap='round' stroke-linejoin='round'/></svg>"); background-repeat: no-repeat; background-position: right 11px center; }
|
||||
.form select:disabled { opacity: .4; cursor: not-allowed; background-color: #0a0f18; }
|
||||
.form .full { grid-column: 1 / -1; }
|
||||
.ckline { margin-top: 12px; font-size: 12px; color: var(--dim); } .ckline.bad { color: var(--bad); } .ckline .warn { color: var(--warn); }
|
||||
.demobar { background: #fbbf2418; border: 1px solid #fbbf2455; color: var(--warn); border-radius: 10px; padding: 8px 12px; margin: 10px 0; font-size: 13px; }
|
||||
.dlg .acts { display: flex; justify-content: flex-end; gap: 10px; margin-top: 18px; }
|
||||
.dlg .warnbox { margin-top: 12px; font-size: 12px; color: var(--warn); background: #fbbf2410; border: 1px solid #fbbf2433; border-radius: 8px; padding: 8px 10px; display: none; }
|
||||
.dlg .acts { display: flex; align-items: center; justify-content: flex-end; gap: 10px; padding: 16px 22px 19px; }
|
||||
.dlg .acts .sp { flex: 1; }
|
||||
button.ghost { background: transparent; }
|
||||
button.ghost:hover { background: #ffffff0a; border-color: #3d4c68; }
|
||||
.dlg .warnbox { margin: 15px 0 0; font-size: 12.5px; line-height: 1.6; color: var(--warn); background: #fbbf240d; border: 1px solid #fbbf2430; border-left: 3px solid #fbbf24; border-radius: 10px; padding: 10px 13px; display: none; }
|
||||
.toast { position: fixed; bottom: 22px; left: 50%; transform: translateX(-50%); background: #1b2433; border: 1px solid var(--line2); padding: 9px 16px; border-radius: 10px; font-size: 13px; z-index: 50; display: none; box-shadow: 0 10px 30px #0008; }
|
||||
@media (max-width: 900px) { .hero { grid-template-columns: 1fr; } .charts { grid-template-columns: 1fr; } main { padding: 14px; } }
|
||||
</style>
|
||||
@@ -168,6 +194,7 @@
|
||||
</header>
|
||||
|
||||
<main>
|
||||
<div id="alert"></div>
|
||||
<section class="card"><h2><span>流水线</span><span class="r" id="stepnote"></span></h2><div class="steps" id="steps"></div></section>
|
||||
|
||||
<section class="hero">
|
||||
@@ -192,22 +219,59 @@
|
||||
<div class="tip" id="tip"></div>
|
||||
<div class="toast" id="toast"></div>
|
||||
<div class="modal" id="modal"><div class="dlg">
|
||||
<h3>开始训练</h3>
|
||||
<div class="d">后台执行 <code>bash ml/pipeline.sh all</code>,从对拍一路跑到验收。已完成的步骤自动跳过;训练每 10 分钟存档、生成按条续写,断了再点一次就从断点接着跑。网页和控制台窗口都可以关,训练不受影响。</div>
|
||||
<div class="form">
|
||||
<label>运行方式<select id="fMode"><option value="resume">续跑:跳过已完成的步骤</option><option value="fresh">从头重跑(旧产物移到 ml/archive/)</option></select></label>
|
||||
<label>动作头 CFG<select id="fCfg"><option value="ml/configs/base.json">MeanFlow(base,M5 不过自动转 WTA)</option><option value="ml/configs/wta.json">WTA(wta)</option><option value="ml/configs/small.json">小模型 d=128(R4 不过时用)</option></select></label>
|
||||
<label>共识过滤 CONSENSUS<select id="fCons"><option value="">按老师门禁自动</option><option value="0">关</option><option value="1">开(生成约慢 5 倍)</option></select></label>
|
||||
<label>任务数 N_TASKS<input id="fN" value="50000"></label>
|
||||
<label>生成进程数 NPROC<input id="fP" placeholder="默认 = CPU 核数"></label>
|
||||
<label>目标样本 TARGET_SAMPLES<input id="fT" value="1700000"></label>
|
||||
<label>挖错轮数 ROUNDS<input id="fR" value="6"></label>
|
||||
<label>老师间隙下限 MIN_GAP_MM<input id="fG" value="3"></label>
|
||||
<div class="hd">
|
||||
<div class="ic">▶</div>
|
||||
<div>
|
||||
<h3 id="mdTitle">开始训练</h3>
|
||||
<div class="d2">一键跑 <code>bash ml/pipeline.sh all</code>:已完成的步骤自动跳过,训练每 10 分钟存档、生成按条续写,断了再点一次就从断点接着跑。网页和控制台窗口都可以关,训练不受影响。</div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="bd">
|
||||
<div class="sec">运行方式</div>
|
||||
<div class="form">
|
||||
<label>运行方式<select id="fMode"><option value="resume">续跑:跳过已完成的步骤</option><option value="from">从指定阶段重训(该阶段及之后全部重做)</option><option value="fresh">全部重跑(数据也重新生成)</option></select><span class="hint">续跑 = 从上次断点接着来;重训 = 把该阶段及之后的旧产物移进 ml/archive/(不删)</span></label>
|
||||
<label>起始阶段<select id="fFrom"></select><span class="hint">按流水线顺序排列;选「预训练」只重训模型:数据、打包样本、一致性门禁都保留</span></label>
|
||||
</div>
|
||||
<div class="sec">模型与数据</div>
|
||||
<div class="form">
|
||||
<label>动作头 CFG<select id="fCfg"><option value="ml/configs/base.json">MeanFlow(base,M5 不过自动转 WTA)</option><option value="ml/configs/wta.json">WTA(wta)</option><option value="ml/configs/small.json">小模型 d=128(R4 不过时用)</option></select></label>
|
||||
<label>共识过滤 CONSENSUS<select id="fCons"><option value="">按老师门禁自动</option><option value="0">关</option><option value="1">开(生成约慢 5 倍)</option></select></label>
|
||||
<label>任务数 N_TASKS<input id="fN" value="50000"></label>
|
||||
<label>生成进程数 NPROC<input id="fP" placeholder="默认 = CPU 核数"></label>
|
||||
<label>目标样本 TARGET_SAMPLES<input id="fT" value="2000000"></label>
|
||||
<label>老师间隙下限 MIN_GAP_MM<input id="fG" value="3"></label>
|
||||
<label>挖错轮数 ROUNDS<input id="fR" value="6"></label>
|
||||
</div>
|
||||
<div class="warnbox" id="fWarn"></div>
|
||||
</div>
|
||||
<div class="acts">
|
||||
<span class="sp"></span>
|
||||
<button class="ghost" id="fCancel">取消</button>
|
||||
<button id="fGo" class="primary">▶ 开始</button>
|
||||
</div>
|
||||
<div class="warnbox" id="fWarn"></div>
|
||||
<div class="acts"><button id="fCancel">取消</button><button id="fGo" class="primary">▶ 开始</button></div>
|
||||
</div></div>
|
||||
|
||||
<script>
|
||||
/* 兜底:正文脚本一旦有语法错误或抛异常,整页会空白(2026-10-18 真踩过:一行字符串写断 → 页面什么都不显示)。
|
||||
这段在正文之前跑,负责"坏也坏得看得见":看门狗 + 全局错误显示。 */
|
||||
window.__boot = { ok: false };
|
||||
function __pageFail(msg) {
|
||||
const m = document.querySelector('main') || document.body;
|
||||
let d = document.getElementById('pagefail');
|
||||
if (!d) {
|
||||
d = document.createElement('div'); d.id = 'pagefail';
|
||||
d.style.cssText = 'margin:10px 24px 0;padding:14px 16px;border-radius:12px;background:#7f1d1dcc;border:1px solid #f87171;' +
|
||||
'color:#ffe3e3;font:600 15px/1.6 system-ui;position:sticky;top:8px;z-index:99;box-shadow:0 8px 26px #00000088';
|
||||
m.prepend(d);
|
||||
}
|
||||
d.textContent = msg;
|
||||
}
|
||||
window.addEventListener('error', e => __pageFail('页面脚本出错:' + (e.message || '未知错误') +
|
||||
'(' + (e.filename || '').split('/').pop() + ':' + (e.lineno || '?') + ')— 面板可能不完整,但训练本身没受影响;按 F12 看 Console'));
|
||||
setTimeout(() => { if (!window.__boot.ok) __pageFail('页面脚本没有启动(多半是 ml/monitor.html 有语法错误):卡片空白不是训练出问题。' +
|
||||
'F12 → Console 看第一条报错,或把该文件回滚到上一个提交。'); }, 3000);
|
||||
</script>
|
||||
|
||||
<script>
|
||||
'use strict';
|
||||
/* =====================================================================
|
||||
@@ -221,7 +285,7 @@ const $ = id => document.getElementById(id);
|
||||
const isNum = v => typeof v === 'number' && isFinite(v);
|
||||
const fmt = (v, d = 3) => !isNum(v) ? '—' : (Math.abs(v) >= 1000 ? Math.round(v).toLocaleString() : Math.abs(v) >= 100 ? v.toFixed(1) : Math.abs(v) >= 1 ? v.toFixed(Math.min(d, 3)) : v === 0 ? '0' : v.toPrecision(3));
|
||||
const pct = (v, d = 1) => !isNum(v) ? '—' : (100 * v).toFixed(d) + '%';
|
||||
const hms = s => { if (!isNum(s)) return '—'; s = Math.max(0, s); const h = Math.floor(s / 3600), m = Math.floor(s % 3600 / 60); return h ? `${h} 小时 ${m} 分` : m ? `${m} 分 ${Math.floor(s % 60)} 秒` : `${Math.floor(s)} 秒`; };
|
||||
const hms = s => { if (!isNum(s)) return '—'; s = Math.max(0, s); const h = Math.floor(s / 3600), m = Math.floor(s % 3600 / 60); return h ? `${h} 小时 ${m} 分 ${Math.floor(s % 60)} 秒` : m ? `${m} 分 ${Math.floor(s % 60)} 秒` : `${Math.floor(s)} 秒`; }; // 用时精确到秒(页面 1 秒轮询,秒位是有意义的)
|
||||
const hmsShort = s => !isNum(s) ? '—' : s >= 3600 ? `${(s / 3600).toFixed(1)} h` : s >= 60 ? `${Math.round(s / 60)} 分` : `${Math.round(s)} 秒`;
|
||||
const ago = t => { const s = Date.now() / 1000 - t; return s < 60 ? `${Math.max(0, Math.floor(s))} 秒前` : s < 3600 ? `${Math.floor(s / 60)} 分钟前` : `${(s / 3600).toFixed(1)} 小时前`; };
|
||||
const ICON_WAIT = '<svg viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2"><circle cx="12" cy="12" r="9"/><path d="M12 7v5l3 2"/></svg>';
|
||||
@@ -344,13 +408,26 @@ let logyOn = false, job = null, userPicked = false, conOff = 0;
|
||||
下面这份只给示例数据用,数值与 pstate.py 一致。 */
|
||||
let STAGE_META = [
|
||||
['setup', '安装依赖', 0.2], ['kin', '对拍', 0.2], ['r4', '推理延迟', 0.2], ['gate1', '一致性门禁', 0.5], ['gen', '生成老师轨迹', 5.0],
|
||||
['prep', '打包样本', 0.5], ['train', '预训练', 3.0], ['m5', '头门禁', 0.5], ['apg', 'APG 微调', 4.0], ['mine', '挖错循环', 9.0],
|
||||
['prep', '打包样本', 2.0], ['train', '预训练', 4.5], ['dagger', 'DAgger 纠偏', 3.5], ['m5', '头门禁', 0.5], ['apg', 'APG 微调', 4.0], ['mine', '挖错循环', 9.0],
|
||||
['export', '导出', 0.1], ['bench', '部署评测', 1.0], ['accept', '验收汇总', 0.5]];
|
||||
let SM = Object.fromEntries(STAGE_META.map(([k, n, w]) => [k, { n, w, to: Math.max(1, 3 * w) }]));
|
||||
/* 起始阶段下拉的编号按 README 步骤表(1 环境、2 对拍、2′ R4、3 门禁、…、12 验收);setup 不进下拉(建议手动装依赖) */
|
||||
const FROM_NO = { setup: '1', kin: '2', r4: '2′', gate1: '3', gen: '4', prep: '5', train: '6', dagger: '6′', m5: '7', apg: '8', mine: '9', export: '10', bench: '11', accept: '12' };
|
||||
const FROM_NOTE = { gen: '(约 7 小时)' };
|
||||
function fillFrom() {
|
||||
const sel = $('fFrom'); if (!sel) return;
|
||||
const want = STAGE_META.map(x => x[0]).filter(k => k !== 'setup');
|
||||
if ([...sel.options].map(o => o.value).join() === want.join()) return; // 阶段表没变就别重建,免得每秒轮询把已选项重置
|
||||
const keep = sel.value;
|
||||
sel.innerHTML = want.map(k => `<option value="${k}">${FROM_NO[k] ? FROM_NO[k] + ' ' : ''}${SM[k].n} ${k}${FROM_NOTE[k] || ''}</option>`).join('');
|
||||
sel.value = want.includes(keep) ? keep : want.includes('train') ? 'train' : want[0];
|
||||
}
|
||||
function applyStageMeta(m) {
|
||||
if (!m || !m.length) return;
|
||||
STAGE_META = m.map(x => [x.id, x.name, x.est_h]); SM = Object.fromEntries(m.map(x => [x.id, { n: x.name, w: x.est_h, to: x.timeout_h }]));
|
||||
fillFrom();
|
||||
}
|
||||
fillFrom();
|
||||
/* 开始 / 停止请求带自定义头:跨站网页发不出这种请求(会触发 CORS 预检,控制台不应答),见 monitor.py 审查 N-8 */
|
||||
const post = (path, body) => fetch(path, { method: 'POST', headers: { 'X-Console': '1', 'Content-Type': 'application/json' }, body: body || '{}' }).then(x => x.json()).catch(e => ({ error: String(e) }));
|
||||
const smoothv = () => +$('smooth').value;
|
||||
@@ -373,6 +450,10 @@ function buildCharts(kind) {
|
||||
b: new Chart(c, '损失分量', { logy: () => logyOn, xmax: total, xmin0: true, hidden: ['smooth'] }),
|
||||
e: new Chart(c, '闭环评估(val,部署节奏,关安全层)', { percent: true, range: [0, 1], xmax: total, xmin0: true, note: kind === 'bench' ? '测速运行不评估' : '每 2000 步',
|
||||
emptyText: kind === 'bench' ? '测速运行不做闭环评估' : '第一次评估在第 2000 步' }),
|
||||
o: new Chart(c, '开环误差(mm)', { xmax: total, xmin0: true, note: kind === 'bench' ? '测速运行不评估' : 'val 样本、老师状态 · 16 步末端误差 · 越低越好',
|
||||
emptyText: kind === 'bench' ? '测速运行不做闭环评估' : '第一次评估在第 2000 步' }),
|
||||
d: new Chart(c, '候选多样性', { xmax: total, xmin0: true, note: kind === 'bench' ? '测速运行不评估' : '候选间标准差 / 动作幅度 · 塌成单峰时接近 0',
|
||||
emptyText: kind === 'bench' ? '测速运行不做闭环评估' : '第一次评估在第 2000 步' }),
|
||||
t: new Chart(c, '吞吐(样本/s)', { xmax: total, xmin0: true, zero: true, refs: [{ y: 5000, label: 'R10 通过线 5000', color: css('--bad') }] }),
|
||||
};
|
||||
}
|
||||
@@ -394,6 +475,11 @@ function render() {
|
||||
}
|
||||
const E = (k, label, best) => { const pts = evals.filter(e => isNum(e[k])).map(e => [e.x, e[k]]); let b = null; if (best && pts.length) b = pts.reduce((m, p) => p[1] > m[1] ? p : m); return { name: k, label, dots: true, pts, best: b }; };
|
||||
charts.e.set([E('success', '到达率', true), E('collision_rate', '碰撞率'), E('time_ok_1p3', '≤ 老师时长×1.3'), E('start_fail', '起步失败')]);
|
||||
if (charts.o) { // 开环指标只随 train.py 的评估写进日志;APG 分支没建这两张图
|
||||
const Emin = (k, label) => { const pts = evals.filter(e => isNum(e[k])).map(e => [e.x, e[k]]); let b = null; if (pts.length) b = pts.reduce((m, p) => p[1] < m[1] ? p : m); return { name: k, label, dots: true, pts, best: b }; };
|
||||
charts.o.set([Emin('ol_fk_mm', '候选 0'), { name: 'ol_best_mm', label: '8 候选最好', dots: true, pts: evals.filter(e => isNum(e.ol_best_mm)).map(e => [e.x, e.ol_best_mm]) }]);
|
||||
charts.d.set([{ name: 'ol_div', label: '多样性', dots: true, pts: evals.filter(e => isNum(e.ol_div)).map(e => [e.x, e.ol_div]) }]);
|
||||
}
|
||||
renderProgress(); renderKpis();
|
||||
}
|
||||
|
||||
@@ -417,7 +503,7 @@ function renderProgress() {
|
||||
<div class="facts">
|
||||
${fact('步数', `${step.toLocaleString()}${total ? ' / ' + total.toLocaleString() : ''}`)}
|
||||
${fact('已用时间', hms(elapsed))}
|
||||
${fact('剩余时间', finished ? '—' : hms(eta))}
|
||||
${fact('剩余时间', finished ? '—' : hmsShort(eta))}
|
||||
${fact('预计完成', fin)}
|
||||
${kind === 'apg' ? fact('本批展开', r.plans ? `${r.plans} 次推理` : '—') : fact('学习率', isNum(r.lr) ? r.lr.toExponential(2) : '—')}
|
||||
${fact('日志更新', lastMtime ? ago(lastMtime) : '—', running ? '' : 'warn')}
|
||||
@@ -427,6 +513,7 @@ function renderProgress() {
|
||||
function renderKpis() {
|
||||
const r = recs[recs.length - 1] || {}, e = evals[evals.length - 1] || {}, pe = evals[evals.length - 2] || {}, kind = cur.kind;
|
||||
const best = evals.length ? Math.max(...evals.map(v => v.success ?? -1)) : null;
|
||||
const bestE = evals.length ? evals.reduce((m, v) => ((v.success ?? -1) > (m.success ?? -1) ? v : m)) : null;
|
||||
const dlt = (k, good = 1) => { if (!isNum(e[k]) || !isNum(pe[k])) return ''; const d = (e[k] - pe[k]) * 100; if (Math.abs(d) < 0.05) return ''; const up = d > 0; return `<span class="delta ${(up ? 1 : -1) * good > 0 ? 'ok' : 'bad'}">${up ? '▲' : '▼'} ${Math.abs(d).toFixed(1)}pp</span>`; };
|
||||
const last = k => recs.filter(v => isNum(v[k])).slice(-300).map(v => [xOf(v), v[k]]);
|
||||
const evs = k => evals.filter(v => isNum(v[k])).map(v => [v.x, v[k]]);
|
||||
@@ -442,7 +529,7 @@ function renderKpis() {
|
||||
const sps = r.samples_per_s;
|
||||
tile('总损失', fmt(r.loss, 4), `MF ${fmt(r.mf)} · FK ${fmt(r.fk_cm)} cm`, '', ema(last('loss'), 0.9), COLORS[0]);
|
||||
tile('吞吐', sps ? `${(sps / 1000).toFixed(2)}<small>k 样本/s</small>` : '—', sps ? (sps >= 5000 ? 'R10 达标(≥ 5k/s)' : 'R10 未达标(< 5k/s)') : 'R10 通过线 5k/s', sps ? (sps >= 5000 ? 'ok' : 'bad') : '', last('samples_per_s'), COLORS[5]);
|
||||
tile('闭环到达率', pct(e.success) + dlt('success'), evals.length ? `最好 ${pct(best)} · @${e.x.toLocaleString()} 步` : '尚未评估', '', evs('success'), COLORS[2]);
|
||||
tile('闭环到达率', pct(e.success) + dlt('success'), bestE ? `最好 ${pct(best)} · @${bestE.x.toLocaleString()} 步` : '尚未评估', '', evs('success'), COLORS[2]);
|
||||
tile('碰撞率', pct(e.collision_rate, 2) + dlt('collision_rate', -1), '关安全层,部署另有护栏', e.collision_rate > 0.03 ? 'warn' : '', evs('collision_rate'), COLORS[3]);
|
||||
tile('梯度范数', fmt(r.gnorm), '裁剪阈值 1.0', r.gnorm > 5 ? 'warn' : '', ema(last('gnorm'), 0.9), COLORS[4]);
|
||||
tile('显存峰值', isNum(r.vram) ? `${r.vram.toFixed(2)}<small>GiB</small>` : '—', '预算 19–20 GB(整卡 24,桌面占 4–5)', r.vram > 17 ? 'bad' : r.vram > 14 ? 'warn' : '', last('vram'), COLORS[1]);
|
||||
@@ -485,8 +572,13 @@ async function poll() {
|
||||
if (DEMO) { setPill('demo', '示例数据'); if (!$('demobar')) $('pipe').closest('.hero').insertAdjacentHTML('beforebegin', `<div class="demobar" id="demobar">这是示例数据,不是你的训练。${location.protocol === 'file:' ? '直接双击 monitor.html 只能看示例;' : ''}看真实训练请双击 ml/console.bat(或 python -m ml.monitor --open)。</div>`); }
|
||||
else if (busy) setPill('live', `运行中 · ${P && P.current ? SM[P.current]?.n || P.current : '启动中'}`);
|
||||
else if (cur && Date.now() / 1000 - lastMtime < 120) setPill('live', `运行中 · ${ago(lastMtime)}更新`);
|
||||
else if (P && P.plan && P.plan.some(k => P.stages[k] && P.stages[k].rc != null && P.stages[k].rc !== 0)) {
|
||||
const k = P.plan.find(k => P.stages[k] && P.stages[k].rc != null && P.stages[k].rc !== 0), rc = P.stages[k].rc, nm = SM[k]?.n || k;
|
||||
setPill(rc === -15 || rc === 143 ? 'stale' : 'dead', `${rc === -15 || rc === 143 ? '已停止' : '失败'} · 停在「${nm}」返回码 ${rc}`);
|
||||
}
|
||||
else setPill('', '空闲');
|
||||
renderStatus(lastStatus); renderPipe(lastStatus); renderJob();
|
||||
try { renderStatus(lastStatus); renderPipe(lastStatus); renderJob(); }
|
||||
catch (e) { __pageFail('渲染面板出错:' + e.message + ' —— 其余面板可能不更新;训练本身不受影响'); }
|
||||
await pollConsole();
|
||||
} catch (err) {
|
||||
setPill('dead', '连不上控制台(双击 ml/console.bat 或 python -m ml.monitor)'); console.error(err);
|
||||
@@ -495,16 +587,21 @@ async function poll() {
|
||||
function setPill(cls, text) { const st = $('state'); st.className = 'pill ' + cls; st.lastElementChild.textContent = text; }
|
||||
|
||||
/* ===================== 流水线 / GPU / 各阶段面板 ===================== */
|
||||
/* 「全流程」进度与上方步骤条同一口径:步骤 = 本次计划 ∪ 记录里已有的阶段。
|
||||
从中间阶段重训时,前面已完成的阶段不在本次 plan 里,但记录还在(步骤条就靠这个显示打勾),
|
||||
总进度/剩余时间/完成计数也必须算上它们,否则总进度会归零。 */
|
||||
const pipeKeys = (P, plan) => STAGE_META.map(x => x[0]).filter(k => plan.includes(k) || P.stages[k]);
|
||||
function stages(s) {
|
||||
const P = s.pipeline;
|
||||
if (P) { // 有流水线记录:以它为准
|
||||
const plan = P.plan && P.plan.length ? P.plan : STAGE_META.map(x => x[0]).filter(k => k !== 'setup');
|
||||
const keys = STAGE_META.map(x => x[0]).filter(k => plan.includes(k) || P.stages[k]);
|
||||
const keys = pipeKeys(P, plan);
|
||||
const el = P.elapsed || {}; // 累计用时(跨停止/续跑),没有就退回本段
|
||||
return keys.map(k => {
|
||||
const r = P.stages[k], nm = SM[k].n;
|
||||
if (!r) return [nm, 'pending', '—'];
|
||||
if (P.current === k) return [nm, 'active', P.progress && P.progress.frac != null ? pct(P.progress.frac, 0) : '进行中'];
|
||||
if (r.rc === 0) return [nm, 'done', '用时 ' + hmsShort(r.end - r.start)];
|
||||
if (r.rc === 0) return [nm, 'done', '用时 ' + hms(el[k] != null ? el[k] : r.end - r.start)];
|
||||
if (r.rc === -15 || r.rc === 143) return [nm, 'fail', '已停止'];
|
||||
if (r.rc != null) return [nm, 'fail', `失败(返回码 ${r.rc})`];
|
||||
return [nm, 'fail', '中断'];
|
||||
@@ -531,34 +628,39 @@ function stages(s) {
|
||||
}
|
||||
function renderStatus(s) {
|
||||
const L = stages(s);
|
||||
$('steps').innerHTML = L.map(([nm, st, ds], i) => `<div class="step ${st}"><div class="ic">${st === 'done' ? '✓' : st === 'fail' ? '!' : i + 1}</div><div class="nm">${nm}</div><div class="ds">${ds}</div></div>`).join('');
|
||||
$('steps').innerHTML = L.map(([nm, st, ds], i) => `<div class="step ${st}"><div class="ic">${st === 'done' ? '✓' : st === 'fail' ? '!' : i + 1}</div><div class="nm">${nm}</div><div class="ds${st === 'active' ? ' big' : ''}">${ds}</div></div>`).join('');
|
||||
const nDone = L.filter(x => x[1] === 'done').length; $('stepnote').textContent = `${nDone} / ${L.length} 完成`;
|
||||
|
||||
// GPU + CPU(同一个框:上面是 GPU 环 + 显存/温度/功耗,下面接 CPU 总占用与每核条)
|
||||
const G = s.gpu || [], CP = s.cpu || {};
|
||||
const BUD = (s.vram_budget_gb || 24) * 1024; // 显存预算(MB):桌面常驻占 4–5 GB,不能按整卡 24 GB 看
|
||||
const RESV = s.vram_reserve_gb ?? 1; // 给系统留的余量(GB):训练进程启动时按"整卡 − 已用 − 余量"自设上限(浮动)
|
||||
const hr = '<hr style="border:none;border-top:1px solid var(--line);margin:14px 0">';
|
||||
const gr = (k, v, p, c) => `<div class="gr"><div class="l"><span>${k}</span><b>${v}</b></div><div class="mini"><div style="width:${Math.min(100, p)}%;background:${c}"></div></div></div>`;
|
||||
const gr = (k, v, p, c, t) => `<div class="gr"><div class="l"><span${t ? ` title="${t}"` : ''}>${k}</span><b>${v}</b></div><div class="mini"><div style="width:${Math.min(100, p)}%;background:${c}"></div></div></div>`;
|
||||
$('gputime').textContent = (G.length || CP.n) ? '实时' : '';
|
||||
const gpuBlk = G.length ? G.map(g => {
|
||||
const u = g.util ?? 0, C = 2 * Math.PI * 50, col = u > 85 ? css('--ok') : u > 50 ? css('--accent') : css('--warn');
|
||||
const BUD = (s.vram_budget_gb || (g.mem_total / 1024 - RESV)) * 1024, avail = BUD / 1024 - g.mem_used / 1024;
|
||||
const memP = 100 * g.mem_used / BUD, tc = g.temp > 83 ? css('--bad') : g.temp > 75 ? css('--warn') : css('--c6');
|
||||
return `<div class="gpuwrap"><div class="ring"><svg width="116" height="116"><circle cx="58" cy="58" r="50" stroke="#1b2433" stroke-width="10" fill="none"/>
|
||||
<circle cx="58" cy="58" r="50" stroke="${col}" stroke-width="10" fill="none" stroke-linecap="round" stroke-dasharray="${C}" stroke-dashoffset="${C * (1 - u / 100)}" style="transition:stroke-dashoffset .6s"/></svg>
|
||||
<div class="c"><div><b>${u}%</b><span>利用率</span></div></div></div>
|
||||
<div class="gpurows">${gr('显存', `${(g.mem_used / 1024).toFixed(1)} / ${(BUD / 1024).toFixed(0)} GB 预算(整卡 ${(g.mem_total / 1024).toFixed(0)})`, memP, memP > 100 ? css('--bad') : memP > 85 ? css('--warn') : css('--accent2'))}
|
||||
<div class="gpurows">${gr('显存', `${(g.mem_used / 1024).toFixed(1)} / ${(BUD / 1024).toFixed(0)} GB · 空 ${avail.toFixed(1)}`, memP, memP > 100 ? css('--bad') : memP > 85 ? css('--warn') : css('--accent2'), `可用 ${(BUD / 1024).toFixed(0)} GB = 整卡 ${(g.mem_total / 1024).toFixed(0)} GB − 给系统的余量 ${RESV} GB;训练进程启动时按「可用 − 那一刻已用」再自设本进程上限,别人占多少自动扣掉(ml/common/vram.py)`)}
|
||||
${gr('温度', `${g.temp} °C`, g.temp, tc)}${gr('功耗', g.power != null ? `${Math.round(g.power)} W` : '—', g.power ? g.power / 3.5 : 0, css('--c7'))}</div></div>
|
||||
<div class="gname">${g.name}</div>`;
|
||||
}).join(hr) : '';
|
||||
const cpuBlk = CP.n ? (() => {
|
||||
const tot = CP.total ?? 0, C = 2 * Math.PI * 50, col = tot > 85 ? css('--bad') : tot > 60 ? css('--warn') : css('--ok');
|
||||
const bars = (CP.per || []).map((v, i) => `<div title="核 ${i + 1}: ${v}%" style="flex:1;min-width:0;height:100%;border-radius:2px;background:${v > 85 ? css('--bad') : v > 60 ? css('--warn') : css('--accent2')};opacity:${0.18 + 0.82 * Math.min(1, v / 100)}"></div>`).join('');
|
||||
const ct = CP.coreT || [], ctMax = ct.length ? Math.max(...ct) : null;
|
||||
const tcol = v => v > 98 ? css('--bad') : v > 90 ? css('--warn') : css('--c6');
|
||||
const bars = (CP.per || []).map((v, i) => `<div title="核 ${i + 1}: ${v}%${ct[i] != null ? ' · ' + ct[i] + ' °C' : ''}" style="flex:1;min-width:0;height:100%;border-radius:2px;background:${v > 85 ? css('--bad') : v > 60 ? css('--warn') : css('--accent2')};opacity:${0.18 + 0.82 * Math.min(1, v / 100)}"></div>`).join('');
|
||||
return `<div class="gpuwrap"><div class="ring"><svg width="116" height="116"><circle cx="58" cy="58" r="50" stroke="#1b2433" stroke-width="10" fill="none"/>
|
||||
<circle cx="58" cy="58" r="50" stroke="${col}" stroke-width="10" fill="none" stroke-linecap="round" stroke-dasharray="${C}" stroke-dashoffset="${C * (1 - tot / 100)}" style="transition:stroke-dashoffset .6s"/></svg>
|
||||
<div class="c"><div><b>${tot}%</b><span>CPU</span></div></div></div>
|
||||
<div class="gpurows">${gr('占用', `${tot}% · ${CP.n} 核`, tot, col)}
|
||||
${CP.temp != null ? gr('温度', `${CP.temp} °C${ctMax != null && ctMax !== CP.temp ? `(最高核 ${ctMax})` : ''}`, CP.temp, tcol(CP.temp), 'MSI Afterburner 的共享内存;鼠标停在核条上还能看每核温度') : ''}
|
||||
${CP.power != null ? gr('功耗', `${Math.round(CP.power)} W`, 100 * CP.power / 253, css('--c7'), 'CPU 封装功耗(满标度按 14700K 的 PL2 253 W 画)') : ''}
|
||||
<div class="cores">${bars}</div></div></div>
|
||||
<div class="gname">每核占用(深色=忙)</div>`;
|
||||
<div class="gname">${CP.model || 'CPU'}${CP.temp == null ? ' · 开 MSI Afterburner 才能读 CPU 温度/功耗' : ''} · 每核占用(深色=忙)</div>`;
|
||||
})() : '';
|
||||
$('gpu').innerHTML = (gpuBlk + (gpuBlk && cpuBlk ? hr : '') + cpuBlk) || empty('未检测到 nvidia-smi / psutil');
|
||||
|
||||
@@ -611,32 +713,39 @@ function renderPipe(s) {
|
||||
: ck.python ? `<div class="ckline">Python ${ck.python} · torch ${ck.torch}${ck.gpu ? ' · ' + ck.gpu : ''}${ck.warn ? ` · <span class="warn">${ck.warn}</span>` : ''}</div>` : '';
|
||||
const who = job && job.external ? `由${(lk && lk.owner || '').startsWith('网页') ? '上一个控制台窗口' : '终端'}启动(pid ${lk && lk.pid},${Math.round(lk && lk.age_s || 0)} 秒前心跳)` : job && job.pid ? `PID ${job.pid}` : '';
|
||||
const btns = `${ckline}<div class="ctlrow">${running || (P && P.current)
|
||||
? `<button class="danger" onclick="stopJob()">■ 停止</button><span class="hint">${who}</span>`
|
||||
: `<button class="primary" onclick="openStart()" ${ck.problems && ck.problems.length ? 'disabled' : ''}>▶ ${P ? '继续训练' : '开始训练'}</button><span class="hint">${job && job.rc != null ? (job.stopped ? '上次已手动停止,再点即从停下的那步继续' : job.rc === 0 ? '上次运行已全部完成' : `上次运行失败(返回码 ${job.rc}),看终端输出;修好后再点即续跑`) : '已完成的步骤会自动跳过'}</span>`}</div>`;
|
||||
? `<button class="danger" onclick="stopJob()">■ 停止</button><button class="ghost" disabled title="训练进行中:先点「停止」,再点「重新训练」">⟳ 重新训练</button><span class="hint">${who}</span>`
|
||||
: `<button class="primary" onclick="openStart()" ${ck.problems && ck.problems.length ? 'disabled' : ''}>▶ ${P ? '继续训练' : '开始训练'}</button><button class="ghost" onclick="openStart({mode:'from',from:'train',title:'重新训练'})" ${ck.problems && ck.problems.length ? 'disabled' : ''} title="从 train 阶段重做:旧模型与评测移到 ml/archive/(不删),生成的数据、打包样本、门禁都保留">⟳ 重新训练</button><span class="hint">${job && job.rc != null ? (job.stopped ? '上次已手动停止,再点即从停下的那步继续' : job.rc === 0 ? '上次运行已全部完成' : `上次运行失败(返回码 ${job.rc}),看终端输出;修好后再点即续跑`) : '已完成的步骤会自动跳过'}</span>`}</div>`;
|
||||
if (!P) { $('pipe').innerHTML = `<h2><span>全流程</span><span class="r">尚未开始</span></h2>${empty('还没运行过流水线。点「开始训练」从对拍一直跑到验收;中途每一步的进度都显示在这里。')}${btns}`; return; }
|
||||
const plan = P.plan && P.plan.length ? P.plan : Object.keys(P.stages);
|
||||
const keys = pipeKeys(P, plan); // 含重训时保留的已完成阶段,与上方步骤条同口径
|
||||
let tot = 0, got = 0;
|
||||
for (const k of plan) { const w = SM[k]?.w || 0.1, r = P.stages[k]; tot += w; if (r && r.rc === 0 && P.current !== k) got += w; else if (P.current === k) got += w * ((P.progress && P.progress.frac) || 0); }
|
||||
for (const k of keys) { const w = SM[k]?.w || 0.1, r = P.stages[k]; tot += w; if (r && r.rc === 0 && P.current !== k) got += w; else if (P.current === k) got += w * ((P.progress && P.progress.frac) || 0); }
|
||||
const frac = tot ? got / tot : 0, now = Date.now() / 1000;
|
||||
const cur = P.current, cr = cur && P.stages[cur], pr = P.progress || {};
|
||||
const doneN = plan.filter(k => P.stages[k] && P.stages[k].rc === 0 && P.current !== k).length;
|
||||
const doneN = keys.filter(k => P.stages[k] && P.stages[k].rc === 0 && P.current !== k).length;
|
||||
const failed = plan.find(k => P.stages[k] && P.stages[k].rc != null && P.stages[k].rc !== 0);
|
||||
const allDone = !cur && doneN === plan.length;
|
||||
let remain = 0; for (const k of plan) { const r = P.stages[k]; if (!r || r.rc == null) remain += (SM[k]?.w || 0.1) * (k === cur ? 1 - (pr.frac || 0) : 1); }
|
||||
const allDone = !cur && doneN === keys.length;
|
||||
let remain = 0, calib = false; for (const k of keys) { const r = P.stages[k]; if (!r || r.rc == null) { if (k === cur && pr.eta_s != null) { remain += pr.eta_s / 3600; calib = true; } else remain += (SM[k]?.w || 0.1) * (k === cur ? 1 - (pr.frac || 0) : 1); } }
|
||||
const head = cur ? `<div class="now">正在运行<b>${SM[cur]?.n || cur}</b>${pr.label || ''}</div>`
|
||||
: allDone ? `<div class="now">状态<b class="ok">全部完成</b>共 ${plan.length} 步</div>`
|
||||
: allDone ? `<div class="now">状态<b class="ok">全部完成</b>共 ${keys.length} 步</div>`
|
||||
: failed ? `<div class="now">状态<b class="bad">停在「${SM[failed]?.n || failed}」</b>${(P.stages[failed].info || {}).timed_out || P.stages[failed].rc === 124 ? '超过时间上限,判定卡死' : P.stages[failed].rc === -15 || P.stages[failed].rc === 143 ? '已停止' : '失败,看终端输出'}</div>`
|
||||
: `<div class="now">状态<b>未在运行</b>已完成 ${doneN} / ${plan.length} 步</div>`;
|
||||
$('pipe').innerHTML = `<h2><span>全流程</span><span class="r">${doneN} / ${plan.length} 步完成</span></h2>
|
||||
: `<div class="now">状态<b>未在运行</b>已完成 ${doneN} / ${keys.length} 步</div>`;
|
||||
const esc2 = x => String(x).replace(/&/g, '&').replace(/</g, '<');
|
||||
const fr = failed && P.stages[failed], stopped = !!fr && (fr.rc === -15 || fr.rc === 143);
|
||||
const timedOut = !!fr && ((fr.info || {}).timed_out || fr.rc === 124);
|
||||
const why = !fr ? '' : stopped ? '已手动停止 —— 点「继续训练」从停下的地方接着跑' : timedOut ? '超过时间上限,判定卡死' : `失败(返回码 ${fr.rc})`;
|
||||
const lastErr = (() => { const t = $('term'); if (!t) return ''; const ls = (t.innerText || '').split(/\r?\n/).filter(x => /Error|error:|Traceback|不通过|失败|超过时间上限/.test(x)); return ls.length ? ls[ls.length - 1].slice(0, 300) : ''; })();
|
||||
$('alert').innerHTML = failed ? `<div class="alert ${stopped ? 'warn' : 'bad'}"><div class="alerth">✕ 停在「${SM[failed]?.n || failed}」 · ${why}</div>${lastErr ? `<div class="alerterr">${esc2(lastErr)}</div>` : ''}<div class="alerttip">完整报错见下方「终端输出」;修好后点「继续训练」,已完成的步骤会自动跳过${stopped ? '、从停下的那一步接着跑' : ''}</div></div>` : '';
|
||||
$('pipe').innerHTML = `<h2><span>全流程</span><span class="r">${doneN} / ${keys.length} 步完成</span></h2>
|
||||
<div class="overall">${head}<div class="bigpct">${(100 * frac).toFixed(1)}<small>%</small></div></div>
|
||||
<div class="bar ${cur ? '' : 'idle'}"><div class="fill" style="width:${100 * frac}%"></div></div>
|
||||
${cur ? `<div class="sublabel"><span>当前步骤 · ${SM[cur]?.n}</span><span>${pr.frac != null ? pct(pr.frac) : '无法计量,按用时估'}</span></div>
|
||||
${cur ? `<div class="sublabel"><span>当前步骤 · ${SM[cur]?.n}</span>${pr.frac != null ? `<b>${pct(pr.frac)}</b>` : '<span>无法计量,按用时估</span>'}</div>
|
||||
<div class="bar thin ${pr.frac == null ? 'ind' : ''}"><div class="fill" style="width:${pr.frac != null ? 100 * pr.frac : 30}%"></div></div>` : ''}
|
||||
<div class="facts">
|
||||
<div class="fact"><div class="k">总用时</div><div class="v">${hms((cur ? now : Math.max(...Object.values(P.stages).map(r => r.end || r.start))) - P.started)}</div></div>
|
||||
<div class="fact"><div class="k">本步用时</div><div class="v">${cr ? hms(now - cr.start) : '—'}</div></div>
|
||||
<div class="fact"><div class="k">本步粗估 / 上限</div><div class="v">${cur ? hmsShort((SM[cur]?.w || 0) * 3600) + ' / ' + hmsShort((cr && cr.timeout_h || SM[cur]?.to || 0) * 3600) : '—'}</div></div>
|
||||
<div class="fact"><div class="k">全流程剩余(粗估)</div><div class="v">${allDone ? '—' : '约 ' + hmsShort(remain * 3600)}</div></div>
|
||||
<div class="fact"><div class="k">总用时</div><div class="v">${hms(keys.reduce((a, k) => a + ((P.elapsed || {})[k] || 0), 0))}</div></div>
|
||||
<div class="fact"><div class="k">本步用时</div><div class="v">${cur ? hms((P.elapsed || {})[cur] || 0) : '—'}</div></div>
|
||||
<div class="fact"><div class="k">${cur && pr.eta_s != null ? '本步预计剩余(实测速率)' : '本步粗估 / 上限'}</div><div class="v">${cur ? (pr.eta_s != null ? hmsShort(pr.eta_s) + ' / 上限 ' + hmsShort((cr && cr.timeout_h || SM[cur]?.to || 0) * 3600) : hmsShort((SM[cur]?.w || 0) * 3600) + ' / ' + hmsShort((cr && cr.timeout_h || SM[cur]?.to || 0) * 3600)) : '—'}</div></div>
|
||||
<div class="fact"><div class="k">全流程剩余(${calib ? '实测外推' : '粗估'})</div><div class="v">${allDone ? '—' : '约 ' + hmsShort(remain * 3600)}</div></div>
|
||||
</div>${btns}`;
|
||||
}
|
||||
function renderJob() {
|
||||
@@ -656,20 +765,28 @@ async function pollConsole() {
|
||||
conOff = d.offset; if (atBottom || d.reset) t.scrollTop = t.scrollHeight;
|
||||
}
|
||||
function toast(msg) { const t = $('toast'); t.textContent = msg; t.style.display = 'block'; clearTimeout(toast.h); toast.h = setTimeout(() => t.style.display = 'none', 3500); }
|
||||
function openStart() {
|
||||
function openStart(preset) {
|
||||
const P = lastStatus && lastStatus.pipeline, ck = lastStatus && lastStatus.check;
|
||||
if (ck && ck.problems && ck.problems.length) { toast('环境自检未通过:' + ck.problems.join(';')); return; }
|
||||
const pre = preset || {};
|
||||
if (pre.mode) $('fMode').value = pre.mode;
|
||||
if (pre.from) $('fFrom').value = pre.from;
|
||||
$('mdTitle').textContent = pre.title || (P ? '继续训练' : '开始训练');
|
||||
const w = $('fWarn'); const upd = () => { const msgs = [];
|
||||
const done = P ? (P.plan || []).filter(k => P.stages[k] && P.stages[k].rc === 0) : [];
|
||||
if ($('fMode').value === 'resume' && done.length) msgs.push(`将跳过已完成的 ${done.length} 步:${done.map(k => SM[k]?.n || k).join('、')}。改了参数会自动改为从头重跑。`);
|
||||
if ($('fMode').value === 'fresh' && P) msgs.push('从头重跑:上次的数据、模型、评测结果整体移到 ml/archive/<时间>/(不删除)。');
|
||||
if ($('fMode').value === 'from' && P) { const nm = SM[$('fFrom').value]?.n || $('fFrom').value; msgs.push(`从「${nm}」开始重训:该阶段及之后的旧产物移到 ml/archive/<时间>/(不删除),之前的阶段保持不动。${$('fFrom').value === 'gen' ? '注意:生成的数据会被归档,重跑约 7 小时。' : ''}`); }
|
||||
if ($('fMode').value === 'fresh' && P) msgs.push('全部重跑:上次的数据、模型、评测结果整体移到 ml/archive/<时间>/(不删除)。生成数据要重跑(约 7 小时)。');
|
||||
$('fFrom').disabled = $('fMode').value !== 'from';
|
||||
$('fGo').textContent = $('fMode').value === 'from' ? '▶ 开始重训' : ($('fMode').value === 'fresh' ? '▶ 全部重跑' : '▶ 开始');
|
||||
w.style.display = msgs.length ? 'block' : 'none'; w.innerHTML = msgs.join('<br>'); };
|
||||
$('fMode').onchange = upd; upd();
|
||||
$('modal').classList.add('show');
|
||||
}
|
||||
async function startJob() {
|
||||
const env = { CFG: $('fCfg').value, CONSENSUS: $('fCons').value, N_TASKS: $('fN').value.trim(), NPROC: $('fP').value.trim(), TARGET_SAMPLES: $('fT').value.trim(), ROUNDS: $('fR').value.trim(), MIN_GAP_MM: $('fG').value.trim() };
|
||||
const body = JSON.stringify({ fresh: $('fMode').value === 'fresh', env });
|
||||
const mode = $('fMode').value, from = mode === 'from' ? $('fFrom').value : 'kin';
|
||||
const env = { CFG: $('fCfg').value, CONSENSUS: $('fCons').value, N_TASKS: $('fN').value.trim(), NPROC: $('fP').value.trim(), TARGET_SAMPLES: $('fT').value.trim(), ROUNDS: $('fR').value.trim(), MIN_GAP_MM: $('fG').value.trim(), FRESH_FROM: mode === 'from' ? from : '' };
|
||||
const body = JSON.stringify({ fresh: mode === 'fresh', from, to: 'accept', env });
|
||||
const r = DEMO ? await api('/api/start', body) : await post('/api/start', body);
|
||||
if (r.error) { toast('没能开始:' + r.error); return; }
|
||||
$('modal').classList.remove('show'); toast('已开始'); conOff = 0; userPicked = false; poll();
|
||||
@@ -683,10 +800,38 @@ async function stopJob() {
|
||||
$('btnStart').onclick = openStart; $('btnStop').onclick = stopJob;
|
||||
$('fCancel').onclick = () => $('modal').classList.remove('show'); $('fGo').onclick = startJob;
|
||||
$('modal').onclick = e => { if (e.target === $('modal')) $('modal').classList.remove('show'); };
|
||||
/* 记住上次的设置(localStorage,按站点保存):下次开页面直接是这套参数。
|
||||
注意「运行方式」不记忆——万一记成"全部重跑",点开始会把数据和模型全归档,风险不对等。 */
|
||||
const PREF = 'mlmon.pref.v1', PREF_IDS = ['fFrom', 'fCfg', 'fCons', 'fN', 'fP', 'fT', 'fR', 'fG'];
|
||||
function prefSave() {
|
||||
try {
|
||||
const o = {};
|
||||
for (const id of PREF_IDS) o[id] = $(id).value;
|
||||
o.smooth = $('smooth').value; o.logy = logyOn ? 1 : 0;
|
||||
localStorage.setItem(PREF, JSON.stringify(o));
|
||||
} catch (e) { }
|
||||
}
|
||||
function prefLoad() {
|
||||
let o = null; try { o = JSON.parse(localStorage.getItem(PREF) || 'null'); } catch (e) { }
|
||||
if (!o) return;
|
||||
for (const id of PREF_IDS) {
|
||||
const v = o[id]; if (v == null || v === '') continue;
|
||||
const el = $(id); if (!el) continue;
|
||||
if (el.tagName === 'SELECT') { if ([...el.options].some(x => x.value === v)) el.value = v; }
|
||||
else if (/^[0-9]*\.?[0-9]*$/.test(String(v))) el.value = v; // 参数框只接受数字,脏值不落地
|
||||
}
|
||||
if (o.smooth != null) { $('smooth').value = o.smooth; $('smoothv').textContent = Number(o.smooth).toFixed(2); }
|
||||
if (o.logy) { logyOn = true; $('logy').classList.add('on'); }
|
||||
}
|
||||
/* 直接开「重新训练」弹窗:?start=train / ?start=gen…(便于收藏和脚本化截图验证) */
|
||||
{ const m = location.search.match(/[?&]start=([a-z0-9]+)/);
|
||||
if (m && SM[m[1]]) setTimeout(() => openStart({ mode: 'from', from: m[1], title: '重新训练' }), 1200); }
|
||||
|
||||
/* ===================== 控件 ===================== */
|
||||
$('smooth').oninput = () => { $('smoothv').textContent = smoothv().toFixed(2); render(); };
|
||||
$('logy').onclick = () => { logyOn = !logyOn; $('logy').classList.toggle('on', logyOn); render(); };
|
||||
prefLoad();
|
||||
$('smooth').oninput = () => { $('smoothv').textContent = smoothv().toFixed(2); prefSave(); render(); };
|
||||
$('logy').onclick = () => { logyOn = !logyOn; $('logy').classList.toggle('on', logyOn); prefSave(); render(); };
|
||||
for (const id of PREF_IDS) { $(id).onchange = prefSave; $(id).oninput = prefSave; }
|
||||
$('run').onchange = () => { userPicked = true; cur = null; poll(); };
|
||||
$('scenario').onchange = () => { api.scenario($('scenario').value); cur = null; userPicked = false; conOff = 0; $('run').innerHTML = ''; poll(); };
|
||||
$('auto').onclick = () => { const on = !$('auto').classList.contains('on'); $('auto').classList.toggle('on', on); clearInterval(timer); if (on) timer = setInterval(poll, 1000); };
|
||||
@@ -711,7 +856,8 @@ function demoApi() {
|
||||
function trainEval(x, t0, sps) {
|
||||
const q = 1 - Math.exp(-x / 14000);
|
||||
return { eval: { step: x, t: t0 + x * 1024 / sps, n: 512, n_ref: 512, success: Math.min(0.975, 0.08 + 0.87 * q + 0.012 * gauss()), collision_rate: Math.max(0.004, 0.31 * (1 - q) + 0.012 + 0.005 * gauss()),
|
||||
time_ok_1p3: Math.min(0.95, 0.05 + 0.8 * q + 0.012 * gauss()), start_fail: Math.max(0, 0.05 * (1 - q) + 0.003 * gauss()) } };
|
||||
time_ok_1p3: Math.min(0.95, 0.05 + 0.8 * q + 0.012 * gauss()), start_fail: Math.max(0, 0.05 * (1 - q) + 0.003 * gauss()),
|
||||
ol_fk_mm: 5.5 * Math.exp(-x / 12000) + 2.0 + 0.25 * gauss(), ol_best_mm: 3.8 * Math.exp(-x / 12000) + 1.7 + 0.16 * gauss(), ol_div: Math.min(0.9, 0.16 + 0.45 * (1 - Math.exp(-x / 9000)) + 0.02 * gauss()) } };
|
||||
}
|
||||
function trainLog(total, from, to, sps, t0, evalEvery = 2000, wta = false) {
|
||||
const L = [];
|
||||
@@ -730,10 +876,10 @@ function demoApi() {
|
||||
}
|
||||
let sc, runs, logs, pipe, jobS, con, genAt = 0;
|
||||
// 示例流水线:stages 里的 [阶段, 开始距今小时, 用时小时]
|
||||
const mkPipe = (done, cur, curStartH) => { const t = now(); const st = {}; let tt = t - curStartH * 3600;
|
||||
for (let i = done.length - 1; i >= 0; i--) { const [k, dur] = done[i]; st[k] = { start: tt - dur * 3600, end: tt, rc: 0, info: {} }; tt -= dur * 3600; }
|
||||
if (cur) st[cur] = { start: t - curStartH * 3600, end: null, rc: null, info: {} };
|
||||
return { started: tt, plan: ['kin', 'r4', 'gate1', 'gen', 'prep', 'train', 'm5', 'apg', 'mine', 'export', 'bench', 'accept'], stages: st, current: cur }; };
|
||||
const mkPipe = (done, cur, curStartH) => { const t = now(); const st = {}, el = {}; let tt = t - curStartH * 3600;
|
||||
for (let i = done.length - 1; i >= 0; i--) { const [k, dur] = done[i]; st[k] = { start: tt - dur * 3600, end: tt, rc: 0, info: {} }; el[k] = dur * 3600; tt -= dur * 3600; }
|
||||
if (cur) { st[cur] = { start: t - curStartH * 3600, end: null, rc: null, info: {} }; el[cur] = curStartH * 3600; }
|
||||
return { started: tt, plan: ['kin', 'r4', 'gate1', 'gen', 'prep', 'train', 'dagger', 'm5', 'apg', 'mine', 'export', 'bench', 'accept'], stages: st, elapsed: el, current: cur }; };
|
||||
const CON = {
|
||||
train: '$ FROM=kin TO=accept bash ml/pipeline.sh all\n计划:kin r4 gate1 gen prep train m5 apg mine export bench accept\n== node ml/prep/export_kin.cjs\n== node ml/eval/dump_parity.cjs 2000\n== python -m ml.eval.check_parity\n{"PASS": true, "fk_mm": 0.0004, "gap_mm": 0.0011}\n== node ml/gen/consistency.cjs 300 4242\n{"consistentByGateRule": 0.86}\n一致性门禁结果 → CONSENSUS=1\n== bash ml/gen/run_parallel.sh 1786 28 1 --consensus 4\n样本数 1702000 ≥ 目标,完成\n== bash ml/prep/run_prep.sh 28\n== python -m ml.eval.check_r2 --samples ml/data/samples --out ml/data/r2.json\n{"pairs": 250, "label_std_mean": 0.071, "pass": true}\n== python -m ml.train --samples ml/data/samples --out ml/runs/bench_base --steps 2000 --no-eval\nstep 2000 loss 1.912 6150 samples/s\n== python -m ml.train --samples ml/data/samples --out ml/runs/base\n',
|
||||
gen: '$ FROM=kin TO=accept bash ml/pipeline.sh all\n计划:kin r4 gate1 gen prep train m5 apg mine export bench accept\n== node ml/prep/export_kin.cjs\n== python -m ml.eval.check_parity\n{"PASS": true}\n== node ml/gen/consistency.cjs 300 4242\n{"consistentByGateRule": 0.86}\n一致性门禁结果 → CONSENSUS=1\n== bash ml/gen/run_parallel.sh 1786 28 1 --consensus 4\n',
|
||||
@@ -819,6 +965,7 @@ function demoApi() {
|
||||
fn.scenario = setup;
|
||||
return fn;
|
||||
}
|
||||
window.__boot.ok = true; // 正文脚本跑到了最后一行,看门狗不用再报"脚本没启动"
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
+200
-28
@@ -17,7 +17,7 @@
|
||||
N-11 Windows 子进程用 CREATE_NO_WINDOW 自带隐藏控制台:关掉控制台黑窗口训练照跑,重开控制台可接着看、可停止;
|
||||
N-9 生成阶段的行数按文件增量统计(只读新增字节),其他阶段不扫 shard;
|
||||
N-12 停止后的状态改写走 ml.pstate(单一写者、原子替换);N-14 阶段名/估时/权重由 ml.pstate.STAGES 下发。"""
|
||||
import argparse, glob, json, os, re, shutil, signal, subprocess, sys, threading, time, webbrowser
|
||||
import argparse, glob, json, os, platform, re, shutil, signal, subprocess, sys, threading, time, webbrowser
|
||||
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
|
||||
from ml import pstate
|
||||
from http.server import ThreadingHTTPServer, BaseHTTPRequestHandler
|
||||
@@ -71,6 +71,22 @@ def read_log(name, offset):
|
||||
_CPU = {'v': {}}
|
||||
|
||||
|
||||
def cpu_name():
|
||||
"""CPU 型号:psutil 只有核数,型号得自己读——Windows 在注册表,Linux 在 /proc/cpuinfo。"""
|
||||
try:
|
||||
if os.name == 'nt':
|
||||
import winreg
|
||||
with winreg.OpenKey(winreg.HKEY_LOCAL_MACHINE, r'HARDWARE\DESCRIPTION\System\CentralProcessor\0') as k:
|
||||
return re.sub(r'\s+', ' ', winreg.QueryValueEx(k, 'ProcessorNameString')[0]).strip()
|
||||
with open('/proc/cpuinfo') as f:
|
||||
for ln in f:
|
||||
if ln.startswith('model name'):
|
||||
return ln.split(':', 1)[1].strip()
|
||||
except Exception:
|
||||
pass
|
||||
return platform.processor().strip()
|
||||
|
||||
|
||||
class CpuSampler(threading.Thread):
|
||||
"""后台每秒阻塞采样一次 CPU。cpu_percent 是"距上次调用"的差值:放在请求里算,窗口取决于两次
|
||||
HTTP 请求的间隔(网页 1 秒轮询 + 手动 curl 会把它压成几毫秒,读数恒为 0)——所以单独一个线程按固定
|
||||
@@ -80,16 +96,66 @@ class CpuSampler(threading.Thread):
|
||||
def run(self):
|
||||
try:
|
||||
import psutil
|
||||
name = cpu_name() # 型号只读一次,别每秒翻注册表
|
||||
psutil.cpu_percent(interval=None, percpu=True) # 打桩:第一次恒为 0,丢掉
|
||||
while True:
|
||||
per = psutil.cpu_percent(interval=1.0, percpu=True)
|
||||
_CPU['v'] = {'total': round(sum(per) / max(1, len(per)), 1), 'per': [round(x) for x in per], 'n': len(per)}
|
||||
_CPU['v'] = {'total': round(sum(per) / max(1, len(per)), 1), 'per': [round(x) for x in per], 'n': len(per), 'model': name}
|
||||
except Exception:
|
||||
_CPU['v'] = {}
|
||||
|
||||
|
||||
def cpu():
|
||||
return _CPU['v']
|
||||
return {**_CPU['v'], **mahm()}
|
||||
|
||||
|
||||
_MAHM = {'t': 0.0, 'v': {}}
|
||||
|
||||
|
||||
def mahm():
|
||||
"""借 MSI Afterburner 的共享内存(MAHMSharedMemory)读 CPU 温度 / 封装功耗——Windows 上没有别的免驱动口子:
|
||||
psutil 在 Windows 没有 sensors_temperatures;ACPI 热区只有个 27.9 °C 的板温;Power Meter 计数器没有实例。
|
||||
条目布局实测:头 32 字节(签名 MAHM / 版本 / 头大小 / 条目数 / 条目大小 / 时间戳),每条 1324 字节,
|
||||
名字在前 260 字节、单位在其后 260、数值是偏移 1300 的 float(用 GPU1 power 对 nvidia-smi 校准过)。
|
||||
读数比 30 秒旧才当没读到:Afterburner 的硬件轮询周期实测 10~13 秒(不是 1 秒),阈值卡 10 秒会
|
||||
正好落在两次刷新之间、把还在用的读数丢掉(表现为温度和功耗一闪一闪地消失)。"""
|
||||
now = time.time()
|
||||
if now - _MAHM['t'] < 1.0: return _MAHM['v']
|
||||
_MAHM['t'] = now
|
||||
out = {}
|
||||
try:
|
||||
import ctypes, struct
|
||||
k = ctypes.windll.kernel32
|
||||
k.OpenFileMappingW.restype = ctypes.c_void_p
|
||||
k.OpenFileMappingW.argtypes = [ctypes.c_uint32, ctypes.c_int, ctypes.c_wchar_p]
|
||||
k.MapViewOfFile.restype = ctypes.c_void_p
|
||||
k.MapViewOfFile.argtypes = [ctypes.c_void_p, ctypes.c_uint32, ctypes.c_uint32, ctypes.c_uint32, ctypes.c_size_t]
|
||||
h = k.OpenFileMappingW(0x0004, 0, 'MAHMSharedMemory')
|
||||
p = k.MapViewOfFile(h, 0x0004, 0, 0, 0) if h else 0
|
||||
if p:
|
||||
hd = ctypes.string_at(p, 32)
|
||||
num, esize, hsize = (int.from_bytes(hd[i * 4:i * 4 + 4], 'little') for i in (3, 4, 2))
|
||||
upd = int.from_bytes(hd[20:24], 'little')
|
||||
if 0 < num < 5000 and 32 < esize < 65536 and now - upd <= 30:
|
||||
buf = ctypes.string_at(p, hsize + num * esize)
|
||||
core = {}
|
||||
for i in range(num):
|
||||
e = buf[hsize + i * esize: hsize + (i + 1) * esize]
|
||||
nm = e[:260].split(b'\x00')[0].decode('latin1')
|
||||
if not nm: continue
|
||||
val = struct.unpack('<f', e[1300:1304])[0]
|
||||
if nm == 'CPU temperature' and 0 < val < 130: out['temp'] = round(val)
|
||||
elif nm == 'CPU power' and 0 < val < 1000: out['power'] = round(val, 1)
|
||||
elif nm.startswith('CPU') and nm.endswith(' temperature'):
|
||||
try:
|
||||
j = int(nm[3:-12])
|
||||
if 0 < val < 130: core[j] = round(val)
|
||||
except ValueError: pass
|
||||
if core: out['coreT'] = [core[k] for k in sorted(core)]
|
||||
except Exception:
|
||||
out = {}
|
||||
_MAHM['v'] = out
|
||||
return out
|
||||
|
||||
|
||||
_GPU_CACHE = {'t': 0.0, 'v': []}
|
||||
@@ -156,7 +222,7 @@ def newer(pattern, t0):
|
||||
|
||||
|
||||
def stage_progress(st, s):
|
||||
"""当前阶段的进度:返回 {'frac': 0..1 或 None(无法计量), 'label': 文字}。"""
|
||||
"""当前阶段的进度:返回 {'frac': 0..1 或 None(无法计量), 'label': 文字, 'eta_s': 按实测速率外推的剩余秒数或 None}。"""
|
||||
t0, info = s['start'], s.get('info') or {}
|
||||
if st == 'gen':
|
||||
# 分子算全部(含之前轮次已完成的分片),分母是累计目标:补量时显示 50,008/79,688 而不是 0/79,688。
|
||||
@@ -165,43 +231,125 @@ def stage_progress(st, s):
|
||||
n = sum(LC.count(f) for f in allf)
|
||||
tgt = info.get('target_tasks') or 0
|
||||
new = sum(LC.count(f) for f in newer(f'{DATA}/shard_*.jsonl', t0) + newer(f'{DATA}/shard_*.jsonl.part', t0))
|
||||
lab = f'已生成 {n:,} / {tgt:,} 条任务' + (f'(本轮补 {new:,} 条)' if 0 < new < n else '')
|
||||
return {'frac': min(1, n / tgt) if tgt else None, 'label': lab}
|
||||
# 剩余时间按"本轮已写条数 ÷ 本轮已用时"外推(条数是实时数出来的,速率就是实测的),不再用静态估时
|
||||
el = max(1.0, time.time() - t0)
|
||||
rate = new / el if new > 0 else 0
|
||||
eta = ((tgt - n) / rate if rate > 0 else None) if tgt > n else (0.0 if rate > 0 else None)
|
||||
lab = f'已生成 {n:,} / {tgt:,} 条任务' + (f'(本轮补 {new:,} 条 · {rate * 60:.0f} 条/分)' if 0 < new < n else '')
|
||||
return {'frac': min(1, n / tgt) if tgt else None, 'label': lab, 'eta_s': eta}
|
||||
if st == 'gate1':
|
||||
# 一致性门禁每 10 个任务往 stderr 打一行 "k/300"(consistency.cjs:41)。进度只读控制台日志最后 64 KB,
|
||||
# 不跑任何计算、不占 CPU —— 正在生成的节点不受影响。日志里可能还留着上一次门禁的进度行,
|
||||
# 所以只数最后一个 "consistency.cjs" 命令行之后的部分。
|
||||
txt = console(0).get('text') or ''
|
||||
cut = txt.rfind('consistency.cjs')
|
||||
m = re.findall(r'^(\d+)/(\d+)\s*$', txt[cut:] if cut >= 0 else '', re.M)
|
||||
n, N = (int(m[-1][0]), int(m[-1][1])) if m else (0, 300)
|
||||
el = time.time() - t0
|
||||
rate = n / el if n > 0 and el > 2 else 0
|
||||
return {'frac': n / N if N else None,
|
||||
'label': f'门禁 {n} / {N} 个任务' + (f' · {rate * 60:.0f} 个/分' if rate else '') if n else '准备中(日志里还没有进度行)',
|
||||
'eta_s': (N - n) / rate if rate > 0 else None}
|
||||
if st == 'prep':
|
||||
nsh = len(glob.glob(os.path.join(ROOT, DATA, 'shard_*.jsonl')))
|
||||
done = len(newer(f'{DATA}/samples/*.X.npy', t0))
|
||||
if os.path.exists(os.path.join(ROOT, DATA, 'r2.json')) and newer(f'{DATA}/r2.json', t0): return {'frac': 1, 'label': '打包完成,R2 检查完成'}
|
||||
return {'frac': min(0.95, done / nsh) if nsh else None, 'label': f'已打包 {done} / {nsh} 个分片' + (',R2 检查中' if nsh and done >= nsh else '')}
|
||||
fin = newer(f'{DATA}/samples/*.X.npy', t0)
|
||||
done = len(fin)
|
||||
if os.path.exists(os.path.join(ROOT, DATA, 'r2.json')) and newer(f'{DATA}/r2.json', t0): return {'frac': 1, 'label': '打包完成,R2 检查完成', 'eta_s': 0}
|
||||
# 2026-10-19:build_dataset 每 10% 打一行 [prep] shard_xxxx a/b;用它把"整片完成"之间的进度和 ETA 算出来
|
||||
fin_n = {os.path.basename(p).split('.')[0] for p in fin}
|
||||
part = {}
|
||||
for ln in console(0)['text'].splitlines()[-800:]:
|
||||
m = re.search(r'\[prep\] (shard_\d+) (\d+)/(\d+)', ln)
|
||||
if m: part[m.group(1)] = int(m.group(2)) / max(1, int(m.group(3)))
|
||||
run = {k: v for k, v in part.items() if k not in fin_n}
|
||||
frac = (done + sum(run.values())) / nsh if nsh else None
|
||||
el = time.time() - t0
|
||||
eta = el * (1 - frac) / frac if frac and 0.02 < frac < 1 and el > 5 else None
|
||||
lab = f'已打包 {done} / {nsh} 个分片'
|
||||
if run: lab += f' · 进行中 {len(run)} 个(平均 {100 * sum(run.values()) / len(run):.0f}%)'
|
||||
elif done == 0: lab += ' · 首波处理中,第一批完成后显示预估'
|
||||
elif nsh and done >= nsh: lab += ',R2 检查中'
|
||||
return {'frac': min(0.95, frac) if frac is not None else None, 'label': lab, 'eta_s': eta}
|
||||
if st == 'train':
|
||||
tag, bs = info.get('tag', 'base'), info.get('bench_steps', 2000)
|
||||
b, m = last_rec(f'bench_{tag}') if newer(f'{RUNS}/bench_{tag}/log.jsonl', t0) else None, last_rec(tag) if newer(f'{RUNS}/{tag}/log.jsonl', t0) else None
|
||||
if m and m.get('total'):
|
||||
return {'frac': (bs + m['step']) / (bs + m['total']), 'label': f'正式训练 {m["step"]:,} / {m["total"]:,} 步', 'run': tag}
|
||||
if b: return {'frac': b['step'] / (bs + 60000), 'label': f'测速 {b["step"]:,} / {bs:,} 步', 'run': f'bench_{tag}'}
|
||||
return {'frac': (bs + m['step']) / (bs + m['total']), 'label': f'正式训练 {m["step"]:,} / {m["total"]:,} 步', 'run': tag,
|
||||
'eta_s': m['eta_h'] * 3600 if m.get('eta_h') else None} # 训练脚本自己按实测样本/s 算的
|
||||
if b: return {'frac': b['step'] / (bs + 24000), # 2026-10-19:正式步数按 configs/base.json 的 24000 估
|
||||
'label': f'测速 {b["step"]:,} / {bs:,} 步', 'run': f'bench_{tag}'}
|
||||
return {'frac': 0, 'label': '加载数据中'}
|
||||
if st == 'dagger': # 2026-10-18 新增:每轮 = 采起点 → 老师重规划 → 打包 → 微调(微调占大头,进度按 0.1/0.25/0.05/0.6 分)
|
||||
tag, R = info.get('tag', 'base'), int(info.get('rounds', 2))
|
||||
k = len(newer(f'{RUNS}/{tag}_dag*/done', t0)); k = min(k, R)
|
||||
base = f'第 {min(k + 1, R)} / {R} 轮'
|
||||
if k >= R: return {'frac': 1, 'label': 'DAgger 完成'}
|
||||
D, run = f'{DATA}/dag_{tag}_r{k + 1}', f'{tag}_dag{k + 1}'
|
||||
m = last_rec(run) if newer(f'{RUNS}/{run}/log.jsonl', t0) else None
|
||||
if m and m.get('total'):
|
||||
return {'frac': (k + 0.4 + 0.6 * m['step'] / m['total']) / R, 'label': f'{base}:微调 {m["step"]:,} / {m["total"]:,} 步', 'run': run,
|
||||
'eta_s': m['eta_h'] * 3600 if m.get('eta_h') else None}
|
||||
if os.path.exists(os.path.join(ROOT, D, 'collect.json')):
|
||||
nt = sum(LC.count(f) for f in glob.glob(os.path.join(ROOT, D, 'tasks_*.jsonl')))
|
||||
nd = sum(LC.count(f) for f in glob.glob(os.path.join(ROOT, D, 'shard_dag_*.jsonl')) + glob.glob(os.path.join(ROOT, D, 'shard_dag_*.jsonl.part')))
|
||||
return {'frac': (k + 0.1 + 0.25 * (nd / nt if nt else 0)) / R, 'label': f'{base}:老师重新规划 {nd:,} / {nt:,} 个起点'}
|
||||
return {'frac': k / R, 'label': f'{base}:闭环采起点'}
|
||||
if st == 'apg':
|
||||
r = last_rec('apg') if newer(f'{RUNS}/apg/log.jsonl', t0) else None
|
||||
return {'frac': r['it'] / r['total'] if r and r.get('total') else 0, 'label': f'APG {r["it"]:,} / {r["total"]:,} 步' if r else '准备中', 'run': 'apg'}
|
||||
eta = None
|
||||
if r and r.get('total'):
|
||||
it, mn = r.get('it') or 0, r.get('min') or 0
|
||||
eta = (r['total'] - it) / (it / mn) * 60 if it > 0 and mn > 0.1 else None # 步/分实测外推
|
||||
return {'frac': r['it'] / r['total'] if r and r.get('total') else 0, 'label': f'APG {r["it"]:,} / {r["total"]:,} 步' if r else '准备中', 'run': 'apg', 'eta_s': eta}
|
||||
if st == 'mine':
|
||||
R = info.get('rounds', 6)
|
||||
k = len(newer(f'{RUNS}/mine/r*/gate.json', t0))
|
||||
cur = sorted(newer(f'{RUNS}/apg_r*/log.jsonl', t0), key=os.path.getmtime)
|
||||
frac_in, lab = 0, f'第 {k + 1} 轮:挖错出题'
|
||||
if cur and len(cur) > k:
|
||||
name = os.path.basename(os.path.dirname(cur[-1])); r = last_rec(name)
|
||||
if r and r.get('total'): frac_in = 0.3 + 0.6 * r['it'] / r['total']; lab = f'第 {k + 1} 轮:APG {r["it"]:,} / {r["total"]:,} 步'
|
||||
return {'frac': min(1, (k + frac_in) / R), 'label': f'{lab}(共最多 {R} 轮,提升不足会提前停)', 'run': name}
|
||||
lab = f'第 {k + 1} 轮:挖错出题'
|
||||
if cur:
|
||||
name = os.path.basename(os.path.dirname(cur[-1])) # ml/runs/apg_rN
|
||||
rn = re.sub(r'\D', '', name) or '1'
|
||||
gate = os.path.join(ROOT, RUNS, 'mine', f'r{rn}', 'gate.json')
|
||||
r = last_rec(name)
|
||||
if r and r.get('total') and not os.path.exists(gate): # 这一轮还在跑(还没出 gate)
|
||||
frac_in = 0.3 + 0.6 * r['it'] / r['total']
|
||||
lab = f'第 {k + 1} 轮:APG {r["it"]:,} / {r["total"]:,} 步'
|
||||
it, mn = r.get('it') or 0, r.get('min') or 0
|
||||
eta = (r['total'] - it) / (it / mn) * 60 if it > 0 and mn > 0.1 else None
|
||||
if eta is not None and k >= 1: # 剩余轮次按已完成轮次的平均时长外推
|
||||
eta += max(0, R - k - 1) * (time.time() - t0) / k
|
||||
return {'frac': min(1, (k + frac_in) / R), 'label': f'{lab}(共最多 {R} 轮,提升不足会提前停)', 'run': name, 'eta_s': eta}
|
||||
return {'frac': min(1, k / R), 'label': f'{lab}(共最多 {R} 轮,提升不足会提前停)'}
|
||||
if st == 'm5':
|
||||
tag = info.get('tag', 'base')
|
||||
n = len(newer(f'{RUNS}/m5_{tag}_policy.json', t0)) + len(newer(f'{RUNS}/m5_{tag}_safety.json', t0))
|
||||
return {'frac': n / 2, 'label': ['纯策略评估中', '加安全层评估中', '汇总'][min(n, 2)]}
|
||||
el = time.time() - t0
|
||||
eta = (2 - n) * (el / n) if n == 1 else (0.0 if n >= 2 else None) # 两次评估(纯策略 / 加安全层)按已完成的平均耗时外推
|
||||
return {'frac': n / 2, 'label': ['纯策略评估中', '加安全层评估中', '汇总'][min(n, 2)], 'eta_s': eta}
|
||||
return {'frac': None, 'label': '进行中'}
|
||||
|
||||
|
||||
def stage_elapsed(pl, now=None):
|
||||
"""每个阶段的累计用时(秒):pipeline.json 的 time{阶段: 秒} 是不含当前段的累计,运行中的阶段
|
||||
再把"这一段"实时加上。老记录(没有 time 字段)用 start/end 的差值兜底。
|
||||
网页的「总用时」=各阶段之和,「本步用时」=当前阶段这条 —— 确定/续跑都不归零。"""
|
||||
now = now or time.time()
|
||||
acc = pl.get('time') or {}
|
||||
cur = pl.get('current')
|
||||
out = {}
|
||||
for k, r in (pl.get('stages') or {}).items():
|
||||
t = acc.get(k)
|
||||
if t is None and r.get('start') and r.get('end'): t = r['end'] - r['start']
|
||||
t = t or 0
|
||||
if k == cur and r.get('start') and r.get('end') is None: t += max(0.0, now - r['start'])
|
||||
out[k] = round(t, 1)
|
||||
return out
|
||||
|
||||
|
||||
def pipeline():
|
||||
d = jload(os.path.join(ROOT, RUNS, 'pipeline.json'))
|
||||
if not d: return None
|
||||
d['elapsed'] = stage_elapsed(d)
|
||||
cur = d.get('current')
|
||||
if cur and cur in d['stages']:
|
||||
try: d['progress'] = stage_progress(cur, d['stages'][cur])
|
||||
@@ -228,7 +376,8 @@ def status():
|
||||
if pb: pb = {k: {kk: v.get(kk) for kk in ('n', 'ok', 'reached', 'collisionFrames', 'brakeRate', 'recoveryRate', 'livelock', 'inferMsP99', 'safetyMsP99')}
|
||||
for k, v in pb.items() if isinstance(v, dict) and 'n' in v}
|
||||
return {'now': time.time(), 'gpu': gpu(), 'cpu': cpu(),
|
||||
'vram_budget_gb': float(os.environ.get('VRAM_BUDGET_GB', 20)), # 本机桌面常驻占 4–5 GB,给训练的预算 19–20 GB
|
||||
'vram_reserve_gb': float(os.environ.get('VRAM_RESERVE_GIB', 1)), # 给系统留的余量(GB):训练进程上限 = 整卡 − 已用 − 这个余量(浮动,见 ml/common/vram.py)
|
||||
'vram_budget_gb': float(os.environ['VRAM_BUDGET_GB']) if os.environ.get('VRAM_BUDGET_GB') else None, # 只有显式指定才覆盖上面的口径
|
||||
'pipeline': pl, 'check': CHECK.get(), 'lock': lock_info(), # 2026-10-18 第八轮修:上一版这三项被行尾注释吞掉,网页步骤条/锁状态拿不到
|
||||
'stage_meta': [{'id': k, 'name': n, 'est_h': h, 'timeout_h': pstate.TIMEOUT_H[k]} for k, n, h in pstate.STAGES],
|
||||
'r4': jload(P(RUNS, 'r4.json')),
|
||||
@@ -243,7 +392,7 @@ def status():
|
||||
STAGES = pstate.ORDER
|
||||
ENV_KEYS = {'N_TASKS': r'\d+', 'NPROC': r'\d+', 'TARGET_SAMPLES': r'\d+', 'CFG': r'ml/configs/(base|wta|small)\.json',
|
||||
'CONSENSUS': r'[01]?', 'ROUNDS': r'\d+', 'APG_STEPS': r'\d+', 'MINE_N': r'\d+', 'MIN_GAP_MM': r'\d+(\.\d+)?',
|
||||
'FRESH': r'[01]', 'R2_STRICT': r'[01]', 'AUTO_WTA': r'[01]'}
|
||||
'FRESH': r'[01]', 'FRESH_FROM': r'(setup|kin|r4|gate1|gen|prep|train|dagger|m5|apg|mine|export|bench|accept)', 'R2_STRICT': r'[01]', 'AUTO_WTA': r'[01]'}
|
||||
CONSOLE = os.path.join(ROOT, RUNS, 'console.log')
|
||||
|
||||
|
||||
@@ -299,6 +448,29 @@ def lock_info():
|
||||
return dict(lk, alive=pstate.lock_alive(lk), age_s=time.time() - lk.get('beat', 0))
|
||||
|
||||
|
||||
NT_NAMES = "'node.exe','xargs.exe','bash.exe','sh.exe','timeout.exe'"
|
||||
NT_SWEEP = (r"pipeline\.sh|run_prep\.sh|run_parallel\.sh|build_dataset\.cjs|"
|
||||
r"gen_dataset\.cjs|teacher\.cjs|policy_bench\.cjs")
|
||||
|
||||
|
||||
def sweep_nt(keep_pid):
|
||||
"""taskkill /T 在 Git Bash 下只杀到 bash,xargs/node 孙进程会逃(10-18 实测:停止后还有 28 个
|
||||
gen_dataset 在跑;当晚二次踩到:两批 build_dataset 抢同一批 shard)。按命令行补扫整条流水线,
|
||||
keep_pid 的祖先链(= 控制台自己拉起的 monitor bash)不动。先杀包装再杀 node,否则 node 一死
|
||||
xargs 会立刻补下一个 shard。返回仍存活个数,-1 = pwsh 没跑成。"""
|
||||
q = ("$anc=@(); $c=%d; while ($c -gt 0 -and ($p=Get-CimInstance Win32_Process -Filter \"ProcessId=$c\" "
|
||||
"-ErrorAction SilentlyContinue)) { $anc+=$p.ProcessId; $c=$p.ParentProcessId }; "
|
||||
"$m={ $_.Name -in %s -and $_.CommandLine -match '%s' -and $anc -notcontains $_.ProcessId }; "
|
||||
"foreach ($x in @(Get-CimInstance Win32_Process | Where-Object $m | "
|
||||
"Sort-Object { if ($_.Name -eq 'node.exe') { 1 } else { 0 } })) "
|
||||
"{ Stop-Process -Id $x.ProcessId -Force -ErrorAction SilentlyContinue }; "
|
||||
"Start-Sleep -Seconds 2; "
|
||||
"@(Get-CimInstance Win32_Process | Where-Object $m).Count" % (int(keep_pid), NT_NAMES, NT_SWEEP))
|
||||
r = subprocess.run(['pwsh', '-NoProfile', '-Command', q], capture_output=True, text=True, timeout=90)
|
||||
lines = [s for s in (r.stdout or '').split() if s.isdigit()]
|
||||
return int(lines[-1]) if lines else -1
|
||||
|
||||
|
||||
def kill_session(sid):
|
||||
"""POSIX:结束会话 sid 里的所有进程(含 timeout 另起的进程组)。"""
|
||||
pids = []
|
||||
@@ -366,7 +538,7 @@ class Job:
|
||||
pid = self.p.pid if ours else int(lk.get('pid') or 0)
|
||||
if not pid: return {'error': '锁里没有 pid,无法停止'}
|
||||
cur = (pstate.load().get('current') or '')
|
||||
if cur in ('train', 'm5', 'apg', 'mine'):
|
||||
if cur in ('train', 'dagger', 'm5', 'apg', 'mine'):
|
||||
open(STOPF, 'w').write(time.strftime('%H:%M:%S'))
|
||||
with open(CONSOLE, 'a', encoding='utf-8') as f: f.write(f'\n[停止请求已发出,等待 {cur} 存档 last.pt…]\n')
|
||||
for _ in range(180):
|
||||
@@ -376,14 +548,14 @@ class Job:
|
||||
with self.lock:
|
||||
if os.name == 'nt':
|
||||
subprocess.run(['taskkill', '/T', '/F', '/PID', str(pid)], capture_output=True)
|
||||
# Git Bash 下 taskkill /T 只杀到 bash,node 孙进程会逃掉(2026-10-18 实测:点了停止还有 28 个
|
||||
# gen_dataset 在跑)。按命令行补扫:只动本仓库 ml/ 下的脚本,绝不碰用户自己的 node 进程。
|
||||
# 补扫漏网的子孙;有残留就写进网页日志,别让用户以为停干净了
|
||||
try:
|
||||
q = ("Get-CimInstance Win32_Process -Filter \"Name='node.exe'\" | "
|
||||
"Where-Object { $_.CommandLine -match 'gen_dataset\\.cjs|run_parallel\\.sh|teacher\\.cjs|policy_bench\\.cjs' } | "
|
||||
"ForEach-Object { Stop-Process -Id $_.ProcessId -Force -ErrorAction SilentlyContinue }")
|
||||
subprocess.run(['pwsh', '-NoProfile', '-Command', q], capture_output=True, timeout=60)
|
||||
except Exception: pass
|
||||
left = sweep_nt(os.getpid())
|
||||
except Exception:
|
||||
left = -1
|
||||
if left != 0:
|
||||
with open(CONSOLE, 'a', encoding='utf-8') as f:
|
||||
f.write(f'\n[停止补扫:仍有 {left} 个流水线子进程存活(-1=补扫没跑成),把这条发我]\n')
|
||||
else:
|
||||
if ours:
|
||||
kill_session(pid)
|
||||
|
||||
+38
-10
@@ -7,6 +7,7 @@
|
||||
# gen 生成老师轨迹(NPROC 进程;CONSENSUS=1 时加共识过滤;样本不足 TARGET_SAMPLES 自动补生成)
|
||||
# prep 打包训练样本 + R2 标签一致性检查
|
||||
# train 预训练(先 2000 步摸吞吐,再正式训练);输出目录随头分开:ml/runs/<CFG 名>
|
||||
# dagger DAgger 纠偏(2026-10-18 新增):策略闭环走到的状态请老师重新规划、并入数据微调 DAG_ROUNDS 轮(ml/dagger/run.sh)
|
||||
# m5 头门禁:贴立柱集穿柱率 ≤ 2%(纯策略口径),同时报仿真安全层口径;通过则记为选定模型,不过自动训 WTA 头再判
|
||||
# apg APG 首轮微调(从选定模型开始)
|
||||
# mine 挖错循环(3–6 轮,含门禁回滚)
|
||||
@@ -45,10 +46,11 @@ SEL=ml/runs/selected_ckpt
|
||||
ckpt() { if [ -n "${CKPT:-}" ]; then echo "$CKPT"; elif [ -f $SEL ]; then cat $SEL; else echo ml/runs/$TAG/best.pt; fi; }
|
||||
stage=${1:-help}
|
||||
run() { echo "== $*" >&2; "$@"; } # 回显走 stderr:gate1 用 tee 把 stdout 存成 consistency.json,回显混进去会让后面读它时 JSONDecodeError
|
||||
ALL_STAGES="setup kin r4 gate1 gen prep train m5 apg mine export bench accept"
|
||||
ALL_STAGES="setup kin r4 gate1 gen prep train dagger m5 apg mine export bench accept"
|
||||
stage_info() { case $1 in
|
||||
gen) echo "target_tasks=$(( (N_TASKS + NPROC - 1) / NPROC * NPROC ))" ;;
|
||||
train) echo "tag=$TAG bench_steps=2000" ;;
|
||||
dagger) echo "tag=$TAG rounds=${DAG_ROUNDS:-2}" ;;
|
||||
m5) echo "tag=$TAG" ;;
|
||||
mine) echo "rounds=$ROUNDS apg_steps=$APG_STEPS" ;;
|
||||
esac; }
|
||||
@@ -76,15 +78,37 @@ on_exit() {
|
||||
trap on_exit EXIT
|
||||
trap 'exit 143' TERM INT HUP # 被停止 / 超时:先走 on_exit 记返回码、放锁
|
||||
# ---- 训练与门禁的单元(m5 自动分支要对两种头各调一次)
|
||||
replay_ok() { # 回放自检结果有效:pass 为真、版本含 oracle 滚动(ver ≥ 2)、指纹 = 当前 kin.json 且 = 数据 shard 的指纹
|
||||
$PY - <<'PY'
|
||||
import json, sys
|
||||
try:
|
||||
r = json.load(open('ml/runs/replay_teacher.json', encoding='utf-8')); k = json.load(open('ml/assets/kin.json', encoding='utf-8'))['fp']
|
||||
except Exception: sys.exit(1)
|
||||
sys.exit(0 if r.get('pass') is True and r.get('ver', 1) >= 2 and r.get('fp') == k and r.get('data_fp') in (None, k) else 1)
|
||||
PY
|
||||
}
|
||||
train_one() { # $1 = 配置文件
|
||||
local cfg=$1 tag; tag=$(basename "$1" .json)
|
||||
# 2026-10-18:开训前先把老师轨迹放进 PyTorch 仿真口径里回放——口径对不上时闭环指标不可信,训多少步都白训(rc 5 停)
|
||||
# 2026-10-19(审查 review_closedloop F6):原来只看文件在不在——replay 失败时也写了文件(pass=false)再 rc 5,用户直接重跑 train 门禁就被跳过。
|
||||
# 现在读 pass 字段 + 指纹(kin.json、数据 shard),不过就重跑一次,仍不过 rc 5;回放同时走完整 rollout()(老师 oracle),覆盖滚动路径。
|
||||
replay_ok || run $PY -m ml.eval.replay_teacher --n 512 --out ml/runs/replay_teacher.json || true
|
||||
replay_ok || { echo "老师回放自检未通过(ml/runs/replay_teacher.json 的 pass / 指纹),闭环指标不可信,不开训"; exit 5; }
|
||||
if [ -f ml/runs/bench_$tag/last.pt ]; then echo "吞吐摸底 ml/runs/bench_$tag 已有,跳过"
|
||||
else run $PY -m ml.train --samples ml/data/samples --out ml/runs/bench_$tag --steps ${BENCH_STEPS:-2000} --no-eval --config $cfg; fi
|
||||
local res=""; [ -f ml/runs/$tag/last.pt ] && { res="--resume ml/runs/$tag/last.pt"; echo "发现 ml/runs/$tag/last.pt → 续训"; }
|
||||
run $PY -m ml.train --samples ml/data/samples --out ml/runs/$tag --config $cfg $res
|
||||
}
|
||||
m5_one() { # $1 = 头标签;返回 0 通过 / 3 不通过
|
||||
local tag=$1 ck=${CKPT:-ml/runs/$1/best.pt} o
|
||||
dagger_one() { # $1 = 头标签:从该头的预训练结果起做 DAgger,最终模型路径写进 ml/runs/<tag>_dagger.ckpt
|
||||
local tag=$1 ck
|
||||
[ -f ml/runs/$tag/best.pt ] || [ -f ml/runs/$tag/last.pt ] || { echo "DAgger:ml/runs/$tag/ 下没有 checkpoint(预训练没完成)"; return 3; }
|
||||
# 2026-10-19(审查 P0-2):闭环 success > 0 用 best.pt,否则 last.pt(都取 EMA 权重)
|
||||
ck=$($PY -m ml.common.ckpt pick ml/runs/$tag)
|
||||
run bash ml/dagger/run.sh $tag $ck
|
||||
}
|
||||
m5_one() { # $1 = 头标签;返回 0 通过 / 3 不通过。起点模型:CKPT > DAgger 结果 > 预训练 best.pt
|
||||
local tag=$1 ck o
|
||||
if [ -n "${CKPT:-}" ]; then ck=$CKPT; elif [ -f ml/runs/${1}_dagger.ckpt ]; then ck=$(cat ml/runs/${1}_dagger.ckpt); else ck=ml/runs/$1/best.pt; fi
|
||||
[ -f "$ck" ] || { echo "M5:$ck 不存在(训练发散或没训完)→ 按不通过处理"; return 3; }
|
||||
# 断点续跑:结果文件比检查点新 → 这一半已经评过,跳过
|
||||
o=ml/runs/m5_${tag}_policy.json; [ -f $o ] && [ $o -nt "$ck" ] && echo "M5:$o 已有,跳过" || run $PY -m ml.sim.closed_loop --ckpt $ck --tasks ml/runs/nc_gate.jsonl --n ${M5_N:-1000} --out $o
|
||||
@@ -127,19 +151,21 @@ case $stage in
|
||||
extra=""; [ "${CONSENSUS:-0}" = "1" ] && extra="--consensus 4"
|
||||
run bash ml/gen/run_parallel.sh $per $NPROC 1 "$extra"
|
||||
# 补量:共识过滤与间隙过滤都会降低保留率,样本不足 TARGET_SAMPLES 时按实测"样本/任务"补生成。
|
||||
# 最多 4 次(第五轮由 2 改 4:两种过滤叠加时 2 次可能补不满);已写完的补量 shard 续跑时跳过。
|
||||
# 最多 5 次(第五轮由 2 改 4;2026-10-18 应用户要求 4 改 5:共识 + 间隙两道过滤叠加时余量更足);已写完的补量 shard 续跑时跳过。
|
||||
s0=$((1 + NPROC))
|
||||
for k in 1 2 3 4; do
|
||||
for k in 1 2 3 4 5; do
|
||||
need=$($PY -c "import json,math;s=json.load(open('ml/data/stats.json', encoding='utf-8'));d=$TARGET_SAMPLES-s['samplesEvery3'];print(0 if d<=0 else math.ceil(d/(s['samplesEvery3']/max(1,s['tasks']))*1.05))")
|
||||
[ "$need" = "0" ] && break
|
||||
echo "样本不足目标 $TARGET_SAMPLES,补生成 $need 条任务(第 $k 次)"
|
||||
per2=$(( (need + NPROC - 1) / NPROC ))
|
||||
old=$($PY -m ml.pstate get gen target_tasks) # 读不到直接失败(审查 N-13:此前空值会被算成 56 这类数)
|
||||
$PY -m ml.pstate set gen target_tasks=$(( old + per2 * NPROC ))
|
||||
# 目标 = 已写完的分片条数 + 本轮要补的;不能用 pstate 里上次的 target_tasks 累加,
|
||||
# 那样每次续跑(点停止后再「继续训练」)分母都加一截,进度条会越续越少(2026-10-18 实测 67368→84728)。
|
||||
done_n=$($PY -c "import json;print(json.load(open('ml/data/stats.json', encoding='utf-8'))['tasks'])")
|
||||
$PY -m ml.pstate set gen target_tasks=$(( done_n + per2 * NPROC ))
|
||||
run bash ml/gen/run_parallel.sh $per2 $NPROC $s0 "$extra"
|
||||
s0=$((s0 + NPROC))
|
||||
done
|
||||
$PY -c "import json;s=json.load(open('ml/data/stats.json', encoding='utf-8'));import sys;sys.exit(0 if s['samplesEvery3']>=0.9*$TARGET_SAMPLES else 7)" || echo "警告:补 4 次后样本仍不足目标的 90%,按现有样本继续(stats.json 见 samplesEvery3)" ;;
|
||||
$PY -c "import json;s=json.load(open('ml/data/stats.json', encoding='utf-8'));import sys;sys.exit(0 if s['samplesEvery3']>=0.9*$TARGET_SAMPLES else 7)" || echo "警告:补 5 次后样本仍不足目标的 90%,按现有样本继续(stats.json 见 samplesEvery3)" ;;
|
||||
prep) # 间隙过滤口径必须与生成时一致(审查 N-1)
|
||||
$PY -c "import json,sys;m=json.load(open('ml/data/stats.json', encoding='utf-8')).get('minGapFilterMm');sys.exit(0 if m is None or float(m)==float($MIN_GAP_MM) else 8)" \
|
||||
|| { echo "MIN_GAP_MM=$MIN_GAP_MM 与生成阶段 stats.json 的 minGapFilterMm 不一致:补量口径与训练集会分叉,请用同一值重跑 gen 或 prep"; exit 8; }
|
||||
@@ -152,6 +178,7 @@ case $stage in
|
||||
echo "警告:R2 不通过(近邻标签不自洽,见 ml/data/r2.json)。按默认继续训练,验收汇总会列出"
|
||||
fi ;;
|
||||
train) train_one $CFG ;;
|
||||
dagger) dagger_one $TAG ;;
|
||||
m5) [ -f ml/runs/nc_gate.jsonl ] || run $PY -m ml.mining.mine_round --make-gate ${M5_N:-1000} --seed 4321 --mix 0,0,1,0 --name nc_gate.jsonl --out ml/runs
|
||||
if m5_one $TAG; then :
|
||||
elif [ "$TAG" != wta ] && [ "${AUTO_WTA:-1}" = 1 ] && [ -z "${CKPT:-}" ]; then
|
||||
@@ -159,11 +186,12 @@ case $stage in
|
||||
echo "M5:$TAG 头不通过 → 自动训练 WTA 回退头(约 3 小时)"
|
||||
$PY -m ml.pstate set m5 branch=wta
|
||||
train_one ml/configs/wta.json
|
||||
dagger_one wta
|
||||
m5_one wta || { echo "M5:两种头都未通过,把 ml/runs/m5_gate_*.json 发我"; exit 3; }
|
||||
elif [ "${M5_STRICT:-1}" = 0 ]; then echo "M5:不通过,M5_STRICT=0(冒烟)→ 照常继续"; echo ml/runs/$TAG/best.pt > $SEL
|
||||
else echo "M5:不通过,把 ml/runs/m5_gate_*.json 发我"; exit 3; fi ;;
|
||||
apg) ck=$(ckpt); res=""; [ -f ml/runs/apg/last.pt ] && { res="--resume ml/runs/apg/last.pt"; echo "发现 ml/runs/apg/last.pt → 续跑"; }
|
||||
run $PY -m ml.apg.apg_finetune --init $ck --samples ml/data/samples --out ml/runs/apg ${APG_FIRST_STEPS:+--steps $APG_FIRST_STEPS} $res; cp ml/runs/apg/best.pt ml/runs/current.pt ;;
|
||||
run $PY -m ml.apg.apg_finetune --init $ck --samples ml/data/samples $(ls -d ml/data/samples_dag_* 2>/dev/null | tr '\n' ' ') --out ml/runs/apg ${APG_FIRST_STEPS:+--steps $APG_FIRST_STEPS} $res; cp ml/runs/apg/best.pt ml/runs/current.pt ;;
|
||||
mine) run bash ml/mining/loop.sh $ROUNDS $APG_STEPS $MINE_N ;;
|
||||
monitor) run $PY -m ml.monitor ${MONITOR_ARGS:-} ;;
|
||||
selftest) run bash ml/selftest.sh ;;
|
||||
@@ -184,7 +212,7 @@ case $stage in
|
||||
# 启动前自检(审查 N-10:不能等到 train 才发现这个 Python 没有 torch)
|
||||
$PY -c "import torch" 2>/dev/null || { echo "当前 Python($($PY -c 'import sys;print(sys.executable)'))没有 torch:请先激活训练环境或设 PYTHON=..."; exit 1; }
|
||||
command -v node >/dev/null || { echo "找不到 node"; exit 1; }
|
||||
fl=""; [ "${FRESH:-0}" = 1 ] && fl="--fresh"
|
||||
fl=""; [ "${FRESH:-0}" = 1 ] && fl="--fresh"; [ -n "${FRESH_FROM:-}" ] && fl="--fresh-from=$FRESH_FROM"
|
||||
$PY -m ml.pstate reset ${PLAN// /,} $fl # 续跑 / 参数变了作废受影响阶段(旧产物移到 ml/archive/<时间>/)
|
||||
TODO=$($PY -m ml.pstate todo)
|
||||
echo "计划:$PLAN"; echo "待跑:${TODO:-(全部已完成)}"
|
||||
|
||||
+49
-19
@@ -2,16 +2,33 @@
|
||||
* node ml/prep/build_dataset.cjs ml/data/shard_0001.jsonl [outDir=ml/data/samples]
|
||||
* 每条 keep=true 的轨迹按 50 Hz 时间律重放,每 3 步取一个样本:
|
||||
* X [N,83] 特征(webapp/js/ik-policy.js features(),与部署同一份代码)
|
||||
* Y [N,16,6] 未来 16 步每步增量 / STEP(到终点后补 0)
|
||||
* Y [N,16,6] 未来 16 步每步增量 / (ACT_SCALE·STEP),夹在 ±1(到终点后补 0)
|
||||
* V [N] 剩余时间(s)/ 5
|
||||
* SG [N,13] 子目标标签:剩余两个途经点相对当前 q 的差 / 180(不足用终点补)+ 是否需要
|
||||
* TID[N] 任务编号(tasks.jsonl 行号所在的全局 id,用于划分与闭环验证)
|
||||
* 增广:30% 样本对 q_t 加 σ=1° 噪声,标签改为 8 步内线性收回老师轨迹(噪声态须无干涉且间隙 > 1 mm);
|
||||
* 已提交前缀长度 r ∈ {0..3} 均匀随机,前缀取老师接下来 r 步的增量。 */
|
||||
* 增广(2026-10-18 首次正式预训练后加大):40% 样本对 q_t 加 σ=AUG_SIGMA(默认 3°)噪声,标签改为 AUG_BACK(默认 12)步内线性收回老师轨迹;
|
||||
* 噪声态须无干涉且间隙 > MIN_GAP_MM,收回途中第 3/6/9/12 步也须无干涉(σ 加大后直接线性拉回可能穿过障碍)。
|
||||
* 原为 30%、σ=1°、8 步:闭环一偏出 1° 就是没见过的状态,首次 21850 步闭环到达 0%。更大的偏离由 DAgger 阶段补(ml/dagger)。
|
||||
* 已提交前缀长度 r ∈ {0..3} 均匀随机,前缀取老师接下来 r 步的增量。
|
||||
* meta [<shard>.meta.json] 打包参数:data_ver、act_scale、增广参数、sig(run_prep.sh 据 sig 判断要不要重新打包)。
|
||||
* 2026-10-19(审查 review_data_actions C2):
|
||||
* - 标签按 ACT_SCALE·STEP 归一化(默认 1.25,DATA_VER 2)。原来按 STEP 归一化,巡航时主导轴标签恰好 = ±1、执行端又夹在 ±1,
|
||||
* 模型在 ±1 附近的对称误差有一半被截掉,主导轴系统性偏慢;留 25% 余量后老师标签最大 0.8,执行端的 ±1 只防越界。
|
||||
* 运行时 inc = clamp(act, ±1)·act_scale·STEP(closed_loop.py / apg / ik-policy.js 同口径,act_scale 经 checkpoint 与 ONNX meta 传递);
|
||||
* - 增广收回标签的亏空结转:被 ±1 截掉的部分记在 prevP 里、下一步补,原来 prevP = 目标点,截掉就丢了(教的是「落后了别追」);
|
||||
* - DAgger 任务带 qPrev / v0(gen_dataset.cjs 写入):t=0 样本用真实上一帧,时间律从当前速度起步。
|
||||
* 特征(83 维)定义不变,FEAT_VER 不变;数据版本由 meta.data_ver 区分,旧打包没有 meta → 训练按 act_scale 1.0 读。 */
|
||||
'use strict';
|
||||
const fs = require('fs'), path = require('path');
|
||||
/* 打包参数(环境变量)与签名:run_prep.sh 用 `node build_dataset.cjs --sig` 取当前签名,与已打包 shard 的 meta.sig 不同就重新打包 */
|
||||
const DATA_VER = 2;
|
||||
const MIN_GAP_MM = +(process.env.MIN_GAP_MM || 3);
|
||||
const AUG_P = +(process.env.AUG_P || 0.4), AUG_SIGMA = +(process.env.AUG_SIGMA || 3), AUG_BACK = +(process.env.AUG_BACK || 12);
|
||||
const ACT_SCALE = +(process.env.ACT_SCALE || 1.25);
|
||||
const SIG = ['v' + DATA_VER, 's' + ACT_SCALE, 'p' + AUG_P, 'sg' + AUG_SIGMA, 'b' + AUG_BACK, 'g' + MIN_GAP_MM].join('|');
|
||||
if (process.argv[2] === '--sig') { console.log(SIG); process.exit(0); }
|
||||
const IN = process.argv[2], OUTDIR = process.argv[3] || path.join(__dirname, '../data/samples');
|
||||
if (!IN) { console.error('usage: build_dataset.cjs <shard.jsonl> [outDir]'); process.exit(1); }
|
||||
if (!IN) { console.error('usage: build_dataset.cjs <shard.jsonl> [outDir] | --sig'); process.exit(1); }
|
||||
const { S } = require('../../tests/route_bench_setup.cjs')();
|
||||
const POL = require('../../webapp/js/ik-policy.js');
|
||||
const { timeParam } = require('../gen/timelaw.cjs');
|
||||
@@ -26,35 +43,45 @@ const hash = s => { let h = 2166136261; for (const c of s) h = Math.imul(h ^ c.c
|
||||
const X = [], Y = [], V = [], SG = [], TID = [], tasks = [];
|
||||
const lines = fs.readFileSync(IN, 'utf8').split('\n').filter(Boolean);
|
||||
let nAug = 0, nAugRej = 0, nGapDrop = 0;
|
||||
/* 2026-10-18:认证最小间隙 < 3 mm 的老师轨迹不进训练集;噪声增广接受地板同步为 3 mm(原 1 mm)。 */
|
||||
const MIN_GAP_MM = +(process.env.MIN_GAP_MM || 3);
|
||||
for (const line of lines) {
|
||||
/* 2026-10-18:认证最小间隙 < 3 mm 的老师轨迹不进训练集(MIN_GAP_MM);噪声增广接受地板同步为 3 mm(原 1 mm)。 */
|
||||
const ASTEP = STEP.map(v => v * ACT_SCALE); // 每单位动作对应的度数
|
||||
const NT = lines.length, PSTEP = Math.max(1, Math.floor(NT / 10));
|
||||
for (let li = 0; li < NT; li++) {
|
||||
if (li % PSTEP === 0) console.log(`[prep] ${base} ${li}/${NT}`); // 每 10% 一行,28 个分片并行也看得见进度
|
||||
const line = lines[li];
|
||||
const r = JSON.parse(line);
|
||||
if (!r.keep) continue;
|
||||
if (r.cert && r.cert.minGap != null && r.cert.minGap < MIN_GAP_MM) { nGapDrop++; continue; } // 与 stats.cjs 同一过滤
|
||||
const tid = hash(r.id) & 0x7fffffff;
|
||||
const split = (hash(r.id + '#split') % 100) < 5 ? 'val' : 'train';
|
||||
const P = r.path;
|
||||
const tp = timeParam(P);
|
||||
const tp = timeParam(P, { v0: r.v0 || 0 }); // DAgger 起点带速度:时间律从 v0 起步(gen_dataset.cjs 写 v0)
|
||||
const traj = tp.traj, taus = tp.taus, N = traj.length - 1;
|
||||
tasks.push({ tid, id: r.id, cat: r.cat, mode: r.mode, q0: r.q0, T: r.T, R: r.R, exec_s: N * POL.DT, travel: r.travel_deg, reached: r.reached, err: r.err, split });
|
||||
tasks.push({ tid, id: r.id, cat: r.cat, mode: r.mode, q0: r.q0, qp0: r.qPrev || null, T: r.T, R: r.R, exec_s: N * POL.DT, travel: r.travel_deg, reached: r.reached, err: r.err, split });
|
||||
const vt = tp.cum; // 顶点 τ
|
||||
const inter = P.slice(1, -1).map((p, i) => ({ p, tau: vt[i + 1] }));
|
||||
for (let t = 0; t <= N; t += 3) {
|
||||
let q = traj[t].slice(), qPrev = (t > 0 ? traj[t - 1] : traj[0]).slice();
|
||||
let q = traj[t].slice(), qPrev = (t > 0 ? traj[t - 1] : (r.qPrev || traj[0])).slice();
|
||||
let off = [0, 0, 0, 0, 0, 0], aug = false;
|
||||
if (rnd() < 0.3) {
|
||||
const qn = q.map((v, j) => { let x = v + gauss(); const t2 = L[O[j]]; if (!t2.cont) x = Math.max(t2.lo, Math.min(t2.hi, x)); return x; });
|
||||
if (rnd() < AUG_P) {
|
||||
const qn = q.map((v, j) => { let x = v + AUG_SIGMA * gauss(); const t2 = L[O[j]]; if (!t2.cont) x = Math.max(t2.lo, Math.min(t2.hi, x)); return x; });
|
||||
nAug++;
|
||||
if (S.collideFree(qn) && S.clearance(qn, 5) > MIN_GAP_MM) { off = qn.map((v, j) => v - q[j]); q = qn; qPrev = qPrev.map((v, j) => v + off[j]); aug = true; }
|
||||
else nAugRej++;
|
||||
const o2 = qn.map((v, j) => v - q[j]);
|
||||
const back = k => traj[Math.min(N, t + k)].map((v, j) => v + o2[j] * Math.max(0, 1 - k / AUG_BACK));
|
||||
if (S.collideFree(qn) && S.clearance(qn, 5) > MIN_GAP_MM && [3, 6, 9, 12].every(k => k >= AUG_BACK || S.collideFree(back(k)))) {
|
||||
off = o2; q = qn; qPrev = qPrev.map((v, j) => v + off[j]); aug = true;
|
||||
} else nAugRej++;
|
||||
}
|
||||
const pos = k => { const i = Math.min(N, t + k); const w = aug ? Math.max(0, 1 - k / 8) : 0; return traj[i].map((v, j) => v + off[j] * w); };
|
||||
const pos = k => { const i = Math.min(N, t + k); const w = aug ? Math.max(0, 1 - k / AUG_BACK) : 0; return traj[i].map((v, j) => v + off[j] * w); };
|
||||
const y = new Float32Array(H * 6);
|
||||
let prevP = q;
|
||||
for (let k = 1; k <= H; k++) { const pk = pos(k); for (let j = 0; j < 6; j++) y[(k - 1) * 6 + j] = Math.max(-1, Math.min(1, (pk[j] - prevP[j]) / STEP[j])); prevP = pk; }
|
||||
/* prevP = 按标签实际会走到的位置:被 ±1 截掉的亏空留在 prevP 与目标之差里,下一步接着补(2026-10-19,C2) */
|
||||
let prevP = q.slice();
|
||||
for (let k = 1; k <= H; k++) {
|
||||
const pk = pos(k);
|
||||
for (let j = 0; j < 6; j++) { const v = Math.max(-1, Math.min(1, (pk[j] - prevP[j]) / ASTEP[j])); y[(k - 1) * 6 + j] = v; prevP[j] += v * ASTEP[j]; }
|
||||
}
|
||||
const rr = Math.floor(rnd() * 4), prefix = [];
|
||||
for (let k = 0; k < rr; k++) { const row = []; for (let j = 0; j < 6; j++) row.push(y[k * 6 + j] * STEP[j]); prefix.push(row); }
|
||||
for (let k = 0; k < rr; k++) { const row = []; for (let j = 0; j < 6; j++) row.push(y[k * 6 + j] * ASTEP[j]); prefix.push(row); }
|
||||
const remain = inter.filter(v => v.tau > taus[t] + 0.02);
|
||||
const sub = remain.length && rnd() < 0.8 ? remain[0].p : null;
|
||||
const gm = S.gapMatrix(q, POL.GAP_CAP);
|
||||
@@ -74,7 +101,10 @@ writeNpy(path.join(OUTDIR, base + '.Y.npy'), cat(Y, H * 6), [n, H, 6]);
|
||||
writeNpy(path.join(OUTDIR, base + '.V.npy'), Float32Array.from(V), [n]);
|
||||
writeNpy(path.join(OUTDIR, base + '.SG.npy'), cat(SG, 13), [n, 13]);
|
||||
writeNpy(path.join(OUTDIR, base + '.TID.npy'), TID, [n], 'int32');
|
||||
/* meta 在 tasks.jsonl 之前写:tasks.jsonl 仍是「本 shard 打包完成」的标记 */
|
||||
fs.writeFileSync(path.join(OUTDIR, base + '.meta.json'), JSON.stringify({ data_ver: DATA_VER, act_scale: ACT_SCALE, aug_p: AUG_P, aug_sigma: AUG_SIGMA,
|
||||
aug_back: AUG_BACK, min_gap_mm: MIN_GAP_MM, aug_carry: true, sig: SIG }));
|
||||
// 2026-10-18 断点续跑:tasks.jsonl 最后写、先写 .tmp 再改名 —— 它存在就代表这个 shard 的 5 个 npy 都已写完,run_prep.sh 据此跳过
|
||||
const tf = path.join(OUTDIR, base + '.tasks.jsonl');
|
||||
fs.writeFileSync(tf + '.tmp', tasks.map(t => JSON.stringify(t)).join('\n') + '\n'); fs.renameSync(tf + '.tmp', tf);
|
||||
console.log(JSON.stringify({ shard: base, tasks: tasks.length, samples: n, aug: nAug, augRejected: nAugRej, gapDrop: nGapDrop, featVer: POL.FEAT_VER }));
|
||||
console.log(JSON.stringify({ shard: base, tasks: tasks.length, samples: n, aug: nAug, augRejected: nAugRej, gapDrop: nGapDrop, featVer: POL.FEAT_VER, dataVer: DATA_VER, actScale: ACT_SCALE }));
|
||||
|
||||
+6
-1
@@ -2,6 +2,8 @@
|
||||
# 并行打包样本:bash ml/prep/run_prep.sh [进程数=CPU 核数]
|
||||
# 2026-10-18 断点续跑:samples/<shard>.tasks.jsonl 存在且比 shard 新 → 该 shard 已打包完,跳过;
|
||||
# shard 已不存在的旧打包结果删掉(不让上一批数据混进来)。中断后重跑只补没打完的 shard。
|
||||
# 2026-10-19(审查 review_closedloop F11):打包参数(数据版本、ACT_SCALE、AUG_P/AUG_SIGMA/AUG_BACK、MIN_GAP_MM)变了也要重打:
|
||||
# build_dataset.cjs --sig 给出当前签名,已打包 shard 的 <shard>.meta.json 里 sig 不同(或没有 meta 的旧打包)→ 重新打包。
|
||||
set -e
|
||||
cd "$(dirname "$0")/../.."
|
||||
P=${1:-$(nproc)}
|
||||
@@ -11,10 +13,13 @@ for t in $D/*.tasks.jsonl; do
|
||||
[ -e "$t" ] || continue; b=$(basename "$t" .tasks.jsonl)
|
||||
[ -f ml/data/$b.jsonl ] || { echo "[prep] 删除旧打包 $b(对应 shard 已不存在)"; rm -f $D/$b.*; }
|
||||
done
|
||||
SIG=$(node ml/prep/build_dataset.cjs --sig)
|
||||
echo "[prep] 打包签名 $SIG"
|
||||
todo=()
|
||||
for f in ml/data/shard_*.jsonl; do
|
||||
b=$(basename "$f" .jsonl)
|
||||
if [ -f $D/$b.tasks.jsonl ] && [ $D/$b.tasks.jsonl -nt "$f" ]; then continue; fi
|
||||
if [ -f $D/$b.tasks.jsonl ] && [ $D/$b.tasks.jsonl -nt "$f" ] && grep -qF "\"sig\":\"$SIG\"" $D/$b.meta.json 2>/dev/null; then continue; fi
|
||||
rm -f $D/$b.tasks.jsonl # 先撤完成标记:重打中途断了不会被当成已完成
|
||||
todo+=("$f")
|
||||
done
|
||||
echo "[prep] 共 $(ls ml/data/shard_*.jsonl | wc -l) 个 shard,待打包 ${#todo[@]} 个"
|
||||
|
||||
+40
-14
@@ -1,6 +1,6 @@
|
||||
"""流水线状态与互斥锁(单一写者:所有对 ml/runs/pipeline.json / pipeline.lock 的写都经过本模块,原子替换)。
|
||||
pipeline.sh 自动调用,一般不需要手动运行:
|
||||
python -m ml.pstate reset <阶段1,...> [--fresh] # `all` 开始:保留 rc=0 的已完成阶段(续跑);参数变了只作废受影响的阶段;--fresh 全部作废
|
||||
python -m ml.pstate reset <阶段1,...> [--fresh] # `all` 开始:保留 rc=0 的已完成阶段(续跑);参数变了只作废受影响的阶段;--fresh 全部作废;--fresh-from=<阶段> 只作废该阶段及之后
|
||||
python -m ml.pstate todo # 打印计划中尚未完成的阶段(空格分隔)
|
||||
python -m ml.pstate begin <阶段> [键=值 ...] # 阶段开始
|
||||
python -m ml.pstate set <阶段> 键=值 ... # 阶段中途更新
|
||||
@@ -13,7 +13,9 @@ pipeline.sh 自动调用,一般不需要手动运行:
|
||||
心跳超过 LOCK_STALE_S 未刷新视为陈旧(进程被强杀),可接管。
|
||||
- 续跑(缺口 1):reset 默认保留已完成阶段;参数变了只作废受它影响的阶段(AFFECTS),其产物移到 ml/archive/,避免新旧参数混用。
|
||||
- 阶段表 STAGES 是估时 / 超时 / 网页进度权重的唯一来源(审查 N-14:网页权重与耗时表同源)。
|
||||
- 停止也走本模块(审查 N-12:此前 monitor 直写 pipeline.json,与子进程竞态)。"""
|
||||
- 停止也走本模块(审查 N-12:此前 monitor 直写 pipeline.json,与子进程竞态)。
|
||||
- 累计用时(2026-10-18):每个阶段跑过的时间并进顶部 time{阶段: 秒},跨「停止 → 继续训练」累加,
|
||||
网页的「总用时 / 本步用时」据此显示;产物被归档(要重做)的阶段清零,其余保留。"""
|
||||
import glob, json, os, shutil, socket, sys, time
|
||||
|
||||
ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
|
||||
@@ -24,27 +26,32 @@ LOCK_STALE_S = 300 # 心跳每 60 s 一次;5 分钟没刷新 = 持锁
|
||||
# 阶段顺序、名称、估时(小时,按 README §耗时表的共识分支上界;网页进度权重即此值)
|
||||
STAGES = [
|
||||
('setup', '安装依赖', 0.2), ('kin', '对拍', 0.2), ('r4', '推理延迟', 0.2), ('gate1', '一致性门禁', 0.5),
|
||||
('gen', '生成老师轨迹', 5.0), ('prep', '打包样本', 0.5), ('train', '预训练', 3.0), ('m5', '头门禁', 0.5),
|
||||
('gen', '生成老师轨迹', 5.0), ('prep', '打包样本', 2.0), ('train', '预训练', 4.5), ('dagger', 'DAgger 纠偏', 3.5), ('m5', '头门禁', 0.5),
|
||||
('apg', 'APG 微调', 4.0), ('mine', '挖错循环', 9.0), ('export', '导出', 0.1), ('bench', '部署评测', 1.0),
|
||||
('accept', '验收汇总', 0.5)]
|
||||
ORDER = [s[0] for s in STAGES]
|
||||
EST_H = {s[0]: s[2] for s in STAGES}
|
||||
# m5 不过时流程内自动训 WTA 头(训练 3 h + 门禁),超时按分支后的总量给
|
||||
TIMEOUT_H = {k: max(1.0, 3 * v) for k, v in EST_H.items()}
|
||||
TIMEOUT_H['m5'] = 3 * (EST_H['train'] + 2 * EST_H['m5'])
|
||||
TIMEOUT_H['m5'] = 3 * (EST_H['train'] + EST_H['dagger'] + 2 * EST_H['m5'])
|
||||
# 参数 → 受它影响的阶段(这些阶段即使上次完成也要重跑,其产物先挪进 ml/archive/)。只改挖错轮数不会让 5 小时的生成作废。
|
||||
_from = lambda st: ORDER[ORDER.index(st):]
|
||||
AFFECTS = {**{k: _from('gen') for k in ('N_TASKS', 'NPROC', 'CONSENSUS', 'MIN_GAP_MM')},
|
||||
# TARGET_SAMPLES 只决定"补量到多少":不作废 gen 的已有分片(重跑 gen 会跳过写完的 shard、按差额补量),
|
||||
# 但样本集会变,所以 prep 及之后作废。2026-10-18 实测:把它当"从 gen 作废"会让 5 小时的 5 万条分片白跑。
|
||||
'TARGET_SAMPLES': ['gen'] + _from('prep'),
|
||||
'CFG': ['r4'] + _from('train'), **{k: _from('mine') for k in ('ROUNDS', 'APG_STEPS', 'MINE_N')}}
|
||||
# 2026-10-19:打包参数(标签尺度、增广)变了 → prep 起作废;run_prep.sh 另按 meta.sig 只重打参数不符的 shard
|
||||
**{k: _from('prep') for k in ('ACT_SCALE', 'AUG_P', 'AUG_SIGMA', 'AUG_BACK')},
|
||||
'CFG': ['r4'] + _from('train'), **{k: _from('dagger') for k in ('DAG_ROUNDS', 'DAG_N', 'DAG_STEPS')},
|
||||
**{k: _from('mine') for k in ('ROUNDS', 'APG_STEPS', 'MINE_N')}}
|
||||
PARAM_KEYS = list(AFFECTS)
|
||||
# 各阶段产物(相对仓库根,可含通配);作废 / 从头重跑时整体移到 ml/archive/<时间>/,不删除
|
||||
PRODUCTS = {
|
||||
'r4': ['ml/runs/r4', 'ml/runs/r4.json'], 'gate1': ['ml/data/consistency.json'],
|
||||
'gen': ['ml/data/shard_*', 'ml/data/log_*', 'ml/data/stats.json'], 'prep': ['ml/data/samples', 'ml/data/r2.json'],
|
||||
'train': ['ml/runs/base', 'ml/runs/wta', 'ml/runs/small', 'ml/runs/bench_*'],
|
||||
'train': ['ml/runs/base', 'ml/runs/wta', 'ml/runs/small', 'ml/runs/bench_*', 'ml/runs/replay_teacher.json',
|
||||
'ml/runs/review_*'], # 评审快照包(_review_pack.py 从 base 打的,内含 log.jsonl 会被监控页当成一条"预训练"运行);重做 train 时一并归档
|
||||
'dagger': ['ml/runs/*_dag[0-9]*', 'ml/runs/*_dagger.ckpt', 'ml/data/dag_*', 'ml/data/samples_dag_*'],
|
||||
'm5': ['ml/runs/m5_*', 'ml/runs/selected_ckpt'], 'apg': ['ml/runs/apg', 'ml/runs/current.pt'],
|
||||
'mine': ['ml/runs/mine', 'ml/runs/apg_r*', 'ml/data/e2', 'ml/data/samples_e2'],
|
||||
'export': ['webapp/models/traj_policy.onnx', 'webapp/models/traj_policy.meta.json'],
|
||||
@@ -69,7 +76,7 @@ def load():
|
||||
try:
|
||||
with open(P, encoding='utf-8') as f: return json.load(f)
|
||||
except Exception:
|
||||
return {'started': time.time(), 'plan': None, 'stages': {}, 'current': None, 'params': {}}
|
||||
return {'started': time.time(), 'plan': None, 'stages': {}, 'current': None, 'params': {}, 'time': {}}
|
||||
|
||||
|
||||
def save(d):
|
||||
@@ -146,11 +153,21 @@ def todo(d=None):
|
||||
return [s for s in (d.get('plan') or []) if (d['stages'].get(s) or {}).get('rc') != 0]
|
||||
|
||||
|
||||
def _acc_time(d, st, now):
|
||||
"""把"这一段"的用时并进累计 time{阶段: 秒}。幂等:stage.end 已写就不再加,停止(mark_stopped)
|
||||
与流水线的 end 都会调,重复调用不会双记。"""
|
||||
s = d['stages'].get(st)
|
||||
if s and s.get('start') and s.get('end') is None:
|
||||
t = d.setdefault('time', {})
|
||||
t[st] = round(t.get(st, 0) + max(0.0, now - s['start']), 1)
|
||||
|
||||
|
||||
def mark_stopped(rc=-15):
|
||||
"""网页「停止」后调用:当前阶段记为被停止,释放锁。"""
|
||||
d = load(); st = d.get('current')
|
||||
"""网页「停止」后调用:当前阶段记为被停止、这段用时并进累计,释放锁。"""
|
||||
d = load(); st = d.get('current'); now = time.time()
|
||||
if st and st in d['stages'] and d['stages'][st].get('end') is None:
|
||||
d['stages'][st].update(end=time.time(), rc=rc)
|
||||
_acc_time(d, st, now)
|
||||
d['stages'][st].update(end=now, rc=rc)
|
||||
d['current'] = None
|
||||
save(d); lock_release()
|
||||
|
||||
@@ -182,11 +199,12 @@ def main():
|
||||
if cmd == 'reset':
|
||||
plan = rest[0].split(',') if rest and rest[0] else []
|
||||
pn = params_now(); prev = d.get('params') or {}
|
||||
changed = []
|
||||
changed = [k for k in PARAM_KEYS if prev and prev.get(k) != pn[k]]
|
||||
ff = next((r.split('=', 1)[1] for r in rest if r.startswith('--fresh-from=')), '')
|
||||
if '--fresh' in rest: bad = set(ORDER)
|
||||
else:
|
||||
changed = [k for k in PARAM_KEYS if prev and prev.get(k) != pn[k]]
|
||||
bad = {st for k in changed for st in AFFECTS[k]}
|
||||
if ff in ORDER: bad |= set(ORDER[ORDER.index(ff):])
|
||||
if changed: print('参数与上次不同:' + json.dumps({k: (prev.get(k), pn[k]) for k in changed}, ensure_ascii=False)
|
||||
+ ';受影响的阶段重跑:' + ' '.join(st for st in ORDER if st in bad), file=sys.stderr)
|
||||
keep = {k: v for k, v in d.get('stages', {}).items() if v.get('rc') == 0 and k not in bad}
|
||||
@@ -194,14 +212,21 @@ def main():
|
||||
# 例外:只改了目标样本量时,gen 要重跑补量,但已有分片必须留着(run_parallel 跳过写完的 shard、
|
||||
# 按差额补量),否则 5 小时的生成白跑。
|
||||
no_archive = {'gen'} if 'TARGET_SAMPLES' in changed else set()
|
||||
arc = archive([st for st in ORDER if st in bad and st not in no_archive]) if bad else None
|
||||
arch = [st for st in ORDER if st in bad and st not in no_archive]
|
||||
arc = archive(arch) if arch else None
|
||||
if arc: print(f'旧产物已移到 {arc}/(未删除)', file=sys.stderr)
|
||||
d = {'started': now, 'plan': plan, 'stages': keep, 'current': None, 'params': pn}
|
||||
# 累计用时跨续跑保留;只有产物被挪走的阶段(要从头重做)才清零
|
||||
tm = {k: v for k, v in (d.get('time') or {}).items() if k not in arch}
|
||||
d = {'started': now, 'plan': plan, 'stages': keep, 'current': None, 'params': pn, 'time': tm}
|
||||
done = [s for s in plan if s in keep]
|
||||
if done: print('跳过已完成的 %d 个阶段:%s' % (len(done), ' '.join(done)), file=sys.stderr)
|
||||
save(d); return
|
||||
elif cmd == 'begin':
|
||||
st = rest[0]
|
||||
prev = d['stages'].get(st) or {}
|
||||
# 老记录(本次改动之前完成的)没有累计字段:用上一次的整段补上,免得续跑时用时从 0 起
|
||||
if prev.get('start') and prev.get('end') and st not in (d.get('time') or {}):
|
||||
d.setdefault('time', {})[st] = round(prev['end'] - prev['start'], 1)
|
||||
d['stages'][st] = {'start': now, 'end': None, 'rc': None, 'info': kv(rest[1:]), 'timeout_h': TIMEOUT_H.get(st)}
|
||||
d['current'] = st
|
||||
elif cmd == 'set':
|
||||
@@ -210,6 +235,7 @@ def main():
|
||||
elif cmd == 'end':
|
||||
st, rc = rest[0], int(rest[1]) if len(rest) > 1 else 0
|
||||
s = d['stages'].setdefault(st, {'start': now, 'info': {}})
|
||||
_acc_time(d, st, now)
|
||||
s['end'], s['rc'] = now, rc
|
||||
if d.get('current') == st: d['current'] = None
|
||||
else:
|
||||
|
||||
+159
-45
@@ -1,19 +1,23 @@
|
||||
"""闭环仿真(验证 / 挖错 / 门禁共用,PyTorch 批量,无梯度)——按部署节奏运行:
|
||||
50 Hz 执行;队列 ≤ 3 步时推理一次(已提交前缀 r = 队列长度),提交接下来 3 步;
|
||||
冷启动用价值头对候选末态打分(短前瞻),之后按与上一计划的一致性选候选;
|
||||
接近目标 < 5 mm 时用局部 DLS 精修;子目标头在新目标时给出途经点(到达 < 5° 或 1.5 s 超时后切下一个)。
|
||||
冷启动用价值头对候选末态打分(短前瞻),之后按与上一条**真正提交**的计划的一致性选候选;
|
||||
接近目标 < 5 mm 时用局部 DLS 精修;子目标头在新目标时给出途经点,按「越过顶点」规则切下一个(与标签同口径,见 sub_advance)。
|
||||
safety=False 时不做任何碰撞拦截(暴露真实错误,挖错用);safety=True 时近似部署安全层(候选提交步有干涉则换下一条,全不过则停一步)。
|
||||
动作反归一化:inc = clamp(act, −1, 1) · act_scale · STEP;act_scale 来自 checkpoint(旧模型缺省 1.0,见 ml/common/ckpt.load_policy)。
|
||||
碰撞与到达只在每题自己的时限内计(短题与长题同批时不把超时后的部分算进去)。
|
||||
python -m ml.sim.closed_loop --ckpt ml/runs/base/best.pt --samples ml/data/samples --split val --n 1000 [--safety]"""
|
||||
import argparse, json, math, os
|
||||
import torch
|
||||
|
||||
from ml.common.kin_torch import Kin
|
||||
from ml.common.features import features, STEP, H, GAP_CAP, D2R, wrap180
|
||||
from ml.common.features import features, STEP, VMAX, H, GAP_CAP, D2R, wrap180
|
||||
|
||||
DT = 0.02
|
||||
|
||||
|
||||
FIRST_DEG = 5.0 # R7 首次方向判定阈值(度):小于它的抖动不算选边
|
||||
SUB_LEAD_S = 0.02 # 子目标「越过顶点」提前量(s):标签在老师 τ > τ_顶点 − 0.02 时就换到下一个顶点(build_dataset.cjs remain 过滤)
|
||||
SUB_REACH_DEG, SUB_TIMEOUT_S = 5.0, 1.5 # 旧规则(sub_rule='timeout'):到达 < 5° 或 1.5 s 超时
|
||||
|
||||
def tasks_to_tensors(tasks, device):
|
||||
q0 = torch.tensor([t['q0'] for t in tasks], dtype=torch.float32, device=device)
|
||||
@@ -28,6 +32,12 @@ def tasks_to_tensors(tasks, device):
|
||||
NOREF_SEC = 6.0 # 无老师参考题的定长时限(s)
|
||||
|
||||
|
||||
def time_limit(ex, max_sec=None):
|
||||
"""每题时限(s):有老师 = 老师时长 × 2(至少 3 s);无老师 = NOREF_SEC;max_sec 覆盖全部。"""
|
||||
if max_sec is not None: return torch.full_like(ex, max_sec)
|
||||
return torch.where(torch.isfinite(ex), torch.clamp(2 * ex, min=3.0), torch.full_like(ex, NOREF_SEC))
|
||||
|
||||
|
||||
def goal_err(kin, q, T, R):
|
||||
xf = kin.cum_xf(q)
|
||||
p = kin.fk(q, xf); Rq = kin.orient(q, xf).reshape(-1, 9)
|
||||
@@ -64,53 +74,139 @@ def dls_steps(kin, q, T, R, n=3, scale=0.3, lam=0.5):
|
||||
return torch.stack(out, 1)
|
||||
|
||||
|
||||
def jdiff(kin, a, b):
|
||||
"""a − b(度),连续轴取最短弧。"""
|
||||
d = a - b
|
||||
return torch.where(kin.cont, wrap180(d), d)
|
||||
|
||||
|
||||
def sub_advance(kin, subs, nsub, sub_prev, sub_t, qe, tnow, rule='cross'):
|
||||
"""子目标推进(每次规划前调用,每次最多推进一个)。subs [B,M,6] 待走途经点(绝对角),nsub [B] 剩余个数,
|
||||
sub_prev [B,6] 上一个已越过的点(冷启动时 = 起点),sub_t [B] 当前子目标开始时刻。返回 (subs, nsub, sub_prev, sub_t)。
|
||||
rule='cross'(默认):qe 沿拐角平分面法向越过当前顶点(提前 SUB_LEAD_S 秒)即切到下一个。
|
||||
2026-10-19(审查 review_data_actions C5 / review_model_train P0-1):标签里 sub = 老师「下一个尚未经过的顶点」,
|
||||
老师 τ 一过顶点(−0.02 s)立刻换;旧规则要等最大关节差 < 5° 或 1.5 s 超时,策略在顶点附近 5–10° 进不去时
|
||||
sub 一直指着身后的点,这种「sub 已在身后」的特征序列训练里从未出现。运行时没有老师 τ,用几何等价物:
|
||||
进入方向 a = 顶点 − 上一点、离开方向 b = 下一顶点 − 顶点(最后一个顶点取 a),各轴按 VMAX 折成秒(与 τ 同量纲),
|
||||
法向 n = â + b̂;(qe − 顶点)·n̂ ≥ −SUB_LEAD_S 判为越过。没有超时(标签也没有)。
|
||||
rule='timeout':旧部署口径,到达 < 5° 或 1.5 s 超时(诊断对照用)。
|
||||
ik-policy.js Runner(subRule)与 apg_finetune.py 用同一规则,三处同步改。"""
|
||||
has = nsub > 0
|
||||
if not bool(has.any()): return subs, nsub, sub_prev, sub_t
|
||||
cur = subs[:, 0]
|
||||
if rule == 'timeout':
|
||||
dd = jdiff(kin, cur, qe).abs().max(-1).values
|
||||
adv = has & ((dd < SUB_REACH_DEG) | (tnow - sub_t > SUB_TIMEOUT_S))
|
||||
elif rule == 'cross':
|
||||
vmax = torch.tensor(VMAX, device=qe.device, dtype=qe.dtype)
|
||||
un = lambda v: v / v.norm(dim=-1, keepdim=True).clamp(min=1e-9)
|
||||
a = jdiff(kin, cur, sub_prev) / vmax
|
||||
nxt = subs[:, 1] if subs.shape[1] > 1 else cur
|
||||
b = torch.where((nsub > 1)[:, None], jdiff(kin, nxt, cur) / vmax, a)
|
||||
n = un(un(a) + un(b))
|
||||
proj = (jdiff(kin, qe, cur) / vmax * n).sum(-1)
|
||||
adv = has & (proj >= -SUB_LEAD_S)
|
||||
else:
|
||||
raise ValueError('sub_rule: ' + str(rule))
|
||||
subs = torch.where(adv[:, None, None], torch.cat([subs[:, 1:], subs[:, -1:]], 1), subs)
|
||||
sub_prev = torch.where(adv[:, None], cur, sub_prev)
|
||||
nsub = torch.where(adv, nsub - 1, nsub); sub_t = torch.where(adv, torch.full_like(sub_t, tnow), sub_t)
|
||||
return subs, nsub, sub_prev, sub_t
|
||||
|
||||
|
||||
class TeacherOracle:
|
||||
"""老师当策略(2026-10-19,审查 review_closedloop D1 / review_data_actions §4.1):
|
||||
规划时从当前时间下标 it 取老师 50 Hz 轨迹的 16 步增量,按与 build_dataset.cjs 相同的标签口径
|
||||
act = clamp(Δ / (act_scale·STEP), −1, 1) 交给 rollout(),其余(队列、前缀、提交、qe、DLS、done、碰撞、时限)一行不改。
|
||||
traj [B,Tmax,6]:每题老师轨迹,末帧重复补齐(按 id 从 ml/data/shard_*.jsonl 取)。
|
||||
sigma > 0:每次规划给 act 加 N(0, σ²)(σ 以 STEP 为单位,各候选独立),测闭环对残差噪声的敏感度。
|
||||
跟随时第 it 步的 q 应等于 traj[:, it](DLS 接管或到达冻结之后除外)。"""
|
||||
head = 'oracle'
|
||||
|
||||
def __init__(self, traj, act_scale=1.0, sigma=0.0):
|
||||
self.traj, self.act_scale, self.sigma = traj, float(act_scale), float(sigma)
|
||||
|
||||
def plan(self, it, K, g):
|
||||
B, Tm, _ = self.traj.shape
|
||||
dev = self.traj.device
|
||||
idx = (it + torch.arange(H + 1, device=dev)).clamp(max=Tm - 1)
|
||||
seg = self.traj[:, idx]
|
||||
astep = torch.tensor(STEP, device=dev) * self.act_scale
|
||||
act = ((seg[:, 1:] - seg[:, :-1]) / astep).clamp(-1, 1)
|
||||
act = act[:, None].expand(B, K, H, 6).contiguous()
|
||||
if self.sigma > 0:
|
||||
act = act + torch.randn(B, K, H, 6, device=dev, generator=g) * (self.sigma / self.act_scale)
|
||||
return act
|
||||
|
||||
|
||||
@torch.no_grad()
|
||||
def rollout(model, kin, tasks, device, K=8, safety=False, seed=0, max_sec=None, return_traj=False):
|
||||
def rollout(model, kin, tasks, device, K=8, safety=False, seed=0, max_sec=None, return_traj=False, use_sub=True, sub_override=None,
|
||||
qp0=None, steps=1, z_scale=1.0, select='consist', oracle=None, replan='r3', sub_rule='cross'):
|
||||
"""use_sub=False:不用子目标(sf=0);sub_override=(subs [B,M,6] 绝对关节角, n [B] 个数 或 need [B] bool):冷启动时用给定途经点代替子目标头;
|
||||
qp0 [B,6]:起点的上一帧(默认 = q0,即静止起步)。steps:MeanFlow 采样段数;z_scale:噪声缩放;
|
||||
select:consist(部署口径:冷启动价值前瞻、之后与上一条已提交计划的一致性)/ value(每次都价值前瞻)/ first(不选,取第 0 条)。
|
||||
oracle:TeacherOracle,代替 model 出动作(model 可为 None,须配 select='first'、use_sub=False)。
|
||||
replan:'r3'(部署:队列 ≤ 3 步就规划,稳态 r=3)/ 'empty'(队列空了才规划,每次 r=0,前缀消融用)。
|
||||
sub_rule:子目标推进规则,见 sub_advance。除默认值外均为诊断用。"""
|
||||
g = torch.Generator(device=device); g.manual_seed(seed)
|
||||
q0, T, R, ex, tr = tasks_to_tensors(tasks, device)
|
||||
B = q0.shape[0]
|
||||
step = torch.tensor(STEP, device=device)
|
||||
act_scale = oracle.act_scale if oracle is not None else float(getattr(model, 'act_scale', 1.0))
|
||||
astep = step * act_scale # 动作反归一化:每单位 act 对应的度数
|
||||
has_ref = torch.isfinite(ex)
|
||||
limit = torch.where(has_ref, torch.clamp(2 * ex, min=3.0), torch.full_like(ex, NOREF_SEC)) if max_sec is None else torch.full_like(ex, max_sec)
|
||||
limit = time_limit(ex, max_sec)
|
||||
nmax = int(math.ceil(limit.max().item() / DT))
|
||||
q, qp = q0.clone(), q0.clone()
|
||||
q, qp = q0.clone(), (qp0.to(device) if qp0 is not None else q0.clone())
|
||||
queue = torch.zeros(B, 0, 6, device=device)
|
||||
last = None
|
||||
subs = torch.zeros(B, 2, 6, device=device); nsub = torch.zeros(B, dtype=torch.long, device=device); sub_t = torch.zeros(B, device=device)
|
||||
last = None; last_ok = torch.zeros(B, dtype=torch.bool, device=device)
|
||||
subs = torch.zeros(B, 2, 6, device=device); nsub = torch.zeros(B, dtype=torch.long, device=device)
|
||||
sub_t = torch.zeros(B, device=device); sub_prev = q0.clone()
|
||||
done = torch.zeros(B, dtype=torch.bool, device=device); t_done = torch.full((B,), float('nan'), device=device)
|
||||
col = torch.zeros(B, device=device); travel = torch.zeros(B, device=device); brakes = torch.zeros(B, device=device)
|
||||
first_hit = torch.full((B,), float('nan'), device=device); dls_it = torch.full((B,), -1, dtype=torch.long, device=device)
|
||||
traj = [q.clone()] if return_traj else None
|
||||
n_start = int(round(0.5 / DT)); moved_start = None # R2 起步失败:0.5 s 内关节总行程 < 1°
|
||||
k2s = int(round(2.0 / DT)); e2 = None
|
||||
e0, _ = goal_err(kin, q0, T, R)
|
||||
j1_acc = torch.zeros(B, device=device); j1_max = torch.zeros(B, device=device); j1_min = torch.zeros(B, device=device)
|
||||
j1_first = torch.zeros(B, device=device) # 2026-10-18 第五轮(审查 N-3):J1 累计偏离首次超过 FIRST_DEG 时的方向(真正的"首次方向")
|
||||
cold = True
|
||||
ar = torch.arange(B, device=device)
|
||||
for it in range(nmax):
|
||||
tnow = it * DT
|
||||
if queue.shape[1] <= 3:
|
||||
if (queue.shape[1] <= 3) if replan == 'r3' else (queue.shape[1] == 0):
|
||||
r = queue.shape[1]
|
||||
pre = torch.zeros(B, 3, 6, device=device); pre[:, :r] = queue[:, :r]
|
||||
qe = q + queue.sum(1)
|
||||
e, oe = goal_err(kin, qe, T, R)
|
||||
near = (e < 5) & ((R.abs().sum(-1) == 0) | (oe < 5))
|
||||
# 子目标推进
|
||||
if (nsub > 0).any():
|
||||
cur = subs[:, 0]
|
||||
dd = torch.where(kin.cont, wrap180(cur - qe), cur - qe).abs().max(-1).values
|
||||
adv = (nsub > 0) & ((dd < 5) | (tnow - sub_t > 1.5))
|
||||
subs = torch.where(adv[:, None, None], torch.cat([subs[:, 1:], subs[:, 1:]], 1), subs)
|
||||
nsub = torch.where(adv, nsub - 1, nsub); sub_t = torch.where(adv, torch.full_like(sub_t, tnow), sub_t)
|
||||
subs, nsub, sub_prev, sub_t = sub_advance(kin, subs, nsub, sub_prev, sub_t, qe, tnow, sub_rule)
|
||||
sf = (nsub > 0).float()
|
||||
feat = features(kin, q, qp, T, R, pre, torch.full((B,), r, device=device), subs[:, 0], sf)
|
||||
z = torch.randn(B * K, H, 6, device=device, generator=g)
|
||||
act, val, sub = model.sample_k(feat, K, z)
|
||||
inc = act.clamp(-1, 1) * step
|
||||
if oracle is not None:
|
||||
act = oracle.plan(it, K, g); sub = torch.zeros(B, 13, device=device)
|
||||
else:
|
||||
z = torch.randn(B * K, H, 6, device=device, generator=g) * z_scale
|
||||
act, val, sub = model.sample_k(feat, K, z, steps=steps)
|
||||
inc = act.float().clamp(-1, 1) * astep
|
||||
if cold:
|
||||
sub0 = sub
|
||||
need = sub0[:, 12] > 0.5
|
||||
subs = q[:, None] + sub0[:, :12].view(B, 2, 6) * 180
|
||||
nsub = torch.where(need, torch.full_like(nsub, 2), torch.zeros_like(nsub)); sub_t.fill_(tnow)
|
||||
need = sub[:, 12] > 0.5
|
||||
subs = q[:, None] + sub[:, :12].float().view(B, 2, 6) * 180
|
||||
if sub_override is not None:
|
||||
subs = sub_override[0].to(device).float()
|
||||
n_ov = sub_override[1].to(device)
|
||||
nsub_ov = n_ov.long() if n_ov.dtype != torch.bool else n_ov.long() * subs.shape[1]
|
||||
need = nsub_ov > 0
|
||||
nsub = nsub_ov
|
||||
else:
|
||||
nsub = torch.where(need, torch.full_like(nsub, 2), torch.zeros_like(nsub))
|
||||
if not use_sub: nsub = torch.zeros_like(nsub)
|
||||
sub_t.fill_(tnow); sub_prev = q.clone()
|
||||
fut = qe[:, None, None] + torch.cumsum(inc[:, :, r:], 2) # [B,K,H-r,6]
|
||||
if last is None:
|
||||
if select == 'first':
|
||||
score = torch.zeros(B, K, device=device); score[:, 0] = 1
|
||||
elif cold or select == 'value':
|
||||
# 短前瞻:候选末态的价值
|
||||
qa = fut[:, :, -1].reshape(B * K, 6); qb = fut[:, :, -2].reshape(B * K, 6)
|
||||
f2 = features(kin, qa, qb, T.repeat_interleave(K, 0), R.repeat_interleave(K, 0), torch.zeros(B * K, 3, 6, device=device),
|
||||
@@ -118,30 +214,39 @@ def rollout(model, kin, tasks, device, K=8, safety=False, seed=0, max_sec=None,
|
||||
v2, _ = model.heads(model.encode(f2))
|
||||
score = -v2.view(B, K)
|
||||
else:
|
||||
n = min(fut.shape[2], last.shape[1] - 3)
|
||||
# 2026-10-19 修错位:last[i] = 上一计划从 qe_old 走 i 步后的位置;提交 3 步后 qe_new = last[3],
|
||||
# 新候选 fut[0] 是 qe_new 再走 1 步 = 对应 last[4]。JS 部署(ik-policy.js)同一处同样修正。
|
||||
n = min(fut.shape[2], last.shape[1] - 4)
|
||||
w = 0.85 ** torch.arange(n, device=device, dtype=torch.float32)
|
||||
d2 = (((fut[:, :, :n] - last[:, None, 3:3 + n]) / step) ** 2).sum(-1)
|
||||
score = -(d2 * w).sum(-1)
|
||||
order = score.argsort(-1, descending=True)
|
||||
d2 = (((fut[:, :, :n] - last[:, None, 4:4 + n]) / step) ** 2).sum(-1)
|
||||
# 2026-10-19(审查 review_closedloop F5):上一次是刹停或 DLS 接管的行没有「已提交计划」,
|
||||
# 与 JS(lastPlan = null → 一致性项缺席、各候选同分 → 取第 0 条)一致:同分,稳定排序取第 0 条
|
||||
score = torch.where(last_ok[:, None], -(d2 * w).sum(-1), torch.zeros_like(d2[..., 0]))
|
||||
order = torch.sort(score, dim=-1, descending=True, stable=True).indices
|
||||
pick = order[:, 0]
|
||||
commit = inc[torch.arange(B), pick, r:r + 3]
|
||||
commit = inc[ar, pick, r:r + 3]
|
||||
committed = torch.ones(B, dtype=torch.bool, device=device)
|
||||
if safety:
|
||||
ok_any = torch.zeros(B, dtype=torch.bool, device=device)
|
||||
chosen = torch.zeros(B, 3, 6, device=device)
|
||||
chosen = torch.zeros(B, 3, 6, device=device); chosen_k = pick.clone()
|
||||
for kk in range(K):
|
||||
ck = inc[torch.arange(B), order[:, kk], r:r + 3]
|
||||
ck = inc[ar, order[:, kk], r:r + 3]
|
||||
pts = qe[:, None] + torch.cumsum(ck, 1)
|
||||
mids = torch.cat([qe[:, None], pts[:, :-1]], 1) + 0.5 * ck
|
||||
allq = torch.cat([pts, mids], 1).reshape(-1, 6)
|
||||
okk = (kin.gaps(allq, 0.5)['gm'].min(-1).values > 0).view(B, 6).all(-1)
|
||||
take = okk & ~ok_any
|
||||
chosen = torch.where(take[:, None, None], ck, chosen); ok_any |= okk
|
||||
chosen = torch.where(take[:, None, None], ck, chosen); chosen_k = torch.where(take, order[:, kk], chosen_k); ok_any |= okk
|
||||
commit = torch.where(ok_any[:, None, None], chosen, torch.zeros_like(chosen))
|
||||
brakes += (~ok_any).float()
|
||||
pick = chosen_k; committed = ok_any
|
||||
if near.any():
|
||||
ref = dls_steps(kin, qe, T, R)
|
||||
commit = torch.where(near[:, None, None], ref, commit)
|
||||
last = torch.cat([qe[:, None], fut[torch.arange(B), pick]], 1)
|
||||
dls_it = torch.where(near & (dls_it < 0), torch.full_like(dls_it, it), dls_it)
|
||||
committed = committed & ~near
|
||||
# 2026-10-19(F5):last 记真正提交的那条候选(安全层换了候选时是换后的那条);刹停 / DLS 接管的行无效
|
||||
last = torch.cat([qe[:, None], fut[ar, pick]], 1); last_ok = committed
|
||||
queue = torch.cat([queue, commit], 1)
|
||||
cold = False
|
||||
d = queue[:, 0]; queue = queue[:, 1:]
|
||||
@@ -154,15 +259,21 @@ def rollout(model, kin, tasks, device, K=8, safety=False, seed=0, max_sec=None,
|
||||
if return_traj: traj.append(q.clone())
|
||||
mid = qp + 0.5 * (q - qp)
|
||||
gm = kin.gaps(torch.cat([q, mid], 0), 0.5)['gm'].min(-1).values
|
||||
hit = ((gm[:B] <= 0) | (gm[B:] <= 0)) & ~done
|
||||
tn = (it + 1) * DT
|
||||
# 2026-10-19(审查 review_closedloop F4):只计每题自己时限内的碰撞(同批长题会把短题多仿真几秒)
|
||||
hit = ((gm[:B] <= 0) | (gm[B:] <= 0)) & ~done & (tn <= limit + 1e-6)
|
||||
col += hit.float()
|
||||
first_hit = torch.where(hit & torch.isnan(first_hit), torch.full_like(first_hit, tn), first_hit)
|
||||
e, oe = goal_err(kin, q, T, R)
|
||||
if it + 1 == k2s: e2 = e.clone()
|
||||
newly = (~done) & (e < 1) & (oe < 0.5)
|
||||
t_done = torch.where(newly, torch.full_like(t_done, (it + 1) * DT), t_done); done |= newly
|
||||
t_done = torch.where(newly, torch.full_like(t_done, tn), t_done); done |= newly
|
||||
if done.all(): break
|
||||
ok = done & (t_done <= limit)
|
||||
ok = done & (t_done <= limit + 1e-6)
|
||||
if moved_start is None: moved_start = travel.clone()
|
||||
e_end, _ = goal_err(kin, q, T, R)
|
||||
if e2 is None: e2 = e_end
|
||||
ratio2 = e2 / e0.clamp(min=1e-3)
|
||||
start_fail = (moved_start < 1.0) & ~(done & (t_done <= 0.5))
|
||||
okr = ok & has_ref & torch.isfinite(tr)
|
||||
res = {'n': B, 'n_ref': int(has_ref.sum()), 'success': ok.float().mean().item(),
|
||||
@@ -170,20 +281,22 @@ def rollout(model, kin, tasks, device, K=8, safety=False, seed=0, max_sec=None,
|
||||
'collision_rate': (col > 0).float().mean().item(), 'collision_steps_mean': col.mean().item(),
|
||||
'travel_ratio_median': (travel[okr] / tr[okr].clamp(min=1)).median().item() if okr.any() else None,
|
||||
'time_mean_s': t_done[ok].mean().item() if ok.any() else None, 'brakes_mean': brakes.mean().item(),
|
||||
'start_fail': start_fail.float().mean().item(),
|
||||
'start_fail': start_fail.float().mean().item(), 'goal_ratio_2s': ratio2.median().item(),
|
||||
'time_budget': 'teacher×2(≥3 s)' if bool(has_ref.all()) else ('fixed %.0f s' % (max_sec or NOREF_SEC) if not has_ref.any() else 'mixed')}
|
||||
# 绕行侧:J1 累计位移的正向/负向极值哪个更大(+1 / −1),供 R7 左右比例门禁
|
||||
# 首次方向;全程没偏离到 FIRST_DEG 的(基本不绕 J1)退回净方向
|
||||
side = torch.where(j1_first != 0, j1_first, torch.where(j1_max >= -j1_min, torch.ones_like(j1_max), -torch.ones_like(j1_max)))
|
||||
per = {'done': done.cpu(), 't_done': t_done.cpu(), 'col': col.cpu(), 'travel': travel.cpu(), 'err': e_end.cpu(),
|
||||
'side': side.cpu(), 'start_fail': start_fail.cpu(), 'brakes': brakes.cpu()}
|
||||
per = {'done': done.cpu(), 'ok': ok.cpu(), 't_done': t_done.cpu(), 'limit': limit.cpu(), 'col': col.cpu(), 'travel': travel.cpu(), 'err': e_end.cpu(),
|
||||
'side': side.cpu(), 'start_fail': start_fail.cpu(), 'brakes': brakes.cpu(), 'first_hit': first_hit.cpu(), 'dls_it': dls_it.cpu(),
|
||||
'goal_ratio_2s': ratio2.cpu()}
|
||||
if return_traj: per['traj'] = torch.stack(traj, 1).cpu()
|
||||
return res, per
|
||||
|
||||
|
||||
def evaluate(model, kin, tasks, device, K=8, safety=False, bs=256, seed=0):
|
||||
if not tasks: # 小样本摸底时 val 可能为空
|
||||
return {'n': 0, 'success': float('nan'), 'collision_rate': float('nan'), 'start_fail': float('nan'), 'brakes_mean': float('nan')}, []
|
||||
return {'n': 0, 'success': float('nan'), 'collision_rate': float('nan'), 'start_fail': float('nan'), 'brakes_mean': float('nan'),
|
||||
'goal_ratio_2s': float('nan')}, []
|
||||
agg, pers = {}, []
|
||||
for i in range(0, len(tasks), bs):
|
||||
r, p = rollout(model, kin, tasks[i:i + bs], device, K=K, safety=safety, seed=seed + i)
|
||||
@@ -194,12 +307,14 @@ def evaluate(model, kin, tasks, device, K=8, safety=False, bs=256, seed=0):
|
||||
n = sum(r['n'] for r, _ in pers)
|
||||
out = {k: v / n for k, v in agg.items()}; out['n'] = n
|
||||
out['n_ref'] = sum(r['n_ref'] for r, _ in pers)
|
||||
# 中位数不能按批加权平均:全体任务重算(2026-10-19,选模分数用它)
|
||||
out['goal_ratio_2s'] = torch.cat([p['goal_ratio_2s'] for _, p in pers]).median().item()
|
||||
out['time_budget'] = pers[0][0]['time_budget'] if len({r['time_budget'] for r, _ in pers}) == 1 else 'mixed'
|
||||
return out, pers
|
||||
|
||||
|
||||
def main():
|
||||
from ml.model.policy import build
|
||||
from ml.common.ckpt import load_policy
|
||||
from ml.data import load_tasks
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument('--ckpt', required=True); ap.add_argument('--samples', default='ml/data/samples')
|
||||
@@ -208,8 +323,7 @@ def main():
|
||||
ap.add_argument('--safety', action='store_true'); ap.add_argument('--K', type=int, default=8)
|
||||
ap.add_argument('--device', default='cuda' if torch.cuda.is_available() else 'cpu'); ap.add_argument('--out', default=None)
|
||||
a = ap.parse_args()
|
||||
ck = torch.load(a.ckpt, map_location=a.device)
|
||||
model = build(ck['cfg']).to(a.device); model.load_state_dict(ck['model']); model.eval()
|
||||
model, ck = load_policy(a.ckpt, a.device)
|
||||
kin = Kin(device=a.device)
|
||||
assert ck.get('fp') in (None, kin.fp), '碰撞模型指纹与 checkpoint 不符'
|
||||
if a.tasks: tasks = [json.loads(l) for l in open(a.tasks, encoding='utf-8') if l.strip()]
|
||||
@@ -217,8 +331,8 @@ def main():
|
||||
tasks = tasks[:a.n]
|
||||
res, pers = evaluate(model, kin, tasks, a.device, K=a.K, safety=a.safety)
|
||||
by = {}
|
||||
flat_done = torch.cat([p['done'] for _, p in pers]); flat_col = torch.cat([p['col'] for _, p in pers])
|
||||
for t, d, c in zip(tasks, flat_done.tolist(), flat_col.tolist()):
|
||||
flat_ok = torch.cat([p['ok'] for _, p in pers]); flat_col = torch.cat([p['col'] for _, p in pers])
|
||||
for t, d, c in zip(tasks, flat_ok.tolist(), flat_col.tolist()):
|
||||
k = t.get('cat', '?') + '/' + t.get('mode', '?'); b = by.setdefault(k, [0, 0, 0]); b[0] += 1; b[1] += int(d); b[2] += int(c > 0)
|
||||
res['by_cat_mode'] = {k: {'n': v[0], 'success': v[1] / v[0], 'collision': v[2] / v[0]} for k, v in by.items()}
|
||||
s = json.dumps(res, indent=1); print(s)
|
||||
|
||||
+123
-33
@@ -1,9 +1,27 @@
|
||||
"""预训练(方案 §6):MeanFlow 一步头 + FK 损失 + 平滑 + 价值头 + 子目标头。
|
||||
python -m ml.train --samples ml/data/samples --out ml/runs/base [--config ml/configs/base.json]
|
||||
吞吐摸底:python -m ml.train --samples ml/data/samples --out ml/runs/bench_base --steps 2000 --no-eval
|
||||
通过线(方案 R10):全程 60000 步 ≤ 3.5 h,即 samples_per_s ≥ 5000(3090 实测 6.2 k/s ≈ 2.8 h);日志 eta_h 为剩余小时数。
|
||||
每 eval_every 步做一次闭环验证(ml/sim/closed_loop.py,部署节奏),按 success 存 best.pt;连续 patience_evals 次不升即停。"""
|
||||
import argparse, json, math, os, sys, time
|
||||
通过线(方案 R10):samples_per_s ≥ 5000;日志 eta_h 为剩余小时数。
|
||||
每 eval_every 步用 EMA 权重做一次闭环验证(ml/sim/closed_loop.py,部署节奏),按选模分数存 best.pt;min_step 之后连续 patience_evals 次不升即停。
|
||||
2026-10-18 首次正式预训练(21850 步,闭环到达 0–0.2%、穿柱 77–97%)后改:
|
||||
- FK / 平滑损失不再加在 one_step(随机噪声) 上(那样等于对任意噪声回归同一个 Y → 条件均值,多峰头塌成单峰),
|
||||
改加在 MF 分支里 r==t 且 t ≤ fk_tmax 的样本的配对去噪估计 x̂ = z − t·u 上(见 ml/model/meanflow.py);
|
||||
- p_neq 前 p_neq_warm 步为 0(纯 flow matching 先把瞬时速度场学稳,再开 r≠t 的自举目标);
|
||||
- 每次评估加开环指标(val 样本上一步采样的 16 步末端 FK 误差 mm、K 个候选的多样性),
|
||||
塌缩或学偏当场可见。
|
||||
2026-10-19(审查 review_model_train P0-2/P0-3、review_data_actions C3):
|
||||
- EMA(train.ema,默认 0.999,前期按 (1+s)/(10+s) 放宽):评估、best/last 里的 'ema' 都是 EMA 权重;下游一律经 ckpt.load_policy 取 ck.get('ema', ck['model']);
|
||||
- 选模:p_neq 预热期内及预热后第一次评估之前不存 best.pt(那时一步映射 u(z,0,1) 还没训过);
|
||||
分数 = success + 0.05·(1 − collision_rate) + 0.05·(1 − min(1, goal_ratio_2s)) − 1e-4·min(ol_best_mm, 500),success 全 0 时闭环指标仍能排序;
|
||||
早停只在 min_step = max(预热 + 2·eval_every, steps/2) 之后计数;
|
||||
- 余弦学习率按真实预算走完(configs 的 steps 改为 24000,原 60000 从来跑不到,14k 步早停时学习率还有 0.87×);
|
||||
- MeanFlow 采样:p_neq 0.5、p_edge 0.1(固定 (r=0,t=1) 的部署点,见 meanflow.sample_rt);
|
||||
- 子目标头只看 sub 特征(68:75)置零的输入:sub 存在时 SG[0:6] 与输入 sd 是同一个量,原来 80% 的样本可以直接抄(标签泄露);
|
||||
运行时子目标头只在冷启动 sf=0 时用,这样训练与使用同口径。价值头仍用完整输入(稳态推理时带 sub);
|
||||
- 条件输入增广 aug_pre_drop / aug_pre_sigma / aug_dq_sigma(见 ml/data.Samples.augment);
|
||||
- 动作尺度 act_scale 取自数据(ml/data.read_meta),存进 checkpoint;续训 / 微调起点与数据不一致直接报错。
|
||||
- DAgger 微调(--init):只有起点 checkpoint 已走过自己的 p_neq 预热时才把预热置 0;起点优先取 EMA 权重。"""
|
||||
import argparse, copy, json, math, os, sys, time
|
||||
import torch
|
||||
|
||||
from ml.common.kin_torch import Kin
|
||||
@@ -12,7 +30,7 @@ from ml.common.ckpt import Saver, atomic_save, rng_state, set_rng_state
|
||||
from ml.common import vram
|
||||
from ml.data import Samples, load_tasks
|
||||
from ml.model.policy import build, wta_loss
|
||||
from ml.model.meanflow import meanflow_loss, one_step
|
||||
from ml.model.meanflow import meanflow_loss
|
||||
from ml.sim.closed_loop import evaluate
|
||||
|
||||
|
||||
@@ -28,46 +46,105 @@ def q_from_feat(X):
|
||||
return torch.atan2(s, c) / D2R
|
||||
|
||||
|
||||
@torch.no_grad()
|
||||
def open_loop(model, kin, vdata, stepv, fk_idx, n=1024, K=8):
|
||||
"""stepv:每单位动作对应的度数(= act_scale·STEP)。"""
|
||||
"""开环指标(val 样本,老师状态上):
|
||||
ol_fk_mm 候选 0 的 16 步末端与老师 Y 末端的胶囊点平均距离(mm);
|
||||
ol_best_mm K 个候选里最接近老师的那条的同一误差(多峰时它应明显小于 ol_fk_mm);
|
||||
ol_div K 个候选间标准差 / 动作平均幅度(塌成单峰时接近 0;首次正式预训练 last.pt 实测 0.20)。"""
|
||||
g = torch.Generator(device=vdata.X.device); g.manual_seed(123)
|
||||
X, Y, _, _ = vdata.batch(min(n, vdata.n), gen=g)
|
||||
act, _, _ = model.sample_k(X, K)
|
||||
act = act.float().clamp(-1, 1)
|
||||
q = q_from_feat(X)
|
||||
qt = q + (Y * stepv).sum(1)
|
||||
qh = q[:, None] + (act * stepv).sum(2) # [B,K,6]
|
||||
pt = fk_points(kin, qt, fk_idx)[:, None]
|
||||
ph = fk_points(kin, qh.reshape(-1, 6), fk_idx).view(X.shape[0], K, -1, 3)
|
||||
err = (ph - pt).norm(dim=-1).mean(-1) # [B,K] mm
|
||||
div = (act.std(1).mean() / act.abs().mean().clamp(min=1e-6)).item()
|
||||
return {'ol_fk_mm': err[:, 0].mean().item(), 'ol_best_mm': err.min(1).values.mean().item(), 'ol_div': div}
|
||||
|
||||
|
||||
def main():
|
||||
ap = argparse.ArgumentParser()
|
||||
ap.add_argument('--samples', nargs='+', default=['ml/data/samples']); ap.add_argument('--out', default='ml/runs/base')
|
||||
ap.add_argument('--config', default='ml/configs/base.json'); ap.add_argument('--steps', type=int, default=None)
|
||||
ap.add_argument('--resume', default=None); ap.add_argument('--no-eval', action='store_true')
|
||||
ap.add_argument('--init', default=None, help='只载入权重(优先 EMA)、优化器与步数从 0 开始(DAgger 微调用)')
|
||||
ap.add_argument('--lr', type=float, default=None); ap.add_argument('--warmup', type=int, default=None)
|
||||
ap.add_argument('--stop-at', type=int, default=0, help='自检用:跑到第 N 步按停止请求处理(存 last.pt、rc 143)')
|
||||
ap.add_argument('--device', default='cuda' if torch.cuda.is_available() else 'cpu'); ap.add_argument('--seed', type=int, default=0)
|
||||
a = ap.parse_args()
|
||||
cfg = json.load(open(a.config)); tc = cfg['train']
|
||||
if a.steps: tc['steps'] = a.steps
|
||||
if a.lr: tc['lr'] = a.lr
|
||||
if a.warmup is not None: tc['warmup'] = a.warmup
|
||||
init_ck = None
|
||||
if a.init:
|
||||
init_ck = torch.load(a.init, map_location='cpu')
|
||||
# 2026-10-19(审查 P0-2):原来 --init 一律置 p_neq_warm=0,前提「起点已训过 r≠t」;best.pt 落在预热期(第 4000 步)时前提不成立,
|
||||
# DAgger 一上来就用没训过的场做 JVP 自举。现在只有起点已走过它自己的预热才置 0。
|
||||
init_warm = (init_ck.get('cfg') or {}).get('train', {}).get('p_neq_warm', 5000)
|
||||
if init_ck.get('step', 0) >= init_warm: tc['p_neq_warm'] = 0
|
||||
print(f'微调起点第 {init_ck.get("step")} 步(其预热 {init_warm} 步)→ 本次 p_neq_warm = {tc.get("p_neq_warm", 5000)}', flush=True)
|
||||
os.makedirs(a.out, exist_ok=True)
|
||||
torch.manual_seed(a.seed)
|
||||
dev = a.device
|
||||
vram.limit()
|
||||
if dev.startswith('cuda'):
|
||||
torch.backends.cuda.matmul.allow_tf32 = True; torch.backends.cudnn.allow_tf32 = True # MF 分支改 fp32 后用 TF32 保吞吐
|
||||
kin = Kin(device=dev)
|
||||
data = Samples(a.samples, device=dev, split='train')
|
||||
val_tasks = [t for t in load_tasks(a.samples) if t['split'] == 'val'][:tc['eval_tasks']]
|
||||
print(f'train samples {data.n}, val tasks {len(val_tasks)}, device {dev}', flush=True)
|
||||
model = build(cfg).to(dev)
|
||||
# 2026-10-19(审查 F11):val 只取主数据目录(第一个 --samples);DAgger 目录的样本全部当 train
|
||||
data = Samples(a.samples, device=dev, split='train', val_dirs=a.samples[:1])
|
||||
act_scale = data.act_scale
|
||||
val_tasks = [t for t in load_tasks(a.samples[:1]) if t['split'] == 'val'][:tc['eval_tasks']]
|
||||
try: vdata = Samples(a.samples[:1], device=dev, split='val')
|
||||
except SystemExit: vdata = None
|
||||
if vdata is not None and vdata.n == 0: vdata = None
|
||||
print(f'train samples {data.n}, val tasks {len(val_tasks)}, act_scale {act_scale}, data_ver {data.data_ver}, device {dev}', flush=True)
|
||||
model = build(cfg).to(dev); model.act_scale = act_scale
|
||||
print(f'params {sum(p.numel() for p in model.parameters()) / 1e6:.2f} M', flush=True)
|
||||
opt = torch.optim.AdamW(model.parameters(), lr=tc['lr'], betas=tuple(tc['betas']), weight_decay=tc['wd'])
|
||||
step0, best, bad_evals = 0, -1.0, 0
|
||||
def check_scale(ck, src):
|
||||
s0 = float(ck.get('act_scale', 1.0))
|
||||
if abs(s0 - act_scale) > 1e-9:
|
||||
raise SystemExit(f'{src} 的 act_scale={s0} 与样本 act_scale={act_scale} 不一致:标签尺度变了,须从头训练(FRESH_FROM=prep)')
|
||||
if init_ck is not None and not a.resume:
|
||||
check_scale(init_ck, a.init)
|
||||
model.load_state_dict(init_ck.get('ema') if init_ck.get('ema') is not None else init_ck['model'])
|
||||
print(f'微调:权重取自 {a.init}({"EMA" if init_ck.get("ema") is not None else "原始"}权重),优化器重置', flush=True)
|
||||
ema = copy.deepcopy(model).eval(); ema.act_scale = act_scale
|
||||
for p_ in ema.parameters(): p_.requires_grad_(False)
|
||||
if a.resume:
|
||||
ck = torch.load(a.resume, map_location=dev); model.load_state_dict(ck['model'])
|
||||
ck = torch.load(a.resume, map_location=dev); check_scale(ck, a.resume)
|
||||
model.load_state_dict(ck['model'])
|
||||
ema.load_state_dict(ck['ema'] if ck.get('ema') is not None else ck['model'])
|
||||
if 'opt' in ck: opt.load_state_dict(ck['opt'])
|
||||
step0, best, bad_evals = ck.get('step', 0), ck.get('best', -1.0), ck.get('bad_evals', 0)
|
||||
if set_rng_state(ck.get('rng')): print('续训:随机数状态已恢复(batch 顺序与不断点一致)', flush=True)
|
||||
print(f'续训:从第 {step0} 步继续(共 {tc["steps"]} 步),当前最好 success {best:.4f}', flush=True)
|
||||
print(f'续训:从第 {step0} 步继续(共 {tc["steps"]} 步),当前最好分数 {best:.4f}', flush=True)
|
||||
del init_ck
|
||||
total = tc['steps']
|
||||
lr_at = lambda s: tc['lr'] * (s / tc['warmup'] if s < tc['warmup'] else 0.5 * (1 + math.cos(math.pi * (s - tc['warmup']) / max(1, total - tc['warmup']))))
|
||||
amp = dev.startswith('cuda') and tc.get('amp') == 'bf16'
|
||||
stepv = torch.tensor(STEP, device=dev)
|
||||
astep = stepv * act_scale # 动作 → 度
|
||||
fk_idx = torch.linspace(0, kin.cap_a.shape[0] - 1, tc['fk_caps'], device=dev).long()
|
||||
ema_d = float(tc.get('ema', 0.999))
|
||||
warm = 0 if model.head == 'wta' else int(tc.get('p_neq_warm', 5000))
|
||||
sel_from = warm + tc['eval_every'] # 第一次可以存 best.pt 的步数:预热结束后的第一次评估
|
||||
min_step = max(warm + 2 * tc['eval_every'], total // 2) # 早停计数起点
|
||||
log = open(os.path.join(a.out, 'log.jsonl'), 'a' if a.resume else 'w') # 新训练覆盖旧日志,续训才追加
|
||||
if not val_tasks and not a.no_eval:
|
||||
print('没有 val 任务(样本太少),本次跳过闭环验证', flush=True); a.no_eval = True
|
||||
t0, seen, diverged = time.time(), 0, False
|
||||
saver = Saver(tc.get('ckpt_min', 10)) # 2026-10-18:每 10 分钟 + 收到停止请求时存 last.pt(原子写)
|
||||
def save(name, step, extra=None):
|
||||
d = {'model': model.state_dict(), 'opt': opt.state_dict(), 'cfg': cfg, 'step': step, 'best': best, 'bad_evals': bad_evals, 'fp': kin.fp}
|
||||
d = {'model': model.state_dict(), 'ema': ema.state_dict(), 'opt': opt.state_dict(), 'cfg': cfg, 'step': step, 'best': best,
|
||||
'bad_evals': bad_evals, 'fp': kin.fp, 'act_scale': act_scale, 'data_ver': data.data_ver}
|
||||
if name == 'last.pt': d['rng'] = rng_state()
|
||||
if extra: d.update(extra)
|
||||
atomic_save(d, os.path.join(a.out, name))
|
||||
@@ -75,26 +152,32 @@ def main():
|
||||
step = step0 - 1 # 续训时已训满:循环一次不跑,下面照常收尾
|
||||
for step in range(step0, total):
|
||||
for gpar in opt.param_groups: gpar['lr'] = lr_at(step)
|
||||
X, Y, V, SG = data.batch(tc['batch'])
|
||||
X, Y, V, SG = data.batch(tc['batch'], aug=tc)
|
||||
model.train()
|
||||
with torch.autocast('cuda', dtype=torch.bfloat16, enabled=amp):
|
||||
mem = model.encode(X)
|
||||
if model.head == 'wta':
|
||||
l_mf, xh = wta_loss(model, mem, Y); mf_raw = l_mf.detach(); xh = xh.float()
|
||||
else:
|
||||
l_mf, mf_raw = meanflow_loss(model, mem, Y, p_neq=tc['p_neq'])
|
||||
e = torch.randn_like(Y)
|
||||
xh = one_step(model, mem, e).float()
|
||||
v, s = model.heads(mem)
|
||||
p_neq = tc['p_neq'] if step >= warm else 0.0
|
||||
l_mf, mf_raw, aux = meanflow_loss(model, mem, Y, p_neq=p_neq, return_aux=True, p_edge=tc.get('p_edge', 0.0))
|
||||
sel = ((aux['r'] == aux['t']) & (aux['t'] <= tc.get('fk_tmax', 0.3))).nonzero(as_tuple=True)[0]
|
||||
tt = aux['t'][sel][:, None, None]
|
||||
xh = aux['z'][sel] - tt * aux['u'][sel] # 配对去噪估计(fp32)
|
||||
v, _ = model.heads(mem)
|
||||
Xs = X.clone(); Xs[:, 68:75] = 0 # 子目标头:sub 特征置零(切断抄输入的捷径,见文件头)
|
||||
_, s = model.heads(model.encode(Xs))
|
||||
l_val = (v.float() - V).abs().mean(); l_sub = (s.float() - SG).abs().mean()
|
||||
d2 = xh[:, 2:] - 2 * xh[:, 1:-1] + xh[:, :-2]
|
||||
l_sm = d2.pow(2).mean()
|
||||
l_fk = torch.zeros((), device=dev)
|
||||
if tc['w_fk'] > 0:
|
||||
q = q_from_feat(X)
|
||||
Xf, Yf = (X, Y) if model.head == 'wta' else (X[sel], Y[sel])
|
||||
l_sm = torch.zeros((), device=dev); l_fk = torch.zeros((), device=dev)
|
||||
if xh.shape[0] > 0:
|
||||
d2 = xh[:, 2:] - 2 * xh[:, 1:-1] + xh[:, :-2]
|
||||
l_sm = d2.pow(2).mean()
|
||||
if tc['w_fk'] > 0 and xh.shape[0] > 0:
|
||||
q = q_from_feat(Xf)
|
||||
ks = torch.tensor([3, 7, 11, 15], device=dev)
|
||||
qh = q[:, None] + torch.cumsum(xh.clamp(-1, 1) * stepv, 1)[:, ks]
|
||||
qt = q[:, None] + torch.cumsum(Y * stepv, 1)[:, ks]
|
||||
qh = q[:, None] + torch.cumsum(xh.clamp(-1, 1) * astep, 1)[:, ks]
|
||||
qt = q[:, None] + torch.cumsum(Yf * astep, 1)[:, ks]
|
||||
B4 = qh.shape[0] * 4
|
||||
ph = fk_points(kin, qh.reshape(B4, 6), fk_idx); pt = fk_points(kin, qt.reshape(B4, 6), fk_idx)
|
||||
l_fk = ((ph - pt).norm(dim=-1) / 10).mean()
|
||||
@@ -105,6 +188,9 @@ def main():
|
||||
if not torch.isfinite(loss):
|
||||
print('非有限损失 → 训练发散(M5 门禁:触发 WTA 回退条件)', flush=True); save('diverged.pt', step); diverged = True; break
|
||||
opt.step(); seen += X.shape[0]
|
||||
with torch.no_grad(): # EMA;前期按 (1+s)/(10+s) 放宽,避免随机初值占权重
|
||||
dk = min(ema_d, (1 + step) / (10 + step))
|
||||
for pe, pm in zip(ema.parameters(), model.parameters()): pe.lerp_(pm.detach(), 1 - dk)
|
||||
if saver.stop() or (a.stop_at and step + 1 >= a.stop_at):
|
||||
save('last.pt', step + 1); log.close()
|
||||
print(f'收到停止请求:已存档 last.pt(第 {step + 1} 步),下次运行自动续训', flush=True); sys.exit(143)
|
||||
@@ -116,21 +202,25 @@ def main():
|
||||
'eta_h': (total - step - 1) * tc['batch'] / max(seen / max(el, 1e-6), 1e-6) / 3600, 'vram': vram.peak_gib()}
|
||||
print(json.dumps(rec), flush=True); log.write(json.dumps(rec) + '\n'); log.flush()
|
||||
if not a.no_eval and (step + 1) % tc['eval_every'] == 0:
|
||||
model.eval()
|
||||
res, _ = evaluate(model, kin, val_tasks, dev)
|
||||
res['step'] = step + 1; res['t'] = time.time()
|
||||
# bs=64 + 归还缓存池:WDDM 卡满时 15.9 GiB 的评估峰值会被驱动换出到内存,eval 从 2 分钟变 1 小时
|
||||
res, _ = evaluate(ema, kin, val_tasks, dev, bs=64)
|
||||
if vdata is not None: res.update(open_loop(ema, kin, vdata, astep, fk_idx))
|
||||
torch.cuda.empty_cache()
|
||||
sc = res['success'] + 0.05 * (1 - res['collision_rate']) + 0.05 * (1 - min(1.0, res.get('goal_ratio_2s', 1.0))) \
|
||||
- 1e-4 * min(res.get('ol_best_mm', 0.0), 500.0)
|
||||
res['step'] = step + 1; res['t'] = time.time(); res['score'] = sc; res['weights'] = 'ema'
|
||||
print('EVAL', json.dumps(res), flush=True); log.write(json.dumps({'eval': res}) + '\n'); log.flush()
|
||||
save('last.pt', step + 1)
|
||||
if res['success'] > best + 1e-4:
|
||||
best = res['success']; bad_evals = 0; save('best.pt', step + 1, {'eval': res})
|
||||
else:
|
||||
if step + 1 >= sel_from and sc > best + 1e-5:
|
||||
best = sc; bad_evals = 0; save('best.pt', step + 1, {'eval': res})
|
||||
elif step + 1 >= min_step:
|
||||
bad_evals += 1
|
||||
if bad_evals >= tc['patience_evals']: print('闭环到达率不再上升,停止', flush=True); break
|
||||
if bad_evals >= tc['patience_evals']: save('last.pt', step + 1); print('闭环分数不再上升,停止', flush=True); break
|
||||
save('last.pt', step + 1)
|
||||
if diverged: # 发散的权重不能覆盖 last.pt / best.pt:last.pt 保留发散前最后一次存档,M5 门禁会因此判不过 → 自动 WTA
|
||||
print('训练发散,last.pt / best.pt 保留发散前的存档', flush=True); return
|
||||
save('last.pt', step + 1)
|
||||
if a.no_eval: save('best.pt', step + 1)
|
||||
print(f'done, best success {best:.4f}, {(time.time() - t0) / 3600:.2f} h', flush=True)
|
||||
if a.no_eval or best <= -1.0: save('best.pt', step + 1) # 没评估过 / 总步数不到 sel_from:best = last
|
||||
print(f'done, best score {best:.4f}, {(time.time() - t0) / 3600:.2f} h', flush=True)
|
||||
|
||||
|
||||
if __name__ == '__main__':
|
||||
|
||||
@@ -71,7 +71,7 @@ const pct = (a, p) => { if (!a.length) return null; const s = a.slice().sort((x,
|
||||
for (let ci = 0; ci < cases.length; ci++) {
|
||||
const c = cases[ci];
|
||||
const R = mode === 'pose' ? c.R : null;
|
||||
const r = new POL.Runner(S, infer, { seed: 1000 + ci });
|
||||
const r = new POL.Runner(S, infer, { seed: 1000 + ci, actScale: meta.act_scale || 1 }); // 2026-10-19:动作尺度随模型
|
||||
r.reset(c.q0); r.setGoal(c.T, R);
|
||||
let prev = c.q0.slice(), cf = 0, tLock = null;
|
||||
for (let k = 0; k < 500 && !r.done; k++) {
|
||||
|
||||
@@ -55,7 +55,7 @@ async function boot() {
|
||||
}
|
||||
if (!sess) sess = await ort.InferenceSession.create(MODEL, { executionProviders: ep, graphOptimizationLevel: 'all' });
|
||||
const infer = ArmPolicy.ortInfer(ort, sess, { encOnce: !!meta.enc_once, featDyn: !!meta.feat_dyn });
|
||||
runner = new ArmPolicy.Runner(S, infer, {});
|
||||
runner = new ArmPolicy.Runner(S, infer, { actScale: meta.act_scale || 1 }); // 2026-10-19:动作尺度随模型(旧模型无此字段 = 1)
|
||||
/* 预热:首次推理有编译开销 */
|
||||
await infer(new Float32Array(8 * ArmPolicy.F_DIM), new Float32Array(8 * ArmPolicy.H * 6), 8);
|
||||
const tick = URL.createObjectURL(new Blob(['let h=null;onmessage=e=>{clearInterval(h);if(e.data>0)h=setInterval(()=>postMessage(0),e.data)}'], { type: 'text/javascript' }));
|
||||
|
||||
+39
-14
@@ -178,8 +178,10 @@ class Runner {
|
||||
this.S = S; this.infer = infer;
|
||||
this.o = Object.assign({ K: 8, commit: 3, margin: 5, /* 2026-10-18:运行时投影余量保持 5 mm(真机护栏,不迁就数据分布);数据侧另行过滤认证间隙 < 3 mm 的老师轨迹,APG 训练余量 3 mm */
|
||||
projMax: 0.5, refineMm: 5, arriveMm: 1, arriveDeg: 0.5,
|
||||
stuckPlans: 5, lockSec: 1.0, subReachDeg: 5, subTimeout: 1.5, seed: 12345, lookahead: true,
|
||||
certBudget: 40, safetyMs: 30, firstMs: 45 }, opts || {}); // 每条候选最多 40 次距离查询;安全层总预算 30 ms,第 1 条候选可用到 45 ms(超出按刹停处理)
|
||||
stuckPlans: 5, lockSec: 1.0, subReachDeg: 5, subTimeout: 1.5, subRule: 'cross', subLeadSec: 0.02, seed: 12345, lookahead: true,
|
||||
actScale: 1, certBudget: 40, safetyMs: 30, firstMs: 45 }, opts || {});
|
||||
/* 2026-10-19:actScale = 模型 meta.act_scale(动作标签按 act_scale·STEP 归一化,旧模型 1);增量 = clamp(act, ±1)·actScale·STEP,
|
||||
与 ml/sim/closed_loop.py 同口径。subRule:'cross'(默认,越过顶点即切,与训练标签同口径;见 _subAdvance)/ 'timeout'(旧:< subReachDeg 或 subTimeout 超时)。 */ // 每条候选最多 40 次距离查询;安全层总预算 30 ms,第 1 条候选可用到 45 ms(超出按刹停处理)
|
||||
this.cert = makeCert(S, { leverTable: this.o.leverTable, cap: 20 });
|
||||
this.rng = makeRng(this.o.seed);
|
||||
/* 2026-10-18 第六轮(审查 B-1/B-2):耗时拆开记——inferMs 主推理、lookaheadMs 前瞻(第二次推理 + 8 份特征)、safetyMs 只含认证、
|
||||
@@ -190,7 +192,7 @@ class Runner {
|
||||
reset(q0) {
|
||||
this.q = q0 ? q0.slice() : null; this.qPrev = this.q ? this.q.slice() : null;
|
||||
this.queue = []; this.brakeTail = []; this.lastPlan = null; this.goal = null;
|
||||
this.subs = []; this.subT = 0; this.vals = []; this.brakeRun = 0; this.lock = null; this.t = 0; this.cold = true; this.done = false;
|
||||
this.subs = []; this.subT = 0; this.subPrev = null; this.vals = []; this.brakeRun = 0; this.lock = null; this.t = 0; this.cold = true; this.done = false;
|
||||
}
|
||||
setGoal(T, R) {
|
||||
const jump = !this.goal || Math.hypot(T[0] - this.goal.T[0], T[1] - this.goal.T[1], T[2] - this.goal.T[2]) > 50 || (!!R !== !!this.goal.R);
|
||||
@@ -220,10 +222,34 @@ class Runner {
|
||||
return this.q;
|
||||
}
|
||||
_cand(act, k, r) { // 第 k 条候选:增量(°),丢掉模型的前 r 步(已提交前缀),取接下来的步
|
||||
const inc = [];
|
||||
for (let s = 0; s < H; s++) { const row = []; for (let j = 0; j < 6; j++) row.push(Math.max(-1, Math.min(1, act[(k * H + s) * 6 + j])) * STEP[j]); inc.push(row); }
|
||||
const inc = [], sc = this.o.actScale || 1;
|
||||
for (let s = 0; s < H; s++) { const row = []; for (let j = 0; j < 6; j++) row.push(Math.max(-1, Math.min(1, act[(k * H + s) * 6 + j])) * sc * STEP[j]); inc.push(row); }
|
||||
return inc;
|
||||
}
|
||||
/* 子目标推进(与 ml/sim/closed_loop.py sub_advance 同一规则,三处同步改)。
|
||||
2026-10-19(审查 C5 / P0-1):训练标签里 sub = 老师「下一个尚未经过的顶点」,老师 τ 一过顶点(−0.02 s)就换;
|
||||
旧规则要 < 5° 或 1.5 s 超时,策略在顶点附近进不去时 sub 一直指着身后的点(训练里从未出现的特征序列)。
|
||||
'cross':进入方向 a = 顶点 − 上一点,离开方向 b = 下一顶点 − 顶点(最后一个取 a),各轴 /VMAX 折成秒;
|
||||
法向 n = â + b̂,(qe − 顶点)·n̂ ≥ −subLeadSec 即越过。没有超时(标签也没有)。 */
|
||||
_subAdvance(qe) {
|
||||
if (!this.subs.length) return;
|
||||
const S = this.S, o = this.o, sg = this.subs[0];
|
||||
const df = (a, b, j) => (S.LIM[S.ORDER[j]].cont ? wrap180(a - b) : a - b);
|
||||
let adv = false;
|
||||
if (o.subRule === 'timeout') {
|
||||
let m = 0; for (let j = 0; j < 6; j++) m = Math.max(m, Math.abs(df(sg[j], qe[j], j)));
|
||||
adv = m < o.subReachDeg || this.t - this.subT > o.subTimeout;
|
||||
} else {
|
||||
const unit = v => { const n = Math.hypot.apply(null, v); return n > 1e-9 ? v.map(x => x / n) : v.map(() => 0); };
|
||||
const prev = this.subPrev || this.q;
|
||||
const a = [0, 1, 2, 3, 4, 5].map(j => df(sg[j], prev[j], j) / VMAX[j]);
|
||||
const b = this.subs.length > 1 ? [0, 1, 2, 3, 4, 5].map(j => df(this.subs[1][j], sg[j], j) / VMAX[j]) : a;
|
||||
const ua = unit(a), ub = unit(b), n = unit(ua.map((x, j) => x + ub[j]));
|
||||
let proj = 0; for (let j = 0; j < 6; j++) proj += df(qe[j], sg[j], j) / VMAX[j] * n[j];
|
||||
adv = proj >= -o.subLeadSec;
|
||||
}
|
||||
if (adv) { this.subPrev = this.subs.shift(); this.subT = this.t; }
|
||||
}
|
||||
_certCommit(qe, steps) { // 认证:提交步 + 刹停段(先用离散判定快速剔除,再做距离下界认证)
|
||||
const pts = [qe.slice()]; let q = qe.slice();
|
||||
for (const d of steps) { q = q.map((v, j) => v + d[j]); pts.push(q.slice()); }
|
||||
@@ -257,13 +283,11 @@ class Runner {
|
||||
const steps = []; let q = qe.slice();
|
||||
for (let k = 0; k < o.commit; k++) { const rs = refineStep(S, q, this.goal.T, this.goal.R, 0.3); steps.push(rs.dq); q = q.map((v, j) => v + rs.dq[j]); }
|
||||
const c = this._certCommit(qe, steps);
|
||||
if (c.ok) { this.queue.push(...steps); this.brakeTail = c.tail; this.stats.refine++; return; }
|
||||
/* 2026-10-19(审查 F5):DLS 接管时没有「已提交的模型计划」,lastPlan 置空(原来保留上一条,下次一致性去对齐一条没执行的轨迹;仿真同步) */
|
||||
if (c.ok) { this.queue.push(...steps); this.brakeTail = c.tail; this.stats.refine++; this.lastPlan = null; return; }
|
||||
}
|
||||
/* 子目标推进 */
|
||||
if (this.subs.length) {
|
||||
const sg = this.subs[0]; let m = 0; for (let j = 0; j < 6; j++) m = Math.max(m, Math.abs((S.LIM[S.ORDER[j]].cont ? wrap180(sg[j] - qe[j]) : sg[j] - qe[j])));
|
||||
if (m < o.subReachDeg || this.t - this.subT > o.subTimeout) { this.subs.shift(); this.subT = this.t; }
|
||||
}
|
||||
this._subAdvance(qe);
|
||||
const gm = S.gapMatrix(this.q, GAP_CAP);
|
||||
const st = { q: this.q, qPrev: this.qPrev, T: this.goal.T, R: this.goal.R, prefix, sub: this.subs[0] || null };
|
||||
const f = features(S, st, gm);
|
||||
@@ -279,7 +303,7 @@ class Runner {
|
||||
if (this.needSub) {
|
||||
this.needSub = false; this.subT = this.t;
|
||||
const sb = out.sub.subarray(best * 13, best * 13 + 13);
|
||||
if (sb[12] > 0.5) this.subs = [0, 1].map(v => this.q.map((x, j) => x + sb[v * 6 + j] * 180));
|
||||
if (sb[12] > 0.5) { this.subs = [0, 1].map(v => this.q.map((x, j) => x + sb[v * 6 + j] * 180)); this.subPrev = this.q.slice(); }
|
||||
}
|
||||
/* 候选:去掉前缀后的提交步,投影,打分 */
|
||||
const cands = [], near = S.criticalPairs(qe, o.margin + 10).slice(0, 64);
|
||||
@@ -297,10 +321,11 @@ class Runner {
|
||||
const fut = []; let qq = qe.slice(); for (let s = r; s < H; s++) { qq = qq.map((v, j) => v + inc[s][j]); fut.push(qq); }
|
||||
let j1 = 0; for (let s = 0; s < H; s++) j1 += inc[s][0];
|
||||
let score = -2.0 * out.value[k];
|
||||
if (this.lastPlan) { // 与上一条已选计划的未来重叠步对齐(时间平移 3 步)
|
||||
if (this.lastPlan) { // 与上一条已选计划的未来重叠步对齐:lastPlan[i] = 旧 qe 走 i 步,新 qe = lastPlan[commit],fut[s] 对应 lastPlan[s+commit+1]
|
||||
// 2026-10-19 修错位(原对齐 lastPlan[s+commit],奖励比上一计划慢一步的候选;Python closed_loop.py 同步修正)
|
||||
let c = 0, w = 1;
|
||||
for (let s = 0; s < fut.length && s + o.commit < this.lastPlan.length; s++, w *= 0.85) {
|
||||
let d2 = 0; for (let j = 0; j < 6; j++) d2 += Math.pow((fut[s][j] - this.lastPlan[s + o.commit][j]) / STEP[j], 2);
|
||||
for (let s = 0; s < fut.length && s + o.commit + 1 < this.lastPlan.length; s++, w *= 0.85) {
|
||||
let d2 = 0; for (let j = 0; j < 6; j++) d2 += Math.pow((fut[s][j] - this.lastPlan[s + o.commit + 1][j]) / STEP[j], 2);
|
||||
c += w * d2;
|
||||
}
|
||||
score -= 0.02 * c;
|
||||
|
||||
Reference in New Issue
Block a user