exp: #27——K16增益不集中在B/C(总体+3.9pp/净10题,B仅+4题不显著);B刹停-29%但到达仅27→32%,证明B/C缺口是策略不产绕行而非闸门
This commit is contained in:
@@ -0,0 +1,107 @@
|
||||
{
|
||||
"config": {
|
||||
"ckpt": "ml/runs/base_dag2_dag1/best.pt",
|
||||
"samples": "ml/data/samples",
|
||||
"tasks": null,
|
||||
"split": "val",
|
||||
"n": 256,
|
||||
"K": 16,
|
||||
"commit": 2,
|
||||
"fb": 2.2,
|
||||
"ki": 0.0,
|
||||
"fb_scale": 0.0,
|
||||
"fb_near": 0.2,
|
||||
"fb_fallback": 0.0,
|
||||
"kdecay": 0.9,
|
||||
"kimax": 0.5,
|
||||
"fb_every": 1,
|
||||
"safety_try": 0,
|
||||
"sel_w": 0.1,
|
||||
"dls_safety": 1,
|
||||
"fb_clear": 0.0,
|
||||
"fb_cmin": 0.3,
|
||||
"sub_refresh": 0,
|
||||
"fb_block_gain": 0.5,
|
||||
"dump": "ml/runs/round42_K16cb1_rows.json",
|
||||
"out": "docs/experiment_round42_K16cb1_dump.json",
|
||||
"trace": null,
|
||||
"select": "endpoint",
|
||||
"replan": "empty",
|
||||
"safety": 1,
|
||||
"partial": 1,
|
||||
"inc_cap": 1,
|
||||
"gap_margin": 0.0,
|
||||
"cert_batch": 1,
|
||||
"steps": 1,
|
||||
"z_scale": 1.0,
|
||||
"len_w": 0.0,
|
||||
"backoff": 0,
|
||||
"max_sec": null,
|
||||
"seed": 0,
|
||||
"dls": "analytic",
|
||||
"device": "cuda"
|
||||
},
|
||||
"batch_size": 256,
|
||||
"batch_seed_rule": "same seed per batch",
|
||||
"checkpoint_sha256": "f7f686fd8ee92fb6948fbcc4ee954636065ad87bb54f661c4f8f575bc16c2ce1",
|
||||
"tasks_sha256": "249bd93457825cda98c59e64d9a096d3473c69aeb3553de1dea6d9e56be71772",
|
||||
"kin_fp": "2ebcbf17",
|
||||
"act_scale": 1.0,
|
||||
"metrics": {
|
||||
"n": 256,
|
||||
"n_ref": 256,
|
||||
"success": 0.5625,
|
||||
"collision_free_success": 0.5625,
|
||||
"time_ok_1p3": 0.296875,
|
||||
"collision_rate": 0.0,
|
||||
"collision_steps_mean": 0.0,
|
||||
"travel_ratio_median": 2.6243183612823486,
|
||||
"time_mean_s": 2.6024999618530273,
|
||||
"brakes_mean": 13.7734375,
|
||||
"start_fail": 0.0,
|
||||
"goal_ratio_2s": 0.2799961566925049,
|
||||
"err_fail_med": 195.448486328125,
|
||||
"err_success_med": 0.15947841107845306,
|
||||
"dls_share": 0.640625,
|
||||
"time_budget": "teacher×2(≥3 s)",
|
||||
"speed_ratio_p50": 1.0000016689300537,
|
||||
"speed_ratio_p95": 1.0000101327896118,
|
||||
"speed_ratio_max": 1.0000101327896118,
|
||||
"over_vmax_rate": 0.0,
|
||||
"accel_ratio_p95": 1.0000016689300537,
|
||||
"accel_ratio_max": 1.0000101327896118,
|
||||
"reverse_p50": 0.26249998807907104,
|
||||
"reverse_mean": 0.2693643867969513,
|
||||
"reverse_moving_p50": 0.3194444477558136,
|
||||
"moving_share_p50": 0.9869565367698669,
|
||||
"brake_rate_mean": 0.07063301652669907,
|
||||
"brake_rate_p50": 0.0,
|
||||
"brake_rate_p90": 0.2589743733406067,
|
||||
"stuck_share": 0.15234375,
|
||||
"n_plans_mean": 195.0,
|
||||
"success_reachable": 0.5925925970077515,
|
||||
"by_cat": {
|
||||
"A": [
|
||||
128,
|
||||
100
|
||||
],
|
||||
"B": [
|
||||
74,
|
||||
24
|
||||
],
|
||||
"C": [
|
||||
32,
|
||||
12
|
||||
],
|
||||
"D": [
|
||||
13,
|
||||
0
|
||||
],
|
||||
"Dr": [
|
||||
9,
|
||||
8
|
||||
]
|
||||
}
|
||||
},
|
||||
"simulation_wall_s": 613.4877918999991
|
||||
}
|
||||
@@ -108,6 +108,23 @@
|
||||
- **对 `backoff` 的预期与判据**:全拒意味着"连第 1 步的端点或中点已经穿进障碍",缩步能不能救取决于**穿得有多深**(穿 0.1 mm 救得回,穿 3 mm 救不回)。这个数转储里没记(只记被选中那条),所以直接由 `backoff_mean`/`reject_all_mean` 来量:
|
||||
`reject_all_mean ≈ 原全拒次数`(=backoff_mean+brakes_mean,与轮次 33 cap 的 19.55 可比),`backoff_mean/(backoff_mean+brakes_mean)` 就是"缩步能救回的比例"。这是补跑(#22)要读的第一个数。
|
||||
|
||||
## #27:K=16 的增益**不集中在 B/C**——B 类缺口是策略能力,不是候选容量(轮次 44,逐题 dump)
|
||||
|
||||
- K=8 与 K=16(都 `--cert_batch 1 --inc_cap 1`,同 256 题同 seed)逐题对齐后按类别:
|
||||
|
||||
| 类别 | 题数 | K8 到达 | K16 到达 | 差 | K8 刹停/题 | K16 刹停/题 |
|
||||
|---|---|---|---|---|---|---|
|
||||
| A 均匀 | 128 | 75% | 78% | +3.1 pp | 7.5 | 4.9 |
|
||||
| B 直线被挡 | 74 | 27% | **32%** | +5.4 pp | 38.1 | **27.0** |
|
||||
| C 贴立柱 | 32 | 34% | 38% | +3.1 pp | 33.3 | 24.5 |
|
||||
| D 不可达 | 13 | 0% | 0% | 0 | 5.2 | 4.8 |
|
||||
| Dr | 9 | 78% | 89% | +11.1 pp | 10.2 | 6.4 |
|
||||
|
||||
逐题翻转:**新救回 12 题(A 6、B 4、C 1、Dr 1),反而丢 2 题(都在 A)**,净 +10 题 = 总体 0.5234 → 0.5625。
|
||||
- **统计诚实**:分类切片里 1–4 题的变化在这个分母上不构成显著(B 类 +4 题 = +5.4 pp,单看会高估);**稳的只有两条**——总体 +3.9 pp(净 10 题)与刹停整体 −29%(B 从 38.1 降到 27.0 次/题,是 74 题上的均值移动)。
|
||||
- **关键读法(决定预算投向)**:候选容量翻倍把 B 的**拒绝次数**砍掉近三成,但 B 的到达只从 27% 到 32%,离 A 类的 78% 还差 46 pp。⇒ **"少被拒"不等于"会绕"**:B/C 的缺口是策略在这些状态下**根本不产出绕行轨迹**,不是被闸门挡住的。继续在排序/认证口径上打磨救不了 B/C;要动的是数据侧(在受阻状态下采集教师轨迹,即 DAgger 第三轮,需要放行)或明确把 B/C 列为当前能力边界之外。
|
||||
- 由此更新候选清单:#24 修正版(`tie='gap'`,等长前缀时按沿路最小间隙定夺)仍然值得测——它影响的是"选了哪条活路",会影响后续拍的存活率;但它**不能**被期待修好 B/C 的 46 pp,那属于训练侧。
|
||||
|
||||
## 分层指标上线并被三重交叉验证(轮次 43)
|
||||
|
||||
- 对照配方(轮次 33 cap:1 步、`--inc_cap 1`)重跑一次带分层指标的评估:**原有 22 个指标逐位不变**,新增键全部产出(`docs/experiment_round43_control_strat.json`):
|
||||
|
||||
Reference in New Issue
Block a user