1254 lines
50 KiB
HTML
1254 lines
50 KiB
HTML
<!DOCTYPE html>
|
||||
|
|
<html xmlns="http://www.w3.org/1999/xhtml" lang="" xml:lang="">
|
|||
|
|
<head>
|
|||
|
|
<meta charset="utf-8" />
|
|||
|
|
<meta name="generator" content="pandoc" />
|
|||
|
|
<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=yes" />
|
|||
|
|
<title>Transformer 避障运动策略:无老师闭环部署方案(v2)</title>
|
|||
|
|
<style>
|
|||
|
|
html {
|
|||
|
|
color: #1a1a1a;
|
|||
|
|
background-color: #fdfdfd;
|
|||
|
|
}
|
|||
|
|
body {
|
|||
|
|
margin: 0 auto;
|
|||
|
|
max-width: 36em;
|
|||
|
|
padding-left: 50px;
|
|||
|
|
padding-right: 50px;
|
|||
|
|
padding-top: 50px;
|
|||
|
|
padding-bottom: 50px;
|
|||
|
|
hyphens: auto;
|
|||
|
|
overflow-wrap: break-word;
|
|||
|
|
text-rendering: optimizeLegibility;
|
|||
|
|
font-kerning: normal;
|
|||
|
|
}
|
|||
|
|
@media (max-width: 600px) {
|
|||
|
|
body {
|
|||
|
|
font-size: 0.9em;
|
|||
|
|
padding: 12px;
|
|||
|
|
}
|
|||
|
|
h1 {
|
|||
|
|
font-size: 1.8em;
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
@media print {
|
|||
|
|
html {
|
|||
|
|
background-color: white;
|
|||
|
|
}
|
|||
|
|
body {
|
|||
|
|
background-color: transparent;
|
|||
|
|
color: black;
|
|||
|
|
font-size: 12pt;
|
|||
|
|
}
|
|||
|
|
p, h2, h3 {
|
|||
|
|
orphans: 3;
|
|||
|
|
widows: 3;
|
|||
|
|
}
|
|||
|
|
h2, h3, h4 {
|
|||
|
|
page-break-after: avoid;
|
|||
|
|
}
|
|||
|
|
}
|
|||
|
|
p {
|
|||
|
|
margin: 1em 0;
|
|||
|
|
}
|
|||
|
|
a {
|
|||
|
|
color: #1a1a1a;
|
|||
|
|
}
|
|||
|
|
a:visited {
|
|||
|
|
color: #1a1a1a;
|
|||
|
|
}
|
|||
|
|
img {
|
|||
|
|
max-width: 100%;
|
|||
|
|
}
|
|||
|
|
svg {
|
|||
|
|
height: auto;
|
|||
|
|
max-width: 100%;
|
|||
|
|
}
|
|||
|
|
h1, h2, h3, h4, h5, h6 {
|
|||
|
|
margin-top: 1.4em;
|
|||
|
|
}
|
|||
|
|
h5, h6 {
|
|||
|
|
font-size: 1em;
|
|||
|
|
font-style: italic;
|
|||
|
|
}
|
|||
|
|
h6 {
|
|||
|
|
font-weight: normal;
|
|||
|
|
}
|
|||
|
|
ol, ul {
|
|||
|
|
padding-left: 1.7em;
|
|||
|
|
margin-top: 1em;
|
|||
|
|
}
|
|||
|
|
li > ol, li > ul {
|
|||
|
|
margin-top: 0;
|
|||
|
|
}
|
|||
|
|
blockquote {
|
|||
|
|
margin: 1em 0 1em 1.7em;
|
|||
|
|
padding-left: 1em;
|
|||
|
|
border-left: 2px solid #e6e6e6;
|
|||
|
|
color: #606060;
|
|||
|
|
}
|
|||
|
|
code {
|
|||
|
|
font-family: Menlo, Monaco, Consolas, 'Lucida Console', monospace;
|
|||
|
|
font-size: 85%;
|
|||
|
|
margin: 0;
|
|||
|
|
hyphens: manual;
|
|||
|
|
}
|
|||
|
|
pre {
|
|||
|
|
margin: 1em 0;
|
|||
|
|
overflow: auto;
|
|||
|
|
}
|
|||
|
|
pre code {
|
|||
|
|
padding: 0;
|
|||
|
|
overflow: visible;
|
|||
|
|
overflow-wrap: normal;
|
|||
|
|
}
|
|||
|
|
.sourceCode {
|
|||
|
|
background-color: transparent;
|
|||
|
|
overflow: visible;
|
|||
|
|
}
|
|||
|
|
hr {
|
|||
|
|
background-color: #1a1a1a;
|
|||
|
|
border: none;
|
|||
|
|
height: 1px;
|
|||
|
|
margin: 1em 0;
|
|||
|
|
}
|
|||
|
|
table {
|
|||
|
|
margin: 1em 0;
|
|||
|
|
border-collapse: collapse;
|
|||
|
|
width: 100%;
|
|||
|
|
overflow-x: auto;
|
|||
|
|
display: block;
|
|||
|
|
font-variant-numeric: lining-nums tabular-nums;
|
|||
|
|
}
|
|||
|
|
table caption {
|
|||
|
|
margin-bottom: 0.75em;
|
|||
|
|
}
|
|||
|
|
tbody {
|
|||
|
|
margin-top: 0.5em;
|
|||
|
|
border-top: 1px solid #1a1a1a;
|
|||
|
|
border-bottom: 1px solid #1a1a1a;
|
|||
|
|
}
|
|||
|
|
th {
|
|||
|
|
border-top: 1px solid #1a1a1a;
|
|||
|
|
padding: 0.25em 0.5em 0.25em 0.5em;
|
|||
|
|
}
|
|||
|
|
td {
|
|||
|
|
padding: 0.125em 0.5em 0.25em 0.5em;
|
|||
|
|
}
|
|||
|
|
header {
|
|||
|
|
margin-bottom: 4em;
|
|||
|
|
text-align: center;
|
|||
|
|
}
|
|||
|
|
#TOC li {
|
|||
|
|
list-style: none;
|
|||
|
|
}
|
|||
|
|
#TOC ul {
|
|||
|
|
padding-left: 1.3em;
|
|||
|
|
}
|
|||
|
|
#TOC > ul {
|
|||
|
|
padding-left: 0;
|
|||
|
|
}
|
|||
|
|
#TOC a:not(:hover) {
|
|||
|
|
text-decoration: none;
|
|||
|
|
}
|
|||
|
|
code{white-space: pre-wrap;}
|
|||
|
|
span.smallcaps{font-variant: small-caps;}
|
|||
|
|
div.columns{display: flex; gap: min(4vw, 1.5em);}
|
|||
|
|
div.column{flex: auto; overflow-x: auto;}
|
|||
|
|
div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;}
|
|||
|
|
/* The extra [class] is a hack that increases specificity enough to
|
|||
|
|
override a similar rule in reveal.js */
|
|||
|
|
ul.task-list[class]{list-style: none;}
|
|||
|
|
ul.task-list li input[type="checkbox"] {
|
|||
|
|
font-size: inherit;
|
|||
|
|
width: 0.8em;
|
|||
|
|
margin: 0 0.8em 0.2em -1.6em;
|
|||
|
|
vertical-align: middle;
|
|||
|
|
}
|
|||
|
|
.display.math{display: block; text-align: center; margin: 0.5rem auto;}
|
|||
|
|
/* CSS for syntax highlighting */
|
|||
|
|
pre > code.sourceCode { white-space: pre; position: relative; }
|
|||
|
|
pre > code.sourceCode > span { line-height: 1.25; }
|
|||
|
|
pre > code.sourceCode > span:empty { height: 1.2em; }
|
|||
|
|
.sourceCode { overflow: visible; }
|
|||
|
|
code.sourceCode > span { color: inherit; text-decoration: inherit; }
|
|||
|
|
div.sourceCode { margin: 1em 0; }
|
|||
|
|
pre.sourceCode { margin: 0; }
|
|||
|
|
@media screen {
|
|||
|
|
div.sourceCode { overflow: auto; }
|
|||
|
|
}
|
|||
|
|
@media print {
|
|||
|
|
pre > code.sourceCode { white-space: pre-wrap; }
|
|||
|
|
pre > code.sourceCode > span { display: inline-block; text-indent: -5em; padding-left: 5em; }
|
|||
|
|
}
|
|||
|
|
pre.numberSource code
|
|||
|
|
{ counter-reset: source-line 0; }
|
|||
|
|
pre.numberSource code > span
|
|||
|
|
{ position: relative; left: -4em; counter-increment: source-line; }
|
|||
|
|
pre.numberSource code > span > a:first-child::before
|
|||
|
|
{ content: counter(source-line);
|
|||
|
|
position: relative; left: -1em; text-align: right; vertical-align: baseline;
|
|||
|
|
border: none; display: inline-block;
|
|||
|
|
-webkit-touch-callout: none; -webkit-user-select: none;
|
|||
|
|
-khtml-user-select: none; -moz-user-select: none;
|
|||
|
|
-ms-user-select: none; user-select: none;
|
|||
|
|
padding: 0 4px; width: 4em;
|
|||
|
|
color: #aaaaaa;
|
|||
|
|
}
|
|||
|
|
pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; }
|
|||
|
|
div.sourceCode
|
|||
|
|
{ }
|
|||
|
|
@media screen {
|
|||
|
|
pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; }
|
|||
|
|
}
|
|||
|
|
code span.al { color: #ff0000; font-weight: bold; } /* Alert */
|
|||
|
|
code span.an { color: #60a0b0; font-weight: bold; font-style: italic; } /* Annotation */
|
|||
|
|
code span.at { color: #7d9029; } /* Attribute */
|
|||
|
|
code span.bn { color: #40a070; } /* BaseN */
|
|||
|
|
code span.bu { color: #008000; } /* BuiltIn */
|
|||
|
|
code span.cf { color: #007020; font-weight: bold; } /* ControlFlow */
|
|||
|
|
code span.ch { color: #4070a0; } /* Char */
|
|||
|
|
code span.cn { color: #880000; } /* Constant */
|
|||
|
|
code span.co { color: #60a0b0; font-style: italic; } /* Comment */
|
|||
|
|
code span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } /* CommentVar */
|
|||
|
|
code span.do { color: #ba2121; font-style: italic; } /* Documentation */
|
|||
|
|
code span.dt { color: #902000; } /* DataType */
|
|||
|
|
code span.dv { color: #40a070; } /* DecVal */
|
|||
|
|
code span.er { color: #ff0000; font-weight: bold; } /* Error */
|
|||
|
|
code span.ex { } /* Extension */
|
|||
|
|
code span.fl { color: #40a070; } /* Float */
|
|||
|
|
code span.fu { color: #06287e; } /* Function */
|
|||
|
|
code span.im { color: #008000; font-weight: bold; } /* Import */
|
|||
|
|
code span.in { color: #60a0b0; font-weight: bold; font-style: italic; } /* Information */
|
|||
|
|
code span.kw { color: #007020; font-weight: bold; } /* Keyword */
|
|||
|
|
code span.op { color: #666666; } /* Operator */
|
|||
|
|
code span.ot { color: #007020; } /* Other */
|
|||
|
|
code span.pp { color: #bc7a00; } /* Preprocessor */
|
|||
|
|
code span.sc { color: #4070a0; } /* SpecialChar */
|
|||
|
|
code span.ss { color: #bb6688; } /* SpecialString */
|
|||
|
|
code span.st { color: #4070a0; } /* String */
|
|||
|
|
code span.va { color: #19177c; } /* Variable */
|
|||
|
|
code span.vs { color: #4070a0; } /* VerbatimString */
|
|||
|
|
code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } /* Warning */
|
|||
|
|
</style>
|
|||
|
|
<style>
|
|||
|
|
:root{--fg:#1d2733;--mut:#5b6b7c;--acc:#1f5fa8;--bd:#d9e1ea;--bg2:#f5f8fb}
|
|||
|
|
body{max-width:1080px;margin:0 auto;padding:32px 28px 80px;font:15px/1.75 -apple-system,"PingFang SC","Microsoft YaHei","Noto Sans CJK SC",sans-serif;color:var(--fg)}
|
|||
|
|
h1{font-size:26px;border-bottom:3px solid var(--acc);padding-bottom:10px}
|
|||
|
|
h2{font-size:21px;margin-top:44px;padding-left:10px;border-left:5px solid var(--acc)}
|
|||
|
|
h3{font-size:17px;margin-top:28px;color:#16324f}
|
|||
|
|
table{border-collapse:collapse;width:100%;margin:14px 0;font-size:13.5px;display:block;overflow-x:auto}
|
|||
|
|
th,td{border:1px solid var(--bd);padding:6px 9px;vertical-align:top}
|
|||
|
|
th{background:#e9f0f7;text-align:left;white-space:nowrap}
|
|||
|
|
tr:nth-child(even) td{background:var(--bg2)}
|
|||
|
|
code{background:#eef2f6;padding:1px 5px;border-radius:4px;font-size:90%}
|
|||
|
|
pre{background:#0f1b2a;color:#e3ecf5;padding:14px 16px;border-radius:8px;overflow-x:auto;line-height:1.5}
|
|||
|
|
pre code{background:none;color:inherit;padding:0}
|
|||
|
|
blockquote{margin:16px 0;padding:10px 16px;background:#fff8e6;border-left:4px solid #e0a800;color:#4a3b00}
|
|||
|
|
#TOC{background:var(--bg2);border:1px solid var(--bd);border-radius:8px;padding:12px 20px;margin:20px 0;font-size:14px}
|
|||
|
|
#TOC ul{margin:4px 0;padding-left:20px}
|
|||
|
|
a{color:var(--acc)}
|
|||
|
|
</style>
|
|||
|
|
</head>
|
|||
|
|
<body>
|
|||
|
|
<nav id="TOC" role="doc-toc">
|
|||
|
|
<ul>
|
|||
|
|
<li><a href="#transformer-避障运动策略无老师闭环部署方案审查稿-v2"
|
|||
|
|
id="toc-transformer-避障运动策略无老师闭环部署方案审查稿-v2">Transformer
|
|||
|
|
避障运动策略:无老师闭环部署方案(审查稿 v2)</a>
|
|||
|
|
<ul>
|
|||
|
|
<li><a href="#0-定位与边界" id="toc-0-定位与边界">0. 定位与边界</a></li>
|
|||
|
|
<li><a href="#1-业界调研具身智能里的神经运动规划"
|
|||
|
|
id="toc-1-业界调研具身智能里的神经运动规划">1.
|
|||
|
|
业界调研:具身智能里的神经运动规划</a></li>
|
|||
|
|
<li><a href="#2-我们的方案和主流的差异以及据此做的优化"
|
|||
|
|
id="toc-2-我们的方案和主流的差异以及据此做的优化">2.
|
|||
|
|
我们的方案和主流的差异,以及据此做的优化</a></li>
|
|||
|
|
<li><a href="#3-训练环境" id="toc-3-训练环境">3. 训练环境</a></li>
|
|||
|
|
<li><a href="#4-第一步离线造预训练数据老师只在这里出现"
|
|||
|
|
id="toc-4-第一步离线造预训练数据老师只在这里出现">4.
|
|||
|
|
第一步:离线造预训练数据(老师只在这里出现)</a></li>
|
|||
|
|
<li><a href="#5-第二步表示与预处理" id="toc-5-第二步表示与预处理">5.
|
|||
|
|
第二步:表示与预处理</a></li>
|
|||
|
|
<li><a href="#6-第三步模型与预训练行为克隆"
|
|||
|
|
id="toc-6-第三步模型与预训练行为克隆">6.
|
|||
|
|
第三步:模型与预训练(行为克隆)</a></li>
|
|||
|
|
<li><a href="#7-第四步后训练无老师为主"
|
|||
|
|
id="toc-7-第四步后训练无老师为主">7.
|
|||
|
|
第四步:后训练(无老师为主)</a></li>
|
|||
|
|
<li><a href="#8-第五步无老师闭环部署" id="toc-8-第五步无老师闭环部署">8.
|
|||
|
|
第五步:无老师闭环部署</a></li>
|
|||
|
|
<li><a href="#9-第六步评估与验收" id="toc-9-第六步评估与验收">9.
|
|||
|
|
第六步:评估与验收</a></li>
|
|||
|
|
<li><a href="#10-风险与维护" id="toc-10-风险与维护">10.
|
|||
|
|
风险与维护</a></li>
|
|||
|
|
<li><a href="#11-目录规划" id="toc-11-目录规划">11. 目录规划</a></li>
|
|||
|
|
<li><a href="#12-里程碑" id="toc-12-里程碑">12. 里程碑</a></li>
|
|||
|
|
<li><a href="#13-参考文献" id="toc-13-参考文献">13. 参考文献</a></li>
|
|||
|
|
<li><a href="#14-需要用户确认" id="toc-14-需要用户确认">14.
|
|||
|
|
需要用户确认</a></li>
|
|||
|
|
</ul></li>
|
|||
|
|
</ul>
|
|||
|
|
</nav>
|
|||
|
|
<h1 id="transformer-避障运动策略无老师闭环部署方案审查稿-v2">Transformer
|
|||
|
|
避障运动策略:无老师闭环部署方案(审查稿 v2)</h1>
|
|||
|
|
<blockquote>
|
|||
|
|
<ul>
|
|||
|
|
<li>v1(2026-10-17 20:06,<code>40af3b4</code>):模型一次生成整条 32
|
|||
|
|
点路径,失败回落到原规划器("有老师部署")。</li>
|
|||
|
|
<li>v1.1(20:25,<code>c51f8f9</code>):加入后训练挖错循环。</li>
|
|||
|
|
<li><strong>v2(本稿)</strong>:按用户决定改为 <strong>无老师部署 +
|
|||
|
|
每个控制周期重算(闭环)</strong>,删除全部"回落原规划器"设计;
|
|||
|
|
加入具身智能领域主流做法的调研与对比(§1–§2),并据此调整模型输出、后训练方法和验收标准。</li>
|
|||
|
|
<li>基于
|
|||
|
|
<code>main@3c7c814</code>(贴立柱修复后的规划器)。本文只是方案,代码尚未编写。</li>
|
|||
|
|
<li>审查重点:§0 定位、§2 差异与优化、§5 输出表示、§7 后训练(APG)、§8
|
|||
|
|
闭环部署与安全层、§9 验收。</li>
|
|||
|
|
</ul>
|
|||
|
|
</blockquote>
|
|||
|
|
<hr />
|
|||
|
|
<h2 id="0-定位与边界">0. 定位与边界</h2>
|
|||
|
|
<table>
|
|||
|
|
<thead>
|
|||
|
|
<tr class="header">
|
|||
|
|
<th>项目</th>
|
|||
|
|
<th>决定</th>
|
|||
|
|
</tr>
|
|||
|
|
</thead>
|
|||
|
|
<tbody>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>部署形态</td>
|
|||
|
|
<td><strong>无老师</strong>:运行时只有「神经策略 + 碰撞安全层」,不加载
|
|||
|
|
IK 大表、PRM 路网、A*/RRT,也不调用 <code>solveDyn</code> /
|
|||
|
|
<code>routeRT</code></td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>运行方式</td>
|
|||
|
|
<td><strong>闭环滚动重算</strong>(MPC 式):每个控制周期(默认 50
|
|||
|
|
Hz,20
|
|||
|
|
ms)读当前关节角和目标,重新推理一段短动作块,只执行最前面一小段,下一周期再算。某一周期算错,下一周期有机会纠正</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>碰撞安全层</td>
|
|||
|
|
<td><strong>保留</strong>。它不是老师:不做搜索,只对模型给出的下一小段做胶囊碰撞检查(已有
|
|||
|
|
<code>collideFree</code> + 包围球预筛),不安全就换候选或刹停</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>老师的角色</td>
|
|||
|
|
<td>只在<strong>离线训练</strong>时出现:生成预训练数据。后训练阶段用可微碰撞代价(§7),基本不再需要老师</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>不做的事</td>
|
|||
|
|
<td>不学动力学(速度/加速度上限由插补器保证);不用相机/点云(环境固定:台架
|
|||
|
|
+ P01 立柱,几何已知)</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>收益目标</td>
|
|||
|
|
<td>冷启动(新目标到第一个动作)< 10 ms;每周期推理 + 安全检查 <
|
|||
|
|
10 ms;拖拽重规划 P99(现状约 321 ms)不再存在</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>底线</td>
|
|||
|
|
<td>实时拖拽<strong>碰撞帧 = 0</strong>(安全层保证);到达率、行程在 §9
|
|||
|
|
的指标上不低于现有规划器</td>
|
|||
|
|
</tr>
|
|||
|
|
</tbody>
|
|||
|
|
</table>
|
|||
|
|
<p>为什么安全层必须保留:调研里最好的神经运动策略在静态场景里仍有约 10%
|
|||
|
|
的轨迹发生碰撞(§1 DRP 附录表 6、ELMP 表 I)。
|
|||
|
|
我们的环境几何完全已知、检查只要几毫秒,保留它代价极小,却把"偶尔撞一下"变成"偶尔停一下"。</p>
|
|||
|
|
<hr />
|
|||
|
|
<h2 id="1-业界调研具身智能里的神经运动规划">1.
|
|||
|
|
业界调研:具身智能里的神经运动规划</h2>
|
|||
|
|
<p>以下数据均来自论文原文(链接见 §13)。</p>
|
|||
|
|
<table>
|
|||
|
|
<thead>
|
|||
|
|
<tr class="header">
|
|||
|
|
<th>工作</th>
|
|||
|
|
<th>年份 / 会议</th>
|
|||
|
|
<th>输入</th>
|
|||
|
|
<th>输出</th>
|
|||
|
|
<th>老师与数据量</th>
|
|||
|
|
<th>关键做法</th>
|
|||
|
|
<th>运行方式</th>
|
|||
|
|
</tr>
|
|||
|
|
</thead>
|
|||
|
|
<tbody>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td><strong>MπNets</strong>(NVIDIA)</td>
|
|||
|
|
<td>2022 CoRL</td>
|
|||
|
|
<td>点云 + 当前关节角 + 目标位姿</td>
|
|||
|
|
<td>下一步关节位置</td>
|
|||
|
|
<td>规划器生成 300 万+ 问题、50 万+ 环境;8×V100 训练一周</td>
|
|||
|
|
<td>端到端行为克隆</td>
|
|||
|
|
<td>闭环 9 Hz,每次执行约 70% 推出的路径,1 kHz PD 跟踪</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td><strong>Neural MP</strong>(CMU)</td>
|
|||
|
|
<td>2024 / IROS 2025</td>
|
|||
|
|
<td>点云 + 关节角 + 目标</td>
|
|||
|
|
<td>GMM(LSTM)</td>
|
|||
|
|
<td>AIT* 等,100 万–300 万条</td>
|
|||
|
|
<td><strong>测试时优化(TTO)</strong>:采样多条,选与场景穿透最少的</td>
|
|||
|
|
<td>开环为主;冷启动约 2970 ms</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td><strong>MπFormer / Avoid Everything</strong>(NVIDIA)</td>
|
|||
|
|
<td>2024 CoRL WS</td>
|
|||
|
|
<td>点云</td>
|
|||
|
|
<td>关节动作</td>
|
|||
|
|
<td>规划器数据</td>
|
|||
|
|
<td>师生微调 + <strong>困难负样本挖掘</strong></td>
|
|||
|
|
<td>闭环</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td><strong>DRP / IMPACT</strong>(CMU)</td>
|
|||
|
|
<td>2025 CoRL</td>
|
|||
|
|
<td>点云 + 当前关节角 + <strong>目标关节角</strong></td>
|
|||
|
|
<td><strong>动作块</strong>:10 步关节增量,MSE 回归</td>
|
|||
|
|
<td>cuRobo 生成 <strong>1000 万条</strong>;4090 训练 3 天</td>
|
|||
|
|
<td>① 大规模预训练 ② <strong>迭代师生微调</strong>(局部控制器 Geometric
|
|||
|
|
Fabrics 当老师,不用全局规划器,因为每步都规划太贵)③ 局部反应层 DCP-RMP
|
|||
|
|
改写目标 ④ 加入"目标被挡→停在最近安全处"的数据</td>
|
|||
|
|
<td>闭环;4.67 M 参数,300 Hz(4090),冷启动 3.48 ms</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td><strong>ELMP</strong>(ETH)</td>
|
|||
|
|
<td>2026 arXiv</td>
|
|||
|
|
<td>点云 + 关节角 + 目标位姿(12 维)</td>
|
|||
|
|
<td>关节增量</td>
|
|||
|
|
<td>AIT* 约 60 万条</td>
|
|||
|
|
<td>行为克隆后做 <strong>APG
|
|||
|
|
微调</strong>:可微运动学展开,碰撞/到达/平滑三项损失直接反传,<strong>不再需要老师数据</strong></td>
|
|||
|
|
<td>MPC 式闭环 30 Hz;冷启动 7.8 ms</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>Diffusion Policy / ACT 一类</td>
|
|||
|
|
<td>2023–</td>
|
|||
|
|
<td>视觉 + 本体</td>
|
|||
|
|
<td>动作块(扩散 / CVAE)</td>
|
|||
|
|
<td>人类示教</td>
|
|||
|
|
<td>动作块 + 滚动执行(receding horizon)+ 时间集成</td>
|
|||
|
|
<td>闭环</td>
|
|||
|
|
</tr>
|
|||
|
|
</tbody>
|
|||
|
|
</table>
|
|||
|
|
<p>几个关键数字:</p>
|
|||
|
|
<ul>
|
|||
|
|
<li>DRP:迭代师生微调使成功率比纯预训练<strong>提高
|
|||
|
|
45%</strong>;纯预训练的 IMPACT "经常有轻微碰撞"。</li>
|
|||
|
|
<li>ELMP:只用 1 万条老师轨迹 + APG(成功率 72.0%、碰撞 19.4%)就超过 60
|
|||
|
|
万条纯行为克隆(57.9%、31.9%),<strong>老师数据少 60 倍</strong>;APG
|
|||
|
|
把环境碰撞率降到原来的 1/3 以下。新环境下,采样 20 万个 APG 问题约 4.5
|
|||
|
|
CPU 小时,而生成同样数量的老师轨迹要 1194 CPU 小时以上。</li>
|
|||
|
|
<li>即便最好的方法,静态场景碰撞率也约 9–10%(ELMP-APG 8.9%,DRP
|
|||
|
|
10.16%)——所以实际部署都配有某种检查或反应层。</li>
|
|||
|
|
</ul>
|
|||
|
|
<p><strong>行业共识</strong>:</p>
|
|||
|
|
<ol type="1">
|
|||
|
|
<li>都是<strong>闭环</strong>策略,每个周期根据当前状态重新出动作,而不是一次性出整条路径。</li>
|
|||
|
|
<li>输出<strong>短动作块</strong>(10
|
|||
|
|
步左右)并滚动执行,兼顾平滑和反应速度。</li>
|
|||
|
|
<li>老师只用于离线造数据;微调阶段倾向于<strong>便宜的老师</strong>(局部控制器)或<strong>不要老师</strong>(可微代价)。</li>
|
|||
|
|
<li>纯模仿有"分布偏移、误差累积"问题,必须做后训练(DAgger / 师生微调 /
|
|||
|
|
APG)。</li>
|
|||
|
|
</ol>
|
|||
|
|
<hr />
|
|||
|
|
<h2 id="2-我们的方案和主流的差异以及据此做的优化">2.
|
|||
|
|
我们的方案和主流的差异,以及据此做的优化</h2>
|
|||
|
|
<h3 id="21-我们的独特条件">2.1 我们的独特条件</h3>
|
|||
|
|
<table>
|
|||
|
|
<thead>
|
|||
|
|
<tr class="header">
|
|||
|
|
<th>条件</th>
|
|||
|
|
<th>主流(DRP/ELMP 等)</th>
|
|||
|
|
<th>本项目</th>
|
|||
|
|
<th>意味着</th>
|
|||
|
|
</tr>
|
|||
|
|
</thead>
|
|||
|
|
<tbody>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>环境</td>
|
|||
|
|
<td>未知、多变,靠点云感知</td>
|
|||
|
|
<td><strong>固定且精确已知</strong>(台架 + 立柱,1936 个胶囊)</td>
|
|||
|
|
<td>不需要点云编码器;模型可以小很多;碰撞检查是<strong>精确</strong>的</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>机器人</td>
|
|||
|
|
<td>Franka 7 轴</td>
|
|||
|
|
<td>自研 6 轴,J1/J3/J6 连续转</td>
|
|||
|
|
<td>需处理连续轴展开</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>碰撞模型</td>
|
|||
|
|
<td>SDF / 球近似,有感知噪声</td>
|
|||
|
|
<td>胶囊解析距离,可微</td>
|
|||
|
|
<td>可以直接做 APG;安全层可信</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>运行平台</td>
|
|||
|
|
<td>台式 GPU</td>
|
|||
|
|
<td>浏览器(后续真机)</td>
|
|||
|
|
<td>模型要小,ONNX + WebGPU/wasm</td>
|
|||
|
|
</tr>
|
|||
|
|
</tbody>
|
|||
|
|
</table>
|
|||
|
|
<p>结论:我们的问题比主流<strong>窄得多也"干净"得多</strong>,主流用来对付感知噪声和环境泛化的大部分复杂度(点云、千万级数据、多环境)都可以省掉,资源应投在<strong>闭环稳定性、多解选择和安全层</strong>上。</p>
|
|||
|
|
<h3 id="22-v1-方案与主流的差异--v2-的调整">2.2 v1 方案与主流的差异 → v2
|
|||
|
|
的调整</h3>
|
|||
|
|
<table>
|
|||
|
|
<thead>
|
|||
|
|
<tr class="header">
|
|||
|
|
<th>#</th>
|
|||
|
|
<th>v1 做法</th>
|
|||
|
|
<th>主流做法</th>
|
|||
|
|
<th>v2 调整</th>
|
|||
|
|
<th>理由</th>
|
|||
|
|
</tr>
|
|||
|
|
</thead>
|
|||
|
|
<tbody>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>1</td>
|
|||
|
|
<td>一次生成整条 32 点路径,开环执行</td>
|
|||
|
|
<td>闭环 + 短动作块滚动执行</td>
|
|||
|
|
<td><strong>改为动作块策略</strong>:每周期输出未来 16 步(320
|
|||
|
|
ms),只执行前 2–4 步</td>
|
|||
|
|
<td>用户要求每周期重算;可纠错、可应对拖拽目标变化</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>2</td>
|
|||
|
|
<td>失败回落原规划器</td>
|
|||
|
|
<td>无回落;靠后训练提升 + 局部反应/检查</td>
|
|||
|
|
<td><strong>删除回落</strong>,改为安全层:换候选 → 刹停保持 →
|
|||
|
|
下周期重算</td>
|
|||
|
|
<td>用户决定无老师部署</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>3</td>
|
|||
|
|
<td>512 桶离散化 + 自回归 186 token</td>
|
|||
|
|
<td>DRP 用 MSE 回归动作块;Neural MP 用 GMM</td>
|
|||
|
|
<td><strong>K 个模式的动作块头(K=4)+
|
|||
|
|
模式分类</strong>,非自回归一次前向</td>
|
|||
|
|
<td>闭环下多峰主要出现在"刚出发选左绕还是右绕",K 模式足够;一次前向比
|
|||
|
|
186 步自回归快一个数量级</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>4</td>
|
|||
|
|
<td>以目标位姿为条件</td>
|
|||
|
|
<td>DRP 用目标关节角(IK 另算);ELMP 用目标位姿</td>
|
|||
|
|
<td><strong>用目标位姿</strong>(位置 + 6D 旋转 +
|
|||
|
|
模式标志),末段可选局部牛顿精修</td>
|
|||
|
|
<td>无老师就没有全量 IK;位姿条件让模型连 IK
|
|||
|
|
一起学。局部牛顿精修只是梯度下降几步,不属于搜索</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>5</td>
|
|||
|
|
<td>后训练:挖错 + 老师重标(每轮 1–2 小时老师计算)</td>
|
|||
|
|
<td>DRP:便宜局部老师;ELMP:APG 无老师</td>
|
|||
|
|
<td><strong>主力换成
|
|||
|
|
APG</strong>(可微胶囊碰撞),挖错循环保留但不再需要老师重标</td>
|
|||
|
|
<td>我们的胶囊模型可微且精确,正好适合 APG;省掉绝大部分老师计算</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>6</td>
|
|||
|
|
<td>拒判头(REJECT)</td>
|
|||
|
|
<td>DRP:目标被挡时学会"停在最近安全处"</td>
|
|||
|
|
<td><strong>改为"停靠"行为</strong>:目标不可达时,数据里老师路径停在离目标最近的安全位形</td>
|
|||
|
|
<td>闭环下"停在最近处"比"拒绝"更有用(目标常在拖拽中短暂进入立柱区)</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>7</td>
|
|||
|
|
<td>20 万条老师数据</td>
|
|||
|
|
<td>主流百万–千万条,但它们要泛化到任意环境</td>
|
|||
|
|
<td><strong>降到 5 万条</strong>(+ 切片增广),其余靠 APG</td>
|
|||
|
|
<td>ELMP 显示 APG 下 1 万条已胜过 60
|
|||
|
|
万条纯克隆;我们的环境固定,需求更小</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>8</td>
|
|||
|
|
<td>无</td>
|
|||
|
|
<td>DRP 的 DCP-RMP 局部反应层</td>
|
|||
|
|
<td>暂不做;安全层已覆盖静态环境。未来真机有人/动态障碍时再加</td>
|
|||
|
|
<td>当前环境静态</td>
|
|||
|
|
</tr>
|
|||
|
|
</tbody>
|
|||
|
|
</table>
|
|||
|
|
<h3 id="23-v2-相对主流的额外优化点">2.3 v2 相对主流的额外优化点</h3>
|
|||
|
|
<ol type="1">
|
|||
|
|
<li><strong>精确安全层</strong>:主流因感知噪声做不到精确检查,我们可以。每周期对即将执行的动作段做
|
|||
|
|
1° 细验 + 半格复验,保证<strong>只执行检查过的动作</strong>。</li>
|
|||
|
|
<li><strong>可刹停不变式</strong>:只有当"执行这一段之后,仍存在一条已检查过的刹停轨迹"时才执行(见
|
|||
|
|
§8.3),保证任何周期推理失败都能安全停住。</li>
|
|||
|
|
<li><strong>模式粘滞</strong>:闭环每周期重算最怕左右摇摆(这一帧选左绕、下一帧选右绕)。部署时对"切换模式"加代价,除非当前模式不安全(§8.2)。</li>
|
|||
|
|
<li><strong>APG
|
|||
|
|
损失直接用我们的胶囊距离</strong>(<code>segSegDist</code>
|
|||
|
|
的可微版本),训练和部署检查同一套几何,消除"训练用球、检查用网格"的口径差(ELMP
|
|||
|
|
的做法有这个差)。</li>
|
|||
|
|
<li><strong>数据便宜</strong>:环境固定,老师离线 5 万条在沙箱约 5
|
|||
|
|
小时即可,不需要集群。</li>
|
|||
|
|
</ol>
|
|||
|
|
<hr />
|
|||
|
|
<h2 id="3-训练环境">3. 训练环境</h2>
|
|||
|
|
<table>
|
|||
|
|
<thead>
|
|||
|
|
<tr class="header">
|
|||
|
|
<th>阶段</th>
|
|||
|
|
<th>在哪里跑</th>
|
|||
|
|
<th>说明</th>
|
|||
|
|
</tr>
|
|||
|
|
</thead>
|
|||
|
|
<tbody>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>预训练数据生成</td>
|
|||
|
|
<td>Hark 沙箱(2 vCPU / 7 GB,Node)或用户机器</td>
|
|||
|
|
<td>纯 JS,无需 GPU</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>预训练 + APG 微调</td>
|
|||
|
|
<td><strong>用户的 GPU 机器</strong></td>
|
|||
|
|
<td>沙箱无 GPU、无 PyTorch。建议 ≥ 8 GB 显存(RTX 3060 级别)</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>部署推理</td>
|
|||
|
|
<td>浏览器(onnxruntime-web,WebGPU 优先,wasm
|
|||
|
|
兜底);后续真机控制器</td>
|
|||
|
|
<td>与现有查看器同页</td>
|
|||
|
|
</tr>
|
|||
|
|
</tbody>
|
|||
|
|
</table>
|
|||
|
|
<div class="sourceCode" id="cb1"><pre
|
|||
|
|
class="sourceCode bash"><code class="sourceCode bash"><span id="cb1-1"><a href="#cb1-1" aria-hidden="true" tabindex="-1"></a><span class="ex">conda</span> create <span class="at">-n</span> armtf python=3.11 <span class="at">-y</span> <span class="kw">&&</span> <span class="ex">conda</span> activate armtf</span>
|
|||
|
|
<span id="cb1-2"><a href="#cb1-2" aria-hidden="true" tabindex="-1"></a><span class="ex">pip</span> install torch <span class="at">--index-url</span> https://download.pytorch.org/whl/cu121 <span class="co"># 按本机 CUDA 版本选</span></span>
|
|||
|
|
<span id="cb1-3"><a href="#cb1-3" aria-hidden="true" tabindex="-1"></a><span class="ex">pip</span> install numpy pyarrow tqdm tensorboard onnx onnxruntime</span></code></pre></div>
|
|||
|
|
<hr />
|
|||
|
|
<h2 id="4-第一步离线造预训练数据老师只在这里出现">4.
|
|||
|
|
第一步:离线造预训练数据(老师只在这里出现)</h2>
|
|||
|
|
<h3 id="41-新增文件">4.1 新增文件</h3>
|
|||
|
|
<table>
|
|||
|
|
<thead>
|
|||
|
|
<tr class="header">
|
|||
|
|
<th>文件</th>
|
|||
|
|
<th>作用</th>
|
|||
|
|
</tr>
|
|||
|
|
</thead>
|
|||
|
|
<tbody>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td><code>ml/gen/gen_dataset.cjs</code></td>
|
|||
|
|
<td>采样任务 → 调老师 → 校验 → 时间参数化 → 写分片</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td><code>ml/gen/sampler.cjs</code></td>
|
|||
|
|
<td>起点/目标采样(4.3)</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td><code>ml/gen/run_parallel.sh</code></td>
|
|||
|
|
<td>多进程,不同种子</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td><code>ml/data/shard_XXXX.jsonl</code></td>
|
|||
|
|
<td>原始样本(不入库)</td>
|
|||
|
|
</tr>
|
|||
|
|
</tbody>
|
|||
|
|
</table>
|
|||
|
|
<p>复用 <code>tests/route_bench_setup.cjs</code>
|
|||
|
|
装配求解器(关节表、<code>docs/travel.json</code>、<code>docs/collide.json</code>)。</p>
|
|||
|
|
<h3 id="42-一条原始样本">4.2 一条原始样本</h3>
|
|||
|
|
<div class="sourceCode" id="cb2"><pre
|
|||
|
|
class="sourceCode json"><code class="sourceCode json"><span id="cb2-1"><a href="#cb2-1" aria-hidden="true" tabindex="-1"></a><span class="fu">{</span></span>
|
|||
|
|
<span id="cb2-2"><a href="#cb2-2" aria-hidden="true" tabindex="-1"></a> <span class="dt">"id"</span><span class="fu">:</span> <span class="st">"s0003_000127"</span><span class="fu">,</span> <span class="dt">"mode"</span><span class="fu">:</span> <span class="st">"pos"</span> <span class="er">|</span> <span class="st">"pose"</span><span class="fu">,</span></span>
|
|||
|
|
<span id="cb2-3"><a href="#cb2-3" aria-hidden="true" tabindex="-1"></a> <span class="dt">"q0"</span><span class="fu">:</span> <span class="ot">[</span><span class="dv">6</span><span class="ot">]</span><span class="fu">,</span> <span class="dt">"T"</span><span class="fu">:</span> <span class="ot">[</span><span class="er">x</span><span class="ot">,</span><span class="er">y</span><span class="ot">,</span><span class="er">z</span><span class="ot">]</span><span class="fu">,</span> <span class="dt">"R"</span><span class="fu">:</span> <span class="ot">[[</span><span class="dv">3</span><span class="er">x3</span><span class="ot">]]</span> <span class="er">|</span> <span class="kw">null</span><span class="fu">,</span> <span class="dt">"tool"</span><span class="fu">:</span> <span class="dv">0</span><span class="fu">,</span></span>
|
|||
|
|
<span id="cb2-4"><a href="#cb2-4" aria-hidden="true" tabindex="-1"></a> <span class="dt">"path"</span><span class="fu">:</span> <span class="ot">[[</span><span class="dv">6</span><span class="ot">],</span> <span class="er">...</span><span class="ot">]</span><span class="fu">,</span> <span class="er">//</span> <span class="er">老师折线</span> <span class="er">q0</span> <span class="er">→</span> <span class="er">via...</span> <span class="er">→</span> <span class="er">q_end</span></span>
|
|||
|
|
<span id="cb2-5"><a href="#cb2-5" aria-hidden="true" tabindex="-1"></a> <span class="dt">"reached"</span><span class="fu">:</span> <span class="kw">true</span><span class="fu">,</span> <span class="er">//</span> <span class="er">false</span> <span class="er">=</span> <span class="er">目标不可达,q_end</span> <span class="er">为离目标最近的安全位形(</span><span class="dt">"停靠"</span><span class="er">)</span></span>
|
|||
|
|
<span id="cb2-6"><a href="#cb2-6" aria-hidden="true" tabindex="-1"></a> <span class="dt">"travel_deg"</span><span class="fu">:</span> <span class="fl">123.4</span><span class="fu">,</span> <span class="dt">"plan_ms"</span><span class="fu">:</span> <span class="fl">812.0</span><span class="fu">,</span></span>
|
|||
|
|
<span id="cb2-7"><a href="#cb2-7" aria-hidden="true" tabindex="-1"></a> <span class="dt">"fp"</span><span class="fu">:</span> <span class="st">"collide.json 指纹"</span></span>
|
|||
|
|
<span id="cb2-8"><a href="#cb2-8" aria-hidden="true" tabindex="-1"></a><span class="fu">}</span></span></code></pre></div>
|
|||
|
|
<h3 id="43-采样分布5-万条任务">4.3 采样分布(5 万条任务)</h3>
|
|||
|
|
<p>关节限位(<code>docs/travel.json</code>):J1/J3/J6 连续 ±180°;J2
|
|||
|
|
[−92, 9],J4 [−111, 16],J5 [−93, 34]。</p>
|
|||
|
|
<table>
|
|||
|
|
<thead>
|
|||
|
|
<tr class="header">
|
|||
|
|
<th>类别</th>
|
|||
|
|
<th style="text-align: right;">占比</th>
|
|||
|
|
<th>采样方式</th>
|
|||
|
|
</tr>
|
|||
|
|
</thead>
|
|||
|
|
<tbody>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>A 均匀</td>
|
|||
|
|
<td style="text-align: right;">45%</td>
|
|||
|
|
<td>q0、目标位形 b 均匀采样且无干涉;目标 = <code>fk(b)</code>(pose
|
|||
|
|
模式加 <code>orient(b)</code>)</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>B 直线被挡</td>
|
|||
|
|
<td style="text-align: right;">30%</td>
|
|||
|
|
<td>q0→b 关节直线 1° 细验有干涉(route_bench 口径)</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>C 贴立柱</td>
|
|||
|
|
<td style="text-align: right;">15%</td>
|
|||
|
|
<td>b 满足 <code>nearColumn</code>,q0 在立柱另一侧(near_column_bench
|
|||
|
|
口径)</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>D 目标不可达</td>
|
|||
|
|
<td style="text-align: right;">10%</td>
|
|||
|
|
<td>目标在立柱内/所有分支都撞/工作空间外;老师标签 =
|
|||
|
|
停在离目标最近的安全位形</td>
|
|||
|
|
</tr>
|
|||
|
|
</tbody>
|
|||
|
|
</table>
|
|||
|
|
<ul>
|
|||
|
|
<li>pos / pose 各半;老师用 <code>solveDyn</code>(松手口径)。</li>
|
|||
|
|
<li><strong>测试集隔离</strong>:固定种子另造 2000
|
|||
|
|
条留出;三个基准案例不进训练集。</li>
|
|||
|
|
</ul>
|
|||
|
|
<h3 id="44-过滤">4.4 过滤</h3>
|
|||
|
|
<ol type="1">
|
|||
|
|
<li>整条路径 1° 细验 + 半格复验,任一段干涉即丢弃。</li>
|
|||
|
|
<li><code>poseRelaxed</code> 样本只当 pos 模式用。</li>
|
|||
|
|
<li>行程 > 同任务 <code>routeRT</code> 1.5
|
|||
|
|
倍的视为绕远(先统计,再定是否丢弃)。</li>
|
|||
|
|
</ol>
|
|||
|
|
<h3 id="45-时间参数化--闭环训练样本v2-新增关键">4.5 时间参数化 →
|
|||
|
|
闭环训练样本(v2 新增,关键)</h3>
|
|||
|
|
<p>闭环策略学的是"在某个状态下,接下来 16
|
|||
|
|
步怎么走",所以要把整条路径切成很多个"状态 → 动作块"样本:</p>
|
|||
|
|
<ol type="1">
|
|||
|
|
<li>按现有 <code>animTo</code> 的轴速比
|
|||
|
|
<code>VREL = [0.6, 0.6, 0.9, 0.9, 1.2, 1.8]</code>
|
|||
|
|
做时间参数化(与查看器动画同口径),控制周期 <strong>dt = 20
|
|||
|
|
ms</strong>。</li>
|
|||
|
|
<li>得到状态序列 q_0, q_1, …, q_N。对每个 t(每隔 3 步取 1
|
|||
|
|
个,减少冗余)生成样本:
|
|||
|
|
<ul>
|
|||
|
|
<li>输入:当前 q_t、目标 (T, R, mode)</li>
|
|||
|
|
<li>标签:未来 S = 16 步的增量 Δq_{t+1..t+16} = q_{t+k} −
|
|||
|
|
q_t(到终点后补零,即"停住")</li>
|
|||
|
|
</ul></li>
|
|||
|
|
<li>每条路径平均约 50–150 个样本 → 5 万条任务约 300 万个训练样本。</li>
|
|||
|
|
<li><strong>路径后缀天然覆盖"从半路出发"</strong>,等于免费的 DAgger
|
|||
|
|
种子数据。</li>
|
|||
|
|
</ol>
|
|||
|
|
<h3 id="46-耗时与存储">4.6 耗时与存储</h3>
|
|||
|
|
<ul>
|
|||
|
|
<li><code>solveDyn</code> 约 0.7 s/条:5 万条在沙箱 2 进程约 5
|
|||
|
|
小时;用户 8 核机器约 1.5 小时。</li>
|
|||
|
|
<li>300 万样本 × 约 0.5 KB ≈ 1.5 GB 原始;Parquet 约 300 MB。</li>
|
|||
|
|
</ul>
|
|||
|
|
<h3 id="47-检查点">4.7 检查点</h3>
|
|||
|
|
<ul class="task-list">
|
|||
|
|
<li><label><input
|
|||
|
|
type="checkbox" /><code>ml/data/stats.json</code>:类别、到达率、行程、样本数分布</label></li>
|
|||
|
|
<li><label><input type="checkbox" />抽 20 条在查看器回放(新增
|
|||
|
|
<code>?replay=</code> URL 参数)</label></li>
|
|||
|
|
<li><label><input
|
|||
|
|
type="checkbox" />测试集与训练集、基准案例无重叠</label></li>
|
|||
|
|
</ul>
|
|||
|
|
<hr />
|
|||
|
|
<h2 id="5-第二步表示与预处理">5. 第二步:表示与预处理</h2>
|
|||
|
|
<p>文件 <code>ml/prep/build_dataset.py</code> →
|
|||
|
|
<code>train/val/test.parquet</code>、<code>norm.json</code>。</p>
|
|||
|
|
<h3 id="51-输入条件">5.1 输入(条件)</h3>
|
|||
|
|
<table>
|
|||
|
|
<thead>
|
|||
|
|
<tr class="header">
|
|||
|
|
<th>量</th>
|
|||
|
|
<th style="text-align: right;">维度</th>
|
|||
|
|
<th>编码</th>
|
|||
|
|
</tr>
|
|||
|
|
</thead>
|
|||
|
|
<tbody>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>当前 q_t</td>
|
|||
|
|
<td style="text-align: right;">6</td>
|
|||
|
|
<td>每轴 (sin, cos) → 12;限位轴另加归一化值 → +3</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>当前 TCP 位置与姿态</td>
|
|||
|
|
<td style="text-align: right;">3 + 6</td>
|
|||
|
|
<td><code>fk(q_t)</code>、<code>orient(q_t)</code> 的 6D
|
|||
|
|
表示(给模型笛卡尔线索)</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>目标位置 T</td>
|
|||
|
|
<td style="text-align: right;">3</td>
|
|||
|
|
<td>/600 mm</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>目标姿态 R</td>
|
|||
|
|
<td style="text-align: right;">6</td>
|
|||
|
|
<td>6D 表示;pos 模式填 0</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>目标 − 当前 TCP</td>
|
|||
|
|
<td style="text-align: right;">3</td>
|
|||
|
|
<td>位置差,/600 mm</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>模式</td>
|
|||
|
|
<td style="text-align: right;">2</td>
|
|||
|
|
<td>one-hot [pos, pose]</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>上一周期选的模式 k_prev</td>
|
|||
|
|
<td style="text-align: right;">K+1</td>
|
|||
|
|
<td>one-hot(含"无"),用于模式粘滞</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>关键胶囊对最小距离</td>
|
|||
|
|
<td style="text-align: right;">8</td>
|
|||
|
|
<td>当前位形下臂各段到立柱的胶囊距离(cheap,现成函数),给模型"离柱子多近"的直接信号</td>
|
|||
|
|
</tr>
|
|||
|
|
</tbody>
|
|||
|
|
</table>
|
|||
|
|
<h3 id="52-输出动作块">5.2 输出(动作块)</h3>
|
|||
|
|
<ul>
|
|||
|
|
<li><strong>K = 4 个模式</strong>,每个模式一整块 16 步 × 6 轴的关节增量
|
|||
|
|
Δq(连续值,按每轴最大步长归一化)。</li>
|
|||
|
|
<li>一个模式分类头:4 类 logits。</li>
|
|||
|
|
<li>训练时
|
|||
|
|
<strong>winner-takes-all</strong>:只对与标签最接近的那个模式回归,分类头学"选哪个"(MTR
|
|||
|
|
/
|
|||
|
|
多模态轨迹预测的常用做法)。这样左绕、右绕会落到不同模式里,不会被平均到穿过立柱。</li>
|
|||
|
|
<li>附加头:到达剩余时间估计(帮助排序和早停)。</li>
|
|||
|
|
</ul>
|
|||
|
|
<h3 id="53-连续轴">5.3 连续轴</h3>
|
|||
|
|
<p>J1/J3/J6 统一用 <code>unwrapTo</code>
|
|||
|
|
口径展开后再做增量;预处理和部署共用一份实现,§6.4 对拍时一并验证。</p>
|
|||
|
|
<hr />
|
|||
|
|
<h2 id="6-第三步模型与预训练行为克隆">6.
|
|||
|
|
第三步:模型与预训练(行为克隆)</h2>
|
|||
|
|
<h3 id="61-结构参考-impact去掉点云部分">6.1 结构(参考
|
|||
|
|
IMPACT,去掉点云部分)</h3>
|
|||
|
|
<pre><code>条件特征(约 60 维)──MLP──▶ 8 个条件 token (d=192)
|
|||
|
|
│
|
|||
|
|
4 层 Transformer encoder
|
|||
|
|
│ (memory)
|
|||
|
|
K×16 个可学习动作 token ──▶ 4 层 Transformer decoder(非自回归,一次前向)
|
|||
|
|
│
|
|||
|
|
├─ 动作头:每 token 输出 6 维 Δq
|
|||
|
|
├─ 模式分类头:K=4
|
|||
|
|
└─ 剩余时间头:1</code></pre>
|
|||
|
|
<table>
|
|||
|
|
<thead>
|
|||
|
|
<tr class="header">
|
|||
|
|
<th>超参</th>
|
|||
|
|
<th>值</th>
|
|||
|
|
</tr>
|
|||
|
|
</thead>
|
|||
|
|
<tbody>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>d_model / 头数 / FFN</td>
|
|||
|
|
<td>192 / 8 / 768</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>encoder / decoder 层数</td>
|
|||
|
|
<td>4 / 4</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>参数量</td>
|
|||
|
|
<td>约 3–4 M(DRP 为 4.67 M,含点云编码)</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>推理</td>
|
|||
|
|
<td>一次前向得到 4 条候选动作块</td>
|
|||
|
|
</tr>
|
|||
|
|
</tbody>
|
|||
|
|
</table>
|
|||
|
|
<h3 id="62-预训练损失">6.2 预训练损失</h3>
|
|||
|
|
<pre><code>L = WTA-L1(动作块, 最近模式) 权重 1.0
|
|||
|
|
+ CE(模式分类) 权重 0.5
|
|||
|
|
+ FK 损失:预测与标签位形下各连杆胶囊端点的位置差 权重 0.5 ← 借鉴 ELMP
|
|||
|
|
+ 平滑:二阶差分 权重 0.05
|
|||
|
|
+ L1(剩余时间) 权重 0.1</code></pre>
|
|||
|
|
<h3 id="63-训练配置">6.3 训练配置</h3>
|
|||
|
|
<table>
|
|||
|
|
<thead>
|
|||
|
|
<tr class="header">
|
|||
|
|
<th>项</th>
|
|||
|
|
<th>值</th>
|
|||
|
|
</tr>
|
|||
|
|
</thead>
|
|||
|
|
<tbody>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>优化器</td>
|
|||
|
|
<td>AdamW,β=(0.9, 0.95),wd 0.01</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>学习率</td>
|
|||
|
|
<td>1e-3,预热 1000 步,余弦退火(同 DRP 量级)</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>batch</td>
|
|||
|
|
<td>512</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>精度</td>
|
|||
|
|
<td>bf16 混合精度</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>步数</td>
|
|||
|
|
<td>约 20 万步(300 万样本约 30
|
|||
|
|
epoch),验证集闭环成功率不再上升即停</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>预计耗时</td>
|
|||
|
|
<td>RTX 3060 约 3–5 小时(估算,跑完首轮校正)</td>
|
|||
|
|
</tr>
|
|||
|
|
</tbody>
|
|||
|
|
</table>
|
|||
|
|
<h3 id="64-对拍与验证每-epoch">6.4 对拍与验证(每 epoch)</h3>
|
|||
|
|
<ul>
|
|||
|
|
<li><code>ml/eval/collide_torch.py</code>:胶囊距离的 PyTorch
|
|||
|
|
批量版(<strong>可微</strong>,§7 要用),先与 JS
|
|||
|
|
<code>collideFree</code> 对拍 10 万个位形,判定 0 误差。</li>
|
|||
|
|
<li>验证用<strong>闭环仿真</strong>而不是单步误差:从 1000 个验证任务的
|
|||
|
|
q0 出发,每 20 ms 推理一次、执行前 3 步、直到到达或 6 s
|
|||
|
|
超时,统计到达率、碰撞步数(安全层关闭时)、行程比。</li>
|
|||
|
|
</ul>
|
|||
|
|
<hr />
|
|||
|
|
<h2 id="7-第四步后训练无老师为主">7. 第四步:后训练(无老师为主)</h2>
|
|||
|
|
<h3 id="71-主力apg-可微微调借鉴-elmp">7.1 主力:APG 可微微调(借鉴
|
|||
|
|
ELMP)</h3>
|
|||
|
|
<p>不再请老师标注,直接让模型在可微仿真里闭环展开,用代价反传:</p>
|
|||
|
|
<pre><code>for 每个 batch(随机采样 q0 和目标,82 ms 级别的便宜采样,不需要老师):
|
|||
|
|
q = q0
|
|||
|
|
for t in 0..H-1 (H = 60 个周期 ≈ 1.2 s,长任务分段展开):
|
|||
|
|
chunk, logits = π(obs(q, 目标))
|
|||
|
|
Δq = chunk[选中模式][0:3] # 执行前 3 步,与部署一致
|
|||
|
|
q = clamp(q + Δq) # 限位
|
|||
|
|
累计 L_col(q), L_smooth(Δq)
|
|||
|
|
L = w_tgt·L_target(q_H) + w_col·ΣL_col + w_smooth·ΣL_smooth (+ w_len·行程)
|
|||
|
|
反传(BPTT),梯度裁剪 ≤ 1.0</code></pre>
|
|||
|
|
<ul>
|
|||
|
|
<li><strong>碰撞项</strong> L_col = Σ_胶囊对 ReLU(margin −
|
|||
|
|
胶囊距离),margin 取 <code>collide.json</code> 的 <code>margin_mm</code>
|
|||
|
|
基础上再加 5 mm 训练余量;包含臂×立柱、臂×台架、臂×臂(非相邻段)。</li>
|
|||
|
|
<li><strong>到达项</strong>:位置误差² + λ_rot·‖R − R*‖_F²(弦距离,ELMP
|
|||
|
|
同款,避免四元数梯度消失)。</li>
|
|||
|
|
<li><strong>平滑项</strong>:速度² + 加速度²。</li>
|
|||
|
|
<li>可加 <strong>行程项</strong>,直接把"关节行程短"写进目标(ELMP
|
|||
|
|
用笛卡尔路径长度项做过同类事情)。</li>
|
|||
|
|
<li>模式选择不可导:用 Gumbel-Softmax,或对 K 个模式都展开 1
|
|||
|
|
步再取最小代价(K=4,代价可接受)。</li>
|
|||
|
|
<li>采样任务分布同 §4.3,但 B/C/D 类提到 60%。</li>
|
|||
|
|
<li>预期:参照 ELMP,APG
|
|||
|
|
主要降低碰撞率、提高末端精度;<strong>不需要任何老师计算</strong>。</li>
|
|||
|
|
</ul>
|
|||
|
|
<h3 id="72-辅助闭环挖错--门禁v11-循环的无老师版">7.2 辅助:闭环挖错 +
|
|||
|
|
门禁(v1.1 循环的无老师版)</h3>
|
|||
|
|
<p>每轮:</p>
|
|||
|
|
<ol type="1">
|
|||
|
|
<li><strong>出题</strong>:新种子 5 万任务 + 2000
|
|||
|
|
段模型在环拖拽仿真(目标沿随机曲线移动)。</li>
|
|||
|
|
<li><strong>闭环作答</strong>:完全按部署方式运行(含模式粘滞,<strong>关闭安全层</strong>以暴露真实错误)。</li>
|
|||
|
|
<li><strong>验算</strong>:GPU
|
|||
|
|
批量胶囊检查;记录到达、碰撞、行程、摇摆次数。</li>
|
|||
|
|
<li><strong>错题分类</strong>:E1 撞了 / E2 没到(超时或卡在局部极小)/
|
|||
|
|
E3 绕远(> 1.2× 参考)/ E4 末端精度差 / E5 模式来回切换 ≥ 3 次 / E6
|
|||
|
|
拖拽失稳。</li>
|
|||
|
|
<li><strong>针对性训练</strong>:错题的起点和目标(以及 ±5 mm / ±3° 的
|
|||
|
|
10 个邻居)加入 APG 的采样池,按类别加权(E1、E6 ×2)。
|
|||
|
|
<ul>
|
|||
|
|
<li>只有
|
|||
|
|
E2(卡在局部极小——可微代价本身解决不了的"需要绕远路"问题)才可选地请老师离线给一条路径做行为克隆补充。这是唯一残留的老师用途,量很小,可以在沙箱慢慢跑。</li>
|
|||
|
|
</ul></li>
|
|||
|
|
<li><strong>门禁</strong>:留出集闭环到达率上升、三个基准不退化、上一轮修好的错题
|
|||
|
|
95% 仍对;否则回滚。</li>
|
|||
|
|
<li><strong>停止</strong>:连续 2 轮提升 < 0.5 个百分点,或满 6
|
|||
|
|
轮,或达到 §9 全部标准。</li>
|
|||
|
|
</ol>
|
|||
|
|
<h3 id="73-可选拒绝采样自蒸馏">7.3 可选:拒绝采样自蒸馏</h3>
|
|||
|
|
<p>对每个任务采样多条闭环轨迹(不同模式、不同温度),只保留验算通过且最短的一条做行为克隆标签——模型自己当老师。适合
|
|||
|
|
APG 收敛后进一步缩短行程。</p>
|
|||
|
|
<hr />
|
|||
|
|
<h2 id="8-第五步无老师闭环部署">8. 第五步:无老师闭环部署</h2>
|
|||
|
|
<h3 id="81-每个控制周期做什么默认-50-hz">8.1 每个控制周期做什么(默认 50
|
|||
|
|
Hz)</h3>
|
|||
|
|
<pre><code>每 20 ms:
|
|||
|
|
1. 读当前 q(真机读编码器;查看器读当前插补位形)和最新目标(拖拽时每周期都在变)
|
|||
|
|
2. 推理一次:得到 K=4 条 16 步动作块 + 模式概率 ≈ 2–5 ms(估算)
|
|||
|
|
3. 按"模式概率 − 切换代价"排序(§8.2)
|
|||
|
|
4. 安全层依次检查候选的前 n=3 步(60 ms 的运动)+ 刹停段(§8.3)
|
|||
|
|
第一个通过的 → 送插补器执行这 3 步;记下 k_prev
|
|||
|
|
5. 全部不通过 → 进入刹停:按已验证的刹停轨迹减速停下,保持;下一周期继续推理
|
|||
|
|
6. 到达判据:TCP 误差 < 1 mm(pose 另需 < 0.5°)且速度≈0 → 结束;
|
|||
|
|
接近目标(< 5 mm)时可选做 1–3 次局部牛顿精修(solveTo 单步,不是全量 IK)</code></pre>
|
|||
|
|
<p>插补器在两个周期之间做平滑过渡(下一块与正在执行的块做时间集成/交叉淡入,ACT
|
|||
|
|
的 temporal ensembling 思路),避免每 20 ms 一顿。</p>
|
|||
|
|
<h3 id="82-模式粘滞防左右摇摆">8.2 模式粘滞(防左右摇摆)</h3>
|
|||
|
|
<ul>
|
|||
|
|
<li>排序分数 = log p(k) − c·[k ≠ k_prev],c 初值 2.0(在 §7.2 的 E5
|
|||
|
|
统计上调)。</li>
|
|||
|
|
<li>训练时 k_prev
|
|||
|
|
作为输入(§5.1),模型自己也学会"保持当前绕行方向"。</li>
|
|||
|
|
</ul>
|
|||
|
|
<h3 id="83-安全层唯一的硬保证">8.3 安全层(唯一的硬保证)</h3>
|
|||
|
|
<ul>
|
|||
|
|
<li>检查内容:候选前 3 步的连续运动做 1° 细验 + 半格复验(与现有
|
|||
|
|
<code>pathFine</code> 同口径),外加关节限位。</li>
|
|||
|
|
<li><strong>可刹停不变式</strong>:执行前还要检查"从第 3
|
|||
|
|
步末状态按最大减速度停下"的刹停段也无干涉。于是任何时候系统都处在"有一条已验证的停车路线"的状态,下一周期就算模型完全失效,也只会停下、不会撞。</li>
|
|||
|
|
<li>耗时:3 步 × 4 候选,最坏约 4 × 10 个采样位形 × 约 1.8 ms ≈ 70
|
|||
|
|
ms(太慢)→ 实际做法:
|
|||
|
|
<ul>
|
|||
|
|
<li>先用现有包围球预筛(大多数位形直接判安全,见 2026-10-15 记录
|
|||
|
|
collideFree 2.9→1.8 ms);</li>
|
|||
|
|
<li>按排序<strong>逐个</strong>检查,第一个通过就停(多数周期只检查 1
|
|||
|
|
条);</li>
|
|||
|
|
<li>目标:每周期安全检查 < 5 ms(需在 M6 实测;不达标则把检查放到 Web
|
|||
|
|
Worker 与推理流水线并行)。</li>
|
|||
|
|
</ul></li>
|
|||
|
|
<li>现有 <code>ik-glue.js</code>
|
|||
|
|
的"每帧碰撞守卫"保留,作为第二道保险。</li>
|
|||
|
|
</ul>
|
|||
|
|
<h3 id="84-部署包里有什么没有什么">8.4 部署包里有什么、没有什么</h3>
|
|||
|
|
<table>
|
|||
|
|
<thead>
|
|||
|
|
<tr class="header">
|
|||
|
|
<th>有</th>
|
|||
|
|
<th>没有</th>
|
|||
|
|
</tr>
|
|||
|
|
</thead>
|
|||
|
|
<tbody>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td><code>traj_v2.onnx</code>(fp16,约 7 MB)+
|
|||
|
|
<code>norm.json</code></td>
|
|||
|
|
<td>PRM 路网(<code>collide.json.roadmap</code>)、IK 候选大表</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>胶囊碰撞检查(现有
|
|||
|
|
<code>collideFree</code>、包围球预筛、<code>pair2d</code> 表)</td>
|
|||
|
|
<td><code>solveDyn</code> / <code>routeRT</code> / RRT / A*</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>正运动学 <code>fk</code> / <code>orient</code>、局部牛顿精修
|
|||
|
|
<code>solveTo</code> 单步</td>
|
|||
|
|
<td>全量 IK 枚举</td>
|
|||
|
|
</tr>
|
|||
|
|
</tbody>
|
|||
|
|
</table>
|
|||
|
|
<p><code>collide.json</code>
|
|||
|
|
指纹写进模型;不一致时前端<strong>拒绝运行并提示重训</strong>(不再回落旧规划器)。</p>
|
|||
|
|
<hr />
|
|||
|
|
<h2 id="9-第六步评估与验收">9. 第六步:评估与验收</h2>
|
|||
|
|
<h3 id="91-测试集合">9.1 测试集合</h3>
|
|||
|
|
<table>
|
|||
|
|
<thead>
|
|||
|
|
<tr class="header">
|
|||
|
|
<th>集合</th>
|
|||
|
|
<th>来源</th>
|
|||
|
|
</tr>
|
|||
|
|
</thead>
|
|||
|
|
<tbody>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>route_bench</td>
|
|||
|
|
<td><code>tests/route_bench.cjs</code>,60 例</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>near_column_bench</td>
|
|||
|
|
<td><code>tests/near_column_bench.cjs</code>,30 例(pos / pose /
|
|||
|
|
rt)</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>rt_drag_bench</td>
|
|||
|
|
<td><code>tests/rt_drag_bench.cjs</code>,12
|
|||
|
|
例(改为模型在环逐周期运行)</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>held-out</td>
|
|||
|
|
<td>§4.3 留出 2000 条</td>
|
|||
|
|
</tr>
|
|||
|
|
</tbody>
|
|||
|
|
</table>
|
|||
|
|
<p>新增 <code>tests/policy_bench.cjs</code>:①现有规划器
|
|||
|
|
②策略(关安全层)③策略 + 安全层(部署形态),输出对比表。</p>
|
|||
|
|
<h3 id="92-验收标准部署形态--策略--安全层">9.2 验收标准(部署形态 = 策略
|
|||
|
|
+ 安全层)</h3>
|
|||
|
|
<table>
|
|||
|
|
<thead>
|
|||
|
|
<tr class="header">
|
|||
|
|
<th>指标</th>
|
|||
|
|
<th>要求</th>
|
|||
|
|
<th>现状参考</th>
|
|||
|
|
</tr>
|
|||
|
|
</thead>
|
|||
|
|
<tbody>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>实际执行的碰撞帧</td>
|
|||
|
|
<td><strong>= 0</strong></td>
|
|||
|
|
<td>rt_drag_bench 0</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>到达率(6 s 内)</td>
|
|||
|
|
<td>route ≥ 60/60,near pos ≥ 29/30,near rt ≥ 29/30,held-out ≥
|
|||
|
|
97%</td>
|
|||
|
|
<td>60/60、30/30、30/30</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>关节行程</td>
|
|||
|
|
<td>≤ 现有规划器 × 1.10</td>
|
|||
|
|
<td>route 平均 467°</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>冷启动(新目标→第一个动作)</td>
|
|||
|
|
<td>< 10 ms</td>
|
|||
|
|
<td>首次全量 IK 最差约 2 s</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>每周期耗时(推理+安全层)P99</td>
|
|||
|
|
<td>< 15 ms</td>
|
|||
|
|
<td>拖拽重规划 P99 约 321 ms</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>刹停发生率</td>
|
|||
|
|
<td>< 2% 的周期</td>
|
|||
|
|
<td>—</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>模式摇摆(≥3 次切换)</td>
|
|||
|
|
<td>< 1% 的任务</td>
|
|||
|
|
<td>—</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>策略单独(关安全层)碰撞率</td>
|
|||
|
|
<td>< 3%(参考:ELMP-APG 8.9%,DRP 10.16%)</td>
|
|||
|
|
<td>—</td>
|
|||
|
|
</tr>
|
|||
|
|
</tbody>
|
|||
|
|
</table>
|
|||
|
|
<p>说明:允许 near_column 少 1
|
|||
|
|
例,因为无老师时最难的贴柱案例可能要靠多次刹停-重算才能到达,6 s
|
|||
|
|
超时内可能失败;这一条请审查时确认能否接受。</p>
|
|||
|
|
<hr />
|
|||
|
|
<h2 id="10-风险与维护">10. 风险与维护</h2>
|
|||
|
|
<ol type="1">
|
|||
|
|
<li><strong>局部极小</strong>:可微代价和闭环策略都可能卡在立柱前"想直穿过去又被安全层挡住"。对策:B/C
|
|||
|
|
类数据加权、模式头学左右绕、E2
|
|||
|
|
错题少量老师补数据(§7.2)。这是无老师方案最大的风险。</li>
|
|||
|
|
<li><strong>碰撞模型/立柱变动 =
|
|||
|
|
重训</strong>:指纹不一致即拒绝运行。重训流程:§4(约 5 h)→ §6(约 4
|
|||
|
|
h)→ §7(约 2–4 h)。</li>
|
|||
|
|
<li><strong>安全层耗时</strong>:最坏周期可能超 5 ms,见 §8.3
|
|||
|
|
的流水线方案。</li>
|
|||
|
|
<li><strong>老师缺陷</strong>:<code>ci27</code>
|
|||
|
|
姿态根枚举漏分支(姿态偏差约 97.5°)会进入预训练数据;APG
|
|||
|
|
的姿态到达项可以部分纠正,必要时先修老师。</li>
|
|||
|
|
<li><strong>真机迁移</strong>:浏览器的"执行"是理想插补;真机需加速度/力矩限制(TOPP
|
|||
|
|
类时间参数化)和跟踪误差余量(把 margin 再加大)。</li>
|
|||
|
|
</ol>
|
|||
|
|
<hr />
|
|||
|
|
<h2 id="11-目录规划">11. 目录规划</h2>
|
|||
|
|
<pre><code>ml/
|
|||
|
|
gen/ gen_dataset.cjs sampler.cjs run_parallel.sh
|
|||
|
|
data/ (不入库)shard_*.jsonl stats.json *.parquet norm.json
|
|||
|
|
prep/ build_dataset.py
|
|||
|
|
model/ policy_transformer.py
|
|||
|
|
sim/ closed_loop.py # 闭环仿真(训练验证、挖错、APG 展开共用)
|
|||
|
|
eval/ collide_torch.py # 可微胶囊距离,与 JS 对拍
|
|||
|
|
apg/ apg_finetune.py
|
|||
|
|
mining/ mine_round.py gate.py loop.sh
|
|||
|
|
configs/ base.yaml apg.yaml
|
|||
|
|
train.py export_onnx.py README.md
|
|||
|
|
webapp/js/ik-policy.js # 闭环推理 + 模式粘滞 + 安全层 + 刹停
|
|||
|
|
webapp/models/traj_v2.onnx traj_v2.norm.json
|
|||
|
|
tests/policy_bench.cjs</code></pre>
|
|||
|
|
<h2 id="12-里程碑">12. 里程碑</h2>
|
|||
|
|
<table>
|
|||
|
|
<thead>
|
|||
|
|
<tr class="header">
|
|||
|
|
<th>#</th>
|
|||
|
|
<th>内容</th>
|
|||
|
|
<th>产出</th>
|
|||
|
|
<th>在哪跑</th>
|
|||
|
|
</tr>
|
|||
|
|
</thead>
|
|||
|
|
<tbody>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>M1</td>
|
|||
|
|
<td>数据生成器 + 时间参数化 + 1000 条试跑</td>
|
|||
|
|
<td><code>ml/gen/*</code>、stats.json</td>
|
|||
|
|
<td>沙箱</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>M2</td>
|
|||
|
|
<td>可微胶囊距离(PyTorch)与 JS 对拍</td>
|
|||
|
|
<td><code>collide_torch.py</code>,10 万位形 0 误差</td>
|
|||
|
|
<td>沙箱(CPU 即可)</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>M3</td>
|
|||
|
|
<td>5 万条老师数据 → 约 300 万闭环样本</td>
|
|||
|
|
<td>Parquet</td>
|
|||
|
|
<td>沙箱约 5 h 或用户机器</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>M4</td>
|
|||
|
|
<td>模型 + 训练工程 + 闭环仿真;小数据过拟合测试</td>
|
|||
|
|
<td><code>train.py</code>、<code>closed_loop.py</code></td>
|
|||
|
|
<td>沙箱 CPU 验证可跑</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>M5</td>
|
|||
|
|
<td>预训练(行为克隆)</td>
|
|||
|
|
<td>checkpoint</td>
|
|||
|
|
<td><strong>用户 GPU</strong></td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>M6</td>
|
|||
|
|
<td>APG 微调 + 挖错循环 3–6 轮</td>
|
|||
|
|
<td>每轮 report.md</td>
|
|||
|
|
<td><strong>用户 GPU</strong></td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="odd">
|
|||
|
|
<td>M7</td>
|
|||
|
|
<td>导出 ONNX + <code>ik-policy.js</code> 闭环部署 + 安全层实测 +
|
|||
|
|
<code>policy_bench</code></td>
|
|||
|
|
<td>对比表</td>
|
|||
|
|
<td>沙箱 + 浏览器</td>
|
|||
|
|
</tr>
|
|||
|
|
<tr class="even">
|
|||
|
|
<td>M8</td>
|
|||
|
|
<td>移除查看器里对
|
|||
|
|
<code>solveDyn</code>/<code>routeRT</code>/路网的运行时依赖(保留在
|
|||
|
|
<code>ml/gen</code> 离线使用)</td>
|
|||
|
|
<td>部署包瘦身</td>
|
|||
|
|
<td>沙箱</td>
|
|||
|
|
</tr>
|
|||
|
|
</tbody>
|
|||
|
|
</table>
|
|||
|
|
<h2 id="13-参考文献">13. 参考文献</h2>
|
|||
|
|
<ul>
|
|||
|
|
<li>Fishman et al., <em>Motion Policy Networks</em>, CoRL 2022. <a
|
|||
|
|
href="https://arxiv.org/abs/2210.12209">https://arxiv.org/abs/2210.12209</a>
|
|||
|
|
;闭环细节见补充材料 <a
|
|||
|
|
href="https://proceedings.mlr.press/v205/fishman23a/fishman23a-supp.pdf">https://proceedings.mlr.press/v205/fishman23a/fishman23a-supp.pdf</a></li>
|
|||
|
|
<li>Dalal et al., <em>Neural MP: A Generalist Neural Motion
|
|||
|
|
Planner</em>, 2024 / IROS 2025. <a
|
|||
|
|
href="https://arxiv.org/abs/2409.05864">https://arxiv.org/abs/2409.05864</a></li>
|
|||
|
|
<li>Yang, Liu et al., <em>Deep Reactive Policy: Learning Reactive
|
|||
|
|
Manipulator Motion Planning for Dynamic Environments</em>, CoRL 2025. <a
|
|||
|
|
href="https://arxiv.org/abs/2509.06953">https://arxiv.org/abs/2509.06953</a></li>
|
|||
|
|
<li>Fishman et al., <em>Avoid Everything: Model-Free Collision Avoidance
|
|||
|
|
with Expert-Guided Fine-Tuning</em>, CoRL 2024 Workshop(DRP 文中引用为
|
|||
|
|
MπFormer)</li>
|
|||
|
|
<li>Li et al., <em>ELMP: Efficient Learning for Motion Planning via
|
|||
|
|
Analytical Policy Gradients</em>, arXiv 2607.00215, 2026. <a
|
|||
|
|
href="https://arxiv.org/abs/2607.00215">https://arxiv.org/abs/2607.00215</a></li>
|
|||
|
|
<li>Ross, Gordon, Bagnell, <em>A Reduction of Imitation Learning and
|
|||
|
|
Structured Prediction to No-Regret Online
|
|||
|
|
Learning</em>(DAgger),AISTATS 2011</li>
|
|||
|
|
</ul>
|
|||
|
|
<h2 id="14-需要用户确认">14. 需要用户确认</h2>
|
|||
|
|
<ol type="1">
|
|||
|
|
<li>训练机显卡型号与显存。</li>
|
|||
|
|
<li>§9.2 中 near_column 允许
|
|||
|
|
29/30(无老师下最难案例可能超时)是否接受。</li>
|
|||
|
|
<li>控制周期 50 Hz、每周期执行 3 步(60
|
|||
|
|
ms)作为默认值是否合适(真机控制器频率如有要求请告知)。</li>
|
|||
|
|
</ol>
|
|||
|
|
</body>
|
|||
|
|
</html>
|