[{"data":1,"prerenderedAt":44577},["ShallowReactive",2],{"blog-post-/blogs/agent":3,"all-blogs-for-navigation":119},{"id":4,"title":5,"body":6,"description":104,"extension":105,"meta":106,"navigation":114,"ogImage":108,"path":115,"seo":116,"stem":117,"__hash__":118},"content/blogs/4. Agent 相关的折腾记录（持续更新）.md","Agent 相关的折腾记录（持续更新）",{"type":7,"value":8,"toc":88},"minimark",[9,13,17,20,23,28,31,35,38,42,45,49,52,56,59,63,66,69,72,76,79,82,85],[10,11,12],"h2",{"id":12},"背景",[14,15,16],"p",{},"最近开始静下心来研究 Agent 生态了，之前一直没有深入研究主要是因为个人感觉生态不够成熟，技术日新月异，发展太快，还没到一个成熟稳定的阶段。最近感觉 Agent 相关概念开始趋于稳定完备，于是开始折腾。",[10,18,19],{"id":19},"介绍一下我有哪些工具",[14,21,22],{},"截至目前，本人共尝试过 Claude Code, Codex, Cursor, 最近又刚开始体验 SOLO Trae（支持国产），先说一下体验。",[24,25,27],"h3",{"id":26},"claude-code","Claude Code",[14,29,30],{},"夯，编程体验最好的一个，插件生态没得说，感觉聪明程度是目前其他 Coding Agent 没得比的，而且能搭配 Curosr/Vscode 之类的编辑器使用，在终端中打开 Claude Code 时可以选中 Cursor/Vscode 视图中某一部分的代码，同步给 Claude Code 作为上下文，选择是否应用修改时也能在 Cursor/Vscode 中进行检查，使用体验极好。",[24,32,34],{"id":33},"codex","Codex",[14,36,37],{},"量大管饱，智力也相当可以，开 xhigh 写代码一点不含糊，感觉生态略差一些。",[24,39,41],{"id":40},"cursor","Cursor",[14,43,44],{},"还可以，使用 GPT 系列经常降智， claude sonnet 和 opus 又容易过度思考，简单任务 auto 又快又省钱，Tab 补全确实惊艳，对于手写古人来说值得体验。",[24,46,48],{"id":47},"solo-trae","SOLO Trae",[14,50,51],{},"现在模型免费，可以白嫖干点简单活，添加模型时自定义能力差，使用 skill 还要指定，用还要排大队，感觉有待提高。",[10,53,55],{"id":54},"skills","Skills",[14,57,58],{},"最近发现 Skills 确实能极大地提高模型的能力，打算开始挖掘 Skills 的潜力。",[10,60,62],{"id":61},"mcp","MCP",[14,64,65],{},"体验不多，目前高度使用 Notion 的 MCP，让 AI 直接把文字写入 Notion 中，非常方便。",[10,67,68],{"id":68},"杂谈",[14,70,71],{},"目前还使用了 CC-Switch 来管理多个 api 渠道，Claude Code 主力使用公益神站 AnyRouter，这个网站唯一的缺点就是太不稳定，最近开始又恢复正常使用了，准备狠狠地用起来。",[10,73,75],{"id":74},"zcf","ZCF",[14,77,78],{},"ZCF 全称 Zero-Config Code Flow。",[14,80,81],{},"ZCF（Zero-Config Code Flow）是一个面向专业开发者的 CLI 工具，目标是在几分钟内完成 Claude Code 与 Codex 的端到端环境初始化。通过 npx zcf 可以一站式完成配置目录创建、API/代理接入、MCP 服务接入、工作流导入、输出风格与记忆配置，以及常用工具安装。",[14,83,84],{},"最近体验了一下 zcf:init-project，发现确实不赖，比Claude Code 和 Codex 自带的 init功能要好不少。这个初始化会生成总体的和目录级的 CLAUDE.md 文件作为指导，还会生成 .claude/index.json 文件作为文件索引，同时会记录修改时间，之后每次使用 Claude Code 修改代码还会同步修改 CLAUDE.md 文件，记录自己在什么时候修改了什么代码，这样可以极大地保证代码的可维护性，防止 AI 拉了一大坨之后无法维护，虽然会加大 Token 消耗，但收益大于付出。",[14,86,87],{},"zcf 在安装时还可以选择性地安装部分 MCP 服务，我个人一般会安装 spec-workflow，context7 和 open-websearch 这三个 MCP 服务，不过最近 spec-workflow 总是连接不上，后面要找个时间修复一下。",{"title":89,"searchDepth":90,"depth":90,"links":91},"",2,[92,93,100,101,102,103],{"id":12,"depth":90,"text":12},{"id":19,"depth":90,"text":19,"children":94},[95,97,98,99],{"id":26,"depth":96,"text":27},3,{"id":33,"depth":96,"text":34},{"id":40,"depth":96,"text":41},{"id":47,"depth":96,"text":48},{"id":54,"depth":90,"text":55},{"id":61,"depth":90,"text":62},{"id":68,"depth":90,"text":68},{"id":74,"depth":90,"text":75},"记录自己折腾一些智能体相关技术的经历","md",{"date":107,"image":108,"alt":109,"tags":110,"published":114},"30 April 2026","/blogs-img/blog4.jpg","Agent",[109,111,112,62,55,113],"折腾笔记","worlflow","harness",true,"/blogs/agent",{"title":5,"description":104},"blogs/4. Agent 相关的折腾记录（持续更新）","MZjPoTh66xeyuEQbTYehdzUZgNUHr_-nXNCmv3pTqWA",[120,12772,44056,44443,44508],{"id":121,"title":122,"body":123,"description":12761,"extension":105,"meta":12762,"navigation":114,"ogImage":12764,"path":12768,"seo":12769,"stem":12770,"__hash__":12771},"content/blogs/1. transformer复习笔记.md","transformer 复习笔记",{"type":7,"value":124,"toc":12745},[125,127,130,147,150,157,160,163,166,426,429,553,869,1301,1307,1578,2497,2500,2726,2729,2968,2975,2978,2981,2986,3318,4665,4682,4908,5367,5370,5465,5468,6055,6416,6772,7129,7402,7405,7408,7741,7898,7901,7904,8096,8235,8238,8750,9149,9153,9156,9768,9772,9775,10775,10778,10781,11072,11075,11232,11243,11837,11840,12547,12550,12728,12731,12736,12741],[10,126,12],{"id":12},[14,128,129],{},"距离首次学习 transformer 已经过去一年，最近在回忆 transformer 的相关内容，发现相关内容忘的差不多了，于是决定复习一下并记录下来。",[14,131,132,133,140,141,146],{},"本文主要记录对transformer的理解和思考，代码实现完全参考 ",[134,135,139],"a",{"href":136,"rel":137},"https://zh.d2l.ai/chapter_attention-mechanisms/transformer.html",[138],"nofollow","动手学深度学习"," 部分，部分理论的解释参考 ",[134,142,145],{"href":143,"rel":144},"https://nlp.seas.harvard.edu/annotated-transformer/",[138],"The Annotated Transformer"," 内容。",[10,148,149],{"id":149},"整体框架",[14,151,152],{},[153,154],"img",{"alt":155,"src":156},"transformer架构图","transformer/transformer.png",[10,158,159],{"id":159},"注意力机制",[24,161,162],{"id":162},"注意力函数",[14,164,165],{},"有两种常见的注意力函数，分别为加性注意力和缩放点积注意力。",[14,167,168,169,425],{},"点积注意力函数与缩放点积注意力函数基本相同，但缩放点积注意力函数有一个额外的缩放因子 ",[170,171,174,213],"span",{"className":172},[173],"katex",[170,175,178],{"className":176},[177],"katex-mathml",[179,180,182],"math",{"xmlns":181},"http://www.w3.org/1998/Math/MathML",[183,184,185,208],"semantics",{},[186,187,188],"mrow",{},[189,190,191,195],"mfrac",{},[192,193,194],"mn",{},"1",[196,197,198],"msqrt",{},[199,200,201,205],"msub",{},[202,203,204],"mi",{},"d",[202,206,207],{},"k",[209,210,212],"annotation",{"encoding":211},"application/x-tex","\\frac{1}{\\sqrt{d_k}}",[170,214,218],{"className":215,"ariaHidden":217},[216],"katex-html","true",[170,219,222,227],{"className":220},[221],"base",[170,223],{"className":224,"style":226},[225],"strut","height:1.3831em;vertical-align:-0.538em;",[170,228,231,236,421],{"className":229},[230],"mord",[170,232],{"className":233},[234,235],"mopen","nulldelimiter",[170,237,239],{"className":238},[189],[170,240,244,412],{"className":241},[242,243],"vlist-t","vlist-t2",[170,245,248,409],{"className":246},[247],"vlist-r",[170,249,253,383,394],{"className":250,"style":252},[251],"vlist","height:0.8451em;",[170,254,256,261],{"style":255},"top:-2.5864em;",[170,257],{"className":258,"style":260},[259],"pstrut","height:3em;",[170,262,268],{"className":263},[264,265,266,267],"sizing","reset-size6","size3","mtight",[170,269,271],{"className":270},[230,267],[170,272,275],{"className":273},[230,274,267],"sqrt",[170,276,278,374],{"className":277},[242,243],[170,279,281,371],{"className":280},[247],[170,282,285,348],{"className":283,"style":284},[251],"height:0.8622em;",[170,286,290,293],{"className":287,"style":289},[288],"svg-align","top:-3em;",[170,291],{"className":292,"style":260},[259],[170,294,297],{"className":295,"style":296},[230,267],"padding-left:0.833em;",[170,298,300,304],{"className":299},[230,267],[170,301,204],{"className":302},[230,303,267],"mathnormal",[170,305,308],{"className":306},[307],"msupsub",[170,309,311,339],{"className":310},[242,243],[170,312,314,334],{"className":313},[247],[170,315,318],{"className":316,"style":317},[251],"height:0.3448em;",[170,319,321,325],{"style":320},"top:-2.3488em;margin-left:0em;margin-right:0.0714em;",[170,322],{"className":323,"style":324},[259],"height:2.5em;",[170,326,330],{"className":327},[264,328,329,267],"reset-size3","size1",[170,331,207],{"className":332,"style":333},[230,303,267],"margin-right:0.03148em;",[170,335,338],{"className":336},[337],"vlist-s","​",[170,340,342],{"className":341},[247],[170,343,346],{"className":344,"style":345},[251],"height:0.1512em;",[170,347],{},[170,349,351,354],{"style":350},"top:-2.8222em;",[170,352],{"className":353,"style":260},[259],[170,355,359],{"className":356,"style":358},[357,267],"hide-tail","min-width:0.853em;height:1.08em;",[360,361,367],"svg",{"xmlns":362,"width":363,"height":364,"viewBox":365,"preserveAspectRatio":366},"http://www.w3.org/2000/svg","400em","1.08em","0 0 400000 1080","xMinYMin slice",[368,369],"path",{"d":370},"M95,702\nc-2.7,0,-7.17,-2.7,-13.5,-8c-5.8,-5.3,-9.5,-10,-9.5,-14\nc0,-2,0.3,-3.3,1,-4c1.3,-2.7,23.83,-20.7,67.5,-54\nc44.2,-33.3,65.8,-50.3,66.5,-51c1.3,-1.3,3,-2,5,-2c4.7,0,8.7,3.3,12,10\ns173,378,173,378c0.7,0,35.3,-71,104,-213c68.7,-142,137.5,-285,206.5,-429\nc69,-144,104.5,-217.7,106.5,-221\nl0 -0\nc5.3,-9.3,12,-14,20,-14\nH400000v40H845.2724\ns-225.272,467,-225.272,467s-235,486,-235,486c-2.7,4.7,-9,7,-19,7\nc-6,0,-10,-1,-12,-3s-194,-422,-194,-422s-65,47,-65,47z\nM834 80h400000v40h-400000z",[170,372,338],{"className":373},[337],[170,375,377],{"className":376},[247],[170,378,381],{"className":379,"style":380},[251],"height:0.1778em;",[170,382],{},[170,384,386,389],{"style":385},"top:-3.23em;",[170,387],{"className":388,"style":260},[259],[170,390],{"className":391,"style":393},[392],"frac-line","border-bottom-width:0.04em;",[170,395,397,400],{"style":396},"top:-3.394em;",[170,398],{"className":399,"style":260},[259],[170,401,403],{"className":402},[264,265,266,267],[170,404,406],{"className":405},[230,267],[170,407,194],{"className":408},[230,267],[170,410,338],{"className":411},[337],[170,413,415],{"className":414},[247],[170,416,419],{"className":417,"style":418},[251],"height:0.538em;",[170,420],{},[170,422],{"className":423},[424,235],"mclose","。",[14,427,428],{},"加性注意力通过一个具有单个隐藏层的前馈网络计算兼容性函数。虽然两者在理论复杂度上相似，但在实际应用中，点积注意力更快且更节省空间，因为它可以使用高度优化的矩阵乘法代码实现。",[14,430,431,432,462,463,492,493,522,523,425],{},"在实践中，我们通常从小批量的角度来考虑提高效率，例如基于 ",[170,433,435,449],{"className":434},[173],[170,436,438],{"className":437},[177],[179,439,440],{"xmlns":181},[183,441,442,447],{},[186,443,444],{},[202,445,446],{},"n",[209,448,446],{"encoding":211},[170,450,452],{"className":451,"ariaHidden":217},[216],[170,453,455,459],{"className":454},[221],[170,456],{"className":457,"style":458},[225],"height:0.4306em;",[170,460,446],{"className":461},[230,303]," 个查询和 ",[170,464,466,480],{"className":465},[173],[170,467,469],{"className":468},[177],[179,470,471],{"xmlns":181},[183,472,473,478],{},[186,474,475],{},[202,476,477],{},"m",[209,479,477],{"encoding":211},[170,481,483],{"className":482,"ariaHidden":217},[216],[170,484,486,489],{"className":485},[221],[170,487],{"className":488,"style":458},[225],[170,490,477],{"className":491},[230,303]," 个键-值对计算注意力，其中查询和键的长度为 ",[170,494,496,509],{"className":495},[173],[170,497,499],{"className":498},[177],[179,500,501],{"xmlns":181},[183,502,503,507],{},[186,504,505],{},[202,506,204],{},[209,508,204],{"encoding":211},[170,510,512],{"className":511,"ariaHidden":217},[216],[170,513,515,519],{"className":514},[221],[170,516],{"className":517,"style":518},[225],"height:0.6944em;",[170,520,204],{"className":521},[230,303],"，值的长度为 ",[170,524,526,540],{"className":525},[173],[170,527,529],{"className":528},[177],[179,530,531],{"xmlns":181},[183,532,533,538],{},[186,534,535],{},[202,536,537],{},"v",[209,539,537],{"encoding":211},[170,541,543],{"className":542,"ariaHidden":217},[216],[170,544,546,549],{"className":545},[221],[170,547],{"className":548,"style":458},[225],[170,550,537],{"className":551,"style":552},[230,303],"margin-right:0.03588em;",[14,554,555,556,668,669,768,769,868],{},"查询 ",[170,557,559,594],{"className":558},[173],[170,560,562],{"className":561},[177],[179,563,564],{"xmlns":181},[183,565,566,591],{},[186,567,568,571,575],{},[202,569,570],{},"Q",[572,573,574],"mo",{},"∈",[576,577,578,582],"msup",{},[202,579,581],{"mathvariant":580},"double-struck","R",[186,583,584,586,589],{},[202,585,446],{},[572,587,588],{},"×",[202,590,204],{},[209,592,593],{"encoding":211},"Q \\in \\mathbb{R}^{n \\times d}",[170,595,597,619],{"className":596,"ariaHidden":217},[216],[170,598,600,604,607,612,616],{"className":599},[221],[170,601],{"className":602,"style":603},[225],"height:0.8778em;vertical-align:-0.1944em;",[170,605,570],{"className":606},[230,303],[170,608],{"className":609,"style":611},[610],"mspace","margin-right:0.2778em;",[170,613,574],{"className":614},[615],"mrel",[170,617],{"className":618,"style":611},[610],[170,620,622,626],{"className":621},[221],[170,623],{"className":624,"style":625},[225],"height:0.8491em;",[170,627,629,633],{"className":628},[230],[170,630,581],{"className":631},[230,632],"mathbb",[170,634,636],{"className":635},[307],[170,637,639],{"className":638},[242],[170,640,642],{"className":641},[247],[170,643,645],{"className":644,"style":625},[251],[170,646,648,652],{"style":647},"top:-3.063em;margin-right:0.05em;",[170,649],{"className":650,"style":651},[259],"height:2.7em;",[170,653,655],{"className":654},[264,265,266,267],[170,656,658,661,665],{"className":657},[230,267],[170,659,446],{"className":660},[230,303,267],[170,662,588],{"className":663},[664,267],"mbin",[170,666,204],{"className":667},[230,303,267],"、键 ",[170,670,672,701],{"className":671},[173],[170,673,675],{"className":674},[177],[179,676,677],{"xmlns":181},[183,678,679,698],{},[186,680,681,684,686],{},[202,682,683],{},"K",[572,685,574],{},[576,687,688,690],{},[202,689,581],{"mathvariant":580},[186,691,692,694,696],{},[202,693,477],{},[572,695,588],{},[202,697,204],{},[209,699,700],{"encoding":211},"K \\in \\mathbb{R}^{m \\times d}",[170,702,704,724],{"className":703,"ariaHidden":217},[216],[170,705,707,711,715,718,721],{"className":706},[221],[170,708],{"className":709,"style":710},[225],"height:0.7224em;vertical-align:-0.0391em;",[170,712,683],{"className":713,"style":714},[230,303],"margin-right:0.07153em;",[170,716],{"className":717,"style":611},[610],[170,719,574],{"className":720},[615],[170,722],{"className":723,"style":611},[610],[170,725,727,730],{"className":726},[221],[170,728],{"className":729,"style":625},[225],[170,731,733,736],{"className":732},[230],[170,734,581],{"className":735},[230,632],[170,737,739],{"className":738},[307],[170,740,742],{"className":741},[242],[170,743,745],{"className":744},[247],[170,746,748],{"className":747,"style":625},[251],[170,749,750,753],{"style":647},[170,751],{"className":752,"style":651},[259],[170,754,756],{"className":755},[264,265,266,267],[170,757,759,762,765],{"className":758},[230,267],[170,760,477],{"className":761},[230,303,267],[170,763,588],{"className":764},[664,267],[170,766,204],{"className":767},[230,303,267]," 和值 ",[170,770,772,801],{"className":771},[173],[170,773,775],{"className":774},[177],[179,776,777],{"xmlns":181},[183,778,779,798],{},[186,780,781,784,786],{},[202,782,783],{},"V",[572,785,574],{},[576,787,788,790],{},[202,789,581],{"mathvariant":580},[186,791,792,794,796],{},[202,793,477],{},[572,795,588],{},[202,797,537],{},[209,799,800],{"encoding":211},"V \\in \\mathbb{R}^{m \\times v}",[170,802,804,823],{"className":803,"ariaHidden":217},[216],[170,805,807,810,814,817,820],{"className":806},[221],[170,808],{"className":809,"style":710},[225],[170,811,783],{"className":812,"style":813},[230,303],"margin-right:0.22222em;",[170,815],{"className":816,"style":611},[610],[170,818,574],{"className":819},[615],[170,821],{"className":822,"style":611},[610],[170,824,826,830],{"className":825},[221],[170,827],{"className":828,"style":829},[225],"height:0.7713em;",[170,831,833,836],{"className":832},[230],[170,834,581],{"className":835},[230,632],[170,837,839],{"className":838},[307],[170,840,842],{"className":841},[242],[170,843,845],{"className":844},[247],[170,846,848],{"className":847,"style":829},[251],[170,849,850,853],{"style":647},[170,851],{"className":852,"style":651},[259],[170,854,856],{"className":855},[264,265,266,267],[170,857,859,862,865],{"className":858},[230,267],[170,860,477],{"className":861},[230,303,267],[170,863,588],{"className":864},[664,267],[170,866,537],{"className":867,"style":552},[230,303,267]," 的缩放点积注意力是：",[14,870,871],{},[170,872,874,979],{"className":873},[173],[170,875,877],{"className":876},[177],[179,878,879],{"xmlns":181},[183,880,881,976],{},[186,882,883,886,889,891,894,896,898,901,904,906,910,912,915,917,919,921,924,927,931,934,957,959,961,973],{},[202,884,885],{},"A",[202,887,888],{},"t",[202,890,888],{},[202,892,893],{},"e",[202,895,446],{},[202,897,888],{},[202,899,900],{},"i",[202,902,903],{},"o",[202,905,446],{},[572,907,909],{"stretchy":908},"false","(",[202,911,570],{},[572,913,914],{"separator":217},",",[202,916,683],{},[572,918,914],{"separator":217},[202,920,783],{},[572,922,923],{"stretchy":908},")",[572,925,926],{},"=",[202,928,930],{"mathvariant":929},"normal","softmax",[572,932,933],{},"⁡",[186,935,936,938,955],{},[572,937,909],{"fence":217},[189,939,940,951],{},[186,941,942,944],{},[202,943,570],{},[576,945,946,948],{},[202,947,683],{},[202,949,950],{"mathvariant":929},"⊤",[196,952,953],{},[202,954,204],{},[572,956,923],{"fence":217},[202,958,783],{},[572,960,574],{},[576,962,963,965],{},[202,964,581],{"mathvariant":580},[186,966,967,969,971],{},[202,968,446],{},[572,970,588],{},[202,972,537],{},[202,974,975],{"mathvariant":929},".",[209,977,978],{"encoding":211}," Attention(Q,K,V) = \\operatorname{softmax}\\left(\\frac{QK^\\top}{\\sqrt{d}}\\right)V \\in \\mathbb{R}^{n \\times v}.",[170,980,982,1052,1254],{"className":981,"ariaHidden":217},[216],[170,983,985,989,992,996,999,1002,1005,1008,1011,1014,1017,1020,1024,1028,1031,1034,1037,1040,1043,1046,1049],{"className":984},[221],[170,986],{"className":987,"style":988},[225],"height:1em;vertical-align:-0.25em;",[170,990,885],{"className":991},[230,303],[170,993,995],{"className":994},[230,303],"tt",[170,997,893],{"className":998},[230,303],[170,1000,446],{"className":1001},[230,303],[170,1003,888],{"className":1004},[230,303],[170,1006,900],{"className":1007},[230,303],[170,1009,903],{"className":1010},[230,303],[170,1012,446],{"className":1013},[230,303],[170,1015,909],{"className":1016},[234],[170,1018,570],{"className":1019},[230,303],[170,1021,914],{"className":1022},[1023],"mpunct",[170,1025],{"className":1026,"style":1027},[610],"margin-right:0.1667em;",[170,1029,683],{"className":1030,"style":714},[230,303],[170,1032,914],{"className":1033},[1023],[170,1035],{"className":1036,"style":1027},[610],[170,1038,783],{"className":1039,"style":813},[230,303],[170,1041,923],{"className":1042},[424],[170,1044],{"className":1045,"style":611},[610],[170,1047,926],{"className":1048},[615],[170,1050],{"className":1051,"style":611},[610],[170,1053,1055,1059,1067,1070,1239,1242,1245,1248,1251],{"className":1054},[221],[170,1056],{"className":1057,"style":1058},[225],"height:1.8em;vertical-align:-0.65em;",[170,1060,1063],{"className":1061},[1062],"mop",[170,1064,930],{"className":1065},[230,1066],"mathrm",[170,1068],{"className":1069,"style":1027},[610],[170,1071,1074,1084,1233],{"className":1072},[1073],"minner",[170,1075,1079],{"className":1076,"style":1078},[234,1077],"delimcenter","top:0em;",[170,1080,909],{"className":1081},[1082,1083],"delimsizing","size2",[170,1085,1087,1090,1230],{"className":1086},[230],[170,1088],{"className":1089},[234,235],[170,1091,1093],{"className":1092},[189],[170,1094,1096,1222],{"className":1095},[242,243],[170,1097,1099,1219],{"className":1098},[247],[170,1100,1103,1165,1173],{"className":1101,"style":1102},[251],"height:1.095em;",[170,1104,1106,1109],{"style":1105},"top:-2.5335em;",[170,1107],{"className":1108,"style":260},[259],[170,1110,1112],{"className":1111},[264,265,266,267],[170,1113,1115],{"className":1114},[230,267],[170,1116,1118],{"className":1117},[230,274,267],[170,1119,1121,1156],{"className":1120},[242,243],[170,1122,1124,1153],{"className":1123},[247],[170,1125,1128,1140],{"className":1126,"style":1127},[251],"height:0.9378em;",[170,1129,1131,1134],{"className":1130,"style":289},[288],[170,1132],{"className":1133,"style":260},[259],[170,1135,1137],{"className":1136,"style":296},[230,267],[170,1138,204],{"className":1139},[230,303,267],[170,1141,1143,1146],{"style":1142},"top:-2.8978em;",[170,1144],{"className":1145,"style":260},[259],[170,1147,1149],{"className":1148,"style":358},[357,267],[360,1150,1151],{"xmlns":362,"width":363,"height":364,"viewBox":365,"preserveAspectRatio":366},[368,1152],{"d":370},[170,1154,338],{"className":1155},[337],[170,1157,1159],{"className":1158},[247],[170,1160,1163],{"className":1161,"style":1162},[251],"height:0.1022em;",[170,1164],{},[170,1166,1167,1170],{"style":385},[170,1168],{"className":1169,"style":260},[259],[170,1171],{"className":1172,"style":393},[392],[170,1174,1176,1179],{"style":1175},"top:-3.4461em;",[170,1177],{"className":1178,"style":260},[259],[170,1180,1182],{"className":1181},[264,265,266,267],[170,1183,1185,1188],{"className":1184},[230,267],[170,1186,570],{"className":1187},[230,303,267],[170,1189,1191,1194],{"className":1190},[230,267],[170,1192,683],{"className":1193,"style":714},[230,303,267],[170,1195,1197],{"className":1196},[307],[170,1198,1200],{"className":1199},[242],[170,1201,1203],{"className":1202},[247],[170,1204,1207],{"className":1205,"style":1206},[251],"height:0.927em;",[170,1208,1210,1213],{"style":1209},"top:-2.931em;margin-right:0.0714em;",[170,1211],{"className":1212,"style":324},[259],[170,1214,1216],{"className":1215},[264,328,329,267],[170,1217,950],{"className":1218},[230,267],[170,1220,338],{"className":1221},[337],[170,1223,1225],{"className":1224},[247],[170,1226,1228],{"className":1227,"style":418},[251],[170,1229],{},[170,1231],{"className":1232},[424,235],[170,1234,1236],{"className":1235,"style":1078},[424,1077],[170,1237,923],{"className":1238},[1082,1083],[170,1240],{"className":1241,"style":1027},[610],[170,1243,783],{"className":1244,"style":813},[230,303],[170,1246],{"className":1247,"style":611},[610],[170,1249,574],{"className":1250},[615],[170,1252],{"className":1253,"style":611},[610],[170,1255,1257,1260,1298],{"className":1256},[221],[170,1258],{"className":1259,"style":829},[225],[170,1261,1263,1266],{"className":1262},[230],[170,1264,581],{"className":1265},[230,632],[170,1267,1269],{"className":1268},[307],[170,1270,1272],{"className":1271},[242],[170,1273,1275],{"className":1274},[247],[170,1276,1278],{"className":1277,"style":829},[251],[170,1279,1280,1283],{"style":647},[170,1281],{"className":1282,"style":651},[259],[170,1284,1286],{"className":1285},[264,265,266,267],[170,1287,1289,1292,1295],{"className":1288},[230,267],[170,1290,446],{"className":1291},[230,303,267],[170,1293,588],{"className":1294},[664,267],[170,1296,537],{"className":1297,"style":552},[230,303,267],[170,1299,975],{"className":1300},[230],[14,1302,1303],{},[153,1304],{"alt":1305,"src":1306},"注意力","transformer/attention.png",[14,1308,1309,1310,1384,1385,1456,1457,1577],{},"缩放点积注意力的输入由维度为 ",[170,1311,1313,1331],{"className":1312},[173],[170,1314,1316],{"className":1315},[177],[179,1317,1318],{"xmlns":181},[183,1319,1320,1328],{},[186,1321,1322],{},[199,1323,1324,1326],{},[202,1325,204],{},[202,1327,207],{},[209,1329,1330],{"encoding":211},"d_k",[170,1332,1334],{"className":1333,"ariaHidden":217},[216],[170,1335,1337,1341],{"className":1336},[221],[170,1338],{"className":1339,"style":1340},[225],"height:0.8444em;vertical-align:-0.15em;",[170,1342,1344,1347],{"className":1343},[230],[170,1345,204],{"className":1346},[230,303],[170,1348,1350],{"className":1349},[307],[170,1351,1353,1375],{"className":1352},[242,243],[170,1354,1356,1372],{"className":1355},[247],[170,1357,1360],{"className":1358,"style":1359},[251],"height:0.3361em;",[170,1361,1363,1366],{"style":1362},"top:-2.55em;margin-left:0em;margin-right:0.05em;",[170,1364],{"className":1365,"style":651},[259],[170,1367,1369],{"className":1368},[264,265,266,267],[170,1370,207],{"className":1371,"style":333},[230,303,267],[170,1373,338],{"className":1374},[337],[170,1376,1378],{"className":1377},[247],[170,1379,1382],{"className":1380,"style":1381},[251],"height:0.15em;",[170,1383],{}," ​的查询和键以及维度为 ",[170,1386,1388,1406],{"className":1387},[173],[170,1389,1391],{"className":1390},[177],[179,1392,1393],{"xmlns":181},[183,1394,1395,1403],{},[186,1396,1397],{},[199,1398,1399,1401],{},[202,1400,204],{},[202,1402,537],{},[209,1404,1405],{"encoding":211},"d_v",[170,1407,1409],{"className":1408,"ariaHidden":217},[216],[170,1410,1412,1415],{"className":1411},[221],[170,1413],{"className":1414,"style":1340},[225],[170,1416,1418,1421],{"className":1417},[230],[170,1419,204],{"className":1420},[230,303],[170,1422,1424],{"className":1423},[307],[170,1425,1427,1448],{"className":1426},[242,243],[170,1428,1430,1445],{"className":1429},[247],[170,1431,1434],{"className":1432,"style":1433},[251],"height:0.1514em;",[170,1435,1436,1439],{"style":1362},[170,1437],{"className":1438,"style":651},[259],[170,1440,1442],{"className":1441},[264,265,266,267],[170,1443,537],{"className":1444,"style":552},[230,303,267],[170,1446,338],{"className":1447},[337],[170,1449,1451],{"className":1450},[247],[170,1452,1454],{"className":1453,"style":1381},[251],[170,1455],{}," ​的值组成。我们计算查询与所有键的点积，将每个点积除以 ",[170,1458,1460,1480],{"className":1459},[173],[170,1461,1463],{"className":1462},[177],[179,1464,1465],{"xmlns":181},[183,1466,1467,1477],{},[186,1468,1469],{},[196,1470,1471],{},[199,1472,1473,1475],{},[202,1474,204],{},[202,1476,207],{},[209,1478,1479],{"encoding":211},"\\sqrt{d_k}",[170,1481,1483],{"className":1482,"ariaHidden":217},[216],[170,1484,1486,1490],{"className":1485},[221],[170,1487],{"className":1488,"style":1489},[225],"height:1.04em;vertical-align:-0.1828em;",[170,1491,1493],{"className":1492},[230,274],[170,1494,1496,1568],{"className":1495},[242,243],[170,1497,1499,1565],{"className":1498},[247],[170,1500,1503,1552],{"className":1501,"style":1502},[251],"height:0.8572em;",[170,1504,1506,1509],{"className":1505,"style":289},[288],[170,1507],{"className":1508,"style":260},[259],[170,1510,1512],{"className":1511,"style":296},[230],[170,1513,1515,1518],{"className":1514},[230],[170,1516,204],{"className":1517},[230,303],[170,1519,1521],{"className":1520},[307],[170,1522,1524,1544],{"className":1523},[242,243],[170,1525,1527,1541],{"className":1526},[247],[170,1528,1530],{"className":1529,"style":1359},[251],[170,1531,1532,1535],{"style":1362},[170,1533],{"className":1534,"style":651},[259],[170,1536,1538],{"className":1537},[264,265,266,267],[170,1539,207],{"className":1540,"style":333},[230,303,267],[170,1542,338],{"className":1543},[337],[170,1545,1547],{"className":1546},[247],[170,1548,1550],{"className":1549,"style":1381},[251],[170,1551],{},[170,1553,1555,1558],{"style":1554},"top:-2.8172em;",[170,1556],{"className":1557,"style":260},[259],[170,1559,1561],{"className":1560,"style":358},[357],[360,1562,1563],{"xmlns":362,"width":363,"height":364,"viewBox":365,"preserveAspectRatio":366},[368,1564],{"d":370},[170,1566,338],{"className":1567},[337],[170,1569,1571],{"className":1570},[247],[170,1572,1575],{"className":1573,"style":1574},[251],"height:0.1828em;",[170,1576],{},"​​，并应用softmax函数以获得值上的权重。",[14,1579,1580,1581,1650,1651,1720,1721,1790,1791,1821,1822,1850,1851,1881,1882,1910,1911,2214,2215,1881,2243,2312,2313,425],{},"对于较小的 ",[170,1582,1584,1601],{"className":1583},[173],[170,1585,1587],{"className":1586},[177],[179,1588,1589],{"xmlns":181},[183,1590,1591,1599],{},[186,1592,1593],{},[199,1594,1595,1597],{},[202,1596,204],{},[202,1598,207],{},[209,1600,1330],{"encoding":211},[170,1602,1604],{"className":1603,"ariaHidden":217},[216],[170,1605,1607,1610],{"className":1606},[221],[170,1608],{"className":1609,"style":1340},[225],[170,1611,1613,1616],{"className":1612},[230],[170,1614,204],{"className":1615},[230,303],[170,1617,1619],{"className":1618},[307],[170,1620,1622,1642],{"className":1621},[242,243],[170,1623,1625,1639],{"className":1624},[247],[170,1626,1628],{"className":1627,"style":1359},[251],[170,1629,1630,1633],{"style":1362},[170,1631],{"className":1632,"style":651},[259],[170,1634,1636],{"className":1635},[264,265,266,267],[170,1637,207],{"className":1638,"style":333},[230,303,267],[170,1640,338],{"className":1641},[337],[170,1643,1645],{"className":1644},[247],[170,1646,1648],{"className":1647,"style":1381},[251],[170,1649],{}," 值来说，加性注意力和点积注意力的表现相似，但加性注意力在 ",[170,1652,1654,1671],{"className":1653},[173],[170,1655,1657],{"className":1656},[177],[179,1658,1659],{"xmlns":181},[183,1660,1661,1669],{},[186,1662,1663],{},[199,1664,1665,1667],{},[202,1666,204],{},[202,1668,207],{},[209,1670,1330],{"encoding":211},[170,1672,1674],{"className":1673,"ariaHidden":217},[216],[170,1675,1677,1680],{"className":1676},[221],[170,1678],{"className":1679,"style":1340},[225],[170,1681,1683,1686],{"className":1682},[230],[170,1684,204],{"className":1685},[230,303],[170,1687,1689],{"className":1688},[307],[170,1690,1692,1712],{"className":1691},[242,243],[170,1693,1695,1709],{"className":1694},[247],[170,1696,1698],{"className":1697,"style":1359},[251],[170,1699,1700,1703],{"style":1362},[170,1701],{"className":1702,"style":651},[259],[170,1704,1706],{"className":1705},[264,265,266,267],[170,1707,207],{"className":1708,"style":333},[230,303,267],[170,1710,338],{"className":1711},[337],[170,1713,1715],{"className":1714},[247],[170,1716,1718],{"className":1717,"style":1381},[251],[170,1719],{}," 值较大时会优于点积注意力。我们怀疑在 ",[170,1722,1724,1741],{"className":1723},[173],[170,1725,1727],{"className":1726},[177],[179,1728,1729],{"xmlns":181},[183,1730,1731,1739],{},[186,1732,1733],{},[199,1734,1735,1737],{},[202,1736,204],{},[202,1738,207],{},[209,1740,1330],{"encoding":211},[170,1742,1744],{"className":1743,"ariaHidden":217},[216],[170,1745,1747,1750],{"className":1746},[221],[170,1748],{"className":1749,"style":1340},[225],[170,1751,1753,1756],{"className":1752},[230],[170,1754,204],{"className":1755},[230,303],[170,1757,1759],{"className":1758},[307],[170,1760,1762,1782],{"className":1761},[242,243],[170,1763,1765,1779],{"className":1764},[247],[170,1766,1768],{"className":1767,"style":1359},[251],[170,1769,1770,1773],{"style":1362},[170,1771],{"className":1772,"style":651},[259],[170,1774,1776],{"className":1775},[264,265,266,267],[170,1777,207],{"className":1778,"style":333},[230,303,267],[170,1780,338],{"className":1781},[337],[170,1783,1785],{"className":1784},[247],[170,1786,1788],{"className":1787,"style":1381},[251],[170,1789],{}," 值较大时，点积的值会很大，导致 softmax 集中在很小的一个区域内。为了更好地说明这一点，我们假设查询 ",[170,1792,1794,1808],{"className":1793},[173],[170,1795,1797],{"className":1796},[177],[179,1798,1799],{"xmlns":181},[183,1800,1801,1806],{},[186,1802,1803],{},[202,1804,1805],{},"q",[209,1807,1805],{"encoding":211},[170,1809,1811],{"className":1810,"ariaHidden":217},[216],[170,1812,1814,1818],{"className":1813},[221],[170,1815],{"className":1816,"style":1817},[225],"height:0.625em;vertical-align:-0.1944em;",[170,1819,1805],{"className":1820,"style":552},[230,303]," 和键 ",[170,1823,1825,1838],{"className":1824},[173],[170,1826,1828],{"className":1827},[177],[179,1829,1830],{"xmlns":181},[183,1831,1832,1836],{},[186,1833,1834],{},[202,1835,207],{},[209,1837,207],{"encoding":211},[170,1839,1841],{"className":1840,"ariaHidden":217},[216],[170,1842,1844,1847],{"className":1843},[221],[170,1845],{"className":1846,"style":518},[225],[170,1848,207],{"className":1849,"style":333},[230,303]," 都符合一个均值为 ",[170,1852,1854,1868],{"className":1853},[173],[170,1855,1857],{"className":1856},[177],[179,1858,1859],{"xmlns":181},[183,1860,1861,1866],{},[186,1862,1863],{},[192,1864,1865],{},"0",[209,1867,1865],{"encoding":211},[170,1869,1871],{"className":1870,"ariaHidden":217},[216],[170,1872,1874,1878],{"className":1873},[221],[170,1875],{"className":1876,"style":1877},[225],"height:0.6444em;",[170,1879,1865],{"className":1880},[230],"，方差为 ",[170,1883,1885,1898],{"className":1884},[173],[170,1886,1888],{"className":1887},[177],[179,1889,1890],{"xmlns":181},[183,1891,1892,1896],{},[186,1893,1894],{},[192,1895,194],{},[209,1897,194],{"encoding":211},[170,1899,1901],{"className":1900,"ariaHidden":217},[216],[170,1902,1904,1907],{"className":1903},[221],[170,1905],{"className":1906,"style":1877},[225],[170,1908,194],{"className":1909},[230]," 的随机变量。他们点积的结果 ",[170,1912,1914,1967],{"className":1913},[173],[170,1915,1917],{"className":1916},[177],[179,1918,1919],{"xmlns":181},[183,1920,1921,1964],{},[186,1922,1923,1925,1928,1930,1932,1952,1958],{},[202,1924,1805],{},[572,1926,1927],{},"⋅",[202,1929,207],{},[572,1931,926],{},[1933,1934,1935,1938,1946],"msubsup",{},[572,1936,1937],{},"∑",[186,1939,1940,1942,1944],{},[202,1941,900],{},[572,1943,926],{},[192,1945,194],{},[199,1947,1948,1950],{},[202,1949,204],{},[202,1951,207],{},[199,1953,1954,1956],{},[202,1955,1805],{},[202,1957,900],{},[199,1959,1960,1962],{},[202,1961,207],{},[202,1963,900],{},[209,1965,1966],{"encoding":211},"q \\cdot k = \\sum_{i=1}^{d_k}q_{i}k_{i}",[170,1968,1970,1990,2008],{"className":1969,"ariaHidden":217},[216],[170,1971,1973,1977,1980,1984,1987],{"className":1972},[221],[170,1974],{"className":1975,"style":1976},[225],"height:0.6389em;vertical-align:-0.1944em;",[170,1978,1805],{"className":1979,"style":552},[230,303],[170,1981],{"className":1982,"style":1983},[610],"margin-right:0.2222em;",[170,1985,1927],{"className":1986},[664],[170,1988],{"className":1989,"style":1983},[610],[170,1991,1993,1996,1999,2002,2005],{"className":1992},[221],[170,1994],{"className":1995,"style":518},[225],[170,1997,207],{"className":1998,"style":333},[230,303],[170,2000],{"className":2001,"style":611},[610],[170,2003,926],{"className":2004},[615],[170,2006],{"className":2007,"style":611},[610],[170,2009,2011,2015,2122,2125,2170],{"className":2010},[221],[170,2012],{"className":2013,"style":2014},[225],"height:1.2887em;vertical-align:-0.2997em;",[170,2016,2018,2024],{"className":2017},[1062],[170,2019,1937],{"className":2020,"style":2023},[1062,2021,2022],"op-symbol","small-op","position:relative;top:0em;",[170,2025,2027],{"className":2026},[307],[170,2028,2030,2113],{"className":2029},[242,243],[170,2031,2033,2110],{"className":2032},[247],[170,2034,2037,2058],{"className":2035,"style":2036},[251],"height:0.989em;",[170,2038,2040,2043],{"style":2039},"top:-2.4003em;margin-left:0em;margin-right:0.05em;",[170,2041],{"className":2042,"style":651},[259],[170,2044,2046],{"className":2045},[264,265,266,267],[170,2047,2049,2052,2055],{"className":2048},[230,267],[170,2050,900],{"className":2051},[230,303,267],[170,2053,926],{"className":2054},[615,267],[170,2056,194],{"className":2057},[230,267],[170,2059,2061,2064],{"style":2060},"top:-3.2029em;margin-right:0.05em;",[170,2062],{"className":2063,"style":651},[259],[170,2065,2067],{"className":2066},[264,265,266,267],[170,2068,2070],{"className":2069},[230,267],[170,2071,2073,2076],{"className":2072},[230,267],[170,2074,204],{"className":2075},[230,303,267],[170,2077,2079],{"className":2078},[307],[170,2080,2082,2102],{"className":2081},[242,243],[170,2083,2085,2099],{"className":2084},[247],[170,2086,2088],{"className":2087,"style":317},[251],[170,2089,2090,2093],{"style":320},[170,2091],{"className":2092,"style":324},[259],[170,2094,2096],{"className":2095},[264,328,329,267],[170,2097,207],{"className":2098,"style":333},[230,303,267],[170,2100,338],{"className":2101},[337],[170,2103,2105],{"className":2104},[247],[170,2106,2108],{"className":2107,"style":345},[251],[170,2109],{},[170,2111,338],{"className":2112},[337],[170,2114,2116],{"className":2115},[247],[170,2117,2120],{"className":2118,"style":2119},[251],"height:0.2997em;",[170,2121],{},[170,2123],{"className":2124,"style":1027},[610],[170,2126,2128,2131],{"className":2127},[230],[170,2129,1805],{"className":2130,"style":552},[230,303],[170,2132,2134],{"className":2133},[307],[170,2135,2137,2162],{"className":2136},[242,243],[170,2138,2140,2159],{"className":2139},[247],[170,2141,2144],{"className":2142,"style":2143},[251],"height:0.3117em;",[170,2145,2147,2150],{"style":2146},"top:-2.55em;margin-left:-0.0359em;margin-right:0.05em;",[170,2148],{"className":2149,"style":651},[259],[170,2151,2153],{"className":2152},[264,265,266,267],[170,2154,2156],{"className":2155},[230,267],[170,2157,900],{"className":2158},[230,303,267],[170,2160,338],{"className":2161},[337],[170,2163,2165],{"className":2164},[247],[170,2166,2168],{"className":2167,"style":1381},[251],[170,2169],{},[170,2171,2173,2176],{"className":2172},[230],[170,2174,207],{"className":2175,"style":333},[230,303],[170,2177,2179],{"className":2178},[307],[170,2180,2182,2206],{"className":2181},[242,243],[170,2183,2185,2203],{"className":2184},[247],[170,2186,2188],{"className":2187,"style":2143},[251],[170,2189,2191,2194],{"style":2190},"top:-2.55em;margin-left:-0.0315em;margin-right:0.05em;",[170,2192],{"className":2193,"style":651},[259],[170,2195,2197],{"className":2196},[264,265,266,267],[170,2198,2200],{"className":2199},[230,267],[170,2201,900],{"className":2202},[230,303,267],[170,2204,338],{"className":2205},[337],[170,2207,2209],{"className":2208},[247],[170,2210,2212],{"className":2211,"style":1381},[251],[170,2213],{}," 的均值为 ",[170,2216,2218,2231],{"className":2217},[173],[170,2219,2221],{"className":2220},[177],[179,2222,2223],{"xmlns":181},[183,2224,2225,2229],{},[186,2226,2227],{},[192,2228,1865],{},[209,2230,1865],{"encoding":211},[170,2232,2234],{"className":2233,"ariaHidden":217},[216],[170,2235,2237,2240],{"className":2236},[221],[170,2238],{"className":2239,"style":1877},[225],[170,2241,1865],{"className":2242},[230],[170,2244,2246,2263],{"className":2245},[173],[170,2247,2249],{"className":2248},[177],[179,2250,2251],{"xmlns":181},[183,2252,2253,2261],{},[186,2254,2255],{},[199,2256,2257,2259],{},[202,2258,204],{},[202,2260,207],{},[209,2262,1330],{"encoding":211},[170,2264,2266],{"className":2265,"ariaHidden":217},[216],[170,2267,2269,2272],{"className":2268},[221],[170,2270],{"className":2271,"style":1340},[225],[170,2273,2275,2278],{"className":2274},[230],[170,2276,204],{"className":2277},[230,303],[170,2279,2281],{"className":2280},[307],[170,2282,2284,2304],{"className":2283},[242,243],[170,2285,2287,2301],{"className":2286},[247],[170,2288,2290],{"className":2289,"style":1359},[251],[170,2291,2292,2295],{"style":1362},[170,2293],{"className":2294,"style":651},[259],[170,2296,2298],{"className":2297},[264,265,266,267],[170,2299,207],{"className":2300,"style":333},[230,303,267],[170,2302,338],{"className":2303},[337],[170,2305,2307],{"className":2306},[247],[170,2308,2310],{"className":2309,"style":1381},[251],[170,2311],{},"，为了抵消这种影响，我们要乘以一个缩放因子 ",[170,2314,2316,2339],{"className":2315},[173],[170,2317,2319],{"className":2318},[177],[179,2320,2321],{"xmlns":181},[183,2322,2323,2337],{},[186,2324,2325],{},[189,2326,2327,2329],{},[192,2328,194],{},[196,2330,2331],{},[199,2332,2333,2335],{},[202,2334,204],{},[202,2336,207],{},[209,2338,212],{"encoding":211},[170,2340,2342],{"className":2341,"ariaHidden":217},[216],[170,2343,2345,2348],{"className":2344},[221],[170,2346],{"className":2347,"style":226},[225],[170,2349,2351,2354,2494],{"className":2350},[230],[170,2352],{"className":2353},[234,235],[170,2355,2357],{"className":2356},[189],[170,2358,2360,2486],{"className":2359},[242,243],[170,2361,2363,2483],{"className":2362},[247],[170,2364,2366,2461,2469],{"className":2365,"style":252},[251],[170,2367,2368,2371],{"style":255},[170,2369],{"className":2370,"style":260},[259],[170,2372,2374],{"className":2373},[264,265,266,267],[170,2375,2377],{"className":2376},[230,267],[170,2378,2380],{"className":2379},[230,274,267],[170,2381,2383,2453],{"className":2382},[242,243],[170,2384,2386,2450],{"className":2385},[247],[170,2387,2389,2438],{"className":2388,"style":284},[251],[170,2390,2392,2395],{"className":2391,"style":289},[288],[170,2393],{"className":2394,"style":260},[259],[170,2396,2398],{"className":2397,"style":296},[230,267],[170,2399,2401,2404],{"className":2400},[230,267],[170,2402,204],{"className":2403},[230,303,267],[170,2405,2407],{"className":2406},[307],[170,2408,2410,2430],{"className":2409},[242,243],[170,2411,2413,2427],{"className":2412},[247],[170,2414,2416],{"className":2415,"style":317},[251],[170,2417,2418,2421],{"style":320},[170,2419],{"className":2420,"style":324},[259],[170,2422,2424],{"className":2423},[264,328,329,267],[170,2425,207],{"className":2426,"style":333},[230,303,267],[170,2428,338],{"className":2429},[337],[170,2431,2433],{"className":2432},[247],[170,2434,2436],{"className":2435,"style":345},[251],[170,2437],{},[170,2439,2440,2443],{"style":350},[170,2441],{"className":2442,"style":260},[259],[170,2444,2446],{"className":2445,"style":358},[357,267],[360,2447,2448],{"xmlns":362,"width":363,"height":364,"viewBox":365,"preserveAspectRatio":366},[368,2449],{"d":370},[170,2451,338],{"className":2452},[337],[170,2454,2456],{"className":2455},[247],[170,2457,2459],{"className":2458,"style":380},[251],[170,2460],{},[170,2462,2463,2466],{"style":385},[170,2464],{"className":2465,"style":260},[259],[170,2467],{"className":2468,"style":393},[392],[170,2470,2471,2474],{"style":396},[170,2472],{"className":2473,"style":260},[259],[170,2475,2477],{"className":2476},[264,265,266,267],[170,2478,2480],{"className":2479},[230,267],[170,2481,194],{"className":2482},[230,267],[170,2484,338],{"className":2485},[337],[170,2487,2489],{"className":2488},[247],[170,2490,2492],{"className":2491,"style":418},[251],[170,2493],{},[170,2495],{"className":2496},[424,235],[14,2498,2499],{},"因为在训练和推理时，并非所有的值都应该加入注意力汇聚操作中，所以我们首先实现 masked_softmax 函数来保证在注意力汇聚时仅加入有意义的值。",[2501,2502,2506],"pre",{"className":2503,"code":2504,"language":2505,"meta":89,"style":89},"language-python shiki shiki-themes github-light dracula","def masked_softmax(X, valid_lens):\n    \"\"\"通过在最后一个轴上掩蔽元素来执行softmax操作\"\"\"\n    # X:3D张量，valid_lens:1D或2D张量\n    if valid_lens is None:\n        return nn.functional.softmax(X, dim=-1)\n    else:\n        shape = X.shape\n        if valid_lens.dim() == 1:\n            valid_lens = torch.repeat_interleave(valid_lens, shape[1])\n        else:\n            valid_lens = valid_lens.reshape(-1)\n        # 最后一轴上被掩蔽的元素使用一个非常大的负值替换，从而其softmax输出为0\n        X = d2l.sequence_mask(X.reshape(-1, shape[-1]), valid_lens,\n                              value=-1e6)\n        return nn.functional.softmax(X.reshape(shape), dim=-1)\n","python",[2507,2508,2509,2537,2543,2549,2568,2589,2597,2608,2625,2641,2649,2666,2672,2697,2710],"code",{"__ignoreMap":89},[170,2510,2513,2517,2521,2524,2528,2531,2534],{"class":2511,"line":2512},"line",1,[170,2514,2516],{"class":2515},"szJfE","def",[170,2518,2520],{"class":2519},"sCLZk"," masked_softmax",[170,2522,909],{"class":2523},"scbbO",[170,2525,2527],{"class":2526},"syNf4","X",[170,2529,2530],{"class":2523},", ",[170,2532,2533],{"class":2526},"valid_lens",[170,2535,2536],{"class":2523},"):\n",[170,2538,2539],{"class":2511,"line":90},[170,2540,2542],{"class":2541},"seLWX","    \"\"\"通过在最后一个轴上掩蔽元素来执行softmax操作\"\"\"\n",[170,2544,2545],{"class":2511,"line":96},[170,2546,2548],{"class":2547},"sfgPZ","    # X:3D张量，valid_lens:1D或2D张量\n",[170,2550,2552,2555,2558,2561,2565],{"class":2511,"line":2551},4,[170,2553,2554],{"class":2515},"    if",[170,2556,2557],{"class":2523}," valid_lens ",[170,2559,2560],{"class":2515},"is",[170,2562,2564],{"class":2563},"soDru"," None",[170,2566,2567],{"class":2523},":\n",[170,2569,2571,2574,2577,2581,2584,2586],{"class":2511,"line":2570},5,[170,2572,2573],{"class":2515},"        return",[170,2575,2576],{"class":2523}," nn.functional.softmax(X, ",[170,2578,2580],{"class":2579},"sQkXh","dim",[170,2582,2583],{"class":2515},"=-",[170,2585,194],{"class":2563},[170,2587,2588],{"class":2523},")\n",[170,2590,2592,2595],{"class":2511,"line":2591},6,[170,2593,2594],{"class":2515},"    else",[170,2596,2567],{"class":2523},[170,2598,2600,2603,2605],{"class":2511,"line":2599},7,[170,2601,2602],{"class":2523},"        shape ",[170,2604,926],{"class":2515},[170,2606,2607],{"class":2523}," X.shape\n",[170,2609,2611,2614,2617,2620,2623],{"class":2511,"line":2610},8,[170,2612,2613],{"class":2515},"        if",[170,2615,2616],{"class":2523}," valid_lens.dim() ",[170,2618,2619],{"class":2515},"==",[170,2621,2622],{"class":2563}," 1",[170,2624,2567],{"class":2523},[170,2626,2628,2631,2633,2636,2638],{"class":2511,"line":2627},9,[170,2629,2630],{"class":2523},"            valid_lens ",[170,2632,926],{"class":2515},[170,2634,2635],{"class":2523}," torch.repeat_interleave(valid_lens, shape[",[170,2637,194],{"class":2563},[170,2639,2640],{"class":2523},"])\n",[170,2642,2644,2647],{"class":2511,"line":2643},10,[170,2645,2646],{"class":2515},"        else",[170,2648,2567],{"class":2523},[170,2650,2652,2654,2656,2659,2662,2664],{"class":2511,"line":2651},11,[170,2653,2630],{"class":2523},[170,2655,926],{"class":2515},[170,2657,2658],{"class":2523}," valid_lens.reshape(",[170,2660,2661],{"class":2515},"-",[170,2663,194],{"class":2563},[170,2665,2588],{"class":2523},[170,2667,2669],{"class":2511,"line":2668},12,[170,2670,2671],{"class":2547},"        # 最后一轴上被掩蔽的元素使用一个非常大的负值替换，从而其softmax输出为0\n",[170,2673,2675,2678,2680,2683,2685,2687,2690,2692,2694],{"class":2511,"line":2674},13,[170,2676,2677],{"class":2523},"        X ",[170,2679,926],{"class":2515},[170,2681,2682],{"class":2523}," d2l.sequence_mask(X.reshape(",[170,2684,2661],{"class":2515},[170,2686,194],{"class":2563},[170,2688,2689],{"class":2523},", shape[",[170,2691,2661],{"class":2515},[170,2693,194],{"class":2563},[170,2695,2696],{"class":2523},"]), valid_lens,\n",[170,2698,2700,2703,2705,2708],{"class":2511,"line":2699},14,[170,2701,2702],{"class":2579},"                              value",[170,2704,2583],{"class":2515},[170,2706,2707],{"class":2563},"1e6",[170,2709,2588],{"class":2523},[170,2711,2713,2715,2718,2720,2722,2724],{"class":2511,"line":2712},15,[170,2714,2573],{"class":2515},[170,2716,2717],{"class":2523}," nn.functional.softmax(X.reshape(shape), ",[170,2719,2580],{"class":2579},[170,2721,2583],{"class":2515},[170,2723,194],{"class":2563},[170,2725,2588],{"class":2523},[14,2727,2728],{},"下面实现缩放点积注意力。",[2501,2730,2732],{"className":2503,"code":2731,"language":2505,"meta":89,"style":89},"class DotProductAttention(nn.Module):\n    \"\"\"缩放点积注意力\"\"\"\n    def __init__(self, dropout, **kwargs):\n        super(DotProductAttention, self).__init__(**kwargs)\n        self.dropout = nn.Dropout(dropout)\n\n    # queries的形状：(batch_size，查询的个数，d)\n    # keys的形状：(batch_size，“键－值”对的个数，d)\n    # values的形状：(batch_size，“键－值”对的个数，值的维度)\n    # valid_lens的形状:(batch_size，)或者(batch_size，查询的个数)\n    def forward(self, queries, keys, values, valid_lens=None):\n        d = queries.shape[-1]\n        # 设置transpose_b=True为了交换keys的最后两个维度\n        scores = torch.bmm(queries, keys.transpose(1,2)) / math.sqrt(d)\n        self.attention_weights = masked_softmax(scores, valid_lens)\n        return torch.bmm(self.dropout(self.attention_weights), values)\n",[2507,2733,2734,2756,2761,2790,2815,2828,2833,2838,2843,2848,2853,2890,2907,2912,2938,2950],{"__ignoreMap":89},[170,2735,2736,2739,2743,2745,2749,2751,2754],{"class":2511,"line":2512},[170,2737,2738],{"class":2515},"class",[170,2740,2742],{"class":2741},"skCyd"," DotProductAttention",[170,2744,909],{"class":2523},[170,2746,2748],{"class":2747},"sDP9b","nn",[170,2750,975],{"class":2523},[170,2752,2753],{"class":2747},"Module",[170,2755,2536],{"class":2523},[170,2757,2758],{"class":2511,"line":90},[170,2759,2760],{"class":2541},"    \"\"\"缩放点积注意力\"\"\"\n",[170,2762,2763,2766,2769,2771,2775,2777,2780,2782,2785,2788],{"class":2511,"line":96},[170,2764,2765],{"class":2515},"    def",[170,2767,2768],{"class":2563}," __init__",[170,2770,909],{"class":2523},[170,2772,2774],{"class":2773},"sD3jg","self",[170,2776,2530],{"class":2523},[170,2778,2779],{"class":2526},"dropout",[170,2781,2530],{"class":2523},[170,2783,2784],{"class":2515},"**",[170,2786,2787],{"class":2526},"kwargs",[170,2789,2536],{"class":2523},[170,2791,2792,2796,2799,2802,2805,2808,2810,2812],{"class":2511,"line":2551},[170,2793,2795],{"class":2794},"sPGBF","        super",[170,2797,2798],{"class":2523},"(DotProductAttention, ",[170,2800,2774],{"class":2801},"sJti5",[170,2803,2804],{"class":2523},").",[170,2806,2807],{"class":2563},"__init__",[170,2809,909],{"class":2523},[170,2811,2784],{"class":2515},[170,2813,2814],{"class":2523},"kwargs)\n",[170,2816,2817,2820,2823,2825],{"class":2511,"line":2570},[170,2818,2819],{"class":2801},"        self",[170,2821,2822],{"class":2523},".dropout ",[170,2824,926],{"class":2515},[170,2826,2827],{"class":2523}," nn.Dropout(dropout)\n",[170,2829,2830],{"class":2511,"line":2591},[170,2831,2832],{"emptyLinePlaceholder":114},"\n",[170,2834,2835],{"class":2511,"line":2599},[170,2836,2837],{"class":2547},"    # queries的形状：(batch_size，查询的个数，d)\n",[170,2839,2840],{"class":2511,"line":2610},[170,2841,2842],{"class":2547},"    # keys的形状：(batch_size，“键－值”对的个数，d)\n",[170,2844,2845],{"class":2511,"line":2627},[170,2846,2847],{"class":2547},"    # values的形状：(batch_size，“键－值”对的个数，值的维度)\n",[170,2849,2850],{"class":2511,"line":2643},[170,2851,2852],{"class":2547},"    # valid_lens的形状:(batch_size，)或者(batch_size，查询的个数)\n",[170,2854,2855,2857,2860,2862,2864,2866,2869,2871,2874,2876,2879,2881,2883,2885,2888],{"class":2511,"line":2651},[170,2856,2765],{"class":2515},[170,2858,2859],{"class":2519}," forward",[170,2861,909],{"class":2523},[170,2863,2774],{"class":2773},[170,2865,2530],{"class":2523},[170,2867,2868],{"class":2526},"queries",[170,2870,2530],{"class":2523},[170,2872,2873],{"class":2526},"keys",[170,2875,2530],{"class":2523},[170,2877,2878],{"class":2526},"values",[170,2880,2530],{"class":2523},[170,2882,2533],{"class":2526},[170,2884,926],{"class":2515},[170,2886,2887],{"class":2563},"None",[170,2889,2536],{"class":2523},[170,2891,2892,2895,2897,2900,2902,2904],{"class":2511,"line":2668},[170,2893,2894],{"class":2523},"        d ",[170,2896,926],{"class":2515},[170,2898,2899],{"class":2523}," queries.shape[",[170,2901,2661],{"class":2515},[170,2903,194],{"class":2563},[170,2905,2906],{"class":2523},"]\n",[170,2908,2909],{"class":2511,"line":2674},[170,2910,2911],{"class":2547},"        # 设置transpose_b=True为了交换keys的最后两个维度\n",[170,2913,2914,2917,2919,2922,2924,2926,2929,2932,2935],{"class":2511,"line":2699},[170,2915,2916],{"class":2523},"        scores ",[170,2918,926],{"class":2515},[170,2920,2921],{"class":2523}," torch.bmm(queries, keys.transpose(",[170,2923,194],{"class":2563},[170,2925,914],{"class":2523},[170,2927,2928],{"class":2563},"2",[170,2930,2931],{"class":2523},")) ",[170,2933,2934],{"class":2515},"/",[170,2936,2937],{"class":2523}," math.sqrt(d)\n",[170,2939,2940,2942,2945,2947],{"class":2511,"line":2712},[170,2941,2819],{"class":2801},[170,2943,2944],{"class":2523},".attention_weights ",[170,2946,926],{"class":2515},[170,2948,2949],{"class":2523}," masked_softmax(scores, valid_lens)\n",[170,2951,2953,2955,2958,2960,2963,2965],{"class":2511,"line":2952},16,[170,2954,2573],{"class":2515},[170,2956,2957],{"class":2523}," torch.bmm(",[170,2959,2774],{"class":2801},[170,2961,2962],{"class":2523},".dropout(",[170,2964,2774],{"class":2801},[170,2966,2967],{"class":2523},".attention_weights), values)\n",[14,2969,2970,2971,2974],{},"代码中函数的 ",[2507,2972,2973],{},"**kwargs","表示任意关键字参数，为了符合 Pytorch 的继承约定。",[24,2976,2977],{"id":2977},"多头注意力",[14,2979,2980],{},"多头注意力机制允许模型在不同位置上同时关注来自不同表示子空间的信息。使用单一注意力头时，平均操作会抑制这种能力。",[14,2982,2983],{},[153,2984],{"alt":89,"src":2985},"transformer/MHA.png",[14,2987,2988],{},[170,2989,2991,3107],{"className":2990},[173],[170,2992,2994],{"className":2993},[177],[179,2995,2996],{"xmlns":181},[183,2997,2998,3104],{},[186,2999,3000,3024,3026,3028,3030,3032,3034,3036,3038,3040,3056,3058,3073,3075,3078,3080,3094,3096],{},[186,3001,3002,3005,3008,3011,3013,3015,3018,3020,3022],{},[202,3003,3004],{"mathvariant":929},"M",[202,3006,3007],{"mathvariant":929},"u",[202,3009,3010],{"mathvariant":929},"l",[202,3012,888],{"mathvariant":929},[202,3014,900],{"mathvariant":929},[202,3016,3017],{"mathvariant":929},"H",[202,3019,893],{"mathvariant":929},[202,3021,134],{"mathvariant":929},[202,3023,204],{"mathvariant":929},[572,3025,909],{"stretchy":908},[202,3027,570],{},[572,3029,914],{"separator":217},[202,3031,683],{},[572,3033,914],{"separator":217},[202,3035,783],{},[572,3037,923],{"stretchy":908},[572,3039,926],{},[186,3041,3042,3045,3047,3049,3052,3054],{},[202,3043,3044],{"mathvariant":929},"C",[202,3046,903],{"mathvariant":929},[202,3048,446],{"mathvariant":929},[202,3050,3051],{"mathvariant":929},"c",[202,3053,134],{"mathvariant":929},[202,3055,888],{"mathvariant":929},[572,3057,909],{"stretchy":908},[199,3059,3060,3071],{},[186,3061,3062,3065,3067,3069],{},[202,3063,3064],{"mathvariant":929},"h",[202,3066,893],{"mathvariant":929},[202,3068,134],{"mathvariant":929},[202,3070,204],{"mathvariant":929},[192,3072,194],{},[572,3074,914],{"separator":217},[572,3076,3077],{},"…",[572,3079,914],{"separator":217},[199,3081,3082,3092],{},[186,3083,3084,3086,3088,3090],{},[202,3085,3064],{"mathvariant":929},[202,3087,893],{"mathvariant":929},[202,3089,134],{"mathvariant":929},[202,3091,204],{"mathvariant":929},[202,3093,3064],{},[572,3095,923],{"stretchy":908},[576,3097,3098,3101],{},[202,3099,3100],{},"W",[202,3102,3103],{},"O",[209,3105,3106],{"encoding":211},"\\mathrm{MultiHead}(Q, K, V) = \\mathrm{Concat}(\\mathrm{head}_1, \\ldots, \\mathrm{head}_h) W^O",[170,3108,3110,3159],{"className":3109,"ariaHidden":217},[216],[170,3111,3113,3116,3123,3126,3129,3132,3135,3138,3141,3144,3147,3150,3153,3156],{"className":3112},[221],[170,3114],{"className":3115,"style":988},[225],[170,3117,3119],{"className":3118},[230],[170,3120,3122],{"className":3121},[230,1066],"MultiHead",[170,3124,909],{"className":3125},[234],[170,3127,570],{"className":3128},[230,303],[170,3130,914],{"className":3131},[1023],[170,3133],{"className":3134,"style":1027},[610],[170,3136,683],{"className":3137,"style":714},[230,303],[170,3139,914],{"className":3140},[1023],[170,3142],{"className":3143,"style":1027},[610],[170,3145,783],{"className":3146,"style":813},[230,303],[170,3148,923],{"className":3149},[424],[170,3151],{"className":3152,"style":611},[610],[170,3154,926],{"className":3155},[615],[170,3157],{"className":3158,"style":611},[610],[170,3160,3162,3166,3173,3176,3222,3225,3228,3231,3234,3237,3240,3283,3286],{"className":3161},[221],[170,3163],{"className":3164,"style":3165},[225],"height:1.0913em;vertical-align:-0.25em;",[170,3167,3169],{"className":3168},[230],[170,3170,3172],{"className":3171},[230,1066],"Concat",[170,3174,909],{"className":3175},[234],[170,3177,3179,3186],{"className":3178},[230],[170,3180,3182],{"className":3181},[230],[170,3183,3185],{"className":3184},[230,1066],"head",[170,3187,3189],{"className":3188},[307],[170,3190,3192,3214],{"className":3191},[242,243],[170,3193,3195,3211],{"className":3194},[247],[170,3196,3199],{"className":3197,"style":3198},[251],"height:0.3011em;",[170,3200,3202,3205],{"style":3201},"top:-2.55em;margin-right:0.05em;",[170,3203],{"className":3204,"style":651},[259],[170,3206,3208],{"className":3207},[264,265,266,267],[170,3209,194],{"className":3210},[230,267],[170,3212,338],{"className":3213},[337],[170,3215,3217],{"className":3216},[247],[170,3218,3220],{"className":3219,"style":1381},[251],[170,3221],{},[170,3223,914],{"className":3224},[1023],[170,3226],{"className":3227,"style":1027},[610],[170,3229,3077],{"className":3230},[1073],[170,3232],{"className":3233,"style":1027},[610],[170,3235,914],{"className":3236},[1023],[170,3238],{"className":3239,"style":1027},[610],[170,3241,3243,3249],{"className":3242},[230],[170,3244,3246],{"className":3245},[230],[170,3247,3185],{"className":3248},[230,1066],[170,3250,3252],{"className":3251},[307],[170,3253,3255,3275],{"className":3254},[242,243],[170,3256,3258,3272],{"className":3257},[247],[170,3259,3261],{"className":3260,"style":1359},[251],[170,3262,3263,3266],{"style":3201},[170,3264],{"className":3265,"style":651},[259],[170,3267,3269],{"className":3268},[264,265,266,267],[170,3270,3064],{"className":3271},[230,303,267],[170,3273,338],{"className":3274},[337],[170,3276,3278],{"className":3277},[247],[170,3279,3281],{"className":3280,"style":1381},[251],[170,3282],{},[170,3284,923],{"className":3285},[424],[170,3287,3289,3293],{"className":3288},[230],[170,3290,3100],{"className":3291,"style":3292},[230,303],"margin-right:0.13889em;",[170,3294,3296],{"className":3295},[307],[170,3297,3299],{"className":3298},[242],[170,3300,3302],{"className":3301},[247],[170,3303,3306],{"className":3304,"style":3305},[251],"height:0.8413em;",[170,3307,3308,3311],{"style":647},[170,3309],{"className":3310,"style":651},[259],[170,3312,3314],{"className":3313},[264,265,266,267],[170,3315,3103],{"className":3316,"style":3317},[230,303,267],"margin-right:0.02778em;",[14,3319,3320,3321,3683,3684,3933,3934,3933,4183,4434,4435,975],{},"其中 ",[170,3322,3324,3416],{"className":3323},[173],[170,3325,3327],{"className":3326},[177],[179,3328,3329],{"xmlns":181},[183,3330,3331,3413],{},[186,3332,3333,3347,3349,3369,3371,3373,3381,3383,3387,3389,3397,3399,3401,3403,3411],{},[199,3334,3335,3345],{},[186,3336,3337,3339,3341,3343],{},[202,3338,3064],{"mathvariant":929},[202,3340,893],{"mathvariant":929},[202,3342,134],{"mathvariant":929},[202,3344,204],{"mathvariant":929},[202,3346,900],{},[572,3348,926],{},[186,3350,3351,3353,3355,3357,3359,3361,3363,3365,3367],{},[202,3352,885],{"mathvariant":929},[202,3354,888],{"mathvariant":929},[202,3356,888],{"mathvariant":929},[202,3358,893],{"mathvariant":929},[202,3360,446],{"mathvariant":929},[202,3362,888],{"mathvariant":929},[202,3364,900],{"mathvariant":929},[202,3366,903],{"mathvariant":929},[202,3368,446],{"mathvariant":929},[572,3370,909],{"stretchy":908},[202,3372,570],{},[1933,3374,3375,3377,3379],{},[202,3376,3100],{},[202,3378,900],{},[202,3380,570],{},[572,3382,914],{"separator":217},[3384,3385,3386],"mtext",{},"  ",[202,3388,683],{},[1933,3390,3391,3393,3395],{},[202,3392,3100],{},[202,3394,900],{},[202,3396,683],{},[572,3398,914],{"separator":217},[3384,3400,3386],{},[202,3402,783],{},[1933,3404,3405,3407,3409],{},[202,3406,3100],{},[202,3408,900],{},[202,3410,783],{},[572,3412,923],{"stretchy":908},[209,3414,3415],{"encoding":211},"\\mathrm{head}_i=\\mathrm{Attention}(QW_i^Q,\\; KW_i^K,\\; VW_i^V)",[170,3417,3419,3477],{"className":3418,"ariaHidden":217},[216],[170,3420,3422,3425,3468,3471,3474],{"className":3421},[221],[170,3423],{"className":3424,"style":1340},[225],[170,3426,3428,3434],{"className":3427},[230],[170,3429,3431],{"className":3430},[230],[170,3432,3185],{"className":3433},[230,1066],[170,3435,3437],{"className":3436},[307],[170,3438,3440,3460],{"className":3439},[242,243],[170,3441,3443,3457],{"className":3442},[247],[170,3444,3446],{"className":3445,"style":2143},[251],[170,3447,3448,3451],{"style":3201},[170,3449],{"className":3450,"style":651},[259],[170,3452,3454],{"className":3453},[264,265,266,267],[170,3455,900],{"className":3456},[230,303,267],[170,3458,338],{"className":3459},[337],[170,3461,3463],{"className":3462},[247],[170,3464,3466],{"className":3465,"style":1381},[251],[170,3467],{},[170,3469],{"className":3470,"style":611},[610],[170,3472,926],{"className":3473},[615],[170,3475],{"className":3476,"style":611},[610],[170,3478,3480,3484,3491,3494,3497,3552,3555,3558,3561,3564,3617,3620,3623,3626,3629,3680],{"className":3479},[221],[170,3481],{"className":3482,"style":3483},[225],"height:1.2361em;vertical-align:-0.2769em;",[170,3485,3487],{"className":3486},[230],[170,3488,3490],{"className":3489},[230,1066],"Attention",[170,3492,909],{"className":3493},[234],[170,3495,570],{"className":3496},[230,303],[170,3498,3500,3503],{"className":3499},[230],[170,3501,3100],{"className":3502,"style":3292},[230,303],[170,3504,3506],{"className":3505},[307],[170,3507,3509,3543],{"className":3508},[242,243],[170,3510,3512,3540],{"className":3511},[247],[170,3513,3516,3528],{"className":3514,"style":3515},[251],"height:0.9592em;",[170,3517,3519,3522],{"style":3518},"top:-2.4231em;margin-left:-0.1389em;margin-right:0.05em;",[170,3520],{"className":3521,"style":651},[259],[170,3523,3525],{"className":3524},[264,265,266,267],[170,3526,900],{"className":3527},[230,303,267],[170,3529,3531,3534],{"style":3530},"top:-3.1809em;margin-right:0.05em;",[170,3532],{"className":3533,"style":651},[259],[170,3535,3537],{"className":3536},[264,265,266,267],[170,3538,570],{"className":3539},[230,303,267],[170,3541,338],{"className":3542},[337],[170,3544,3546],{"className":3545},[247],[170,3547,3550],{"className":3548,"style":3549},[251],"height:0.2769em;",[170,3551],{},[170,3553,914],{"className":3554},[1023],[170,3556],{"className":3557,"style":611},[610],[170,3559],{"className":3560,"style":1027},[610],[170,3562,683],{"className":3563,"style":714},[230,303],[170,3565,3567,3570],{"className":3566},[230],[170,3568,3100],{"className":3569,"style":3292},[230,303],[170,3571,3573],{"className":3572},[307],[170,3574,3576,3608],{"className":3575},[242,243],[170,3577,3579,3605],{"className":3578},[247],[170,3580,3582,3594],{"className":3581,"style":3305},[251],[170,3583,3585,3588],{"style":3584},"top:-2.4413em;margin-left:-0.1389em;margin-right:0.05em;",[170,3586],{"className":3587,"style":651},[259],[170,3589,3591],{"className":3590},[264,265,266,267],[170,3592,900],{"className":3593},[230,303,267],[170,3595,3596,3599],{"style":647},[170,3597],{"className":3598,"style":651},[259],[170,3600,3602],{"className":3601},[264,265,266,267],[170,3603,683],{"className":3604,"style":714},[230,303,267],[170,3606,338],{"className":3607},[337],[170,3609,3611],{"className":3610},[247],[170,3612,3615],{"className":3613,"style":3614},[251],"height:0.2587em;",[170,3616],{},[170,3618,914],{"className":3619},[1023],[170,3621],{"className":3622,"style":611},[610],[170,3624],{"className":3625,"style":1027},[610],[170,3627,783],{"className":3628,"style":813},[230,303],[170,3630,3632,3635],{"className":3631},[230],[170,3633,3100],{"className":3634,"style":3292},[230,303],[170,3636,3638],{"className":3637},[307],[170,3639,3641,3672],{"className":3640},[242,243],[170,3642,3644,3669],{"className":3643},[247],[170,3645,3647,3658],{"className":3646,"style":3305},[251],[170,3648,3649,3652],{"style":3584},[170,3650],{"className":3651,"style":651},[259],[170,3653,3655],{"className":3654},[264,265,266,267],[170,3656,900],{"className":3657},[230,303,267],[170,3659,3660,3663],{"style":647},[170,3661],{"className":3662,"style":651},[259],[170,3664,3666],{"className":3665},[264,265,266,267],[170,3667,783],{"className":3668,"style":813},[230,303,267],[170,3670,338],{"className":3671},[337],[170,3673,3675],{"className":3674},[247],[170,3676,3678],{"className":3677,"style":3614},[251],[170,3679],{},[170,3681,923],{"className":3682},[424],"，投影参数是矩阵\n",[170,3685,3687,3739],{"className":3686},[173],[170,3688,3690],{"className":3689},[177],[179,3691,3692],{"xmlns":181},[183,3693,3694,3736],{},[186,3695,3696,3704,3706],{},[1933,3697,3698,3700,3702],{},[202,3699,3100],{},[202,3701,900],{},[202,3703,570],{},[572,3705,574],{},[576,3707,3708,3710],{},[202,3709,581],{"mathvariant":580},[186,3711,3712,3728,3730],{},[199,3713,3714,3716],{},[202,3715,204],{},[186,3717,3718,3720,3722,3724,3726],{},[202,3719,477],{"mathvariant":929},[202,3721,903],{"mathvariant":929},[202,3723,204],{"mathvariant":929},[202,3725,893],{"mathvariant":929},[202,3727,3010],{"mathvariant":929},[572,3729,588],{},[199,3731,3732,3734],{},[202,3733,204],{},[202,3735,207],{},[209,3737,3738],{"encoding":211},"W_i^Q \\in \\mathbb{R}^{d_{\\mathrm{model}} \\times d_k}",[170,3740,3742,3808],{"className":3741,"ariaHidden":217},[216],[170,3743,3745,3748,3799,3802,3805],{"className":3744},[221],[170,3746],{"className":3747,"style":3483},[225],[170,3749,3751,3754],{"className":3750},[230],[170,3752,3100],{"className":3753,"style":3292},[230,303],[170,3755,3757],{"className":3756},[307],[170,3758,3760,3791],{"className":3759},[242,243],[170,3761,3763,3788],{"className":3762},[247],[170,3764,3766,3777],{"className":3765,"style":3515},[251],[170,3767,3768,3771],{"style":3518},[170,3769],{"className":3770,"style":651},[259],[170,3772,3774],{"className":3773},[264,265,266,267],[170,3775,900],{"className":3776},[230,303,267],[170,3778,3779,3782],{"style":3530},[170,3780],{"className":3781,"style":651},[259],[170,3783,3785],{"className":3784},[264,265,266,267],[170,3786,570],{"className":3787},[230,303,267],[170,3789,338],{"className":3790},[337],[170,3792,3794],{"className":3793},[247],[170,3795,3797],{"className":3796,"style":3549},[251],[170,3798],{},[170,3800],{"className":3801,"style":611},[610],[170,3803,574],{"className":3804},[615],[170,3806],{"className":3807,"style":611},[610],[170,3809,3811,3814],{"className":3810},[221],[170,3812],{"className":3813,"style":625},[225],[170,3815,3817,3820],{"className":3816},[230],[170,3818,581],{"className":3819},[230,632],[170,3821,3823],{"className":3822},[307],[170,3824,3826],{"className":3825},[242],[170,3827,3829],{"className":3828},[247],[170,3830,3832],{"className":3831,"style":625},[251],[170,3833,3834,3837],{"style":647},[170,3835],{"className":3836,"style":651},[259],[170,3838,3840],{"className":3839},[264,265,266,267],[170,3841,3843,3890,3893],{"className":3842},[230,267],[170,3844,3846,3849],{"className":3845},[230,267],[170,3847,204],{"className":3848},[230,303,267],[170,3850,3852],{"className":3851},[307],[170,3853,3855,3882],{"className":3854},[242,243],[170,3856,3858,3879],{"className":3857},[247],[170,3859,3861],{"className":3860,"style":317},[251],[170,3862,3863,3866],{"style":320},[170,3864],{"className":3865,"style":324},[259],[170,3867,3869],{"className":3868},[264,328,329,267],[170,3870,3872],{"className":3871},[230,267],[170,3873,3875],{"className":3874},[230,267],[170,3876,3878],{"className":3877},[230,1066,267],"model",[170,3880,338],{"className":3881},[337],[170,3883,3885],{"className":3884},[247],[170,3886,3888],{"className":3887,"style":345},[251],[170,3889],{},[170,3891,588],{"className":3892},[664,267],[170,3894,3896,3899],{"className":3895},[230,267],[170,3897,204],{"className":3898},[230,303,267],[170,3900,3902],{"className":3901},[307],[170,3903,3905,3925],{"className":3904},[242,243],[170,3906,3908,3922],{"className":3907},[247],[170,3909,3911],{"className":3910,"style":317},[251],[170,3912,3913,3916],{"style":320},[170,3914],{"className":3915,"style":324},[259],[170,3917,3919],{"className":3918},[264,328,329,267],[170,3920,207],{"className":3921,"style":333},[230,303,267],[170,3923,338],{"className":3924},[337],[170,3926,3928],{"className":3927},[247],[170,3929,3931],{"className":3930,"style":345},[251],[170,3932],{},",\n",[170,3935,3937,3989],{"className":3936},[173],[170,3938,3940],{"className":3939},[177],[179,3941,3942],{"xmlns":181},[183,3943,3944,3986],{},[186,3945,3946,3954,3956],{},[1933,3947,3948,3950,3952],{},[202,3949,3100],{},[202,3951,900],{},[202,3953,683],{},[572,3955,574],{},[576,3957,3958,3960],{},[202,3959,581],{"mathvariant":580},[186,3961,3962,3978,3980],{},[199,3963,3964,3966],{},[202,3965,204],{},[186,3967,3968,3970,3972,3974,3976],{},[202,3969,477],{"mathvariant":929},[202,3971,903],{"mathvariant":929},[202,3973,204],{"mathvariant":929},[202,3975,893],{"mathvariant":929},[202,3977,3010],{"mathvariant":929},[572,3979,588],{},[199,3981,3982,3984],{},[202,3983,204],{},[202,3985,207],{},[209,3987,3988],{"encoding":211},"W_i^K \\in \\mathbb{R}^{d_{\\mathrm{model}} \\times d_k}",[170,3990,3992,4059],{"className":3991,"ariaHidden":217},[216],[170,3993,3995,3999,4050,4053,4056],{"className":3994},[221],[170,3996],{"className":3997,"style":3998},[225],"height:1.1em;vertical-align:-0.2587em;",[170,4000,4002,4005],{"className":4001},[230],[170,4003,3100],{"className":4004,"style":3292},[230,303],[170,4006,4008],{"className":4007},[307],[170,4009,4011,4042],{"className":4010},[242,243],[170,4012,4014,4039],{"className":4013},[247],[170,4015,4017,4028],{"className":4016,"style":3305},[251],[170,4018,4019,4022],{"style":3584},[170,4020],{"className":4021,"style":651},[259],[170,4023,4025],{"className":4024},[264,265,266,267],[170,4026,900],{"className":4027},[230,303,267],[170,4029,4030,4033],{"style":647},[170,4031],{"className":4032,"style":651},[259],[170,4034,4036],{"className":4035},[264,265,266,267],[170,4037,683],{"className":4038,"style":714},[230,303,267],[170,4040,338],{"className":4041},[337],[170,4043,4045],{"className":4044},[247],[170,4046,4048],{"className":4047,"style":3614},[251],[170,4049],{},[170,4051],{"className":4052,"style":611},[610],[170,4054,574],{"className":4055},[615],[170,4057],{"className":4058,"style":611},[610],[170,4060,4062,4065],{"className":4061},[221],[170,4063],{"className":4064,"style":625},[225],[170,4066,4068,4071],{"className":4067},[230],[170,4069,581],{"className":4070},[230,632],[170,4072,4074],{"className":4073},[307],[170,4075,4077],{"className":4076},[242],[170,4078,4080],{"className":4079},[247],[170,4081,4083],{"className":4082,"style":625},[251],[170,4084,4085,4088],{"style":647},[170,4086],{"className":4087,"style":651},[259],[170,4089,4091],{"className":4090},[264,265,266,267],[170,4092,4094,4140,4143],{"className":4093},[230,267],[170,4095,4097,4100],{"className":4096},[230,267],[170,4098,204],{"className":4099},[230,303,267],[170,4101,4103],{"className":4102},[307],[170,4104,4106,4132],{"className":4105},[242,243],[170,4107,4109,4129],{"className":4108},[247],[170,4110,4112],{"className":4111,"style":317},[251],[170,4113,4114,4117],{"style":320},[170,4115],{"className":4116,"style":324},[259],[170,4118,4120],{"className":4119},[264,328,329,267],[170,4121,4123],{"className":4122},[230,267],[170,4124,4126],{"className":4125},[230,267],[170,4127,3878],{"className":4128},[230,1066,267],[170,4130,338],{"className":4131},[337],[170,4133,4135],{"className":4134},[247],[170,4136,4138],{"className":4137,"style":345},[251],[170,4139],{},[170,4141,588],{"className":4142},[664,267],[170,4144,4146,4149],{"className":4145},[230,267],[170,4147,204],{"className":4148},[230,303,267],[170,4150,4152],{"className":4151},[307],[170,4153,4155,4175],{"className":4154},[242,243],[170,4156,4158,4172],{"className":4157},[247],[170,4159,4161],{"className":4160,"style":317},[251],[170,4162,4163,4166],{"style":320},[170,4164],{"className":4165,"style":324},[259],[170,4167,4169],{"className":4168},[264,328,329,267],[170,4170,207],{"className":4171,"style":333},[230,303,267],[170,4173,338],{"className":4174},[337],[170,4176,4178],{"className":4177},[247],[170,4179,4181],{"className":4180,"style":345},[251],[170,4182],{},[170,4184,4186,4238],{"className":4185},[173],[170,4187,4189],{"className":4188},[177],[179,4190,4191],{"xmlns":181},[183,4192,4193,4235],{},[186,4194,4195,4203,4205],{},[1933,4196,4197,4199,4201],{},[202,4198,3100],{},[202,4200,900],{},[202,4202,783],{},[572,4204,574],{},[576,4206,4207,4209],{},[202,4208,581],{"mathvariant":580},[186,4210,4211,4227,4229],{},[199,4212,4213,4215],{},[202,4214,204],{},[186,4216,4217,4219,4221,4223,4225],{},[202,4218,477],{"mathvariant":929},[202,4220,903],{"mathvariant":929},[202,4222,204],{"mathvariant":929},[202,4224,893],{"mathvariant":929},[202,4226,3010],{"mathvariant":929},[572,4228,588],{},[199,4230,4231,4233],{},[202,4232,204],{},[202,4234,537],{},[209,4236,4237],{"encoding":211},"W_i^V \\in \\mathbb{R}^{d_{\\mathrm{model}} \\times d_v}",[170,4239,4241,4307],{"className":4240,"ariaHidden":217},[216],[170,4242,4244,4247,4298,4301,4304],{"className":4243},[221],[170,4245],{"className":4246,"style":3998},[225],[170,4248,4250,4253],{"className":4249},[230],[170,4251,3100],{"className":4252,"style":3292},[230,303],[170,4254,4256],{"className":4255},[307],[170,4257,4259,4290],{"className":4258},[242,243],[170,4260,4262,4287],{"className":4261},[247],[170,4263,4265,4276],{"className":4264,"style":3305},[251],[170,4266,4267,4270],{"style":3584},[170,4268],{"className":4269,"style":651},[259],[170,4271,4273],{"className":4272},[264,265,266,267],[170,4274,900],{"className":4275},[230,303,267],[170,4277,4278,4281],{"style":647},[170,4279],{"className":4280,"style":651},[259],[170,4282,4284],{"className":4283},[264,265,266,267],[170,4285,783],{"className":4286,"style":813},[230,303,267],[170,4288,338],{"className":4289},[337],[170,4291,4293],{"className":4292},[247],[170,4294,4296],{"className":4295,"style":3614},[251],[170,4297],{},[170,4299],{"className":4300,"style":611},[610],[170,4302,574],{"className":4303},[615],[170,4305],{"className":4306,"style":611},[610],[170,4308,4310,4313],{"className":4309},[221],[170,4311],{"className":4312,"style":625},[225],[170,4314,4316,4319],{"className":4315},[230],[170,4317,581],{"className":4318},[230,632],[170,4320,4322],{"className":4321},[307],[170,4323,4325],{"className":4324},[242],[170,4326,4328],{"className":4327},[247],[170,4329,4331],{"className":4330,"style":625},[251],[170,4332,4333,4336],{"style":647},[170,4334],{"className":4335,"style":651},[259],[170,4337,4339],{"className":4338},[264,265,266,267],[170,4340,4342,4388,4391],{"className":4341},[230,267],[170,4343,4345,4348],{"className":4344},[230,267],[170,4346,204],{"className":4347},[230,303,267],[170,4349,4351],{"className":4350},[307],[170,4352,4354,4380],{"className":4353},[242,243],[170,4355,4357,4377],{"className":4356},[247],[170,4358,4360],{"className":4359,"style":317},[251],[170,4361,4362,4365],{"style":320},[170,4363],{"className":4364,"style":324},[259],[170,4366,4368],{"className":4367},[264,328,329,267],[170,4369,4371],{"className":4370},[230,267],[170,4372,4374],{"className":4373},[230,267],[170,4375,3878],{"className":4376},[230,1066,267],[170,4378,338],{"className":4379},[337],[170,4381,4383],{"className":4382},[247],[170,4384,4386],{"className":4385,"style":345},[251],[170,4387],{},[170,4389,588],{"className":4390},[664,267],[170,4392,4394,4397],{"className":4393},[230,267],[170,4395,204],{"className":4396},[230,303,267],[170,4398,4400],{"className":4399},[307],[170,4401,4403,4425],{"className":4402},[242,243],[170,4404,4406,4422],{"className":4405},[247],[170,4407,4410],{"className":4408,"style":4409},[251],"height:0.1645em;",[170,4411,4413,4416],{"style":4412},"top:-2.357em;margin-left:0em;margin-right:0.0714em;",[170,4414],{"className":4415,"style":324},[259],[170,4417,4419],{"className":4418},[264,328,329,267],[170,4420,537],{"className":4421,"style":552},[230,303,267],[170,4423,338],{"className":4424},[337],[170,4426,4428],{"className":4427},[247],[170,4429,4432],{"className":4430,"style":4431},[251],"height:0.143em;",[170,4433],{},"\n和\n",[170,4436,4438,4490],{"className":4437},[173],[170,4439,4441],{"className":4440},[177],[179,4442,4443],{"xmlns":181},[183,4444,4445,4487],{},[186,4446,4447,4453,4455],{},[576,4448,4449,4451],{},[202,4450,3100],{},[202,4452,3103],{},[572,4454,574],{},[576,4456,4457,4459],{},[202,4458,581],{"mathvariant":580},[186,4460,4461,4463,4469,4471],{},[202,4462,3064],{},[199,4464,4465,4467],{},[202,4466,204],{},[202,4468,537],{},[572,4470,588],{},[199,4472,4473,4475],{},[202,4474,204],{},[186,4476,4477,4479,4481,4483,4485],{},[202,4478,477],{"mathvariant":929},[202,4480,903],{"mathvariant":929},[202,4482,204],{"mathvariant":929},[202,4484,893],{"mathvariant":929},[202,4486,3010],{"mathvariant":929},[209,4488,4489],{"encoding":211},"W^O \\in \\mathbb{R}^{h d_v \\times d_{\\mathrm{model}}}",[170,4491,4493,4538],{"className":4492,"ariaHidden":217},[216],[170,4494,4496,4500,4529,4532,4535],{"className":4495},[221],[170,4497],{"className":4498,"style":4499},[225],"height:0.8804em;vertical-align:-0.0391em;",[170,4501,4503,4506],{"className":4502},[230],[170,4504,3100],{"className":4505,"style":3292},[230,303],[170,4507,4509],{"className":4508},[307],[170,4510,4512],{"className":4511},[242],[170,4513,4515],{"className":4514},[247],[170,4516,4518],{"className":4517,"style":3305},[251],[170,4519,4520,4523],{"style":647},[170,4521],{"className":4522,"style":651},[259],[170,4524,4526],{"className":4525},[264,265,266,267],[170,4527,3103],{"className":4528,"style":3317},[230,303,267],[170,4530],{"className":4531,"style":611},[610],[170,4533,574],{"className":4534},[615],[170,4536],{"className":4537,"style":611},[610],[170,4539,4541,4544],{"className":4540},[221],[170,4542],{"className":4543,"style":625},[225],[170,4545,4547,4550],{"className":4546},[230],[170,4548,581],{"className":4549},[230,632],[170,4551,4553],{"className":4552},[307],[170,4554,4556],{"className":4555},[242],[170,4557,4559],{"className":4558},[247],[170,4560,4562],{"className":4561,"style":625},[251],[170,4563,4564,4567],{"style":647},[170,4565],{"className":4566,"style":651},[259],[170,4568,4570],{"className":4569},[264,265,266,267],[170,4571,4573,4576,4616,4619],{"className":4572},[230,267],[170,4574,3064],{"className":4575},[230,303,267],[170,4577,4579,4582],{"className":4578},[230,267],[170,4580,204],{"className":4581},[230,303,267],[170,4583,4585],{"className":4584},[307],[170,4586,4588,4608],{"className":4587},[242,243],[170,4589,4591,4605],{"className":4590},[247],[170,4592,4594],{"className":4593,"style":4409},[251],[170,4595,4596,4599],{"style":4412},[170,4597],{"className":4598,"style":324},[259],[170,4600,4602],{"className":4601},[264,328,329,267],[170,4603,537],{"className":4604,"style":552},[230,303,267],[170,4606,338],{"className":4607},[337],[170,4609,4611],{"className":4610},[247],[170,4612,4614],{"className":4613,"style":4431},[251],[170,4615],{},[170,4617,588],{"className":4618},[664,267],[170,4620,4622,4625],{"className":4621},[230,267],[170,4623,204],{"className":4624},[230,303,267],[170,4626,4628],{"className":4627},[307],[170,4629,4631,4657],{"className":4630},[242,243],[170,4632,4634,4654],{"className":4633},[247],[170,4635,4637],{"className":4636,"style":317},[251],[170,4638,4639,4642],{"style":320},[170,4640],{"className":4641,"style":324},[259],[170,4643,4645],{"className":4644},[264,328,329,267],[170,4646,4648],{"className":4647},[230,267],[170,4649,4651],{"className":4650},[230,267],[170,4652,3878],{"className":4653},[230,1066,267],[170,4655,338],{"className":4656},[337],[170,4658,4660],{"className":4659},[247],[170,4661,4663],{"className":4662,"style":345},[251],[170,4664],{},[14,4666,4667,4668,4671,4672,4675,4676,4678,4679,4681],{},"为了能使多个头并行计算，我们定义两个转置函数 ",[2507,4669,4670],{},"transpose_output"," 和 ",[2507,4673,4674],{},"transpose_qkv","，其中 ",[2507,4677,4670],{}," 还原了 ",[2507,4680,4674],{}," 的结果。",[2501,4683,4685],{"className":2503,"code":4684,"language":2505,"meta":89,"style":89},"def transpose_qkv(X, num_heads):\n    \"\"\"为了多注意力头的并行计算而变换形状\"\"\"\n    # 输入X的形状:(batch_size，查询或者“键－值”对的个数，num_hiddens)\n    # 输出X的形状:(batch_size，查询或者“键－值”对的个数，num_heads，num_hiddens/num_heads)\n    X = X.reshape(X.shape[0], X.shape[1], num_heads, -1)\n\n    # 输出X的形状:(batch_size，num_heads，查询或者“键－值”对的个数, num_hiddens/num_heads)\n    X = X.permute(0, 2, 1, 3)\n\n    # 最终输出的形状:(batch_size*num_heads,查询或者“键－值”对的个数, num_hiddens/num_heads)\n    return X.reshape(-1, X.shape[2], X.shape[3])\n\ndef transpose_output(X, num_heads):\n    \"\"\"逆转transpose_qkv函数的操作\"\"\"\n    X = X.reshape(-1, num_heads, X.shape[1], X.shape[2])\n    X = X.permute(0, 2, 1, 3)\n    return X.reshape(X.shape[0], X.shape[1], -1)\n",[2507,4686,4687,4705,4710,4715,4720,4746,4750,4755,4781,4785,4790,4813,4817,4834,4839,4862,4886],{"__ignoreMap":89},[170,4688,4689,4691,4694,4696,4698,4700,4703],{"class":2511,"line":2512},[170,4690,2516],{"class":2515},[170,4692,4693],{"class":2519}," transpose_qkv",[170,4695,909],{"class":2523},[170,4697,2527],{"class":2526},[170,4699,2530],{"class":2523},[170,4701,4702],{"class":2526},"num_heads",[170,4704,2536],{"class":2523},[170,4706,4707],{"class":2511,"line":90},[170,4708,4709],{"class":2541},"    \"\"\"为了多注意力头的并行计算而变换形状\"\"\"\n",[170,4711,4712],{"class":2511,"line":96},[170,4713,4714],{"class":2547},"    # 输入X的形状:(batch_size，查询或者“键－值”对的个数，num_hiddens)\n",[170,4716,4717],{"class":2511,"line":2551},[170,4718,4719],{"class":2547},"    # 输出X的形状:(batch_size，查询或者“键－值”对的个数，num_heads，num_hiddens/num_heads)\n",[170,4721,4722,4725,4727,4730,4732,4735,4737,4740,4742,4744],{"class":2511,"line":2570},[170,4723,4724],{"class":2523},"    X ",[170,4726,926],{"class":2515},[170,4728,4729],{"class":2523}," X.reshape(X.shape[",[170,4731,1865],{"class":2563},[170,4733,4734],{"class":2523},"], X.shape[",[170,4736,194],{"class":2563},[170,4738,4739],{"class":2523},"], num_heads, ",[170,4741,2661],{"class":2515},[170,4743,194],{"class":2563},[170,4745,2588],{"class":2523},[170,4747,4748],{"class":2511,"line":2591},[170,4749,2832],{"emptyLinePlaceholder":114},[170,4751,4752],{"class":2511,"line":2599},[170,4753,4754],{"class":2547},"    # 输出X的形状:(batch_size，num_heads，查询或者“键－值”对的个数, num_hiddens/num_heads)\n",[170,4756,4757,4759,4761,4764,4766,4768,4770,4772,4774,4776,4779],{"class":2511,"line":2610},[170,4758,4724],{"class":2523},[170,4760,926],{"class":2515},[170,4762,4763],{"class":2523}," X.permute(",[170,4765,1865],{"class":2563},[170,4767,2530],{"class":2523},[170,4769,2928],{"class":2563},[170,4771,2530],{"class":2523},[170,4773,194],{"class":2563},[170,4775,2530],{"class":2523},[170,4777,4778],{"class":2563},"3",[170,4780,2588],{"class":2523},[170,4782,4783],{"class":2511,"line":2627},[170,4784,2832],{"emptyLinePlaceholder":114},[170,4786,4787],{"class":2511,"line":2643},[170,4788,4789],{"class":2547},"    # 最终输出的形状:(batch_size*num_heads,查询或者“键－值”对的个数, num_hiddens/num_heads)\n",[170,4791,4792,4795,4798,4800,4802,4805,4807,4809,4811],{"class":2511,"line":2651},[170,4793,4794],{"class":2515},"    return",[170,4796,4797],{"class":2523}," X.reshape(",[170,4799,2661],{"class":2515},[170,4801,194],{"class":2563},[170,4803,4804],{"class":2523},", X.shape[",[170,4806,2928],{"class":2563},[170,4808,4734],{"class":2523},[170,4810,4778],{"class":2563},[170,4812,2640],{"class":2523},[170,4814,4815],{"class":2511,"line":2668},[170,4816,2832],{"emptyLinePlaceholder":114},[170,4818,4819,4821,4824,4826,4828,4830,4832],{"class":2511,"line":2674},[170,4820,2516],{"class":2515},[170,4822,4823],{"class":2519}," transpose_output",[170,4825,909],{"class":2523},[170,4827,2527],{"class":2526},[170,4829,2530],{"class":2523},[170,4831,4702],{"class":2526},[170,4833,2536],{"class":2523},[170,4835,4836],{"class":2511,"line":2699},[170,4837,4838],{"class":2541},"    \"\"\"逆转transpose_qkv函数的操作\"\"\"\n",[170,4840,4841,4843,4845,4847,4849,4851,4854,4856,4858,4860],{"class":2511,"line":2712},[170,4842,4724],{"class":2523},[170,4844,926],{"class":2515},[170,4846,4797],{"class":2523},[170,4848,2661],{"class":2515},[170,4850,194],{"class":2563},[170,4852,4853],{"class":2523},", num_heads, X.shape[",[170,4855,194],{"class":2563},[170,4857,4734],{"class":2523},[170,4859,2928],{"class":2563},[170,4861,2640],{"class":2523},[170,4863,4864,4866,4868,4870,4872,4874,4876,4878,4880,4882,4884],{"class":2511,"line":2952},[170,4865,4724],{"class":2523},[170,4867,926],{"class":2515},[170,4869,4763],{"class":2523},[170,4871,1865],{"class":2563},[170,4873,2530],{"class":2523},[170,4875,2928],{"class":2563},[170,4877,2530],{"class":2523},[170,4879,194],{"class":2563},[170,4881,2530],{"class":2523},[170,4883,4778],{"class":2563},[170,4885,2588],{"class":2523},[170,4887,4889,4891,4893,4895,4897,4899,4902,4904,4906],{"class":2511,"line":4888},17,[170,4890,4794],{"class":2515},[170,4892,4729],{"class":2523},[170,4894,1865],{"class":2563},[170,4896,4734],{"class":2523},[170,4898,194],{"class":2563},[170,4900,4901],{"class":2523},"], ",[170,4903,2661],{"class":2515},[170,4905,194],{"class":2563},[170,4907,2588],{"class":2523},[2501,4909,4911],{"className":2503,"code":4910,"language":2505,"meta":89,"style":89},"#@save\nclass MultiHeadAttention(nn.Module):\n    \"\"\"多头注意力\"\"\"\n    def __init__(self, key_size, query_size, value_size, num_hiddens,\n                 num_heads, dropout, bias=False, **kwargs):\n        super(MultiHeadAttention, self).__init__(**kwargs)\n        self.num_heads = num_heads\n        self.attention = d2l.DotProductAttention(dropout)\n        self.W_q = nn.Linear(query_size, num_hiddens, bias=bias)\n        self.W_k = nn.Linear(key_size, num_hiddens, bias=bias)\n        self.W_v = nn.Linear(value_size, num_hiddens, bias=bias)\n        self.W_o = nn.Linear(num_hiddens, num_hiddens, bias=bias)\n\n    def forward(self, queries, keys, values, valid_lens):\n        # queries，keys，values的形状:\n        # (batch_size，查询或者“键－值”对的个数，num_hiddens)\n        # valid_lens　的形状:\n        # (batch_size，)或(batch_size，查询的个数)\n        # 经过变换后，输出的queries，keys，values 的形状:\n        # (batch_size*num_heads，查询或者“键－值”对的个数，num_hiddens/num_heads)\n        queries = transpose_qkv(self.W_q(queries), self.num_heads)\n        keys = transpose_qkv(self.W_k(keys), self.num_heads)\n        values = transpose_qkv(self.W_v(values), self.num_heads)\n\n        if valid_lens is not None:\n            # 在轴0，将第一项（标量或者矢量）复制num_heads次，\n            # 然后如此复制第二项，然后诸如此类。\n            valid_lens = torch.repeat_interleave(\n                valid_lens, repeats=self.num_heads, dim=0)\n\n        # output的形状:(batch_size*num_heads，查询的个数， num_hiddens/num_heads)\n        output = self.attention(queries, keys, values, valid_lens)\n\n        # output_concat的形状:(batch_size，查询的个数，num_hiddens)\n        output_concat = transpose_output(output, self.num_heads)\n        return self.W_o(output_concat)\n",[2507,4912,4913,4918,4935,4940,4972,4999,5018,5030,5042,5061,5079,5097,5115,5119,5147,5152,5157,5162,5168,5174,5180,5201,5220,5239,5244,5260,5266,5272,5282,5306,5311,5317,5331,5336,5342,5357],{"__ignoreMap":89},[170,4914,4915],{"class":2511,"line":2512},[170,4916,4917],{"class":2547},"#@save\n",[170,4919,4920,4922,4925,4927,4929,4931,4933],{"class":2511,"line":90},[170,4921,2738],{"class":2515},[170,4923,4924],{"class":2741}," MultiHeadAttention",[170,4926,909],{"class":2523},[170,4928,2748],{"class":2747},[170,4930,975],{"class":2523},[170,4932,2753],{"class":2747},[170,4934,2536],{"class":2523},[170,4936,4937],{"class":2511,"line":96},[170,4938,4939],{"class":2541},"    \"\"\"多头注意力\"\"\"\n",[170,4941,4942,4944,4946,4948,4950,4952,4955,4957,4960,4962,4965,4967,4970],{"class":2511,"line":2551},[170,4943,2765],{"class":2515},[170,4945,2768],{"class":2563},[170,4947,909],{"class":2523},[170,4949,2774],{"class":2773},[170,4951,2530],{"class":2523},[170,4953,4954],{"class":2526},"key_size",[170,4956,2530],{"class":2523},[170,4958,4959],{"class":2526},"query_size",[170,4961,2530],{"class":2523},[170,4963,4964],{"class":2526},"value_size",[170,4966,2530],{"class":2523},[170,4968,4969],{"class":2526},"num_hiddens",[170,4971,3933],{"class":2523},[170,4973,4974,4977,4979,4981,4983,4986,4988,4991,4993,4995,4997],{"class":2511,"line":2570},[170,4975,4976],{"class":2526},"                 num_heads",[170,4978,2530],{"class":2523},[170,4980,2779],{"class":2526},[170,4982,2530],{"class":2523},[170,4984,4985],{"class":2526},"bias",[170,4987,926],{"class":2515},[170,4989,4990],{"class":2563},"False",[170,4992,2530],{"class":2523},[170,4994,2784],{"class":2515},[170,4996,2787],{"class":2526},[170,4998,2536],{"class":2523},[170,5000,5001,5003,5006,5008,5010,5012,5014,5016],{"class":2511,"line":2591},[170,5002,2795],{"class":2794},[170,5004,5005],{"class":2523},"(MultiHeadAttention, ",[170,5007,2774],{"class":2801},[170,5009,2804],{"class":2523},[170,5011,2807],{"class":2563},[170,5013,909],{"class":2523},[170,5015,2784],{"class":2515},[170,5017,2814],{"class":2523},[170,5019,5020,5022,5025,5027],{"class":2511,"line":2599},[170,5021,2819],{"class":2801},[170,5023,5024],{"class":2523},".num_heads ",[170,5026,926],{"class":2515},[170,5028,5029],{"class":2523}," num_heads\n",[170,5031,5032,5034,5037,5039],{"class":2511,"line":2610},[170,5033,2819],{"class":2801},[170,5035,5036],{"class":2523},".attention ",[170,5038,926],{"class":2515},[170,5040,5041],{"class":2523}," d2l.DotProductAttention(dropout)\n",[170,5043,5044,5046,5049,5051,5054,5056,5058],{"class":2511,"line":2627},[170,5045,2819],{"class":2801},[170,5047,5048],{"class":2523},".W_q ",[170,5050,926],{"class":2515},[170,5052,5053],{"class":2523}," nn.Linear(query_size, num_hiddens, ",[170,5055,4985],{"class":2579},[170,5057,926],{"class":2515},[170,5059,5060],{"class":2523},"bias)\n",[170,5062,5063,5065,5068,5070,5073,5075,5077],{"class":2511,"line":2643},[170,5064,2819],{"class":2801},[170,5066,5067],{"class":2523},".W_k ",[170,5069,926],{"class":2515},[170,5071,5072],{"class":2523}," nn.Linear(key_size, num_hiddens, ",[170,5074,4985],{"class":2579},[170,5076,926],{"class":2515},[170,5078,5060],{"class":2523},[170,5080,5081,5083,5086,5088,5091,5093,5095],{"class":2511,"line":2651},[170,5082,2819],{"class":2801},[170,5084,5085],{"class":2523},".W_v ",[170,5087,926],{"class":2515},[170,5089,5090],{"class":2523}," nn.Linear(value_size, num_hiddens, ",[170,5092,4985],{"class":2579},[170,5094,926],{"class":2515},[170,5096,5060],{"class":2523},[170,5098,5099,5101,5104,5106,5109,5111,5113],{"class":2511,"line":2668},[170,5100,2819],{"class":2801},[170,5102,5103],{"class":2523},".W_o ",[170,5105,926],{"class":2515},[170,5107,5108],{"class":2523}," nn.Linear(num_hiddens, num_hiddens, ",[170,5110,4985],{"class":2579},[170,5112,926],{"class":2515},[170,5114,5060],{"class":2523},[170,5116,5117],{"class":2511,"line":2674},[170,5118,2832],{"emptyLinePlaceholder":114},[170,5120,5121,5123,5125,5127,5129,5131,5133,5135,5137,5139,5141,5143,5145],{"class":2511,"line":2699},[170,5122,2765],{"class":2515},[170,5124,2859],{"class":2519},[170,5126,909],{"class":2523},[170,5128,2774],{"class":2773},[170,5130,2530],{"class":2523},[170,5132,2868],{"class":2526},[170,5134,2530],{"class":2523},[170,5136,2873],{"class":2526},[170,5138,2530],{"class":2523},[170,5140,2878],{"class":2526},[170,5142,2530],{"class":2523},[170,5144,2533],{"class":2526},[170,5146,2536],{"class":2523},[170,5148,5149],{"class":2511,"line":2712},[170,5150,5151],{"class":2547},"        # queries，keys，values的形状:\n",[170,5153,5154],{"class":2511,"line":2952},[170,5155,5156],{"class":2547},"        # (batch_size，查询或者“键－值”对的个数，num_hiddens)\n",[170,5158,5159],{"class":2511,"line":4888},[170,5160,5161],{"class":2547},"        # valid_lens　的形状:\n",[170,5163,5165],{"class":2511,"line":5164},18,[170,5166,5167],{"class":2547},"        # (batch_size，)或(batch_size，查询的个数)\n",[170,5169,5171],{"class":2511,"line":5170},19,[170,5172,5173],{"class":2547},"        # 经过变换后，输出的queries，keys，values 的形状:\n",[170,5175,5177],{"class":2511,"line":5176},20,[170,5178,5179],{"class":2547},"        # (batch_size*num_heads，查询或者“键－值”对的个数，num_hiddens/num_heads)\n",[170,5181,5183,5186,5188,5191,5193,5196,5198],{"class":2511,"line":5182},21,[170,5184,5185],{"class":2523},"        queries ",[170,5187,926],{"class":2515},[170,5189,5190],{"class":2523}," transpose_qkv(",[170,5192,2774],{"class":2801},[170,5194,5195],{"class":2523},".W_q(queries), ",[170,5197,2774],{"class":2801},[170,5199,5200],{"class":2523},".num_heads)\n",[170,5202,5204,5207,5209,5211,5213,5216,5218],{"class":2511,"line":5203},22,[170,5205,5206],{"class":2523},"        keys ",[170,5208,926],{"class":2515},[170,5210,5190],{"class":2523},[170,5212,2774],{"class":2801},[170,5214,5215],{"class":2523},".W_k(keys), ",[170,5217,2774],{"class":2801},[170,5219,5200],{"class":2523},[170,5221,5223,5226,5228,5230,5232,5235,5237],{"class":2511,"line":5222},23,[170,5224,5225],{"class":2523},"        values ",[170,5227,926],{"class":2515},[170,5229,5190],{"class":2523},[170,5231,2774],{"class":2801},[170,5233,5234],{"class":2523},".W_v(values), ",[170,5236,2774],{"class":2801},[170,5238,5200],{"class":2523},[170,5240,5242],{"class":2511,"line":5241},24,[170,5243,2832],{"emptyLinePlaceholder":114},[170,5245,5247,5249,5251,5253,5256,5258],{"class":2511,"line":5246},25,[170,5248,2613],{"class":2515},[170,5250,2557],{"class":2523},[170,5252,2560],{"class":2515},[170,5254,5255],{"class":2515}," not",[170,5257,2564],{"class":2563},[170,5259,2567],{"class":2523},[170,5261,5263],{"class":2511,"line":5262},26,[170,5264,5265],{"class":2547},"            # 在轴0，将第一项（标量或者矢量）复制num_heads次，\n",[170,5267,5269],{"class":2511,"line":5268},27,[170,5270,5271],{"class":2547},"            # 然后如此复制第二项，然后诸如此类。\n",[170,5273,5275,5277,5279],{"class":2511,"line":5274},28,[170,5276,2630],{"class":2523},[170,5278,926],{"class":2515},[170,5280,5281],{"class":2523}," torch.repeat_interleave(\n",[170,5283,5285,5288,5291,5293,5295,5298,5300,5302,5304],{"class":2511,"line":5284},29,[170,5286,5287],{"class":2523},"                valid_lens, ",[170,5289,5290],{"class":2579},"repeats",[170,5292,926],{"class":2515},[170,5294,2774],{"class":2801},[170,5296,5297],{"class":2523},".num_heads, ",[170,5299,2580],{"class":2579},[170,5301,926],{"class":2515},[170,5303,1865],{"class":2563},[170,5305,2588],{"class":2523},[170,5307,5309],{"class":2511,"line":5308},30,[170,5310,2832],{"emptyLinePlaceholder":114},[170,5312,5314],{"class":2511,"line":5313},31,[170,5315,5316],{"class":2547},"        # output的形状:(batch_size*num_heads，查询的个数， num_hiddens/num_heads)\n",[170,5318,5320,5323,5325,5328],{"class":2511,"line":5319},32,[170,5321,5322],{"class":2523},"        output ",[170,5324,926],{"class":2515},[170,5326,5327],{"class":2801}," self",[170,5329,5330],{"class":2523},".attention(queries, keys, values, valid_lens)\n",[170,5332,5334],{"class":2511,"line":5333},33,[170,5335,2832],{"emptyLinePlaceholder":114},[170,5337,5339],{"class":2511,"line":5338},34,[170,5340,5341],{"class":2547},"        # output_concat的形状:(batch_size，查询的个数，num_hiddens)\n",[170,5343,5345,5348,5350,5353,5355],{"class":2511,"line":5344},35,[170,5346,5347],{"class":2523},"        output_concat ",[170,5349,926],{"class":2515},[170,5351,5352],{"class":2523}," transpose_output(output, ",[170,5354,2774],{"class":2801},[170,5356,5200],{"class":2523},[170,5358,5360,5362,5364],{"class":2511,"line":5359},36,[170,5361,2573],{"class":2515},[170,5363,5327],{"class":2801},[170,5365,5366],{"class":2523},".W_o(output_concat)\n",[10,5368,5369],{"id":5369},"位置编码",[14,5371,5372,5373,5464],{},"在处理 token 序列时 RNN 采用了按顺序逐个处理 token 的，但注意力机制为了并行计算放弃了顺序处理，这会导致其失去位置信息。为了使模型能够使用到位置信息，我们需要在 token 中注入一些和位置相关的信息，我们使用位置编码来注入绝对的或相对的位置信息。位置编码与嵌入的维度相同，均为 ",[170,5374,5376,5402],{"className":5375},[173],[170,5377,5379],{"className":5378},[177],[179,5380,5381],{"xmlns":181},[183,5382,5383,5399],{},[186,5384,5385,5391,5393,5395,5397],{},[199,5386,5387,5389],{},[202,5388,204],{},[202,5390,477],{},[202,5392,903],{},[202,5394,204],{},[202,5396,893],{},[202,5398,3010],{},[209,5400,5401],{"encoding":211},"d_model",[170,5403,5405],{"className":5404,"ariaHidden":217},[216],[170,5406,5408,5411,5451,5454,5457,5460],{"className":5407},[221],[170,5409],{"className":5410,"style":1340},[225],[170,5412,5414,5417],{"className":5413},[230],[170,5415,204],{"className":5416},[230,303],[170,5418,5420],{"className":5419},[307],[170,5421,5423,5443],{"className":5422},[242,243],[170,5424,5426,5440],{"className":5425},[247],[170,5427,5429],{"className":5428,"style":1433},[251],[170,5430,5431,5434],{"style":1362},[170,5432],{"className":5433,"style":651},[259],[170,5435,5437],{"className":5436},[264,265,266,267],[170,5438,477],{"className":5439},[230,303,267],[170,5441,338],{"className":5442},[337],[170,5444,5446],{"className":5445},[247],[170,5447,5449],{"className":5448,"style":1381},[251],[170,5450],{},[170,5452,903],{"className":5453},[230,303],[170,5455,204],{"className":5456},[230,303],[170,5458,893],{"className":5459},[230,303],[170,5461,3010],{"className":5462,"style":5463},[230,303],"margin-right:0.01968em;","，因此两者可以相加。位置编码可以通过学习得到也可以直接固定得到。",[14,5466,5467],{},"在这里，我们使用的是基于正弦函数和余弦函数的位置编码。",[14,5469,5470,5471,5636,5637,5665,5666,5694,5695,5867,5868,5929,5930,5959,5960,5997,5998,6054],{},"假设输入表示 ",[170,5472,5474,5517],{"className":5473},[173],[170,5475,5477],{"className":5476},[177],[179,5478,5479],{"xmlns":181},[183,5480,5481,5514],{},[186,5482,5483,5486,5488],{},[202,5484,2527],{"mathvariant":5485},"bold",[572,5487,574],{},[576,5489,5490,5492],{},[202,5491,581],{"mathvariant":580},[186,5493,5494,5496,5498],{},[202,5495,446],{},[572,5497,588],{},[199,5499,5500,5502],{},[202,5501,204],{},[186,5503,5504,5506,5508,5510,5512],{},[202,5505,477],{},[202,5507,903],{},[202,5509,204],{},[202,5511,893],{},[202,5513,3010],{},[209,5515,5516],{"encoding":211},"\\mathbf{X} \\in \\mathbb{R}^{n \\times d_{model}}",[170,5518,5520,5540],{"className":5519,"ariaHidden":217},[216],[170,5521,5523,5527,5531,5534,5537],{"className":5522},[221],[170,5524],{"className":5525,"style":5526},[225],"height:0.7252em;vertical-align:-0.0391em;",[170,5528,2527],{"className":5529},[230,5530],"mathbf",[170,5532],{"className":5533,"style":611},[610],[170,5535,574],{"className":5536},[615],[170,5538],{"className":5539,"style":611},[610],[170,5541,5543,5546],{"className":5542},[221],[170,5544],{"className":5545,"style":625},[225],[170,5547,5549,5552],{"className":5548},[230],[170,5550,581],{"className":5551},[230,632],[170,5553,5555],{"className":5554},[307],[170,5556,5558],{"className":5557},[242],[170,5559,5561],{"className":5560},[247],[170,5562,5564],{"className":5563,"style":625},[251],[170,5565,5566,5569],{"style":647},[170,5567],{"className":5568,"style":651},[259],[170,5570,5572],{"className":5571},[264,265,266,267],[170,5573,5575,5578,5581],{"className":5574},[230,267],[170,5576,446],{"className":5577},[230,303,267],[170,5579,588],{"className":5580},[664,267],[170,5582,5584,5587],{"className":5583},[230,267],[170,5585,204],{"className":5586},[230,303,267],[170,5588,5590],{"className":5589},[307],[170,5591,5593,5628],{"className":5592},[242,243],[170,5594,5596,5625],{"className":5595},[247],[170,5597,5599],{"className":5598,"style":317},[251],[170,5600,5601,5604],{"style":320},[170,5602],{"className":5603,"style":324},[259],[170,5605,5607],{"className":5606},[264,328,329,267],[170,5608,5610,5613,5616,5619,5622],{"className":5609},[230,267],[170,5611,477],{"className":5612},[230,303,267],[170,5614,903],{"className":5615},[230,303,267],[170,5617,204],{"className":5618},[230,303,267],[170,5620,893],{"className":5621},[230,303,267],[170,5623,3010],{"className":5624,"style":5463},[230,303,267],[170,5626,338],{"className":5627},[337],[170,5629,5631],{"className":5630},[247],[170,5632,5634],{"className":5633,"style":345},[251],[170,5635],{}," 包含一个序列中 ",[170,5638,5640,5653],{"className":5639},[173],[170,5641,5643],{"className":5642},[177],[179,5644,5645],{"xmlns":181},[183,5646,5647,5651],{},[186,5648,5649],{},[202,5650,446],{},[209,5652,446],{"encoding":211},[170,5654,5656],{"className":5655,"ariaHidden":217},[216],[170,5657,5659,5662],{"className":5658},[221],[170,5660],{"className":5661,"style":458},[225],[170,5663,446],{"className":5664},[230,303]," 个词元的 ",[170,5667,5669,5682],{"className":5668},[173],[170,5670,5672],{"className":5671},[177],[179,5673,5674],{"xmlns":181},[183,5675,5676,5680],{},[186,5677,5678],{},[202,5679,204],{},[209,5681,204],{"encoding":211},[170,5683,5685],{"className":5684,"ariaHidden":217},[216],[170,5686,5688,5691],{"className":5687},[221],[170,5689],{"className":5690,"style":518},[225],[170,5692,204],{"className":5693},[230,303]," 维嵌入表示。位置编码使用相同形状的位置嵌入矩阵 ",[170,5696,5698,5746],{"className":5697},[173],[170,5699,5701],{"className":5700},[177],[179,5702,5703],{"xmlns":181},[183,5704,5705,5743],{},[186,5706,5707,5715,5717],{},[186,5708,5709,5712],{},[202,5710,5711],{"mathvariant":5485},"P",[202,5713,5714],{"mathvariant":5485},"E",[572,5716,574],{},[576,5718,5719,5721],{},[202,5720,581],{"mathvariant":580},[186,5722,5723,5725,5727],{},[202,5724,446],{},[572,5726,588],{},[199,5728,5729,5731],{},[202,5730,204],{},[186,5732,5733,5735,5737,5739,5741],{},[202,5734,477],{},[202,5736,903],{},[202,5738,204],{},[202,5740,893],{},[202,5742,3010],{},[209,5744,5745],{"encoding":211},"\\mathbf{PE} \\in \\mathbb{R}^{n \\times d_{model}}",[170,5747,5749,5771],{"className":5748,"ariaHidden":217},[216],[170,5750,5752,5755,5762,5765,5768],{"className":5751},[221],[170,5753],{"className":5754,"style":5526},[225],[170,5756,5758],{"className":5757},[230],[170,5759,5761],{"className":5760},[230,5530],"PE",[170,5763],{"className":5764,"style":611},[610],[170,5766,574],{"className":5767},[615],[170,5769],{"className":5770,"style":611},[610],[170,5772,5774,5777],{"className":5773},[221],[170,5775],{"className":5776,"style":625},[225],[170,5778,5780,5783],{"className":5779},[230],[170,5781,581],{"className":5782},[230,632],[170,5784,5786],{"className":5785},[307],[170,5787,5789],{"className":5788},[242],[170,5790,5792],{"className":5791},[247],[170,5793,5795],{"className":5794,"style":625},[251],[170,5796,5797,5800],{"style":647},[170,5798],{"className":5799,"style":651},[259],[170,5801,5803],{"className":5802},[264,265,266,267],[170,5804,5806,5809,5812],{"className":5805},[230,267],[170,5807,446],{"className":5808},[230,303,267],[170,5810,588],{"className":5811},[664,267],[170,5813,5815,5818],{"className":5814},[230,267],[170,5816,204],{"className":5817},[230,303,267],[170,5819,5821],{"className":5820},[307],[170,5822,5824,5859],{"className":5823},[242,243],[170,5825,5827,5856],{"className":5826},[247],[170,5828,5830],{"className":5829,"style":317},[251],[170,5831,5832,5835],{"style":320},[170,5833],{"className":5834,"style":324},[259],[170,5836,5838],{"className":5837},[264,328,329,267],[170,5839,5841,5844,5847,5850,5853],{"className":5840},[230,267],[170,5842,477],{"className":5843},[230,303,267],[170,5845,903],{"className":5846},[230,303,267],[170,5848,204],{"className":5849},[230,303,267],[170,5851,893],{"className":5852},[230,303,267],[170,5854,3010],{"className":5855,"style":5463},[230,303,267],[170,5857,338],{"className":5858},[337],[170,5860,5862],{"className":5861},[247],[170,5863,5865],{"className":5864,"style":345},[251],[170,5866],{}," 输出 ",[170,5869,5871,5894],{"className":5870},[173],[170,5872,5874],{"className":5873},[177],[179,5875,5876],{"xmlns":181},[183,5877,5878,5891],{},[186,5879,5880,5882,5885],{},[202,5881,2527],{"mathvariant":5485},[572,5883,5884],{},"+",[186,5886,5887,5889],{},[202,5888,5711],{"mathvariant":5485},[202,5890,5714],{"mathvariant":5485},[209,5892,5893],{"encoding":211},"\\mathbf{X} + \\mathbf{PE}",[170,5895,5897,5916],{"className":5896,"ariaHidden":217},[216],[170,5898,5900,5904,5907,5910,5913],{"className":5899},[221],[170,5901],{"className":5902,"style":5903},[225],"height:0.7694em;vertical-align:-0.0833em;",[170,5905,2527],{"className":5906},[230,5530],[170,5908],{"className":5909,"style":1983},[610],[170,5911,5884],{"className":5912},[664],[170,5914],{"className":5915,"style":1983},[610],[170,5917,5919,5923],{"className":5918},[221],[170,5920],{"className":5921,"style":5922},[225],"height:0.6861em;",[170,5924,5926],{"className":5925},[230],[170,5927,5761],{"className":5928},[230,5530],"，矩阵第 ",[170,5931,5933,5946],{"className":5932},[173],[170,5934,5936],{"className":5935},[177],[179,5937,5938],{"xmlns":181},[183,5939,5940,5944],{},[186,5941,5942],{},[202,5943,900],{},[209,5945,900],{"encoding":211},[170,5947,5949],{"className":5948,"ariaHidden":217},[216],[170,5950,5952,5956],{"className":5951},[221],[170,5953],{"className":5954,"style":5955},[225],"height:0.6595em;",[170,5957,900],{"className":5958},[230,303]," 行、第 ",[170,5961,5963,5980],{"className":5962},[173],[170,5964,5966],{"className":5965},[177],[179,5967,5968],{"xmlns":181},[183,5969,5970,5977],{},[186,5971,5972,5974],{},[192,5973,2928],{},[202,5975,5976],{},"j",[209,5978,5979],{"encoding":211},"2j",[170,5981,5983],{"className":5982,"ariaHidden":217},[216],[170,5984,5986,5990,5993],{"className":5985},[221],[170,5987],{"className":5988,"style":5989},[225],"height:0.854em;vertical-align:-0.1944em;",[170,5991,2928],{"className":5992},[230],[170,5994,5976],{"className":5995,"style":5996},[230,303],"margin-right:0.05724em;"," 列和 ",[170,5999,6001,6021],{"className":6000},[173],[170,6002,6004],{"className":6003},[177],[179,6005,6006],{"xmlns":181},[183,6007,6008,6018],{},[186,6009,6010,6012,6014,6016],{},[192,6011,2928],{},[202,6013,5976],{},[572,6015,5884],{},[192,6017,194],{},[209,6019,6020],{"encoding":211},"2j+1",[170,6022,6024,6045],{"className":6023,"ariaHidden":217},[216],[170,6025,6027,6030,6033,6036,6039,6042],{"className":6026},[221],[170,6028],{"className":6029,"style":5989},[225],[170,6031,2928],{"className":6032},[230],[170,6034,5976],{"className":6035,"style":5996},[230,303],[170,6037],{"className":6038,"style":1983},[610],[170,6040,5884],{"className":6041},[664],[170,6043],{"className":6044,"style":1983},[610],[170,6046,6048,6051],{"className":6047},[221],[170,6049],{"className":6050,"style":1877},[225],[170,6052,194],{"className":6053},[230]," 列上的元素为：",[14,6056,6057],{},[170,6058,6060,6138],{"className":6059},[173],[170,6061,6063],{"className":6062},[177],[179,6064,6065],{"xmlns":181},[183,6066,6067,6135],{},[186,6068,6069,6071,6089,6091,6094,6096],{},[202,6070,5711],{},[199,6072,6073,6075],{},[202,6074,5714],{},[186,6076,6077,6079,6081,6083,6085,6087],{},[572,6078,909],{"stretchy":908},[202,6080,900],{},[572,6082,914],{"separator":217},[192,6084,2928],{},[202,6086,5976],{},[572,6088,923],{"stretchy":908},[572,6090,926],{},[202,6092,6093],{},"sin",[572,6095,933],{},[186,6097,6098,6100,6133],{},[572,6099,909],{"fence":217},[189,6101,6102,6104],{},[202,6103,900],{},[576,6105,6106,6109],{},[192,6107,6108],{},"10000",[186,6110,6111,6113,6115,6117],{},[192,6112,2928],{},[202,6114,5976],{},[202,6116,2934],{"mathvariant":929},[199,6118,6119,6121],{},[202,6120,204],{},[186,6122,6123,6125,6127,6129,6131],{},[202,6124,477],{},[202,6126,903],{},[202,6128,204],{},[202,6130,893],{},[202,6132,3010],{},[572,6134,923],{"fence":217},[209,6136,6137],{"encoding":211}," PE_{(i,2j)} = \\sin\\left(\\frac{i}{10000^{2j/d_{model}}}\\right)",[170,6139,6141,6221],{"className":6140,"ariaHidden":217},[216],[170,6142,6144,6148,6151,6212,6215,6218],{"className":6143},[221],[170,6145],{"className":6146,"style":6147},[225],"height:1.0385em;vertical-align:-0.3552em;",[170,6149,5711],{"className":6150,"style":3292},[230,303],[170,6152,6154,6158],{"className":6153},[230],[170,6155,5714],{"className":6156,"style":6157},[230,303],"margin-right:0.05764em;",[170,6159,6161],{"className":6160},[307],[170,6162,6164,6203],{"className":6163},[242,243],[170,6165,6167,6200],{"className":6166},[247],[170,6168,6170],{"className":6169,"style":317},[251],[170,6171,6173,6176],{"style":6172},"top:-2.5198em;margin-left:-0.0576em;margin-right:0.05em;",[170,6174],{"className":6175,"style":651},[259],[170,6177,6179],{"className":6178},[264,265,266,267],[170,6180,6182,6185,6188,6191,6194,6197],{"className":6181},[230,267],[170,6183,909],{"className":6184},[234,267],[170,6186,900],{"className":6187},[230,303,267],[170,6189,914],{"className":6190},[1023,267],[170,6192,2928],{"className":6193},[230,267],[170,6195,5976],{"className":6196,"style":5996},[230,303,267],[170,6198,923],{"className":6199},[424,267],[170,6201,338],{"className":6202},[337],[170,6204,6206],{"className":6205},[247],[170,6207,6210],{"className":6208,"style":6209},[251],"height:0.3552em;",[170,6211],{},[170,6213],{"className":6214,"style":611},[610],[170,6216,926],{"className":6217},[615],[170,6219],{"className":6220,"style":611},[610],[170,6222,6224,6227,6230,6233],{"className":6223},[221],[170,6225],{"className":6226,"style":1058},[225],[170,6228,6093],{"className":6229},[1062],[170,6231],{"className":6232,"style":1027},[610],[170,6234,6236,6242,6410],{"className":6235},[1073],[170,6237,6239],{"className":6238,"style":1078},[234,1077],[170,6240,909],{"className":6241},[1082,1083],[170,6243,6245,6248,6407],{"className":6244},[230],[170,6246],{"className":6247},[234,235],[170,6249,6251],{"className":6250},[189],[170,6252,6254,6398],{"className":6253},[242,243],[170,6255,6257,6395],{"className":6256},[247],[170,6258,6261,6373,6381],{"className":6259,"style":6260},[251],"height:0.8557em;",[170,6262,6264,6267],{"style":6263},"top:-2.5648em;",[170,6265],{"className":6266,"style":260},[259],[170,6268,6270],{"className":6269},[264,265,266,267],[170,6271,6273,6277],{"className":6272},[230,267],[170,6274,6276],{"className":6275},[230,267],"1000",[170,6278,6280,6283],{"className":6279},[230,267],[170,6281,1865],{"className":6282},[230,267],[170,6284,6286],{"className":6285},[307],[170,6287,6289],{"className":6288},[242],[170,6290,6292],{"className":6291},[247],[170,6293,6296],{"className":6294,"style":6295},[251],"height:0.8932em;",[170,6297,6299,6303],{"style":6298},"top:-2.8932em;margin-right:0.0714em;",[170,6300],{"className":6301,"style":6302},[259],"height:2.5357em;",[170,6304,6306],{"className":6305},[264,328,329,267],[170,6307,6309,6312,6315,6318],{"className":6308},[230,267],[170,6310,2928],{"className":6311},[230,267],[170,6313,5976],{"className":6314,"style":5996},[230,303,267],[170,6316,2934],{"className":6317},[230,267],[170,6319,6321,6324],{"className":6320},[230,267],[170,6322,204],{"className":6323},[230,303,267],[170,6325,6327],{"className":6326},[307],[170,6328,6330,6364],{"className":6329},[242,243],[170,6331,6333,6361],{"className":6332},[247],[170,6334,6336],{"className":6335,"style":317},[251],[170,6337,6339,6343],{"style":6338},"top:-2.3448em;margin-left:0em;margin-right:0.1em;",[170,6340],{"className":6341,"style":6342},[259],"height:2.6944em;",[170,6344,6346,6349,6352,6355,6358],{"className":6345},[230,267],[170,6347,477],{"className":6348},[230,303,267],[170,6350,903],{"className":6351},[230,303,267],[170,6353,204],{"className":6354},[230,303,267],[170,6356,893],{"className":6357},[230,303,267],[170,6359,3010],{"className":6360,"style":5463},[230,303,267],[170,6362,338],{"className":6363},[337],[170,6365,6367],{"className":6366},[247],[170,6368,6371],{"className":6369,"style":6370},[251],"height:0.3496em;",[170,6372],{},[170,6374,6375,6378],{"style":385},[170,6376],{"className":6377,"style":260},[259],[170,6379],{"className":6380,"style":393},[392],[170,6382,6383,6386],{"style":396},[170,6384],{"className":6385,"style":260},[259],[170,6387,6389],{"className":6388},[264,265,266,267],[170,6390,6392],{"className":6391},[230,267],[170,6393,900],{"className":6394},[230,303,267],[170,6396,338],{"className":6397},[337],[170,6399,6401],{"className":6400},[247],[170,6402,6405],{"className":6403,"style":6404},[251],"height:0.4352em;",[170,6406],{},[170,6408],{"className":6409},[424,235],[170,6411,6413],{"className":6412,"style":1078},[424,1077],[170,6414,923],{"className":6415},[1082,1083],[14,6417,6418],{},[170,6419,6421,6502],{"className":6420},[173],[170,6422,6424],{"className":6423},[177],[179,6425,6426],{"xmlns":181},[183,6427,6428,6499],{},[186,6429,6430,6432,6454,6456,6459,6461],{},[202,6431,5711],{},[199,6433,6434,6436],{},[202,6435,5714],{},[186,6437,6438,6440,6442,6444,6446,6448,6450,6452],{},[572,6439,909],{"stretchy":908},[202,6441,900],{},[572,6443,914],{"separator":217},[192,6445,2928],{},[202,6447,5976],{},[572,6449,5884],{},[192,6451,194],{},[572,6453,923],{"stretchy":908},[572,6455,926],{},[202,6457,6458],{},"cos",[572,6460,933],{},[186,6462,6463,6465,6497],{},[572,6464,909],{"fence":217},[189,6466,6467,6469],{},[202,6468,900],{},[576,6470,6471,6473],{},[192,6472,6108],{},[186,6474,6475,6477,6479,6481],{},[192,6476,2928],{},[202,6478,5976],{},[202,6480,2934],{"mathvariant":929},[199,6482,6483,6485],{},[202,6484,204],{},[186,6486,6487,6489,6491,6493,6495],{},[202,6488,477],{},[202,6490,903],{},[202,6492,204],{},[202,6494,893],{},[202,6496,3010],{},[572,6498,923],{"fence":217},[209,6500,6501],{"encoding":211}," PE_{(i,2j+1)} = \\cos\\left(\\frac{i}{10000^{2j/d_{model}}}\\right)",[170,6503,6505,6587],{"className":6504,"ariaHidden":217},[216],[170,6506,6508,6511,6514,6578,6581,6584],{"className":6507},[221],[170,6509],{"className":6510,"style":6147},[225],[170,6512,5711],{"className":6513,"style":3292},[230,303],[170,6515,6517,6520],{"className":6516},[230],[170,6518,5714],{"className":6519,"style":6157},[230,303],[170,6521,6523],{"className":6522},[307],[170,6524,6526,6570],{"className":6525},[242,243],[170,6527,6529,6567],{"className":6528},[247],[170,6530,6532],{"className":6531,"style":317},[251],[170,6533,6534,6537],{"style":6172},[170,6535],{"className":6536,"style":651},[259],[170,6538,6540],{"className":6539},[264,265,266,267],[170,6541,6543,6546,6549,6552,6555,6558,6561,6564],{"className":6542},[230,267],[170,6544,909],{"className":6545},[234,267],[170,6547,900],{"className":6548},[230,303,267],[170,6550,914],{"className":6551},[1023,267],[170,6553,2928],{"className":6554},[230,267],[170,6556,5976],{"className":6557,"style":5996},[230,303,267],[170,6559,5884],{"className":6560},[664,267],[170,6562,194],{"className":6563},[230,267],[170,6565,923],{"className":6566},[424,267],[170,6568,338],{"className":6569},[337],[170,6571,6573],{"className":6572},[247],[170,6574,6576],{"className":6575,"style":6209},[251],[170,6577],{},[170,6579],{"className":6580,"style":611},[610],[170,6582,926],{"className":6583},[615],[170,6585],{"className":6586,"style":611},[610],[170,6588,6590,6593,6596,6599],{"className":6589},[221],[170,6591],{"className":6592,"style":1058},[225],[170,6594,6458],{"className":6595},[1062],[170,6597],{"className":6598,"style":1027},[610],[170,6600,6602,6608,6766],{"className":6601},[1073],[170,6603,6605],{"className":6604,"style":1078},[234,1077],[170,6606,909],{"className":6607},[1082,1083],[170,6609,6611,6614,6763],{"className":6610},[230],[170,6612],{"className":6613},[234,235],[170,6615,6617],{"className":6616},[189],[170,6618,6620,6755],{"className":6619},[242,243],[170,6621,6623,6752],{"className":6622},[247],[170,6624,6626,6730,6738],{"className":6625,"style":6260},[251],[170,6627,6628,6631],{"style":6263},[170,6629],{"className":6630,"style":260},[259],[170,6632,6634],{"className":6633},[264,265,266,267],[170,6635,6637,6640],{"className":6636},[230,267],[170,6638,6276],{"className":6639},[230,267],[170,6641,6643,6646],{"className":6642},[230,267],[170,6644,1865],{"className":6645},[230,267],[170,6647,6649],{"className":6648},[307],[170,6650,6652],{"className":6651},[242],[170,6653,6655],{"className":6654},[247],[170,6656,6658],{"className":6657,"style":6295},[251],[170,6659,6660,6663],{"style":6298},[170,6661],{"className":6662,"style":6302},[259],[170,6664,6666],{"className":6665},[264,328,329,267],[170,6667,6669,6672,6675,6678],{"className":6668},[230,267],[170,6670,2928],{"className":6671},[230,267],[170,6673,5976],{"className":6674,"style":5996},[230,303,267],[170,6676,2934],{"className":6677},[230,267],[170,6679,6681,6684],{"className":6680},[230,267],[170,6682,204],{"className":6683},[230,303,267],[170,6685,6687],{"className":6686},[307],[170,6688,6690,6722],{"className":6689},[242,243],[170,6691,6693,6719],{"className":6692},[247],[170,6694,6696],{"className":6695,"style":317},[251],[170,6697,6698,6701],{"style":6338},[170,6699],{"className":6700,"style":6342},[259],[170,6702,6704,6707,6710,6713,6716],{"className":6703},[230,267],[170,6705,477],{"className":6706},[230,303,267],[170,6708,903],{"className":6709},[230,303,267],[170,6711,204],{"className":6712},[230,303,267],[170,6714,893],{"className":6715},[230,303,267],[170,6717,3010],{"className":6718,"style":5463},[230,303,267],[170,6720,338],{"className":6721},[337],[170,6723,6725],{"className":6724},[247],[170,6726,6728],{"className":6727,"style":6370},[251],[170,6729],{},[170,6731,6732,6735],{"style":385},[170,6733],{"className":6734,"style":260},[259],[170,6736],{"className":6737,"style":393},[392],[170,6739,6740,6743],{"style":396},[170,6741],{"className":6742,"style":260},[259],[170,6744,6746],{"className":6745},[264,265,266,267],[170,6747,6749],{"className":6748},[230,267],[170,6750,900],{"className":6751},[230,303,267],[170,6753,338],{"className":6754},[337],[170,6756,6758],{"className":6757},[247],[170,6759,6761],{"className":6760,"style":6404},[251],[170,6762],{},[170,6764],{"className":6765},[424,235],[170,6767,6769],{"className":6768,"style":1078},[424,1077],[170,6770,923],{"className":6771},[1082,1083],[14,6773,6774,6775,6803,6804,6832,6833,6868,6869,6925,6926,6954,6955,7048,7049,7128],{},"其中，",[170,6776,6778,6791],{"className":6777},[173],[170,6779,6781],{"className":6780},[177],[179,6782,6783],{"xmlns":181},[183,6784,6785,6789],{},[186,6786,6787],{},[202,6788,900],{},[209,6790,900],{"encoding":211},[170,6792,6794],{"className":6793,"ariaHidden":217},[216],[170,6795,6797,6800],{"className":6796},[221],[170,6798],{"className":6799,"style":5955},[225],[170,6801,900],{"className":6802},[230,303]," 表示位置，",[170,6805,6807,6820],{"className":6806},[173],[170,6808,6810],{"className":6809},[177],[179,6811,6812],{"xmlns":181},[183,6813,6814,6818],{},[186,6815,6816],{},[202,6817,5976],{},[209,6819,5976],{"encoding":211},[170,6821,6823],{"className":6822,"ariaHidden":217},[216],[170,6824,6826,6829],{"className":6825},[221],[170,6827],{"className":6828,"style":5989},[225],[170,6830,5976],{"className":6831,"style":5996},[230,303]," 表示维度。也就是说，位置编码的每一个维度都对应一个正弦曲线。其波长从 ",[170,6834,6836,6853],{"className":6835},[173],[170,6837,6839],{"className":6838},[177],[179,6840,6841],{"xmlns":181},[183,6842,6843,6850],{},[186,6844,6845,6847],{},[192,6846,2928],{},[202,6848,6849],{},"π",[209,6851,6852],{"encoding":211},"2\\pi",[170,6854,6856],{"className":6855,"ariaHidden":217},[216],[170,6857,6859,6862,6865],{"className":6858},[221],[170,6860],{"className":6861,"style":1877},[225],[170,6863,2928],{"className":6864},[230],[170,6866,6849],{"className":6867,"style":552},[230,303]," 到 ",[170,6870,6872,6892],{"className":6871},[173],[170,6873,6875],{"className":6874},[177],[179,6876,6877],{"xmlns":181},[183,6878,6879,6889],{},[186,6880,6881,6883,6885,6887],{},[192,6882,6108],{},[572,6884,1927],{},[192,6886,2928],{},[202,6888,6849],{},[209,6890,6891],{"encoding":211},"10000 \\cdot 2\\pi",[170,6893,6895,6913],{"className":6894,"ariaHidden":217},[216],[170,6896,6898,6901,6904,6907,6910],{"className":6897},[221],[170,6899],{"className":6900,"style":1877},[225],[170,6902,6108],{"className":6903},[230],[170,6905],{"className":6906,"style":1983},[610],[170,6908,1927],{"className":6909},[664],[170,6911],{"className":6912,"style":1983},[610],[170,6914,6916,6919,6922],{"className":6915},[221],[170,6917],{"className":6918,"style":1877},[225],[170,6920,2928],{"className":6921},[230],[170,6923,6849],{"className":6924,"style":552},[230,303]," 按几何级数增长。我们选择这个函数，是因为我们假设它能够使模型更容易学习关注相对位置，因为对于任意固定的偏移量 ",[170,6927,6929,6942],{"className":6928},[173],[170,6930,6932],{"className":6931},[177],[179,6933,6934],{"xmlns":181},[183,6935,6936,6940],{},[186,6937,6938],{},[202,6939,207],{},[209,6941,207],{"encoding":211},[170,6943,6945],{"className":6944,"ariaHidden":217},[216],[170,6946,6948,6951],{"className":6947},[221],[170,6949],{"className":6950,"style":518},[225],[170,6952,207],{"className":6953,"style":333},[230,303],"，",[170,6956,6958,6984],{"className":6957},[173],[170,6959,6961],{"className":6960},[177],[179,6962,6963],{"xmlns":181},[183,6964,6965,6981],{},[186,6966,6967,6969],{},[202,6968,5711],{},[199,6970,6971,6973],{},[202,6972,5714],{},[186,6974,6975,6977,6979],{},[202,6976,900],{},[572,6978,5884],{},[202,6980,207],{},[209,6982,6983],{"encoding":211},"PE_{i+k}",[170,6985,6987],{"className":6986,"ariaHidden":217},[216],[170,6988,6990,6994,6997],{"className":6989},[221],[170,6991],{"className":6992,"style":6993},[225],"height:0.8917em;vertical-align:-0.2083em;",[170,6995,5711],{"className":6996,"style":3292},[230,303],[170,6998,7000,7003],{"className":6999},[230],[170,7001,5714],{"className":7002,"style":6157},[230,303],[170,7004,7006],{"className":7005},[307],[170,7007,7009,7039],{"className":7008},[242,243],[170,7010,7012,7036],{"className":7011},[247],[170,7013,7015],{"className":7014,"style":1359},[251],[170,7016,7018,7021],{"style":7017},"top:-2.55em;margin-left:-0.0576em;margin-right:0.05em;",[170,7019],{"className":7020,"style":651},[259],[170,7022,7024],{"className":7023},[264,265,266,267],[170,7025,7027,7030,7033],{"className":7026},[230,267],[170,7028,900],{"className":7029},[230,303,267],[170,7031,5884],{"className":7032},[664,267],[170,7034,207],{"className":7035,"style":333},[230,303,267],[170,7037,338],{"className":7038},[337],[170,7040,7042],{"className":7041},[247],[170,7043,7046],{"className":7044,"style":7045},[251],"height:0.2083em;",[170,7047],{}," 都可以表示为 ",[170,7050,7052,7072],{"className":7051},[173],[170,7053,7055],{"className":7054},[177],[179,7056,7057],{"xmlns":181},[183,7058,7059,7069],{},[186,7060,7061,7063],{},[202,7062,5711],{},[199,7064,7065,7067],{},[202,7066,5714],{},[202,7068,900],{},[209,7070,7071],{"encoding":211},"PE_{i}",[170,7073,7075],{"className":7074,"ariaHidden":217},[216],[170,7076,7078,7082,7085],{"className":7077},[221],[170,7079],{"className":7080,"style":7081},[225],"height:0.8333em;vertical-align:-0.15em;",[170,7083,5711],{"className":7084,"style":3292},[230,303],[170,7086,7088,7091],{"className":7087},[230],[170,7089,5714],{"className":7090,"style":6157},[230,303],[170,7092,7094],{"className":7093},[307],[170,7095,7097,7120],{"className":7096},[242,243],[170,7098,7100,7117],{"className":7099},[247],[170,7101,7103],{"className":7102,"style":2143},[251],[170,7104,7105,7108],{"style":7017},[170,7106],{"className":7107,"style":651},[259],[170,7109,7111],{"className":7110},[264,265,266,267],[170,7112,7114],{"className":7113},[230,267],[170,7115,900],{"className":7116},[230,303,267],[170,7118,338],{"className":7119},[337],[170,7121,7123],{"className":7122},[247],[170,7124,7126],{"className":7125,"style":1381},[251],[170,7127],{}," 的线性函数。",[2501,7130,7132],{"className":2503,"code":7131,"language":2505,"meta":89,"style":89},"class PositionalEncoding(nn.Block):\n    \"\"\"位置编码\"\"\"\n    def __init__(self, num_hiddens, dropout, max_len=1000):\n        super(PositionalEncoding, self).__init__()\n        self.dropout = nn.Dropout(dropout)\n        # 创建一个足够长的P\n        self.P = np.zeros((1, max_len, num_hiddens))\n        X = np.arange(max_len).reshape(-1, 1) / np.power(\n            10000, np.arange(0, num_hiddens, 2) / num_hiddens)\n        self.P[:, :, 0::2] = np.sin(X)\n        self.P[:, :, 1::2] = np.cos(X)\n\n    def forward(self, X):\n        X = X + self.P[:, :X.shape[1], :].as_in_ctx(X.ctx)\n        return self.dropout(X)\n",[2507,7133,7134,7152,7157,7186,7202,7212,7217,7234,7259,7281,7313,7340,7344,7360,7393],{"__ignoreMap":89},[170,7135,7136,7138,7141,7143,7145,7147,7150],{"class":2511,"line":2512},[170,7137,2738],{"class":2515},[170,7139,7140],{"class":2741}," PositionalEncoding",[170,7142,909],{"class":2523},[170,7144,2748],{"class":2747},[170,7146,975],{"class":2523},[170,7148,7149],{"class":2747},"Block",[170,7151,2536],{"class":2523},[170,7153,7154],{"class":2511,"line":90},[170,7155,7156],{"class":2541},"    \"\"\"位置编码\"\"\"\n",[170,7158,7159,7161,7163,7165,7167,7169,7171,7173,7175,7177,7180,7182,7184],{"class":2511,"line":96},[170,7160,2765],{"class":2515},[170,7162,2768],{"class":2563},[170,7164,909],{"class":2523},[170,7166,2774],{"class":2773},[170,7168,2530],{"class":2523},[170,7170,4969],{"class":2526},[170,7172,2530],{"class":2523},[170,7174,2779],{"class":2526},[170,7176,2530],{"class":2523},[170,7178,7179],{"class":2526},"max_len",[170,7181,926],{"class":2515},[170,7183,6276],{"class":2563},[170,7185,2536],{"class":2523},[170,7187,7188,7190,7193,7195,7197,7199],{"class":2511,"line":2551},[170,7189,2795],{"class":2794},[170,7191,7192],{"class":2523},"(PositionalEncoding, ",[170,7194,2774],{"class":2801},[170,7196,2804],{"class":2523},[170,7198,2807],{"class":2563},[170,7200,7201],{"class":2523},"()\n",[170,7203,7204,7206,7208,7210],{"class":2511,"line":2570},[170,7205,2819],{"class":2801},[170,7207,2822],{"class":2523},[170,7209,926],{"class":2515},[170,7211,2827],{"class":2523},[170,7213,7214],{"class":2511,"line":2591},[170,7215,7216],{"class":2547},"        # 创建一个足够长的P\n",[170,7218,7219,7221,7224,7226,7229,7231],{"class":2511,"line":2599},[170,7220,2819],{"class":2801},[170,7222,7223],{"class":2523},".P ",[170,7225,926],{"class":2515},[170,7227,7228],{"class":2523}," np.zeros((",[170,7230,194],{"class":2563},[170,7232,7233],{"class":2523},", max_len, num_hiddens))\n",[170,7235,7236,7238,7240,7243,7245,7247,7249,7251,7254,7256],{"class":2511,"line":2610},[170,7237,2677],{"class":2523},[170,7239,926],{"class":2515},[170,7241,7242],{"class":2523}," np.arange(max_len).reshape(",[170,7244,2661],{"class":2515},[170,7246,194],{"class":2563},[170,7248,2530],{"class":2523},[170,7250,194],{"class":2563},[170,7252,7253],{"class":2523},") ",[170,7255,2934],{"class":2515},[170,7257,7258],{"class":2523}," np.power(\n",[170,7260,7261,7264,7267,7269,7272,7274,7276,7278],{"class":2511,"line":2627},[170,7262,7263],{"class":2563},"            10000",[170,7265,7266],{"class":2523},", np.arange(",[170,7268,1865],{"class":2563},[170,7270,7271],{"class":2523},", num_hiddens, ",[170,7273,2928],{"class":2563},[170,7275,7253],{"class":2523},[170,7277,2934],{"class":2515},[170,7279,7280],{"class":2523}," num_hiddens)\n",[170,7282,7283,7285,7288,7292,7294,7296,7298,7300,7303,7305,7308,7310],{"class":2511,"line":2643},[170,7284,2819],{"class":2801},[170,7286,7287],{"class":2523},".P[",[170,7289,7291],{"class":7290},"sDoOe",":",[170,7293,2530],{"class":2523},[170,7295,7291],{"class":7290},[170,7297,2530],{"class":2523},[170,7299,1865],{"class":2563},[170,7301,7302],{"class":7290},"::",[170,7304,2928],{"class":2563},[170,7306,7307],{"class":2523},"] ",[170,7309,926],{"class":2515},[170,7311,7312],{"class":2523}," np.sin(X)\n",[170,7314,7315,7317,7319,7321,7323,7325,7327,7329,7331,7333,7335,7337],{"class":2511,"line":2651},[170,7316,2819],{"class":2801},[170,7318,7287],{"class":2523},[170,7320,7291],{"class":7290},[170,7322,2530],{"class":2523},[170,7324,7291],{"class":7290},[170,7326,2530],{"class":2523},[170,7328,194],{"class":2563},[170,7330,7302],{"class":7290},[170,7332,2928],{"class":2563},[170,7334,7307],{"class":2523},[170,7336,926],{"class":2515},[170,7338,7339],{"class":2523}," np.cos(X)\n",[170,7341,7342],{"class":2511,"line":2668},[170,7343,2832],{"emptyLinePlaceholder":114},[170,7345,7346,7348,7350,7352,7354,7356,7358],{"class":2511,"line":2674},[170,7347,2765],{"class":2515},[170,7349,2859],{"class":2519},[170,7351,909],{"class":2523},[170,7353,2774],{"class":2773},[170,7355,2530],{"class":2523},[170,7357,2527],{"class":2526},[170,7359,2536],{"class":2523},[170,7361,7362,7364,7366,7369,7371,7373,7375,7377,7379,7381,7384,7386,7388,7390],{"class":2511,"line":2699},[170,7363,2677],{"class":2523},[170,7365,926],{"class":2515},[170,7367,7368],{"class":2523}," X ",[170,7370,5884],{"class":2515},[170,7372,5327],{"class":2801},[170,7374,7287],{"class":2523},[170,7376,7291],{"class":7290},[170,7378,2530],{"class":2523},[170,7380,7291],{"class":7290},[170,7382,7383],{"class":2523},"X.shape[",[170,7385,194],{"class":2563},[170,7387,4901],{"class":2523},[170,7389,7291],{"class":7290},[170,7391,7392],{"class":2523},"].as_in_ctx(X.ctx)\n",[170,7394,7395,7397,7399],{"class":2511,"line":2712},[170,7396,2573],{"class":2515},[170,7398,5327],{"class":2801},[170,7400,7401],{"class":2523},".dropout(X)\n",[10,7403,7404],{"id":7404},"前馈神经网络",[14,7406,7407],{},"除了注意力子层外，我们的编码器和解码器的每一层中都包含一个全连接的前馈神经网络，该网络分别且相同地应用于每个位置。这包括两个线性变换，中间使用ReLU激活函数。",[14,7409,7410],{},[170,7411,7413,7486],{"className":7412},[173],[170,7414,7416],{"className":7415},[177],[179,7417,7418],{"xmlns":181},[183,7419,7420,7483],{},[186,7421,7422,7425,7427,7430,7432,7435,7437,7439,7442,7444,7446,7448,7450,7452,7458,7460,7467,7469,7475,7477],{},[202,7423,7424],{},"F",[202,7426,7424],{},[202,7428,7429],{},"N",[572,7431,909],{"stretchy":908},[202,7433,7434],{},"x",[572,7436,923],{"stretchy":908},[572,7438,926],{},[202,7440,7441],{},"max",[572,7443,933],{},[572,7445,909],{"stretchy":908},[192,7447,1865],{},[572,7449,914],{"separator":217},[202,7451,7434],{},[199,7453,7454,7456],{},[202,7455,3100],{},[192,7457,194],{},[572,7459,5884],{},[199,7461,7462,7465],{},[202,7463,7464],{},"b",[192,7466,194],{},[572,7468,923],{"stretchy":908},[199,7470,7471,7473],{},[202,7472,3100],{},[192,7474,2928],{},[572,7476,5884],{},[199,7478,7479,7481],{},[202,7480,7464],{},[192,7482,2928],{},[209,7484,7485],{"encoding":211}," FFN(x) = \\max(0,xW_1+b_1)W_2+b_2",[170,7487,7489,7523,7597,7695],{"className":7488,"ariaHidden":217},[216],[170,7490,7492,7495,7498,7501,7505,7508,7511,7514,7517,7520],{"className":7491},[221],[170,7493],{"className":7494,"style":988},[225],[170,7496,7424],{"className":7497,"style":3292},[230,303],[170,7499,7424],{"className":7500,"style":3292},[230,303],[170,7502,7429],{"className":7503,"style":7504},[230,303],"margin-right:0.10903em;",[170,7506,909],{"className":7507},[234],[170,7509,7434],{"className":7510},[230,303],[170,7512,923],{"className":7513},[424],[170,7515],{"className":7516,"style":611},[610],[170,7518,926],{"className":7519},[615],[170,7521],{"className":7522,"style":611},[610],[170,7524,7526,7529,7532,7535,7538,7541,7544,7547,7588,7591,7594],{"className":7525},[221],[170,7527],{"className":7528,"style":988},[225],[170,7530,7441],{"className":7531},[1062],[170,7533,909],{"className":7534},[234],[170,7536,1865],{"className":7537},[230],[170,7539,914],{"className":7540},[1023],[170,7542],{"className":7543,"style":1027},[610],[170,7545,7434],{"className":7546},[230,303],[170,7548,7550,7553],{"className":7549},[230],[170,7551,3100],{"className":7552,"style":3292},[230,303],[170,7554,7556],{"className":7555},[307],[170,7557,7559,7580],{"className":7558},[242,243],[170,7560,7562,7577],{"className":7561},[247],[170,7563,7565],{"className":7564,"style":3198},[251],[170,7566,7568,7571],{"style":7567},"top:-2.55em;margin-left:-0.1389em;margin-right:0.05em;",[170,7569],{"className":7570,"style":651},[259],[170,7572,7574],{"className":7573},[264,265,266,267],[170,7575,194],{"className":7576},[230,267],[170,7578,338],{"className":7579},[337],[170,7581,7583],{"className":7582},[247],[170,7584,7586],{"className":7585,"style":1381},[251],[170,7587],{},[170,7589],{"className":7590,"style":1983},[610],[170,7592,5884],{"className":7593},[664],[170,7595],{"className":7596,"style":1983},[610],[170,7598,7600,7603,7643,7646,7686,7689,7692],{"className":7599},[221],[170,7601],{"className":7602,"style":988},[225],[170,7604,7606,7609],{"className":7605},[230],[170,7607,7464],{"className":7608},[230,303],[170,7610,7612],{"className":7611},[307],[170,7613,7615,7635],{"className":7614},[242,243],[170,7616,7618,7632],{"className":7617},[247],[170,7619,7621],{"className":7620,"style":3198},[251],[170,7622,7623,7626],{"style":1362},[170,7624],{"className":7625,"style":651},[259],[170,7627,7629],{"className":7628},[264,265,266,267],[170,7630,194],{"className":7631},[230,267],[170,7633,338],{"className":7634},[337],[170,7636,7638],{"className":7637},[247],[170,7639,7641],{"className":7640,"style":1381},[251],[170,7642],{},[170,7644,923],{"className":7645},[424],[170,7647,7649,7652],{"className":7648},[230],[170,7650,3100],{"className":7651,"style":3292},[230,303],[170,7653,7655],{"className":7654},[307],[170,7656,7658,7678],{"className":7657},[242,243],[170,7659,7661,7675],{"className":7660},[247],[170,7662,7664],{"className":7663,"style":3198},[251],[170,7665,7666,7669],{"style":7567},[170,7667],{"className":7668,"style":651},[259],[170,7670,7672],{"className":7671},[264,265,266,267],[170,7673,2928],{"className":7674},[230,267],[170,7676,338],{"className":7677},[337],[170,7679,7681],{"className":7680},[247],[170,7682,7684],{"className":7683,"style":1381},[251],[170,7685],{},[170,7687],{"className":7688,"style":1983},[610],[170,7690,5884],{"className":7691},[664],[170,7693],{"className":7694,"style":1983},[610],[170,7696,7698,7701],{"className":7697},[221],[170,7699],{"className":7700,"style":1340},[225],[170,7702,7704,7707],{"className":7703},[230],[170,7705,7464],{"className":7706},[230,303],[170,7708,7710],{"className":7709},[307],[170,7711,7713,7733],{"className":7712},[242,243],[170,7714,7716,7730],{"className":7715},[247],[170,7717,7719],{"className":7718,"style":3198},[251],[170,7720,7721,7724],{"style":1362},[170,7722],{"className":7723,"style":651},[259],[170,7725,7727],{"className":7726},[264,265,266,267],[170,7728,2928],{"className":7729},[230,267],[170,7731,338],{"className":7732},[337],[170,7734,7736],{"className":7735},[247],[170,7737,7739],{"className":7738,"style":1381},[251],[170,7740],{},[2501,7742,7744],{"className":2503,"code":7743,"language":2505,"meta":89,"style":89},"class PositionWiseFFN(nn.Module):\n    \"\"\"基于位置的前馈网络\"\"\"\n    def __init__(self, ffn_num_input, ffn_num_hiddens, ffn_num_outputs,\n                 **kwargs):\n        super(PositionWiseFFN, self).__init__(**kwargs)\n        self.dense1 = nn.Linear(ffn_num_input, ffn_num_hiddens)\n        self.relu = nn.ReLU()\n        self.dense2 = nn.Linear(ffn_num_hiddens, ffn_num_outputs)\n\n    def forward(self, X):\n        return self.dense2(self.relu(self.dense1(X)))\n",[2507,7745,7746,7763,7768,7795,7804,7823,7835,7847,7859,7863,7879],{"__ignoreMap":89},[170,7747,7748,7750,7753,7755,7757,7759,7761],{"class":2511,"line":2512},[170,7749,2738],{"class":2515},[170,7751,7752],{"class":2741}," PositionWiseFFN",[170,7754,909],{"class":2523},[170,7756,2748],{"class":2747},[170,7758,975],{"class":2523},[170,7760,2753],{"class":2747},[170,7762,2536],{"class":2523},[170,7764,7765],{"class":2511,"line":90},[170,7766,7767],{"class":2541},"    \"\"\"基于位置的前馈网络\"\"\"\n",[170,7769,7770,7772,7774,7776,7778,7780,7783,7785,7788,7790,7793],{"class":2511,"line":96},[170,7771,2765],{"class":2515},[170,7773,2768],{"class":2563},[170,7775,909],{"class":2523},[170,7777,2774],{"class":2773},[170,7779,2530],{"class":2523},[170,7781,7782],{"class":2526},"ffn_num_input",[170,7784,2530],{"class":2523},[170,7786,7787],{"class":2526},"ffn_num_hiddens",[170,7789,2530],{"class":2523},[170,7791,7792],{"class":2526},"ffn_num_outputs",[170,7794,3933],{"class":2523},[170,7796,7797,7800,7802],{"class":2511,"line":2551},[170,7798,7799],{"class":2515},"                 **",[170,7801,2787],{"class":2526},[170,7803,2536],{"class":2523},[170,7805,7806,7808,7811,7813,7815,7817,7819,7821],{"class":2511,"line":2570},[170,7807,2795],{"class":2794},[170,7809,7810],{"class":2523},"(PositionWiseFFN, ",[170,7812,2774],{"class":2801},[170,7814,2804],{"class":2523},[170,7816,2807],{"class":2563},[170,7818,909],{"class":2523},[170,7820,2784],{"class":2515},[170,7822,2814],{"class":2523},[170,7824,7825,7827,7830,7832],{"class":2511,"line":2591},[170,7826,2819],{"class":2801},[170,7828,7829],{"class":2523},".dense1 ",[170,7831,926],{"class":2515},[170,7833,7834],{"class":2523}," nn.Linear(ffn_num_input, ffn_num_hiddens)\n",[170,7836,7837,7839,7842,7844],{"class":2511,"line":2599},[170,7838,2819],{"class":2801},[170,7840,7841],{"class":2523},".relu ",[170,7843,926],{"class":2515},[170,7845,7846],{"class":2523}," nn.ReLU()\n",[170,7848,7849,7851,7854,7856],{"class":2511,"line":2610},[170,7850,2819],{"class":2801},[170,7852,7853],{"class":2523},".dense2 ",[170,7855,926],{"class":2515},[170,7857,7858],{"class":2523}," nn.Linear(ffn_num_hiddens, ffn_num_outputs)\n",[170,7860,7861],{"class":2511,"line":2627},[170,7862,2832],{"emptyLinePlaceholder":114},[170,7864,7865,7867,7869,7871,7873,7875,7877],{"class":2511,"line":2643},[170,7866,2765],{"class":2515},[170,7868,2859],{"class":2519},[170,7870,909],{"class":2523},[170,7872,2774],{"class":2773},[170,7874,2530],{"class":2523},[170,7876,2527],{"class":2526},[170,7878,2536],{"class":2523},[170,7880,7881,7883,7885,7888,7890,7893,7895],{"class":2511,"line":2651},[170,7882,2573],{"class":2515},[170,7884,5327],{"class":2801},[170,7886,7887],{"class":2523},".dense2(",[170,7889,2774],{"class":2801},[170,7891,7892],{"class":2523},".relu(",[170,7894,2774],{"class":2801},[170,7896,7897],{"class":2523},".dense1(X)))\n",[10,7899,7900],{"id":7900},"残差连接和层规范化",[14,7902,7903],{},"我们在每个子层之间加入了一个残差连接和层归一化。",[14,7905,7906,7907,4675,8032,8095],{},"因此，每个子层的输出都是 ",[170,7908,7910,7978],{"className":7909},[173],[170,7911,7913],{"className":7912},[177],[179,7914,7915],{"xmlns":181},[183,7916,7917,7975],{},[186,7918,7919,7942,7944,7946,7948,7967,7969,7971,7973],{},[186,7920,7921,7924,7926,7929,7931,7934,7936,7938,7940],{},[202,7922,7923],{"mathvariant":929},"L",[202,7925,134],{"mathvariant":929},[202,7927,7928],{"mathvariant":929},"y",[202,7930,893],{"mathvariant":929},[202,7932,7933],{"mathvariant":929},"r",[202,7935,7429],{"mathvariant":929},[202,7937,903],{"mathvariant":929},[202,7939,7933],{"mathvariant":929},[202,7941,477],{"mathvariant":929},[572,7943,909],{"stretchy":908},[202,7945,7434],{},[572,7947,5884],{},[186,7949,7950,7953,7955,7957,7959,7961,7963,7965],{},[202,7951,7952],{"mathvariant":929},"S",[202,7954,3007],{"mathvariant":929},[202,7956,7464],{"mathvariant":929},[202,7958,3010],{"mathvariant":929},[202,7960,134],{"mathvariant":929},[202,7962,7928],{"mathvariant":929},[202,7964,893],{"mathvariant":929},[202,7966,7933],{"mathvariant":929},[572,7968,909],{"stretchy":908},[202,7970,7434],{},[572,7972,923],{"stretchy":908},[572,7974,923],{"stretchy":908},[209,7976,7977],{"encoding":211},"\\mathrm{LayerNorm}(x + \\mathrm{Sublayer}(x))",[170,7979,7981,8009],{"className":7980,"ariaHidden":217},[216],[170,7982,7984,7987,7994,7997,8000,8003,8006],{"className":7983},[221],[170,7985],{"className":7986,"style":988},[225],[170,7988,7990],{"className":7989},[230],[170,7991,7993],{"className":7992},[230,1066],"LayerNorm",[170,7995,909],{"className":7996},[234],[170,7998,7434],{"className":7999},[230,303],[170,8001],{"className":8002,"style":1983},[610],[170,8004,5884],{"className":8005},[664],[170,8007],{"className":8008,"style":1983},[610],[170,8010,8012,8015,8022,8025,8028],{"className":8011},[221],[170,8013],{"className":8014,"style":988},[225],[170,8016,8018],{"className":8017},[230],[170,8019,8021],{"className":8020},[230,1066],"Sublayer",[170,8023,909],{"className":8024},[234],[170,8026,7434],{"className":8027},[230,303],[170,8029,8031],{"className":8030},[424],"))",[170,8033,8035,8071],{"className":8034},[173],[170,8036,8038],{"className":8037},[177],[179,8039,8040],{"xmlns":181},[183,8041,8042,8068],{},[186,8043,8044,8062,8064,8066],{},[186,8045,8046,8048,8050,8052,8054,8056,8058,8060],{},[202,8047,7952],{"mathvariant":929},[202,8049,3007],{"mathvariant":929},[202,8051,7464],{"mathvariant":929},[202,8053,3010],{"mathvariant":929},[202,8055,134],{"mathvariant":929},[202,8057,7928],{"mathvariant":929},[202,8059,893],{"mathvariant":929},[202,8061,7933],{"mathvariant":929},[572,8063,909],{"stretchy":908},[202,8065,7434],{},[572,8067,923],{"stretchy":908},[209,8069,8070],{"encoding":211},"\\mathrm{Sublayer}(x)",[170,8072,8074],{"className":8073,"ariaHidden":217},[216],[170,8075,8077,8080,8086,8089,8092],{"className":8076},[221],[170,8078],{"className":8079,"style":988},[225],[170,8081,8083],{"className":8082},[230],[170,8084,8021],{"className":8085},[230,1066],[170,8087,909],{"className":8088},[234],[170,8090,7434],{"className":8091},[230,303],[170,8093,923],{"className":8094},[424]," 是由该子层本身实现的函数。我们在每个子层的输出上应用 dropout，然后再将其加到子层输入并进行归一化。",[2501,8097,8099],{"className":2503,"code":8098,"language":2505,"meta":89,"style":89},"class AddNorm(nn.Module):\n    \"\"\"残差连接后进行层规范化\"\"\"\n    def __init__(self, normalized_shape, dropout, **kwargs):\n        super(AddNorm, self).__init__(**kwargs)\n        self.dropout = nn.Dropout(dropout)\n        self.ln = nn.LayerNorm(normalized_shape)\n\n    def forward(self, X, Y):\n        return self.ln(self.dropout(Y) + X)\n",[2507,8100,8101,8118,8123,8150,8169,8179,8191,8195,8216],{"__ignoreMap":89},[170,8102,8103,8105,8108,8110,8112,8114,8116],{"class":2511,"line":2512},[170,8104,2738],{"class":2515},[170,8106,8107],{"class":2741}," AddNorm",[170,8109,909],{"class":2523},[170,8111,2748],{"class":2747},[170,8113,975],{"class":2523},[170,8115,2753],{"class":2747},[170,8117,2536],{"class":2523},[170,8119,8120],{"class":2511,"line":90},[170,8121,8122],{"class":2541},"    \"\"\"残差连接后进行层规范化\"\"\"\n",[170,8124,8125,8127,8129,8131,8133,8135,8138,8140,8142,8144,8146,8148],{"class":2511,"line":96},[170,8126,2765],{"class":2515},[170,8128,2768],{"class":2563},[170,8130,909],{"class":2523},[170,8132,2774],{"class":2773},[170,8134,2530],{"class":2523},[170,8136,8137],{"class":2526},"normalized_shape",[170,8139,2530],{"class":2523},[170,8141,2779],{"class":2526},[170,8143,2530],{"class":2523},[170,8145,2784],{"class":2515},[170,8147,2787],{"class":2526},[170,8149,2536],{"class":2523},[170,8151,8152,8154,8157,8159,8161,8163,8165,8167],{"class":2511,"line":2551},[170,8153,2795],{"class":2794},[170,8155,8156],{"class":2523},"(AddNorm, ",[170,8158,2774],{"class":2801},[170,8160,2804],{"class":2523},[170,8162,2807],{"class":2563},[170,8164,909],{"class":2523},[170,8166,2784],{"class":2515},[170,8168,2814],{"class":2523},[170,8170,8171,8173,8175,8177],{"class":2511,"line":2570},[170,8172,2819],{"class":2801},[170,8174,2822],{"class":2523},[170,8176,926],{"class":2515},[170,8178,2827],{"class":2523},[170,8180,8181,8183,8186,8188],{"class":2511,"line":2591},[170,8182,2819],{"class":2801},[170,8184,8185],{"class":2523},".ln ",[170,8187,926],{"class":2515},[170,8189,8190],{"class":2523}," nn.LayerNorm(normalized_shape)\n",[170,8192,8193],{"class":2511,"line":2599},[170,8194,2832],{"emptyLinePlaceholder":114},[170,8196,8197,8199,8201,8203,8205,8207,8209,8211,8214],{"class":2511,"line":2610},[170,8198,2765],{"class":2515},[170,8200,2859],{"class":2519},[170,8202,909],{"class":2523},[170,8204,2774],{"class":2773},[170,8206,2530],{"class":2523},[170,8208,2527],{"class":2526},[170,8210,2530],{"class":2523},[170,8212,8213],{"class":2526},"Y",[170,8215,2536],{"class":2523},[170,8217,8218,8220,8222,8225,8227,8230,8232],{"class":2511,"line":2627},[170,8219,2573],{"class":2515},[170,8221,5327],{"class":2801},[170,8223,8224],{"class":2523},".ln(",[170,8226,2774],{"class":2801},[170,8228,8229],{"class":2523},".dropout(Y) ",[170,8231,5884],{"class":2515},[170,8233,8234],{"class":2523}," X)\n",[10,8236,8237],{"id":8237},"编码器-解码器架构",[14,8239,8240,8241,8391,8392,8568,8569,8598,8599,8749],{},"大多数主流的神经序列转换模型都采用编码器—解码器结构。其中，编码器将符号表示组成的输入序列 ",[170,8242,8244,8278],{"className":8243},[173],[170,8245,8247],{"className":8246},[177],[179,8248,8249],{"xmlns":181},[183,8250,8251,8275],{},[186,8252,8253,8255,8261,8263,8265,8267,8273],{},[572,8254,909],{"stretchy":908},[199,8256,8257,8259],{},[202,8258,7434],{},[192,8260,194],{},[572,8262,914],{"separator":217},[572,8264,3077],{},[572,8266,914],{"separator":217},[199,8268,8269,8271],{},[202,8270,7434],{},[202,8272,446],{},[572,8274,923],{"stretchy":908},[209,8276,8277],{"encoding":211},"(x_1, \\ldots, x_n)",[170,8279,8281],{"className":8280,"ariaHidden":217},[216],[170,8282,8284,8287,8290,8330,8333,8336,8339,8342,8345,8348,8388],{"className":8283},[221],[170,8285],{"className":8286,"style":988},[225],[170,8288,909],{"className":8289},[234],[170,8291,8293,8296],{"className":8292},[230],[170,8294,7434],{"className":8295},[230,303],[170,8297,8299],{"className":8298},[307],[170,8300,8302,8322],{"className":8301},[242,243],[170,8303,8305,8319],{"className":8304},[247],[170,8306,8308],{"className":8307,"style":3198},[251],[170,8309,8310,8313],{"style":1362},[170,8311],{"className":8312,"style":651},[259],[170,8314,8316],{"className":8315},[264,265,266,267],[170,8317,194],{"className":8318},[230,267],[170,8320,338],{"className":8321},[337],[170,8323,8325],{"className":8324},[247],[170,8326,8328],{"className":8327,"style":1381},[251],[170,8329],{},[170,8331,914],{"className":8332},[1023],[170,8334],{"className":8335,"style":1027},[610],[170,8337,3077],{"className":8338},[1073],[170,8340],{"className":8341,"style":1027},[610],[170,8343,914],{"className":8344},[1023],[170,8346],{"className":8347,"style":1027},[610],[170,8349,8351,8354],{"className":8350},[230],[170,8352,7434],{"className":8353},[230,303],[170,8355,8357],{"className":8356},[307],[170,8358,8360,8380],{"className":8359},[242,243],[170,8361,8363,8377],{"className":8362},[247],[170,8364,8366],{"className":8365,"style":1433},[251],[170,8367,8368,8371],{"style":1362},[170,8369],{"className":8370,"style":651},[259],[170,8372,8374],{"className":8373},[264,265,266,267],[170,8375,446],{"className":8376},[230,303,267],[170,8378,338],{"className":8379},[337],[170,8381,8383],{"className":8382},[247],[170,8384,8386],{"className":8385,"style":1381},[251],[170,8387],{},[170,8389,923],{"className":8390},[424]," 映射为连续表示序列 ",[170,8393,8395,8434],{"className":8394},[173],[170,8396,8398],{"className":8397},[177],[179,8399,8400],{"xmlns":181},[183,8401,8402,8431],{},[186,8403,8404,8407,8409,8411,8417,8419,8421,8423,8429],{},[202,8405,8406],{"mathvariant":5485},"z",[572,8408,926],{},[572,8410,909],{"stretchy":908},[199,8412,8413,8415],{},[202,8414,8406],{},[192,8416,194],{},[572,8418,914],{"separator":217},[572,8420,3077],{},[572,8422,914],{"separator":217},[199,8424,8425,8427],{},[202,8426,8406],{},[202,8428,446],{},[572,8430,923],{"stretchy":908},[209,8432,8433],{"encoding":211},"\\mathbf{z} = (z_1, \\ldots, z_n)",[170,8435,8437,8456],{"className":8436,"ariaHidden":217},[216],[170,8438,8440,8444,8447,8450,8453],{"className":8439},[221],[170,8441],{"className":8442,"style":8443},[225],"height:0.4444em;",[170,8445,8406],{"className":8446},[230,5530],[170,8448],{"className":8449,"style":611},[610],[170,8451,926],{"className":8452},[615],[170,8454],{"className":8455,"style":611},[610],[170,8457,8459,8462,8465,8507,8510,8513,8516,8519,8522,8525,8565],{"className":8458},[221],[170,8460],{"className":8461,"style":988},[225],[170,8463,909],{"className":8464},[234],[170,8466,8468,8472],{"className":8467},[230],[170,8469,8406],{"className":8470,"style":8471},[230,303],"margin-right:0.04398em;",[170,8473,8475],{"className":8474},[307],[170,8476,8478,8499],{"className":8477},[242,243],[170,8479,8481,8496],{"className":8480},[247],[170,8482,8484],{"className":8483,"style":3198},[251],[170,8485,8487,8490],{"style":8486},"top:-2.55em;margin-left:-0.044em;margin-right:0.05em;",[170,8488],{"className":8489,"style":651},[259],[170,8491,8493],{"className":8492},[264,265,266,267],[170,8494,194],{"className":8495},[230,267],[170,8497,338],{"className":8498},[337],[170,8500,8502],{"className":8501},[247],[170,8503,8505],{"className":8504,"style":1381},[251],[170,8506],{},[170,8508,914],{"className":8509},[1023],[170,8511],{"className":8512,"style":1027},[610],[170,8514,3077],{"className":8515},[1073],[170,8517],{"className":8518,"style":1027},[610],[170,8520,914],{"className":8521},[1023],[170,8523],{"className":8524,"style":1027},[610],[170,8526,8528,8531],{"className":8527},[230],[170,8529,8406],{"className":8530,"style":8471},[230,303],[170,8532,8534],{"className":8533},[307],[170,8535,8537,8557],{"className":8536},[242,243],[170,8538,8540,8554],{"className":8539},[247],[170,8541,8543],{"className":8542,"style":1433},[251],[170,8544,8545,8548],{"style":8486},[170,8546],{"className":8547,"style":651},[259],[170,8549,8551],{"className":8550},[264,265,266,267],[170,8552,446],{"className":8553},[230,303,267],[170,8555,338],{"className":8556},[337],[170,8558,8560],{"className":8559},[247],[170,8561,8563],{"className":8562,"style":1381},[251],[170,8564],{},[170,8566,923],{"className":8567},[424],"。给定 ",[170,8570,8572,8586],{"className":8571},[173],[170,8573,8575],{"className":8574},[177],[179,8576,8577],{"xmlns":181},[183,8578,8579,8583],{},[186,8580,8581],{},[202,8582,8406],{"mathvariant":5485},[209,8584,8585],{"encoding":211},"\\mathbf{z}",[170,8587,8589],{"className":8588,"ariaHidden":217},[216],[170,8590,8592,8595],{"className":8591},[221],[170,8593],{"className":8594,"style":8443},[225],[170,8596,8406],{"className":8597},[230,5530],"，解码器随后一次生成一个元素的输出符号序列 ",[170,8600,8602,8636],{"className":8601},[173],[170,8603,8605],{"className":8604},[177],[179,8606,8607],{"xmlns":181},[183,8608,8609,8633],{},[186,8610,8611,8613,8619,8621,8623,8625,8631],{},[572,8612,909],{"stretchy":908},[199,8614,8615,8617],{},[202,8616,7928],{},[192,8618,194],{},[572,8620,914],{"separator":217},[572,8622,3077],{},[572,8624,914],{"separator":217},[199,8626,8627,8629],{},[202,8628,7928],{},[202,8630,477],{},[572,8632,923],{"stretchy":908},[209,8634,8635],{"encoding":211},"(y_1, \\ldots, y_m)",[170,8637,8639],{"className":8638,"ariaHidden":217},[216],[170,8640,8642,8645,8648,8688,8691,8694,8697,8700,8703,8706,8746],{"className":8641},[221],[170,8643],{"className":8644,"style":988},[225],[170,8646,909],{"className":8647},[234],[170,8649,8651,8654],{"className":8650},[230],[170,8652,7928],{"className":8653,"style":552},[230,303],[170,8655,8657],{"className":8656},[307],[170,8658,8660,8680],{"className":8659},[242,243],[170,8661,8663,8677],{"className":8662},[247],[170,8664,8666],{"className":8665,"style":3198},[251],[170,8667,8668,8671],{"style":2146},[170,8669],{"className":8670,"style":651},[259],[170,8672,8674],{"className":8673},[264,265,266,267],[170,8675,194],{"className":8676},[230,267],[170,8678,338],{"className":8679},[337],[170,8681,8683],{"className":8682},[247],[170,8684,8686],{"className":8685,"style":1381},[251],[170,8687],{},[170,8689,914],{"className":8690},[1023],[170,8692],{"className":8693,"style":1027},[610],[170,8695,3077],{"className":8696},[1073],[170,8698],{"className":8699,"style":1027},[610],[170,8701,914],{"className":8702},[1023],[170,8704],{"className":8705,"style":1027},[610],[170,8707,8709,8712],{"className":8708},[230],[170,8710,7928],{"className":8711,"style":552},[230,303],[170,8713,8715],{"className":8714},[307],[170,8716,8718,8738],{"className":8717},[242,243],[170,8719,8721,8735],{"className":8720},[247],[170,8722,8724],{"className":8723,"style":1433},[251],[170,8725,8726,8729],{"style":2146},[170,8727],{"className":8728,"style":651},[259],[170,8730,8732],{"className":8731},[264,265,266,267],[170,8733,477],{"className":8734},[230,303,267],[170,8736,338],{"className":8737},[337],[170,8739,8741],{"className":8740},[247],[170,8742,8744],{"className":8743,"style":1381},[251],[170,8745],{},[170,8747,923],{"className":8748},[424],"。在每一个时间步，模型都是自回归的，即在生成下一个符号时，会将之前已经生成的符号作为额外输入。",[2501,8751,8753],{"className":2503,"code":8752,"language":2505,"meta":89,"style":89},"class Encoder(nn.Module):\n    \"\"\"编码器-解码器架构的基本编码器接口\"\"\"\n    def __init__(self, **kwargs):\n        super(Encoder, self).__init__(**kwargs)\n\n    def forward(self, X, *args):\n        raise NotImplementedError\n\nclass Decoder(nn.Module):\n    \"\"\"编码器-解码器架构的基本解码器接口\"\"\"\n    def __init__(self, **kwargs):\n        super(Decoder, self).__init__(**kwargs)\n\n    def init_state(self, enc_outputs, *args):\n        raise NotImplementedError\n\n    def forward(self, X, state):\n        raise NotImplementedError\n\nclass EncoderDecoder(nn.Module):\n    \"\"\"编码器-解码器架构的基类\"\"\"\n    def __init__(self, encoder, decoder, **kwargs):\n        super(EncoderDecoder, self).__init__(**kwargs)\n        self.encoder = encoder\n        self.decoder = decoder\n\n    def forward(self, enc_X, dec_X, *args):\n        enc_outputs = self.encoder(enc_X, *args)\n        dec_state = self.decoder.init_state(enc_outputs, *args)\n        return self.decoder(dec_X, dec_state)\n",[2507,8754,8755,8772,8777,8795,8814,8818,8842,8850,8854,8871,8876,8894,8913,8917,8941,8947,8951,8972,8978,8982,8999,9004,9032,9051,9063,9075,9079,9107,9124,9140],{"__ignoreMap":89},[170,8756,8757,8759,8762,8764,8766,8768,8770],{"class":2511,"line":2512},[170,8758,2738],{"class":2515},[170,8760,8761],{"class":2741}," Encoder",[170,8763,909],{"class":2523},[170,8765,2748],{"class":2747},[170,8767,975],{"class":2523},[170,8769,2753],{"class":2747},[170,8771,2536],{"class":2523},[170,8773,8774],{"class":2511,"line":90},[170,8775,8776],{"class":2541},"    \"\"\"编码器-解码器架构的基本编码器接口\"\"\"\n",[170,8778,8779,8781,8783,8785,8787,8789,8791,8793],{"class":2511,"line":96},[170,8780,2765],{"class":2515},[170,8782,2768],{"class":2563},[170,8784,909],{"class":2523},[170,8786,2774],{"class":2773},[170,8788,2530],{"class":2523},[170,8790,2784],{"class":2515},[170,8792,2787],{"class":2526},[170,8794,2536],{"class":2523},[170,8796,8797,8799,8802,8804,8806,8808,8810,8812],{"class":2511,"line":2551},[170,8798,2795],{"class":2794},[170,8800,8801],{"class":2523},"(Encoder, ",[170,8803,2774],{"class":2801},[170,8805,2804],{"class":2523},[170,8807,2807],{"class":2563},[170,8809,909],{"class":2523},[170,8811,2784],{"class":2515},[170,8813,2814],{"class":2523},[170,8815,8816],{"class":2511,"line":2570},[170,8817,2832],{"emptyLinePlaceholder":114},[170,8819,8820,8822,8824,8826,8828,8830,8832,8834,8837,8840],{"class":2511,"line":2591},[170,8821,2765],{"class":2515},[170,8823,2859],{"class":2519},[170,8825,909],{"class":2523},[170,8827,2774],{"class":2773},[170,8829,2530],{"class":2523},[170,8831,2527],{"class":2526},[170,8833,2530],{"class":2523},[170,8835,8836],{"class":2515},"*",[170,8838,8839],{"class":2526},"args",[170,8841,2536],{"class":2523},[170,8843,8844,8847],{"class":2511,"line":2599},[170,8845,8846],{"class":2515},"        raise",[170,8848,8849],{"class":2794}," NotImplementedError\n",[170,8851,8852],{"class":2511,"line":2610},[170,8853,2832],{"emptyLinePlaceholder":114},[170,8855,8856,8858,8861,8863,8865,8867,8869],{"class":2511,"line":2627},[170,8857,2738],{"class":2515},[170,8859,8860],{"class":2741}," Decoder",[170,8862,909],{"class":2523},[170,8864,2748],{"class":2747},[170,8866,975],{"class":2523},[170,8868,2753],{"class":2747},[170,8870,2536],{"class":2523},[170,8872,8873],{"class":2511,"line":2643},[170,8874,8875],{"class":2541},"    \"\"\"编码器-解码器架构的基本解码器接口\"\"\"\n",[170,8877,8878,8880,8882,8884,8886,8888,8890,8892],{"class":2511,"line":2651},[170,8879,2765],{"class":2515},[170,8881,2768],{"class":2563},[170,8883,909],{"class":2523},[170,8885,2774],{"class":2773},[170,8887,2530],{"class":2523},[170,8889,2784],{"class":2515},[170,8891,2787],{"class":2526},[170,8893,2536],{"class":2523},[170,8895,8896,8898,8901,8903,8905,8907,8909,8911],{"class":2511,"line":2668},[170,8897,2795],{"class":2794},[170,8899,8900],{"class":2523},"(Decoder, ",[170,8902,2774],{"class":2801},[170,8904,2804],{"class":2523},[170,8906,2807],{"class":2563},[170,8908,909],{"class":2523},[170,8910,2784],{"class":2515},[170,8912,2814],{"class":2523},[170,8914,8915],{"class":2511,"line":2674},[170,8916,2832],{"emptyLinePlaceholder":114},[170,8918,8919,8921,8924,8926,8928,8930,8933,8935,8937,8939],{"class":2511,"line":2699},[170,8920,2765],{"class":2515},[170,8922,8923],{"class":2519}," init_state",[170,8925,909],{"class":2523},[170,8927,2774],{"class":2773},[170,8929,2530],{"class":2523},[170,8931,8932],{"class":2526},"enc_outputs",[170,8934,2530],{"class":2523},[170,8936,8836],{"class":2515},[170,8938,8839],{"class":2526},[170,8940,2536],{"class":2523},[170,8942,8943,8945],{"class":2511,"line":2712},[170,8944,8846],{"class":2515},[170,8946,8849],{"class":2794},[170,8948,8949],{"class":2511,"line":2952},[170,8950,2832],{"emptyLinePlaceholder":114},[170,8952,8953,8955,8957,8959,8961,8963,8965,8967,8970],{"class":2511,"line":4888},[170,8954,2765],{"class":2515},[170,8956,2859],{"class":2519},[170,8958,909],{"class":2523},[170,8960,2774],{"class":2773},[170,8962,2530],{"class":2523},[170,8964,2527],{"class":2526},[170,8966,2530],{"class":2523},[170,8968,8969],{"class":2526},"state",[170,8971,2536],{"class":2523},[170,8973,8974,8976],{"class":2511,"line":5164},[170,8975,8846],{"class":2515},[170,8977,8849],{"class":2794},[170,8979,8980],{"class":2511,"line":5170},[170,8981,2832],{"emptyLinePlaceholder":114},[170,8983,8984,8986,8989,8991,8993,8995,8997],{"class":2511,"line":5176},[170,8985,2738],{"class":2515},[170,8987,8988],{"class":2741}," EncoderDecoder",[170,8990,909],{"class":2523},[170,8992,2748],{"class":2747},[170,8994,975],{"class":2523},[170,8996,2753],{"class":2747},[170,8998,2536],{"class":2523},[170,9000,9001],{"class":2511,"line":5182},[170,9002,9003],{"class":2541},"    \"\"\"编码器-解码器架构的基类\"\"\"\n",[170,9005,9006,9008,9010,9012,9014,9016,9019,9021,9024,9026,9028,9030],{"class":2511,"line":5203},[170,9007,2765],{"class":2515},[170,9009,2768],{"class":2563},[170,9011,909],{"class":2523},[170,9013,2774],{"class":2773},[170,9015,2530],{"class":2523},[170,9017,9018],{"class":2526},"encoder",[170,9020,2530],{"class":2523},[170,9022,9023],{"class":2526},"decoder",[170,9025,2530],{"class":2523},[170,9027,2784],{"class":2515},[170,9029,2787],{"class":2526},[170,9031,2536],{"class":2523},[170,9033,9034,9036,9039,9041,9043,9045,9047,9049],{"class":2511,"line":5222},[170,9035,2795],{"class":2794},[170,9037,9038],{"class":2523},"(EncoderDecoder, ",[170,9040,2774],{"class":2801},[170,9042,2804],{"class":2523},[170,9044,2807],{"class":2563},[170,9046,909],{"class":2523},[170,9048,2784],{"class":2515},[170,9050,2814],{"class":2523},[170,9052,9053,9055,9058,9060],{"class":2511,"line":5241},[170,9054,2819],{"class":2801},[170,9056,9057],{"class":2523},".encoder ",[170,9059,926],{"class":2515},[170,9061,9062],{"class":2523}," encoder\n",[170,9064,9065,9067,9070,9072],{"class":2511,"line":5246},[170,9066,2819],{"class":2801},[170,9068,9069],{"class":2523},".decoder ",[170,9071,926],{"class":2515},[170,9073,9074],{"class":2523}," decoder\n",[170,9076,9077],{"class":2511,"line":5262},[170,9078,2832],{"emptyLinePlaceholder":114},[170,9080,9081,9083,9085,9087,9089,9091,9094,9096,9099,9101,9103,9105],{"class":2511,"line":5268},[170,9082,2765],{"class":2515},[170,9084,2859],{"class":2519},[170,9086,909],{"class":2523},[170,9088,2774],{"class":2773},[170,9090,2530],{"class":2523},[170,9092,9093],{"class":2526},"enc_X",[170,9095,2530],{"class":2523},[170,9097,9098],{"class":2526},"dec_X",[170,9100,2530],{"class":2523},[170,9102,8836],{"class":2515},[170,9104,8839],{"class":2526},[170,9106,2536],{"class":2523},[170,9108,9109,9112,9114,9116,9119,9121],{"class":2511,"line":5274},[170,9110,9111],{"class":2523},"        enc_outputs ",[170,9113,926],{"class":2515},[170,9115,5327],{"class":2801},[170,9117,9118],{"class":2523},".encoder(enc_X, ",[170,9120,8836],{"class":2515},[170,9122,9123],{"class":2523},"args)\n",[170,9125,9126,9129,9131,9133,9136,9138],{"class":2511,"line":5284},[170,9127,9128],{"class":2523},"        dec_state ",[170,9130,926],{"class":2515},[170,9132,5327],{"class":2801},[170,9134,9135],{"class":2523},".decoder.init_state(enc_outputs, ",[170,9137,8836],{"class":2515},[170,9139,9123],{"class":2523},[170,9141,9142,9144,9146],{"class":2511,"line":5308},[170,9143,2573],{"class":2515},[170,9145,5327],{"class":2801},[170,9147,9148],{"class":2523},".decoder(dec_X, dec_state)\n",[10,9150,9152],{"id":9151},"transformer-编码器","transformer 编码器",[14,9154,9155],{},"代码实现如下：",[2501,9157,9159],{"className":2503,"code":9158,"language":2505,"meta":89,"style":89},"class EncoderBlock(nn.Module):\n    \"\"\"Transformer编码器块\"\"\"\n    def __init__(self, key_size, query_size, value_size, num_hiddens,\n                 norm_shape, ffn_num_input, ffn_num_hiddens, num_heads,\n                 dropout, use_bias=False, **kwargs):\n        super(EncoderBlock, self).__init__(**kwargs)\n        self.attention = d2l.MultiHeadAttention(\n            key_size, query_size, value_size, num_hiddens, num_heads, dropout,\n            use_bias)\n        self.addnorm1 = AddNorm(norm_shape, dropout)\n        self.ffn = PositionWiseFFN(\n            ffn_num_input, ffn_num_hiddens, num_hiddens)\n        self.addnorm2 = AddNorm(norm_shape, dropout)\n\n    def forward(self, X, valid_lens):\n        Y = self.addnorm1(X, self.attention(X, X, X, valid_lens))\n        return self.addnorm2(Y, self.ffn(Y))\n\nclass TransformerEncoder(Encoder):\n    \"\"\"Transformer编码器\"\"\"\n    def __init__(self, vocab_size, key_size, query_size, value_size,\n                 num_hiddens, norm_shape, ffn_num_input, ffn_num_hiddens,\n                 num_heads, num_layers, dropout, use_bias=False, **kwargs):\n        super(TransformerEncoder, self).__init__(**kwargs)\n        self.num_hiddens = num_hiddens\n        self.embedding = nn.Embedding(vocab_size, num_hiddens)\n        self.pos_encoding = PositionalEncoding(num_hiddens, dropout)\n        self.blks = nn.Sequential()\n        for i in range(num_layers):\n            self.blks.add_module(\"block\"+str(i),\n                EncoderBlock(key_size, query_size, value_size, num_hiddens,\n                             norm_shape, ffn_num_input, ffn_num_hiddens,\n                             num_heads, dropout, use_bias))\n\n    def forward(self, X, valid_lens, *args):\n        # 因为位置编码值在-1和1之间，\n        # 因此嵌入值乘以嵌入维度的平方根进行缩放，\n        # 然后再与位置编码相加。\n        X = self.pos_encoding(self.embedding(X) * math.sqrt(self.num_hiddens))\n        self.attention_weights = [None] * len(self.blks)\n        for i, blk in enumerate(self.blks):\n            X = blk(X, valid_lens)\n            self.attention_weights[i] = blk.attention.attention.attention_weights\n        return X\n",[2507,9160,9161,9178,9183,9211,9230,9252,9271,9282,9287,9292,9304,9316,9321,9332,9336,9356,9373,9387,9391,9405,9410,9439,9459,9488,9507,9519,9531,9543,9555,9573,9599,9604,9609,9614,9618,9644,9649,9655,9661,9688,9716,9736,9747,9760],{"__ignoreMap":89},[170,9162,9163,9165,9168,9170,9172,9174,9176],{"class":2511,"line":2512},[170,9164,2738],{"class":2515},[170,9166,9167],{"class":2741}," EncoderBlock",[170,9169,909],{"class":2523},[170,9171,2748],{"class":2747},[170,9173,975],{"class":2523},[170,9175,2753],{"class":2747},[170,9177,2536],{"class":2523},[170,9179,9180],{"class":2511,"line":90},[170,9181,9182],{"class":2541},"    \"\"\"Transformer编码器块\"\"\"\n",[170,9184,9185,9187,9189,9191,9193,9195,9197,9199,9201,9203,9205,9207,9209],{"class":2511,"line":96},[170,9186,2765],{"class":2515},[170,9188,2768],{"class":2563},[170,9190,909],{"class":2523},[170,9192,2774],{"class":2773},[170,9194,2530],{"class":2523},[170,9196,4954],{"class":2526},[170,9198,2530],{"class":2523},[170,9200,4959],{"class":2526},[170,9202,2530],{"class":2523},[170,9204,4964],{"class":2526},[170,9206,2530],{"class":2523},[170,9208,4969],{"class":2526},[170,9210,3933],{"class":2523},[170,9212,9213,9216,9218,9220,9222,9224,9226,9228],{"class":2511,"line":2551},[170,9214,9215],{"class":2526},"                 norm_shape",[170,9217,2530],{"class":2523},[170,9219,7782],{"class":2526},[170,9221,2530],{"class":2523},[170,9223,7787],{"class":2526},[170,9225,2530],{"class":2523},[170,9227,4702],{"class":2526},[170,9229,3933],{"class":2523},[170,9231,9232,9235,9237,9240,9242,9244,9246,9248,9250],{"class":2511,"line":2570},[170,9233,9234],{"class":2526},"                 dropout",[170,9236,2530],{"class":2523},[170,9238,9239],{"class":2526},"use_bias",[170,9241,926],{"class":2515},[170,9243,4990],{"class":2563},[170,9245,2530],{"class":2523},[170,9247,2784],{"class":2515},[170,9249,2787],{"class":2526},[170,9251,2536],{"class":2523},[170,9253,9254,9256,9259,9261,9263,9265,9267,9269],{"class":2511,"line":2591},[170,9255,2795],{"class":2794},[170,9257,9258],{"class":2523},"(EncoderBlock, ",[170,9260,2774],{"class":2801},[170,9262,2804],{"class":2523},[170,9264,2807],{"class":2563},[170,9266,909],{"class":2523},[170,9268,2784],{"class":2515},[170,9270,2814],{"class":2523},[170,9272,9273,9275,9277,9279],{"class":2511,"line":2599},[170,9274,2819],{"class":2801},[170,9276,5036],{"class":2523},[170,9278,926],{"class":2515},[170,9280,9281],{"class":2523}," d2l.MultiHeadAttention(\n",[170,9283,9284],{"class":2511,"line":2610},[170,9285,9286],{"class":2523},"            key_size, query_size, value_size, num_hiddens, num_heads, dropout,\n",[170,9288,9289],{"class":2511,"line":2627},[170,9290,9291],{"class":2523},"            use_bias)\n",[170,9293,9294,9296,9299,9301],{"class":2511,"line":2643},[170,9295,2819],{"class":2801},[170,9297,9298],{"class":2523},".addnorm1 ",[170,9300,926],{"class":2515},[170,9302,9303],{"class":2523}," AddNorm(norm_shape, dropout)\n",[170,9305,9306,9308,9311,9313],{"class":2511,"line":2651},[170,9307,2819],{"class":2801},[170,9309,9310],{"class":2523},".ffn ",[170,9312,926],{"class":2515},[170,9314,9315],{"class":2523}," PositionWiseFFN(\n",[170,9317,9318],{"class":2511,"line":2668},[170,9319,9320],{"class":2523},"            ffn_num_input, ffn_num_hiddens, num_hiddens)\n",[170,9322,9323,9325,9328,9330],{"class":2511,"line":2674},[170,9324,2819],{"class":2801},[170,9326,9327],{"class":2523},".addnorm2 ",[170,9329,926],{"class":2515},[170,9331,9303],{"class":2523},[170,9333,9334],{"class":2511,"line":2699},[170,9335,2832],{"emptyLinePlaceholder":114},[170,9337,9338,9340,9342,9344,9346,9348,9350,9352,9354],{"class":2511,"line":2712},[170,9339,2765],{"class":2515},[170,9341,2859],{"class":2519},[170,9343,909],{"class":2523},[170,9345,2774],{"class":2773},[170,9347,2530],{"class":2523},[170,9349,2527],{"class":2526},[170,9351,2530],{"class":2523},[170,9353,2533],{"class":2526},[170,9355,2536],{"class":2523},[170,9357,9358,9361,9363,9365,9368,9370],{"class":2511,"line":2952},[170,9359,9360],{"class":2523},"        Y ",[170,9362,926],{"class":2515},[170,9364,5327],{"class":2801},[170,9366,9367],{"class":2523},".addnorm1(X, ",[170,9369,2774],{"class":2801},[170,9371,9372],{"class":2523},".attention(X, X, X, valid_lens))\n",[170,9374,9375,9377,9379,9382,9384],{"class":2511,"line":4888},[170,9376,2573],{"class":2515},[170,9378,5327],{"class":2801},[170,9380,9381],{"class":2523},".addnorm2(Y, ",[170,9383,2774],{"class":2801},[170,9385,9386],{"class":2523},".ffn(Y))\n",[170,9388,9389],{"class":2511,"line":5164},[170,9390,2832],{"emptyLinePlaceholder":114},[170,9392,9393,9395,9398,9400,9403],{"class":2511,"line":5170},[170,9394,2738],{"class":2515},[170,9396,9397],{"class":2741}," TransformerEncoder",[170,9399,909],{"class":2523},[170,9401,9402],{"class":2747},"Encoder",[170,9404,2536],{"class":2523},[170,9406,9407],{"class":2511,"line":5176},[170,9408,9409],{"class":2541},"    \"\"\"Transformer编码器\"\"\"\n",[170,9411,9412,9414,9416,9418,9420,9422,9425,9427,9429,9431,9433,9435,9437],{"class":2511,"line":5182},[170,9413,2765],{"class":2515},[170,9415,2768],{"class":2563},[170,9417,909],{"class":2523},[170,9419,2774],{"class":2773},[170,9421,2530],{"class":2523},[170,9423,9424],{"class":2526},"vocab_size",[170,9426,2530],{"class":2523},[170,9428,4954],{"class":2526},[170,9430,2530],{"class":2523},[170,9432,4959],{"class":2526},[170,9434,2530],{"class":2523},[170,9436,4964],{"class":2526},[170,9438,3933],{"class":2523},[170,9440,9441,9444,9446,9449,9451,9453,9455,9457],{"class":2511,"line":5203},[170,9442,9443],{"class":2526},"                 num_hiddens",[170,9445,2530],{"class":2523},[170,9447,9448],{"class":2526},"norm_shape",[170,9450,2530],{"class":2523},[170,9452,7782],{"class":2526},[170,9454,2530],{"class":2523},[170,9456,7787],{"class":2526},[170,9458,3933],{"class":2523},[170,9460,9461,9463,9465,9468,9470,9472,9474,9476,9478,9480,9482,9484,9486],{"class":2511,"line":5222},[170,9462,4976],{"class":2526},[170,9464,2530],{"class":2523},[170,9466,9467],{"class":2526},"num_layers",[170,9469,2530],{"class":2523},[170,9471,2779],{"class":2526},[170,9473,2530],{"class":2523},[170,9475,9239],{"class":2526},[170,9477,926],{"class":2515},[170,9479,4990],{"class":2563},[170,9481,2530],{"class":2523},[170,9483,2784],{"class":2515},[170,9485,2787],{"class":2526},[170,9487,2536],{"class":2523},[170,9489,9490,9492,9495,9497,9499,9501,9503,9505],{"class":2511,"line":5241},[170,9491,2795],{"class":2794},[170,9493,9494],{"class":2523},"(TransformerEncoder, ",[170,9496,2774],{"class":2801},[170,9498,2804],{"class":2523},[170,9500,2807],{"class":2563},[170,9502,909],{"class":2523},[170,9504,2784],{"class":2515},[170,9506,2814],{"class":2523},[170,9508,9509,9511,9514,9516],{"class":2511,"line":5246},[170,9510,2819],{"class":2801},[170,9512,9513],{"class":2523},".num_hiddens ",[170,9515,926],{"class":2515},[170,9517,9518],{"class":2523}," num_hiddens\n",[170,9520,9521,9523,9526,9528],{"class":2511,"line":5262},[170,9522,2819],{"class":2801},[170,9524,9525],{"class":2523},".embedding ",[170,9527,926],{"class":2515},[170,9529,9530],{"class":2523}," nn.Embedding(vocab_size, num_hiddens)\n",[170,9532,9533,9535,9538,9540],{"class":2511,"line":5268},[170,9534,2819],{"class":2801},[170,9536,9537],{"class":2523},".pos_encoding ",[170,9539,926],{"class":2515},[170,9541,9542],{"class":2523}," PositionalEncoding(num_hiddens, dropout)\n",[170,9544,9545,9547,9550,9552],{"class":2511,"line":5274},[170,9546,2819],{"class":2801},[170,9548,9549],{"class":2523},".blks ",[170,9551,926],{"class":2515},[170,9553,9554],{"class":2523}," nn.Sequential()\n",[170,9556,9557,9560,9563,9566,9570],{"class":2511,"line":5284},[170,9558,9559],{"class":2515},"        for",[170,9561,9562],{"class":2523}," i ",[170,9564,9565],{"class":2515},"in",[170,9567,9569],{"class":9568},"sDgm9"," range",[170,9571,9572],{"class":2523},"(num_layers):\n",[170,9574,9575,9578,9581,9585,9589,9591,9593,9596],{"class":2511,"line":5308},[170,9576,9577],{"class":2801},"            self",[170,9579,9580],{"class":2523},".blks.add_module(",[170,9582,9584],{"class":9583},"sMWOi","\"",[170,9586,9588],{"class":9587},"sEzAm","block",[170,9590,9584],{"class":9583},[170,9592,5884],{"class":2515},[170,9594,9595],{"class":2794},"str",[170,9597,9598],{"class":2523},"(i),\n",[170,9600,9601],{"class":2511,"line":5313},[170,9602,9603],{"class":2523},"                EncoderBlock(key_size, query_size, value_size, num_hiddens,\n",[170,9605,9606],{"class":2511,"line":5319},[170,9607,9608],{"class":2523},"                             norm_shape, ffn_num_input, ffn_num_hiddens,\n",[170,9610,9611],{"class":2511,"line":5333},[170,9612,9613],{"class":2523},"                             num_heads, dropout, use_bias))\n",[170,9615,9616],{"class":2511,"line":5338},[170,9617,2832],{"emptyLinePlaceholder":114},[170,9619,9620,9622,9624,9626,9628,9630,9632,9634,9636,9638,9640,9642],{"class":2511,"line":5344},[170,9621,2765],{"class":2515},[170,9623,2859],{"class":2519},[170,9625,909],{"class":2523},[170,9627,2774],{"class":2773},[170,9629,2530],{"class":2523},[170,9631,2527],{"class":2526},[170,9633,2530],{"class":2523},[170,9635,2533],{"class":2526},[170,9637,2530],{"class":2523},[170,9639,8836],{"class":2515},[170,9641,8839],{"class":2526},[170,9643,2536],{"class":2523},[170,9645,9646],{"class":2511,"line":5359},[170,9647,9648],{"class":2547},"        # 因为位置编码值在-1和1之间，\n",[170,9650,9652],{"class":2511,"line":9651},37,[170,9653,9654],{"class":2547},"        # 因此嵌入值乘以嵌入维度的平方根进行缩放，\n",[170,9656,9658],{"class":2511,"line":9657},38,[170,9659,9660],{"class":2547},"        # 然后再与位置编码相加。\n",[170,9662,9664,9666,9668,9670,9673,9675,9678,9680,9683,9685],{"class":2511,"line":9663},39,[170,9665,2677],{"class":2523},[170,9667,926],{"class":2515},[170,9669,5327],{"class":2801},[170,9671,9672],{"class":2523},".pos_encoding(",[170,9674,2774],{"class":2801},[170,9676,9677],{"class":2523},".embedding(X) ",[170,9679,8836],{"class":2515},[170,9681,9682],{"class":2523}," math.sqrt(",[170,9684,2774],{"class":2801},[170,9686,9687],{"class":2523},".num_hiddens))\n",[170,9689,9691,9693,9695,9697,9700,9702,9704,9706,9709,9711,9713],{"class":2511,"line":9690},40,[170,9692,2819],{"class":2801},[170,9694,2944],{"class":2523},[170,9696,926],{"class":2515},[170,9698,9699],{"class":2523}," [",[170,9701,2887],{"class":2563},[170,9703,7307],{"class":2523},[170,9705,8836],{"class":2515},[170,9707,9708],{"class":9568}," len",[170,9710,909],{"class":2523},[170,9712,2774],{"class":2801},[170,9714,9715],{"class":2523},".blks)\n",[170,9717,9719,9721,9724,9726,9729,9731,9733],{"class":2511,"line":9718},41,[170,9720,9559],{"class":2515},[170,9722,9723],{"class":2523}," i, blk ",[170,9725,9565],{"class":2515},[170,9727,9728],{"class":9568}," enumerate",[170,9730,909],{"class":2523},[170,9732,2774],{"class":2801},[170,9734,9735],{"class":2523},".blks):\n",[170,9737,9739,9742,9744],{"class":2511,"line":9738},42,[170,9740,9741],{"class":2523},"            X ",[170,9743,926],{"class":2515},[170,9745,9746],{"class":2523}," blk(X, valid_lens)\n",[170,9748,9750,9752,9755,9757],{"class":2511,"line":9749},43,[170,9751,9577],{"class":2801},[170,9753,9754],{"class":2523},".attention_weights[i] ",[170,9756,926],{"class":2515},[170,9758,9759],{"class":2523}," blk.attention.attention.attention_weights\n",[170,9761,9763,9765],{"class":2511,"line":9762},44,[170,9764,2573],{"class":2515},[170,9766,9767],{"class":2523}," X\n",[10,9769,9771],{"id":9770},"transformer-解码器","transformer 解码器",[14,9773,9774],{},"代码如下：",[2501,9776,9778],{"className":2503,"code":9777,"language":2505,"meta":89,"style":89},"class DecoderBlock(nn.Module):\n    \"\"\"解码器中第i个块\"\"\"\n    def __init__(self, key_size, query_size, value_size, num_hiddens,\n                 norm_shape, ffn_num_input, ffn_num_hiddens, num_heads,\n                 dropout, i, **kwargs):\n        super(DecoderBlock, self).__init__(**kwargs)\n        self.i = i\n        self.attention1 = d2l.MultiHeadAttention(\n            key_size, query_size, value_size, num_hiddens, num_heads, dropout)\n        self.addnorm1 = AddNorm(norm_shape, dropout)\n        self.attention2 = d2l.MultiHeadAttention(\n            key_size, query_size, value_size, num_hiddens, num_heads, dropout)\n        self.addnorm2 = AddNorm(norm_shape, dropout)\n        self.ffn = PositionWiseFFN(ffn_num_input, ffn_num_hiddens,\n                                   num_hiddens)\n        self.addnorm3 = AddNorm(norm_shape, dropout)\n\n    def forward(self, X, state):\n        enc_outputs, enc_valid_lens = state[0], state[1]\n        # 训练阶段，输出序列的所有词元都在同一时间处理，\n        # 因此state[2][self.i]初始化为None。\n        # 预测阶段，输出序列是通过词元一个接着一个解码的，\n        # 因此state[2][self.i]包含着直到当前时间步第i个块解码的输出表示\n        if state[2][self.i] is None:\n            key_values = X\n        else:\n            key_values = torch.cat((state[2][self.i], X), axis=1)\n        state[2][self.i] = key_values\n        if self.training:\n            batch_size, num_steps, _ = X.shape\n            # dec_valid_lens的开头:(batch_size,num_steps),\n            # 其中每一行是[1,2,...,num_steps]\n            dec_valid_lens = torch.arange(\n                1, num_steps + 1, device=X.device).repeat(batch_size, 1)\n        else:\n            dec_valid_lens = None\n\n        # 自注意力\n        X2 = self.attention1(X, key_values, key_values, dec_valid_lens)\n        Y = self.addnorm1(X, X2)\n        # 编码器－解码器注意力。\n        # enc_outputs的开头:(batch_size,num_steps,num_hiddens)\n        Y2 = self.attention2(Y, enc_outputs, enc_outputs, enc_valid_lens)\n        Z = self.addnorm2(Y, Y2)\n        return self.addnorm3(Z, self.ffn(Z)), state\n\nclass TransformerDecoder(AttentionDecoder):\n    def __init__(self, vocab_size, key_size, query_size, value_size,\n                 num_hiddens, norm_shape, ffn_num_input, ffn_num_hiddens,\n                 num_heads, num_layers, dropout, **kwargs):\n        super(TransformerDecoder, self).__init__(**kwargs)\n        self.num_hiddens = num_hiddens\n        self.num_layers = num_layers\n        self.embedding = nn.Embedding(vocab_size, num_hiddens)\n        self.pos_encoding = PositionalEncoding(num_hiddens, dropout)\n        self.blks = nn.Sequential()\n        for i in range(num_layers):\n            self.blks.add_module(\"block\"+str(i),\n                DecoderBlock(key_size, query_size, value_size, num_hiddens,\n                             norm_shape, ffn_num_input, ffn_num_hiddens,\n                             num_heads, dropout, i))\n        self.dense = nn.Linear(num_hiddens, vocab_size)\n\n    def init_state(self, enc_outputs, enc_valid_lens, *args):\n        return [enc_outputs, enc_valid_lens, [None] * self.num_layers]\n\n    def forward(self, X, state):\n        X = self.pos_encoding(self.embedding(X) * math.sqrt(self.num_hiddens))\n        self._attention_weights = [[None] * len(self.blks) for _ in range (2)]\n        for i, blk in enumerate(self.blks):\n            X, state = blk(X, state)\n            # 解码器自注意力权重\n            self._attention_weights[0][i] = blk.attention1.attention.attention_weights\n            # “编码器－解码器”自注意力权重\n            self._attention_weights[1][i] = blk.attention2.attention.attention_weights\n        return self.dense(X), state\n\n    @property\n    def attention_weights(self):\n        return self._attention_weights\n",[2507,9779,9780,9797,9802,9830,9848,9864,9883,9895,9906,9911,9921,9932,9936,9946,9957,9962,9973,9977,9997,10016,10021,10026,10031,10036,10058,10067,10073,10100,10118,10127,10136,10141,10146,10156,10182,10188,10197,10201,10206,10218,10229,10234,10239,10251,10263,10278,10283,10298,10327,10346,10367,10387,10398,10411,10422,10433,10444,10457,10476,10482,10487,10493,10506,10511,10539,10558,10563,10584,10607,10653,10670,10681,10687,10705,10711,10727,10737,10742,10751,10765],{"__ignoreMap":89},[170,9781,9782,9784,9787,9789,9791,9793,9795],{"class":2511,"line":2512},[170,9783,2738],{"class":2515},[170,9785,9786],{"class":2741}," DecoderBlock",[170,9788,909],{"class":2523},[170,9790,2748],{"class":2747},[170,9792,975],{"class":2523},[170,9794,2753],{"class":2747},[170,9796,2536],{"class":2523},[170,9798,9799],{"class":2511,"line":90},[170,9800,9801],{"class":2541},"    \"\"\"解码器中第i个块\"\"\"\n",[170,9803,9804,9806,9808,9810,9812,9814,9816,9818,9820,9822,9824,9826,9828],{"class":2511,"line":96},[170,9805,2765],{"class":2515},[170,9807,2768],{"class":2563},[170,9809,909],{"class":2523},[170,9811,2774],{"class":2773},[170,9813,2530],{"class":2523},[170,9815,4954],{"class":2526},[170,9817,2530],{"class":2523},[170,9819,4959],{"class":2526},[170,9821,2530],{"class":2523},[170,9823,4964],{"class":2526},[170,9825,2530],{"class":2523},[170,9827,4969],{"class":2526},[170,9829,3933],{"class":2523},[170,9831,9832,9834,9836,9838,9840,9842,9844,9846],{"class":2511,"line":2551},[170,9833,9215],{"class":2526},[170,9835,2530],{"class":2523},[170,9837,7782],{"class":2526},[170,9839,2530],{"class":2523},[170,9841,7787],{"class":2526},[170,9843,2530],{"class":2523},[170,9845,4702],{"class":2526},[170,9847,3933],{"class":2523},[170,9849,9850,9852,9854,9856,9858,9860,9862],{"class":2511,"line":2570},[170,9851,9234],{"class":2526},[170,9853,2530],{"class":2523},[170,9855,900],{"class":2526},[170,9857,2530],{"class":2523},[170,9859,2784],{"class":2515},[170,9861,2787],{"class":2526},[170,9863,2536],{"class":2523},[170,9865,9866,9868,9871,9873,9875,9877,9879,9881],{"class":2511,"line":2591},[170,9867,2795],{"class":2794},[170,9869,9870],{"class":2523},"(DecoderBlock, ",[170,9872,2774],{"class":2801},[170,9874,2804],{"class":2523},[170,9876,2807],{"class":2563},[170,9878,909],{"class":2523},[170,9880,2784],{"class":2515},[170,9882,2814],{"class":2523},[170,9884,9885,9887,9890,9892],{"class":2511,"line":2599},[170,9886,2819],{"class":2801},[170,9888,9889],{"class":2523},".i ",[170,9891,926],{"class":2515},[170,9893,9894],{"class":2523}," i\n",[170,9896,9897,9899,9902,9904],{"class":2511,"line":2610},[170,9898,2819],{"class":2801},[170,9900,9901],{"class":2523},".attention1 ",[170,9903,926],{"class":2515},[170,9905,9281],{"class":2523},[170,9907,9908],{"class":2511,"line":2627},[170,9909,9910],{"class":2523},"            key_size, query_size, value_size, num_hiddens, num_heads, dropout)\n",[170,9912,9913,9915,9917,9919],{"class":2511,"line":2643},[170,9914,2819],{"class":2801},[170,9916,9298],{"class":2523},[170,9918,926],{"class":2515},[170,9920,9303],{"class":2523},[170,9922,9923,9925,9928,9930],{"class":2511,"line":2651},[170,9924,2819],{"class":2801},[170,9926,9927],{"class":2523},".attention2 ",[170,9929,926],{"class":2515},[170,9931,9281],{"class":2523},[170,9933,9934],{"class":2511,"line":2668},[170,9935,9910],{"class":2523},[170,9937,9938,9940,9942,9944],{"class":2511,"line":2674},[170,9939,2819],{"class":2801},[170,9941,9327],{"class":2523},[170,9943,926],{"class":2515},[170,9945,9303],{"class":2523},[170,9947,9948,9950,9952,9954],{"class":2511,"line":2699},[170,9949,2819],{"class":2801},[170,9951,9310],{"class":2523},[170,9953,926],{"class":2515},[170,9955,9956],{"class":2523}," PositionWiseFFN(ffn_num_input, ffn_num_hiddens,\n",[170,9958,9959],{"class":2511,"line":2712},[170,9960,9961],{"class":2523},"                                   num_hiddens)\n",[170,9963,9964,9966,9969,9971],{"class":2511,"line":2952},[170,9965,2819],{"class":2801},[170,9967,9968],{"class":2523},".addnorm3 ",[170,9970,926],{"class":2515},[170,9972,9303],{"class":2523},[170,9974,9975],{"class":2511,"line":4888},[170,9976,2832],{"emptyLinePlaceholder":114},[170,9978,9979,9981,9983,9985,9987,9989,9991,9993,9995],{"class":2511,"line":5164},[170,9980,2765],{"class":2515},[170,9982,2859],{"class":2519},[170,9984,909],{"class":2523},[170,9986,2774],{"class":2773},[170,9988,2530],{"class":2523},[170,9990,2527],{"class":2526},[170,9992,2530],{"class":2523},[170,9994,8969],{"class":2526},[170,9996,2536],{"class":2523},[170,9998,9999,10002,10004,10007,10009,10012,10014],{"class":2511,"line":5170},[170,10000,10001],{"class":2523},"        enc_outputs, enc_valid_lens ",[170,10003,926],{"class":2515},[170,10005,10006],{"class":2523}," state[",[170,10008,1865],{"class":2563},[170,10010,10011],{"class":2523},"], state[",[170,10013,194],{"class":2563},[170,10015,2906],{"class":2523},[170,10017,10018],{"class":2511,"line":5176},[170,10019,10020],{"class":2547},"        # 训练阶段，输出序列的所有词元都在同一时间处理，\n",[170,10022,10023],{"class":2511,"line":5182},[170,10024,10025],{"class":2547},"        # 因此state[2][self.i]初始化为None。\n",[170,10027,10028],{"class":2511,"line":5203},[170,10029,10030],{"class":2547},"        # 预测阶段，输出序列是通过词元一个接着一个解码的，\n",[170,10032,10033],{"class":2511,"line":5222},[170,10034,10035],{"class":2547},"        # 因此state[2][self.i]包含着直到当前时间步第i个块解码的输出表示\n",[170,10037,10038,10040,10042,10044,10047,10049,10052,10054,10056],{"class":2511,"line":5241},[170,10039,2613],{"class":2515},[170,10041,10006],{"class":2523},[170,10043,2928],{"class":2563},[170,10045,10046],{"class":2523},"][",[170,10048,2774],{"class":2801},[170,10050,10051],{"class":2523},".i] ",[170,10053,2560],{"class":2515},[170,10055,2564],{"class":2563},[170,10057,2567],{"class":2523},[170,10059,10060,10063,10065],{"class":2511,"line":5246},[170,10061,10062],{"class":2523},"            key_values ",[170,10064,926],{"class":2515},[170,10066,9767],{"class":2523},[170,10068,10069,10071],{"class":2511,"line":5262},[170,10070,2646],{"class":2515},[170,10072,2567],{"class":2523},[170,10074,10075,10077,10079,10082,10084,10086,10088,10091,10094,10096,10098],{"class":2511,"line":5268},[170,10076,10062],{"class":2523},[170,10078,926],{"class":2515},[170,10080,10081],{"class":2523}," torch.cat((state[",[170,10083,2928],{"class":2563},[170,10085,10046],{"class":2523},[170,10087,2774],{"class":2801},[170,10089,10090],{"class":2523},".i], X), ",[170,10092,10093],{"class":2579},"axis",[170,10095,926],{"class":2515},[170,10097,194],{"class":2563},[170,10099,2588],{"class":2523},[170,10101,10102,10105,10107,10109,10111,10113,10115],{"class":2511,"line":5274},[170,10103,10104],{"class":2523},"        state[",[170,10106,2928],{"class":2563},[170,10108,10046],{"class":2523},[170,10110,2774],{"class":2801},[170,10112,10051],{"class":2523},[170,10114,926],{"class":2515},[170,10116,10117],{"class":2523}," key_values\n",[170,10119,10120,10122,10124],{"class":2511,"line":5284},[170,10121,2613],{"class":2515},[170,10123,5327],{"class":2801},[170,10125,10126],{"class":2523},".training:\n",[170,10128,10129,10132,10134],{"class":2511,"line":5308},[170,10130,10131],{"class":2523},"            batch_size, num_steps, _ ",[170,10133,926],{"class":2515},[170,10135,2607],{"class":2523},[170,10137,10138],{"class":2511,"line":5313},[170,10139,10140],{"class":2547},"            # dec_valid_lens的开头:(batch_size,num_steps),\n",[170,10142,10143],{"class":2511,"line":5319},[170,10144,10145],{"class":2547},"            # 其中每一行是[1,2,...,num_steps]\n",[170,10147,10148,10151,10153],{"class":2511,"line":5333},[170,10149,10150],{"class":2523},"            dec_valid_lens ",[170,10152,926],{"class":2515},[170,10154,10155],{"class":2523}," torch.arange(\n",[170,10157,10158,10161,10164,10166,10168,10170,10173,10175,10178,10180],{"class":2511,"line":5338},[170,10159,10160],{"class":2563},"                1",[170,10162,10163],{"class":2523},", num_steps ",[170,10165,5884],{"class":2515},[170,10167,2622],{"class":2563},[170,10169,2530],{"class":2523},[170,10171,10172],{"class":2579},"device",[170,10174,926],{"class":2515},[170,10176,10177],{"class":2523},"X.device).repeat(batch_size, ",[170,10179,194],{"class":2563},[170,10181,2588],{"class":2523},[170,10183,10184,10186],{"class":2511,"line":5344},[170,10185,2646],{"class":2515},[170,10187,2567],{"class":2523},[170,10189,10190,10192,10194],{"class":2511,"line":5359},[170,10191,10150],{"class":2523},[170,10193,926],{"class":2515},[170,10195,10196],{"class":2563}," None\n",[170,10198,10199],{"class":2511,"line":9651},[170,10200,2832],{"emptyLinePlaceholder":114},[170,10202,10203],{"class":2511,"line":9657},[170,10204,10205],{"class":2547},"        # 自注意力\n",[170,10207,10208,10211,10213,10215],{"class":2511,"line":9663},[170,10209,10210],{"class":2523},"        X2 ",[170,10212,926],{"class":2515},[170,10214,5327],{"class":2801},[170,10216,10217],{"class":2523},".attention1(X, key_values, key_values, dec_valid_lens)\n",[170,10219,10220,10222,10224,10226],{"class":2511,"line":9690},[170,10221,9360],{"class":2523},[170,10223,926],{"class":2515},[170,10225,5327],{"class":2801},[170,10227,10228],{"class":2523},".addnorm1(X, X2)\n",[170,10230,10231],{"class":2511,"line":9718},[170,10232,10233],{"class":2547},"        # 编码器－解码器注意力。\n",[170,10235,10236],{"class":2511,"line":9738},[170,10237,10238],{"class":2547},"        # enc_outputs的开头:(batch_size,num_steps,num_hiddens)\n",[170,10240,10241,10244,10246,10248],{"class":2511,"line":9749},[170,10242,10243],{"class":2523},"        Y2 ",[170,10245,926],{"class":2515},[170,10247,5327],{"class":2801},[170,10249,10250],{"class":2523},".attention2(Y, enc_outputs, enc_outputs, enc_valid_lens)\n",[170,10252,10253,10256,10258,10260],{"class":2511,"line":9762},[170,10254,10255],{"class":2523},"        Z ",[170,10257,926],{"class":2515},[170,10259,5327],{"class":2801},[170,10261,10262],{"class":2523},".addnorm2(Y, Y2)\n",[170,10264,10266,10268,10270,10273,10275],{"class":2511,"line":10265},45,[170,10267,2573],{"class":2515},[170,10269,5327],{"class":2801},[170,10271,10272],{"class":2523},".addnorm3(Z, ",[170,10274,2774],{"class":2801},[170,10276,10277],{"class":2523},".ffn(Z)), state\n",[170,10279,10281],{"class":2511,"line":10280},46,[170,10282,2832],{"emptyLinePlaceholder":114},[170,10284,10286,10288,10291,10293,10296],{"class":2511,"line":10285},47,[170,10287,2738],{"class":2515},[170,10289,10290],{"class":2741}," TransformerDecoder",[170,10292,909],{"class":2523},[170,10294,10295],{"class":2747},"AttentionDecoder",[170,10297,2536],{"class":2523},[170,10299,10301,10303,10305,10307,10309,10311,10313,10315,10317,10319,10321,10323,10325],{"class":2511,"line":10300},48,[170,10302,2765],{"class":2515},[170,10304,2768],{"class":2563},[170,10306,909],{"class":2523},[170,10308,2774],{"class":2773},[170,10310,2530],{"class":2523},[170,10312,9424],{"class":2526},[170,10314,2530],{"class":2523},[170,10316,4954],{"class":2526},[170,10318,2530],{"class":2523},[170,10320,4959],{"class":2526},[170,10322,2530],{"class":2523},[170,10324,4964],{"class":2526},[170,10326,3933],{"class":2523},[170,10328,10330,10332,10334,10336,10338,10340,10342,10344],{"class":2511,"line":10329},49,[170,10331,9443],{"class":2526},[170,10333,2530],{"class":2523},[170,10335,9448],{"class":2526},[170,10337,2530],{"class":2523},[170,10339,7782],{"class":2526},[170,10341,2530],{"class":2523},[170,10343,7787],{"class":2526},[170,10345,3933],{"class":2523},[170,10347,10349,10351,10353,10355,10357,10359,10361,10363,10365],{"class":2511,"line":10348},50,[170,10350,4976],{"class":2526},[170,10352,2530],{"class":2523},[170,10354,9467],{"class":2526},[170,10356,2530],{"class":2523},[170,10358,2779],{"class":2526},[170,10360,2530],{"class":2523},[170,10362,2784],{"class":2515},[170,10364,2787],{"class":2526},[170,10366,2536],{"class":2523},[170,10368,10370,10372,10375,10377,10379,10381,10383,10385],{"class":2511,"line":10369},51,[170,10371,2795],{"class":2794},[170,10373,10374],{"class":2523},"(TransformerDecoder, ",[170,10376,2774],{"class":2801},[170,10378,2804],{"class":2523},[170,10380,2807],{"class":2563},[170,10382,909],{"class":2523},[170,10384,2784],{"class":2515},[170,10386,2814],{"class":2523},[170,10388,10390,10392,10394,10396],{"class":2511,"line":10389},52,[170,10391,2819],{"class":2801},[170,10393,9513],{"class":2523},[170,10395,926],{"class":2515},[170,10397,9518],{"class":2523},[170,10399,10401,10403,10406,10408],{"class":2511,"line":10400},53,[170,10402,2819],{"class":2801},[170,10404,10405],{"class":2523},".num_layers ",[170,10407,926],{"class":2515},[170,10409,10410],{"class":2523}," num_layers\n",[170,10412,10414,10416,10418,10420],{"class":2511,"line":10413},54,[170,10415,2819],{"class":2801},[170,10417,9525],{"class":2523},[170,10419,926],{"class":2515},[170,10421,9530],{"class":2523},[170,10423,10425,10427,10429,10431],{"class":2511,"line":10424},55,[170,10426,2819],{"class":2801},[170,10428,9537],{"class":2523},[170,10430,926],{"class":2515},[170,10432,9542],{"class":2523},[170,10434,10436,10438,10440,10442],{"class":2511,"line":10435},56,[170,10437,2819],{"class":2801},[170,10439,9549],{"class":2523},[170,10441,926],{"class":2515},[170,10443,9554],{"class":2523},[170,10445,10447,10449,10451,10453,10455],{"class":2511,"line":10446},57,[170,10448,9559],{"class":2515},[170,10450,9562],{"class":2523},[170,10452,9565],{"class":2515},[170,10454,9569],{"class":9568},[170,10456,9572],{"class":2523},[170,10458,10460,10462,10464,10466,10468,10470,10472,10474],{"class":2511,"line":10459},58,[170,10461,9577],{"class":2801},[170,10463,9580],{"class":2523},[170,10465,9584],{"class":9583},[170,10467,9588],{"class":9587},[170,10469,9584],{"class":9583},[170,10471,5884],{"class":2515},[170,10473,9595],{"class":2794},[170,10475,9598],{"class":2523},[170,10477,10479],{"class":2511,"line":10478},59,[170,10480,10481],{"class":2523},"                DecoderBlock(key_size, query_size, value_size, num_hiddens,\n",[170,10483,10485],{"class":2511,"line":10484},60,[170,10486,9608],{"class":2523},[170,10488,10490],{"class":2511,"line":10489},61,[170,10491,10492],{"class":2523},"                             num_heads, dropout, i))\n",[170,10494,10496,10498,10501,10503],{"class":2511,"line":10495},62,[170,10497,2819],{"class":2801},[170,10499,10500],{"class":2523},".dense ",[170,10502,926],{"class":2515},[170,10504,10505],{"class":2523}," nn.Linear(num_hiddens, vocab_size)\n",[170,10507,10509],{"class":2511,"line":10508},63,[170,10510,2832],{"emptyLinePlaceholder":114},[170,10512,10514,10516,10518,10520,10522,10524,10526,10528,10531,10533,10535,10537],{"class":2511,"line":10513},64,[170,10515,2765],{"class":2515},[170,10517,8923],{"class":2519},[170,10519,909],{"class":2523},[170,10521,2774],{"class":2773},[170,10523,2530],{"class":2523},[170,10525,8932],{"class":2526},[170,10527,2530],{"class":2523},[170,10529,10530],{"class":2526},"enc_valid_lens",[170,10532,2530],{"class":2523},[170,10534,8836],{"class":2515},[170,10536,8839],{"class":2526},[170,10538,2536],{"class":2523},[170,10540,10542,10544,10547,10549,10551,10553,10555],{"class":2511,"line":10541},65,[170,10543,2573],{"class":2515},[170,10545,10546],{"class":2523}," [enc_outputs, enc_valid_lens, [",[170,10548,2887],{"class":2563},[170,10550,7307],{"class":2523},[170,10552,8836],{"class":2515},[170,10554,5327],{"class":2801},[170,10556,10557],{"class":2523},".num_layers]\n",[170,10559,10561],{"class":2511,"line":10560},66,[170,10562,2832],{"emptyLinePlaceholder":114},[170,10564,10566,10568,10570,10572,10574,10576,10578,10580,10582],{"class":2511,"line":10565},67,[170,10567,2765],{"class":2515},[170,10569,2859],{"class":2519},[170,10571,909],{"class":2523},[170,10573,2774],{"class":2773},[170,10575,2530],{"class":2523},[170,10577,2527],{"class":2526},[170,10579,2530],{"class":2523},[170,10581,8969],{"class":2526},[170,10583,2536],{"class":2523},[170,10585,10587,10589,10591,10593,10595,10597,10599,10601,10603,10605],{"class":2511,"line":10586},68,[170,10588,2677],{"class":2523},[170,10590,926],{"class":2515},[170,10592,5327],{"class":2801},[170,10594,9672],{"class":2523},[170,10596,2774],{"class":2801},[170,10598,9677],{"class":2523},[170,10600,8836],{"class":2515},[170,10602,9682],{"class":2523},[170,10604,2774],{"class":2801},[170,10606,9687],{"class":2523},[170,10608,10610,10612,10615,10617,10620,10622,10624,10626,10628,10630,10632,10635,10638,10641,10643,10645,10648,10650],{"class":2511,"line":10609},69,[170,10611,2819],{"class":2801},[170,10613,10614],{"class":2523},"._attention_weights ",[170,10616,926],{"class":2515},[170,10618,10619],{"class":2523}," [[",[170,10621,2887],{"class":2563},[170,10623,7307],{"class":2523},[170,10625,8836],{"class":2515},[170,10627,9708],{"class":9568},[170,10629,909],{"class":2523},[170,10631,2774],{"class":2801},[170,10633,10634],{"class":2523},".blks) ",[170,10636,10637],{"class":2515},"for",[170,10639,10640],{"class":2523}," _ ",[170,10642,9565],{"class":2515},[170,10644,9569],{"class":9568},[170,10646,10647],{"class":2523}," (",[170,10649,2928],{"class":2563},[170,10651,10652],{"class":2523},")]\n",[170,10654,10656,10658,10660,10662,10664,10666,10668],{"class":2511,"line":10655},70,[170,10657,9559],{"class":2515},[170,10659,9723],{"class":2523},[170,10661,9565],{"class":2515},[170,10663,9728],{"class":9568},[170,10665,909],{"class":2523},[170,10667,2774],{"class":2801},[170,10669,9735],{"class":2523},[170,10671,10673,10676,10678],{"class":2511,"line":10672},71,[170,10674,10675],{"class":2523},"            X, state ",[170,10677,926],{"class":2515},[170,10679,10680],{"class":2523}," blk(X, state)\n",[170,10682,10684],{"class":2511,"line":10683},72,[170,10685,10686],{"class":2547},"            # 解码器自注意力权重\n",[170,10688,10690,10692,10695,10697,10700,10702],{"class":2511,"line":10689},73,[170,10691,9577],{"class":2801},[170,10693,10694],{"class":2523},"._attention_weights[",[170,10696,1865],{"class":2563},[170,10698,10699],{"class":2523},"][i] ",[170,10701,926],{"class":2515},[170,10703,10704],{"class":2523}," blk.attention1.attention.attention_weights\n",[170,10706,10708],{"class":2511,"line":10707},74,[170,10709,10710],{"class":2547},"            # “编码器－解码器”自注意力权重\n",[170,10712,10714,10716,10718,10720,10722,10724],{"class":2511,"line":10713},75,[170,10715,9577],{"class":2801},[170,10717,10694],{"class":2523},[170,10719,194],{"class":2563},[170,10721,10699],{"class":2523},[170,10723,926],{"class":2515},[170,10725,10726],{"class":2523}," blk.attention2.attention.attention_weights\n",[170,10728,10730,10732,10734],{"class":2511,"line":10729},76,[170,10731,2573],{"class":2515},[170,10733,5327],{"class":2801},[170,10735,10736],{"class":2523},".dense(X), state\n",[170,10738,10740],{"class":2511,"line":10739},77,[170,10741,2832],{"emptyLinePlaceholder":114},[170,10743,10745,10748],{"class":2511,"line":10744},78,[170,10746,10747],{"class":2519},"    @",[170,10749,10750],{"class":2794},"property\n",[170,10752,10754,10756,10759,10761,10763],{"class":2511,"line":10753},79,[170,10755,2765],{"class":2515},[170,10757,10758],{"class":2519}," attention_weights",[170,10760,909],{"class":2523},[170,10762,2774],{"class":2773},[170,10764,2536],{"class":2523},[170,10766,10768,10770,10772],{"class":2511,"line":10767},80,[170,10769,2573],{"class":2515},[170,10771,5327],{"class":2801},[170,10773,10774],{"class":2523},"._attention_weights\n",[10,10776,10777],{"id":10777},"训练",[14,10779,10780],{},"同样，训练时需要屏蔽部分不需要的内容：",[2501,10782,10784],{"className":2503,"code":10783,"language":2505,"meta":89,"style":89},"def sequence_mask(X, valid_len, value=0):\n    \"\"\"在序列中屏蔽不相关的项\"\"\"\n    maxlen = X.size(1)\n    mask = torch.arange((maxlen), dtype=torch.float32,\n                        device=X.device)[None, :] \u003C valid_len[:, None]\n    X[~mask] = value\n    return X\n\nclass MaskedSoftmaxCELoss(nn.CrossEntropyLoss):\n    \"\"\"带遮蔽的softmax交叉熵损失函数\"\"\"\n    # pred的形状：(batch_size,num_steps,vocab_size)\n    # label的形状：(batch_size,num_steps)\n    # valid_len的形状：(batch_size,)\n    def forward(self, pred, label, valid_len):\n        weights = torch.ones_like(label)\n        weights = sequence_mask(weights, valid_len)\n        self.reduction='none'\n        unweighted_loss = super(MaskedSoftmaxCELoss, self).forward(\n            pred.permute(0, 2, 1), label)\n        weighted_loss = (unweighted_loss * weights).mean(dim=1)\n        return weighted_loss\n",[2507,10785,10786,10813,10818,10832,10850,10879,10895,10901,10905,10923,10928,10933,10938,10943,10969,10979,10988,11006,11024,11042,11065],{"__ignoreMap":89},[170,10787,10788,10790,10793,10795,10797,10799,10802,10804,10807,10809,10811],{"class":2511,"line":2512},[170,10789,2516],{"class":2515},[170,10791,10792],{"class":2519}," sequence_mask",[170,10794,909],{"class":2523},[170,10796,2527],{"class":2526},[170,10798,2530],{"class":2523},[170,10800,10801],{"class":2526},"valid_len",[170,10803,2530],{"class":2523},[170,10805,10806],{"class":2526},"value",[170,10808,926],{"class":2515},[170,10810,1865],{"class":2563},[170,10812,2536],{"class":2523},[170,10814,10815],{"class":2511,"line":90},[170,10816,10817],{"class":2541},"    \"\"\"在序列中屏蔽不相关的项\"\"\"\n",[170,10819,10820,10823,10825,10828,10830],{"class":2511,"line":96},[170,10821,10822],{"class":2523},"    maxlen ",[170,10824,926],{"class":2515},[170,10826,10827],{"class":2523}," X.size(",[170,10829,194],{"class":2563},[170,10831,2588],{"class":2523},[170,10833,10834,10837,10839,10842,10845,10847],{"class":2511,"line":2551},[170,10835,10836],{"class":2523},"    mask ",[170,10838,926],{"class":2515},[170,10840,10841],{"class":2523}," torch.arange((maxlen), ",[170,10843,10844],{"class":2579},"dtype",[170,10846,926],{"class":2515},[170,10848,10849],{"class":2523},"torch.float32,\n",[170,10851,10852,10855,10857,10860,10862,10865,10868,10871,10873,10875,10877],{"class":2511,"line":2570},[170,10853,10854],{"class":2579},"                        device",[170,10856,926],{"class":2515},[170,10858,10859],{"class":2523},"X.device)[",[170,10861,2887],{"class":2563},[170,10863,10864],{"class":2523},", :] ",[170,10866,10867],{"class":2515},"\u003C",[170,10869,10870],{"class":2523}," valid_len[",[170,10872,7291],{"class":7290},[170,10874,2530],{"class":2523},[170,10876,2887],{"class":2563},[170,10878,2906],{"class":2523},[170,10880,10881,10884,10887,10890,10892],{"class":2511,"line":2591},[170,10882,10883],{"class":2523},"    X[",[170,10885,10886],{"class":2515},"~",[170,10888,10889],{"class":2523},"mask] ",[170,10891,926],{"class":2515},[170,10893,10894],{"class":2523}," value\n",[170,10896,10897,10899],{"class":2511,"line":2599},[170,10898,4794],{"class":2515},[170,10900,9767],{"class":2523},[170,10902,10903],{"class":2511,"line":2610},[170,10904,2832],{"emptyLinePlaceholder":114},[170,10906,10907,10909,10912,10914,10916,10918,10921],{"class":2511,"line":2627},[170,10908,2738],{"class":2515},[170,10910,10911],{"class":2741}," MaskedSoftmaxCELoss",[170,10913,909],{"class":2523},[170,10915,2748],{"class":2747},[170,10917,975],{"class":2523},[170,10919,10920],{"class":2747},"CrossEntropyLoss",[170,10922,2536],{"class":2523},[170,10924,10925],{"class":2511,"line":2643},[170,10926,10927],{"class":2541},"    \"\"\"带遮蔽的softmax交叉熵损失函数\"\"\"\n",[170,10929,10930],{"class":2511,"line":2651},[170,10931,10932],{"class":2547},"    # pred的形状：(batch_size,num_steps,vocab_size)\n",[170,10934,10935],{"class":2511,"line":2668},[170,10936,10937],{"class":2547},"    # label的形状：(batch_size,num_steps)\n",[170,10939,10940],{"class":2511,"line":2674},[170,10941,10942],{"class":2547},"    # valid_len的形状：(batch_size,)\n",[170,10944,10945,10947,10949,10951,10953,10955,10958,10960,10963,10965,10967],{"class":2511,"line":2699},[170,10946,2765],{"class":2515},[170,10948,2859],{"class":2519},[170,10950,909],{"class":2523},[170,10952,2774],{"class":2773},[170,10954,2530],{"class":2523},[170,10956,10957],{"class":2526},"pred",[170,10959,2530],{"class":2523},[170,10961,10962],{"class":2526},"label",[170,10964,2530],{"class":2523},[170,10966,10801],{"class":2526},[170,10968,2536],{"class":2523},[170,10970,10971,10974,10976],{"class":2511,"line":2712},[170,10972,10973],{"class":2523},"        weights ",[170,10975,926],{"class":2515},[170,10977,10978],{"class":2523}," torch.ones_like(label)\n",[170,10980,10981,10983,10985],{"class":2511,"line":2952},[170,10982,10973],{"class":2523},[170,10984,926],{"class":2515},[170,10986,10987],{"class":2523}," sequence_mask(weights, valid_len)\n",[170,10989,10990,10992,10995,10997,11000,11003],{"class":2511,"line":4888},[170,10991,2819],{"class":2801},[170,10993,10994],{"class":2523},".reduction",[170,10996,926],{"class":2515},[170,10998,10999],{"class":9583},"'",[170,11001,11002],{"class":9587},"none",[170,11004,11005],{"class":9583},"'\n",[170,11007,11008,11011,11013,11016,11019,11021],{"class":2511,"line":5164},[170,11009,11010],{"class":2523},"        unweighted_loss ",[170,11012,926],{"class":2515},[170,11014,11015],{"class":2794}," super",[170,11017,11018],{"class":2523},"(MaskedSoftmaxCELoss, ",[170,11020,2774],{"class":2801},[170,11022,11023],{"class":2523},").forward(\n",[170,11025,11026,11029,11031,11033,11035,11037,11039],{"class":2511,"line":5170},[170,11027,11028],{"class":2523},"            pred.permute(",[170,11030,1865],{"class":2563},[170,11032,2530],{"class":2523},[170,11034,2928],{"class":2563},[170,11036,2530],{"class":2523},[170,11038,194],{"class":2563},[170,11040,11041],{"class":2523},"), label)\n",[170,11043,11044,11047,11049,11052,11054,11057,11059,11061,11063],{"class":2511,"line":5176},[170,11045,11046],{"class":2523},"        weighted_loss ",[170,11048,926],{"class":2515},[170,11050,11051],{"class":2523}," (unweighted_loss ",[170,11053,8836],{"class":2515},[170,11055,11056],{"class":2523}," weights).mean(",[170,11058,2580],{"class":2579},[170,11060,926],{"class":2515},[170,11062,194],{"class":2563},[170,11064,2588],{"class":2523},[170,11066,11067,11069],{"class":2511,"line":5182},[170,11068,2573],{"class":2515},[170,11070,11071],{"class":2523}," weighted_loss\n",[14,11073,11074],{},"梯度裁剪函数：",[2501,11076,11078],{"className":2503,"code":11077,"language":2505,"meta":89,"style":89},"def grad_clipping(net, theta):\n    \"\"\"裁剪梯度\"\"\"\n    if isinstance(net, nn.Module):\n        params = [p for p in net.parameters() if p.requires_grad]\n    else:\n        params = net.params\n    norm = torch.sqrt(sum(torch.sum((p.grad ** 2)) for p in params))\n    if norm > theta:\n        for param in params:\n            param.grad[:] *= theta / norm\n",[2507,11079,11080,11099,11104,11114,11140,11146,11155,11187,11200,11212],{"__ignoreMap":89},[170,11081,11082,11084,11087,11089,11092,11094,11097],{"class":2511,"line":2512},[170,11083,2516],{"class":2515},[170,11085,11086],{"class":2519}," grad_clipping",[170,11088,909],{"class":2523},[170,11090,11091],{"class":2526},"net",[170,11093,2530],{"class":2523},[170,11095,11096],{"class":2526},"theta",[170,11098,2536],{"class":2523},[170,11100,11101],{"class":2511,"line":90},[170,11102,11103],{"class":2541},"    \"\"\"裁剪梯度\"\"\"\n",[170,11105,11106,11108,11111],{"class":2511,"line":96},[170,11107,2554],{"class":2515},[170,11109,11110],{"class":9568}," isinstance",[170,11112,11113],{"class":2523},"(net, nn.Module):\n",[170,11115,11116,11119,11121,11124,11126,11129,11131,11134,11137],{"class":2511,"line":2551},[170,11117,11118],{"class":2523},"        params ",[170,11120,926],{"class":2515},[170,11122,11123],{"class":2523}," [p ",[170,11125,10637],{"class":2515},[170,11127,11128],{"class":2523}," p ",[170,11130,9565],{"class":2515},[170,11132,11133],{"class":2523}," net.parameters() ",[170,11135,11136],{"class":2515},"if",[170,11138,11139],{"class":2523}," p.requires_grad]\n",[170,11141,11142,11144],{"class":2511,"line":2570},[170,11143,2594],{"class":2515},[170,11145,2567],{"class":2523},[170,11147,11148,11150,11152],{"class":2511,"line":2591},[170,11149,11118],{"class":2523},[170,11151,926],{"class":2515},[170,11153,11154],{"class":2523}," net.params\n",[170,11156,11157,11160,11162,11165,11168,11171,11173,11176,11178,11180,11182,11184],{"class":2511,"line":2599},[170,11158,11159],{"class":2523},"    norm ",[170,11161,926],{"class":2515},[170,11163,11164],{"class":2523}," torch.sqrt(",[170,11166,11167],{"class":9568},"sum",[170,11169,11170],{"class":2523},"(torch.sum((p.grad ",[170,11172,2784],{"class":2515},[170,11174,11175],{"class":2563}," 2",[170,11177,2931],{"class":2523},[170,11179,10637],{"class":2515},[170,11181,11128],{"class":2523},[170,11183,9565],{"class":2515},[170,11185,11186],{"class":2523}," params))\n",[170,11188,11189,11191,11194,11197],{"class":2511,"line":2610},[170,11190,2554],{"class":2515},[170,11192,11193],{"class":2523}," norm ",[170,11195,11196],{"class":2515},">",[170,11198,11199],{"class":2523}," theta:\n",[170,11201,11202,11204,11207,11209],{"class":2511,"line":2627},[170,11203,9559],{"class":2515},[170,11205,11206],{"class":2523}," param ",[170,11208,9565],{"class":2515},[170,11210,11211],{"class":2523}," params:\n",[170,11213,11214,11217,11219,11221,11224,11227,11229],{"class":2511,"line":2643},[170,11215,11216],{"class":2523},"            param.grad[",[170,11218,7291],{"class":7290},[170,11220,7307],{"class":2523},[170,11222,11223],{"class":2515},"*=",[170,11225,11226],{"class":2523}," theta ",[170,11228,2934],{"class":2515},[170,11230,11231],{"class":2523}," norm\n",[14,11233,11234,11235],{},"之后，在训练时，特定的序列开始词元（“",[11236,11237,11238,11239],"bos",{},"”）和 原始的输出序列（不包括序列结束词元“",[11240,11241,11242],"eos",{},"”） 拼接在一起作为解码器的输入。 这被称为强制教学（teacher forcing）， 因为原始的输出序列（词元的标签）被送入解码器。 或者，将来自上一个时间步的预测得到的词元作为解码器的当前输入。",[2501,11244,11246],{"className":2503,"code":11245,"language":2505,"meta":89,"style":89},"def train_seq2seq(net, data_iter, lr, num_epochs, tgt_vocab, device):\n    \"\"\"训练序列到序列模型\"\"\"\n    def xavier_init_weights(m):\n        if type(m) == nn.Linear:\n            nn.init.xavier_uniform_(m.weight)\n        if type(m) == nn.GRU:\n            for param in m._flat_weights_names:\n                if \"weight\" in param:\n                    nn.init.xavier_uniform_(m._parameters[param])\n\n    net.apply(xavier_init_weights)\n    net.to(device)\n    optimizer = torch.optim.Adam(net.parameters(), lr=lr)\n    loss = MaskedSoftmaxCELoss()\n    net.train()\n    animator = d2l.Animator(xlabel='epoch', ylabel='loss',\n                     xlim=[10, num_epochs])\n    for epoch in range(num_epochs):\n        timer = d2l.Timer()\n        metric = d2l.Accumulator(2)  # 训练损失总和，词元数量\n        for batch in data_iter:\n            optimizer.zero_grad()\n            X, X_valid_len, Y, Y_valid_len = [x.to(device) for x in batch]\n            bos = torch.tensor([tgt_vocab['\u003Cbos>']] * Y.shape[0],\n                          device=device).reshape(-1, 1)\n            dec_input = torch.cat([bos, Y[:, :-1]], 1)  # 强制教学\n            Y_hat, _ = net(X, dec_input, X_valid_len)\n            l = loss(Y_hat, Y, Y_valid_len)\n            l.sum().backward()      # 损失函数的标量进行“反向传播”\n            grad_clipping(net, 1)\n            num_tokens = Y_valid_len.sum()\n            optimizer.step()\n            with torch.no_grad():\n                metric.add(l.sum(), num_tokens)\n        if (epoch + 1) % 10 == 0:\n            animator.add(epoch + 1, (metric[0] / metric[1],))\n    print(f'loss {metric[0] / metric[1]:.3f}, {metric[1] / timer.stop():.1f} '\n        f'tokens/sec on {str(device)}')\n",[2507,11247,11248,11285,11290,11303,11318,11323,11341,11353,11372,11377,11381,11386,11391,11408,11418,11423,11461,11477,11492,11502,11520,11532,11537,11557,11587,11607,11637,11647,11657,11665,11674,11684,11689,11697,11702,11729,11755,11816],{"__ignoreMap":89},[170,11249,11250,11252,11255,11257,11259,11261,11264,11266,11269,11271,11274,11276,11279,11281,11283],{"class":2511,"line":2512},[170,11251,2516],{"class":2515},[170,11253,11254],{"class":2519}," train_seq2seq",[170,11256,909],{"class":2523},[170,11258,11091],{"class":2526},[170,11260,2530],{"class":2523},[170,11262,11263],{"class":2526},"data_iter",[170,11265,2530],{"class":2523},[170,11267,11268],{"class":2526},"lr",[170,11270,2530],{"class":2523},[170,11272,11273],{"class":2526},"num_epochs",[170,11275,2530],{"class":2523},[170,11277,11278],{"class":2526},"tgt_vocab",[170,11280,2530],{"class":2523},[170,11282,10172],{"class":2526},[170,11284,2536],{"class":2523},[170,11286,11287],{"class":2511,"line":90},[170,11288,11289],{"class":2541},"    \"\"\"训练序列到序列模型\"\"\"\n",[170,11291,11292,11294,11297,11299,11301],{"class":2511,"line":96},[170,11293,2765],{"class":2515},[170,11295,11296],{"class":2519}," xavier_init_weights",[170,11298,909],{"class":2523},[170,11300,477],{"class":2526},[170,11302,2536],{"class":2523},[170,11304,11305,11307,11310,11313,11315],{"class":2511,"line":2551},[170,11306,2613],{"class":2515},[170,11308,11309],{"class":2794}," type",[170,11311,11312],{"class":2523},"(m) ",[170,11314,2619],{"class":2515},[170,11316,11317],{"class":2523}," nn.Linear:\n",[170,11319,11320],{"class":2511,"line":2570},[170,11321,11322],{"class":2523},"            nn.init.xavier_uniform_(m.weight)\n",[170,11324,11325,11327,11329,11331,11333,11336,11339],{"class":2511,"line":2591},[170,11326,2613],{"class":2515},[170,11328,11309],{"class":2794},[170,11330,11312],{"class":2523},[170,11332,2619],{"class":2515},[170,11334,11335],{"class":2523}," nn.",[170,11337,11338],{"class":2563},"GRU",[170,11340,2567],{"class":2523},[170,11342,11343,11346,11348,11350],{"class":2511,"line":2599},[170,11344,11345],{"class":2515},"            for",[170,11347,11206],{"class":2523},[170,11349,9565],{"class":2515},[170,11351,11352],{"class":2523}," m._flat_weights_names:\n",[170,11354,11355,11358,11361,11364,11366,11369],{"class":2511,"line":2610},[170,11356,11357],{"class":2515},"                if",[170,11359,11360],{"class":9583}," \"",[170,11362,11363],{"class":9587},"weight",[170,11365,9584],{"class":9583},[170,11367,11368],{"class":2515}," in",[170,11370,11371],{"class":2523}," param:\n",[170,11373,11374],{"class":2511,"line":2627},[170,11375,11376],{"class":2523},"                    nn.init.xavier_uniform_(m._parameters[param])\n",[170,11378,11379],{"class":2511,"line":2643},[170,11380,2832],{"emptyLinePlaceholder":114},[170,11382,11383],{"class":2511,"line":2651},[170,11384,11385],{"class":2523},"    net.apply(xavier_init_weights)\n",[170,11387,11388],{"class":2511,"line":2668},[170,11389,11390],{"class":2523},"    net.to(device)\n",[170,11392,11393,11396,11398,11401,11403,11405],{"class":2511,"line":2674},[170,11394,11395],{"class":2523},"    optimizer ",[170,11397,926],{"class":2515},[170,11399,11400],{"class":2523}," torch.optim.Adam(net.parameters(), ",[170,11402,11268],{"class":2579},[170,11404,926],{"class":2515},[170,11406,11407],{"class":2523},"lr)\n",[170,11409,11410,11413,11415],{"class":2511,"line":2699},[170,11411,11412],{"class":2523},"    loss ",[170,11414,926],{"class":2515},[170,11416,11417],{"class":2523}," MaskedSoftmaxCELoss()\n",[170,11419,11420],{"class":2511,"line":2712},[170,11421,11422],{"class":2523},"    net.train()\n",[170,11424,11425,11428,11430,11433,11436,11438,11440,11443,11445,11447,11450,11452,11454,11457,11459],{"class":2511,"line":2952},[170,11426,11427],{"class":2523},"    animator ",[170,11429,926],{"class":2515},[170,11431,11432],{"class":2523}," d2l.Animator(",[170,11434,11435],{"class":2579},"xlabel",[170,11437,926],{"class":2515},[170,11439,10999],{"class":9583},[170,11441,11442],{"class":9587},"epoch",[170,11444,10999],{"class":9583},[170,11446,2530],{"class":2523},[170,11448,11449],{"class":2579},"ylabel",[170,11451,926],{"class":2515},[170,11453,10999],{"class":9583},[170,11455,11456],{"class":9587},"loss",[170,11458,10999],{"class":9583},[170,11460,3933],{"class":2523},[170,11462,11463,11466,11468,11471,11474],{"class":2511,"line":4888},[170,11464,11465],{"class":2579},"                     xlim",[170,11467,926],{"class":2515},[170,11469,11470],{"class":2523},"[",[170,11472,11473],{"class":2563},"10",[170,11475,11476],{"class":2523},", num_epochs])\n",[170,11478,11479,11482,11485,11487,11489],{"class":2511,"line":5164},[170,11480,11481],{"class":2515},"    for",[170,11483,11484],{"class":2523}," epoch ",[170,11486,9565],{"class":2515},[170,11488,9569],{"class":9568},[170,11490,11491],{"class":2523},"(num_epochs):\n",[170,11493,11494,11497,11499],{"class":2511,"line":5170},[170,11495,11496],{"class":2523},"        timer ",[170,11498,926],{"class":2515},[170,11500,11501],{"class":2523}," d2l.Timer()\n",[170,11503,11504,11507,11509,11512,11514,11517],{"class":2511,"line":5176},[170,11505,11506],{"class":2523},"        metric ",[170,11508,926],{"class":2515},[170,11510,11511],{"class":2523}," d2l.Accumulator(",[170,11513,2928],{"class":2563},[170,11515,11516],{"class":2523},")  ",[170,11518,11519],{"class":2547},"# 训练损失总和，词元数量\n",[170,11521,11522,11524,11527,11529],{"class":2511,"line":5182},[170,11523,9559],{"class":2515},[170,11525,11526],{"class":2523}," batch ",[170,11528,9565],{"class":2515},[170,11530,11531],{"class":2523}," data_iter:\n",[170,11533,11534],{"class":2511,"line":5203},[170,11535,11536],{"class":2523},"            optimizer.zero_grad()\n",[170,11538,11539,11542,11544,11547,11549,11552,11554],{"class":2511,"line":5222},[170,11540,11541],{"class":2523},"            X, X_valid_len, Y, Y_valid_len ",[170,11543,926],{"class":2515},[170,11545,11546],{"class":2523}," [x.to(device) ",[170,11548,10637],{"class":2515},[170,11550,11551],{"class":2523}," x ",[170,11553,9565],{"class":2515},[170,11555,11556],{"class":2523}," batch]\n",[170,11558,11559,11562,11564,11567,11569,11572,11574,11577,11579,11582,11584],{"class":2511,"line":5241},[170,11560,11561],{"class":2523},"            bos ",[170,11563,926],{"class":2515},[170,11565,11566],{"class":2523}," torch.tensor([tgt_vocab[",[170,11568,10999],{"class":9583},[170,11570,11571],{"class":9587},"\u003Cbos>",[170,11573,10999],{"class":9583},[170,11575,11576],{"class":2523},"]] ",[170,11578,8836],{"class":2515},[170,11580,11581],{"class":2523}," Y.shape[",[170,11583,1865],{"class":2563},[170,11585,11586],{"class":2523},"],\n",[170,11588,11589,11592,11594,11597,11599,11601,11603,11605],{"class":2511,"line":5246},[170,11590,11591],{"class":2579},"                          device",[170,11593,926],{"class":2515},[170,11595,11596],{"class":2523},"device).reshape(",[170,11598,2661],{"class":2515},[170,11600,194],{"class":2563},[170,11602,2530],{"class":2523},[170,11604,194],{"class":2563},[170,11606,2588],{"class":2523},[170,11608,11609,11612,11614,11617,11619,11621,11623,11625,11627,11630,11632,11634],{"class":2511,"line":5262},[170,11610,11611],{"class":2523},"            dec_input ",[170,11613,926],{"class":2515},[170,11615,11616],{"class":2523}," torch.cat([bos, Y[",[170,11618,7291],{"class":7290},[170,11620,2530],{"class":2523},[170,11622,7291],{"class":7290},[170,11624,2661],{"class":2515},[170,11626,194],{"class":2563},[170,11628,11629],{"class":2523},"]], ",[170,11631,194],{"class":2563},[170,11633,11516],{"class":2523},[170,11635,11636],{"class":2547},"# 强制教学\n",[170,11638,11639,11642,11644],{"class":2511,"line":5268},[170,11640,11641],{"class":2523},"            Y_hat, _ ",[170,11643,926],{"class":2515},[170,11645,11646],{"class":2523}," net(X, dec_input, X_valid_len)\n",[170,11648,11649,11652,11654],{"class":2511,"line":5274},[170,11650,11651],{"class":2523},"            l ",[170,11653,926],{"class":2515},[170,11655,11656],{"class":2523}," loss(Y_hat, Y, Y_valid_len)\n",[170,11658,11659,11662],{"class":2511,"line":5284},[170,11660,11661],{"class":2523},"            l.sum().backward()      ",[170,11663,11664],{"class":2547},"# 损失函数的标量进行“反向传播”\n",[170,11666,11667,11670,11672],{"class":2511,"line":5308},[170,11668,11669],{"class":2523},"            grad_clipping(net, ",[170,11671,194],{"class":2563},[170,11673,2588],{"class":2523},[170,11675,11676,11679,11681],{"class":2511,"line":5313},[170,11677,11678],{"class":2523},"            num_tokens ",[170,11680,926],{"class":2515},[170,11682,11683],{"class":2523}," Y_valid_len.sum()\n",[170,11685,11686],{"class":2511,"line":5319},[170,11687,11688],{"class":2523},"            optimizer.step()\n",[170,11690,11691,11694],{"class":2511,"line":5333},[170,11692,11693],{"class":2515},"            with",[170,11695,11696],{"class":2523}," torch.no_grad():\n",[170,11698,11699],{"class":2511,"line":5338},[170,11700,11701],{"class":2523},"                metric.add(l.sum(), num_tokens)\n",[170,11703,11704,11706,11709,11711,11713,11715,11718,11721,11724,11727],{"class":2511,"line":5344},[170,11705,2613],{"class":2515},[170,11707,11708],{"class":2523}," (epoch ",[170,11710,5884],{"class":2515},[170,11712,2622],{"class":2563},[170,11714,7253],{"class":2523},[170,11716,11717],{"class":2515},"%",[170,11719,11720],{"class":2563}," 10",[170,11722,11723],{"class":2515}," ==",[170,11725,11726],{"class":2563}," 0",[170,11728,2567],{"class":2523},[170,11730,11731,11734,11736,11738,11741,11743,11745,11747,11750,11752],{"class":2511,"line":5359},[170,11732,11733],{"class":2523},"            animator.add(epoch ",[170,11735,5884],{"class":2515},[170,11737,2622],{"class":2563},[170,11739,11740],{"class":2523},", (metric[",[170,11742,1865],{"class":2563},[170,11744,7307],{"class":2523},[170,11746,2934],{"class":2515},[170,11748,11749],{"class":2523}," metric[",[170,11751,194],{"class":2563},[170,11753,11754],{"class":2523},"],))\n",[170,11756,11757,11760,11762,11765,11768,11771,11774,11776,11778,11780,11782,11784,11787,11790,11793,11795,11797,11799,11801,11803,11805,11808,11811,11813],{"class":2511,"line":9651},[170,11758,11759],{"class":9568},"    print",[170,11761,909],{"class":2523},[170,11763,11764],{"class":2515},"f",[170,11766,11767],{"class":9587},"'loss ",[170,11769,11770],{"class":2563},"{",[170,11772,11773],{"class":2523},"metric[",[170,11775,1865],{"class":2563},[170,11777,7307],{"class":2523},[170,11779,2934],{"class":2515},[170,11781,11749],{"class":2523},[170,11783,194],{"class":2563},[170,11785,11786],{"class":2523},"]",[170,11788,11789],{"class":2515},":.3f",[170,11791,11792],{"class":2563},"}",[170,11794,2530],{"class":9587},[170,11796,11770],{"class":2563},[170,11798,11773],{"class":2523},[170,11800,194],{"class":2563},[170,11802,7307],{"class":2523},[170,11804,2934],{"class":2515},[170,11806,11807],{"class":2523}," timer.stop()",[170,11809,11810],{"class":2515},":.1f",[170,11812,11792],{"class":2563},[170,11814,11815],{"class":9587}," '\n",[170,11817,11818,11821,11824,11826,11828,11831,11833,11835],{"class":2511,"line":9657},[170,11819,11820],{"class":2515},"        f",[170,11822,11823],{"class":9587},"'tokens/sec on ",[170,11825,11770],{"class":2563},[170,11827,9595],{"class":2794},[170,11829,11830],{"class":2523},"(device)",[170,11832,11792],{"class":2563},[170,11834,10999],{"class":9587},[170,11836,2588],{"class":2523},[14,11838,11839],{},"进行数据集的数据处理",[2501,11841,11843],{"className":2503,"code":11842,"language":2505,"meta":89,"style":89},"def read_data_nmt():\n    \"\"\"载入“英语－法语”数据集\n\n    Defined in :numref:`sec_machine_translation`\"\"\"\n    data_dir = d2l.download_extract('fra-eng')\n    with open(os.path.join(data_dir, 'fra.txt'), 'r',\n             encoding='utf-8') as f:\n        return f.read()\n\ndef preprocess_nmt(text):\n    \"\"\"预处理“英语－法语”数据集\n\n    Defined in :numref:`sec_machine_translation`\"\"\"\n    def no_space(char, prev_char):\n        return char in set(',.!?') and prev_char != ' '\n\n    # 使用空格替换不间断空格\n    # 使用小写字母替换大写字母\n    text = text.replace('\\u202f', ' ').replace('\\xa0', ' ').lower()\n    # 在单词和标点符号之间插入空格\n    out = [' ' + char if i > 0 and no_space(char, text[i - 1]) else char\n           for i, char in enumerate(text)]\n    return ''.join(out)\n\ndef tokenize_nmt(text, num_examples=None):\n    \"\"\"词元化“英语－法语”数据数据集\n\n    Defined in :numref:`sec_machine_translation`\"\"\"\n    source, target = [], []\n    for i, line in enumerate(text.split('\\n')):\n        if num_examples and i > num_examples:\n            break\n        parts = line.split('\\t')\n        if len(parts) == 2:\n            source.append(parts[0].split(' '))\n            target.append(parts[1].split(' '))\n    return source, target\n\ndef load_data_nmt(batch_size, num_steps, num_examples=600):\n    \"\"\"返回翻译数据集的迭代器和词表\n\n    Defined in :numref:`subsec_mt_data_loading`\"\"\"\n    text = preprocess_nmt(read_data_nmt())\n    source, target = tokenize_nmt(text, num_examples)\n    src_vocab = d2l.Vocab(source, min_freq=2,\n                          reserved_tokens=['\u003Cpad>', '\u003Cbos>', '\u003Ceos>'])\n    tgt_vocab = d2l.Vocab(target, min_freq=2,\n                          reserved_tokens=['\u003Cpad>', '\u003Cbos>', '\u003Ceos>'])\n    src_array, src_valid_len = build_array_nmt(source, src_vocab, num_steps)\n    tgt_array, tgt_valid_len = build_array_nmt(target, tgt_vocab, num_steps)\n    data_arrays = (src_array, src_valid_len, tgt_array, tgt_valid_len)\n    data_iter = d2l.load_array(data_arrays, batch_size)\n    return data_iter, src_vocab, tgt_vocab\n",[2507,11844,11845,11855,11860,11864,11869,11888,11917,11939,11946,11950,11964,11969,11973,11977,11996,12033,12037,12042,12047,12090,12095,12140,12155,12165,12169,12191,12196,12200,12204,12214,12238,12254,12259,12278,12293,12310,12325,12332,12336,12364,12369,12373,12378,12387,12396,12415,12450,12468,12500,12510,12520,12530,12540],{"__ignoreMap":89},[170,11846,11847,11849,11852],{"class":2511,"line":2512},[170,11848,2516],{"class":2515},[170,11850,11851],{"class":2519}," read_data_nmt",[170,11853,11854],{"class":2523},"():\n",[170,11856,11857],{"class":2511,"line":90},[170,11858,11859],{"class":2541},"    \"\"\"载入“英语－法语”数据集\n",[170,11861,11862],{"class":2511,"line":96},[170,11863,2832],{"emptyLinePlaceholder":114},[170,11865,11866],{"class":2511,"line":2551},[170,11867,11868],{"class":2541},"    Defined in :numref:`sec_machine_translation`\"\"\"\n",[170,11870,11871,11874,11876,11879,11881,11884,11886],{"class":2511,"line":2570},[170,11872,11873],{"class":2523},"    data_dir ",[170,11875,926],{"class":2515},[170,11877,11878],{"class":2523}," d2l.download_extract(",[170,11880,10999],{"class":9583},[170,11882,11883],{"class":9587},"fra-eng",[170,11885,10999],{"class":9583},[170,11887,2588],{"class":2523},[170,11889,11890,11893,11896,11899,11901,11904,11906,11909,11911,11913,11915],{"class":2511,"line":2591},[170,11891,11892],{"class":2515},"    with",[170,11894,11895],{"class":9568}," open",[170,11897,11898],{"class":2523},"(os.path.join(data_dir, ",[170,11900,10999],{"class":9583},[170,11902,11903],{"class":9587},"fra.txt",[170,11905,10999],{"class":9583},[170,11907,11908],{"class":2523},"), ",[170,11910,10999],{"class":9583},[170,11912,7933],{"class":9587},[170,11914,10999],{"class":9583},[170,11916,3933],{"class":2523},[170,11918,11919,11922,11924,11926,11929,11931,11933,11936],{"class":2511,"line":2599},[170,11920,11921],{"class":2579},"             encoding",[170,11923,926],{"class":2515},[170,11925,10999],{"class":9583},[170,11927,11928],{"class":9587},"utf-8",[170,11930,10999],{"class":9583},[170,11932,7253],{"class":2523},[170,11934,11935],{"class":2515},"as",[170,11937,11938],{"class":2523}," f:\n",[170,11940,11941,11943],{"class":2511,"line":2610},[170,11942,2573],{"class":2515},[170,11944,11945],{"class":2523}," f.read()\n",[170,11947,11948],{"class":2511,"line":2627},[170,11949,2832],{"emptyLinePlaceholder":114},[170,11951,11952,11954,11957,11959,11962],{"class":2511,"line":2643},[170,11953,2516],{"class":2515},[170,11955,11956],{"class":2519}," preprocess_nmt",[170,11958,909],{"class":2523},[170,11960,11961],{"class":2526},"text",[170,11963,2536],{"class":2523},[170,11965,11966],{"class":2511,"line":2651},[170,11967,11968],{"class":2541},"    \"\"\"预处理“英语－法语”数据集\n",[170,11970,11971],{"class":2511,"line":2668},[170,11972,2832],{"emptyLinePlaceholder":114},[170,11974,11975],{"class":2511,"line":2674},[170,11976,11868],{"class":2541},[170,11978,11979,11981,11984,11986,11989,11991,11994],{"class":2511,"line":2699},[170,11980,2765],{"class":2515},[170,11982,11983],{"class":2519}," no_space",[170,11985,909],{"class":2523},[170,11987,11988],{"class":2526},"char",[170,11990,2530],{"class":2523},[170,11992,11993],{"class":2526},"prev_char",[170,11995,2536],{"class":2523},[170,11997,11998,12000,12003,12005,12008,12010,12012,12015,12017,12019,12022,12025,12028,12031],{"class":2511,"line":2712},[170,11999,2573],{"class":2515},[170,12001,12002],{"class":2523}," char ",[170,12004,9565],{"class":2515},[170,12006,12007],{"class":2794}," set",[170,12009,909],{"class":2523},[170,12011,10999],{"class":9583},[170,12013,12014],{"class":9587},",.!?",[170,12016,10999],{"class":9583},[170,12018,7253],{"class":2523},[170,12020,12021],{"class":2515},"and",[170,12023,12024],{"class":2523}," prev_char ",[170,12026,12027],{"class":2515},"!=",[170,12029,12030],{"class":9583}," '",[170,12032,11815],{"class":9583},[170,12034,12035],{"class":2511,"line":2952},[170,12036,2832],{"emptyLinePlaceholder":114},[170,12038,12039],{"class":2511,"line":4888},[170,12040,12041],{"class":2547},"    # 使用空格替换不间断空格\n",[170,12043,12044],{"class":2511,"line":5164},[170,12045,12046],{"class":2547},"    # 使用小写字母替换大写字母\n",[170,12048,12049,12052,12054,12057,12059,12063,12065,12067,12069,12071,12074,12076,12079,12081,12083,12085,12087],{"class":2511,"line":5170},[170,12050,12051],{"class":2523},"    text ",[170,12053,926],{"class":2515},[170,12055,12056],{"class":2523}," text.replace(",[170,12058,10999],{"class":9583},[170,12060,12062],{"class":12061},"sRfyP","\\u202f",[170,12064,10999],{"class":9583},[170,12066,2530],{"class":2523},[170,12068,10999],{"class":9583},[170,12070,12030],{"class":9583},[170,12072,12073],{"class":2523},").replace(",[170,12075,10999],{"class":9583},[170,12077,12078],{"class":12061},"\\xa0",[170,12080,10999],{"class":9583},[170,12082,2530],{"class":2523},[170,12084,10999],{"class":9583},[170,12086,12030],{"class":9583},[170,12088,12089],{"class":2523},").lower()\n",[170,12091,12092],{"class":2511,"line":5176},[170,12093,12094],{"class":2547},"    # 在单词和标点符号之间插入空格\n",[170,12096,12097,12100,12102,12104,12106,12108,12111,12113,12115,12117,12119,12121,12124,12127,12129,12131,12134,12137],{"class":2511,"line":5182},[170,12098,12099],{"class":2523},"    out ",[170,12101,926],{"class":2515},[170,12103,9699],{"class":2523},[170,12105,10999],{"class":9583},[170,12107,12030],{"class":9583},[170,12109,12110],{"class":2515}," +",[170,12112,12002],{"class":2523},[170,12114,11136],{"class":2515},[170,12116,9562],{"class":2523},[170,12118,11196],{"class":2515},[170,12120,11726],{"class":2563},[170,12122,12123],{"class":2515}," and",[170,12125,12126],{"class":2523}," no_space(char, text[i ",[170,12128,2661],{"class":2515},[170,12130,2622],{"class":2563},[170,12132,12133],{"class":2523},"]) ",[170,12135,12136],{"class":2515},"else",[170,12138,12139],{"class":2523}," char\n",[170,12141,12142,12145,12148,12150,12152],{"class":2511,"line":5203},[170,12143,12144],{"class":2515},"           for",[170,12146,12147],{"class":2523}," i, char ",[170,12149,9565],{"class":2515},[170,12151,9728],{"class":9568},[170,12153,12154],{"class":2523},"(text)]\n",[170,12156,12157,12159,12162],{"class":2511,"line":5222},[170,12158,4794],{"class":2515},[170,12160,12161],{"class":9583}," ''",[170,12163,12164],{"class":2523},".join(out)\n",[170,12166,12167],{"class":2511,"line":5241},[170,12168,2832],{"emptyLinePlaceholder":114},[170,12170,12171,12173,12176,12178,12180,12182,12185,12187,12189],{"class":2511,"line":5246},[170,12172,2516],{"class":2515},[170,12174,12175],{"class":2519}," tokenize_nmt",[170,12177,909],{"class":2523},[170,12179,11961],{"class":2526},[170,12181,2530],{"class":2523},[170,12183,12184],{"class":2526},"num_examples",[170,12186,926],{"class":2515},[170,12188,2887],{"class":2563},[170,12190,2536],{"class":2523},[170,12192,12193],{"class":2511,"line":5262},[170,12194,12195],{"class":2541},"    \"\"\"词元化“英语－法语”数据数据集\n",[170,12197,12198],{"class":2511,"line":5268},[170,12199,2832],{"emptyLinePlaceholder":114},[170,12201,12202],{"class":2511,"line":5274},[170,12203,11868],{"class":2541},[170,12205,12206,12209,12211],{"class":2511,"line":5284},[170,12207,12208],{"class":2523},"    source, target ",[170,12210,926],{"class":2515},[170,12212,12213],{"class":2523}," [], []\n",[170,12215,12216,12218,12221,12223,12225,12228,12230,12233,12235],{"class":2511,"line":5308},[170,12217,11481],{"class":2515},[170,12219,12220],{"class":2523}," i, line ",[170,12222,9565],{"class":2515},[170,12224,9728],{"class":9568},[170,12226,12227],{"class":2523},"(text.split(",[170,12229,10999],{"class":9583},[170,12231,12232],{"class":12061},"\\n",[170,12234,10999],{"class":9583},[170,12236,12237],{"class":2523},")):\n",[170,12239,12240,12242,12245,12247,12249,12251],{"class":2511,"line":5313},[170,12241,2613],{"class":2515},[170,12243,12244],{"class":2523}," num_examples ",[170,12246,12021],{"class":2515},[170,12248,9562],{"class":2523},[170,12250,11196],{"class":2515},[170,12252,12253],{"class":2523}," num_examples:\n",[170,12255,12256],{"class":2511,"line":5319},[170,12257,12258],{"class":2515},"            break\n",[170,12260,12261,12264,12266,12269,12271,12274,12276],{"class":2511,"line":5333},[170,12262,12263],{"class":2523},"        parts ",[170,12265,926],{"class":2515},[170,12267,12268],{"class":2523}," line.split(",[170,12270,10999],{"class":9583},[170,12272,12273],{"class":12061},"\\t",[170,12275,10999],{"class":9583},[170,12277,2588],{"class":2523},[170,12279,12280,12282,12284,12287,12289,12291],{"class":2511,"line":5338},[170,12281,2613],{"class":2515},[170,12283,9708],{"class":9568},[170,12285,12286],{"class":2523},"(parts) ",[170,12288,2619],{"class":2515},[170,12290,11175],{"class":2563},[170,12292,2567],{"class":2523},[170,12294,12295,12298,12300,12303,12305,12307],{"class":2511,"line":5344},[170,12296,12297],{"class":2523},"            source.append(parts[",[170,12299,1865],{"class":2563},[170,12301,12302],{"class":2523},"].split(",[170,12304,10999],{"class":9583},[170,12306,12030],{"class":9583},[170,12308,12309],{"class":2523},"))\n",[170,12311,12312,12315,12317,12319,12321,12323],{"class":2511,"line":5359},[170,12313,12314],{"class":2523},"            target.append(parts[",[170,12316,194],{"class":2563},[170,12318,12302],{"class":2523},[170,12320,10999],{"class":9583},[170,12322,12030],{"class":9583},[170,12324,12309],{"class":2523},[170,12326,12327,12329],{"class":2511,"line":9651},[170,12328,4794],{"class":2515},[170,12330,12331],{"class":2523}," source, target\n",[170,12333,12334],{"class":2511,"line":9657},[170,12335,2832],{"emptyLinePlaceholder":114},[170,12337,12338,12340,12343,12345,12348,12350,12353,12355,12357,12359,12362],{"class":2511,"line":9663},[170,12339,2516],{"class":2515},[170,12341,12342],{"class":2519}," load_data_nmt",[170,12344,909],{"class":2523},[170,12346,12347],{"class":2526},"batch_size",[170,12349,2530],{"class":2523},[170,12351,12352],{"class":2526},"num_steps",[170,12354,2530],{"class":2523},[170,12356,12184],{"class":2526},[170,12358,926],{"class":2515},[170,12360,12361],{"class":2563},"600",[170,12363,2536],{"class":2523},[170,12365,12366],{"class":2511,"line":9690},[170,12367,12368],{"class":2541},"    \"\"\"返回翻译数据集的迭代器和词表\n",[170,12370,12371],{"class":2511,"line":9718},[170,12372,2832],{"emptyLinePlaceholder":114},[170,12374,12375],{"class":2511,"line":9738},[170,12376,12377],{"class":2541},"    Defined in :numref:`subsec_mt_data_loading`\"\"\"\n",[170,12379,12380,12382,12384],{"class":2511,"line":9749},[170,12381,12051],{"class":2523},[170,12383,926],{"class":2515},[170,12385,12386],{"class":2523}," preprocess_nmt(read_data_nmt())\n",[170,12388,12389,12391,12393],{"class":2511,"line":9762},[170,12390,12208],{"class":2523},[170,12392,926],{"class":2515},[170,12394,12395],{"class":2523}," tokenize_nmt(text, num_examples)\n",[170,12397,12398,12401,12403,12406,12409,12411,12413],{"class":2511,"line":10265},[170,12399,12400],{"class":2523},"    src_vocab ",[170,12402,926],{"class":2515},[170,12404,12405],{"class":2523}," d2l.Vocab(source, ",[170,12407,12408],{"class":2579},"min_freq",[170,12410,926],{"class":2515},[170,12412,2928],{"class":2563},[170,12414,3933],{"class":2523},[170,12416,12417,12420,12422,12424,12426,12429,12431,12433,12435,12437,12439,12441,12443,12446,12448],{"class":2511,"line":10280},[170,12418,12419],{"class":2579},"                          reserved_tokens",[170,12421,926],{"class":2515},[170,12423,11470],{"class":2523},[170,12425,10999],{"class":9583},[170,12427,12428],{"class":9587},"\u003Cpad>",[170,12430,10999],{"class":9583},[170,12432,2530],{"class":2523},[170,12434,10999],{"class":9583},[170,12436,11571],{"class":9587},[170,12438,10999],{"class":9583},[170,12440,2530],{"class":2523},[170,12442,10999],{"class":9583},[170,12444,12445],{"class":9587},"\u003Ceos>",[170,12447,10999],{"class":9583},[170,12449,2640],{"class":2523},[170,12451,12452,12455,12457,12460,12462,12464,12466],{"class":2511,"line":10285},[170,12453,12454],{"class":2523},"    tgt_vocab ",[170,12456,926],{"class":2515},[170,12458,12459],{"class":2523}," d2l.Vocab(target, ",[170,12461,12408],{"class":2579},[170,12463,926],{"class":2515},[170,12465,2928],{"class":2563},[170,12467,3933],{"class":2523},[170,12469,12470,12472,12474,12476,12478,12480,12482,12484,12486,12488,12490,12492,12494,12496,12498],{"class":2511,"line":10300},[170,12471,12419],{"class":2579},[170,12473,926],{"class":2515},[170,12475,11470],{"class":2523},[170,12477,10999],{"class":9583},[170,12479,12428],{"class":9587},[170,12481,10999],{"class":9583},[170,12483,2530],{"class":2523},[170,12485,10999],{"class":9583},[170,12487,11571],{"class":9587},[170,12489,10999],{"class":9583},[170,12491,2530],{"class":2523},[170,12493,10999],{"class":9583},[170,12495,12445],{"class":9587},[170,12497,10999],{"class":9583},[170,12499,2640],{"class":2523},[170,12501,12502,12505,12507],{"class":2511,"line":10329},[170,12503,12504],{"class":2523},"    src_array, src_valid_len ",[170,12506,926],{"class":2515},[170,12508,12509],{"class":2523}," build_array_nmt(source, src_vocab, num_steps)\n",[170,12511,12512,12515,12517],{"class":2511,"line":10348},[170,12513,12514],{"class":2523},"    tgt_array, tgt_valid_len ",[170,12516,926],{"class":2515},[170,12518,12519],{"class":2523}," build_array_nmt(target, tgt_vocab, num_steps)\n",[170,12521,12522,12525,12527],{"class":2511,"line":10369},[170,12523,12524],{"class":2523},"    data_arrays ",[170,12526,926],{"class":2515},[170,12528,12529],{"class":2523}," (src_array, src_valid_len, tgt_array, tgt_valid_len)\n",[170,12531,12532,12535,12537],{"class":2511,"line":10389},[170,12533,12534],{"class":2523},"    data_iter ",[170,12536,926],{"class":2515},[170,12538,12539],{"class":2523}," d2l.load_array(data_arrays, batch_size)\n",[170,12541,12542,12544],{"class":2511,"line":10400},[170,12543,4794],{"class":2515},[170,12545,12546],{"class":2523}," data_iter, src_vocab, tgt_vocab\n",[14,12548,12549],{},"之后按照 transformer 架构创建编码器-解码器架构，",[2501,12551,12553],{"className":2503,"code":12552,"language":2505,"meta":89,"style":89},"num_hiddens, num_layers, dropout, batch_size, num_steps = 32, 2, 0.1, 64, 10\nlr, num_epochs, device = 0.005, 200, torch.device('cuda:0') if torch.cuda.device_count() >= 1 else torch.device('cpu')\nffn_num_hiddens, num_heads = 64, 4\n\ntrain_iter, src_vocab, tgt_vocab = d2l.load_data_nmt(batch_size, num_steps)\n\nencoder = TransformerEncoder(\n    len(src_vocab), num_hiddens, ffn_num_hiddens, num_heads, num_layers,\n    dropout)\ndecoder = TransformerDecoder(\n    len(tgt_vocab), num_hiddens, ffn_num_hiddens, num_heads, num_layers,\n    dropout)\nnet = EncoderDecoder(encoder, decoder)\ntrain_seq2seq(net, train_iter, lr, num_epochs, tgt_vocab, device)\n",[2507,12554,12555,12584,12636,12651,12655,12665,12669,12679,12687,12692,12702,12709,12713,12723],{"__ignoreMap":89},[170,12556,12557,12560,12562,12565,12567,12569,12571,12574,12576,12579,12581],{"class":2511,"line":2512},[170,12558,12559],{"class":2523},"num_hiddens, num_layers, dropout, batch_size, num_steps ",[170,12561,926],{"class":2515},[170,12563,12564],{"class":2563}," 32",[170,12566,2530],{"class":2523},[170,12568,2928],{"class":2563},[170,12570,2530],{"class":2523},[170,12572,12573],{"class":2563},"0.1",[170,12575,2530],{"class":2523},[170,12577,12578],{"class":2563},"64",[170,12580,2530],{"class":2523},[170,12582,12583],{"class":2563},"10\n",[170,12585,12586,12589,12591,12594,12596,12599,12602,12604,12607,12609,12611,12613,12616,12619,12621,12624,12627,12629,12632,12634],{"class":2511,"line":90},[170,12587,12588],{"class":2523},"lr, num_epochs, device ",[170,12590,926],{"class":2515},[170,12592,12593],{"class":2563}," 0.005",[170,12595,2530],{"class":2523},[170,12597,12598],{"class":2563},"200",[170,12600,12601],{"class":2523},", torch.device(",[170,12603,10999],{"class":9583},[170,12605,12606],{"class":9587},"cuda:0",[170,12608,10999],{"class":9583},[170,12610,7253],{"class":2523},[170,12612,11136],{"class":2515},[170,12614,12615],{"class":2523}," torch.cuda.device_count() ",[170,12617,12618],{"class":2515},">=",[170,12620,2622],{"class":2563},[170,12622,12623],{"class":2515}," else",[170,12625,12626],{"class":2523}," torch.device(",[170,12628,10999],{"class":9583},[170,12630,12631],{"class":9587},"cpu",[170,12633,10999],{"class":9583},[170,12635,2588],{"class":2523},[170,12637,12638,12641,12643,12646,12648],{"class":2511,"line":96},[170,12639,12640],{"class":2523},"ffn_num_hiddens, num_heads ",[170,12642,926],{"class":2515},[170,12644,12645],{"class":2563}," 64",[170,12647,2530],{"class":2523},[170,12649,12650],{"class":2563},"4\n",[170,12652,12653],{"class":2511,"line":2551},[170,12654,2832],{"emptyLinePlaceholder":114},[170,12656,12657,12660,12662],{"class":2511,"line":2570},[170,12658,12659],{"class":2523},"train_iter, src_vocab, tgt_vocab ",[170,12661,926],{"class":2515},[170,12663,12664],{"class":2523}," d2l.load_data_nmt(batch_size, num_steps)\n",[170,12666,12667],{"class":2511,"line":2591},[170,12668,2832],{"emptyLinePlaceholder":114},[170,12670,12671,12674,12676],{"class":2511,"line":2599},[170,12672,12673],{"class":2523},"encoder ",[170,12675,926],{"class":2515},[170,12677,12678],{"class":2523}," TransformerEncoder(\n",[170,12680,12681,12684],{"class":2511,"line":2610},[170,12682,12683],{"class":9568},"    len",[170,12685,12686],{"class":2523},"(src_vocab), num_hiddens, ffn_num_hiddens, num_heads, num_layers,\n",[170,12688,12689],{"class":2511,"line":2627},[170,12690,12691],{"class":2523},"    dropout)\n",[170,12693,12694,12697,12699],{"class":2511,"line":2643},[170,12695,12696],{"class":2523},"decoder ",[170,12698,926],{"class":2515},[170,12700,12701],{"class":2523}," TransformerDecoder(\n",[170,12703,12704,12706],{"class":2511,"line":2651},[170,12705,12683],{"class":9568},[170,12707,12708],{"class":2523},"(tgt_vocab), num_hiddens, ffn_num_hiddens, num_heads, num_layers,\n",[170,12710,12711],{"class":2511,"line":2668},[170,12712,12691],{"class":2523},[170,12714,12715,12718,12720],{"class":2511,"line":2674},[170,12716,12717],{"class":2523},"net ",[170,12719,926],{"class":2515},[170,12721,12722],{"class":2523}," EncoderDecoder(encoder, decoder)\n",[170,12724,12725],{"class":2511,"line":2699},[170,12726,12727],{"class":2523},"train_seq2seq(net, train_iter, lr, num_epochs, tgt_vocab, device)\n",[10,12729,12730],{"id":12730},"参考文献",[14,12732,12733],{},[134,12734,145],{"href":143,"rel":12735},[138],[14,12737,12738],{},[134,12739,139],{"href":136,"rel":12740},[138],[12742,12743,12744],"style",{},"html pre.shiki code .szJfE, html code.shiki .szJfE{--shiki-default:#D73A49;--shiki-dark:#FF79C6}html pre.shiki code .sCLZk, html code.shiki .sCLZk{--shiki-default:#6F42C1;--shiki-dark:#50FA7B}html pre.shiki code .scbbO, html code.shiki .scbbO{--shiki-default:#24292E;--shiki-dark:#F8F8F2}html pre.shiki code .syNf4, html code.shiki .syNf4{--shiki-default:#24292E;--shiki-default-font-style:inherit;--shiki-dark:#FFB86C;--shiki-dark-font-style:italic}html pre.shiki code .seLWX, html code.shiki .seLWX{--shiki-default:#032F62;--shiki-dark:#6272A4}html pre.shiki code .sfgPZ, html code.shiki .sfgPZ{--shiki-default:#6A737D;--shiki-dark:#6272A4}html pre.shiki code .soDru, html code.shiki .soDru{--shiki-default:#005CC5;--shiki-dark:#BD93F9}html pre.shiki code .sQkXh, html code.shiki .sQkXh{--shiki-default:#E36209;--shiki-default-font-style:inherit;--shiki-dark:#FFB86C;--shiki-dark-font-style:italic}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .skCyd, html code.shiki .skCyd{--shiki-default:#6F42C1;--shiki-dark:#8BE9FD}html pre.shiki code .sDP9b, html code.shiki .sDP9b{--shiki-default:#6F42C1;--shiki-default-font-style:inherit;--shiki-dark:#8BE9FD;--shiki-dark-font-style:italic}html pre.shiki code .sD3jg, html code.shiki .sD3jg{--shiki-default:#24292E;--shiki-default-font-style:inherit;--shiki-dark:#BD93F9;--shiki-dark-font-style:italic}html pre.shiki code .sPGBF, html code.shiki .sPGBF{--shiki-default:#005CC5;--shiki-default-font-style:inherit;--shiki-dark:#8BE9FD;--shiki-dark-font-style:italic}html pre.shiki code .sJti5, html code.shiki .sJti5{--shiki-default:#005CC5;--shiki-default-font-style:inherit;--shiki-dark:#BD93F9;--shiki-dark-font-style:italic}html pre.shiki code .sDoOe, html code.shiki .sDoOe{--shiki-default:#24292E;--shiki-dark:#FF79C6}html pre.shiki code .sDgm9, html code.shiki .sDgm9{--shiki-default:#005CC5;--shiki-dark:#8BE9FD}html pre.shiki code .sMWOi, html code.shiki .sMWOi{--shiki-default:#032F62;--shiki-dark:#E9F284}html pre.shiki code .sEzAm, html code.shiki .sEzAm{--shiki-default:#032F62;--shiki-dark:#F1FA8C}html pre.shiki code .sRfyP, html code.shiki .sRfyP{--shiki-default:#005CC5;--shiki-dark:#FF79C6}",{"title":89,"searchDepth":90,"depth":90,"links":12746},[12747,12748,12749,12753,12754,12755,12756,12757,12758,12759,12760],{"id":12,"depth":90,"text":12},{"id":149,"depth":90,"text":149},{"id":159,"depth":90,"text":159,"children":12750},[12751,12752],{"id":162,"depth":96,"text":162},{"id":2977,"depth":96,"text":2977},{"id":5369,"depth":90,"text":5369},{"id":7404,"depth":90,"text":7404},{"id":7900,"depth":90,"text":7900},{"id":8237,"depth":90,"text":8237},{"id":9151,"depth":90,"text":9152},{"id":9770,"depth":90,"text":9771},{"id":10777,"depth":90,"text":10777},{"id":12730,"depth":90,"text":12730},"距离首次学习 transformer 已经过去一年，内容忘的差不多了，决定复习一下。",{"date":12763,"image":12764,"alt":122,"tags":12765,"published":114},"30st Jan 2026","/blogs-img/blog1.jpg",[12766,12767],"deep-learning","学习笔记","/blogs/transformer",{"title":122,"description":12761},"blogs/1. transformer复习笔记","59tIbWfqdpJGJRh5PHVoEHC170F6vnech8kVXdG8MGw",{"id":12773,"title":12774,"body":12775,"description":44045,"extension":105,"meta":44046,"navigation":114,"ogImage":44048,"path":44052,"seo":44053,"stem":44054,"__hash__":44055},"content/blogs/2. PPO学习笔记.md","零基础速通 PPO 学习笔记",{"type":7,"value":12776,"toc":44032},[12777,12780,12783,12786,13294,13597,14046,14049,14052,14114,14453,18445,19046,19565,19569,19644,20308,20552,20671,20999,21002,21688,22827,22929,22932,25564,25670,27542,27546,27549,27583,30365,30368,30371,30374,31861,31864,31868,31871,31874,32009,33673,33676,33680,34323,34514,39039,39042,40087,40090,40095,40131,41422,41426,41429,42038,44022,44025],[10,12778,12779],{"id":12779},"学习背景",[14,12781,12782],{},"最近陶瓷到了港科的一个 RA，组里在做 Agent 强化学习相关内容，听从学长的指导来学习 PPO 算法相关知识。",[10,12784,12785],{"id":12785},"符号定义",[14,12787,12788,12789,12861,12862,12891,12892,12963,12964,12992,12993,13022,13023,13160,13161,13232,13233,13261,13262,13293],{},"在强化学习中，",[170,12790,12792,12810],{"className":12791},[173],[170,12793,12795],{"className":12794},[177],[179,12796,12797],{"xmlns":181},[183,12798,12799,12807],{},[186,12800,12801],{},[199,12802,12803,12805],{},[202,12804,134],{},[202,12806,888],{},[209,12808,12809],{"encoding":211},"a_t",[170,12811,12813],{"className":12812,"ariaHidden":217},[216],[170,12814,12816,12820],{"className":12815},[221],[170,12817],{"className":12818,"style":12819},[225],"height:0.5806em;vertical-align:-0.15em;",[170,12821,12823,12826],{"className":12822},[230],[170,12824,134],{"className":12825},[230,303],[170,12827,12829],{"className":12828},[307],[170,12830,12832,12853],{"className":12831},[242,243],[170,12833,12835,12850],{"className":12834},[247],[170,12836,12839],{"className":12837,"style":12838},[251],"height:0.2806em;",[170,12840,12841,12844],{"style":1362},[170,12842],{"className":12843,"style":651},[259],[170,12845,12847],{"className":12846},[264,265,266,267],[170,12848,888],{"className":12849},[230,303,267],[170,12851,338],{"className":12852},[337],[170,12854,12856],{"className":12855},[247],[170,12857,12859],{"className":12858,"style":1381},[251],[170,12860],{}," 一般表示 ",[170,12863,12865,12878],{"className":12864},[173],[170,12866,12868],{"className":12867},[177],[179,12869,12870],{"xmlns":181},[183,12871,12872,12876],{},[186,12873,12874],{},[202,12875,888],{},[209,12877,888],{"encoding":211},[170,12879,12881],{"className":12880,"ariaHidden":217},[216],[170,12882,12884,12888],{"className":12883},[221],[170,12885],{"className":12886,"style":12887},[225],"height:0.6151em;",[170,12889,888],{"className":12890},[230,303]," 时刻 Agent 采取的动作（action）；",[170,12893,12895,12914],{"className":12894},[173],[170,12896,12898],{"className":12897},[177],[179,12899,12900],{"xmlns":181},[183,12901,12902,12911],{},[186,12903,12904],{},[199,12905,12906,12909],{},[202,12907,12908],{},"s",[202,12910,888],{},[209,12912,12913],{"encoding":211},"s_t",[170,12915,12917],{"className":12916,"ariaHidden":217},[216],[170,12918,12920,12923],{"className":12919},[221],[170,12921],{"className":12922,"style":12819},[225],[170,12924,12926,12929],{"className":12925},[230],[170,12927,12908],{"className":12928},[230,303],[170,12930,12932],{"className":12931},[307],[170,12933,12935,12955],{"className":12934},[242,243],[170,12936,12938,12952],{"className":12937},[247],[170,12939,12941],{"className":12940,"style":12838},[251],[170,12942,12943,12946],{"style":1362},[170,12944],{"className":12945,"style":651},[259],[170,12947,12949],{"className":12948},[264,265,266,267],[170,12950,888],{"className":12951},[230,303,267],[170,12953,338],{"className":12954},[337],[170,12956,12958],{"className":12957},[247],[170,12959,12961],{"className":12960,"style":1381},[251],[170,12962],{}," 表示 ",[170,12965,12967,12980],{"className":12966},[173],[170,12968,12970],{"className":12969},[177],[179,12971,12972],{"xmlns":181},[183,12973,12974,12978],{},[186,12975,12976],{},[202,12977,888],{},[209,12979,888],{"encoding":211},[170,12981,12983],{"className":12982,"ariaHidden":217},[216],[170,12984,12986,12989],{"className":12985},[221],[170,12987],{"className":12988,"style":12887},[225],[170,12990,888],{"className":12991},[230,303]," 时刻 Agent 所处的状态（state）；",[170,12994,12996,13010],{"className":12995},[173],[170,12997,12999],{"className":12998},[177],[179,13000,13001],{"xmlns":181},[183,13002,13003,13007],{},[186,13004,13005],{},[202,13006,6849],{},[209,13008,13009],{"encoding":211},"\\pi",[170,13011,13013],{"className":13012,"ariaHidden":217},[216],[170,13014,13016,13019],{"className":13015},[221],[170,13017],{"className":13018,"style":458},[225],[170,13020,6849],{"className":13021,"style":552},[230,303]," 表示 Agent的策略函数（policy），输入状态，输入采取每个动作的概率分布 ",[170,13024,13026,13059],{"className":13025},[173],[170,13027,13029],{"className":13028},[177],[179,13030,13031],{"xmlns":181},[183,13032,13033,13056],{},[186,13034,13035,13037,13039,13045,13048,13054],{},[202,13036,6849],{},[572,13038,909],{"stretchy":908},[199,13040,13041,13043],{},[202,13042,134],{},[202,13044,888],{},[202,13046,13047],{"mathvariant":929},"∣",[199,13049,13050,13052],{},[202,13051,12908],{},[202,13053,888],{},[572,13055,923],{"stretchy":908},[209,13057,13058],{"encoding":211},"\\pi(a_t|s_t)",[170,13060,13062],{"className":13061,"ariaHidden":217},[216],[170,13063,13065,13068,13071,13074,13114,13117,13157],{"className":13064},[221],[170,13066],{"className":13067,"style":988},[225],[170,13069,6849],{"className":13070,"style":552},[230,303],[170,13072,909],{"className":13073},[234],[170,13075,13077,13080],{"className":13076},[230],[170,13078,134],{"className":13079},[230,303],[170,13081,13083],{"className":13082},[307],[170,13084,13086,13106],{"className":13085},[242,243],[170,13087,13089,13103],{"className":13088},[247],[170,13090,13092],{"className":13091,"style":12838},[251],[170,13093,13094,13097],{"style":1362},[170,13095],{"className":13096,"style":651},[259],[170,13098,13100],{"className":13099},[264,265,266,267],[170,13101,888],{"className":13102},[230,303,267],[170,13104,338],{"className":13105},[337],[170,13107,13109],{"className":13108},[247],[170,13110,13112],{"className":13111,"style":1381},[251],[170,13113],{},[170,13115,13047],{"className":13116},[230],[170,13118,13120,13123],{"className":13119},[230],[170,13121,12908],{"className":13122},[230,303],[170,13124,13126],{"className":13125},[307],[170,13127,13129,13149],{"className":13128},[242,243],[170,13130,13132,13146],{"className":13131},[247],[170,13133,13135],{"className":13134,"style":12838},[251],[170,13136,13137,13140],{"style":1362},[170,13138],{"className":13139,"style":651},[259],[170,13141,13143],{"className":13142},[264,265,266,267],[170,13144,888],{"className":13145},[230,303,267],[170,13147,338],{"className":13148},[337],[170,13150,13152],{"className":13151},[247],[170,13153,13155],{"className":13154,"style":1381},[251],[170,13156],{},[170,13158,923],{"className":13159},[424],"；",[170,13162,13164,13182],{"className":13163},[173],[170,13165,13167],{"className":13166},[177],[179,13168,13169],{"xmlns":181},[183,13170,13171,13179],{},[186,13172,13173],{},[199,13174,13175,13177],{},[202,13176,7933],{},[202,13178,888],{},[209,13180,13181],{"encoding":211},"r_t",[170,13183,13185],{"className":13184,"ariaHidden":217},[216],[170,13186,13188,13191],{"className":13187},[221],[170,13189],{"className":13190,"style":12819},[225],[170,13192,13194,13197],{"className":13193},[230],[170,13195,7933],{"className":13196,"style":3317},[230,303],[170,13198,13200],{"className":13199},[307],[170,13201,13203,13224],{"className":13202},[242,243],[170,13204,13206,13221],{"className":13205},[247],[170,13207,13209],{"className":13208,"style":12838},[251],[170,13210,13212,13215],{"style":13211},"top:-2.55em;margin-left:-0.0278em;margin-right:0.05em;",[170,13213],{"className":13214,"style":651},[259],[170,13216,13218],{"className":13217},[264,265,266,267],[170,13219,888],{"className":13220},[230,303,267],[170,13222,338],{"className":13223},[337],[170,13225,13227],{"className":13226},[247],[170,13228,13230],{"className":13229,"style":1381},[251],[170,13231],{}," 一边表示 ",[170,13234,13236,13249],{"className":13235},[173],[170,13237,13239],{"className":13238},[177],[179,13240,13241],{"xmlns":181},[183,13242,13243,13247],{},[186,13244,13245],{},[202,13246,888],{},[209,13248,888],{"encoding":211},[170,13250,13252],{"className":13251,"ariaHidden":217},[216],[170,13253,13255,13258],{"className":13254},[221],[170,13256],{"className":13257,"style":12887},[225],[170,13259,888],{"className":13260},[230,303]," 时刻 Agent 采取行动获得的奖励（Reward）；",[170,13263,13265,13280],{"className":13264},[173],[170,13266,13268],{"className":13267},[177],[179,13269,13270],{"xmlns":181},[183,13271,13272,13277],{},[186,13273,13274],{},[202,13275,13276],{},"τ",[209,13278,13279],{"encoding":211},"\\tau",[170,13281,13283],{"className":13282,"ariaHidden":217},[216],[170,13284,13286,13289],{"className":13285},[221],[170,13287],{"className":13288,"style":458},[225],[170,13290,13276],{"className":13291,"style":13292},[230,303],"margin-right:0.1132em;"," 表示轨迹（Trajectory），Episode 指一次完整的交互过程：从环境开始，到终止结束。Rollout 指按某个策略实际“跑出来”的一段轨迹数据。它不一定非得是完整的一局，也可以只是中间截取的一段。",[14,13295,13296],{},[170,13297,13299,13354],{"className":13298},[173],[170,13300,13302],{"className":13301},[177],[179,13303,13304],{"xmlns":181},[183,13305,13306,13351],{},[186,13307,13308,13310,13316,13318,13324,13326,13332,13334,13340,13342,13349],{},[572,13309,909],{"stretchy":908},[199,13311,13312,13314],{},[202,13313,12908],{},[192,13315,1865],{},[572,13317,914],{"separator":217},[199,13319,13320,13322],{},[202,13321,134],{},[192,13323,1865],{},[572,13325,914],{"separator":217},[199,13327,13328,13330],{},[202,13329,12908],{},[192,13331,194],{},[572,13333,914],{"separator":217},[199,13335,13336,13338],{},[202,13337,134],{},[192,13339,194],{},[572,13341,3077],{},[199,13343,13344,13346],{},[202,13345,12908],{},[202,13347,13348],{},"T",[572,13350,923],{"stretchy":908},[209,13352,13353],{"encoding":211},"(s_0,a_0,s_1,a_1 \\dots s_T)",[170,13355,13357],{"className":13356,"ariaHidden":217},[216],[170,13358,13360,13363,13366,13406,13409,13412,13452,13455,13458,13498,13501,13504,13544,13547,13550,13553,13594],{"className":13359},[221],[170,13361],{"className":13362,"style":988},[225],[170,13364,909],{"className":13365},[234],[170,13367,13369,13372],{"className":13368},[230],[170,13370,12908],{"className":13371},[230,303],[170,13373,13375],{"className":13374},[307],[170,13376,13378,13398],{"className":13377},[242,243],[170,13379,13381,13395],{"className":13380},[247],[170,13382,13384],{"className":13383,"style":3198},[251],[170,13385,13386,13389],{"style":1362},[170,13387],{"className":13388,"style":651},[259],[170,13390,13392],{"className":13391},[264,265,266,267],[170,13393,1865],{"className":13394},[230,267],[170,13396,338],{"className":13397},[337],[170,13399,13401],{"className":13400},[247],[170,13402,13404],{"className":13403,"style":1381},[251],[170,13405],{},[170,13407,914],{"className":13408},[1023],[170,13410],{"className":13411,"style":1027},[610],[170,13413,13415,13418],{"className":13414},[230],[170,13416,134],{"className":13417},[230,303],[170,13419,13421],{"className":13420},[307],[170,13422,13424,13444],{"className":13423},[242,243],[170,13425,13427,13441],{"className":13426},[247],[170,13428,13430],{"className":13429,"style":3198},[251],[170,13431,13432,13435],{"style":1362},[170,13433],{"className":13434,"style":651},[259],[170,13436,13438],{"className":13437},[264,265,266,267],[170,13439,1865],{"className":13440},[230,267],[170,13442,338],{"className":13443},[337],[170,13445,13447],{"className":13446},[247],[170,13448,13450],{"className":13449,"style":1381},[251],[170,13451],{},[170,13453,914],{"className":13454},[1023],[170,13456],{"className":13457,"style":1027},[610],[170,13459,13461,13464],{"className":13460},[230],[170,13462,12908],{"className":13463},[230,303],[170,13465,13467],{"className":13466},[307],[170,13468,13470,13490],{"className":13469},[242,243],[170,13471,13473,13487],{"className":13472},[247],[170,13474,13476],{"className":13475,"style":3198},[251],[170,13477,13478,13481],{"style":1362},[170,13479],{"className":13480,"style":651},[259],[170,13482,13484],{"className":13483},[264,265,266,267],[170,13485,194],{"className":13486},[230,267],[170,13488,338],{"className":13489},[337],[170,13491,13493],{"className":13492},[247],[170,13494,13496],{"className":13495,"style":1381},[251],[170,13497],{},[170,13499,914],{"className":13500},[1023],[170,13502],{"className":13503,"style":1027},[610],[170,13505,13507,13510],{"className":13506},[230],[170,13508,134],{"className":13509},[230,303],[170,13511,13513],{"className":13512},[307],[170,13514,13516,13536],{"className":13515},[242,243],[170,13517,13519,13533],{"className":13518},[247],[170,13520,13522],{"className":13521,"style":3198},[251],[170,13523,13524,13527],{"style":1362},[170,13525],{"className":13526,"style":651},[259],[170,13528,13530],{"className":13529},[264,265,266,267],[170,13531,194],{"className":13532},[230,267],[170,13534,338],{"className":13535},[337],[170,13537,13539],{"className":13538},[247],[170,13540,13542],{"className":13541,"style":1381},[251],[170,13543],{},[170,13545],{"className":13546,"style":1027},[610],[170,13548,3077],{"className":13549},[1073],[170,13551],{"className":13552,"style":1027},[610],[170,13554,13556,13559],{"className":13555},[230],[170,13557,12908],{"className":13558},[230,303],[170,13560,13562],{"className":13561},[307],[170,13563,13565,13586],{"className":13564},[242,243],[170,13566,13568,13583],{"className":13567},[247],[170,13569,13572],{"className":13570,"style":13571},[251],"height:0.3283em;",[170,13573,13574,13577],{"style":1362},[170,13575],{"className":13576,"style":651},[259],[170,13578,13580],{"className":13579},[264,265,266,267],[170,13581,13348],{"className":13582,"style":3292},[230,303,267],[170,13584,338],{"className":13585},[337],[170,13587,13589],{"className":13588},[247],[170,13590,13592],{"className":13591,"style":1381},[251],[170,13593],{},[170,13595,923],{"className":13596},[424],[14,13598,13599,13818,13819,425],{},[170,13600,13602,13648],{"className":13601},[173],[170,13603,13605],{"className":13604},[177],[179,13606,13607],{"xmlns":181},[183,13608,13609,13645],{},[186,13610,13611,13623,13625,13627,13629,13635,13637,13643],{},[199,13612,13613,13615],{},[202,13614,12908],{},[186,13616,13617,13619,13621],{},[202,13618,888],{},[572,13620,5884],{},[192,13622,194],{},[572,13624,926],{},[202,13626,11764],{},[572,13628,909],{"stretchy":908},[199,13630,13631,13633],{},[202,13632,12908],{},[202,13634,888],{},[572,13636,914],{"separator":217},[199,13638,13639,13641],{},[202,13640,134],{},[202,13642,888],{},[572,13644,923],{"stretchy":908},[209,13646,13647],{"encoding":211}," s_{t+1} = f(s_t,a_t)",[170,13649,13651,13716],{"className":13650,"ariaHidden":217},[216],[170,13652,13654,13658,13707,13710,13713],{"className":13653},[221],[170,13655],{"className":13656,"style":13657},[225],"height:0.6389em;vertical-align:-0.2083em;",[170,13659,13661,13664],{"className":13660},[230],[170,13662,12908],{"className":13663},[230,303],[170,13665,13667],{"className":13666},[307],[170,13668,13670,13699],{"className":13669},[242,243],[170,13671,13673,13696],{"className":13672},[247],[170,13674,13676],{"className":13675,"style":3198},[251],[170,13677,13678,13681],{"style":1362},[170,13679],{"className":13680,"style":651},[259],[170,13682,13684],{"className":13683},[264,265,266,267],[170,13685,13687,13690,13693],{"className":13686},[230,267],[170,13688,888],{"className":13689},[230,303,267],[170,13691,5884],{"className":13692},[664,267],[170,13694,194],{"className":13695},[230,267],[170,13697,338],{"className":13698},[337],[170,13700,13702],{"className":13701},[247],[170,13703,13705],{"className":13704,"style":7045},[251],[170,13706],{},[170,13708],{"className":13709,"style":611},[610],[170,13711,926],{"className":13712},[615],[170,13714],{"className":13715,"style":611},[610],[170,13717,13719,13722,13726,13729,13769,13772,13775,13815],{"className":13718},[221],[170,13720],{"className":13721,"style":988},[225],[170,13723,11764],{"className":13724,"style":13725},[230,303],"margin-right:0.10764em;",[170,13727,909],{"className":13728},[234],[170,13730,13732,13735],{"className":13731},[230],[170,13733,12908],{"className":13734},[230,303],[170,13736,13738],{"className":13737},[307],[170,13739,13741,13761],{"className":13740},[242,243],[170,13742,13744,13758],{"className":13743},[247],[170,13745,13747],{"className":13746,"style":12838},[251],[170,13748,13749,13752],{"style":1362},[170,13750],{"className":13751,"style":651},[259],[170,13753,13755],{"className":13754},[264,265,266,267],[170,13756,888],{"className":13757},[230,303,267],[170,13759,338],{"className":13760},[337],[170,13762,13764],{"className":13763},[247],[170,13765,13767],{"className":13766,"style":1381},[251],[170,13768],{},[170,13770,914],{"className":13771},[1023],[170,13773],{"className":13774,"style":1027},[610],[170,13776,13778,13781],{"className":13777},[230],[170,13779,134],{"className":13780},[230,303],[170,13782,13784],{"className":13783},[307],[170,13785,13787,13807],{"className":13786},[242,243],[170,13788,13790,13804],{"className":13789},[247],[170,13791,13793],{"className":13792,"style":12838},[251],[170,13794,13795,13798],{"style":1362},[170,13796],{"className":13797,"style":651},[259],[170,13799,13801],{"className":13800},[264,265,266,267],[170,13802,888],{"className":13803},[230,303,267],[170,13805,338],{"className":13806},[337],[170,13808,13810],{"className":13809},[247],[170,13811,13813],{"className":13812,"style":1381},[251],[170,13814],{},[170,13816,923],{"className":13817},[424]," 或者 ",[170,13820,13822,13872],{"className":13821},[173],[170,13823,13825],{"className":13824},[177],[179,13826,13827],{"xmlns":181},[183,13828,13829,13869],{},[186,13830,13831,13843,13845,13847,13849,13851,13853,13859,13861,13867],{},[199,13832,13833,13835],{},[202,13834,12908],{},[186,13836,13837,13839,13841],{},[202,13838,888],{},[572,13840,5884],{},[192,13842,194],{},[572,13844,926],{},[202,13846,5711],{},[572,13848,909],{"stretchy":908},[572,13850,1927],{},[202,13852,13047],{"mathvariant":929},[199,13854,13855,13857],{},[202,13856,12908],{},[202,13858,888],{},[572,13860,914],{"separator":217},[199,13862,13863,13865],{},[202,13864,134],{},[202,13866,888],{},[572,13868,923],{"stretchy":908},[209,13870,13871],{"encoding":211},"s_{t+1} = P(\\cdot|s_t,a_t)",[170,13873,13875,13939],{"className":13874,"ariaHidden":217},[216],[170,13876,13878,13881,13930,13933,13936],{"className":13877},[221],[170,13879],{"className":13880,"style":13657},[225],[170,13882,13884,13887],{"className":13883},[230],[170,13885,12908],{"className":13886},[230,303],[170,13888,13890],{"className":13889},[307],[170,13891,13893,13922],{"className":13892},[242,243],[170,13894,13896,13919],{"className":13895},[247],[170,13897,13899],{"className":13898,"style":3198},[251],[170,13900,13901,13904],{"style":1362},[170,13902],{"className":13903,"style":651},[259],[170,13905,13907],{"className":13906},[264,265,266,267],[170,13908,13910,13913,13916],{"className":13909},[230,267],[170,13911,888],{"className":13912},[230,303,267],[170,13914,5884],{"className":13915},[664,267],[170,13917,194],{"className":13918},[230,267],[170,13920,338],{"className":13921},[337],[170,13923,13925],{"className":13924},[247],[170,13926,13928],{"className":13927,"style":7045},[251],[170,13929],{},[170,13931],{"className":13932,"style":611},[610],[170,13934,926],{"className":13935},[615],[170,13937],{"className":13938,"style":611},[610],[170,13940,13942,13945,13948,13951,13954,13957,13997,14000,14003,14043],{"className":13941},[221],[170,13943],{"className":13944,"style":988},[225],[170,13946,5711],{"className":13947,"style":3292},[230,303],[170,13949,909],{"className":13950},[234],[170,13952,1927],{"className":13953},[230],[170,13955,13047],{"className":13956},[230],[170,13958,13960,13963],{"className":13959},[230],[170,13961,12908],{"className":13962},[230,303],[170,13964,13966],{"className":13965},[307],[170,13967,13969,13989],{"className":13968},[242,243],[170,13970,13972,13986],{"className":13971},[247],[170,13973,13975],{"className":13974,"style":12838},[251],[170,13976,13977,13980],{"style":1362},[170,13978],{"className":13979,"style":651},[259],[170,13981,13983],{"className":13982},[264,265,266,267],[170,13984,888],{"className":13985},[230,303,267],[170,13987,338],{"className":13988},[337],[170,13990,13992],{"className":13991},[247],[170,13993,13995],{"className":13994,"style":1381},[251],[170,13996],{},[170,13998,914],{"className":13999},[1023],[170,14001],{"className":14002,"style":1027},[610],[170,14004,14006,14009],{"className":14005},[230],[170,14007,134],{"className":14008},[230,303],[170,14010,14012],{"className":14011},[307],[170,14013,14015,14035],{"className":14014},[242,243],[170,14016,14018,14032],{"className":14017},[247],[170,14019,14021],{"className":14020,"style":12838},[251],[170,14022,14023,14026],{"style":1362},[170,14024],{"className":14025,"style":651},[259],[170,14027,14029],{"className":14028},[264,265,266,267],[170,14030,888],{"className":14031},[230,303,267],[170,14033,338],{"className":14034},[337],[170,14036,14038],{"className":14037},[247],[170,14039,14041],{"className":14040,"style":1381},[251],[170,14042],{},[170,14044,923],{"className":14045},[424],[14,14047,14048],{},"Return 一般表示回报，指从当前时间点到游戏结束时奖励的累积和或者加权累积和。",[10,14050,14051],{"id":14051},"训练目标",[14,14053,14054,14055,14083,14084,14113],{},"训练一个 Policy 神经网络 ",[170,14056,14058,14071],{"className":14057},[173],[170,14059,14061],{"className":14060},[177],[179,14062,14063],{"xmlns":181},[183,14064,14065,14069],{},[186,14066,14067],{},[202,14068,6849],{},[209,14070,13009],{"encoding":211},[170,14072,14074],{"className":14073,"ariaHidden":217},[216],[170,14075,14077,14080],{"className":14076},[221],[170,14078],{"className":14079,"style":458},[225],[170,14081,6849],{"className":14082,"style":552},[230,303],"，在所有状态 ",[170,14085,14087,14100],{"className":14086},[173],[170,14088,14090],{"className":14089},[177],[179,14091,14092],{"xmlns":181},[183,14093,14094,14098],{},[186,14095,14096],{},[202,14097,7952],{},[209,14099,7952],{"encoding":211},[170,14101,14103],{"className":14102,"ariaHidden":217},[216],[170,14104,14106,14110],{"className":14105},[221],[170,14107],{"className":14108,"style":14109},[225],"height:0.6833em;",[170,14111,7952],{"className":14112,"style":6157},[230,303]," 下，给出相应的 Action，得到的 Return 的期望最大。",[14,14115,14116],{},[170,14117,14119,14196],{"className":14118},[173],[170,14120,14122],{"className":14121},[177],[179,14123,14124],{"xmlns":181},[183,14125,14126,14193],{},[186,14127,14128,14130,14132,14134,14136,14138,14140,14164,14166,14173,14175,14177,14179,14181,14187,14189,14191],{},[202,14129,5714],{},[572,14131,909],{"stretchy":908},[202,14133,581],{},[572,14135,909],{"stretchy":908},[202,14137,13276],{},[572,14139,923],{"stretchy":908},[199,14141,14142,14144],{},[572,14143,923],{"stretchy":908},[186,14145,14146,14148,14151,14158,14160,14162],{},[202,14147,13276],{},[572,14149,14150],{},"∼",[199,14152,14153,14155],{},[202,14154,5711],{},[202,14156,14157],{},"θ",[572,14159,909],{"stretchy":908},[202,14161,13276],{},[572,14163,923],{"stretchy":908},[572,14165,926],{},[14167,14168,14169,14171],"munder",{},[572,14170,1937],{},[202,14172,13276],{},[202,14174,581],{},[572,14176,909],{"stretchy":908},[202,14178,13276],{},[572,14180,923],{"stretchy":908},[199,14182,14183,14185],{},[202,14184,5711],{},[202,14186,14157],{},[572,14188,909],{"stretchy":908},[202,14190,13276],{},[572,14192,923],{"stretchy":908},[209,14194,14195],{"encoding":211}," E (R(\\tau))_{\\tau \\sim P_{\\theta}(\\tau)} = \\sum\\limits_\\tau R(\\tau)P_\\theta(\\tau)",[170,14197,14199,14334],{"className":14198,"ariaHidden":217},[216],[170,14200,14202,14206,14209,14212,14216,14219,14222,14225,14325,14328,14331],{"className":14201},[221],[170,14203],{"className":14204,"style":14205},[225],"height:1.1052em;vertical-align:-0.3552em;",[170,14207,5714],{"className":14208,"style":6157},[230,303],[170,14210,909],{"className":14211},[234],[170,14213,581],{"className":14214,"style":14215},[230,303],"margin-right:0.00773em;",[170,14217,909],{"className":14218},[234],[170,14220,13276],{"className":14221,"style":13292},[230,303],[170,14223,923],{"className":14224},[424],[170,14226,14228,14231],{"className":14227},[424],[170,14229,923],{"className":14230},[424],[170,14232,14234],{"className":14233},[307],[170,14235,14237,14317],{"className":14236},[242,243],[170,14238,14240,14314],{"className":14239},[247],[170,14241,14243],{"className":14242,"style":317},[251],[170,14244,14246,14249],{"style":14245},"top:-2.5198em;margin-left:0em;margin-right:0.05em;",[170,14247],{"className":14248,"style":651},[259],[170,14250,14252],{"className":14251},[264,265,266,267],[170,14253,14255,14258,14261,14305,14308,14311],{"className":14254},[230,267],[170,14256,13276],{"className":14257,"style":13292},[230,303,267],[170,14259,14150],{"className":14260},[615,267],[170,14262,14264,14267],{"className":14263},[230,267],[170,14265,5711],{"className":14266,"style":3292},[230,303,267],[170,14268,14270],{"className":14269},[307],[170,14271,14273,14297],{"className":14272},[242,243],[170,14274,14276,14294],{"className":14275},[247],[170,14277,14279],{"className":14278,"style":317},[251],[170,14280,14282,14285],{"style":14281},"top:-2.3488em;margin-left:-0.1389em;margin-right:0.0714em;",[170,14283],{"className":14284,"style":324},[259],[170,14286,14288],{"className":14287},[264,328,329,267],[170,14289,14291],{"className":14290},[230,267],[170,14292,14157],{"className":14293,"style":3317},[230,303,267],[170,14295,338],{"className":14296},[337],[170,14298,14300],{"className":14299},[247],[170,14301,14303],{"className":14302,"style":345},[251],[170,14304],{},[170,14306,909],{"className":14307},[234,267],[170,14309,13276],{"className":14310,"style":13292},[230,303,267],[170,14312,923],{"className":14313},[424,267],[170,14315,338],{"className":14316},[337],[170,14318,14320],{"className":14319},[247],[170,14321,14323],{"className":14322,"style":6209},[251],[170,14324],{},[170,14326],{"className":14327,"style":611},[610],[170,14329,926],{"className":14330},[615],[170,14332],{"className":14333,"style":611},[610],[170,14335,14337,14341,14389,14392,14395,14398,14401,14404,14444,14447,14450],{"className":14336},[221],[170,14338],{"className":14339,"style":14340},[225],"height:1.7em;vertical-align:-0.95em;",[170,14342,14345],{"className":14343},[1062,14344],"op-limits",[170,14346,14348,14380],{"className":14347},[242,243],[170,14349,14351,14377],{"className":14350},[247],[170,14352,14355,14367],{"className":14353,"style":14354},[251],"height:0.75em;",[170,14356,14358,14361],{"style":14357},"top:-2.15em;margin-left:0em;",[170,14359],{"className":14360,"style":260},[259],[170,14362,14364],{"className":14363},[264,265,266,267],[170,14365,13276],{"className":14366,"style":13292},[230,303,267],[170,14368,14369,14372],{"style":289},[170,14370],{"className":14371,"style":260},[259],[170,14373,14374],{},[170,14375,1937],{"className":14376},[1062,2021,2022],[170,14378,338],{"className":14379},[337],[170,14381,14383],{"className":14382},[247],[170,14384,14387],{"className":14385,"style":14386},[251],"height:0.95em;",[170,14388],{},[170,14390],{"className":14391,"style":1027},[610],[170,14393,581],{"className":14394,"style":14215},[230,303],[170,14396,909],{"className":14397},[234],[170,14399,13276],{"className":14400,"style":13292},[230,303],[170,14402,923],{"className":14403},[424],[170,14405,14407,14410],{"className":14406},[230],[170,14408,5711],{"className":14409,"style":3292},[230,303],[170,14411,14413],{"className":14412},[307],[170,14414,14416,14436],{"className":14415},[242,243],[170,14417,14419,14433],{"className":14418},[247],[170,14420,14422],{"className":14421,"style":1359},[251],[170,14423,14424,14427],{"style":7567},[170,14425],{"className":14426,"style":651},[259],[170,14428,14430],{"className":14429},[264,265,266,267],[170,14431,14157],{"className":14432,"style":3317},[230,303,267],[170,14434,338],{"className":14435},[337],[170,14437,14439],{"className":14438},[247],[170,14440,14442],{"className":14441,"style":1381},[251],[170,14443],{},[170,14445,909],{"className":14446},[234],[170,14448,13276],{"className":14449,"style":13292},[230,303],[170,14451,923],{"className":14452},[424],[170,14454,14457],{"className":14455},[14456],"katex-display",[170,14458,14460,15240],{"className":14459},[173],[170,14461,14463],{"className":14462},[177],[179,14464,14465],{"xmlns":181,"display":9588},[183,14466,14467,15237],{},[14468,14469,14473,14559,14605,14685,14763,14857,14930,15033,15135],"mtable",{"rowspacing":14470,"columnalign":14471,"columnspacing":14472},"0.25em","right left","0em",[14474,14475,14476,14521],"mtr",{},[14477,14478,14479],"mtd",{},[14480,14481,14482],"mstyle",{"scriptlevel":1865,"displaystyle":217},[186,14483,14484,14487,14489,14491,14493,14495,14497,14499],{},[202,14485,14486],{"mathvariant":929},"∇",[202,14488,5714],{},[572,14490,909],{"stretchy":908},[202,14492,581],{},[572,14494,909],{"stretchy":908},[202,14496,13276],{},[572,14498,923],{"stretchy":908},[199,14500,14501,14503],{},[572,14502,923],{"stretchy":908},[186,14504,14505,14507,14509,14515,14517,14519],{},[202,14506,13276],{},[572,14508,14150],{},[199,14510,14511,14513],{},[202,14512,5711],{},[202,14514,14157],{},[572,14516,909],{"stretchy":908},[202,14518,13276],{},[572,14520,923],{"stretchy":908},[14477,14522,14523],{},[14480,14524,14525],{"scriptlevel":1865,"displaystyle":217},[186,14526,14527,14529,14531,14533,14539,14541,14543,14545,14547,14553,14555,14557],{},[186,14528],{},[572,14530,926],{},[202,14532,14486],{"mathvariant":929},[14167,14534,14535,14537],{},[572,14536,1937],{},[202,14538,13276],{},[202,14540,581],{},[572,14542,909],{"stretchy":908},[202,14544,13276],{},[572,14546,923],{"stretchy":908},[199,14548,14549,14551],{},[202,14550,5711],{},[202,14552,14157],{},[572,14554,909],{"stretchy":908},[202,14556,13276],{},[572,14558,923],{"stretchy":908},[14474,14560,14561,14567],{},[14477,14562,14563],{},[14480,14564,14565],{"scriptlevel":1865,"displaystyle":217},[186,14566],{},[14477,14568,14569],{},[14480,14570,14571],{"scriptlevel":1865,"displaystyle":217},[186,14572,14573,14575,14577,14583,14585,14587,14589,14591,14593,14599,14601,14603],{},[186,14574],{},[572,14576,926],{},[14167,14578,14579,14581],{},[572,14580,1937],{},[202,14582,13276],{},[202,14584,581],{},[572,14586,909],{"stretchy":908},[202,14588,13276],{},[572,14590,923],{"stretchy":908},[202,14592,14486],{"mathvariant":929},[199,14594,14595,14597],{},[202,14596,5711],{},[202,14598,14157],{},[572,14600,909],{"stretchy":908},[202,14602,13276],{},[572,14604,923],{"stretchy":908},[14474,14606,14607,14613],{},[14477,14608,14609],{},[14480,14610,14611],{"scriptlevel":1865,"displaystyle":217},[186,14612],{},[14477,14614,14615],{},[14480,14616,14617],{"scriptlevel":1865,"displaystyle":217},[186,14618,14619,14621,14623,14629,14631,14633,14635,14637,14639,14645,14647,14649,14651,14653],{},[186,14620],{},[572,14622,926],{},[14167,14624,14625,14627],{},[572,14626,1937],{},[202,14628,13276],{},[202,14630,581],{},[572,14632,909],{"stretchy":908},[202,14634,13276],{},[572,14636,923],{"stretchy":908},[202,14638,14486],{"mathvariant":929},[199,14640,14641,14643],{},[202,14642,5711],{},[202,14644,14157],{},[572,14646,909],{"stretchy":908},[202,14648,13276],{},[572,14650,923],{"stretchy":908},[572,14652,1927],{},[14480,14654,14655],{"scriptlevel":1865,"displaystyle":217},[189,14656,14657,14671],{},[186,14658,14659,14665,14667,14669],{},[199,14660,14661,14663],{},[202,14662,5711],{},[202,14664,14157],{},[572,14666,909],{"stretchy":908},[202,14668,13276],{},[572,14670,923],{"stretchy":908},[186,14672,14673,14679,14681,14683],{},[199,14674,14675,14677],{},[202,14676,5711],{},[202,14678,14157],{},[572,14680,909],{"stretchy":908},[202,14682,13276],{},[572,14684,923],{"stretchy":908},[14474,14686,14687,14693],{},[14477,14688,14689],{},[14480,14690,14691],{"scriptlevel":1865,"displaystyle":217},[186,14692],{},[14477,14694,14695],{},[14480,14696,14697],{"scriptlevel":1865,"displaystyle":217},[186,14698,14699,14701,14703,14709,14715,14717,14719,14721,14723,14725,14727,14729],{},[186,14700],{},[572,14702,926],{},[14167,14704,14705,14707],{},[572,14706,1937],{},[202,14708,13276],{},[199,14710,14711,14713],{},[202,14712,5711],{},[202,14714,14157],{},[572,14716,909],{"stretchy":908},[202,14718,13276],{},[572,14720,923],{"stretchy":908},[202,14722,581],{},[572,14724,909],{"stretchy":908},[202,14726,13276],{},[572,14728,923],{"stretchy":908},[14480,14730,14731],{"scriptlevel":1865,"displaystyle":217},[189,14732,14733,14749],{},[186,14734,14735,14737,14743,14745,14747],{},[202,14736,14486],{"mathvariant":929},[199,14738,14739,14741],{},[202,14740,5711],{},[202,14742,14157],{},[572,14744,909],{"stretchy":908},[202,14746,13276],{},[572,14748,923],{"stretchy":908},[186,14750,14751,14757,14759,14761],{},[199,14752,14753,14755],{},[202,14754,5711],{},[202,14756,14157],{},[572,14758,909],{"stretchy":908},[202,14760,13276],{},[572,14762,923],{"stretchy":908},[14474,14764,14765,14771],{},[14477,14766,14767],{},[14480,14768,14769],{"scriptlevel":1865,"displaystyle":217},[186,14770],{},[14477,14772,14773],{},[14480,14774,14775],{"scriptlevel":1865,"displaystyle":217},[186,14776,14777,14779,14782,14788,14803,14805,14807,14813,14815],{},[186,14778],{},[572,14780,14781],{},"≈",[189,14783,14784,14786],{},[192,14785,194],{},[202,14787,7429],{},[14789,14790,14791,14793,14801],"munderover",{},[572,14792,1937],{},[186,14794,14795,14797,14799],{},[202,14796,446],{},[572,14798,926],{},[192,14800,194],{},[202,14802,7429],{},[202,14804,581],{},[572,14806,909],{"stretchy":908},[576,14808,14809,14811],{},[202,14810,13276],{},[202,14812,446],{},[572,14814,923],{"stretchy":908},[14480,14816,14817],{"scriptlevel":1865,"displaystyle":217},[189,14818,14819,14839],{},[186,14820,14821,14823,14829,14831,14837],{},[202,14822,14486],{"mathvariant":929},[199,14824,14825,14827],{},[202,14826,14],{},[202,14828,14157],{},[572,14830,909],{"stretchy":908},[576,14832,14833,14835],{},[202,14834,13276],{},[202,14836,446],{},[572,14838,923],{"stretchy":908},[186,14840,14841,14847,14849,14855],{},[199,14842,14843,14845],{},[202,14844,5711],{},[202,14846,14157],{},[572,14848,909],{"stretchy":908},[576,14850,14851,14853],{},[202,14852,13276],{},[202,14854,446],{},[572,14856,923],{"stretchy":908},[14474,14858,14859,14865],{},[14477,14860,14861],{},[14480,14862,14863],{"scriptlevel":1865,"displaystyle":217},[186,14864],{},[14477,14866,14867],{},[14480,14868,14869],{"scriptlevel":1865,"displaystyle":217},[186,14870,14871,14873,14875,14881,14895,14897,14899,14905,14907,14909,14912,14914,14920,14922,14928],{},[186,14872],{},[572,14874,926],{},[189,14876,14877,14879],{},[192,14878,194],{},[202,14880,7429],{},[14789,14882,14883,14885,14893],{},[572,14884,1937],{},[186,14886,14887,14889,14891],{},[202,14888,446],{},[572,14890,926],{},[192,14892,194],{},[202,14894,7429],{},[202,14896,581],{},[572,14898,909],{"stretchy":908},[576,14900,14901,14903],{},[202,14902,13276],{},[202,14904,446],{},[572,14906,923],{"stretchy":908},[202,14908,14486],{"mathvariant":929},[202,14910,14911],{},"log",[572,14913,933],{},[199,14915,14916,14918],{},[202,14917,5711],{},[202,14919,14157],{},[572,14921,909],{"stretchy":908},[576,14923,14924,14926],{},[202,14925,13276],{},[202,14927,446],{},[572,14929,923],{"stretchy":908},[14474,14931,14932,14938],{},[14477,14933,14934],{},[14480,14935,14936],{"scriptlevel":1865,"displaystyle":217},[186,14937],{},[14477,14939,14940],{},[14480,14941,14942],{"scriptlevel":1865,"displaystyle":217},[186,14943,14944,14946,14948,14954,14968,14970,14972,14978,14980,14982,14984,14986,15005,15011,15013,15021,15023,15031],{},[186,14945],{},[572,14947,926],{},[189,14949,14950,14952],{},[192,14951,194],{},[202,14953,7429],{},[14789,14955,14956,14958,14966],{},[572,14957,1937],{},[186,14959,14960,14962,14964],{},[202,14961,446],{},[572,14963,926],{},[192,14965,194],{},[202,14967,7429],{},[202,14969,581],{},[572,14971,909],{"stretchy":908},[576,14973,14974,14976],{},[202,14975,13276],{},[202,14977,446],{},[572,14979,923],{"stretchy":908},[202,14981,14486],{"mathvariant":929},[202,14983,14911],{},[572,14985,933],{},[14789,14987,14988,14991,14999],{},[572,14989,14990],{},"∏",[186,14992,14993,14995,14997],{},[202,14994,888],{},[572,14996,926],{},[192,14998,194],{},[199,15000,15001,15003],{},[202,15002,13348],{},[202,15004,446],{},[199,15006,15007,15009],{},[202,15008,5711],{},[202,15010,14157],{},[572,15012,909],{"stretchy":908},[1933,15014,15015,15017,15019],{},[202,15016,134],{},[202,15018,446],{},[202,15020,888],{},[202,15022,13047],{"mathvariant":929},[1933,15024,15025,15027,15029],{},[202,15026,12908],{},[202,15028,446],{},[202,15030,888],{},[572,15032,923],{"stretchy":908},[14474,15034,15035,15041],{},[14477,15036,15037],{},[14480,15038,15039],{"scriptlevel":1865,"displaystyle":217},[186,15040],{},[14477,15042,15043],{},[14480,15044,15045],{"scriptlevel":1865,"displaystyle":217},[186,15046,15047,15049,15051,15057,15071,15073,15075,15081,15083,15101,15103,15105,15107,15113,15115,15123,15125,15133],{},[186,15048],{},[572,15050,926],{},[189,15052,15053,15055],{},[192,15054,194],{},[202,15056,7429],{},[14789,15058,15059,15061,15069],{},[572,15060,1937],{},[186,15062,15063,15065,15067],{},[202,15064,446],{},[572,15066,926],{},[192,15068,194],{},[202,15070,7429],{},[202,15072,581],{},[572,15074,909],{"stretchy":908},[576,15076,15077,15079],{},[202,15078,13276],{},[202,15080,446],{},[572,15082,923],{"stretchy":908},[14789,15084,15085,15087,15095],{},[572,15086,1937],{},[186,15088,15089,15091,15093],{},[202,15090,888],{},[572,15092,926],{},[192,15094,194],{},[199,15096,15097,15099],{},[202,15098,13348],{},[202,15100,446],{},[202,15102,14486],{"mathvariant":929},[202,15104,14911],{},[572,15106,933],{},[199,15108,15109,15111],{},[202,15110,5711],{},[202,15112,14157],{},[572,15114,909],{"stretchy":908},[1933,15116,15117,15119,15121],{},[202,15118,134],{},[202,15120,446],{},[202,15122,888],{},[202,15124,13047],{"mathvariant":929},[1933,15126,15127,15129,15131],{},[202,15128,12908],{},[202,15130,446],{},[202,15132,888],{},[572,15134,923],{"stretchy":908},[14474,15136,15137,15143],{},[14477,15138,15139],{},[14480,15140,15141],{"scriptlevel":1865,"displaystyle":217},[186,15142],{},[14477,15144,15145],{},[14480,15146,15147],{"scriptlevel":1865,"displaystyle":217},[186,15148,15149,15151,15153,15159,15173,15191,15193,15195,15201,15203,15205,15207,15209,15215,15217,15225,15227,15235],{},[186,15150],{},[572,15152,926],{},[189,15154,15155,15157],{},[192,15156,194],{},[202,15158,7429],{},[14789,15160,15161,15163,15171],{},[572,15162,1937],{},[186,15164,15165,15167,15169],{},[202,15166,446],{},[572,15168,926],{},[192,15170,194],{},[202,15172,7429],{},[14789,15174,15175,15177,15185],{},[572,15176,1937],{},[186,15178,15179,15181,15183],{},[202,15180,888],{},[572,15182,926],{},[192,15184,194],{},[199,15186,15187,15189],{},[202,15188,13348],{},[202,15190,446],{},[202,15192,581],{},[572,15194,909],{"stretchy":908},[576,15196,15197,15199],{},[202,15198,13276],{},[202,15200,446],{},[572,15202,923],{"stretchy":908},[202,15204,14486],{"mathvariant":929},[202,15206,14911],{},[572,15208,933],{},[199,15210,15211,15213],{},[202,15212,5711],{},[202,15214,14157],{},[572,15216,909],{"stretchy":908},[1933,15218,15219,15221,15223],{},[202,15220,134],{},[202,15222,446],{},[202,15224,888],{},[202,15226,13047],{"mathvariant":929},[1933,15228,15229,15231,15233],{},[202,15230,12908],{},[202,15232,446],{},[202,15234,888],{},[572,15236,923],{"stretchy":908},[209,15238,15239],{"encoding":211},"\\begin{aligned}\n\\nabla E(R(\\tau))_{\\tau \\sim P_\\theta(\\tau)} &= \\nabla \\sum_\\tau R(\\tau)P_\\theta(\\tau)\\\\\n&= \\sum_\\tau R(\\tau)\\nabla P_\\theta(\\tau)\\\\\n&= \\sum_\\tau R(\\tau)\\nabla P_\\theta(\\tau) \\cdot \\dfrac{P_\\theta(\\tau)}{P_\\theta(\\tau)}\\\\\n&= \\sum_\\tau P_\\theta(\\tau)R(\\tau) \\dfrac{\\nabla P_\\theta(\\tau)}{P_\\theta(\\tau)}\\\\\n&\\approx \\frac{1}{N}\\sum\\limits_{n=1}^N R(\\tau^n)\\dfrac{\\nabla p_\\theta(\\tau^n)}{P_\\theta(\\tau^n)}\\\\\n&= \\frac{1}{N}\\sum\\limits_{n=1}^N R(\\tau^n)\\nabla \\log P_\\theta(\\tau^n)\\\\\n&= \\frac{1}{N}\\sum\\limits_{n=1}^N R(\\tau^n)\\nabla \\log \\prod\\limits_{t=1}^{T_n}P_\\theta(a_n^t|s_n^t)\\\\\n&= \\frac{1}{N}\\sum\\limits_{n=1}^N R(\\tau^n)\\sum\\limits_{t=1}^{T_n}\\nabla \\log P_\\theta(a_n^t|s_n^t)\\\\\n&= \\frac{1}{N}\\sum\\limits_{n=1}^N \\sum\\limits_{t=1}^{T_n} R(\\tau^n)\\nabla \\log P_\\theta(a_n^t|s_n^t)\\\\\n\\end{aligned}",[170,15241,15243],{"className":15242,"ariaHidden":217},[216],[170,15244,15246,15250],{"className":15245},[221],[170,15247],{"className":15248,"style":15249},[225],"height:28.1646em;vertical-align:-13.8323em;",[170,15251,15253],{"className":15252},[230],[170,15254,15256,15480],{"className":15255},[14468],[170,15257,15260],{"className":15258},[15259],"col-align-r",[170,15261,15263,15471],{"className":15262},[242,243],[170,15264,15266,15468],{"className":15265},[247],[170,15267,15270,15396,15405,15414,15423,15432,15441,15450,15459],{"className":15268,"style":15269},[251],"height:14.3323em;",[170,15271,15273,15277],{"style":15272},"top:-17.1217em;",[170,15274],{"className":15275,"style":15276},[259],"height:3.8394em;",[170,15278,15280,15283,15286,15289,15292,15295,15298,15301],{"className":15279},[230],[170,15281,14486],{"className":15282},[230],[170,15284,5714],{"className":15285,"style":6157},[230,303],[170,15287,909],{"className":15288},[234],[170,15290,581],{"className":15291,"style":14215},[230,303],[170,15293,909],{"className":15294},[234],[170,15296,13276],{"className":15297,"style":13292},[230,303],[170,15299,923],{"className":15300},[424],[170,15302,15304,15307],{"className":15303},[424],[170,15305,923],{"className":15306},[424],[170,15308,15310],{"className":15309},[307],[170,15311,15313,15388],{"className":15312},[242,243],[170,15314,15316,15385],{"className":15315},[247],[170,15317,15319],{"className":15318,"style":317},[251],[170,15320,15321,15324],{"style":14245},[170,15322],{"className":15323,"style":651},[259],[170,15325,15327],{"className":15326},[264,265,266,267],[170,15328,15330,15333,15336,15376,15379,15382],{"className":15329},[230,267],[170,15331,13276],{"className":15332,"style":13292},[230,303,267],[170,15334,14150],{"className":15335},[615,267],[170,15337,15339,15342],{"className":15338},[230,267],[170,15340,5711],{"className":15341,"style":3292},[230,303,267],[170,15343,15345],{"className":15344},[307],[170,15346,15348,15368],{"className":15347},[242,243],[170,15349,15351,15365],{"className":15350},[247],[170,15352,15354],{"className":15353,"style":317},[251],[170,15355,15356,15359],{"style":14281},[170,15357],{"className":15358,"style":324},[259],[170,15360,15362],{"className":15361},[264,328,329,267],[170,15363,14157],{"className":15364,"style":3317},[230,303,267],[170,15366,338],{"className":15367},[337],[170,15369,15371],{"className":15370},[247],[170,15372,15374],{"className":15373,"style":345},[251],[170,15375],{},[170,15377,909],{"className":15378},[234,267],[170,15380,13276],{"className":15381,"style":13292},[230,303,267],[170,15383,923],{"className":15384},[424,267],[170,15386,338],{"className":15387},[337],[170,15389,15391],{"className":15390},[247],[170,15392,15394],{"className":15393,"style":6209},[251],[170,15395],{},[170,15397,15399,15402],{"style":15398},"top:-14.5217em;",[170,15400],{"className":15401,"style":15276},[259],[170,15403],{"className":15404},[230],[170,15406,15408,15411],{"style":15407},"top:-11.5447em;",[170,15409],{"className":15410,"style":15276},[259],[170,15412],{"className":15413},[230],[170,15415,15417,15420],{"style":15416},"top:-8.5677em;",[170,15418],{"className":15419,"style":15276},[259],[170,15421],{"className":15422},[230],[170,15424,15426,15429],{"style":15425},"top:-5.1894em;",[170,15427],{"className":15428,"style":15276},[259],[170,15430],{"className":15431},[230],[170,15433,15435,15438],{"style":15434},"top:-1.7939em;",[170,15436],{"className":15437,"style":15276},[259],[170,15439],{"className":15440},[230],[170,15442,15444,15447],{"style":15443},"top:1.6126em;",[170,15445],{"className":15446,"style":15276},[259],[170,15448],{"className":15449},[230],[170,15451,15453,15456],{"style":15452},"top:5.0192em;",[170,15454],{"className":15455,"style":15276},[259],[170,15457],{"className":15458},[230],[170,15460,15462,15465],{"style":15461},"top:8.4257em;",[170,15463],{"className":15464,"style":15276},[259],[170,15466],{"className":15467},[230],[170,15469,338],{"className":15470},[337],[170,15472,15474],{"className":15473},[247],[170,15475,15478],{"className":15476,"style":15477},[251],"height:13.8323em;",[170,15479],{},[170,15481,15484],{"className":15482},[15483],"col-align-l",[170,15485,15487,18437],{"className":15486},[242,243],[170,15488,15490,18434],{"className":15489},[247],[170,15491,15493,15633,15764,16062,16347,16757,17039,17507,17972],{"className":15492,"style":15269},[251],[170,15494,15495,15498],{"style":15272},[170,15496],{"className":15497,"style":15276},[259],[170,15499,15501,15504,15507,15510,15513,15516,15519,15569,15572,15575,15578,15581,15584,15624,15627,15630],{"className":15500},[230],[170,15502],{"className":15503},[230],[170,15505],{"className":15506,"style":611},[610],[170,15508,926],{"className":15509},[615],[170,15511],{"className":15512,"style":611},[610],[170,15514,14486],{"className":15515},[230],[170,15517],{"className":15518,"style":1027},[610],[170,15520,15522],{"className":15521},[1062,14344],[170,15523,15525,15560],{"className":15524},[242,243],[170,15526,15528,15557],{"className":15527},[247],[170,15529,15532,15545],{"className":15530,"style":15531},[251],"height:1.05em;",[170,15533,15535,15539],{"style":15534},"top:-1.9em;margin-left:0em;",[170,15536],{"className":15537,"style":15538},[259],"height:3.05em;",[170,15540,15542],{"className":15541},[264,265,266,267],[170,15543,13276],{"className":15544,"style":13292},[230,303,267],[170,15546,15548,15551],{"style":15547},"top:-3.05em;",[170,15549],{"className":15550,"style":15538},[259],[170,15552,15553],{},[170,15554,1937],{"className":15555},[1062,2021,15556],"large-op",[170,15558,338],{"className":15559},[337],[170,15561,15563],{"className":15562},[247],[170,15564,15567],{"className":15565,"style":15566},[251],"height:1.25em;",[170,15568],{},[170,15570],{"className":15571,"style":1027},[610],[170,15573,581],{"className":15574,"style":14215},[230,303],[170,15576,909],{"className":15577},[234],[170,15579,13276],{"className":15580,"style":13292},[230,303],[170,15582,923],{"className":15583},[424],[170,15585,15587,15590],{"className":15586},[230],[170,15588,5711],{"className":15589,"style":3292},[230,303],[170,15591,15593],{"className":15592},[307],[170,15594,15596,15616],{"className":15595},[242,243],[170,15597,15599,15613],{"className":15598},[247],[170,15600,15602],{"className":15601,"style":1359},[251],[170,15603,15604,15607],{"style":7567},[170,15605],{"className":15606,"style":651},[259],[170,15608,15610],{"className":15609},[264,265,266,267],[170,15611,14157],{"className":15612,"style":3317},[230,303,267],[170,15614,338],{"className":15615},[337],[170,15617,15619],{"className":15618},[247],[170,15620,15622],{"className":15621,"style":1381},[251],[170,15623],{},[170,15625,909],{"className":15626},[234],[170,15628,13276],{"className":15629,"style":13292},[230,303],[170,15631,923],{"className":15632},[424],[170,15634,15635,15638],{"style":15398},[170,15636],{"className":15637,"style":15276},[259],[170,15639,15641,15644,15647,15650,15653,15697,15700,15703,15706,15709,15712,15715,15755,15758,15761],{"className":15640},[230],[170,15642],{"className":15643},[230],[170,15645],{"className":15646,"style":611},[610],[170,15648,926],{"className":15649},[615],[170,15651],{"className":15652,"style":611},[610],[170,15654,15656],{"className":15655},[1062,14344],[170,15657,15659,15689],{"className":15658},[242,243],[170,15660,15662,15686],{"className":15661},[247],[170,15663,15665,15676],{"className":15664,"style":15531},[251],[170,15666,15667,15670],{"style":15534},[170,15668],{"className":15669,"style":15538},[259],[170,15671,15673],{"className":15672},[264,265,266,267],[170,15674,13276],{"className":15675,"style":13292},[230,303,267],[170,15677,15678,15681],{"style":15547},[170,15679],{"className":15680,"style":15538},[259],[170,15682,15683],{},[170,15684,1937],{"className":15685},[1062,2021,15556],[170,15687,338],{"className":15688},[337],[170,15690,15692],{"className":15691},[247],[170,15693,15695],{"className":15694,"style":15566},[251],[170,15696],{},[170,15698],{"className":15699,"style":1027},[610],[170,15701,581],{"className":15702,"style":14215},[230,303],[170,15704,909],{"className":15705},[234],[170,15707,13276],{"className":15708,"style":13292},[230,303],[170,15710,923],{"className":15711},[424],[170,15713,14486],{"className":15714},[230],[170,15716,15718,15721],{"className":15717},[230],[170,15719,5711],{"className":15720,"style":3292},[230,303],[170,15722,15724],{"className":15723},[307],[170,15725,15727,15747],{"className":15726},[242,243],[170,15728,15730,15744],{"className":15729},[247],[170,15731,15733],{"className":15732,"style":1359},[251],[170,15734,15735,15738],{"style":7567},[170,15736],{"className":15737,"style":651},[259],[170,15739,15741],{"className":15740},[264,265,266,267],[170,15742,14157],{"className":15743,"style":3317},[230,303,267],[170,15745,338],{"className":15746},[337],[170,15748,15750],{"className":15749},[247],[170,15751,15753],{"className":15752,"style":1381},[251],[170,15754],{},[170,15756,909],{"className":15757},[234],[170,15759,13276],{"className":15760,"style":13292},[230,303],[170,15762,923],{"className":15763},[424],[170,15765,15766,15769],{"style":15407},[170,15767],{"className":15768,"style":15276},[259],[170,15770,15772,15775,15778,15781,15784,15828,15831,15834,15837,15840,15843,15846,15886,15889,15892,15895,15898,15901,15904],{"className":15771},[230],[170,15773],{"className":15774},[230],[170,15776],{"className":15777,"style":611},[610],[170,15779,926],{"className":15780},[615],[170,15782],{"className":15783,"style":611},[610],[170,15785,15787],{"className":15786},[1062,14344],[170,15788,15790,15820],{"className":15789},[242,243],[170,15791,15793,15817],{"className":15792},[247],[170,15794,15796,15807],{"className":15795,"style":15531},[251],[170,15797,15798,15801],{"style":15534},[170,15799],{"className":15800,"style":15538},[259],[170,15802,15804],{"className":15803},[264,265,266,267],[170,15805,13276],{"className":15806,"style":13292},[230,303,267],[170,15808,15809,15812],{"style":15547},[170,15810],{"className":15811,"style":15538},[259],[170,15813,15814],{},[170,15815,1937],{"className":15816},[1062,2021,15556],[170,15818,338],{"className":15819},[337],[170,15821,15823],{"className":15822},[247],[170,15824,15826],{"className":15825,"style":15566},[251],[170,15827],{},[170,15829],{"className":15830,"style":1027},[610],[170,15832,581],{"className":15833,"style":14215},[230,303],[170,15835,909],{"className":15836},[234],[170,15838,13276],{"className":15839,"style":13292},[230,303],[170,15841,923],{"className":15842},[424],[170,15844,14486],{"className":15845},[230],[170,15847,15849,15852],{"className":15848},[230],[170,15850,5711],{"className":15851,"style":3292},[230,303],[170,15853,15855],{"className":15854},[307],[170,15856,15858,15878],{"className":15857},[242,243],[170,15859,15861,15875],{"className":15860},[247],[170,15862,15864],{"className":15863,"style":1359},[251],[170,15865,15866,15869],{"style":7567},[170,15867],{"className":15868,"style":651},[259],[170,15870,15872],{"className":15871},[264,265,266,267],[170,15873,14157],{"className":15874,"style":3317},[230,303,267],[170,15876,338],{"className":15877},[337],[170,15879,15881],{"className":15880},[247],[170,15882,15884],{"className":15883,"style":1381},[251],[170,15885],{},[170,15887,909],{"className":15888},[234],[170,15890,13276],{"className":15891,"style":13292},[230,303],[170,15893,923],{"className":15894},[424],[170,15896],{"className":15897,"style":1983},[610],[170,15899,1927],{"className":15900},[664],[170,15902],{"className":15903,"style":1983},[610],[170,15905,15907,15910,16059],{"className":15906},[230],[170,15908],{"className":15909},[234,235],[170,15911,15913],{"className":15912},[189],[170,15914,15916,16050],{"className":15915},[242,243],[170,15917,15919,16047],{"className":15918},[247],[170,15920,15923,15981,15989],{"className":15921,"style":15922},[251],"height:1.427em;",[170,15924,15926,15929],{"style":15925},"top:-2.314em;",[170,15927],{"className":15928,"style":260},[259],[170,15930,15932,15972,15975,15978],{"className":15931},[230],[170,15933,15935,15938],{"className":15934},[230],[170,15936,5711],{"className":15937,"style":3292},[230,303],[170,15939,15941],{"className":15940},[307],[170,15942,15944,15964],{"className":15943},[242,243],[170,15945,15947,15961],{"className":15946},[247],[170,15948,15950],{"className":15949,"style":1359},[251],[170,15951,15952,15955],{"style":7567},[170,15953],{"className":15954,"style":651},[259],[170,15956,15958],{"className":15957},[264,265,266,267],[170,15959,14157],{"className":15960,"style":3317},[230,303,267],[170,15962,338],{"className":15963},[337],[170,15965,15967],{"className":15966},[247],[170,15968,15970],{"className":15969,"style":1381},[251],[170,15971],{},[170,15973,909],{"className":15974},[234],[170,15976,13276],{"className":15977,"style":13292},[230,303],[170,15979,923],{"className":15980},[424],[170,15982,15983,15986],{"style":385},[170,15984],{"className":15985,"style":260},[259],[170,15987],{"className":15988,"style":393},[392],[170,15990,15992,15995],{"style":15991},"top:-3.677em;",[170,15993],{"className":15994,"style":260},[259],[170,15996,15998,16038,16041,16044],{"className":15997},[230],[170,15999,16001,16004],{"className":16000},[230],[170,16002,5711],{"className":16003,"style":3292},[230,303],[170,16005,16007],{"className":16006},[307],[170,16008,16010,16030],{"className":16009},[242,243],[170,16011,16013,16027],{"className":16012},[247],[170,16014,16016],{"className":16015,"style":1359},[251],[170,16017,16018,16021],{"style":7567},[170,16019],{"className":16020,"style":651},[259],[170,16022,16024],{"className":16023},[264,265,266,267],[170,16025,14157],{"className":16026,"style":3317},[230,303,267],[170,16028,338],{"className":16029},[337],[170,16031,16033],{"className":16032},[247],[170,16034,16036],{"className":16035,"style":1381},[251],[170,16037],{},[170,16039,909],{"className":16040},[234],[170,16042,13276],{"className":16043,"style":13292},[230,303],[170,16045,923],{"className":16046},[424],[170,16048,338],{"className":16049},[337],[170,16051,16053],{"className":16052},[247],[170,16054,16057],{"className":16055,"style":16056},[251],"height:0.936em;",[170,16058],{},[170,16060],{"className":16061},[424,235],[170,16063,16064,16067],{"style":15416},[170,16065],{"className":16066,"style":15276},[259],[170,16068,16070,16073,16076,16079,16082,16126,16129,16169,16172,16175,16178,16181,16184,16187,16190],{"className":16069},[230],[170,16071],{"className":16072},[230],[170,16074],{"className":16075,"style":611},[610],[170,16077,926],{"className":16078},[615],[170,16080],{"className":16081,"style":611},[610],[170,16083,16085],{"className":16084},[1062,14344],[170,16086,16088,16118],{"className":16087},[242,243],[170,16089,16091,16115],{"className":16090},[247],[170,16092,16094,16105],{"className":16093,"style":15531},[251],[170,16095,16096,16099],{"style":15534},[170,16097],{"className":16098,"style":15538},[259],[170,16100,16102],{"className":16101},[264,265,266,267],[170,16103,13276],{"className":16104,"style":13292},[230,303,267],[170,16106,16107,16110],{"style":15547},[170,16108],{"className":16109,"style":15538},[259],[170,16111,16112],{},[170,16113,1937],{"className":16114},[1062,2021,15556],[170,16116,338],{"className":16117},[337],[170,16119,16121],{"className":16120},[247],[170,16122,16124],{"className":16123,"style":15566},[251],[170,16125],{},[170,16127],{"className":16128,"style":1027},[610],[170,16130,16132,16135],{"className":16131},[230],[170,16133,5711],{"className":16134,"style":3292},[230,303],[170,16136,16138],{"className":16137},[307],[170,16139,16141,16161],{"className":16140},[242,243],[170,16142,16144,16158],{"className":16143},[247],[170,16145,16147],{"className":16146,"style":1359},[251],[170,16148,16149,16152],{"style":7567},[170,16150],{"className":16151,"style":651},[259],[170,16153,16155],{"className":16154},[264,265,266,267],[170,16156,14157],{"className":16157,"style":3317},[230,303,267],[170,16159,338],{"className":16160},[337],[170,16162,16164],{"className":16163},[247],[170,16165,16167],{"className":16166,"style":1381},[251],[170,16168],{},[170,16170,909],{"className":16171},[234],[170,16173,13276],{"className":16174,"style":13292},[230,303],[170,16176,923],{"className":16177},[424],[170,16179,581],{"className":16180,"style":14215},[230,303],[170,16182,909],{"className":16183},[234],[170,16185,13276],{"className":16186,"style":13292},[230,303],[170,16188,923],{"className":16189},[424],[170,16191,16193,16196,16344],{"className":16192},[230],[170,16194],{"className":16195},[234,235],[170,16197,16199],{"className":16198},[189],[170,16200,16202,16336],{"className":16201},[242,243],[170,16203,16205,16333],{"className":16204},[247],[170,16206,16208,16265,16273],{"className":16207,"style":15922},[251],[170,16209,16210,16213],{"style":15925},[170,16211],{"className":16212,"style":260},[259],[170,16214,16216,16256,16259,16262],{"className":16215},[230],[170,16217,16219,16222],{"className":16218},[230],[170,16220,5711],{"className":16221,"style":3292},[230,303],[170,16223,16225],{"className":16224},[307],[170,16226,16228,16248],{"className":16227},[242,243],[170,16229,16231,16245],{"className":16230},[247],[170,16232,16234],{"className":16233,"style":1359},[251],[170,16235,16236,16239],{"style":7567},[170,16237],{"className":16238,"style":651},[259],[170,16240,16242],{"className":16241},[264,265,266,267],[170,16243,14157],{"className":16244,"style":3317},[230,303,267],[170,16246,338],{"className":16247},[337],[170,16249,16251],{"className":16250},[247],[170,16252,16254],{"className":16253,"style":1381},[251],[170,16255],{},[170,16257,909],{"className":16258},[234],[170,16260,13276],{"className":16261,"style":13292},[230,303],[170,16263,923],{"className":16264},[424],[170,16266,16267,16270],{"style":385},[170,16268],{"className":16269,"style":260},[259],[170,16271],{"className":16272,"style":393},[392],[170,16274,16275,16278],{"style":15991},[170,16276],{"className":16277,"style":260},[259],[170,16279,16281,16284,16324,16327,16330],{"className":16280},[230],[170,16282,14486],{"className":16283},[230],[170,16285,16287,16290],{"className":16286},[230],[170,16288,5711],{"className":16289,"style":3292},[230,303],[170,16291,16293],{"className":16292},[307],[170,16294,16296,16316],{"className":16295},[242,243],[170,16297,16299,16313],{"className":16298},[247],[170,16300,16302],{"className":16301,"style":1359},[251],[170,16303,16304,16307],{"style":7567},[170,16305],{"className":16306,"style":651},[259],[170,16308,16310],{"className":16309},[264,265,266,267],[170,16311,14157],{"className":16312,"style":3317},[230,303,267],[170,16314,338],{"className":16315},[337],[170,16317,16319],{"className":16318},[247],[170,16320,16322],{"className":16321,"style":1381},[251],[170,16323],{},[170,16325,909],{"className":16326},[234],[170,16328,13276],{"className":16329,"style":13292},[230,303],[170,16331,923],{"className":16332},[424],[170,16334,338],{"className":16335},[337],[170,16337,16339],{"className":16338},[247],[170,16340,16342],{"className":16341,"style":16056},[251],[170,16343],{},[170,16345],{"className":16346},[424,235],[170,16348,16349,16352],{"style":15425},[170,16350],{"className":16351,"style":15276},[259],[170,16353,16355,16358,16361,16364,16367,16431,16434,16502,16505,16508,16511,16542,16545],{"className":16354},[230],[170,16356],{"className":16357},[230],[170,16359],{"className":16360,"style":611},[610],[170,16362,14781],{"className":16363},[615],[170,16365],{"className":16366,"style":611},[610],[170,16368,16370,16373,16428],{"className":16369},[230],[170,16371],{"className":16372},[234,235],[170,16374,16376],{"className":16375},[189],[170,16377,16379,16419],{"className":16378},[242,243],[170,16380,16382,16416],{"className":16381},[247],[170,16383,16386,16397,16405],{"className":16384,"style":16385},[251],"height:1.3214em;",[170,16387,16388,16391],{"style":15925},[170,16389],{"className":16390,"style":260},[259],[170,16392,16394],{"className":16393},[230],[170,16395,7429],{"className":16396,"style":7504},[230,303],[170,16398,16399,16402],{"style":385},[170,16400],{"className":16401,"style":260},[259],[170,16403],{"className":16404,"style":393},[392],[170,16406,16407,16410],{"style":15991},[170,16408],{"className":16409,"style":260},[259],[170,16411,16413],{"className":16412},[230],[170,16414,194],{"className":16415},[230],[170,16417,338],{"className":16418},[337],[170,16420,16422],{"className":16421},[247],[170,16423,16426],{"className":16424,"style":16425},[251],"height:0.686em;",[170,16427],{},[170,16429],{"className":16430},[424,235],[170,16432],{"className":16433,"style":1027},[610],[170,16435,16437],{"className":16436},[1062,14344],[170,16438,16440,16493],{"className":16439},[242,243],[170,16441,16443,16490],{"className":16442},[247],[170,16444,16447,16468,16478],{"className":16445,"style":16446},[251],"height:1.8283em;",[170,16448,16450,16453],{"style":16449},"top:-1.8829em;margin-left:0em;",[170,16451],{"className":16452,"style":15538},[259],[170,16454,16456],{"className":16455},[264,265,266,267],[170,16457,16459,16462,16465],{"className":16458},[230,267],[170,16460,446],{"className":16461},[230,303,267],[170,16463,926],{"className":16464},[615,267],[170,16466,194],{"className":16467},[230,267],[170,16469,16470,16473],{"style":15547},[170,16471],{"className":16472,"style":15538},[259],[170,16474,16475],{},[170,16476,1937],{"className":16477},[1062,2021,15556],[170,16479,16481,16484],{"style":16480},"top:-4.3em;margin-left:0em;",[170,16482],{"className":16483,"style":15538},[259],[170,16485,16487],{"className":16486},[264,265,266,267],[170,16488,7429],{"className":16489,"style":7504},[230,303,267],[170,16491,338],{"className":16492},[337],[170,16494,16496],{"className":16495},[247],[170,16497,16500],{"className":16498,"style":16499},[251],"height:1.2671em;",[170,16501],{},[170,16503],{"className":16504,"style":1027},[610],[170,16506,581],{"className":16507,"style":14215},[230,303],[170,16509,909],{"className":16510},[234],[170,16512,16514,16517],{"className":16513},[230],[170,16515,13276],{"className":16516,"style":13292},[230,303],[170,16518,16520],{"className":16519},[307],[170,16521,16523],{"className":16522},[242],[170,16524,16526],{"className":16525},[247],[170,16527,16530],{"className":16528,"style":16529},[251],"height:0.7144em;",[170,16531,16533,16536],{"style":16532},"top:-3.113em;margin-right:0.05em;",[170,16534],{"className":16535,"style":651},[259],[170,16537,16539],{"className":16538},[264,265,266,267],[170,16540,446],{"className":16541},[230,303,267],[170,16543,923],{"className":16544},[424],[170,16546,16548,16551,16754],{"className":16547},[230],[170,16549],{"className":16550},[234,235],[170,16552,16554],{"className":16553},[189],[170,16555,16557,16746],{"className":16556},[242,243],[170,16558,16560,16743],{"className":16559},[247],[170,16561,16563,16648,16656],{"className":16562,"style":15922},[251],[170,16564,16565,16568],{"style":15925},[170,16566],{"className":16567,"style":260},[259],[170,16569,16571,16611,16614,16645],{"className":16570},[230],[170,16572,16574,16577],{"className":16573},[230],[170,16575,5711],{"className":16576,"style":3292},[230,303],[170,16578,16580],{"className":16579},[307],[170,16581,16583,16603],{"className":16582},[242,243],[170,16584,16586,16600],{"className":16585},[247],[170,16587,16589],{"className":16588,"style":1359},[251],[170,16590,16591,16594],{"style":7567},[170,16592],{"className":16593,"style":651},[259],[170,16595,16597],{"className":16596},[264,265,266,267],[170,16598,14157],{"className":16599,"style":3317},[230,303,267],[170,16601,338],{"className":16602},[337],[170,16604,16606],{"className":16605},[247],[170,16607,16609],{"className":16608,"style":1381},[251],[170,16610],{},[170,16612,909],{"className":16613},[234],[170,16615,16617,16620],{"className":16616},[230],[170,16618,13276],{"className":16619,"style":13292},[230,303],[170,16621,16623],{"className":16622},[307],[170,16624,16626],{"className":16625},[242],[170,16627,16629],{"className":16628},[247],[170,16630,16633],{"className":16631,"style":16632},[251],"height:0.5904em;",[170,16634,16636,16639],{"style":16635},"top:-2.989em;margin-right:0.05em;",[170,16637],{"className":16638,"style":651},[259],[170,16640,16642],{"className":16641},[264,265,266,267],[170,16643,446],{"className":16644},[230,303,267],[170,16646,923],{"className":16647},[424],[170,16649,16650,16653],{"style":385},[170,16651],{"className":16652,"style":260},[259],[170,16654],{"className":16655,"style":393},[392],[170,16657,16658,16661],{"style":15991},[170,16659],{"className":16660,"style":260},[259],[170,16662,16664,16667,16707,16710,16740],{"className":16663},[230],[170,16665,14486],{"className":16666},[230],[170,16668,16670,16673],{"className":16669},[230],[170,16671,14],{"className":16672},[230,303],[170,16674,16676],{"className":16675},[307],[170,16677,16679,16699],{"className":16678},[242,243],[170,16680,16682,16696],{"className":16681},[247],[170,16683,16685],{"className":16684,"style":1359},[251],[170,16686,16687,16690],{"style":1362},[170,16688],{"className":16689,"style":651},[259],[170,16691,16693],{"className":16692},[264,265,266,267],[170,16694,14157],{"className":16695,"style":3317},[230,303,267],[170,16697,338],{"className":16698},[337],[170,16700,16702],{"className":16701},[247],[170,16703,16705],{"className":16704,"style":1381},[251],[170,16706],{},[170,16708,909],{"className":16709},[234],[170,16711,16713,16716],{"className":16712},[230],[170,16714,13276],{"className":16715,"style":13292},[230,303],[170,16717,16719],{"className":16718},[307],[170,16720,16722],{"className":16721},[242],[170,16723,16725],{"className":16724},[247],[170,16726,16729],{"className":16727,"style":16728},[251],"height:0.6644em;",[170,16730,16731,16734],{"style":647},[170,16732],{"className":16733,"style":651},[259],[170,16735,16737],{"className":16736},[264,265,266,267],[170,16738,446],{"className":16739},[230,303,267],[170,16741,923],{"className":16742},[424],[170,16744,338],{"className":16745},[337],[170,16747,16749],{"className":16748},[247],[170,16750,16752],{"className":16751,"style":16056},[251],[170,16753],{},[170,16755],{"className":16756},[424,235],[170,16758,16759,16762],{"style":15434},[170,16760],{"className":16761,"style":15276},[259],[170,16763,16765,16768,16771,16774,16777,16839,16842,16906,16909,16912,16915,16944,16947,16950,16953,16961,16964,17004,17007,17036],{"className":16764},[230],[170,16766],{"className":16767},[230],[170,16769],{"className":16770,"style":611},[610],[170,16772,926],{"className":16773},[615],[170,16775],{"className":16776,"style":611},[610],[170,16778,16780,16783,16836],{"className":16779},[230],[170,16781],{"className":16782},[234,235],[170,16784,16786],{"className":16785},[189],[170,16787,16789,16828],{"className":16788},[242,243],[170,16790,16792,16825],{"className":16791},[247],[170,16793,16795,16806,16814],{"className":16794,"style":16385},[251],[170,16796,16797,16800],{"style":15925},[170,16798],{"className":16799,"style":260},[259],[170,16801,16803],{"className":16802},[230],[170,16804,7429],{"className":16805,"style":7504},[230,303],[170,16807,16808,16811],{"style":385},[170,16809],{"className":16810,"style":260},[259],[170,16812],{"className":16813,"style":393},[392],[170,16815,16816,16819],{"style":15991},[170,16817],{"className":16818,"style":260},[259],[170,16820,16822],{"className":16821},[230],[170,16823,194],{"className":16824},[230],[170,16826,338],{"className":16827},[337],[170,16829,16831],{"className":16830},[247],[170,16832,16834],{"className":16833,"style":16425},[251],[170,16835],{},[170,16837],{"className":16838},[424,235],[170,16840],{"className":16841,"style":1027},[610],[170,16843,16845],{"className":16844},[1062,14344],[170,16846,16848,16898],{"className":16847},[242,243],[170,16849,16851,16895],{"className":16850},[247],[170,16852,16854,16874,16884],{"className":16853,"style":16446},[251],[170,16855,16856,16859],{"style":16449},[170,16857],{"className":16858,"style":15538},[259],[170,16860,16862],{"className":16861},[264,265,266,267],[170,16863,16865,16868,16871],{"className":16864},[230,267],[170,16866,446],{"className":16867},[230,303,267],[170,16869,926],{"className":16870},[615,267],[170,16872,194],{"className":16873},[230,267],[170,16875,16876,16879],{"style":15547},[170,16877],{"className":16878,"style":15538},[259],[170,16880,16881],{},[170,16882,1937],{"className":16883},[1062,2021,15556],[170,16885,16886,16889],{"style":16480},[170,16887],{"className":16888,"style":15538},[259],[170,16890,16892],{"className":16891},[264,265,266,267],[170,16893,7429],{"className":16894,"style":7504},[230,303,267],[170,16896,338],{"className":16897},[337],[170,16899,16901],{"className":16900},[247],[170,16902,16904],{"className":16903,"style":16499},[251],[170,16905],{},[170,16907],{"className":16908,"style":1027},[610],[170,16910,581],{"className":16911,"style":14215},[230,303],[170,16913,909],{"className":16914},[234],[170,16916,16918,16921],{"className":16917},[230],[170,16919,13276],{"className":16920,"style":13292},[230,303],[170,16922,16924],{"className":16923},[307],[170,16925,16927],{"className":16926},[242],[170,16928,16930],{"className":16929},[247],[170,16931,16933],{"className":16932,"style":16529},[251],[170,16934,16935,16938],{"style":16532},[170,16936],{"className":16937,"style":651},[259],[170,16939,16941],{"className":16940},[264,265,266,267],[170,16942,446],{"className":16943},[230,303,267],[170,16945,923],{"className":16946},[424],[170,16948,14486],{"className":16949},[230],[170,16951],{"className":16952,"style":1027},[610],[170,16954,16956,16957],{"className":16955},[1062],"lo",[170,16958,16960],{"style":16959},"margin-right:0.01389em;","g",[170,16962],{"className":16963,"style":1027},[610],[170,16965,16967,16970],{"className":16966},[230],[170,16968,5711],{"className":16969,"style":3292},[230,303],[170,16971,16973],{"className":16972},[307],[170,16974,16976,16996],{"className":16975},[242,243],[170,16977,16979,16993],{"className":16978},[247],[170,16980,16982],{"className":16981,"style":1359},[251],[170,16983,16984,16987],{"style":7567},[170,16985],{"className":16986,"style":651},[259],[170,16988,16990],{"className":16989},[264,265,266,267],[170,16991,14157],{"className":16992,"style":3317},[230,303,267],[170,16994,338],{"className":16995},[337],[170,16997,16999],{"className":16998},[247],[170,17000,17002],{"className":17001,"style":1381},[251],[170,17003],{},[170,17005,909],{"className":17006},[234],[170,17008,17010,17013],{"className":17009},[230],[170,17011,13276],{"className":17012,"style":13292},[230,303],[170,17014,17016],{"className":17015},[307],[170,17017,17019],{"className":17018},[242],[170,17020,17022],{"className":17021},[247],[170,17023,17025],{"className":17024,"style":16529},[251],[170,17026,17027,17030],{"style":16532},[170,17028],{"className":17029,"style":651},[259],[170,17031,17033],{"className":17032},[264,265,266,267],[170,17034,446],{"className":17035},[230,303,267],[170,17037,923],{"className":17038},[424],[170,17040,17041,17044],{"style":15443},[170,17042],{"className":17043,"style":15276},[259],[170,17045,17047,17050,17053,17056,17059,17121,17124,17188,17191,17194,17197,17226,17229,17232,17235,17240,17243,17350,17353,17393,17396,17450,17453,17504],{"className":17046},[230],[170,17048],{"className":17049},[230],[170,17051],{"className":17052,"style":611},[610],[170,17054,926],{"className":17055},[615],[170,17057],{"className":17058,"style":611},[610],[170,17060,17062,17065,17118],{"className":17061},[230],[170,17063],{"className":17064},[234,235],[170,17066,17068],{"className":17067},[189],[170,17069,17071,17110],{"className":17070},[242,243],[170,17072,17074,17107],{"className":17073},[247],[170,17075,17077,17088,17096],{"className":17076,"style":16385},[251],[170,17078,17079,17082],{"style":15925},[170,17080],{"className":17081,"style":260},[259],[170,17083,17085],{"className":17084},[230],[170,17086,7429],{"className":17087,"style":7504},[230,303],[170,17089,17090,17093],{"style":385},[170,17091],{"className":17092,"style":260},[259],[170,17094],{"className":17095,"style":393},[392],[170,17097,17098,17101],{"style":15991},[170,17099],{"className":17100,"style":260},[259],[170,17102,17104],{"className":17103},[230],[170,17105,194],{"className":17106},[230],[170,17108,338],{"className":17109},[337],[170,17111,17113],{"className":17112},[247],[170,17114,17116],{"className":17115,"style":16425},[251],[170,17117],{},[170,17119],{"className":17120},[424,235],[170,17122],{"className":17123,"style":1027},[610],[170,17125,17127],{"className":17126},[1062,14344],[170,17128,17130,17180],{"className":17129},[242,243],[170,17131,17133,17177],{"className":17132},[247],[170,17134,17136,17156,17166],{"className":17135,"style":16446},[251],[170,17137,17138,17141],{"style":16449},[170,17139],{"className":17140,"style":15538},[259],[170,17142,17144],{"className":17143},[264,265,266,267],[170,17145,17147,17150,17153],{"className":17146},[230,267],[170,17148,446],{"className":17149},[230,303,267],[170,17151,926],{"className":17152},[615,267],[170,17154,194],{"className":17155},[230,267],[170,17157,17158,17161],{"style":15547},[170,17159],{"className":17160,"style":15538},[259],[170,17162,17163],{},[170,17164,1937],{"className":17165},[1062,2021,15556],[170,17167,17168,17171],{"style":16480},[170,17169],{"className":17170,"style":15538},[259],[170,17172,17174],{"className":17173},[264,265,266,267],[170,17175,7429],{"className":17176,"style":7504},[230,303,267],[170,17178,338],{"className":17179},[337],[170,17181,17183],{"className":17182},[247],[170,17184,17186],{"className":17185,"style":16499},[251],[170,17187],{},[170,17189],{"className":17190,"style":1027},[610],[170,17192,581],{"className":17193,"style":14215},[230,303],[170,17195,909],{"className":17196},[234],[170,17198,17200,17203],{"className":17199},[230],[170,17201,13276],{"className":17202,"style":13292},[230,303],[170,17204,17206],{"className":17205},[307],[170,17207,17209],{"className":17208},[242],[170,17210,17212],{"className":17211},[247],[170,17213,17215],{"className":17214,"style":16529},[251],[170,17216,17217,17220],{"style":16532},[170,17218],{"className":17219,"style":651},[259],[170,17221,17223],{"className":17222},[264,265,266,267],[170,17224,446],{"className":17225},[230,303,267],[170,17227,923],{"className":17228},[424],[170,17230,14486],{"className":17231},[230],[170,17233],{"className":17234,"style":1027},[610],[170,17236,16956,17238],{"className":17237},[1062],[170,17239,16960],{"style":16959},[170,17241],{"className":17242,"style":1027},[610],[170,17244,17246],{"className":17245},[1062,14344],[170,17247,17249,17342],{"className":17248},[242,243],[170,17250,17252,17339],{"className":17251},[247],[170,17253,17256,17276,17286],{"className":17254,"style":17255},[251],"height:1.8394em;",[170,17257,17258,17261],{"style":16449},[170,17259],{"className":17260,"style":15538},[259],[170,17262,17264],{"className":17263},[264,265,266,267],[170,17265,17267,17270,17273],{"className":17266},[230,267],[170,17268,888],{"className":17269},[230,303,267],[170,17271,926],{"className":17272},[615,267],[170,17274,194],{"className":17275},[230,267],[170,17277,17278,17281],{"style":15547},[170,17279],{"className":17280,"style":15538},[259],[170,17282,17283],{},[170,17284,14990],{"className":17285},[1062,2021,15556],[170,17287,17289,17292],{"style":17288},"top:-4.3111em;margin-left:0em;",[170,17290],{"className":17291,"style":15538},[259],[170,17293,17295],{"className":17294},[264,265,266,267],[170,17296,17298],{"className":17297},[230,267],[170,17299,17301,17304],{"className":17300},[230,267],[170,17302,13348],{"className":17303,"style":3292},[230,303,267],[170,17305,17307],{"className":17306},[307],[170,17308,17310,17331],{"className":17309},[242,243],[170,17311,17313,17328],{"className":17312},[247],[170,17314,17316],{"className":17315,"style":4409},[251],[170,17317,17319,17322],{"style":17318},"top:-2.357em;margin-left:-0.1389em;margin-right:0.0714em;",[170,17320],{"className":17321,"style":324},[259],[170,17323,17325],{"className":17324},[264,328,329,267],[170,17326,446],{"className":17327},[230,303,267],[170,17329,338],{"className":17330},[337],[170,17332,17334],{"className":17333},[247],[170,17335,17337],{"className":17336,"style":4431},[251],[170,17338],{},[170,17340,338],{"className":17341},[337],[170,17343,17345],{"className":17344},[247],[170,17346,17348],{"className":17347,"style":16499},[251],[170,17349],{},[170,17351],{"className":17352,"style":1027},[610],[170,17354,17356,17359],{"className":17355},[230],[170,17357,5711],{"className":17358,"style":3292},[230,303],[170,17360,17362],{"className":17361},[307],[170,17363,17365,17385],{"className":17364},[242,243],[170,17366,17368,17382],{"className":17367},[247],[170,17369,17371],{"className":17370,"style":1359},[251],[170,17372,17373,17376],{"style":7567},[170,17374],{"className":17375,"style":651},[259],[170,17377,17379],{"className":17378},[264,265,266,267],[170,17380,14157],{"className":17381,"style":3317},[230,303,267],[170,17383,338],{"className":17384},[337],[170,17386,17388],{"className":17387},[247],[170,17389,17391],{"className":17390,"style":1381},[251],[170,17392],{},[170,17394,909],{"className":17395},[234],[170,17397,17399,17402],{"className":17398},[230],[170,17400,134],{"className":17401},[230,303],[170,17403,17405],{"className":17404},[307],[170,17406,17408,17441],{"className":17407},[242,243],[170,17409,17411,17438],{"className":17410},[247],[170,17412,17415,17427],{"className":17413,"style":17414},[251],"height:0.8436em;",[170,17416,17418,17421],{"style":17417},"top:-2.453em;margin-left:0em;margin-right:0.05em;",[170,17419],{"className":17420,"style":651},[259],[170,17422,17424],{"className":17423},[264,265,266,267],[170,17425,446],{"className":17426},[230,303,267],[170,17428,17429,17432],{"style":16532},[170,17430],{"className":17431,"style":651},[259],[170,17433,17435],{"className":17434},[264,265,266,267],[170,17436,888],{"className":17437},[230,303,267],[170,17439,338],{"className":17440},[337],[170,17442,17444],{"className":17443},[247],[170,17445,17448],{"className":17446,"style":17447},[251],"height:0.247em;",[170,17449],{},[170,17451,13047],{"className":17452},[230],[170,17454,17456,17459],{"className":17455},[230],[170,17457,12908],{"className":17458},[230,303],[170,17460,17462],{"className":17461},[307],[170,17463,17465,17496],{"className":17464},[242,243],[170,17466,17468,17493],{"className":17467},[247],[170,17469,17471,17482],{"className":17470,"style":17414},[251],[170,17472,17473,17476],{"style":17417},[170,17474],{"className":17475,"style":651},[259],[170,17477,17479],{"className":17478},[264,265,266,267],[170,17480,446],{"className":17481},[230,303,267],[170,17483,17484,17487],{"style":16532},[170,17485],{"className":17486,"style":651},[259],[170,17488,17490],{"className":17489},[264,265,266,267],[170,17491,888],{"className":17492},[230,303,267],[170,17494,338],{"className":17495},[337],[170,17497,17499],{"className":17498},[247],[170,17500,17502],{"className":17501,"style":17447},[251],[170,17503],{},[170,17505,923],{"className":17506},[424],[170,17508,17509,17512],{"style":15452},[170,17510],{"className":17511,"style":15276},[259],[170,17513,17515,17518,17521,17524,17527,17589,17592,17656,17659,17662,17665,17694,17697,17700,17804,17807,17810,17813,17818,17821,17861,17864,17915,17918,17969],{"className":17514},[230],[170,17516],{"className":17517},[230],[170,17519],{"className":17520,"style":611},[610],[170,17522,926],{"className":17523},[615],[170,17525],{"className":17526,"style":611},[610],[170,17528,17530,17533,17586],{"className":17529},[230],[170,17531],{"className":17532},[234,235],[170,17534,17536],{"className":17535},[189],[170,17537,17539,17578],{"className":17538},[242,243],[170,17540,17542,17575],{"className":17541},[247],[170,17543,17545,17556,17564],{"className":17544,"style":16385},[251],[170,17546,17547,17550],{"style":15925},[170,17548],{"className":17549,"style":260},[259],[170,17551,17553],{"className":17552},[230],[170,17554,7429],{"className":17555,"style":7504},[230,303],[170,17557,17558,17561],{"style":385},[170,17559],{"className":17560,"style":260},[259],[170,17562],{"className":17563,"style":393},[392],[170,17565,17566,17569],{"style":15991},[170,17567],{"className":17568,"style":260},[259],[170,17570,17572],{"className":17571},[230],[170,17573,194],{"className":17574},[230],[170,17576,338],{"className":17577},[337],[170,17579,17581],{"className":17580},[247],[170,17582,17584],{"className":17583,"style":16425},[251],[170,17585],{},[170,17587],{"className":17588},[424,235],[170,17590],{"className":17591,"style":1027},[610],[170,17593,17595],{"className":17594},[1062,14344],[170,17596,17598,17648],{"className":17597},[242,243],[170,17599,17601,17645],{"className":17600},[247],[170,17602,17604,17624,17634],{"className":17603,"style":16446},[251],[170,17605,17606,17609],{"style":16449},[170,17607],{"className":17608,"style":15538},[259],[170,17610,17612],{"className":17611},[264,265,266,267],[170,17613,17615,17618,17621],{"className":17614},[230,267],[170,17616,446],{"className":17617},[230,303,267],[170,17619,926],{"className":17620},[615,267],[170,17622,194],{"className":17623},[230,267],[170,17625,17626,17629],{"style":15547},[170,17627],{"className":17628,"style":15538},[259],[170,17630,17631],{},[170,17632,1937],{"className":17633},[1062,2021,15556],[170,17635,17636,17639],{"style":16480},[170,17637],{"className":17638,"style":15538},[259],[170,17640,17642],{"className":17641},[264,265,266,267],[170,17643,7429],{"className":17644,"style":7504},[230,303,267],[170,17646,338],{"className":17647},[337],[170,17649,17651],{"className":17650},[247],[170,17652,17654],{"className":17653,"style":16499},[251],[170,17655],{},[170,17657],{"className":17658,"style":1027},[610],[170,17660,581],{"className":17661,"style":14215},[230,303],[170,17663,909],{"className":17664},[234],[170,17666,17668,17671],{"className":17667},[230],[170,17669,13276],{"className":17670,"style":13292},[230,303],[170,17672,17674],{"className":17673},[307],[170,17675,17677],{"className":17676},[242],[170,17678,17680],{"className":17679},[247],[170,17681,17683],{"className":17682,"style":16529},[251],[170,17684,17685,17688],{"style":16532},[170,17686],{"className":17687,"style":651},[259],[170,17689,17691],{"className":17690},[264,265,266,267],[170,17692,446],{"className":17693},[230,303,267],[170,17695,923],{"className":17696},[424],[170,17698],{"className":17699,"style":1027},[610],[170,17701,17703],{"className":17702},[1062,14344],[170,17704,17706,17796],{"className":17705},[242,243],[170,17707,17709,17793],{"className":17708},[247],[170,17710,17712,17732,17742],{"className":17711,"style":17255},[251],[170,17713,17714,17717],{"style":16449},[170,17715],{"className":17716,"style":15538},[259],[170,17718,17720],{"className":17719},[264,265,266,267],[170,17721,17723,17726,17729],{"className":17722},[230,267],[170,17724,888],{"className":17725},[230,303,267],[170,17727,926],{"className":17728},[615,267],[170,17730,194],{"className":17731},[230,267],[170,17733,17734,17737],{"style":15547},[170,17735],{"className":17736,"style":15538},[259],[170,17738,17739],{},[170,17740,1937],{"className":17741},[1062,2021,15556],[170,17743,17744,17747],{"style":17288},[170,17745],{"className":17746,"style":15538},[259],[170,17748,17750],{"className":17749},[264,265,266,267],[170,17751,17753],{"className":17752},[230,267],[170,17754,17756,17759],{"className":17755},[230,267],[170,17757,13348],{"className":17758,"style":3292},[230,303,267],[170,17760,17762],{"className":17761},[307],[170,17763,17765,17785],{"className":17764},[242,243],[170,17766,17768,17782],{"className":17767},[247],[170,17769,17771],{"className":17770,"style":4409},[251],[170,17772,17773,17776],{"style":17318},[170,17774],{"className":17775,"style":324},[259],[170,17777,17779],{"className":17778},[264,328,329,267],[170,17780,446],{"className":17781},[230,303,267],[170,17783,338],{"className":17784},[337],[170,17786,17788],{"className":17787},[247],[170,17789,17791],{"className":17790,"style":4431},[251],[170,17792],{},[170,17794,338],{"className":17795},[337],[170,17797,17799],{"className":17798},[247],[170,17800,17802],{"className":17801,"style":16499},[251],[170,17803],{},[170,17805],{"className":17806,"style":1027},[610],[170,17808,14486],{"className":17809},[230],[170,17811],{"className":17812,"style":1027},[610],[170,17814,16956,17816],{"className":17815},[1062],[170,17817,16960],{"style":16959},[170,17819],{"className":17820,"style":1027},[610],[170,17822,17824,17827],{"className":17823},[230],[170,17825,5711],{"className":17826,"style":3292},[230,303],[170,17828,17830],{"className":17829},[307],[170,17831,17833,17853],{"className":17832},[242,243],[170,17834,17836,17850],{"className":17835},[247],[170,17837,17839],{"className":17838,"style":1359},[251],[170,17840,17841,17844],{"style":7567},[170,17842],{"className":17843,"style":651},[259],[170,17845,17847],{"className":17846},[264,265,266,267],[170,17848,14157],{"className":17849,"style":3317},[230,303,267],[170,17851,338],{"className":17852},[337],[170,17854,17856],{"className":17855},[247],[170,17857,17859],{"className":17858,"style":1381},[251],[170,17860],{},[170,17862,909],{"className":17863},[234],[170,17865,17867,17870],{"className":17866},[230],[170,17868,134],{"className":17869},[230,303],[170,17871,17873],{"className":17872},[307],[170,17874,17876,17907],{"className":17875},[242,243],[170,17877,17879,17904],{"className":17878},[247],[170,17880,17882,17893],{"className":17881,"style":17414},[251],[170,17883,17884,17887],{"style":17417},[170,17885],{"className":17886,"style":651},[259],[170,17888,17890],{"className":17889},[264,265,266,267],[170,17891,446],{"className":17892},[230,303,267],[170,17894,17895,17898],{"style":16532},[170,17896],{"className":17897,"style":651},[259],[170,17899,17901],{"className":17900},[264,265,266,267],[170,17902,888],{"className":17903},[230,303,267],[170,17905,338],{"className":17906},[337],[170,17908,17910],{"className":17909},[247],[170,17911,17913],{"className":17912,"style":17447},[251],[170,17914],{},[170,17916,13047],{"className":17917},[230],[170,17919,17921,17924],{"className":17920},[230],[170,17922,12908],{"className":17923},[230,303],[170,17925,17927],{"className":17926},[307],[170,17928,17930,17961],{"className":17929},[242,243],[170,17931,17933,17958],{"className":17932},[247],[170,17934,17936,17947],{"className":17935,"style":17414},[251],[170,17937,17938,17941],{"style":17417},[170,17939],{"className":17940,"style":651},[259],[170,17942,17944],{"className":17943},[264,265,266,267],[170,17945,446],{"className":17946},[230,303,267],[170,17948,17949,17952],{"style":16532},[170,17950],{"className":17951,"style":651},[259],[170,17953,17955],{"className":17954},[264,265,266,267],[170,17956,888],{"className":17957},[230,303,267],[170,17959,338],{"className":17960},[337],[170,17962,17964],{"className":17963},[247],[170,17965,17967],{"className":17966,"style":17447},[251],[170,17968],{},[170,17970,923],{"className":17971},[424],[170,17973,17974,17977],{"style":15461},[170,17975],{"className":17976,"style":15276},[259],[170,17978,17980,17983,17986,17989,17992,18054,18057,18121,18124,18228,18231,18234,18237,18266,18269,18272,18275,18280,18283,18323,18326,18377,18380,18431],{"className":17979},[230],[170,17981],{"className":17982},[230],[170,17984],{"className":17985,"style":611},[610],[170,17987,926],{"className":17988},[615],[170,17990],{"className":17991,"style":611},[610],[170,17993,17995,17998,18051],{"className":17994},[230],[170,17996],{"className":17997},[234,235],[170,17999,18001],{"className":18000},[189],[170,18002,18004,18043],{"className":18003},[242,243],[170,18005,18007,18040],{"className":18006},[247],[170,18008,18010,18021,18029],{"className":18009,"style":16385},[251],[170,18011,18012,18015],{"style":15925},[170,18013],{"className":18014,"style":260},[259],[170,18016,18018],{"className":18017},[230],[170,18019,7429],{"className":18020,"style":7504},[230,303],[170,18022,18023,18026],{"style":385},[170,18024],{"className":18025,"style":260},[259],[170,18027],{"className":18028,"style":393},[392],[170,18030,18031,18034],{"style":15991},[170,18032],{"className":18033,"style":260},[259],[170,18035,18037],{"className":18036},[230],[170,18038,194],{"className":18039},[230],[170,18041,338],{"className":18042},[337],[170,18044,18046],{"className":18045},[247],[170,18047,18049],{"className":18048,"style":16425},[251],[170,18050],{},[170,18052],{"className":18053},[424,235],[170,18055],{"className":18056,"style":1027},[610],[170,18058,18060],{"className":18059},[1062,14344],[170,18061,18063,18113],{"className":18062},[242,243],[170,18064,18066,18110],{"className":18065},[247],[170,18067,18069,18089,18099],{"className":18068,"style":16446},[251],[170,18070,18071,18074],{"style":16449},[170,18072],{"className":18073,"style":15538},[259],[170,18075,18077],{"className":18076},[264,265,266,267],[170,18078,18080,18083,18086],{"className":18079},[230,267],[170,18081,446],{"className":18082},[230,303,267],[170,18084,926],{"className":18085},[615,267],[170,18087,194],{"className":18088},[230,267],[170,18090,18091,18094],{"style":15547},[170,18092],{"className":18093,"style":15538},[259],[170,18095,18096],{},[170,18097,1937],{"className":18098},[1062,2021,15556],[170,18100,18101,18104],{"style":16480},[170,18102],{"className":18103,"style":15538},[259],[170,18105,18107],{"className":18106},[264,265,266,267],[170,18108,7429],{"className":18109,"style":7504},[230,303,267],[170,18111,338],{"className":18112},[337],[170,18114,18116],{"className":18115},[247],[170,18117,18119],{"className":18118,"style":16499},[251],[170,18120],{},[170,18122],{"className":18123,"style":1027},[610],[170,18125,18127],{"className":18126},[1062,14344],[170,18128,18130,18220],{"className":18129},[242,243],[170,18131,18133,18217],{"className":18132},[247],[170,18134,18136,18156,18166],{"className":18135,"style":17255},[251],[170,18137,18138,18141],{"style":16449},[170,18139],{"className":18140,"style":15538},[259],[170,18142,18144],{"className":18143},[264,265,266,267],[170,18145,18147,18150,18153],{"className":18146},[230,267],[170,18148,888],{"className":18149},[230,303,267],[170,18151,926],{"className":18152},[615,267],[170,18154,194],{"className":18155},[230,267],[170,18157,18158,18161],{"style":15547},[170,18159],{"className":18160,"style":15538},[259],[170,18162,18163],{},[170,18164,1937],{"className":18165},[1062,2021,15556],[170,18167,18168,18171],{"style":17288},[170,18169],{"className":18170,"style":15538},[259],[170,18172,18174],{"className":18173},[264,265,266,267],[170,18175,18177],{"className":18176},[230,267],[170,18178,18180,18183],{"className":18179},[230,267],[170,18181,13348],{"className":18182,"style":3292},[230,303,267],[170,18184,18186],{"className":18185},[307],[170,18187,18189,18209],{"className":18188},[242,243],[170,18190,18192,18206],{"className":18191},[247],[170,18193,18195],{"className":18194,"style":4409},[251],[170,18196,18197,18200],{"style":17318},[170,18198],{"className":18199,"style":324},[259],[170,18201,18203],{"className":18202},[264,328,329,267],[170,18204,446],{"className":18205},[230,303,267],[170,18207,338],{"className":18208},[337],[170,18210,18212],{"className":18211},[247],[170,18213,18215],{"className":18214,"style":4431},[251],[170,18216],{},[170,18218,338],{"className":18219},[337],[170,18221,18223],{"className":18222},[247],[170,18224,18226],{"className":18225,"style":16499},[251],[170,18227],{},[170,18229],{"className":18230,"style":1027},[610],[170,18232,581],{"className":18233,"style":14215},[230,303],[170,18235,909],{"className":18236},[234],[170,18238,18240,18243],{"className":18239},[230],[170,18241,13276],{"className":18242,"style":13292},[230,303],[170,18244,18246],{"className":18245},[307],[170,18247,18249],{"className":18248},[242],[170,18250,18252],{"className":18251},[247],[170,18253,18255],{"className":18254,"style":16529},[251],[170,18256,18257,18260],{"style":16532},[170,18258],{"className":18259,"style":651},[259],[170,18261,18263],{"className":18262},[264,265,266,267],[170,18264,446],{"className":18265},[230,303,267],[170,18267,923],{"className":18268},[424],[170,18270,14486],{"className":18271},[230],[170,18273],{"className":18274,"style":1027},[610],[170,18276,16956,18278],{"className":18277},[1062],[170,18279,16960],{"style":16959},[170,18281],{"className":18282,"style":1027},[610],[170,18284,18286,18289],{"className":18285},[230],[170,18287,5711],{"className":18288,"style":3292},[230,303],[170,18290,18292],{"className":18291},[307],[170,18293,18295,18315],{"className":18294},[242,243],[170,18296,18298,18312],{"className":18297},[247],[170,18299,18301],{"className":18300,"style":1359},[251],[170,18302,18303,18306],{"style":7567},[170,18304],{"className":18305,"style":651},[259],[170,18307,18309],{"className":18308},[264,265,266,267],[170,18310,14157],{"className":18311,"style":3317},[230,303,267],[170,18313,338],{"className":18314},[337],[170,18316,18318],{"className":18317},[247],[170,18319,18321],{"className":18320,"style":1381},[251],[170,18322],{},[170,18324,909],{"className":18325},[234],[170,18327,18329,18332],{"className":18328},[230],[170,18330,134],{"className":18331},[230,303],[170,18333,18335],{"className":18334},[307],[170,18336,18338,18369],{"className":18337},[242,243],[170,18339,18341,18366],{"className":18340},[247],[170,18342,18344,18355],{"className":18343,"style":17414},[251],[170,18345,18346,18349],{"style":17417},[170,18347],{"className":18348,"style":651},[259],[170,18350,18352],{"className":18351},[264,265,266,267],[170,18353,446],{"className":18354},[230,303,267],[170,18356,18357,18360],{"style":16532},[170,18358],{"className":18359,"style":651},[259],[170,18361,18363],{"className":18362},[264,265,266,267],[170,18364,888],{"className":18365},[230,303,267],[170,18367,338],{"className":18368},[337],[170,18370,18372],{"className":18371},[247],[170,18373,18375],{"className":18374,"style":17447},[251],[170,18376],{},[170,18378,13047],{"className":18379},[230],[170,18381,18383,18386],{"className":18382},[230],[170,18384,12908],{"className":18385},[230,303],[170,18387,18389],{"className":18388},[307],[170,18390,18392,18423],{"className":18391},[242,243],[170,18393,18395,18420],{"className":18394},[247],[170,18396,18398,18409],{"className":18397,"style":17414},[251],[170,18399,18400,18403],{"style":17417},[170,18401],{"className":18402,"style":651},[259],[170,18404,18406],{"className":18405},[264,265,266,267],[170,18407,446],{"className":18408},[230,303,267],[170,18410,18411,18414],{"style":16532},[170,18412],{"className":18413,"style":651},[259],[170,18415,18417],{"className":18416},[264,265,266,267],[170,18418,888],{"className":18419},[230,303,267],[170,18421,338],{"className":18422},[337],[170,18424,18426],{"className":18425},[247],[170,18427,18429],{"className":18428,"style":17447},[251],[170,18430],{},[170,18432,923],{"className":18433},[424],[170,18435,338],{"className":18436},[337],[170,18438,18440],{"className":18439},[247],[170,18441,18443],{"className":18442,"style":15477},[251],[170,18444],{},[14,18446,18447],{},[170,18448,18450,18557],{"className":18449},[173],[170,18451,18453],{"className":18452},[177],[179,18454,18455],{"xmlns":181},[183,18456,18457,18554],{},[186,18458,18459,18461,18463,18465,18467,18469,18472,18478,18492,18510,18512,18514,18520,18522,18524,18526,18532,18534,18542,18544,18552],{},[202,18460,7923],{},[202,18462,903],{},[202,18464,12908],{},[202,18466,12908],{},[572,18468,926],{},[572,18470,18471],{},"−",[189,18473,18474,18476],{},[192,18475,194],{},[202,18477,7429],{},[1933,18479,18480,18482,18490],{},[572,18481,1937],{},[186,18483,18484,18486,18488],{},[202,18485,446],{},[572,18487,926],{},[192,18489,194],{},[202,18491,7429],{},[1933,18493,18494,18496,18504],{},[572,18495,1937],{},[186,18497,18498,18500,18502],{},[202,18499,888],{},[572,18501,926],{},[192,18503,194],{},[199,18505,18506,18508],{},[202,18507,13348],{},[202,18509,446],{},[202,18511,581],{},[572,18513,909],{"stretchy":908},[576,18515,18516,18518],{},[202,18517,13276],{},[202,18519,446],{},[572,18521,923],{"stretchy":908},[202,18523,14911],{},[572,18525,933],{},[199,18527,18528,18530],{},[202,18529,5711],{},[202,18531,14157],{},[572,18533,909],{"stretchy":908},[1933,18535,18536,18538,18540],{},[202,18537,134],{},[202,18539,446],{},[202,18541,888],{},[202,18543,13047],{"mathvariant":929},[1933,18545,18546,18548,18550],{},[202,18547,12908],{},[202,18549,446],{},[202,18551,888],{},[572,18553,923],{"stretchy":908},[209,18555,18556],{"encoding":211},"Loss = -\\frac{1}{N}\\sum\\limits_{n=1}^N \\sum\\limits_{t=1}^{T_n} R(\\tau^n) \\log P_\\theta(a_n^t|s_n^t)",[170,18558,18560,18582],{"className":18559,"ariaHidden":217},[216],[170,18561,18563,18566,18569,18573,18576,18579],{"className":18562},[221],[170,18564],{"className":18565,"style":14109},[225],[170,18567,7923],{"className":18568},[230,303],[170,18570,18572],{"className":18571},[230,303],"oss",[170,18574],{"className":18575,"style":611},[610],[170,18577,926],{"className":18578},[615],[170,18580],{"className":18581,"style":611},[610],[170,18583,18585,18589,18592,18662,18665,18733,18736,18842,18845,18848,18851,18880,18883,18886,18891,18894,18934,18937,18989,18992,19043],{"className":18584},[221],[170,18586],{"className":18587,"style":18588},[225],"height:2.5065em;vertical-align:-0.9671em;",[170,18590,18471],{"className":18591},[230],[170,18593,18595,18598,18659],{"className":18594},[230],[170,18596],{"className":18597},[234,235],[170,18599,18601],{"className":18600},[189],[170,18602,18604,18650],{"className":18603},[242,243],[170,18605,18607,18647],{"className":18606},[247],[170,18608,18610,18625,18633],{"className":18609,"style":252},[251],[170,18611,18613,18616],{"style":18612},"top:-2.655em;",[170,18614],{"className":18615,"style":260},[259],[170,18617,18619],{"className":18618},[264,265,266,267],[170,18620,18622],{"className":18621},[230,267],[170,18623,7429],{"className":18624,"style":7504},[230,303,267],[170,18626,18627,18630],{"style":385},[170,18628],{"className":18629,"style":260},[259],[170,18631],{"className":18632,"style":393},[392],[170,18634,18635,18638],{"style":396},[170,18636],{"className":18637,"style":260},[259],[170,18639,18641],{"className":18640},[264,265,266,267],[170,18642,18644],{"className":18643},[230,267],[170,18645,194],{"className":18646},[230,267],[170,18648,338],{"className":18649},[337],[170,18651,18653],{"className":18652},[247],[170,18654,18657],{"className":18655,"style":18656},[251],"height:0.345em;",[170,18658],{},[170,18660],{"className":18661},[424,235],[170,18663],{"className":18664,"style":1027},[610],[170,18666,18668],{"className":18667},[1062,14344],[170,18669,18671,18724],{"className":18670},[242,243],[170,18672,18674,18721],{"className":18673},[247],[170,18675,18678,18699,18709],{"className":18676,"style":18677},[251],"height:1.5283em;",[170,18679,18681,18684],{"style":18680},"top:-2.1329em;margin-left:0em;",[170,18682],{"className":18683,"style":260},[259],[170,18685,18687],{"className":18686},[264,265,266,267],[170,18688,18690,18693,18696],{"className":18689},[230,267],[170,18691,446],{"className":18692},[230,303,267],[170,18694,926],{"className":18695},[615,267],[170,18697,194],{"className":18698},[230,267],[170,18700,18701,18704],{"style":289},[170,18702],{"className":18703,"style":260},[259],[170,18705,18706],{},[170,18707,1937],{"className":18708},[1062,2021,2022],[170,18710,18712,18715],{"style":18711},"top:-3.95em;margin-left:0em;",[170,18713],{"className":18714,"style":260},[259],[170,18716,18718],{"className":18717},[264,265,266,267],[170,18719,7429],{"className":18720,"style":7504},[230,303,267],[170,18722,338],{"className":18723},[337],[170,18725,18727],{"className":18726},[247],[170,18728,18731],{"className":18729,"style":18730},[251],"height:0.9671em;",[170,18732],{},[170,18734],{"className":18735,"style":1027},[610],[170,18737,18739],{"className":18738},[1062,14344],[170,18740,18742,18834],{"className":18741},[242,243],[170,18743,18745,18831],{"className":18744},[247],[170,18746,18749,18769,18779],{"className":18747,"style":18748},[251],"height:1.5394em;",[170,18750,18751,18754],{"style":18680},[170,18752],{"className":18753,"style":260},[259],[170,18755,18757],{"className":18756},[264,265,266,267],[170,18758,18760,18763,18766],{"className":18759},[230,267],[170,18761,888],{"className":18762},[230,303,267],[170,18764,926],{"className":18765},[615,267],[170,18767,194],{"className":18768},[230,267],[170,18770,18771,18774],{"style":289},[170,18772],{"className":18773,"style":260},[259],[170,18775,18776],{},[170,18777,1937],{"className":18778},[1062,2021,2022],[170,18780,18782,18785],{"style":18781},"top:-3.9611em;margin-left:0em;",[170,18783],{"className":18784,"style":260},[259],[170,18786,18788],{"className":18787},[264,265,266,267],[170,18789,18791],{"className":18790},[230,267],[170,18792,18794,18797],{"className":18793},[230,267],[170,18795,13348],{"className":18796,"style":3292},[230,303,267],[170,18798,18800],{"className":18799},[307],[170,18801,18803,18823],{"className":18802},[242,243],[170,18804,18806,18820],{"className":18805},[247],[170,18807,18809],{"className":18808,"style":4409},[251],[170,18810,18811,18814],{"style":17318},[170,18812],{"className":18813,"style":324},[259],[170,18815,18817],{"className":18816},[264,328,329,267],[170,18818,446],{"className":18819},[230,303,267],[170,18821,338],{"className":18822},[337],[170,18824,18826],{"className":18825},[247],[170,18827,18829],{"className":18828,"style":4431},[251],[170,18830],{},[170,18832,338],{"className":18833},[337],[170,18835,18837],{"className":18836},[247],[170,18838,18840],{"className":18839,"style":18730},[251],[170,18841],{},[170,18843],{"className":18844,"style":1027},[610],[170,18846,581],{"className":18847,"style":14215},[230,303],[170,18849,909],{"className":18850},[234],[170,18852,18854,18857],{"className":18853},[230],[170,18855,13276],{"className":18856,"style":13292},[230,303],[170,18858,18860],{"className":18859},[307],[170,18861,18863],{"className":18862},[242],[170,18864,18866],{"className":18865},[247],[170,18867,18869],{"className":18868,"style":16728},[251],[170,18870,18871,18874],{"style":647},[170,18872],{"className":18873,"style":651},[259],[170,18875,18877],{"className":18876},[264,265,266,267],[170,18878,446],{"className":18879},[230,303,267],[170,18881,923],{"className":18882},[424],[170,18884],{"className":18885,"style":1027},[610],[170,18887,16956,18889],{"className":18888},[1062],[170,18890,16960],{"style":16959},[170,18892],{"className":18893,"style":1027},[610],[170,18895,18897,18900],{"className":18896},[230],[170,18898,5711],{"className":18899,"style":3292},[230,303],[170,18901,18903],{"className":18902},[307],[170,18904,18906,18926],{"className":18905},[242,243],[170,18907,18909,18923],{"className":18908},[247],[170,18910,18912],{"className":18911,"style":1359},[251],[170,18913,18914,18917],{"style":7567},[170,18915],{"className":18916,"style":651},[259],[170,18918,18920],{"className":18919},[264,265,266,267],[170,18921,14157],{"className":18922,"style":3317},[230,303,267],[170,18924,338],{"className":18925},[337],[170,18927,18929],{"className":18928},[247],[170,18930,18932],{"className":18931,"style":1381},[251],[170,18933],{},[170,18935,909],{"className":18936},[234],[170,18938,18940,18943],{"className":18939},[230],[170,18941,134],{"className":18942},[230,303],[170,18944,18946],{"className":18945},[307],[170,18947,18949,18981],{"className":18948},[242,243],[170,18950,18952,18978],{"className":18951},[247],[170,18953,18956,18967],{"className":18954,"style":18955},[251],"height:0.7936em;",[170,18957,18958,18961],{"style":17417},[170,18959],{"className":18960,"style":651},[259],[170,18962,18964],{"className":18963},[264,265,266,267],[170,18965,446],{"className":18966},[230,303,267],[170,18968,18969,18972],{"style":647},[170,18970],{"className":18971,"style":651},[259],[170,18973,18975],{"className":18974},[264,265,266,267],[170,18976,888],{"className":18977},[230,303,267],[170,18979,338],{"className":18980},[337],[170,18982,18984],{"className":18983},[247],[170,18985,18987],{"className":18986,"style":17447},[251],[170,18988],{},[170,18990,13047],{"className":18991},[230],[170,18993,18995,18998],{"className":18994},[230],[170,18996,12908],{"className":18997},[230,303],[170,18999,19001],{"className":19000},[307],[170,19002,19004,19035],{"className":19003},[242,243],[170,19005,19007,19032],{"className":19006},[247],[170,19008,19010,19021],{"className":19009,"style":18955},[251],[170,19011,19012,19015],{"style":17417},[170,19013],{"className":19014,"style":651},[259],[170,19016,19018],{"className":19017},[264,265,266,267],[170,19019,446],{"className":19020},[230,303,267],[170,19022,19023,19026],{"style":647},[170,19024],{"className":19025,"style":651},[259],[170,19027,19029],{"className":19028},[264,265,266,267],[170,19030,888],{"className":19031},[230,303,267],[170,19033,338],{"className":19034},[337],[170,19036,19038],{"className":19037},[247],[170,19039,19041],{"className":19040,"style":17447},[251],[170,19042],{},[170,19044,923],{"className":19045},[424],[14,19047,19048],{},[170,19049,19051,19139],{"className":19050},[173],[170,19052,19054],{"className":19053},[177],[179,19055,19056],{"xmlns":181},[183,19057,19058,19136],{},[186,19059,19060,19062,19064,19070,19072,19074,19097,19114,19126,19128],{},[202,19061,581],{},[572,19063,909],{"stretchy":908},[576,19065,19066,19068],{},[202,19067,13276],{},[202,19069,446],{},[572,19071,923],{"stretchy":908},[572,19073,926],{},[1933,19075,19076,19078,19091],{},[572,19077,1937],{},[186,19079,19080,19082,19084],{},[202,19081,888],{},[572,19083,926],{},[576,19085,19086,19088],{},[202,19087,888],{},[572,19089,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},"′",[199,19092,19093,19095],{},[202,19094,13348],{},[202,19096,446],{},[576,19098,19099,19102],{},[202,19100,19101],{},"γ",[186,19103,19104,19106,19108],{},[202,19105,888],{},[572,19107,18471],{},[576,19109,19110,19112],{},[202,19111,888],{},[572,19113,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[1933,19115,19116,19118,19124],{},[202,19117,7933],{},[576,19119,19120,19122],{},[202,19121,888],{},[572,19123,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[202,19125,446],{},[572,19127,926],{},[1933,19129,19130,19132,19134],{},[202,19131,581],{},[202,19133,888],{},[202,19135,446],{},[209,19137,19138],{"encoding":211},"R(\\tau^n) = \\sum\\limits_{t=t'}^{T_n} \\gamma^{t-t'} r_{t'}^n = R_t^n",[170,19140,19142,19195,19506],{"className":19141,"ariaHidden":217},[216],[170,19143,19145,19148,19151,19154,19183,19186,19189,19192],{"className":19144},[221],[170,19146],{"className":19147,"style":988},[225],[170,19149,581],{"className":19150,"style":14215},[230,303],[170,19152,909],{"className":19153},[234],[170,19155,19157,19160],{"className":19156},[230],[170,19158,13276],{"className":19159,"style":13292},[230,303],[170,19161,19163],{"className":19162},[307],[170,19164,19166],{"className":19165},[242],[170,19167,19169],{"className":19168},[247],[170,19170,19172],{"className":19171,"style":16728},[251],[170,19173,19174,19177],{"style":647},[170,19175],{"className":19176,"style":651},[259],[170,19178,19180],{"className":19179},[264,265,266,267],[170,19181,446],{"className":19182},[230,303,267],[170,19184,923],{"className":19185},[424],[170,19187],{"className":19188,"style":611},[610],[170,19190,926],{"className":19191},[615],[170,19193],{"className":19194,"style":611},[610],[170,19196,19198,19202,19339,19342,19412,19497,19500,19503],{"className":19197},[221],[170,19199],{"className":19200,"style":19201},[225],"height:2.5334em;vertical-align:-0.994em;",[170,19203,19205],{"className":19204},[1062,14344],[170,19206,19208,19330],{"className":19207},[242,243],[170,19209,19211,19327],{"className":19210},[247],[170,19212,19214,19266,19276],{"className":19213,"style":18748},[251],[170,19215,19217,19220],{"style":19216},"top:-2.106em;margin-left:0em;",[170,19218],{"className":19219,"style":260},[259],[170,19221,19223],{"className":19222},[264,265,266,267],[170,19224,19226,19229,19232],{"className":19225},[230,267],[170,19227,888],{"className":19228},[230,303,267],[170,19230,926],{"className":19231},[615,267],[170,19233,19235,19238],{"className":19234},[230,267],[170,19236,888],{"className":19237},[230,303,267],[170,19239,19241],{"className":19240},[307],[170,19242,19244],{"className":19243},[242],[170,19245,19247],{"className":19246},[247],[170,19248,19251],{"className":19249,"style":19250},[251],"height:0.6828em;",[170,19252,19254,19257],{"style":19253},"top:-2.786em;margin-right:0.0714em;",[170,19255],{"className":19256,"style":324},[259],[170,19258,19260],{"className":19259},[264,328,329,267],[170,19261,19263],{"className":19262},[230,267],[170,19264,19090],{"className":19265},[230,267],[170,19267,19268,19271],{"style":289},[170,19269],{"className":19270,"style":260},[259],[170,19272,19273],{},[170,19274,1937],{"className":19275},[1062,2021,2022],[170,19277,19278,19281],{"style":18781},[170,19279],{"className":19280,"style":260},[259],[170,19282,19284],{"className":19283},[264,265,266,267],[170,19285,19287],{"className":19286},[230,267],[170,19288,19290,19293],{"className":19289},[230,267],[170,19291,13348],{"className":19292,"style":3292},[230,303,267],[170,19294,19296],{"className":19295},[307],[170,19297,19299,19319],{"className":19298},[242,243],[170,19300,19302,19316],{"className":19301},[247],[170,19303,19305],{"className":19304,"style":4409},[251],[170,19306,19307,19310],{"style":17318},[170,19308],{"className":19309,"style":324},[259],[170,19311,19313],{"className":19312},[264,328,329,267],[170,19314,446],{"className":19315},[230,303,267],[170,19317,338],{"className":19318},[337],[170,19320,19322],{"className":19321},[247],[170,19323,19325],{"className":19324,"style":4431},[251],[170,19326],{},[170,19328,338],{"className":19329},[337],[170,19331,19333],{"className":19332},[247],[170,19334,19337],{"className":19335,"style":19336},[251],"height:0.994em;",[170,19338],{},[170,19340],{"className":19341,"style":1027},[610],[170,19343,19345,19349],{"className":19344},[230],[170,19346,19101],{"className":19347,"style":19348},[230,303],"margin-right:0.05556em;",[170,19350,19352],{"className":19351},[307],[170,19353,19355],{"className":19354},[242],[170,19356,19358],{"className":19357},[247],[170,19359,19362],{"className":19360,"style":19361},[251],"height:0.9425em;",[170,19363,19364,19367],{"style":647},[170,19365],{"className":19366,"style":651},[259],[170,19368,19370],{"className":19369},[264,265,266,267],[170,19371,19373,19376,19379],{"className":19372},[230,267],[170,19374,888],{"className":19375},[230,303,267],[170,19377,18471],{"className":19378},[664,267],[170,19380,19382,19385],{"className":19381},[230,267],[170,19383,888],{"className":19384},[230,303,267],[170,19386,19388],{"className":19387},[307],[170,19389,19391],{"className":19390},[242],[170,19392,19394],{"className":19393},[247],[170,19395,19398],{"className":19396,"style":19397},[251],"height:0.8278em;",[170,19399,19400,19403],{"style":1209},[170,19401],{"className":19402,"style":324},[259],[170,19404,19406],{"className":19405},[264,328,329,267],[170,19407,19409],{"className":19408},[230,267],[170,19410,19090],{"className":19411},[230,267],[170,19413,19415,19418],{"className":19414},[230],[170,19416,7933],{"className":19417,"style":3317},[230,303],[170,19419,19421],{"className":19420},[307],[170,19422,19424,19488],{"className":19423},[242,243],[170,19425,19427,19485],{"className":19426},[247],[170,19428,19430,19474],{"className":19429,"style":16728},[251],[170,19431,19433,19436],{"style":19432},"top:-2.425em;margin-left:-0.0278em;margin-right:0.05em;",[170,19434],{"className":19435,"style":651},[259],[170,19437,19439],{"className":19438},[264,265,266,267],[170,19440,19442],{"className":19441},[230,267],[170,19443,19445,19448],{"className":19444},[230,267],[170,19446,888],{"className":19447},[230,303,267],[170,19449,19451],{"className":19450},[307],[170,19452,19454],{"className":19453},[242],[170,19455,19457],{"className":19456},[247],[170,19458,19460],{"className":19459,"style":19250},[251],[170,19461,19462,19465],{"style":19253},[170,19463],{"className":19464,"style":324},[259],[170,19466,19468],{"className":19467},[264,328,329,267],[170,19469,19471],{"className":19470},[230,267],[170,19472,19090],{"className":19473},[230,267],[170,19475,19476,19479],{"style":647},[170,19477],{"className":19478,"style":651},[259],[170,19480,19482],{"className":19481},[264,265,266,267],[170,19483,446],{"className":19484},[230,303,267],[170,19486,338],{"className":19487},[337],[170,19489,19491],{"className":19490},[247],[170,19492,19495],{"className":19493,"style":19494},[251],"height:0.275em;",[170,19496],{},[170,19498],{"className":19499,"style":611},[610],[170,19501,926],{"className":19502},[615],[170,19504],{"className":19505,"style":611},[610],[170,19507,19509,19513],{"className":19508},[221],[170,19510],{"className":19511,"style":19512},[225],"height:0.9303em;vertical-align:-0.247em;",[170,19514,19516,19519],{"className":19515},[230],[170,19517,581],{"className":19518,"style":14215},[230,303],[170,19520,19522],{"className":19521},[307],[170,19523,19525,19557],{"className":19524},[242,243],[170,19526,19528,19554],{"className":19527},[247],[170,19529,19531,19543],{"className":19530,"style":16728},[251],[170,19532,19534,19537],{"style":19533},"top:-2.453em;margin-left:-0.0077em;margin-right:0.05em;",[170,19535],{"className":19536,"style":651},[259],[170,19538,19540],{"className":19539},[264,265,266,267],[170,19541,888],{"className":19542},[230,303,267],[170,19544,19545,19548],{"style":647},[170,19546],{"className":19547,"style":651},[259],[170,19549,19551],{"className":19550},[264,265,266,267],[170,19552,446],{"className":19553},[230,303,267],[170,19555,338],{"className":19556},[337],[170,19558,19560],{"className":19559},[247],[170,19561,19563],{"className":19562,"style":17447},[251],[170,19564],{},[10,19566,19568],{"id":19567},"actor-critic-演员-评论家算法","Actor-Critic 演员-评论家算法",[14,19570,19571,19572,19643],{},"为了衡量每个动作的相对好坏的情况，我们令奖励 ",[170,19573,19575,19593],{"className":19574},[173],[170,19576,19578],{"className":19577},[177],[179,19579,19580],{"xmlns":181},[183,19581,19582,19590],{},[186,19583,19584],{},[199,19585,19586,19588],{},[202,19587,581],{},[202,19589,888],{},[209,19591,19592],{"encoding":211},"R_t",[170,19594,19596],{"className":19595,"ariaHidden":217},[216],[170,19597,19599,19602],{"className":19598},[221],[170,19600],{"className":19601,"style":7081},[225],[170,19603,19605,19608],{"className":19604},[230],[170,19606,581],{"className":19607,"style":14215},[230,303],[170,19609,19611],{"className":19610},[307],[170,19612,19614,19635],{"className":19613},[242,243],[170,19615,19617,19632],{"className":19616},[247],[170,19618,19620],{"className":19619,"style":12838},[251],[170,19621,19623,19626],{"style":19622},"top:-2.55em;margin-left:-0.0077em;margin-right:0.05em;",[170,19624],{"className":19625,"style":651},[259],[170,19627,19629],{"className":19628},[264,265,266,267],[170,19630,888],{"className":19631},[230,303,267],[170,19633,338],{"className":19634},[337],[170,19636,19638],{"className":19637},[247],[170,19639,19641],{"className":19640,"style":1381},[251],[170,19642],{}," 减去一个基准值。",[14,19645,19646],{},[170,19647,19649,19762],{"className":19648},[173],[170,19650,19652],{"className":19651},[177],[179,19653,19654],{"xmlns":181},[183,19655,19656,19759],{},[186,19657,19658,19664,19678,19696,19698,19706,19708,19711,19713,19721,19723,19725,19727,19729,19731,19737,19739,19747,19749,19757],{},[189,19659,19660,19662],{},[192,19661,194],{},[202,19663,7429],{},[1933,19665,19666,19668,19676],{},[572,19667,1937],{},[186,19669,19670,19672,19674],{},[202,19671,446],{},[572,19673,926],{},[192,19675,194],{},[202,19677,7429],{},[1933,19679,19680,19682,19690],{},[572,19681,1937],{},[186,19683,19684,19686,19688],{},[202,19685,888],{},[572,19687,926],{},[192,19689,194],{},[199,19691,19692,19694],{},[202,19693,13348],{},[202,19695,446],{},[572,19697,909],{"stretchy":908},[1933,19699,19700,19702,19704],{},[202,19701,581],{},[202,19703,888],{},[202,19705,446],{},[572,19707,18471],{},[202,19709,19710],{},"B",[572,19712,909],{"stretchy":908},[1933,19714,19715,19717,19719],{},[202,19716,12908],{},[202,19718,888],{},[202,19720,446],{},[572,19722,923],{"stretchy":908},[572,19724,923],{"stretchy":908},[202,19726,14486],{"mathvariant":929},[202,19728,14911],{},[572,19730,933],{},[199,19732,19733,19735],{},[202,19734,5711],{},[202,19736,14157],{},[572,19738,909],{"stretchy":908},[1933,19740,19741,19743,19745],{},[202,19742,134],{},[202,19744,888],{},[202,19746,446],{},[202,19748,13047],{"mathvariant":929},[1933,19750,19751,19753,19755],{},[202,19752,12908],{},[202,19754,888],{},[202,19756,446],{},[572,19758,923],{"stretchy":908},[209,19760,19761],{"encoding":211}," \\frac{1}{N} \\sum\\limits_{n=1}^N \\sum\\limits_{t=1}^{T_n} (R_t^n - B(s_t^n))\\nabla \\log P_\\theta(a_t^n|s_t^n)",[170,19763,19765,20076],{"className":19764,"ariaHidden":217},[216],[170,19766,19768,19771,19839,19842,19906,19909,20013,20016,20067,20070,20073],{"className":19767},[221],[170,19769],{"className":19770,"style":18588},[225],[170,19772,19774,19777,19836],{"className":19773},[230],[170,19775],{"className":19776},[234,235],[170,19778,19780],{"className":19779},[189],[170,19781,19783,19828],{"className":19782},[242,243],[170,19784,19786,19825],{"className":19785},[247],[170,19787,19789,19803,19811],{"className":19788,"style":252},[251],[170,19790,19791,19794],{"style":18612},[170,19792],{"className":19793,"style":260},[259],[170,19795,19797],{"className":19796},[264,265,266,267],[170,19798,19800],{"className":19799},[230,267],[170,19801,7429],{"className":19802,"style":7504},[230,303,267],[170,19804,19805,19808],{"style":385},[170,19806],{"className":19807,"style":260},[259],[170,19809],{"className":19810,"style":393},[392],[170,19812,19813,19816],{"style":396},[170,19814],{"className":19815,"style":260},[259],[170,19817,19819],{"className":19818},[264,265,266,267],[170,19820,19822],{"className":19821},[230,267],[170,19823,194],{"className":19824},[230,267],[170,19826,338],{"className":19827},[337],[170,19829,19831],{"className":19830},[247],[170,19832,19834],{"className":19833,"style":18656},[251],[170,19835],{},[170,19837],{"className":19838},[424,235],[170,19840],{"className":19841,"style":1027},[610],[170,19843,19845],{"className":19844},[1062,14344],[170,19846,19848,19898],{"className":19847},[242,243],[170,19849,19851,19895],{"className":19850},[247],[170,19852,19854,19874,19884],{"className":19853,"style":18677},[251],[170,19855,19856,19859],{"style":18680},[170,19857],{"className":19858,"style":260},[259],[170,19860,19862],{"className":19861},[264,265,266,267],[170,19863,19865,19868,19871],{"className":19864},[230,267],[170,19866,446],{"className":19867},[230,303,267],[170,19869,926],{"className":19870},[615,267],[170,19872,194],{"className":19873},[230,267],[170,19875,19876,19879],{"style":289},[170,19877],{"className":19878,"style":260},[259],[170,19880,19881],{},[170,19882,1937],{"className":19883},[1062,2021,2022],[170,19885,19886,19889],{"style":18711},[170,19887],{"className":19888,"style":260},[259],[170,19890,19892],{"className":19891},[264,265,266,267],[170,19893,7429],{"className":19894,"style":7504},[230,303,267],[170,19896,338],{"className":19897},[337],[170,19899,19901],{"className":19900},[247],[170,19902,19904],{"className":19903,"style":18730},[251],[170,19905],{},[170,19907],{"className":19908,"style":1027},[610],[170,19910,19912],{"className":19911},[1062,14344],[170,19913,19915,20005],{"className":19914},[242,243],[170,19916,19918,20002],{"className":19917},[247],[170,19919,19921,19941,19951],{"className":19920,"style":18748},[251],[170,19922,19923,19926],{"style":18680},[170,19924],{"className":19925,"style":260},[259],[170,19927,19929],{"className":19928},[264,265,266,267],[170,19930,19932,19935,19938],{"className":19931},[230,267],[170,19933,888],{"className":19934},[230,303,267],[170,19936,926],{"className":19937},[615,267],[170,19939,194],{"className":19940},[230,267],[170,19942,19943,19946],{"style":289},[170,19944],{"className":19945,"style":260},[259],[170,19947,19948],{},[170,19949,1937],{"className":19950},[1062,2021,2022],[170,19952,19953,19956],{"style":18781},[170,19954],{"className":19955,"style":260},[259],[170,19957,19959],{"className":19958},[264,265,266,267],[170,19960,19962],{"className":19961},[230,267],[170,19963,19965,19968],{"className":19964},[230,267],[170,19966,13348],{"className":19967,"style":3292},[230,303,267],[170,19969,19971],{"className":19970},[307],[170,19972,19974,19994],{"className":19973},[242,243],[170,19975,19977,19991],{"className":19976},[247],[170,19978,19980],{"className":19979,"style":4409},[251],[170,19981,19982,19985],{"style":17318},[170,19983],{"className":19984,"style":324},[259],[170,19986,19988],{"className":19987},[264,328,329,267],[170,19989,446],{"className":19990},[230,303,267],[170,19992,338],{"className":19993},[337],[170,19995,19997],{"className":19996},[247],[170,19998,20000],{"className":19999,"style":4431},[251],[170,20001],{},[170,20003,338],{"className":20004},[337],[170,20006,20008],{"className":20007},[247],[170,20009,20011],{"className":20010,"style":18730},[251],[170,20012],{},[170,20014,909],{"className":20015},[234],[170,20017,20019,20022],{"className":20018},[230],[170,20020,581],{"className":20021,"style":14215},[230,303],[170,20023,20025],{"className":20024},[307],[170,20026,20028,20059],{"className":20027},[242,243],[170,20029,20031,20056],{"className":20030},[247],[170,20032,20034,20045],{"className":20033,"style":16728},[251],[170,20035,20036,20039],{"style":19533},[170,20037],{"className":20038,"style":651},[259],[170,20040,20042],{"className":20041},[264,265,266,267],[170,20043,888],{"className":20044},[230,303,267],[170,20046,20047,20050],{"style":647},[170,20048],{"className":20049,"style":651},[259],[170,20051,20053],{"className":20052},[264,265,266,267],[170,20054,446],{"className":20055},[230,303,267],[170,20057,338],{"className":20058},[337],[170,20060,20062],{"className":20061},[247],[170,20063,20065],{"className":20064,"style":17447},[251],[170,20066],{},[170,20068],{"className":20069,"style":1983},[610],[170,20071,18471],{"className":20072},[664],[170,20074],{"className":20075,"style":1983},[610],[170,20077,20079,20082,20086,20089,20140,20143,20146,20149,20154,20157,20197,20200,20251,20254,20305],{"className":20078},[221],[170,20080],{"className":20081,"style":988},[225],[170,20083,19710],{"className":20084,"style":20085},[230,303],"margin-right:0.05017em;",[170,20087,909],{"className":20088},[234],[170,20090,20092,20095],{"className":20091},[230],[170,20093,12908],{"className":20094},[230,303],[170,20096,20098],{"className":20097},[307],[170,20099,20101,20132],{"className":20100},[242,243],[170,20102,20104,20129],{"className":20103},[247],[170,20105,20107,20118],{"className":20106,"style":16728},[251],[170,20108,20109,20112],{"style":17417},[170,20110],{"className":20111,"style":651},[259],[170,20113,20115],{"className":20114},[264,265,266,267],[170,20116,888],{"className":20117},[230,303,267],[170,20119,20120,20123],{"style":647},[170,20121],{"className":20122,"style":651},[259],[170,20124,20126],{"className":20125},[264,265,266,267],[170,20127,446],{"className":20128},[230,303,267],[170,20130,338],{"className":20131},[337],[170,20133,20135],{"className":20134},[247],[170,20136,20138],{"className":20137,"style":17447},[251],[170,20139],{},[170,20141,8031],{"className":20142},[424],[170,20144,14486],{"className":20145},[230],[170,20147],{"className":20148,"style":1027},[610],[170,20150,16956,20152],{"className":20151},[1062],[170,20153,16960],{"style":16959},[170,20155],{"className":20156,"style":1027},[610],[170,20158,20160,20163],{"className":20159},[230],[170,20161,5711],{"className":20162,"style":3292},[230,303],[170,20164,20166],{"className":20165},[307],[170,20167,20169,20189],{"className":20168},[242,243],[170,20170,20172,20186],{"className":20171},[247],[170,20173,20175],{"className":20174,"style":1359},[251],[170,20176,20177,20180],{"style":7567},[170,20178],{"className":20179,"style":651},[259],[170,20181,20183],{"className":20182},[264,265,266,267],[170,20184,14157],{"className":20185,"style":3317},[230,303,267],[170,20187,338],{"className":20188},[337],[170,20190,20192],{"className":20191},[247],[170,20193,20195],{"className":20194,"style":1381},[251],[170,20196],{},[170,20198,909],{"className":20199},[234],[170,20201,20203,20206],{"className":20202},[230],[170,20204,134],{"className":20205},[230,303],[170,20207,20209],{"className":20208},[307],[170,20210,20212,20243],{"className":20211},[242,243],[170,20213,20215,20240],{"className":20214},[247],[170,20216,20218,20229],{"className":20217,"style":16728},[251],[170,20219,20220,20223],{"style":17417},[170,20221],{"className":20222,"style":651},[259],[170,20224,20226],{"className":20225},[264,265,266,267],[170,20227,888],{"className":20228},[230,303,267],[170,20230,20231,20234],{"style":647},[170,20232],{"className":20233,"style":651},[259],[170,20235,20237],{"className":20236},[264,265,266,267],[170,20238,446],{"className":20239},[230,303,267],[170,20241,338],{"className":20242},[337],[170,20244,20246],{"className":20245},[247],[170,20247,20249],{"className":20248,"style":17447},[251],[170,20250],{},[170,20252,13047],{"className":20253},[230],[170,20255,20257,20260],{"className":20256},[230],[170,20258,12908],{"className":20259},[230,303],[170,20261,20263],{"className":20262},[307],[170,20264,20266,20297],{"className":20265},[242,243],[170,20267,20269,20294],{"className":20268},[247],[170,20270,20272,20283],{"className":20271,"style":16728},[251],[170,20273,20274,20277],{"style":17417},[170,20275],{"className":20276,"style":651},[259],[170,20278,20280],{"className":20279},[264,265,266,267],[170,20281,888],{"className":20282},[230,303,267],[170,20284,20285,20288],{"style":647},[170,20286],{"className":20287,"style":651},[259],[170,20289,20291],{"className":20290},[264,265,266,267],[170,20292,446],{"className":20293},[230,303,267],[170,20295,338],{"className":20296},[337],[170,20298,20300],{"className":20299},[247],[170,20301,20303],{"className":20302,"style":17447},[251],[170,20304],{},[170,20306,923],{"className":20307},[424],[14,20309,20310,20311,20394,20395,20493,20494,20522,20523,20551],{},"但是，",[170,20312,20314,20334],{"className":20313},[173],[170,20315,20317],{"className":20316},[177],[179,20318,20319],{"xmlns":181},[183,20320,20321,20331],{},[186,20322,20323],{},[1933,20324,20325,20327,20329],{},[202,20326,581],{},[202,20328,888],{},[202,20330,446],{},[209,20332,20333],{"encoding":211},"R_t^n",[170,20335,20337],{"className":20336,"ariaHidden":217},[216],[170,20338,20340,20343],{"className":20339},[221],[170,20341],{"className":20342,"style":19512},[225],[170,20344,20346,20349],{"className":20345},[230],[170,20347,581],{"className":20348,"style":14215},[230,303],[170,20350,20352],{"className":20351},[307],[170,20353,20355,20386],{"className":20354},[242,243],[170,20356,20358,20383],{"className":20357},[247],[170,20359,20361,20372],{"className":20360,"style":16728},[251],[170,20362,20363,20366],{"style":19533},[170,20364],{"className":20365,"style":651},[259],[170,20367,20369],{"className":20368},[264,265,266,267],[170,20370,888],{"className":20371},[230,303,267],[170,20373,20374,20377],{"style":647},[170,20375],{"className":20376,"style":651},[259],[170,20378,20380],{"className":20379},[264,265,266,267],[170,20381,446],{"className":20382},[230,303,267],[170,20384,338],{"className":20385},[337],[170,20387,20389],{"className":20388},[247],[170,20390,20392],{"className":20391,"style":17447},[251],[170,20393],{}," 都是一次随机采样，方差很大，训练不稳定。我们引入 ",[170,20396,20398,20426],{"className":20397},[173],[170,20399,20401],{"className":20400},[177],[179,20402,20403],{"xmlns":181},[183,20404,20405,20423],{},[186,20406,20407,20413,20415,20417,20419,20421],{},[199,20408,20409,20411],{},[202,20410,570],{},[202,20412,14157],{},[572,20414,909],{"stretchy":908},[202,20416,12908],{},[572,20418,914],{"separator":217},[202,20420,134],{},[572,20422,923],{"stretchy":908},[209,20424,20425],{"encoding":211},"Q_\\theta(s,a)",[170,20427,20429],{"className":20428,"ariaHidden":217},[216],[170,20430,20432,20435,20475,20478,20481,20484,20487,20490],{"className":20431},[221],[170,20433],{"className":20434,"style":988},[225],[170,20436,20438,20441],{"className":20437},[230],[170,20439,570],{"className":20440},[230,303],[170,20442,20444],{"className":20443},[307],[170,20445,20447,20467],{"className":20446},[242,243],[170,20448,20450,20464],{"className":20449},[247],[170,20451,20453],{"className":20452,"style":1359},[251],[170,20454,20455,20458],{"style":1362},[170,20456],{"className":20457,"style":651},[259],[170,20459,20461],{"className":20460},[264,265,266,267],[170,20462,14157],{"className":20463,"style":3317},[230,303,267],[170,20465,338],{"className":20466},[337],[170,20468,20470],{"className":20469},[247],[170,20471,20473],{"className":20472,"style":1381},[251],[170,20474],{},[170,20476,909],{"className":20477},[234],[170,20479,12908],{"className":20480},[230,303],[170,20482,914],{"className":20483},[1023],[170,20485],{"className":20486,"style":1027},[610],[170,20488,134],{"className":20489},[230,303],[170,20491,923],{"className":20492},[424],"，表示在状态 ",[170,20495,20497,20510],{"className":20496},[173],[170,20498,20500],{"className":20499},[177],[179,20501,20502],{"xmlns":181},[183,20503,20504,20508],{},[186,20505,20506],{},[202,20507,12908],{},[209,20509,12908],{"encoding":211},[170,20511,20513],{"className":20512,"ariaHidden":217},[216],[170,20514,20516,20519],{"className":20515},[221],[170,20517],{"className":20518,"style":458},[225],[170,20520,12908],{"className":20521},[230,303],"，下，做出动作 ",[170,20524,20526,20539],{"className":20525},[173],[170,20527,20529],{"className":20528},[177],[179,20530,20531],{"xmlns":181},[183,20532,20533,20537],{},[186,20534,20535],{},[202,20536,134],{},[209,20538,134],{"encoding":211},[170,20540,20542],{"className":20541,"ariaHidden":217},[216],[170,20543,20545,20548],{"className":20544},[221],[170,20546],{"className":20547,"style":458},[225],[170,20549,134],{"className":20550},[230,303]," 后期望的回报，即动作价值函数。",[14,20553,20554,20555,20641,20642,20670],{},"令 ",[170,20556,20558,20582],{"className":20557},[173],[170,20559,20561],{"className":20560},[177],[179,20562,20563],{"xmlns":181},[183,20564,20565,20579],{},[186,20566,20567,20573,20575,20577],{},[199,20568,20569,20571],{},[202,20570,783],{},[202,20572,14157],{},[572,20574,909],{"stretchy":908},[202,20576,12908],{},[572,20578,923],{"stretchy":908},[209,20580,20581],{"encoding":211},"V_\\theta(s)",[170,20583,20585],{"className":20584,"ariaHidden":217},[216],[170,20586,20588,20591,20632,20635,20638],{"className":20587},[221],[170,20589],{"className":20590,"style":988},[225],[170,20592,20594,20597],{"className":20593},[230],[170,20595,783],{"className":20596,"style":813},[230,303],[170,20598,20600],{"className":20599},[307],[170,20601,20603,20624],{"className":20602},[242,243],[170,20604,20606,20621],{"className":20605},[247],[170,20607,20609],{"className":20608,"style":1359},[251],[170,20610,20612,20615],{"style":20611},"top:-2.55em;margin-left:-0.2222em;margin-right:0.05em;",[170,20613],{"className":20614,"style":651},[259],[170,20616,20618],{"className":20617},[264,265,266,267],[170,20619,14157],{"className":20620,"style":3317},[230,303,267],[170,20622,338],{"className":20623},[337],[170,20625,20627],{"className":20626},[247],[170,20628,20630],{"className":20629,"style":1381},[251],[170,20631],{},[170,20633,909],{"className":20634},[234],[170,20636,12908],{"className":20637},[230,303],[170,20639,923],{"className":20640},[424]," 在状态 ",[170,20643,20645,20658],{"className":20644},[173],[170,20646,20648],{"className":20647},[177],[179,20649,20650],{"xmlns":181},[183,20651,20652,20656],{},[186,20653,20654],{},[202,20655,12908],{},[209,20657,12908],{"encoding":211},[170,20659,20661],{"className":20660,"ariaHidden":217},[216],[170,20662,20664,20667],{"className":20663},[221],[170,20665],{"className":20666,"style":458},[225],[170,20668,12908],{"className":20669},[230,303]," 下期望的回报，即状态价值函数",[14,20672,20673,20940,20941,20969,20970,20998],{},[170,20674,20676,20736],{"className":20675},[173],[170,20677,20679],{"className":20678},[177],[179,20680,20681],{"xmlns":181},[183,20682,20683,20733],{},[186,20684,20685,20691,20693,20695,20697,20699,20701,20703,20709,20711,20713,20715,20717,20719,20721,20727,20729,20731],{},[199,20686,20687,20689],{},[202,20688,885],{},[202,20690,14157],{},[572,20692,909],{"stretchy":908},[202,20694,12908],{},[572,20696,914],{"separator":217},[202,20698,134],{},[572,20700,923],{"stretchy":908},[572,20702,926],{},[199,20704,20705,20707],{},[202,20706,570],{},[202,20708,14157],{},[572,20710,909],{"stretchy":908},[202,20712,12908],{},[572,20714,914],{"separator":217},[202,20716,134],{},[572,20718,923],{"stretchy":908},[572,20720,18471],{},[199,20722,20723,20725],{},[202,20724,783],{},[202,20726,14157],{},[572,20728,909],{"stretchy":908},[202,20730,12908],{},[572,20732,923],{"stretchy":908},[209,20734,20735],{"encoding":211},"A_\\theta(s,a) = Q_\\theta(s,a) - V_\\theta(s)",[170,20737,20739,20812,20885],{"className":20738,"ariaHidden":217},[216],[170,20740,20742,20745,20785,20788,20791,20794,20797,20800,20803,20806,20809],{"className":20741},[221],[170,20743],{"className":20744,"style":988},[225],[170,20746,20748,20751],{"className":20747},[230],[170,20749,885],{"className":20750},[230,303],[170,20752,20754],{"className":20753},[307],[170,20755,20757,20777],{"className":20756},[242,243],[170,20758,20760,20774],{"className":20759},[247],[170,20761,20763],{"className":20762,"style":1359},[251],[170,20764,20765,20768],{"style":1362},[170,20766],{"className":20767,"style":651},[259],[170,20769,20771],{"className":20770},[264,265,266,267],[170,20772,14157],{"className":20773,"style":3317},[230,303,267],[170,20775,338],{"className":20776},[337],[170,20778,20780],{"className":20779},[247],[170,20781,20783],{"className":20782,"style":1381},[251],[170,20784],{},[170,20786,909],{"className":20787},[234],[170,20789,12908],{"className":20790},[230,303],[170,20792,914],{"className":20793},[1023],[170,20795],{"className":20796,"style":1027},[610],[170,20798,134],{"className":20799},[230,303],[170,20801,923],{"className":20802},[424],[170,20804],{"className":20805,"style":611},[610],[170,20807,926],{"className":20808},[615],[170,20810],{"className":20811,"style":611},[610],[170,20813,20815,20818,20858,20861,20864,20867,20870,20873,20876,20879,20882],{"className":20814},[221],[170,20816],{"className":20817,"style":988},[225],[170,20819,20821,20824],{"className":20820},[230],[170,20822,570],{"className":20823},[230,303],[170,20825,20827],{"className":20826},[307],[170,20828,20830,20850],{"className":20829},[242,243],[170,20831,20833,20847],{"className":20832},[247],[170,20834,20836],{"className":20835,"style":1359},[251],[170,20837,20838,20841],{"style":1362},[170,20839],{"className":20840,"style":651},[259],[170,20842,20844],{"className":20843},[264,265,266,267],[170,20845,14157],{"className":20846,"style":3317},[230,303,267],[170,20848,338],{"className":20849},[337],[170,20851,20853],{"className":20852},[247],[170,20854,20856],{"className":20855,"style":1381},[251],[170,20857],{},[170,20859,909],{"className":20860},[234],[170,20862,12908],{"className":20863},[230,303],[170,20865,914],{"className":20866},[1023],[170,20868],{"className":20869,"style":1027},[610],[170,20871,134],{"className":20872},[230,303],[170,20874,923],{"className":20875},[424],[170,20877],{"className":20878,"style":1983},[610],[170,20880,18471],{"className":20881},[664],[170,20883],{"className":20884,"style":1983},[610],[170,20886,20888,20891,20931,20934,20937],{"className":20887},[221],[170,20889],{"className":20890,"style":988},[225],[170,20892,20894,20897],{"className":20893},[230],[170,20895,783],{"className":20896,"style":813},[230,303],[170,20898,20900],{"className":20899},[307],[170,20901,20903,20923],{"className":20902},[242,243],[170,20904,20906,20920],{"className":20905},[247],[170,20907,20909],{"className":20908,"style":1359},[251],[170,20910,20911,20914],{"style":20611},[170,20912],{"className":20913,"style":651},[259],[170,20915,20917],{"className":20916},[264,265,266,267],[170,20918,14157],{"className":20919,"style":3317},[230,303,267],[170,20921,338],{"className":20922},[337],[170,20924,20926],{"className":20925},[247],[170,20927,20929],{"className":20928,"style":1381},[251],[170,20930],{},[170,20932,909],{"className":20933},[234],[170,20935,12908],{"className":20936},[230,303],[170,20938,923],{"className":20939},[424]," 表示优势函数（Advantage Function），衡量在状态 ",[170,20942,20944,20957],{"className":20943},[173],[170,20945,20947],{"className":20946},[177],[179,20948,20949],{"xmlns":181},[183,20950,20951,20955],{},[186,20952,20953],{},[202,20954,12908],{},[209,20956,12908],{"encoding":211},[170,20958,20960],{"className":20959,"ariaHidden":217},[216],[170,20961,20963,20966],{"className":20962},[221],[170,20964],{"className":20965,"style":458},[225],[170,20967,12908],{"className":20968},[230,303]," 下，动作 ",[170,20971,20973,20986],{"className":20972},[173],[170,20974,20976],{"className":20975},[177],[179,20977,20978],{"xmlns":181},[183,20979,20980,20984],{},[186,20981,20982],{},[202,20983,134],{},[209,20985,134],{"encoding":211},[170,20987,20989],{"className":20988,"ariaHidden":217},[216],[170,20990,20992,20995],{"className":20991},[221],[170,20993],{"className":20994,"style":458},[225],[170,20996,134],{"className":20997},[230,303]," 相对于平均水平的优势。",[14,21000,21001],{},"讲上述的公式改写为",[14,21003,21004],{},[170,21005,21007,21121],{"className":21006},[173],[170,21008,21010],{"className":21009},[177],[179,21011,21012],{"xmlns":181},[183,21013,21014,21118],{},[186,21015,21016,21024,21038,21056,21062,21064,21072,21074,21082,21084,21086,21088,21090,21096,21098,21106,21108,21116],{},[14480,21017,21018],{"scriptlevel":1865,"displaystyle":217},[189,21019,21020,21022],{},[192,21021,194],{},[202,21023,7429],{},[1933,21025,21026,21028,21036],{},[572,21027,1937],{},[186,21029,21030,21032,21034],{},[202,21031,446],{},[572,21033,926],{},[192,21035,194],{},[202,21037,7429],{},[1933,21039,21040,21042,21050],{},[572,21041,1937],{},[186,21043,21044,21046,21048],{},[202,21045,888],{},[572,21047,926],{},[192,21049,194],{},[199,21051,21052,21054],{},[202,21053,13348],{},[202,21055,446],{},[199,21057,21058,21060],{},[202,21059,885],{},[202,21061,14157],{},[572,21063,909],{"stretchy":908},[1933,21065,21066,21068,21070],{},[202,21067,12908],{},[202,21069,888],{},[202,21071,446],{},[572,21073,914],{"separator":217},[1933,21075,21076,21078,21080],{},[202,21077,134],{},[202,21079,888],{},[202,21081,446],{},[572,21083,923],{"stretchy":908},[202,21085,14486],{"mathvariant":929},[202,21087,14911],{},[572,21089,933],{},[199,21091,21092,21094],{},[202,21093,5711],{},[202,21095,14157],{},[572,21097,909],{"stretchy":908},[1933,21099,21100,21102,21104],{},[202,21101,134],{},[202,21103,888],{},[202,21105,446],{},[202,21107,13047],{"mathvariant":929},[1933,21109,21110,21112,21114],{},[202,21111,12908],{},[202,21113,888],{},[202,21115,446],{},[572,21117,923],{"stretchy":908},[209,21119,21120],{"encoding":211},"\\dfrac{1}{N}\\sum\\limits_{n=1}^N \\sum\\limits_{t=1}^{T_n} A_\\theta(s_t^n,a_t^n)\\nabla \\log P_\\theta(a_t^n|s_t^n)",[170,21122,21124],{"className":21123,"ariaHidden":217},[216],[170,21125,21127,21130,21192,21195,21259,21262,21366,21369,21409,21412,21463,21466,21469,21520,21523,21526,21529,21534,21537,21577,21580,21631,21634,21685],{"className":21126},[221],[170,21128],{"className":21129,"style":18588},[225],[170,21131,21133,21136,21189],{"className":21132},[230],[170,21134],{"className":21135},[234,235],[170,21137,21139],{"className":21138},[189],[170,21140,21142,21181],{"className":21141},[242,243],[170,21143,21145,21178],{"className":21144},[247],[170,21146,21148,21159,21167],{"className":21147,"style":16385},[251],[170,21149,21150,21153],{"style":15925},[170,21151],{"className":21152,"style":260},[259],[170,21154,21156],{"className":21155},[230],[170,21157,7429],{"className":21158,"style":7504},[230,303],[170,21160,21161,21164],{"style":385},[170,21162],{"className":21163,"style":260},[259],[170,21165],{"className":21166,"style":393},[392],[170,21168,21169,21172],{"style":15991},[170,21170],{"className":21171,"style":260},[259],[170,21173,21175],{"className":21174},[230],[170,21176,194],{"className":21177},[230],[170,21179,338],{"className":21180},[337],[170,21182,21184],{"className":21183},[247],[170,21185,21187],{"className":21186,"style":16425},[251],[170,21188],{},[170,21190],{"className":21191},[424,235],[170,21193],{"className":21194,"style":1027},[610],[170,21196,21198],{"className":21197},[1062,14344],[170,21199,21201,21251],{"className":21200},[242,243],[170,21202,21204,21248],{"className":21203},[247],[170,21205,21207,21227,21237],{"className":21206,"style":18677},[251],[170,21208,21209,21212],{"style":18680},[170,21210],{"className":21211,"style":260},[259],[170,21213,21215],{"className":21214},[264,265,266,267],[170,21216,21218,21221,21224],{"className":21217},[230,267],[170,21219,446],{"className":21220},[230,303,267],[170,21222,926],{"className":21223},[615,267],[170,21225,194],{"className":21226},[230,267],[170,21228,21229,21232],{"style":289},[170,21230],{"className":21231,"style":260},[259],[170,21233,21234],{},[170,21235,1937],{"className":21236},[1062,2021,2022],[170,21238,21239,21242],{"style":18711},[170,21240],{"className":21241,"style":260},[259],[170,21243,21245],{"className":21244},[264,265,266,267],[170,21246,7429],{"className":21247,"style":7504},[230,303,267],[170,21249,338],{"className":21250},[337],[170,21252,21254],{"className":21253},[247],[170,21255,21257],{"className":21256,"style":18730},[251],[170,21258],{},[170,21260],{"className":21261,"style":1027},[610],[170,21263,21265],{"className":21264},[1062,14344],[170,21266,21268,21358],{"className":21267},[242,243],[170,21269,21271,21355],{"className":21270},[247],[170,21272,21274,21294,21304],{"className":21273,"style":18748},[251],[170,21275,21276,21279],{"style":18680},[170,21277],{"className":21278,"style":260},[259],[170,21280,21282],{"className":21281},[264,265,266,267],[170,21283,21285,21288,21291],{"className":21284},[230,267],[170,21286,888],{"className":21287},[230,303,267],[170,21289,926],{"className":21290},[615,267],[170,21292,194],{"className":21293},[230,267],[170,21295,21296,21299],{"style":289},[170,21297],{"className":21298,"style":260},[259],[170,21300,21301],{},[170,21302,1937],{"className":21303},[1062,2021,2022],[170,21305,21306,21309],{"style":18781},[170,21307],{"className":21308,"style":260},[259],[170,21310,21312],{"className":21311},[264,265,266,267],[170,21313,21315],{"className":21314},[230,267],[170,21316,21318,21321],{"className":21317},[230,267],[170,21319,13348],{"className":21320,"style":3292},[230,303,267],[170,21322,21324],{"className":21323},[307],[170,21325,21327,21347],{"className":21326},[242,243],[170,21328,21330,21344],{"className":21329},[247],[170,21331,21333],{"className":21332,"style":4409},[251],[170,21334,21335,21338],{"style":17318},[170,21336],{"className":21337,"style":324},[259],[170,21339,21341],{"className":21340},[264,328,329,267],[170,21342,446],{"className":21343},[230,303,267],[170,21345,338],{"className":21346},[337],[170,21348,21350],{"className":21349},[247],[170,21351,21353],{"className":21352,"style":4431},[251],[170,21354],{},[170,21356,338],{"className":21357},[337],[170,21359,21361],{"className":21360},[247],[170,21362,21364],{"className":21363,"style":18730},[251],[170,21365],{},[170,21367],{"className":21368,"style":1027},[610],[170,21370,21372,21375],{"className":21371},[230],[170,21373,885],{"className":21374},[230,303],[170,21376,21378],{"className":21377},[307],[170,21379,21381,21401],{"className":21380},[242,243],[170,21382,21384,21398],{"className":21383},[247],[170,21385,21387],{"className":21386,"style":1359},[251],[170,21388,21389,21392],{"style":1362},[170,21390],{"className":21391,"style":651},[259],[170,21393,21395],{"className":21394},[264,265,266,267],[170,21396,14157],{"className":21397,"style":3317},[230,303,267],[170,21399,338],{"className":21400},[337],[170,21402,21404],{"className":21403},[247],[170,21405,21407],{"className":21406,"style":1381},[251],[170,21408],{},[170,21410,909],{"className":21411},[234],[170,21413,21415,21418],{"className":21414},[230],[170,21416,12908],{"className":21417},[230,303],[170,21419,21421],{"className":21420},[307],[170,21422,21424,21455],{"className":21423},[242,243],[170,21425,21427,21452],{"className":21426},[247],[170,21428,21430,21441],{"className":21429,"style":16728},[251],[170,21431,21432,21435],{"style":17417},[170,21433],{"className":21434,"style":651},[259],[170,21436,21438],{"className":21437},[264,265,266,267],[170,21439,888],{"className":21440},[230,303,267],[170,21442,21443,21446],{"style":647},[170,21444],{"className":21445,"style":651},[259],[170,21447,21449],{"className":21448},[264,265,266,267],[170,21450,446],{"className":21451},[230,303,267],[170,21453,338],{"className":21454},[337],[170,21456,21458],{"className":21457},[247],[170,21459,21461],{"className":21460,"style":17447},[251],[170,21462],{},[170,21464,914],{"className":21465},[1023],[170,21467],{"className":21468,"style":1027},[610],[170,21470,21472,21475],{"className":21471},[230],[170,21473,134],{"className":21474},[230,303],[170,21476,21478],{"className":21477},[307],[170,21479,21481,21512],{"className":21480},[242,243],[170,21482,21484,21509],{"className":21483},[247],[170,21485,21487,21498],{"className":21486,"style":16728},[251],[170,21488,21489,21492],{"style":17417},[170,21490],{"className":21491,"style":651},[259],[170,21493,21495],{"className":21494},[264,265,266,267],[170,21496,888],{"className":21497},[230,303,267],[170,21499,21500,21503],{"style":647},[170,21501],{"className":21502,"style":651},[259],[170,21504,21506],{"className":21505},[264,265,266,267],[170,21507,446],{"className":21508},[230,303,267],[170,21510,338],{"className":21511},[337],[170,21513,21515],{"className":21514},[247],[170,21516,21518],{"className":21517,"style":17447},[251],[170,21519],{},[170,21521,923],{"className":21522},[424],[170,21524,14486],{"className":21525},[230],[170,21527],{"className":21528,"style":1027},[610],[170,21530,16956,21532],{"className":21531},[1062],[170,21533,16960],{"style":16959},[170,21535],{"className":21536,"style":1027},[610],[170,21538,21540,21543],{"className":21539},[230],[170,21541,5711],{"className":21542,"style":3292},[230,303],[170,21544,21546],{"className":21545},[307],[170,21547,21549,21569],{"className":21548},[242,243],[170,21550,21552,21566],{"className":21551},[247],[170,21553,21555],{"className":21554,"style":1359},[251],[170,21556,21557,21560],{"style":7567},[170,21558],{"className":21559,"style":651},[259],[170,21561,21563],{"className":21562},[264,265,266,267],[170,21564,14157],{"className":21565,"style":3317},[230,303,267],[170,21567,338],{"className":21568},[337],[170,21570,21572],{"className":21571},[247],[170,21573,21575],{"className":21574,"style":1381},[251],[170,21576],{},[170,21578,909],{"className":21579},[234],[170,21581,21583,21586],{"className":21582},[230],[170,21584,134],{"className":21585},[230,303],[170,21587,21589],{"className":21588},[307],[170,21590,21592,21623],{"className":21591},[242,243],[170,21593,21595,21620],{"className":21594},[247],[170,21596,21598,21609],{"className":21597,"style":16728},[251],[170,21599,21600,21603],{"style":17417},[170,21601],{"className":21602,"style":651},[259],[170,21604,21606],{"className":21605},[264,265,266,267],[170,21607,888],{"className":21608},[230,303,267],[170,21610,21611,21614],{"style":647},[170,21612],{"className":21613,"style":651},[259],[170,21615,21617],{"className":21616},[264,265,266,267],[170,21618,446],{"className":21619},[230,303,267],[170,21621,338],{"className":21622},[337],[170,21624,21626],{"className":21625},[247],[170,21627,21629],{"className":21628,"style":17447},[251],[170,21630],{},[170,21632,13047],{"className":21633},[230],[170,21635,21637,21640],{"className":21636},[230],[170,21638,12908],{"className":21639},[230,303],[170,21641,21643],{"className":21642},[307],[170,21644,21646,21677],{"className":21645},[242,243],[170,21647,21649,21674],{"className":21648},[247],[170,21650,21652,21663],{"className":21651,"style":16728},[251],[170,21653,21654,21657],{"style":17417},[170,21655],{"className":21656,"style":651},[259],[170,21658,21660],{"className":21659},[264,265,266,267],[170,21661,888],{"className":21662},[230,303,267],[170,21664,21665,21668],{"style":647},[170,21666],{"className":21667,"style":651},[259],[170,21669,21671],{"className":21670},[264,265,266,267],[170,21672,446],{"className":21673},[230,303,267],[170,21675,338],{"className":21676},[337],[170,21678,21680],{"className":21679},[247],[170,21681,21683],{"className":21682,"style":17447},[251],[170,21684],{},[170,21686,923],{"className":21687},[424],[170,21689,21691],{"className":21690},[14456],[170,21692,21694,21899],{"className":21693},[173],[170,21695,21697],{"className":21696},[177],[179,21698,21699],{"xmlns":181,"display":9588},[183,21700,21701,21896],{},[186,21702,21703,21709,21711,21717,21719,21721,21723,21725,21731,21733,21735,21741,21743,21755,21757,21760,21766,21768,21774,21776,21778,21780,21782,21788,21790,21792,21798,21800,21812,21814,21816,21822,21824,21830,21832,21834,21840,21842,21854,21856,21858,21870,21872,21874,21880,21882,21894],{},[199,21704,21705,21707],{},[202,21706,570],{},[202,21708,14157],{},[572,21710,909],{"stretchy":908},[199,21712,21713,21715],{},[202,21714,12908],{},[202,21716,888],{},[572,21718,914],{"separator":217},[202,21720,134],{},[572,21722,923],{"stretchy":908},[572,21724,926],{},[199,21726,21727,21729],{},[202,21728,7933],{},[202,21730,888],{},[572,21732,5884],{},[202,21734,19101],{},[199,21736,21737,21739],{},[202,21738,783],{},[202,21740,14157],{},[572,21742,909],{"stretchy":908},[199,21744,21745,21747],{},[202,21746,12908],{},[186,21748,21749,21751,21753],{},[202,21750,888],{},[572,21752,5884],{},[192,21754,194],{},[572,21756,923],{"stretchy":908},[610,21758],{"linebreak":21759},"newline",[199,21761,21762,21764],{},[202,21763,885],{},[202,21765,14157],{},[572,21767,909],{"stretchy":908},[199,21769,21770,21772],{},[202,21771,12908],{},[202,21773,888],{},[572,21775,914],{"separator":217},[202,21777,134],{},[572,21779,923],{"stretchy":908},[572,21781,926],{},[199,21783,21784,21786],{},[202,21785,7933],{},[202,21787,888],{},[572,21789,5884],{},[202,21791,19101],{},[199,21793,21794,21796],{},[202,21795,783],{},[202,21797,14157],{},[572,21799,909],{"stretchy":908},[199,21801,21802,21804],{},[202,21803,12908],{},[186,21805,21806,21808,21810],{},[202,21807,888],{},[572,21809,5884],{},[192,21811,194],{},[572,21813,923],{"stretchy":908},[572,21815,18471],{},[199,21817,21818,21820],{},[202,21819,783],{},[202,21821,14157],{},[572,21823,909],{"stretchy":908},[199,21825,21826,21828],{},[202,21827,12908],{},[202,21829,888],{},[572,21831,923],{"stretchy":908},[610,21833],{"linebreak":21759},[199,21835,21836,21838],{},[202,21837,783],{},[202,21839,14157],{},[572,21841,909],{"stretchy":908},[199,21843,21844,21846],{},[202,21845,12908],{},[186,21847,21848,21850,21852],{},[202,21849,888],{},[572,21851,5884],{},[192,21853,194],{},[572,21855,923],{"stretchy":908},[572,21857,14781],{},[199,21859,21860,21862],{},[202,21861,7933],{},[186,21863,21864,21866,21868],{},[202,21865,888],{},[572,21867,5884],{},[192,21869,194],{},[572,21871,5884],{},[202,21873,19101],{},[199,21875,21876,21878],{},[202,21877,783],{},[202,21879,14157],{},[572,21881,909],{"stretchy":908},[199,21883,21884,21886],{},[202,21885,12908],{},[186,21887,21888,21890,21892],{},[202,21889,888],{},[572,21891,5884],{},[192,21893,2928],{},[572,21895,923],{"stretchy":908},[209,21897,21898],{"encoding":211}," Q_\\theta(s_t, a) = r_t + \\gamma V_\\theta(s_{t+1}) \\\\\n\n A_\\theta(s_t, a) = r_t + \\gamma V_\\theta(s_{t+1}) - V_\\theta(s_t)\\\\\n V_\\theta(s_{t+1}) \\approx r_{t+1} + \\gamma V_\\theta(s_{t+2})",[170,21900,21902,22012,22068,22172,22175,22285,22340,22453,22545,22548,22658,22723],{"className":21901,"ariaHidden":217},[216],[170,21903,21905,21908,21948,21951,21991,21994,21997,22000,22003,22006,22009],{"className":21904},[221],[170,21906],{"className":21907,"style":988},[225],[170,21909,21911,21914],{"className":21910},[230],[170,21912,570],{"className":21913},[230,303],[170,21915,21917],{"className":21916},[307],[170,21918,21920,21940],{"className":21919},[242,243],[170,21921,21923,21937],{"className":21922},[247],[170,21924,21926],{"className":21925,"style":1359},[251],[170,21927,21928,21931],{"style":1362},[170,21929],{"className":21930,"style":651},[259],[170,21932,21934],{"className":21933},[264,265,266,267],[170,21935,14157],{"className":21936,"style":3317},[230,303,267],[170,21938,338],{"className":21939},[337],[170,21941,21943],{"className":21942},[247],[170,21944,21946],{"className":21945,"style":1381},[251],[170,21947],{},[170,21949,909],{"className":21950},[234],[170,21952,21954,21957],{"className":21953},[230],[170,21955,12908],{"className":21956},[230,303],[170,21958,21960],{"className":21959},[307],[170,21961,21963,21983],{"className":21962},[242,243],[170,21964,21966,21980],{"className":21965},[247],[170,21967,21969],{"className":21968,"style":12838},[251],[170,21970,21971,21974],{"style":1362},[170,21972],{"className":21973,"style":651},[259],[170,21975,21977],{"className":21976},[264,265,266,267],[170,21978,888],{"className":21979},[230,303,267],[170,21981,338],{"className":21982},[337],[170,21984,21986],{"className":21985},[247],[170,21987,21989],{"className":21988,"style":1381},[251],[170,21990],{},[170,21992,914],{"className":21993},[1023],[170,21995],{"className":21996,"style":1027},[610],[170,21998,134],{"className":21999},[230,303],[170,22001,923],{"className":22002},[424],[170,22004],{"className":22005,"style":611},[610],[170,22007,926],{"className":22008},[615],[170,22010],{"className":22011,"style":611},[610],[170,22013,22015,22019,22059,22062,22065],{"className":22014},[221],[170,22016],{"className":22017,"style":22018},[225],"height:0.7333em;vertical-align:-0.15em;",[170,22020,22022,22025],{"className":22021},[230],[170,22023,7933],{"className":22024,"style":3317},[230,303],[170,22026,22028],{"className":22027},[307],[170,22029,22031,22051],{"className":22030},[242,243],[170,22032,22034,22048],{"className":22033},[247],[170,22035,22037],{"className":22036,"style":12838},[251],[170,22038,22039,22042],{"style":13211},[170,22040],{"className":22041,"style":651},[259],[170,22043,22045],{"className":22044},[264,265,266,267],[170,22046,888],{"className":22047},[230,303,267],[170,22049,338],{"className":22050},[337],[170,22052,22054],{"className":22053},[247],[170,22055,22057],{"className":22056,"style":1381},[251],[170,22058],{},[170,22060],{"className":22061,"style":1983},[610],[170,22063,5884],{"className":22064},[664],[170,22066],{"className":22067,"style":1983},[610],[170,22069,22071,22074,22077,22117,22120,22169],{"className":22070},[221],[170,22072],{"className":22073,"style":988},[225],[170,22075,19101],{"className":22076,"style":19348},[230,303],[170,22078,22080,22083],{"className":22079},[230],[170,22081,783],{"className":22082,"style":813},[230,303],[170,22084,22086],{"className":22085},[307],[170,22087,22089,22109],{"className":22088},[242,243],[170,22090,22092,22106],{"className":22091},[247],[170,22093,22095],{"className":22094,"style":1359},[251],[170,22096,22097,22100],{"style":20611},[170,22098],{"className":22099,"style":651},[259],[170,22101,22103],{"className":22102},[264,265,266,267],[170,22104,14157],{"className":22105,"style":3317},[230,303,267],[170,22107,338],{"className":22108},[337],[170,22110,22112],{"className":22111},[247],[170,22113,22115],{"className":22114,"style":1381},[251],[170,22116],{},[170,22118,909],{"className":22119},[234],[170,22121,22123,22126],{"className":22122},[230],[170,22124,12908],{"className":22125},[230,303],[170,22127,22129],{"className":22128},[307],[170,22130,22132,22161],{"className":22131},[242,243],[170,22133,22135,22158],{"className":22134},[247],[170,22136,22138],{"className":22137,"style":3198},[251],[170,22139,22140,22143],{"style":1362},[170,22141],{"className":22142,"style":651},[259],[170,22144,22146],{"className":22145},[264,265,266,267],[170,22147,22149,22152,22155],{"className":22148},[230,267],[170,22150,888],{"className":22151},[230,303,267],[170,22153,5884],{"className":22154},[664,267],[170,22156,194],{"className":22157},[230,267],[170,22159,338],{"className":22160},[337],[170,22162,22164],{"className":22163},[247],[170,22165,22167],{"className":22166,"style":7045},[251],[170,22168],{},[170,22170,923],{"className":22171},[424],[170,22173],{"className":22174},[610,21759],[170,22176,22178,22181,22221,22224,22264,22267,22270,22273,22276,22279,22282],{"className":22177},[221],[170,22179],{"className":22180,"style":988},[225],[170,22182,22184,22187],{"className":22183},[230],[170,22185,885],{"className":22186},[230,303],[170,22188,22190],{"className":22189},[307],[170,22191,22193,22213],{"className":22192},[242,243],[170,22194,22196,22210],{"className":22195},[247],[170,22197,22199],{"className":22198,"style":1359},[251],[170,22200,22201,22204],{"style":1362},[170,22202],{"className":22203,"style":651},[259],[170,22205,22207],{"className":22206},[264,265,266,267],[170,22208,14157],{"className":22209,"style":3317},[230,303,267],[170,22211,338],{"className":22212},[337],[170,22214,22216],{"className":22215},[247],[170,22217,22219],{"className":22218,"style":1381},[251],[170,22220],{},[170,22222,909],{"className":22223},[234],[170,22225,22227,22230],{"className":22226},[230],[170,22228,12908],{"className":22229},[230,303],[170,22231,22233],{"className":22232},[307],[170,22234,22236,22256],{"className":22235},[242,243],[170,22237,22239,22253],{"className":22238},[247],[170,22240,22242],{"className":22241,"style":12838},[251],[170,22243,22244,22247],{"style":1362},[170,22245],{"className":22246,"style":651},[259],[170,22248,22250],{"className":22249},[264,265,266,267],[170,22251,888],{"className":22252},[230,303,267],[170,22254,338],{"className":22255},[337],[170,22257,22259],{"className":22258},[247],[170,22260,22262],{"className":22261,"style":1381},[251],[170,22263],{},[170,22265,914],{"className":22266},[1023],[170,22268],{"className":22269,"style":1027},[610],[170,22271,134],{"className":22272},[230,303],[170,22274,923],{"className":22275},[424],[170,22277],{"className":22278,"style":611},[610],[170,22280,926],{"className":22281},[615],[170,22283],{"className":22284,"style":611},[610],[170,22286,22288,22291,22331,22334,22337],{"className":22287},[221],[170,22289],{"className":22290,"style":22018},[225],[170,22292,22294,22297],{"className":22293},[230],[170,22295,7933],{"className":22296,"style":3317},[230,303],[170,22298,22300],{"className":22299},[307],[170,22301,22303,22323],{"className":22302},[242,243],[170,22304,22306,22320],{"className":22305},[247],[170,22307,22309],{"className":22308,"style":12838},[251],[170,22310,22311,22314],{"style":13211},[170,22312],{"className":22313,"style":651},[259],[170,22315,22317],{"className":22316},[264,265,266,267],[170,22318,888],{"className":22319},[230,303,267],[170,22321,338],{"className":22322},[337],[170,22324,22326],{"className":22325},[247],[170,22327,22329],{"className":22328,"style":1381},[251],[170,22330],{},[170,22332],{"className":22333,"style":1983},[610],[170,22335,5884],{"className":22336},[664],[170,22338],{"className":22339,"style":1983},[610],[170,22341,22343,22346,22349,22389,22392,22441,22444,22447,22450],{"className":22342},[221],[170,22344],{"className":22345,"style":988},[225],[170,22347,19101],{"className":22348,"style":19348},[230,303],[170,22350,22352,22355],{"className":22351},[230],[170,22353,783],{"className":22354,"style":813},[230,303],[170,22356,22358],{"className":22357},[307],[170,22359,22361,22381],{"className":22360},[242,243],[170,22362,22364,22378],{"className":22363},[247],[170,22365,22367],{"className":22366,"style":1359},[251],[170,22368,22369,22372],{"style":20611},[170,22370],{"className":22371,"style":651},[259],[170,22373,22375],{"className":22374},[264,265,266,267],[170,22376,14157],{"className":22377,"style":3317},[230,303,267],[170,22379,338],{"className":22380},[337],[170,22382,22384],{"className":22383},[247],[170,22385,22387],{"className":22386,"style":1381},[251],[170,22388],{},[170,22390,909],{"className":22391},[234],[170,22393,22395,22398],{"className":22394},[230],[170,22396,12908],{"className":22397},[230,303],[170,22399,22401],{"className":22400},[307],[170,22402,22404,22433],{"className":22403},[242,243],[170,22405,22407,22430],{"className":22406},[247],[170,22408,22410],{"className":22409,"style":3198},[251],[170,22411,22412,22415],{"style":1362},[170,22413],{"className":22414,"style":651},[259],[170,22416,22418],{"className":22417},[264,265,266,267],[170,22419,22421,22424,22427],{"className":22420},[230,267],[170,22422,888],{"className":22423},[230,303,267],[170,22425,5884],{"className":22426},[664,267],[170,22428,194],{"className":22429},[230,267],[170,22431,338],{"className":22432},[337],[170,22434,22436],{"className":22435},[247],[170,22437,22439],{"className":22438,"style":7045},[251],[170,22440],{},[170,22442,923],{"className":22443},[424],[170,22445],{"className":22446,"style":1983},[610],[170,22448,18471],{"className":22449},[664],[170,22451],{"className":22452,"style":1983},[610],[170,22454,22456,22459,22499,22502,22542],{"className":22455},[221],[170,22457],{"className":22458,"style":988},[225],[170,22460,22462,22465],{"className":22461},[230],[170,22463,783],{"className":22464,"style":813},[230,303],[170,22466,22468],{"className":22467},[307],[170,22469,22471,22491],{"className":22470},[242,243],[170,22472,22474,22488],{"className":22473},[247],[170,22475,22477],{"className":22476,"style":1359},[251],[170,22478,22479,22482],{"style":20611},[170,22480],{"className":22481,"style":651},[259],[170,22483,22485],{"className":22484},[264,265,266,267],[170,22486,14157],{"className":22487,"style":3317},[230,303,267],[170,22489,338],{"className":22490},[337],[170,22492,22494],{"className":22493},[247],[170,22495,22497],{"className":22496,"style":1381},[251],[170,22498],{},[170,22500,909],{"className":22501},[234],[170,22503,22505,22508],{"className":22504},[230],[170,22506,12908],{"className":22507},[230,303],[170,22509,22511],{"className":22510},[307],[170,22512,22514,22534],{"className":22513},[242,243],[170,22515,22517,22531],{"className":22516},[247],[170,22518,22520],{"className":22519,"style":12838},[251],[170,22521,22522,22525],{"style":1362},[170,22523],{"className":22524,"style":651},[259],[170,22526,22528],{"className":22527},[264,265,266,267],[170,22529,888],{"className":22530},[230,303,267],[170,22532,338],{"className":22533},[337],[170,22535,22537],{"className":22536},[247],[170,22538,22540],{"className":22539,"style":1381},[251],[170,22541],{},[170,22543,923],{"className":22544},[424],[170,22546],{"className":22547},[610,21759],[170,22549,22551,22554,22594,22597,22646,22649,22652,22655],{"className":22550},[221],[170,22552],{"className":22553,"style":988},[225],[170,22555,22557,22560],{"className":22556},[230],[170,22558,783],{"className":22559,"style":813},[230,303],[170,22561,22563],{"className":22562},[307],[170,22564,22566,22586],{"className":22565},[242,243],[170,22567,22569,22583],{"className":22568},[247],[170,22570,22572],{"className":22571,"style":1359},[251],[170,22573,22574,22577],{"style":20611},[170,22575],{"className":22576,"style":651},[259],[170,22578,22580],{"className":22579},[264,265,266,267],[170,22581,14157],{"className":22582,"style":3317},[230,303,267],[170,22584,338],{"className":22585},[337],[170,22587,22589],{"className":22588},[247],[170,22590,22592],{"className":22591,"style":1381},[251],[170,22593],{},[170,22595,909],{"className":22596},[234],[170,22598,22600,22603],{"className":22599},[230],[170,22601,12908],{"className":22602},[230,303],[170,22604,22606],{"className":22605},[307],[170,22607,22609,22638],{"className":22608},[242,243],[170,22610,22612,22635],{"className":22611},[247],[170,22613,22615],{"className":22614,"style":3198},[251],[170,22616,22617,22620],{"style":1362},[170,22618],{"className":22619,"style":651},[259],[170,22621,22623],{"className":22622},[264,265,266,267],[170,22624,22626,22629,22632],{"className":22625},[230,267],[170,22627,888],{"className":22628},[230,303,267],[170,22630,5884],{"className":22631},[664,267],[170,22633,194],{"className":22634},[230,267],[170,22636,338],{"className":22637},[337],[170,22639,22641],{"className":22640},[247],[170,22642,22644],{"className":22643,"style":7045},[251],[170,22645],{},[170,22647,923],{"className":22648},[424],[170,22650],{"className":22651,"style":611},[610],[170,22653,14781],{"className":22654},[615],[170,22656],{"className":22657,"style":611},[610],[170,22659,22661,22665,22714,22717,22720],{"className":22660},[221],[170,22662],{"className":22663,"style":22664},[225],"height:0.7917em;vertical-align:-0.2083em;",[170,22666,22668,22671],{"className":22667},[230],[170,22669,7933],{"className":22670,"style":3317},[230,303],[170,22672,22674],{"className":22673},[307],[170,22675,22677,22706],{"className":22676},[242,243],[170,22678,22680,22703],{"className":22679},[247],[170,22681,22683],{"className":22682,"style":3198},[251],[170,22684,22685,22688],{"style":13211},[170,22686],{"className":22687,"style":651},[259],[170,22689,22691],{"className":22690},[264,265,266,267],[170,22692,22694,22697,22700],{"className":22693},[230,267],[170,22695,888],{"className":22696},[230,303,267],[170,22698,5884],{"className":22699},[664,267],[170,22701,194],{"className":22702},[230,267],[170,22704,338],{"className":22705},[337],[170,22707,22709],{"className":22708},[247],[170,22710,22712],{"className":22711,"style":7045},[251],[170,22713],{},[170,22715],{"className":22716,"style":1983},[610],[170,22718,5884],{"className":22719},[664],[170,22721],{"className":22722,"style":1983},[610],[170,22724,22726,22729,22732,22772,22775,22824],{"className":22725},[221],[170,22727],{"className":22728,"style":988},[225],[170,22730,19101],{"className":22731,"style":19348},[230,303],[170,22733,22735,22738],{"className":22734},[230],[170,22736,783],{"className":22737,"style":813},[230,303],[170,22739,22741],{"className":22740},[307],[170,22742,22744,22764],{"className":22743},[242,243],[170,22745,22747,22761],{"className":22746},[247],[170,22748,22750],{"className":22749,"style":1359},[251],[170,22751,22752,22755],{"style":20611},[170,22753],{"className":22754,"style":651},[259],[170,22756,22758],{"className":22757},[264,265,266,267],[170,22759,14157],{"className":22760,"style":3317},[230,303,267],[170,22762,338],{"className":22763},[337],[170,22765,22767],{"className":22766},[247],[170,22768,22770],{"className":22769,"style":1381},[251],[170,22771],{},[170,22773,909],{"className":22774},[234],[170,22776,22778,22781],{"className":22777},[230],[170,22779,12908],{"className":22780},[230,303],[170,22782,22784],{"className":22783},[307],[170,22785,22787,22816],{"className":22786},[242,243],[170,22788,22790,22813],{"className":22789},[247],[170,22791,22793],{"className":22792,"style":3198},[251],[170,22794,22795,22798],{"style":1362},[170,22796],{"className":22797,"style":651},[259],[170,22799,22801],{"className":22800},[264,265,266,267],[170,22802,22804,22807,22810],{"className":22803},[230,267],[170,22805,888],{"className":22806},[230,303,267],[170,22808,5884],{"className":22809},[664,267],[170,22811,2928],{"className":22812},[230,267],[170,22814,338],{"className":22815},[337],[170,22817,22819],{"className":22818},[247],[170,22820,22822],{"className":22821,"style":7045},[251],[170,22823],{},[170,22825,923],{"className":22826},[424],[14,22828,22829,22830,22928],{},"通过上述公式变化，关于 ",[170,22831,22833,22861],{"className":22832},[173],[170,22834,22836],{"className":22835},[177],[179,22837,22838],{"xmlns":181},[183,22839,22840,22858],{},[186,22841,22842,22848,22850,22852,22854,22856],{},[199,22843,22844,22846],{},[202,22845,885],{},[202,22847,14157],{},[572,22849,909],{"stretchy":908},[202,22851,12908],{},[572,22853,914],{"separator":217},[202,22855,134],{},[572,22857,923],{"stretchy":908},[209,22859,22860],{"encoding":211},"A_\\theta(s,a)",[170,22862,22864],{"className":22863,"ariaHidden":217},[216],[170,22865,22867,22870,22910,22913,22916,22919,22922,22925],{"className":22866},[221],[170,22868],{"className":22869,"style":988},[225],[170,22871,22873,22876],{"className":22872},[230],[170,22874,885],{"className":22875},[230,303],[170,22877,22879],{"className":22878},[307],[170,22880,22882,22902],{"className":22881},[242,243],[170,22883,22885,22899],{"className":22884},[247],[170,22886,22888],{"className":22887,"style":1359},[251],[170,22889,22890,22893],{"style":1362},[170,22891],{"className":22892,"style":651},[259],[170,22894,22896],{"className":22895},[264,265,266,267],[170,22897,14157],{"className":22898,"style":3317},[230,303,267],[170,22900,338],{"className":22901},[337],[170,22903,22905],{"className":22904},[247],[170,22906,22908],{"className":22907,"style":1381},[251],[170,22909],{},[170,22911,909],{"className":22912},[234],[170,22914,12908],{"className":22915},[230,303],[170,22917,914],{"className":22918},[1023],[170,22920],{"className":22921,"style":1027},[610],[170,22923,134],{"className":22924},[230,303],[170,22926,923],{"className":22927},[424]," 的推到公式仅剩一个相关变量，可以降低公式的复杂性。",[14,22930,22931],{},"之后再根据采样步数进行估算。采样越多，情况越多，整体估计的偏差就会偏小，同样的，碰到极值的概率越大，方差就越大。",[170,22933,22935],{"className":22934},[14456],[170,22936,22938,23388],{"className":22937},[173],[170,22939,22941],{"className":22940},[177],[179,22942,22943],{"xmlns":181,"display":9588},[183,22944,22945,23385],{},[186,22946,22947,22955,22957,22963,22965,22967,22969,22971,22977,22979,22981,22987,22989,23001,23003,23005,23011,23013,23019,23021,23023,23031,23033,23039,23041,23043,23045,23047,23053,23055,23057,23069,23071,23077,23083,23085,23097,23099,23101,23107,23109,23115,23117,23119,23127,23129,23135,23137,23139,23141,23143,23149,23151,23153,23156,23168,23170,23176,23178,23190,23192,23198,23204,23206,23218,23220,23222,23228,23230,23236,23238,23240,23252,23254,23262,23264,23270,23272,23274,23276,23278,23284,23286,23288,23290,23302,23304,23310,23312,23324,23326,23332,23334,23346,23348,23351,23353,23359,23361,23367,23369,23375,23377,23383],{},[1933,22948,22949,22951,22953],{},[202,22950,885],{},[202,22952,14157],{},[192,22954,194],{},[572,22956,909],{"stretchy":908},[199,22958,22959,22961],{},[202,22960,12908],{},[202,22962,888],{},[572,22964,914],{"separator":217},[202,22966,134],{},[572,22968,923],{"stretchy":908},[572,22970,926],{},[199,22972,22973,22975],{},[202,22974,7933],{},[202,22976,888],{},[572,22978,5884],{},[202,22980,19101],{},[199,22982,22983,22985],{},[202,22984,783],{},[202,22986,14157],{},[572,22988,909],{"stretchy":908},[199,22990,22991,22993],{},[202,22992,12908],{},[186,22994,22995,22997,22999],{},[202,22996,888],{},[572,22998,5884],{},[192,23000,194],{},[572,23002,923],{"stretchy":908},[572,23004,18471],{},[199,23006,23007,23009],{},[202,23008,783],{},[202,23010,14157],{},[572,23012,909],{"stretchy":908},[199,23014,23015,23017],{},[202,23016,12908],{},[202,23018,888],{},[572,23020,923],{"stretchy":908},[610,23022],{"linebreak":21759},[1933,23024,23025,23027,23029],{},[202,23026,885],{},[202,23028,14157],{},[192,23030,2928],{},[572,23032,909],{"stretchy":908},[199,23034,23035,23037],{},[202,23036,12908],{},[202,23038,888],{},[572,23040,914],{"separator":217},[202,23042,134],{},[572,23044,923],{"stretchy":908},[572,23046,926],{},[199,23048,23049,23051],{},[202,23050,7933],{},[202,23052,888],{},[572,23054,5884],{},[202,23056,19101],{},[199,23058,23059,23061],{},[202,23060,7933],{},[186,23062,23063,23065,23067],{},[202,23064,888],{},[572,23066,5884],{},[192,23068,194],{},[572,23070,5884],{},[576,23072,23073,23075],{},[202,23074,19101],{},[192,23076,2928],{},[199,23078,23079,23081],{},[202,23080,783],{},[202,23082,14157],{},[572,23084,909],{"stretchy":908},[199,23086,23087,23089],{},[202,23088,12908],{},[186,23090,23091,23093,23095],{},[202,23092,888],{},[572,23094,5884],{},[192,23096,2928],{},[572,23098,923],{"stretchy":908},[572,23100,18471],{},[199,23102,23103,23105],{},[202,23104,783],{},[202,23106,14157],{},[572,23108,909],{"stretchy":908},[199,23110,23111,23113],{},[202,23112,12908],{},[202,23114,888],{},[572,23116,923],{"stretchy":908},[610,23118],{"linebreak":21759},[1933,23120,23121,23123,23125],{},[202,23122,885],{},[202,23124,14157],{},[192,23126,4778],{},[572,23128,909],{"stretchy":908},[199,23130,23131,23133],{},[202,23132,12908],{},[202,23134,888],{},[572,23136,914],{"separator":217},[202,23138,134],{},[572,23140,923],{"stretchy":908},[572,23142,926],{},[199,23144,23145,23147],{},[202,23146,7933],{},[202,23148,888],{},[572,23150,5884],{},[202,23152,19101],{},[572,23154,23155],{},"∗",[199,23157,23158,23160],{},[202,23159,7933],{},[186,23161,23162,23164,23166],{},[202,23163,888],{},[572,23165,5884],{},[192,23167,194],{},[572,23169,5884],{},[576,23171,23172,23174],{},[202,23173,19101],{},[192,23175,2928],{},[572,23177,23155],{},[199,23179,23180,23182],{},[202,23181,7933],{},[186,23183,23184,23186,23188],{},[202,23185,888],{},[572,23187,5884],{},[192,23189,2928],{},[572,23191,5884],{},[576,23193,23194,23196],{},[202,23195,19101],{},[192,23197,4778],{},[199,23199,23200,23202],{},[202,23201,783],{},[202,23203,14157],{},[572,23205,909],{"stretchy":908},[199,23207,23208,23210],{},[202,23209,12908],{},[186,23211,23212,23214,23216],{},[202,23213,888],{},[572,23215,5884],{},[192,23217,4778],{},[572,23219,923],{"stretchy":908},[572,23221,18471],{},[199,23223,23224,23226],{},[202,23225,783],{},[202,23227,14157],{},[572,23229,909],{"stretchy":908},[199,23231,23232,23234],{},[202,23233,12908],{},[202,23235,888],{},[572,23237,923],{"stretchy":908},[610,23239],{"linebreak":21759},[186,23241,23242,23245],{},[202,23243,23244],{"mathvariant":929},"⋮",[23246,23247,23248],"mpadded",{"height":14472,"voffset":14472},[610,23249],{"mathbackground":23250,"width":14472,"height":23251},"black","1.5em",[610,23253],{"linebreak":21759},[1933,23255,23256,23258,23260],{},[202,23257,885],{},[202,23259,14157],{},[202,23261,13348],{},[572,23263,909],{"stretchy":908},[199,23265,23266,23268],{},[202,23267,12908],{},[202,23269,888],{},[572,23271,914],{"separator":217},[202,23273,134],{},[572,23275,923],{"stretchy":908},[572,23277,926],{},[199,23279,23280,23282],{},[202,23281,7933],{},[202,23283,888],{},[572,23285,5884],{},[202,23287,19101],{},[572,23289,23155],{},[199,23291,23292,23294],{},[202,23293,7933],{},[186,23295,23296,23298,23300],{},[202,23297,888],{},[572,23299,5884],{},[192,23301,194],{},[572,23303,5884],{},[576,23305,23306,23308],{},[202,23307,19101],{},[192,23309,2928],{},[572,23311,23155],{},[199,23313,23314,23316],{},[202,23315,7933],{},[186,23317,23318,23320,23322],{},[202,23319,888],{},[572,23321,5884],{},[192,23323,2928],{},[572,23325,5884],{},[576,23327,23328,23330],{},[202,23329,19101],{},[192,23331,4778],{},[572,23333,23155],{},[199,23335,23336,23338],{},[202,23337,7933],{},[186,23339,23340,23342,23344],{},[202,23341,888],{},[572,23343,5884],{},[192,23345,4778],{},[572,23347,5884],{},[572,23349,23350],{},"⋯",[572,23352,5884],{},[576,23354,23355,23357],{},[202,23356,19101],{},[202,23358,13348],{},[572,23360,23155],{},[199,23362,23363,23365],{},[202,23364,7933],{},[202,23366,13348],{},[572,23368,18471],{},[199,23370,23371,23373],{},[202,23372,783],{},[202,23374,14157],{},[572,23376,909],{"stretchy":908},[199,23378,23379,23381],{},[202,23380,12908],{},[202,23382,888],{},[572,23384,923],{"stretchy":908},[209,23386,23387],{"encoding":211}," A_\\theta^1(s_t, a) = r_t + \\gamma V_\\theta(s_{t+1}) - V_\\theta(s_t)\\\\\n A_\\theta^2(s_t, a) = r_t + \\gamma r_{t+1} + \\gamma^2 V_\\theta(s_{t+2}) - V_\\theta(s_t)\\\\\n A_\\theta^3(s_t, a) = r_t + \\gamma * r_{t+1} + \\gamma^2 * r_{t+2} + \\gamma^3 V_\\theta(s_{t+3}) - V_\\theta(s_t)\\\\\n \\vdots \\\\\n A_\\theta^T(s_t, a) = r_t + \\gamma * r_{t+1} + \\gamma^2 * r_{t+2} + \\gamma^3 * r_{t+3} + \\cdots + \\gamma^T * r_T - V_\\theta(s_t)",[170,23389,23391,23514,23569,23682,23774,23777,23898,23953,24020,24159,24251,24254,24375,24430,24449,24513,24558,24622,24761,24853,24856,24874,24877,25000,25055,25073,25137,25181,25245,25289,25353,25372,25417,25472],{"className":23390,"ariaHidden":217},[216],[170,23392,23394,23398,23450,23453,23493,23496,23499,23502,23505,23508,23511],{"className":23393},[221],[170,23395],{"className":23396,"style":23397},[225],"height:1.1141em;vertical-align:-0.25em;",[170,23399,23401,23404],{"className":23400},[230],[170,23402,885],{"className":23403},[230,303],[170,23405,23407],{"className":23406},[307],[170,23408,23410,23442],{"className":23409},[242,243],[170,23411,23413,23439],{"className":23412},[247],[170,23414,23417,23428],{"className":23415,"style":23416},[251],"height:0.8641em;",[170,23418,23419,23422],{"style":17417},[170,23420],{"className":23421,"style":651},[259],[170,23423,23425],{"className":23424},[264,265,266,267],[170,23426,14157],{"className":23427,"style":3317},[230,303,267],[170,23429,23430,23433],{"style":16532},[170,23431],{"className":23432,"style":651},[259],[170,23434,23436],{"className":23435},[264,265,266,267],[170,23437,194],{"className":23438},[230,267],[170,23440,338],{"className":23441},[337],[170,23443,23445],{"className":23444},[247],[170,23446,23448],{"className":23447,"style":17447},[251],[170,23449],{},[170,23451,909],{"className":23452},[234],[170,23454,23456,23459],{"className":23455},[230],[170,23457,12908],{"className":23458},[230,303],[170,23460,23462],{"className":23461},[307],[170,23463,23465,23485],{"className":23464},[242,243],[170,23466,23468,23482],{"className":23467},[247],[170,23469,23471],{"className":23470,"style":12838},[251],[170,23472,23473,23476],{"style":1362},[170,23474],{"className":23475,"style":651},[259],[170,23477,23479],{"className":23478},[264,265,266,267],[170,23480,888],{"className":23481},[230,303,267],[170,23483,338],{"className":23484},[337],[170,23486,23488],{"className":23487},[247],[170,23489,23491],{"className":23490,"style":1381},[251],[170,23492],{},[170,23494,914],{"className":23495},[1023],[170,23497],{"className":23498,"style":1027},[610],[170,23500,134],{"className":23501},[230,303],[170,23503,923],{"className":23504},[424],[170,23506],{"className":23507,"style":611},[610],[170,23509,926],{"className":23510},[615],[170,23512],{"className":23513,"style":611},[610],[170,23515,23517,23520,23560,23563,23566],{"className":23516},[221],[170,23518],{"className":23519,"style":22018},[225],[170,23521,23523,23526],{"className":23522},[230],[170,23524,7933],{"className":23525,"style":3317},[230,303],[170,23527,23529],{"className":23528},[307],[170,23530,23532,23552],{"className":23531},[242,243],[170,23533,23535,23549],{"className":23534},[247],[170,23536,23538],{"className":23537,"style":12838},[251],[170,23539,23540,23543],{"style":13211},[170,23541],{"className":23542,"style":651},[259],[170,23544,23546],{"className":23545},[264,265,266,267],[170,23547,888],{"className":23548},[230,303,267],[170,23550,338],{"className":23551},[337],[170,23553,23555],{"className":23554},[247],[170,23556,23558],{"className":23557,"style":1381},[251],[170,23559],{},[170,23561],{"className":23562,"style":1983},[610],[170,23564,5884],{"className":23565},[664],[170,23567],{"className":23568,"style":1983},[610],[170,23570,23572,23575,23578,23618,23621,23670,23673,23676,23679],{"className":23571},[221],[170,23573],{"className":23574,"style":988},[225],[170,23576,19101],{"className":23577,"style":19348},[230,303],[170,23579,23581,23584],{"className":23580},[230],[170,23582,783],{"className":23583,"style":813},[230,303],[170,23585,23587],{"className":23586},[307],[170,23588,23590,23610],{"className":23589},[242,243],[170,23591,23593,23607],{"className":23592},[247],[170,23594,23596],{"className":23595,"style":1359},[251],[170,23597,23598,23601],{"style":20611},[170,23599],{"className":23600,"style":651},[259],[170,23602,23604],{"className":23603},[264,265,266,267],[170,23605,14157],{"className":23606,"style":3317},[230,303,267],[170,23608,338],{"className":23609},[337],[170,23611,23613],{"className":23612},[247],[170,23614,23616],{"className":23615,"style":1381},[251],[170,23617],{},[170,23619,909],{"className":23620},[234],[170,23622,23624,23627],{"className":23623},[230],[170,23625,12908],{"className":23626},[230,303],[170,23628,23630],{"className":23629},[307],[170,23631,23633,23662],{"className":23632},[242,243],[170,23634,23636,23659],{"className":23635},[247],[170,23637,23639],{"className":23638,"style":3198},[251],[170,23640,23641,23644],{"style":1362},[170,23642],{"className":23643,"style":651},[259],[170,23645,23647],{"className":23646},[264,265,266,267],[170,23648,23650,23653,23656],{"className":23649},[230,267],[170,23651,888],{"className":23652},[230,303,267],[170,23654,5884],{"className":23655},[664,267],[170,23657,194],{"className":23658},[230,267],[170,23660,338],{"className":23661},[337],[170,23663,23665],{"className":23664},[247],[170,23666,23668],{"className":23667,"style":7045},[251],[170,23669],{},[170,23671,923],{"className":23672},[424],[170,23674],{"className":23675,"style":1983},[610],[170,23677,18471],{"className":23678},[664],[170,23680],{"className":23681,"style":1983},[610],[170,23683,23685,23688,23728,23731,23771],{"className":23684},[221],[170,23686],{"className":23687,"style":988},[225],[170,23689,23691,23694],{"className":23690},[230],[170,23692,783],{"className":23693,"style":813},[230,303],[170,23695,23697],{"className":23696},[307],[170,23698,23700,23720],{"className":23699},[242,243],[170,23701,23703,23717],{"className":23702},[247],[170,23704,23706],{"className":23705,"style":1359},[251],[170,23707,23708,23711],{"style":20611},[170,23709],{"className":23710,"style":651},[259],[170,23712,23714],{"className":23713},[264,265,266,267],[170,23715,14157],{"className":23716,"style":3317},[230,303,267],[170,23718,338],{"className":23719},[337],[170,23721,23723],{"className":23722},[247],[170,23724,23726],{"className":23725,"style":1381},[251],[170,23727],{},[170,23729,909],{"className":23730},[234],[170,23732,23734,23737],{"className":23733},[230],[170,23735,12908],{"className":23736},[230,303],[170,23738,23740],{"className":23739},[307],[170,23741,23743,23763],{"className":23742},[242,243],[170,23744,23746,23760],{"className":23745},[247],[170,23747,23749],{"className":23748,"style":12838},[251],[170,23750,23751,23754],{"style":1362},[170,23752],{"className":23753,"style":651},[259],[170,23755,23757],{"className":23756},[264,265,266,267],[170,23758,888],{"className":23759},[230,303,267],[170,23761,338],{"className":23762},[337],[170,23764,23766],{"className":23765},[247],[170,23767,23769],{"className":23768,"style":1381},[251],[170,23770],{},[170,23772,923],{"className":23773},[424],[170,23775],{"className":23776},[610,21759],[170,23778,23780,23783,23834,23837,23877,23880,23883,23886,23889,23892,23895],{"className":23779},[221],[170,23781],{"className":23782,"style":23397},[225],[170,23784,23786,23789],{"className":23785},[230],[170,23787,885],{"className":23788},[230,303],[170,23790,23792],{"className":23791},[307],[170,23793,23795,23826],{"className":23794},[242,243],[170,23796,23798,23823],{"className":23797},[247],[170,23799,23801,23812],{"className":23800,"style":23416},[251],[170,23802,23803,23806],{"style":17417},[170,23804],{"className":23805,"style":651},[259],[170,23807,23809],{"className":23808},[264,265,266,267],[170,23810,14157],{"className":23811,"style":3317},[230,303,267],[170,23813,23814,23817],{"style":16532},[170,23815],{"className":23816,"style":651},[259],[170,23818,23820],{"className":23819},[264,265,266,267],[170,23821,2928],{"className":23822},[230,267],[170,23824,338],{"className":23825},[337],[170,23827,23829],{"className":23828},[247],[170,23830,23832],{"className":23831,"style":17447},[251],[170,23833],{},[170,23835,909],{"className":23836},[234],[170,23838,23840,23843],{"className":23839},[230],[170,23841,12908],{"className":23842},[230,303],[170,23844,23846],{"className":23845},[307],[170,23847,23849,23869],{"className":23848},[242,243],[170,23850,23852,23866],{"className":23851},[247],[170,23853,23855],{"className":23854,"style":12838},[251],[170,23856,23857,23860],{"style":1362},[170,23858],{"className":23859,"style":651},[259],[170,23861,23863],{"className":23862},[264,265,266,267],[170,23864,888],{"className":23865},[230,303,267],[170,23867,338],{"className":23868},[337],[170,23870,23872],{"className":23871},[247],[170,23873,23875],{"className":23874,"style":1381},[251],[170,23876],{},[170,23878,914],{"className":23879},[1023],[170,23881],{"className":23882,"style":1027},[610],[170,23884,134],{"className":23885},[230,303],[170,23887,923],{"className":23888},[424],[170,23890],{"className":23891,"style":611},[610],[170,23893,926],{"className":23894},[615],[170,23896],{"className":23897,"style":611},[610],[170,23899,23901,23904,23944,23947,23950],{"className":23900},[221],[170,23902],{"className":23903,"style":22018},[225],[170,23905,23907,23910],{"className":23906},[230],[170,23908,7933],{"className":23909,"style":3317},[230,303],[170,23911,23913],{"className":23912},[307],[170,23914,23916,23936],{"className":23915},[242,243],[170,23917,23919,23933],{"className":23918},[247],[170,23920,23922],{"className":23921,"style":12838},[251],[170,23923,23924,23927],{"style":13211},[170,23925],{"className":23926,"style":651},[259],[170,23928,23930],{"className":23929},[264,265,266,267],[170,23931,888],{"className":23932},[230,303,267],[170,23934,338],{"className":23935},[337],[170,23937,23939],{"className":23938},[247],[170,23940,23942],{"className":23941,"style":1381},[251],[170,23943],{},[170,23945],{"className":23946,"style":1983},[610],[170,23948,5884],{"className":23949},[664],[170,23951],{"className":23952,"style":1983},[610],[170,23954,23956,23959,23962,24011,24014,24017],{"className":23955},[221],[170,23957],{"className":23958,"style":22664},[225],[170,23960,19101],{"className":23961,"style":19348},[230,303],[170,23963,23965,23968],{"className":23964},[230],[170,23966,7933],{"className":23967,"style":3317},[230,303],[170,23969,23971],{"className":23970},[307],[170,23972,23974,24003],{"className":23973},[242,243],[170,23975,23977,24000],{"className":23976},[247],[170,23978,23980],{"className":23979,"style":3198},[251],[170,23981,23982,23985],{"style":13211},[170,23983],{"className":23984,"style":651},[259],[170,23986,23988],{"className":23987},[264,265,266,267],[170,23989,23991,23994,23997],{"className":23990},[230,267],[170,23992,888],{"className":23993},[230,303,267],[170,23995,5884],{"className":23996},[664,267],[170,23998,194],{"className":23999},[230,267],[170,24001,338],{"className":24002},[337],[170,24004,24006],{"className":24005},[247],[170,24007,24009],{"className":24008,"style":7045},[251],[170,24010],{},[170,24012],{"className":24013,"style":1983},[610],[170,24015,5884],{"className":24016},[664],[170,24018],{"className":24019,"style":1983},[610],[170,24021,24023,24026,24055,24095,24098,24147,24150,24153,24156],{"className":24022},[221],[170,24024],{"className":24025,"style":23397},[225],[170,24027,24029,24032],{"className":24028},[230],[170,24030,19101],{"className":24031,"style":19348},[230,303],[170,24033,24035],{"className":24034},[307],[170,24036,24038],{"className":24037},[242],[170,24039,24041],{"className":24040},[247],[170,24042,24044],{"className":24043,"style":23416},[251],[170,24045,24046,24049],{"style":16532},[170,24047],{"className":24048,"style":651},[259],[170,24050,24052],{"className":24051},[264,265,266,267],[170,24053,2928],{"className":24054},[230,267],[170,24056,24058,24061],{"className":24057},[230],[170,24059,783],{"className":24060,"style":813},[230,303],[170,24062,24064],{"className":24063},[307],[170,24065,24067,24087],{"className":24066},[242,243],[170,24068,24070,24084],{"className":24069},[247],[170,24071,24073],{"className":24072,"style":1359},[251],[170,24074,24075,24078],{"style":20611},[170,24076],{"className":24077,"style":651},[259],[170,24079,24081],{"className":24080},[264,265,266,267],[170,24082,14157],{"className":24083,"style":3317},[230,303,267],[170,24085,338],{"className":24086},[337],[170,24088,24090],{"className":24089},[247],[170,24091,24093],{"className":24092,"style":1381},[251],[170,24094],{},[170,24096,909],{"className":24097},[234],[170,24099,24101,24104],{"className":24100},[230],[170,24102,12908],{"className":24103},[230,303],[170,24105,24107],{"className":24106},[307],[170,24108,24110,24139],{"className":24109},[242,243],[170,24111,24113,24136],{"className":24112},[247],[170,24114,24116],{"className":24115,"style":3198},[251],[170,24117,24118,24121],{"style":1362},[170,24119],{"className":24120,"style":651},[259],[170,24122,24124],{"className":24123},[264,265,266,267],[170,24125,24127,24130,24133],{"className":24126},[230,267],[170,24128,888],{"className":24129},[230,303,267],[170,24131,5884],{"className":24132},[664,267],[170,24134,2928],{"className":24135},[230,267],[170,24137,338],{"className":24138},[337],[170,24140,24142],{"className":24141},[247],[170,24143,24145],{"className":24144,"style":7045},[251],[170,24146],{},[170,24148,923],{"className":24149},[424],[170,24151],{"className":24152,"style":1983},[610],[170,24154,18471],{"className":24155},[664],[170,24157],{"className":24158,"style":1983},[610],[170,24160,24162,24165,24205,24208,24248],{"className":24161},[221],[170,24163],{"className":24164,"style":988},[225],[170,24166,24168,24171],{"className":24167},[230],[170,24169,783],{"className":24170,"style":813},[230,303],[170,24172,24174],{"className":24173},[307],[170,24175,24177,24197],{"className":24176},[242,243],[170,24178,24180,24194],{"className":24179},[247],[170,24181,24183],{"className":24182,"style":1359},[251],[170,24184,24185,24188],{"style":20611},[170,24186],{"className":24187,"style":651},[259],[170,24189,24191],{"className":24190},[264,265,266,267],[170,24192,14157],{"className":24193,"style":3317},[230,303,267],[170,24195,338],{"className":24196},[337],[170,24198,24200],{"className":24199},[247],[170,24201,24203],{"className":24202,"style":1381},[251],[170,24204],{},[170,24206,909],{"className":24207},[234],[170,24209,24211,24214],{"className":24210},[230],[170,24212,12908],{"className":24213},[230,303],[170,24215,24217],{"className":24216},[307],[170,24218,24220,24240],{"className":24219},[242,243],[170,24221,24223,24237],{"className":24222},[247],[170,24224,24226],{"className":24225,"style":12838},[251],[170,24227,24228,24231],{"style":1362},[170,24229],{"className":24230,"style":651},[259],[170,24232,24234],{"className":24233},[264,265,266,267],[170,24235,888],{"className":24236},[230,303,267],[170,24238,338],{"className":24239},[337],[170,24241,24243],{"className":24242},[247],[170,24244,24246],{"className":24245,"style":1381},[251],[170,24247],{},[170,24249,923],{"className":24250},[424],[170,24252],{"className":24253},[610,21759],[170,24255,24257,24260,24311,24314,24354,24357,24360,24363,24366,24369,24372],{"className":24256},[221],[170,24258],{"className":24259,"style":23397},[225],[170,24261,24263,24266],{"className":24262},[230],[170,24264,885],{"className":24265},[230,303],[170,24267,24269],{"className":24268},[307],[170,24270,24272,24303],{"className":24271},[242,243],[170,24273,24275,24300],{"className":24274},[247],[170,24276,24278,24289],{"className":24277,"style":23416},[251],[170,24279,24280,24283],{"style":17417},[170,24281],{"className":24282,"style":651},[259],[170,24284,24286],{"className":24285},[264,265,266,267],[170,24287,14157],{"className":24288,"style":3317},[230,303,267],[170,24290,24291,24294],{"style":16532},[170,24292],{"className":24293,"style":651},[259],[170,24295,24297],{"className":24296},[264,265,266,267],[170,24298,4778],{"className":24299},[230,267],[170,24301,338],{"className":24302},[337],[170,24304,24306],{"className":24305},[247],[170,24307,24309],{"className":24308,"style":17447},[251],[170,24310],{},[170,24312,909],{"className":24313},[234],[170,24315,24317,24320],{"className":24316},[230],[170,24318,12908],{"className":24319},[230,303],[170,24321,24323],{"className":24322},[307],[170,24324,24326,24346],{"className":24325},[242,243],[170,24327,24329,24343],{"className":24328},[247],[170,24330,24332],{"className":24331,"style":12838},[251],[170,24333,24334,24337],{"style":1362},[170,24335],{"className":24336,"style":651},[259],[170,24338,24340],{"className":24339},[264,265,266,267],[170,24341,888],{"className":24342},[230,303,267],[170,24344,338],{"className":24345},[337],[170,24347,24349],{"className":24348},[247],[170,24350,24352],{"className":24351,"style":1381},[251],[170,24353],{},[170,24355,914],{"className":24356},[1023],[170,24358],{"className":24359,"style":1027},[610],[170,24361,134],{"className":24362},[230,303],[170,24364,923],{"className":24365},[424],[170,24367],{"className":24368,"style":611},[610],[170,24370,926],{"className":24371},[615],[170,24373],{"className":24374,"style":611},[610],[170,24376,24378,24381,24421,24424,24427],{"className":24377},[221],[170,24379],{"className":24380,"style":22018},[225],[170,24382,24384,24387],{"className":24383},[230],[170,24385,7933],{"className":24386,"style":3317},[230,303],[170,24388,24390],{"className":24389},[307],[170,24391,24393,24413],{"className":24392},[242,243],[170,24394,24396,24410],{"className":24395},[247],[170,24397,24399],{"className":24398,"style":12838},[251],[170,24400,24401,24404],{"style":13211},[170,24402],{"className":24403,"style":651},[259],[170,24405,24407],{"className":24406},[264,265,266,267],[170,24408,888],{"className":24409},[230,303,267],[170,24411,338],{"className":24412},[337],[170,24414,24416],{"className":24415},[247],[170,24417,24419],{"className":24418,"style":1381},[251],[170,24420],{},[170,24422],{"className":24423,"style":1983},[610],[170,24425,5884],{"className":24426},[664],[170,24428],{"className":24429,"style":1983},[610],[170,24431,24433,24437,24440,24443,24446],{"className":24432},[221],[170,24434],{"className":24435,"style":24436},[225],"height:0.6597em;vertical-align:-0.1944em;",[170,24438,19101],{"className":24439,"style":19348},[230,303],[170,24441],{"className":24442,"style":1983},[610],[170,24444,23155],{"className":24445},[664],[170,24447],{"className":24448,"style":1983},[610],[170,24450,24452,24455,24504,24507,24510],{"className":24451},[221],[170,24453],{"className":24454,"style":22664},[225],[170,24456,24458,24461],{"className":24457},[230],[170,24459,7933],{"className":24460,"style":3317},[230,303],[170,24462,24464],{"className":24463},[307],[170,24465,24467,24496],{"className":24466},[242,243],[170,24468,24470,24493],{"className":24469},[247],[170,24471,24473],{"className":24472,"style":3198},[251],[170,24474,24475,24478],{"style":13211},[170,24476],{"className":24477,"style":651},[259],[170,24479,24481],{"className":24480},[264,265,266,267],[170,24482,24484,24487,24490],{"className":24483},[230,267],[170,24485,888],{"className":24486},[230,303,267],[170,24488,5884],{"className":24489},[664,267],[170,24491,194],{"className":24492},[230,267],[170,24494,338],{"className":24495},[337],[170,24497,24499],{"className":24498},[247],[170,24500,24502],{"className":24501,"style":7045},[251],[170,24503],{},[170,24505],{"className":24506,"style":1983},[610],[170,24508,5884],{"className":24509},[664],[170,24511],{"className":24512,"style":1983},[610],[170,24514,24516,24520,24549,24552,24555],{"className":24515},[221],[170,24517],{"className":24518,"style":24519},[225],"height:1.0585em;vertical-align:-0.1944em;",[170,24521,24523,24526],{"className":24522},[230],[170,24524,19101],{"className":24525,"style":19348},[230,303],[170,24527,24529],{"className":24528},[307],[170,24530,24532],{"className":24531},[242],[170,24533,24535],{"className":24534},[247],[170,24536,24538],{"className":24537,"style":23416},[251],[170,24539,24540,24543],{"style":16532},[170,24541],{"className":24542,"style":651},[259],[170,24544,24546],{"className":24545},[264,265,266,267],[170,24547,2928],{"className":24548},[230,267],[170,24550],{"className":24551,"style":1983},[610],[170,24553,23155],{"className":24554},[664],[170,24556],{"className":24557,"style":1983},[610],[170,24559,24561,24564,24613,24616,24619],{"className":24560},[221],[170,24562],{"className":24563,"style":22664},[225],[170,24565,24567,24570],{"className":24566},[230],[170,24568,7933],{"className":24569,"style":3317},[230,303],[170,24571,24573],{"className":24572},[307],[170,24574,24576,24605],{"className":24575},[242,243],[170,24577,24579,24602],{"className":24578},[247],[170,24580,24582],{"className":24581,"style":3198},[251],[170,24583,24584,24587],{"style":13211},[170,24585],{"className":24586,"style":651},[259],[170,24588,24590],{"className":24589},[264,265,266,267],[170,24591,24593,24596,24599],{"className":24592},[230,267],[170,24594,888],{"className":24595},[230,303,267],[170,24597,5884],{"className":24598},[664,267],[170,24600,2928],{"className":24601},[230,267],[170,24603,338],{"className":24604},[337],[170,24606,24608],{"className":24607},[247],[170,24609,24611],{"className":24610,"style":7045},[251],[170,24612],{},[170,24614],{"className":24615,"style":1983},[610],[170,24617,5884],{"className":24618},[664],[170,24620],{"className":24621,"style":1983},[610],[170,24623,24625,24628,24657,24697,24700,24749,24752,24755,24758],{"className":24624},[221],[170,24626],{"className":24627,"style":23397},[225],[170,24629,24631,24634],{"className":24630},[230],[170,24632,19101],{"className":24633,"style":19348},[230,303],[170,24635,24637],{"className":24636},[307],[170,24638,24640],{"className":24639},[242],[170,24641,24643],{"className":24642},[247],[170,24644,24646],{"className":24645,"style":23416},[251],[170,24647,24648,24651],{"style":16532},[170,24649],{"className":24650,"style":651},[259],[170,24652,24654],{"className":24653},[264,265,266,267],[170,24655,4778],{"className":24656},[230,267],[170,24658,24660,24663],{"className":24659},[230],[170,24661,783],{"className":24662,"style":813},[230,303],[170,24664,24666],{"className":24665},[307],[170,24667,24669,24689],{"className":24668},[242,243],[170,24670,24672,24686],{"className":24671},[247],[170,24673,24675],{"className":24674,"style":1359},[251],[170,24676,24677,24680],{"style":20611},[170,24678],{"className":24679,"style":651},[259],[170,24681,24683],{"className":24682},[264,265,266,267],[170,24684,14157],{"className":24685,"style":3317},[230,303,267],[170,24687,338],{"className":24688},[337],[170,24690,24692],{"className":24691},[247],[170,24693,24695],{"className":24694,"style":1381},[251],[170,24696],{},[170,24698,909],{"className":24699},[234],[170,24701,24703,24706],{"className":24702},[230],[170,24704,12908],{"className":24705},[230,303],[170,24707,24709],{"className":24708},[307],[170,24710,24712,24741],{"className":24711},[242,243],[170,24713,24715,24738],{"className":24714},[247],[170,24716,24718],{"className":24717,"style":3198},[251],[170,24719,24720,24723],{"style":1362},[170,24721],{"className":24722,"style":651},[259],[170,24724,24726],{"className":24725},[264,265,266,267],[170,24727,24729,24732,24735],{"className":24728},[230,267],[170,24730,888],{"className":24731},[230,303,267],[170,24733,5884],{"className":24734},[664,267],[170,24736,4778],{"className":24737},[230,267],[170,24739,338],{"className":24740},[337],[170,24742,24744],{"className":24743},[247],[170,24745,24747],{"className":24746,"style":7045},[251],[170,24748],{},[170,24750,923],{"className":24751},[424],[170,24753],{"className":24754,"style":1983},[610],[170,24756,18471],{"className":24757},[664],[170,24759],{"className":24760,"style":1983},[610],[170,24762,24764,24767,24807,24810,24850],{"className":24763},[221],[170,24765],{"className":24766,"style":988},[225],[170,24768,24770,24773],{"className":24769},[230],[170,24771,783],{"className":24772,"style":813},[230,303],[170,24774,24776],{"className":24775},[307],[170,24777,24779,24799],{"className":24778},[242,243],[170,24780,24782,24796],{"className":24781},[247],[170,24783,24785],{"className":24784,"style":1359},[251],[170,24786,24787,24790],{"style":20611},[170,24788],{"className":24789,"style":651},[259],[170,24791,24793],{"className":24792},[264,265,266,267],[170,24794,14157],{"className":24795,"style":3317},[230,303,267],[170,24797,338],{"className":24798},[337],[170,24800,24802],{"className":24801},[247],[170,24803,24805],{"className":24804,"style":1381},[251],[170,24806],{},[170,24808,909],{"className":24809},[234],[170,24811,24813,24816],{"className":24812},[230],[170,24814,12908],{"className":24815},[230,303],[170,24817,24819],{"className":24818},[307],[170,24820,24822,24842],{"className":24821},[242,243],[170,24823,24825,24839],{"className":24824},[247],[170,24826,24828],{"className":24827,"style":12838},[251],[170,24829,24830,24833],{"style":1362},[170,24831],{"className":24832,"style":651},[259],[170,24834,24836],{"className":24835},[264,265,266,267],[170,24837,888],{"className":24838},[230,303,267],[170,24840,338],{"className":24841},[337],[170,24843,24845],{"className":24844},[247],[170,24846,24848],{"className":24847,"style":1381},[251],[170,24849],{},[170,24851,923],{"className":24852},[424],[170,24854],{"className":24855},[610,21759],[170,24857,24859,24863],{"className":24858},[221],[170,24860],{"className":24861,"style":24862},[225],"height:1.53em;vertical-align:-0.03em;",[170,24864,24866,24869],{"className":24865},[230],[170,24867,23244],{"className":24868},[230],[170,24870],{"className":24871,"style":24873},[230,24872],"rule","border-right-width:0em;border-top-width:1.5em;bottom:0em;",[170,24875],{"className":24876},[610,21759],[170,24878,24880,24884,24936,24939,24979,24982,24985,24988,24991,24994,24997],{"className":24879},[221],[170,24881],{"className":24882,"style":24883},[225],"height:1.1413em;vertical-align:-0.25em;",[170,24885,24887,24890],{"className":24886},[230],[170,24888,885],{"className":24889},[230,303],[170,24891,24893],{"className":24892},[307],[170,24894,24896,24928],{"className":24895},[242,243],[170,24897,24899,24925],{"className":24898},[247],[170,24900,24903,24914],{"className":24901,"style":24902},[251],"height:0.8913em;",[170,24904,24905,24908],{"style":17417},[170,24906],{"className":24907,"style":651},[259],[170,24909,24911],{"className":24910},[264,265,266,267],[170,24912,14157],{"className":24913,"style":3317},[230,303,267],[170,24915,24916,24919],{"style":16532},[170,24917],{"className":24918,"style":651},[259],[170,24920,24922],{"className":24921},[264,265,266,267],[170,24923,13348],{"className":24924,"style":3292},[230,303,267],[170,24926,338],{"className":24927},[337],[170,24929,24931],{"className":24930},[247],[170,24932,24934],{"className":24933,"style":17447},[251],[170,24935],{},[170,24937,909],{"className":24938},[234],[170,24940,24942,24945],{"className":24941},[230],[170,24943,12908],{"className":24944},[230,303],[170,24946,24948],{"className":24947},[307],[170,24949,24951,24971],{"className":24950},[242,243],[170,24952,24954,24968],{"className":24953},[247],[170,24955,24957],{"className":24956,"style":12838},[251],[170,24958,24959,24962],{"style":1362},[170,24960],{"className":24961,"style":651},[259],[170,24963,24965],{"className":24964},[264,265,266,267],[170,24966,888],{"className":24967},[230,303,267],[170,24969,338],{"className":24970},[337],[170,24972,24974],{"className":24973},[247],[170,24975,24977],{"className":24976,"style":1381},[251],[170,24978],{},[170,24980,914],{"className":24981},[1023],[170,24983],{"className":24984,"style":1027},[610],[170,24986,134],{"className":24987},[230,303],[170,24989,923],{"className":24990},[424],[170,24992],{"className":24993,"style":611},[610],[170,24995,926],{"className":24996},[615],[170,24998],{"className":24999,"style":611},[610],[170,25001,25003,25006,25046,25049,25052],{"className":25002},[221],[170,25004],{"className":25005,"style":22018},[225],[170,25007,25009,25012],{"className":25008},[230],[170,25010,7933],{"className":25011,"style":3317},[230,303],[170,25013,25015],{"className":25014},[307],[170,25016,25018,25038],{"className":25017},[242,243],[170,25019,25021,25035],{"className":25020},[247],[170,25022,25024],{"className":25023,"style":12838},[251],[170,25025,25026,25029],{"style":13211},[170,25027],{"className":25028,"style":651},[259],[170,25030,25032],{"className":25031},[264,265,266,267],[170,25033,888],{"className":25034},[230,303,267],[170,25036,338],{"className":25037},[337],[170,25039,25041],{"className":25040},[247],[170,25042,25044],{"className":25043,"style":1381},[251],[170,25045],{},[170,25047],{"className":25048,"style":1983},[610],[170,25050,5884],{"className":25051},[664],[170,25053],{"className":25054,"style":1983},[610],[170,25056,25058,25061,25064,25067,25070],{"className":25057},[221],[170,25059],{"className":25060,"style":24436},[225],[170,25062,19101],{"className":25063,"style":19348},[230,303],[170,25065],{"className":25066,"style":1983},[610],[170,25068,23155],{"className":25069},[664],[170,25071],{"className":25072,"style":1983},[610],[170,25074,25076,25079,25128,25131,25134],{"className":25075},[221],[170,25077],{"className":25078,"style":22664},[225],[170,25080,25082,25085],{"className":25081},[230],[170,25083,7933],{"className":25084,"style":3317},[230,303],[170,25086,25088],{"className":25087},[307],[170,25089,25091,25120],{"className":25090},[242,243],[170,25092,25094,25117],{"className":25093},[247],[170,25095,25097],{"className":25096,"style":3198},[251],[170,25098,25099,25102],{"style":13211},[170,25100],{"className":25101,"style":651},[259],[170,25103,25105],{"className":25104},[264,265,266,267],[170,25106,25108,25111,25114],{"className":25107},[230,267],[170,25109,888],{"className":25110},[230,303,267],[170,25112,5884],{"className":25113},[664,267],[170,25115,194],{"className":25116},[230,267],[170,25118,338],{"className":25119},[337],[170,25121,25123],{"className":25122},[247],[170,25124,25126],{"className":25125,"style":7045},[251],[170,25127],{},[170,25129],{"className":25130,"style":1983},[610],[170,25132,5884],{"className":25133},[664],[170,25135],{"className":25136,"style":1983},[610],[170,25138,25140,25143,25172,25175,25178],{"className":25139},[221],[170,25141],{"className":25142,"style":24519},[225],[170,25144,25146,25149],{"className":25145},[230],[170,25147,19101],{"className":25148,"style":19348},[230,303],[170,25150,25152],{"className":25151},[307],[170,25153,25155],{"className":25154},[242],[170,25156,25158],{"className":25157},[247],[170,25159,25161],{"className":25160,"style":23416},[251],[170,25162,25163,25166],{"style":16532},[170,25164],{"className":25165,"style":651},[259],[170,25167,25169],{"className":25168},[264,265,266,267],[170,25170,2928],{"className":25171},[230,267],[170,25173],{"className":25174,"style":1983},[610],[170,25176,23155],{"className":25177},[664],[170,25179],{"className":25180,"style":1983},[610],[170,25182,25184,25187,25236,25239,25242],{"className":25183},[221],[170,25185],{"className":25186,"style":22664},[225],[170,25188,25190,25193],{"className":25189},[230],[170,25191,7933],{"className":25192,"style":3317},[230,303],[170,25194,25196],{"className":25195},[307],[170,25197,25199,25228],{"className":25198},[242,243],[170,25200,25202,25225],{"className":25201},[247],[170,25203,25205],{"className":25204,"style":3198},[251],[170,25206,25207,25210],{"style":13211},[170,25208],{"className":25209,"style":651},[259],[170,25211,25213],{"className":25212},[264,265,266,267],[170,25214,25216,25219,25222],{"className":25215},[230,267],[170,25217,888],{"className":25218},[230,303,267],[170,25220,5884],{"className":25221},[664,267],[170,25223,2928],{"className":25224},[230,267],[170,25226,338],{"className":25227},[337],[170,25229,25231],{"className":25230},[247],[170,25232,25234],{"className":25233,"style":7045},[251],[170,25235],{},[170,25237],{"className":25238,"style":1983},[610],[170,25240,5884],{"className":25241},[664],[170,25243],{"className":25244,"style":1983},[610],[170,25246,25248,25251,25280,25283,25286],{"className":25247},[221],[170,25249],{"className":25250,"style":24519},[225],[170,25252,25254,25257],{"className":25253},[230],[170,25255,19101],{"className":25256,"style":19348},[230,303],[170,25258,25260],{"className":25259},[307],[170,25261,25263],{"className":25262},[242],[170,25264,25266],{"className":25265},[247],[170,25267,25269],{"className":25268,"style":23416},[251],[170,25270,25271,25274],{"style":16532},[170,25272],{"className":25273,"style":651},[259],[170,25275,25277],{"className":25276},[264,265,266,267],[170,25278,4778],{"className":25279},[230,267],[170,25281],{"className":25282,"style":1983},[610],[170,25284,23155],{"className":25285},[664],[170,25287],{"className":25288,"style":1983},[610],[170,25290,25292,25295,25344,25347,25350],{"className":25291},[221],[170,25293],{"className":25294,"style":22664},[225],[170,25296,25298,25301],{"className":25297},[230],[170,25299,7933],{"className":25300,"style":3317},[230,303],[170,25302,25304],{"className":25303},[307],[170,25305,25307,25336],{"className":25306},[242,243],[170,25308,25310,25333],{"className":25309},[247],[170,25311,25313],{"className":25312,"style":3198},[251],[170,25314,25315,25318],{"style":13211},[170,25316],{"className":25317,"style":651},[259],[170,25319,25321],{"className":25320},[264,265,266,267],[170,25322,25324,25327,25330],{"className":25323},[230,267],[170,25325,888],{"className":25326},[230,303,267],[170,25328,5884],{"className":25329},[664,267],[170,25331,4778],{"className":25332},[230,267],[170,25334,338],{"className":25335},[337],[170,25337,25339],{"className":25338},[247],[170,25340,25342],{"className":25341,"style":7045},[251],[170,25343],{},[170,25345],{"className":25346,"style":1983},[610],[170,25348,5884],{"className":25349},[664],[170,25351],{"className":25352,"style":1983},[610],[170,25354,25356,25360,25363,25366,25369],{"className":25355},[221],[170,25357],{"className":25358,"style":25359},[225],"height:0.6667em;vertical-align:-0.0833em;",[170,25361,23350],{"className":25362},[1073],[170,25364],{"className":25365,"style":1983},[610],[170,25367,5884],{"className":25368},[664],[170,25370],{"className":25371,"style":1983},[610],[170,25373,25375,25379,25408,25411,25414],{"className":25374},[221],[170,25376],{"className":25377,"style":25378},[225],"height:1.0858em;vertical-align:-0.1944em;",[170,25380,25382,25385],{"className":25381},[230],[170,25383,19101],{"className":25384,"style":19348},[230,303],[170,25386,25388],{"className":25387},[307],[170,25389,25391],{"className":25390},[242],[170,25392,25394],{"className":25393},[247],[170,25395,25397],{"className":25396,"style":24902},[251],[170,25398,25399,25402],{"style":16532},[170,25400],{"className":25401,"style":651},[259],[170,25403,25405],{"className":25404},[264,265,266,267],[170,25406,13348],{"className":25407,"style":3292},[230,303,267],[170,25409],{"className":25410,"style":1983},[610],[170,25412,23155],{"className":25413},[664],[170,25415],{"className":25416,"style":1983},[610],[170,25418,25420,25423,25463,25466,25469],{"className":25419},[221],[170,25421],{"className":25422,"style":22018},[225],[170,25424,25426,25429],{"className":25425},[230],[170,25427,7933],{"className":25428,"style":3317},[230,303],[170,25430,25432],{"className":25431},[307],[170,25433,25435,25455],{"className":25434},[242,243],[170,25436,25438,25452],{"className":25437},[247],[170,25439,25441],{"className":25440,"style":13571},[251],[170,25442,25443,25446],{"style":13211},[170,25444],{"className":25445,"style":651},[259],[170,25447,25449],{"className":25448},[264,265,266,267],[170,25450,13348],{"className":25451,"style":3292},[230,303,267],[170,25453,338],{"className":25454},[337],[170,25456,25458],{"className":25457},[247],[170,25459,25461],{"className":25460,"style":1381},[251],[170,25462],{},[170,25464],{"className":25465,"style":1983},[610],[170,25467,18471],{"className":25468},[664],[170,25470],{"className":25471,"style":1983},[610],[170,25473,25475,25478,25518,25521,25561],{"className":25474},[221],[170,25476],{"className":25477,"style":988},[225],[170,25479,25481,25484],{"className":25480},[230],[170,25482,783],{"className":25483,"style":813},[230,303],[170,25485,25487],{"className":25486},[307],[170,25488,25490,25510],{"className":25489},[242,243],[170,25491,25493,25507],{"className":25492},[247],[170,25494,25496],{"className":25495,"style":1359},[251],[170,25497,25498,25501],{"style":20611},[170,25499],{"className":25500,"style":651},[259],[170,25502,25504],{"className":25503},[264,265,266,267],[170,25505,14157],{"className":25506,"style":3317},[230,303,267],[170,25508,338],{"className":25509},[337],[170,25511,25513],{"className":25512},[247],[170,25514,25516],{"className":25515,"style":1381},[251],[170,25517],{},[170,25519,909],{"className":25520},[234],[170,25522,25524,25527],{"className":25523},[230],[170,25525,12908],{"className":25526},[230,303],[170,25528,25530],{"className":25529},[307],[170,25531,25533,25553],{"className":25532},[242,243],[170,25534,25536,25550],{"className":25535},[247],[170,25537,25539],{"className":25538,"style":12838},[251],[170,25540,25541,25544],{"style":1362},[170,25542],{"className":25543,"style":651},[259],[170,25545,25547],{"className":25546},[264,265,266,267],[170,25548,888],{"className":25549},[230,303,267],[170,25551,338],{"className":25552},[337],[170,25554,25556],{"className":25555},[247],[170,25557,25559],{"className":25558,"style":1381},[251],[170,25560],{},[170,25562,923],{"className":25563},[424],[14,25565,25566,25567,25640,25641,25669],{},"为了让式子表示整洁，我们定义一个中间变量，令 ",[170,25568,25570,25589],{"className":25569},[173],[170,25571,25573],{"className":25572},[177],[179,25574,25575],{"xmlns":181},[183,25576,25577,25586],{},[186,25578,25579],{},[199,25580,25581,25584],{},[202,25582,25583],{},"δ",[202,25585,888],{},[209,25587,25588],{"encoding":211},"\\delta_t",[170,25590,25592],{"className":25591,"ariaHidden":217},[216],[170,25593,25595,25598],{"className":25594},[221],[170,25596],{"className":25597,"style":1340},[225],[170,25599,25601,25605],{"className":25600},[230],[170,25602,25583],{"className":25603,"style":25604},[230,303],"margin-right:0.03785em;",[170,25606,25608],{"className":25607},[307],[170,25609,25611,25632],{"className":25610},[242,243],[170,25612,25614,25629],{"className":25613},[247],[170,25615,25617],{"className":25616,"style":12838},[251],[170,25618,25620,25623],{"style":25619},"top:-2.55em;margin-left:-0.0379em;margin-right:0.05em;",[170,25621],{"className":25622,"style":651},[259],[170,25624,25626],{"className":25625},[264,265,266,267],[170,25627,888],{"className":25628},[230,303,267],[170,25630,338],{"className":25631},[337],[170,25633,25635],{"className":25634},[247],[170,25636,25638],{"className":25637,"style":1381},[251],[170,25639],{}," 表示在第 ",[170,25642,25644,25657],{"className":25643},[173],[170,25645,25647],{"className":25646},[177],[179,25648,25649],{"xmlns":181},[183,25650,25651,25655],{},[186,25652,25653],{},[202,25654,888],{},[209,25656,888],{"encoding":211},[170,25658,25660],{"className":25659,"ariaHidden":217},[216],[170,25661,25663,25666],{"className":25662},[221],[170,25664],{"className":25665,"style":12887},[225],[170,25667,888],{"className":25668},[230,303]," 步采取特定动作带来的优势。",[170,25671,25673],{"className":25672},[14456],[170,25674,25676,26002],{"className":25675},[173],[170,25677,25679],{"className":25678},[177],[179,25680,25681],{"xmlns":181,"display":9588},[183,25682,25683,25999],{},[186,25684,25685,25693,25695,25701,25703,25705,25707,25713,25715,25727,25729,25731,25737,25739,25745,25747,25749,25763,25765,25777,25779,25781,25783,25789,25791,25803,25805,25807,25813,25815,25827,25829,25831,25839,25841,25847,25849,25851,25853,25855,25863,25865,25873,25875,25881,25883,25885,25887,25889,25897,25899,25901,25915,25917,25925,25927,25933,25935,25937,25939,25941,25949,25951,25953,25967,25969,25975,25989,25991],{},[1933,25686,25687,25689,25691],{},[202,25688,25583],{},[202,25690,888],{},[202,25692,783],{},[572,25694,926],{},[199,25696,25697,25699],{},[202,25698,7933],{},[202,25700,888],{},[572,25702,5884],{},[202,25704,19101],{},[572,25706,23155],{},[199,25708,25709,25711],{},[202,25710,783],{},[202,25712,14157],{},[572,25714,909],{"stretchy":908},[199,25716,25717,25719],{},[202,25718,12908],{},[186,25720,25721,25723,25725],{},[202,25722,888],{},[572,25724,5884],{},[192,25726,194],{},[572,25728,923],{"stretchy":908},[572,25730,18471],{},[199,25732,25733,25735],{},[202,25734,783],{},[202,25736,14157],{},[572,25738,909],{"stretchy":908},[199,25740,25741,25743],{},[202,25742,12908],{},[202,25744,888],{},[572,25746,923],{"stretchy":908},[610,25748],{"linebreak":21759},[1933,25750,25751,25753,25761],{},[202,25752,25583],{},[186,25754,25755,25757,25759],{},[202,25756,888],{},[572,25758,5884],{},[192,25760,194],{},[202,25762,783],{},[572,25764,926],{},[199,25766,25767,25769],{},[202,25768,7933],{},[186,25770,25771,25773,25775],{},[202,25772,888],{},[572,25774,5884],{},[192,25776,194],{},[572,25778,5884],{},[202,25780,19101],{},[572,25782,23155],{},[199,25784,25785,25787],{},[202,25786,783],{},[202,25788,14157],{},[572,25790,909],{"stretchy":908},[199,25792,25793,25795],{},[202,25794,12908],{},[186,25796,25797,25799,25801],{},[202,25798,888],{},[572,25800,5884],{},[192,25802,2928],{},[572,25804,923],{"stretchy":908},[572,25806,18471],{},[199,25808,25809,25811],{},[202,25810,783],{},[202,25812,14157],{},[572,25814,909],{"stretchy":908},[199,25816,25817,25819],{},[202,25818,12908],{},[186,25820,25821,25823,25825],{},[202,25822,888],{},[572,25824,5884],{},[192,25826,194],{},[572,25828,923],{"stretchy":908},[610,25830],{"linebreak":21759},[1933,25832,25833,25835,25837],{},[202,25834,885],{},[202,25836,14157],{},[192,25838,194],{},[572,25840,909],{"stretchy":908},[199,25842,25843,25845],{},[202,25844,12908],{},[202,25846,888],{},[572,25848,914],{"separator":217},[202,25850,134],{},[572,25852,923],{"stretchy":908},[572,25854,926],{},[1933,25856,25857,25859,25861],{},[202,25858,25583],{},[202,25860,888],{},[202,25862,783],{},[610,25864],{"linebreak":21759},[1933,25866,25867,25869,25871],{},[202,25868,885],{},[202,25870,14157],{},[192,25872,2928],{},[572,25874,909],{"stretchy":908},[199,25876,25877,25879],{},[202,25878,12908],{},[202,25880,888],{},[572,25882,914],{"separator":217},[202,25884,134],{},[572,25886,923],{"stretchy":908},[572,25888,926],{},[1933,25890,25891,25893,25895],{},[202,25892,25583],{},[202,25894,888],{},[202,25896,783],{},[572,25898,5884],{},[202,25900,19101],{},[1933,25902,25903,25905,25913],{},[202,25904,25583],{},[186,25906,25907,25909,25911],{},[202,25908,888],{},[572,25910,5884],{},[192,25912,194],{},[202,25914,783],{},[610,25916],{"linebreak":21759},[1933,25918,25919,25921,25923],{},[202,25920,885],{},[202,25922,14157],{},[192,25924,4778],{},[572,25926,909],{"stretchy":908},[199,25928,25929,25931],{},[202,25930,12908],{},[202,25932,888],{},[572,25934,914],{"separator":217},[202,25936,134],{},[572,25938,923],{"stretchy":908},[572,25940,926],{},[1933,25942,25943,25945,25947],{},[202,25944,25583],{},[202,25946,888],{},[202,25948,783],{},[572,25950,5884],{},[202,25952,19101],{},[1933,25954,25955,25957,25965],{},[202,25956,25583],{},[186,25958,25959,25961,25963],{},[202,25960,888],{},[572,25962,5884],{},[192,25964,194],{},[202,25966,783],{},[572,25968,5884],{},[576,25970,25971,25973],{},[202,25972,19101],{},[192,25974,2928],{},[1933,25976,25977,25979,25987],{},[202,25978,25583],{},[186,25980,25981,25983,25985],{},[202,25982,888],{},[572,25984,5884],{},[192,25986,2928],{},[202,25988,783],{},[610,25990],{"linebreak":21759},[186,25992,25993,25995],{},[202,25994,23244],{"mathvariant":929},[23246,25996,25997],{"height":14472,"voffset":14472},[610,25998],{"mathbackground":23250,"width":14472,"height":23251},[209,26000,26001],{"encoding":211},"\n\\delta_t^V = r_t + \\gamma * V_\\theta(s_{t+1}) - V_\\theta(s_t)\\\\\n\n\\delta_{t+1}^V = r_{t+1} + \\gamma * V_\\theta(s_{t+2}) - V_\\theta(s_{t+1})\\\\\n\nA_\\theta^1(s_t, a) = \\delta_t^V\\\\\n\nA_\\theta^2(s_t, a) = \\delta_t^V + \\gamma \\delta_{t+1}^V\\\\\n\nA_\\theta^3(s_t, a) = \\delta_t^V + \\gamma \\delta_{t+1}^V + \\gamma^2 \\delta_{t+2}^V\\\\\n\n\\vdots",[170,26003,26005,26073,26128,26146,26256,26348,26351,26428,26492,26510,26620,26721,26724,26845,26902,26905,27026,27092,27161,27164,27285,27351,27429,27524,27527],{"className":26004,"ariaHidden":217},[216],[170,26006,26008,26012,26064,26067,26070],{"className":26007},[221],[170,26009],{"className":26010,"style":26011},[225],"height:1.1383em;vertical-align:-0.247em;",[170,26013,26015,26018],{"className":26014},[230],[170,26016,25583],{"className":26017,"style":25604},[230,303],[170,26019,26021],{"className":26020},[307],[170,26022,26024,26056],{"className":26023},[242,243],[170,26025,26027,26053],{"className":26026},[247],[170,26028,26030,26042],{"className":26029,"style":24902},[251],[170,26031,26033,26036],{"style":26032},"top:-2.453em;margin-left:-0.0379em;margin-right:0.05em;",[170,26034],{"className":26035,"style":651},[259],[170,26037,26039],{"className":26038},[264,265,266,267],[170,26040,888],{"className":26041},[230,303,267],[170,26043,26044,26047],{"style":16532},[170,26045],{"className":26046,"style":651},[259],[170,26048,26050],{"className":26049},[264,265,266,267],[170,26051,783],{"className":26052,"style":813},[230,303,267],[170,26054,338],{"className":26055},[337],[170,26057,26059],{"className":26058},[247],[170,26060,26062],{"className":26061,"style":17447},[251],[170,26063],{},[170,26065],{"className":26066,"style":611},[610],[170,26068,926],{"className":26069},[615],[170,26071],{"className":26072,"style":611},[610],[170,26074,26076,26079,26119,26122,26125],{"className":26075},[221],[170,26077],{"className":26078,"style":22018},[225],[170,26080,26082,26085],{"className":26081},[230],[170,26083,7933],{"className":26084,"style":3317},[230,303],[170,26086,26088],{"className":26087},[307],[170,26089,26091,26111],{"className":26090},[242,243],[170,26092,26094,26108],{"className":26093},[247],[170,26095,26097],{"className":26096,"style":12838},[251],[170,26098,26099,26102],{"style":13211},[170,26100],{"className":26101,"style":651},[259],[170,26103,26105],{"className":26104},[264,265,266,267],[170,26106,888],{"className":26107},[230,303,267],[170,26109,338],{"className":26110},[337],[170,26112,26114],{"className":26113},[247],[170,26115,26117],{"className":26116,"style":1381},[251],[170,26118],{},[170,26120],{"className":26121,"style":1983},[610],[170,26123,5884],{"className":26124},[664],[170,26126],{"className":26127,"style":1983},[610],[170,26129,26131,26134,26137,26140,26143],{"className":26130},[221],[170,26132],{"className":26133,"style":24436},[225],[170,26135,19101],{"className":26136,"style":19348},[230,303],[170,26138],{"className":26139,"style":1983},[610],[170,26141,23155],{"className":26142},[664],[170,26144],{"className":26145,"style":1983},[610],[170,26147,26149,26152,26192,26195,26244,26247,26250,26253],{"className":26148},[221],[170,26150],{"className":26151,"style":988},[225],[170,26153,26155,26158],{"className":26154},[230],[170,26156,783],{"className":26157,"style":813},[230,303],[170,26159,26161],{"className":26160},[307],[170,26162,26164,26184],{"className":26163},[242,243],[170,26165,26167,26181],{"className":26166},[247],[170,26168,26170],{"className":26169,"style":1359},[251],[170,26171,26172,26175],{"style":20611},[170,26173],{"className":26174,"style":651},[259],[170,26176,26178],{"className":26177},[264,265,266,267],[170,26179,14157],{"className":26180,"style":3317},[230,303,267],[170,26182,338],{"className":26183},[337],[170,26185,26187],{"className":26186},[247],[170,26188,26190],{"className":26189,"style":1381},[251],[170,26191],{},[170,26193,909],{"className":26194},[234],[170,26196,26198,26201],{"className":26197},[230],[170,26199,12908],{"className":26200},[230,303],[170,26202,26204],{"className":26203},[307],[170,26205,26207,26236],{"className":26206},[242,243],[170,26208,26210,26233],{"className":26209},[247],[170,26211,26213],{"className":26212,"style":3198},[251],[170,26214,26215,26218],{"style":1362},[170,26216],{"className":26217,"style":651},[259],[170,26219,26221],{"className":26220},[264,265,266,267],[170,26222,26224,26227,26230],{"className":26223},[230,267],[170,26225,888],{"className":26226},[230,303,267],[170,26228,5884],{"className":26229},[664,267],[170,26231,194],{"className":26232},[230,267],[170,26234,338],{"className":26235},[337],[170,26237,26239],{"className":26238},[247],[170,26240,26242],{"className":26241,"style":7045},[251],[170,26243],{},[170,26245,923],{"className":26246},[424],[170,26248],{"className":26249,"style":1983},[610],[170,26251,18471],{"className":26252},[664],[170,26254],{"className":26255,"style":1983},[610],[170,26257,26259,26262,26302,26305,26345],{"className":26258},[221],[170,26260],{"className":26261,"style":988},[225],[170,26263,26265,26268],{"className":26264},[230],[170,26266,783],{"className":26267,"style":813},[230,303],[170,26269,26271],{"className":26270},[307],[170,26272,26274,26294],{"className":26273},[242,243],[170,26275,26277,26291],{"className":26276},[247],[170,26278,26280],{"className":26279,"style":1359},[251],[170,26281,26282,26285],{"style":20611},[170,26283],{"className":26284,"style":651},[259],[170,26286,26288],{"className":26287},[264,265,266,267],[170,26289,14157],{"className":26290,"style":3317},[230,303,267],[170,26292,338],{"className":26293},[337],[170,26295,26297],{"className":26296},[247],[170,26298,26300],{"className":26299,"style":1381},[251],[170,26301],{},[170,26303,909],{"className":26304},[234],[170,26306,26308,26311],{"className":26307},[230],[170,26309,12908],{"className":26310},[230,303],[170,26312,26314],{"className":26313},[307],[170,26315,26317,26337],{"className":26316},[242,243],[170,26318,26320,26334],{"className":26319},[247],[170,26321,26323],{"className":26322,"style":12838},[251],[170,26324,26325,26328],{"style":1362},[170,26326],{"className":26327,"style":651},[259],[170,26329,26331],{"className":26330},[264,265,266,267],[170,26332,888],{"className":26333},[230,303,267],[170,26335,338],{"className":26336},[337],[170,26338,26340],{"className":26339},[247],[170,26341,26343],{"className":26342,"style":1381},[251],[170,26344],{},[170,26346,923],{"className":26347},[424],[170,26349],{"className":26350},[610,21759],[170,26352,26354,26358,26419,26422,26425],{"className":26353},[221],[170,26355],{"className":26356,"style":26357},[225],"height:1.1967em;vertical-align:-0.3053em;",[170,26359,26361,26364],{"className":26360},[230],[170,26362,25583],{"className":26363,"style":25604},[230,303],[170,26365,26367],{"className":26366},[307],[170,26368,26370,26410],{"className":26369},[242,243],[170,26371,26373,26407],{"className":26372},[247],[170,26374,26376,26396],{"className":26375,"style":24902},[251],[170,26377,26378,26381],{"style":26032},[170,26379],{"className":26380,"style":651},[259],[170,26382,26384],{"className":26383},[264,265,266,267],[170,26385,26387,26390,26393],{"className":26386},[230,267],[170,26388,888],{"className":26389},[230,303,267],[170,26391,5884],{"className":26392},[664,267],[170,26394,194],{"className":26395},[230,267],[170,26397,26398,26401],{"style":16532},[170,26399],{"className":26400,"style":651},[259],[170,26402,26404],{"className":26403},[264,265,266,267],[170,26405,783],{"className":26406,"style":813},[230,303,267],[170,26408,338],{"className":26409},[337],[170,26411,26413],{"className":26412},[247],[170,26414,26417],{"className":26415,"style":26416},[251],"height:0.3053em;",[170,26418],{},[170,26420],{"className":26421,"style":611},[610],[170,26423,926],{"className":26424},[615],[170,26426],{"className":26427,"style":611},[610],[170,26429,26431,26434,26483,26486,26489],{"className":26430},[221],[170,26432],{"className":26433,"style":22664},[225],[170,26435,26437,26440],{"className":26436},[230],[170,26438,7933],{"className":26439,"style":3317},[230,303],[170,26441,26443],{"className":26442},[307],[170,26444,26446,26475],{"className":26445},[242,243],[170,26447,26449,26472],{"className":26448},[247],[170,26450,26452],{"className":26451,"style":3198},[251],[170,26453,26454,26457],{"style":13211},[170,26455],{"className":26456,"style":651},[259],[170,26458,26460],{"className":26459},[264,265,266,267],[170,26461,26463,26466,26469],{"className":26462},[230,267],[170,26464,888],{"className":26465},[230,303,267],[170,26467,5884],{"className":26468},[664,267],[170,26470,194],{"className":26471},[230,267],[170,26473,338],{"className":26474},[337],[170,26476,26478],{"className":26477},[247],[170,26479,26481],{"className":26480,"style":7045},[251],[170,26482],{},[170,26484],{"className":26485,"style":1983},[610],[170,26487,5884],{"className":26488},[664],[170,26490],{"className":26491,"style":1983},[610],[170,26493,26495,26498,26501,26504,26507],{"className":26494},[221],[170,26496],{"className":26497,"style":24436},[225],[170,26499,19101],{"className":26500,"style":19348},[230,303],[170,26502],{"className":26503,"style":1983},[610],[170,26505,23155],{"className":26506},[664],[170,26508],{"className":26509,"style":1983},[610],[170,26511,26513,26516,26556,26559,26608,26611,26614,26617],{"className":26512},[221],[170,26514],{"className":26515,"style":988},[225],[170,26517,26519,26522],{"className":26518},[230],[170,26520,783],{"className":26521,"style":813},[230,303],[170,26523,26525],{"className":26524},[307],[170,26526,26528,26548],{"className":26527},[242,243],[170,26529,26531,26545],{"className":26530},[247],[170,26532,26534],{"className":26533,"style":1359},[251],[170,26535,26536,26539],{"style":20611},[170,26537],{"className":26538,"style":651},[259],[170,26540,26542],{"className":26541},[264,265,266,267],[170,26543,14157],{"className":26544,"style":3317},[230,303,267],[170,26546,338],{"className":26547},[337],[170,26549,26551],{"className":26550},[247],[170,26552,26554],{"className":26553,"style":1381},[251],[170,26555],{},[170,26557,909],{"className":26558},[234],[170,26560,26562,26565],{"className":26561},[230],[170,26563,12908],{"className":26564},[230,303],[170,26566,26568],{"className":26567},[307],[170,26569,26571,26600],{"className":26570},[242,243],[170,26572,26574,26597],{"className":26573},[247],[170,26575,26577],{"className":26576,"style":3198},[251],[170,26578,26579,26582],{"style":1362},[170,26580],{"className":26581,"style":651},[259],[170,26583,26585],{"className":26584},[264,265,266,267],[170,26586,26588,26591,26594],{"className":26587},[230,267],[170,26589,888],{"className":26590},[230,303,267],[170,26592,5884],{"className":26593},[664,267],[170,26595,2928],{"className":26596},[230,267],[170,26598,338],{"className":26599},[337],[170,26601,26603],{"className":26602},[247],[170,26604,26606],{"className":26605,"style":7045},[251],[170,26607],{},[170,26609,923],{"className":26610},[424],[170,26612],{"className":26613,"style":1983},[610],[170,26615,18471],{"className":26616},[664],[170,26618],{"className":26619,"style":1983},[610],[170,26621,26623,26626,26666,26669,26718],{"className":26622},[221],[170,26624],{"className":26625,"style":988},[225],[170,26627,26629,26632],{"className":26628},[230],[170,26630,783],{"className":26631,"style":813},[230,303],[170,26633,26635],{"className":26634},[307],[170,26636,26638,26658],{"className":26637},[242,243],[170,26639,26641,26655],{"className":26640},[247],[170,26642,26644],{"className":26643,"style":1359},[251],[170,26645,26646,26649],{"style":20611},[170,26647],{"className":26648,"style":651},[259],[170,26650,26652],{"className":26651},[264,265,266,267],[170,26653,14157],{"className":26654,"style":3317},[230,303,267],[170,26656,338],{"className":26657},[337],[170,26659,26661],{"className":26660},[247],[170,26662,26664],{"className":26663,"style":1381},[251],[170,26665],{},[170,26667,909],{"className":26668},[234],[170,26670,26672,26675],{"className":26671},[230],[170,26673,12908],{"className":26674},[230,303],[170,26676,26678],{"className":26677},[307],[170,26679,26681,26710],{"className":26680},[242,243],[170,26682,26684,26707],{"className":26683},[247],[170,26685,26687],{"className":26686,"style":3198},[251],[170,26688,26689,26692],{"style":1362},[170,26690],{"className":26691,"style":651},[259],[170,26693,26695],{"className":26694},[264,265,266,267],[170,26696,26698,26701,26704],{"className":26697},[230,267],[170,26699,888],{"className":26700},[230,303,267],[170,26702,5884],{"className":26703},[664,267],[170,26705,194],{"className":26706},[230,267],[170,26708,338],{"className":26709},[337],[170,26711,26713],{"className":26712},[247],[170,26714,26716],{"className":26715,"style":7045},[251],[170,26717],{},[170,26719,923],{"className":26720},[424],[170,26722],{"className":26723},[610,21759],[170,26725,26727,26730,26781,26784,26824,26827,26830,26833,26836,26839,26842],{"className":26726},[221],[170,26728],{"className":26729,"style":23397},[225],[170,26731,26733,26736],{"className":26732},[230],[170,26734,885],{"className":26735},[230,303],[170,26737,26739],{"className":26738},[307],[170,26740,26742,26773],{"className":26741},[242,243],[170,26743,26745,26770],{"className":26744},[247],[170,26746,26748,26759],{"className":26747,"style":23416},[251],[170,26749,26750,26753],{"style":17417},[170,26751],{"className":26752,"style":651},[259],[170,26754,26756],{"className":26755},[264,265,266,267],[170,26757,14157],{"className":26758,"style":3317},[230,303,267],[170,26760,26761,26764],{"style":16532},[170,26762],{"className":26763,"style":651},[259],[170,26765,26767],{"className":26766},[264,265,266,267],[170,26768,194],{"className":26769},[230,267],[170,26771,338],{"className":26772},[337],[170,26774,26776],{"className":26775},[247],[170,26777,26779],{"className":26778,"style":17447},[251],[170,26780],{},[170,26782,909],{"className":26783},[234],[170,26785,26787,26790],{"className":26786},[230],[170,26788,12908],{"className":26789},[230,303],[170,26791,26793],{"className":26792},[307],[170,26794,26796,26816],{"className":26795},[242,243],[170,26797,26799,26813],{"className":26798},[247],[170,26800,26802],{"className":26801,"style":12838},[251],[170,26803,26804,26807],{"style":1362},[170,26805],{"className":26806,"style":651},[259],[170,26808,26810],{"className":26809},[264,265,266,267],[170,26811,888],{"className":26812},[230,303,267],[170,26814,338],{"className":26815},[337],[170,26817,26819],{"className":26818},[247],[170,26820,26822],{"className":26821,"style":1381},[251],[170,26823],{},[170,26825,914],{"className":26826},[1023],[170,26828],{"className":26829,"style":1027},[610],[170,26831,134],{"className":26832},[230,303],[170,26834,923],{"className":26835},[424],[170,26837],{"className":26838,"style":611},[610],[170,26840,926],{"className":26841},[615],[170,26843],{"className":26844,"style":611},[610],[170,26846,26848,26851],{"className":26847},[221],[170,26849],{"className":26850,"style":26011},[225],[170,26852,26854,26857],{"className":26853},[230],[170,26855,25583],{"className":26856,"style":25604},[230,303],[170,26858,26860],{"className":26859},[307],[170,26861,26863,26894],{"className":26862},[242,243],[170,26864,26866,26891],{"className":26865},[247],[170,26867,26869,26880],{"className":26868,"style":24902},[251],[170,26870,26871,26874],{"style":26032},[170,26872],{"className":26873,"style":651},[259],[170,26875,26877],{"className":26876},[264,265,266,267],[170,26878,888],{"className":26879},[230,303,267],[170,26881,26882,26885],{"style":16532},[170,26883],{"className":26884,"style":651},[259],[170,26886,26888],{"className":26887},[264,265,266,267],[170,26889,783],{"className":26890,"style":813},[230,303,267],[170,26892,338],{"className":26893},[337],[170,26895,26897],{"className":26896},[247],[170,26898,26900],{"className":26899,"style":17447},[251],[170,26901],{},[170,26903],{"className":26904},[610,21759],[170,26906,26908,26911,26962,26965,27005,27008,27011,27014,27017,27020,27023],{"className":26907},[221],[170,26909],{"className":26910,"style":23397},[225],[170,26912,26914,26917],{"className":26913},[230],[170,26915,885],{"className":26916},[230,303],[170,26918,26920],{"className":26919},[307],[170,26921,26923,26954],{"className":26922},[242,243],[170,26924,26926,26951],{"className":26925},[247],[170,26927,26929,26940],{"className":26928,"style":23416},[251],[170,26930,26931,26934],{"style":17417},[170,26932],{"className":26933,"style":651},[259],[170,26935,26937],{"className":26936},[264,265,266,267],[170,26938,14157],{"className":26939,"style":3317},[230,303,267],[170,26941,26942,26945],{"style":16532},[170,26943],{"className":26944,"style":651},[259],[170,26946,26948],{"className":26947},[264,265,266,267],[170,26949,2928],{"className":26950},[230,267],[170,26952,338],{"className":26953},[337],[170,26955,26957],{"className":26956},[247],[170,26958,26960],{"className":26959,"style":17447},[251],[170,26961],{},[170,26963,909],{"className":26964},[234],[170,26966,26968,26971],{"className":26967},[230],[170,26969,12908],{"className":26970},[230,303],[170,26972,26974],{"className":26973},[307],[170,26975,26977,26997],{"className":26976},[242,243],[170,26978,26980,26994],{"className":26979},[247],[170,26981,26983],{"className":26982,"style":12838},[251],[170,26984,26985,26988],{"style":1362},[170,26986],{"className":26987,"style":651},[259],[170,26989,26991],{"className":26990},[264,265,266,267],[170,26992,888],{"className":26993},[230,303,267],[170,26995,338],{"className":26996},[337],[170,26998,27000],{"className":26999},[247],[170,27001,27003],{"className":27002,"style":1381},[251],[170,27004],{},[170,27006,914],{"className":27007},[1023],[170,27009],{"className":27010,"style":1027},[610],[170,27012,134],{"className":27013},[230,303],[170,27015,923],{"className":27016},[424],[170,27018],{"className":27019,"style":611},[610],[170,27021,926],{"className":27022},[615],[170,27024],{"className":27025,"style":611},[610],[170,27027,27029,27032,27083,27086,27089],{"className":27028},[221],[170,27030],{"className":27031,"style":26011},[225],[170,27033,27035,27038],{"className":27034},[230],[170,27036,25583],{"className":27037,"style":25604},[230,303],[170,27039,27041],{"className":27040},[307],[170,27042,27044,27075],{"className":27043},[242,243],[170,27045,27047,27072],{"className":27046},[247],[170,27048,27050,27061],{"className":27049,"style":24902},[251],[170,27051,27052,27055],{"style":26032},[170,27053],{"className":27054,"style":651},[259],[170,27056,27058],{"className":27057},[264,265,266,267],[170,27059,888],{"className":27060},[230,303,267],[170,27062,27063,27066],{"style":16532},[170,27064],{"className":27065,"style":651},[259],[170,27067,27069],{"className":27068},[264,265,266,267],[170,27070,783],{"className":27071,"style":813},[230,303,267],[170,27073,338],{"className":27074},[337],[170,27076,27078],{"className":27077},[247],[170,27079,27081],{"className":27080,"style":17447},[251],[170,27082],{},[170,27084],{"className":27085,"style":1983},[610],[170,27087,5884],{"className":27088},[664],[170,27090],{"className":27091,"style":1983},[610],[170,27093,27095,27098,27101],{"className":27094},[221],[170,27096],{"className":27097,"style":26357},[225],[170,27099,19101],{"className":27100,"style":19348},[230,303],[170,27102,27104,27107],{"className":27103},[230],[170,27105,25583],{"className":27106,"style":25604},[230,303],[170,27108,27110],{"className":27109},[307],[170,27111,27113,27153],{"className":27112},[242,243],[170,27114,27116,27150],{"className":27115},[247],[170,27117,27119,27139],{"className":27118,"style":24902},[251],[170,27120,27121,27124],{"style":26032},[170,27122],{"className":27123,"style":651},[259],[170,27125,27127],{"className":27126},[264,265,266,267],[170,27128,27130,27133,27136],{"className":27129},[230,267],[170,27131,888],{"className":27132},[230,303,267],[170,27134,5884],{"className":27135},[664,267],[170,27137,194],{"className":27138},[230,267],[170,27140,27141,27144],{"style":16532},[170,27142],{"className":27143,"style":651},[259],[170,27145,27147],{"className":27146},[264,265,266,267],[170,27148,783],{"className":27149,"style":813},[230,303,267],[170,27151,338],{"className":27152},[337],[170,27154,27156],{"className":27155},[247],[170,27157,27159],{"className":27158,"style":26416},[251],[170,27160],{},[170,27162],{"className":27163},[610,21759],[170,27165,27167,27170,27221,27224,27264,27267,27270,27273,27276,27279,27282],{"className":27166},[221],[170,27168],{"className":27169,"style":23397},[225],[170,27171,27173,27176],{"className":27172},[230],[170,27174,885],{"className":27175},[230,303],[170,27177,27179],{"className":27178},[307],[170,27180,27182,27213],{"className":27181},[242,243],[170,27183,27185,27210],{"className":27184},[247],[170,27186,27188,27199],{"className":27187,"style":23416},[251],[170,27189,27190,27193],{"style":17417},[170,27191],{"className":27192,"style":651},[259],[170,27194,27196],{"className":27195},[264,265,266,267],[170,27197,14157],{"className":27198,"style":3317},[230,303,267],[170,27200,27201,27204],{"style":16532},[170,27202],{"className":27203,"style":651},[259],[170,27205,27207],{"className":27206},[264,265,266,267],[170,27208,4778],{"className":27209},[230,267],[170,27211,338],{"className":27212},[337],[170,27214,27216],{"className":27215},[247],[170,27217,27219],{"className":27218,"style":17447},[251],[170,27220],{},[170,27222,909],{"className":27223},[234],[170,27225,27227,27230],{"className":27226},[230],[170,27228,12908],{"className":27229},[230,303],[170,27231,27233],{"className":27232},[307],[170,27234,27236,27256],{"className":27235},[242,243],[170,27237,27239,27253],{"className":27238},[247],[170,27240,27242],{"className":27241,"style":12838},[251],[170,27243,27244,27247],{"style":1362},[170,27245],{"className":27246,"style":651},[259],[170,27248,27250],{"className":27249},[264,265,266,267],[170,27251,888],{"className":27252},[230,303,267],[170,27254,338],{"className":27255},[337],[170,27257,27259],{"className":27258},[247],[170,27260,27262],{"className":27261,"style":1381},[251],[170,27263],{},[170,27265,914],{"className":27266},[1023],[170,27268],{"className":27269,"style":1027},[610],[170,27271,134],{"className":27272},[230,303],[170,27274,923],{"className":27275},[424],[170,27277],{"className":27278,"style":611},[610],[170,27280,926],{"className":27281},[615],[170,27283],{"className":27284,"style":611},[610],[170,27286,27288,27291,27342,27345,27348],{"className":27287},[221],[170,27289],{"className":27290,"style":26011},[225],[170,27292,27294,27297],{"className":27293},[230],[170,27295,25583],{"className":27296,"style":25604},[230,303],[170,27298,27300],{"className":27299},[307],[170,27301,27303,27334],{"className":27302},[242,243],[170,27304,27306,27331],{"className":27305},[247],[170,27307,27309,27320],{"className":27308,"style":24902},[251],[170,27310,27311,27314],{"style":26032},[170,27312],{"className":27313,"style":651},[259],[170,27315,27317],{"className":27316},[264,265,266,267],[170,27318,888],{"className":27319},[230,303,267],[170,27321,27322,27325],{"style":16532},[170,27323],{"className":27324,"style":651},[259],[170,27326,27328],{"className":27327},[264,265,266,267],[170,27329,783],{"className":27330,"style":813},[230,303,267],[170,27332,338],{"className":27333},[337],[170,27335,27337],{"className":27336},[247],[170,27338,27340],{"className":27339,"style":17447},[251],[170,27341],{},[170,27343],{"className":27344,"style":1983},[610],[170,27346,5884],{"className":27347},[664],[170,27349],{"className":27350,"style":1983},[610],[170,27352,27354,27357,27360,27420,27423,27426],{"className":27353},[221],[170,27355],{"className":27356,"style":26357},[225],[170,27358,19101],{"className":27359,"style":19348},[230,303],[170,27361,27363,27366],{"className":27362},[230],[170,27364,25583],{"className":27365,"style":25604},[230,303],[170,27367,27369],{"className":27368},[307],[170,27370,27372,27412],{"className":27371},[242,243],[170,27373,27375,27409],{"className":27374},[247],[170,27376,27378,27398],{"className":27377,"style":24902},[251],[170,27379,27380,27383],{"style":26032},[170,27381],{"className":27382,"style":651},[259],[170,27384,27386],{"className":27385},[264,265,266,267],[170,27387,27389,27392,27395],{"className":27388},[230,267],[170,27390,888],{"className":27391},[230,303,267],[170,27393,5884],{"className":27394},[664,267],[170,27396,194],{"className":27397},[230,267],[170,27399,27400,27403],{"style":16532},[170,27401],{"className":27402,"style":651},[259],[170,27404,27406],{"className":27405},[264,265,266,267],[170,27407,783],{"className":27408,"style":813},[230,303,267],[170,27410,338],{"className":27411},[337],[170,27413,27415],{"className":27414},[247],[170,27416,27418],{"className":27417,"style":26416},[251],[170,27419],{},[170,27421],{"className":27422,"style":1983},[610],[170,27424,5884],{"className":27425},[664],[170,27427],{"className":27428,"style":1983},[610],[170,27430,27432,27435,27464],{"className":27431},[221],[170,27433],{"className":27434,"style":26357},[225],[170,27436,27438,27441],{"className":27437},[230],[170,27439,19101],{"className":27440,"style":19348},[230,303],[170,27442,27444],{"className":27443},[307],[170,27445,27447],{"className":27446},[242],[170,27448,27450],{"className":27449},[247],[170,27451,27453],{"className":27452,"style":23416},[251],[170,27454,27455,27458],{"style":16532},[170,27456],{"className":27457,"style":651},[259],[170,27459,27461],{"className":27460},[264,265,266,267],[170,27462,2928],{"className":27463},[230,267],[170,27465,27467,27470],{"className":27466},[230],[170,27468,25583],{"className":27469,"style":25604},[230,303],[170,27471,27473],{"className":27472},[307],[170,27474,27476,27516],{"className":27475},[242,243],[170,27477,27479,27513],{"className":27478},[247],[170,27480,27482,27502],{"className":27481,"style":24902},[251],[170,27483,27484,27487],{"style":26032},[170,27485],{"className":27486,"style":651},[259],[170,27488,27490],{"className":27489},[264,265,266,267],[170,27491,27493,27496,27499],{"className":27492},[230,267],[170,27494,888],{"className":27495},[230,303,267],[170,27497,5884],{"className":27498},[664,267],[170,27500,2928],{"className":27501},[230,267],[170,27503,27504,27507],{"style":16532},[170,27505],{"className":27506,"style":651},[259],[170,27508,27510],{"className":27509},[264,265,266,267],[170,27511,783],{"className":27512,"style":813},[230,303,267],[170,27514,338],{"className":27515},[337],[170,27517,27519],{"className":27518},[247],[170,27520,27522],{"className":27521,"style":26416},[251],[170,27523],{},[170,27525],{"className":27526},[610,21759],[170,27528,27530,27533],{"className":27529},[221],[170,27531],{"className":27532,"style":24862},[225],[170,27534,27536,27539],{"className":27535},[230],[170,27537,23244],{"className":27538},[230],[170,27540],{"className":27541,"style":24873},[230,24872],[24,27543,27545],{"id":27544},"generalized-advantage-estimation-gae","Generalized Advantage Estimation (GAE)",[14,27547,27548],{},"这时我们考虑，在运行时我们应该向后采样几步呢？GAE 的答案是，小孩子才做选择，我全都要！",[14,27550,27551,27552,27582],{},"GAE考虑全部的采样结果，并引入一个衰减因子 ",[170,27553,27555,27570],{"className":27554},[173],[170,27556,27558],{"className":27557},[177],[179,27559,27560],{"xmlns":181},[183,27561,27562,27567],{},[186,27563,27564],{},[202,27565,27566],{},"λ",[209,27568,27569],{"encoding":211},"\\lambda",[170,27571,27573],{"className":27572,"ariaHidden":217},[216],[170,27574,27576,27579],{"className":27575},[221],[170,27577],{"className":27578,"style":518},[225],[170,27580,27566],{"className":27581},[230,303]," 。",[170,27584,27586],{"className":27585},[14456],[170,27587,27589,28180],{"className":27588},[173],[170,27590,27592],{"className":27591},[177],[179,27593,27594],{"xmlns":181,"display":9588},[183,27595,27596,28177],{},[14468,27597,27598,27776,27918,28028,28118],{"rowspacing":14470,"columnalign":14471,"columnspacing":14472},[14474,27599,27600,27635],{},[14477,27601,27602],{},[14480,27603,27604],{"scriptlevel":1865,"displaystyle":217},[186,27605,27606,27621,27623,27629,27631,27633],{},[1933,27607,27608,27610,27612],{},[202,27609,885],{},[202,27611,14157],{},[186,27613,27614,27617,27619],{},[202,27615,27616],{},"G",[202,27618,885],{},[202,27620,5714],{},[572,27622,909],{"stretchy":908},[199,27624,27625,27627],{},[202,27626,12908],{},[202,27628,888],{},[572,27630,914],{"separator":217},[202,27632,134],{},[572,27634,923],{"stretchy":908},[14477,27636,27637],{},[14480,27638,27639],{"scriptlevel":1865,"displaystyle":217},[186,27640,27641,27643,27645,27647,27649,27651,27653,27655,27657,27665,27667,27669,27671,27679,27681,27687,27695,27697,27699,27702,27704,27707,27709,27711,27713,27716,27718,27720,27734,27736,27738,27746,27748,27751,27759,27761,27764,27772,27774],{},[186,27642],{},[572,27644,926],{},[572,27646,909],{"stretchy":908},[192,27648,194],{},[572,27650,18471],{},[202,27652,27566],{},[572,27654,923],{"stretchy":908},[572,27656,909],{"stretchy":908},[1933,27658,27659,27661,27663],{},[202,27660,885],{},[202,27662,14157],{},[192,27664,194],{},[572,27666,5884],{},[202,27668,27566],{},[572,27670,23155],{},[1933,27672,27673,27675,27677],{},[202,27674,885],{},[202,27676,14157],{},[192,27678,2928],{},[572,27680,5884],{},[576,27682,27683,27685],{},[202,27684,27566],{},[192,27686,2928],{},[1933,27688,27689,27691,27693],{},[202,27690,885],{},[202,27692,14157],{},[192,27694,4778],{},[572,27696,5884],{},[572,27698,23350],{},[3384,27700,27701],{}," ",[572,27703,923],{"stretchy":908},[610,27705],{"width":27706},"1em",[610,27708],{"width":27706},[202,27710,27566],{},[572,27712,926],{},[192,27714,27715],{},"0.9",[572,27717,7291],{},[610,27719],{"width":27706},[1933,27721,27722,27724,27726],{},[202,27723,885],{},[202,27725,14157],{},[186,27727,27728,27730,27732],{},[202,27729,27616],{},[202,27731,885],{},[202,27733,5714],{},[572,27735,926],{},[192,27737,12573],{},[1933,27739,27740,27742,27744],{},[202,27741,885],{},[202,27743,14157],{},[192,27745,194],{},[572,27747,5884],{},[192,27749,27750],{},"0.09",[1933,27752,27753,27755,27757],{},[202,27754,885],{},[202,27756,14157],{},[192,27758,2928],{},[572,27760,5884],{},[192,27762,27763],{},"0.081",[1933,27765,27766,27768,27770],{},[202,27767,885],{},[202,27769,14157],{},[192,27771,4778],{},[572,27773,5884],{},[572,27775,23350],{},[14474,27777,27778,27784],{},[14477,27779,27780],{},[14480,27781,27782],{"scriptlevel":1865,"displaystyle":217},[186,27783],{},[14477,27785,27786],{},[14480,27787,27788],{"scriptlevel":1865,"displaystyle":217},[186,27789,27790,27792,27794,27796,27798,27800,27802,27804,27806,27814,27816,27818,27820,27822,27830,27832,27834,27848,27850,27852,27858,27860,27868,27870,27872,27886,27888,27894,27908,27910,27912,27914,27916],{},[186,27791],{},[572,27793,926],{},[572,27795,909],{"stretchy":908},[192,27797,194],{},[572,27799,18471],{},[202,27801,27566],{},[572,27803,923],{"stretchy":908},[572,27805,909],{"stretchy":908},[1933,27807,27808,27810,27812],{},[202,27809,25583],{},[202,27811,888],{},[202,27813,783],{},[572,27815,5884],{},[202,27817,27566],{},[572,27819,23155],{},[572,27821,909],{"stretchy":908},[1933,27823,27824,27826,27828],{},[202,27825,25583],{},[202,27827,888],{},[202,27829,783],{},[572,27831,5884],{},[202,27833,19101],{},[1933,27835,27836,27838,27846],{},[202,27837,25583],{},[186,27839,27840,27842,27844],{},[202,27841,888],{},[572,27843,5884],{},[192,27845,194],{},[202,27847,783],{},[572,27849,923],{"stretchy":908},[572,27851,5884],{},[576,27853,27854,27856],{},[202,27855,27566],{},[192,27857,2928],{},[572,27859,909],{"stretchy":908},[1933,27861,27862,27864,27866],{},[202,27863,25583],{},[202,27865,888],{},[202,27867,783],{},[572,27869,5884],{},[202,27871,19101],{},[1933,27873,27874,27876,27884],{},[202,27875,25583],{},[186,27877,27878,27880,27882],{},[202,27879,888],{},[572,27881,5884],{},[192,27883,194],{},[202,27885,783],{},[572,27887,5884],{},[576,27889,27890,27892],{},[202,27891,19101],{},[192,27893,2928],{},[1933,27895,27896,27898,27906],{},[202,27897,25583],{},[186,27899,27900,27902,27904],{},[202,27901,888],{},[572,27903,5884],{},[192,27905,2928],{},[202,27907,783],{},[572,27909,923],{"stretchy":908},[572,27911,5884],{},[572,27913,23350],{},[3384,27915,27701],{},[572,27917,923],{"stretchy":908},[14474,27919,27920,27926],{},[14477,27921,27922],{},[14480,27923,27924],{"scriptlevel":1865,"displaystyle":217},[186,27925],{},[14477,27927,27928],{},[14480,27929,27930],{"scriptlevel":1865,"displaystyle":217},[186,27931,27932,27934,27936,27938,27940,27942,27944,27946,27948,27956,27958,27960,27962,27964,27966,27972,27974,27976,27978,27980,27982,27984,27998,28000,28002,28004,28006,28012,28014,28016,28018,28020,28022,28024,28026],{},[186,27933],{},[572,27935,926],{},[572,27937,909],{"stretchy":908},[192,27939,194],{},[572,27941,18471],{},[202,27943,27566],{},[572,27945,923],{"stretchy":908},[572,27947,909],{"stretchy":908},[1933,27949,27950,27952,27954],{},[202,27951,25583],{},[202,27953,888],{},[202,27955,783],{},[572,27957,909],{"stretchy":908},[192,27959,194],{},[572,27961,5884],{},[202,27963,27566],{},[572,27965,5884],{},[576,27967,27968,27970],{},[202,27969,27566],{},[192,27971,2928],{},[572,27973,5884],{},[572,27975,23350],{},[3384,27977,27701],{},[572,27979,923],{"stretchy":908},[572,27981,5884],{},[202,27983,19101],{},[1933,27985,27986,27988,27996],{},[202,27987,25583],{},[186,27989,27990,27992,27994],{},[202,27991,888],{},[572,27993,5884],{},[192,27995,194],{},[202,27997,783],{},[572,27999,23155],{},[572,28001,909],{"stretchy":908},[202,28003,27566],{},[572,28005,5884],{},[576,28007,28008,28010],{},[202,28009,27566],{},[192,28011,2928],{},[572,28013,5884],{},[572,28015,23350],{},[3384,28017,27701],{},[572,28019,923],{"stretchy":908},[572,28021,5884],{},[572,28023,23350],{},[3384,28025,27701],{},[572,28027,923],{"stretchy":908},[14474,28029,28030,28036],{},[14477,28031,28032],{},[14480,28033,28034],{"scriptlevel":1865,"displaystyle":217},[186,28035],{},[14477,28037,28038],{},[14480,28039,28040],{"scriptlevel":1865,"displaystyle":217},[186,28041,28042,28044,28046,28048,28050,28052,28054,28056],{},[186,28043],{},[572,28045,926],{},[572,28047,909],{"stretchy":908},[192,28049,194],{},[572,28051,18471],{},[202,28053,27566],{},[572,28055,923],{"stretchy":908},[186,28057,28058,28060,28068,28080,28082,28084,28098,28110,28112,28114,28116],{},[572,28059,909],{"fence":217},[1933,28061,28062,28064,28066],{},[202,28063,25583],{},[202,28065,888],{},[202,28067,783],{},[189,28069,28070,28072],{},[192,28071,194],{},[186,28073,28074,28076,28078],{},[192,28075,194],{},[572,28077,18471],{},[202,28079,27566],{},[572,28081,5884],{},[202,28083,19101],{},[1933,28085,28086,28088,28096],{},[202,28087,25583],{},[186,28089,28090,28092,28094],{},[202,28091,888],{},[572,28093,5884],{},[192,28095,194],{},[202,28097,783],{},[189,28099,28100,28102],{},[202,28101,27566],{},[186,28103,28104,28106,28108],{},[192,28105,194],{},[572,28107,18471],{},[202,28109,27566],{},[572,28111,5884],{},[572,28113,23350],{},[3384,28115,27701],{},[572,28117,923],{"fence":217},[14474,28119,28120,28126],{},[14477,28121,28122],{},[14480,28123,28124],{"scriptlevel":1865,"displaystyle":217},[186,28125],{},[14477,28127,28128],{},[14480,28129,28130],{"scriptlevel":1865,"displaystyle":217},[186,28131,28132,28134,28136,28151,28153,28155,28157,28163],{},[186,28133],{},[572,28135,926],{},[14789,28137,28138,28140,28148],{},[572,28139,1937],{},[186,28141,28142,28144,28146],{},[202,28143,7464],{},[572,28145,926],{},[192,28147,1865],{},[202,28149,28150],{"mathvariant":929},"∞",[572,28152,909],{"stretchy":908},[202,28154,19101],{},[202,28156,27566],{},[576,28158,28159,28161],{},[572,28160,923],{"stretchy":908},[202,28162,7464],{},[1933,28164,28165,28167,28175],{},[202,28166,25583],{},[186,28168,28169,28171,28173],{},[202,28170,888],{},[572,28172,5884],{},[202,28174,7464],{},[202,28176,783],{},[209,28178,28179],{"encoding":211},"\\begin{aligned}\nA_\\theta^{GAE}(s_t, a) &= (1 - \\lambda)(A_\\theta^1 + \\lambda * A_\\theta^2 + \\lambda^2 A_\\theta^3 + \\cdots) \\quad\\quad \\lambda = 0.9: \\quad A_\\theta^{GAE} = 0.1A_\\theta^1 + 0.09A_\\theta^2 + 0.081A_\\theta^3 + \\cdots \\\\\n&= (1 - \\lambda)(\\delta_t^V + \\lambda * (\\delta_t^V + \\gamma \\delta_{t+1}^V) + \\lambda^2 (\\delta_t^V + \\gamma \\delta_{t+1}^V + \\gamma^2 \\delta_{t+2}^V) + \\cdots) \\\\\n&= (1 - \\lambda)(\\delta_t^V (1 + \\lambda + \\lambda^2 + \\cdots) + \\gamma \\delta_{t+1}^V * (\\lambda + \\lambda^2 + \\cdots) + \\cdots) \\\\\n&= (1 - \\lambda)\\left(\\delta_t^V \\frac{1}{1 - \\lambda} + \\gamma \\delta_{t+1}^V \\frac{\\lambda}{1 - \\lambda} + \\cdots\\right) \\\\\n&= \\sum\\limits_{b=0}^\\infty (\\gamma\\lambda)^b \\delta_{t+b}^V\n\\end{aligned}",[170,28181,28183],{"className":28182,"ariaHidden":217},[216],[170,28184,28186,28190],{"className":28185},[221],[170,28187],{"className":28188,"style":28189},[225],"height:10.6075em;vertical-align:-5.0538em;",[170,28191,28193],{"className":28192},[230],[170,28194,28196,28382],{"className":28195},[14468],[170,28197,28199],{"className":28198},[15259],[170,28200,28202,28373],{"className":28201},[242,243],[170,28203,28205,28370],{"className":28204},[247],[170,28206,28209,28334,28343,28352,28361],{"className":28207,"style":28208},[251],"height:5.5538em;",[170,28210,28212,28216],{"style":28211},"top:-8.3138em;",[170,28213],{"className":28214,"style":28215},[259],"height:3.6514em;",[170,28217,28219,28279,28282,28322,28325,28328,28331],{"className":28218},[230],[170,28220,28222,28225],{"className":28221},[230],[170,28223,885],{"className":28224},[230,303],[170,28226,28228],{"className":28227},[307],[170,28229,28231,28271],{"className":28230},[242,243],[170,28232,28234,28268],{"className":28233},[247],[170,28235,28237,28248],{"className":28236,"style":24902},[251],[170,28238,28239,28242],{"style":17417},[170,28240],{"className":28241,"style":651},[259],[170,28243,28245],{"className":28244},[264,265,266,267],[170,28246,14157],{"className":28247,"style":3317},[230,303,267],[170,28249,28250,28253],{"style":16532},[170,28251],{"className":28252,"style":651},[259],[170,28254,28256],{"className":28255},[264,265,266,267],[170,28257,28259,28262,28265],{"className":28258},[230,267],[170,28260,27616],{"className":28261},[230,303,267],[170,28263,885],{"className":28264},[230,303,267],[170,28266,5714],{"className":28267,"style":6157},[230,303,267],[170,28269,338],{"className":28270},[337],[170,28272,28274],{"className":28273},[247],[170,28275,28277],{"className":28276,"style":17447},[251],[170,28278],{},[170,28280,909],{"className":28281},[234],[170,28283,28285,28288],{"className":28284},[230],[170,28286,12908],{"className":28287},[230,303],[170,28289,28291],{"className":28290},[307],[170,28292,28294,28314],{"className":28293},[242,243],[170,28295,28297,28311],{"className":28296},[247],[170,28298,28300],{"className":28299,"style":12838},[251],[170,28301,28302,28305],{"style":1362},[170,28303],{"className":28304,"style":651},[259],[170,28306,28308],{"className":28307},[264,265,266,267],[170,28309,888],{"className":28310},[230,303,267],[170,28312,338],{"className":28313},[337],[170,28315,28317],{"className":28316},[247],[170,28318,28320],{"className":28319,"style":1381},[251],[170,28321],{},[170,28323,914],{"className":28324},[1023],[170,28326],{"className":28327,"style":1027},[610],[170,28329,134],{"className":28330},[230,303],[170,28332,923],{"className":28333},[424],[170,28335,28337,28340],{"style":28336},"top:-6.7625em;",[170,28338],{"className":28339,"style":28215},[259],[170,28341],{"className":28342},[230],[170,28344,28346,28349],{"style":28345},"top:-5.2112em;",[170,28347],{"className":28348,"style":28215},[259],[170,28350],{"className":28351},[230],[170,28353,28355,28358],{"style":28354},"top:-3.1012em;",[170,28356],{"className":28357,"style":28215},[259],[170,28359],{"className":28360},[230],[170,28362,28364,28367],{"style":28363},"top:-0.1997em;",[170,28365],{"className":28366,"style":28215},[259],[170,28368],{"className":28369},[230],[170,28371,338],{"className":28372},[337],[170,28374,28376],{"className":28375},[247],[170,28377,28380],{"className":28378,"style":28379},[251],"height:5.0538em;",[170,28381],{},[170,28383,28385],{"className":28384},[15483],[170,28386,28388,30357],{"className":28387},[242,243],[170,28389,28391,30354],{"className":28390},[247],[170,28392,28394,28963,29491,29821,30168],{"className":28393,"style":28208},[251],[170,28395,28396,28399],{"style":28211},[170,28397],{"className":28398,"style":28215},[259],[170,28400,28402,28405,28408,28411,28414,28417,28420,28423,28426,28429,28432,28435,28438,28489,28492,28495,28498,28501,28504,28507,28510,28561,28564,28567,28570,28599,28650,28653,28656,28659,28662,28665,28668,28672,28675,28678,28681,28684,28687,28690,28693,28696,28699,28702,28762,28765,28768,28771,28774,28825,28828,28831,28834,28837,28888,28891,28894,28897,28900,28951,28954,28957,28960],{"className":28401},[230],[170,28403],{"className":28404},[230],[170,28406],{"className":28407,"style":611},[610],[170,28409,926],{"className":28410},[615],[170,28412],{"className":28413,"style":611},[610],[170,28415,909],{"className":28416},[234],[170,28418,194],{"className":28419},[230],[170,28421],{"className":28422,"style":1983},[610],[170,28424,18471],{"className":28425},[664],[170,28427],{"className":28428,"style":1983},[610],[170,28430,27566],{"className":28431},[230,303],[170,28433,923],{"className":28434},[424],[170,28436,909],{"className":28437},[234],[170,28439,28441,28444],{"className":28440},[230],[170,28442,885],{"className":28443},[230,303],[170,28445,28447],{"className":28446},[307],[170,28448,28450,28481],{"className":28449},[242,243],[170,28451,28453,28478],{"className":28452},[247],[170,28454,28456,28467],{"className":28455,"style":23416},[251],[170,28457,28458,28461],{"style":17417},[170,28459],{"className":28460,"style":651},[259],[170,28462,28464],{"className":28463},[264,265,266,267],[170,28465,14157],{"className":28466,"style":3317},[230,303,267],[170,28468,28469,28472],{"style":16532},[170,28470],{"className":28471,"style":651},[259],[170,28473,28475],{"className":28474},[264,265,266,267],[170,28476,194],{"className":28477},[230,267],[170,28479,338],{"className":28480},[337],[170,28482,28484],{"className":28483},[247],[170,28485,28487],{"className":28486,"style":17447},[251],[170,28488],{},[170,28490],{"className":28491,"style":1983},[610],[170,28493,5884],{"className":28494},[664],[170,28496],{"className":28497,"style":1983},[610],[170,28499,27566],{"className":28500},[230,303],[170,28502],{"className":28503,"style":1983},[610],[170,28505,23155],{"className":28506},[664],[170,28508],{"className":28509,"style":1983},[610],[170,28511,28513,28516],{"className":28512},[230],[170,28514,885],{"className":28515},[230,303],[170,28517,28519],{"className":28518},[307],[170,28520,28522,28553],{"className":28521},[242,243],[170,28523,28525,28550],{"className":28524},[247],[170,28526,28528,28539],{"className":28527,"style":23416},[251],[170,28529,28530,28533],{"style":17417},[170,28531],{"className":28532,"style":651},[259],[170,28534,28536],{"className":28535},[264,265,266,267],[170,28537,14157],{"className":28538,"style":3317},[230,303,267],[170,28540,28541,28544],{"style":16532},[170,28542],{"className":28543,"style":651},[259],[170,28545,28547],{"className":28546},[264,265,266,267],[170,28548,2928],{"className":28549},[230,267],[170,28551,338],{"className":28552},[337],[170,28554,28556],{"className":28555},[247],[170,28557,28559],{"className":28558,"style":17447},[251],[170,28560],{},[170,28562],{"className":28563,"style":1983},[610],[170,28565,5884],{"className":28566},[664],[170,28568],{"className":28569,"style":1983},[610],[170,28571,28573,28576],{"className":28572},[230],[170,28574,27566],{"className":28575},[230,303],[170,28577,28579],{"className":28578},[307],[170,28580,28582],{"className":28581},[242],[170,28583,28585],{"className":28584},[247],[170,28586,28588],{"className":28587,"style":23416},[251],[170,28589,28590,28593],{"style":16532},[170,28591],{"className":28592,"style":651},[259],[170,28594,28596],{"className":28595},[264,265,266,267],[170,28597,2928],{"className":28598},[230,267],[170,28600,28602,28605],{"className":28601},[230],[170,28603,885],{"className":28604},[230,303],[170,28606,28608],{"className":28607},[307],[170,28609,28611,28642],{"className":28610},[242,243],[170,28612,28614,28639],{"className":28613},[247],[170,28615,28617,28628],{"className":28616,"style":23416},[251],[170,28618,28619,28622],{"style":17417},[170,28620],{"className":28621,"style":651},[259],[170,28623,28625],{"className":28624},[264,265,266,267],[170,28626,14157],{"className":28627,"style":3317},[230,303,267],[170,28629,28630,28633],{"style":16532},[170,28631],{"className":28632,"style":651},[259],[170,28634,28636],{"className":28635},[264,265,266,267],[170,28637,4778],{"className":28638},[230,267],[170,28640,338],{"className":28641},[337],[170,28643,28645],{"className":28644},[247],[170,28646,28648],{"className":28647,"style":17447},[251],[170,28649],{},[170,28651],{"className":28652,"style":1983},[610],[170,28654,5884],{"className":28655},[664],[170,28657],{"className":28658,"style":1983},[610],[170,28660,23350],{"className":28661},[1073],[170,28663],{"className":28664,"style":1027},[610],[170,28666,923],{"className":28667},[424],[170,28669],{"className":28670,"style":28671},[610],"margin-right:1em;",[170,28673],{"className":28674,"style":28671},[610],[170,28676,27566],{"className":28677},[230,303],[170,28679],{"className":28680,"style":611},[610],[170,28682,926],{"className":28683},[615],[170,28685],{"className":28686,"style":611},[610],[170,28688,27715],{"className":28689},[230],[170,28691],{"className":28692,"style":611},[610],[170,28694,7291],{"className":28695},[615],[170,28697],{"className":28698,"style":28671},[610],[170,28700],{"className":28701,"style":611},[610],[170,28703,28705,28708],{"className":28704},[230],[170,28706,885],{"className":28707},[230,303],[170,28709,28711],{"className":28710},[307],[170,28712,28714,28754],{"className":28713},[242,243],[170,28715,28717,28751],{"className":28716},[247],[170,28718,28720,28731],{"className":28719,"style":24902},[251],[170,28721,28722,28725],{"style":17417},[170,28723],{"className":28724,"style":651},[259],[170,28726,28728],{"className":28727},[264,265,266,267],[170,28729,14157],{"className":28730,"style":3317},[230,303,267],[170,28732,28733,28736],{"style":16532},[170,28734],{"className":28735,"style":651},[259],[170,28737,28739],{"className":28738},[264,265,266,267],[170,28740,28742,28745,28748],{"className":28741},[230,267],[170,28743,27616],{"className":28744},[230,303,267],[170,28746,885],{"className":28747},[230,303,267],[170,28749,5714],{"className":28750,"style":6157},[230,303,267],[170,28752,338],{"className":28753},[337],[170,28755,28757],{"className":28756},[247],[170,28758,28760],{"className":28759,"style":17447},[251],[170,28761],{},[170,28763],{"className":28764,"style":611},[610],[170,28766,926],{"className":28767},[615],[170,28769],{"className":28770,"style":611},[610],[170,28772,12573],{"className":28773},[230],[170,28775,28777,28780],{"className":28776},[230],[170,28778,885],{"className":28779},[230,303],[170,28781,28783],{"className":28782},[307],[170,28784,28786,28817],{"className":28785},[242,243],[170,28787,28789,28814],{"className":28788},[247],[170,28790,28792,28803],{"className":28791,"style":23416},[251],[170,28793,28794,28797],{"style":17417},[170,28795],{"className":28796,"style":651},[259],[170,28798,28800],{"className":28799},[264,265,266,267],[170,28801,14157],{"className":28802,"style":3317},[230,303,267],[170,28804,28805,28808],{"style":16532},[170,28806],{"className":28807,"style":651},[259],[170,28809,28811],{"className":28810},[264,265,266,267],[170,28812,194],{"className":28813},[230,267],[170,28815,338],{"className":28816},[337],[170,28818,28820],{"className":28819},[247],[170,28821,28823],{"className":28822,"style":17447},[251],[170,28824],{},[170,28826],{"className":28827,"style":1983},[610],[170,28829,5884],{"className":28830},[664],[170,28832],{"className":28833,"style":1983},[610],[170,28835,27750],{"className":28836},[230],[170,28838,28840,28843],{"className":28839},[230],[170,28841,885],{"className":28842},[230,303],[170,28844,28846],{"className":28845},[307],[170,28847,28849,28880],{"className":28848},[242,243],[170,28850,28852,28877],{"className":28851},[247],[170,28853,28855,28866],{"className":28854,"style":23416},[251],[170,28856,28857,28860],{"style":17417},[170,28858],{"className":28859,"style":651},[259],[170,28861,28863],{"className":28862},[264,265,266,267],[170,28864,14157],{"className":28865,"style":3317},[230,303,267],[170,28867,28868,28871],{"style":16532},[170,28869],{"className":28870,"style":651},[259],[170,28872,28874],{"className":28873},[264,265,266,267],[170,28875,2928],{"className":28876},[230,267],[170,28878,338],{"className":28879},[337],[170,28881,28883],{"className":28882},[247],[170,28884,28886],{"className":28885,"style":17447},[251],[170,28887],{},[170,28889],{"className":28890,"style":1983},[610],[170,28892,5884],{"className":28893},[664],[170,28895],{"className":28896,"style":1983},[610],[170,28898,27763],{"className":28899},[230],[170,28901,28903,28906],{"className":28902},[230],[170,28904,885],{"className":28905},[230,303],[170,28907,28909],{"className":28908},[307],[170,28910,28912,28943],{"className":28911},[242,243],[170,28913,28915,28940],{"className":28914},[247],[170,28916,28918,28929],{"className":28917,"style":23416},[251],[170,28919,28920,28923],{"style":17417},[170,28921],{"className":28922,"style":651},[259],[170,28924,28926],{"className":28925},[264,265,266,267],[170,28927,14157],{"className":28928,"style":3317},[230,303,267],[170,28930,28931,28934],{"style":16532},[170,28932],{"className":28933,"style":651},[259],[170,28935,28937],{"className":28936},[264,265,266,267],[170,28938,4778],{"className":28939},[230,267],[170,28941,338],{"className":28942},[337],[170,28944,28946],{"className":28945},[247],[170,28947,28949],{"className":28948,"style":17447},[251],[170,28950],{},[170,28952],{"className":28953,"style":1983},[610],[170,28955,5884],{"className":28956},[664],[170,28958],{"className":28959,"style":1983},[610],[170,28961,23350],{"className":28962},[1073],[170,28964,28965,28968],{"style":28336},[170,28966],{"className":28967,"style":28215},[259],[170,28969,28971,28974,28977,28980,28983,28986,28989,28992,28995,28998,29001,29004,29007,29058,29061,29064,29067,29070,29073,29076,29079,29082,29133,29136,29139,29142,29145,29205,29208,29211,29214,29217,29246,29249,29300,29303,29306,29309,29312,29372,29375,29378,29381,29410,29470,29473,29476,29479,29482,29485,29488],{"className":28970},[230],[170,28972],{"className":28973},[230],[170,28975],{"className":28976,"style":611},[610],[170,28978,926],{"className":28979},[615],[170,28981],{"className":28982,"style":611},[610],[170,28984,909],{"className":28985},[234],[170,28987,194],{"className":28988},[230],[170,28990],{"className":28991,"style":1983},[610],[170,28993,18471],{"className":28994},[664],[170,28996],{"className":28997,"style":1983},[610],[170,28999,27566],{"className":29000},[230,303],[170,29002,923],{"className":29003},[424],[170,29005,909],{"className":29006},[234],[170,29008,29010,29013],{"className":29009},[230],[170,29011,25583],{"className":29012,"style":25604},[230,303],[170,29014,29016],{"className":29015},[307],[170,29017,29019,29050],{"className":29018},[242,243],[170,29020,29022,29047],{"className":29021},[247],[170,29023,29025,29036],{"className":29024,"style":24902},[251],[170,29026,29027,29030],{"style":26032},[170,29028],{"className":29029,"style":651},[259],[170,29031,29033],{"className":29032},[264,265,266,267],[170,29034,888],{"className":29035},[230,303,267],[170,29037,29038,29041],{"style":16532},[170,29039],{"className":29040,"style":651},[259],[170,29042,29044],{"className":29043},[264,265,266,267],[170,29045,783],{"className":29046,"style":813},[230,303,267],[170,29048,338],{"className":29049},[337],[170,29051,29053],{"className":29052},[247],[170,29054,29056],{"className":29055,"style":17447},[251],[170,29057],{},[170,29059],{"className":29060,"style":1983},[610],[170,29062,5884],{"className":29063},[664],[170,29065],{"className":29066,"style":1983},[610],[170,29068,27566],{"className":29069},[230,303],[170,29071],{"className":29072,"style":1983},[610],[170,29074,23155],{"className":29075},[664],[170,29077],{"className":29078,"style":1983},[610],[170,29080,909],{"className":29081},[234],[170,29083,29085,29088],{"className":29084},[230],[170,29086,25583],{"className":29087,"style":25604},[230,303],[170,29089,29091],{"className":29090},[307],[170,29092,29094,29125],{"className":29093},[242,243],[170,29095,29097,29122],{"className":29096},[247],[170,29098,29100,29111],{"className":29099,"style":24902},[251],[170,29101,29102,29105],{"style":26032},[170,29103],{"className":29104,"style":651},[259],[170,29106,29108],{"className":29107},[264,265,266,267],[170,29109,888],{"className":29110},[230,303,267],[170,29112,29113,29116],{"style":16532},[170,29114],{"className":29115,"style":651},[259],[170,29117,29119],{"className":29118},[264,265,266,267],[170,29120,783],{"className":29121,"style":813},[230,303,267],[170,29123,338],{"className":29124},[337],[170,29126,29128],{"className":29127},[247],[170,29129,29131],{"className":29130,"style":17447},[251],[170,29132],{},[170,29134],{"className":29135,"style":1983},[610],[170,29137,5884],{"className":29138},[664],[170,29140],{"className":29141,"style":1983},[610],[170,29143,19101],{"className":29144,"style":19348},[230,303],[170,29146,29148,29151],{"className":29147},[230],[170,29149,25583],{"className":29150,"style":25604},[230,303],[170,29152,29154],{"className":29153},[307],[170,29155,29157,29197],{"className":29156},[242,243],[170,29158,29160,29194],{"className":29159},[247],[170,29161,29163,29183],{"className":29162,"style":24902},[251],[170,29164,29165,29168],{"style":26032},[170,29166],{"className":29167,"style":651},[259],[170,29169,29171],{"className":29170},[264,265,266,267],[170,29172,29174,29177,29180],{"className":29173},[230,267],[170,29175,888],{"className":29176},[230,303,267],[170,29178,5884],{"className":29179},[664,267],[170,29181,194],{"className":29182},[230,267],[170,29184,29185,29188],{"style":16532},[170,29186],{"className":29187,"style":651},[259],[170,29189,29191],{"className":29190},[264,265,266,267],[170,29192,783],{"className":29193,"style":813},[230,303,267],[170,29195,338],{"className":29196},[337],[170,29198,29200],{"className":29199},[247],[170,29201,29203],{"className":29202,"style":26416},[251],[170,29204],{},[170,29206,923],{"className":29207},[424],[170,29209],{"className":29210,"style":1983},[610],[170,29212,5884],{"className":29213},[664],[170,29215],{"className":29216,"style":1983},[610],[170,29218,29220,29223],{"className":29219},[230],[170,29221,27566],{"className":29222},[230,303],[170,29224,29226],{"className":29225},[307],[170,29227,29229],{"className":29228},[242],[170,29230,29232],{"className":29231},[247],[170,29233,29235],{"className":29234,"style":23416},[251],[170,29236,29237,29240],{"style":16532},[170,29238],{"className":29239,"style":651},[259],[170,29241,29243],{"className":29242},[264,265,266,267],[170,29244,2928],{"className":29245},[230,267],[170,29247,909],{"className":29248},[234],[170,29250,29252,29255],{"className":29251},[230],[170,29253,25583],{"className":29254,"style":25604},[230,303],[170,29256,29258],{"className":29257},[307],[170,29259,29261,29292],{"className":29260},[242,243],[170,29262,29264,29289],{"className":29263},[247],[170,29265,29267,29278],{"className":29266,"style":24902},[251],[170,29268,29269,29272],{"style":26032},[170,29270],{"className":29271,"style":651},[259],[170,29273,29275],{"className":29274},[264,265,266,267],[170,29276,888],{"className":29277},[230,303,267],[170,29279,29280,29283],{"style":16532},[170,29281],{"className":29282,"style":651},[259],[170,29284,29286],{"className":29285},[264,265,266,267],[170,29287,783],{"className":29288,"style":813},[230,303,267],[170,29290,338],{"className":29291},[337],[170,29293,29295],{"className":29294},[247],[170,29296,29298],{"className":29297,"style":17447},[251],[170,29299],{},[170,29301],{"className":29302,"style":1983},[610],[170,29304,5884],{"className":29305},[664],[170,29307],{"className":29308,"style":1983},[610],[170,29310,19101],{"className":29311,"style":19348},[230,303],[170,29313,29315,29318],{"className":29314},[230],[170,29316,25583],{"className":29317,"style":25604},[230,303],[170,29319,29321],{"className":29320},[307],[170,29322,29324,29364],{"className":29323},[242,243],[170,29325,29327,29361],{"className":29326},[247],[170,29328,29330,29350],{"className":29329,"style":24902},[251],[170,29331,29332,29335],{"style":26032},[170,29333],{"className":29334,"style":651},[259],[170,29336,29338],{"className":29337},[264,265,266,267],[170,29339,29341,29344,29347],{"className":29340},[230,267],[170,29342,888],{"className":29343},[230,303,267],[170,29345,5884],{"className":29346},[664,267],[170,29348,194],{"className":29349},[230,267],[170,29351,29352,29355],{"style":16532},[170,29353],{"className":29354,"style":651},[259],[170,29356,29358],{"className":29357},[264,265,266,267],[170,29359,783],{"className":29360,"style":813},[230,303,267],[170,29362,338],{"className":29363},[337],[170,29365,29367],{"className":29366},[247],[170,29368,29370],{"className":29369,"style":26416},[251],[170,29371],{},[170,29373],{"className":29374,"style":1983},[610],[170,29376,5884],{"className":29377},[664],[170,29379],{"className":29380,"style":1983},[610],[170,29382,29384,29387],{"className":29383},[230],[170,29385,19101],{"className":29386,"style":19348},[230,303],[170,29388,29390],{"className":29389},[307],[170,29391,29393],{"className":29392},[242],[170,29394,29396],{"className":29395},[247],[170,29397,29399],{"className":29398,"style":23416},[251],[170,29400,29401,29404],{"style":16532},[170,29402],{"className":29403,"style":651},[259],[170,29405,29407],{"className":29406},[264,265,266,267],[170,29408,2928],{"className":29409},[230,267],[170,29411,29413,29416],{"className":29412},[230],[170,29414,25583],{"className":29415,"style":25604},[230,303],[170,29417,29419],{"className":29418},[307],[170,29420,29422,29462],{"className":29421},[242,243],[170,29423,29425,29459],{"className":29424},[247],[170,29426,29428,29448],{"className":29427,"style":24902},[251],[170,29429,29430,29433],{"style":26032},[170,29431],{"className":29432,"style":651},[259],[170,29434,29436],{"className":29435},[264,265,266,267],[170,29437,29439,29442,29445],{"className":29438},[230,267],[170,29440,888],{"className":29441},[230,303,267],[170,29443,5884],{"className":29444},[664,267],[170,29446,2928],{"className":29447},[230,267],[170,29449,29450,29453],{"style":16532},[170,29451],{"className":29452,"style":651},[259],[170,29454,29456],{"className":29455},[264,265,266,267],[170,29457,783],{"className":29458,"style":813},[230,303,267],[170,29460,338],{"className":29461},[337],[170,29463,29465],{"className":29464},[247],[170,29466,29468],{"className":29467,"style":26416},[251],[170,29469],{},[170,29471,923],{"className":29472},[424],[170,29474],{"className":29475,"style":1983},[610],[170,29477,5884],{"className":29478},[664],[170,29480],{"className":29481,"style":1983},[610],[170,29483,23350],{"className":29484},[1073],[170,29486],{"className":29487,"style":1027},[610],[170,29489,923],{"className":29490},[424],[170,29492,29493,29496],{"style":28345},[170,29494],{"className":29495,"style":28215},[259],[170,29497,29499,29502,29505,29508,29511,29514,29517,29520,29523,29526,29529,29532,29535,29586,29589,29592,29595,29598,29601,29604,29607,29610,29613,29642,29645,29648,29651,29654,29657,29660,29663,29666,29669,29672,29732,29735,29738,29741,29744,29747,29750,29753,29756,29785,29788,29791,29794,29797,29800,29803,29806,29809,29812,29815,29818],{"className":29498},[230],[170,29500],{"className":29501},[230],[170,29503],{"className":29504,"style":611},[610],[170,29506,926],{"className":29507},[615],[170,29509],{"className":29510,"style":611},[610],[170,29512,909],{"className":29513},[234],[170,29515,194],{"className":29516},[230],[170,29518],{"className":29519,"style":1983},[610],[170,29521,18471],{"className":29522},[664],[170,29524],{"className":29525,"style":1983},[610],[170,29527,27566],{"className":29528},[230,303],[170,29530,923],{"className":29531},[424],[170,29533,909],{"className":29534},[234],[170,29536,29538,29541],{"className":29537},[230],[170,29539,25583],{"className":29540,"style":25604},[230,303],[170,29542,29544],{"className":29543},[307],[170,29545,29547,29578],{"className":29546},[242,243],[170,29548,29550,29575],{"className":29549},[247],[170,29551,29553,29564],{"className":29552,"style":24902},[251],[170,29554,29555,29558],{"style":26032},[170,29556],{"className":29557,"style":651},[259],[170,29559,29561],{"className":29560},[264,265,266,267],[170,29562,888],{"className":29563},[230,303,267],[170,29565,29566,29569],{"style":16532},[170,29567],{"className":29568,"style":651},[259],[170,29570,29572],{"className":29571},[264,265,266,267],[170,29573,783],{"className":29574,"style":813},[230,303,267],[170,29576,338],{"className":29577},[337],[170,29579,29581],{"className":29580},[247],[170,29582,29584],{"className":29583,"style":17447},[251],[170,29585],{},[170,29587,909],{"className":29588},[234],[170,29590,194],{"className":29591},[230],[170,29593],{"className":29594,"style":1983},[610],[170,29596,5884],{"className":29597},[664],[170,29599],{"className":29600,"style":1983},[610],[170,29602,27566],{"className":29603},[230,303],[170,29605],{"className":29606,"style":1983},[610],[170,29608,5884],{"className":29609},[664],[170,29611],{"className":29612,"style":1983},[610],[170,29614,29616,29619],{"className":29615},[230],[170,29617,27566],{"className":29618},[230,303],[170,29620,29622],{"className":29621},[307],[170,29623,29625],{"className":29624},[242],[170,29626,29628],{"className":29627},[247],[170,29629,29631],{"className":29630,"style":23416},[251],[170,29632,29633,29636],{"style":16532},[170,29634],{"className":29635,"style":651},[259],[170,29637,29639],{"className":29638},[264,265,266,267],[170,29640,2928],{"className":29641},[230,267],[170,29643],{"className":29644,"style":1983},[610],[170,29646,5884],{"className":29647},[664],[170,29649],{"className":29650,"style":1983},[610],[170,29652,23350],{"className":29653},[1073],[170,29655],{"className":29656,"style":1027},[610],[170,29658,923],{"className":29659},[424],[170,29661],{"className":29662,"style":1983},[610],[170,29664,5884],{"className":29665},[664],[170,29667],{"className":29668,"style":1983},[610],[170,29670,19101],{"className":29671,"style":19348},[230,303],[170,29673,29675,29678],{"className":29674},[230],[170,29676,25583],{"className":29677,"style":25604},[230,303],[170,29679,29681],{"className":29680},[307],[170,29682,29684,29724],{"className":29683},[242,243],[170,29685,29687,29721],{"className":29686},[247],[170,29688,29690,29710],{"className":29689,"style":24902},[251],[170,29691,29692,29695],{"style":26032},[170,29693],{"className":29694,"style":651},[259],[170,29696,29698],{"className":29697},[264,265,266,267],[170,29699,29701,29704,29707],{"className":29700},[230,267],[170,29702,888],{"className":29703},[230,303,267],[170,29705,5884],{"className":29706},[664,267],[170,29708,194],{"className":29709},[230,267],[170,29711,29712,29715],{"style":16532},[170,29713],{"className":29714,"style":651},[259],[170,29716,29718],{"className":29717},[264,265,266,267],[170,29719,783],{"className":29720,"style":813},[230,303,267],[170,29722,338],{"className":29723},[337],[170,29725,29727],{"className":29726},[247],[170,29728,29730],{"className":29729,"style":26416},[251],[170,29731],{},[170,29733],{"className":29734,"style":1983},[610],[170,29736,23155],{"className":29737},[664],[170,29739],{"className":29740,"style":1983},[610],[170,29742,909],{"className":29743},[234],[170,29745,27566],{"className":29746},[230,303],[170,29748],{"className":29749,"style":1983},[610],[170,29751,5884],{"className":29752},[664],[170,29754],{"className":29755,"style":1983},[610],[170,29757,29759,29762],{"className":29758},[230],[170,29760,27566],{"className":29761},[230,303],[170,29763,29765],{"className":29764},[307],[170,29766,29768],{"className":29767},[242],[170,29769,29771],{"className":29770},[247],[170,29772,29774],{"className":29773,"style":23416},[251],[170,29775,29776,29779],{"style":16532},[170,29777],{"className":29778,"style":651},[259],[170,29780,29782],{"className":29781},[264,265,266,267],[170,29783,2928],{"className":29784},[230,267],[170,29786],{"className":29787,"style":1983},[610],[170,29789,5884],{"className":29790},[664],[170,29792],{"className":29793,"style":1983},[610],[170,29795,23350],{"className":29796},[1073],[170,29798],{"className":29799,"style":1027},[610],[170,29801,923],{"className":29802},[424],[170,29804],{"className":29805,"style":1983},[610],[170,29807,5884],{"className":29808},[664],[170,29810],{"className":29811,"style":1983},[610],[170,29813,23350],{"className":29814},[1073],[170,29816],{"className":29817,"style":1027},[610],[170,29819,923],{"className":29820},[424],[170,29822,29823,29826],{"style":28354},[170,29824],{"className":29825,"style":28215},[259],[170,29827,29829,29832,29835,29838,29841,29844,29847,29850,29853,29856,29859,29862,29865],{"className":29828},[230],[170,29830],{"className":29831},[230],[170,29833],{"className":29834,"style":611},[610],[170,29836,926],{"className":29837},[615],[170,29839],{"className":29840,"style":611},[610],[170,29842,909],{"className":29843},[234],[170,29845,194],{"className":29846},[230],[170,29848],{"className":29849,"style":1983},[610],[170,29851,18471],{"className":29852},[664],[170,29854],{"className":29855,"style":1983},[610],[170,29857,27566],{"className":29858},[230,303],[170,29860,923],{"className":29861},[424],[170,29863],{"className":29864,"style":1027},[610],[170,29866,29868,29874,29925,30000,30003,30006,30009,30012,30072,30147,30150,30153,30156,30159,30162],{"className":29867},[1073],[170,29869,29871],{"className":29870,"style":1078},[234,1077],[170,29872,909],{"className":29873},[1082,266],[170,29875,29877,29880],{"className":29876},[230],[170,29878,25583],{"className":29879,"style":25604},[230,303],[170,29881,29883],{"className":29882},[307],[170,29884,29886,29917],{"className":29885},[242,243],[170,29887,29889,29914],{"className":29888},[247],[170,29890,29892,29903],{"className":29891,"style":24902},[251],[170,29893,29894,29897],{"style":26032},[170,29895],{"className":29896,"style":651},[259],[170,29898,29900],{"className":29899},[264,265,266,267],[170,29901,888],{"className":29902},[230,303,267],[170,29904,29905,29908],{"style":16532},[170,29906],{"className":29907,"style":651},[259],[170,29909,29911],{"className":29910},[264,265,266,267],[170,29912,783],{"className":29913,"style":813},[230,303,267],[170,29915,338],{"className":29916},[337],[170,29918,29920],{"className":29919},[247],[170,29921,29923],{"className":29922,"style":17447},[251],[170,29924],{},[170,29926,29928,29931,29997],{"className":29927},[230],[170,29929],{"className":29930},[234,235],[170,29932,29934],{"className":29933},[189],[170,29935,29937,29988],{"className":29936},[242,243],[170,29938,29940,29985],{"className":29939},[247],[170,29941,29943,29966,29974],{"className":29942,"style":16385},[251],[170,29944,29945,29948],{"style":15925},[170,29946],{"className":29947,"style":260},[259],[170,29949,29951,29954,29957,29960,29963],{"className":29950},[230],[170,29952,194],{"className":29953},[230],[170,29955],{"className":29956,"style":1983},[610],[170,29958,18471],{"className":29959},[664],[170,29961],{"className":29962,"style":1983},[610],[170,29964,27566],{"className":29965},[230,303],[170,29967,29968,29971],{"style":385},[170,29969],{"className":29970,"style":260},[259],[170,29972],{"className":29973,"style":393},[392],[170,29975,29976,29979],{"style":15991},[170,29977],{"className":29978,"style":260},[259],[170,29980,29982],{"className":29981},[230],[170,29983,194],{"className":29984},[230],[170,29986,338],{"className":29987},[337],[170,29989,29991],{"className":29990},[247],[170,29992,29995],{"className":29993,"style":29994},[251],"height:0.7693em;",[170,29996],{},[170,29998],{"className":29999},[424,235],[170,30001],{"className":30002,"style":1983},[610],[170,30004,5884],{"className":30005},[664],[170,30007],{"className":30008,"style":1983},[610],[170,30010,19101],{"className":30011,"style":19348},[230,303],[170,30013,30015,30018],{"className":30014},[230],[170,30016,25583],{"className":30017,"style":25604},[230,303],[170,30019,30021],{"className":30020},[307],[170,30022,30024,30064],{"className":30023},[242,243],[170,30025,30027,30061],{"className":30026},[247],[170,30028,30030,30050],{"className":30029,"style":24902},[251],[170,30031,30032,30035],{"style":26032},[170,30033],{"className":30034,"style":651},[259],[170,30036,30038],{"className":30037},[264,265,266,267],[170,30039,30041,30044,30047],{"className":30040},[230,267],[170,30042,888],{"className":30043},[230,303,267],[170,30045,5884],{"className":30046},[664,267],[170,30048,194],{"className":30049},[230,267],[170,30051,30052,30055],{"style":16532},[170,30053],{"className":30054,"style":651},[259],[170,30056,30058],{"className":30057},[264,265,266,267],[170,30059,783],{"className":30060,"style":813},[230,303,267],[170,30062,338],{"className":30063},[337],[170,30065,30067],{"className":30066},[247],[170,30068,30070],{"className":30069,"style":26416},[251],[170,30071],{},[170,30073,30075,30078,30144],{"className":30074},[230],[170,30076],{"className":30077},[234,235],[170,30079,30081],{"className":30080},[189],[170,30082,30084,30136],{"className":30083},[242,243],[170,30085,30087,30133],{"className":30086},[247],[170,30088,30091,30114,30122],{"className":30089,"style":30090},[251],"height:1.3714em;",[170,30092,30093,30096],{"style":15925},[170,30094],{"className":30095,"style":260},[259],[170,30097,30099,30102,30105,30108,30111],{"className":30098},[230],[170,30100,194],{"className":30101},[230],[170,30103],{"className":30104,"style":1983},[610],[170,30106,18471],{"className":30107},[664],[170,30109],{"className":30110,"style":1983},[610],[170,30112,27566],{"className":30113},[230,303],[170,30115,30116,30119],{"style":385},[170,30117],{"className":30118,"style":260},[259],[170,30120],{"className":30121,"style":393},[392],[170,30123,30124,30127],{"style":15991},[170,30125],{"className":30126,"style":260},[259],[170,30128,30130],{"className":30129},[230],[170,30131,27566],{"className":30132},[230,303],[170,30134,338],{"className":30135},[337],[170,30137,30139],{"className":30138},[247],[170,30140,30142],{"className":30141,"style":29994},[251],[170,30143],{},[170,30145],{"className":30146},[424,235],[170,30148],{"className":30149,"style":1983},[610],[170,30151,5884],{"className":30152},[664],[170,30154],{"className":30155,"style":1983},[610],[170,30157,23350],{"className":30158},[1073],[170,30160],{"className":30161,"style":1027},[610],[170,30163,30165],{"className":30164,"style":1078},[424,1077],[170,30166,923],{"className":30167},[1082,266],[170,30169,30170,30173],{"style":28363},[170,30171],{"className":30172,"style":28215},[259],[170,30174,30176,30179,30182,30185,30188,30255,30258,30261,30264,30294],{"className":30175},[230],[170,30177],{"className":30178},[230],[170,30180],{"className":30181,"style":611},[610],[170,30183,926],{"className":30184},[615],[170,30186],{"className":30187,"style":611},[610],[170,30189,30191],{"className":30190},[1062,14344],[170,30192,30194,30246],{"className":30193},[242,243],[170,30195,30197,30243],{"className":30196},[247],[170,30198,30201,30222,30232],{"className":30199,"style":30200},[251],"height:1.6514em;",[170,30202,30204,30207],{"style":30203},"top:-1.8479em;margin-left:0em;",[170,30205],{"className":30206,"style":15538},[259],[170,30208,30210],{"className":30209},[264,265,266,267],[170,30211,30213,30216,30219],{"className":30212},[230,267],[170,30214,7464],{"className":30215},[230,303,267],[170,30217,926],{"className":30218},[615,267],[170,30220,1865],{"className":30221},[230,267],[170,30223,30224,30227],{"style":15547},[170,30225],{"className":30226,"style":15538},[259],[170,30228,30229],{},[170,30230,1937],{"className":30231},[1062,2021,15556],[170,30233,30234,30237],{"style":16480},[170,30235],{"className":30236,"style":15538},[259],[170,30238,30240],{"className":30239},[264,265,266,267],[170,30241,28150],{"className":30242},[230,267],[170,30244,338],{"className":30245},[337],[170,30247,30249],{"className":30248},[247],[170,30250,30253],{"className":30251,"style":30252},[251],"height:1.3021em;",[170,30254],{},[170,30256,909],{"className":30257},[234],[170,30259,19101],{"className":30260,"style":19348},[230,303],[170,30262,27566],{"className":30263},[230,303],[170,30265,30267,30270],{"className":30266},[424],[170,30268,923],{"className":30269},[424],[170,30271,30273],{"className":30272},[307],[170,30274,30276],{"className":30275},[242],[170,30277,30279],{"className":30278},[247],[170,30280,30283],{"className":30281,"style":30282},[251],"height:0.8991em;",[170,30284,30285,30288],{"style":16532},[170,30286],{"className":30287,"style":651},[259],[170,30289,30291],{"className":30290},[264,265,266,267],[170,30292,7464],{"className":30293},[230,303,267],[170,30295,30297,30300],{"className":30296},[230],[170,30298,25583],{"className":30299,"style":25604},[230,303],[170,30301,30303],{"className":30302},[307],[170,30304,30306,30346],{"className":30305},[242,243],[170,30307,30309,30343],{"className":30308},[247],[170,30310,30312,30332],{"className":30311,"style":24902},[251],[170,30313,30314,30317],{"style":26032},[170,30315],{"className":30316,"style":651},[259],[170,30318,30320],{"className":30319},[264,265,266,267],[170,30321,30323,30326,30329],{"className":30322},[230,267],[170,30324,888],{"className":30325},[230,303,267],[170,30327,5884],{"className":30328},[664,267],[170,30330,7464],{"className":30331},[230,303,267],[170,30333,30334,30337],{"style":16532},[170,30335],{"className":30336,"style":651},[259],[170,30338,30340],{"className":30339},[264,265,266,267],[170,30341,783],{"className":30342,"style":813},[230,303,267],[170,30344,338],{"className":30345},[337],[170,30347,30349],{"className":30348},[247],[170,30350,30352],{"className":30351,"style":26416},[251],[170,30353],{},[170,30355,338],{"className":30356},[337],[170,30358,30360],{"className":30359},[247],[170,30361,30363],{"className":30362,"style":28379},[251],[170,30364],{},[14,30366,30367],{},"整理后发现其中有等比数列，利用等比数列求和得到最终的 GAE 优势函数的表达式。",[14,30369,30370],{},"GAE 优势函数平衡了采样不同步带来的方差和偏差的问题。",[14,30372,30373],{},"现在我们来看上面得之不易的三个表达式：",[170,30375,30377],{"className":30376},[14456],[170,30378,30380,30636],{"className":30379},[173],[170,30381,30383],{"className":30382},[177],[179,30384,30385],{"xmlns":181,"display":9588},[183,30386,30387,30633],{},[186,30388,30389,30397,30399,30405,30407,30409,30415,30417,30429,30431,30433,30439,30441,30447,30449,30451,30465,30467,30473,30475,30477,30479,30481,30495,30497,30499,30501,30507,30521,30523,30529,30543,30561,30575,30577,30585,30587,30595,30597,30599,30601,30603,30605,30611,30613,30621,30623,30631],{},[1933,30390,30391,30393,30395],{},[202,30392,25583],{},[202,30394,888],{},[202,30396,783],{},[572,30398,926],{},[199,30400,30401,30403],{},[202,30402,7933],{},[202,30404,888],{},[572,30406,5884],{},[202,30408,19101],{},[199,30410,30411,30413],{},[202,30412,783],{},[202,30414,14157],{},[572,30416,909],{"stretchy":908},[199,30418,30419,30421],{},[202,30420,12908],{},[186,30422,30423,30425,30427],{},[202,30424,888],{},[572,30426,5884],{},[192,30428,194],{},[572,30430,923],{"stretchy":908},[572,30432,18471],{},[199,30434,30435,30437],{},[202,30436,783],{},[202,30438,14157],{},[572,30440,909],{"stretchy":908},[199,30442,30443,30445],{},[202,30444,12908],{},[202,30446,888],{},[572,30448,923],{"stretchy":908},[610,30450],{"linebreak":21759},[1933,30452,30453,30455,30457],{},[202,30454,885],{},[202,30456,14157],{},[186,30458,30459,30461,30463],{},[202,30460,27616],{},[202,30462,885],{},[202,30464,5714],{},[572,30466,909],{"stretchy":908},[199,30468,30469,30471],{},[202,30470,12908],{},[202,30472,888],{},[572,30474,914],{"separator":217},[202,30476,134],{},[572,30478,923],{"stretchy":908},[572,30480,926],{},[14789,30482,30483,30485,30493],{},[572,30484,1937],{},[186,30486,30487,30489,30491],{},[202,30488,7464],{},[572,30490,926],{},[192,30492,1865],{},[202,30494,28150],{"mathvariant":929},[572,30496,909],{"stretchy":908},[202,30498,19101],{},[202,30500,27566],{},[576,30502,30503,30505],{},[572,30504,923],{"stretchy":908},[202,30506,7464],{},[1933,30508,30509,30511,30519],{},[202,30510,25583],{},[186,30512,30513,30515,30517],{},[202,30514,888],{},[572,30516,5884],{},[202,30518,7464],{},[202,30520,783],{},[610,30522],{"linebreak":21759},[189,30524,30525,30527],{},[192,30526,194],{},[202,30528,7429],{},[14789,30530,30531,30533,30541],{},[572,30532,1937],{},[186,30534,30535,30537,30539],{},[202,30536,446],{},[572,30538,926],{},[192,30540,194],{},[202,30542,7429],{},[14789,30544,30545,30547,30555],{},[572,30546,1937],{},[186,30548,30549,30551,30553],{},[202,30550,888],{},[572,30552,926],{},[192,30554,194],{},[199,30556,30557,30559],{},[202,30558,13348],{},[202,30560,446],{},[1933,30562,30563,30565,30567],{},[202,30564,885],{},[202,30566,14157],{},[186,30568,30569,30571,30573],{},[202,30570,27616],{},[202,30572,885],{},[202,30574,5714],{},[572,30576,909],{"stretchy":908},[1933,30578,30579,30581,30583],{},[202,30580,12908],{},[202,30582,446],{},[202,30584,888],{},[572,30586,914],{"separator":217},[1933,30588,30589,30591,30593],{},[202,30590,134],{},[202,30592,446],{},[202,30594,888],{},[572,30596,923],{"stretchy":908},[3384,30598,27701],{},[202,30600,14486],{"mathvariant":929},[202,30602,14911],{},[572,30604,933],{},[199,30606,30607,30609],{},[202,30608,5711],{},[202,30610,14157],{},[572,30612,909],{"stretchy":908},[1933,30614,30615,30617,30619],{},[202,30616,134],{},[202,30618,446],{},[202,30620,888],{},[202,30622,13047],{"mathvariant":929},[1933,30624,30625,30627,30629],{},[202,30626,12908],{},[202,30628,446],{},[202,30630,888],{},[572,30632,923],{"stretchy":908},[209,30634,30635],{"encoding":211},"\\delta_t^V = r_t + \\gamma V_\\theta(s_{t+1}) - V_\\theta(s_t) \\\\\nA_\\theta^{GAE}(s_t, a) = \\sum_{b=0}^{\\infty} (\\gamma\\lambda)^b \\delta_{t+b}^V \\\\\n\\frac{1}{N}\\sum_{n=1}^N\\sum_{t=1}^{T_n} A_\\theta^{GAE}(s_n^t, a_n^t)\\, \\nabla \\log P_\\theta(a_n^t|s_n^t)",[170,30637,30639,30705,30760,30873,30965,30968,31098,31270,31273],{"className":30638,"ariaHidden":217},[216],[170,30640,30642,30645,30696,30699,30702],{"className":30641},[221],[170,30643],{"className":30644,"style":26011},[225],[170,30646,30648,30651],{"className":30647},[230],[170,30649,25583],{"className":30650,"style":25604},[230,303],[170,30652,30654],{"className":30653},[307],[170,30655,30657,30688],{"className":30656},[242,243],[170,30658,30660,30685],{"className":30659},[247],[170,30661,30663,30674],{"className":30662,"style":24902},[251],[170,30664,30665,30668],{"style":26032},[170,30666],{"className":30667,"style":651},[259],[170,30669,30671],{"className":30670},[264,265,266,267],[170,30672,888],{"className":30673},[230,303,267],[170,30675,30676,30679],{"style":16532},[170,30677],{"className":30678,"style":651},[259],[170,30680,30682],{"className":30681},[264,265,266,267],[170,30683,783],{"className":30684,"style":813},[230,303,267],[170,30686,338],{"className":30687},[337],[170,30689,30691],{"className":30690},[247],[170,30692,30694],{"className":30693,"style":17447},[251],[170,30695],{},[170,30697],{"className":30698,"style":611},[610],[170,30700,926],{"className":30701},[615],[170,30703],{"className":30704,"style":611},[610],[170,30706,30708,30711,30751,30754,30757],{"className":30707},[221],[170,30709],{"className":30710,"style":22018},[225],[170,30712,30714,30717],{"className":30713},[230],[170,30715,7933],{"className":30716,"style":3317},[230,303],[170,30718,30720],{"className":30719},[307],[170,30721,30723,30743],{"className":30722},[242,243],[170,30724,30726,30740],{"className":30725},[247],[170,30727,30729],{"className":30728,"style":12838},[251],[170,30730,30731,30734],{"style":13211},[170,30732],{"className":30733,"style":651},[259],[170,30735,30737],{"className":30736},[264,265,266,267],[170,30738,888],{"className":30739},[230,303,267],[170,30741,338],{"className":30742},[337],[170,30744,30746],{"className":30745},[247],[170,30747,30749],{"className":30748,"style":1381},[251],[170,30750],{},[170,30752],{"className":30753,"style":1983},[610],[170,30755,5884],{"className":30756},[664],[170,30758],{"className":30759,"style":1983},[610],[170,30761,30763,30766,30769,30809,30812,30861,30864,30867,30870],{"className":30762},[221],[170,30764],{"className":30765,"style":988},[225],[170,30767,19101],{"className":30768,"style":19348},[230,303],[170,30770,30772,30775],{"className":30771},[230],[170,30773,783],{"className":30774,"style":813},[230,303],[170,30776,30778],{"className":30777},[307],[170,30779,30781,30801],{"className":30780},[242,243],[170,30782,30784,30798],{"className":30783},[247],[170,30785,30787],{"className":30786,"style":1359},[251],[170,30788,30789,30792],{"style":20611},[170,30790],{"className":30791,"style":651},[259],[170,30793,30795],{"className":30794},[264,265,266,267],[170,30796,14157],{"className":30797,"style":3317},[230,303,267],[170,30799,338],{"className":30800},[337],[170,30802,30804],{"className":30803},[247],[170,30805,30807],{"className":30806,"style":1381},[251],[170,30808],{},[170,30810,909],{"className":30811},[234],[170,30813,30815,30818],{"className":30814},[230],[170,30816,12908],{"className":30817},[230,303],[170,30819,30821],{"className":30820},[307],[170,30822,30824,30853],{"className":30823},[242,243],[170,30825,30827,30850],{"className":30826},[247],[170,30828,30830],{"className":30829,"style":3198},[251],[170,30831,30832,30835],{"style":1362},[170,30833],{"className":30834,"style":651},[259],[170,30836,30838],{"className":30837},[264,265,266,267],[170,30839,30841,30844,30847],{"className":30840},[230,267],[170,30842,888],{"className":30843},[230,303,267],[170,30845,5884],{"className":30846},[664,267],[170,30848,194],{"className":30849},[230,267],[170,30851,338],{"className":30852},[337],[170,30854,30856],{"className":30855},[247],[170,30857,30859],{"className":30858,"style":7045},[251],[170,30860],{},[170,30862,923],{"className":30863},[424],[170,30865],{"className":30866,"style":1983},[610],[170,30868,18471],{"className":30869},[664],[170,30871],{"className":30872,"style":1983},[610],[170,30874,30876,30879,30919,30922,30962],{"className":30875},[221],[170,30877],{"className":30878,"style":988},[225],[170,30880,30882,30885],{"className":30881},[230],[170,30883,783],{"className":30884,"style":813},[230,303],[170,30886,30888],{"className":30887},[307],[170,30889,30891,30911],{"className":30890},[242,243],[170,30892,30894,30908],{"className":30893},[247],[170,30895,30897],{"className":30896,"style":1359},[251],[170,30898,30899,30902],{"style":20611},[170,30900],{"className":30901,"style":651},[259],[170,30903,30905],{"className":30904},[264,265,266,267],[170,30906,14157],{"className":30907,"style":3317},[230,303,267],[170,30909,338],{"className":30910},[337],[170,30912,30914],{"className":30913},[247],[170,30915,30917],{"className":30916,"style":1381},[251],[170,30918],{},[170,30920,909],{"className":30921},[234],[170,30923,30925,30928],{"className":30924},[230],[170,30926,12908],{"className":30927},[230,303],[170,30929,30931],{"className":30930},[307],[170,30932,30934,30954],{"className":30933},[242,243],[170,30935,30937,30951],{"className":30936},[247],[170,30938,30940],{"className":30939,"style":12838},[251],[170,30941,30942,30945],{"style":1362},[170,30943],{"className":30944,"style":651},[259],[170,30946,30948],{"className":30947},[264,265,266,267],[170,30949,888],{"className":30950},[230,303,267],[170,30952,338],{"className":30953},[337],[170,30955,30957],{"className":30956},[247],[170,30958,30960],{"className":30959,"style":1381},[251],[170,30961],{},[170,30963,923],{"className":30964},[424],[170,30966],{"className":30967},[610,21759],[170,30969,30971,30974,31034,31037,31077,31080,31083,31086,31089,31092,31095],{"className":30970},[221],[170,30972],{"className":30973,"style":24883},[225],[170,30975,30977,30980],{"className":30976},[230],[170,30978,885],{"className":30979},[230,303],[170,30981,30983],{"className":30982},[307],[170,30984,30986,31026],{"className":30985},[242,243],[170,30987,30989,31023],{"className":30988},[247],[170,30990,30992,31003],{"className":30991,"style":24902},[251],[170,30993,30994,30997],{"style":17417},[170,30995],{"className":30996,"style":651},[259],[170,30998,31000],{"className":30999},[264,265,266,267],[170,31001,14157],{"className":31002,"style":3317},[230,303,267],[170,31004,31005,31008],{"style":16532},[170,31006],{"className":31007,"style":651},[259],[170,31009,31011],{"className":31010},[264,265,266,267],[170,31012,31014,31017,31020],{"className":31013},[230,267],[170,31015,27616],{"className":31016},[230,303,267],[170,31018,885],{"className":31019},[230,303,267],[170,31021,5714],{"className":31022,"style":6157},[230,303,267],[170,31024,338],{"className":31025},[337],[170,31027,31029],{"className":31028},[247],[170,31030,31032],{"className":31031,"style":17447},[251],[170,31033],{},[170,31035,909],{"className":31036},[234],[170,31038,31040,31043],{"className":31039},[230],[170,31041,12908],{"className":31042},[230,303],[170,31044,31046],{"className":31045},[307],[170,31047,31049,31069],{"className":31048},[242,243],[170,31050,31052,31066],{"className":31051},[247],[170,31053,31055],{"className":31054,"style":12838},[251],[170,31056,31057,31060],{"style":1362},[170,31058],{"className":31059,"style":651},[259],[170,31061,31063],{"className":31062},[264,265,266,267],[170,31064,888],{"className":31065},[230,303,267],[170,31067,338],{"className":31068},[337],[170,31070,31072],{"className":31071},[247],[170,31073,31075],{"className":31074,"style":1381},[251],[170,31076],{},[170,31078,914],{"className":31079},[1023],[170,31081],{"className":31082,"style":1027},[610],[170,31084,134],{"className":31085},[230,303],[170,31087,923],{"className":31088},[424],[170,31090],{"className":31091,"style":611},[610],[170,31093,926],{"className":31094},[615],[170,31096],{"className":31097,"style":611},[610],[170,31099,31101,31105,31172,31175,31178,31181,31210],{"className":31100},[221],[170,31102],{"className":31103,"style":31104},[225],"height:2.9535em;vertical-align:-1.3021em;",[170,31106,31108],{"className":31107},[1062,14344],[170,31109,31111,31164],{"className":31110},[242,243],[170,31112,31114,31161],{"className":31113},[247],[170,31115,31117,31137,31147],{"className":31116,"style":30200},[251],[170,31118,31119,31122],{"style":30203},[170,31120],{"className":31121,"style":15538},[259],[170,31123,31125],{"className":31124},[264,265,266,267],[170,31126,31128,31131,31134],{"className":31127},[230,267],[170,31129,7464],{"className":31130},[230,303,267],[170,31132,926],{"className":31133},[615,267],[170,31135,1865],{"className":31136},[230,267],[170,31138,31139,31142],{"style":15547},[170,31140],{"className":31141,"style":15538},[259],[170,31143,31144],{},[170,31145,1937],{"className":31146},[1062,2021,15556],[170,31148,31149,31152],{"style":16480},[170,31150],{"className":31151,"style":15538},[259],[170,31153,31155],{"className":31154},[264,265,266,267],[170,31156,31158],{"className":31157},[230,267],[170,31159,28150],{"className":31160},[230,267],[170,31162,338],{"className":31163},[337],[170,31165,31167],{"className":31166},[247],[170,31168,31170],{"className":31169,"style":30252},[251],[170,31171],{},[170,31173,909],{"className":31174},[234],[170,31176,19101],{"className":31177,"style":19348},[230,303],[170,31179,27566],{"className":31180},[230,303],[170,31182,31184,31187],{"className":31183},[424],[170,31185,923],{"className":31186},[424],[170,31188,31190],{"className":31189},[307],[170,31191,31193],{"className":31192},[242],[170,31194,31196],{"className":31195},[247],[170,31197,31199],{"className":31198,"style":30282},[251],[170,31200,31201,31204],{"style":16532},[170,31202],{"className":31203,"style":651},[259],[170,31205,31207],{"className":31206},[264,265,266,267],[170,31208,7464],{"className":31209},[230,303,267],[170,31211,31213,31216],{"className":31212},[230],[170,31214,25583],{"className":31215,"style":25604},[230,303],[170,31217,31219],{"className":31218},[307],[170,31220,31222,31262],{"className":31221},[242,243],[170,31223,31225,31259],{"className":31224},[247],[170,31226,31228,31248],{"className":31227,"style":24902},[251],[170,31229,31230,31233],{"style":26032},[170,31231],{"className":31232,"style":651},[259],[170,31234,31236],{"className":31235},[264,265,266,267],[170,31237,31239,31242,31245],{"className":31238},[230,267],[170,31240,888],{"className":31241},[230,303,267],[170,31243,5884],{"className":31244},[664,267],[170,31246,7464],{"className":31247},[230,303,267],[170,31249,31250,31253],{"style":16532},[170,31251],{"className":31252,"style":651},[259],[170,31254,31256],{"className":31255},[264,265,266,267],[170,31257,783],{"className":31258,"style":813},[230,303,267],[170,31260,338],{"className":31261},[337],[170,31263,31265],{"className":31264},[247],[170,31266,31268],{"className":31267,"style":26416},[251],[170,31269],{},[170,31271],{"className":31272},[610,21759],[170,31274,31276,31280,31342,31345,31409,31412,31516,31519,31579,31582,31633,31636,31639,31690,31693,31696,31699,31702,31707,31710,31750,31753,31804,31807,31858],{"className":31275},[221],[170,31277],{"className":31278,"style":31279},[225],"height:3.1065em;vertical-align:-1.2671em;",[170,31281,31283,31286,31339],{"className":31282},[230],[170,31284],{"className":31285},[234,235],[170,31287,31289],{"className":31288},[189],[170,31290,31292,31331],{"className":31291},[242,243],[170,31293,31295,31328],{"className":31294},[247],[170,31296,31298,31309,31317],{"className":31297,"style":16385},[251],[170,31299,31300,31303],{"style":15925},[170,31301],{"className":31302,"style":260},[259],[170,31304,31306],{"className":31305},[230],[170,31307,7429],{"className":31308,"style":7504},[230,303],[170,31310,31311,31314],{"style":385},[170,31312],{"className":31313,"style":260},[259],[170,31315],{"className":31316,"style":393},[392],[170,31318,31319,31322],{"style":15991},[170,31320],{"className":31321,"style":260},[259],[170,31323,31325],{"className":31324},[230],[170,31326,194],{"className":31327},[230],[170,31329,338],{"className":31330},[337],[170,31332,31334],{"className":31333},[247],[170,31335,31337],{"className":31336,"style":16425},[251],[170,31338],{},[170,31340],{"className":31341},[424,235],[170,31343],{"className":31344,"style":1027},[610],[170,31346,31348],{"className":31347},[1062,14344],[170,31349,31351,31401],{"className":31350},[242,243],[170,31352,31354,31398],{"className":31353},[247],[170,31355,31357,31377,31387],{"className":31356,"style":16446},[251],[170,31358,31359,31362],{"style":16449},[170,31360],{"className":31361,"style":15538},[259],[170,31363,31365],{"className":31364},[264,265,266,267],[170,31366,31368,31371,31374],{"className":31367},[230,267],[170,31369,446],{"className":31370},[230,303,267],[170,31372,926],{"className":31373},[615,267],[170,31375,194],{"className":31376},[230,267],[170,31378,31379,31382],{"style":15547},[170,31380],{"className":31381,"style":15538},[259],[170,31383,31384],{},[170,31385,1937],{"className":31386},[1062,2021,15556],[170,31388,31389,31392],{"style":16480},[170,31390],{"className":31391,"style":15538},[259],[170,31393,31395],{"className":31394},[264,265,266,267],[170,31396,7429],{"className":31397,"style":7504},[230,303,267],[170,31399,338],{"className":31400},[337],[170,31402,31404],{"className":31403},[247],[170,31405,31407],{"className":31406,"style":16499},[251],[170,31408],{},[170,31410],{"className":31411,"style":1027},[610],[170,31413,31415],{"className":31414},[1062,14344],[170,31416,31418,31508],{"className":31417},[242,243],[170,31419,31421,31505],{"className":31420},[247],[170,31422,31424,31444,31454],{"className":31423,"style":17255},[251],[170,31425,31426,31429],{"style":16449},[170,31427],{"className":31428,"style":15538},[259],[170,31430,31432],{"className":31431},[264,265,266,267],[170,31433,31435,31438,31441],{"className":31434},[230,267],[170,31436,888],{"className":31437},[230,303,267],[170,31439,926],{"className":31440},[615,267],[170,31442,194],{"className":31443},[230,267],[170,31445,31446,31449],{"style":15547},[170,31447],{"className":31448,"style":15538},[259],[170,31450,31451],{},[170,31452,1937],{"className":31453},[1062,2021,15556],[170,31455,31456,31459],{"style":17288},[170,31457],{"className":31458,"style":15538},[259],[170,31460,31462],{"className":31461},[264,265,266,267],[170,31463,31465],{"className":31464},[230,267],[170,31466,31468,31471],{"className":31467},[230,267],[170,31469,13348],{"className":31470,"style":3292},[230,303,267],[170,31472,31474],{"className":31473},[307],[170,31475,31477,31497],{"className":31476},[242,243],[170,31478,31480,31494],{"className":31479},[247],[170,31481,31483],{"className":31482,"style":4409},[251],[170,31484,31485,31488],{"style":17318},[170,31486],{"className":31487,"style":324},[259],[170,31489,31491],{"className":31490},[264,328,329,267],[170,31492,446],{"className":31493},[230,303,267],[170,31495,338],{"className":31496},[337],[170,31498,31500],{"className":31499},[247],[170,31501,31503],{"className":31502,"style":4431},[251],[170,31504],{},[170,31506,338],{"className":31507},[337],[170,31509,31511],{"className":31510},[247],[170,31512,31514],{"className":31513,"style":16499},[251],[170,31515],{},[170,31517],{"className":31518,"style":1027},[610],[170,31520,31522,31525],{"className":31521},[230],[170,31523,885],{"className":31524},[230,303],[170,31526,31528],{"className":31527},[307],[170,31529,31531,31571],{"className":31530},[242,243],[170,31532,31534,31568],{"className":31533},[247],[170,31535,31537,31548],{"className":31536,"style":24902},[251],[170,31538,31539,31542],{"style":17417},[170,31540],{"className":31541,"style":651},[259],[170,31543,31545],{"className":31544},[264,265,266,267],[170,31546,14157],{"className":31547,"style":3317},[230,303,267],[170,31549,31550,31553],{"style":16532},[170,31551],{"className":31552,"style":651},[259],[170,31554,31556],{"className":31555},[264,265,266,267],[170,31557,31559,31562,31565],{"className":31558},[230,267],[170,31560,27616],{"className":31561},[230,303,267],[170,31563,885],{"className":31564},[230,303,267],[170,31566,5714],{"className":31567,"style":6157},[230,303,267],[170,31569,338],{"className":31570},[337],[170,31572,31574],{"className":31573},[247],[170,31575,31577],{"className":31576,"style":17447},[251],[170,31578],{},[170,31580,909],{"className":31581},[234],[170,31583,31585,31588],{"className":31584},[230],[170,31586,12908],{"className":31587},[230,303],[170,31589,31591],{"className":31590},[307],[170,31592,31594,31625],{"className":31593},[242,243],[170,31595,31597,31622],{"className":31596},[247],[170,31598,31600,31611],{"className":31599,"style":17414},[251],[170,31601,31602,31605],{"style":17417},[170,31603],{"className":31604,"style":651},[259],[170,31606,31608],{"className":31607},[264,265,266,267],[170,31609,446],{"className":31610},[230,303,267],[170,31612,31613,31616],{"style":16532},[170,31614],{"className":31615,"style":651},[259],[170,31617,31619],{"className":31618},[264,265,266,267],[170,31620,888],{"className":31621},[230,303,267],[170,31623,338],{"className":31624},[337],[170,31626,31628],{"className":31627},[247],[170,31629,31631],{"className":31630,"style":17447},[251],[170,31632],{},[170,31634,914],{"className":31635},[1023],[170,31637],{"className":31638,"style":1027},[610],[170,31640,31642,31645],{"className":31641},[230],[170,31643,134],{"className":31644},[230,303],[170,31646,31648],{"className":31647},[307],[170,31649,31651,31682],{"className":31650},[242,243],[170,31652,31654,31679],{"className":31653},[247],[170,31655,31657,31668],{"className":31656,"style":17414},[251],[170,31658,31659,31662],{"style":17417},[170,31660],{"className":31661,"style":651},[259],[170,31663,31665],{"className":31664},[264,265,266,267],[170,31666,446],{"className":31667},[230,303,267],[170,31669,31670,31673],{"style":16532},[170,31671],{"className":31672,"style":651},[259],[170,31674,31676],{"className":31675},[264,265,266,267],[170,31677,888],{"className":31678},[230,303,267],[170,31680,338],{"className":31681},[337],[170,31683,31685],{"className":31684},[247],[170,31686,31688],{"className":31687,"style":17447},[251],[170,31689],{},[170,31691,923],{"className":31692},[424],[170,31694],{"className":31695,"style":1027},[610],[170,31697,14486],{"className":31698},[230],[170,31700],{"className":31701,"style":1027},[610],[170,31703,16956,31705],{"className":31704},[1062],[170,31706,16960],{"style":16959},[170,31708],{"className":31709,"style":1027},[610],[170,31711,31713,31716],{"className":31712},[230],[170,31714,5711],{"className":31715,"style":3292},[230,303],[170,31717,31719],{"className":31718},[307],[170,31720,31722,31742],{"className":31721},[242,243],[170,31723,31725,31739],{"className":31724},[247],[170,31726,31728],{"className":31727,"style":1359},[251],[170,31729,31730,31733],{"style":7567},[170,31731],{"className":31732,"style":651},[259],[170,31734,31736],{"className":31735},[264,265,266,267],[170,31737,14157],{"className":31738,"style":3317},[230,303,267],[170,31740,338],{"className":31741},[337],[170,31743,31745],{"className":31744},[247],[170,31746,31748],{"className":31747,"style":1381},[251],[170,31749],{},[170,31751,909],{"className":31752},[234],[170,31754,31756,31759],{"className":31755},[230],[170,31757,134],{"className":31758},[230,303],[170,31760,31762],{"className":31761},[307],[170,31763,31765,31796],{"className":31764},[242,243],[170,31766,31768,31793],{"className":31767},[247],[170,31769,31771,31782],{"className":31770,"style":17414},[251],[170,31772,31773,31776],{"style":17417},[170,31774],{"className":31775,"style":651},[259],[170,31777,31779],{"className":31778},[264,265,266,267],[170,31780,446],{"className":31781},[230,303,267],[170,31783,31784,31787],{"style":16532},[170,31785],{"className":31786,"style":651},[259],[170,31788,31790],{"className":31789},[264,265,266,267],[170,31791,888],{"className":31792},[230,303,267],[170,31794,338],{"className":31795},[337],[170,31797,31799],{"className":31798},[247],[170,31800,31802],{"className":31801,"style":17447},[251],[170,31803],{},[170,31805,13047],{"className":31806},[230],[170,31808,31810,31813],{"className":31809},[230],[170,31811,12908],{"className":31812},[230,303],[170,31814,31816],{"className":31815},[307],[170,31817,31819,31850],{"className":31818},[242,243],[170,31820,31822,31847],{"className":31821},[247],[170,31823,31825,31836],{"className":31824,"style":17414},[251],[170,31826,31827,31830],{"style":17417},[170,31828],{"className":31829,"style":651},[259],[170,31831,31833],{"className":31832},[264,265,266,267],[170,31834,446],{"className":31835},[230,303,267],[170,31837,31838,31841],{"style":16532},[170,31839],{"className":31840,"style":651},[259],[170,31842,31844],{"className":31843},[264,265,266,267],[170,31845,888],{"className":31846},[230,303,267],[170,31848,338],{"className":31849},[337],[170,31851,31853],{"className":31852},[247],[170,31854,31856],{"className":31855,"style":17447},[251],[170,31857],{},[170,31859,923],{"className":31860},[424],[14,31862,31863],{},"这里的价值状态函数我们一般用一个神经网路进行拟合，一般可以和策略网络公用参数，只是最后一层不同。",[10,31865,31867],{"id":31866},"proximal-policy-optimization-ppo-近邻策略优化算法","Proximal Policy Optimization (PPO) 近邻策略优化算法",[14,31869,31870],{},"在经典的强化学习环境中，我们一般是一边采集数据，一边进行训练模型，采集完的数据用完就会丢掉。这导致我们采集的数据发生了一定的浪费，且模型训练的成本也会增高。但如果我们有一种方法，能让模型在其他模型的表现中学习自己的表现，参考其他数据就能实现自身的更新，这样我们的训练成本就会大大减少。",[24,31872,31873],{"id":31873},"重要性采样",[14,31875,31876,31877,31921,31922,31950,31951,31979,31980,32008],{},"将 ",[170,31878,31880,31900],{"className":31879},[173],[170,31881,31883],{"className":31882},[177],[179,31884,31885],{"xmlns":181},[183,31886,31887,31897],{},[186,31888,31889,31891,31893,31895],{},[202,31890,11764],{},[572,31892,909],{"stretchy":908},[202,31894,7434],{},[572,31896,923],{"stretchy":908},[209,31898,31899],{"encoding":211},"f(x)",[170,31901,31903],{"className":31902,"ariaHidden":217},[216],[170,31904,31906,31909,31912,31915,31918],{"className":31905},[221],[170,31907],{"className":31908,"style":988},[225],[170,31910,11764],{"className":31911,"style":13725},[230,303],[170,31913,909],{"className":31914},[234],[170,31916,7434],{"className":31917},[230,303],[170,31919,923],{"className":31920},[424]," 在分布 ",[170,31923,31925,31938],{"className":31924},[173],[170,31926,31928],{"className":31927},[177],[179,31929,31930],{"xmlns":181},[183,31931,31932,31936],{},[186,31933,31934],{},[202,31935,14],{},[209,31937,14],{"encoding":211},[170,31939,31941],{"className":31940,"ariaHidden":217},[216],[170,31942,31944,31947],{"className":31943},[221],[170,31945],{"className":31946,"style":1817},[225],[170,31948,14],{"className":31949},[230,303]," 下的期望改写为在另一分布 ",[170,31952,31954,31967],{"className":31953},[173],[170,31955,31957],{"className":31956},[177],[179,31958,31959],{"xmlns":181},[183,31960,31961,31965],{},[186,31962,31963],{},[202,31964,1805],{},[209,31966,1805],{"encoding":211},[170,31968,31970],{"className":31969,"ariaHidden":217},[216],[170,31971,31973,31976],{"className":31972},[221],[170,31974],{"className":31975,"style":1817},[225],[170,31977,1805],{"className":31978,"style":552},[230,303],"（proposal）下的期望，从而可用从 ",[170,31981,31983,31996],{"className":31982},[173],[170,31984,31986],{"className":31985},[177],[179,31987,31988],{"xmlns":181},[183,31989,31990,31994],{},[186,31991,31992],{},[202,31993,1805],{},[209,31995,1805],{"encoding":211},[170,31997,31999],{"className":31998,"ariaHidden":217},[216],[170,32000,32002,32005],{"className":32001},[221],[170,32003],{"className":32004,"style":1817},[225],[170,32006,1805],{"className":32007,"style":552},[230,303]," 采样的数据估计：",[170,32010,32012],{"className":32011},[14456],[170,32013,32015,32399],{"className":32014},[173],[170,32016,32018],{"className":32017},[177],[179,32019,32020],{"xmlns":181,"display":9588},[183,32021,32022,32396],{},[14468,32023,32024,32096,32160,32224,32296],{"rowspacing":14470,"columnalign":14471,"columnspacing":14472},[14474,32025,32026,32062],{},[14477,32027,32028],{},[14480,32029,32030],{"scriptlevel":1865,"displaystyle":217},[186,32031,32032,32034,32036,32038,32040,32042,32044],{},[202,32033,5714],{"mathvariant":580},[572,32035,909],{"stretchy":908},[202,32037,11764],{},[572,32039,909],{"stretchy":908},[202,32041,7434],{},[572,32043,923],{"stretchy":908},[199,32045,32046,32048],{},[572,32047,923],{"stretchy":908},[186,32049,32050,32052,32054,32056,32058,32060],{},[202,32051,7434],{},[572,32053,14150],{},[202,32055,14],{},[572,32057,909],{"stretchy":908},[202,32059,7434],{},[572,32061,923],{"stretchy":908},[14477,32063,32064],{},[14480,32065,32066],{"scriptlevel":1865,"displaystyle":217},[186,32067,32068,32070,32072,32078,32080,32082,32084,32086,32088,32090,32092,32094],{},[186,32069],{},[572,32071,926],{},[14167,32073,32074,32076],{},[572,32075,1937],{},[202,32077,7434],{},[202,32079,11764],{},[572,32081,909],{"stretchy":908},[202,32083,7434],{},[572,32085,923],{"stretchy":908},[572,32087,1927],{},[202,32089,14],{},[572,32091,909],{"stretchy":908},[202,32093,7434],{},[572,32095,923],{"stretchy":908},[14474,32097,32098,32104],{},[14477,32099,32100],{},[14480,32101,32102],{"scriptlevel":1865,"displaystyle":217},[186,32103],{},[14477,32105,32106],{},[14480,32107,32108],{"scriptlevel":1865,"displaystyle":217},[186,32109,32110,32112,32114,32120,32122,32124,32126,32128,32130,32132,32134,32136,32138],{},[186,32111],{},[572,32113,926],{},[14167,32115,32116,32118],{},[572,32117,1937],{},[202,32119,7434],{},[202,32121,11764],{},[572,32123,909],{"stretchy":908},[202,32125,7434],{},[572,32127,923],{"stretchy":908},[572,32129,1927],{},[202,32131,14],{},[572,32133,909],{"stretchy":908},[202,32135,7434],{},[572,32137,923],{"stretchy":908},[189,32139,32140,32150],{},[186,32141,32142,32144,32146,32148],{},[202,32143,1805],{},[572,32145,909],{"stretchy":908},[202,32147,7434],{},[572,32149,923],{"stretchy":908},[186,32151,32152,32154,32156,32158],{},[202,32153,1805],{},[572,32155,909],{"stretchy":908},[202,32157,7434],{},[572,32159,923],{"stretchy":908},[14474,32161,32162,32168],{},[14477,32163,32164],{},[14480,32165,32166],{"scriptlevel":1865,"displaystyle":217},[186,32167],{},[14477,32169,32170],{},[14480,32171,32172],{"scriptlevel":1865,"displaystyle":217},[186,32173,32174,32176,32178,32184,32186,32188,32190,32192,32214,32216,32218,32220,32222],{},[186,32175],{},[572,32177,926],{},[14167,32179,32180,32182],{},[572,32181,1937],{},[202,32183,7434],{},[202,32185,11764],{},[572,32187,909],{"stretchy":908},[202,32189,7434],{},[572,32191,923],{"stretchy":908},[189,32193,32194,32204],{},[186,32195,32196,32198,32200,32202],{},[202,32197,14],{},[572,32199,909],{"stretchy":908},[202,32201,7434],{},[572,32203,923],{"stretchy":908},[186,32205,32206,32208,32210,32212],{},[202,32207,1805],{},[572,32209,909],{"stretchy":908},[202,32211,7434],{},[572,32213,923],{"stretchy":908},[572,32215,1927],{},[202,32217,1805],{},[572,32219,909],{"stretchy":908},[202,32221,7434],{},[572,32223,923],{"stretchy":908},[14474,32225,32226,32232],{},[14477,32227,32228],{},[14480,32229,32230],{"scriptlevel":1865,"displaystyle":217},[186,32231],{},[14477,32233,32234],{},[14480,32235,32236],{"scriptlevel":1865,"displaystyle":217},[186,32237,32238,32240,32242,32244],{},[186,32239],{},[572,32241,926],{},[202,32243,5714],{"mathvariant":580},[199,32245,32246,32282],{},[186,32247,32248,32250,32252,32254,32256,32258,32280],{},[572,32249,909],{"fence":217},[202,32251,11764],{},[572,32253,909],{"stretchy":908},[202,32255,7434],{},[572,32257,923],{"stretchy":908},[189,32259,32260,32270],{},[186,32261,32262,32264,32266,32268],{},[202,32263,14],{},[572,32265,909],{"stretchy":908},[202,32267,7434],{},[572,32269,923],{"stretchy":908},[186,32271,32272,32274,32276,32278],{},[202,32273,1805],{},[572,32275,909],{"stretchy":908},[202,32277,7434],{},[572,32279,923],{"stretchy":908},[572,32281,923],{"fence":217},[186,32283,32284,32286,32288,32290,32292,32294],{},[202,32285,7434],{},[572,32287,14150],{},[202,32289,1805],{},[572,32291,909],{"stretchy":908},[202,32293,7434],{},[572,32295,923],{"stretchy":908},[14474,32297,32298,32304],{},[14477,32299,32300],{},[14480,32301,32302],{"scriptlevel":1865,"displaystyle":217},[186,32303],{},[14477,32305,32306],{},[14480,32307,32308],{"scriptlevel":1865,"displaystyle":217},[186,32309,32310,32312,32314,32320,32334,32336,32338,32344,32346,32376,32378,32380,32386,32388,32390,32392,32394],{},[186,32311],{},[572,32313,14781],{},[189,32315,32316,32318],{},[192,32317,194],{},[202,32319,7429],{},[14789,32321,32322,32324,32332],{},[572,32323,1937],{},[186,32325,32326,32328,32330],{},[202,32327,446],{},[572,32329,926],{},[192,32331,194],{},[202,32333,7429],{},[202,32335,11764],{},[572,32337,909],{"stretchy":908},[199,32339,32340,32342],{},[202,32341,7434],{},[202,32343,446],{},[572,32345,923],{"stretchy":908},[189,32347,32348,32362],{},[186,32349,32350,32352,32354,32360],{},[202,32351,14],{},[572,32353,909],{"stretchy":908},[199,32355,32356,32358],{},[202,32357,7434],{},[202,32359,446],{},[572,32361,923],{"stretchy":908},[186,32363,32364,32366,32368,32374],{},[202,32365,1805],{},[572,32367,909],{"stretchy":908},[199,32369,32370,32372],{},[202,32371,7434],{},[202,32373,446],{},[572,32375,923],{"stretchy":908},[572,32377,914],{"separator":217},[610,32379],{"width":27706},[199,32381,32382,32384],{},[202,32383,7434],{},[202,32385,446],{},[572,32387,14150],{},[202,32389,1805],{},[572,32391,909],{"stretchy":908},[202,32393,7434],{},[572,32395,923],{"stretchy":908},[209,32397,32398],{"encoding":211},"\\begin{aligned}\n\\mathbb{E}(f(x))_{x \\sim p(x)} &= \\sum_{x} f(x) \\cdot p(x) \\\\\n&= \\sum_{x} f(x) \\cdot p(x) \\frac{q(x)}{q(x)} \\\\\n&= \\sum_{x} f(x) \\frac{p(x)}{q(x)} \\cdot q(x) \\\\\n&= \\mathbb{E}\\left(f(x) \\frac{p(x)}{q(x)}\\right)_{x \\sim q(x)} \\\\\n&\\approx \\frac{1}{N} \\sum_{n=1}^{N} f(x_n) \\frac{p(x_n)}{q(x_n)}, \\quad x_n \\sim q(x)\n\\end{aligned}",[170,32400,32402],{"className":32401,"ariaHidden":217},[216],[170,32403,32405,32409],{"className":32404},[221],[170,32406],{"className":32407,"style":32408},[225],"height:14.8742em;vertical-align:-7.1871em;",[170,32410,32412],{"className":32411},[230],[170,32413,32415,32562],{"className":32414},[14468],[170,32416,32418],{"className":32417},[15259],[170,32419,32421,32553],{"className":32420},[242,243],[170,32422,32424,32550],{"className":32423},[247],[170,32425,32428,32514,32523,32532,32541],{"className":32426,"style":32427},[251],"height:7.6871em;",[170,32429,32431,32435],{"style":32430},"top:-10.4654em;",[170,32432],{"className":32433,"style":32434},[259],"height:3.8283em;",[170,32436,32438,32441,32444,32447,32450,32453,32456],{"className":32437},[230],[170,32439,5714],{"className":32440},[230,632],[170,32442,909],{"className":32443},[234],[170,32445,11764],{"className":32446,"style":13725},[230,303],[170,32448,909],{"className":32449},[234],[170,32451,7434],{"className":32452},[230,303],[170,32454,923],{"className":32455},[424],[170,32457,32459,32462],{"className":32458},[424],[170,32460,923],{"className":32461},[424],[170,32463,32465],{"className":32464},[307],[170,32466,32468,32506],{"className":32467},[242,243],[170,32469,32471,32503],{"className":32470},[247],[170,32472,32474],{"className":32473,"style":317},[251],[170,32475,32476,32479],{"style":14245},[170,32477],{"className":32478,"style":651},[259],[170,32480,32482],{"className":32481},[264,265,266,267],[170,32483,32485,32488,32491,32494,32497,32500],{"className":32484},[230,267],[170,32486,7434],{"className":32487},[230,303,267],[170,32489,14150],{"className":32490},[615,267],[170,32492,14],{"className":32493},[230,303,267],[170,32495,909],{"className":32496},[234,267],[170,32498,7434],{"className":32499},[230,303,267],[170,32501,923],{"className":32502},[424,267],[170,32504,338],{"className":32505},[337],[170,32507,32509],{"className":32508},[247],[170,32510,32512],{"className":32511,"style":6209},[251],[170,32513],{},[170,32515,32517,32520],{"style":32516},"top:-7.4884em;",[170,32518],{"className":32519,"style":32434},[259],[170,32521],{"className":32522},[230],[170,32524,32526,32529],{"style":32525},"top:-4.5114em;",[170,32527],{"className":32528,"style":32434},[259],[170,32530],{"className":32531},[230],[170,32533,32535,32538],{"style":32534},"top:-1.5114em;",[170,32536],{"className":32537,"style":32434},[259],[170,32539],{"className":32540},[230],[170,32542,32544,32547],{"style":32543},"top:1.7917em;",[170,32545],{"className":32546,"style":32434},[259],[170,32548],{"className":32549},[230],[170,32551,338],{"className":32552},[337],[170,32554,32556],{"className":32555},[247],[170,32557,32560],{"className":32558,"style":32559},[251],"height:7.1871em;",[170,32561],{},[170,32563,32565],{"className":32564},[15483],[170,32566,32568,33665],{"className":32567},[242,243],[170,32569,32571,33662],{"className":32570},[247],[170,32572,32574,32677,32860,33043,33234],{"className":32573,"style":32427},[251],[170,32575,32576,32579],{"style":32430},[170,32577],{"className":32578,"style":32434},[259],[170,32580,32582,32585,32588,32591,32594,32641,32644,32647,32650,32653,32656,32659,32662,32665,32668,32671,32674],{"className":32581},[230],[170,32583],{"className":32584},[230],[170,32586],{"className":32587,"style":611},[610],[170,32589,926],{"className":32590},[615],[170,32592],{"className":32593,"style":611},[610],[170,32595,32597],{"className":32596},[1062,14344],[170,32598,32600,32633],{"className":32599},[242,243],[170,32601,32603,32630],{"className":32602},[247],[170,32604,32606,32620],{"className":32605,"style":15531},[251],[170,32607,32608,32611],{"style":15534},[170,32609],{"className":32610,"style":15538},[259],[170,32612,32614],{"className":32613},[264,265,266,267],[170,32615,32617],{"className":32616},[230,267],[170,32618,7434],{"className":32619},[230,303,267],[170,32621,32622,32625],{"style":15547},[170,32623],{"className":32624,"style":15538},[259],[170,32626,32627],{},[170,32628,1937],{"className":32629},[1062,2021,15556],[170,32631,338],{"className":32632},[337],[170,32634,32636],{"className":32635},[247],[170,32637,32639],{"className":32638,"style":15566},[251],[170,32640],{},[170,32642],{"className":32643,"style":1027},[610],[170,32645,11764],{"className":32646,"style":13725},[230,303],[170,32648,909],{"className":32649},[234],[170,32651,7434],{"className":32652},[230,303],[170,32654,923],{"className":32655},[424],[170,32657],{"className":32658,"style":1983},[610],[170,32660,1927],{"className":32661},[664],[170,32663],{"className":32664,"style":1983},[610],[170,32666,14],{"className":32667},[230,303],[170,32669,909],{"className":32670},[234],[170,32672,7434],{"className":32673},[230,303],[170,32675,923],{"className":32676},[424],[170,32678,32679,32682],{"style":32516},[170,32680],{"className":32681,"style":32434},[259],[170,32683,32685,32688,32691,32694,32697,32744,32747,32750,32753,32756,32759,32762,32765,32768,32771,32774,32777,32780],{"className":32684},[230],[170,32686],{"className":32687},[230],[170,32689],{"className":32690,"style":611},[610],[170,32692,926],{"className":32693},[615],[170,32695],{"className":32696,"style":611},[610],[170,32698,32700],{"className":32699},[1062,14344],[170,32701,32703,32736],{"className":32702},[242,243],[170,32704,32706,32733],{"className":32705},[247],[170,32707,32709,32723],{"className":32708,"style":15531},[251],[170,32710,32711,32714],{"style":15534},[170,32712],{"className":32713,"style":15538},[259],[170,32715,32717],{"className":32716},[264,265,266,267],[170,32718,32720],{"className":32719},[230,267],[170,32721,7434],{"className":32722},[230,303,267],[170,32724,32725,32728],{"style":15547},[170,32726],{"className":32727,"style":15538},[259],[170,32729,32730],{},[170,32731,1937],{"className":32732},[1062,2021,15556],[170,32734,338],{"className":32735},[337],[170,32737,32739],{"className":32738},[247],[170,32740,32742],{"className":32741,"style":15566},[251],[170,32743],{},[170,32745],{"className":32746,"style":1027},[610],[170,32748,11764],{"className":32749,"style":13725},[230,303],[170,32751,909],{"className":32752},[234],[170,32754,7434],{"className":32755},[230,303],[170,32757,923],{"className":32758},[424],[170,32760],{"className":32761,"style":1983},[610],[170,32763,1927],{"className":32764},[664],[170,32766],{"className":32767,"style":1983},[610],[170,32769,14],{"className":32770},[230,303],[170,32772,909],{"className":32773},[234],[170,32775,7434],{"className":32776},[230,303],[170,32778,923],{"className":32779},[424],[170,32781,32783,32786,32857],{"className":32782},[230],[170,32784],{"className":32785},[234,235],[170,32787,32789],{"className":32788},[189],[170,32790,32792,32849],{"className":32791},[242,243],[170,32793,32795,32846],{"className":32794},[247],[170,32796,32798,32818,32826],{"className":32797,"style":15922},[251],[170,32799,32800,32803],{"style":15925},[170,32801],{"className":32802,"style":260},[259],[170,32804,32806,32809,32812,32815],{"className":32805},[230],[170,32807,1805],{"className":32808,"style":552},[230,303],[170,32810,909],{"className":32811},[234],[170,32813,7434],{"className":32814},[230,303],[170,32816,923],{"className":32817},[424],[170,32819,32820,32823],{"style":385},[170,32821],{"className":32822,"style":260},[259],[170,32824],{"className":32825,"style":393},[392],[170,32827,32828,32831],{"style":15991},[170,32829],{"className":32830,"style":260},[259],[170,32832,32834,32837,32840,32843],{"className":32833},[230],[170,32835,1805],{"className":32836,"style":552},[230,303],[170,32838,909],{"className":32839},[234],[170,32841,7434],{"className":32842},[230,303],[170,32844,923],{"className":32845},[424],[170,32847,338],{"className":32848},[337],[170,32850,32852],{"className":32851},[247],[170,32853,32855],{"className":32854,"style":16056},[251],[170,32856],{},[170,32858],{"className":32859},[424,235],[170,32861,32862,32865],{"style":32525},[170,32863],{"className":32864,"style":32434},[259],[170,32866,32868,32871,32874,32877,32880,32927,32930,32933,32936,32939,32942,33022,33025,33028,33031,33034,33037,33040],{"className":32867},[230],[170,32869],{"className":32870},[230],[170,32872],{"className":32873,"style":611},[610],[170,32875,926],{"className":32876},[615],[170,32878],{"className":32879,"style":611},[610],[170,32881,32883],{"className":32882},[1062,14344],[170,32884,32886,32919],{"className":32885},[242,243],[170,32887,32889,32916],{"className":32888},[247],[170,32890,32892,32906],{"className":32891,"style":15531},[251],[170,32893,32894,32897],{"style":15534},[170,32895],{"className":32896,"style":15538},[259],[170,32898,32900],{"className":32899},[264,265,266,267],[170,32901,32903],{"className":32902},[230,267],[170,32904,7434],{"className":32905},[230,303,267],[170,32907,32908,32911],{"style":15547},[170,32909],{"className":32910,"style":15538},[259],[170,32912,32913],{},[170,32914,1937],{"className":32915},[1062,2021,15556],[170,32917,338],{"className":32918},[337],[170,32920,32922],{"className":32921},[247],[170,32923,32925],{"className":32924,"style":15566},[251],[170,32926],{},[170,32928],{"className":32929,"style":1027},[610],[170,32931,11764],{"className":32932,"style":13725},[230,303],[170,32934,909],{"className":32935},[234],[170,32937,7434],{"className":32938},[230,303],[170,32940,923],{"className":32941},[424],[170,32943,32945,32948,33019],{"className":32944},[230],[170,32946],{"className":32947},[234,235],[170,32949,32951],{"className":32950},[189],[170,32952,32954,33011],{"className":32953},[242,243],[170,32955,32957,33008],{"className":32956},[247],[170,32958,32960,32980,32988],{"className":32959,"style":15922},[251],[170,32961,32962,32965],{"style":15925},[170,32963],{"className":32964,"style":260},[259],[170,32966,32968,32971,32974,32977],{"className":32967},[230],[170,32969,1805],{"className":32970,"style":552},[230,303],[170,32972,909],{"className":32973},[234],[170,32975,7434],{"className":32976},[230,303],[170,32978,923],{"className":32979},[424],[170,32981,32982,32985],{"style":385},[170,32983],{"className":32984,"style":260},[259],[170,32986],{"className":32987,"style":393},[392],[170,32989,32990,32993],{"style":15991},[170,32991],{"className":32992,"style":260},[259],[170,32994,32996,32999,33002,33005],{"className":32995},[230],[170,32997,14],{"className":32998},[230,303],[170,33000,909],{"className":33001},[234],[170,33003,7434],{"className":33004},[230,303],[170,33006,923],{"className":33007},[424],[170,33009,338],{"className":33010},[337],[170,33012,33014],{"className":33013},[247],[170,33015,33017],{"className":33016,"style":16056},[251],[170,33018],{},[170,33020],{"className":33021},[424,235],[170,33023],{"className":33024,"style":1983},[610],[170,33026,1927],{"className":33027},[664],[170,33029],{"className":33030,"style":1983},[610],[170,33032,1805],{"className":33033,"style":552},[230,303],[170,33035,909],{"className":33036},[234],[170,33038,7434],{"className":33039},[230,303],[170,33041,923],{"className":33042},[424],[170,33044,33045,33048],{"style":32534},[170,33046],{"className":33047,"style":32434},[259],[170,33049,33051,33054,33057,33060,33063,33066,33069],{"className":33050},[230],[170,33052],{"className":33053},[230],[170,33055],{"className":33056,"style":611},[610],[170,33058,926],{"className":33059},[615],[170,33061],{"className":33062,"style":611},[610],[170,33064,5714],{"className":33065},[230,632],[170,33067],{"className":33068,"style":1027},[610],[170,33070,33072,33179],{"className":33071},[1073],[170,33073,33075,33081,33084,33087,33090,33093,33173],{"className":33074},[1073],[170,33076,33078],{"className":33077,"style":1078},[234,1077],[170,33079,909],{"className":33080},[1082,266],[170,33082,11764],{"className":33083,"style":13725},[230,303],[170,33085,909],{"className":33086},[234],[170,33088,7434],{"className":33089},[230,303],[170,33091,923],{"className":33092},[424],[170,33094,33096,33099,33170],{"className":33095},[230],[170,33097],{"className":33098},[234,235],[170,33100,33102],{"className":33101},[189],[170,33103,33105,33162],{"className":33104},[242,243],[170,33106,33108,33159],{"className":33107},[247],[170,33109,33111,33131,33139],{"className":33110,"style":15922},[251],[170,33112,33113,33116],{"style":15925},[170,33114],{"className":33115,"style":260},[259],[170,33117,33119,33122,33125,33128],{"className":33118},[230],[170,33120,1805],{"className":33121,"style":552},[230,303],[170,33123,909],{"className":33124},[234],[170,33126,7434],{"className":33127},[230,303],[170,33129,923],{"className":33130},[424],[170,33132,33133,33136],{"style":385},[170,33134],{"className":33135,"style":260},[259],[170,33137],{"className":33138,"style":393},[392],[170,33140,33141,33144],{"style":15991},[170,33142],{"className":33143,"style":260},[259],[170,33145,33147,33150,33153,33156],{"className":33146},[230],[170,33148,14],{"className":33149},[230,303],[170,33151,909],{"className":33152},[234],[170,33154,7434],{"className":33155},[230,303],[170,33157,923],{"className":33158},[424],[170,33160,338],{"className":33161},[337],[170,33163,33165],{"className":33164},[247],[170,33166,33168],{"className":33167,"style":16056},[251],[170,33169],{},[170,33171],{"className":33172},[424,235],[170,33174,33176],{"className":33175,"style":1078},[424,1077],[170,33177,923],{"className":33178},[1082,266],[170,33180,33182],{"className":33181},[307],[170,33183,33185,33225],{"className":33184},[242,243],[170,33186,33188,33222],{"className":33187},[247],[170,33189,33192],{"className":33190,"style":33191},[251],"height:-0.4747em;",[170,33193,33195,33198],{"style":33194},"top:-1.7003em;margin-right:0.05em;",[170,33196],{"className":33197,"style":651},[259],[170,33199,33201],{"className":33200},[264,265,266,267],[170,33202,33204,33207,33210,33213,33216,33219],{"className":33203},[230,267],[170,33205,7434],{"className":33206},[230,303,267],[170,33208,14150],{"className":33209},[615,267],[170,33211,1805],{"className":33212,"style":552},[230,303,267],[170,33214,909],{"className":33215},[234,267],[170,33217,7434],{"className":33218},[230,303,267],[170,33220,923],{"className":33221},[424,267],[170,33223,338],{"className":33224},[337],[170,33226,33228],{"className":33227},[247],[170,33229,33232],{"className":33230,"style":33231},[251],"height:1.1747em;",[170,33233],{},[170,33235,33236,33239],{"style":32543},[170,33237],{"className":33238,"style":32434},[259],[170,33240,33242,33245,33248,33251,33254,33316,33319,33386,33389,33392,33395,33435,33438,33592,33595,33598,33601,33641,33644,33647,33650,33653,33656,33659],{"className":33241},[230],[170,33243],{"className":33244},[230],[170,33246],{"className":33247,"style":611},[610],[170,33249,14781],{"className":33250},[615],[170,33252],{"className":33253,"style":611},[610],[170,33255,33257,33260,33313],{"className":33256},[230],[170,33258],{"className":33259},[234,235],[170,33261,33263],{"className":33262},[189],[170,33264,33266,33305],{"className":33265},[242,243],[170,33267,33269,33302],{"className":33268},[247],[170,33270,33272,33283,33291],{"className":33271,"style":16385},[251],[170,33273,33274,33277],{"style":15925},[170,33275],{"className":33276,"style":260},[259],[170,33278,33280],{"className":33279},[230],[170,33281,7429],{"className":33282,"style":7504},[230,303],[170,33284,33285,33288],{"style":385},[170,33286],{"className":33287,"style":260},[259],[170,33289],{"className":33290,"style":393},[392],[170,33292,33293,33296],{"style":15991},[170,33294],{"className":33295,"style":260},[259],[170,33297,33299],{"className":33298},[230],[170,33300,194],{"className":33301},[230],[170,33303,338],{"className":33304},[337],[170,33306,33308],{"className":33307},[247],[170,33309,33311],{"className":33310,"style":16425},[251],[170,33312],{},[170,33314],{"className":33315},[424,235],[170,33317],{"className":33318,"style":1027},[610],[170,33320,33322],{"className":33321},[1062,14344],[170,33323,33325,33378],{"className":33324},[242,243],[170,33326,33328,33375],{"className":33327},[247],[170,33329,33331,33351,33361],{"className":33330,"style":16446},[251],[170,33332,33333,33336],{"style":16449},[170,33334],{"className":33335,"style":15538},[259],[170,33337,33339],{"className":33338},[264,265,266,267],[170,33340,33342,33345,33348],{"className":33341},[230,267],[170,33343,446],{"className":33344},[230,303,267],[170,33346,926],{"className":33347},[615,267],[170,33349,194],{"className":33350},[230,267],[170,33352,33353,33356],{"style":15547},[170,33354],{"className":33355,"style":15538},[259],[170,33357,33358],{},[170,33359,1937],{"className":33360},[1062,2021,15556],[170,33362,33363,33366],{"style":16480},[170,33364],{"className":33365,"style":15538},[259],[170,33367,33369],{"className":33368},[264,265,266,267],[170,33370,33372],{"className":33371},[230,267],[170,33373,7429],{"className":33374,"style":7504},[230,303,267],[170,33376,338],{"className":33377},[337],[170,33379,33381],{"className":33380},[247],[170,33382,33384],{"className":33383,"style":16499},[251],[170,33385],{},[170,33387],{"className":33388,"style":1027},[610],[170,33390,11764],{"className":33391,"style":13725},[230,303],[170,33393,909],{"className":33394},[234],[170,33396,33398,33401],{"className":33397},[230],[170,33399,7434],{"className":33400},[230,303],[170,33402,33404],{"className":33403},[307],[170,33405,33407,33427],{"className":33406},[242,243],[170,33408,33410,33424],{"className":33409},[247],[170,33411,33413],{"className":33412,"style":1433},[251],[170,33414,33415,33418],{"style":1362},[170,33416],{"className":33417,"style":651},[259],[170,33419,33421],{"className":33420},[264,265,266,267],[170,33422,446],{"className":33423},[230,303,267],[170,33425,338],{"className":33426},[337],[170,33428,33430],{"className":33429},[247],[170,33431,33433],{"className":33432,"style":1381},[251],[170,33434],{},[170,33436,923],{"className":33437},[424],[170,33439,33441,33444,33589],{"className":33440},[230],[170,33442],{"className":33443},[234,235],[170,33445,33447],{"className":33446},[189],[170,33448,33450,33581],{"className":33449},[242,243],[170,33451,33453,33578],{"className":33452},[247],[170,33454,33456,33513,33521],{"className":33455,"style":15922},[251],[170,33457,33458,33461],{"style":15925},[170,33459],{"className":33460,"style":260},[259],[170,33462,33464,33467,33470,33510],{"className":33463},[230],[170,33465,1805],{"className":33466,"style":552},[230,303],[170,33468,909],{"className":33469},[234],[170,33471,33473,33476],{"className":33472},[230],[170,33474,7434],{"className":33475},[230,303],[170,33477,33479],{"className":33478},[307],[170,33480,33482,33502],{"className":33481},[242,243],[170,33483,33485,33499],{"className":33484},[247],[170,33486,33488],{"className":33487,"style":1433},[251],[170,33489,33490,33493],{"style":1362},[170,33491],{"className":33492,"style":651},[259],[170,33494,33496],{"className":33495},[264,265,266,267],[170,33497,446],{"className":33498},[230,303,267],[170,33500,338],{"className":33501},[337],[170,33503,33505],{"className":33504},[247],[170,33506,33508],{"className":33507,"style":1381},[251],[170,33509],{},[170,33511,923],{"className":33512},[424],[170,33514,33515,33518],{"style":385},[170,33516],{"className":33517,"style":260},[259],[170,33519],{"className":33520,"style":393},[392],[170,33522,33523,33526],{"style":15991},[170,33524],{"className":33525,"style":260},[259],[170,33527,33529,33532,33535,33575],{"className":33528},[230],[170,33530,14],{"className":33531},[230,303],[170,33533,909],{"className":33534},[234],[170,33536,33538,33541],{"className":33537},[230],[170,33539,7434],{"className":33540},[230,303],[170,33542,33544],{"className":33543},[307],[170,33545,33547,33567],{"className":33546},[242,243],[170,33548,33550,33564],{"className":33549},[247],[170,33551,33553],{"className":33552,"style":1433},[251],[170,33554,33555,33558],{"style":1362},[170,33556],{"className":33557,"style":651},[259],[170,33559,33561],{"className":33560},[264,265,266,267],[170,33562,446],{"className":33563},[230,303,267],[170,33565,338],{"className":33566},[337],[170,33568,33570],{"className":33569},[247],[170,33571,33573],{"className":33572,"style":1381},[251],[170,33574],{},[170,33576,923],{"className":33577},[424],[170,33579,338],{"className":33580},[337],[170,33582,33584],{"className":33583},[247],[170,33585,33587],{"className":33586,"style":16056},[251],[170,33588],{},[170,33590],{"className":33591},[424,235],[170,33593,914],{"className":33594},[1023],[170,33596],{"className":33597,"style":28671},[610],[170,33599],{"className":33600,"style":1027},[610],[170,33602,33604,33607],{"className":33603},[230],[170,33605,7434],{"className":33606},[230,303],[170,33608,33610],{"className":33609},[307],[170,33611,33613,33633],{"className":33612},[242,243],[170,33614,33616,33630],{"className":33615},[247],[170,33617,33619],{"className":33618,"style":1433},[251],[170,33620,33621,33624],{"style":1362},[170,33622],{"className":33623,"style":651},[259],[170,33625,33627],{"className":33626},[264,265,266,267],[170,33628,446],{"className":33629},[230,303,267],[170,33631,338],{"className":33632},[337],[170,33634,33636],{"className":33635},[247],[170,33637,33639],{"className":33638,"style":1381},[251],[170,33640],{},[170,33642],{"className":33643,"style":611},[610],[170,33645,14150],{"className":33646},[615],[170,33648],{"className":33649,"style":611},[610],[170,33651,1805],{"className":33652,"style":552},[230,303],[170,33654,909],{"className":33655},[234],[170,33657,7434],{"className":33658},[230,303],[170,33660,923],{"className":33661},[424],[170,33663,338],{"className":33664},[337],[170,33666,33668],{"className":33667},[247],[170,33669,33671],{"className":33670,"style":32559},[251],[170,33672],{},[14,33674,33675],{},"这样我们可以用重要性采样更新我们的目标函数的梯度公式，这样我们可以把 on-policy 的训练改为 off-policy 的训练。",[24,33677,33679],{"id":33678},"off-policy","Off-policy",[14,33681,33682,33683,33746,33747,33776,33777,33914,33915,34106,34107,34322],{},"记 ",[170,33684,33686,33704],{"className":33685},[173],[170,33687,33689],{"className":33688},[177],[179,33690,33691],{"xmlns":181},[183,33692,33693,33701],{},[186,33694,33695],{},[576,33696,33697,33699],{},[202,33698,14157],{},[572,33700,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[209,33702,33703],{"encoding":211},"\\theta'",[170,33705,33707],{"className":33706,"ariaHidden":217},[216],[170,33708,33710,33714],{"className":33709},[221],[170,33711],{"className":33712,"style":33713},[225],"height:0.7519em;",[170,33715,33717,33720],{"className":33716},[230],[170,33718,14157],{"className":33719,"style":3317},[230,303],[170,33721,33723],{"className":33722},[307],[170,33724,33726],{"className":33725},[242],[170,33727,33729],{"className":33728},[247],[170,33730,33732],{"className":33731,"style":33713},[251],[170,33733,33734,33737],{"style":647},[170,33735],{"className":33736,"style":651},[259],[170,33738,33740],{"className":33739},[264,265,266,267],[170,33741,33743],{"className":33742},[230,267],[170,33744,19090],{"className":33745},[230,267]," 为采集数据时的旧策略，",[170,33748,33750,33764],{"className":33749},[173],[170,33751,33753],{"className":33752},[177],[179,33754,33755],{"xmlns":181},[183,33756,33757,33761],{},[186,33758,33759],{},[202,33760,14157],{},[209,33762,33763],{"encoding":211},"\\theta",[170,33765,33767],{"className":33766,"ariaHidden":217},[216],[170,33768,33770,33773],{"className":33769},[221],[170,33771],{"className":33772,"style":518},[225],[170,33774,14157],{"className":33775,"style":3317},[230,303]," 为当前要优化的策略；优势 ",[170,33778,33780,33810],{"className":33779},[173],[170,33781,33783],{"className":33782},[177],[179,33784,33785],{"xmlns":181},[183,33786,33787,33807],{},[186,33788,33789],{},[1933,33790,33791,33793,33799],{},[202,33792,885],{},[576,33794,33795,33797],{},[202,33796,14157],{},[572,33798,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[186,33800,33801,33803,33805],{},[202,33802,27616],{},[202,33804,885],{},[202,33806,5714],{},[209,33808,33809],{"encoding":211},"A_{\\theta'}^{GAE}",[170,33811,33813],{"className":33812,"ariaHidden":217},[216],[170,33814,33816,33820],{"className":33815},[221],[170,33817],{"className":33818,"style":33819},[225],"height:1.1244em;vertical-align:-0.2831em;",[170,33821,33823,33826],{"className":33822},[230],[170,33824,885],{"className":33825},[230,303],[170,33827,33829],{"className":33828},[307],[170,33830,33832,33905],{"className":33831},[242,243],[170,33833,33835,33902],{"className":33834},[247],[170,33836,33838,33882],{"className":33837,"style":3305},[251],[170,33839,33841,33844],{"style":33840},"top:-2.4169em;margin-left:0em;margin-right:0.05em;",[170,33842],{"className":33843,"style":651},[259],[170,33845,33847],{"className":33846},[264,265,266,267],[170,33848,33850],{"className":33849},[230,267],[170,33851,33853,33856],{"className":33852},[230,267],[170,33854,14157],{"className":33855,"style":3317},[230,303,267],[170,33857,33859],{"className":33858},[307],[170,33860,33862],{"className":33861},[242],[170,33863,33865],{"className":33864},[247],[170,33866,33868],{"className":33867,"style":19250},[251],[170,33869,33870,33873],{"style":19253},[170,33871],{"className":33872,"style":324},[259],[170,33874,33876],{"className":33875},[264,328,329,267],[170,33877,33879],{"className":33878},[230,267],[170,33880,19090],{"className":33881},[230,267],[170,33883,33884,33887],{"style":647},[170,33885],{"className":33886,"style":651},[259],[170,33888,33890],{"className":33889},[264,265,266,267],[170,33891,33893,33896,33899],{"className":33892},[230,267],[170,33894,27616],{"className":33895},[230,303,267],[170,33897,885],{"className":33898},[230,303,267],[170,33900,5714],{"className":33901,"style":6157},[230,303,267],[170,33903,338],{"className":33904},[337],[170,33906,33908],{"className":33907},[247],[170,33909,33912],{"className":33910,"style":33911},[251],"height:0.2831em;",[170,33913],{}," 由旧策略下的价值网络估计。利用恒等式 ",[170,33916,33918,33972],{"className":33917},[173],[170,33919,33921],{"className":33920},[177],[179,33922,33923],{"xmlns":181},[183,33924,33925,33969],{},[186,33926,33927,33929,33931,33933,33935,33937,33939,33941,33943],{},[202,33928,14486],{"mathvariant":929},[202,33930,14911],{},[572,33932,933],{},[202,33934,11764],{},[572,33936,909],{"stretchy":908},[202,33938,7434],{},[572,33940,923],{"stretchy":908},[572,33942,926],{},[14480,33944,33945],{"scriptlevel":1865,"displaystyle":217},[189,33946,33947,33959],{},[186,33948,33949,33951,33953,33955,33957],{},[202,33950,14486],{"mathvariant":929},[202,33952,11764],{},[572,33954,909],{"stretchy":908},[202,33956,7434],{},[572,33958,923],{"stretchy":908},[186,33960,33961,33963,33965,33967],{},[202,33962,11764],{},[572,33964,909],{"stretchy":908},[202,33966,7434],{},[572,33968,923],{"stretchy":908},[209,33970,33971],{"encoding":211},"\\nabla \\log f(x) = \\dfrac{\\nabla f(x)}{f(x)}",[170,33973,33975,34016],{"className":33974,"ariaHidden":217},[216],[170,33976,33978,33981,33984,33987,33992,33995,33998,34001,34004,34007,34010,34013],{"className":33977},[221],[170,33979],{"className":33980,"style":988},[225],[170,33982,14486],{"className":33983},[230],[170,33985],{"className":33986,"style":1027},[610],[170,33988,16956,33990],{"className":33989},[1062],[170,33991,16960],{"style":16959},[170,33993],{"className":33994,"style":1027},[610],[170,33996,11764],{"className":33997,"style":13725},[230,303],[170,33999,909],{"className":34000},[234],[170,34002,7434],{"className":34003},[230,303],[170,34005,923],{"className":34006},[424],[170,34008],{"className":34009,"style":611},[610],[170,34011,926],{"className":34012},[615],[170,34014],{"className":34015,"style":611},[610],[170,34017,34019,34023],{"className":34018},[221],[170,34020],{"className":34021,"style":34022},[225],"height:2.363em;vertical-align:-0.936em;",[170,34024,34026,34029,34103],{"className":34025},[230],[170,34027],{"className":34028},[234,235],[170,34030,34032],{"className":34031},[189],[170,34033,34035,34095],{"className":34034},[242,243],[170,34036,34038,34092],{"className":34037},[247],[170,34039,34041,34061,34069],{"className":34040,"style":15922},[251],[170,34042,34043,34046],{"style":15925},[170,34044],{"className":34045,"style":260},[259],[170,34047,34049,34052,34055,34058],{"className":34048},[230],[170,34050,11764],{"className":34051,"style":13725},[230,303],[170,34053,909],{"className":34054},[234],[170,34056,7434],{"className":34057},[230,303],[170,34059,923],{"className":34060},[424],[170,34062,34063,34066],{"style":385},[170,34064],{"className":34065,"style":260},[259],[170,34067],{"className":34068,"style":393},[392],[170,34070,34071,34074],{"style":15991},[170,34072],{"className":34073,"style":260},[259],[170,34075,34077,34080,34083,34086,34089],{"className":34076},[230],[170,34078,14486],{"className":34079},[230],[170,34081,11764],{"className":34082,"style":13725},[230,303],[170,34084,909],{"className":34085},[234],[170,34087,7434],{"className":34088},[230,303],[170,34090,923],{"className":34091},[424],[170,34093,338],{"className":34094},[337],[170,34096,34098],{"className":34097},[247],[170,34099,34101],{"className":34100,"style":16056},[251],[170,34102],{},[170,34104],{"className":34105},[424,235],"，可将策略梯度写成重要性采样比 ",[170,34108,34110,34142],{"className":34109},[173],[170,34111,34113],{"className":34112},[177],[179,34114,34115],{"xmlns":181},[183,34116,34117,34139],{},[186,34118,34119],{},[14480,34120,34121],{"scriptlevel":1865,"displaystyle":217},[189,34122,34123,34129],{},[199,34124,34125,34127],{},[202,34126,5711],{},[202,34128,14157],{},[199,34130,34131,34133],{},[202,34132,5711],{},[576,34134,34135,34137],{},[202,34136,14157],{},[572,34138,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[209,34140,34141],{"encoding":211},"\\dfrac{P_\\theta}{P_{\\theta'}}",[170,34143,34145],{"className":34144,"ariaHidden":217},[216],[170,34146,34148,34152],{"className":34147},[221],[170,34149],{"className":34150,"style":34151},[225],"height:2.1963em;vertical-align:-0.836em;",[170,34153,34155,34158,34319],{"className":34154},[230],[170,34156],{"className":34157},[234,235],[170,34159,34161],{"className":34160},[189],[170,34162,34164,34310],{"className":34163},[242,243],[170,34165,34167,34307],{"className":34166},[247],[170,34168,34171,34251,34259],{"className":34169,"style":34170},[251],"height:1.3603em;",[170,34172,34173,34176],{"style":15925},[170,34174],{"className":34175,"style":260},[259],[170,34177,34179],{"className":34178},[230],[170,34180,34182,34185],{"className":34181},[230],[170,34183,5711],{"className":34184,"style":3292},[230,303],[170,34186,34188],{"className":34187},[307],[170,34189,34191,34243],{"className":34190},[242,243],[170,34192,34194,34240],{"className":34193},[247],[170,34195,34197],{"className":34196,"style":1359},[251],[170,34198,34199,34202],{"style":7567},[170,34200],{"className":34201,"style":651},[259],[170,34203,34205],{"className":34204},[264,265,266,267],[170,34206,34208],{"className":34207},[230,267],[170,34209,34211,34214],{"className":34210},[230,267],[170,34212,14157],{"className":34213,"style":3317},[230,303,267],[170,34215,34217],{"className":34216},[307],[170,34218,34220],{"className":34219},[242],[170,34221,34223],{"className":34222},[247],[170,34224,34226],{"className":34225,"style":19250},[251],[170,34227,34228,34231],{"style":19253},[170,34229],{"className":34230,"style":324},[259],[170,34232,34234],{"className":34233},[264,328,329,267],[170,34235,34237],{"className":34236},[230,267],[170,34238,19090],{"className":34239},[230,267],[170,34241,338],{"className":34242},[337],[170,34244,34246],{"className":34245},[247],[170,34247,34249],{"className":34248,"style":1381},[251],[170,34250],{},[170,34252,34253,34256],{"style":385},[170,34254],{"className":34255,"style":260},[259],[170,34257],{"className":34258,"style":393},[392],[170,34260,34261,34264],{"style":15991},[170,34262],{"className":34263,"style":260},[259],[170,34265,34267],{"className":34266},[230],[170,34268,34270,34273],{"className":34269},[230],[170,34271,5711],{"className":34272,"style":3292},[230,303],[170,34274,34276],{"className":34275},[307],[170,34277,34279,34299],{"className":34278},[242,243],[170,34280,34282,34296],{"className":34281},[247],[170,34283,34285],{"className":34284,"style":1359},[251],[170,34286,34287,34290],{"style":7567},[170,34288],{"className":34289,"style":651},[259],[170,34291,34293],{"className":34292},[264,265,266,267],[170,34294,14157],{"className":34295,"style":3317},[230,303,267],[170,34297,338],{"className":34298},[337],[170,34300,34302],{"className":34301},[247],[170,34303,34305],{"className":34304,"style":1381},[251],[170,34306],{},[170,34308,338],{"className":34309},[337],[170,34311,34313],{"className":34312},[247],[170,34314,34317],{"className":34315,"style":34316},[251],"height:0.836em;",[170,34318],{},[170,34320],{"className":34321},[424,235]," 的形式：",[170,34324,34326],{"className":34325},[14456],[170,34327,34329,34381],{"className":34328},[173],[170,34330,34332],{"className":34331},[177],[179,34333,34334],{"xmlns":181,"display":9588},[183,34335,34336,34378],{},[186,34337,34338,34340,34342,34344,34346,34348,34350,34352,34354],{},[202,34339,14486],{"mathvariant":929},[202,34341,14911],{},[572,34343,933],{},[202,34345,11764],{},[572,34347,909],{"stretchy":908},[202,34349,7434],{},[572,34351,923],{"stretchy":908},[572,34353,926],{},[189,34355,34356,34368],{},[186,34357,34358,34360,34362,34364,34366],{},[202,34359,14486],{"mathvariant":929},[202,34361,11764],{},[572,34363,909],{"stretchy":908},[202,34365,7434],{},[572,34367,923],{"stretchy":908},[186,34369,34370,34372,34374,34376],{},[202,34371,11764],{},[572,34373,909],{"stretchy":908},[202,34375,7434],{},[572,34377,923],{"stretchy":908},[209,34379,34380],{"encoding":211},"\\nabla \\log f(x) = \\frac{\\nabla f(x)}{f(x)}",[170,34382,34384,34425],{"className":34383,"ariaHidden":217},[216],[170,34385,34387,34390,34393,34396,34401,34404,34407,34410,34413,34416,34419,34422],{"className":34386},[221],[170,34388],{"className":34389,"style":988},[225],[170,34391,14486],{"className":34392},[230],[170,34394],{"className":34395,"style":1027},[610],[170,34397,16956,34399],{"className":34398},[1062],[170,34400,16960],{"style":16959},[170,34402],{"className":34403,"style":1027},[610],[170,34405,11764],{"className":34406,"style":13725},[230,303],[170,34408,909],{"className":34409},[234],[170,34411,7434],{"className":34412},[230,303],[170,34414,923],{"className":34415},[424],[170,34417],{"className":34418,"style":611},[610],[170,34420,926],{"className":34421},[615],[170,34423],{"className":34424,"style":611},[610],[170,34426,34428,34431],{"className":34427},[221],[170,34429],{"className":34430,"style":34022},[225],[170,34432,34434,34437,34511],{"className":34433},[230],[170,34435],{"className":34436},[234,235],[170,34438,34440],{"className":34439},[189],[170,34441,34443,34503],{"className":34442},[242,243],[170,34444,34446,34500],{"className":34445},[247],[170,34447,34449,34469,34477],{"className":34448,"style":15922},[251],[170,34450,34451,34454],{"style":15925},[170,34452],{"className":34453,"style":260},[259],[170,34455,34457,34460,34463,34466],{"className":34456},[230],[170,34458,11764],{"className":34459,"style":13725},[230,303],[170,34461,909],{"className":34462},[234],[170,34464,7434],{"className":34465},[230,303],[170,34467,923],{"className":34468},[424],[170,34470,34471,34474],{"style":385},[170,34472],{"className":34473,"style":260},[259],[170,34475],{"className":34476,"style":393},[392],[170,34478,34479,34482],{"style":15991},[170,34480],{"className":34481,"style":260},[259],[170,34483,34485,34488,34491,34494,34497],{"className":34484},[230],[170,34486,14486],{"className":34487},[230],[170,34489,11764],{"className":34490,"style":13725},[230,303],[170,34492,909],{"className":34493},[234],[170,34495,7434],{"className":34496},[230,303],[170,34498,923],{"className":34499},[424],[170,34501,338],{"className":34502},[337],[170,34504,34506],{"className":34505},[247],[170,34507,34509],{"className":34508,"style":16056},[251],[170,34510],{},[170,34512],{"className":34513},[424,235],[170,34515,34517],{"className":34516},[14456],[170,34518,34520,35242],{"className":34519},[173],[170,34521,34523],{"className":34522},[177],[179,34524,34525],{"xmlns":181,"display":9588},[183,34526,34527,35239],{},[14468,34528,34529,34653,34849,35075],{"rowspacing":14470,"columnalign":14471,"columnspacing":14472},[14474,34530,34531,34537],{},[14477,34532,34533],{},[14480,34534,34535],{"scriptlevel":1865,"displaystyle":217},[186,34536],{},[14477,34538,34539],{},[14480,34540,34541],{"scriptlevel":1865,"displaystyle":217},[186,34542,34543,34545,34551,34565,34583,34597,34599,34607,34609,34617,34619,34621,34623,34625,34631,34633,34641,34643,34651],{},[186,34544],{},[189,34546,34547,34549],{},[192,34548,194],{},[202,34550,7429],{},[14789,34552,34553,34555,34563],{},[572,34554,1937],{},[186,34556,34557,34559,34561],{},[202,34558,446],{},[572,34560,926],{},[192,34562,194],{},[202,34564,7429],{},[14789,34566,34567,34569,34577],{},[572,34568,1937],{},[186,34570,34571,34573,34575],{},[202,34572,888],{},[572,34574,926],{},[192,34576,194],{},[199,34578,34579,34581],{},[202,34580,13348],{},[202,34582,446],{},[1933,34584,34585,34587,34589],{},[202,34586,885],{},[202,34588,14157],{},[186,34590,34591,34593,34595],{},[202,34592,27616],{},[202,34594,885],{},[202,34596,5714],{},[572,34598,909],{"stretchy":908},[1933,34600,34601,34603,34605],{},[202,34602,12908],{},[202,34604,446],{},[202,34606,888],{},[572,34608,914],{"separator":217},[1933,34610,34611,34613,34615],{},[202,34612,134],{},[202,34614,446],{},[202,34616,888],{},[572,34618,923],{"stretchy":908},[202,34620,14486],{"mathvariant":929},[202,34622,14911],{},[572,34624,933],{},[199,34626,34627,34629],{},[202,34628,5711],{},[202,34630,14157],{},[572,34632,909],{"stretchy":908},[1933,34634,34635,34637,34639],{},[202,34636,134],{},[202,34638,446],{},[202,34640,888],{},[202,34642,13047],{"mathvariant":929},[1933,34644,34645,34647,34649],{},[202,34646,12908],{},[202,34648,446],{},[202,34650,888],{},[572,34652,923],{"stretchy":908},[14474,34654,34655,34661],{},[14477,34656,34657],{},[14480,34658,34659],{"scriptlevel":1865,"displaystyle":217},[186,34660],{},[14477,34662,34663],{},[14480,34664,34665],{"scriptlevel":1865,"displaystyle":217},[186,34666,34667,34669,34671,34677,34691,34709,34727,34729,34737,34739,34747,34749,34815,34817,34819,34821,34827,34829,34837,34839,34847],{},[186,34668],{},[572,34670,926],{},[189,34672,34673,34675],{},[192,34674,194],{},[202,34676,7429],{},[14789,34678,34679,34681,34689],{},[572,34680,1937],{},[186,34682,34683,34685,34687],{},[202,34684,446],{},[572,34686,926],{},[192,34688,194],{},[202,34690,7429],{},[14789,34692,34693,34695,34703],{},[572,34694,1937],{},[186,34696,34697,34699,34701],{},[202,34698,888],{},[572,34700,926],{},[192,34702,194],{},[199,34704,34705,34707],{},[202,34706,13348],{},[202,34708,446],{},[1933,34710,34711,34713,34719],{},[202,34712,885],{},[576,34714,34715,34717],{},[202,34716,14157],{},[572,34718,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[186,34720,34721,34723,34725],{},[202,34722,27616],{},[202,34724,885],{},[202,34726,5714],{},[572,34728,909],{"stretchy":908},[1933,34730,34731,34733,34735],{},[202,34732,12908],{},[202,34734,446],{},[202,34736,888],{},[572,34738,914],{"separator":217},[1933,34740,34741,34743,34745],{},[202,34742,134],{},[202,34744,446],{},[202,34746,888],{},[572,34748,923],{"stretchy":908},[189,34750,34751,34781],{},[186,34752,34753,34759,34761,34769,34771,34779],{},[199,34754,34755,34757],{},[202,34756,5711],{},[202,34758,14157],{},[572,34760,909],{"stretchy":908},[1933,34762,34763,34765,34767],{},[202,34764,134],{},[202,34766,446],{},[202,34768,888],{},[202,34770,13047],{"mathvariant":929},[1933,34772,34773,34775,34777],{},[202,34774,12908],{},[202,34776,446],{},[202,34778,888],{},[572,34780,923],{"stretchy":908},[186,34782,34783,34793,34795,34803,34805,34813],{},[199,34784,34785,34787],{},[202,34786,5711],{},[576,34788,34789,34791],{},[202,34790,14157],{},[572,34792,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[572,34794,909],{"stretchy":908},[1933,34796,34797,34799,34801],{},[202,34798,134],{},[202,34800,446],{},[202,34802,888],{},[202,34804,13047],{"mathvariant":929},[1933,34806,34807,34809,34811],{},[202,34808,12908],{},[202,34810,446],{},[202,34812,888],{},[572,34814,923],{"stretchy":908},[202,34816,14486],{"mathvariant":929},[202,34818,14911],{},[572,34820,933],{},[199,34822,34823,34825],{},[202,34824,5711],{},[202,34826,14157],{},[572,34828,909],{"stretchy":908},[1933,34830,34831,34833,34835],{},[202,34832,134],{},[202,34834,446],{},[202,34836,888],{},[202,34838,13047],{"mathvariant":929},[1933,34840,34841,34843,34845],{},[202,34842,12908],{},[202,34844,446],{},[202,34846,888],{},[572,34848,923],{"stretchy":908},[14474,34850,34851,34857],{},[14477,34852,34853],{},[14480,34854,34855],{"scriptlevel":1865,"displaystyle":217},[186,34856],{},[14477,34858,34859],{},[14480,34860,34861],{"scriptlevel":1865,"displaystyle":217},[186,34862,34863,34865,34867,34873,34887,34905,34923,34925,34933,34935,34943,34945,35011],{},[186,34864],{},[572,34866,926],{},[189,34868,34869,34871],{},[192,34870,194],{},[202,34872,7429],{},[14789,34874,34875,34877,34885],{},[572,34876,1937],{},[186,34878,34879,34881,34883],{},[202,34880,446],{},[572,34882,926],{},[192,34884,194],{},[202,34886,7429],{},[14789,34888,34889,34891,34899],{},[572,34890,1937],{},[186,34892,34893,34895,34897],{},[202,34894,888],{},[572,34896,926],{},[192,34898,194],{},[199,34900,34901,34903],{},[202,34902,13348],{},[202,34904,446],{},[1933,34906,34907,34909,34915],{},[202,34908,885],{},[576,34910,34911,34913],{},[202,34912,14157],{},[572,34914,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[186,34916,34917,34919,34921],{},[202,34918,27616],{},[202,34920,885],{},[202,34922,5714],{},[572,34924,909],{"stretchy":908},[1933,34926,34927,34929,34931],{},[202,34928,12908],{},[202,34930,446],{},[202,34932,888],{},[572,34934,914],{"separator":217},[1933,34936,34937,34939,34941],{},[202,34938,134],{},[202,34940,446],{},[202,34942,888],{},[572,34944,923],{"stretchy":908},[189,34946,34947,34977],{},[186,34948,34949,34955,34957,34965,34967,34975],{},[199,34950,34951,34953],{},[202,34952,5711],{},[202,34954,14157],{},[572,34956,909],{"stretchy":908},[1933,34958,34959,34961,34963],{},[202,34960,134],{},[202,34962,446],{},[202,34964,888],{},[202,34966,13047],{"mathvariant":929},[1933,34968,34969,34971,34973],{},[202,34970,12908],{},[202,34972,446],{},[202,34974,888],{},[572,34976,923],{"stretchy":908},[186,34978,34979,34989,34991,34999,35001,35009],{},[199,34980,34981,34983],{},[202,34982,5711],{},[576,34984,34985,34987],{},[202,34986,14157],{},[572,34988,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[572,34990,909],{"stretchy":908},[1933,34992,34993,34995,34997],{},[202,34994,134],{},[202,34996,446],{},[202,34998,888],{},[202,35000,13047],{"mathvariant":929},[1933,35002,35003,35005,35007],{},[202,35004,12908],{},[202,35006,446],{},[202,35008,888],{},[572,35010,923],{"stretchy":908},[189,35012,35013,35045],{},[186,35014,35015,35017,35023,35025,35033,35035,35043],{},[202,35016,14486],{"mathvariant":929},[199,35018,35019,35021],{},[202,35020,5711],{},[202,35022,14157],{},[572,35024,909],{"stretchy":908},[1933,35026,35027,35029,35031],{},[202,35028,134],{},[202,35030,446],{},[202,35032,888],{},[202,35034,13047],{"mathvariant":929},[1933,35036,35037,35039,35041],{},[202,35038,12908],{},[202,35040,446],{},[202,35042,888],{},[572,35044,923],{"stretchy":908},[186,35046,35047,35053,35055,35063,35065,35073],{},[199,35048,35049,35051],{},[202,35050,5711],{},[202,35052,14157],{},[572,35054,909],{"stretchy":908},[1933,35056,35057,35059,35061],{},[202,35058,134],{},[202,35060,446],{},[202,35062,888],{},[202,35064,13047],{"mathvariant":929},[1933,35066,35067,35069,35071],{},[202,35068,12908],{},[202,35070,446],{},[202,35072,888],{},[572,35074,923],{"stretchy":908},[14474,35076,35077,35083],{},[14477,35078,35079],{},[14480,35080,35081],{"scriptlevel":1865,"displaystyle":217},[186,35082],{},[14477,35084,35085],{},[14480,35086,35087],{"scriptlevel":1865,"displaystyle":217},[186,35088,35089,35091,35093,35099,35113,35131,35149,35151,35159,35161,35169,35171],{},[186,35090],{},[572,35092,926],{},[189,35094,35095,35097],{},[192,35096,194],{},[202,35098,7429],{},[14789,35100,35101,35103,35111],{},[572,35102,1937],{},[186,35104,35105,35107,35109],{},[202,35106,446],{},[572,35108,926],{},[192,35110,194],{},[202,35112,7429],{},[14789,35114,35115,35117,35125],{},[572,35116,1937],{},[186,35118,35119,35121,35123],{},[202,35120,888],{},[572,35122,926],{},[192,35124,194],{},[199,35126,35127,35129],{},[202,35128,13348],{},[202,35130,446],{},[1933,35132,35133,35135,35141],{},[202,35134,885],{},[576,35136,35137,35139],{},[202,35138,14157],{},[572,35140,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[186,35142,35143,35145,35147],{},[202,35144,27616],{},[202,35146,885],{},[202,35148,5714],{},[572,35150,909],{"stretchy":908},[1933,35152,35153,35155,35157],{},[202,35154,12908],{},[202,35156,446],{},[202,35158,888],{},[572,35160,914],{"separator":217},[1933,35162,35163,35165,35167],{},[202,35164,134],{},[202,35166,446],{},[202,35168,888],{},[572,35170,923],{"stretchy":908},[189,35172,35173,35205],{},[186,35174,35175,35177,35183,35185,35193,35195,35203],{},[202,35176,14486],{"mathvariant":929},[199,35178,35179,35181],{},[202,35180,5711],{},[202,35182,14157],{},[572,35184,909],{"stretchy":908},[1933,35186,35187,35189,35191],{},[202,35188,134],{},[202,35190,446],{},[202,35192,888],{},[202,35194,13047],{"mathvariant":929},[1933,35196,35197,35199,35201],{},[202,35198,12908],{},[202,35200,446],{},[202,35202,888],{},[572,35204,923],{"stretchy":908},[186,35206,35207,35217,35219,35227,35229,35237],{},[199,35208,35209,35211],{},[202,35210,5711],{},[576,35212,35213,35215],{},[202,35214,14157],{},[572,35216,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[572,35218,909],{"stretchy":908},[1933,35220,35221,35223,35225],{},[202,35222,134],{},[202,35224,446],{},[202,35226,888],{},[202,35228,13047],{"mathvariant":929},[1933,35230,35231,35233,35235],{},[202,35232,12908],{},[202,35234,446],{},[202,35236,888],{},[572,35238,923],{"stretchy":908},[209,35240,35241],{"encoding":211},"\\begin{aligned}\n&\\frac{1}{N} \\sum_{n=1}^{N} \\sum_{t=1}^{T_n} A_\\theta^{GAE}(s_n^t, a_n^t) \\nabla \\log P_\\theta(a_n^t | s_n^t) \\\\\n&= \\frac{1}{N} \\sum_{n=1}^{N} \\sum_{t=1}^{T_n} A_{\\theta'}^{GAE}(s_n^t, a_n^t) \\frac{P_\\theta(a_n^t | s_n^t)}{P_{\\theta'}(a_n^t | s_n^t)} \\nabla \\log P_\\theta(a_n^t | s_n^t) \\\\\n&= \\frac{1}{N} \\sum_{n=1}^{N} \\sum_{t=1}^{T_n} A_{\\theta'}^{GAE}(s_n^t, a_n^t) \\frac{P_\\theta(a_n^t | s_n^t)}{P_{\\theta'}(a_n^t | s_n^t)} \\frac{\\nabla P_\\theta(a_n^t | s_n^t)}{P_\\theta(a_n^t | s_n^t)} \\\\\n&= \\frac{1}{N} \\sum_{n=1}^{N} \\sum_{t=1}^{T_n} A_{\\theta'}^{GAE}(s_n^t, a_n^t) \\frac{\\nabla P_\\theta(a_n^t | s_n^t)}{P_{\\theta'}(a_n^t | s_n^t)}\n\\end{aligned}",[170,35243,35245],{"className":35244,"ariaHidden":217},[216],[170,35246,35248,35252],{"className":35247},[221],[170,35249],{"className":35250,"style":35251},[225],"height:13.6262em;vertical-align:-6.5631em;",[170,35253,35255],{"className":35254},[230],[170,35256,35258,35319],{"className":35257},[14468],[170,35259,35261],{"className":35260},[15259],[170,35262,35264,35310],{"className":35263},[242,243],[170,35265,35267,35307],{"className":35266},[247],[170,35268,35271,35280,35289,35298],{"className":35269,"style":35270},[251],"height:7.0631em;",[170,35272,35274,35277],{"style":35273},"top:-9.0631em;",[170,35275],{"className":35276,"style":15276},[259],[170,35278],{"className":35279},[230],[170,35281,35283,35286],{"style":35282},"top:-5.6565em;",[170,35284],{"className":35285,"style":15276},[259],[170,35287],{"className":35288},[230],[170,35290,35292,35295],{"style":35291},"top:-2.25em;",[170,35293],{"className":35294,"style":15276},[259],[170,35296],{"className":35297},[230],[170,35299,35301,35304],{"style":35300},"top:1.1565em;",[170,35302],{"className":35303,"style":15276},[259],[170,35305],{"className":35306},[230],[170,35308,338],{"className":35309},[337],[170,35311,35313],{"className":35312},[247],[170,35314,35317],{"className":35315,"style":35316},[251],"height:6.5631em;",[170,35318],{},[170,35320,35322],{"className":35321},[15483],[170,35323,35325,39031],{"className":35324},[242,243],[170,35326,35328,39028],{"className":35327},[247],[170,35329,35331,35923,36948,38167],{"className":35330,"style":35270},[251],[170,35332,35333,35336],{"style":35273},[170,35334],{"className":35335,"style":15276},[259],[170,35337,35339,35342,35404,35407,35474,35477,35581,35584,35644,35647,35698,35701,35704,35755,35758,35761,35764,35769,35772,35812,35815,35866,35869,35920],{"className":35338},[230],[170,35340],{"className":35341},[230],[170,35343,35345,35348,35401],{"className":35344},[230],[170,35346],{"className":35347},[234,235],[170,35349,35351],{"className":35350},[189],[170,35352,35354,35393],{"className":35353},[242,243],[170,35355,35357,35390],{"className":35356},[247],[170,35358,35360,35371,35379],{"className":35359,"style":16385},[251],[170,35361,35362,35365],{"style":15925},[170,35363],{"className":35364,"style":260},[259],[170,35366,35368],{"className":35367},[230],[170,35369,7429],{"className":35370,"style":7504},[230,303],[170,35372,35373,35376],{"style":385},[170,35374],{"className":35375,"style":260},[259],[170,35377],{"className":35378,"style":393},[392],[170,35380,35381,35384],{"style":15991},[170,35382],{"className":35383,"style":260},[259],[170,35385,35387],{"className":35386},[230],[170,35388,194],{"className":35389},[230],[170,35391,338],{"className":35392},[337],[170,35394,35396],{"className":35395},[247],[170,35397,35399],{"className":35398,"style":16425},[251],[170,35400],{},[170,35402],{"className":35403},[424,235],[170,35405],{"className":35406,"style":1027},[610],[170,35408,35410],{"className":35409},[1062,14344],[170,35411,35413,35466],{"className":35412},[242,243],[170,35414,35416,35463],{"className":35415},[247],[170,35417,35419,35439,35449],{"className":35418,"style":16446},[251],[170,35420,35421,35424],{"style":16449},[170,35422],{"className":35423,"style":15538},[259],[170,35425,35427],{"className":35426},[264,265,266,267],[170,35428,35430,35433,35436],{"className":35429},[230,267],[170,35431,446],{"className":35432},[230,303,267],[170,35434,926],{"className":35435},[615,267],[170,35437,194],{"className":35438},[230,267],[170,35440,35441,35444],{"style":15547},[170,35442],{"className":35443,"style":15538},[259],[170,35445,35446],{},[170,35447,1937],{"className":35448},[1062,2021,15556],[170,35450,35451,35454],{"style":16480},[170,35452],{"className":35453,"style":15538},[259],[170,35455,35457],{"className":35456},[264,265,266,267],[170,35458,35460],{"className":35459},[230,267],[170,35461,7429],{"className":35462,"style":7504},[230,303,267],[170,35464,338],{"className":35465},[337],[170,35467,35469],{"className":35468},[247],[170,35470,35472],{"className":35471,"style":16499},[251],[170,35473],{},[170,35475],{"className":35476,"style":1027},[610],[170,35478,35480],{"className":35479},[1062,14344],[170,35481,35483,35573],{"className":35482},[242,243],[170,35484,35486,35570],{"className":35485},[247],[170,35487,35489,35509,35519],{"className":35488,"style":17255},[251],[170,35490,35491,35494],{"style":16449},[170,35492],{"className":35493,"style":15538},[259],[170,35495,35497],{"className":35496},[264,265,266,267],[170,35498,35500,35503,35506],{"className":35499},[230,267],[170,35501,888],{"className":35502},[230,303,267],[170,35504,926],{"className":35505},[615,267],[170,35507,194],{"className":35508},[230,267],[170,35510,35511,35514],{"style":15547},[170,35512],{"className":35513,"style":15538},[259],[170,35515,35516],{},[170,35517,1937],{"className":35518},[1062,2021,15556],[170,35520,35521,35524],{"style":17288},[170,35522],{"className":35523,"style":15538},[259],[170,35525,35527],{"className":35526},[264,265,266,267],[170,35528,35530],{"className":35529},[230,267],[170,35531,35533,35536],{"className":35532},[230,267],[170,35534,13348],{"className":35535,"style":3292},[230,303,267],[170,35537,35539],{"className":35538},[307],[170,35540,35542,35562],{"className":35541},[242,243],[170,35543,35545,35559],{"className":35544},[247],[170,35546,35548],{"className":35547,"style":4409},[251],[170,35549,35550,35553],{"style":17318},[170,35551],{"className":35552,"style":324},[259],[170,35554,35556],{"className":35555},[264,328,329,267],[170,35557,446],{"className":35558},[230,303,267],[170,35560,338],{"className":35561},[337],[170,35563,35565],{"className":35564},[247],[170,35566,35568],{"className":35567,"style":4431},[251],[170,35569],{},[170,35571,338],{"className":35572},[337],[170,35574,35576],{"className":35575},[247],[170,35577,35579],{"className":35578,"style":16499},[251],[170,35580],{},[170,35582],{"className":35583,"style":1027},[610],[170,35585,35587,35590],{"className":35586},[230],[170,35588,885],{"className":35589},[230,303],[170,35591,35593],{"className":35592},[307],[170,35594,35596,35636],{"className":35595},[242,243],[170,35597,35599,35633],{"className":35598},[247],[170,35600,35602,35613],{"className":35601,"style":24902},[251],[170,35603,35604,35607],{"style":17417},[170,35605],{"className":35606,"style":651},[259],[170,35608,35610],{"className":35609},[264,265,266,267],[170,35611,14157],{"className":35612,"style":3317},[230,303,267],[170,35614,35615,35618],{"style":16532},[170,35616],{"className":35617,"style":651},[259],[170,35619,35621],{"className":35620},[264,265,266,267],[170,35622,35624,35627,35630],{"className":35623},[230,267],[170,35625,27616],{"className":35626},[230,303,267],[170,35628,885],{"className":35629},[230,303,267],[170,35631,5714],{"className":35632,"style":6157},[230,303,267],[170,35634,338],{"className":35635},[337],[170,35637,35639],{"className":35638},[247],[170,35640,35642],{"className":35641,"style":17447},[251],[170,35643],{},[170,35645,909],{"className":35646},[234],[170,35648,35650,35653],{"className":35649},[230],[170,35651,12908],{"className":35652},[230,303],[170,35654,35656],{"className":35655},[307],[170,35657,35659,35690],{"className":35658},[242,243],[170,35660,35662,35687],{"className":35661},[247],[170,35663,35665,35676],{"className":35664,"style":17414},[251],[170,35666,35667,35670],{"style":17417},[170,35668],{"className":35669,"style":651},[259],[170,35671,35673],{"className":35672},[264,265,266,267],[170,35674,446],{"className":35675},[230,303,267],[170,35677,35678,35681],{"style":16532},[170,35679],{"className":35680,"style":651},[259],[170,35682,35684],{"className":35683},[264,265,266,267],[170,35685,888],{"className":35686},[230,303,267],[170,35688,338],{"className":35689},[337],[170,35691,35693],{"className":35692},[247],[170,35694,35696],{"className":35695,"style":17447},[251],[170,35697],{},[170,35699,914],{"className":35700},[1023],[170,35702],{"className":35703,"style":1027},[610],[170,35705,35707,35710],{"className":35706},[230],[170,35708,134],{"className":35709},[230,303],[170,35711,35713],{"className":35712},[307],[170,35714,35716,35747],{"className":35715},[242,243],[170,35717,35719,35744],{"className":35718},[247],[170,35720,35722,35733],{"className":35721,"style":17414},[251],[170,35723,35724,35727],{"style":17417},[170,35725],{"className":35726,"style":651},[259],[170,35728,35730],{"className":35729},[264,265,266,267],[170,35731,446],{"className":35732},[230,303,267],[170,35734,35735,35738],{"style":16532},[170,35736],{"className":35737,"style":651},[259],[170,35739,35741],{"className":35740},[264,265,266,267],[170,35742,888],{"className":35743},[230,303,267],[170,35745,338],{"className":35746},[337],[170,35748,35750],{"className":35749},[247],[170,35751,35753],{"className":35752,"style":17447},[251],[170,35754],{},[170,35756,923],{"className":35757},[424],[170,35759,14486],{"className":35760},[230],[170,35762],{"className":35763,"style":1027},[610],[170,35765,16956,35767],{"className":35766},[1062],[170,35768,16960],{"style":16959},[170,35770],{"className":35771,"style":1027},[610],[170,35773,35775,35778],{"className":35774},[230],[170,35776,5711],{"className":35777,"style":3292},[230,303],[170,35779,35781],{"className":35780},[307],[170,35782,35784,35804],{"className":35783},[242,243],[170,35785,35787,35801],{"className":35786},[247],[170,35788,35790],{"className":35789,"style":1359},[251],[170,35791,35792,35795],{"style":7567},[170,35793],{"className":35794,"style":651},[259],[170,35796,35798],{"className":35797},[264,265,266,267],[170,35799,14157],{"className":35800,"style":3317},[230,303,267],[170,35802,338],{"className":35803},[337],[170,35805,35807],{"className":35806},[247],[170,35808,35810],{"className":35809,"style":1381},[251],[170,35811],{},[170,35813,909],{"className":35814},[234],[170,35816,35818,35821],{"className":35817},[230],[170,35819,134],{"className":35820},[230,303],[170,35822,35824],{"className":35823},[307],[170,35825,35827,35858],{"className":35826},[242,243],[170,35828,35830,35855],{"className":35829},[247],[170,35831,35833,35844],{"className":35832,"style":17414},[251],[170,35834,35835,35838],{"style":17417},[170,35836],{"className":35837,"style":651},[259],[170,35839,35841],{"className":35840},[264,265,266,267],[170,35842,446],{"className":35843},[230,303,267],[170,35845,35846,35849],{"style":16532},[170,35847],{"className":35848,"style":651},[259],[170,35850,35852],{"className":35851},[264,265,266,267],[170,35853,888],{"className":35854},[230,303,267],[170,35856,338],{"className":35857},[337],[170,35859,35861],{"className":35860},[247],[170,35862,35864],{"className":35863,"style":17447},[251],[170,35865],{},[170,35867,13047],{"className":35868},[230],[170,35870,35872,35875],{"className":35871},[230],[170,35873,12908],{"className":35874},[230,303],[170,35876,35878],{"className":35877},[307],[170,35879,35881,35912],{"className":35880},[242,243],[170,35882,35884,35909],{"className":35883},[247],[170,35885,35887,35898],{"className":35886,"style":17414},[251],[170,35888,35889,35892],{"style":17417},[170,35890],{"className":35891,"style":651},[259],[170,35893,35895],{"className":35894},[264,265,266,267],[170,35896,446],{"className":35897},[230,303,267],[170,35899,35900,35903],{"style":16532},[170,35901],{"className":35902,"style":651},[259],[170,35904,35906],{"className":35905},[264,265,266,267],[170,35907,888],{"className":35908},[230,303,267],[170,35910,338],{"className":35911},[337],[170,35913,35915],{"className":35914},[247],[170,35916,35918],{"className":35917,"style":17447},[251],[170,35919],{},[170,35921,923],{"className":35922},[424],[170,35924,35925,35928],{"style":35282},[170,35926],{"className":35927,"style":15276},[259],[170,35929,35931,35934,35937,35940,35943,36005,36008,36075,36078,36182,36185,36277,36280,36331,36334,36337,36388,36391,36783,36786,36789,36794,36797,36837,36840,36891,36894,36945],{"className":35930},[230],[170,35932],{"className":35933},[230],[170,35935],{"className":35936,"style":611},[610],[170,35938,926],{"className":35939},[615],[170,35941],{"className":35942,"style":611},[610],[170,35944,35946,35949,36002],{"className":35945},[230],[170,35947],{"className":35948},[234,235],[170,35950,35952],{"className":35951},[189],[170,35953,35955,35994],{"className":35954},[242,243],[170,35956,35958,35991],{"className":35957},[247],[170,35959,35961,35972,35980],{"className":35960,"style":16385},[251],[170,35962,35963,35966],{"style":15925},[170,35964],{"className":35965,"style":260},[259],[170,35967,35969],{"className":35968},[230],[170,35970,7429],{"className":35971,"style":7504},[230,303],[170,35973,35974,35977],{"style":385},[170,35975],{"className":35976,"style":260},[259],[170,35978],{"className":35979,"style":393},[392],[170,35981,35982,35985],{"style":15991},[170,35983],{"className":35984,"style":260},[259],[170,35986,35988],{"className":35987},[230],[170,35989,194],{"className":35990},[230],[170,35992,338],{"className":35993},[337],[170,35995,35997],{"className":35996},[247],[170,35998,36000],{"className":35999,"style":16425},[251],[170,36001],{},[170,36003],{"className":36004},[424,235],[170,36006],{"className":36007,"style":1027},[610],[170,36009,36011],{"className":36010},[1062,14344],[170,36012,36014,36067],{"className":36013},[242,243],[170,36015,36017,36064],{"className":36016},[247],[170,36018,36020,36040,36050],{"className":36019,"style":16446},[251],[170,36021,36022,36025],{"style":16449},[170,36023],{"className":36024,"style":15538},[259],[170,36026,36028],{"className":36027},[264,265,266,267],[170,36029,36031,36034,36037],{"className":36030},[230,267],[170,36032,446],{"className":36033},[230,303,267],[170,36035,926],{"className":36036},[615,267],[170,36038,194],{"className":36039},[230,267],[170,36041,36042,36045],{"style":15547},[170,36043],{"className":36044,"style":15538},[259],[170,36046,36047],{},[170,36048,1937],{"className":36049},[1062,2021,15556],[170,36051,36052,36055],{"style":16480},[170,36053],{"className":36054,"style":15538},[259],[170,36056,36058],{"className":36057},[264,265,266,267],[170,36059,36061],{"className":36060},[230,267],[170,36062,7429],{"className":36063,"style":7504},[230,303,267],[170,36065,338],{"className":36066},[337],[170,36068,36070],{"className":36069},[247],[170,36071,36073],{"className":36072,"style":16499},[251],[170,36074],{},[170,36076],{"className":36077,"style":1027},[610],[170,36079,36081],{"className":36080},[1062,14344],[170,36082,36084,36174],{"className":36083},[242,243],[170,36085,36087,36171],{"className":36086},[247],[170,36088,36090,36110,36120],{"className":36089,"style":17255},[251],[170,36091,36092,36095],{"style":16449},[170,36093],{"className":36094,"style":15538},[259],[170,36096,36098],{"className":36097},[264,265,266,267],[170,36099,36101,36104,36107],{"className":36100},[230,267],[170,36102,888],{"className":36103},[230,303,267],[170,36105,926],{"className":36106},[615,267],[170,36108,194],{"className":36109},[230,267],[170,36111,36112,36115],{"style":15547},[170,36113],{"className":36114,"style":15538},[259],[170,36116,36117],{},[170,36118,1937],{"className":36119},[1062,2021,15556],[170,36121,36122,36125],{"style":17288},[170,36123],{"className":36124,"style":15538},[259],[170,36126,36128],{"className":36127},[264,265,266,267],[170,36129,36131],{"className":36130},[230,267],[170,36132,36134,36137],{"className":36133},[230,267],[170,36135,13348],{"className":36136,"style":3292},[230,303,267],[170,36138,36140],{"className":36139},[307],[170,36141,36143,36163],{"className":36142},[242,243],[170,36144,36146,36160],{"className":36145},[247],[170,36147,36149],{"className":36148,"style":4409},[251],[170,36150,36151,36154],{"style":17318},[170,36152],{"className":36153,"style":324},[259],[170,36155,36157],{"className":36156},[264,328,329,267],[170,36158,446],{"className":36159},[230,303,267],[170,36161,338],{"className":36162},[337],[170,36164,36166],{"className":36165},[247],[170,36167,36169],{"className":36168,"style":4431},[251],[170,36170],{},[170,36172,338],{"className":36173},[337],[170,36175,36177],{"className":36176},[247],[170,36178,36180],{"className":36179,"style":16499},[251],[170,36181],{},[170,36183],{"className":36184,"style":1027},[610],[170,36186,36188,36191],{"className":36187},[230],[170,36189,885],{"className":36190},[230,303],[170,36192,36194],{"className":36193},[307],[170,36195,36197,36269],{"className":36196},[242,243],[170,36198,36200,36266],{"className":36199},[247],[170,36201,36203,36246],{"className":36202,"style":24902},[251],[170,36204,36205,36208],{"style":17417},[170,36206],{"className":36207,"style":651},[259],[170,36209,36211],{"className":36210},[264,265,266,267],[170,36212,36214],{"className":36213},[230,267],[170,36215,36217,36220],{"className":36216},[230,267],[170,36218,14157],{"className":36219,"style":3317},[230,303,267],[170,36221,36223],{"className":36222},[307],[170,36224,36226],{"className":36225},[242],[170,36227,36229],{"className":36228},[247],[170,36230,36232],{"className":36231,"style":19250},[251],[170,36233,36234,36237],{"style":19253},[170,36235],{"className":36236,"style":324},[259],[170,36238,36240],{"className":36239},[264,328,329,267],[170,36241,36243],{"className":36242},[230,267],[170,36244,19090],{"className":36245},[230,267],[170,36247,36248,36251],{"style":16532},[170,36249],{"className":36250,"style":651},[259],[170,36252,36254],{"className":36253},[264,265,266,267],[170,36255,36257,36260,36263],{"className":36256},[230,267],[170,36258,27616],{"className":36259},[230,303,267],[170,36261,885],{"className":36262},[230,303,267],[170,36264,5714],{"className":36265,"style":6157},[230,303,267],[170,36267,338],{"className":36268},[337],[170,36270,36272],{"className":36271},[247],[170,36273,36275],{"className":36274,"style":17447},[251],[170,36276],{},[170,36278,909],{"className":36279},[234],[170,36281,36283,36286],{"className":36282},[230],[170,36284,12908],{"className":36285},[230,303],[170,36287,36289],{"className":36288},[307],[170,36290,36292,36323],{"className":36291},[242,243],[170,36293,36295,36320],{"className":36294},[247],[170,36296,36298,36309],{"className":36297,"style":17414},[251],[170,36299,36300,36303],{"style":17417},[170,36301],{"className":36302,"style":651},[259],[170,36304,36306],{"className":36305},[264,265,266,267],[170,36307,446],{"className":36308},[230,303,267],[170,36310,36311,36314],{"style":16532},[170,36312],{"className":36313,"style":651},[259],[170,36315,36317],{"className":36316},[264,265,266,267],[170,36318,888],{"className":36319},[230,303,267],[170,36321,338],{"className":36322},[337],[170,36324,36326],{"className":36325},[247],[170,36327,36329],{"className":36328,"style":17447},[251],[170,36330],{},[170,36332,914],{"className":36333},[1023],[170,36335],{"className":36336,"style":1027},[610],[170,36338,36340,36343],{"className":36339},[230],[170,36341,134],{"className":36342},[230,303],[170,36344,36346],{"className":36345},[307],[170,36347,36349,36380],{"className":36348},[242,243],[170,36350,36352,36377],{"className":36351},[247],[170,36353,36355,36366],{"className":36354,"style":17414},[251],[170,36356,36357,36360],{"style":17417},[170,36358],{"className":36359,"style":651},[259],[170,36361,36363],{"className":36362},[264,265,266,267],[170,36364,446],{"className":36365},[230,303,267],[170,36367,36368,36371],{"style":16532},[170,36369],{"className":36370,"style":651},[259],[170,36372,36374],{"className":36373},[264,265,266,267],[170,36375,888],{"className":36376},[230,303,267],[170,36378,338],{"className":36379},[337],[170,36381,36383],{"className":36382},[247],[170,36384,36386],{"className":36385,"style":17447},[251],[170,36387],{},[170,36389,923],{"className":36390},[424],[170,36392,36394,36397,36780],{"className":36393},[230],[170,36395],{"className":36396},[234,235],[170,36398,36400],{"className":36399},[189],[170,36401,36403,36772],{"className":36402},[242,243],[170,36404,36406,36769],{"className":36405},[247],[170,36407,36410,36602,36610],{"className":36408,"style":36409},[251],"height:1.4706em;",[170,36411,36412,36415],{"style":15925},[170,36413],{"className":36414,"style":260},[259],[170,36416,36418,36490,36493,36545,36548,36599],{"className":36417},[230],[170,36419,36421,36424],{"className":36420},[230],[170,36422,5711],{"className":36423,"style":3292},[230,303],[170,36425,36427],{"className":36426},[307],[170,36428,36430,36482],{"className":36429},[242,243],[170,36431,36433,36479],{"className":36432},[247],[170,36434,36436],{"className":36435,"style":1359},[251],[170,36437,36438,36441],{"style":7567},[170,36439],{"className":36440,"style":651},[259],[170,36442,36444],{"className":36443},[264,265,266,267],[170,36445,36447],{"className":36446},[230,267],[170,36448,36450,36453],{"className":36449},[230,267],[170,36451,14157],{"className":36452,"style":3317},[230,303,267],[170,36454,36456],{"className":36455},[307],[170,36457,36459],{"className":36458},[242],[170,36460,36462],{"className":36461},[247],[170,36463,36465],{"className":36464,"style":19250},[251],[170,36466,36467,36470],{"style":19253},[170,36468],{"className":36469,"style":324},[259],[170,36471,36473],{"className":36472},[264,328,329,267],[170,36474,36476],{"className":36475},[230,267],[170,36477,19090],{"className":36478},[230,267],[170,36480,338],{"className":36481},[337],[170,36483,36485],{"className":36484},[247],[170,36486,36488],{"className":36487,"style":1381},[251],[170,36489],{},[170,36491,909],{"className":36492},[234],[170,36494,36496,36499],{"className":36495},[230],[170,36497,134],{"className":36498},[230,303],[170,36500,36502],{"className":36501},[307],[170,36503,36505,36537],{"className":36504},[242,243],[170,36506,36508,36534],{"className":36507},[247],[170,36509,36512,36523],{"className":36510,"style":36511},[251],"height:0.7196em;",[170,36513,36514,36517],{"style":17417},[170,36515],{"className":36516,"style":651},[259],[170,36518,36520],{"className":36519},[264,265,266,267],[170,36521,446],{"className":36522},[230,303,267],[170,36524,36525,36528],{"style":16635},[170,36526],{"className":36527,"style":651},[259],[170,36529,36531],{"className":36530},[264,265,266,267],[170,36532,888],{"className":36533},[230,303,267],[170,36535,338],{"className":36536},[337],[170,36538,36540],{"className":36539},[247],[170,36541,36543],{"className":36542,"style":17447},[251],[170,36544],{},[170,36546,13047],{"className":36547},[230],[170,36549,36551,36554],{"className":36550},[230],[170,36552,12908],{"className":36553},[230,303],[170,36555,36557],{"className":36556},[307],[170,36558,36560,36591],{"className":36559},[242,243],[170,36561,36563,36588],{"className":36562},[247],[170,36564,36566,36577],{"className":36565,"style":36511},[251],[170,36567,36568,36571],{"style":17417},[170,36569],{"className":36570,"style":651},[259],[170,36572,36574],{"className":36573},[264,265,266,267],[170,36575,446],{"className":36576},[230,303,267],[170,36578,36579,36582],{"style":16635},[170,36580],{"className":36581,"style":651},[259],[170,36583,36585],{"className":36584},[264,265,266,267],[170,36586,888],{"className":36587},[230,303,267],[170,36589,338],{"className":36590},[337],[170,36592,36594],{"className":36593},[247],[170,36595,36597],{"className":36596,"style":17447},[251],[170,36598],{},[170,36600,923],{"className":36601},[424],[170,36603,36604,36607],{"style":385},[170,36605],{"className":36606,"style":260},[259],[170,36608],{"className":36609,"style":393},[392],[170,36611,36612,36615],{"style":15991},[170,36613],{"className":36614,"style":260},[259],[170,36616,36618,36658,36661,36712,36715,36766],{"className":36617},[230],[170,36619,36621,36624],{"className":36620},[230],[170,36622,5711],{"className":36623,"style":3292},[230,303],[170,36625,36627],{"className":36626},[307],[170,36628,36630,36650],{"className":36629},[242,243],[170,36631,36633,36647],{"className":36632},[247],[170,36634,36636],{"className":36635,"style":1359},[251],[170,36637,36638,36641],{"style":7567},[170,36639],{"className":36640,"style":651},[259],[170,36642,36644],{"className":36643},[264,265,266,267],[170,36645,14157],{"className":36646,"style":3317},[230,303,267],[170,36648,338],{"className":36649},[337],[170,36651,36653],{"className":36652},[247],[170,36654,36656],{"className":36655,"style":1381},[251],[170,36657],{},[170,36659,909],{"className":36660},[234],[170,36662,36664,36667],{"className":36663},[230],[170,36665,134],{"className":36666},[230,303],[170,36668,36670],{"className":36669},[307],[170,36671,36673,36704],{"className":36672},[242,243],[170,36674,36676,36701],{"className":36675},[247],[170,36677,36679,36690],{"className":36678,"style":18955},[251],[170,36680,36681,36684],{"style":17417},[170,36682],{"className":36683,"style":651},[259],[170,36685,36687],{"className":36686},[264,265,266,267],[170,36688,446],{"className":36689},[230,303,267],[170,36691,36692,36695],{"style":647},[170,36693],{"className":36694,"style":651},[259],[170,36696,36698],{"className":36697},[264,265,266,267],[170,36699,888],{"className":36700},[230,303,267],[170,36702,338],{"className":36703},[337],[170,36705,36707],{"className":36706},[247],[170,36708,36710],{"className":36709,"style":17447},[251],[170,36711],{},[170,36713,13047],{"className":36714},[230],[170,36716,36718,36721],{"className":36717},[230],[170,36719,12908],{"className":36720},[230,303],[170,36722,36724],{"className":36723},[307],[170,36725,36727,36758],{"className":36726},[242,243],[170,36728,36730,36755],{"className":36729},[247],[170,36731,36733,36744],{"className":36732,"style":18955},[251],[170,36734,36735,36738],{"style":17417},[170,36736],{"className":36737,"style":651},[259],[170,36739,36741],{"className":36740},[264,265,266,267],[170,36742,446],{"className":36743},[230,303,267],[170,36745,36746,36749],{"style":647},[170,36747],{"className":36748,"style":651},[259],[170,36750,36752],{"className":36751},[264,265,266,267],[170,36753,888],{"className":36754},[230,303,267],[170,36756,338],{"className":36757},[337],[170,36759,36761],{"className":36760},[247],[170,36762,36764],{"className":36763,"style":17447},[251],[170,36765],{},[170,36767,923],{"className":36768},[424],[170,36770,338],{"className":36771},[337],[170,36773,36775],{"className":36774},[247],[170,36776,36778],{"className":36777,"style":16056},[251],[170,36779],{},[170,36781],{"className":36782},[424,235],[170,36784,14486],{"className":36785},[230],[170,36787],{"className":36788,"style":1027},[610],[170,36790,16956,36792],{"className":36791},[1062],[170,36793,16960],{"style":16959},[170,36795],{"className":36796,"style":1027},[610],[170,36798,36800,36803],{"className":36799},[230],[170,36801,5711],{"className":36802,"style":3292},[230,303],[170,36804,36806],{"className":36805},[307],[170,36807,36809,36829],{"className":36808},[242,243],[170,36810,36812,36826],{"className":36811},[247],[170,36813,36815],{"className":36814,"style":1359},[251],[170,36816,36817,36820],{"style":7567},[170,36818],{"className":36819,"style":651},[259],[170,36821,36823],{"className":36822},[264,265,266,267],[170,36824,14157],{"className":36825,"style":3317},[230,303,267],[170,36827,338],{"className":36828},[337],[170,36830,36832],{"className":36831},[247],[170,36833,36835],{"className":36834,"style":1381},[251],[170,36836],{},[170,36838,909],{"className":36839},[234],[170,36841,36843,36846],{"className":36842},[230],[170,36844,134],{"className":36845},[230,303],[170,36847,36849],{"className":36848},[307],[170,36850,36852,36883],{"className":36851},[242,243],[170,36853,36855,36880],{"className":36854},[247],[170,36856,36858,36869],{"className":36857,"style":17414},[251],[170,36859,36860,36863],{"style":17417},[170,36861],{"className":36862,"style":651},[259],[170,36864,36866],{"className":36865},[264,265,266,267],[170,36867,446],{"className":36868},[230,303,267],[170,36870,36871,36874],{"style":16532},[170,36872],{"className":36873,"style":651},[259],[170,36875,36877],{"className":36876},[264,265,266,267],[170,36878,888],{"className":36879},[230,303,267],[170,36881,338],{"className":36882},[337],[170,36884,36886],{"className":36885},[247],[170,36887,36889],{"className":36888,"style":17447},[251],[170,36890],{},[170,36892,13047],{"className":36893},[230],[170,36895,36897,36900],{"className":36896},[230],[170,36898,12908],{"className":36899},[230,303],[170,36901,36903],{"className":36902},[307],[170,36904,36906,36937],{"className":36905},[242,243],[170,36907,36909,36934],{"className":36908},[247],[170,36910,36912,36923],{"className":36911,"style":17414},[251],[170,36913,36914,36917],{"style":17417},[170,36915],{"className":36916,"style":651},[259],[170,36918,36920],{"className":36919},[264,265,266,267],[170,36921,446],{"className":36922},[230,303,267],[170,36924,36925,36928],{"style":16532},[170,36926],{"className":36927,"style":651},[259],[170,36929,36931],{"className":36930},[264,265,266,267],[170,36932,888],{"className":36933},[230,303,267],[170,36935,338],{"className":36936},[337],[170,36938,36940],{"className":36939},[247],[170,36941,36943],{"className":36942,"style":17447},[251],[170,36944],{},[170,36946,923],{"className":36947},[424],[170,36949,36950,36953],{"style":35291},[170,36951],{"className":36952,"style":15276},[259],[170,36954,36956,36959,36962,36965,36968,37030,37033,37100,37103,37207,37210,37302,37305,37356,37359,37362,37413,37416,37806],{"className":36955},[230],[170,36957],{"className":36958},[230],[170,36960],{"className":36961,"style":611},[610],[170,36963,926],{"className":36964},[615],[170,36966],{"className":36967,"style":611},[610],[170,36969,36971,36974,37027],{"className":36970},[230],[170,36972],{"className":36973},[234,235],[170,36975,36977],{"className":36976},[189],[170,36978,36980,37019],{"className":36979},[242,243],[170,36981,36983,37016],{"className":36982},[247],[170,36984,36986,36997,37005],{"className":36985,"style":16385},[251],[170,36987,36988,36991],{"style":15925},[170,36989],{"className":36990,"style":260},[259],[170,36992,36994],{"className":36993},[230],[170,36995,7429],{"className":36996,"style":7504},[230,303],[170,36998,36999,37002],{"style":385},[170,37000],{"className":37001,"style":260},[259],[170,37003],{"className":37004,"style":393},[392],[170,37006,37007,37010],{"style":15991},[170,37008],{"className":37009,"style":260},[259],[170,37011,37013],{"className":37012},[230],[170,37014,194],{"className":37015},[230],[170,37017,338],{"className":37018},[337],[170,37020,37022],{"className":37021},[247],[170,37023,37025],{"className":37024,"style":16425},[251],[170,37026],{},[170,37028],{"className":37029},[424,235],[170,37031],{"className":37032,"style":1027},[610],[170,37034,37036],{"className":37035},[1062,14344],[170,37037,37039,37092],{"className":37038},[242,243],[170,37040,37042,37089],{"className":37041},[247],[170,37043,37045,37065,37075],{"className":37044,"style":16446},[251],[170,37046,37047,37050],{"style":16449},[170,37048],{"className":37049,"style":15538},[259],[170,37051,37053],{"className":37052},[264,265,266,267],[170,37054,37056,37059,37062],{"className":37055},[230,267],[170,37057,446],{"className":37058},[230,303,267],[170,37060,926],{"className":37061},[615,267],[170,37063,194],{"className":37064},[230,267],[170,37066,37067,37070],{"style":15547},[170,37068],{"className":37069,"style":15538},[259],[170,37071,37072],{},[170,37073,1937],{"className":37074},[1062,2021,15556],[170,37076,37077,37080],{"style":16480},[170,37078],{"className":37079,"style":15538},[259],[170,37081,37083],{"className":37082},[264,265,266,267],[170,37084,37086],{"className":37085},[230,267],[170,37087,7429],{"className":37088,"style":7504},[230,303,267],[170,37090,338],{"className":37091},[337],[170,37093,37095],{"className":37094},[247],[170,37096,37098],{"className":37097,"style":16499},[251],[170,37099],{},[170,37101],{"className":37102,"style":1027},[610],[170,37104,37106],{"className":37105},[1062,14344],[170,37107,37109,37199],{"className":37108},[242,243],[170,37110,37112,37196],{"className":37111},[247],[170,37113,37115,37135,37145],{"className":37114,"style":17255},[251],[170,37116,37117,37120],{"style":16449},[170,37118],{"className":37119,"style":15538},[259],[170,37121,37123],{"className":37122},[264,265,266,267],[170,37124,37126,37129,37132],{"className":37125},[230,267],[170,37127,888],{"className":37128},[230,303,267],[170,37130,926],{"className":37131},[615,267],[170,37133,194],{"className":37134},[230,267],[170,37136,37137,37140],{"style":15547},[170,37138],{"className":37139,"style":15538},[259],[170,37141,37142],{},[170,37143,1937],{"className":37144},[1062,2021,15556],[170,37146,37147,37150],{"style":17288},[170,37148],{"className":37149,"style":15538},[259],[170,37151,37153],{"className":37152},[264,265,266,267],[170,37154,37156],{"className":37155},[230,267],[170,37157,37159,37162],{"className":37158},[230,267],[170,37160,13348],{"className":37161,"style":3292},[230,303,267],[170,37163,37165],{"className":37164},[307],[170,37166,37168,37188],{"className":37167},[242,243],[170,37169,37171,37185],{"className":37170},[247],[170,37172,37174],{"className":37173,"style":4409},[251],[170,37175,37176,37179],{"style":17318},[170,37177],{"className":37178,"style":324},[259],[170,37180,37182],{"className":37181},[264,328,329,267],[170,37183,446],{"className":37184},[230,303,267],[170,37186,338],{"className":37187},[337],[170,37189,37191],{"className":37190},[247],[170,37192,37194],{"className":37193,"style":4431},[251],[170,37195],{},[170,37197,338],{"className":37198},[337],[170,37200,37202],{"className":37201},[247],[170,37203,37205],{"className":37204,"style":16499},[251],[170,37206],{},[170,37208],{"className":37209,"style":1027},[610],[170,37211,37213,37216],{"className":37212},[230],[170,37214,885],{"className":37215},[230,303],[170,37217,37219],{"className":37218},[307],[170,37220,37222,37294],{"className":37221},[242,243],[170,37223,37225,37291],{"className":37224},[247],[170,37226,37228,37271],{"className":37227,"style":24902},[251],[170,37229,37230,37233],{"style":17417},[170,37231],{"className":37232,"style":651},[259],[170,37234,37236],{"className":37235},[264,265,266,267],[170,37237,37239],{"className":37238},[230,267],[170,37240,37242,37245],{"className":37241},[230,267],[170,37243,14157],{"className":37244,"style":3317},[230,303,267],[170,37246,37248],{"className":37247},[307],[170,37249,37251],{"className":37250},[242],[170,37252,37254],{"className":37253},[247],[170,37255,37257],{"className":37256,"style":19250},[251],[170,37258,37259,37262],{"style":19253},[170,37260],{"className":37261,"style":324},[259],[170,37263,37265],{"className":37264},[264,328,329,267],[170,37266,37268],{"className":37267},[230,267],[170,37269,19090],{"className":37270},[230,267],[170,37272,37273,37276],{"style":16532},[170,37274],{"className":37275,"style":651},[259],[170,37277,37279],{"className":37278},[264,265,266,267],[170,37280,37282,37285,37288],{"className":37281},[230,267],[170,37283,27616],{"className":37284},[230,303,267],[170,37286,885],{"className":37287},[230,303,267],[170,37289,5714],{"className":37290,"style":6157},[230,303,267],[170,37292,338],{"className":37293},[337],[170,37295,37297],{"className":37296},[247],[170,37298,37300],{"className":37299,"style":17447},[251],[170,37301],{},[170,37303,909],{"className":37304},[234],[170,37306,37308,37311],{"className":37307},[230],[170,37309,12908],{"className":37310},[230,303],[170,37312,37314],{"className":37313},[307],[170,37315,37317,37348],{"className":37316},[242,243],[170,37318,37320,37345],{"className":37319},[247],[170,37321,37323,37334],{"className":37322,"style":17414},[251],[170,37324,37325,37328],{"style":17417},[170,37326],{"className":37327,"style":651},[259],[170,37329,37331],{"className":37330},[264,265,266,267],[170,37332,446],{"className":37333},[230,303,267],[170,37335,37336,37339],{"style":16532},[170,37337],{"className":37338,"style":651},[259],[170,37340,37342],{"className":37341},[264,265,266,267],[170,37343,888],{"className":37344},[230,303,267],[170,37346,338],{"className":37347},[337],[170,37349,37351],{"className":37350},[247],[170,37352,37354],{"className":37353,"style":17447},[251],[170,37355],{},[170,37357,914],{"className":37358},[1023],[170,37360],{"className":37361,"style":1027},[610],[170,37363,37365,37368],{"className":37364},[230],[170,37366,134],{"className":37367},[230,303],[170,37369,37371],{"className":37370},[307],[170,37372,37374,37405],{"className":37373},[242,243],[170,37375,37377,37402],{"className":37376},[247],[170,37378,37380,37391],{"className":37379,"style":17414},[251],[170,37381,37382,37385],{"style":17417},[170,37383],{"className":37384,"style":651},[259],[170,37386,37388],{"className":37387},[264,265,266,267],[170,37389,446],{"className":37390},[230,303,267],[170,37392,37393,37396],{"style":16532},[170,37394],{"className":37395,"style":651},[259],[170,37397,37399],{"className":37398},[264,265,266,267],[170,37400,888],{"className":37401},[230,303,267],[170,37403,338],{"className":37404},[337],[170,37406,37408],{"className":37407},[247],[170,37409,37411],{"className":37410,"style":17447},[251],[170,37412],{},[170,37414,923],{"className":37415},[424],[170,37417,37419,37422,37803],{"className":37418},[230],[170,37420],{"className":37421},[234,235],[170,37423,37425],{"className":37424},[189],[170,37426,37428,37795],{"className":37427},[242,243],[170,37429,37431,37792],{"className":37430},[247],[170,37432,37434,37625,37633],{"className":37433,"style":36409},[251],[170,37435,37436,37439],{"style":15925},[170,37437],{"className":37438,"style":260},[259],[170,37440,37442,37514,37517,37568,37571,37622],{"className":37441},[230],[170,37443,37445,37448],{"className":37444},[230],[170,37446,5711],{"className":37447,"style":3292},[230,303],[170,37449,37451],{"className":37450},[307],[170,37452,37454,37506],{"className":37453},[242,243],[170,37455,37457,37503],{"className":37456},[247],[170,37458,37460],{"className":37459,"style":1359},[251],[170,37461,37462,37465],{"style":7567},[170,37463],{"className":37464,"style":651},[259],[170,37466,37468],{"className":37467},[264,265,266,267],[170,37469,37471],{"className":37470},[230,267],[170,37472,37474,37477],{"className":37473},[230,267],[170,37475,14157],{"className":37476,"style":3317},[230,303,267],[170,37478,37480],{"className":37479},[307],[170,37481,37483],{"className":37482},[242],[170,37484,37486],{"className":37485},[247],[170,37487,37489],{"className":37488,"style":19250},[251],[170,37490,37491,37494],{"style":19253},[170,37492],{"className":37493,"style":324},[259],[170,37495,37497],{"className":37496},[264,328,329,267],[170,37498,37500],{"className":37499},[230,267],[170,37501,19090],{"className":37502},[230,267],[170,37504,338],{"className":37505},[337],[170,37507,37509],{"className":37508},[247],[170,37510,37512],{"className":37511,"style":1381},[251],[170,37513],{},[170,37515,909],{"className":37516},[234],[170,37518,37520,37523],{"className":37519},[230],[170,37521,134],{"className":37522},[230,303],[170,37524,37526],{"className":37525},[307],[170,37527,37529,37560],{"className":37528},[242,243],[170,37530,37532,37557],{"className":37531},[247],[170,37533,37535,37546],{"className":37534,"style":36511},[251],[170,37536,37537,37540],{"style":17417},[170,37538],{"className":37539,"style":651},[259],[170,37541,37543],{"className":37542},[264,265,266,267],[170,37544,446],{"className":37545},[230,303,267],[170,37547,37548,37551],{"style":16635},[170,37549],{"className":37550,"style":651},[259],[170,37552,37554],{"className":37553},[264,265,266,267],[170,37555,888],{"className":37556},[230,303,267],[170,37558,338],{"className":37559},[337],[170,37561,37563],{"className":37562},[247],[170,37564,37566],{"className":37565,"style":17447},[251],[170,37567],{},[170,37569,13047],{"className":37570},[230],[170,37572,37574,37577],{"className":37573},[230],[170,37575,12908],{"className":37576},[230,303],[170,37578,37580],{"className":37579},[307],[170,37581,37583,37614],{"className":37582},[242,243],[170,37584,37586,37611],{"className":37585},[247],[170,37587,37589,37600],{"className":37588,"style":36511},[251],[170,37590,37591,37594],{"style":17417},[170,37592],{"className":37593,"style":651},[259],[170,37595,37597],{"className":37596},[264,265,266,267],[170,37598,446],{"className":37599},[230,303,267],[170,37601,37602,37605],{"style":16635},[170,37603],{"className":37604,"style":651},[259],[170,37606,37608],{"className":37607},[264,265,266,267],[170,37609,888],{"className":37610},[230,303,267],[170,37612,338],{"className":37613},[337],[170,37615,37617],{"className":37616},[247],[170,37618,37620],{"className":37619,"style":17447},[251],[170,37621],{},[170,37623,923],{"className":37624},[424],[170,37626,37627,37630],{"style":385},[170,37628],{"className":37629,"style":260},[259],[170,37631],{"className":37632,"style":393},[392],[170,37634,37635,37638],{"style":15991},[170,37636],{"className":37637,"style":260},[259],[170,37639,37641,37681,37684,37735,37738,37789],{"className":37640},[230],[170,37642,37644,37647],{"className":37643},[230],[170,37645,5711],{"className":37646,"style":3292},[230,303],[170,37648,37650],{"className":37649},[307],[170,37651,37653,37673],{"className":37652},[242,243],[170,37654,37656,37670],{"className":37655},[247],[170,37657,37659],{"className":37658,"style":1359},[251],[170,37660,37661,37664],{"style":7567},[170,37662],{"className":37663,"style":651},[259],[170,37665,37667],{"className":37666},[264,265,266,267],[170,37668,14157],{"className":37669,"style":3317},[230,303,267],[170,37671,338],{"className":37672},[337],[170,37674,37676],{"className":37675},[247],[170,37677,37679],{"className":37678,"style":1381},[251],[170,37680],{},[170,37682,909],{"className":37683},[234],[170,37685,37687,37690],{"className":37686},[230],[170,37688,134],{"className":37689},[230,303],[170,37691,37693],{"className":37692},[307],[170,37694,37696,37727],{"className":37695},[242,243],[170,37697,37699,37724],{"className":37698},[247],[170,37700,37702,37713],{"className":37701,"style":18955},[251],[170,37703,37704,37707],{"style":17417},[170,37705],{"className":37706,"style":651},[259],[170,37708,37710],{"className":37709},[264,265,266,267],[170,37711,446],{"className":37712},[230,303,267],[170,37714,37715,37718],{"style":647},[170,37716],{"className":37717,"style":651},[259],[170,37719,37721],{"className":37720},[264,265,266,267],[170,37722,888],{"className":37723},[230,303,267],[170,37725,338],{"className":37726},[337],[170,37728,37730],{"className":37729},[247],[170,37731,37733],{"className":37732,"style":17447},[251],[170,37734],{},[170,37736,13047],{"className":37737},[230],[170,37739,37741,37744],{"className":37740},[230],[170,37742,12908],{"className":37743},[230,303],[170,37745,37747],{"className":37746},[307],[170,37748,37750,37781],{"className":37749},[242,243],[170,37751,37753,37778],{"className":37752},[247],[170,37754,37756,37767],{"className":37755,"style":18955},[251],[170,37757,37758,37761],{"style":17417},[170,37759],{"className":37760,"style":651},[259],[170,37762,37764],{"className":37763},[264,265,266,267],[170,37765,446],{"className":37766},[230,303,267],[170,37768,37769,37772],{"style":647},[170,37770],{"className":37771,"style":651},[259],[170,37773,37775],{"className":37774},[264,265,266,267],[170,37776,888],{"className":37777},[230,303,267],[170,37779,338],{"className":37780},[337],[170,37782,37784],{"className":37783},[247],[170,37785,37787],{"className":37786,"style":17447},[251],[170,37788],{},[170,37790,923],{"className":37791},[424],[170,37793,338],{"className":37794},[337],[170,37796,37798],{"className":37797},[247],[170,37799,37801],{"className":37800,"style":16056},[251],[170,37802],{},[170,37804],{"className":37805},[424,235],[170,37807,37809,37812,38164],{"className":37808},[230],[170,37810],{"className":37811},[234,235],[170,37813,37815],{"className":37814},[189],[170,37816,37818,38156],{"className":37817},[242,243],[170,37819,37821,38153],{"className":37820},[247],[170,37822,37824,37983,37991],{"className":37823,"style":36409},[251],[170,37825,37826,37829],{"style":15925},[170,37827],{"className":37828,"style":260},[259],[170,37830,37832,37872,37875,37926,37929,37980],{"className":37831},[230],[170,37833,37835,37838],{"className":37834},[230],[170,37836,5711],{"className":37837,"style":3292},[230,303],[170,37839,37841],{"className":37840},[307],[170,37842,37844,37864],{"className":37843},[242,243],[170,37845,37847,37861],{"className":37846},[247],[170,37848,37850],{"className":37849,"style":1359},[251],[170,37851,37852,37855],{"style":7567},[170,37853],{"className":37854,"style":651},[259],[170,37856,37858],{"className":37857},[264,265,266,267],[170,37859,14157],{"className":37860,"style":3317},[230,303,267],[170,37862,338],{"className":37863},[337],[170,37865,37867],{"className":37866},[247],[170,37868,37870],{"className":37869,"style":1381},[251],[170,37871],{},[170,37873,909],{"className":37874},[234],[170,37876,37878,37881],{"className":37877},[230],[170,37879,134],{"className":37880},[230,303],[170,37882,37884],{"className":37883},[307],[170,37885,37887,37918],{"className":37886},[242,243],[170,37888,37890,37915],{"className":37889},[247],[170,37891,37893,37904],{"className":37892,"style":36511},[251],[170,37894,37895,37898],{"style":17417},[170,37896],{"className":37897,"style":651},[259],[170,37899,37901],{"className":37900},[264,265,266,267],[170,37902,446],{"className":37903},[230,303,267],[170,37905,37906,37909],{"style":16635},[170,37907],{"className":37908,"style":651},[259],[170,37910,37912],{"className":37911},[264,265,266,267],[170,37913,888],{"className":37914},[230,303,267],[170,37916,338],{"className":37917},[337],[170,37919,37921],{"className":37920},[247],[170,37922,37924],{"className":37923,"style":17447},[251],[170,37925],{},[170,37927,13047],{"className":37928},[230],[170,37930,37932,37935],{"className":37931},[230],[170,37933,12908],{"className":37934},[230,303],[170,37936,37938],{"className":37937},[307],[170,37939,37941,37972],{"className":37940},[242,243],[170,37942,37944,37969],{"className":37943},[247],[170,37945,37947,37958],{"className":37946,"style":36511},[251],[170,37948,37949,37952],{"style":17417},[170,37950],{"className":37951,"style":651},[259],[170,37953,37955],{"className":37954},[264,265,266,267],[170,37956,446],{"className":37957},[230,303,267],[170,37959,37960,37963],{"style":16635},[170,37961],{"className":37962,"style":651},[259],[170,37964,37966],{"className":37965},[264,265,266,267],[170,37967,888],{"className":37968},[230,303,267],[170,37970,338],{"className":37971},[337],[170,37973,37975],{"className":37974},[247],[170,37976,37978],{"className":37977,"style":17447},[251],[170,37979],{},[170,37981,923],{"className":37982},[424],[170,37984,37985,37988],{"style":385},[170,37986],{"className":37987,"style":260},[259],[170,37989],{"className":37990,"style":393},[392],[170,37992,37993,37996],{"style":15991},[170,37994],{"className":37995,"style":260},[259],[170,37997,37999,38002,38042,38045,38096,38099,38150],{"className":37998},[230],[170,38000,14486],{"className":38001},[230],[170,38003,38005,38008],{"className":38004},[230],[170,38006,5711],{"className":38007,"style":3292},[230,303],[170,38009,38011],{"className":38010},[307],[170,38012,38014,38034],{"className":38013},[242,243],[170,38015,38017,38031],{"className":38016},[247],[170,38018,38020],{"className":38019,"style":1359},[251],[170,38021,38022,38025],{"style":7567},[170,38023],{"className":38024,"style":651},[259],[170,38026,38028],{"className":38027},[264,265,266,267],[170,38029,14157],{"className":38030,"style":3317},[230,303,267],[170,38032,338],{"className":38033},[337],[170,38035,38037],{"className":38036},[247],[170,38038,38040],{"className":38039,"style":1381},[251],[170,38041],{},[170,38043,909],{"className":38044},[234],[170,38046,38048,38051],{"className":38047},[230],[170,38049,134],{"className":38050},[230,303],[170,38052,38054],{"className":38053},[307],[170,38055,38057,38088],{"className":38056},[242,243],[170,38058,38060,38085],{"className":38059},[247],[170,38061,38063,38074],{"className":38062,"style":18955},[251],[170,38064,38065,38068],{"style":17417},[170,38066],{"className":38067,"style":651},[259],[170,38069,38071],{"className":38070},[264,265,266,267],[170,38072,446],{"className":38073},[230,303,267],[170,38075,38076,38079],{"style":647},[170,38077],{"className":38078,"style":651},[259],[170,38080,38082],{"className":38081},[264,265,266,267],[170,38083,888],{"className":38084},[230,303,267],[170,38086,338],{"className":38087},[337],[170,38089,38091],{"className":38090},[247],[170,38092,38094],{"className":38093,"style":17447},[251],[170,38095],{},[170,38097,13047],{"className":38098},[230],[170,38100,38102,38105],{"className":38101},[230],[170,38103,12908],{"className":38104},[230,303],[170,38106,38108],{"className":38107},[307],[170,38109,38111,38142],{"className":38110},[242,243],[170,38112,38114,38139],{"className":38113},[247],[170,38115,38117,38128],{"className":38116,"style":18955},[251],[170,38118,38119,38122],{"style":17417},[170,38120],{"className":38121,"style":651},[259],[170,38123,38125],{"className":38124},[264,265,266,267],[170,38126,446],{"className":38127},[230,303,267],[170,38129,38130,38133],{"style":647},[170,38131],{"className":38132,"style":651},[259],[170,38134,38136],{"className":38135},[264,265,266,267],[170,38137,888],{"className":38138},[230,303,267],[170,38140,338],{"className":38141},[337],[170,38143,38145],{"className":38144},[247],[170,38146,38148],{"className":38147,"style":17447},[251],[170,38149],{},[170,38151,923],{"className":38152},[424],[170,38154,338],{"className":38155},[337],[170,38157,38159],{"className":38158},[247],[170,38160,38162],{"className":38161,"style":16056},[251],[170,38163],{},[170,38165],{"className":38166},[424,235],[170,38168,38169,38172],{"style":35300},[170,38170],{"className":38171,"style":15276},[259],[170,38173,38175,38178,38181,38184,38187,38249,38252,38319,38322,38426,38429,38521,38524,38575,38578,38581,38632,38635],{"className":38174},[230],[170,38176],{"className":38177},[230],[170,38179],{"className":38180,"style":611},[610],[170,38182,926],{"className":38183},[615],[170,38185],{"className":38186,"style":611},[610],[170,38188,38190,38193,38246],{"className":38189},[230],[170,38191],{"className":38192},[234,235],[170,38194,38196],{"className":38195},[189],[170,38197,38199,38238],{"className":38198},[242,243],[170,38200,38202,38235],{"className":38201},[247],[170,38203,38205,38216,38224],{"className":38204,"style":16385},[251],[170,38206,38207,38210],{"style":15925},[170,38208],{"className":38209,"style":260},[259],[170,38211,38213],{"className":38212},[230],[170,38214,7429],{"className":38215,"style":7504},[230,303],[170,38217,38218,38221],{"style":385},[170,38219],{"className":38220,"style":260},[259],[170,38222],{"className":38223,"style":393},[392],[170,38225,38226,38229],{"style":15991},[170,38227],{"className":38228,"style":260},[259],[170,38230,38232],{"className":38231},[230],[170,38233,194],{"className":38234},[230],[170,38236,338],{"className":38237},[337],[170,38239,38241],{"className":38240},[247],[170,38242,38244],{"className":38243,"style":16425},[251],[170,38245],{},[170,38247],{"className":38248},[424,235],[170,38250],{"className":38251,"style":1027},[610],[170,38253,38255],{"className":38254},[1062,14344],[170,38256,38258,38311],{"className":38257},[242,243],[170,38259,38261,38308],{"className":38260},[247],[170,38262,38264,38284,38294],{"className":38263,"style":16446},[251],[170,38265,38266,38269],{"style":16449},[170,38267],{"className":38268,"style":15538},[259],[170,38270,38272],{"className":38271},[264,265,266,267],[170,38273,38275,38278,38281],{"className":38274},[230,267],[170,38276,446],{"className":38277},[230,303,267],[170,38279,926],{"className":38280},[615,267],[170,38282,194],{"className":38283},[230,267],[170,38285,38286,38289],{"style":15547},[170,38287],{"className":38288,"style":15538},[259],[170,38290,38291],{},[170,38292,1937],{"className":38293},[1062,2021,15556],[170,38295,38296,38299],{"style":16480},[170,38297],{"className":38298,"style":15538},[259],[170,38300,38302],{"className":38301},[264,265,266,267],[170,38303,38305],{"className":38304},[230,267],[170,38306,7429],{"className":38307,"style":7504},[230,303,267],[170,38309,338],{"className":38310},[337],[170,38312,38314],{"className":38313},[247],[170,38315,38317],{"className":38316,"style":16499},[251],[170,38318],{},[170,38320],{"className":38321,"style":1027},[610],[170,38323,38325],{"className":38324},[1062,14344],[170,38326,38328,38418],{"className":38327},[242,243],[170,38329,38331,38415],{"className":38330},[247],[170,38332,38334,38354,38364],{"className":38333,"style":17255},[251],[170,38335,38336,38339],{"style":16449},[170,38337],{"className":38338,"style":15538},[259],[170,38340,38342],{"className":38341},[264,265,266,267],[170,38343,38345,38348,38351],{"className":38344},[230,267],[170,38346,888],{"className":38347},[230,303,267],[170,38349,926],{"className":38350},[615,267],[170,38352,194],{"className":38353},[230,267],[170,38355,38356,38359],{"style":15547},[170,38357],{"className":38358,"style":15538},[259],[170,38360,38361],{},[170,38362,1937],{"className":38363},[1062,2021,15556],[170,38365,38366,38369],{"style":17288},[170,38367],{"className":38368,"style":15538},[259],[170,38370,38372],{"className":38371},[264,265,266,267],[170,38373,38375],{"className":38374},[230,267],[170,38376,38378,38381],{"className":38377},[230,267],[170,38379,13348],{"className":38380,"style":3292},[230,303,267],[170,38382,38384],{"className":38383},[307],[170,38385,38387,38407],{"className":38386},[242,243],[170,38388,38390,38404],{"className":38389},[247],[170,38391,38393],{"className":38392,"style":4409},[251],[170,38394,38395,38398],{"style":17318},[170,38396],{"className":38397,"style":324},[259],[170,38399,38401],{"className":38400},[264,328,329,267],[170,38402,446],{"className":38403},[230,303,267],[170,38405,338],{"className":38406},[337],[170,38408,38410],{"className":38409},[247],[170,38411,38413],{"className":38412,"style":4431},[251],[170,38414],{},[170,38416,338],{"className":38417},[337],[170,38419,38421],{"className":38420},[247],[170,38422,38424],{"className":38423,"style":16499},[251],[170,38425],{},[170,38427],{"className":38428,"style":1027},[610],[170,38430,38432,38435],{"className":38431},[230],[170,38433,885],{"className":38434},[230,303],[170,38436,38438],{"className":38437},[307],[170,38439,38441,38513],{"className":38440},[242,243],[170,38442,38444,38510],{"className":38443},[247],[170,38445,38447,38490],{"className":38446,"style":24902},[251],[170,38448,38449,38452],{"style":17417},[170,38450],{"className":38451,"style":651},[259],[170,38453,38455],{"className":38454},[264,265,266,267],[170,38456,38458],{"className":38457},[230,267],[170,38459,38461,38464],{"className":38460},[230,267],[170,38462,14157],{"className":38463,"style":3317},[230,303,267],[170,38465,38467],{"className":38466},[307],[170,38468,38470],{"className":38469},[242],[170,38471,38473],{"className":38472},[247],[170,38474,38476],{"className":38475,"style":19250},[251],[170,38477,38478,38481],{"style":19253},[170,38479],{"className":38480,"style":324},[259],[170,38482,38484],{"className":38483},[264,328,329,267],[170,38485,38487],{"className":38486},[230,267],[170,38488,19090],{"className":38489},[230,267],[170,38491,38492,38495],{"style":16532},[170,38493],{"className":38494,"style":651},[259],[170,38496,38498],{"className":38497},[264,265,266,267],[170,38499,38501,38504,38507],{"className":38500},[230,267],[170,38502,27616],{"className":38503},[230,303,267],[170,38505,885],{"className":38506},[230,303,267],[170,38508,5714],{"className":38509,"style":6157},[230,303,267],[170,38511,338],{"className":38512},[337],[170,38514,38516],{"className":38515},[247],[170,38517,38519],{"className":38518,"style":17447},[251],[170,38520],{},[170,38522,909],{"className":38523},[234],[170,38525,38527,38530],{"className":38526},[230],[170,38528,12908],{"className":38529},[230,303],[170,38531,38533],{"className":38532},[307],[170,38534,38536,38567],{"className":38535},[242,243],[170,38537,38539,38564],{"className":38538},[247],[170,38540,38542,38553],{"className":38541,"style":17414},[251],[170,38543,38544,38547],{"style":17417},[170,38545],{"className":38546,"style":651},[259],[170,38548,38550],{"className":38549},[264,265,266,267],[170,38551,446],{"className":38552},[230,303,267],[170,38554,38555,38558],{"style":16532},[170,38556],{"className":38557,"style":651},[259],[170,38559,38561],{"className":38560},[264,265,266,267],[170,38562,888],{"className":38563},[230,303,267],[170,38565,338],{"className":38566},[337],[170,38568,38570],{"className":38569},[247],[170,38571,38573],{"className":38572,"style":17447},[251],[170,38574],{},[170,38576,914],{"className":38577},[1023],[170,38579],{"className":38580,"style":1027},[610],[170,38582,38584,38587],{"className":38583},[230],[170,38585,134],{"className":38586},[230,303],[170,38588,38590],{"className":38589},[307],[170,38591,38593,38624],{"className":38592},[242,243],[170,38594,38596,38621],{"className":38595},[247],[170,38597,38599,38610],{"className":38598,"style":17414},[251],[170,38600,38601,38604],{"style":17417},[170,38602],{"className":38603,"style":651},[259],[170,38605,38607],{"className":38606},[264,265,266,267],[170,38608,446],{"className":38609},[230,303,267],[170,38611,38612,38615],{"style":16532},[170,38613],{"className":38614,"style":651},[259],[170,38616,38618],{"className":38617},[264,265,266,267],[170,38619,888],{"className":38620},[230,303,267],[170,38622,338],{"className":38623},[337],[170,38625,38627],{"className":38626},[247],[170,38628,38630],{"className":38629,"style":17447},[251],[170,38631],{},[170,38633,923],{"className":38634},[424],[170,38636,38638,38641,39025],{"className":38637},[230],[170,38639],{"className":38640},[234,235],[170,38642,38644],{"className":38643},[189],[170,38645,38647,39017],{"className":38646},[242,243],[170,38648,38650,39014],{"className":38649},[247],[170,38651,38653,38844,38852],{"className":38652,"style":36409},[251],[170,38654,38655,38658],{"style":15925},[170,38656],{"className":38657,"style":260},[259],[170,38659,38661,38733,38736,38787,38790,38841],{"className":38660},[230],[170,38662,38664,38667],{"className":38663},[230],[170,38665,5711],{"className":38666,"style":3292},[230,303],[170,38668,38670],{"className":38669},[307],[170,38671,38673,38725],{"className":38672},[242,243],[170,38674,38676,38722],{"className":38675},[247],[170,38677,38679],{"className":38678,"style":1359},[251],[170,38680,38681,38684],{"style":7567},[170,38682],{"className":38683,"style":651},[259],[170,38685,38687],{"className":38686},[264,265,266,267],[170,38688,38690],{"className":38689},[230,267],[170,38691,38693,38696],{"className":38692},[230,267],[170,38694,14157],{"className":38695,"style":3317},[230,303,267],[170,38697,38699],{"className":38698},[307],[170,38700,38702],{"className":38701},[242],[170,38703,38705],{"className":38704},[247],[170,38706,38708],{"className":38707,"style":19250},[251],[170,38709,38710,38713],{"style":19253},[170,38711],{"className":38712,"style":324},[259],[170,38714,38716],{"className":38715},[264,328,329,267],[170,38717,38719],{"className":38718},[230,267],[170,38720,19090],{"className":38721},[230,267],[170,38723,338],{"className":38724},[337],[170,38726,38728],{"className":38727},[247],[170,38729,38731],{"className":38730,"style":1381},[251],[170,38732],{},[170,38734,909],{"className":38735},[234],[170,38737,38739,38742],{"className":38738},[230],[170,38740,134],{"className":38741},[230,303],[170,38743,38745],{"className":38744},[307],[170,38746,38748,38779],{"className":38747},[242,243],[170,38749,38751,38776],{"className":38750},[247],[170,38752,38754,38765],{"className":38753,"style":36511},[251],[170,38755,38756,38759],{"style":17417},[170,38757],{"className":38758,"style":651},[259],[170,38760,38762],{"className":38761},[264,265,266,267],[170,38763,446],{"className":38764},[230,303,267],[170,38766,38767,38770],{"style":16635},[170,38768],{"className":38769,"style":651},[259],[170,38771,38773],{"className":38772},[264,265,266,267],[170,38774,888],{"className":38775},[230,303,267],[170,38777,338],{"className":38778},[337],[170,38780,38782],{"className":38781},[247],[170,38783,38785],{"className":38784,"style":17447},[251],[170,38786],{},[170,38788,13047],{"className":38789},[230],[170,38791,38793,38796],{"className":38792},[230],[170,38794,12908],{"className":38795},[230,303],[170,38797,38799],{"className":38798},[307],[170,38800,38802,38833],{"className":38801},[242,243],[170,38803,38805,38830],{"className":38804},[247],[170,38806,38808,38819],{"className":38807,"style":36511},[251],[170,38809,38810,38813],{"style":17417},[170,38811],{"className":38812,"style":651},[259],[170,38814,38816],{"className":38815},[264,265,266,267],[170,38817,446],{"className":38818},[230,303,267],[170,38820,38821,38824],{"style":16635},[170,38822],{"className":38823,"style":651},[259],[170,38825,38827],{"className":38826},[264,265,266,267],[170,38828,888],{"className":38829},[230,303,267],[170,38831,338],{"className":38832},[337],[170,38834,38836],{"className":38835},[247],[170,38837,38839],{"className":38838,"style":17447},[251],[170,38840],{},[170,38842,923],{"className":38843},[424],[170,38845,38846,38849],{"style":385},[170,38847],{"className":38848,"style":260},[259],[170,38850],{"className":38851,"style":393},[392],[170,38853,38854,38857],{"style":15991},[170,38855],{"className":38856,"style":260},[259],[170,38858,38860,38863,38903,38906,38957,38960,39011],{"className":38859},[230],[170,38861,14486],{"className":38862},[230],[170,38864,38866,38869],{"className":38865},[230],[170,38867,5711],{"className":38868,"style":3292},[230,303],[170,38870,38872],{"className":38871},[307],[170,38873,38875,38895],{"className":38874},[242,243],[170,38876,38878,38892],{"className":38877},[247],[170,38879,38881],{"className":38880,"style":1359},[251],[170,38882,38883,38886],{"style":7567},[170,38884],{"className":38885,"style":651},[259],[170,38887,38889],{"className":38888},[264,265,266,267],[170,38890,14157],{"className":38891,"style":3317},[230,303,267],[170,38893,338],{"className":38894},[337],[170,38896,38898],{"className":38897},[247],[170,38899,38901],{"className":38900,"style":1381},[251],[170,38902],{},[170,38904,909],{"className":38905},[234],[170,38907,38909,38912],{"className":38908},[230],[170,38910,134],{"className":38911},[230,303],[170,38913,38915],{"className":38914},[307],[170,38916,38918,38949],{"className":38917},[242,243],[170,38919,38921,38946],{"className":38920},[247],[170,38922,38924,38935],{"className":38923,"style":18955},[251],[170,38925,38926,38929],{"style":17417},[170,38927],{"className":38928,"style":651},[259],[170,38930,38932],{"className":38931},[264,265,266,267],[170,38933,446],{"className":38934},[230,303,267],[170,38936,38937,38940],{"style":647},[170,38938],{"className":38939,"style":651},[259],[170,38941,38943],{"className":38942},[264,265,266,267],[170,38944,888],{"className":38945},[230,303,267],[170,38947,338],{"className":38948},[337],[170,38950,38952],{"className":38951},[247],[170,38953,38955],{"className":38954,"style":17447},[251],[170,38956],{},[170,38958,13047],{"className":38959},[230],[170,38961,38963,38966],{"className":38962},[230],[170,38964,12908],{"className":38965},[230,303],[170,38967,38969],{"className":38968},[307],[170,38970,38972,39003],{"className":38971},[242,243],[170,38973,38975,39000],{"className":38974},[247],[170,38976,38978,38989],{"className":38977,"style":18955},[251],[170,38979,38980,38983],{"style":17417},[170,38981],{"className":38982,"style":651},[259],[170,38984,38986],{"className":38985},[264,265,266,267],[170,38987,446],{"className":38988},[230,303,267],[170,38990,38991,38994],{"style":647},[170,38992],{"className":38993,"style":651},[259],[170,38995,38997],{"className":38996},[264,265,266,267],[170,38998,888],{"className":38999},[230,303,267],[170,39001,338],{"className":39002},[337],[170,39004,39006],{"className":39005},[247],[170,39007,39009],{"className":39008,"style":17447},[251],[170,39010],{},[170,39012,923],{"className":39013},[424],[170,39015,338],{"className":39016},[337],[170,39018,39020],{"className":39019},[247],[170,39021,39023],{"className":39022,"style":16056},[251],[170,39024],{},[170,39026],{"className":39027},[424,235],[170,39029,338],{"className":39030},[337],[170,39032,39034],{"className":39033},[247],[170,39035,39037],{"className":39036,"style":35316},[251],[170,39038],{},[14,39040,39041],{},"对应地，极大化目标时可用如下损失（最小化时取负号）：",[170,39043,39045],{"className":39044},[14456],[170,39046,39048,39216],{"className":39047},[173],[170,39049,39051],{"className":39050},[177],[179,39052,39053],{"xmlns":181,"display":9588},[183,39054,39055,39213],{},[186,39056,39057,39059,39061,39063,39065,39067,39069,39075,39089,39107,39125,39127,39135,39137,39145,39147],{},[202,39058,7923],{},[202,39060,903],{},[202,39062,12908],{},[202,39064,12908],{},[572,39066,926],{},[572,39068,18471],{},[189,39070,39071,39073],{},[192,39072,194],{},[202,39074,7429],{},[14789,39076,39077,39079,39087],{},[572,39078,1937],{},[186,39080,39081,39083,39085],{},[202,39082,446],{},[572,39084,926],{},[192,39086,194],{},[202,39088,7429],{},[14789,39090,39091,39093,39101],{},[572,39092,1937],{},[186,39094,39095,39097,39099],{},[202,39096,888],{},[572,39098,926],{},[192,39100,194],{},[199,39102,39103,39105],{},[202,39104,13348],{},[202,39106,446],{},[1933,39108,39109,39111,39117],{},[202,39110,885],{},[576,39112,39113,39115],{},[202,39114,14157],{},[572,39116,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[186,39118,39119,39121,39123],{},[202,39120,27616],{},[202,39122,885],{},[202,39124,5714],{},[572,39126,909],{"stretchy":908},[1933,39128,39129,39131,39133],{},[202,39130,12908],{},[202,39132,446],{},[202,39134,888],{},[572,39136,914],{"separator":217},[1933,39138,39139,39141,39143],{},[202,39140,134],{},[202,39142,446],{},[202,39144,888],{},[572,39146,923],{"stretchy":908},[189,39148,39149,39179],{},[186,39150,39151,39157,39159,39167,39169,39177],{},[199,39152,39153,39155],{},[202,39154,5711],{},[202,39156,14157],{},[572,39158,909],{"stretchy":908},[1933,39160,39161,39163,39165],{},[202,39162,134],{},[202,39164,446],{},[202,39166,888],{},[202,39168,13047],{"mathvariant":929},[1933,39170,39171,39173,39175],{},[202,39172,12908],{},[202,39174,446],{},[202,39176,888],{},[572,39178,923],{"stretchy":908},[186,39180,39181,39191,39193,39201,39203,39211],{},[199,39182,39183,39185],{},[202,39184,5711],{},[576,39186,39187,39189],{},[202,39188,14157],{},[572,39190,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[572,39192,909],{"stretchy":908},[1933,39194,39195,39197,39199],{},[202,39196,134],{},[202,39198,446],{},[202,39200,888],{},[202,39202,13047],{"mathvariant":929},[1933,39204,39205,39207,39209],{},[202,39206,12908],{},[202,39208,446],{},[202,39210,888],{},[572,39212,923],{"stretchy":908},[209,39214,39215],{"encoding":211},"Loss = -\\frac{1}{N} \\sum_{n=1}^{N} \\sum_{t=1}^{T_n} A_{\\theta'}^{GAE}(s_n^t, a_n^t) \\frac{P_\\theta(a_n^t | s_n^t)}{P_{\\theta'}(a_n^t | s_n^t)}",[170,39217,39219,39240],{"className":39218,"ariaHidden":217},[216],[170,39220,39222,39225,39228,39231,39234,39237],{"className":39221},[221],[170,39223],{"className":39224,"style":14109},[225],[170,39226,7923],{"className":39227},[230,303],[170,39229,18572],{"className":39230},[230,303],[170,39232],{"className":39233,"style":611},[610],[170,39235,926],{"className":39236},[615],[170,39238],{"className":39239,"style":611},[610],[170,39241,39243,39246,39249,39311,39314,39381,39384,39488,39491,39583,39586,39637,39640,39643,39694,39697],{"className":39242},[221],[170,39244],{"className":39245,"style":31279},[225],[170,39247,18471],{"className":39248},[230],[170,39250,39252,39255,39308],{"className":39251},[230],[170,39253],{"className":39254},[234,235],[170,39256,39258],{"className":39257},[189],[170,39259,39261,39300],{"className":39260},[242,243],[170,39262,39264,39297],{"className":39263},[247],[170,39265,39267,39278,39286],{"className":39266,"style":16385},[251],[170,39268,39269,39272],{"style":15925},[170,39270],{"className":39271,"style":260},[259],[170,39273,39275],{"className":39274},[230],[170,39276,7429],{"className":39277,"style":7504},[230,303],[170,39279,39280,39283],{"style":385},[170,39281],{"className":39282,"style":260},[259],[170,39284],{"className":39285,"style":393},[392],[170,39287,39288,39291],{"style":15991},[170,39289],{"className":39290,"style":260},[259],[170,39292,39294],{"className":39293},[230],[170,39295,194],{"className":39296},[230],[170,39298,338],{"className":39299},[337],[170,39301,39303],{"className":39302},[247],[170,39304,39306],{"className":39305,"style":16425},[251],[170,39307],{},[170,39309],{"className":39310},[424,235],[170,39312],{"className":39313,"style":1027},[610],[170,39315,39317],{"className":39316},[1062,14344],[170,39318,39320,39373],{"className":39319},[242,243],[170,39321,39323,39370],{"className":39322},[247],[170,39324,39326,39346,39356],{"className":39325,"style":16446},[251],[170,39327,39328,39331],{"style":16449},[170,39329],{"className":39330,"style":15538},[259],[170,39332,39334],{"className":39333},[264,265,266,267],[170,39335,39337,39340,39343],{"className":39336},[230,267],[170,39338,446],{"className":39339},[230,303,267],[170,39341,926],{"className":39342},[615,267],[170,39344,194],{"className":39345},[230,267],[170,39347,39348,39351],{"style":15547},[170,39349],{"className":39350,"style":15538},[259],[170,39352,39353],{},[170,39354,1937],{"className":39355},[1062,2021,15556],[170,39357,39358,39361],{"style":16480},[170,39359],{"className":39360,"style":15538},[259],[170,39362,39364],{"className":39363},[264,265,266,267],[170,39365,39367],{"className":39366},[230,267],[170,39368,7429],{"className":39369,"style":7504},[230,303,267],[170,39371,338],{"className":39372},[337],[170,39374,39376],{"className":39375},[247],[170,39377,39379],{"className":39378,"style":16499},[251],[170,39380],{},[170,39382],{"className":39383,"style":1027},[610],[170,39385,39387],{"className":39386},[1062,14344],[170,39388,39390,39480],{"className":39389},[242,243],[170,39391,39393,39477],{"className":39392},[247],[170,39394,39396,39416,39426],{"className":39395,"style":17255},[251],[170,39397,39398,39401],{"style":16449},[170,39399],{"className":39400,"style":15538},[259],[170,39402,39404],{"className":39403},[264,265,266,267],[170,39405,39407,39410,39413],{"className":39406},[230,267],[170,39408,888],{"className":39409},[230,303,267],[170,39411,926],{"className":39412},[615,267],[170,39414,194],{"className":39415},[230,267],[170,39417,39418,39421],{"style":15547},[170,39419],{"className":39420,"style":15538},[259],[170,39422,39423],{},[170,39424,1937],{"className":39425},[1062,2021,15556],[170,39427,39428,39431],{"style":17288},[170,39429],{"className":39430,"style":15538},[259],[170,39432,39434],{"className":39433},[264,265,266,267],[170,39435,39437],{"className":39436},[230,267],[170,39438,39440,39443],{"className":39439},[230,267],[170,39441,13348],{"className":39442,"style":3292},[230,303,267],[170,39444,39446],{"className":39445},[307],[170,39447,39449,39469],{"className":39448},[242,243],[170,39450,39452,39466],{"className":39451},[247],[170,39453,39455],{"className":39454,"style":4409},[251],[170,39456,39457,39460],{"style":17318},[170,39458],{"className":39459,"style":324},[259],[170,39461,39463],{"className":39462},[264,328,329,267],[170,39464,446],{"className":39465},[230,303,267],[170,39467,338],{"className":39468},[337],[170,39470,39472],{"className":39471},[247],[170,39473,39475],{"className":39474,"style":4431},[251],[170,39476],{},[170,39478,338],{"className":39479},[337],[170,39481,39483],{"className":39482},[247],[170,39484,39486],{"className":39485,"style":16499},[251],[170,39487],{},[170,39489],{"className":39490,"style":1027},[610],[170,39492,39494,39497],{"className":39493},[230],[170,39495,885],{"className":39496},[230,303],[170,39498,39500],{"className":39499},[307],[170,39501,39503,39575],{"className":39502},[242,243],[170,39504,39506,39572],{"className":39505},[247],[170,39507,39509,39552],{"className":39508,"style":24902},[251],[170,39510,39511,39514],{"style":17417},[170,39512],{"className":39513,"style":651},[259],[170,39515,39517],{"className":39516},[264,265,266,267],[170,39518,39520],{"className":39519},[230,267],[170,39521,39523,39526],{"className":39522},[230,267],[170,39524,14157],{"className":39525,"style":3317},[230,303,267],[170,39527,39529],{"className":39528},[307],[170,39530,39532],{"className":39531},[242],[170,39533,39535],{"className":39534},[247],[170,39536,39538],{"className":39537,"style":19250},[251],[170,39539,39540,39543],{"style":19253},[170,39541],{"className":39542,"style":324},[259],[170,39544,39546],{"className":39545},[264,328,329,267],[170,39547,39549],{"className":39548},[230,267],[170,39550,19090],{"className":39551},[230,267],[170,39553,39554,39557],{"style":16532},[170,39555],{"className":39556,"style":651},[259],[170,39558,39560],{"className":39559},[264,265,266,267],[170,39561,39563,39566,39569],{"className":39562},[230,267],[170,39564,27616],{"className":39565},[230,303,267],[170,39567,885],{"className":39568},[230,303,267],[170,39570,5714],{"className":39571,"style":6157},[230,303,267],[170,39573,338],{"className":39574},[337],[170,39576,39578],{"className":39577},[247],[170,39579,39581],{"className":39580,"style":17447},[251],[170,39582],{},[170,39584,909],{"className":39585},[234],[170,39587,39589,39592],{"className":39588},[230],[170,39590,12908],{"className":39591},[230,303],[170,39593,39595],{"className":39594},[307],[170,39596,39598,39629],{"className":39597},[242,243],[170,39599,39601,39626],{"className":39600},[247],[170,39602,39604,39615],{"className":39603,"style":17414},[251],[170,39605,39606,39609],{"style":17417},[170,39607],{"className":39608,"style":651},[259],[170,39610,39612],{"className":39611},[264,265,266,267],[170,39613,446],{"className":39614},[230,303,267],[170,39616,39617,39620],{"style":16532},[170,39618],{"className":39619,"style":651},[259],[170,39621,39623],{"className":39622},[264,265,266,267],[170,39624,888],{"className":39625},[230,303,267],[170,39627,338],{"className":39628},[337],[170,39630,39632],{"className":39631},[247],[170,39633,39635],{"className":39634,"style":17447},[251],[170,39636],{},[170,39638,914],{"className":39639},[1023],[170,39641],{"className":39642,"style":1027},[610],[170,39644,39646,39649],{"className":39645},[230],[170,39647,134],{"className":39648},[230,303],[170,39650,39652],{"className":39651},[307],[170,39653,39655,39686],{"className":39654},[242,243],[170,39656,39658,39683],{"className":39657},[247],[170,39659,39661,39672],{"className":39660,"style":17414},[251],[170,39662,39663,39666],{"style":17417},[170,39664],{"className":39665,"style":651},[259],[170,39667,39669],{"className":39668},[264,265,266,267],[170,39670,446],{"className":39671},[230,303,267],[170,39673,39674,39677],{"style":16532},[170,39675],{"className":39676,"style":651},[259],[170,39678,39680],{"className":39679},[264,265,266,267],[170,39681,888],{"className":39682},[230,303,267],[170,39684,338],{"className":39685},[337],[170,39687,39689],{"className":39688},[247],[170,39690,39692],{"className":39691,"style":17447},[251],[170,39693],{},[170,39695,923],{"className":39696},[424],[170,39698,39700,39703,40084],{"className":39699},[230],[170,39701],{"className":39702},[234,235],[170,39704,39706],{"className":39705},[189],[170,39707,39709,40076],{"className":39708},[242,243],[170,39710,39712,40073],{"className":39711},[247],[170,39713,39715,39906,39914],{"className":39714,"style":36409},[251],[170,39716,39717,39720],{"style":15925},[170,39718],{"className":39719,"style":260},[259],[170,39721,39723,39795,39798,39849,39852,39903],{"className":39722},[230],[170,39724,39726,39729],{"className":39725},[230],[170,39727,5711],{"className":39728,"style":3292},[230,303],[170,39730,39732],{"className":39731},[307],[170,39733,39735,39787],{"className":39734},[242,243],[170,39736,39738,39784],{"className":39737},[247],[170,39739,39741],{"className":39740,"style":1359},[251],[170,39742,39743,39746],{"style":7567},[170,39744],{"className":39745,"style":651},[259],[170,39747,39749],{"className":39748},[264,265,266,267],[170,39750,39752],{"className":39751},[230,267],[170,39753,39755,39758],{"className":39754},[230,267],[170,39756,14157],{"className":39757,"style":3317},[230,303,267],[170,39759,39761],{"className":39760},[307],[170,39762,39764],{"className":39763},[242],[170,39765,39767],{"className":39766},[247],[170,39768,39770],{"className":39769,"style":19250},[251],[170,39771,39772,39775],{"style":19253},[170,39773],{"className":39774,"style":324},[259],[170,39776,39778],{"className":39777},[264,328,329,267],[170,39779,39781],{"className":39780},[230,267],[170,39782,19090],{"className":39783},[230,267],[170,39785,338],{"className":39786},[337],[170,39788,39790],{"className":39789},[247],[170,39791,39793],{"className":39792,"style":1381},[251],[170,39794],{},[170,39796,909],{"className":39797},[234],[170,39799,39801,39804],{"className":39800},[230],[170,39802,134],{"className":39803},[230,303],[170,39805,39807],{"className":39806},[307],[170,39808,39810,39841],{"className":39809},[242,243],[170,39811,39813,39838],{"className":39812},[247],[170,39814,39816,39827],{"className":39815,"style":36511},[251],[170,39817,39818,39821],{"style":17417},[170,39819],{"className":39820,"style":651},[259],[170,39822,39824],{"className":39823},[264,265,266,267],[170,39825,446],{"className":39826},[230,303,267],[170,39828,39829,39832],{"style":16635},[170,39830],{"className":39831,"style":651},[259],[170,39833,39835],{"className":39834},[264,265,266,267],[170,39836,888],{"className":39837},[230,303,267],[170,39839,338],{"className":39840},[337],[170,39842,39844],{"className":39843},[247],[170,39845,39847],{"className":39846,"style":17447},[251],[170,39848],{},[170,39850,13047],{"className":39851},[230],[170,39853,39855,39858],{"className":39854},[230],[170,39856,12908],{"className":39857},[230,303],[170,39859,39861],{"className":39860},[307],[170,39862,39864,39895],{"className":39863},[242,243],[170,39865,39867,39892],{"className":39866},[247],[170,39868,39870,39881],{"className":39869,"style":36511},[251],[170,39871,39872,39875],{"style":17417},[170,39873],{"className":39874,"style":651},[259],[170,39876,39878],{"className":39877},[264,265,266,267],[170,39879,446],{"className":39880},[230,303,267],[170,39882,39883,39886],{"style":16635},[170,39884],{"className":39885,"style":651},[259],[170,39887,39889],{"className":39888},[264,265,266,267],[170,39890,888],{"className":39891},[230,303,267],[170,39893,338],{"className":39894},[337],[170,39896,39898],{"className":39897},[247],[170,39899,39901],{"className":39900,"style":17447},[251],[170,39902],{},[170,39904,923],{"className":39905},[424],[170,39907,39908,39911],{"style":385},[170,39909],{"className":39910,"style":260},[259],[170,39912],{"className":39913,"style":393},[392],[170,39915,39916,39919],{"style":15991},[170,39917],{"className":39918,"style":260},[259],[170,39920,39922,39962,39965,40016,40019,40070],{"className":39921},[230],[170,39923,39925,39928],{"className":39924},[230],[170,39926,5711],{"className":39927,"style":3292},[230,303],[170,39929,39931],{"className":39930},[307],[170,39932,39934,39954],{"className":39933},[242,243],[170,39935,39937,39951],{"className":39936},[247],[170,39938,39940],{"className":39939,"style":1359},[251],[170,39941,39942,39945],{"style":7567},[170,39943],{"className":39944,"style":651},[259],[170,39946,39948],{"className":39947},[264,265,266,267],[170,39949,14157],{"className":39950,"style":3317},[230,303,267],[170,39952,338],{"className":39953},[337],[170,39955,39957],{"className":39956},[247],[170,39958,39960],{"className":39959,"style":1381},[251],[170,39961],{},[170,39963,909],{"className":39964},[234],[170,39966,39968,39971],{"className":39967},[230],[170,39969,134],{"className":39970},[230,303],[170,39972,39974],{"className":39973},[307],[170,39975,39977,40008],{"className":39976},[242,243],[170,39978,39980,40005],{"className":39979},[247],[170,39981,39983,39994],{"className":39982,"style":18955},[251],[170,39984,39985,39988],{"style":17417},[170,39986],{"className":39987,"style":651},[259],[170,39989,39991],{"className":39990},[264,265,266,267],[170,39992,446],{"className":39993},[230,303,267],[170,39995,39996,39999],{"style":647},[170,39997],{"className":39998,"style":651},[259],[170,40000,40002],{"className":40001},[264,265,266,267],[170,40003,888],{"className":40004},[230,303,267],[170,40006,338],{"className":40007},[337],[170,40009,40011],{"className":40010},[247],[170,40012,40014],{"className":40013,"style":17447},[251],[170,40015],{},[170,40017,13047],{"className":40018},[230],[170,40020,40022,40025],{"className":40021},[230],[170,40023,12908],{"className":40024},[230,303],[170,40026,40028],{"className":40027},[307],[170,40029,40031,40062],{"className":40030},[242,243],[170,40032,40034,40059],{"className":40033},[247],[170,40035,40037,40048],{"className":40036,"style":18955},[251],[170,40038,40039,40042],{"style":17417},[170,40040],{"className":40041,"style":651},[259],[170,40043,40045],{"className":40044},[264,265,266,267],[170,40046,446],{"className":40047},[230,303,267],[170,40049,40050,40053],{"style":647},[170,40051],{"className":40052,"style":651},[259],[170,40054,40056],{"className":40055},[264,265,266,267],[170,40057,888],{"className":40058},[230,303,267],[170,40060,338],{"className":40061},[337],[170,40063,40065],{"className":40064},[247],[170,40066,40068],{"className":40067,"style":17447},[251],[170,40069],{},[170,40071,923],{"className":40072},[424],[170,40074,338],{"className":40075},[337],[170,40077,40079],{"className":40078},[247],[170,40080,40082],{"className":40081,"style":16056},[251],[170,40083],{},[170,40085],{"className":40086},[424,235],[14,40088,40089],{},"同时，你学习的目标模型不能和我们的差距过大，否则很难学到有用的经验和教训。",[40091,40092,40094],"h4",{"id":40093},"ppo-penalty在-surrogate-上加-kl-惩罚","PPO-Penalty：在 surrogate 上加 KL 惩罚",[14,40096,40097,40098,40130],{},"如何给我们训练的策略和参考的策略增加相差不能过大这个约束呢？那就是加上 KL 散度的约束。KL 散度就是衡量两个分布差异大小的指标。两个分布差异越小，KL散度越小，分布差异越大，KL散度越大。通过因子 ",[170,40099,40101,40116],{"className":40100},[173],[170,40102,40104],{"className":40103},[177],[179,40105,40106],{"xmlns":181},[183,40107,40108,40113],{},[186,40109,40110],{},[202,40111,40112],{},"β",[209,40114,40115],{"encoding":211},"\\beta",[170,40117,40119],{"className":40118,"ariaHidden":217},[216],[170,40120,40122,40126],{"className":40121},[221],[170,40123],{"className":40124,"style":40125},[225],"height:0.8889em;vertical-align:-0.1944em;",[170,40127,40112],{"className":40128,"style":40129},[230,303],"margin-right:0.05278em;"," 来调整 KL 散度影响的大小。",[170,40132,40134],{"className":40133},[14456],[170,40135,40137,40344],{"className":40136},[173],[170,40138,40140],{"className":40139},[177],[179,40141,40142],{"xmlns":181,"display":9588},[183,40143,40144,40341],{},[186,40145,40146,40148,40150,40152,40159,40161,40163,40169,40183,40201,40219,40221,40229,40231,40239,40241,40307,40309,40311,40313,40319,40321,40327,40329,40339],{},[202,40147,7923],{},[202,40149,903],{},[202,40151,12908],{},[199,40153,40154,40156],{},[202,40155,12908],{},[3384,40157,40158],{},"ppo",[572,40160,926],{},[572,40162,18471],{},[189,40164,40165,40167],{},[192,40166,194],{},[202,40168,7429],{},[14789,40170,40171,40173,40181],{},[572,40172,1937],{},[186,40174,40175,40177,40179],{},[202,40176,446],{},[572,40178,926],{},[192,40180,194],{},[202,40182,7429],{},[14789,40184,40185,40187,40195],{},[572,40186,1937],{},[186,40188,40189,40191,40193],{},[202,40190,888],{},[572,40192,926],{},[192,40194,194],{},[199,40196,40197,40199],{},[202,40198,13348],{},[202,40200,446],{},[1933,40202,40203,40205,40211],{},[202,40204,885],{},[576,40206,40207,40209],{},[202,40208,14157],{},[572,40210,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[186,40212,40213,40215,40217],{},[202,40214,27616],{},[202,40216,885],{},[202,40218,5714],{},[572,40220,909],{"stretchy":908},[1933,40222,40223,40225,40227],{},[202,40224,12908],{},[202,40226,446],{},[202,40228,888],{},[572,40230,914],{"separator":217},[1933,40232,40233,40235,40237],{},[202,40234,134],{},[202,40236,446],{},[202,40238,888],{},[572,40240,923],{"stretchy":908},[189,40242,40243,40273],{},[186,40244,40245,40251,40253,40261,40263,40271],{},[199,40246,40247,40249],{},[202,40248,5711],{},[202,40250,14157],{},[572,40252,909],{"stretchy":908},[1933,40254,40255,40257,40259],{},[202,40256,134],{},[202,40258,446],{},[202,40260,888],{},[202,40262,13047],{"mathvariant":929},[1933,40264,40265,40267,40269],{},[202,40266,12908],{},[202,40268,446],{},[202,40270,888],{},[572,40272,923],{"stretchy":908},[186,40274,40275,40285,40287,40295,40297,40305],{},[199,40276,40277,40279],{},[202,40278,5711],{},[576,40280,40281,40283],{},[202,40282,14157],{},[572,40284,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[572,40286,909],{"stretchy":908},[1933,40288,40289,40291,40293],{},[202,40290,134],{},[202,40292,446],{},[202,40294,888],{},[202,40296,13047],{"mathvariant":929},[1933,40298,40299,40301,40303],{},[202,40300,12908],{},[202,40302,446],{},[202,40304,888],{},[572,40306,923],{"stretchy":908},[572,40308,5884],{},[202,40310,40112],{},[3384,40312,27701],{},[186,40314,40315,40317],{},[202,40316,683],{"mathvariant":929},[202,40318,7923],{"mathvariant":929},[572,40320,909],{"stretchy":908},[199,40322,40323,40325],{},[202,40324,5711],{},[202,40326,14157],{},[572,40328,914],{"separator":217},[199,40330,40331,40333],{},[202,40332,5711],{},[576,40334,40335,40337],{},[202,40336,14157],{},[572,40338,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[572,40340,923],{"stretchy":908},[209,40342,40343],{"encoding":211},"Loss_{\\text{ppo}} = -\\frac{1}{N} \\sum_{n=1}^{N} \\sum_{t=1}^{T_n} A_{\\theta'}^{GAE}(s_n^t, a_n^t) \\frac{P_{\\theta}(a_n^t|s_n^t)}{P_{\\theta'}(a_n^t|s_n^t)} + \\beta\\,\\mathrm{KL}(P_{\\theta}, P_{\\theta'})",[170,40345,40347,40417,41276],{"className":40346,"ariaHidden":217},[216],[170,40348,40350,40354,40357,40361,40408,40411,40414],{"className":40349},[221],[170,40351],{"className":40352,"style":40353},[225],"height:0.9694em;vertical-align:-0.2861em;",[170,40355,7923],{"className":40356},[230,303],[170,40358,40360],{"className":40359},[230,303],"os",[170,40362,40364,40367],{"className":40363},[230],[170,40365,12908],{"className":40366},[230,303],[170,40368,40370],{"className":40369},[307],[170,40371,40373,40399],{"className":40372},[242,243],[170,40374,40376,40396],{"className":40375},[247],[170,40377,40379],{"className":40378,"style":1433},[251],[170,40380,40381,40384],{"style":1362},[170,40382],{"className":40383,"style":651},[259],[170,40385,40387],{"className":40386},[264,265,266,267],[170,40388,40390],{"className":40389},[230,267],[170,40391,40393],{"className":40392},[230,11961,267],[170,40394,40158],{"className":40395},[230,267],[170,40397,338],{"className":40398},[337],[170,40400,40402],{"className":40401},[247],[170,40403,40406],{"className":40404,"style":40405},[251],"height:0.2861em;",[170,40407],{},[170,40409],{"className":40410,"style":611},[610],[170,40412,926],{"className":40413},[615],[170,40415],{"className":40416,"style":611},[610],[170,40418,40420,40423,40426,40488,40491,40558,40561,40665,40668,40760,40763,40814,40817,40820,40871,40874,41267,41270,41273],{"className":40419},[221],[170,40421],{"className":40422,"style":31279},[225],[170,40424,18471],{"className":40425},[230],[170,40427,40429,40432,40485],{"className":40428},[230],[170,40430],{"className":40431},[234,235],[170,40433,40435],{"className":40434},[189],[170,40436,40438,40477],{"className":40437},[242,243],[170,40439,40441,40474],{"className":40440},[247],[170,40442,40444,40455,40463],{"className":40443,"style":16385},[251],[170,40445,40446,40449],{"style":15925},[170,40447],{"className":40448,"style":260},[259],[170,40450,40452],{"className":40451},[230],[170,40453,7429],{"className":40454,"style":7504},[230,303],[170,40456,40457,40460],{"style":385},[170,40458],{"className":40459,"style":260},[259],[170,40461],{"className":40462,"style":393},[392],[170,40464,40465,40468],{"style":15991},[170,40466],{"className":40467,"style":260},[259],[170,40469,40471],{"className":40470},[230],[170,40472,194],{"className":40473},[230],[170,40475,338],{"className":40476},[337],[170,40478,40480],{"className":40479},[247],[170,40481,40483],{"className":40482,"style":16425},[251],[170,40484],{},[170,40486],{"className":40487},[424,235],[170,40489],{"className":40490,"style":1027},[610],[170,40492,40494],{"className":40493},[1062,14344],[170,40495,40497,40550],{"className":40496},[242,243],[170,40498,40500,40547],{"className":40499},[247],[170,40501,40503,40523,40533],{"className":40502,"style":16446},[251],[170,40504,40505,40508],{"style":16449},[170,40506],{"className":40507,"style":15538},[259],[170,40509,40511],{"className":40510},[264,265,266,267],[170,40512,40514,40517,40520],{"className":40513},[230,267],[170,40515,446],{"className":40516},[230,303,267],[170,40518,926],{"className":40519},[615,267],[170,40521,194],{"className":40522},[230,267],[170,40524,40525,40528],{"style":15547},[170,40526],{"className":40527,"style":15538},[259],[170,40529,40530],{},[170,40531,1937],{"className":40532},[1062,2021,15556],[170,40534,40535,40538],{"style":16480},[170,40536],{"className":40537,"style":15538},[259],[170,40539,40541],{"className":40540},[264,265,266,267],[170,40542,40544],{"className":40543},[230,267],[170,40545,7429],{"className":40546,"style":7504},[230,303,267],[170,40548,338],{"className":40549},[337],[170,40551,40553],{"className":40552},[247],[170,40554,40556],{"className":40555,"style":16499},[251],[170,40557],{},[170,40559],{"className":40560,"style":1027},[610],[170,40562,40564],{"className":40563},[1062,14344],[170,40565,40567,40657],{"className":40566},[242,243],[170,40568,40570,40654],{"className":40569},[247],[170,40571,40573,40593,40603],{"className":40572,"style":17255},[251],[170,40574,40575,40578],{"style":16449},[170,40576],{"className":40577,"style":15538},[259],[170,40579,40581],{"className":40580},[264,265,266,267],[170,40582,40584,40587,40590],{"className":40583},[230,267],[170,40585,888],{"className":40586},[230,303,267],[170,40588,926],{"className":40589},[615,267],[170,40591,194],{"className":40592},[230,267],[170,40594,40595,40598],{"style":15547},[170,40596],{"className":40597,"style":15538},[259],[170,40599,40600],{},[170,40601,1937],{"className":40602},[1062,2021,15556],[170,40604,40605,40608],{"style":17288},[170,40606],{"className":40607,"style":15538},[259],[170,40609,40611],{"className":40610},[264,265,266,267],[170,40612,40614],{"className":40613},[230,267],[170,40615,40617,40620],{"className":40616},[230,267],[170,40618,13348],{"className":40619,"style":3292},[230,303,267],[170,40621,40623],{"className":40622},[307],[170,40624,40626,40646],{"className":40625},[242,243],[170,40627,40629,40643],{"className":40628},[247],[170,40630,40632],{"className":40631,"style":4409},[251],[170,40633,40634,40637],{"style":17318},[170,40635],{"className":40636,"style":324},[259],[170,40638,40640],{"className":40639},[264,328,329,267],[170,40641,446],{"className":40642},[230,303,267],[170,40644,338],{"className":40645},[337],[170,40647,40649],{"className":40648},[247],[170,40650,40652],{"className":40651,"style":4431},[251],[170,40653],{},[170,40655,338],{"className":40656},[337],[170,40658,40660],{"className":40659},[247],[170,40661,40663],{"className":40662,"style":16499},[251],[170,40664],{},[170,40666],{"className":40667,"style":1027},[610],[170,40669,40671,40674],{"className":40670},[230],[170,40672,885],{"className":40673},[230,303],[170,40675,40677],{"className":40676},[307],[170,40678,40680,40752],{"className":40679},[242,243],[170,40681,40683,40749],{"className":40682},[247],[170,40684,40686,40729],{"className":40685,"style":24902},[251],[170,40687,40688,40691],{"style":17417},[170,40689],{"className":40690,"style":651},[259],[170,40692,40694],{"className":40693},[264,265,266,267],[170,40695,40697],{"className":40696},[230,267],[170,40698,40700,40703],{"className":40699},[230,267],[170,40701,14157],{"className":40702,"style":3317},[230,303,267],[170,40704,40706],{"className":40705},[307],[170,40707,40709],{"className":40708},[242],[170,40710,40712],{"className":40711},[247],[170,40713,40715],{"className":40714,"style":19250},[251],[170,40716,40717,40720],{"style":19253},[170,40718],{"className":40719,"style":324},[259],[170,40721,40723],{"className":40722},[264,328,329,267],[170,40724,40726],{"className":40725},[230,267],[170,40727,19090],{"className":40728},[230,267],[170,40730,40731,40734],{"style":16532},[170,40732],{"className":40733,"style":651},[259],[170,40735,40737],{"className":40736},[264,265,266,267],[170,40738,40740,40743,40746],{"className":40739},[230,267],[170,40741,27616],{"className":40742},[230,303,267],[170,40744,885],{"className":40745},[230,303,267],[170,40747,5714],{"className":40748,"style":6157},[230,303,267],[170,40750,338],{"className":40751},[337],[170,40753,40755],{"className":40754},[247],[170,40756,40758],{"className":40757,"style":17447},[251],[170,40759],{},[170,40761,909],{"className":40762},[234],[170,40764,40766,40769],{"className":40765},[230],[170,40767,12908],{"className":40768},[230,303],[170,40770,40772],{"className":40771},[307],[170,40773,40775,40806],{"className":40774},[242,243],[170,40776,40778,40803],{"className":40777},[247],[170,40779,40781,40792],{"className":40780,"style":17414},[251],[170,40782,40783,40786],{"style":17417},[170,40784],{"className":40785,"style":651},[259],[170,40787,40789],{"className":40788},[264,265,266,267],[170,40790,446],{"className":40791},[230,303,267],[170,40793,40794,40797],{"style":16532},[170,40795],{"className":40796,"style":651},[259],[170,40798,40800],{"className":40799},[264,265,266,267],[170,40801,888],{"className":40802},[230,303,267],[170,40804,338],{"className":40805},[337],[170,40807,40809],{"className":40808},[247],[170,40810,40812],{"className":40811,"style":17447},[251],[170,40813],{},[170,40815,914],{"className":40816},[1023],[170,40818],{"className":40819,"style":1027},[610],[170,40821,40823,40826],{"className":40822},[230],[170,40824,134],{"className":40825},[230,303],[170,40827,40829],{"className":40828},[307],[170,40830,40832,40863],{"className":40831},[242,243],[170,40833,40835,40860],{"className":40834},[247],[170,40836,40838,40849],{"className":40837,"style":17414},[251],[170,40839,40840,40843],{"style":17417},[170,40841],{"className":40842,"style":651},[259],[170,40844,40846],{"className":40845},[264,265,266,267],[170,40847,446],{"className":40848},[230,303,267],[170,40850,40851,40854],{"style":16532},[170,40852],{"className":40853,"style":651},[259],[170,40855,40857],{"className":40856},[264,265,266,267],[170,40858,888],{"className":40859},[230,303,267],[170,40861,338],{"className":40862},[337],[170,40864,40866],{"className":40865},[247],[170,40867,40869],{"className":40868,"style":17447},[251],[170,40870],{},[170,40872,923],{"className":40873},[424],[170,40875,40877,40880,41264],{"className":40876},[230],[170,40878],{"className":40879},[234,235],[170,40881,40883],{"className":40882},[189],[170,40884,40886,41256],{"className":40885},[242,243],[170,40887,40889,41253],{"className":40888},[247],[170,40890,40892,41083,41091],{"className":40891,"style":36409},[251],[170,40893,40894,40897],{"style":15925},[170,40895],{"className":40896,"style":260},[259],[170,40898,40900,40972,40975,41026,41029,41080],{"className":40899},[230],[170,40901,40903,40906],{"className":40902},[230],[170,40904,5711],{"className":40905,"style":3292},[230,303],[170,40907,40909],{"className":40908},[307],[170,40910,40912,40964],{"className":40911},[242,243],[170,40913,40915,40961],{"className":40914},[247],[170,40916,40918],{"className":40917,"style":1359},[251],[170,40919,40920,40923],{"style":7567},[170,40921],{"className":40922,"style":651},[259],[170,40924,40926],{"className":40925},[264,265,266,267],[170,40927,40929],{"className":40928},[230,267],[170,40930,40932,40935],{"className":40931},[230,267],[170,40933,14157],{"className":40934,"style":3317},[230,303,267],[170,40936,40938],{"className":40937},[307],[170,40939,40941],{"className":40940},[242],[170,40942,40944],{"className":40943},[247],[170,40945,40947],{"className":40946,"style":19250},[251],[170,40948,40949,40952],{"style":19253},[170,40950],{"className":40951,"style":324},[259],[170,40953,40955],{"className":40954},[264,328,329,267],[170,40956,40958],{"className":40957},[230,267],[170,40959,19090],{"className":40960},[230,267],[170,40962,338],{"className":40963},[337],[170,40965,40967],{"className":40966},[247],[170,40968,40970],{"className":40969,"style":1381},[251],[170,40971],{},[170,40973,909],{"className":40974},[234],[170,40976,40978,40981],{"className":40977},[230],[170,40979,134],{"className":40980},[230,303],[170,40982,40984],{"className":40983},[307],[170,40985,40987,41018],{"className":40986},[242,243],[170,40988,40990,41015],{"className":40989},[247],[170,40991,40993,41004],{"className":40992,"style":36511},[251],[170,40994,40995,40998],{"style":17417},[170,40996],{"className":40997,"style":651},[259],[170,40999,41001],{"className":41000},[264,265,266,267],[170,41002,446],{"className":41003},[230,303,267],[170,41005,41006,41009],{"style":16635},[170,41007],{"className":41008,"style":651},[259],[170,41010,41012],{"className":41011},[264,265,266,267],[170,41013,888],{"className":41014},[230,303,267],[170,41016,338],{"className":41017},[337],[170,41019,41021],{"className":41020},[247],[170,41022,41024],{"className":41023,"style":17447},[251],[170,41025],{},[170,41027,13047],{"className":41028},[230],[170,41030,41032,41035],{"className":41031},[230],[170,41033,12908],{"className":41034},[230,303],[170,41036,41038],{"className":41037},[307],[170,41039,41041,41072],{"className":41040},[242,243],[170,41042,41044,41069],{"className":41043},[247],[170,41045,41047,41058],{"className":41046,"style":36511},[251],[170,41048,41049,41052],{"style":17417},[170,41050],{"className":41051,"style":651},[259],[170,41053,41055],{"className":41054},[264,265,266,267],[170,41056,446],{"className":41057},[230,303,267],[170,41059,41060,41063],{"style":16635},[170,41061],{"className":41062,"style":651},[259],[170,41064,41066],{"className":41065},[264,265,266,267],[170,41067,888],{"className":41068},[230,303,267],[170,41070,338],{"className":41071},[337],[170,41073,41075],{"className":41074},[247],[170,41076,41078],{"className":41077,"style":17447},[251],[170,41079],{},[170,41081,923],{"className":41082},[424],[170,41084,41085,41088],{"style":385},[170,41086],{"className":41087,"style":260},[259],[170,41089],{"className":41090,"style":393},[392],[170,41092,41093,41096],{"style":15991},[170,41094],{"className":41095,"style":260},[259],[170,41097,41099,41142,41145,41196,41199,41250],{"className":41098},[230],[170,41100,41102,41105],{"className":41101},[230],[170,41103,5711],{"className":41104,"style":3292},[230,303],[170,41106,41108],{"className":41107},[307],[170,41109,41111,41134],{"className":41110},[242,243],[170,41112,41114,41131],{"className":41113},[247],[170,41115,41117],{"className":41116,"style":1359},[251],[170,41118,41119,41122],{"style":7567},[170,41120],{"className":41121,"style":651},[259],[170,41123,41125],{"className":41124},[264,265,266,267],[170,41126,41128],{"className":41127},[230,267],[170,41129,14157],{"className":41130,"style":3317},[230,303,267],[170,41132,338],{"className":41133},[337],[170,41135,41137],{"className":41136},[247],[170,41138,41140],{"className":41139,"style":1381},[251],[170,41141],{},[170,41143,909],{"className":41144},[234],[170,41146,41148,41151],{"className":41147},[230],[170,41149,134],{"className":41150},[230,303],[170,41152,41154],{"className":41153},[307],[170,41155,41157,41188],{"className":41156},[242,243],[170,41158,41160,41185],{"className":41159},[247],[170,41161,41163,41174],{"className":41162,"style":18955},[251],[170,41164,41165,41168],{"style":17417},[170,41166],{"className":41167,"style":651},[259],[170,41169,41171],{"className":41170},[264,265,266,267],[170,41172,446],{"className":41173},[230,303,267],[170,41175,41176,41179],{"style":647},[170,41177],{"className":41178,"style":651},[259],[170,41180,41182],{"className":41181},[264,265,266,267],[170,41183,888],{"className":41184},[230,303,267],[170,41186,338],{"className":41187},[337],[170,41189,41191],{"className":41190},[247],[170,41192,41194],{"className":41193,"style":17447},[251],[170,41195],{},[170,41197,13047],{"className":41198},[230],[170,41200,41202,41205],{"className":41201},[230],[170,41203,12908],{"className":41204},[230,303],[170,41206,41208],{"className":41207},[307],[170,41209,41211,41242],{"className":41210},[242,243],[170,41212,41214,41239],{"className":41213},[247],[170,41215,41217,41228],{"className":41216,"style":18955},[251],[170,41218,41219,41222],{"style":17417},[170,41220],{"className":41221,"style":651},[259],[170,41223,41225],{"className":41224},[264,265,266,267],[170,41226,446],{"className":41227},[230,303,267],[170,41229,41230,41233],{"style":647},[170,41231],{"className":41232,"style":651},[259],[170,41234,41236],{"className":41235},[264,265,266,267],[170,41237,888],{"className":41238},[230,303,267],[170,41240,338],{"className":41241},[337],[170,41243,41245],{"className":41244},[247],[170,41246,41248],{"className":41247,"style":17447},[251],[170,41249],{},[170,41251,923],{"className":41252},[424],[170,41254,338],{"className":41255},[337],[170,41257,41259],{"className":41258},[247],[170,41260,41262],{"className":41261,"style":16056},[251],[170,41263],{},[170,41265],{"className":41266},[424,235],[170,41268],{"className":41269,"style":1983},[610],[170,41271,5884],{"className":41272},[664],[170,41274],{"className":41275,"style":1983},[610],[170,41277,41279,41282,41285,41288,41295,41298,41341,41344,41347,41419],{"className":41278},[221],[170,41280],{"className":41281,"style":988},[225],[170,41283,40112],{"className":41284,"style":40129},[230,303],[170,41286],{"className":41287,"style":1027},[610],[170,41289,41291],{"className":41290},[230],[170,41292,41294],{"className":41293},[230,1066],"KL",[170,41296,909],{"className":41297},[234],[170,41299,41301,41304],{"className":41300},[230],[170,41302,5711],{"className":41303,"style":3292},[230,303],[170,41305,41307],{"className":41306},[307],[170,41308,41310,41333],{"className":41309},[242,243],[170,41311,41313,41330],{"className":41312},[247],[170,41314,41316],{"className":41315,"style":1359},[251],[170,41317,41318,41321],{"style":7567},[170,41319],{"className":41320,"style":651},[259],[170,41322,41324],{"className":41323},[264,265,266,267],[170,41325,41327],{"className":41326},[230,267],[170,41328,14157],{"className":41329,"style":3317},[230,303,267],[170,41331,338],{"className":41332},[337],[170,41334,41336],{"className":41335},[247],[170,41337,41339],{"className":41338,"style":1381},[251],[170,41340],{},[170,41342,914],{"className":41343},[1023],[170,41345],{"className":41346,"style":1027},[610],[170,41348,41350,41353],{"className":41349},[230],[170,41351,5711],{"className":41352,"style":3292},[230,303],[170,41354,41356],{"className":41355},[307],[170,41357,41359,41411],{"className":41358},[242,243],[170,41360,41362,41408],{"className":41361},[247],[170,41363,41365],{"className":41364,"style":1359},[251],[170,41366,41367,41370],{"style":7567},[170,41368],{"className":41369,"style":651},[259],[170,41371,41373],{"className":41372},[264,265,266,267],[170,41374,41376],{"className":41375},[230,267],[170,41377,41379,41382],{"className":41378},[230,267],[170,41380,14157],{"className":41381,"style":3317},[230,303,267],[170,41383,41385],{"className":41384},[307],[170,41386,41388],{"className":41387},[242],[170,41389,41391],{"className":41390},[247],[170,41392,41394],{"className":41393,"style":19250},[251],[170,41395,41396,41399],{"style":19253},[170,41397],{"className":41398,"style":324},[259],[170,41400,41402],{"className":41401},[264,328,329,267],[170,41403,41405],{"className":41404},[230,267],[170,41406,19090],{"className":41407},[230,267],[170,41409,338],{"className":41410},[337],[170,41412,41414],{"className":41413},[247],[170,41415,41417],{"className":41416,"style":1381},[251],[170,41418],{},[170,41420,923],{"className":41421},[424],[40091,41423,41425],{"id":41424},"ppo-clip截断重要性采样比","PPO-Clip：截断重要性采样比",[14,41427,41428],{},"PPO 还有一种实现，是通过截断函数来替代 KL 散度，防止训练的策略和参考的策略偏差过大，可以看出它是由红色和蓝色两个部分构成，取两个部分的最小值。",[14,41430,41431,41432,41932,41933,6868,41986,42037],{},"其中红色部分是原始公式，蓝色部分是一个截断函数，截断函数内由三个部分组成。如果 ",[170,41433,41435,41513],{"className":41434},[173],[170,41436,41438],{"className":41437},[177],[179,41439,41440],{"xmlns":181},[183,41441,41442,41510],{},[186,41443,41444],{},[189,41445,41446,41476],{},[186,41447,41448,41454,41456,41464,41466,41474],{},[199,41449,41450,41452],{},[202,41451,5711],{},[202,41453,14157],{},[572,41455,909],{"stretchy":908},[1933,41457,41458,41460,41462],{},[202,41459,134],{},[202,41461,446],{},[202,41463,888],{},[202,41465,13047],{"mathvariant":929},[1933,41467,41468,41470,41472],{},[202,41469,12908],{},[202,41471,446],{},[202,41473,888],{},[572,41475,923],{"stretchy":908},[186,41477,41478,41488,41490,41498,41500,41508],{},[199,41479,41480,41482],{},[202,41481,5711],{},[576,41483,41484,41486],{},[202,41485,14157],{},[572,41487,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[572,41489,909],{"stretchy":908},[1933,41491,41492,41494,41496],{},[202,41493,134],{},[202,41495,446],{},[202,41497,888],{},[202,41499,13047],{"mathvariant":929},[1933,41501,41502,41504,41506],{},[202,41503,12908],{},[202,41505,446],{},[202,41507,888],{},[572,41509,923],{"stretchy":908},[209,41511,41512],{"encoding":211},"\\frac{P_{\\theta}(a_n^t|s_n^t)}{P_{\\theta'}(a_n^t|s_n^t)}",[170,41514,41516],{"className":41515,"ariaHidden":217},[216],[170,41517,41519,41523],{"className":41518},[221],[170,41520],{"className":41521,"style":41522},[225],"height:1.6646em;vertical-align:-0.5452em;",[170,41524,41526,41529,41929],{"className":41525},[230],[170,41527],{"className":41528},[234,235],[170,41530,41532],{"className":41531},[189],[170,41533,41535,41920],{"className":41534},[242,243],[170,41536,41538,41917],{"className":41537},[247],[170,41539,41542,41742,41750],{"className":41540,"style":41541},[251],"height:1.1194em;",[170,41543,41544,41547],{"style":18612},[170,41545],{"className":41546,"style":260},[259],[170,41548,41550],{"className":41549},[264,265,266,267],[170,41551,41553,41628,41631,41685,41688,41739],{"className":41552},[230,267],[170,41554,41556,41559],{"className":41555},[230,267],[170,41557,5711],{"className":41558,"style":3292},[230,303,267],[170,41560,41562],{"className":41561},[307],[170,41563,41565,41619],{"className":41564},[242,243],[170,41566,41568,41616],{"className":41567},[247],[170,41569,41571],{"className":41570,"style":317},[251],[170,41572,41574,41578],{"style":41573},"top:-2.3448em;margin-left:-0.1389em;margin-right:0.0714em;",[170,41575],{"className":41576,"style":41577},[259],"height:2.6068em;",[170,41579,41581],{"className":41580},[264,328,329,267],[170,41582,41584],{"className":41583},[230,267],[170,41585,41587,41590],{"className":41586},[230,267],[170,41588,14157],{"className":41589,"style":3317},[230,303,267],[170,41591,41593],{"className":41592},[307],[170,41594,41596],{"className":41595},[242],[170,41597,41599],{"className":41598},[247],[170,41600,41603],{"className":41601,"style":41602},[251],"height:0.8496em;",[170,41604,41606,41610],{"style":41605},"top:-2.8496em;margin-right:0.1em;",[170,41607],{"className":41608,"style":41609},[259],"height:2.5556em;",[170,41611,41613],{"className":41612},[230,267],[170,41614,19090],{"className":41615},[230,267],[170,41617,338],{"className":41618},[337],[170,41620,41622],{"className":41621},[247],[170,41623,41626],{"className":41624,"style":41625},[251],"height:0.262em;",[170,41627],{},[170,41629,909],{"className":41630},[234,267],[170,41632,41634,41637],{"className":41633},[230,267],[170,41635,134],{"className":41636},[230,303,267],[170,41638,41640],{"className":41639},[307],[170,41641,41643,41676],{"className":41642},[242,243],[170,41644,41646,41673],{"className":41645},[247],[170,41647,41650,41662],{"className":41648,"style":41649},[251],"height:0.7253em;",[170,41651,41653,41656],{"style":41652},"top:-2.214em;margin-left:0em;margin-right:0.0714em;",[170,41654],{"className":41655,"style":324},[259],[170,41657,41659],{"className":41658},[264,328,329,267],[170,41660,446],{"className":41661},[230,303,267],[170,41663,41664,41667],{"style":19253},[170,41665],{"className":41666,"style":324},[259],[170,41668,41670],{"className":41669},[264,328,329,267],[170,41671,888],{"className":41672},[230,303,267],[170,41674,338],{"className":41675},[337],[170,41677,41679],{"className":41678},[247],[170,41680,41683],{"className":41681,"style":41682},[251],"height:0.286em;",[170,41684],{},[170,41686,13047],{"className":41687},[230,267],[170,41689,41691,41694],{"className":41690},[230,267],[170,41692,12908],{"className":41693},[230,303,267],[170,41695,41697],{"className":41696},[307],[170,41698,41700,41731],{"className":41699},[242,243],[170,41701,41703,41728],{"className":41702},[247],[170,41704,41706,41717],{"className":41705,"style":41649},[251],[170,41707,41708,41711],{"style":41652},[170,41709],{"className":41710,"style":324},[259],[170,41712,41714],{"className":41713},[264,328,329,267],[170,41715,446],{"className":41716},[230,303,267],[170,41718,41719,41722],{"style":19253},[170,41720],{"className":41721,"style":324},[259],[170,41723,41725],{"className":41724},[264,328,329,267],[170,41726,888],{"className":41727},[230,303,267],[170,41729,338],{"className":41730},[337],[170,41732,41734],{"className":41733},[247],[170,41735,41737],{"className":41736,"style":41682},[251],[170,41738],{},[170,41740,923],{"className":41741},[424,267],[170,41743,41744,41747],{"style":385},[170,41745],{"className":41746,"style":260},[259],[170,41748],{"className":41749,"style":393},[392],[170,41751,41753,41756],{"style":41752},"top:-3.5102em;",[170,41754],{"className":41755,"style":260},[259],[170,41757,41759],{"className":41758},[264,265,266,267],[170,41760,41762,41805,41808,41860,41863,41914],{"className":41761},[230,267],[170,41763,41765,41768],{"className":41764},[230,267],[170,41766,5711],{"className":41767,"style":3292},[230,303,267],[170,41769,41771],{"className":41770},[307],[170,41772,41774,41797],{"className":41773},[242,243],[170,41775,41777,41794],{"className":41776},[247],[170,41778,41780],{"className":41779,"style":317},[251],[170,41781,41782,41785],{"style":14281},[170,41783],{"className":41784,"style":324},[259],[170,41786,41788],{"className":41787},[264,328,329,267],[170,41789,41791],{"className":41790},[230,267],[170,41792,14157],{"className":41793,"style":3317},[230,303,267],[170,41795,338],{"className":41796},[337],[170,41798,41800],{"className":41799},[247],[170,41801,41803],{"className":41802,"style":345},[251],[170,41804],{},[170,41806,909],{"className":41807},[234,267],[170,41809,41811,41814],{"className":41810},[230,267],[170,41812,134],{"className":41813},[230,303,267],[170,41815,41817],{"className":41816},[307],[170,41818,41820,41852],{"className":41819},[242,243],[170,41821,41823,41849],{"className":41822},[247],[170,41824,41827,41838],{"className":41825,"style":41826},[251],"height:0.8703em;",[170,41828,41829,41832],{"style":41652},[170,41830],{"className":41831,"style":324},[259],[170,41833,41835],{"className":41834},[264,328,329,267],[170,41836,446],{"className":41837},[230,303,267],[170,41839,41840,41843],{"style":1209},[170,41841],{"className":41842,"style":324},[259],[170,41844,41846],{"className":41845},[264,328,329,267],[170,41847,888],{"className":41848},[230,303,267],[170,41850,338],{"className":41851},[337],[170,41853,41855],{"className":41854},[247],[170,41856,41858],{"className":41857,"style":41682},[251],[170,41859],{},[170,41861,13047],{"className":41862},[230,267],[170,41864,41866,41869],{"className":41865},[230,267],[170,41867,12908],{"className":41868},[230,303,267],[170,41870,41872],{"className":41871},[307],[170,41873,41875,41906],{"className":41874},[242,243],[170,41876,41878,41903],{"className":41877},[247],[170,41879,41881,41892],{"className":41880,"style":41826},[251],[170,41882,41883,41886],{"style":41652},[170,41884],{"className":41885,"style":324},[259],[170,41887,41889],{"className":41888},[264,328,329,267],[170,41890,446],{"className":41891},[230,303,267],[170,41893,41894,41897],{"style":1209},[170,41895],{"className":41896,"style":324},[259],[170,41898,41900],{"className":41899},[264,328,329,267],[170,41901,888],{"className":41902},[230,303,267],[170,41904,338],{"className":41905},[337],[170,41907,41909],{"className":41908},[247],[170,41910,41912],{"className":41911,"style":41682},[251],[170,41913],{},[170,41915,923],{"className":41916},[424,267],[170,41918,338],{"className":41919},[337],[170,41921,41923],{"className":41922},[247],[170,41924,41927],{"className":41925,"style":41926},[251],"height:0.5452em;",[170,41928],{},[170,41930],{"className":41931},[424,235]," 的值介于 ",[170,41934,41936,41955],{"className":41935},[173],[170,41937,41939],{"className":41938},[177],[179,41940,41941],{"xmlns":181},[183,41942,41943,41952],{},[186,41944,41945,41947,41949],{},[192,41946,194],{},[572,41948,18471],{},[202,41950,41951],{},"ϵ",[209,41953,41954],{"encoding":211},"1-\\epsilon",[170,41956,41958,41977],{"className":41957,"ariaHidden":217},[216],[170,41959,41961,41965,41968,41971,41974],{"className":41960},[221],[170,41962],{"className":41963,"style":41964},[225],"height:0.7278em;vertical-align:-0.0833em;",[170,41966,194],{"className":41967},[230],[170,41969],{"className":41970,"style":1983},[610],[170,41972,18471],{"className":41973},[664],[170,41975],{"className":41976,"style":1983},[610],[170,41978,41980,41983],{"className":41979},[221],[170,41981],{"className":41982,"style":458},[225],[170,41984,41951],{"className":41985},[230,303],[170,41987,41989,42007],{"className":41988},[173],[170,41990,41992],{"className":41991},[177],[179,41993,41994],{"xmlns":181},[183,41995,41996,42004],{},[186,41997,41998,42000,42002],{},[192,41999,194],{},[572,42001,5884],{},[202,42003,41951],{},[209,42005,42006],{"encoding":211},"1+\\epsilon",[170,42008,42010,42028],{"className":42009,"ariaHidden":217},[216],[170,42011,42013,42016,42019,42022,42025],{"className":42012},[221],[170,42014],{"className":42015,"style":41964},[225],[170,42017,194],{"className":42018},[230],[170,42020],{"className":42021,"style":1983},[610],[170,42023,5884],{"className":42024},[664],[170,42026],{"className":42027,"style":1983},[610],[170,42029,42031,42034],{"className":42030},[221],[170,42032],{"className":42033,"style":458},[225],[170,42035,41951],{"className":42036},[230,303]," 之间，就返回原值，否则返回其中更接近的一个。这就限制的参考模型和训练模型差异的值不能太大。",[170,42039,42041],{"className":42040},[14456],[170,42042,42044,42380],{"className":42043},[173],[170,42045,42047],{"className":42046},[177],[179,42048,42049],{"xmlns":181,"display":9588},[183,42050,42051,42377],{},[186,42052,42053,42055,42057,42059,42066,42068,42070,42076,42090,42108,42111,42113],{},[202,42054,7923],{},[202,42056,903],{},[202,42058,12908],{},[199,42060,42061,42063],{},[202,42062,12908],{},[3384,42064,42065],{},"ppo2",[572,42067,926],{},[572,42069,18471],{},[189,42071,42072,42074],{},[192,42073,194],{},[202,42075,7429],{},[14789,42077,42078,42080,42088],{},[572,42079,1937],{},[186,42081,42082,42084,42086],{},[202,42083,446],{},[572,42085,926],{},[192,42087,194],{},[202,42089,7429],{},[14789,42091,42092,42094,42102],{},[572,42093,1937],{},[186,42095,42096,42098,42100],{},[202,42097,888],{},[572,42099,926],{},[192,42101,194],{},[199,42103,42104,42106],{},[202,42105,13348],{},[202,42107,446],{},[202,42109,42110],{},"min",[572,42112,933],{},[186,42114,42115,42117,42226,42228,42230,42375],{},[572,42116,909],{"fence":217},[14480,42118,42120,42138,42140,42148,42150,42158,42160],{"mathcolor":42119},"red",[1933,42121,42122,42124,42130],{},[202,42123,885],{},[576,42125,42126,42128],{},[202,42127,14157],{},[572,42129,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[186,42131,42132,42134,42136],{},[202,42133,27616],{},[202,42135,885],{},[202,42137,5714],{},[572,42139,909],{"stretchy":908},[1933,42141,42142,42144,42146],{},[202,42143,12908],{},[202,42145,446],{},[202,42147,888],{},[572,42149,914],{"separator":217},[1933,42151,42152,42154,42156],{},[202,42153,134],{},[202,42155,446],{},[202,42157,888],{},[572,42159,923],{"stretchy":908},[189,42161,42162,42192],{},[186,42163,42164,42170,42172,42180,42182,42190],{},[199,42165,42166,42168],{},[202,42167,5711],{},[202,42169,14157],{},[572,42171,909],{"stretchy":908},[1933,42173,42174,42176,42178],{},[202,42175,134],{},[202,42177,446],{},[202,42179,888],{},[202,42181,13047],{"mathvariant":929},[1933,42183,42184,42186,42188],{},[202,42185,12908],{},[202,42187,446],{},[202,42189,888],{},[572,42191,923],{"stretchy":908},[186,42193,42194,42204,42206,42214,42216,42224],{},[199,42195,42196,42198],{},[202,42197,5711],{},[576,42199,42200,42202],{},[202,42201,14157],{},[572,42203,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[572,42205,909],{"stretchy":908},[1933,42207,42208,42210,42212],{},[202,42209,134],{},[202,42211,446],{},[202,42213,888],{},[202,42215,13047],{"mathvariant":929},[1933,42217,42218,42220,42222],{},[202,42219,12908],{},[202,42221,446],{},[202,42223,888],{},[572,42225,923],{"stretchy":908},[572,42227,914],{"separator":217},[3384,42229,27701],{},[14480,42231,42233,42243,42335,42353,42355,42363,42365,42373],{"mathcolor":42232},"skyblue",[186,42234,42235,42237,42239,42241],{},[202,42236,3051],{"mathvariant":929},[202,42238,3010],{"mathvariant":929},[202,42240,900],{"mathvariant":929},[202,42242,14],{"mathvariant":929},[186,42244,42245,42247,42313,42315,42317,42319,42321,42323,42325,42327,42329,42331,42333],{},[572,42246,909],{"fence":217},[189,42248,42249,42279],{},[186,42250,42251,42257,42259,42267,42269,42277],{},[199,42252,42253,42255],{},[202,42254,5711],{},[202,42256,14157],{},[572,42258,909],{"stretchy":908},[1933,42260,42261,42263,42265],{},[202,42262,134],{},[202,42264,446],{},[202,42266,888],{},[202,42268,13047],{"mathvariant":929},[1933,42270,42271,42273,42275],{},[202,42272,12908],{},[202,42274,446],{},[202,42276,888],{},[572,42278,923],{"stretchy":908},[186,42280,42281,42291,42293,42301,42303,42311],{},[199,42282,42283,42285],{},[202,42284,5711],{},[576,42286,42287,42289],{},[202,42288,14157],{},[572,42290,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[572,42292,909],{"stretchy":908},[1933,42294,42295,42297,42299],{},[202,42296,134],{},[202,42298,446],{},[202,42300,888],{},[202,42302,13047],{"mathvariant":929},[1933,42304,42305,42307,42309],{},[202,42306,12908],{},[202,42308,446],{},[202,42310,888],{},[572,42312,923],{"stretchy":908},[572,42314,914],{"separator":217},[3384,42316,27701],{},[192,42318,194],{},[572,42320,18471],{},[202,42322,41951],{},[572,42324,914],{"separator":217},[3384,42326,27701],{},[192,42328,194],{},[572,42330,5884],{},[202,42332,41951],{},[572,42334,923],{"fence":217},[1933,42336,42337,42339,42345],{},[202,42338,885],{},[576,42340,42341,42343],{},[202,42342,14157],{},[572,42344,19090],{"mathvariant":929,"lspace":14472,"rspace":14472},[186,42346,42347,42349,42351],{},[202,42348,27616],{},[202,42350,885],{},[202,42352,5714],{},[572,42354,909],{"stretchy":908},[1933,42356,42357,42359,42361],{},[202,42358,12908],{},[202,42360,446],{},[202,42362,888],{},[572,42364,914],{"separator":217},[1933,42366,42367,42369,42371],{},[202,42368,134],{},[202,42370,446],{},[202,42372,888],{},[572,42374,923],{"stretchy":908},[572,42376,923],{"fence":217},[209,42378,42379],{"encoding":211},"Loss_{\\text{ppo2}} = -\\frac{1}{N} \\sum_{n=1}^{N} \\sum_{t=1}^{T_n} \\min\\left(\\textcolor{red}{A_{\\theta'}^{GAE}(s_n^t, a_n^t) \\frac{P_{\\theta}(a_n^t|s_n^t)}{P_{\\theta'}(a_n^t|s_n^t)}},\\,\\textcolor{skyblue}{\\mathrm{clip}\\left(\\frac{P_{\\theta}(a_n^t|s_n^t)}{P_{\\theta'}(a_n^t|s_n^t)},\\, 1-\\epsilon,\\, 1+\\epsilon\\right) A_{\\theta'}^{GAE}(s_n^t, a_n^t)}\\right)",[170,42381,42383,42450],{"className":42382,"ariaHidden":217},[216],[170,42384,42386,42389,42392,42395,42441,42444,42447],{"className":42385},[221],[170,42387],{"className":42388,"style":40353},[225],[170,42390,7923],{"className":42391},[230,303],[170,42393,40360],{"className":42394},[230,303],[170,42396,42398,42401],{"className":42397},[230],[170,42399,12908],{"className":42400},[230,303],[170,42402,42404],{"className":42403},[307],[170,42405,42407,42433],{"className":42406},[242,243],[170,42408,42410,42430],{"className":42409},[247],[170,42411,42413],{"className":42412,"style":3198},[251],[170,42414,42415,42418],{"style":1362},[170,42416],{"className":42417,"style":651},[259],[170,42419,42421],{"className":42420},[264,265,266,267],[170,42422,42424],{"className":42423},[230,267],[170,42425,42427],{"className":42426},[230,11961,267],[170,42428,42065],{"className":42429},[230,267],[170,42431,338],{"className":42432},[337],[170,42434,42436],{"className":42435},[247],[170,42437,42439],{"className":42438,"style":40405},[251],[170,42440],{},[170,42442],{"className":42443,"style":611},[610],[170,42445,926],{"className":42446},[615],[170,42448],{"className":42449,"style":611},[610],[170,42451,42453,42456,42459,42521,42524,42591,42594,42698,42701,42704,42707],{"className":42452},[221],[170,42454],{"className":42455,"style":31279},[225],[170,42457,18471],{"className":42458},[230],[170,42460,42462,42465,42518],{"className":42461},[230],[170,42463],{"className":42464},[234,235],[170,42466,42468],{"className":42467},[189],[170,42469,42471,42510],{"className":42470},[242,243],[170,42472,42474,42507],{"className":42473},[247],[170,42475,42477,42488,42496],{"className":42476,"style":16385},[251],[170,42478,42479,42482],{"style":15925},[170,42480],{"className":42481,"style":260},[259],[170,42483,42485],{"className":42484},[230],[170,42486,7429],{"className":42487,"style":7504},[230,303],[170,42489,42490,42493],{"style":385},[170,42491],{"className":42492,"style":260},[259],[170,42494],{"className":42495,"style":393},[392],[170,42497,42498,42501],{"style":15991},[170,42499],{"className":42500,"style":260},[259],[170,42502,42504],{"className":42503},[230],[170,42505,194],{"className":42506},[230],[170,42508,338],{"className":42509},[337],[170,42511,42513],{"className":42512},[247],[170,42514,42516],{"className":42515,"style":16425},[251],[170,42517],{},[170,42519],{"className":42520},[424,235],[170,42522],{"className":42523,"style":1027},[610],[170,42525,42527],{"className":42526},[1062,14344],[170,42528,42530,42583],{"className":42529},[242,243],[170,42531,42533,42580],{"className":42532},[247],[170,42534,42536,42556,42566],{"className":42535,"style":16446},[251],[170,42537,42538,42541],{"style":16449},[170,42539],{"className":42540,"style":15538},[259],[170,42542,42544],{"className":42543},[264,265,266,267],[170,42545,42547,42550,42553],{"className":42546},[230,267],[170,42548,446],{"className":42549},[230,303,267],[170,42551,926],{"className":42552},[615,267],[170,42554,194],{"className":42555},[230,267],[170,42557,42558,42561],{"style":15547},[170,42559],{"className":42560,"style":15538},[259],[170,42562,42563],{},[170,42564,1937],{"className":42565},[1062,2021,15556],[170,42567,42568,42571],{"style":16480},[170,42569],{"className":42570,"style":15538},[259],[170,42572,42574],{"className":42573},[264,265,266,267],[170,42575,42577],{"className":42576},[230,267],[170,42578,7429],{"className":42579,"style":7504},[230,303,267],[170,42581,338],{"className":42582},[337],[170,42584,42586],{"className":42585},[247],[170,42587,42589],{"className":42588,"style":16499},[251],[170,42590],{},[170,42592],{"className":42593,"style":1027},[610],[170,42595,42597],{"className":42596},[1062,14344],[170,42598,42600,42690],{"className":42599},[242,243],[170,42601,42603,42687],{"className":42602},[247],[170,42604,42606,42626,42636],{"className":42605,"style":17255},[251],[170,42607,42608,42611],{"style":16449},[170,42609],{"className":42610,"style":15538},[259],[170,42612,42614],{"className":42613},[264,265,266,267],[170,42615,42617,42620,42623],{"className":42616},[230,267],[170,42618,888],{"className":42619},[230,303,267],[170,42621,926],{"className":42622},[615,267],[170,42624,194],{"className":42625},[230,267],[170,42627,42628,42631],{"style":15547},[170,42629],{"className":42630,"style":15538},[259],[170,42632,42633],{},[170,42634,1937],{"className":42635},[1062,2021,15556],[170,42637,42638,42641],{"style":17288},[170,42639],{"className":42640,"style":15538},[259],[170,42642,42644],{"className":42643},[264,265,266,267],[170,42645,42647],{"className":42646},[230,267],[170,42648,42650,42653],{"className":42649},[230,267],[170,42651,13348],{"className":42652,"style":3292},[230,303,267],[170,42654,42656],{"className":42655},[307],[170,42657,42659,42679],{"className":42658},[242,243],[170,42660,42662,42676],{"className":42661},[247],[170,42663,42665],{"className":42664,"style":4409},[251],[170,42666,42667,42670],{"style":17318},[170,42668],{"className":42669,"style":324},[259],[170,42671,42673],{"className":42672},[264,328,329,267],[170,42674,446],{"className":42675},[230,303,267],[170,42677,338],{"className":42678},[337],[170,42680,42682],{"className":42681},[247],[170,42683,42685],{"className":42684,"style":4431},[251],[170,42686],{},[170,42688,338],{"className":42689},[337],[170,42691,42693],{"className":42692},[247],[170,42694,42696],{"className":42695,"style":16499},[251],[170,42697],{},[170,42699],{"className":42700,"style":1027},[610],[170,42702,42110],{"className":42703},[1062],[170,42705],{"className":42706,"style":1027},[610],[170,42708,42710,42716,42811,42814,42865,42868,42871,42922,42925,43320,43323,43326,43329,43337,43340,43806,43809,43902,43905,43956,43959,43962,44013,44016],{"className":42709},[1073],[170,42711,42713],{"className":42712,"style":1078},[234,1077],[170,42714,909],{"className":42715},[1082,266],[170,42717,42720,42723],{"className":42718,"style":42719},[230],"color:red;",[170,42721,885],{"className":42722,"style":42719},[230,303],[170,42724,42726],{"className":42725},[307],[170,42727,42729,42803],{"className":42728},[242,243],[170,42730,42732,42800],{"className":42731},[247],[170,42733,42735,42779],{"className":42734,"style":24902},[251],[170,42736,42737,42740],{"style":17417},[170,42738],{"className":42739,"style":651},[259],[170,42741,42743],{"className":42742,"style":42719},[264,265,266,267],[170,42744,42746],{"className":42745,"style":42719},[230,267],[170,42747,42749,42753],{"className":42748,"style":42719},[230,267],[170,42750,14157],{"className":42751,"style":42752},[230,303,267],"margin-right:0.02778em;color:red;",[170,42754,42756],{"className":42755},[307],[170,42757,42759],{"className":42758},[242],[170,42760,42762],{"className":42761},[247],[170,42763,42765],{"className":42764,"style":19250},[251],[170,42766,42767,42770],{"style":19253},[170,42768],{"className":42769,"style":324},[259],[170,42771,42773],{"className":42772,"style":42719},[264,328,329,267],[170,42774,42776],{"className":42775,"style":42719},[230,267],[170,42777,19090],{"className":42778,"style":42719},[230,267],[170,42780,42781,42784],{"style":16532},[170,42782],{"className":42783,"style":651},[259],[170,42785,42787],{"className":42786,"style":42719},[264,265,266,267],[170,42788,42790,42793,42796],{"className":42789,"style":42719},[230,267],[170,42791,27616],{"className":42792,"style":42719},[230,303,267],[170,42794,885],{"className":42795,"style":42719},[230,303,267],[170,42797,5714],{"className":42798,"style":42799},[230,303,267],"margin-right:0.05764em;color:red;",[170,42801,338],{"className":42802},[337],[170,42804,42806],{"className":42805},[247],[170,42807,42809],{"className":42808,"style":17447},[251],[170,42810],{},[170,42812,909],{"className":42813,"style":42719},[234],[170,42815,42817,42820],{"className":42816,"style":42719},[230],[170,42818,12908],{"className":42819,"style":42719},[230,303],[170,42821,42823],{"className":42822},[307],[170,42824,42826,42857],{"className":42825},[242,243],[170,42827,42829,42854],{"className":42828},[247],[170,42830,42832,42843],{"className":42831,"style":17414},[251],[170,42833,42834,42837],{"style":17417},[170,42835],{"className":42836,"style":651},[259],[170,42838,42840],{"className":42839,"style":42719},[264,265,266,267],[170,42841,446],{"className":42842,"style":42719},[230,303,267],[170,42844,42845,42848],{"style":16532},[170,42846],{"className":42847,"style":651},[259],[170,42849,42851],{"className":42850,"style":42719},[264,265,266,267],[170,42852,888],{"className":42853,"style":42719},[230,303,267],[170,42855,338],{"className":42856},[337],[170,42858,42860],{"className":42859},[247],[170,42861,42863],{"className":42862,"style":17447},[251],[170,42864],{},[170,42866,914],{"className":42867,"style":42719},[1023],[170,42869],{"className":42870,"style":1027},[610],[170,42872,42874,42877],{"className":42873,"style":42719},[230],[170,42875,134],{"className":42876,"style":42719},[230,303],[170,42878,42880],{"className":42879},[307],[170,42881,42883,42914],{"className":42882},[242,243],[170,42884,42886,42911],{"className":42885},[247],[170,42887,42889,42900],{"className":42888,"style":17414},[251],[170,42890,42891,42894],{"style":17417},[170,42892],{"className":42893,"style":651},[259],[170,42895,42897],{"className":42896,"style":42719},[264,265,266,267],[170,42898,446],{"className":42899,"style":42719},[230,303,267],[170,42901,42902,42905],{"style":16532},[170,42903],{"className":42904,"style":651},[259],[170,42906,42908],{"className":42907,"style":42719},[264,265,266,267],[170,42909,888],{"className":42910,"style":42719},[230,303,267],[170,42912,338],{"className":42913},[337],[170,42915,42917],{"className":42916},[247],[170,42918,42920],{"className":42919,"style":17447},[251],[170,42921],{},[170,42923,923],{"className":42924,"style":42719},[424],[170,42926,42928,42931,43317],{"className":42927,"style":42719},[230],[170,42929],{"className":42930},[234,235],[170,42932,42934],{"className":42933},[189],[170,42935,42937,43309],{"className":42936},[242,243],[170,42938,42940,43306],{"className":42939},[247],[170,42941,42943,43135,43144],{"className":42942,"style":36409},[251],[170,42944,42945,42948],{"style":15925},[170,42946],{"className":42947,"style":260},[259],[170,42949,42951,43024,43027,43078,43081,43132],{"className":42950,"style":42719},[230],[170,42952,42954,42958],{"className":42953,"style":42719},[230],[170,42955,5711],{"className":42956,"style":42957},[230,303],"margin-right:0.13889em;color:red;",[170,42959,42961],{"className":42960},[307],[170,42962,42964,43016],{"className":42963},[242,243],[170,42965,42967,43013],{"className":42966},[247],[170,42968,42970],{"className":42969,"style":1359},[251],[170,42971,42972,42975],{"style":7567},[170,42973],{"className":42974,"style":651},[259],[170,42976,42978],{"className":42977,"style":42719},[264,265,266,267],[170,42979,42981],{"className":42980,"style":42719},[230,267],[170,42982,42984,42987],{"className":42983,"style":42719},[230,267],[170,42985,14157],{"className":42986,"style":42752},[230,303,267],[170,42988,42990],{"className":42989},[307],[170,42991,42993],{"className":42992},[242],[170,42994,42996],{"className":42995},[247],[170,42997,42999],{"className":42998,"style":19250},[251],[170,43000,43001,43004],{"style":19253},[170,43002],{"className":43003,"style":324},[259],[170,43005,43007],{"className":43006,"style":42719},[264,328,329,267],[170,43008,43010],{"className":43009,"style":42719},[230,267],[170,43011,19090],{"className":43012,"style":42719},[230,267],[170,43014,338],{"className":43015},[337],[170,43017,43019],{"className":43018},[247],[170,43020,43022],{"className":43021,"style":1381},[251],[170,43023],{},[170,43025,909],{"className":43026,"style":42719},[234],[170,43028,43030,43033],{"className":43029,"style":42719},[230],[170,43031,134],{"className":43032,"style":42719},[230,303],[170,43034,43036],{"className":43035},[307],[170,43037,43039,43070],{"className":43038},[242,243],[170,43040,43042,43067],{"className":43041},[247],[170,43043,43045,43056],{"className":43044,"style":36511},[251],[170,43046,43047,43050],{"style":17417},[170,43048],{"className":43049,"style":651},[259],[170,43051,43053],{"className":43052,"style":42719},[264,265,266,267],[170,43054,446],{"className":43055,"style":42719},[230,303,267],[170,43057,43058,43061],{"style":16635},[170,43059],{"className":43060,"style":651},[259],[170,43062,43064],{"className":43063,"style":42719},[264,265,266,267],[170,43065,888],{"className":43066,"style":42719},[230,303,267],[170,43068,338],{"className":43069},[337],[170,43071,43073],{"className":43072},[247],[170,43074,43076],{"className":43075,"style":17447},[251],[170,43077],{},[170,43079,13047],{"className":43080,"style":42719},[230],[170,43082,43084,43087],{"className":43083,"style":42719},[230],[170,43085,12908],{"className":43086,"style":42719},[230,303],[170,43088,43090],{"className":43089},[307],[170,43091,43093,43124],{"className":43092},[242,243],[170,43094,43096,43121],{"className":43095},[247],[170,43097,43099,43110],{"className":43098,"style":36511},[251],[170,43100,43101,43104],{"style":17417},[170,43102],{"className":43103,"style":651},[259],[170,43105,43107],{"className":43106,"style":42719},[264,265,266,267],[170,43108,446],{"className":43109,"style":42719},[230,303,267],[170,43111,43112,43115],{"style":16635},[170,43113],{"className":43114,"style":651},[259],[170,43116,43118],{"className":43117,"style":42719},[264,265,266,267],[170,43119,888],{"className":43120,"style":42719},[230,303,267],[170,43122,338],{"className":43123},[337],[170,43125,43127],{"className":43126},[247],[170,43128,43130],{"className":43129,"style":17447},[251],[170,43131],{},[170,43133,923],{"className":43134,"style":42719},[424],[170,43136,43137,43140],{"style":385},[170,43138],{"className":43139,"style":260},[259],[170,43141],{"className":43142,"style":43143},[392],"color:red;border-bottom-width:0.04em;",[170,43145,43146,43149],{"style":15991},[170,43147],{"className":43148,"style":260},[259],[170,43150,43152,43195,43198,43249,43252,43303],{"className":43151,"style":42719},[230],[170,43153,43155,43158],{"className":43154,"style":42719},[230],[170,43156,5711],{"className":43157,"style":42957},[230,303],[170,43159,43161],{"className":43160},[307],[170,43162,43164,43187],{"className":43163},[242,243],[170,43165,43167,43184],{"className":43166},[247],[170,43168,43170],{"className":43169,"style":1359},[251],[170,43171,43172,43175],{"style":7567},[170,43173],{"className":43174,"style":651},[259],[170,43176,43178],{"className":43177,"style":42719},[264,265,266,267],[170,43179,43181],{"className":43180,"style":42719},[230,267],[170,43182,14157],{"className":43183,"style":42752},[230,303,267],[170,43185,338],{"className":43186},[337],[170,43188,43190],{"className":43189},[247],[170,43191,43193],{"className":43192,"style":1381},[251],[170,43194],{},[170,43196,909],{"className":43197,"style":42719},[234],[170,43199,43201,43204],{"className":43200,"style":42719},[230],[170,43202,134],{"className":43203,"style":42719},[230,303],[170,43205,43207],{"className":43206},[307],[170,43208,43210,43241],{"className":43209},[242,243],[170,43211,43213,43238],{"className":43212},[247],[170,43214,43216,43227],{"className":43215,"style":18955},[251],[170,43217,43218,43221],{"style":17417},[170,43219],{"className":43220,"style":651},[259],[170,43222,43224],{"className":43223,"style":42719},[264,265,266,267],[170,43225,446],{"className":43226,"style":42719},[230,303,267],[170,43228,43229,43232],{"style":647},[170,43230],{"className":43231,"style":651},[259],[170,43233,43235],{"className":43234,"style":42719},[264,265,266,267],[170,43236,888],{"className":43237,"style":42719},[230,303,267],[170,43239,338],{"className":43240},[337],[170,43242,43244],{"className":43243},[247],[170,43245,43247],{"className":43246,"style":17447},[251],[170,43248],{},[170,43250,13047],{"className":43251,"style":42719},[230],[170,43253,43255,43258],{"className":43254,"style":42719},[230],[170,43256,12908],{"className":43257,"style":42719},[230,303],[170,43259,43261],{"className":43260},[307],[170,43262,43264,43295],{"className":43263},[242,243],[170,43265,43267,43292],{"className":43266},[247],[170,43268,43270,43281],{"className":43269,"style":18955},[251],[170,43271,43272,43275],{"style":17417},[170,43273],{"className":43274,"style":651},[259],[170,43276,43278],{"className":43277,"style":42719},[264,265,266,267],[170,43279,446],{"className":43280,"style":42719},[230,303,267],[170,43282,43283,43286],{"style":647},[170,43284],{"className":43285,"style":651},[259],[170,43287,43289],{"className":43288,"style":42719},[264,265,266,267],[170,43290,888],{"className":43291,"style":42719},[230,303,267],[170,43293,338],{"className":43294},[337],[170,43296,43298],{"className":43297},[247],[170,43299,43301],{"className":43300,"style":17447},[251],[170,43302],{},[170,43304,923],{"className":43305,"style":42719},[424],[170,43307,338],{"className":43308},[337],[170,43310,43312],{"className":43311},[247],[170,43313,43315],{"className":43314,"style":16056},[251],[170,43316],{},[170,43318],{"className":43319},[424,235],[170,43321,914],{"className":43322},[1023],[170,43324],{"className":43325,"style":1027},[610],[170,43327],{"className":43328,"style":1027},[610],[170,43330,43333],{"className":43331,"style":43332},[230],"color:skyblue;",[170,43334,43336],{"className":43335,"style":43332},[230,1066],"clip",[170,43338],{"className":43339,"style":1027},[610],[170,43341,43343,43352,43748,43751,43755,43758,43761,43765,43768,43771,43774,43777,43780,43783,43786,43789,43792,43795,43798],{"className":43342,"style":43332},[1073],[170,43344,43347],{"className":43345,"style":43346},[234,1077],"color:skyblue;top:0em;",[170,43348,43350],{"className":43349,"style":43332},[1082,266],[170,43351,909],{"style":43332},[170,43353,43355,43358,43745],{"className":43354,"style":43332},[230],[170,43356],{"className":43357},[234,235],[170,43359,43361],{"className":43360},[189],[170,43362,43364,43737],{"className":43363},[242,243],[170,43365,43367,43734],{"className":43366},[247],[170,43368,43370,43563,43572],{"className":43369,"style":36409},[251],[170,43371,43372,43375],{"style":15925},[170,43373],{"className":43374,"style":260},[259],[170,43376,43378,43452,43455,43506,43509,43560],{"className":43377,"style":43332},[230],[170,43379,43381,43385],{"className":43380,"style":43332},[230],[170,43382,5711],{"className":43383,"style":43384},[230,303],"margin-right:0.13889em;color:skyblue;",[170,43386,43388],{"className":43387},[307],[170,43389,43391,43444],{"className":43390},[242,243],[170,43392,43394,43441],{"className":43393},[247],[170,43395,43397],{"className":43396,"style":1359},[251],[170,43398,43399,43402],{"style":7567},[170,43400],{"className":43401,"style":651},[259],[170,43403,43405],{"className":43404,"style":43332},[264,265,266,267],[170,43406,43408],{"className":43407,"style":43332},[230,267],[170,43409,43411,43415],{"className":43410,"style":43332},[230,267],[170,43412,14157],{"className":43413,"style":43414},[230,303,267],"margin-right:0.02778em;color:skyblue;",[170,43416,43418],{"className":43417},[307],[170,43419,43421],{"className":43420},[242],[170,43422,43424],{"className":43423},[247],[170,43425,43427],{"className":43426,"style":19250},[251],[170,43428,43429,43432],{"style":19253},[170,43430],{"className":43431,"style":324},[259],[170,43433,43435],{"className":43434,"style":43332},[264,328,329,267],[170,43436,43438],{"className":43437,"style":43332},[230,267],[170,43439,19090],{"className":43440,"style":43332},[230,267],[170,43442,338],{"className":43443},[337],[170,43445,43447],{"className":43446},[247],[170,43448,43450],{"className":43449,"style":1381},[251],[170,43451],{},[170,43453,909],{"className":43454,"style":43332},[234],[170,43456,43458,43461],{"className":43457,"style":43332},[230],[170,43459,134],{"className":43460,"style":43332},[230,303],[170,43462,43464],{"className":43463},[307],[170,43465,43467,43498],{"className":43466},[242,243],[170,43468,43470,43495],{"className":43469},[247],[170,43471,43473,43484],{"className":43472,"style":36511},[251],[170,43474,43475,43478],{"style":17417},[170,43476],{"className":43477,"style":651},[259],[170,43479,43481],{"className":43480,"style":43332},[264,265,266,267],[170,43482,446],{"className":43483,"style":43332},[230,303,267],[170,43485,43486,43489],{"style":16635},[170,43487],{"className":43488,"style":651},[259],[170,43490,43492],{"className":43491,"style":43332},[264,265,266,267],[170,43493,888],{"className":43494,"style":43332},[230,303,267],[170,43496,338],{"className":43497},[337],[170,43499,43501],{"className":43500},[247],[170,43502,43504],{"className":43503,"style":17447},[251],[170,43505],{},[170,43507,13047],{"className":43508,"style":43332},[230],[170,43510,43512,43515],{"className":43511,"style":43332},[230],[170,43513,12908],{"className":43514,"style":43332},[230,303],[170,43516,43518],{"className":43517},[307],[170,43519,43521,43552],{"className":43520},[242,243],[170,43522,43524,43549],{"className":43523},[247],[170,43525,43527,43538],{"className":43526,"style":36511},[251],[170,43528,43529,43532],{"style":17417},[170,43530],{"className":43531,"style":651},[259],[170,43533,43535],{"className":43534,"style":43332},[264,265,266,267],[170,43536,446],{"className":43537,"style":43332},[230,303,267],[170,43539,43540,43543],{"style":16635},[170,43541],{"className":43542,"style":651},[259],[170,43544,43546],{"className":43545,"style":43332},[264,265,266,267],[170,43547,888],{"className":43548,"style":43332},[230,303,267],[170,43550,338],{"className":43551},[337],[170,43553,43555],{"className":43554},[247],[170,43556,43558],{"className":43557,"style":17447},[251],[170,43559],{},[170,43561,923],{"className":43562,"style":43332},[424],[170,43564,43565,43568],{"style":385},[170,43566],{"className":43567,"style":260},[259],[170,43569],{"className":43570,"style":43571},[392],"color:skyblue;border-bottom-width:0.04em;",[170,43573,43574,43577],{"style":15991},[170,43575],{"className":43576,"style":260},[259],[170,43578,43580,43623,43626,43677,43680,43731],{"className":43579,"style":43332},[230],[170,43581,43583,43586],{"className":43582,"style":43332},[230],[170,43584,5711],{"className":43585,"style":43384},[230,303],[170,43587,43589],{"className":43588},[307],[170,43590,43592,43615],{"className":43591},[242,243],[170,43593,43595,43612],{"className":43594},[247],[170,43596,43598],{"className":43597,"style":1359},[251],[170,43599,43600,43603],{"style":7567},[170,43601],{"className":43602,"style":651},[259],[170,43604,43606],{"className":43605,"style":43332},[264,265,266,267],[170,43607,43609],{"className":43608,"style":43332},[230,267],[170,43610,14157],{"className":43611,"style":43414},[230,303,267],[170,43613,338],{"className":43614},[337],[170,43616,43618],{"className":43617},[247],[170,43619,43621],{"className":43620,"style":1381},[251],[170,43622],{},[170,43624,909],{"className":43625,"style":43332},[234],[170,43627,43629,43632],{"className":43628,"style":43332},[230],[170,43630,134],{"className":43631,"style":43332},[230,303],[170,43633,43635],{"className":43634},[307],[170,43636,43638,43669],{"className":43637},[242,243],[170,43639,43641,43666],{"className":43640},[247],[170,43642,43644,43655],{"className":43643,"style":18955},[251],[170,43645,43646,43649],{"style":17417},[170,43647],{"className":43648,"style":651},[259],[170,43650,43652],{"className":43651,"style":43332},[264,265,266,267],[170,43653,446],{"className":43654,"style":43332},[230,303,267],[170,43656,43657,43660],{"style":647},[170,43658],{"className":43659,"style":651},[259],[170,43661,43663],{"className":43662,"style":43332},[264,265,266,267],[170,43664,888],{"className":43665,"style":43332},[230,303,267],[170,43667,338],{"className":43668},[337],[170,43670,43672],{"className":43671},[247],[170,43673,43675],{"className":43674,"style":17447},[251],[170,43676],{},[170,43678,13047],{"className":43679,"style":43332},[230],[170,43681,43683,43686],{"className":43682,"style":43332},[230],[170,43684,12908],{"className":43685,"style":43332},[230,303],[170,43687,43689],{"className":43688},[307],[170,43690,43692,43723],{"className":43691},[242,243],[170,43693,43695,43720],{"className":43694},[247],[170,43696,43698,43709],{"className":43697,"style":18955},[251],[170,43699,43700,43703],{"style":17417},[170,43701],{"className":43702,"style":651},[259],[170,43704,43706],{"className":43705,"style":43332},[264,265,266,267],[170,43707,446],{"className":43708,"style":43332},[230,303,267],[170,43710,43711,43714],{"style":647},[170,43712],{"className":43713,"style":651},[259],[170,43715,43717],{"className":43716,"style":43332},[264,265,266,267],[170,43718,888],{"className":43719,"style":43332},[230,303,267],[170,43721,338],{"className":43722},[337],[170,43724,43726],{"className":43725},[247],[170,43727,43729],{"className":43728,"style":17447},[251],[170,43730],{},[170,43732,923],{"className":43733,"style":43332},[424],[170,43735,338],{"className":43736},[337],[170,43738,43740],{"className":43739},[247],[170,43741,43743],{"className":43742,"style":16056},[251],[170,43744],{},[170,43746],{"className":43747},[424,235],[170,43749,914],{"className":43750,"style":43332},[1023],[170,43752],{"className":43753,"style":43754},[610],"color:skyblue;margin-right:0.1667em;",[170,43756],{"className":43757,"style":43754},[610],[170,43759,194],{"className":43760,"style":43332},[230],[170,43762],{"className":43763,"style":43764},[610],"color:skyblue;margin-right:0.2222em;",[170,43766,18471],{"className":43767,"style":43332},[664],[170,43769],{"className":43770,"style":43764},[610],[170,43772,41951],{"className":43773,"style":43332},[230,303],[170,43775,914],{"className":43776,"style":43332},[1023],[170,43778],{"className":43779,"style":43754},[610],[170,43781],{"className":43782,"style":43754},[610],[170,43784,194],{"className":43785,"style":43332},[230],[170,43787],{"className":43788,"style":43764},[610],[170,43790,5884],{"className":43791,"style":43332},[664],[170,43793],{"className":43794,"style":43764},[610],[170,43796,41951],{"className":43797,"style":43332},[230,303],[170,43799,43801],{"className":43800,"style":43346},[424,1077],[170,43802,43804],{"className":43803,"style":43332},[1082,266],[170,43805,923],{"style":43332},[170,43807],{"className":43808,"style":1027},[610],[170,43810,43812,43815],{"className":43811,"style":43332},[230],[170,43813,885],{"className":43814,"style":43332},[230,303],[170,43816,43818],{"className":43817},[307],[170,43819,43821,43894],{"className":43820},[242,243],[170,43822,43824,43891],{"className":43823},[247],[170,43825,43827,43870],{"className":43826,"style":24902},[251],[170,43828,43829,43832],{"style":17417},[170,43830],{"className":43831,"style":651},[259],[170,43833,43835],{"className":43834,"style":43332},[264,265,266,267],[170,43836,43838],{"className":43837,"style":43332},[230,267],[170,43839,43841,43844],{"className":43840,"style":43332},[230,267],[170,43842,14157],{"className":43843,"style":43414},[230,303,267],[170,43845,43847],{"className":43846},[307],[170,43848,43850],{"className":43849},[242],[170,43851,43853],{"className":43852},[247],[170,43854,43856],{"className":43855,"style":19250},[251],[170,43857,43858,43861],{"style":19253},[170,43859],{"className":43860,"style":324},[259],[170,43862,43864],{"className":43863,"style":43332},[264,328,329,267],[170,43865,43867],{"className":43866,"style":43332},[230,267],[170,43868,19090],{"className":43869,"style":43332},[230,267],[170,43871,43872,43875],{"style":16532},[170,43873],{"className":43874,"style":651},[259],[170,43876,43878],{"className":43877,"style":43332},[264,265,266,267],[170,43879,43881,43884,43887],{"className":43880,"style":43332},[230,267],[170,43882,27616],{"className":43883,"style":43332},[230,303,267],[170,43885,885],{"className":43886,"style":43332},[230,303,267],[170,43888,5714],{"className":43889,"style":43890},[230,303,267],"margin-right:0.05764em;color:skyblue;",[170,43892,338],{"className":43893},[337],[170,43895,43897],{"className":43896},[247],[170,43898,43900],{"className":43899,"style":17447},[251],[170,43901],{},[170,43903,909],{"className":43904,"style":43332},[234],[170,43906,43908,43911],{"className":43907,"style":43332},[230],[170,43909,12908],{"className":43910,"style":43332},[230,303],[170,43912,43914],{"className":43913},[307],[170,43915,43917,43948],{"className":43916},[242,243],[170,43918,43920,43945],{"className":43919},[247],[170,43921,43923,43934],{"className":43922,"style":17414},[251],[170,43924,43925,43928],{"style":17417},[170,43926],{"className":43927,"style":651},[259],[170,43929,43931],{"className":43930,"style":43332},[264,265,266,267],[170,43932,446],{"className":43933,"style":43332},[230,303,267],[170,43935,43936,43939],{"style":16532},[170,43937],{"className":43938,"style":651},[259],[170,43940,43942],{"className":43941,"style":43332},[264,265,266,267],[170,43943,888],{"className":43944,"style":43332},[230,303,267],[170,43946,338],{"className":43947},[337],[170,43949,43951],{"className":43950},[247],[170,43952,43954],{"className":43953,"style":17447},[251],[170,43955],{},[170,43957,914],{"className":43958,"style":43332},[1023],[170,43960],{"className":43961,"style":1027},[610],[170,43963,43965,43968],{"className":43964,"style":43332},[230],[170,43966,134],{"className":43967,"style":43332},[230,303],[170,43969,43971],{"className":43970},[307],[170,43972,43974,44005],{"className":43973},[242,243],[170,43975,43977,44002],{"className":43976},[247],[170,43978,43980,43991],{"className":43979,"style":17414},[251],[170,43981,43982,43985],{"style":17417},[170,43983],{"className":43984,"style":651},[259],[170,43986,43988],{"className":43987,"style":43332},[264,265,266,267],[170,43989,446],{"className":43990,"style":43332},[230,303,267],[170,43992,43993,43996],{"style":16532},[170,43994],{"className":43995,"style":651},[259],[170,43997,43999],{"className":43998,"style":43332},[264,265,266,267],[170,44000,888],{"className":44001,"style":43332},[230,303,267],[170,44003,338],{"className":44004},[337],[170,44006,44008],{"className":44007},[247],[170,44009,44011],{"className":44010,"style":17447},[251],[170,44012],{},[170,44014,923],{"className":44015,"style":43332},[424],[170,44017,44019],{"className":44018,"style":1078},[424,1077],[170,44020,923],{"className":44021},[1082,266],[10,44023,44024],{"id":44024},"参考资料",[14,44026,44027],{},[134,44028,44031],{"href":44029,"rel":44030},"https://www.bilibili.com/video/BV1iz421h7gb/",[138],"零基础学习强化学习算法：ppo",{"title":89,"searchDepth":90,"depth":90,"links":44033},[44034,44035,44036,44037,44040,44044],{"id":12779,"depth":90,"text":12779},{"id":12785,"depth":90,"text":12785},{"id":14051,"depth":90,"text":14051},{"id":19567,"depth":90,"text":19568,"children":44038},[44039],{"id":27544,"depth":96,"text":27545},{"id":31866,"depth":90,"text":31867,"children":44041},[44042,44043],{"id":31873,"depth":96,"text":31873},{"id":33678,"depth":96,"text":33679},{"id":44024,"depth":90,"text":44024},"记一下学习 PPO中学习到的东西",{"date":44047,"image":44048,"alt":44049,"tags":44050,"published":114},"8 Mar 2026","/blogs-img/blog2.jpg","PPO学习笔记",[44051,12767],"Reinforcement learning","/blogs/ppo",{"title":12774,"description":44045},"blogs/2. PPO学习笔记","Sx3SA-gd0nzBc5-IShm_PpZrHX9EMT8A7rF2k0GLT9I",{"id":44057,"title":44058,"body":44059,"description":44058,"extension":105,"meta":44433,"navigation":114,"ogImage":44435,"path":44439,"seo":44440,"stem":44441,"__hash__":44442},"content/blogs/3. 记录一下我的 zsh 配置.md","记录一下我的 zsh 配置",{"type":7,"value":44060,"toc":44416},[44061,44063,44066,44070,44073,44076,44079,44082,44085,44128,44131,44135,44138,44141,44207,44210,44229,44232,44235,44238,44241,44244,44264,44267,44270,44272,44291,44293,44300,44305,44308,44311,44314,44317,44324,44332,44335,44338,44341,44344,44347,44396,44402,44405,44408,44413],[10,44062,12],{"id":12},[14,44064,44065],{},"最近想写点东西记录一下自己折腾这些工具的经历，之前没想到记录什么，今年实验室的服务器迁移需要重新配置一下 zsh,于是想着干脆顺手写一下自己折腾 zsh 出来的方案。",[10,44067,44069],{"id":44068},"什么是-zsh","什么是 Zsh",[14,44071,44072],{},"Zsh 也是一种 shell 软件，可以看成是一个更好用的 Bash，目前 MacOS 中的默认 shell 已经是 Zsh 了。Zsh 完全兼容 Bash，拥有极其丰富的插件，其强大的自动补全参数、文件名以及自定义功能，可以大大提高我们使用 Linux 的效率。",[14,44074,44075],{},"Zsh 可以通过大部分主流 Linux 发行版的包管理器进行安装，若在没有 sudo 权限的服务器上也可以通过conda进行安装。",[10,44077,44078],{"id":44078},"oh-my-zsh",[14,44080,44081],{},"Zsh 固然好用，但初始时只有白板没有任何插件，距离理想的使用体验还有一些差距，这里我们介绍 oh-my-zsh。",[14,44083,44084],{},"oh-my-zsh 是社区为 zsh 收集的一个第三方插件和主题合集，我们可以通过国内镜像快速安装，这里使用了南京大学镜像站。",[2501,44086,44090],{"className":44087,"code":44088,"language":44089,"meta":89,"style":89},"language-bash shiki shiki-themes github-light dracula","git clone https://mirror.nju.edu.cn/git/ohmyzsh.git\ncd ohmyzsh/tools\nREMOTE=https://mirror.nju.edu.cn/git/ohmyzsh.git sh install.sh\n","bash",[2507,44091,44092,44103,44111],{"__ignoreMap":89},[170,44093,44094,44097,44100],{"class":2511,"line":2512},[170,44095,44096],{"class":2519},"git",[170,44098,44099],{"class":9587}," clone",[170,44101,44102],{"class":9587}," https://mirror.nju.edu.cn/git/ohmyzsh.git\n",[170,44104,44105,44108],{"class":2511,"line":90},[170,44106,44107],{"class":9568},"cd",[170,44109,44110],{"class":9587}," ohmyzsh/tools\n",[170,44112,44113,44117,44119,44122,44125],{"class":2511,"line":96},[170,44114,44116],{"class":44115},"scEb-","REMOTE",[170,44118,926],{"class":2515},[170,44120,44121],{"class":9587},"https://mirror.nju.edu.cn/git/ohmyzsh.git",[170,44123,44124],{"class":2519}," sh",[170,44126,44127],{"class":9587}," install.sh\n",[14,44129,44130],{},"通过这三条指令可以在境内快速安装我们的 oh-my-zsh，同时也可以选择将 zsh 设置为默认 shell。",[10,44132,44134],{"id":44133},"powerlevel10k","Powerlevel10k",[14,44136,44137],{},"oh-my-zsh 中的主题我认为还是不够美观和定制，我希望有个更好的体验，这里我们引入 Powerlevel10k 主题。在众多主题中，Powerlevel10k 因其美观性和高度定制的特性脱颖而出，截止到 2026年4月28日，Powerlevel10k 在 github 已累计收获 54k 个 star。",[14,44139,44140],{},"在国内，安装 Powerlevel10k 可以使用 gitee 的镜像进行安装，安装指令如下：",[2501,44142,44144],{"className":44087,"code":44143,"language":44089,"meta":89,"style":89},"git clone --depth=1 https://gitee.com/romkatv/powerlevel10k.git \"${ZSH_CUSTOM:-$HOME/.oh-my-zsh/custom}/themes/powerlevel10k\"\necho 'source /usr/share/zsh-theme-powerlevel10k/powerlevel10k.zsh-theme' >>~/.zshrc\n",[2507,44145,44146,44189],{"__ignoreMap":89},[170,44147,44148,44150,44152,44155,44158,44160,44163,44166,44169,44172,44174,44176,44178,44180,44183,44186],{"class":2511,"line":2512},[170,44149,44096],{"class":2519},[170,44151,44099],{"class":9587},[170,44153,44154],{"class":2563}," --depth=1",[170,44156,44157],{"class":9587}," https://gitee.com/romkatv/powerlevel10k.git",[170,44159,11360],{"class":9583},[170,44161,44162],{"class":9587},"${",[170,44164,44165],{"class":44115},"ZSH_CUSTOM",[170,44167,44168],{"class":2515},":-",[170,44170,44171],{"class":44115},"$HOME",[170,44173,2934],{"class":2515},[170,44175,975],{"class":9587},[170,44177,44078],{"class":44115},[170,44179,2934],{"class":2515},[170,44181,44182],{"class":44115},"custom",[170,44184,44185],{"class":9587},"}/themes/powerlevel10k",[170,44187,44188],{"class":9583},"\"\n",[170,44190,44191,44194,44196,44199,44201,44204],{"class":2511,"line":90},[170,44192,44193],{"class":9568},"echo",[170,44195,12030],{"class":9583},[170,44197,44198],{"class":9587},"source /usr/share/zsh-theme-powerlevel10k/powerlevel10k.zsh-theme",[170,44200,10999],{"class":9583},[170,44202,44203],{"class":2515}," >>",[170,44205,44206],{"class":9587},"~/.zshrc\n",[14,44208,44209],{},"之后需要安装一款推荐字体，这里我个人使用的 0xProtoNerdFont 字体，大家也可以在其他博客或者官方仓库中使用其他人推荐的字体。",[14,44211,44212,44213,44216,44217,44220,44221,44224,44225,44228],{},"之后修改 ",[2507,44214,44215],{},"~/.zshrc"," 文件，将变量 ",[2507,44218,44219],{},"ZSH_THEME"," 修改为 ",[2507,44222,44223],{},"powerlevel10k/powerlevel10k",",即 ",[2507,44226,44227],{},"ZSH_THEME=\"powerlevel10k/powerlevel10k\""," 即可，之后重启终端即可根据个人喜好开始配置。",[10,44230,44231],{"id":44231},"插件设置",[14,44233,44234],{},"现在主题已经搞定了，下一步开始设置真正能帮助我们提高生产力的插件。",[24,44236,44237],{"id":44237},"zsh-autosuggestions",[14,44239,44240],{},"一个非常有用的插件，它根据历史记录和补全信息在输入时建议命令。",[14,44242,44243],{},"安装指令：",[2501,44245,44247],{"className":44087,"code":44246,"language":44089,"meta":89,"style":89},"git clone https://github.com/zsh-users/zsh-autosuggestions $ZSH_CUSTOM/plugins/zsh-autosuggestions\n",[2507,44248,44249],{"__ignoreMap":89},[170,44250,44251,44253,44255,44258,44261],{"class":2511,"line":2512},[170,44252,44096],{"class":2519},[170,44254,44099],{"class":9587},[170,44256,44257],{"class":9587}," https://github.com/zsh-users/zsh-autosuggestions",[170,44259,44260],{"class":44115}," $ZSH_CUSTOM",[170,44262,44263],{"class":9587},"/plugins/zsh-autosuggestions\n",[24,44265,44266],{"id":44266},"zsh-syntax-highlighting",[14,44268,44269],{},"提供指令和文件路径的高亮限制，方便你检查指令和路径的正确性。",[14,44271,44243],{},[2501,44273,44275],{"className":44087,"code":44274,"language":44089,"meta":89,"style":89},"git clone https://github.com/zsh-users/zsh-syntax-highlighting $ZSH_CUSTOM/plugins/zsh-syntax-highlighting\n",[2507,44276,44277],{"__ignoreMap":89},[170,44278,44279,44281,44283,44286,44288],{"class":2511,"line":2512},[170,44280,44096],{"class":2519},[170,44282,44099],{"class":9587},[170,44284,44285],{"class":9587}," https://github.com/zsh-users/zsh-syntax-highlighting",[170,44287,44260],{"class":44115},[170,44289,44290],{"class":9587},"/plugins/zsh-syntax-highlighting\n",[24,44292,44096],{"id":44096},[14,44294,44295,44296,44299],{},"oh-my-zsh 自带的插件，将大量常用的 git 指令简化，具体怎么使用可以参考 ",[2507,44297,44298],{},"$ZSH/plugins/git/git.plugin.zsh "," 这个文件。",[14,44301,44302],{},[153,44303],{"alt":44096,"src":44304},"zsh/git.png",[14,44306,44307],{},"如图所示 alias 表示别名，等号左边为缩写指令，右边为原指令。",[10,44309,44310],{"id":44310},"一些软件",[24,44312,44313],{"id":44313},"nvm",[14,44315,44316],{},"一个非常好用的 node.js 版本管理器，无需 sudo 权限。",[14,44318,44319,44320,44323],{},"解决使用系统级 node 时 ",[2507,44321,44322],{},"npm install -g "," 安装引起的一些问题。",[14,44325,44326,44327,44331],{},"参考 ",[134,44328,44329],{"href":44329,"rel":44330},"https://github.com/nvm-sh/nvm",[138]," 安装。",[24,44333,44334],{"id":44334},"conda",[14,44336,44337],{},"无需多言。",[24,44339,44340],{"id":44340},"fastfetch",[14,44342,44343],{},"获取本机发行版和硬件信息，比较酷炫，装逼使用。",[14,44345,44346],{},"可以进行美化，参考指令：",[2501,44348,44350],{"className":44087,"code":44349,"language":44089,"meta":89,"style":89},"cd $HOME/.config\n\n# 下载并解压然后删除\nwget https://github.com/sqlsec/fastfetch/releases/download/v0.1/fastfetch-for-Linux.zip\nunzip fastfetch-for-Linux.zip && rm fastfetch-for-Linux.zip\n",[2507,44351,44352,44362,44366,44371,44379],{"__ignoreMap":89},[170,44353,44354,44356,44359],{"class":2511,"line":2512},[170,44355,44107],{"class":9568},[170,44357,44358],{"class":44115}," $HOME",[170,44360,44361],{"class":9587},"/.config\n",[170,44363,44364],{"class":2511,"line":90},[170,44365,2832],{"emptyLinePlaceholder":114},[170,44367,44368],{"class":2511,"line":96},[170,44369,44370],{"class":2547},"# 下载并解压然后删除\n",[170,44372,44373,44376],{"class":2511,"line":2551},[170,44374,44375],{"class":2519},"wget",[170,44377,44378],{"class":9587}," https://github.com/sqlsec/fastfetch/releases/download/v0.1/fastfetch-for-Linux.zip\n",[170,44380,44381,44384,44387,44390,44393],{"class":2511,"line":2570},[170,44382,44383],{"class":2519},"unzip",[170,44385,44386],{"class":9587}," fastfetch-for-Linux.zip",[170,44388,44389],{"class":2523}," && ",[170,44391,44392],{"class":2519},"rm",[170,44394,44395],{"class":9587}," fastfetch-for-Linux.zip\n",[14,44397,44398,44399],{},"效果如图：\n",[153,44400],{"alt":44340,"src":44401},"zsh/fastfetch.png",[24,44403,44404],{"id":44404},"onefetch",[14,44406,44407],{},"获取当前 git 项目信息，使用时需保证当前目录已经 git 初始化。",[14,44409,44398,44410],{},[153,44411],{"alt":44404,"src":44412},"zsh/onefetch.png",[12742,44414,44415],{},"html pre.shiki code .sCLZk, html code.shiki .sCLZk{--shiki-default:#6F42C1;--shiki-dark:#50FA7B}html pre.shiki code .sEzAm, html code.shiki .sEzAm{--shiki-default:#032F62;--shiki-dark:#F1FA8C}html pre.shiki code .sDgm9, html code.shiki .sDgm9{--shiki-default:#005CC5;--shiki-dark:#8BE9FD}html pre.shiki code .scEb-, html code.shiki .scEb-{--shiki-default:#24292E;--shiki-dark:#BD93F9}html pre.shiki code .szJfE, html code.shiki .szJfE{--shiki-default:#D73A49;--shiki-dark:#FF79C6}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html pre.shiki code .soDru, html code.shiki .soDru{--shiki-default:#005CC5;--shiki-dark:#BD93F9}html pre.shiki code .sMWOi, html code.shiki .sMWOi{--shiki-default:#032F62;--shiki-dark:#E9F284}html pre.shiki code .sfgPZ, html code.shiki .sfgPZ{--shiki-default:#6A737D;--shiki-dark:#6272A4}html pre.shiki code .scbbO, html code.shiki .scbbO{--shiki-default:#24292E;--shiki-dark:#F8F8F2}",{"title":89,"searchDepth":90,"depth":90,"links":44417},[44418,44419,44420,44421,44422,44427],{"id":12,"depth":90,"text":12},{"id":44068,"depth":90,"text":44069},{"id":44078,"depth":90,"text":44078},{"id":44133,"depth":90,"text":44134},{"id":44231,"depth":90,"text":44231,"children":44423},[44424,44425,44426],{"id":44237,"depth":96,"text":44237},{"id":44266,"depth":96,"text":44266},{"id":44096,"depth":96,"text":44096},{"id":44310,"depth":90,"text":44310,"children":44428},[44429,44430,44431,44432],{"id":44313,"depth":96,"text":44313},{"id":44334,"depth":96,"text":44334},{"id":44340,"depth":96,"text":44340},{"id":44404,"depth":96,"text":44404},{"date":44434,"image":44435,"alt":44436,"tags":44437,"published":114},"28 April 2026","/blogs-img/blog3.jpg","Zsh 配置",[44438,111],"Linux","/blogs/zsh",{"title":44058,"description":44058},"blogs/3. 记录一下我的 zsh 配置","PJV8pr0i_NIcto4IrLugXMoDWoeE7cWFgoETPzSjgXk",{"id":4,"title":5,"body":44444,"description":104,"extension":105,"meta":44505,"navigation":114,"ogImage":108,"path":115,"seo":44507,"stem":117,"__hash__":118},{"type":7,"value":44445,"toc":44492},[44446,44448,44450,44452,44454,44456,44458,44460,44462,44464,44466,44468,44470,44472,44474,44476,44478,44480,44482,44484,44486,44488,44490],[10,44447,12],{"id":12},[14,44449,16],{},[10,44451,19],{"id":19},[14,44453,22],{},[24,44455,27],{"id":26},[14,44457,30],{},[24,44459,34],{"id":33},[14,44461,37],{},[24,44463,41],{"id":40},[14,44465,44],{},[24,44467,48],{"id":47},[14,44469,51],{},[10,44471,55],{"id":54},[14,44473,58],{},[10,44475,62],{"id":61},[14,44477,65],{},[10,44479,68],{"id":68},[14,44481,71],{},[10,44483,75],{"id":74},[14,44485,78],{},[14,44487,81],{},[14,44489,84],{},[14,44491,87],{},{"title":89,"searchDepth":90,"depth":90,"links":44493},[44494,44495,44501,44502,44503,44504],{"id":12,"depth":90,"text":12},{"id":19,"depth":90,"text":19,"children":44496},[44497,44498,44499,44500],{"id":26,"depth":96,"text":27},{"id":33,"depth":96,"text":34},{"id":40,"depth":96,"text":41},{"id":47,"depth":96,"text":48},{"id":54,"depth":90,"text":55},{"id":61,"depth":90,"text":62},{"id":68,"depth":90,"text":68},{"id":74,"depth":90,"text":75},{"date":107,"image":108,"alt":109,"tags":44506,"published":114},[109,111,112,62,55,113],{"title":5,"description":104},{"id":44509,"title":44510,"body":44511,"description":44567,"extension":105,"meta":44568,"navigation":114,"ogImage":108,"path":44573,"seo":44574,"stem":44575,"__hash__":44576},"content/blogs/5. OpenMP 学习笔记.md","OpenMP 学习笔记",{"type":7,"value":44512,"toc":44562},[44513,44515,44518,44522,44528,44537,44539],[10,44514,12],{"id":12},[14,44516,44517],{},"这学期选修了并行计算这门课程，但学期过半一点还没开始学（），之前的实验啥的全仰仗 GPT 大人，最近开始打算好好学一学，先从课程的第一部分：CPU 并行学习。",[10,44519,44521],{"id":44520},"openmp","OpenMP",[44523,44524,44525],"blockquote",{},[14,44526,44527],{},"OpenMP（Open Multi-Processing）是一套支持跨平台共享内存方式的多线程并发的编程API，使用C,C++和Fortran语言，可以在大多数的处理器体系和操作系统中运行，包括Solaris, AIX, HP-UX, GNU/Linux, Mac OS X, 和Microsoft Windows。包括一套编译器指令、库和一些能够影响运行行为的环境变量。",[14,44529,44530,44531,44536],{},"这个介绍摘自 ",[134,44532,44535],{"href":44533,"rel":44534},"https://zh.wikipedia.org/wiki/OpenMP",[138],"维基百科 OpenMP","，在目前的并行计算框架中，OpenMP 是最容易上手的一个框架，通过许多高层抽象隐藏了实现的具体细节，只需要串行程序中加入OpenMP预处理指令, 就可以实现串行程序的并行化。",[10,44538,44024],{"id":44024},[44540,44541,44542,44549,44555],"ul",{},[44543,44544,44545],"li",{},[134,44546,44510],{"href":44547,"rel":44548},"https://parallel.zhangjikai.com/openmp.html",[138],[44543,44550,44551],{},[134,44552,44554],{"href":44533,"rel":44553},[138],"维基百科",[44543,44556,44557],{},[134,44558,44561],{"href":44559,"rel":44560},"https://www.bilibili.com/video/BV1SW411s7ST",[138],"Introduction to OpenMP - Tim Mattson (Intel)",{"title":89,"searchDepth":90,"depth":90,"links":44563},[44564,44565,44566],{"id":12,"depth":90,"text":12},{"id":44520,"depth":90,"text":44521},{"id":44024,"depth":90,"text":44024},"2026年春季学期并行计算课程的OpenMP学习笔记。",{"date":44569,"image":108,"alt":44570,"tags":44571,"published":114},"1 May 2026","并行计算学习笔记",[44438,44572],"并行计算","/blogs/openmp",{"title":44510,"description":44567},"blogs/5. OpenMP 学习笔记","3cqC4qDQLnD-zXRvzydKPsXhvOzyB39QnVWQ326QAKQ",1777735910278]